Entity embeddings, attention mechanisms and ordinal regression for predicting life satisfaction: a study of young south Korean male workers

Background: Young male workers in South Korea face unique challenges that can impact their subjective life satisfaction. This study aimed to develop and evaluate a novel machine learning model to predict their life satisfaction. Methods: We integrated Entity Embeddings with Attention mechanisms into an ordinal logistic regression framework. Entity Embeddings transformed categorical variables into dense vectors, while the Attention mechanism prioritized the most influential features. Model performance was compared against Random Forest, Gradient Boosting, and K-Nearest Neighbors (KNN) using metrics such as Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R-squared (R2). Results: The Entity Embeddings with Attention model demonstrated superior predictive accuracy compared to all baseline models across all evaluation metrics. Ordinal logistic regression facilitated model interpretability, revealing key predictors of life satisfaction, including monthly salary, job satisfaction, company size, weekly working hours, and educational background. Conclusions: This study provides valuable insights for policymakers and employers to enhance the well-being of young male workers in South Korea. The proposed model offers a robust and interpretable approach for predicting subjective life satisfaction, enabling targeted interventions to address the specific needs and challenges of this demographic.

Entity embeddings; Attention mechanism; Ordinal regression; Subjective life satisfaction; Predictive modeling

[1] Doede M. Race as a predictor of job satisfaction and turnover in US nurses. Journal of Nursing Management. 2017; 25: 207–214.

[2] Lazzari M, Alvarez JM, Ruggieri S. Predicting and explaining employee turnover intention. International Journal of Data Science and Analytics. 2022; 14: 279–292.

[3] Ozkan A, Elci M, Karabay M, Kitapci H, Garip C. Antecedents of turnover intention: a meta-analysis study in the United States. Ekonomics and Management. 2020; 23: 93–110.

[4] Al Akasheh M, Hujran O, Malik EF, Zaki N. Enhancing the prediction of employee turnover with knowledge graphs and explainable AI. IEEE Access. 2024; 12: 77041–77053.

[5] Hahs-Vaughn DL, Lomax RG. An introduction to statistical concepts. 4th edn. Routledge: New York, NY. 2020.

[6] Skelton AR, Nattress D, Dwyer RJ. Predicting manufacturing employee turnover intentions. Journal of Economics, Finance and Administrative Science. 2020; 25: 101–117.

[7] Dahouda MK, Joe I. A deep-learned embedding technique for categorical features encoding. IEEE Access. 2021; 9: 114381–114391.

[8] Rezaei F, Houmansadr A. FINN: fingerprinting network flows using neural networks. Proceedings of the 37th Annual Computer Security Applications Conference 2021. Annual Computer Security Applications Conference: Austin, Texas, USA. 2021.

[9] Dong Y, Cordonnier JB, Loukas A. Attention is not all you need: pure attention loses rank doubly exponentially with depth. International Conference on Machine Learning 2021. Proceedings of Machine Learning Research: Seattle, WA, USA. 2021.

[10] Umuroglu Y, Fraser NJ, Gambardella G, Blott M, Leong P, Jahre M, et al. Finn: a framework for fast, scalable binarized neural network inference. Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. February 2017. ACM/SIGDA International Symposium on Field-Programmable Gate Arrays: Monterey, California, USA. 2017.

[11] Abowd JM, Kramarz F. The analysis of labor markets using matched employer-employee data. In Abowd JM, Kramarz F (eds.) Handbook of Labor Economics (pp. 2629–2710). Elsevier: Amsterdam. 1999.

[12] Crandall RW, Lehr W, Litan RE. The effects of broadband deployment on output and employment: a cross-sectional analysis of US data. Brookings Institution. 2007; 6: 1–36.

[13] Bastani K, Asgari E, Namavari H. Wide and deep learning for peer-to-peer lending. Expert Systems with Applications. 2019; 134: 209–224.

[14] Zhou F, Zhou HM, Yang Z, Yang L. EMD2FNN: a strategy combining empirical mode decomposition and factorization machine based neural network for stock market trend prediction. Expert Systems with Applications. 2019; 115: 136–151.

[15] Lundberg SM, Lee SI. A unified approach to interpreting model predictions. 2017. Available at: https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf (Accessed: 15 November 2024).

[16] Pardoe I. Applied regression modeling. John Wiley & Sons: New York, NY. 2020.

[17] Liang J, Bi G, Zhan C. Multinomial and ordinal logistic regression analyses with multi-categorical variables using R. Annals of Translational Medicine. 2020; 8: 982.

[18] Li L, Zhao J, Hou L, Zhai Y, Shi J, Cui F. An attention-based deep learning model for clinical named entity recognition of Chinese electronic medical records. BMC Medical Informatics and Decision Making. 2019; 19: 235.