The position of belief functions structure in performance evaluation measures: a new approach

Document Type : Original Article

Authors

1 Department of industrial engineering/ Payame Noor University/ Tehran/ Iran

2 Department of Management, Payam Noor University, Tehran, Iran

10.22080/shrm.2023.4393

Abstract

Performance evaluation is one of the management processes,, the accuracy and validity of the results of this process is of great importance. In the performance evaluation process. common performance evaluation scales do not have the appropriate capability of modeling and dealing with uncertainty. Among the methods of dealing with uncertainty, the theory of belief functions is a suitable structure for gathering inconclusive opinions from evaluators and it has the ability to simultaneously face the uncertainty caused by the variability and uncertainty of the lack of knowledge. In this paper, a new way to get opinions from evaluators based on belief functions structure was presented and the advantages and improvements of the presented model were compared to the common metrics used in the performance evaluation process. Two questionnaires were prepared to evaluate different scales including Likert , Fuzzy Linguistic , Visual Analogue and Belief Functions. The first questionnaire contains 5 simple questions and the second questionnaire contains 3 complex questions. after summarizing the results of each questionnaire, a survey was conducted regarding the acceptability of the obtained results. According to the data conditions, one-way ANOVA test and Friedman's non-parametric test were used to analyze. The results of the implementation of the model in evaluating the performance of university lecturers showed that the advantages of the belief functions structure in the face of uncertainty can facilitate the possibility of expressing opinions for evaluators and improve the final results of the evaluation.

Keywords

References

AbuDahab, K., Xu, D. L., & Chen, Y. W. (2016). A new belief rule base knowledge representation scheme and inference methodology using the evidential reasoning rule for evidence combination. Expert Systems with Applications, 51, 218–230. https://doi.org/10.1016/j.eswa.2015.12.013
Aiken, L. R. (1996). Rating scales and checklists: Evaluating behavior, personality, and attitudes. John Wiley & Sons.
Arbaiy, N., & Suradi, Z. (2007). Staff performance appraisal using fuzzy evaluation. Artificial Intelligence and Innovations 2007: From Theory to Applications, 195–203. https://doi.org/10.1007/978-0-387-74161-1_21
Austin, J. T., & Villanova, P. (1992). The criterion problem: 1917-1992. Journal of Applied Psychology, 77(6), 836–874. https://doi.org/10.1037/0021-9010.77.6.836
Awogbami, G., Agana, N., Nazmi, S., & Homaifar, A. (2018). A New Combination Rule Based on the Average Belief Function. SoutheastCon 2018, 1–5. https://doi.org/10.1109/SECON.2018.8478815
Azwir, H. H., & Kalinggo, B. A. (2019). Multistage Fuzzy Inference System for Solving Problems in Performance Appraisal. 2019 International Conference on Sustainable Engineering and Creative Computing (ICSECC), 200–205. https://doi.org/10.1109/ICSECC.2019.8907212
Baraldi, P., Mangili, F., & Zio, E. (2015). A belief function theory based approach to combining different representation of uncertainty in prognostics. Information Sciences, 303, 134–149. https://doi.org/https://doi.org/10.1016/j.ins.2014.12.051
Borman, W. C., Buck, D. E., Hanson, M. A., Motowidlo, S. J., Stark, S., & Drasgow, F. (2001a). An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86(5), 965.
Borman, W. C., Buck, D. E., Hanson, M. A., Motowidlo, S. J., Stark, S., & Drasgow, F. (2001b). An examination of the comparative reliability, validity, and accuracy of performance ratings made using computerized adaptive rating scales. Journal of Applied Psychology, 86(5), 965–973.
Brutus, S. S. (2010). Words versus numbers: A theoretical exploration of giving and receiving narrative comments in performance appraisal. Human Resource Management Review, 20(2), 144–157. https://doi.org/10.1016/j.hrmr.2009.06.003
Chang, J.-R., Cheng, C.-H., & Chen, L.-S. (2007). A fuzzy-based military officer performance appraisal system. Applied Soft Computing, 7(3), 936–945. https://doi.org/http://dx.doi.org/10.1016/j.asoc.2006.03.003
Cotton, D. R. E., Nash, T., & Kneale, P. (2017). Supporting the retention of non-traditional students in Higher Education using a resilience framework. European Educational Research Journal, 16(1), 62–79.
De La Rosa De Sáa, S., Gil, M. Á., García, M. T. L., & Lubiano, M. A. (2013). Fuzzy rating vs. fuzzy conversion scales: An empirical comparison through the MSE. Advances in Intelligent Systems and Computing, 190 AISC, 135–143. https://doi.org/10.1007/978-3-642-33042-1_15
Deng, Y., Sadiq, R., Jiang, W., & Tesfamariam, S. (2011). Risk analysis in a linguistic environment: A fuzzy evidential reasoning-based approach. Expert Systems with Applications, 38(12), 15438–15446. https://doi.org/10.1016/j.eswa.2011.06.018
Dezert, J., Wang, P., & Tchamova, A. (2012). On the validity of Dempster-Shafer theory. Information Fusion (FUSION), 2012 15th International Conference On, 655–660.
Ding, S., Ma, X.-J., & Yang, S.-L. (2012). A software trustworthiness evaluation model using objective weight based evidential reasoning approach. Knowledge and Information Systems, 33(1), 171–189.
Esen, H., Hatipoğlu, T., & Boyacı, A. İ. (2016). A Fuzzy Approach for Performance Appraisal: The Evaluation of a Purchasing Specialist. In Studies in Computational Intelligence (Vol. 620, pp. 235–250). Springer. https://doi.org/10.1007/978-3-319-26393-9_14
Espinilla, M., de Andrés, R., Martínez, F. J., & Martínez, L. (2013). A 360-degree performance appraisal model dealing with heterogeneous information and dependent criteria. Information Sciences, 222, 459–471. https://doi.org/http://dx.doi.org/10.1016/j.ins.2012.08.015
Fu, C., Yang, J.-B., & Yang, S.-L. (2015). A group evidential reasoning approach based on expert reliability. European Journal of Operational Research, 246(3), 886–893. https://doi.org/http://dx.doi.org/10.1016/j.ejor.2015.05.042
Fu, C., & Yang, S. (2012). An evidential reasoning based consensus model for multiple attribute group decision analysis problems with interval-valued group consensus requirements. European Journal of Operational Research, 223(1), 167–176. https://doi.org/10.1016/j.ejor.2012.05.048
Golec, A., & Kahya, E. (2007). A fuzzy model for competency-based employee evaluation and selection. Computers & Industrial Engineering, 52(1), 143–161.
Golman, R., & Bhatia, S. (2012). Performance evaluation inflation and compression. Accounting, Organizations and Society, 37(8), 534–543.
Gürbüz, T., & Albayrak, Y. E. (2014). An engineering approach to human resources performance evaluation: Hybrid MCDM application with interactions. Applied Soft Computing Journal, 21, 365–375. https://doi.org/10.1016/j.asoc.2014.03.025
Ha-Duong, M. (2008). Hierarchical fusion of expert opinions in the Transferable Belief Model, application to climate sensitivity. International Journal of Approximate Reasoning, 49(3), 555–574. https://doi.org/10.1016/j.ijar.2008.05.003
Hedge, J. W., & Teachout, M. S. (2000). Exploring the Concept of Acceptability as a Criterion for Evaluating Performance Measures. Group & Organization Management, 25(1), 22–44. https://doi.org/10.1177/1059601100251003
Hesketh, T., Pryor, R., & Hesketh, B. (1988). An application of a computerized fuzzy graphic rating scale to the psychological measurement of individual differences. International Journal of Man-Machine Studies, 29(1), 21-35.
Hoffman, B. J., Gorman, C. A., Blair, C. A., Meriac, J. P., Overstreet, B., & Atchley, E. K. (2012). Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology. Personnel Psychology, 65(3), 531–563. https://doi.org/10.1111/j.1744-6570.2012.01252.x
Hooft, E. A. J., Flier, H., & Minne, M. R. (2006). Construct Validity of Multi‐Source Performance Ratings: An Examination of the Relationship of Self‐, Supervisor‐, and Peer‐Ratings with Cognitive and Personality Measures. International Journal of Selection and Assessment, 14(1), 67–81.
Huang, L., Ruan, S., & Denœux, T. (2021). Belief Function-Based Semi-Supervised Learning For Brain Tumor Segmentation. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), 160–164. https://doi.org/10.1109/ISBI48211.2021.9433885
Jamsandekar, S. S., & Mudholkar, R. R. (2013). Performance Evaluation by Fuzzy Inference Technique. 2, 158–164.
Jelley, R. B., Goffm, R. D., & Goffin, R. D. (2001). Can performance-feedback accuracy be improved? Effects of rater priming and rating-scale format on rating accuracy. Journal of Applied Psychology, 86(1), 134.
Jiang, W., Wei, B., Xie, C., & Zhou, D. (2016). An evidential sensor fusion method in fault diagnosis. Advances in Mechanical Engineering, 8(3), 1-7.
Jin, L., Fang, X., & Xu, Y. (2015). A Method for Multi-attribute Decision Making Under Uncertainty Using Evidential Reasoning and Prospect Theory. International Journal of Computational Intelligence Systems, 8(sup1), 48–62. https://doi.org/10.1080/18756891.2015.1129578
John Bernardin, H., Thomason, S., Ronald Buckley, M., & Kane, J. S. (2016). Rater rating‐level bias and accuracy in performance appraisals: The impact of rater personality, performance management competence, and rater accountability. Human Resource Management, 55(2), 321-340.
Klein, J., & Colot, O. (2011). Singular sources mining using evidential conflict analysis. International Journal of Approximate Reasoning, 52(9), 1433–1451. https://doi.org/10.1016/j.ijar.2011.08.005
Lefevre, E., Colot, O., & Vannoorenberghe, P. (2002). Belief functions combination and conflict management. Information Fusion Journal, 3(2), 149–162.
Lefèvre, E., & Elouedi, Z. (2013). How to preserve the conflict as an alarm in the combination of belief functions? Decision Support Systems, 56(1), 326–333. https://doi.org/10.1016/j.dss.2013.06.012
Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30(6), 881–905.
Liu, Z. G., Dezert, J., Pan, Q., & Mercier, G. (2011). Combination of sources of evidence with different discounting factors based on a new dissimilarity measure. Decision Support Systems, 52(1), 133–141. https://doi.org/10.1016/j.dss.2011.06.002
Lubiano, M. A., de La Rosa De Sáa, S., Montenegro, M., Sinova, B., & Gil, M. Á. (2016a). Descriptive analysis of responses to items in questionnaires. Why not using a fuzzy rating scale? Information Sciences, 360, 131–148. https://doi.org/10.1016/j.ins.2016.04.029
Lubiano, M. A., Montenegro, M., Sinova, B., de la Rosa de Sáa, S., & Gil, M. Á. (2016b). Hypothesis testing for means in connection with fuzzy rating scale-based data: algorithms and applications. European Journal of Operational Research, 251(3), 918–929. https://doi.org/http://dx.doi.org/10.1016/j.ejor.2015.11.016
Macwan, N., & Sajja, D. P. S. (2013). Performance appraisal using fuzzy evaluation methodology. International Journal of Engineering and Innovative Technology, 3(3), 324-329.
Manoharan, T. R., Muralidharan, C., & Deshmukh, S. G. (2011). An integrated fuzzy multi-attribute decision-making model for employees’ performance appraisal. The International Journal of Human Resource Management, 22(3), 722–745. https://doi.org/10.1080/09585192.2011.543763
Maseleno, A., Hasan, Md. M., Tuah, N., & Tabbu, C. R. (2015). Fuzzy Logic and Mathematical Theory of Evidence to Detect the Risk of Disease Spreading of Highly Pathogenic Avian Influenza H5N1. Procedia Computer Science, 57, 348–357. https://doi.org/http://dx.doi.org/10.1016/j.procs.2015.07.349
Meenakshi, G. (2012). Multi source feedback based performance appraisal system using Fuzzy logic decision support system. International Journal on Soft Computing, 3(1), 91–106. https://doi.org/10.5121/ijsc.2012.3108
Min, N., Sirisrisakulchai, J., Sriboonchitta, S. (2015). Forecasting Tourist Arrivals to Thailand Using Belief Functions. In: Huynh, VN., Kreinovich, V., Sriboonchitta, S., Suriya, K. (eds) Econometrics of Risk. Studies in Computational Intelligence, vol 583. Springer, Cham. https://doi.org/10.1007/978-3-319-13449-9_24
Moon, C., Lee, J., & Lim, S. (2010). A performance appraisal and promotion ranking system based on fuzzy logic: An implementation case in military organizations. Applied Soft Computing, 10(2), 512–519. https://doi.org/http://dx.doi.org/10.1016/j.asoc.2009.08.035
Murphy, C. K. (2000). Combining belief functions when evidence conflicts. Decision Support Systems, 29(1), 1–9. https://doi.org/10.1016/S0167-9236(99)00084-6
Murphy, K. R. (2008a). Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance. Industrial and Organizational Psychology, 1(2), 148–160. https://doi.org/10.1111/j.1754-9434.2008.00030.x
Murphy, K. R. (2008b). Explaining the Weak Relationship Between Job Performance and Ratings of Job Performance. Industrial and Organizational Psychology, 1(2), 148–160. https://doi.org/10.1111/j.1754-9434.2008.00030.x
Murphy, K. R., & Cleveland, J. (1995). Understanding performance appraisal: Social, organizational, and goal-based perspectives. Sage.
Neogi, A., Mondal, A. C. A. A. C. A., & Mandal, S. S. K. (2011). A cascaded fuzzy inference system for university non-teaching staff performance appraisal. Journal of Information Processing Systems, 7(4), 595–612. https://doi.org/10.3745/JIPS.2011.7.4.595
Ng, C. Y. (2016). Evidential reasoning-based Fuzzy AHP approach for the evaluation of design alternatives’ environmental performances. Applied Soft Computing Journal, 46, 381–397. https://doi.org/10.1016/j.asoc.2016.05.022
Nowack, K. M., & Mashihi, S. (2012). Evidence-based answers to 15 questions about leveraging 360-degree feedback. Consulting Psychology Journal: Practice and Research, 64(3), 157-182.
Oberkampf, W. L., DeLand, S. M., Rutherford, B. M., Diegert, K. v., & Alvin, K. F. (2002). Error and uncertainty in modeling and simulation. Reliability Engineering & System Safety, 75(3), 333–357. https://doi.org/10.1016/S0951-8320(01)00120-X
Oukhellou, L., Debiolles, A., Denœux, T., & Aknin, P. (2010). Fault diagnosis in railway track circuits using Dempster–Shafer classifier fusion. Engineering Applications of Artificial Intelligence, 23(1), 117–128. https://doi.org/http://dx.doi.org/10.1016/j.engappai.2009.06.005
Paksoy, A., & Göktürk, M. (2011). Information fusion with dempster-shafer evidence theory for software defect prediction. Procedia Computer Science, 3, 600–605.
Qiang, C., & Deng, Y. (2022a). A new correlation coefficient of mass function in evidence theory and its application in fault diagnosis. Applied Intelligence, 52(7), 7832–7842. https://doi.org/10.1007/s10489-021-02797-2
Qiang, C., & Deng, Y. (2022b). A new correlation coefficient of mass function in evidence theory and its application in fault diagnosis. Applied Intelligence, 52, 1–11. https://doi.org/10.1007/s10489-021-02797-2
Saris, W. E., & van Meurs, A. (1990). Evaluation of measurement instruments by meta-analysis of multitrait multi-method studies. Proceedings Amsterdam: North Holland.
Schubert, J. (2011). Conflict management in Dempster-Shafer theory using the degree of falsity. International Journal of Approximate Reasoning, 52(3), 449–460. https://doi.org/10.1016/j.ijar.2010.10.004
Sentz, K., & Ferson, S. (2002). Combination of evidence in Dempster-Shafer theory (Vol. 4015). Citeseer, Sandia National Laboratories.
Shaout, A., Yousif, M. K., & Khalid Yousif, M. (2014). Employee Performance Appraisal System Using Fuzzy Logic. International Journal of Computer Science and Information Technology, 6(4), 1–19. https://doi.org/10.5121/ijcsit.2014.6401
Smets, P. (2005). Decision making in the TBM: The necessity of the pignistic transformation. International Journal of Approximate Reasoning, 38(2), 133–147. https://doi.org/10.1016/j.ijar.2004.05.003
Smets, P. (2007). Analyzing the combination of conflicting belief functions. Information Fusion, 8(4), 387–412. https://doi.org/10.1016/j.inffus.2006.04.003
Tang, D., Wong, T. C., Chin, K. S., & Kwong, C. K. (2014). Evaluation of user satisfaction using evidential reasoning-based methodology. Neurocomputing, 142, 86–94. https://doi.org/http://dx.doi.org/10.1016/j.neucom.2014.01.055
Taroun, A., & Yang, J.-B. (2011). Dempster-Shafer Theory of Evidence: Potential usage for decision making and risk analysis in construction project management. The Built & Human Environment Review, 4(1), 155-166.
Tong, Z., Xu, P., & Denœux, T. (2021). An evidential classifier based on Dempster-Shafer theory and deep learning. Neurocomputing, 450, 275–293. https://doi.org/https://doi.org/10.1016/j.neucom.2021.03.066
Turgut, H., & Mert, I. S. (2014). Evaluation of Performance Appraisal Methods through Appraisal Errors by Using Fuzzy VIKOR Method. International Business Research, 7(10), 170–178. https://doi.org/10.5539/ibr.v7n10p170
Vasu, J. Z., Deb, A. K., & Mukhopadhyay, S. (2015). MVEM-based fault diagnosis of automotive engines using Dempster–Shafer theory and multiple hypotheses testing. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45(7), 977-989.
Wagner, S. H., & Goffin, R. D. (1997). Differences in accuracy of absolute and comparative performance appraisal methods. Organizational Behavior & Human Decision Processes, 70(2), 95–103. https://doi.org/10.1006/obhd.1997.2698
Wang, Y., Zhang, K., & Deng, Y. (2019). Base belief function: an efficient method of conflict management. Journal of Ambient Intelligence and Humanized Computing, 10(9), 3427–3437. https://doi.org/10.1007/s12652-018-1099-2
Wilson, N. (1993). Decision-making with Belief Functions and pignistic probabilities. In M. Clarke, R. Kruse, & S. Moral (Eds.), Symbolic and Quantitative Approaches to Reasoning and Uncertainty: European Conference ECSQARU ’93 Granada, Spain, November 8--10, 1993 Proceedings (pp. 364–371). Springer Berlin Heidelberg. https://doi.org/10.1007/BFb0028222
Woehr, D. J. (2008). On the Relationship Between Job Performance and Ratings of Job Performance: What Do We Really Know? Industrial and Organizational Psychology, 1(2), 161–166.
Woehr, D. J., & Huffcutt, A. I. (1994a). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67(3), 189–205.
Woehr, D. J., & Huffcutt, A. I. (1994b). Rater training for performance appraisal: A quantitative review. Journal of Occupational and Organizational Psychology, 67(3), 189–205.
Xiao, Z., Yang, X., Pang, Y., & Dang, X. (2012). The prediction for listed companies’ financial distress by using multiple prediction methods with rough set and Dempster–Shafer evidence theory. Knowledge-Based Systems, 26, 196–206. https://doi.org/http://dx.doi.org/10.1016/j.knosys.2011.08.001
Xu, D.-L. (2012). An introduction and survey of the evidential reasoning approach for multiple criteria decision analysis. Ann Oper Res, 195(1), 163–187. https://doi.org/10.1007/s10479-011-0945-9
Yang, J. B., & Singh, M. G. (1994). An evidential reasoning approach for multiple-attribute decision making with uncertainty. IEEE Transactions on systems, Man, and Cybernetics, 24(1), 1-18.
Ying-Feng, K., & Ling-Show, C. (2002). Using the fuzzy synthetic decision approach to assess the performance of university teachers in Taiwan. International Journal of  …, 19(4), 593–604.
Zadeh, L. A. (1975). The concept of a linguistic variable and its application to approximate reasoning—I. Information Sciences, 8(3), 199–249.
Zadeh, L. A. (1986). A simple view of the Dempster-Shafer theory of evidence and its implication for the rule of combination. AI Magazine, 7(2), 85.
 

Files

Share

How to cite

Statistics