European Journal of Social Sciences – Volume 7, Number 3 (2009)
A New Framework for Selection of the Best Performance Appraisal Method Mostafa Jafari Industrial Engineering Department, Iran University of Science & Technology Narmak, Tehran, Iran E-mail: [email protected] Tel: +98-912-1382658 Atieh Bourouni Industrial Engineering Department, Iran University of Science & Technology Narmak, Tehran, Iran E-mail: [email protected] Tel: +98-912-5007402 Roozbeh Hesam Amiri Industrial Engineering Department, Iran University of Science & Technology Narmak, Tehran, Iran E-mail: [email protected] Tel: +98-912-5356487; Fax: +98-21-88814576 Abstract Performance appraisal is one of the most important processes in human resource management, because it has a great effect on both the financial and program components of any organization. There is a verity of methods for the appraisal of employees' performance. Obviously, no method can claim that it has an integrated approach in performance appraisal. Therefore, human resource managers should select an appraisal method which is most efficient in their organizations. In this paper, we propose a framework for the selection of appraisal methods and compare some performance appraisal methods in order to facilitate the selection process for organizations. The value of this framework is that, with use of it, organisations can evaluate their performance appraisal method with respect to the key features of it before implementing any method as well as expending extra costs. This framework is theoretical in nature, and is build based on a review of related literature. Keywords: Human Resource Management, Absolute Standards, Relative Standards, Objectives Approach, Simple Additive Weighting (SAW), Personnel Psychology
1. Introduction One of the basic and major needs in any organization is to evaluate its employee’s performance contiguously and continually to find out whether they improve or not and know their situation in organization. There are many methods for performance appraisal which are suitable for different situations and organization’s characteristics. the major focus of enquiry for several authors (e.g. Freeman, 2002; Seddon, 2001; Wilson and Western, 2000; Randell, 1994; Segall, 1989; Long, 1986) has been examining why, in such a volatile environment, performance appraisal systems tend not to 92
European Journal of Social Sciences – Volume 7, Number 3 (2009) incorporate goals and the direction or strategic needs of the business, or the personal aspirations of the employees and their future development. The HR strategies adopted by an organization (Harris, 1994) may have an effect on the type of appraisal system adopted by an organization. The purpose of this paper is to present a framework for performance appraisal comparison and show how organization can use it. The purpose of this comparison can be summarized in two objects (Avison, 2006): 1. Academic purpose: for better understanding of nature of techniques in order to doing classifications and improving coming knowledge mapping 2. Practical purpose: to select a technique or part of one or more techniques for specific application or the whole of organization. There are many successful (Pettijohn et al., 2001b; Roberts & Reed, 1996; Pettijohn et al., 2001; Roberts, 2003; Shah & Murphy, 1995; Martey, 2002; Austin, 2008; Simmons, 2005; Ford, 2004; Khoury, 2004; Grifell-Tatjé, 2008) and unsuccessful (Blackmore, 2005; Shih, 2005; McAdam, 2005) cases because of suitable or unsuitable performance appraisal methods. In additional, Verbeeten (2008) argued that the definition of clear and measurable goals is positively associated with quantity and quality performance. The results suggest that the behavioral effects of performance management practices are as important as the economic effects in organizations. In this paper, in order to select an appropriate appraisal method, the Simple Additive Weighting (SAW) is used. This method has been described in literature (Ching-Lai, 1981; Kwangsun, 1981; Fiala et al., 1997; Tyc et al., 2004; Azar, 2000; Smith, 2006; Hwang and Yoon, 1981; Zavadskas and Kaklauskas, 1996; Balcomb and Curtner, 2000; Triantaphyllou, 2000).
2. Review of Literature 2.1. Performance Appraisal Employees want to know how well they perform on their jobs. A simple statement, almost axiomatic in any organization, yet it has probably caused more controversy, applied research and practical advice than any other assertion in the history of management writing and thinking (Kavanagh, 1997). Performance appraisal has many definitions. “Performance appraisal” is a process within the overall performance management process (Dowling et al., 1999), and is defined as “the evaluation of an individual’s work performance in order to arrive at objective personnel decisions” (Robbins et al., 2000). 2.1.1. Performance Appraisal Purpose Taylor and Wherry (1951) proposed that ratings collected for administrative purposes would be more lenient than ratings collected for research or developmental purposes. Over the last few years, there are no new empirical works which have been conducted (Greguras et al., 2003; Harris et al., 1995) but a useful review of it has been written (Jawahar & Williams, 1997). While the majority of the research on performance appraisal purpose has focused on the rater, some work has also been conducted on rater effects (Boswell & Boudreau, 2000, 2002). There are numerous methods to measure employee’s performance appraisal but some of these methods are not suitable in some cases. Effective appraisal systems should address clarity, openness, and fairness; recognize productivity through rewards; and be cognizant of appraiser leadership qualities (Winston & Creamer, 1997). 2.2. Performance Appraisal Methods Decenzo and Robbins (1998) denominate that there are three existent approaches for measuring performance appraisal. These are (1) absolute standards (2) relative standards and (3) objectives.
93
European Journal of Social Sciences – Volume 7, Number 3 (2009) 2.2.1. Absolute Standards One group of appraisal methods use absolute standard. This means that employees compare to a standard, and their evaluation is independent of any other employee in a week group (Dessler, 2000). Included in this group are the following methods: the essay appraisal, the critical incident appraisal, the checklist, the graphic rating scale, forced choice and behaviorally anchored rating scales. The essay appraisal: It is the simplest evaluating method in which evaluator writes an explanation about employee’s strength and weakness points, previous performance, positional and suggestion for his (her) improvement at the end of evaluation term. This kind of evaluations usually includes some parts of other systems to cause their flexibility. This method often combines with other methods. In essay appraisal, we attempt to focus on behaviors (Mondy, 2008). The critical incident appraisal: It focuses on key factors which make difference in performing a job efficiently. This method is more credible because it is more related to job and based on individual’s performance than characteristic. The necessity of this system is to try to measure individuals’ performance in term of incidents and special episodes which take place in job performance. These incidents are known as critical incident. In this method, the manager writes down the positive and negative individuals’ performance behavior in evaluation term (Mondy, 2008). The checklist: In this method, the evaluator has a list of situations and statements and compares it with employees. The checklist is a presentation of employee’s characteristics and performance. The results can be quantitative and give weight to characteristics. Answers of checklist are often “Yes” or “No” (Decenzo, 2002). The graphic rating scale: This is the most commonly used method of performance appraisal because they are less time-consuming to develop and administer and allow for quantitative analysis and comparison. It is a scale that lists some characteristics and rang of performance of each individual. Therefore, employees are ranked by determining a score which shows their performance level. The utility of this technique can be enhanced by using it in conjunction with the essay appraisal technique (Mondy, 2008). Forced choice: This method evolved after a great deal of research conducted for the military services during World War II. It is a method in which the evaluator should rank individual work behavior between two or more states. Each state may be favorable or unfavorable. The activity of evaluator is to determine which state has an explanation of employee most (Mondy, 2008). Behaviorally anchored rating scales (BARS): This method replaces traditional numerical anchors tools with behavioral prototypes of real work behaviors. BARS lets evaluator to rank employee based on observable behavioral dimension. The elements of this method are result of combination of major elements of critical incident and adjective rating scale appraisal methods (Wiese, 1998). BARS has five stages (Decenzo, 2002): 1) Generate Critical Incidents, 2) Develop performance dimensions, 3) Relocate incidents, 4) Rating of level of performance for each incident and 5) Development of the final instrument. 2.2.2. Relative Standards In the second general category of appraisal methods, individuals are compared against other individuals. These methods are relative standards rather than absolute measuring device. The most popular of the relative method are group order ranking, individual ranking and paired comparison. Group order ranking: In this method, employees are placed into a particular classification, such as “top one-fifth”. For example, if a rater has 20 employees, only 4 can be in the top fifth and 4 must be relegated to the bottom fifth (Decenzo, 2002). Individual ranking: In this type of appraisal, individuals are ranked from highest to lowest. It is assumed that the difference between the first and second employee is equal to difference between 21st and 22nd employee. In this method, the manager compares each person with others than work standards (Dessler, 2000). 94
European Journal of Social Sciences – Volume 7, Number 3 (2009) Paired comparison: In this method, employees are compared with all others in pairs. The N ⋅ (N − 1) number of comparison is followed as in which N shows the number of employees. After 2 doing all comparisons, the best person is determined for each characteristic (Mondy, 2008). 2.2.3. Objectives The third approach to appraisal makes use of objectives. Employees are evaluated on how well they accomplished a specific set of objectives that have been determined to be critical in the successful completion of their job. This approach is frequently referred to as Management by Objectives (MBO). Management by objectives is a process that converts organizational objectives in to individual objectives. It consists of four steps (Ingham, 1995): goal setting, action planning, self-control and periodic reviews. 2.2.4. 360 Degree Feedback Appraisal 360 degree evaluations are the latest approach to evaluating performance. It is a popular performance appraisal method that involves evaluation input from multiple levels within the firm as well as external sources. There are numerous authors who propose definitions of the 360 degree feedback process. “Feedback from multiple sources or ‘360 degree feedback’ is a performance appraisal approach that relies on the input of an employee’s superiors, colleagues, subordinates, sometimes customers, suppliers and/or spouses” (Yukl and Lepsinger, 1995). In a special edition of Human Resource Management on 360 degree feedback, Tornow (1993) observes that in 360 degree feedback programmes, feedback about a target individual is solicited from significant others using a standardized instrument. Jones and Bearley (1996) refer to 360 degree feedback as the practice of gathering and processing multi-rater assessments on individuals and feeding back the results to the recipients. Hoffman (1995) explains that 360 degree feedback is an approach that gathers behavioral observations from many layers within the organization and includes self-assessment. The 360-degree evaluation can help one person be rated from different sides, different people which can give the wider prospective of the employee’s competencies (Shrestha, 2007). It has been used for human resource development, appraisal and pay decisions (Armstrong, 1998; Stone, 2002).
3. The comparison of Performance Appraisal Methods As mentioned above, each method has advantages and disadvantages which make appraisal method more precise and candid by using of several methods. In order to decision making and determining the best appraisal method, a question should be answered: “The best for what?” It means that the appraisal method should be achieve to which goal or measure which performance? Therefore the manager should determine organization’s goals, objectives and expectations of performance appraisal. Performance appraisal objectives are usually evaluating and nurturing. On the other hand, an effective appraisal method should be reliable, trustworthy and free of error. Moreover, it can be able to compare employees between them and organization’s parts. In our comparison framework, we have seven important criteria which are upheld by experts. These are Training needs evaluation, Coincidence with institutes, Excite staff to be better, Ability to compare, Cost of method and Being free of error. Training needs evaluation: this criterion determines that the appraisal method has the ability to distinguish staff’s training needs. Coincidence with institutes: this criterion determines that a method can coincidence itself with organization’s institutes or not. Excite staff to be better: this criterion focuses on employees, their motivation, creating feedback and assist to human resource planning. 95
European Journal of Social Sciences – Volume 7, Number 3 (2009) Ability to compare: whenever an organization wants to make decision about preferment, eviction, privation, wage and payment and displacing, it should evaluate its employees; compare them and organization’s parts. Therefore an appraisal method should have the ability to compare staffs. Cost of method: it contains cost of procurement, codifying and implementation cost of method. Being free of error: halos effect, leniency, severity and attribution bias. In table 1, the defined appraisal methods were compared in some cases which can be important to selection. Table 1:
Performance appraisal methods' comparison Ranking
MBO
360 Degree Feedback
B A C B A C
BARS
A A B C C B
Forced choice
B- Average
A A B C B C
The graphic rating
A- High
B B C C A C
The checklist
Training needs evaluation Coincidence with institutes Excite staff to be better Ability to compare Cost of method Free of error
The critical incident
The essay
Methods Criteria
C C C B C B
A A B B C B
C C C A A A
A A B A C B
A B A A B A
C- Low
4. A Procedure for the Selection of the Best Appraisal Method Our matrix is extracted from table 1 which A is replaced by 3, B with 2 and C with 1. A1 A2 A3 A4 A5 A6 A7 A8 A9
X 1 ⎛2 X 2 ⎜⎜ 2 X 3 ⎜1 ⎜ X 4 ⎜1 X 5 ⎜3 ⎜ X 6 ⎜⎝ 1
3 3 2 1 3 1 3 3⎞ ⎟ 3 3 3 1 3 1 3 2⎟ 2 2 1 1 2 1 2 3⎟ ⎟ 1 1 2 2 2 3 3 3⎟ 2 1 3 1 1 3 1 2⎟ ⎟ 1 2 1 2 2 3 2 3 ⎟⎠
At first, the scores are normalized (converted) by Linear Scale Transformation which is done by following formulas: x ij x min Benefit : rij = max , Cost : rij = i xi x ij Therefore, our normalized matrix becomes: A1 A 2 A 3 A 4 A 5 A 6 A 7 A8 A9 X 1 ⎛ 0.67 1 1 0.67 0.33 1 0.33 1 1 ⎞ ⎜ ⎟ X 2 ⎜ 0.67 1 1 1 0.33 1 0.33 1 0.67 ⎟ X 3 ⎜ 0.33 0.67 0.67 0.33 0.33 0.67 0.33 0.67 1 ⎟ ⎜ ⎟ X 4 ⎜ 0.33 0.33 0.33 0.67 0.67 0.67 1 1 1 ⎟ X 5 ⎜ 0.33 0.5 1 0.33 1 1 0.33 1 0.5 ⎟ ⎜⎜ ⎟ X 6 ⎝ 0.33 0.33 0.67 0.33 0.67 0.67 1 0.67 1 ⎟⎠
96
European Journal of Social Sciences – Volume 7, Number 3 (2009) To define normalized weight of each criterion, multiple linear regression was introduced to define the straight rank of each criterion, then the resulted ranks were involved in the following formula: n − rj + 1 wj = n ∑ (n − rk + 1) k =1
Where w j is the normalized weight for the j
th
criterion, n is the number of criteria under
consideration and r j is the rank position of the criterion. In table 2, we can see criteria, their rank, weight and their w Table 2:
j
Rank, weight and wj of each criterion
Criteria
Rank ( r j )
Weight ( n − r j + 1)
wj
4 6 5 1 2 3
3 1 2 6 5 4
0.14 0.05 0.1 0.29 0.24 0.19
Training needs evaluation Coincidence with institutes Excite staff to be better Ability to compare Cost of method Free of error
The ranking results were shown in table 3: Table 3:
Final result of SAW analysis
Methods A8 (MBO) A9 (360 Degree Feedback) A6 (BARS) A3 (The checklist) A5 (Forced choice), A7 (Ranking) A2 (The critical incident) A4 (The graphic rating scale) A1 (The essay)
Methods’ grades 0.91 0.87 0.82 0.72 0.66 0.54 0.51 0.4
As shown above, in this example MBO method is in the top of the list, therefore MBO is the best method to apply in putative organization as a performance appraisal method. After that, 360 degree feedback, BARS and checklist are the most suitable methods for appraisal. Forced choice method and Ranking, include group order ranking, individual ranking and paired comparison, are indifferent. It means that if the human resource manager uses each of them for their employees’ performance appraisal, his (or her) consent will be the same. The critical incident, the graphic rating scale and the essay are the worst method to use. They are not even in the mean.
5. Conclusion A part of human resource management is performance appraisal to improve corporate performance, all the while harming the targeted individuals and even undermining the commitment and energy of the survivors. Developing clear, realistic performance standards can also reduce communication problems in performance appraisal feedback among managers, supervisors, and employees. Proposed framework is based on six factors which are training needs evaluation, coincidence with institutes, excite staff to be better, ability to compare, cost of method, and free of error. 97
European Journal of Social Sciences – Volume 7, Number 3 (2009) Along with this selection framework, a comparison of methods is represented with attention to the key features of each performance appraisal method. These features play an important role in successful appraisal of employees. The represented framework helps human resource managers to select their suitable method with attention to methods’ features and apply it in organization and be sure of doing appraisal successfully.
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18]
Armstrong, M. 1998. Performance management: The new realities. London: Institute of personnel and development. Austin, Manila S. & Harkins, Debra A. 2008. Assessing change: can organizational learning “work” for schools? Journal of The Learning Organization, Vol. 15, No. 2, pp. 105-125. Avison, D. and Fitzgerald, G. 2006. Information Systems Development Methodologies, Techniques & Tools. 4th Edition, McGraw Hill. Azar, Fred S. 2000. Multiattribute decision-making: Use of three scoring methods to compare the performance of imaging techniques for breast cancer detection. Department of BioEngineering, Technical report MS-BE-00-01. Balcomb, J. D. and Curtner, A. 2000. MCDM-23: A multi-criteria decision making tool for buildings. Proceedings of International Conference on Sustainable Building, Maastricht, The Netherlands, pp. 219–221. Blackmore, Jacqueline A. 2005. A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management. Vol. 19, No. 3, pp. 218-232. Boswell, W. R. and Boudreau, J. W. 2000. Employee satisfaction with performance appraisals and appraisers: The role of perceived appraisal use. Human Resource Development Quarterly, Vol. 11 No.3, pp. 283. Boswell,W. R. and Boudreau, J.W. 2002. Separating the developmental and evaluative performance appraisal uses. Journal of Business & Psychology, Vol. 16 No.3, pp. 391-412. Ching-Lai, H. and Kwangsun, Y. 1981. Multiple attribute decision making. New York: Springer Verlang, Berlin-Heidelberg. Decenzo, D. A. and Robbins, Stephen P. 2002. Human resource management. John Wiley and sons, 7th edition. Dessler, Gary. 2000. Human Resource Management, Prentice Hall, 8th edition. Dowling, P.J. and Welch, D.E. and Schuler, R.S. 1999. International Human Resource management. South-Western, Cincinnati, OH. Fiala P. and Jablonský J. and Maňas M. 1997. Vícekriteriální rozhodování. VŠE, Praha. Ford, D. K. 2004. Development of a performance appraisal training program for the Rehabilitation Institute of Chicago. Journal of European Industrial Training, Vol. 28, No. 7, pp. 550-563. Freeman, J. 2002. How to improve the effectiveness of performance management and appraisal by overcoming the root cause of the problem. The HRM Guide Network, available from: www.hrmguide.com/performance. Greguras, G. J. and Robie, C. and Schleicher, D. J. and Goff, M. 2003. A field study of the effects of rating purpose on the quality of multisource ratings. Personnel Psychology, Vol. 56 No.1, pp. 1–21. Grifell-Tatjé, E. and Marques-Gou, P. 2008. Internal performance evaluation: the case of bank branches. International Journal of Service Industry Management, Vol. 19, No. 3, pp. 302-324. Harris, M. M. 1994. Rater motivation in the performance appraisal context: A theoretical framework, Journal of Management, 20(4): 737. 98
European Journal of Social Sciences – Volume 7, Number 3 (2009) [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41]
Harris, M. M. and Smith, D. E., and Champagne, D. 1995. A field study of performance appraisal purpose: Research versus administrative-based ratings, Personnel Psychology, Vol. 48 No.1, pp. 151. Hoffman. 1995. Ten reasons why you should be using 360-degree feedback. HR Magazine, Vol. 40 No.4, pp.82-6. Hwang, Ch. L. and Yoon, K. 1981. Multiple Attribute Decision Making. Springer-Verlag, Berlin Heidelberg, New York, pp. 260. Ingham, Terry. 1995. Management by objectives: a lesson in commitment and co-operation. Managing Service Quality, Vol. 5 No. 6, pp. 35-38. Jawahar, I. M. and Williams, C. R. 1997. Where all the children are above average: The performance appraisal purpose effect, Personnel Psychology, Vol. 50 No.4, pp. 905. Jones, J.E and Bearley, W.L. 1996. 360° Feedback: Strategies, Tactics and Techniques for Developing Leaders. HRD Press, Armhurst, MA. Kavanagh, M. 1997. I simply want to know how well I am doing! Group and Organization Management, Vol. 22 No.2, pp.147-148. Khoury, Grace C. and Analoui, Farhad. 2004. Innovative management model for performance appraisal: the case of the Palestinian public universities, Journal of Management Research News, Vol. 27, No. 1/2. Long, P. 1986. Performance Appraisal Revised. London: Institute of Personnel Management. Martey, A.K. 2002. Appraising the performance of library staff in a Ghanaian academic library. Journal of Library Management, Vol. 23, No. 8/9, pp. 403- 416. McAdam, Rodney, Hazlett, Shirley-Ann and Casey, Ch. 2005. Performance management in the UK public sector. International Journal of Public Sector Management, Vol. 18, No. 3, pp. 256273. Mondy, W., and Noe, R. 2008. Human Resource Management. Prentice-Hall, 10th edition. Pettijohn, C. E., Pettijohn, L. S., and d’Amico, M. 2001. Characteristics of performance appraisals and their impact on sales force satisfaction. Journal of Human Resource Development Quarterly, 12(2): 127–146. Pettijohn, C., Pettijohn, L. S., Taylor, A. J., and Keillor, B. D. 2001. Are performance appraisals a bureaucratic exercise or can they be used to enhance sales-force satisfaction and commitment? Journal of Psychology & Marketing, 18(4): 337–364. Randell, G. 1994. Employee appraisal, in Sisson, K. (Ed.), Personnel Management: A Comprehensive Guide to Theory and Practice in Britain, Oxford: Blackwell Publishers Ltd. Robbins, S., Bergman, R., Stagg, I. and Coulter, M. 2000. Management, 2nd ed., Australia, Sydney: Prentice-Hall. Roberts, G. E. 2003. Employee performance appraisal system participation: A technique that works. Journal of Public Personnel Management, 32(1): 89. Roberts, G. E. and Reed, T. (1996). Performance appraisal participation, goal setting and feedback, Review of Public Personnel Administration, 16(4): 29. Seddon, J. 2001. Performance a miracle: praise the workers, The Observer, 11 March. Segella, L.J. 1989. Kiss appraisal woes goodbye. Journal of Supervisory Management, Vol. 34, December, p. 24. Shah, J. B., and Murphy, J. 1995. Performance appraisals for improved productivity. Journal of Management in Engineering, 11(2): 26 Shih, Hsi-An, Chiang, Yun-Hwa and Kim, 2005. Expatriate performance management from MNEs of different national origins. International Journal of Manpower, Vol. 26, No. 2, pp. 157-176. Shrestha, Shruti. 2007. Improving employee performance appraisal method through web-based appraisal support system: system development from the study on Thai companies. IEICE TRANS. INF. & SYST., Vol.E90–D, No.10, pp. 1621-1629. 99
European Journal of Social Sciences – Volume 7, Number 3 (2009) [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54]
Simmons, John and Lovegrove, Ian. 2005. Bridging the conceptual divide: lessons from stakeholder analysis. Journal of Organizational Change Management, Vol. 18, No. 5, pp. 495513. Smith, P. N. 2006. Flexible aggregation in multiple attribute decision making: application to the kuranda range road upgrade, Cybernetics and systems: an international journal, Vol. 37, No. 1, pp. 1–22. Stone, R. 2002. Human Resource Management. 4th ed., Wiley, Brisbane. Taylor, E. K., and Wherry, R. J. 1951. A study of leniency in two rating systems, Personnel Psychology, Vol. 4, pp. 39–47. Tornow, W. 1993. Perceptions or reality: is multi-perspective measurement a means to an end? Human Resource Management, Vol. 32 No.2 and 3, pp. 221-229. Triantaphyllou, E. 2000. Multi-Criteria Decision Making Methods: A Comparative Study. Kluwer Academic Publishers, Dordrechet, pp. 290. Tyc O. and Holoubek J. 2004. Poznámky k problematice vícekriteriálního hodnocení konečného počtu variant. Proceedings of conference Kvantitativné metódy v ekonómii, SPU, Nitra. Verbeeten, Frank H.M. 2008. Performance management practices in public sector organizations. Accounting, Auditing & Accountability Journal, Vol. 21, No. 3, pp. 427-454. Wiese, Danielle S. and Buckley, M. Ronald. 1998. The evolution of the performance appraisal process. Journal of Management History, Vol. 4 No. 3, pp. 233-249. Wilson, J.P. and Western, S. 2000. Performance appraisal: an obstacle to training and development? Journal of European Industrial Training, Vol. 24 No. 7, pp. 384-91. Winston and Creamer, 1997. Performance Appraisal. Available from: http://filebox.vt.edu/users/dgc2/staffinghandbook/performanceappraisalsingleunit. Yukl, G and Lepsinger, R. 1995. How to get the most out of 360° feedback. Training, Vol. 32 No.12, pp.45-50. Zavadskas, E. K. and Kaklauskas, A. 1996. Systemotechnical evaluation of buildings (Pastatų sistemotechninis įvertinimas). Vilnius: Technika, pp. 280 (in Lithuanian).
100
A New Framework for Selection of the Best Performance Appraisal Method Mostafa Jafari Industrial Engineering Department, Iran University of Science & Technology Narmak, Tehran, Iran E-mail: [email protected] Tel: +98-912-1382658 Atieh Bourouni Industrial Engineering Department, Iran University of Science & Technology Narmak, Tehran, Iran E-mail: [email protected] Tel: +98-912-5007402 Roozbeh Hesam Amiri Industrial Engineering Department, Iran University of Science & Technology Narmak, Tehran, Iran E-mail: [email protected] Tel: +98-912-5356487; Fax: +98-21-88814576 Abstract Performance appraisal is one of the most important processes in human resource management, because it has a great effect on both the financial and program components of any organization. There is a verity of methods for the appraisal of employees' performance. Obviously, no method can claim that it has an integrated approach in performance appraisal. Therefore, human resource managers should select an appraisal method which is most efficient in their organizations. In this paper, we propose a framework for the selection of appraisal methods and compare some performance appraisal methods in order to facilitate the selection process for organizations. The value of this framework is that, with use of it, organisations can evaluate their performance appraisal method with respect to the key features of it before implementing any method as well as expending extra costs. This framework is theoretical in nature, and is build based on a review of related literature. Keywords: Human Resource Management, Absolute Standards, Relative Standards, Objectives Approach, Simple Additive Weighting (SAW), Personnel Psychology
1. Introduction One of the basic and major needs in any organization is to evaluate its employee’s performance contiguously and continually to find out whether they improve or not and know their situation in organization. There are many methods for performance appraisal which are suitable for different situations and organization’s characteristics. the major focus of enquiry for several authors (e.g. Freeman, 2002; Seddon, 2001; Wilson and Western, 2000; Randell, 1994; Segall, 1989; Long, 1986) has been examining why, in such a volatile environment, performance appraisal systems tend not to 92
European Journal of Social Sciences – Volume 7, Number 3 (2009) incorporate goals and the direction or strategic needs of the business, or the personal aspirations of the employees and their future development. The HR strategies adopted by an organization (Harris, 1994) may have an effect on the type of appraisal system adopted by an organization. The purpose of this paper is to present a framework for performance appraisal comparison and show how organization can use it. The purpose of this comparison can be summarized in two objects (Avison, 2006): 1. Academic purpose: for better understanding of nature of techniques in order to doing classifications and improving coming knowledge mapping 2. Practical purpose: to select a technique or part of one or more techniques for specific application or the whole of organization. There are many successful (Pettijohn et al., 2001b; Roberts & Reed, 1996; Pettijohn et al., 2001; Roberts, 2003; Shah & Murphy, 1995; Martey, 2002; Austin, 2008; Simmons, 2005; Ford, 2004; Khoury, 2004; Grifell-Tatjé, 2008) and unsuccessful (Blackmore, 2005; Shih, 2005; McAdam, 2005) cases because of suitable or unsuitable performance appraisal methods. In additional, Verbeeten (2008) argued that the definition of clear and measurable goals is positively associated with quantity and quality performance. The results suggest that the behavioral effects of performance management practices are as important as the economic effects in organizations. In this paper, in order to select an appropriate appraisal method, the Simple Additive Weighting (SAW) is used. This method has been described in literature (Ching-Lai, 1981; Kwangsun, 1981; Fiala et al., 1997; Tyc et al., 2004; Azar, 2000; Smith, 2006; Hwang and Yoon, 1981; Zavadskas and Kaklauskas, 1996; Balcomb and Curtner, 2000; Triantaphyllou, 2000).
2. Review of Literature 2.1. Performance Appraisal Employees want to know how well they perform on their jobs. A simple statement, almost axiomatic in any organization, yet it has probably caused more controversy, applied research and practical advice than any other assertion in the history of management writing and thinking (Kavanagh, 1997). Performance appraisal has many definitions. “Performance appraisal” is a process within the overall performance management process (Dowling et al., 1999), and is defined as “the evaluation of an individual’s work performance in order to arrive at objective personnel decisions” (Robbins et al., 2000). 2.1.1. Performance Appraisal Purpose Taylor and Wherry (1951) proposed that ratings collected for administrative purposes would be more lenient than ratings collected for research or developmental purposes. Over the last few years, there are no new empirical works which have been conducted (Greguras et al., 2003; Harris et al., 1995) but a useful review of it has been written (Jawahar & Williams, 1997). While the majority of the research on performance appraisal purpose has focused on the rater, some work has also been conducted on rater effects (Boswell & Boudreau, 2000, 2002). There are numerous methods to measure employee’s performance appraisal but some of these methods are not suitable in some cases. Effective appraisal systems should address clarity, openness, and fairness; recognize productivity through rewards; and be cognizant of appraiser leadership qualities (Winston & Creamer, 1997). 2.2. Performance Appraisal Methods Decenzo and Robbins (1998) denominate that there are three existent approaches for measuring performance appraisal. These are (1) absolute standards (2) relative standards and (3) objectives.
93
European Journal of Social Sciences – Volume 7, Number 3 (2009) 2.2.1. Absolute Standards One group of appraisal methods use absolute standard. This means that employees compare to a standard, and their evaluation is independent of any other employee in a week group (Dessler, 2000). Included in this group are the following methods: the essay appraisal, the critical incident appraisal, the checklist, the graphic rating scale, forced choice and behaviorally anchored rating scales. The essay appraisal: It is the simplest evaluating method in which evaluator writes an explanation about employee’s strength and weakness points, previous performance, positional and suggestion for his (her) improvement at the end of evaluation term. This kind of evaluations usually includes some parts of other systems to cause their flexibility. This method often combines with other methods. In essay appraisal, we attempt to focus on behaviors (Mondy, 2008). The critical incident appraisal: It focuses on key factors which make difference in performing a job efficiently. This method is more credible because it is more related to job and based on individual’s performance than characteristic. The necessity of this system is to try to measure individuals’ performance in term of incidents and special episodes which take place in job performance. These incidents are known as critical incident. In this method, the manager writes down the positive and negative individuals’ performance behavior in evaluation term (Mondy, 2008). The checklist: In this method, the evaluator has a list of situations and statements and compares it with employees. The checklist is a presentation of employee’s characteristics and performance. The results can be quantitative and give weight to characteristics. Answers of checklist are often “Yes” or “No” (Decenzo, 2002). The graphic rating scale: This is the most commonly used method of performance appraisal because they are less time-consuming to develop and administer and allow for quantitative analysis and comparison. It is a scale that lists some characteristics and rang of performance of each individual. Therefore, employees are ranked by determining a score which shows their performance level. The utility of this technique can be enhanced by using it in conjunction with the essay appraisal technique (Mondy, 2008). Forced choice: This method evolved after a great deal of research conducted for the military services during World War II. It is a method in which the evaluator should rank individual work behavior between two or more states. Each state may be favorable or unfavorable. The activity of evaluator is to determine which state has an explanation of employee most (Mondy, 2008). Behaviorally anchored rating scales (BARS): This method replaces traditional numerical anchors tools with behavioral prototypes of real work behaviors. BARS lets evaluator to rank employee based on observable behavioral dimension. The elements of this method are result of combination of major elements of critical incident and adjective rating scale appraisal methods (Wiese, 1998). BARS has five stages (Decenzo, 2002): 1) Generate Critical Incidents, 2) Develop performance dimensions, 3) Relocate incidents, 4) Rating of level of performance for each incident and 5) Development of the final instrument. 2.2.2. Relative Standards In the second general category of appraisal methods, individuals are compared against other individuals. These methods are relative standards rather than absolute measuring device. The most popular of the relative method are group order ranking, individual ranking and paired comparison. Group order ranking: In this method, employees are placed into a particular classification, such as “top one-fifth”. For example, if a rater has 20 employees, only 4 can be in the top fifth and 4 must be relegated to the bottom fifth (Decenzo, 2002). Individual ranking: In this type of appraisal, individuals are ranked from highest to lowest. It is assumed that the difference between the first and second employee is equal to difference between 21st and 22nd employee. In this method, the manager compares each person with others than work standards (Dessler, 2000). 94
European Journal of Social Sciences – Volume 7, Number 3 (2009) Paired comparison: In this method, employees are compared with all others in pairs. The N ⋅ (N − 1) number of comparison is followed as in which N shows the number of employees. After 2 doing all comparisons, the best person is determined for each characteristic (Mondy, 2008). 2.2.3. Objectives The third approach to appraisal makes use of objectives. Employees are evaluated on how well they accomplished a specific set of objectives that have been determined to be critical in the successful completion of their job. This approach is frequently referred to as Management by Objectives (MBO). Management by objectives is a process that converts organizational objectives in to individual objectives. It consists of four steps (Ingham, 1995): goal setting, action planning, self-control and periodic reviews. 2.2.4. 360 Degree Feedback Appraisal 360 degree evaluations are the latest approach to evaluating performance. It is a popular performance appraisal method that involves evaluation input from multiple levels within the firm as well as external sources. There are numerous authors who propose definitions of the 360 degree feedback process. “Feedback from multiple sources or ‘360 degree feedback’ is a performance appraisal approach that relies on the input of an employee’s superiors, colleagues, subordinates, sometimes customers, suppliers and/or spouses” (Yukl and Lepsinger, 1995). In a special edition of Human Resource Management on 360 degree feedback, Tornow (1993) observes that in 360 degree feedback programmes, feedback about a target individual is solicited from significant others using a standardized instrument. Jones and Bearley (1996) refer to 360 degree feedback as the practice of gathering and processing multi-rater assessments on individuals and feeding back the results to the recipients. Hoffman (1995) explains that 360 degree feedback is an approach that gathers behavioral observations from many layers within the organization and includes self-assessment. The 360-degree evaluation can help one person be rated from different sides, different people which can give the wider prospective of the employee’s competencies (Shrestha, 2007). It has been used for human resource development, appraisal and pay decisions (Armstrong, 1998; Stone, 2002).
3. The comparison of Performance Appraisal Methods As mentioned above, each method has advantages and disadvantages which make appraisal method more precise and candid by using of several methods. In order to decision making and determining the best appraisal method, a question should be answered: “The best for what?” It means that the appraisal method should be achieve to which goal or measure which performance? Therefore the manager should determine organization’s goals, objectives and expectations of performance appraisal. Performance appraisal objectives are usually evaluating and nurturing. On the other hand, an effective appraisal method should be reliable, trustworthy and free of error. Moreover, it can be able to compare employees between them and organization’s parts. In our comparison framework, we have seven important criteria which are upheld by experts. These are Training needs evaluation, Coincidence with institutes, Excite staff to be better, Ability to compare, Cost of method and Being free of error. Training needs evaluation: this criterion determines that the appraisal method has the ability to distinguish staff’s training needs. Coincidence with institutes: this criterion determines that a method can coincidence itself with organization’s institutes or not. Excite staff to be better: this criterion focuses on employees, their motivation, creating feedback and assist to human resource planning. 95
European Journal of Social Sciences – Volume 7, Number 3 (2009) Ability to compare: whenever an organization wants to make decision about preferment, eviction, privation, wage and payment and displacing, it should evaluate its employees; compare them and organization’s parts. Therefore an appraisal method should have the ability to compare staffs. Cost of method: it contains cost of procurement, codifying and implementation cost of method. Being free of error: halos effect, leniency, severity and attribution bias. In table 1, the defined appraisal methods were compared in some cases which can be important to selection. Table 1:
Performance appraisal methods' comparison Ranking
MBO
360 Degree Feedback
B A C B A C
BARS
A A B C C B
Forced choice
B- Average
A A B C B C
The graphic rating
A- High
B B C C A C
The checklist
Training needs evaluation Coincidence with institutes Excite staff to be better Ability to compare Cost of method Free of error
The critical incident
The essay
Methods Criteria
C C C B C B
A A B B C B
C C C A A A
A A B A C B
A B A A B A
C- Low
4. A Procedure for the Selection of the Best Appraisal Method Our matrix is extracted from table 1 which A is replaced by 3, B with 2 and C with 1. A1 A2 A3 A4 A5 A6 A7 A8 A9
X 1 ⎛2 X 2 ⎜⎜ 2 X 3 ⎜1 ⎜ X 4 ⎜1 X 5 ⎜3 ⎜ X 6 ⎜⎝ 1
3 3 2 1 3 1 3 3⎞ ⎟ 3 3 3 1 3 1 3 2⎟ 2 2 1 1 2 1 2 3⎟ ⎟ 1 1 2 2 2 3 3 3⎟ 2 1 3 1 1 3 1 2⎟ ⎟ 1 2 1 2 2 3 2 3 ⎟⎠
At first, the scores are normalized (converted) by Linear Scale Transformation which is done by following formulas: x ij x min Benefit : rij = max , Cost : rij = i xi x ij Therefore, our normalized matrix becomes: A1 A 2 A 3 A 4 A 5 A 6 A 7 A8 A9 X 1 ⎛ 0.67 1 1 0.67 0.33 1 0.33 1 1 ⎞ ⎜ ⎟ X 2 ⎜ 0.67 1 1 1 0.33 1 0.33 1 0.67 ⎟ X 3 ⎜ 0.33 0.67 0.67 0.33 0.33 0.67 0.33 0.67 1 ⎟ ⎜ ⎟ X 4 ⎜ 0.33 0.33 0.33 0.67 0.67 0.67 1 1 1 ⎟ X 5 ⎜ 0.33 0.5 1 0.33 1 1 0.33 1 0.5 ⎟ ⎜⎜ ⎟ X 6 ⎝ 0.33 0.33 0.67 0.33 0.67 0.67 1 0.67 1 ⎟⎠
96
European Journal of Social Sciences – Volume 7, Number 3 (2009) To define normalized weight of each criterion, multiple linear regression was introduced to define the straight rank of each criterion, then the resulted ranks were involved in the following formula: n − rj + 1 wj = n ∑ (n − rk + 1) k =1
Where w j is the normalized weight for the j
th
criterion, n is the number of criteria under
consideration and r j is the rank position of the criterion. In table 2, we can see criteria, their rank, weight and their w Table 2:
j
Rank, weight and wj of each criterion
Criteria
Rank ( r j )
Weight ( n − r j + 1)
wj
4 6 5 1 2 3
3 1 2 6 5 4
0.14 0.05 0.1 0.29 0.24 0.19
Training needs evaluation Coincidence with institutes Excite staff to be better Ability to compare Cost of method Free of error
The ranking results were shown in table 3: Table 3:
Final result of SAW analysis
Methods A8 (MBO) A9 (360 Degree Feedback) A6 (BARS) A3 (The checklist) A5 (Forced choice), A7 (Ranking) A2 (The critical incident) A4 (The graphic rating scale) A1 (The essay)
Methods’ grades 0.91 0.87 0.82 0.72 0.66 0.54 0.51 0.4
As shown above, in this example MBO method is in the top of the list, therefore MBO is the best method to apply in putative organization as a performance appraisal method. After that, 360 degree feedback, BARS and checklist are the most suitable methods for appraisal. Forced choice method and Ranking, include group order ranking, individual ranking and paired comparison, are indifferent. It means that if the human resource manager uses each of them for their employees’ performance appraisal, his (or her) consent will be the same. The critical incident, the graphic rating scale and the essay are the worst method to use. They are not even in the mean.
5. Conclusion A part of human resource management is performance appraisal to improve corporate performance, all the while harming the targeted individuals and even undermining the commitment and energy of the survivors. Developing clear, realistic performance standards can also reduce communication problems in performance appraisal feedback among managers, supervisors, and employees. Proposed framework is based on six factors which are training needs evaluation, coincidence with institutes, excite staff to be better, ability to compare, cost of method, and free of error. 97
European Journal of Social Sciences – Volume 7, Number 3 (2009) Along with this selection framework, a comparison of methods is represented with attention to the key features of each performance appraisal method. These features play an important role in successful appraisal of employees. The represented framework helps human resource managers to select their suitable method with attention to methods’ features and apply it in organization and be sure of doing appraisal successfully.
References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18]
Armstrong, M. 1998. Performance management: The new realities. London: Institute of personnel and development. Austin, Manila S. & Harkins, Debra A. 2008. Assessing change: can organizational learning “work” for schools? Journal of The Learning Organization, Vol. 15, No. 2, pp. 105-125. Avison, D. and Fitzgerald, G. 2006. Information Systems Development Methodologies, Techniques & Tools. 4th Edition, McGraw Hill. Azar, Fred S. 2000. Multiattribute decision-making: Use of three scoring methods to compare the performance of imaging techniques for breast cancer detection. Department of BioEngineering, Technical report MS-BE-00-01. Balcomb, J. D. and Curtner, A. 2000. MCDM-23: A multi-criteria decision making tool for buildings. Proceedings of International Conference on Sustainable Building, Maastricht, The Netherlands, pp. 219–221. Blackmore, Jacqueline A. 2005. A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management. Vol. 19, No. 3, pp. 218-232. Boswell, W. R. and Boudreau, J. W. 2000. Employee satisfaction with performance appraisals and appraisers: The role of perceived appraisal use. Human Resource Development Quarterly, Vol. 11 No.3, pp. 283. Boswell,W. R. and Boudreau, J.W. 2002. Separating the developmental and evaluative performance appraisal uses. Journal of Business & Psychology, Vol. 16 No.3, pp. 391-412. Ching-Lai, H. and Kwangsun, Y. 1981. Multiple attribute decision making. New York: Springer Verlang, Berlin-Heidelberg. Decenzo, D. A. and Robbins, Stephen P. 2002. Human resource management. John Wiley and sons, 7th edition. Dessler, Gary. 2000. Human Resource Management, Prentice Hall, 8th edition. Dowling, P.J. and Welch, D.E. and Schuler, R.S. 1999. International Human Resource management. South-Western, Cincinnati, OH. Fiala P. and Jablonský J. and Maňas M. 1997. Vícekriteriální rozhodování. VŠE, Praha. Ford, D. K. 2004. Development of a performance appraisal training program for the Rehabilitation Institute of Chicago. Journal of European Industrial Training, Vol. 28, No. 7, pp. 550-563. Freeman, J. 2002. How to improve the effectiveness of performance management and appraisal by overcoming the root cause of the problem. The HRM Guide Network, available from: www.hrmguide.com/performance. Greguras, G. J. and Robie, C. and Schleicher, D. J. and Goff, M. 2003. A field study of the effects of rating purpose on the quality of multisource ratings. Personnel Psychology, Vol. 56 No.1, pp. 1–21. Grifell-Tatjé, E. and Marques-Gou, P. 2008. Internal performance evaluation: the case of bank branches. International Journal of Service Industry Management, Vol. 19, No. 3, pp. 302-324. Harris, M. M. 1994. Rater motivation in the performance appraisal context: A theoretical framework, Journal of Management, 20(4): 737. 98
European Journal of Social Sciences – Volume 7, Number 3 (2009) [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39] [40] [41]
Harris, M. M. and Smith, D. E., and Champagne, D. 1995. A field study of performance appraisal purpose: Research versus administrative-based ratings, Personnel Psychology, Vol. 48 No.1, pp. 151. Hoffman. 1995. Ten reasons why you should be using 360-degree feedback. HR Magazine, Vol. 40 No.4, pp.82-6. Hwang, Ch. L. and Yoon, K. 1981. Multiple Attribute Decision Making. Springer-Verlag, Berlin Heidelberg, New York, pp. 260. Ingham, Terry. 1995. Management by objectives: a lesson in commitment and co-operation. Managing Service Quality, Vol. 5 No. 6, pp. 35-38. Jawahar, I. M. and Williams, C. R. 1997. Where all the children are above average: The performance appraisal purpose effect, Personnel Psychology, Vol. 50 No.4, pp. 905. Jones, J.E and Bearley, W.L. 1996. 360° Feedback: Strategies, Tactics and Techniques for Developing Leaders. HRD Press, Armhurst, MA. Kavanagh, M. 1997. I simply want to know how well I am doing! Group and Organization Management, Vol. 22 No.2, pp.147-148. Khoury, Grace C. and Analoui, Farhad. 2004. Innovative management model for performance appraisal: the case of the Palestinian public universities, Journal of Management Research News, Vol. 27, No. 1/2. Long, P. 1986. Performance Appraisal Revised. London: Institute of Personnel Management. Martey, A.K. 2002. Appraising the performance of library staff in a Ghanaian academic library. Journal of Library Management, Vol. 23, No. 8/9, pp. 403- 416. McAdam, Rodney, Hazlett, Shirley-Ann and Casey, Ch. 2005. Performance management in the UK public sector. International Journal of Public Sector Management, Vol. 18, No. 3, pp. 256273. Mondy, W., and Noe, R. 2008. Human Resource Management. Prentice-Hall, 10th edition. Pettijohn, C. E., Pettijohn, L. S., and d’Amico, M. 2001. Characteristics of performance appraisals and their impact on sales force satisfaction. Journal of Human Resource Development Quarterly, 12(2): 127–146. Pettijohn, C., Pettijohn, L. S., Taylor, A. J., and Keillor, B. D. 2001. Are performance appraisals a bureaucratic exercise or can they be used to enhance sales-force satisfaction and commitment? Journal of Psychology & Marketing, 18(4): 337–364. Randell, G. 1994. Employee appraisal, in Sisson, K. (Ed.), Personnel Management: A Comprehensive Guide to Theory and Practice in Britain, Oxford: Blackwell Publishers Ltd. Robbins, S., Bergman, R., Stagg, I. and Coulter, M. 2000. Management, 2nd ed., Australia, Sydney: Prentice-Hall. Roberts, G. E. 2003. Employee performance appraisal system participation: A technique that works. Journal of Public Personnel Management, 32(1): 89. Roberts, G. E. and Reed, T. (1996). Performance appraisal participation, goal setting and feedback, Review of Public Personnel Administration, 16(4): 29. Seddon, J. 2001. Performance a miracle: praise the workers, The Observer, 11 March. Segella, L.J. 1989. Kiss appraisal woes goodbye. Journal of Supervisory Management, Vol. 34, December, p. 24. Shah, J. B., and Murphy, J. 1995. Performance appraisals for improved productivity. Journal of Management in Engineering, 11(2): 26 Shih, Hsi-An, Chiang, Yun-Hwa and Kim, 2005. Expatriate performance management from MNEs of different national origins. International Journal of Manpower, Vol. 26, No. 2, pp. 157-176. Shrestha, Shruti. 2007. Improving employee performance appraisal method through web-based appraisal support system: system development from the study on Thai companies. IEICE TRANS. INF. & SYST., Vol.E90–D, No.10, pp. 1621-1629. 99
European Journal of Social Sciences – Volume 7, Number 3 (2009) [42] [43] [44] [45] [46] [47] [48] [49] [50] [51] [52] [53] [54]
Simmons, John and Lovegrove, Ian. 2005. Bridging the conceptual divide: lessons from stakeholder analysis. Journal of Organizational Change Management, Vol. 18, No. 5, pp. 495513. Smith, P. N. 2006. Flexible aggregation in multiple attribute decision making: application to the kuranda range road upgrade, Cybernetics and systems: an international journal, Vol. 37, No. 1, pp. 1–22. Stone, R. 2002. Human Resource Management. 4th ed., Wiley, Brisbane. Taylor, E. K., and Wherry, R. J. 1951. A study of leniency in two rating systems, Personnel Psychology, Vol. 4, pp. 39–47. Tornow, W. 1993. Perceptions or reality: is multi-perspective measurement a means to an end? Human Resource Management, Vol. 32 No.2 and 3, pp. 221-229. Triantaphyllou, E. 2000. Multi-Criteria Decision Making Methods: A Comparative Study. Kluwer Academic Publishers, Dordrechet, pp. 290. Tyc O. and Holoubek J. 2004. Poznámky k problematice vícekriteriálního hodnocení konečného počtu variant. Proceedings of conference Kvantitativné metódy v ekonómii, SPU, Nitra. Verbeeten, Frank H.M. 2008. Performance management practices in public sector organizations. Accounting, Auditing & Accountability Journal, Vol. 21, No. 3, pp. 427-454. Wiese, Danielle S. and Buckley, M. Ronald. 1998. The evolution of the performance appraisal process. Journal of Management History, Vol. 4 No. 3, pp. 233-249. Wilson, J.P. and Western, S. 2000. Performance appraisal: an obstacle to training and development? Journal of European Industrial Training, Vol. 24 No. 7, pp. 384-91. Winston and Creamer, 1997. Performance Appraisal. Available from: http://filebox.vt.edu/users/dgc2/staffinghandbook/performanceappraisalsingleunit. Yukl, G and Lepsinger, R. 1995. How to get the most out of 360° feedback. Training, Vol. 32 No.12, pp.45-50. Zavadskas, E. K. and Kaklauskas, A. 1996. Systemotechnical evaluation of buildings (Pastatų sistemotechninis įvertinimas). Vilnius: Technika, pp. 280 (in Lithuanian).
100
