timer Asked: Apr 12th, 2020

Question Description

Here is my response to the required post where I needed to consider how I would combine the statistical scores.

Post by Day 4, an explanation of how you might combine multiple raters’ evaluations of an individual on a measure of job performance. Provide a specific example of this use. Then explain the psychometric implications of using multiple raters. Finally, explain steps you could take to improve the reliability of a multi-rater assessment. Support your response using the Learning Resources and the current literature.

Combine Multiple Raters’ Evaluations of an Individual on a Measure of Job Performance and Example.

About 34% of all organizations are using multiple-rater feedback evaluations with their employees (Aamodt, 2016). Many are using multi-rater evaluations to help develop their leaders (Van Velsor, Leslie, & Fleenor, 1997). Multi-rater evaluations have its benefits in that you get input from different sources (Lauby, 2016). Instead of having one point of view, the individual has feedback from many people that can provide varying insights based on their experiences with that person (Lauby, 2016). One example of a combined approach would be one that a past employer used. As a global insurance company that had a four-year leadership development program, they invested in their leaders. Year one was called Management Essentials, and year two was Avanza. Year three of the program was called the Management Development Program (MDP), and year four was the International Management Development Program. Everyone wanted to make it to year four since you got to go to Spain for two weeks of training. Each of these programs required a year-long commitment and two weeks of residency. During MDP, a combined self-report DiSC and multi-rater 360 evaluation were done. The assessments were completed in parts. Part one was a DiSC assessment, and part two was a 360-degree review. This allowed for the individual to self-assess their personality type and get feedback on how others saw them. The goal was to have the leaders make personalized development plans to work on their strengths and weaknesses and understanding how they like to work with others. DiSC (n.d.) helped the leaders identify their personality type, which could be a combination of the four categories:

Dominance - Person places emphasis on accomplishing results, the bottom line, confidence

Influence - Person places emphasis on influencing or persuading others, openness, relationships

Steadiness - Person places emphasis on cooperation, sincerity, dependability

Conscientiousness - Person places emphasis on quality and accuracy, expertise, competency

Moreover, it aligned this information with behaviors. For example, I am a Di, meaning I am a high dominant and influencer. Some of the behaviors associated with this are seeing the big picture, can be blunt, accepts challenges, gets straight to the point, shows enthusiasm, is optimistic and likes to collaborate.

Psychometric Implications

McMahon (2019) defined psychometrics as a "field of study devoted to tests and measurements." Psychometric tests like the ones we discussed above are used to evaluate an individual's capabilities (McMahon, 2019). Some implications of psychometrics are the wether they are single rater or multi-rater assessments. McCrae (1994) stated that test scores could only be considered or understood when the trait and the method are taken into consideration. So, in other words, psychometric implications can be the trait and the method used to evaluate that trait. An important consideration is that measuring traits can not be done via self-reports (McCrae, 1994).

Improving the Reliability of a Multi-rater Assessment

Reliability is the degree a test score or evolution is free of errors and is consistent (Aamodt, 2016). In multi-rater assessments, reliability can be reached when raters are familiar with what is being measured and aggregated (McCrae, 1994). When considering raters, we should look at their familiarity with the person being rated and the role that they fill. How often do they interact with the person being rated? This is why it is recommended that the person being rated ask people that work with them regularly and know them to give them feedback. Another consideration may be time. If I have not worked with a person in the last year, I would not ask them to evaluate me. In a study by Rothstein (1990), they found that reliability is changed with time. The increase in reliability by having time to observe will improve the accuracy of the ratings given (Rothstein, 1990).


Aamodt, M. (2016). Industrial/Organizational Psychology: An Applied Approach.Eighth Edition. Belmont, CA: Wadsworth, ©2010.

DiSC Overview. (n.d.). Retrieved from

Lauby, S. (2016). Multi-Rater Reviews: Advantages and Disadvantages. The SHRM Blog. Retrieved from

McCrae, R. R. (1994). The counterpoint of personality assessment: Self-reports and observer ratings. Assessment, 1(2), 159–172.

McMahon, M. (2019). Psychometrics. Salem Press Encyclopedia. Retrieved from

Rothstein, H. R. (1990). Interrater reliability of job performance ratings: Growth to asymptote level with increasing opportunity to observe. Journal of Applied Psychology, 75(3), 322–327.
Retrieved from the Walden Library databases.

Van Velsor, E., Leslie, J. B., & Fleenor, J. (1997). Choosing 360: A guide to evaluating multi-rater feedback instruments for management development. Center for Creative Leadership.

Student has agreed that all tutoring, explanations, and answers provided by the tutor will be used to help in the learning process and in accordance with Studypool's honor code & terms of service.

This question has not been answered.

Create a free account to get help with this and any other question!

Similar Questions
Related Tags

Brown University

1271 Tutors

California Institute of Technology

2131 Tutors

Carnegie Mellon University

982 Tutors

Columbia University

1256 Tutors

Dartmouth University

2113 Tutors

Emory University

2279 Tutors

Harvard University

599 Tutors

Massachusetts Institute of Technology

2319 Tutors

New York University

1645 Tutors

Notre Dam University

1911 Tutors

Oklahoma University

2122 Tutors

Pennsylvania State University

932 Tutors

Princeton University

1211 Tutors

Stanford University

983 Tutors

University of California

1282 Tutors

Oxford University

123 Tutors

Yale University

2325 Tutors