Assignment 3: Evaluating Selection Techniques and Employee Performance
Part I: Rating Errors (see Ch. 7). Think of 5 professors that you had at Lynn University (do not use me). Write down their
names in places below and rate each professor on the five dimensions. Your answers will be confidential (in fact, you
don’t even need to name the professors if you do not wish to).
Professor A (Name):
Knowledge of Subject
Fairness of Grades
Organization
Speaking Skills
Interest in Students
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
1
1
1
1
1
2
2
2
2
2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
Professor B (Name):
Knowledge of Subject
Fairness of Grades
Organization
Speaking Skills
Interest in Students
Professor C (Name):
Knowledge of Subject
Fairness of Grades
Organization
Speaking Skills
Interest in Students
Professor D (Name):
Knowledge of Subject
Fairness of Grades
Organization
Speaking Skills
Interest in Students
Professor E (Name):
Knowledge of Subject
Fairness of Grades
Organization
Speaking Skills
Interest in Students
Part II: Paired-Comparison Technique (see Ch. 7). Now, use a paired-comparison technique for ranking performance as
was discussed in your text. With this technique, rather than ranking several employees at one time, employees are
compared one pair at a time.
Step 1. List the names of the professors that you used above.
Professor A:
Professor B:
Professor C:
Professor D:
Professor E:
Step 2. Write the names of the professors in the appropriate spaces below. Once these names have been written, circle
the name of the professor in each pair that you thought was the better of the two
A.
B.
A.
C.
A.
D.
A.
E.
B.
C.
B.
D.
B.
E.
C.
D
C.
E
D.
E.
Step 3. Count the number of times you circled each name and place that number of the chart below:
Professor
Times Chosen
Professor A:
Professor B:
Professor C:
Professor D:
Professor E:
When you are finished, look at the pattern of ratings that you made in Part I. Did your ratings suffer from any of the
rating errors discussed in your text? In around two pages, discuss rating errors in-depth. Then, compare and contrast the
paired-comparison technique of evaluating employee performance with the rank order method and the forced
distribution method. Please submit this document and your two-page summary to Canvas.
Chapter 6
Understand
how to determine the reliability
of a test and the factors that affect test
reliability
Understand the five ways to validate a test
Learn how to find information about tests
Understand how to determine the utility of a
selection test
Be able to evaluate a test for potential legal
problems
Understand how to use test scores to make
personnel selection decisions
© 2017 Cengage Learning. All Rights Reserved.
Are Reliable
Are Valid
The
Based on a job analysis (content validity)
Predict work-related behavior (criterion validity)
Reduce the Chance of a Legal Challenge
Face valid
Don’t invade privacy
Don’t intentionally discriminate
Minimize adverse impact
extent to which a score from a test is
consistent and free from errors of
measurement
Methods of Determining Reliability
Are Cost Effective
Test-retest (temporal stability)
Alternate forms (form stability)
Internal reliability (item stability)
Scorer reliability
Cost to purchase/create
Cost to administer
Cost to score
Measures
temporal stability
Administration
Same applicants
Same test
Two testing periods
Scores
at time one are correlated with scores
at time two
Correlation should be above 0.70
Sources
of measurement errors
Characteristic or attribute being measured may
change over time
Reactivity
Carry over effects
Practical
problems
Time consuming
Expensive
Inappropriate for some types of tests
Two
forms of the same test are developed,
and to the highest degree possible, are
equivalent in terms of content, response
process, and statistical characteristics
One form is administered to examinees, and
at some later date, the same examinees take
the second form
Scores
Difficult
to develop
Defines
Content
sampling errors
Time
sampling errors
Split-Half
method (most common)
Test items are divided into two equal parts
Scores for the two parts are correlated to get a
measure of internal reliability.
Spearman-Brown
prophecy formula:
(2 x split half reliability) ÷ (1 + split-half
reliability)
from the first form of test are
correlated with scores from the second form
If the scores are highly correlated, the test
has form stability
measurement error strictly in terms
of consistency or inconsistency in the
content of the test.
Used when it is impractical to administer two
separate forms of a test.
With this form of reliability the test is
administered only once and measures item
stability.
(2
(1
If
x split-half correlation)
+ split-half correlation)
we have a split-half correlation of 0.60,
the corrected reliability would be:
(2 x 0.60) ÷ (1 + 0.60) = 1.2 ÷ 1.6 = 0.75
Used
Cronbach’s
Coefficient Alpha
Used with ratio or interval data.
Kuder-Richardson
when human judgment of performance
is involved in the selection process
Refers to the degree of agreement between 2
or more raters
Formula
Used for test with dichotomous items (yes-no
true-false)
The
higher the reliability of a selection test
the better. Reliability should be 0.70 or
higher
Reliability can be affected by many factors
If a selection test is not reliable, it is useless
as a tool for selecting individuals
Definition:
The degree to which inferences
from scores on tests or assessments are
justified by the evidence
Common Ways to Measure
Content Validity
Criterion Validity
Construct Validity
The
extent to which test items sample the
content that they are supposed to measure
In
industry the appropriate content of a test
of test battery is determined by a job
analysis
Criterion
validity refers to the extent to
which a test score is related to some
measure of job performance called a
criterion
Established using one of the following
research designs:
Uses
current employees
Range
restriction can be a problem
Concurrent Validity
Predictive Validity
Validity Generalization
Correlates
test scores with future behavior
the problem of range restriction
May not be practical
Reduces
Method
Validity
Method
Validity
Structured Interview
0.57
Experience
0.27
Cognitive ability
0.51
Situational judgment tests
0.26
Biodata
0.51
Conscientiousness
0.24
Job knowledge
0.45
Unstructured interviews
0.20
Work samples (verbal)
0.48
Integrity tests
0.18
Assessment centers
0.38
Interest inventories
0.10
College grades
0.32
Handwriting analysis
0.02
References
0.29
Projective personality tests
0.00
Validity
Generalization is the extent to which
a test found valid for a job in one location is
valid for the same job in a different location
The key to establishing validity
generalization is meta-analysis and job
analysis
The
extent to which a test actually measures
the construct that it purports to measure
Is concerned with inferences about test
scores
Determined by correlating scores on a test
with scores from other test
The
extent to which a test appears to be job
related
Reduces the chance of legal challenge
Increasing face validity
Workbook Exercise 6.1
The
degree to which a selection device
improves the quality of a personnel system,
above and beyond what would have occurred
had the instrument not been used.
You
Taylor-Russell
You
have many job openings
have many more applicants than
openings
You have a valid test
The job in question has a high salary
The job is not easily performed or easily
trained
Proportion
Tables
of Correct Decisions
The Brogden-Cronbach-Gleser Model
Estimates
the percentage of future
employees that will be successful
Three components
Validity
Base rate (successful employees ÷ total
employees)
Selection ratio (hired ÷ applicants)
50%
10
0.05
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
0.95
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.50
0.10
0.58
0.57
0.56
0.55
0.54
0.53
0.53
0.52
0.51
0.51
0.50
0.20
0.67
0.64
0.61
0.59
0.58
0.56
0.55
0.54
0.53
0.52
0.51
0.30
0.74
0.71
0.67
0.64
0.62
0.60
0.58
0.56
0.54
0.52
0.51
0.40
0.82
0.78
0.73
0.69
0.66
0.63
0.61
0.58
0.56
0.53
0.52
0.50
0.88
0.84
0.76
0.74
0.70
0.67
0.63
0.60
0.57
0.54
0.52
0.60
0.94
0.90
0.84
0.79
0.75
0.70
0.66
0.62
0.59
0.54
0.52
0.70
0.98
0.95
0.90
0.85
0.80
0.75
0.70
0.65
0.60
0.55
0.53
0.80
1.0
0.99
0.95
0.90
0.85
0.80
0.73
0.67
0.61
0.55
0.53
0.90
1.0
1.0
0.99
0.97
0.92
0.86
0.78
0.70
0.62
0.56
0.53
x
x
x
x
II
x
8
x
7
x
6
the Taylor-Russell Tables what
percentage of future employees would be
successful?
0.00
I
x
4
3
x
1
x
1
2
Proportion of Correct Decisions With Test
(10 + 11)
÷
(5 + 10 + 4 + 11)
Quadrant II Quadrant IV
Quadrants I+II+III+IV
= 21 ÷ 30 = 0.70
Baseline of Correct Decisions
5 + 10
÷
Quadrants I + II
= 15 ÷ 30 = 0.50
x
x
x
x
5
6
Test Score (x)
7
Successful employees ÷ Total employees
Quadrants I + II
Quadrants I+II+III+IV
x
x
4
Baseline of Correct Decisions
x
x
x
3
(Correct rejections + correct acceptances) ÷ Total
employees
Quadrant II Quadrant IV
Quadrants I+II+III+IV
III
x
Proportion of Correct Decisions With Test
x
x
x
x
2
x
x
x
x
x
IV
5
x
we have
a test validity of 0.40
a selection ratio of 0.30
a base rate of 0.50
Using
r
9
C
r
i
t
e
r
i
o
n
Suppose
8
9
10
5 + 10 + 4 + 11
Quadrants I+II+III+IV
9
x
8
x
7
Workbook Exercise 6.3
x
6
5
I
4
x
x
3
x
x
x
x
II
x
x
x
2
1
x
x
x
x
x
x
x
x
IV
1
2
3
III
4
5
6
7
8
9
Test Scores
Proportion of Correct Decisions With Test
(8 + 6)
÷
(4 + 8 + 6 + 2)
Quadrant II Quadrant IV
Quadrants I+II+III+IV
= 14 ÷ 20 = 0.70
Baseline of Correct Decisions
4+8
÷
Quadrants I + II
= 12 ÷ 20 = 0.60
Selection ratio
The ratio between the number of openings to the
number of applicants
Validity coefficient
4+8+6+2
Quadrants I+II+III+IV
Base rate of current performance
The percentage of employees currently on the job
who are considered successful.
Sdy
The difference in performance (measured in dollars)
between a good and average worker (workers one
standard deviation apart)
Gives
an estimate of utility by estimating the
amount of money an organization would save
if it used the test to select employees.
Savings =(n) (t) (r) (SDy) (m) – cost of testing
n = Number of employees hired per year
t = average tenure
r = test validity
SDy = standard deviation of performance in
dollars
m = mean standardized predictor score of
selected applicants
For
example, we administer a test of mental
ability to a group of 100 applicants and hire
the 10 with the highest scores. The average
score of the 10 hired applicants was 34.6,
the average test score of the other 90
applicants was 28.4, and the standard
deviation of all test scores was 8.3. The
desired figure would be:
(34.6 – 28.4) ÷ 8.3 = 6.2 ÷ 8.3 = ?
You
administer a test of mental ability to a
group of 150 applicants, and hire 35 with the
highest scores. The average score of the 35
hired applicants was 35.7, the average test
score of the other 115 applicants was 24.6,
and the standard deviation of all test scores
was 11.2. The desired figure would be:
m
0.00
0.90
0.20
0.80
0.35
0.70
0.50
0.60
0.64
0.50
0.80
0.40
0.97
0.30
1.17
0.20
1.40
0.10
1.76
0.05
2.08
Suppose:
(35.7 – 24.6) ÷ 11.2 = ?
SR
1.00
we hire 10 auditors per year
the average person in this position stays 2 years
the validity coefficient is 0.40
the average annual salary for the position is $30,000
we have 50 applicants for ten openings.
Workbook Exercise 6.2
Our utility would be:
(10 x 2 x 0.40 x $12,000 x 1.40) – (50 x 10) =
$133,900
Selection Ratio
80%
1. Selection Ratio
Base rate
Validity
% of future successful
employees
250 ÷ 500 =
0.50
800 ÷ 1000 =
0.80
0.40
89%
r
0.05
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
0.95
0.00
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.10
0.85
0.85
0.84
0.83
0.83
0.82
0.82
0.81
0.81
0.81
0.80
0.20
0.90
0.89
0.87
0.86
0.85
0.84
0.84
0.83
0.82
0.81
0.81
0.30
0.94
0.92
0.90
0.89
0.88
0.87
0.86
0.84
0.83
0.82
0.81
0.40
0.96
0.95
0.93
0.92
0.90
0.89
0.88
0.86
0.85
0.83
0.82
0.50
0.98
0.97
0.96
0.94
0.93
0.91
0.90
0.88
0.86
0.84
0.82
0.60
0.99
0.99
0.98
0.96
0.95
0.94
0.92
0.90
0.87
0.84
0.83
0.70
1.0
1.0
0.99
0.98
0.97
0.96
0.94
0.92
0.89
0.85
0.83
0.80
1.0
1.0
1.0
1.0
0.99
0.98
0.96
0.94
0.91
0.87
0.84
0.90
1.0
1.0
1.0
1.0
1.0
1.0
0.99
0.97
0.94
0.88
0.84
Components:
We will hire 250 people
The average person in this position stays 4 years
The validity coefficient is 0.30
The average annual salary for the position is $70,000
We have 500 applicants for 250 openings.
Our utility would be:
(250 x 4 x 0.30 x $28,000 x 0.80) – (500 x 15) =
$6,720,000 - $7,500 = $6,712,500
Test
We will hire 250 people
The average person in this position stays 4 years
The validity coefficient is 0.40
The average annual salary for the position is $70,000
We have 500 applicants for 200 openings.
Our utility would be:
(250 x 4 x 0.40 x $28,000 x 0.80) – (500 x 10) =
$8,960,000 - $5,000 = $8,955,000
SR
m
1.00
0.00
$8,955,000
0.90
0.20
0.80
0.35
$6,712,500
0.70
0.50
0.60
0.64
$2,242,500
0.50
0.80
0.40
0.97
0.30
1.17
0.20
1.40
0.10
1.76
0.05
2.08
Utility
New Test: Reilly Statistical Logic
Test
Old Test: Tribble Math
Savings
Method
Components:
Validity
Method
Validity
Cognitive ability
0.39
References
0.18
Biodata
0.36
Grades
0.16
Structured
Interview
0.34
Integrity tests
0.13
Assessment centers
0.28
Agreeableness
0.13
Work samples
0.26
Unstructured interviews
0.11
Experience
0.22
Interest inventories
0.10
Conscientiousness
0.21
Emotional stability
0.08
Situational
judgment
0.20
Openness
0.06
1.
Selection Ratio
0.50
Base rate
0.80
Validity
% of future successful
employees
0.34
0.87 (round r down)
0.89 (round r up)
Selection Ratio
80%
0.05
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
0.95
0.00
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.80
0.10
0.85
0.85
0.84
0.83
0.83
0.82
0.82
0.81
0.81
0.81
0.80
0.20
0.90
0.89
0.87
0.86
0.85
0.84
0.84
0.83
0.82
0.81
0.81
0.30
0.94
0.92
0.90
0.89
0.88
0.87
0.86
0.84
0.83
0.82
0.81
0.40
0.96
0.95
0.93
0.92
0.90
0.89
0.88
0.86
0.85
0.83
0.82
0.50
0.98
0.97
0.96
0.94
0.93
0.91
0.90
0.88
0.86
0.84
0.82
0.60
0.99
0.99
0.98
0.96
0.95
0.94
0.92
0.90
0.87
0.84
0.83
0.70
1.0
1.0
0.99
0.98
0.97
0.96
0.94
0.92
0.89
0.85
0.83
0.80
1.0
1.0
1.0
1.0
0.99
0.98
0.96
0.94
0.91
0.87
0.84
0.90
1.0
1.0
1.0
1.0
1.0
1.0
0.99
0.97
0.94
0.88
0.84
Components:
r
We will hire 200 people
The average person in this position stays 4 years
The observed validity coefficient is 0.34
The average annual salary for the position is $60,000
We have 500 applicants for 200 openings.
Our utility would be:
(250 x 4 x 0.34 x $28,000 x 0.80) – (500 x 15) =
$7,616,000 - $12,500 = $7,603,500
Components:
We will hire 250 people
The average person in this position stays 4 years
The validity coefficient is 0.11
The average annual salary for the position is $70,000
We have 500 applicants for 250 openings.
Our utility would be:
(250 x 4 x 0.11 x $28,000 x 0.80) – (500 x 25) =
$2,464,000 - $12,500 = $2,451,500
Test
Utility
New Test: Structured Interview $7,603,500
Old Test: Unstructured
Interview
Savings
Measurement
$5,152,000
Bias
Technical aspects of the test
A test is biased if there are group differences in
test scores (e.g., race, gender) that are
unrelated to the construct being measured (e.g.,
integrity)
Predictive
$2,451,500
Bias
A test is fair if people of equal probability of
success on a job have an equal chance of being
hired
Number
of applicants
Number hired
Selection ratio
Male
50
20
0.40
Female
30
10
0.33
Number
of applicants
Number hired
Selection ratio
Male
40
20
0.50
Female
20
4
0.20
0.33/0.40 = 0.83 > 0.80 (no adverse impact)
0.20/0.50 = 0.40 < 0.80 (adverse impact)
1. Compute Standard Deviation
1. Compute Standard Deviation
female applicants male applicants
x
x total hired
total applicants total applicants
2. Multiply standard deviation by 2
3. Compute expected number of females to be
hired (female applicants/total applicants) x total
hired
4. Compute confidence interval (expected ± 2 SD)
5. Determine if number of females hired falls
within the confidence interval
Single-Group
Differential
Validity
Test predicts for one group but not another
Very rare
Validity
Test predicts for both groups but better for one
Also very rare
10 40
x x 20 = .20 x .80 x 20 = 3.2 = 1.79
50 50
2. Multiply standard deviation by 2 = 1.79 * 2 =
3.58
3. Compute expected number of females to be
hired (10/50) x 20 = 0.2 x 20 = 4
4. Compute confidence interval (0.42 4
7.58)
5. Determine if number of females hired falls
within the confidence interval
Unadjusted
Top-down Selection
Scores
Banding
A
Advantages
Who
Passing
Higher
quality of selected applicants
Objective decision making
Disadvantages
Less flexibility in decision making
Adverse impact = less workforce diversity
Ignores measurement error
Assumes test score accounts for all the
variance in performance (Zedeck, Cascio,
Goldstein & Outtz, 1996).
Applicant
Sex
Score
Omar
M
98
Eric
M
80
Mia
F
70 (passing score)
Morris
M
69
Tammy
F
58
Drew
M
40
“performance first” hiring formula
Applicant
Drew
Eric
Lenny
Omar
Mia
Morris
Test Score
99
98
91
90
88
87
will perform at an acceptable level?
A passing score is a point in a distribution of
scores that distinguishes acceptable from
unacceptable performance (Kane, 1994).
Uniform
Sex
M
M
M
M
F
M
Guidelines (1978) Section 5H:
Passing scores should be reasonable and
consistent with expectations of acceptable
proficiency
Advantages
Increased flexibility in decision making
Less adverse impact against protected groups
Disadvantages
Lowered utility
Can be difficult to set
Top-down
(most inflexibility)
of “Three” or “Five”
Traditional banding
Expectancy bands
SEM banding (standard error of
measurement)
Rules
Applicant
Drew
Eric
Lenny
Omar
Mia
Morris
Testing differences between scores for statistical
significance.
Pass/Fail
bands (most flexibility)
Based
Applicant
Drew
Eric
Lenny
Omar
Jerry
Morris
Sex
M
M
M
M
F
M
Test Score
99
98
91
90
88
87
Test Score
Probability
A
522 – 574
85%
B
483 – 521
75%
C
419 – 482
66%
D
0 – 418
56%
Test Score
99
98
91
90
88
87
on expert judgment
ease
Administrative
A
Band
Sex
M
M
M
M
F
M
e.g., college grading system
e.g., level of job qualifications
compromise between the top-down and
passing scores approach.
It takes into account that tests are not
perfectly reliable (error).
Applicant
Armstrong
Glenn
Grissom
Aldren
Ride
Irwin
Carpenter
Gibson
McAuliffe
Carr
Teshkova
Jamison
Pogue
Resnick
Anders
Borman
Lovell
Slayton
Kubasov
Compromise between top-down selection and
passing scores
Based on the concept of the standard error of
measurement
To compute you need the standard deviation and
reliability of the test
Standard error =
SD 1 reliability
Band is established by multiplying 1.96 times the
standard error
Applicant
Sex
Score
Band 1
Band 2
Clancy
m
97
x
hired
King
m
95
x
x
Band 3
hired
Band 4
Koontz
m
94
x
x
x
hired
Follot
m
92
x
x
x
x
hired
Saunders
m
88
x
x
x
x
Crichton
m
87
x
x
x
x
Sanford
m
84
x
x
x
Dixon
m
80
Wolfe
m
75
Grisham
m
72
Clussler
m
70
Turow
m
65
Cornwell
f
64
Clark
f
61
= 12.8 * 0.316
Brown
f
60
= 4.04
Sex
m
m
m
m
f
m
m
m
f
m
f
m
m
f
m
m
m
m
f
Score
99
98
94
92
88
87
84
80
75
72
70
65
64
61
60
58
57
55
53
Band 1 Band 2 Band 3 Band 4
x
hired
hired
hired
x
x
hired
hired
x
x
x
hired
x
x
x
x
x
hired
x
x
x
Band 5
Fixed
Sliding
x
Diversity-based
Band = 4.04 * 1.96 = 7.92 ~ 8
Applicant Sex
Score
Omar
M
98
Eric
M
80
Mia
F
70 (cutoff)
Morris
M
69
Tammy
F
58
Drew
M
40
Females and minorities are given preference
when selecting from within a band.
12.8 1 .90
12.8 .10
Helps
reduce adverse impact, increase
workforce diversity,and increase perceptions
of fairness (Zedeck et al., 1996).
Allows you to consider secondary criteria
relevant to the job (Campion et al., 2001).
Lose
valuable information
the quality of people selected
Sliding bands may be difficult to apply in the
private sector
Banding without minority preference may not
reduce adverse impact
Narrow
Lower
Consequences
Banding
The
has generally been approved by the
courts
Bridgeport Guardians v. City of Bridgeport, 1991
Chicago Firefighters Union Local No.2 v. City of
Chicago, 1999
Officers for Justice v. Civil Service Commission,
1992
Minority
Sample Test Information
Reliability = 0.80
Mean = 72.85
Standard deviation = 9.1
The Band
Band = Standard error *
1.96
Band = 4.07 * 1.96 =
7.98 ~ 8
The Standard Error
SD 1 reliability
Example 1
9.1 1 .80
9.1 .20
=
9.1 * 0.447
We have four openings
We would like to hire
more females
Example 2
= 4.07
company should have established rules
and procedures for making choices within a
band
Applicants should be informed about the use
and logic behind banding in addition to
company values and objectives (Campion et
al., 2001).
Preference
bands are preferred
or errors in selection
Criterion space covered by selection device
Reliability of selection device
Validity evidence
Diversity issues
Reliability = 0.90
Standard deviation =
12.8
Workbook Exercise 6.4
1. Standard Error
2. Band
3.06
3.06 * 1.96 = 6.0 points
3. Hire using nonsliding band
McCoy
Crane
Robinette
Carmichael
4. Hire using sliding band
Carmichael
McCoy
Ross
Crane
5. Hire using a passing score of 80
Applicant
Sex
Score
McCoy
Crane
Robinette
Schiff
m
m
m
m
97
95
94
94
Band 1
x
x
x
x
Carmichael f
Carver
Ross
91
m
f
x
89
89
hire
Cutter
Kincaid
Cabot
Stone
m
f
f
m
88
87
86
86
Lewin
f
85
Shore
Branch
Sack
m
m
m
83
80
78
Band 2
Band 3
Band 4
Band 5
x
x
x
x
hire
x
x
x
hired
x
x
x
hired
hire
x
hired
hired
hired
x
x
hire
x
hired
7.91 1 .85
= 7.91 * .387
= 3.06
Band = 3.06 * 1.96 = 6
Carmichael
McCoy
Ross
Crane
Should
the top scores on a test always get
the job?
Diversity
Efforts
Applied
Case Study: Thomas A. Edison’s
Employment Test
To increase diversity, it is often legal to consider race or
gender as a factor in selecting employees. Although legal,
do you think it is ethical that race or gender be a factor in
making an employment decision? How much of a role
should it play?
Is it ethical to hire a person with a lower test score
because he or she seems to be a better personality fit for
an organization?
If an I/O psychologist is employed by a company that
appears to be discriminating against Hispanics, is it ethical
for her to stay with the company? What ethical obligations
does she have?
Know how to create a performance appraisal
instrument
Know how to administer a performance appraisal
system
Understand the problems associated with
performance ratings
Be able to conduct a performance appraisal
review
Understand how to legally terminate an
unproductive employee
Learn how to monitor the legality of a
performance appraisal system
Chapter 7
© 2017 Cengage Learning. All Rights Reserved.
Are job-related and based on a job analysis
Are properly administered
Formal
Standardized
Not too complicated
Provide specific feedback
Focus on behaviors rather than traits
Ratings are documented
Use appropriate raters
Raters are trained
All relevant raters are used
Raters used must be relevant and qualified
Determine the sources to be used in appraising
performance.
Assumptions (Coens & Jenkins, 2000)
Supervisors and raters are fair, objective, and unbiased
Supervisors and employees will not try to manipulate
performance ratings to get desired outcomes
Raters can adequately distinguish an individual’s
performance from the situational constraints (the system)
Coens & Jenkins believe that if these assumptions are
not true, you should not conduct performance appraisals
Determine the best method to accomplish your
goal
Create
a task-force that includes all levels in
the organization
Determine why you are evaluating
performance (your goal)
Most systems have no goal
90% of systems do not work (SHRM survey)
Will improving individual performance improve
organizational performance?
Identify
environmental and cultural variables
that could affect the system
Feedback
Personnel
and training (65%)
decisions
Raises (86%)
Promotions (45%)
Termination decisions (30%)
Research
Legal
or certification reasons
7
Upper
Direct
management
supervisor
Peers
Subordinates
Support
staff
Customers/the
public
Vendors
Self
Use
28% of organizations (Mercer Consulting, 2005)
SHRM Survey
18% for nonexempt positions
29% for exempt positions
32% for executive level positions
Suggestions
4-10 raters
15 minutes to complete
Provide feedback within 6 weeks of evaluation
Agreement Between
Two supervisors
Two peers
Two subordinates
Supervisor and peer
Supervisor and subordinate
Supervisor and self
Peer and subordinate
Peer and self
Correlation
0.50
0.37
0.30
0.34
0.22
0.22
0.22
0.19
Would
Workbook Exercise 7.1
you implement a 360-degree feedback
system in your organization?
Decision 1
Goal Focus (Results)
Prevent crimes from
occurring
Finish shift without
personal injury
Have arrests and
citations stand up in
court
Trait Focus
Honesty
Courtesy
Responsibility
Dependability
Cooperation
Competency Focus
Report writing skills
Driving skills
Public speaking skills
Knowledge of the law
Task focus
Crime prevention
Arrest procedures
Court testimony
Use of vehicle
Workbook Exercise 7.2
Employee
Comparisons
Rank order
Paired comparison
Forced distribution
Decision 2
Results
Quantity
Accidents
Absenteeism
Tardiness
Subjective
Ratings
Rating Dimension
Rank
Order
Comparison
Forced Distribution
Paired
Employee
Knowledge
Dependability
Quality
Total
Barrino
1
1
1
1.00
Underwood
2
3
2
2.33
Hicks
3
2
3
2.67
Sparks
4
5
4
4.33
Cook
5
4
5
4.67
Aniston
Employees
Green
Briscoe
Rey
Logan
Ceretta
Paired Comparisons
Green
Briscoe
Green
Rey
Green
Logan
Green
Ceretta
Briscoe
Rey
Briscoe
Logan
Briscoe
Ceretta
Rey
Logan
Rey
Ceretta
Logan
Ceretta
Stone
Scoring
Green
Briscoe
Rey
Logan
Ceretta
4
3
1
2
0
Portman
Theron
Lawrence
Roberts
Kunis
Bullock
Stewart
Jolie
10%
20%
40%
20%
10%
Terrible
Below
Average
Average
Above
Average
Excellent
Quantity
Workbook Exercise 7.3
of work
of work
Attendance
Quality
Absenteeism
Tardiness
Time theft
Safety
Graphic
Rating Scales
Methods
Behavior-Based
Behavioral checklists
Behaviorally anchored rating scales (BARS)
Behavioral observation scales (BOS)
Behavioral expectation scales (BES)
Mixed-standard scales
Forced choice scales
Job
Patrol
2
Poor 1
4
5 Excellent
2
3
4
5 Excellent
4
5 Excellent
4
5 Excellent
making
Poor 1
Use
3
activity
Decision
5—
knowledge
Poor 1
2
3
of weapons
Poor 1
A customer wanted to deposit a large amount of money. The teller
explained to the customer that he could earn more interest on a
money market account than with a savings account
2
3
Radio Procedures
Behavioral Elements
4—
A customer applied for a new auto loan and had and E/I too high
for approval. The employee suggested a lower-priced auto with a
lower payment to reduce his E/I
3—
When a customer called, this employee accurately answered her
question about finance charges
2—
When a customer came to the bank for a loan, this employee had
to search for instructions and kept the customer waiting
A customer wanted to cash a large check. The teller said that it
could not be cashed but did not realize that it was all right as long
as the customer had that amount in her account
1—
___
___
___
___
___
___
___
___
___
___
Always uses proper codes and signals when sending information
Always understands codes and signals when receiving information
Voice is clear and easy to understand in normal situations
Voice is clear, easy to understand, and does not indicate panic in high stress situations
Always follows proper radio procedures
Always monitors the proper channels
Always knows the location of all district officers
Never communicates improper information over the radio
Keeps control informed of current status
Treats communications officers with respect and courtesy
Dimension Rating
___
___
___
___
___
5
4
3
2
1
Consistently exceed requirements, no improvements needed
Exceeds most requirements
Usually meets requirements, acceptable performance
Usually meets most requirements, but needs much improvement
Does not meet minimum requirements, needs immediate and extensive improvement
Rating
Most Least
1 a)
.
Teller is always on time (neutral)
b)
Teller smiles at each customer (excellent)
2 a)
.
Teller usually cross-sells (excellent)
c)
Teller constantly argues with other employees (P)
2.
Teller smiles at customers (A)
3.
Teller asks customers how their families are doing (E)
4.
Teller helps other employees when possible (A)
5.
Teller is always friendly to other employees (E)
6.
Teller asks customers what they want (P)
Teller is never short at end of the day (poor)
c)
b)
1.
Teller keeps work area neat and orderly (poor)
Teller is friendly to other employees (neutral)
Items 1, 4, and 5 are from the Employee Relations dimension
Items 2, 3, and 6 and from the Customer Relations dimension
Performance
Job Knowledge
1. ___ Is aware of current interest rates.
2. ___ Offers suggestions to customers about how they can
make the most interest
3. ___ Knows various strategies for converting IRAs.
Employee Relations
1. ___ Offers to help other employees when own workload
is down.
2. ___ Praises other employee when they do well
based (Extent to which
expectations have been met)
Normative
based (Comparison to other
employees)
above average
average
Frequency
Dimension: Refers to Customers by Name
___ Much better than other tellers
___ Better than other tellers
___ The same as other tellers
___ Worse than other tellers
___ Much worse than other tellers
exceeds expectations
meets expectations
based
always
sometimes
Dimension: Refers to Customers by Name
___ Always
___ Almost always
___ Often
___ Seldom
___ Never
Workbook Exercise 7.4
Dimension: Refers to Customers by Name
___ Greatly exceeds expectations
___ Exceeds expectations
___ Meets expectations
___ Falls below expectations
___ Falls well below expectations
Forces supervisor to
look for behaviors
Aids in recall during
evaluation
Provides examples to
use when reviewing
performance
Provides concrete
data to support
personnel decisions
Who
Stored
Recalled
Behavior
1
Observed
100
90
80
70
60
50
40
30
20
10
0
5
is the toughest rater?
Sgt. Esterhaus
Sgt. Jablonski
Capt. Furillo
is a low rating
is the highest rating
Sgt. Esterhous
Bobby
Andy
Joe
Hill
5
1
3
5
1
3
4
2
Hunter
Washington 3
2
Coffey
Norman
Lucy
Buntz
Bates
Harry
Capt. Furillo
Renko
Howard
Neal
Sgt. Jablonski
4
3
4
4
2
2
3
3
5
3
3
1
Schnitz
2
4
1
Ray
Callentano
1
3
5
Larue
1
4
4
3
5
2
2
4
4
2
5
2
Henry
Tina
Goldblum
Russo
Michael
Belker
Sgt. Esterhous
Bobby
Andy
Neal
Joe
5
1
3
5
1
3
4
2
Hunter
Washington 3
Coffey
Buntz
Bates
Harry
Garibaldi
2
3
4
3
4
4
2
2
3
3
5
3
3
1
Schnitz
2
4
1
Ray
Callentano
1
3
5
Larue
1
4
4
3
5
2
2
4
4
2
5
2
Henry
Tina
Goldblum
Russo
Michael
Belker
Details
The
source of the information
First
impressions
behaviors
Unusual behaviors
Extreme behaviors
Behavior consistent with opinion
Recent
3
Leo
John
is the toughest rater?
Sgt. Esterhaus
Sgt. Jablonski
Capt. Furillo
Capt. Furillo
Hill
Norman
Lucy
Sgt. Jablonski
Renko
Howard
Who
Leo
John
Garibaldi
3
3
Are
examples of poor or excellent
performance
Provide behavioral examples
Are usually collected through logs
Employee performance record can be used
Dimension
Knowledge of
Law
Poor
Performance
Excellent
Performance
3/17 Arrest procedure
ruled OK by court
Report Writing
1/25 accident report 1/30 well written
contained five typos report
2/07 well written
report
Patrol
Performance
3/16 complaint from
motorist that officer
did not help
Read
critical incident logs
and review objective data
Halo
Obtain
traffic citations
arrests
complaints
commendations
accidents
leniency
strictness
central tendency
Location
or Horns
Restriction
Range
Andy Sipowicz
Errors
proximity
contrast
John Sullivan
Job Knowledge
1
2
3
4
5
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Judgment
1
2
3
4
5
Bobby Simone
5
Ty Davis
Job Knowledge
1
2
3
4
5
Job Knowledge
1
2
3
4
Report Writing
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Judgment
1
2
3
4
5
James Martinez
Maurice “Bosco” Boscorelli
Job Knowledge
1
2
3
4
5
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Judgment
1
2
3
4
5
Frank Pembleton
Jon Baker
Job Knowledge
1
2
3
4
5
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Judgment
1
2
3
4
5
Tim Bayless
Frank Poncherello
Job Knowledge
1
2
3
4
5
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Judgment
1
2
3
4
5
Meldrick Lewis
Barry Baricza
Job Knowledge
1
2
3
4
5
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Judgment
1
2
3
4
5
Lenny Briscoe
Job Knowledge
1
2
3
4
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Mike Logan
5
Raters
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
often don’t agree
Why?
Common rating errors
Different standards and comparisons
Observation of different behaviors
Ed Green
Job Knowledge
1
2
3
4
5
Report Writing
1
2
3
4
5
Patrol Activity
1
2
3
4
5
Citizen Relations
1
2
3
4
5
Judgment
1
2
3
4
5
Shift
Geographic
area
Supervisor
Peers
Equipment
Incentives
Training
Season/time
of year
Workbook Exercise 7.5
Day Shift
Day Shift
Area A Malloy (33)
Reed (34)
Area B
Evening Shift Night Shift
Romano (19)
Hooker (23)
Boscarelli (17) Fife (14)
Yokus (23)
Taylor (15)
Area C Briscoe (15)
Green (17)
Area D Cagney (7)
Lacey (9)
128
Friday (8)
Gannon (8)
85
Sullivan (11)
Davis (10)
Bates (12)
Coffey (3)
68
Hill (6)
Renko (4)
Poncherello
(5)
Baker (2)
33
155
Evaluation
Ho (10)
Williams (9)
102
57
Malloy
Reed
Area B
Boscarelli (- 3.90)
Fife
Yokus
(+2.10)
Taylor
20.9
(+ 0.20)
(+ 1.20)
Briscoe
Green
( 0.00)
(- 1.00)
Area C
Area D
Cagney
Lacey
314
Night Shift
(+1.40)
Romano (-1.80)
Ho
(- 1.60)
(+2.40)
Hooker (+2.20)
Williams (-2.60)
31.6
20.8
11.6
(-1.80)
Sullivan
(+0.20)
Davis
16.8
(- 1.10)
(+ 0.90)
Hill
Renko
155
Friday
Gannon
(+ 0.30)
(+ 0.30)
13.8
Bates
Coffey
(+ 0.60)
(- 1.40)
85
(+ 5.80)
(- 3.20)
68
6.2
Poncherello (+ 2.00)
Baker
(- 1.00)
5.4
102
33
3.0
57
Frequency
v. informal
Since my last report, this employee has reached rockbottom and shows signs of starting to dig
His men would follow him anywhere, but only out of
morbid curiosity
I would not allow this employee to breed
This associate is really not so much of a has-been but more
of a definitely won’t be
He would be out of his depth in a parking lot puddle
This young lady has delusions of adequacy
This employee should go far – and the sooner the better
This employee is depriving a village somewhere of an idiot
He sets low personal standards and then consistently fails
to achieve them
128
7.7
11.0
8.1
semi-annual
annual
more frequent during probation?
Formal
Evening Shift
Area A
Workbook Exercise 7.6
314
Employee input prior
to the meeting
High level of
employee
participation
Helpful supervisor
attitude
Focus on behaviors
rather than traits
Identification of
solutions to problems
Mutual setting of
specific goals
Consistent application
of standards
Rater is familiar with
the employee’s work
Employee
Preparation
Give blank copy of forms to employee
Have the employee rate him/herself
Scheduling
Schedule at least an hour for the performance
review
Schedule the review in a private, neutral
location
Supervisor
Preparation
Review ratings and reasons for the ratings
Determine goals for the review
Quiet,
private location
rapport
Have employee discuss self-ratings
Discuss your ratings
Identify problems and determine solutions
Set goals
Establish
Workbook Exercise 7.7
Public
Sector
Probationary period
Violation of rules
Inability to perform
Layoff
Private
Sector
Employment-at-Will
A
Employee
rule must exist
must have known about the rule
Violation of the rule must be proven
Rule must be equally enforced
Progressive discipline must occur
Punishment must fit the offense
A reasonable attempt must be made to change
behavior
Standard
of performance
concrete
reasonable
job-related
failure to meet the standard
critical incident logs
work samples
performance ratings
Progressive
Ensure
Concept
Limitations
Documented
Counseling/training
Oral warning
Written warning
Probation
Suspension with pay
Suspension without pay
Transfer
Reduction in pay
Demotion
Termination
federal or state law
public policy or interest
implied contracts
covenants of good faith and fair dealing
At-Will
Statements
discipline
that the legal process has been
followed
Decide how much help you want to offer the
employee
Choose a neutral, private place
Plan enough time for the meeting
Schedule the meeting for the beginning of
the week
Get
to the point
state the reasons for the decision
Express gratitude for the employee’s efforts
(if sincere)
Offer whatever assistance you wish to
provide
Rationally
Severance pay
Recommendation
Perform
administrative duties
employee to gather personal belongings
and leave
Ask
Maintain
Protect
your self-esteem
yourself from guilt by reviewing the
facts
Help
other employees cope
Be honest with them
This will avoid “water cooler whispers”
Avoid
negative public statements about the
fired employee’s character
Notify all organizational units affected by
employee’s departure
Has the organization provided training?
Are there gender, race, ethnicity, disability, or
age differences in the ratings?
Do employees with similar ratings receive similar
rewards or consequences?
Are there gender, race, ethnicity, disability, or
age differences in the opportunities given to
improve performance?
Are there employees who historically have
received high ratings suddenly being given a low
rating?
Applied
The
Case Study: Kohl’s Department Store
At-Will Doctrine
What are other ethical dilemmas in the at-will
doctrine?
Which argument do you agree with: that of
critics of the doctrine or supporters?
Are there other limitations that should be
imposed on the doctrine, other than those
outlined in your textbook, that would make the
doctrine more ethical, and therefore, fairer?
Do you agree that having different laws for
public and private sector employees (at-will
versus for just cause) is fair?
Purchase answer to see full
attachment