Part One
Framework for
Organizational
Assessment
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
CHAPTER 1
INDIVIDUAL DIFFERENCES
THAT INFLUENCE
PERFORMANCE AND
EFFECTIVENESS
What Should We Assess?
Kevin R. Murphy
Assessment in organizations can be carried out for a variety of
purposes, many with high stakes for both individuals and organizations. The stakes can be particularly high when assessments
are used to make decisions about personnel selection and placement or about advancement and development of individuals
once they have been hired. If assessments focus on traits, attributes, or outcomes that are not relevant to success and effectiveness, both organizations and individuals may end up making
poor decisions about the fit between people and jobs. If assessments are appropriately focused but poorly executed (perhaps
the right attributes are measured, but they are measured with
very low levels of reliability and precision), these assessments
may lead to poor decisions on the parts of both organizations
and individuals.
In this chapter, I focus on broad questions about the content
of assessments (for example, What sorts of human attributes
should assessments attempt to measure?) and say very little
about the execution of assessments (the choice of specific tests
3
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
4
Handbook of Workplace Assessment
or assessment methods, for example) or even the use of assessment data. My discussion is general rather than specific, focusing
on general dimensions of assessment (whether to assess cognitive abilities or broad versus narrow abilities, for example) rather
than on the specifics of assessment for a particular job (say, the
best set of assessments for selecting among applicants for a job as
a firefighter).
This chapter provides a general foundation for many of the
chapters that follow. It sets the stage by discussing broad dimensions of individual differences that are likely to be relevant for
understanding performance, effectiveness, and development in
the workplace. The remaining chapters in Part One start addressing more specific questions that arise when attempting to assess
these dimensions. Chapter Two reviews the range of methods that
can be used to assess the quality of measures, and Chapters Three
through Eight provide a more detailed examination of specific
domains: cognitive abilities, personality, background and experience, knowledge and skill, physical and psychomotor skills and
abilities, and competencies.
Part Two of this book discusses assessment for selection, promotion, and development, and Parts Three and Four deal with
strategic assessment programs and with emerging trends and
issues.
I begin this chapter by noting two general strategies for determining what to assess in organizations: one that focuses on the
work and the other that focuses on the person. The personoriented approaches are likely to provide the most useful guidance in determining what to assess for the purpose of selection
and placement in entry-level jobs, and work-oriented assessments
might prove more useful for identifying opportunities for and
challenges to development and advancement.
Two Perspectives for Determining What to Assess
A number of important decisions must be made in determining what to assess, but the first is to determine whether the focus
should be on the person or the work. That is, it is possible to
build assessment strategies around the things people do in organizations in carrying out their work roles (work oriented) or
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
5
around the characteristics of individuals that influence what they
do and how well they do it in the workplace (person oriented).
For example, it is common to start the process of selecting and
deciding how to use assessments with a careful job analysis on the
assumption that a detailed examination of what people do, how
they do it, and how their work relates to the work of others will
shed light on the knowledge, skills, abilities, and other attributes
(KSAOs) required to perform the job well. An alternative strategy is to start by examining the individual difference domains
that underlie most assessments and to use knowledge about the
structure and content of those domains to drive choices about
what to assess.
The choice of specific assessments is a three-step process that
starts with choosing between a broadly person-oriented or workoriented approach, then making choices about the domains
within each approach to emphasize (for example, whether to focus
on cognitive ability or on personality), and finally narrowing down
the choice of specific attributes (say, spatial ability) and assessment
methods (perhaps computerized tests). As I noted earlier, this
chapter focuses on the first two of these steps.
Work-Oriented Strategies
Different jobs involve very different tasks and duties and may call
on very different sorts of knowledge or skill, but it is possible to
describe the domain of work in general terms that are relevant
across a wide range of jobs and organizations; such a wide-ranging
description provides the basis for worker-oriented strategies
for determining what to assess. Starting in the late 1960s, considerable progress was made in the development of structured
questionnaires and inventories for analyzing jobs (for example,
the Position Analysis Questionnaire; McCormick, Jeanneret, &
Mecham, 1972). These analysis instruments in turn helped to
define the contents and structure of the O*NET (Occupational
Information Network; Peterson, Mumford, Borman, Jeanneret, &
Fleishman, 1999) Generalized Work Activities Taxonomy, arguably the most comprehensive attempt to describe the content
and nature of work. Table 1.1 lists the major dimensions of this
taxonomy.
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
6
Handbook of Workplace Assessment
Table 1.1. O*NET Generalized Work Activities
Information input
Looking for and receiving job-related
information
Identifying and evaluating job-relevant
information
Mental processes
Information and data processing
Reasoning and decision making
Work output
Performing physical and manual work
activities
Performing complex and technical activities
Interacting with others
Communicating and interacting
Coordinating, developing, managing, and
advising
Administering
If you were to ask, “What do people do when they work?”
Table 1.1 suggests that the answer would be that they gather
information, process and make sense of that information, make
decisions, perform physical and technical tasks, and interact with
others. The specifics might vary across jobs, but it is reasonable to
argue that Table 1.1 provides a general structure for describing
jobs of all sorts and for describing, in particular, what it is that
people do at work. Each of these major dimensions can be broken down into subdimensions (which are shown in this table),
most of which can be broken down even further (for example,
administering can be broken down into performing administrative activities, staffing organizational units, and monitoring and
controlling resources) to provide a more detailed picture of the
activities that make up most jobs.
In the field of human resource (HR) management, the detailed
analysis of jobs has largely been replaced with assessments of competencies. The term competency refers to an individual’s demonstrated
knowledge, skills, or abilities (Shippmann et al., 2000). The precise definition of competencies and the similarities and differences
between traditional job analysis and competency modeling are
matters that have been sharply debated (Shippmann et al., 2000),
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
7
and it is not clear whether competency modeling is really anything
other than unstructured and informal job analysis. Nevertheless,
the business world has adopted the language of competencies,
and competency-based descriptions of work are becoming increasingly common.
Some competency models are based on careful analysis and
compelling data, most notably the Great Eight model (Bartram,
2005):
Great Eight Competency Model
•
•
•
•
•
•
•
•
Leading and deciding
Supporting and cooperating
Interacting and presenting
Analyzing and interpreting
Creating and conceptualizing
Organizing and executing
Adapting and coping
Enterprising and performing
Bartram summarizes evidence of the validity of a range of individual difference measures for predicting the Great Eight. Unlike
some other competency models, assessment of these particular
competencies is often done on the basis of psychometrically sound
measurement instruments.
Drilling Deeper
Work can be described in general terms such as the competencies
detailed in the previous section. A more detailed analysis of what
people do at work is likely to lead to an assessment of more specific skills and an evaluation of background and experience factors that are likely to be related to these skills. In this context, skill
has a specific meaning: the consistent performance of complex
tasks with a high level of accuracy, effectiveness, or efficiency. Skills
are distinct from abilities in three ways: (1) they involve the performance of specific tasks, (2) they involve automatic rather than
controlled performance, and (3) they are the result of practice.
These last two features of skills are especially critical. The acquisition and mastery of skills usually requires a substantial amount of
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
8
Handbook of Workplace Assessment
practice or rehearsal, which suggests a link between assessment of
skills and assessments of background and experience. In the past
two decades, considerable progress has been made in assessments
of background and experience (Mael, 1991), but it is fair to say
that there are not well-established taxonomies of job-related skills
or of background and experience factors, making it difficult to
describe these domains in a great deal of detail.
Inferring Job Requirements
One of the most difficult challenges that proponents of workeroriented approaches face is to convincingly translate information about what people do at work into judgments about the
KSAOs required for performing well in particular jobs. This is
sometimes done on an empirical basis (for example, the Position
Analysis Questionnaire provides data that can be used to determine the predicted validity of a range of ability and skill tests),
but it is most often done on the basis of subjective judgments.
Virtually all methods of job analysis and competency modeling
involve inferences about the attributes required for successful
performance, but these judgments are rarely themselves validated. Indeed, there is little scientific evidence that given a good
description of the job, analysts can make valid inferences about
what attributes are required for successful performance beyond a
handful of obvious prerequisites; knowing that electricians work
with wires that are often color-coded, it is not hard to infer that
color vision is required for this job, for example. Usually inferences of this sort are based on the assumption that if the content
of the test matches the content of the assessments, those tests will
be valid predictors of performance on the job.
Murphy, Dzieweczynski, and Yang (2009) reviewed a large
number of studies testing the hypothesis that the match between
job content and test content influences the validity of tests and
found little support for this hypothesis. Nevertheless, an analysis
of the job, whether it is done in terms of competencies, generalized work activities, or detailed questionnaires, is often the first
step in making a decision about the content and the focus of
workplace assessments.
Work-oriented approaches to assessment are likely to be particularly useful as part of the process of making decisions about
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
9
placement and development. In particular, comparisons between
the content of previous and current jobs and the content of
future jobs are useful for identifying developmental needs and
gaps between the knowledge, skills, and experiences developed
in previous jobs and those required in future assignments.
Person-Oriented Analyses
A very different strategy for making decisions about what attributes should or should not be included in assessments starts
from the perspective of differential psychology: using what we
know about individual differences to drive what we assess. In
particular, this approach takes our knowledge of the dimensions
and structure of human cognitive ability, normal personality, and
interests and value orientations as a starting point for determining what to assess.
Cognitive Ability
There are enduring and stable individual differences in performance on virtually all tasks that involve the active processing of
information; these individual differences form the core of the
domain we refer to as cognitive ability.
The key to understanding the structure of human cognitive
abilities is the fact that scores on almost any reliable measure that
calls for active information processing will be positively correlated with any other reliable measure that also involves cognitive
activity. That is, scores on virtually all cognitively demanding tasks
exhibit positive manifold (Carroll, 1993). Thus, scores on paragraph comprehension measures will be correlated with scores
on numerical problem solving, which will be correlated with
scores on spatial relations tests and so on. The existence of positive manifold virtually guarantees that the structure of human
abilities will be hierarchically arranged, with virtually all specific
abilities (or groups of abilities) positively correlated with more
general ability factors. Theories of cognitive ability that give little
emphasis to g or deny the utility of a general factor do not seem
to provide any convincing explanation for positive manifold.
Carroll’s (1993) three-stratum model of cognitive ability (based
on the results of a large number of factor-analytic studies) nicely
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
10
Handbook of Workplace Assessment
illustrates the nature of modern hierarchical models. The essential features of this model are shown in Figure 1.1. At the most
general level, there is a g factor, which implies stable differences
in performance on a wide range of cognitively demanding tasks.
At the next level (the broad stratum) are a number of areas of
ability, which imply that the rank ordering of individuals’ task performance will not be exactly the same across all cognitive tasks,
but rather will show some clustering. Finally, each of these broad
ability areas can be characterized in terms of a number of more
specific abilities (the narrow stratum) that are more homogeneous still than those at the next highest level.
The hierarchical structure of the domain of cognitive abilities
has important implications for understanding three key aspects
of cognitive ability tests: (1) the validity of these tests as predictors of job performance and effectiveness, (2) the relationships
among abilities and the relative importance of general versus
specific abilities for predicting performance, and (3) adverse
impact. First, abundant evidence shows that cognitive ability is
highly relevant in a wide range of jobs and settings and that measures of general cognitive ability represent perhaps the best predictors of performance (Schmidt & Hunter, 1998). The validity
of measures of general cognitive ability has been established in
all sorts of jobs and settings, and it is reasonable to believe that a
good ability test will be a valid predictor of performance in virtually any application of testing.
The hierarchical structure of the cognitive domain is almost
certainly a key to the widespread evidence of the validity of cognitive tests. All jobs require active information processing (such as
retrieving and processing information, making judgments), and
Figure 1.1. The Cognitive Domain
g
Spatial/
Mechanical
Verbal
Comprehension
Vocabulary
Mathematics
Spatial
Rotation
Inference and
Diagnosis
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
11
even when the content of the job focuses on very specific tasks or
types of ability (a job might require spatial visualization abilities,
for example), the strong intercorrelations among abilities virtually guarantee that measures of general ability will predict performance. This intercorrelation among cognitive abilities also has
important implications for evaluating the importance of general
versus specific abilities.
A good deal of evidence exists that the incremental contribution of specific abilities (over and above general ability) to the
prediction of performance or training outcomes is often minimal (Ree, Earles, & Teachout, 1994). Because of the correlation
among measures of general and specific abilities, payoff for the
specific abilities required in a job is usually small. Measures of
general ability will usually be as good as, and often better than,
measures of specific abilities as a predictor of performance and
effectiveness.
The strong pattern of intercorrelation among cognitive abilities
poses a strong challenge to the hypotheses that many types of intelligence exist (Gardner, 1999) or that important abilities have not
yet been fully uncovered. In particular, the overwhelming evidence
of positive correlations among virtually all abilities raises important
questions about the nature of emotional intelligence.
Organizations have shown considerable interest in the concept of emotional intelligence (EI: Murphy, 2006). There are
many different definitions and models of EI, but the claim that
it is a distinct type of intelligence is at the heart of the debate
over its meaning and value. On the whole, little evidence exists
that emotional intelligence is related to other cognitive abilities,
casting doubts on its status as an “intelligence.” Some evidence
suggests that EI is related to a variety of organizationally relevant
criteria, but on the whole, the claim that EI is a distinct type of
intelligence and an important predictor of performance and
effectiveness does not hold up to close scrutiny (Murphy, 2006).
More generally, the idea that there are distinct types of intelligence does not square with the evidence.
Finally, the hierarchical structure of the cognitive domain has
important implications for the likelihood that ability measures will
lead to different outcomes for members of different ethnic and
racial groups. Black (and, to a lesser extent, Hispanic) examinees
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
12
Handbook of Workplace Assessment
consistently receive lower scores on cognitive ability tests than
white examinees, and the use of cognitive ability tests in personnel selection or placement will normally lead to adverse impact
against black and Hispanic examinees (Schmitt, Rogers, Chan,
Sheppard, & Jennings, 1997). Some differences in the amount of
racial disparity are expected with measures of different specific
abilities (in general, the stronger the correlation of specific abilities with g, the larger the racial disparities), but one consequence
of the positive manifold among measures of various abilities is
that adverse impact will be expected almost regardless of what
specific abilities are measured. The hierarchical structure of the
cognitive ability domain has several implications for research and
practice in personnel assessment, including:
• General abilities have broad relevance in most settings.
• Identifying the right specific abilities is not necessarily
important.
• The faults of general abilities will be shared with specific
ones.
• The belief in multiple types of intelligence or newly discovered intelligences is not consistent with the data.
First, the hierarchical structure of cognitive abilities means that
general abilities are more likely to be useful for predicting and
understanding behavior in organizations than more narrowly
defined specific abilities. This structure guarantees that even if it is
the specific ability that is important, general abilities will also turn
out to be good predictors in most settings. Because general abilities are usually measured with more reliability and more precision,
making the case for focusing on specific rather than on general
abilities is often hard.
Second, if the goal is predicting future performance and
effectiveness, this structure suggests a diminishing payoff for getting it exactly right when drawing inferences about the abilities
required by a job. For example, the spatial-perceptual branch of
most hierarchical models of cognitive ability includes a number
of specific abilities (say, three-dimensional spatial visualization
versus the ability to estimate distance and range). The further down
the chain of related abilities one goes (from general to spatial to
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
13
three-dimensional spatial visualization), the less difference choices
among branches of the ability tree are likely to make in determining the eventual value and criterion-related validity of ability tests.
Third, the use of ability tests in making decisions about people
in organizations such as personnel selection or placement will lead
to adverse impact against members of a number of racial and ethnic groups, and the use of specific rather than general ability measures will rarely change this fundamentally. Specific ability measures
do show slightly lower levels of adverse impact than general ones,
but they also typically show lower levels of criterion-related validity.
The decision to use cognitive ability tests in organizations is necessarily also a decision to accept a certain level of adverse impact; the
decision to refrain from using such tests is almost always also a decision to sacrifice validity.
Finally, the long-standing assumption and hope of many
researchers and practitioners (especially in educational settings)
that we can identify many separate types of intelligence is exactly
that: an assumption and an aspiration. Models that posit multiple
intelligences or suggest that specific types of content such as emotional information require their own type of intelligence are popular but not well supported. In the case of emotional intelligence,
which has attracted a great deal of attention in both educational
and organizational settings, improvements in the models and measures of this construct may eventually lead to the acceptance of EI
as a distinct and important domain of human cognitive ability, but
there are few data on the immediate horizon to lead us to believe
that current conclusions about the structure and nature of human
cognitive ability will need to be radically changed to accommodate
separate intelligences such as EI.
Personality
The link between personality and behavior in organizations has a
long history of interest. In a highly influential review, Guion and
Gottier (1965) cast considerable doubt on the value of personality measures, especially as predictors of job performance. They
concluded that “there is no generalizable evidence that personality measures can be recommended as good or practical tools for
employee selection” (p. 159) and that “it is difficult to advocate,
with a clear conscience, the use of personality measures in most
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
14
Handbook of Workplace Assessment
situations as a basis for making employment decisions about people” (p. 160). This review led to a long period of skepticism about
the relevance of personality in understanding performance and
effectiveness in the workplace. Not until the 1990s did personality
reemerge as a viable tool for understanding and predicting performance and effectiveness (Barrick & Mount, 1991). It is now widely
accepted that measures of normal personality have some value as
predictors of performance, but the validities of these measures are
often low. Nevertheless, they are also viewed as useful measures
for helping to structure and manage development and placement
programs.
As with cognitive ability, one of the keys to understanding
the relevance and value of personality measures is to examine the structure and the contents of this domain. The Five
Factor Model, often referred to as the “Big Five,” has emerged
as a dominant model for describing normal personality. This
model has been replicated across a number of methods, settings, and cultures, and it provides a good starting point for
describing what exactly personality means. This model suggests
that normal personality can be described largely in terms of
five broad factors that are at best weakly related to one another
and (with the exception of Openness to Experience) with cognitive abilities:
• Neuroticism: emotional instability, tendency to experience
negative emotions easily
• Extraversion: outgoing, energetic, tending toward positive
emotions
• Agreeableness: cooperates with, is compassionate and considerate toward others
• Conscientiousness: reliability, self-discipline, achievement oriented, planfulness
• Openness to Experience: curiosity, imagination, appreciation
for new ideas and experiences, appreciation of art, emotion,
adventure
The two structural aspects of the domain of normal personality that are most important for understanding the ways broad personality dimensions might be used in assessment are the relatively
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
15
weak correlations among the dimensions of normal personality
and the relatively weak relationships between personality and
cognitive ability. The weak correlations among the Big Five mean
that different dimensions of personality really do convey different information and that all sorts of personality profiles are possible. The weak correlations between personality and cognitive
ability have three very different and very important implications.
First, personality measures will contribute unique information not
captured by cognitive ability. That is, whatever variance in performance, behavior, or effectiveness is explained by personality will
almost certainly be distinct from variance explained by cognitive
ability. Second, personality measures will not share some of the
characteristics common to ability measures. In particular, measures
of normal personality are typically unrelated to the respondent’s
race, ethnicity, or gender.
Whereas the use of cognitive ability tests is a major cause of
adverse impact in personnel selection, the use of personality
measures can reduce adverse impact. Unfortunately, the reduction in adverse impact when ability and personality measures
are combined is not as large as one might expect; the combination of ability tests (which have adverse impact) and personality inventories (which do not) leads to some reduction in
adverse impact, but it will not cut it in half (Ryan, Ployhart, &
Friedel, 1998). Third, the weak relationships between personality and cognitive ability are consistent with one of the most
contentious issues in research on personality assessment in
organizations: the validity of broad personality dimensions as
predictors of performance and effectiveness. Although there is
considerable interest in the use of personality assessments in
organizations, studies of the validity of personality measures
as predictors of performance have consistently shown that the
correlations between personality and performance are small
(Morgeson et al., 2007). If the goal is to predict performance
and effectiveness, it is unlikely that measures of broad personality dimensions will help very much.
The two alternatives to using broad personality dimensions
in assessment might yield higher levels of validity. First, it is possible to use finely grained measures. For example, measures of
the Big Five often provide separate assessments of multiple facets
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
16
Handbook of Workplace Assessment
of each major dimension. For example, the NEO-PI (Costa &
McCrae, 1995) yields scores on the Big Five and on several facets
of each dimension; these are shown in Table 1.2. For example,
Conscientiousness can be broken down into Competence, Order,
Dutifulness, Achievement-striving, Self-discipline, and Deliberation.
It is possible that different facets are relevant in different jobs or
situations and from assessment of specific facets will yield different
levels of validity from those that have been exhibited by measures
of the Big Five.
An alternative to the use of finely grained measures is the
use of composite measures. For example, there is evidence that
Table 1.2. Facets of the Big Five
Neuroticism
Anxiety
Hostility
Depression
Self-consciousness
Impulsiveness
Vulnerability
Extraversion
Warmth
Gregariousness
Assertiveness
Activity
Excitement seeking
Positive emotions
Conscientiousness
Competence
Order
Dutifulness
Achievement-striving
Self-discipline
Deliberation
Agreeableness
Trust
Straightforwardness
Altruism
Compliance
Modesty
Tender-mindedness
Openness
Fantasy
Aesthetics
Feelings
Actions
Ideas
Values
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
17
integrity tests capture aspects of Conscientiousness, Neuroticism,
and Agreeableness (Ones, Viswesvaran, & Schmidt, 1993); the
breadth of the domain these tests cover may help to explain their
validity as a predictor of a fairly wide range of criteria. In principle, there might be no effective limit to the types of composite
personality tests that might be created, and some of these might
plausibly show very respectable levels of validity. However, this
strategy almost certainly involves a trade-off between the potential
for validity and interpretability.
The use of personality assessments to make high-stakes decisions about individuals is controversial (Morgeson et al., 2007),
in large part because most personality inventories are self-reports
that are potentially vulnerable to faking. The research literature
examining faking in personality assessment is broad and complex (Ones, Viswesvaran, & Reiss, 1996), but there is consensus
about a few key points. First, people can fake, in the sense that
they can often identify test responses that will paint them in the
most favorable light. Second, while faking can influence the outcomes of testing, it often does not greatly affect the validity of
tests. This is because positive self-presentation biases are often in
play when job applicants and incumbents respond to personality inventories, meaning that everyone’s scores might be inflated.
Although faking is a legitimate concern, it is probably more
realistic to be worried about the possibility of differential faking. That is, if some people inflate their scores more than others,
faking could change both the mean score and the rank order of
respondents. In other words, if everyone fakes, it might not be a
big problem, but if some people fake more or better than others,
faking could seriously affect the decisions based on personality
inventories.
As with cognitive ability, the structure and nature of the domain
of normal personality have important implications for research
and practice in organizational assessment:
• The relative independence of major personality dimensions puts a greater premium on identifying the right
dimensions and the right rules for combining information
from separate dimensions.
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
18
Handbook of Workplace Assessment
• Personality measures provide information that is distinct from
that provided by ability measures.
• The relatively low correlations with ability suggest that personality measures will be poor predictors of performance and
effectiveness; the available evidence seems to confirm this
prediction.
• Narrow dimensions of personality are easiest to interpret, but
are often similarly narrow in terms of what they predict. The
broadest dimensions show more predictive power but are
hard to sensibly interpret.
First, the broad dimensions that characterize the Big Five
are relatively distinct, which poses both opportunities and challenges. On the opportunity side, it is more likely that the complex models (for example, configural models, in which the
meaning of a score on one dimension depends on a person’s
score on other dimensions) will pay off in the domain of personality than in the domain of cognitive ability. In the ability
domain, the pervasive pattern of positive correlations among virtually all ability measures means it is hard to go too far wrong.
Even if you fail to identify the exact set of abilities that is most
important, you can be pretty certain of capturing relevant variance with measures of general abilities. In the personality
domain, choices of which dimensions to assess and how to combine them are likely to matter. This also means that identifying
the best way to use personality information is likely to be a much
more difficult challenge than identifying the best way to use
information about abilities.
Second, personality and ability seem to be largely independent
domains. There are some broad personality dimensions that may
be related to g, but most are not. This means that potential exists
for personality measures to contribute to the prediction of performance and effectiveness above and beyond the contributions of
ability measures. Unfortunately, as noted in our third point, this
often does not happen. The validities of personality measures are
statistically different from zero but are often not much greater
than zero (Morgeson et al., 2007).
Finally, personality assessment often poses trade-offs. One
trade-off is often between predictive power and interpretability
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
19
and another between ease of use and trustworthiness. Personality measures are usually self-reports, and they are not necessarily
hard to develop. They are, however, vulnerable to faking. Ability
tests have many defects, but at least it is hard to “fake smart.”
A personality inventory that shows an applicant to be high on
Conscientiousness and Agreeableness might mean exactly what
it appears to mean—or it might mean that the respondent knows
that high scores on these dimensions are viewed favorably, and
is faking.
Interests and Value Orientations
Organizational assessments are used not only to predict performance and efficiency but also to evaluate the fit between people and environments or jobs. Ability and personality measures
can be very useful in assessing fit, but many discussions of fit
focus on interests and value orientation, based on the argument
that the congruence between the interests and the values of an
individual and the environment in which he or she functions is
an important determinant of long-term success and satisfaction.
There are important questions about the extent to which fit
can be adequately measured and about the importance of
person-environment fit (Tinsley, 2000), but the idea of congruence between individuals and environments is widely accepted
in areas such as career development and counseling. Numerous
models have been used to describe the congruence between
individuals and environments; Lofquist and Dawis’s (1969)
Theory of Work Adjustment represents the most comprehensive and influential model of fit. The theory examines the links
between the worker’s needs and values and the job’s ability to
satisfy those needs, and it also considers the match between the
skills an individual brings to the job and the skills required for
effective performance in that job.
Assessments of interests have long been an important part of
matching individuals with jobs. Strong (1943) defined an interest
as “a response of liking” (p. 6). It is a learned affective response
to an object or activity. Things in which we are interested elicit
positive feelings, things in which we have little interest elicit little
affect, and things in which we are totally disinterested elicit apathy
or even feelings of aversion. Interest measures are widely used to
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
20
Handbook of Workplace Assessment
help individuals identify vocations and jobs that are likely to satisfy
and engage them.
The dominant theory of vocational choice was developed by
Holland (1973), who suggested that vocational interests can be
broken down into six basic types: realistic (interest in things),
investigative (interest in ideas), artistic (interest in creating),
social (interest in people), enterprising (interest in getting
ahead), and conventional (interest in order and predictability).
The Holland RIASEC model is shown in Figure 1.2.
The hexagonal structure of Figure 1.2 reflects one of the key
aspects of the Holland model. Interests that are close together
on the Holland hexagon, such as Realistic and Investigative, are
more likely to co-occur than interests that are far apart such as
Realistic and Social. The great majority of measures of vocational
interests and theories of vocational choice are based on the
Holland model.
Unlike the field of interest measurement, there is no single dominant model of work-related values. Probably the best-researched
Figure 1.2. Holland Taxonomy of Vocational Interests
Realistic
Conventional
Investigative
Enterprising
Artistic
Social
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
21
model is that proposed by Lofquist and Dawis (1969). Their taxonomy of work-related values, shown in Table 1.3, was adopted by
O*NET as a way of characterizing the values most relevant to various occupations.
Like many other taxonomies, the O*NET Work Value Taxonomy
is hierarchically structured. At the highest level of abstraction, jobs
can be characterized in terms of the extent to which they are likely
to satisfy value related to opportunities for achievement, favorable working conditions, opportunities for recognition, emphasis
on relationships, support, and opportunities for independence.
One of the many uses of O*NET is to match jobs to people’s
values. For example, individuals who value achievement and recognition can use O*NET to identify jobs that are likely to satisfy those
preferences. The lower level of the taxonomy helps to clarify the
meaning of each of the higher-order values and provides a basis
Table 1.3. O*NET Work Value Taxonomy
Achievement
Ability utilization
Achievement
Working conditions
Activity
Independence
Variety
Compensation
Security
Working conditions
Recognition
Advancement
Recognition
Authority
Social status
Relationships
Coworkers
Social service
Moral values
Support
Company policies and practices
Supervision, human relations
Supervision, technical
Independence
Creativity
Responsibility
Autonomy
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
22
Handbook of Workplace Assessment
for a more finely grained assessment of person-job fit. For example,
good working conditions might refer to almost any combination of
opportunities for activity, independence, variety, compensation, or
job security.
Assessments of cognitive abilities and personality traits are
often used to predict criteria such as performance and effectiveness. Assessments of interests and values are not likely to reveal as
much about performance, but are related to criteria such as satisfaction, burnout, and retention. Good person-job fit is thought
to enhance the attractiveness and the motivational potential of a
job, and in theory these assessments can be used for both individual counseling and placement. In practice, systematic placement
(hiring an individual first and deciding afterward what job or
even what occupational family to assign that person to) is rarely
practiced outside the armed services. However, interest measures
might be quite useful for career planning activities at both the
individual and the organizational levels. For example, executive
development programs often involve a sequence of jobs or assignments, and the use of interest and value assessments might help
in fine-tuning the sequence of assignments that is most likely to
lead to successful development.
Implications for Assessment in Organizations
Individual differences in cognitive ability, personality, values, and
interests are likely to affect the performance, effectiveness, motivation, and long-term success of workers at all levels in an organization. A general familiarity with the structure and the content of
each of these domains provides a good starting point for designing organizational assessments.
The essential feature of cognitive abilities is their interrelatedness. This presents both opportunities and challenges when using
ability tests in organizations. Because virtually all abilities are correlated (often substantially) with general abilities, it is hard to
go seriously wrong with the choice of ability measures; jobs that
require one ability also tend to require the constellation of other
related abilities. Because virtually all jobs require and involve
the core activities that define cognitive ability (the acquisition,
manipulation, and use of information), it is generally a safe bet
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
23
that ability measures will turn out to be valid predictors of performance. Unfortunately, the interconnectedness of abilities also
implies that any of the shortcomings of general cognitive ability
as a predictor will be broadly shared by more specific measures.
In particular, ability measures of all sorts are likely to show substantial adverse impact on the employment opportunities of black
and Hispanic applicants and employees, and this impact has both
legal and ethical implications. Depending on the weight you
give to predictive validity versus the social impact of using ability
tests to make high-stakes decisions, you might come to very different conclusions about whether including these measures in organizational assessments makes sense (Murphy, 2010).
The domain of normal personality has a much different structure. The Big Five personality factors are interrelated, but the
correlations among dimensions are generally quite weak, and no
general factor describes human personality. Like many other taxonomic structures, the Big Five can be broken down into facets,
or they can be combined into composites, but moving from the
level of the Big Five to either higher (composite) or lower (facet)
levels of abstraction often involves trade-offs between interpretability and predictive value.
Two issues seem especially important when using personality measures as part of assessment in organizations. First, these
are usually self-reports and are vulnerable to manipulation and
misrepresentation. There are important debates about the
actual effects of faking on validity and the outcomes of selection (Morgeson et al., 2007), but the possibility that respondents
might be able to consciously inflate their scores on high-stakes
assessments is likely to be a realistic barrier to their use in many
settings. More important, the validity of these measures as predictors of criteria such as performance or effectiveness is often disappointing, and the value of obtaining these assessments is not
always clear.
Vocational interests are well understood and are captured nicely
by Holland’s hexagonal model. This model posits relationships
among interests that can be captured by the distance between any
pair of interests on the hexagon; this model has been applied with
considerable success in vocational counseling. However, it is not
always clear how to use assessments of interests or values to make
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
24
Handbook of Workplace Assessment
more detailed predictions of judgments. There are many models
of person-job fit, and different models often depend on different
sets of values. No single agreed-on taxonomy adequately captures
the universe of organizationally relevant values. Nevertheless, the
general proposition that some jobs are more likely than others
to fit an individual’s values and that some individuals are more
likely than others to fit any specific job seems well established, and
the measurement of work-related values has potential for both
research and practice.
This chapter has been intentionally broad in its focus, and the
implications for assessment laid out in the preceding paragraphs
are similarly broad. Chapters Two through Eight examine more
specific issues in assessments of domains ranging from abilities
to personality to background and experience. Chapters Nine
through Fourteen show how assessments of these domains are
used in making decisions in occupations ranging from hourly
or skilled work to executive and managerial positions. Chapters
Fifteen through Twenty-Four discuss a wide range of questions
encountered when developing and using assessments in a range
of organizational contexts.
References
Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology,
44, 1–26.
Bartram, D. (2005). The great eight competencies: A criterioncentric approach to validation. Journal of Applied Psychology, 90,
1185–1203.
Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic
studies. New York: Cambridge University Press.
Costa, P. T., & McCrae, R. R. (1995). Domains and facets: Hierarchical
personality assessment using the Revised NEO Personality Inventory.
Journal of Personality Assessment, 64, 21–50.
Gardner, H. (1999). Intelligence reframed: Multiple intelligences for the 21st
century. New York: Basic Books.
Guion, R. M., & Gottier, R. F. (1965). Validity of personality measures in
personnel selection. Personnel Psychology, 18, 135–164.
Holland, J. L. (1973). Making vocational choices: A theory of careers. Upper
Saddle River, NJ: Prentice Hall.
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Individual Differences That Influence Performance
25
Lofquist, L. H., & Dawis, R. V. (1969). Adjustment to work. New York:
Appleton–Century–Crofts.
Mael, F. A. (1991). A conceptual rationale for the domain of attributes
of biodata items. Personnel Psychology, 44, 763–792.
McCormick, E. J., Jeanneret, P. R., & Mecham, R. C. (1972). A study
of job characteristics and job dimensions as based on the Position
Analysis Questionnaire (PAQ). Journal of Applied Psychology, 56,
347–368.
Morgeson, F. P., Campion, M. A., Dipboye, R. L., Hollenbeck, J. R.,
Murphy, K., & Schmitt, N. (2007). Reconsidering the use of personality tests in personnel selection contexts. Personnel Psychology,
60, 683–729.
Murphy, K. R. (2000). What constructs underlie measures of honesty
or integrity? In R. Goffin & E. Helmes (Eds.), Problems and solutions in human assessment: A festschrift to Douglas N. Jackson at seventy
(pp. 265–284). Norwell, MA: Kluwer.
Murphy, K. R. (2006). A critique of emotional intelligence. Mahwah,
NJ: Erlbaum.
Murphy, K. (2010). How a broader definition of the criterion domain
changes our thinking about adverse impact. In J. Outtz (Ed.),
Adverse impact (pp. 137–160). San Francisco: Jossey-Bass.
Murphy, K. R., Dzieweczynski, J. L., & Yang, Z. (2009). Positive manifold
limits the relevance of content-matching strategies for validating
selection test batteries. Journal of Applied Psychology, 94, 1018–1031.
Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996). Role of social desirability in personality testing for personnel selection: The red herring.
Journal of Applied Psychology, 81, 660–679.
Ones, D. S., Viswesvaran, C., & Schmidt, F. L (1993). Comprehensive
meta-analysis of integrity test validities. Journal of Applied Psychology,
78, 679–703.
Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., &
Fleishman, E. A. (1999). An occupational information system for the
21st century: The development of O*NET. Washington, DC: American
Psychological Association.
Ree, M. J., Earles, J. A., & Teachout, M. S. (1994). Predicting job performance: Not much more than g. Journal of Applied Psychology, 79,
518–524.
Ryan, A. M., Ployhart, R. E., & Friedel, L. A. (1998). Using personality testing to reduce adverse impact: A cautionary note. Journal of
Applied Psychology, 83, 298–307.
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
26
Handbook of Workplace Assessment
Schippmann, J. S., Ash, R. A., Carr, L., Hesketh, B., Pearlman, K., Battista,
M. et al. (2000). The practice of competency modeling. Personnel Psychology, 53, 703–740.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical
implications of 85 years of research findings. Psychological Bulletin,
124, 262–274.
Schmitt, N., Rogers, W., Chan, D., Sheppard, L., & Jennings, D. (1997).
Adverse impact and predictive efficiency of various predictor combinations. Journal of Applied Psychology, 82, 719–730.
Strong, E. K. (1943). Vocational interests of men and women. Stanford,
CA: Stanford University Press.
Tinsley, H. E. (2000). The congruence myth: An analysis of the efficacy
of the person-environment fit model. Journal of Vocational Behavior,
56, 147–179.
08/08/2018 - RS0000000000000000000000984657 (Savior Wright) Handbook of Workplace Assessment
Purchase answer to see full
attachment