The Dubious Management Fad
Sweeping Corporate America
NPS—or net promoter score—is a measure of customer satisfaction that has
developed a cultlike following among CEOs. It also may be misleading.
By
Khadeeja Safdar and
Inti Pacheco
May 15, 2019
Best Buy and American Express use it to dole out employee bonuses. Target
and Intuit point to it to justify investments. Delta Air Lines andUnitedHealth can’t
stop talking about it.
Much of Corporate America is obsessed with its net promoter score, or NPS, a
measure of customer satisfaction that has developed a cultlike following among
CEOs in recent years. Unlike profits or sales, which are measured and audited,
NPS is usually calculated from a one-question survey that companies often
administer themselves.
Last year, “net promoter” or “NPS” was cited more than 150 times in earnings
conference calls by 50 S&P 500 companies, according to a Wall Street Journal
analysis of transcripts. That’s more than four times as many mentions, and nearly
three times as many companies, compared with five years earlier.
Executives pointed to strong or rising NPS as proof that shoppers preferred to
pick up orders at Target Corp. stores or that Google’s newest Pixel smartphone
was off to a good start. Out of all the mentions the Journal tracked on earnings
calls, no executive has ever said the score declined.
Dozens of public companies are reporting the score in securities filings. Last year,
“net promoter” was mentioned in 56 proxy filings. Some companies, including
American Express Co., Best Buy Co. andCitigroup Inc., list the metric as a criteria
for executive compensation, alongside traditional measures such as revenue
growth and earnings per share.
The score was introduced in 2003 in a Harvard Business Review article titled “The
One Number You Need to Grow.” The Bain & Co. consultant who wrote the article
called NPS the “simplest, most intuitive and best predictor of customer behavior”
and a “useful predictor of growth.”
Keeping Score
The number of mentions of Net Promoter Score, or NPS*—which companies are
touting as a measure of customer satisfaction—in earnings calls has ballooned.
*NPS or Net Promoter found in over 40,000 earnings-call transcripts from 2003 to 2018
Source: WSJ analysis of companies’ earnings calls
Since then, the metric has taken on a life of its own, so much so that the inventor,
Fred Reichheld, said he is astonished companies are using NPS to determine
bonuses and as a performance indicator. “That’s completely bogus,” Mr.
Reichheld, who still consults for Bain, said in an interview. “I had no idea how
people would mess with the score to bend it, to make it serve their selfish
objectives.”
The score is typically derived from customer responses to a single question that
companies ask at the checkout register of a store or in an email or web pop-up
online: On a scale of 0 to 10, how likely are you to recommend the company’s
product or service to a friend? The survey usually includes a follow-up question
asking customers to explain their ratings.
NPS is based on the premise that every company’s customers can be divided into
three groups. People who answer 9 or 10 are “promoters,” or loyal enthusiasts
who keep buying. Those who give a score of 0 to 6 are “detractors,” or unhappy
customers. Those who answer 7 or 8 are considered “passives,” satisfied but
easily wooed by competitors.
Would You Recommend?
Companies using NPS ask customers to rate their experience by answering the
question, 'How likely are you to recommend' the company's product or service
to a friend, on a scale of zero to 10?
Source: Bain & Co.
The score is determined by subtracting the percentage of customers who are
detractors from those who are promoters, with the result falling in a range
between -100 to 100. Passives are ignored in the calculation.
Management consultants are notorious for pushing ideas to CEOs using jargon
and claims of improved business performance. Total quality management, or
TQM, which advocated installing quality programs at companies, and business reengineering process, or BRP, which was a way to restructure companies, gained
traction in the 1990s and then faded. NPS has outlived such fads, spawning a
cottage industry of consultants and software firms that help businesses
implement and boost their score.
Some academics have questioned the whole idea, suggesting that NPS has been
oversold. Two 2007 studies analyzing thousands of customer interviews said NPS
doesn’t correlate with revenue or predict customer behavior any better than
other survey-based metric. A 2015 study examining data on 80,000 customers
from hundreds of brands said the score doesn’t explain the way people allocate
their money.
New Measure for Companies
The number of companies talking about Net Promoter Score, or NPS* in
earnings calls has steadily risen as it has grown in popularity.
*NPS or Net Promoter found in over 40,000 earnings-call transcripts from 2003 to 2018
Source: WSJ analysis of companies’ earnings calls
“The science behind NPS is bad,” said Timothy Keiningham, a marketing professor
at St. John’s University in New York, and one of the co-authors of the three
studies. He said the creators of NPS haven’t provided peer-reviewed research to
support their original claims of a strong correlation to growth. “When people
change their net promoter score, that has almost no relationship to how they
divide their spending.”
Some data scientists said the way NPS is calculated, in which one survey metric is
subtracted from another, increases the margin of error and requires a larger
sample size to get useful results.
“It’s common for companies to track NPS data as if it’s gospel—not knowing that
it’s super noisy by design,” said Kim Larsen, who has worked as a data scientist at
several companies, including Charles Schwab Corp.
Bain, which now refers to NPS as “net promoter system,” said some companies
are focusing too heavily on the score, but still defended the approach for some
practical benefits. It is simple to communicate to employees, provides an easy
way to follow up with customers and can be used to benchmark against rivals.
The firm also said third-party analyses, including the 2007 studies, of whether NPS
correlates with revenue aren’t as good as the analyses companies conduct
internally.
“These are not stupid people. They are running large, successful companies,” said
Rob Markey, a Bain partner who helps clients use NPS. “They have demonstrated
to their own satisfaction that it’s good.”
Among the first companies to implement NPS were General Electric Co., Intuit Inc.
and Schwab, whose leaders were convinced of the benefits after meeting with
Mr. Reichheld and other Bain consultants. Now, hundreds of companies are using
the score and many have tweaked the methodology, such as making the
numerical scale 1 to 5 or including additional survey questions.
International Business Machines Corp. said it switched from a three-question
survey to NPS in 2015. Employees in different departments can see the NPS
feedback on their phones. “What it’s become here is a shared truth,” said Kathy
McGettrick, vice president of market development and insights at IBM.
Some NPS users in the Journal analysis said the score correlates with revenue
growth, though no company would disclose data to prove that point. Several
companies said NPS is just one of many metrics they use to make decisions and
that it helps them improve products or services.
Intuit, which makes QuickBooks and TurboTax, began asking its customers the
NPS question around 2003. Shortly after, CEO Steve Bennett attributed the
company’s growth in self-prepared taxes to its net promoter efforts, since
feedback had spurred improvements in some products.
Intuit has referred to NPS more than 60 times on earnings calls since the survey
was first implemented. The company has used it to explain investments, including
its 2007 acquisition of web-hosting company Homestead Technologies and the
expansion of the ad budget for its QuickBooks Online service last year.
“We have not intended to imply that NPS is related to, or a driver of, revenue,”
said an Intuit spokeswoman, adding that NPS is one of many metrics the company
uses. “It allows our teams to gather customer insights so they can make decisions
internally to deliver customer benefits.”
Mr. Bennett, who left Intuit at the end of 2007, later went on to serve as CEO of
antivirus software maker Symantec, where he asked employees to stop using NPS,
according to people familiar with his tenure. In response to the Journal, Mr.
Bennett said he came to learn that the score is less meaningful than the openended question that can follow the rating.
“A big challenge with the methodology is that organizations tend to focus on the
metric as the objective instead of gaining the insight to learn and act on to
improve the customer experience,” he said. “When organizations manage to the
metric, they find ways to game the system.”
The results are easy to manipulate, whether intentionally or unintentionally. On
Reddit posts, Best Buy employees share tips and tricks to improve NPS, which the
company derives from a random sample of customers. They said they can get
better results when they explain to customers how the scoring works, or tell them
their compensation is connected to the result. Some said they remind only the
happiest customers to take the survey.
“When horrible NPS comments would come in, the management would rail at the
employees,” said Alan Sabido, a former Best Buy employee who worked at a Las
Vegas store for three years until he quit last year. Mr. Sabido recalled an instance
when his store team received a bad score because a customer had a poor
experience at a different Best Buy location.
NPS took on a greater role after Hubert Joly joined as Best Buy’s chief executive in
2012. The company said it was administering the NPS survey question to
customers who bought products as well as those who didn’t. Best Buy also made
the metric one of the criteria used to determine bonuses.
Since then, Best Buy has mentioned NPS or net promoter more than 50 times on
earnings calls. The company has created a team of employees at its Richfield,
Minn., headquarters called “Enterprise NPS” and some of its store workers are
also tasked with “driving positive NPS results,” according to job postings.
“We use a number of methods to measure customer satisfaction and NPS is just
one of them. We are aware of its limitations but believe it is valuable,” said a Best
Buy spokesman. “Our revenue and earnings have increased at the same time
we’ve seen NPS improvements.”
When Mr. Joly announced he was stepping aside as CEO in April, Best Buy listed
among his accomplishments that NPS grew by three points in the past fiscal year.
In securities filings, Delta Air Lines Inc. has said it measures “customer service
performance” as the percentage point improvement in its average NPS. The
methodology, the company said, was approved by a board committee and its
progress is reported periodically to the board. NPS is also listed as criteria for
executive bonuses.
Carol Campbell, Delta’s managing director of customer experience, said most of
her work entails using NPS to find opportunities to make investments. The airline
added fresh-baked cookies on trans-Atlantic flights, chose to put nine seats in
each Boeing 777 row instead of the standard 10 and made enhancements to its
app because of NPS.
Delta executives describe NPS as the “true North Star,” she said, though the
airline uses other customer metrics as well. “We have been able to statistically
correlate our NPS performance with our revenue premium,” she said, referring to
how much more Delta is able to charge than a competitor because of its brand.
UnitedHealth Group Inc. has mentioned NPS more times on earnings calls than
any other S&P 500 company in the Journal’s analysis. In April, CEO David
Wichmann said the insurer’s net promoter scores “continued to advance
meaningfully in the first quarter 2019 as we march toward an aggressive target of
70 by 2025.”
“UnitedHealth Group uses NPS as an important measure to understand and
continually improve upon the quality of the experience for those we serve,” a
spokesman for the company said.
It’s hard for investors to interpret the score because companies don’t typically
share response rates, margin of error, or whether results are adjusted for cultural
and other biases. Research shows NPS tends to be higher when response rates
are lower, and Americans tend to give higher scores than consumers in some
countries such as Japan and Korea.
Since so many NPS surveys are sent to customers through emails, web pop-ups
and phone calls, it has become harder to elicit a response these days, survey firms
said. Email response rates are usually less than 5%, according to an estimate by
Qualaroo, a software startup that helps companies conduct user research.
Software providers like Medallia and Qualtrics said they help companies get
higher response rates. Satmetrix, which originally helped Mr. Reichheld develop
the score and is now a division of software-provider NICE, said it sells access to
the “world’s best net promoter certification program.” J.D. Power said it signed an
agreement with Bain to “become the officially recognized authority for
benchmarking the Net Promoter Score.”
METHODOLOGY
To find the number of mentions of the terms “net promoter” or “NPS,” The Wall
Street Journal used transcripts of company conference calls discussing quarterly
earnings, retrieved from Factiva, a business research tool owned by WSJ publisher
Dow Jones & Co.
The analysis started with more than 40,000 transcripts for calls held from 2003
through 2018, for 688 existing companies that were members of the S&P 500
index at any point during that period, as indicated by S&P Dow Jones Indices.
The Journal analyzed transcripts from companies that are still publicly traded. It
excludes 209 companies that were acquired during the period but includes 188
that left the index yet remain public.
Nearly 400 transcripts contained at least one of the terms. A paragraph was
counted as a single mention if it contained one or more uses of the terms.
Portions of the transcripts consisting of questions from analysts were ignored.
Questions
1. Based on this article, what's your opinion of Net Promoter Score? Do you
feel this metric reflects a firm’s performing in the minds of consumers?
Please provide details for your response.
2. Are there certain industries in which NPS usage is more useful than others?
Describe one industry that should use NPS in their customer satisfaction
assessment and list another industry where NPS is not an accurate measure
of success. Please provide a 1 or 2 sentence rationale for your selection.
3. In addition to NPS, what are 3 additional metrics firms can reference when
gauging customer satisfaction? Please list all three with a 1 or 2 sentence
rationale for your selection.
Purchase answer to see full
attachment