qotd: Berenson on the fallacy of “If you can’t measure it, you can’t manage it”

JAMA Forum
January 13, 2016
If You Can't Measure Performance, Can You Improve It?
By Robert A. Berenson, MD

"If you can't measure it, you can't manage it" is an often-quoted
admonition commonly attributed to the late W. Edwards Deming, a leader
in the field of quality improvement. Some well-respected health policy
experts have adopted as a truism a popular variation of the Deming quote
— "if something cannot be measured, it cannot be improved" — and point
to the recent enactment of the Medicare Access and CHIP Reauthorization
Act of 2015 (MACRA) as a confirmation of "the broadening societal
embrace" of this concept.

The problem is that Deming actually wrote, "It is wrong to suppose that
if you can't measure it, you can't manage it — a costly myth" — the
exact opposite. Deming consistently cautioned against requiring
measurement to guide management decisions, observing that the most
important data needed to manage often are unknown and unknowable.

Many Routes to Improvement

The requirement for measurement as essential to management and
improvement is a fallacy, not a self-evident truth and not supported by
Deming, other management experts, or common sense. There are many routes
to improvement, such as doing things better based on experience,
example, as well as evidence from research studies.

Comparative public performance using meaningful and accurate measures
has led to quality improvements, as clinicians and hospitals reflect on
their own comparative performance and seek to improve their public
standing. Examples include improved hospital care for patients
experiencing heart attacks and improved renal dialysis. In most clinical
areas, however, we lack readily available measures to use as valid
benchmarks to assess performance.

Not deterred, however, last year a rarely bipartisan Congress passed the
MACRA legislation. Its core element was repealing the unsustainable
sustainable growth rate mechanism threatening huge payment cuts to
physicians caring for Medicare patients. The law called for development
of "value based" payment approaches that would pay for quality and cost
outcomes, rather than just for the myriad services physicians provide or
order, whether or not the services are needed or well performed. "Paying
for value, not volume" has become the slogan du jour, itself assuming a
mostly unchallenged position in health policy circles.

Now comes the hard part: actually achieving greater value, rather than
fashioning an increasingly complex, intrusive, and likely doomed attempt
to measure value.

After the MACRA's Merit-Based Incentive Payment System (MIPS) is fully
phased in early in the next decade, a physician caring for Medicare
patients under MIPS stands to lose up to 9% of their Medicare payments
or conceivably gain 27%, based on their performance on measures of
quality, their use of health care resources, the extent to which they
have implemented electronic health records, and their participation in
quality improvement activities.

But performance on a few, random and often unreliable measures of
performance can provide a highly misleading snapshot of any physician's
value.

A Bad Idea?

Practical challenges aside, pay for performance for health professionals
may simply be a bad idea. Behavioral economists find that tangible
rewards can undermine motivation for tasks that are intrinsically
interesting or rewarding. Furthermore, such rewards have their strongest
negative impact when they are perceived as being large, controlling,
contingent on very specific task performance, or associated with
surveillance, deadlines, or threats, as with MIPS.

Another major problem with the current preoccupation with measurement as
the central route to improvement is the assumption that if a quality
problem isn't being measured, it basically doesn't exist. A prime
example is diagnosis errors. Recently, an Institute of Medicine (IOM)
committee, on which I was a member, issued Improving Diagnosis in Health
Care, documenting serious errors of diagnosis in 5% to 15% of
interactions with the health care system.

As the report emphasizes, we cannot now measure the accuracy of
diagnoses, which means MIPS scores will not include performance on this
core physician competency. Still, the IOM committee proposed numerous
improvement strategies. These include development of immediate feedback
programs to erring clinicians from patients and other health
professionals when a serious misdiagnosis occurs (making errors
memorable if not measureable), greater attention in medical education to
the cognitive bias that commonly clouds clinicians' judgment, improved
systems to ensure that abnormal test results are promptly communicated
to patients and diagnostic team members, and giving patients direct
access to their medical records so they can introduce relevant, missing
information and correct the misinformation that is common in clinical
records.

These and other IOM recommendations represent better practices that
might dramatically improve diagnostic accuracy, relying not on
performance measures but on adopting better work processes and focused
education. Measures would help, but substantial progress can be made
regardless.

The overarching concern is that under MIPS and similar programs,
physicians will focus on the money while their intrinsic motivation to
make accurate, timely diagnoses as a core responsibility will be crowded
out. If so, the worthwhile recommendations in the IOM report will likely
sit on the shelf, gathering dust, thanks to the misguided supposition
that "if you can't measure it, you can't manage it."

http://newsatjama.jama.com/2016/01/13/jama-forum-if-you-cant-measure-performance-can-you-improve-it/

***

AMA Wire
January 13, 2016
CMS chief vows to replace meaningful use with better policy

Centers for Medicare & Medicaid Services (CMS) Acting Administrator Andy
Slavitt on Monday said that the agency is changing its culture to focus
more on listening to physician needs and giving them the freedom they
need to keep patients at the center of the practice of medicine.

Referring to execution of the electronic health record (EHR) meaningful
use program, Slavitt noted that the agency's previous regulatory
approach created difficulties. "When in doubt, I think, do less and
figure it out."

"The meaningful use program as it has existed will now be effectively
over and replaced with something better," Slavitt said.

In its place will be the new Merit-Based Incentive Payment System
(MIPS), called for in the Medicare Access and CHIP Reauthorization Act
of 2015, which is intended to sunset the three existing reporting
programs and streamline them into a single program.

"The stakes are high for this program," Slavitt said. "As any physician
will tell you, physician burden and frustration levels are real.
Programs designed to improve often distract. Done poorly, measures are
divorced from how physicians practice and add to the cynicism that the
people who build these programs just don't get it."

http://www.ama-assn.org/ama/ama-wire/post/cms-chief-vows-replace-meaningful-use-better-policy

***


Comment by Don McCanne

As several prior Quote of the Day messages warned, MACRA's Merit-Based
Incentive Payment System (MIPS) is a horrendous trade-off for getting
rid of the flawed SGR payment system. At least for the next decade, we
are going to have to live with a system which supposedly will reward or
penalize physicians based on measured performance when "the most
important data needed to manage often are unknown and unknowable."

Robert Berenson writes, "a few, random and often unreliable measures of
performance can provide a highly misleading snapshot of any physician's
value." Further, under MIPS, "physicians will focus on the money while
their intrinsic motivation to make accurate, timely diagnoses as a core
responsibility will be crowded out."

CMS is responding to the great dissatisfaction with the administrative
burden of the "meaningful use" program for electronic health records,
which would have been the source of many of these performance
measurements. But what is their response? Acting CMS Administrator Andy
Slavitt says, "The meaningful use program as it has existed will now be
effectively over and replaced with something better" - MIPS! He says,
"Done poorly, measures are divorced from how physicians practice and add
to the cynicism that the people who build these programs just don't get it."

Well, yes. They just don't get it.