“That which is essential is invisible to the eyes.”
— Antoine de Saint-Exupéry, The Little Prince.
As someone who leads an integrated health care delivery system, this is a question I frequently find myself asking.
The obvious answer, of course, is to develop measurements based on treatment protocols. Of which we have plenty. It seems these days that we have a measurement and documentation requirement for just about everything. In fact, quality measurement in health care has become an industry unto itself. Hospitals and health care systems across the country pay a lot of money to have their quality of care scrutinized and, hopefully, lauded, by a number of companies that charge them for such assessments. In many cases, those assessments are valuable.
Nevertheless, I began to think about the value of measurement after exchanging some emails with my friend and college mentor, Deborah Stone. Deborah is a professor at Brandeis University’s Heller School for Social Policy and Management who’s been doing a lot of thinking lately about counting, measurement, and statistic and the ways in which numbers are used to distort and distract from reality. In a lecture she recently gave to the American Political Science Association, Deborah announced to her audience, “Numbers are figments of our imagination, fictions really, no more true than poems or drawings. In this sense, all statistics are lies.”
I’m not sure I’m willing to go as far as Deborah, who’s quite a provocative thinker, but she did make me wonder whether our current health quality measures are offering the right information and, moreover, whether everything valuable in health care can be easily measured. At some level, I suspect, things that are important are not always quantifiable. For example, even if my facilities are spotless and my clinical staff is expert at avoiding preventable infections, does that mean they’re good at explaining diagnoses to their patients? Do they know how to communicate effectively and sympathetically when delivering bad news? Do they return patient calls at night? In today’s health care climate, physicians are often required to see a specific number of patients each day. But how effective are our measurements if a physician misses that quota because she devoted extra time to a single patient who really needed the extra attention and care?
These questions echo others raised in a recent study headed by Arnold Milstein, who directs the Clinical Excellence Research Center at Stanford University. In the study, Milstein and his colleagues used commercial health insurance claims to identify physician-practice sites across the country that delivered high-quality care with a lower overall cost. They then conducted extensive site visits to determine what these practices were doing right.
The researchers found some common themes, including the implementation of a concept that Milstein termed “care-traffic control.” As Milstein describes it, “We found that physicians at these sites were thinking more deeply about what each individual patient needs to navigate in the periods between primary care office visits… Does their illness affect their executive functioning? Are they following through on laboratory tests? Are they taking their medicines as prescribed? Are all of the doctors and specialists a patient sees aware of important aspects of their care plan, such as the existence of an advance directive? Although this is unknown territory to physicians in average-performing primary care practices, it is actively surveilled and supported by their high-value peers.” Needless to say, many of the attributes that Milstein describes are not often measured by quality measurement organizations. For example, the researchers found that high-value practices usually welcome complaints, offer same-day appointments and expanded hours, and, interestingly, are located in “modest” office space.
Another researcher who’s done important research into how to measure care is the child psychiatrist Gordon Harper of Harvard Medical School. Harper has noted that, in child and adolescent mental health policy, “much emphasis has been placed on demonstrating processes.” He suggests that’s because “process, indeed, is much easier to measure than outcomes.”
For example, Harper told me about a resident he once supervised. The resident saw a patient who presented common symptoms of depression, which she said began when her child died. The physician asked how the child died. “I don’t know,” replied the resident, who explained that she never asked because that question wasn’t on her diagnostic checklist. The physician explained that the question was important, not just for establishing a professional rapport with the patient, but because a question like that can help a psychiatrist determine the right diagnosis. A good psychiatrist, says Harper, considers the trauma the parent may have witnessed — or even the possibility that the child may not have existed. “When people try to reduce everything to checklists, it puts everything in danger,” he says.
Someone who’s not ready to write off the value of measurement is Dr. Lori Tishler, at Commonwealth Care Alliance, a not-for-profit, community-based healthcare organization. “Some metrics are really good,” Tishler told me recently. She points to checklists that ensure patients receive mammograms and other regular screenings. “Screening tests can save somebody’s life,” she says. “They’re reasonable ways to measure quality when we look at data points.”
Another way Tishler measures quality is through patient surveys that ask questions like, “Did your doctor listen to you? Did he or she take the time to explain the plan for your health? Did they speak to you in words you could understand?”
“Hospitals should be clean and employees should be courteous. But that’s just the beginning,” says Tishler, who says patient surveys are valuable because they help her ensure that her staff is “doing things because they’re right for the patient. And often what’s right for the patient is right for the bottom line.”
Tishler was one of my primary care preceptors in residency. As we talked, she noted that, as a medical school professor, it was much easier to measure performance using tools such as the Objective Structured Clinical Examination (OSCE), where students are asked to perform histories and examinations on standarized patients. But, she says, “Measuring for physicians in practice is a lot more difficult.”
Perhaps. But the physician and writer Atul Gawande suggests an out-of-the box approach. In an article in The New Yorker, he discusses his experience performing surgery under the watchful eye of a coach, who helped him identify ways to improve his technique. “Since I have taken on a coach, my complication rate has gone down,” Gawande happily reports.
It may not be practical for every physician to have a coach present for every patient encounter or for every health system to create enough coaching capacity (perhaps this will be a feature of Gawande’s new Haven venture with Amazon, Berkshire Hathaway, and JP Morgan Chase). But Gawande’s experience is a reminder to all health providers that checklists, standards and protocols are not enough to ensure quality in our health care system. In an era when nurse practitioners are doing the work of doctors, generalists are doing the work of specialists, and specialists are doing the work of sub-specialists, it’s vital that we have strong, common sense supervisory systems in place that bring a culture of quality to the place where it’s most needed in American healthcare: the exam room.