CQI - The Good, the Bad, and the Ugly
With apologies to Sergio Leone, whose 1966 “spaghetti western” used the title, I have applied it to this discussion of continuous quality assurance. The movie was a not-so-serious send up of the standard tropes of the Westerns of the 1950’s, and was a good fit for the Zeitgeist of the time. Perhaps the choice is appropriate for a seriously important topic in healthcare, where CQI can be the good, the bad, or the ugly, depending on its use and misuse. In this discussion I am going to use examples from the care of dialysis patients, as that is where my expertise lies, but I am sure you can think of examples in other fields.
First, the good. In 1986 I met with my dialysis leaders and discussed choosing subjects for a test CQI project that would be important to us and to our patients. We decided to focus on delivering asymptomatic dialysis treatments, on the assumption patients did not want us to make them pass out, cramp, or vomit, as a result of dialysis. The literature suggested this occurred in about 20% of treatments. Now there was a major problem defining hypotension. If the patient came in with hypertension and went home in the normal range, that was success, so we could not use an arbitrary difference in pre- and post-dialysis readings. Likewise, some patients walk around every day with readings that are low, but are not symptomatic, and we did not want to count them either. What we wanted to do was avoid hypotensive episodes related to ultrafiltration of fluid, which, in turn, reflects the difficulty in deciding on the target “dry weight” in the particular patient.
We ended up deciding to use an operational definition. If the nurse took an action—giving fluids or elevating the feet—then that counted. Reducing the ultrafiltration goal without an intervention did not. We then set about training the staff to count accurately, but also training in how to avoid the issues in the first place. After several iterations, we got symptomatic hypotensive episodes down to less than 5% of treatments, where it remained for years. We thought our success contributed to our relatively low rate of skipping scheduled treatments. Given the counting problem, though, CMS has never attempted making this a quality goal, preferring instead to use measures that are easier to audit.
The bad, although it had some good aspects, is illustrated by the “Fistula First” campaign. The Achilles’ heel of maintenance hemodialysis is the need to obtain repeated vascular access to remove and replace large amounts of blood. The original access was an external Scribner shunt, now a tunneled dual-lumen catheter, which was prone to clotting, infection, and had a limited life. This was replaced by creation of an internal fistula between the radial artery and vein at the wrist. But by the 1990’s, fistulas were uncommon, and the majority of patients had interposition grafts of synthetic material between the artery and vein used for vascular access. Graft accesses were also prone to clotting, infection, and generally were worn out after two years and had to be replaced. CMS looked at the data and decided it would be “better” (and cheaper) fistulas were the prevalent type of access. The did adjust payments so that surgeons were not penalized and created codes to pay for preoperative assessment to increase success rates, which was good. But it also set a goal of 80% of patients having a fistula, when only one place in the country was reporting that sort of success rate. Over the years it became apparent that surgeons were placing fistulas and units were counting them, even though they were never usable for dialysis. Not only was the 80% goal too ambitious, it did not benefit patients, and would have, perhaps, been more effective if it were fashioned as a “Catheter Last” campaign.
The ugly is illustrated by a paper recently published by Sheetz and associates, examining the impact of the “Quality Incentive Program” initiated by CMS in 2012, examining the effect of financial penalties applied to 19% of units in 2017, on their quality performance in 2017 and 2018. Perhaps to the authors’ surprise, but not to mine, they found no change in performance could be measured in penalized units. Now the items that are included in the score represent issues that impact patients at the individual level, but are mostly not modifiable by changes in dialysis center practices. Further, units are penalized for being in the bottom quintile, not for being statistical outliers. Units subject to penalties turned out to be located in ZIP codes with higher percentages of non-white residents, and those with lower median income. Since these units also tend to serve fewer patients with commercial insurance, they are more dependent upon Medicare funding, so one would predict they were highly motivated to “fix” the issues.
This study cannot illuminate motivation, but perhaps these units are like the duck in a story I used to read when I was a child. Every day at dark, the ducks had to get on their home ship, but the last duck on always got a beating. One day the last duck (Ping) decided not to get on, so avoiding the beating, but facing all sorts of dangers. When he finally got a chance to get on board the ship again, even though he was last, he took his beating. I’m not sure why my mother thought the book was one to bring home, but I did not introduce my children to it. The story of what is inherently random punishment, much like Jack Welch’s approach to firing the bottom 5% of employees every year units, actually serves to create an anxiety-ridden, toxic culture, which is not one conducive to providing care to others. Units subjected to beatings for things over which they have no control may take their beating for a while, but eventually will decide to quit. Since there is no evidence that these units are “killing” patients, this would represent a major disservice from a program designed to make things better.
So CQI is good when used by care teams to work on problems that matter to them and their patients, and for which there are actions that can be taken to make things better. It is occasionally bad, though, because our inability to predict the future means that our efforts will produce unintended consequences which may be worse than the original problem. If the program is local, this can be chalked up as a learning experience and the program can be re-designed. When it is a federal program, though, that sort of change takes more than an Act of Congress. Finally, if success or failure is arbitrarily defined, not statistically defined as being an outlier, it gets ugly.
Like the spaghetti western, CQI is a perfect metaphor of the times—a useful method for improving and evaluating medical care has been corrupted by being used mostly to reduce expenditures. The resulting cynicism is contributing, along with EMR’s, and the pandemic with its associated stressors, to widespread demoralization of physicians, nurses and other clinical workers. Maybe, like Ping, they will come home and take the beating, but I would not bet on it.
10 October 2021
 Sheetz KH, Gerhardinger L, Ryan AM, Waits SA. Changes in Dialysis Center Quality Associated With the End-Stage Renal Disease Quality Incentive Program: an Observational Study With a Regression Discontinuity Design. Ann Intern Med 2021;174:1058-1064. doi. 10.7326/M20-6662.
 Flack, Marjorie and Weise, Kurt. The Story About Ping. Viking Press, 1933.
Confronting The Quality Paradox - Part 1
Improving Clinical Quality Under Fire
Touro Infirmary has been in the news but not in a good way. What should it do?
More on the Quality Paradox
The quality paradox is the number may improve while the experience of care worsens. What's new?
Paying for What We Don't Want
Do you believe the proverb "you get what you pay for"? What if you pay for what you don't want?
Quality Improvement 3.0
Is it time for clinical quality improvement 3.0?