A Broken Process

Submitted by francois.debrantes@hci3.org on Friday, September 11, 2015 - 01:48

Newtown, CT – September 11, 2015

Let's start this week with a seemingly simple quiz. Assume you are part of an Advisory Committee to the NQF and you have to vote on the following measures. Which would you recommend for endorsement?

  • A process measure that has no demonstrable reliability to differentiate provider performance, using data sources that are heterogeneous and potentially unreliable, and that has not been tested empirically outside of one health system with 124 patients.
  • A composite measure of avoidable complications that is defined as an outcomes measure, with testing on multiple very large datasets across millions of health plan members and thousands of providers, showing robust and statistically valid reliability scores, and therefore able to differentiate provider performance.

Unfortunately for us, you're not members of the Committee, so your vote doesn't count. And, as incredulous as it sounds, prior to rejecting many of our measures of avoidable complications, the NQF committee voted to recommend the first measure listed above for endorsement. Only in a place so divorced from reality as Washington, D.C. does this make sense. For the rest of us, it's just part of the Theater of the Absurd.

What this means to you – To say that the NQF process is broken is clearly an understatement. The personal biases of the measure reviewers who lead the discussion and vote on a specific measure, was laid bare and the overwhelming bias is for the type of very tightly defined and scoped accountability that has given us the useless measures that we mostly have today. It's pretty clear to us that the threat our measures represented to them was very real and they took it personally. As one reviewer stated: "if you were to measure these potentially avoidable complications at the health system or ACO level, we would be ok with it." In other words it's fine to measure an organization behind which individuals can hide and always point to someone else as the culprit. In fact, they ended up by recommending one of our measures for endorsement because it was at the facility level and not at the clinician level. As another Committee member stated: "I'm fine with many of the avoidable complications you have that are directly related to the procedure, but I can't be responsible for all that patient safety stuff." And herein lies the rub, the paradox, and the problem for the country. What mattered to the majority of the Committee was how the measure would impact them, not whether it was a reliable and useful measure for consumers, employers, or for those managing patients across settings. And this, at a time when more and more providers are taking on risk contracts to do just that. As I reminded one of the Committee members, in Medicare's recently announced mandatory joint replacement bundle, all of the PACs we've defined are in, and then some. So rest assured, in this instance, and until things at the NQF change dramatically, not being recommended for endorsement is probably the best endorsement we can get. Because as Jack Welch was fond of saying: "You know you're hitting the target when you're getting a lot of flack." As far as we're concerned, it's bombs away folks, and for all purchasers and payers out there, if you want to see change happen in this country, stop looking to DC, because that's truly the swamp of the status quo.