CLIA Final Rule

CLIA Final Rule:
Lies, Damn Lies, and "Equivalent" QC

When the CLIA Final Rule Interpretive Guidelines came out, a new practice called "Equivalent" QC was introduced. Just what is it equivalent to? Well, if it was applied to fire safety, CMS would be telling you to turn off your smoke detectors.

• Equivalent to what?
  • Eqc option 1
  • Eqc option 2
  • Eqc option 3
• Smoke and fire, or smoke and mirrors?
• Who's clinically and legally responsible?
• What's the bottom line?

January 2004

The February 2004 issue of Clinical Laboratory News features an article “CLIA Interpretive Guidelines Debut on CMS Web Site: EQC options now applicable to broader spectrum of laboratory testing.” There’s considerable difference of opinion on the validity of the proposed equivalent QC procedures, at least, I beg to differ with any assertion that these equivalent QC procedures will provide the same level of performance and quality provided by the traditional QC procedures they supposedly can replace. It’s important for laboratories to make a careful assessment of these “equivalent QC options” because they may have a major impact on how QC is done, or more importantly, how QC is not done very often.

QC has remained one of the major deficiencies identified during laboratory inspections. Given this problem, it is difficult to understand the rationale for new Equivalent QC procedures that allow further reductions from daily QC to weekly or even monthly QC. We think this practice should be designated Eqc, i.e., big E little q little c, because it may lead to big Errors, little quality, and little control.

Eqc may also represent “easier quality compliance”! Many hospitals and laboratories will be happy to adopt this practice to minimize inspection issues. My sage advice is to move slowly and cautiously in this direction, because Eqc is a flawed concept and the recommended evaluation protocols are also flawed. Eqc is a step in the wrong direction.

Equivalent to what?

The regulatory language states that QC should “detect immediate errors”, but the Interpretive Guidelines allow a practice that reduces QC from once a day to once a week or perhaps only once a month. We've already discussed “Equivalent QC Practices” in another essay, but I think the total inadequacy of this idea needs to be repeated and repeated until everyone understands its absurdity. So let me try once more

• Eqc option 1 is for test systems with internal procedural controls that monitor the entire analytic process. How do you know the entire analytical process is being monitored? You’re supposed to ask the manufacturer and trust his answer. But it is the laboratory’s responsibility to evaluate the effectiveness of the internal procedural controls. The CMS guidelines specify that the evaluation requires the laboratory to run two levels of external control daily for 10 days. If the internal and external control results are acceptable throughout the 10 day period, the laboratory may reduce the frequency of external control testing to once every 30 days. How does that work? Observing no problems in a 10 day period somehow provides evidence that there won’t be any problems in a 30 day period! That’s absurd - a leap of faith, not a scientific evaluation!
• Eqc option 2 is intended for test systems with internal procedural controls that monitor a portion of the analytic process. The evaluation guidelines specify running two levels of external controls for 30 days. If the internal and external control results are acceptable for the 30 day period, the laboratory may reduce the frequency of external control testing to once a week. Well, at least the time dimension makes sense for this option. However, the evaluation protocol does NOT assure that the external QC procedure will detect medically important errors if they occur. How does the absence of out-of-control signals demonstrate that errors will be immediately detected if they occur? That’s absurd – the error detection capability isn’t even being tested, to say nothing of the lack of immediacy of 5 days!
• Eqc option 3 is intended for test systems without any internal procedure controls. The evaluation guidelines specify testing 2 levels of external controls for 60 days. If the external controls have not identified any problems during this time period, then they need only be analyzed once a week instead of daily. That’s absurd – again, the error detection capability isn’t even being tested.

Smoke and fire, or smoke and mirrors?

I often use the example of a smoke detector to explain quality control. There are many parallels.

• First of all, quality control is like safety – you do it regularly to be sure there aren’t any problems, even if the historical data shows there haven’t been any problems. No one would argue against the required use of fire alarms and smoke detectors in public and private buildings. People safety is too important to disregard!
• Second, the smoke alarm is a fire detector, whereas QC is an error detector. Smoke and fire are related, just like errors and quality. We want to detect fire immediately to assure the safety of the people, likewise, we want to detect errors in laboratory testing immediately to assure proper test interpretation and treatment for our patients, that is, patient safety. To do that, the detector must be carefully designed and properly implemented, e.g., smoke detectors must meet certain standards for performance, as well as being properly located and installed. One smoke detector in one classroom of a medical center is obviously inadequate, even if it performs correctly. Just having a smoke detector does not assure people safety! Just running controls does not assure patient safety!
• Third, the performance characteristics are similar – there may be both false alarms and true alarms. Everyone has experienced a false fire alarm and knows that it modifies their behavior when a second alarm occurs. Likewise, laboratory analysts experience false alarms, particularly when they employ 2 SD control limits, and have modified their behavior - often just repeating the controls rather than looking to see if there really is a problem.
• Concerning the other important performance characteristic – error detection, we recognize that the response of the detector will be related to the size of the fire. What size fire will be detected by your alarm system? For example, if you light one match right now, it is unlikely to set off the fire alarm. However, if you light a waste basket full of paper, there’s a much higher probability the alarm will go off. What size error will be detected by a laboratory QC procedure? It shouldn’t be a surprise that not all errors will be detected! But it is critical to know whether a medically important error will be detected if we are to assure patient safety? Both the smoke detector and the laboratory QC procedure need to be properly designed to have the necessary sensitivity or error detection.

If we were to test a new smoke detector according to “equivalent QC practices”, this is what we would do. Most smoke detectors would qualify for Eqc option 2 because they usually have an internal mechanism (procedural control) that allows us to manually check the alarm. That check doesn’t actually test the whole analytic process because it doesn’t introduce smoke, it only shorts out the detector to demonstrate that the batteries are still good and the horn still sounds. To prove that the new detector works okay, according to Eqc, we check each day for 30 days to see whether or not there is a fire. In the absence of observing a fire for 30 days and the absence of any alarms from the smoke detector, we now only need to check once a week to see if there is a fire. We trust the smoke detector will be watching out for us the rest of the time, even though we’ve never checked whether it can or will actually detect a real fire.

In the absence of any internal check for battery life and a working horn, a smoke detector would quality for Eqc option 3. That means we have to test the new smoke detector for 60 days and if we don’t observe anything, then we only need to check for fire once a week. Once again, we trust that the smoke detector actually works, even though we’ve never tested its error detection capability.

Eqc is really about smoke and mirrors, not smoke and fire. Without proper design and implementation of the daily statistical QC procedure, there is no evidence that the lack of alarms can be equated to the lack of errors and the presence of quality. Given that proper implementation of appropriate statistical QC procedures is still a major deficiency in laboratories, it is not realistic to expect laboratories to now use statistical QC to prove that QC can be further reduced from daily to weekly or even monthly.

Who’s clinically and legally responsible?

The most damning words about equivalent QC practices come from the Interpretive Guidelines themselves in section D5445:

NOTE: Since the purpose of control testing is to detect immediate errors and monitor performance over time, increasing the interval between control testing (i.e. weekly, or monthly) will require a more extensive evaluation of patient test results when control failure occurs (see 493.1282). The director must consider the laboratory's clinical and legal responsibility for providing accurate and reliable patient test results versus the cost implications of reducing the quality control testing frequency. [CMS emphasis, not ours]

The government is proposing a practice that it is not willing to condone. You can do this, you'll be in compliance with the letter of this particular law, but the government isn't willing to guarantee that it's clinically or legally acceptable. Does that inspire confidence in you? Would that inspire confidence in the physician users and patient customers you serve?

What’s the bottom line?

In an era where we are hearing more and more about medical errors in healthcare, where there are louder and louder calls for better quality and more safety, why are we degrading the practice of QC? Observations of current QC practices, even CMS’s own evidence from laboratory inspections, suggest that we need more QC, not less, in healthcare. Yet the effect of CLIA Final Rule and Interpretive Guidelines will be a further reduction in QC. Both manufacturers and laboratories are to benefit and to blame.

As of this writing, the US is in the middle of what's called a “jobless recovery.” That is, the economy is growing, but jobs are not being added. For some, “jobless recovery” is a contradiction in terms. For those who lost their jobs during the last recession and haven't found new employment, a jobless recovery simply isn't a recovery at all.

To “jobless recovery”, we need to add a new phrase “quality-less compliance.” The government is off the hook now that they've issued the Final CLIA Rule, and even more so because they have now said, in effect, “If these regulations aren't clinically responsible and legally defensible, that's not our fault, it's the fault of the Laboratory who decides to follow the regulations.” Manufacturers are off the hook, having avoided the need to obtain clearance of their QC instructions, shifting rather than sharing responsibility for quality with their laboratory users and customers. The Laboratory is responsible, which would be good if the Laboratory exercises that responsibility in a professional manner.

But Eqc is not a responsible option! If evidence-based medicine and patient safety are important in healthcare today, then professional responsibility requires that we own up to the potential shortcomings of Eqc - big errors, little quality, and little control. As laboratory professionals, we owe it to our patients to aim for excellence, not quality-less compliance!.