Essays

November 2007

Despite the universal complaints, Despite the CMS admission that "We blew it",Despite a complete lack of scientific validity, "EQC" options are coming to a laboratory near you. See how it's going to happen.

• EQC is now officially compliant
• It's your Laboratory Director's decision and responsibility
• It's your Laboratory's risk and liability
• For more information about "Equivalent QC"

On December 6, 2007, AACC is sponsoring an audio conference on “New Developments in CLIA and QC”, which is described as follows:

“QC remains the No. 1 deficiency identified by CLIA surveyors and is a continuing concern for many in the clinical laboratory. During this audio conference, CMS’ CLIA administrator, Judith Yost, MA, MT(ASCP), will describe the imminent policy changes CMS has planned, and will discuss QC under CLIA, providing an overview of the most common QC deficiencies and a glimpse into what the future holds for QC. James Nichols will discuss strategies for reducing some of these common QC problems, and will describe how his laboratory succeeded in implementing alternative QC measures for several test systems.”

For a preview of the CMS update regarding the CLIA QC policies related to what they call "Equivalent QC" (EQC) – see CMS Guidance Document: Survey and Policy Letter, posted on the CMS website: Memo 07-33: Continuation and Revision of the Components of the CLIA Educational Period Regarding QC

EQC is now officially compliant!

In this updated guidance to State Survey Agencies, CMS now officially implements the flawed EQC procedures originally introduced in the State Operations Manual in 2004.

“Laboratories that meet the QC regulations at 42 CFR 493.1256(c)(3) or perform the CMS EQC protocol correctly, as written in the Surveyors’ Interpretive Guidelines, will be determined to be in compliance with CLIA QC procedures and will not receive a letter or CMS-2567 indicating non-compliance.”

Remember, the protocol for EQC option 1 (which will be most desirable and commonly adopted option) requires a laboratory to test two levels of external controls along with the manufacturer’s internal controls for a period of 10 days; if no out-of-control situations are observed, then the laboratory can reduce external QC to two levels once per month. Performing this EQC validation protocol correctly is not really an issue because the protocol itself is not correct! The protocol is obviously not valid – how can 10 days of testing prove the method will be stable for 30 days?

There probably is no one, except for CMS, CDC, and FDA, who will defend this EQC validation protocol as correct! Even manufacturers have objected to the option #1 protocol. Through CLSI, they initiated development of an option #4 to provide a rigorous validation of QC instructions that could be “cleared” by FDA. Unfortunately, after 2 years of development, option #4 didn’t happen and that effort is now being redirected to the development of “alternate QC” or “customized QC.” This approach calls for manufacturers to identify the risks of failure in their analytic systems and measurement procedures, eliminate those risks when possible, implement procedures to mitigate remaining risks, and identify the “residual risks” that must be managed in the laboratory. And all this is supposed to allow the reduction of external QC measurements. In this day when QC is still the number 1 deficiency in laboratory inspections, is it really good advice to do less QC, regardless whether it is called equivalent QC, alternate QC, or, more accurately, reduced QC?

It’s your Laboratory Director’s decision and responsibility!

Also keep in mind some of the clarifications provided by CMS in their new “EQC fact sheet”:

“For non-waived laboratories EQC is a choice to be determined by the Laboratory Director who selects the EQC option based on written manufacturer’s information regarding the extent internal QC monitors the analytic process (operator, analysis, environment) and the laboratory’s circumstances, i.e., staff competency, turnover, device stability, etc. The laboratory may use option 1 or 2 at their discretion.”

“If the laboratory chooses EQC, the surveyor can provide guidance by clarifying eligibility, and explaining the protocol and evaluation process, etc.”

“EQC permits the laboratory to decrease the frequency of external QC to save costs as long as the test system is stable, eligible, the laboratory successfully completes their evaluation process, and their quality system is functioning within acceptable limits.”

This last point is especially interesting – how will the laboratory know their quality system is functioning within acceptable limits when external controls are hardly ever being analyzed? The laboratory will only have 1 external control measurement per material per month. Given it takes a minimum of 20 control observations to get a reasonably good estimate of the standard deviation of a method, it will take a laboratory well over a year (20 months, 600 days) to accumulate enough data to make a quantitative assessment of the variance due to the operators, analysis, and environment.

How will you know that the variance due to the method, operators, and environmental conditions remains stable if you don’t have any data to properly characterize method performance? CMS thinks that can be done by annual competency assessment of operators, six month calibration verification and patient comparison checks, three proficiency testing events per year, plus following manufacturers directions for maintenance and function checks. Unfortunately that still leaves very little data that can be used to quantitatively assess whether the laboratory’s quality system is functioning within acceptable limits.

Oh, by the way, what are acceptable limits?

CMS is really assuming that a manufacture’s internal QC checks will detect any problems and will assure the quality of test results. But remember that manufacturer’s QC procedures and directions have not been “cleared” by FDA as originally intended by the CLIA regulations. Nor was the development of EQC Option #4 successful. Therefore, the laboratory must also assume that a manufacturer’s internal QC procedures are effective without any external review or verification by the FDA. Plus, you have to assume that the manufacturer never has any control problems with the production of reagents, parts, and materials, because there is hardly any control data that would allow the detection of those problems.

It’s your laboratory’s risk and liability!

In this age of patient safety and risk management, it is unquestionably risky to implement any EQC procedure on the basis of a protocol that is so obviously invalid. A laboratory should recognize its liability if it chooses to implement EQC. CMS clearly states that the laboratory director is responsible for any decision to do this. That also suggests that the laboratory can no longer get off the hook by saying it satisfies the CLIA QC requirements. CMS is no longer taking any responsibility for the effectiveness of EQC; they’re just telling you that you can do it to save money, but if you do it, you are liable for the effectiveness.

I’ll be happy to meet you in the courtroom to discuss this further!

We cover the topic of “How to formulate an analytical quality control system” in chapter 14 of Assuring the Right Quality Right. The graphic displayed at right shows a portion of the advice given there, detailing a "cause-and-effect diagram" with all the factors that must be addressed by the quality control system. If you want to customize your QC procedures, please proceed carefully and take into account the quality required for the particular test, as well as the performance observed for your method under the operating conditions in your laboratory.

Like it or not, it is still necessary to properly design your statistical QC procedures in order to monitor the “residual risks” of the testing processes in your laboratory. And it is very risky to reduce QC below CLIA’s already low minimum of 2 levels per day; in fact, many testing processes actually need more than 2 levels of control per day! Remember, compliance management isn’t the same as quality management!

Other materials on EQC on this website

We’ve written a lot about EQC over the last 4 years. Here are many of those materials, in the order of their appearance, beginning in late 2003.

1. QC – Quality or Compliance? Discusses the back requirements for analytic systems and describes the laboratory’s responsibilities for quality control.
2. Equivalent” Quality Control Practices. Raises the question of the meaning of the word “equivalent” in the context of QC.
3. Appropriate QC Procedures. Describes how sigma-metrics can be used to characterize method performance and to select the right QC.
4. Lies, Damn Lies, and “Equivalent” QC. Illustrates the issue of QC performance by analogy with a fire alarm.
5. More on CLIA QC and Q-less Compliance. Discusses how CMS’s earlier “approval” of electronic QC set the stage for introduction of equivalent QC.
6. Testing Equivalent Quality: A Better Way. Provides another discussion of the use of sigma-metrics for selection of the right QC procedures.
7. Final Rule vs Final Word on Quality. Argues for professional responsibility for quality rather than just compliance to regulations.
8. An Open Letter Regarding the Scientific Rationale for Equivalent QC. Here Westgard Web published a letter from a laboratory professional who asks CMS to explain their reasoning about EQC. Includes an official response from CMS.
9. An Examination of the Scientific Rationale for Equivalent QC. Discusses CMS’s rationale which cited the CLSI M2-A guideline “Performance Standards for Antimicrobial Disk Susceptibility Tests.”
10. How I Learned to Stop Worrying and Love EQC. A personal discussion of my beliefs in EQC in the spirit of Jonathon Swift.
11. EQC Option 4: Back to the Future. A review of the CLSI/CMS sponsored workshop on “QC for the Future,” including a proposal to develop an EQC Option #4.
12. The Current State of Equivocal QC Practices. An update of the views on EQC from our professional and accreditation organizations (AACC, CLMA, CAP, JCAHO, ASCLS).
13. An Open Letter: Mitigating the Future Risk of Poor Quality Laboratory Tests. Some advice about the risk of using risk management to develop the QC for the Future.
14. CLSI Faces the Challenge of Quality. See the section that provides an update on EQC Option #4.

James O. Westgard, PhD, is a professor emeritus of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.