Essays

Previous articles in this series:

• 2008: The year of EQC, AQC or No QC at all?
• Equivocal QC: Coming to a Lab near you
• CLSI faces the challenge of quality

August 2008

At the AACC meeting in Washington DC, CLSI, CMS, and FDA presented a session on “Perspectives on Quality: Updates – EP22 and EP23 (CLSI), Accreditation Preparedness (CMS), and IVD Regulations (FDA).

EP22 and EP23 form the basis for what is now called “Alternate QC” (AQC). EP22 is titled Presentation of Manufacturer’s Risk Information and EP23 is now titled Laboratory Quality Control Based on Risk Management. Both have been preceded by EP18-2A Risk Management Techniques to Identify and Control Laboratory Error Sources; Proposed Guideline - Second Edition which provides guidance to manufacturers on how to perform risk assessment.

Greg Cooper, chairman of the EP22 subcommittee, provided the historical context for these documents and the original purpose of EP22 to “provide statistical and scientific guidance to allow manufacturers to make QC frequency recommendations in their labeling.” Originally proposed by ADVAMED as “EQC option #4”, EP22 was hamstrung by a lack of peer-reviewed science in clinical diagnostics on the issue of frequency of QC. Without that science, it became impossible to develop a guideline that could provide quantitative methodology and guidance for establishing the optimum frequency of QC. But the committee didn't give up. Instead, the committee changed its focus to the use of risk assessment to identify specific “failure modes” that should be mitigated by the manufacturer to reduce their possible impact on the quality of testing.

The current objective of EP22 is to provide guidance to manufacturers on how to present the results of their risk assessment studies in an understandable way that will help laboratory users identify those “residual risks” that must be mitigated and/or monitored in the laboratory. The recommended format includes information about “(1)device feature or recommended action, (2) targeted failure mode, (3) description of how feature or recommended action performs its intended function, (4) known limitations of feature or recommendation action, (5) actions required to address known limitations, and (6) studies performed to verify feature/recommendation achieves intended purpose.” Item #5 would alert laboratory users to “residual risks” that have been passed on to the laboratory.

James Nichols, chairman of the EP23 subcommittee, indicated this document is “intended for users of laboratory and POC systems with alternative control processes.” The purpose is to provide users the guidance needed to “develop effective, cost-efficient QC protocols that will ensure appropriate application of local regulatory requirements based on the technologies selected by the lab and reflective of the lab’s unique environmental aspects” (such as personnel competency, temperature, storage conditions, clinical use of test results, etc.).

Basically, EP23 hopes to provide more flexibility to meet the CLIA requirements for QC, particularly alternatives to the minimum requirement of two levels of QC each day of testing. To accomplish this, EP23 will provide a QC toolbox of available quality control techniques, procedures, and processes that can be employed in place of the traditional “surrogate” QC, i.e., analysis of liquid control materials.

EP22 has been voted on by the subcommittee and will next be submitted to the EP Area Committee for approval. EP23 is in the final stage of drafting and will need to be voted on by both the subcommittee and area committee. The plan is to release both documents at the same time because they are so closely coupled. The target is early 2009 for the “proposed” versions of these documents.

Judy Yost from CMS discussed “How to avoid the most frequently cited deficiencies”. CMS’s “top ten” include the following:

• 493.1236. Alternative assessment for tests not covered by PT (5.9%);
• 493.1252. Laboratory must select test system (5.7%);
• 493.1252. Laboratory defines storage conditions (5.6%);
• 493.1289. Analytical system QA (5.6%);
• 493.1252. Procedure manual content (5.0%);
• 493.1239. General laboratory systems QA (4.2%);
• 493.1407. Lab director established QA plan (4.2%)
• 493.1291. Lab report content (4.0%)
• 493.1252. No expired reagents (3.8%)
• 493.1403. Unqualified lab director (3.5%)

It is important to note that this information is based only on CMS’s own surveys, therefore it probably represents smaller laboratories than those accredited by CAP and JC. It is interesting to notice that QC no longer appears in the top ten (unless “analytical systems QA” means QC). I find that puzzling, but that may be because many CMS labs have implemented EQC and no longer do much QC. Remember that CMS has allowed labs to implement EQC for some 3 years now and stated publicly that inspectors have not observed any problems in the field. (I still wonder how they would know whether or not problems are occurring, since they have no QC data to inspect!)

Perhaps the most interesting information came from Alberto Gutierrez who spoke on the “FDA Regulation of IVDs: Assuring Quality?” The question mark in the title is his, not mine, but I certainty have my own concerns because FDA has never implemented the “QC clearance” process that was intended as part of the original 1992 CLIA rules. It was also the purpose of “EQC Option #4” to have a manufacturer’s recommendations for QC reviewed and approved by the FDA, but that didn’t happen, either. Now with EP22, manufacturers may provide information on the failure modes of their systems to help users manage the quality of testing, but remember that supplying risk information is not mandatory.

Dr. Gutierrez was asked whether such risk information would be considered part of labeling and therefore subject to the “truth in labeling” provision. He responded that it was considered part of labeling, but would not be subject to any prior review or approval. When further asked what would happen if the labeling wasn’t truthful, he indicated that the FDA could always choose to review the labeling as part of its “misbranding authority.” That makes me wonder whether manufacturers will choose to provide any information on failure modes when there is no pre-approval, but later on may be subjected to review. It seems to me that would be risky business for a manufacturer and, in the spirit of risk mitigation, they might decide it is safest NOT to provide customers with that information!

Another question raised the issue - if manufacturers don’t provide laboratories with risk information, then what would be the value of EP23? One response was that EP23 would "stand on its own" and still be useful to laboratories. That means the laboratory director would now be responsible for making his/her own assessment of failure modes and risks, as well as identifying those measures for mitigating and monitoring risks. That’s a daunting task, and I worry that those 3.5% of unqualified laboratory directors might not know any better and I’m not so sure about the remaining 96.5% either! I would think most laboratory directors would find it too difficult and would resort to an easier way, such as the default QC of 2 levels per day.

Remember, with EQC and AQC, we’re primarily considering how to save money by doing less QC. The objective is to reduce the number of failures, which will reduce internal failure-costs, such as the cost of QC, cost of repeating QC, cost of repeating patients, cost of trouble-shooting, etc.). We should also worry about external failure-costs, i.e., the costs of producing wrong test results, which may lead to the wrong diagnosis, wrong treatment, or wrong assessment of treatment, which may have much larger costs. Those costs won’t show up on the laboratory budget, however, so they may not receive as much attention.

In summary, the original 2 year timeline for resolution of the issues with EQC has now become a 4 year time period during which EQC has been successfully established de facto by CMS, and AQC is now being proposed as a further alternative to doing statistical QC. The original focus on POC tests has morphed to analytic systems in general and AQC will most likely be adopted and adapted by large laboratories with skilled analysts, rather than POC applications where EQC represents the simplest way to reduce the QC and still be in compliance with the CLIA regulations.

AQC may indeed turn out to be valuable for customizing QC to fit the specific needs of an analytic system operated in a particular laboratory. However, I would caution laboratories to keep in mind that statistical QC is still an important tool in the QC toolbox and may, in fact, be the most universal and powerful tool that is available. When all is said and done, you will need statistical QC to monitor that “residual risk” that manufacturers pass on to the laboratory.

James O. Westgard, PhD, is a professor emeritus of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.