Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Essays
  4. Trends
  5. A momentous happening - A New Way to do QC!

Trends

A momentous happening - A New Way to do QC!

A new QC technology has been cleared that replaces the use of traditional external quality controls. This happened because a manufacturer submited a claim for this new QC technology and provided the documentation to defend that claim. Read all about it!

A major shortcoming of the US CLIA regulations has been the delay in the implementing the QC clearance provision. That provision has been postponed again and again and again and again, delaying the final CLIA regulations by ten years - so far.

Until the QC clearance provision is implemented, the complete QC regulations are not in effect. Laboratories are only required to analyze two or three levels of control materials each day of testing, without validating the effectiveness of the QC procedures being used. In this ten-year interlude, QC has come to mean simply running controls, with little or no attention to setting the control limits correctly, utilizing appropriate control rules, or analyzing a sufficient number of control materials. The adequacy of the QC procedure for the intended medical use of the test has not been validated, which would be required - either by the manufacturer or the laboratory - if the complete regulations were implemented.

Meanwhile, electronic QC seems to have been accepted for POC applications, primarily because it is better than doing nothing. There actually has been no formal approval; more importantly, there has been no formal disapproval. Instead a temporary allowance has been made, pending the release of the final, final, final, final CLIA rules. While there is little scientific validity for electronic QC, it will continue to be used until the CLIA QC standard is fully implemented. For further discussion, see https://westgard.com/essay17.htm.

CLIA Standard for QC

Here's the exact language of the standard for quality control, as given in section 493.1218 of the Federal Register [Vol 57, No. 40, page 7166, February 28, 1992]:

"Control procedures are performed on a routine basis to monitor the stability of the method or test system; control and calibration materials provide a means to indirectly assess the accuracy and precision of patient test results. Control procedures must be performed as defined in this section unless otherwise specified in 493.1223 through 493.1285 of this subpart.

(a) For each method cleared by the FDA as meeting the CLIA requirements for general quality control, the laboratory must, at a minimum, follow the manufacturer's instructions for control procedures. In addition, the laboratory must meet the requirements under paragraphs (c) through (e) of this section and, as applicable, paragraph (f) of this section.

(b) For each method that is developed in-house, is a modification of the manufacturer's test procedure, or is a method that has not been cleared by the FDA as meeting the CLIA requirements for general quality control, the laboratory must evaluate instrument and reagent stability and operator variance in determining the number, type, and frequency of testing calibration or control materials and establish criteria for acceptability used to monitor test performance during a run of patient specimen(s). A run is an interval within which the accuracy and precision of a testing system is expected to be stable, but cannot be greater than 24 hours…"

Note the phrase in italics. This means that the laboratory must account for the observed performance of the method (instrument and reagent stability and operator variance) in determining the number of controls analyzed and the statistical control rules (criteria) used to judge acceptability. The laboratory is supposed to demonstrate that the statistical QC procedures are valid or follow a manufacturer's QC instructions have been cleared by FDA.

The QC clearance provision was intended to provide manufacturers with flexibility to develop new and non-traditional QC procedures that would be appropriate for new measurement technology, as long as they demonstrated that their QC instructions satisfied the CLIA requirements. Laboratories would then be allowed to follow the manufacturer's QC instructions and be in compliance with the CLIA rules. FDA was expected to provide guidelines that tell a manufacturer how to validate their QC instructions, but that never happened. Meanwhile, for the past ten years, manufacturers have only provided generic QC instructions without any documentation of their validity.

A momentous happening!

In September 2002, the FDA approved the labeling of a new product called iQM (intelligent Quality Management). Here's the exact 510(k) submission claim that was approved:

Intelligent Quality Management (iQM™) is being introduced on the GEM® Premier 3000 as an active quality process control program designed to provide continuous monitoring of the analytical process with real-time automatic error detection, automatic correction of the system and automatic documentation of all corrective actions, replacing the use of traditional external quality controls.

A new QC technology has been cleared that replaces the use of traditional external quality controls. This happened because a manufacturer - Instrumentation Laboratory - submited a claim for a new QC technology and provided the documentation to defend that claim.

What is iQM?

iQM is a totally automated statistical QC process for IL's GEM Premier 3000 analyzer. The GEM analyzer has a closed and disposable cartridge that contains all the reagents and sensors for performing a variety of tests, such as blood gas measurements, sodium, potassium, ionized calcium, glucose, lactate, and hematocrit. Three internal process control solutions (PCSs) are used to monitor test performance throughout the lifetime of the cartridge, which is a maximum of twenty-one days. Internal drift limits act as statistical control limits in the iQM software, alerting the instrument to potential problems. iQM also includes diagnostic pattern recognition algorithms to identify certain problems, such as micro clots, which can then be remedied automatically by corrective actions initiated and controlled by the instrument itself. If corrective actions do not resolve the problems, then the instrument disables the tests that are affected. The entire monitoring and corrective action process takes place automatically without any intervention from the operator.

How does iQM work?

The three process control solutions are run at different frequencies, e.g., PCS C is analyzed every 24 hours, PCS A every 4 hours, and PCS B every 3 to 30 minutes. Obviously, the frequency of analysis of PCS B makes it the material that is most likely to detect problems the quickest. Solution B follows every patient sample, which accounts for a 3 minute frequency during heavy workload. In applications with low workload, PCS B is analyzed at 30 minute intervals even is there are no patient samples being analyzed. Thus, control samples are being analyzed much more frequently than required by the CLIA regulations (which is 1 control every 8 hours).

Each test on each of these three process control solutions has its own statistical limits, which are defined by the manufacturer's "drift specifications". iQM is statistical process control and, because of the frequent monitoring via the internal PCS materials, can be expected to provide better performance than traditional QC procedures.

How does a manufacturer validate QC performance?

A new ISO document that is nearing completion provides guidelines for "validation of manufacturer's recommended procedures for user quality control" [1]. It recommends the following information be provided, when applicable:

  • the type of error that the quality control procedure is intended to detect;
  • control materials that may be used;
  • control materials that may not be used;
  • recommended analyte concentrations;
  • guidelines for determining acceptability criteria (control limits); and
  • the probabilities of detecting and not detecting an inaccurate result.

For electrochemical measurements such as performed by the GEM analyzer, the type of error of interest is electrode drift, which is a systematic error. Ideally, that drift should be detected before it reaches a size that would cause medically important errors. The size of medically important errors can be determined by defining the quality required for a test, e.g., the quality required for successful performance of proficiency testing as described by the CLIA regulations "criteria for acceptable performance". The internal process control solutions provide the recommended analyte concentrations. No other control materials are needed. The drift specifications define the acceptability criteria or control limits. These specifications can be understood in statistical terms by dividing the drift limits by the observed SDs for the tests, which converts them to statistical control limits or statistical control rules. Once the statistical limit or rule is known, it is possible to determine the probabilities for false rejection and error detection.

These probabilities can be converted to "average run lengths" (ARLs), i.e., the average number of times a PCS solution needs to be measured before a run will be rejected. Given the time intervals for measuring the different PCS solutions, the ARLs can be multiplied by the sampling time to provide a practical measure of performance in terms of the "average time to error detection," e.g., the average time needed to detect medically important errors.

What performance is expected for iQM?

Based on the manufacturer's drift specifications, the method performance observed on twenty-four cartridges, and the CLIA quality requirements for acceptability in proficiency testing events, the probabilities of detecting medically important errors have been determined, converted to average run lengths, and finally converted to average times for error detection. [The details of these calculations are described in the lesson Areas Under a Table, which discusses utilizing a table of areas under a normal curve to determine probabilities for false rejection and error detection.]

The average times for error detection are expected to be 3 to 33 minutes for pH, PCO2, PO2, K+, Ca++, lactate, and hematocrit for PCS B, which is the material analyzed most frequently. For glucose, the average detection time is 7 to 71 minutes; for sodium, 10 to 102 minutes. The lower figures correspond to the heavy workload situation where PCS B is analyzed every 3 minutes and the longer times are for the low workload situation or "standby" when PCS B is analyzed every 30 minutes.

How does iQM performance compare with traditional QC?

For laboratories that perform the minimum QC required by CLIA, the earliest that errors can be detected is 8 hours. That assumes the QC procedure achieves a Ped of 1.00, which we know will be difficult for critical care tests such as PO2, PCO2, and Na+ [2]. Even so, which means making an optimistic assessment of current QC procedures, iQM is expected to detect problems much faster than traditional QC procedures - in minutes instead of hours, shifts, or days.

This is good news for laboratories. Better quality control can be provided without any increased efforts by analysts and operators, without any further laboratory investment in the education and training, and with complete automation and documentation. The bar has been raised! A new standard for quality control has been set. Manufacturers can gain FDA approval for new and innovative QC technology.

References

  1. International Organization for Standardization. In vitro diagnostic medical devices - Validation of manufacturer's recommended procedures for user quality control. ISO?DIS 15198. Geneva, Switzerland:ISO, 2000.
  2. Olafsdottir E, Westgard JO, Ehrmeyer SS, Fallon KD. Matrix effects and the performance and selection of quality-control procedures to monitor PO2 measurements. Clin Chem 1996;42:392-396.

James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.