Guest Essay

Dr. Sharon Ehrmeyer is back again! You may remember her article on regulations in our Basic QC Practices series. In this article, she explains which regulations and standards and agencies are involved with Method Validation. A must-read reference.

• General requirements for quality control
• Method validation recommendations depend on test complexity
• Waived tests
• Unmodified moderate and high complexity tests
• Modified moderate and high complexity tests
• Documentation
• Additional requirements for periodic verification
• JCAHO standards for method verification
• CAP Laboratory Accreditation Program guidelines for method verification
• Whose standards do you need to satisfy?
• References

Please note: Consult our CLIA Final Rules section for the latest on regulations for Method Validation.

The Clinical Laboratory Improvement Amendments of 1988 (CLIA'88) include specific regulations for laboratory quality control, which contain some specific recommendations for method validation [1]. Because of delays in implementing the regulations [2,3], it is difficult to understand which standards are in effect and which ones have been delayed. Therefore, in this discussion, the intent of the regulations is described as "recommendations" rather than "requirements."

Until the CLIA regulations are fully implemented, it will be important to consider the requirements of accreditation organizations with deemed status such as the Joint Commission for Accreditation of Healthcare Organizations (JCAHO) [4], the College of American Pathologists (CAP) [5], and the Commission on Office Laboratory Accreditation (COLA)[6]. There are also state health laboratory organizations in Washington, Oregon, and New York, that are approved by the government (have exempt status) and impose specific requirements [7]. And, it is always important to keep in mind those practices that would be expected as part of professional responsibility and good laboratory practice.

General requirements for quality control of laboratory tests

CLIA provides a broad framework of quality control. Subpart K begins with the statement:

"The laboratory must establish and follow written QC procedures for monitoring and evaluating the quality of the analytical testing process of each method to assure the accuracy and reliability of patient test results and reports."

The regulations then proceed to define a broad set of standards for quality control, which include sections on each of the following:

• Facilities (§493.1204)
• Test methods, equipment, instrumentation, reagents, materials, and supplies (§493.1205)
• Procedure manual (§493.1211)
• Establishment and verification of method performance specifications (§493.1213)
• Equipment maintenance and function checks (§493.1215)
• Calibration and calibration verification procedures (§493.1217)
• Control procedures (§493.1218)
• Remedial actions (§493.1219)
• Quality control records (§493.1221)

The section on the "establishment and verification of method performance specifications" provides the recommendations for method validation studies.

MV recommendations depend on test complexity

The recommendations that must be followed depend on the "complexity" of the test. The categories are waived, moderate complexity, and high complexity. Information about classification of specific tests is available from the CDC web-site at the address: http://www.hcfa.gov/medicaid/clia/cliahome.htm. Each of these categories has different regulatory requirements for personnel, quality control, quality assurance, proficiency testing, etc.

Waived tests. The minimum requirement for waived testing is to follow the manufacturers' directions. There are no recommendations for method validation.

Unmodified moderate and high complexity tests. Most tests performed in laboratories today fall in these categories, thus most method validation studies should follow these recomendations. Section §493.1213(b)(1) identifies what is intended when CLIA is finalized:

"…a laboratory that introduces a new procedure for patient testing using a moderate or high complexity method (instrument, kit, or test system) cleared by the FDA as meeting the CLIA requirements for general quality control, must demonstrate that prior to reporting patient test results, it can obtain the performance specifications for accuracy, precision, and reportable range of patient test results, comparable to those established by the manufacturer. The laboratory must also verify that the manufacturer's reference range is appropriate for the laboratory's patient population."

This would generally mean performing four different experiments:

• a comparison of methods experiment to estimate inaccuracy or bias,
• a replication experiment to estimate imprecision,
• a linearity type experiment to determine the reportable range, and
• collecting reference values to verify the reference range [alternatively, the laboratory medical director can document that the manufacturer's ranges or textbook ranges are appropriate for the clientele being served].

Modified moderate or high complexity tests. Section §493.1213(b)(2) identifies what is intended when CLIA is fully implemented:

• "…a laboratory that introduces a new procedure for patient testing using: a method developed in-house; a modification of the manufacturer's test procedure; or a method (instrument, kit, or test system) that has not been cleared by FDA as meeting the CLIA requirements for general quality control, must, prior to reporting patient test results:
• (i) verify or establish for each method the performance specifications for the following performance characteristics, as applicable:
• (A) Accuracy;
• (B) Precision;
• (C) Analytical sensitivity;
• (D) Analytical specificity to include interfering substances;
• (E) Reportable range of patient test results;
• (F) Reference range(s); and
• (G) Any other performance characteristics required for test performance.
• (ii) Based upon the performance specifications verified or established in accordance with paragraph (b)(2)(i) of this section, establish calibration and control procedures for patient testing as required under §493.1217 and §493.1218.

This would mean performing seven different experiments:

• a comparison of methods experiment to estimate inaccuracy or bias,
• a replication experiment to estimate imprecision
• a detection limit for estimating analytical sensitivity
• interference experiments for estimating constant interferences,
• recovery experiments for estimating proportional interferences,
• a linearity type experiment to determine the reportable range, and
• a more extensive reference value study to estimate the reference range(s).

Documentation. Section §493.1213(c) requires that "the laboratory must have documentation of the verification or establishment of all applicable test performance specifications.

Additional requirements for periodic validation of method performance. Ongoing (at least semi-annual) assessment of accuracy is required under Subpart P - Quality Assurance. The need for ongoing method comparison studies is given in §493.1709 Standard; Comparison of test results", which states:

"If a laboratory performs the same test using different methodologies or instruments, or performs the same test at multiple testing sites, the laboratory must have a system that twice a year evaluates and defines the relationship between test results using different methodologies, instruments, or testing sites.

"In addition, if a laboratory performs tests that are not included under Subpart I, Proficiency Testing, the laboratory must have a system for verifying the accuracy and reliability of its test results at least twice a year."

JCAHO standards for method validation [4]

Waived tests. JCAHO recognizes the waived tests as defined by CLIA'88, but requires more than just following manufacturer's directions. JCAHO's standard WT.1.3, "policies and procedures governing specific testing-related processes are current and readily available," includes equipment performance evaluation. In discussing the standard with Anne Belanger, director of JCAHO's Laboratory Accreditation Services, she indicated this means having some evidence (available for the inspectors) that all tests are meeting the needs of the testing site's clientele. The evidence includes: accuracy, precision, and reportable range (validity of low and high values) data. In addition, someone (i.e., the testing site's director) will need to verify that the reference range(s) taken from the manufacturer or elsewhere are appropriate.

For JCAHO, it is only necessary to evaluate the method, not all instruments, i.e., all glucose meters, employing the method. As part of JCAHO's total quality management philosophy, performance history, including QC data, can be used as the "evidence" of adequate method performance when a method used in one location is instituted in another.

Unmodified Moderate complexity tests. While JCAHO requires the same general QC requirements as CLIA, JCAHO also has specific method validation requirements (QC.1.2). JCAHO identifies the intent of QC.1.2 as follows:

"before a new test method is used to report patient results, the laboratory must verify that the method will produce accurate results on a consistent and reliable basis...the laboratory at least verifies accuracy, precision, and reportable range for a patient-testing procedure that is an approved, unmodified test of moderate complexity for which the manufacturer has established the performance specifications. The laboratory also ensures that the reference range applies to the specific patient population(s) tested."

"For previously established methods, QC data and test performance history are adequate to confirm validity."

High complexity tests. Testing sites that modify moderate complexity tests (this includes not following the manufacturer's directions) or use tests developed in-house or tests classified as high complexity under CLIA must follow all the method validation requirements identified in Section §493.1213 of the CLIA regulations for high complexity tests. These requirements include: accuracy; precision; analytical sensitivity and specificity, if applicable; the reportable range of patient test results, the reference ranges(s) (normal values); and any other applicable performance characteristic that may be appropriate for the test.

CAP Laboratory Accreditation Program standards for method validation [5]

CAP-LAP's philosophy is that all clinical laboratory testing, including even CLIA waived tests, essentially need to meet the requirements defined for high complexity testing under Section §493.1213 of CLIA'88.

The 1998.1 CAP-LAP Checklist (Laboratory General -- Section 1) includes performance specification requirements -- accuracy (01:4202), precision (01:4202), reportable range (01:4204), and reference range (01:4208)-- for each test procedure. Sensitivity is discussed under reportable range and is identified as the "minimal limit of quantification." Specificity (01:4203) implies an evaluation of the method's ability to respond correctly to the concentration of analyte in the presence of interfering substances. CAP's Checklist now states: "that interfering substances may pose a significant problem to the clinical laboratory and healthcare providers...the laboratory must be aware of common interferences by performing studies or having available studies performed elsewhere (such as by the instrument-reagent manufacturer) that are consistent with NCCLS EP7-A."

Whose standards do you need to satisfy?

You can select an approved ("deemed" status) organization whose requirements are at least equivalent to CLIA'88 regulations.

The Health Care Financing Administration (HCFA) will inspect any size laboratory, including physician office laboratories, for adherence to the CLIA requirements; the Commission of Office Laboratory Accreditation (COLA) inspects primarily physicians office laboratories for adherence to COLA standards which closely parallel the CLIA regulations; the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) inspects laboratories and testing sites as well as the entire hospital or healthcare organization; and the College of American Pathologists Laboratory Accreditation Program (CAP-LAP) mainly inspects large laboratories directed by pathologists. There are other government approved (deemed) organizations that have standards for laboratories to follow and some states, such as Washington, Oregon and New York, impose specific requirements.

References

1. U.S. Department of Health and Human Services. Medicare, Medicaid and CLIA programs: Regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). Final rule. Fed Regist 1992; 57:7002-186.
2. U.S. Department of Health and Human Services. Medicare, Medicaid and CLIA programs: Regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA) and Clinical Laboratory Act program fee collection. Fed Regist 1993:58:5215-37.
3. U.S. Department of Health and Human Services. Medicare, Medicaid and CLIA Programs: Extension of Certain Effective Dates for Clinical Laboratory Requirements Under CLIA. Fed Regist 1997;62:25855-58.
4. Accreditation Manual for Pathology and Clinical laboratory Services. Joint Commission on Accreditation of Healthcare Organizations (JCAHO). Oakbrook Terrace, IL, 1998.
5. Laboratory Accreditation Program. College of American Pathologists (CAP). Northfield, IL, 1998
6. Accreditation Manual. Commission on Office Accreditation (COLA). Columbia, MD, 1998.
7. Laessig RH, Ehrmeyer SS: New Poor Man's (Person's) Guide to Meeting the Regulations. R & S Consultants, Madison WI. 1998.