Guest Essay

In the US, when it comes to quality requirements, we have CLIA and clinical decision criteria and a whole host of other choices (many of them bad). But, if you can believe it, the situation in Europe is even more complicated! Dr. Per Hyltoft-Petersen discusses external quality assurance schemes, ISO, analytical goal-setting, and finally biological goal-setting.

• Review of approaches to analytical quality specifications
• Clinical goal setting
• Biological goal setting (basic concepts)
• Recommendations of two European working groups
• EGE-Lab
• EQA Organizers
• QC Selection/Design based on European biologic goals
• Calculation of biologic allowable error
• Calculation of medically important errors
• Example applications
• References
• QC Planning applications using European biologic goals
• Table of European biologic goals

Review of approaches to analytical quality specifications

Since Tonks [1], a number of different approaches to analytical goal-setting have been introduced based on the state of the art’, biology, and clinical usefulness. In the US, the introduction of the CLIA concept has focused on total error criteria for acceptable performance in proficiency testing surveys. These CLIA criteria are available on this website and have been discussed by Westgard in Quality goals, requirements, and specifications.

In Europe, Germany has chosen a concept close to the US, whereas the other European countries prefer external quality assessment with the so called educational schemes’ in order to establish a continuous improvement of quality. Two approaches to goal-setting -- clinical and biological -- have been evaluated and two European working groups (EGE-Lab and EQA-Organizers) have outlined recommendations for analytical quality specifications based on biology. Further, a new group under ECLM is working on quality specifications, also for quantities measured on ordinal and nominal scales. Internationally, ISO has a working group on analytical quality specifications and in the US there have been attempts to evaluate analytical quality specifications based on 'clinical outcomes’.

Clinical goal setting

The Nordic approaches (NORDKEM-projects)[2-3] are based on clinical situations/strategies where specific clinical decisions using a single analytical quantity are described under optimum analytical conditions. Then, increasing analytical bias and imprecision are assumed or simulated resulting in poorer clinical/diagnostical outcome and the minimum analytical quality requirements are defined from the worst, but still acceptable situation. A special evaluation of analytical quality specifications for S-Cholesterol has been performed by Westgard et al. [4] and Klee has now taken up the Nordic concept for S-Calcium [5]. These approaches, all give separate specifications for allowable analytical bias and imprecision.

A comparable approach for evaluation of imprecision specifications, in clinical decisions for pre-decided changes during patient monitoring has been evaluated by Fraser, Lytken Larsen, and Hyltoft Petersen [6-8] and the Barnett approach by questionnaires sent to clinicians [9] has been used by Elion-Gerritzen [10] and Skendzel et al. [11]. A special approach for quality specifications in drug monitoring has been elaborated by Fraser [12] (for imprecision only).

Biological goal-setting (basic concepts)

Two main strategies for analytical quality specifications based on biology have been evaluated for imprecision and bias (in combination with imprecision), respectively.

The Cotlove/Harris concept [13] defines the maximum allowable imprecision, CV_A, as the maximum imprecision, that when added to the within-subject biological variation, CV_I, will maximimally increase the total CV by 12%, which is achieved when:

Gowans et al. [14] based their evaluation on the concept of sharing common reference intervals for homogeneous populations. IFCC’s recommendation for establishing reference intervals has a minimum of 120 reference individuals in order to reduce the confidence intervals about the reference limits [15]. By increasing the sample size considerably (more than 800 individuals), this uncertainty will become negligible. Then, the inevitable uncertainty of the recommendations can be used as the acceptable analytical uncertainty, if a laboratory wants to use these common reference intervals, and thus the analytical quality specifications are defined. This concept assumes reliable method standardization, but when established, no further creations of reference intervals are needed in the laboratories. Thus, the reference intervals are well documented for all laboratories and no extra work on reference intervals need to be done in the single laboratory, where such work has often been difficult to perform. This model has been used for establishment of specifications for nine plasma proteins in the Nordic countries using traceability to the CRM 470 preparation [16]. The specification for analytical bias (for using the common reference intervals) is:

when analytical imprecision is negligible, and where CV_G is the between-subject biological variation [14]. Lists of biological data on CV_I and CV_G are available from references 17 and 18.

Recommendations of Two European working groups

EGE-Lab

A working group under EGE-Lab (European Group for the Evaluation of Reagents and Analytical Systems in Laboratory Medicine) has adapted the two biological concepts, mentioned above, as the basis for recommendations for analytical quality specifications for bias and imprecision [19, 20]. Illustrative examples are shown in the table below; the full list is available in the Table of European biologic goals. For quantities where the biologically derived specifications are too demanding for the current methodology, interim specifications are given in parentheses.

Examples of EGE-Lab biologic goals

EQA-Organizers

A Working group under the auspices of European External Quality Assessment Organizers (EQA-Organizers) has confirmed the EGE-Lab recommendations with a minor difference [21]. Whereas the EGE-Lab recommends separate criteria for bias and imprecision that contribute to a total error, the EQA-Organizer group pays attention to the original work of Gowans et al. [14] and recommends the same figures, but the value for bias is only valid when imprecision is negligible.

This working group further recommends a maximum allowable difference in bias between two instruments in a laboratory measuring the same quantity based on the work of Hyltoft Petersen et al. [8]:

QC Selection/Design based on European biologic goals

The EQA-Organizer working group has also described how the Validator program can be used for design of internal control systems according to the biological criteria [22]. The group pays attention to the importance of keeping the P_fr very low, when effective trouble- shooting procedures are maintained and fail-safe procedures are established. When P_fr is low, e.g. 0.1 %, then the probability of false reject is so small, that it is worth to start an intensive trouble-shooting procedure each time the run is rejected. This means there should be no immediate re-run, but a thorough investigation of the error in order to find the source. When this has been found, then fail-safe procedures are established in order to prevent future errors of the same kind - and consequently the frequency of errors are reduced. Therefore, a P_ed of less than 90 % may be acceptable as the frequency of reported errors is reduced as well, and without frustrations from re-running falsely rejected runs. This concept requires a lot of discipline in the beginning until the technicians (and others) really understand the system.

Calculation of a biologic allowable total error

The European goals for imprecion and inaccuracy can be transformed to a biologic allowable total error (TE_BA), as described by EQA-Organizers [22], according to the following formula:

|TE_BA| = |Bias_A| + 1.65 * s_A (or CV_A)

where the absolute value sign is used to be sure that bias and imprecision add together to give a maximum total error. This calculated total error can then be used as the total error parameter in the Validator program and the QC planning tools and planning process applied as demonstrated on this website.

Calculation of medically important errors

The EGE-Lab specifications are related to analytical bias and imprecision separately, so an alternate QC planning approach is to calculate the critical increases in random and systematic that are important to detect by QC, as shown below:

RE_crit = s_A/s_observed

SE_crit = (Bias_A - Bias_observed )/s_observed

where the division by sobserved is necessary to present the critical errors as multiples of the observed method imprecision; note that sobserved is called smeas and biasobserved is called biasmeas in the terminology used on this website. These calculated critical-sized errors can then be drawn on power curves to prepare critical-error graphs for estimating the probabilities for error detection and false rejection. The EZ Rules program can be used as a source of power curves for most commonly used QC procedures.

Example applications

The EQA-Organizers group included applications for alanine aminotransferase and urate in its description of how to use the EZ Rules/Validator program [22].

• For alanine aminotransferase, the biologic allowable total error was calculated to be 36% (TE_BA = 13.6% + 1.65*13.6%).
• For urate, TE_BA was calculated to be 10.9% (4.0% + 1.65*4.2%);
• Note that if the interim goal of 8.4% for urate inaccuracy were used, TE_BA would be 15.3% (8.4% + 1.65*4.2%).

Additional applications of the European biologic goals are provided on this website for cholesterol and glucose to permit comparison with earlier examples that are used analytical and clinical quality requirements. See QC Planning Applications using European Biologic Goals.

References

1. Tonks DB. A study of the accuracy and precision of clinical chemistry determinations in 1970 Canadian laboratories. Clin Chem 1963;9:217-233.
2. Hørder M, ed: Assessing quality requirements in clinical chemistry. Scand J Clin Lab Invest 1980;40 suppl. 155:1-144.
3. de Verdier C-H, Groth T, Hyltoft Petersen P, eds. Medical need for quality specifications in clinical laboratories. Upsala J Med Sci 1993;98:187-491.
4. Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory process specifications for assuring quality in the US national cholesterol education program. Clin Chem 1991;37:656-61.
5. Klee GG. Analytical performance goals based on direct effect of analytical bias on medical classification decisions. 1995 institute on critical issues in health laboratory practice: Frontiers in laboratory practice research 1995:219-26.
6. Fraser CG, Hyltoft Petersen P, Lytken Larsen M. Setting analytical goals for random analytical error in specific clinical monitoring situations. Clin Chem 1990;36:1625-8.
7. Lytken Larsen M, Fraser CG, Hyltoft Petersen P. A comparison of analytical goals for haemoglobin A1C derived using different strategies. Ann Clin Biochem 1991;28:272-8.
8. Hyltoft Petersen P, Fraser CG, Westgard JO, Lytken Larsen M. Analytical goal- setting for monitoring patients when two analytical methods are used. Clin Chem 1992;38:2256-60.
9. Barnett RN. Medical significance of laboratory results. Am J Clin Pathol 1968;50:671-6.
10. Elion-Gerritzen WE. Analytical precision in clinical chemistry and medical decisions. Am J Clin Pathol 1980;73:183-95.
11. Skendzel LP, Barnett RN, Platt R. Medical useful criteria for analyte performance of laboratory tests. Am J Clin Pathol 1985;83:200-5.
12. Fraser CG, Browning MCK. Deciding the uptimum interval between specimen collections: Theory and nomograms. Clin Chem 1987;33:1436-8.
13. Cotlove E, Harriis EK, Williams GZ. Biological and analytical components of variation in long-term studies of serum constituents in normal subjects. III. Physiological and medical implications. Clin Chem 1970;16:1028-32.
14. Gowans EMS, Hyltoft Petersen P, Blaabjerg O, Hørder M. Analytical goals for the acceptance of common reference intervals for laboratories throughout a geographical area. Scand J Clin Lab Invest 1988;48:757-64.
15. Solberg HE. Approved recommendations (1987) on the theory of reference values. Part 5. Statistical treatment of collected reference values: Determination of reference limits. J Clin Chem Clin Biochem 1987;25:645-56.
16. Hyltoft Petersen P, Blaabjerg O, Irjala K, eds. Assessing quality in measurements of plasma proteins. Upsala J Med Sci 1994;99:195-389.
17. Fraser CG. The application of theoretical goals based on biological variation data in proficiency testing. Arch Pathol Lab Med 1988;112:404-15.
18. Fraser CG. Biological variation in clinical chemistry. An update: Collected data, 1988-1991. Arch Pathol Lab Med 1992;116:916-23.
19. Fraser CG, Hyltoft Petersen P, Ricós C, Haeckel R. Proposed specifications for the imprecision and inaccuracy of analytical systems for clinical chemistry. Eur J Clin Chem Clin Biochem 1992;30:311-7.
20. Fraser CG, Hyltoft Petersen P, Ricós C, Haeckel R. Quality specifications. In Haeckel R ed. Evaluation methods in laboratory medicine. Weinheim:VCH, 1992:87-99.
21. Stöckl D, Baadenhuijsen H, Fraser CG, Libeer J-C, Hyltoft Petersen P, Ricós C. Desirable routine analytical goals for quantities assayed in serum. Discussion paper from the members of the external quality assessment (EQA) working group A on the analytical quality goals in laboratory medicine.Eur J Clin Chem Clin Biochem 1995;33:157-69.
22. Hyltoft Petersen P, Ricós C, Stöckl D, Libeer J-C, Baadenhuijsen H, Fraser CG, Thienpont L. Proposed guidelines for the internal quality control of analytical results in the medical laboratory. Discussion paper from the members of the external quality assessment (EQA) working group A on the analytical quality goals in laboratory medicine. Eur J Clin Chem Clin Biochem 1996;34:983-99.

Biography: Per Hyltoft Peterson

Per Hyltoft Petersen is a biochemist (MS) from University of Copenhagen and has been in the field of clinical chemistry since 1963, first at The Finsen Institute and from 1971 at Odense University Hospital, Denmark. He spent a sabbatical semester as a Visiting Scientist at the University of Wisconsin in 1983. He has been involved in analytical goal-setting (or analytical quality specifications) since the late seventies, when the Nordic approach was initiated in a Nordic project (NORDKEM-project) on analytical quality specifications based on clinical situations/strategies. He has elaborated on this concept for a number of clinical strategies, but also realized that only few clinical strategies are accepted internationally, and that a more general concept is needed as well, e.g., for external quality assessment schemes. So together with Elizabeth M. S. Gowans he evaluated the quality specifications for sharing common reference intervals. These specifications have been applied in a Nordic project on establishing common reference intervals for nine specific plasma proteins, traceable to the certified reference material CRM 470. As a member of European working groups on analytical quality specifications he has been involved in the recommendations from EGE-Lab and EQA- Organizers on quality specifications for external quality assessment schemes based on biology, and he is now a member of an other European working group under ECLM, where the goal is to establish quality specifications also for quantities measured on ordinal and nominal scales. He has further been involved in external as well as internal quality control for which the EQA-Organizers working group has published recommendations as well.