Quality Goal Index

David M. Parry, St. Boniface General Hospital, Winnipeg, MB, Canada.

Dr. David Parry of St. Boniface General Hospital in Winnipeg, provides us with data on Sigma metrics in two laboratories. Dr. Parry's innovation is a Quality Goal Index, a metric that can distinguish between precision and accuracy problems, as well as techniques to deal with calibrator lot changes.

Its Use in Benchmarking and Improving Sigma Quality Performance of Automated Analytic Tests

Sigma Quality Performance
Quality Goal Index
Complete listing of applications, Sigma metrics, and QGI: Laboratory A
Complete listing of applications, Sigma metrics and QGI: Laboratory B
Summary of Laboratory Performance Problems
Sigma Quality Improvement
References

The long-term goal of six sigma quality management is to achieve an error rate of 3.4 or less per million opportunities for all laboratory processes. In percent terms, that’s an error rate of less than 0.001%. This very demanding goal is achievable or within reach for many automated analytic processes, but implicitly requires knowledge of which processes fall short of this goal and why. Hence, a systematic approach to benchmarking sigma quality performance of analytic processes is fundamental to practicing six sigma quality management. In order to achieve six sigma quality, it is necessary to assess and document which analytical processes do and do not achieve six sigma quality performance and to understand why.

Sigma Quality Performance

As discussed elsewhere (1), quality performance of any analytical process in a laboratory can be described in terms of its sigma metric. Calculation of sigma performance is quite simple:

Sigma = (TE_a – Bias) / CV

The harder part is making sure that data used in this calculation really reflects current analytical performance. In my region, we turned to our on-analyzer quality control databases for estimates of imprecision (CV). This data reflects actual analytical imprecision, if and only if, data integrity has been maintained. Data integrity can be assured only when procedures are in place and rigorously practiced to exclude erroneous quality control results due to procedural blunders and statistical outliers. This does not mean excluding out-of-control results; no statistically-valid data should be edited out.

For estimates of test bias, our region makes use of Randox Interlaboratory Quality Assessment Scheme (RIAQS). This external quality assessment program, run out of Ireland, provides biweekly (every 2 weeks) challenges for a broad range of automated chemistry tests. There are two main attributes that make this program of particular value for assessing test bias. First, challenges are blind and second, they are at varying analyte concentrations. These two attributes, combined with its biweekly frequency and relatively large user base, make this database reflective, as close as possible, of “real-world” testing bias. Bias is calculated for each test application based on the average (running mean) bias of the last 10 values obtained over the preceding five month period, updated every two weeks.

Sigma quality management forces one to think in terms of quality needs, not just performance capabilities and this is good since it focuses attention on what really matters. The next step is setting quality goals. This in itself can be a real eye-opener and a little daunting. There are multiple sources for quality goals but the proviso, of course, is objectivity in selecting these goals in deference to analytic capabilities. We selected quality goals for total error (TE_a) based on either biological variation or proficiency testing target data accessible through this website (2, 3), fully recognizing that at least some of these goals may need customizing to specific clinical requirements as our experience with this approach develops. Selecting test specific goals, as uncomfortable or imperfect a process as this may be, it reinforces the fact that good clinical practice is implicitly linked to quality of laboratory testing.

Once current and reliable data for imprecision and bias has been obtained and quality goals set, sigma quality performance is then calculated for each test application. These sigma metrics are date documented and used to benchmark existing sigma performance for inter-instrument and inter-laboratory comparisons within our region and for future historical reviews to assess performance changes.

To illustrate this approach, the sigma metrics for 60 automated test applications over three analytic platforms at one laboratory (site A) in our region indicates that twenty eight (46%) of these applications fall short of meeting six sigma quality performance. Of these, twelve fail to meet minimum sigma quality performance with metrics less than three and another four just meet minimal acceptable performance with sigma metrics between three and four. At another laboratory (site B) in our regional group (seven sites in total), the original data collected indicates more than half (57%) of the automated test applications do not achieve six sigma quality performance.