Advanced Quality Management / Six Sigma

In other lessons, we've discussed how to turn method validation data (method performance claims) into Sigma-metrics. But what about the case when only limited claims are available? Sometimes a complete method validation study is not available - and only the manufacturer claims for precision are readily at hand. How can you turn those claims into something useful on the Sigma scale? (Preview)

• What are Precision Claims?
• Are Precision Claims useful?
• What is Six Sigma?
• How do you go from Precision Claims to Sigma metrics?
• What's a quality requirement and where do I find it?
• Calculating a Six Sigma Estimate
• How useful is the Sigma metric?
• What do you need to do after getting the Sigma metric?
• Conclusion
• Postscript: How would you QC hemoglobin?

Note: This essay is a synthesis of ideas and applications of Six Sigma, and QC Design. It assumes you are familiar with many of these concepts, but if you aren’t, there are links to essays and lessons with in-depth coverage. The main goal of this essay is to show how to put Method Validation together with Six Sigma Metrics in order to make a definitive decision on the suitability of a method for your own application.

There are dozens if not hundreds of factors to consider when purchasing of a new diagnostic instrument, and new issues are emerging all the time. As instruments get bigger and more expensive, they do more. Yet for all the marketing claims of faster, more advanced, more automated, more ”verified”, and integratedinternetworked, the critical characteristic of the device remains its analytical testing performance. (Remember – the device is supposed to perform a test accurately and precisely, not just produce test results faster and cheaper.) That performance can be judged through the manufacturer's precision claims.

However, what is the practical meaning of a precision claim? Is 2% good? How about 3%? Certainly if for test Z, instrument A has a precision claim of 2%, while instrument B has a claim of 3%, you have a better opinion of A. But is that a valid judgement? What is the real precision need for patient care?

The problem with precision claims is that while they can provide some relative comparison between competing products, they are disconnected from the real-world performance needs. By translating them into Six Sigma metric estimates, we can connect claims with the analytical needs of test performance.

What are precision claims?

When diagnostic manufacturers market a product, they are required to make a precision claim for each test. That, they must claim that the coefficient of variation (CV) of the device will be “X% or better.” It's one of the few quantitative claims for analytical performance they are required to make.

Are precision claims useful?

There are two ways to approach a precision claim: as an optomist or a pessimist. For the optimist, a precision claim is actually higher than the CV that the customer will routinely experience. Why? Because the claim has to be high to encompass all the different performances of the instrument across laboratories and locations. So precision claims are useful, but not worth worrying about.

For the pessimist, the precision claim must be taken at face value. If a manufacturer could claim a lower CV, they would, so there's a definite reason that the number is where and what it is. That is, if the manufacturer could claim a 1% CV, they would, so if they claim 2%, there's a reason for it. This is beyond worst-case scenario thinking – the claim indicates that the performance isn't significantly better in the majority of cases. Maybe some labs will be able to get the instrument to perform better, but many won't. So if you are getting a new instrument with a precision claim of 4% CV, you better plan that you will experience 4% CV when the device is installed in your lab.

Ultimately, a precision claim is only useful until you have better information. As soon as you can obtain performance data for the device, either through method validation studies or routine laboratory data, those numbers are far more useful than the precision claims. You want to know how the device will perform in your lab, not anyone else's.

Want to read more of this article?

We invite you to purchase the Six Sigma QC Design and Control manual, Second Edition, available at our online store. You can also download the Table of Contents and a Sample chapter.