Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Lessons
  4. Advanced Quality Management / Six Sigma
  5. Surpassing Six Sigma

Advanced Quality Management / Six Sigma

Surpassing Six Sigma

What does it mean when the Sigma-metric is above Six Sigma? What are the implications of superior performance? Are there any benefits to improving beyond Six Sigma?

Seven Thoughts on Superior Performance

October 2008

Sixsigma2edsmall [Note: This QC application is an extension of the lesson From Method Validation to Six Sigma: Translating Method Performance Claims into Sigma Metrics. This article assumes that you have read that lesson first, and that you are also familiar with the concepts of QC Design, Method Validation, and Six Sigma. If you aren't, follow the link.] Bmv3edsmall

When Six Sigma first entered healthcare and the laboratory, the numbers were grim. David Nevelainen’s landmark paper in 2000 found that most common laboratory processes were not even close to Six Sigma. Indeed, many of the processes were below three sigma – a threshold usually considered the minimum performance by other manufacturing and service industries.

When we first began analyzing analytical testing performance on the Sigma scale, the results there, too, were not entirely encouraging[1]. While chemistry tests, in general, perform very well, there are many analytes where performance is not nearly as good as the quality requirement demands. Worse still, many of the new Point-of-Care devices we examined were not only not as good as the traditional laboratory instruments, the POC devices were simply no good at all [see here and here and here and here].

More recently, we have begun to see better performance. Not only better, but superior performance. On the chemistry side, we are now seeing instruments that can perform not only at Six Sigma performance, but at levels far beyond Six Sigma. Sometimes, we see performance at 10, 15, even 20 Sigma.

This begs the question: if Six Sigma is world class, what is 20 Sigma? If a method is 18 Sigma, is that three times better than 6 Sigma? If a method is that good, are there any practical benefits to achieving higher than 6 Sigma, or is this just “surplus” excellence?

1. Are you Sure this isn’t a fluke?

One of the first questions you should ask when you come across a Sigma-metric beyond 6.0 is, is this an outlier of performance, or is this test always performing this well? Before you contemplate the benefits of world class performance, you need to make sure that the method is truly performing at Six Sigma consistently and at more than one decision level.

Usually, our QC applications analyze one or two decision levels of a test. If you are considering major changes to your laboratory operating procedures based on Sigma-metric results, you might want to look at performance for multiple levels across the range. If a test has several cutoff points or multiple levels where different interpretations might occur, assess those points on with Sigma scale. A test that has beyond Six Sigma performance at the high end of the range, but below Three Sigma at the low end of the range, is not a method where you can make changes. If anything, you’ll have to ignore the level where Six Sigma performance.

This is one of those areas where professional judgment becomes important. Only trained laboratory professionals can understand the different levels of a test and balance which level is most important. For all the power of Six Sigma, it only provides you with information – it does not dictate your actions. You must decide what to do with that information.

2. Are there more Savings we can realize?

Simply put, higher Sigma performance should translate into increased savings. Six Sigma reduces defects to just 3.4 per million opportunities. Higher Sigma performance will mean even fewer defects. Fewer defects, as we know, leads to less rework, fewer work-arounds, less trouble-shooting, and a reduction in the wasted time and resources of the organization.

Certainly, the additional savings experienced by moving from Six to (for example) Seven Sigma will be less than the savings experienced when moving from Five to Six Sigma. At the margin, in economic terms, there are diminishing returns. The difference between 19 and 20 Sigma is quite small.

If the method is already performing at Six Sigma, the financial return of even higher Sigma performance may not merit additional effort by the laboratory. If methods are performing at low Sigma, Six Sigma is still the goal. Moving a Three Sigma process to Six Sigma is a significant task; raising the bar to Seven Sigma is not necessary.

3. Can we better Tolerate Shifts in performance?

One of the important footnotes on Sigma metrics is the difference between short-term and long-term Sigma. Usually, only the short-term Sigma of processes are determined – because the short-term number is the highest. Over the long term, Six Sigma theory posits that small shifts will inevitably occur in a process, usually 1.5s. Achieving Six Sigma will allow these small shifts to occur without a reduction in quality or output. That is, at Six Sigma, you will be able to compensate and tolerate the “natural” variation of the process.

Higher Sigma performance will likewise be able to tolerate larger shifts. A 7 Sigma process can tolerate more variation than a Six Sigma process can. For laboratory testing processes, this extra capacity for variation can be particularly useful. One of the common problems with many tests is the “control lot shift” – the change in the mean caused by a switch to a new lot of controls. Often, with poorly performing tests, a switch in the control lot may shift values to the point where they are now considered “out-of-control.” What follows at that point is a waste of time and resources as the laboratory struggles to finagle the numbers back “in.”

With superior Sigma performance, new control lots need not be feared. The process will be able to tolerate small shifts in the control mean and standard deviation without incident. Note, however, this assumes and requires that the laboratory has properly designed the QC procedure based on the Sigma-metric. If the laboratory has superior performance but is using the old “2s” rules, there is no tolerance for shifts. That’s a case where you’ve got the right test but the wrong QC.

More generally, a superior Sigma process will be able to tolerate more variation of all kinds, not just control lot switches. The method can tolerate greater bias. The method can tolerate greater imprecision. Just how much depends on the size of the Sigma-metric.

4. Is it Time to Tighten our Standards?

Here’s a question we don’t often get to ask: What if the test isn’t performing that well, but actually the quality requirement is too wide? Achieving Six Sigma isn’t hard if the target is large. Perhaps the prudent thing to do with superior Sigma performance is to reduce the size of the target.

Here’s a quick example:

  • Quality Requirement of 30%
  • CV 3%
  • Bias 4%
  • Sigma is 8.6
  • If the quality requirement was 28%, Sigma would be 8.
  • If the quality requirement was 25%, Sigma would be 7.
  • If the quality requirement was 22%, Sigma would be 6.

Is there a benefit to achieving a smaller Sigma with a smaller quality requirement? Yes. Remember that these quality requirements are supposed to represent the medically important changes in test interpretation. So if we have a smaller quality requirement, we have a smaller change we’re able to detect. Between the choice of achieving Six Sigma with a smaller quality requirement and achieving a larger Sigma-metric with larger quality requirement, choose the former. "Mere" world class performance, even Five Sigma performance, with a smaller quality requirement, means that you’re delivering more precise information about the patient.

Another reason to scrutinize the quality requirement is the fact that some of the existing quality requirements don’t adequately reflect the true medical interpretation. Remember, CLIA’s quality requirements are decades old – CLIA just celebrated 20 years of existence - plus the CLIA quality requirements were originally intended as proficiency testing guidelines, so they are probably already larger than the requirements required by individual laboratories. It would be surprising if the interpretation of tests hasn’t changed in all those years.

5. Can we have more confidence in our test results?

Achieving performance beyond Six Sigma means that you and the clinicians can have more confidence in the results you report. With less variation and fewer defects, that means the results generated by the testing process are more signal, less noise. If we look at Dr. Callum Fraser’s work on the Dispersion of test results and the Critical Number of Samples required, you can easily see how better performance on the Six Sigma scale translates into less dispersion in the test results and fewer samples needed.

Increasing confidence in the results should lead to increasing confidence in treatment. When a doctor can be sure that a change in the test results is a real change in the condition of the patient, tests don’t need to be run again, patient stays don’t need to be extended for more testing cycles, and treatments can begin sooner.

6. Are we generating “Surplus Precision”?

Here’s a possibility that is rarely mentioned, but is worth considering. In the Lean Methodology, there is a famous list of types of waste (8 Wastes of a process). One of the last forms of waste is defined as Overproduction. One definition of this is “Doing more, earlier or faster than required by the next process.”[2]

Is it possible that testing processes are more precise than medically needed? If quality requirements are properly set and performance is still far beyond Six Sigma, it may be worth considering whether or not the doctor needs this level of precision. If the clinicians truly do not need this level of precision, it is possible that the engineering of this process can be relaxed – or less expensive components can be used, ones that have reduced costs and inferior performance.

To put it more colorfully, if you’re only trying to hit the side of a barn, you probably don’t need a sniper rifle.

Let’s bring this idea closer to a healthcare example. Imagine a test where the interpretation guidelines really are quite far apart, while the process performance is extremely tight (beyond Six Sigma). The doctors really only make decisions at a cutoff that’s very far away from the normal range. In that case, the method might not need to perform as well.

Please keep in mind, this is really only a hypothetical consideration. There are few cases we can imagine where better precision is not needed. But if a method produces 20+ Sigma, it might be worth considering if the process could “degrade” to only a 10-Sigma method.

The reality of test interpretation is fluid, however, and for manufacturers, the risk of degrading method performance is considerable. If you engineer the method to perform less precisely, you might find that clinicians, in the meantime, have started using cutoffs that are closer together. Suddenly, what used to be “excess precision” by the old treatment guidelines is now necessary precision for the new treatment guidelines. Since clinicians are rarely uniform in their treatment decisions and test interpretation behavior, it may be impossible to determine if improved precision is excessive.

7. Will we need Less Service and Less Technical Support?

If your method is performing beyond Six Sigma, you should experience far fewer defects. Defects like false rejection often lead to a troubling cycle: trouble-shooting that chases the ghosts of false rejects, more calls to technical support, more complaints to the manufacturer, increased field service visits, higher dissatisfaction with the instrument, and worse. Neither the manufacturer nor the customer are happy.

These problems, particularly false rejection, cause the laboratory to over-correct. Increased tinkering, recalibration, and maintenance puts more wear and stress on the instrument, not to mention the staff. When this is unnecessary waste, it merely strains the laboratory and the instrument – and it strains the manufacturer that tries to respond to the situation. High Sigma performance leads to a virtuous cycle of better performance and less tinkering. Low Sigma performance can lead to a death spiral of repeating controls, recalibrating materials, replacing parts, etc.

A process or method that surpasses Six Sigma should require less service and fewer tech calls and visits than a process with lesser performance. But if you're doing the wrong QC, or have poorly skilled staff, those instrument gains will be wiped out by your losses in the laboratory.

Conclusion

As we have said in many applications and examples where processes perform better than Six Sigma, that’s the time when we should celebrate. And any process that operates as well as airline safety is truly worth celebrating. Six Sigma performance in healthcare is a remarkable achievement.

But beyond the revelry, there are other thoughts to consider. You can double-check the results and check other levels in the range to assure that the method is performing well consistently and over the entire range. You can examine the performance to determine whether or not you can widen your limits and tolerate shifts in performance. You can make it harder on yourself by shrinking the quality requirement and increasing the demands on the process (thus lowering the Sigma). You can take – and communicate – better confidence in the results so that clinicians in turn have more confidence in their interpretations and treatment decisions. You can even consider relaxing performance – possibly by using lesser components or less skilled staff – until you’ve “only” reached Six Sigma. Finally, you can sit back and enjoy the reduced need for trouble-shooting, recalibration, maintenance and other service activities typically generated by a poorly-performing process.

Achieving performance beyond Six Sigma doesn’t mean taking excellence for granted – you still need to continuously monitor and measure your performance. But having superlative performance means that there are more possibilities and more freedom of action for laboratories.

References

1. Westgard JO, Six Sigma QC Design and Control, 2nd edition, (Westgard QC: Madison WI) 2006.

2. Creating Lean Healthcare, George Alukal and Robert Chalice, Appendix F in Improving Healthcare using Toyota Lean Production Methods, 2nd Edition, 2007 (Milwaukee, WI: ASQ Press) p.215