Basic Method Validation

You've crunched numbers, plotted and graphed -- now what do you do? Keeping in mind the inner, hidden, deeper, secret meaning of Method Validation, Dr. Westgard explains how to judge your method performance. (Preview)

MV - The Decision on Method Performance

• What's the right approach?
• How should a requirement for medically allowable errors be stated?
• Where can you find recommendations for allowable total errors?
• How are the observed errors compared to an allowable total error?
• How do you construct a method decision chart?
• How do you use the method decision chart?
• Example applications
• Try it!
• Try it with our web tools!
  • Normalized Operating Point Calculator
  • Normalized Method Decision Chart
• References

You've performed the experiments, tabulated the results, plotted the data, and calculated the statistics. Now you have to make a decision on the acceptability of the method. How do you decide whether the method is good enough to use in your laboratory?

What's the right approach?

Remember the inner hidden deeper secret meaning of method validation - ERROR ASSESSMENT. The decision on the acceptability of method performance depends on the size of the observed errors relative to some "standard" or quality requirement that defines the medically allowable error. Method performance is acceptable when the observed errors are smaller than the medically allowable error. Method performance is NOT acceptable when the observed errors are larger than the medically allowable error.

You should actually define the medically allowable errors at important medical decision levels in the beginning to help guide the design of the experiments and the collection of the data. What will remain to be done, then, is to compare your observed errors with the defined medically allowable errors.

How should a requirement for medically allowable errors be stated?

In the scientific literature, requirements for analytical quality have been defined in three different formats - allowable total error, allowable SD, and allowable bias. An allowable total error sets a limit on the combined effect of the random and systematic errors of a method, whereas an allowable SD and an allowable bias set separate limits for random and systematic errors, respectively.

Separate requirements for allowable SD and allowable bias would appear to be useful because these statistics can be calculated directly from the experimental data (e.g., an SD is calculated for the data from a replication experiment and a bias for the data from a comparison of methods experiment). However, the quality of a patient test result is determined by the net or total effect of both the random and systematic errors, therefore the total error is more relevant medically [1].

"The physician thinks rather in terms of the total analytical error, which includes both random and systematic components. From his point of view, all types of analytic error are acceptable as long as the total analytic error is less than a specified amount. Total error is medically more useful; after all, it makes little difference to the patient whether a laboratory value is in error because of random or systematic analytical error, and ultimately he is the one who must live with the error."

We invite you to read the rest of this article.

This article, plus many more revised, updated, and expanded chapters are available in the Basic Method Validation manual, 3rd Edition which is available at our online store. You can download the Table of Contents and other chapters here. You can also enroll in the Basic Method Validation course and access the new materials online.