Interviews
Intro to EP 15: with R. Neill Carey, Ph.D.
Sten Westgard, MS
R. Neill Carey, Ph.D. has served on several Clinical Laboratory Standards Institute committees in the Area of Method Evaluations, most significantly, as the chair of the EP 15 standard. He received his BS degree from the University of Richmond and his Ph.D. from Duke (in absentia). He got his start in Clinical Chemistry when his Ph.D. thesis director at Duke moved to the University of Wisconsin in Madison. He served as Clinical Chemist and Laboraqtory Manager at Peninsula Regional Medical Center in Salisbury, MD, for many years. He is a co-presenter of the longest-running workshop at the AACC Annual Meeting, “Concepts and Practices in the Evaluation of Laboratory Methods.” We were fortunate enough to persuade Dr. Carey to introduce and explain EP 15 at the October 10th Method Validation workshop in Chicago, as well as provide a chapter on EP 15 in the third edition of Basic Method Validation.
Q: What is CLSI EP15: User Verification of Performance for Precision and Trueness?
Dr. Carey: It a guideline that allows users to demonstrate that they can achieve the same performance as the manufacturer claims, thus verifying that method is operating in accordance with manufacturer's claims.
The document suggests that users should first define or derive the allowable standard deviation and allowable bias for the test, then find a method whose manufacturer’s claims meet the allowable standard deviation and bias, then use the EP15 protocol to verify that the method’s performance in the user’s own laboratory meets the manufacturer’s claims.
EP15 describes three specific protocols, a protocol for verification of precision, and two protocols for verification of trueness (bias): one using patient samples, and a second using reference materials with assigned values.
Q: Is it intended that manufacturers use EP5 and EP9, while users use EP15?
Dr. Carey: Yes, EP15 is paired to EP5 and EP9, and builds on the work done by the manufacturer using those guidelines. The estimates of method error that come out of EP5 and EP9 from the manufacturer are very robust claims that can be verified by the user with EP15.
Q: Do manufacturers need to do more work if their customers want to use EP15?
Dr. Carey: No, manufacturers already do EP5 and EP9, so EP15 doesn't require more work from them.
Q: Do Joint Commission, CAP, and COLA accept EP15?
Dr. Carey: I am not aware of any problems encountered by EP15 studies during inspections. In inspections of our laboratory, the inspectors have accepted method evaluations using EP15. You also have to remember that CLSI is a consensus organization, so the government and accrediting organizations are represented and involved during the entire standards-creation process. If there had been an issue, it would have happened earlier, before EP15 became a standard.
Q: What else is there about EP15 that we should know?
Dr. Carey: The guideline for verification of trueness using reference materials protocol has been improved. It allows laboratories to use materials with known values, such as proficiency testing samples, or materials provided by method manufacturers, to verify trueness of the method in the laboratory. EP15 provides a statistically valid verification of performance using these materials.
For example, if you have a failure in PT, you can take the PT sample and use EP15 to demonstrate that your mean is within the expected limits of the group mean.
Q: So EP15 can be used during trouble-shooting, not just during method verification?
Dr. Carey: Right. It's a way of adding some reliability to the follow-up data after a proficiency testing problem. Repeating each survey sample once and obtaining a result within acceptable limits doesn't add much confidence to the method; you could have just gotten lucky. With EP15, testing a few replicates on a few samples allows you to confirm statistically that the method is performing the way it should be.
Our thanks to Dr. Carey to taking the time to answer our questions.
More links to CLSI EP 15
CLSI’s EP 15 standard - purchase the complete standard at CLSI
Basic Method Validation manual, 3rd Edition, which includes Dr. Carey’s chapter and discussion of EP 15
EP15 tools – part of the Method Validation toolkit