Quality Standards

Now that there are new measurement uncertainty goals, it's time to check if any instruments can hit them. Using the performance data from a recent study of the Roche cobas 8000 series in France, we assess the acceptability of the mu of methods.

32 of 37 assays on the Roche cobas 8000 cannot hit desirable measurement uncertainty goals

Sten Westgard, MS
February 2025

A recent study in the looked at a laboratory in Montpellier, France at the performance of the Roche cobas 8000 series. They used a heterogenous mix of performance specifications, as well as a leading approach to risk-management (using Sigma metrics to inform the risk priority number RPN):

Sentinel testing, analytical sigma metrics, and a risk management approach as part of a simplified verification/validation process. Claudio Ilardo, Chehin Lamarti, Batricia Al Muhanna, Michel Bastelica, and Nathalie Benaily. Scandinavian Journal of Clinical and Laboratory Investigation. https://doi.org/10.1080/00365513.2024.2442512

While we've done an assessment of this performance using analytical Sigma-metrics, the data can also be assessed using the new measurement uncertainty goals.

The data was collected over a period of 5 months, which is close to the recommended 6 months for appropriate estimation of lab reproducibility.

Of the analytes assessed in the study, 32 of 37 fail to meet the desirable measurement uncertainty specifications. 27 of the 37 analytes fail to meet the minimum uncertainty specifications. Only 10 analytes pass the minimum mu standards. The EFLM biological database has another set of specifications for minimum CV, of which the 26 out of 34 pass. 

Of course, this is not taking into account any of the bias measured in the study. If that were to be included in the measurement uncertainty, as an additional variance, even fewer analytes would pass.

There may be reasons why so many analytes failed to meet measurement uncertainty specifications. The controls might not be fully commutable or traceable - particularly if they were Roche manufacturer controls - but the kind of controls that are recommended for the proper assessment of mu don't actually exist. The laboratory might not be operating the instrument correctly, or the instrument might have been malfunctioning the 5 months that data was being collected. 

But it's far more likely that the new mu goals are too demanding, unrealistically so, and their widespread application will result in unproductive strife in laboratories around the world. Or, a far more likely outcome, these new mu goals will simply be ignored by most laboratories, much as most labs already ignore the bulk of the measurement uncertainty approach.