CLIA Final Rule
An Outside Review of an IQCP for POC
In 2016 we posted a sample IQCP for a POC device. We asked IQCP experts to review and grade this IQCP. Turns out there is a lot of individuality in these IQCPs, and perhaps this variation is significant.
AN IQCP REVIEW
July 2016
Guest Article from CarePoint Solutions
[Editor's note: the IQCP for a POC device was posted on Westgard Web in an earlier issue. We then asked CarePoint Solutions to review this IQCP.]
The team at CarePoint Solutions found the Sample IQCP, which appeared to be based largely on the manufacturer’s guidance documents, was not sufficiently comprehensive. We compared the output of CarePoint Solutions Pro-QCP, a quality management software application, to the Sample IQCP. Pro-QCP identified 97 likely sources of error for this test system, as compared to the sample IQCP that identified 30. Of course, each IQCP should be custom to the individual test process, and some sources of error may not be present in all situations.
We believe the Sample IQCP does not adequately address sources of error related to the environment or testing personnel, nor does it address verification of performance on multiple identical devices. Also, the Sample IQCP is lacking in historical data relative to the laboratory environment, instrument performance and testing personnel. It was also noted that some of the manufacturer’s minimum requirements (e.g., performing a thermal probe check every 6 months) was missing from the Quality Control Plan (QCP). The Sample IQCP also includes onboard device process controls that eliminate certain risks (e.g., automatic clot detection). In these cases, we believe it is not appropriate to include sources of error in the laboratory’s IQCP that have been effectively eliminated by the device manufacturer.
Comparison Table:
CAP Checklist Requirement | Does Sample IQCP meet requirement? | Comment | Best Practice IQCP | |
COM.50200 | TBD |
|
||
Laboratory has identified all tests using an IQCP and completed CAP forms | TBD | Best Practice is to format IQCPs to meet CAP IQCP form requirements. | ||
Risk Assessment | ||||
COM.50300 |
★★ Partially meets |
The IQCP for a test/device/instrument includes a risk assessment to evaluate potential sources of error to include all of the following: ● Preanalytic, analytic, and postanalytic phases of the testing process ● Intended medical uses of the test and impact if inaccurate results are reported (clinical risk) ● Components of the tests including reagents, environment, specimen, testing personnel, and test system ● Variations in the components based on use of the tests (e.g. use in different environments, by different personnel, or multiple identical devices) ● Data from the laboratory's own environment, instrument/equipment performance, and testing personnel ● Manufacturer's instructions and recommendations |
★★★ Meets |
|
Includes pre-analytic phase |
★★ Partially meets |
Missing the following sources of error – Is there an order to ensure that that correct patient enters the testing process? What about the risk of sample contamination during collection process? |
★★★ Meets |
|
Includes analytic phase |
★★ Partially meets |
The focus on the analytic phase is consistent with manufacturer’s guidance IQCP documents. However, it is difficult to discern the specifics around manufacturer’s requirements from the IQCP document. Missing sources of error in analytic phase - What if the POC device is dropped? What about risks associated with cartridge load – overfill or underfill? |
★★★ Meets Best Practice would be t provide citations to each manufacturer recommendation & requirement |
|
Includes post-analytic phase |
★★ Partially meets |
The details of interferences from the manufacturer’s instructions for use are lacking. How does the laboratory ensure that these are recognized/handled by testing personnel? Are testing personnel trained to recognize and respond to inconsistent test results? |
★★★ Meets |
|
Includes 5 components of testing: Reagent |
★★ Partially meets |
Missing sources of error associated with external quality controls and calibration verification product storage and handling. | We can think of at least 20 possible sources of error compared to 5 in Sample IQCP. | |
Includes 5 components of testing: Environment |
★★ Partially meets |
Missing sources of error that may be associated with altitude (barometric pressure) as well as risks in the work environment such as distractions, noise and interruptions. | We can think of at least 7 possible sources of error compared to 5 in Sample IQCP. | |
Includes 5 components of testing: Specimen |
★★ Partially meets |
Missing error sources associated with collection device instructions for use. Missing error sources associated with sample contamination during collection either from alcohol at collection site or IV fluid contamination. | We can think of at least 17 sources of error compared to 9 in Sample IQCP. | |
Includes 5 components of testing: Testing personnel |
★★ Partially meets |
We can think of at least 8 sources of error compared to 5 in Sample IQCP. | ||
Includes 5 components of testing: Test system |
★★ Partially meets |
Missing error sources associated with intended use, test method limitations and correlation of results. | We can think of at least 45 sources of error compared to 14 in sample IQCP. | |
Variations in the components based on use of the test in different environments |
★ Does not meet |
Not accounted for in IQCP document. |
Best Practice would be to include this in a IQCP |
|
Variations in the components based on use of the test by different personnel |
★ Does not meet |
Not accounted for in IQCP document. | Best Practice would be to include this in a IQCP | |
Variations in the components based on use of the tests with multiple identical devices. |
★ Does not meet |
Not accounted for in IQCP document. | Best Practice would be to include this in a IQCP | |
Data included from the laboratory’s own environment |
★ Does not meet |
No evidence that data from the laboratory’s specific environment was included in the IQCP document. |
Best Practice would be to include this in a IQCP |
|
Data included from the laboratory’s own instrument/equipment performance |
★ Does not meet |
The only hint of the laboratory’s past performance with QC is referenced in the risk assessment under “instrument prevents results from being reported when external liquid QC is unacceptable”. The source of error “external liquid QC is unacceptable” is scored as a frequent occurrence with negligible severity of harm. Based on IQCP document, this scoring appeared to be based on external quality control for new lots of reagents. | ||
Data included from the laboratory’s own testing personnel |
★ Does not meet |
Not accounted for in the IQCP document | Best Practice would be to include this in a IQCP | |
Manufacturer’s instructions and recommendations |
★★ Partially meets |
Difficult to discern as sources of error are defined in more general terms. For example “specimen stability” does not include specifics around manufacturer’s exact instructions for sample stability. | Best Practice would be to have itemized manufacturer recommendation and provide citations to the manufacturer’s source documentation. | |
Quality Control Plan | ||||
COM.50400 | The IQCP includes a written quality control plan approved by the laboratory director prior to implementation. | |||
IQCP includes written quality control plan |
Meets |
Written quality control plan included |
Meets |
|
COM.50500 |
The individualized quality control plan must define all aspects monitored based on the potential errors identified during the risk assessment, including the following parameters as applicable: ● The number, type (external and internal quality control systems), and frequency of quality control ● Criteria for acceptable performance ● Monitoring of the testing environment and reagents ● Specimen quality ● Instrument calibration, maintenance, and function checks ● Training and competency of testing personnel ● Provisions for multiple identical devices and variation for uses covered under one IQCP |
|||
Number, type and frequency of quality control |
★★★ Meets |
★★★ |
||
Criteria for acceptable performance |
★★★ Meets |
From a regulatory perspective, while a proficiency testing score of 80% or higher is graded a satisfactory performance by CMS, corrective actions are still required by CMS for any PT failure or scoring less than 100%. |
★★★ Meets |
|
Monitoring of testing environment and reagents |
★★★ Meets |
★★★ Meets |
||
Specimen quality |
★ Does not meet |
No mention of monitoring criteria for sample quality. |
★★★ Meets |
|
Instrument calibration, maintenance and function checks |
★★ Partially meets |
No mention of thermal probe check in the Quality Control Plan. The manufacturer’s system manual specifies “Check Thermal Control System Every Six Months” |
★★★ Meets |
|
Training and competency of testing personnel |
★★★ Meets |
★★★ Meets |
||
Provisions for multiple identical devices and variance for uses covered under one IQCP |
★ Does not meet |
No mention of provisions for multiple identical devices for the POC Chemistry system. |
★★★ Meets |
|
Quality Assessment | ||||
COM.50600 |
Ongoing quality assessment monitoring is performed by the laboratory to ensure that the quality control plan is effective in mitigating the identified risks for the IQCP and includes the following: ● Review of quality control and instrument/equipment maintenance and function check data at least monthly ● Evaluation of errors relating to preanalytic, analytic and post analytic phases of the testing process ● Review of complaints from clinicians and other healthcare providers regarding the quality of testing to confirm the clinical efficacy of testing, and ● Evaluation of corrective actions taken if problems are identified ● Reapproval of the quality control plan by the laboratory director or designee at least annually
|
|||
Review of quality control and instrument/equipment maintenance and function check data at least monthly |
★★★ Meets |
★★★ Meets |
||
Evaluation of errors relating to preanalytic, analytic and postanalytic phases of the testing process. |
★ Does not meet |
Sample IQCP references a Quality Assurance Year to Date Report, but does not provide enough detail to ensure the evaluation of the three phases of the testing process as part of the QA process. |
★★★ Meets |
|
Review of complaints from clinicians and other healthcare providers |
★★★ Meets |
★★★ Meets |
||
Evaluation of corrective actions if problems identified. |
★★★ Meets |
Quality Assurance Year to Date Report specifies “POCT resource must submit action plan to reduce error rate to allowable target” |
★★★ Meets |
|
Re-approval of the quality control plan by the laboratory director or designee at least annually |
★ Does not meet |
The Sample IQCP does not document re-approval of the QCP as part of the Quality Assessment Monitoring |
★★★ Meets |
|
Reference: CAP 2015 All Common Checklist Requirements for IQCP (NEW)
Scoring:
★★★ Meets the requirements
★★ Partially meets the requirement
★ Does not meet the requirement
TBD: To be determined
About the Reviewers
At CarePoint Solutions, we probably view IQCP’s quite differently than most. We view it as a process, not merely a document.
We believe that IQCP’s are the beginning of a transformative wave in healthcare delivery, as we make the transition to quality management systems that will drive improved efficiency, economy, and quality. Just about every other major industry that has adopted these data driven, outcomes oriented processes, has dramatically transformed the quality of their output. Quality feedback loops drive continuous improvement, and these quality management systems foster a culture of quality that includes (and empowers) each participant in the process.
This is really good news, given the recent studies from CDC, the Institute of Medicine, and John Hopkins that implicate preventable medical error as the #3 killer in the US, just behind cancer and heart disease. This translates to about a quarter of a million needless deaths occurring in our hospitals each year. That’s roughly 40 deaths per US hospital, suggesting a systemic problem. And guess what the number one contributor to these errors is? Diagnostic error.
Worse still, these studies conclude that, as the complexity of healthcare delivery increases, the incidence of diagnostic error is increasing as well.
So the CarePoint team is excited to announce the launch of Pro-QCP, a web app that represents a first-of-its-kind quality management system for point of care testing. This is an expanded software app that not only delivers a superior IQCP, but underpins the processes that drive continuous improvement. Incorporating best practices that have been proven in other industries, Pro-QCP has the potential to drive efficiency, significantly lower healthcare costs, and measurably improve patient outcomes.
Pro-QCP takes you step-by-step through a process of identifying sources of error in your test process, collecting your historical data along the way (e.g., the incidence of sample hemolysis or patient misidentification). The software also suggests key performance indicators (KPIs) that will help trend the effectiveness of your quality efforts over time.
CarePoint is already working on the next generation of Pro-QCP that will provide KPI dashboards and real time inter-laboratory comparisons that allow you to compare your error detection, your error mitigation efforts, and your own performance indicators to those of your peers – in real time. In a collaborative way, user data is leveraged to enhance the program content over time. For example, users will be able to compare what process controls are being employed by their peers, and they can see how effective those process controls have been at improving quality. And the value of the data repository increases as the user base grows.
While the incidence of preventable deaths due to diagnostic error is quite concerning, we believe adopting quality management practices is a appropriate response and offers significant promise to improve the outcomes of patients in our care.