Tools, Technologies and Training for Healthcare Laboratories

An Outside Review of an IQCP for POC

In 2016 we posted a sample IQCP for a POC device. We asked IQCP experts to review and grade this IQCP. Turns out there is a lot of individuality in these IQCPs, and perhaps this variation is significant.

AN IQCP REVIEW

July 2016
Guest Article from CarePoint Solutions

[Editor's note: the IQCP for a POC device was posted on Westgard Web in an earlier issue. We then asked CarePoint Solutions to review this IQCP.]

The team at CarePoint Solutions found the Sample IQCP, which appeared to be based largely on the manufacturer’s guidance documents, was not sufficiently comprehensive. We compared the output of CarePoint Solutions Pro-QCP, a quality management software application, to the Sample IQCP.   Pro-QCP identified 97 likely sources of error for this test system, as compared to the sample IQCP that identified 30. Of course, each IQCP should be custom to the individual test process, and some sources of error may not be present in all situations.

We believe the Sample IQCP does not adequately address sources of error related to the environment or testing personnel, nor does it address verification of performance on multiple identical devices. Also, the Sample IQCP is lacking in historical data relative to the laboratory environment, instrument performance and testing personnel. It was also noted that some of the manufacturer’s minimum requirements (e.g., performing a thermal probe check every 6 months) was missing from the Quality Control Plan (QCP). The Sample IQCP also includes onboard device process controls that eliminate certain risks (e.g., automatic clot detection). In these cases, we believe it is not appropriate to include sources of error in the laboratory’s IQCP that have been effectively eliminated by the device manufacturer.

Comparison Table:

CAP Checklist Requirement Does Sample IQCP meet requirement? Comment Best Practice IQCP
       
COM.50200 TBD
The laboratory has identified all tests using an IQCP and completed the CAP's forms for laboratories using an individualized quality control plan.
 
Laboratory has identified all tests using an IQCP and completed CAP forms TBD   Best Practice is to  format IQCPs to meet CAP IQCP form requirements.
Risk Assessment      
COM.50300

★★

Partially meets

The IQCP for a test/device/instrument includes a risk assessment to evaluate potential sources of error to include all of the following:

● Preanalytic, analytic, and postanalytic phases of the testing process

● Intended medical uses of the test and impact if inaccurate results are reported (clinical risk)

Components of the tests including reagents, environment, specimen, testing personnel, and test system

Variations in the components based on use of the tests (e.g. use in different environments, by different personnel, or multiple identical devices)

Data from the laboratory's own environment, instrument/equipment performance, and testing personnel

Manufacturer's instructions and recommendations

   ★★★        

Meets

Includes pre-analytic phase

★★        

Partially meets

Missing the following sources of error – Is there an order to ensure that that correct patient enters the testing process? What about the risk of sample contamination during collection process?

★★★              

Meets

Includes analytic phase

★★    

Partially meets

The focus on the analytic phase is consistent with manufacturer’s guidance IQCP documents. However, it is difficult to discern the specifics around manufacturer’s requirements from the IQCP document.

Missing sources of error in analytic phase - What if the POC device is dropped?   What about risks associated with cartridge load – overfill or underfill?

 ★★★             

Meets

Best Practice would be t provide citations to each manufacturer recommendation & requirement

Includes post-analytic phase

★★    

Partially meets

The details of interferences from the manufacturer’s instructions for use are lacking. How does the laboratory ensure that these are recognized/handled by testing personnel?

Are testing personnel trained to recognize and respond to inconsistent test results?

★★★              

Meets

Includes 5 components of testing:   Reagent

 ★★     

Partially meets

Missing sources of error associated with external quality controls and calibration verification product storage and handling. We can think of at least 20 possible sources of error compared to 5 in Sample IQCP.
Includes 5 components of testing: Environment

★★     

Partially meets

Missing sources of error that may be associated with altitude (barometric pressure) as well as risks in the work environment such as distractions, noise and interruptions. We can think of at least 7 possible sources of error compared to 5 in Sample IQCP.
Includes 5 components of testing: Specimen

★★       

Partially meets

Missing error sources associated with collection device instructions for use. Missing error sources associated with sample contamination during collection either from alcohol at collection site or IV fluid contamination. We can think of at least 17 sources of error compared to 9 in Sample IQCP.
Includes 5 components of testing: Testing personnel

★★     

Partially meets

  We can think of at least 8 sources of error compared to 5 in Sample IQCP.
Includes 5 components of testing: Test system

★★     

Partially meets

Missing error sources associated with intended use, test method limitations and correlation of results.   We can think of at least 45 sources of error compared to 14 in sample IQCP.
Variations in the components based on use of the test in different environments

Does not meet

Not accounted for in IQCP document.

Best Practice would be to include this in a IQCP

Variations in the components based on use of the test by different personnel

Does not meet

Not accounted for in IQCP document. Best Practice would be to include this in a IQCP
Variations in the components based on use of the tests with multiple identical devices.

Does not meet

Not accounted for in IQCP document. Best Practice would be to include this in a IQCP
Data included from the laboratory’s own environment

Does not meet

No evidence that data from the laboratory’s specific environment was included in the IQCP document.

Best Practice would be to include this in a IQCP

Data included from the laboratory’s own instrument/equipment performance

Does not meet

The only hint of the laboratory’s past performance with QC is referenced in the risk assessment under “instrument prevents results from being reported when external liquid QC is unacceptable”. The source of error “external liquid QC is unacceptable” is scored as a frequent occurrence with negligible severity of harm. Based on IQCP document, this scoring appeared to be based on external quality control for new lots of reagents.  
Data included from the laboratory’s own testing personnel

★  

Does not meet

Not accounted for in the IQCP document Best Practice would be to include this in a IQCP
Manufacturer’s instructions and recommendations

★★

Partially meets

Difficult to discern as sources of error are defined in more general terms. For example “specimen stability” does not include specifics around manufacturer’s exact instructions for sample stability. Best Practice would be to have itemized manufacturer recommendation and provide citations to the manufacturer’s source documentation.
Quality Control Plan      
COM.50400   The IQCP includes a written quality control plan approved by the laboratory director prior to implementation.  
IQCP includes written quality control plan

                

Meets

Written quality control plan included

                

Meets

COM.50500  

The individualized quality control plan must define all aspects monitored based on the potential errors identified during the risk assessment, including the following parameters as applicable:

The number, type (external and internal quality control systems), and frequency of quality control

Criteria for acceptable performance

● Monitoring of the testing environment and reagents

● Specimen quality

● Instrument calibration, maintenance, and function checks

● Training and competency of testing personnel

Provisions for multiple identical devices and variation for uses covered under one IQCP

 
Number, type and frequency of quality control

★★★                    

Meets

 

★★★
Meets

Criteria for acceptable performance

★★★

Meets

From a regulatory perspective, while a proficiency testing score of 80% or higher is graded a satisfactory performance by CMS, corrective actions are still required by CMS for any PT failure or scoring less than 100%.

★★★

Meets

Monitoring of testing environment and reagents

★★★

Meets

 

★★★

Meets

Specimen quality

 ★

Does not meet

No mention of monitoring criteria for sample quality.

★★★

Meets

Instrument calibration, maintenance and function checks

★★      

Partially meets

No mention of thermal probe check in the Quality Control Plan.   The manufacturer’s system manual specifies “Check Thermal Control System Every Six Months”

★★★

Meets

Training and competency of testing personnel

★★★              

Meets

 

★★★

Meets

Provisions for multiple identical devices and variance for uses covered under one IQCP

Does not meet

No mention of provisions for multiple identical devices for the POC Chemistry system.

★★★

Meets

Quality Assessment      
COM.50600  

Ongoing quality assessment monitoring is performed by the laboratory to ensure that the quality control plan is effective in mitigating the identified risks for the IQCP and includes the following:

Review of quality control and instrument/equipment maintenance and function check data at least monthly

Evaluation of errors relating to preanalytic, analytic and post analytic phases of the testing process

Review of complaints from clinicians and other healthcare providers regarding the quality of testing to confirm the clinical efficacy of testing, and

● Evaluation of corrective actions taken if problems are identified

● Reapproval of the quality control plan by the laboratory director or designee at least annually

 

 
Review of quality control and instrument/equipment maintenance and function check data at least monthly

★★★                

Meets

 

★★★

Meets

Evaluation of errors relating to preanalytic, analytic and postanalytic phases of the testing process.  

★  

Does not meet

Sample IQCP references a Quality Assurance Year to Date Report, but does not provide enough detail to ensure the evaluation of the three phases of the testing process as part of the QA process.

★★★

Meets

Review of complaints from clinicians and other healthcare providers

★★★ 

Meets

 

★★★

Meets

Evaluation of corrective actions if problems identified.

★★★

Meets

Quality Assurance Year to Date Report specifies “POCT resource must submit action plan to reduce error rate to allowable target”

★★★

Meets

Re-approval of the quality control plan by the laboratory director or designee at least annually

Does not meet

The Sample IQCP does not document re-approval of the QCP as part of the Quality Assessment Monitoring

★★★

Meets

       

Reference: CAP 2015 All Common Checklist Requirements for IQCP (NEW)

Scoring:

 ★★★    Meets the requirements

 ★★     Partially meets the requirement

 ★      Does not meet the requirement

TBD: To be determined

fishbone comparisonsmall


About the Reviewers

At CarePoint Solutions, we probably view IQCP’s quite differently than most. We view it as a process, not merely a document.

We believe that IQCP’s are the beginning of a transformative wave in healthcare delivery, as we make the transition to quality management systems that will drive improved efficiency, economy, and quality. Just about every other major industry that has adopted these data driven, outcomes oriented processes, has dramatically transformed the quality of their output. Quality feedback loops drive continuous improvement, and these quality management systems foster a culture of quality that includes (and empowers) each participant in the process.

This is really good news, given the recent studies from CDC, the Institute of Medicine, and John Hopkins that implicate preventable medical error as the #3 killer in the US, just behind cancer and heart disease. This translates to about a quarter of a million needless deaths occurring in our hospitals each year. That’s roughly 40 deaths per US hospital, suggesting a systemic problem. And guess what the number one contributor to these errors is? Diagnostic error.

Worse still, these studies conclude that, as the complexity of healthcare delivery increases, the incidence of diagnostic error is increasing as well.

So the CarePoint team is excited to announce the launch of Pro-QCP, a web app that represents a first-of-its-kind quality management system for point of care testing. This is an expanded software app that not only delivers a superior IQCP, but underpins the processes that drive continuous improvement. Incorporating best practices that have been proven in other industries, Pro-QCP has the potential to drive efficiency, significantly lower healthcare costs, and measurably improve patient outcomes.

Pro-QCP takes you step-by-step through a process of identifying sources of error in your test process, collecting your historical data along the way (e.g., the incidence of sample hemolysis or patient misidentification). The software also suggests key performance indicators (KPIs) that will help trend the effectiveness of your quality efforts over time.

CarePoint is already working on the next generation of Pro-QCP that will provide KPI dashboards and real time inter-laboratory comparisons that allow you to compare your error detection, your error mitigation efforts, and your own performance indicators to those of your peers – in real time. In a collaborative way, user data is leveraged to enhance the program content over time. For example, users will be able to compare what process controls are being employed by their peers, and they can see how effective those process controls have been at improving quality. And the value of the data repository increases as the user base grows.

While the incidence of preventable deaths due to diagnostic error is quite concerning, we believe adopting quality management practices is a appropriate response and offers significant promise to improve the outcomes of patients in our care.