Essays
2016 IQCP Users Survey (USA)
Here are the results of our 2016 Survey of IQCP Users. Over 210 results from both within the US and abroad.
2016 IQCP Users Survey (USA)
July 2016
Sten Westgard
Nearly 150 laboratory professionals in the US shared their experiences with us about their implementation of IQCP. We're breaking out the US labs from the labs from the rest of the world because, as you may eventually see, the views and experiences are very different. In the US, IQCP is a regulatory mandate, a required activity for labs that want to run QC less than once a day. Outside the US, labs that adopt IQCP are doing so voluntarily.
when we surveyed labs on their IQCP preparations two years ago, we were able to get many more responses (closer to five hundred, or more than twice what we got in this survey). My jaundiced view of our lower response rate this time is that the labs that actually went through IQCP implementation are exhausted by it and just want to put it behind them; they're not as interested in talking about it now that the policy has become mandatory. Or it could be that we've just talked and talked and talked about IQCP for so long this isn't interesting anymore. In any case, let's move on and look at what US labs are saying.
US Labs: Type of Labs Responding to the IQCP Survey
We had mostly core laboratory users reply to our survey, but very few of the small laboratories.
US Labs: Volume of Tests Per Day
Most of those who responded came from large labs, with between 1,000 and 10,000 tests per day.
US Labs: How many IQCPs have they completed?
Most labs report having only completed a few IQCPs, between one and five.
US Labs: How many sites needed IQCPs?
A plurality of labs have only had to plan IQCPs for a single facility. But close to a third have had to design IQCPs for 3 or more sites.
US Labs: Number of staff involved in IQCP creation
The most common response was that five or more staff were dedicated to the task of creating IQCPs. However, the second most common response was that only one staff person was saddled with the same task. This suggests that some labs are overburdening their staff, having just one person take on a project that should be collaborative and multidisciplinary.
US Labs: Number of Hours required to develop an IQCP
The most common response was that each IQCP took more than 15 hours to complete. If you take that with the most common number of staff involved (5), that's 5 * 15 or 75 staff hours per IQCP. This is adding up to a significant commitment of labor. For high volume labs that have more staff, they may be able to find the time and people to tackle this challenge, but it's hard to see a small volume operation having that much spare time or staff to devote to this task.
Of course more than 1 in 5 labs reported that it only took between one and 5 hours to complete, which may be too short a time to fully analyzer and assess the risks of a test. Another 11% said they use IQCPs provided by the manufacturer or vendor. This might be a worrying finding, if those labs just copied manufacturer information and failed to properly "individualize" their IQCPs.
US Labs: What Resources did they use to develop their IQCPs?
There are a lot of resources out there available to the laboratory, but mostly labs have only taken advantage of the free material from CMS and CAP. The most popular item is a set of brochures on IQCP, followed by the free IQCP booklet from CMS. Neither of these resources is thorough but they do provide a start. The CAP provides a set of forms that must be used by its members if they implement IQCP.
Just over half of the surveyed laboratories made the purchase of EP23 to inform their IQCP development. After that, however, mostly labs have generated home-grown solutions. The commercial resources from COLA and the automated EZ-IQCP software have only been used by a small number of labs.
US Labs: Number of Hazards, Failures Modes, and Risks Assessed
The most common response was that between 10 and 20 hazards, risks, or failure modes were assessed. This may reflect a poor wording of the question. But there should probably be more hazards identified in an IQCP. The examples I have seen and we have posted online show a lot more possible risks, hazards, and failure modes. If most labs are just finding around a dozen risks, they may not be adequately assessing their risks or they may be willfully ignoring them.
We've seen manufacturers provide stock/sample/template IQCPs which have between 10 and 20 failure modes, which means an individual laboratory should be finding more risks of their own (the local lab has its own risks specific to their location).
US Labs: Has the IQCP process revealed any unacceptable risks?
A little under a third of the labs report that the IQCP process helped them discover hazards that heretofore had been unknown or unaddressed. However more than two-thirds of laboratories found no new risks and/or no unacceptable risks.
US Labs: Did the IQCP require you to add additional control mechanisms to handle new risks?
While a little under a third of laboratories uncovered some risks they hadn't noticed before, fewer labs had to introduce new controls to deal with them. Most labs found that either there were no new risks discovered during the IQCP process, or even the new risks that they did uncover were adequately controlled by their current and existing quality control mechanisms. In other words, for nearly 80% of labs, the IQCP process did not generate any new actions for them to take.
The cynical perspective is that the IQCP process is not adding value, but merely consuming the time and effort of our scare labor resources. If an IQCP uncovers nothing, or results in no new changes in QC behavior, why was it necessary in the first place?
US Labs: After IQCP, how often are they performing their QC?
For over two-thirds of labs, IQCP has meant moving to (or staying at) once a month QC. This is suspiciously identical to the QC frequency generated by the now-illegal EQC options. Again, this begs the question - if the EQC options led a majority of labs to once a month QC but we acknowledge that this was an unscientific process, how can we end up in the same place with "Risk Management" and those previously unacceptable practices are now acceptable?
It's a kind of magic act: with a wave of the Risk Assessment wand, we've been able to take the old unscientific once-a-month EQC and presto-chango it's now compliant and acceptable once-a-month RiskQC.
US Labs: Have their IQCPs been inspected yet?
Two-thirds of the labs responding to the survey hadn't been inspected yet, which means we don't even know if their IQCP practices will in fact be considered compliant and acceptable to the regulators. The last two years of "education" and "transition" have been rather disconcerting for labs because the regulators have spent more time marketing the benefits of IQCP than they have spent time providing concrete details of what an IQCP should actually look like. It wasn't until this year that CAP really provided the most detailed forms for the documentation of IQCP.
US Labs: Did your IQCPs survive the Inspection?
Almost two-thirds of the labs that were inspected had their IQCPs found to be adequate. Roughly another third of labs were inspected without having their IQCPs examined (the inspector simply inspected other parts of the lab). Only a small minority of labs found themselves in trouble with their IQCPs. Here are all the reports of problems that inspectors found with IQCPs:
- "Needed to add components to IQCP"
- "We were cited for failure to have an IQCP for the Affirm VPIII test. COLA subsequently accepted our IQCP response [thereafter]."
- "TJC surveyor made recommendations related to improving the format of the IQCP's, but felt the content was adequate."
- "We were cited for not having a count of the actual internal and external QC and failures documented for a particular time"
Notice that the few problems that were found had more to do with form rather than substance, counts and components rather than assessments of risk. It appears that many inspectors are not adequately trained in IQCP inspection, and thus labs are still confused about how to prepare these IQCPs properly. CAP and CMS have provided most guidance in the Table of Contents, components, and formatting of an IQCP, with little attention paid to the proper assessment and reduction of risks.
US Labs: What is your overall satisfaction with the IQCP process?
This is a very well-distributed set of experiences! There are almost the same number of respondents who were somewhat satisfied and very satisfied as there were labs that were somewhat dissatisfied and very dissatisfied. The dissatisfied vote edge out the satisfied vote by just a hair. The most common answer, as you can see, was simple indifference. There are many labs that neither like nor dislike the IQCP.
The overall picture that emerges from the US labs depends on your perspective. If you believe that IQCP wasn't going to be a revolutionary new technique, that all is was going to do was ratify, rationalize and legalize current reduced QC practices, then it has largely succeeded in that task. If you were hoping IQCP might provide a new perspective, one that would encourage laboratories to increase their QC frequency and take more care in what devices they choose for testing, it has not been a success. The cynical perspective is that IQCP has merely been a paperwork exercise, a piece of regulatory theater that allows once-a-month QC POC devices to continue to operate despite the fact that this may be inadequate to the needs of good laboratory practices or good patient care.
For more views, please see the open text comments shared by the survey participants.