Guest Essay

Wayne Dimech revisits the quality challenges of SARS-Cov-2 testing. More than a year later, are we doing any better? Wayne Dimech is Executive Manager, Scientific and Business Relations at NRL, Australia (NRL) in Melbourne. He is a recognized expert in infectious disease serology and laboratory quality.

"Are we there yet"?  COVID19 and the Lab, 18 months later.

Wayne Dimech, B. Appl Sci.; MASM; MBA; FAIMS; FRCPA (science faculty)
July 2021

Recently, I had the pleasure of talking with my microbiology university lecturer at a dinner party. I was fascinated as he recounted his experience as a young scientist working with smallpox in post-war England. It made me wonder how we might discuss our experience as scientists working with COVID-19 in years to come. The pandemic has impacted all countries and all aspects of our lives. Many commentators in the future will review each country’s response in economic and health terms, but it is timely, 18 months into the pandemic, to review the successes of medical laboratory science and assess where we might improve if we are ever faced by a similar situation in the future.

My first thoughts are to acknowledge the medical laboratory scientists and technicians who have been the backbone of all counties’ responses. Quite rightly, patient-facing health workers are lauded for their tireless and brave work; but laboratory staff have quietly and vigilantly tested countless millions of SARS-CoV-2 tests, 24 hours a day; every day of the pandemic and it seems unrelenting. Normal rosters were abandoned, workers re-trained and re-deployed. Overnight shifts implemented and new instrumentation and techniques introduced. Each morning, health minsters all around the world front the media and report the numbers of tests performed and the numbers of positive cases detected. Rarely, if ever, is credit given to the laboratory workers providing the test results. So, I would like to take the opportunity to thank every laboratory worker for their contribution over the past 18 months.
Testing of clinical samples is only a small part of the contribution of medical science. Within weeks of SARS-CoV-2 being detected and recognized as a pathogen of significance, scientists had sequenced the viral genome. Within a month, organizations such as the Peter Doherty Institute in my hometown of Melbourne, had grown the virus enabling researchers, reference laboratories and quality assurance providers to implement their activities. By January 2020, clinical and reference laboratories globally had developed in-house nucleic acid testing (NAT) allowing large scale screening and potential cases detected and isolated. At the same time, China CDC, USA CDC, Charité – Universitätsmedizin Berlin and others had recommended primers and probes for different genomic regions and with different screening and confirmatory strategies. Research into the pathology, spread, epidemiology and immunology of the virus, led by medical scientists, was undertaken and findings accumulated and shared. This information was critical to determining appropriate community responses to the pandemic. It is hard to imagine how the world would have experienced COVID-19 without the contribution of medical scientists.

In a similar vein, the contribution of the in-vitro diagnostic device (IVD) industry has been outstanding. By July 2020, there were over 700 commercial SARS-CoV-2 assays available on the market. This is outstanding given the time to market for a new IVD is usually measured in years, not month. Laboratory testing using in-house assays would not have been sustainable. Access to commercial IVDs, manufactured in quantity, under quality processes, was critical to sustaining a laboratory response. Apart from manufacturers of NAT IVDs, the reality is that many of these IVDs are not necessarily commercially viable but brought to market as a response to the pandemic. IVDs were brought to market with short expiry dating leading losses due to expired reagents. No doubt initial decisions were made for potential financial gain, but the decision to maintain non-viable products allows scientists to choose from a larger range of options. This outcome was achieved in an environment where there was competition for raw materials, restrictions on staff movement and often local transmission of infection in the countries of manufacture, restricting the normal training of engineers and customers. Planned pipelines for development were abandoned and resources re-directed to COVID-19 activities. This research and development of new products and the process for bringing them to market was also led by medical scientists.

Could we have done any better? I believe that there were areas where our approach could have been improved; mostly related to well-established principles not being followed due to the nature of the pandemic.

As outlined above, large numbers of test kits were brought to market in a short period of time. Many of the manufacturers of these test kits had no previous experience in infectious diseases and it would not be unreasonable to assume that some of these tests may not be fit-for-purpose. In situations like this regulatory authorities are expected to support laboratories in the selection of test kits by ensuring that only quality tests are sold in their jurisdiction. Existing regulatory requirements, based on risk of the pathogen to the individual and the community, are used to authorize the marketing of IVDs. The highest risk category relates to test kits used to screen for infections that pose a risk to both the individual and the community. There is little doubt that SARS-CoV-2 falls within this category. Under normal circumstances, manufacturers must demonstrate the IVDs are safe, comply with the essential principles and are fit-for-purpose. A comprehensive dossier detailing the performance of the test kit against pre-defined criteria is required. Although sometimes seen as barriers, these requirements are important mechanisms to ensure the quality of testing.

In the COVID-19 situation, all major regulatory bodies allowed the listing of test kits for SARS-CoV-2 as Emergency Use Authorization (EUA). Regulators relied on minimal performance data provided by the manufacturer rather than instigating independent performance testing. The requirements for comprehensive evidence of the performance of the test kit prior to registration was essentially removed. In European Commission, COVID-19 test kits are self-declared, allowing manufacturers to affix a CE IVD certification to the test kit without independent scrutiny by a notified body. Ironically, from early 2022, SARS-CoV-2 test kits will be classified as the highest risk category (Class D) under the IVDR regulations, similar to HIV and hepatitis assays. The USA FDA “recommends that developers confirm performance of their assay by testing a minimum of 30 positive specimens and 30 negative specimens as determined by an authorized assay. If you do not have access to clinical samples as determined by an authorized assay, contrived clinical specimens may be considered.” Note that the “authorized assays” are registered as EUA. The use of 30 positive/negative samples is barely sufficient for a laboratory-based verification of a previously evaluated IVD. The Australian Therapeutics Goods Administration (TGA) allows the registration of SARS-CoV-2 test kits as Emergency Use. Only rapid device tests (RDT) for serology are required to undergo a “post-market evaluation”, which the National Serology Reference Laboratory, Australia (NRL) is conducting under contract to the Peter Doherty Institute.

This situation has resulted in open access to SARS-CoV-2 test kits but also to limited access to scientifically-sound performance information. Consequently, many countries reported spending millions of dollars purchasing test kits that could not be used due to poor performance or inappropriate for the purpose. While it is understandable that regulators felt the need to remove restrictions to access to important diagnostic tools, this approach sent a signal to potential users unfamiliar with regulatory requirements that test kits registered with EUA had undergone the stringent regulatory processes used for all other IVDs. This situation added extra pressure on laboratories who needed to either evaluate assays themselves of rely on publicly available information. Unfortunately, most laboratories do not have the capacity or the expertise to perform appropriately designed studies leading to significant duplication of effort.

Compounding this situation has been the ready publication of non-peer reviewed articles, both by reputable scientific journals and other media. The need for information particularly early in the pandemic was high and scientists were keen to share their findings. However, the evaluation of IVD performance is a specialized discipline and many of the publications were flawed, with different studies reporting grossly different sensitivities and specificities for the same IVD. This causes confusion and, potentially, incorrect concern that test kits under perform or a false sense of security that certain IVDs are fit for purpose. Like regulators, scientists look to scientific journals, as a source of dependable and trustworthy information. Studies with poorly structured methodology, using small sample sets and few test kits, and having inappropriate conclusions and interpretations led to difficulty in understanding the performance of test kits available.

Some institutions have provided the clarity required. The World Health Organization (WHO) has been a leader in supporting the acquisition of comprehensive performance data for IVDs. NRL has been commissioned by WHO to conduct a comprehensive evaluation of 40 RDTs and EIAs for SARS-CoV-2 serology using the same sample set including seroconversion panels and interfering and cross-reacting samples. Independently, NRL is also providing this service to manufacturers of automated assays. The results will soon be publicly available on the NRL website (www.nrlquality.org.au). The Foundation for Innovative New Diagnostics (FIND), in collaboration with University Hospitals of Geneva (HUG), has also conducted large scale evaluations of molecular assays, also using the same panels of samples. The Paul Ehrlich Institute (Langen, Germany) has conducted evaluations of SARS-CoV-2 antigen test kits using the same sample panel.

Now, more than 18 months since the onset of the pandemic and we are experiencing a new sense of “normal”. Numbers are still high, lock downs still occur, but in many countries coping mechanisms have been implemented. This is the case for laboratory science as well. We have a greater understanding of the tools at our disposal; processes developed; staff trained. We have a little more time to understand and react to the situation. Where will be in 18 months time? In the world we now live in, it is so difficult to predict. What if a new vaccine-breakthrough strain emerges? Or if vaccine hesitancy is heightened? Countries without good access to vaccination will remain a reservoir for shedding into other countries, underlining the need for all countries to support each other’s access to the vaccine.

I do predict that, unlike SARS-CoV-1, the virus will be in the community for years to come, therefore a constant and ever-present challenge to our health systems; in particular laboratories. As populations are vaccinated and the virus suppressed, there will be a need for rapid testing to help contain the spread of remnant virus for those countries that choose that option. Decisions about the efficacy of “vaccine passports” are still to be made. This approach may rely on evidence of vaccination or maybe laboratory-based evidence of antibodies detection. There is precedence for a vaccine passport with the requirement for Yellow Fever vaccination before entry to some countries, but the efficacy of this approach is still debatable.

Medical laboratory scientists and technicians will continue to play a significant role in ensuring the health of our communities and continue to test increasingly large numbers of swabs, develop and validate new procedures, report results with increasing rapidness and to maintain the quality of all other pathology tests requested.

I hope this special effort by medical laboratory staff is publicly acknowledged in the community.

See even more stories about COVID-19 Laboratory Challenges...

About the author, This email address is being protected from spambots. You need JavaScript enabled to view it.

Wayne Dimech is Executive Manager, Scientific and Business Relation of the NRL, a World Health Organization (WHO) Collaborating Centre for Diagnostics and Laboratory Support for HIV and AIDS and Other Blood-borne Infections. Mr Dimech obtained a degree in medical laboratory science at Royal Melbourne Institute of Technology (RMIT) University in Melbourne, before undertaking a microbiology fellowship at the Australian Institute of Medical Scientists and completing an MBA at LaTrobe University in Melbourne. He is also a Fellow of the Faculty of Science (Research) of the Royal College of Pathologists Australasia. He has worked in private and public pathology laboratories predominantly in Microbiology Departments, where he specialised in infectious disease serology. Mr Dimech’s research interests include the control and standardisation of assays that detect and monitor blood-borne and sexually transmitted infectious diseases.  A particularly interest is the standardization of rubella testing and the monitoring of infectious disease assay variability.  He was instrumental in the development of EDCNet, an internet-based program for monitoring quality control test results, which is now used worldwide, and in the optimization of OASYS, software designed to manage the external quality assessment schemes. Mr Dimech is an advisor for numerous national and international working groups, including the Australian Hepatitis B Testing Strategy, Standards Australia and consultancies under the auspice of WHO, International Health Regulations, UNDP and the Global Fund. He has authored or co-authored about 50 articles in international peer-reviewed journals and contributed to three book chapters.