Virolens vs Other COVID-19 Testing Modalities (Part 2)
Part 2: Clinical trial outcome interpretation
Excerpts from a report written by Dr Kerry Mills, Chair of the International Virolens Research Consortium
Direct comparison of Virolens with RT-PCR: results of performance testing
Testing methodologies that measure different things can be compared. For example, the limit of detection, susceptibility to interfering substances, cross-reactivity with other pathogens - these can all be done and compared with RT-PCR. In this way, the inherent ability of the technology to correctly identify viral RNA (in the case of novel PCR tests), viral protein (in the case of novel RATs), or whole virions (Virolens) can be determined. These results represent the best-case scenarios for each of the technologies.
The efficacy of RT-PCR in real-world testing
Due to the necessity for processing of the sample, RT-PCR cannot differentiate between infectious virions, genomes from lysed cells, or non-infectious viral RNA shedding (Figure 1). Furthermore, it has been shown that contamination of commercial primer/probe sets is rife, making false positives inevitable.5 RT-PCR has been shown to have a minimum 5% false positive rate, and a minimum 20% false negative rate in the field.6 That is, although RT-PCR is highly sensitive in itself, a single test in a real life situation will fail to detect a positive case in at least 1 in 5 people. This can be due to the swabbing technique, sample preparation, problems with the machine, the viremia of the tested person, the presence of inhibiting antibodies, vaccination status and day post-infection among others (Figure 3).
Figure 3: real-world sensitivity and specificity of RT-PCR
In fact, the probability of a negative RT-PCR result varies over time, with a median of 67% of cases missed at day 4 post-infection, and a median of 38% at the day of symptom onset (Figure 4). In fact, the greatest correlation between RT-PCR and SARS-CoV-2 infection is three days after symptom onset, a time at which a person has already passed the infection onto others.
Figure 4: probability of a negative test result despite having SARS-CoV-2 infection by days after initial infection (data from Kucirka 2020)7
Therefore, the lack of correlation between RT-PCR and a test that uses a different input source (antigen, antibody, whole virions) in a real-world scenario cannot necessarily be ascribed to the other test. In fact, any “false positives” obtained from a new device before the day of symptom onset (days 1-4) are by definition more likely to result from a failure of RT-PCR than from the other test. Similarly, from day 17 post-infection (day 12 after symptom onset), the likelihood that a negative RT-PCR test is a false negative again rises above 50%, making it more likely that a positive result from a different test is, in fact, the correct result.
Regulatory requirements
Most of the global regulators have harmonised their requirements for approval of SARS-CoV-2 tests based on the WHO recommendations.8 This document sets out the requirements for performance testing of novel nucleic acid and rapid antigen detection tests. However, because there is currently no approved test that does not detect SARS-CoV-2 nucleic acid, antigen, or antibodies against SARs-CoV-2 proteins, there is no guidance for the performance testing of novel technologies.
Importantly, the WHO does not require RATs to be compared with RT-PCR, because these tests are, as described above, actually testing for different things, and the comparison is not valid. Furthermore, under such circumstances, no RAT would be given approval, as they do not perform as well as RT-PCR. Thus novel nucleic acid tests must be tested against an established, characterised RT-PCR test, and novel antigen tests must be tested against an established, characterised antigen test.
Therefore, although many of the requirements for performance testing listed in the WHO document can be performed for Virolens (limit of detection testing, interfering substances testing, cross-reactivity testing), the current options for comparators (PCR, RAT) are not valid for Virolens. Indeed, just as the comparison between PCR and RAT would expect to result in divergent results, a comparison between Virolens and PCR, and Virolens and RAT would equally be expected to result in a significant lack of concordance.
How to interpret test results from Virolens trials
Many international regulatory and government bodies currently require Virolens to be compared with RT-PCR. However, given the information above, this is not appropriate. The following section can help with interpretation of Virolens vs RT-PCR results.
Scenario one: Virolens negative, PCR positive
There are several explanations for this outcome:
1) Virolens is giving a false negative
2) PCR is giving a false positive
3) PCR is detecting viral RNA that does not form part of infectious virions (see Figure 1)
Depending on the timing of the test, different scenarios have different likelihoods. To arrive at the most likely explanation, the following should be done:
1) A characterised SARS-CoV-2 stock should be serially diluted, fixed with PFA and used as a positive control for Virolens, and a known negative sputum sample should be used as a negative control.
2) All inputs into the RT-PCR should be thoroughly tested for contamination.
3) Samples should be incubated with a permissive cell line according to our established protocol.
If the positive and negative controls work in the Virolens, the machine is working correctly. If the RT-PCR inputs (buffers, nucleotides, enzymes etc) have been thoroughly tested, then the PCR is working correctly. This leaves the possibility that the PCR is still giving a false positive for another reason, that the RT-PCR is more sensitive than Virolens, or that the RT-PCR is picking up RNA that is non-infectious, i.e. not in virions. Given the sensitivity of Virolens is known to be at least that of RT-PCR, the most likely conclusion is that there are no infectious virions present in the sample, probably because the person has already cleared the infection. NOTE: as with all testing methodologies, this does not necessarily mean that the person is not infected. Should they go on to develop symptoms, the person should seek further testing.
Scenario two: Virolens positive, PCR negative
There are several explanations for this outcome:
1) Virolens is giving a false positive
2) PCR is giving a false negative
3) The sample used for PCR testing was done poorly, either deliberately (if done by the person themselves) or accidentally (poorly trained staff, human error etc)
Depending on the timing of the test, different scenarios have different likelihoods. To arrive at the most likely explanation, the following should be done:
1) A characterised SARS-CoV-2 stock should be serially diluted, fixed with PFA and used as a positive control for Virolens, and a known negative sputum sample should be used as a negative control.
2) The person should be given a small amount of water to drink, and should be asked to wait for 15-30 minutes, and given a repeat Virolens test.
3) A repeat PCR test should be carried out every two or three days for up to 14 days.
If the positive and negative controls work in the Virolens, the machine is working correctly. If the person rinses their mouth, waits for the correct amount of time, and tests positive again, then the Virolens test is correct. If the test is done soon after exposure, before the onset of symptoms, or in a screening situation (i.e. testing large numbers of people who are all asymptomatic), the most likely conclusion is that the Virolens result is correct. As seen in Figure 4, PCR is highly inaccurate early in infection.
Conclusion
Virolens, RT-PCR and Rapid Antigen Testing all have specific targets and all detect different things. As such, the direct comparison of one with another is not appropriate, and is not required by the WHO. The testing undertaken on the Virolens machine shows it to be at least as sensitive as RT-PCR, over 99.9% specific to SARS-CoV-2. As such, once the machine has correctly processed the negative and positive controls, the results can be reliably interpreted.
References
