Computation of the “Gray Zone” Based on Experimental Data and Its Effect on the Trueness in HIV Results in the Blood, Cells and Tissues Banks

Department of Quality Assurance, Portuguese Institute of Blood and Transplantation, Avenida Miguel Bombarda, Portugal.

Corresponding Author: Paulo Pereira, Department of Quality Assurance, Portuguese Institute of Blood and Transplantation, Avenida Miguel Bombarda 6, 1000-208 Lisboa, Portugal, Tel: +351-210063047; E-Mail:[email protected]

Received Date: 22 Dec 2016    Accepted Date: 02 Mar 2017   Published Date: 03  Mar 2017

Copyright© 2017 Pereira P

Citation: Pereira P. (2017). Computation of the “gray zone” based on experimental data and its effect on the trueness in HIV results in the blood, cells, and tissues banks. Mathews J HIV AIDS. 2(1): 015

ABSTRACT

Background: The impact of false negative HIV results in the clinical decision is a major concern, principally in the blood, cells, and tissues banks due to the high risk of post-transfusion/post-transplant infection. The use of the “gray zone” in medical laboratory tests is not systematically used. Thus, it is up to the laboratory to decide on its use. This text analyses a model to determine the “gray zone” based on the Guide to the expression of uncertainty in measurement and using experimental data. Materials and methods: Usually, the selected decision limit relies on a theoretical zone. Nevertheless, an empirical “gray zone” could be computed established on data already available in the medical laboratory offering a more realistic interval. An empirical model conforming to the “Uncertainty Approach” principles using intra-laboratory method is applied using short-term and long-term data. Results and discussion: The expanded measurement uncertainty of the combination of the within-laboratory uncertainty and the bias uncertainty is 14% and 23% on the short-term and long-term, respectively. The impact of the indeterminate results of true negatives is non-significant (0.04%) to the budget. 

Conclusions: The use of a “gray zone” based on HIV experimental data should be classified as a good laboratory practice, contributing to decreasing the residual risk related to post-transfusion and post-transplant infection.

KEYWORDS

Gray Zone; Gum; HIV; ISO 15189; Measurement Uncertainty; Window Period.

INTRODUCTION

INTRODUCTION A screening test is recognized as a technique “used to evaluate large populations of individuals for the presence of a disease or analyte” [1, 2]. The physician interprets the HIV result tested in a screening immunoassay as the in vivo condition/true result. Based on this result a clinical decision is taken. However, there is a chance for some binary results (positive/negative) in medical laboratory tests to be false. This situation represents a serious risk of incorrect clinical decision with an impact on the patient safety. This could also be viewed not only at healthcare but also at other stages, such as on the validation of components in human blood components, cells, and tissues banks [3, 4]. The risk to the customer significantly increases in these banks when compared to a hospital laboratory where typically negative results from individuals in groups of risk are retested on a second collection due to the chance of a seroconversion window period occurs. On the banks’ scenario, the collection of a second sample occurs only on the vigilance and surveillance when a post-transfusion or post-transplant infection is reported, so, it is not related directly to post-transfusion or post-transplant infection safety but indirectly with the corrective actions/preventive actions (CAPA) development. Let consider a result equal to the cutoff and the condition: positive if equal or higher than the cutoff. In this case, a true positive result of a Gaussian distribution is 50% false negative in an infinite number of determinations. It could be interpreted that true positive results close to the decision value have a significant statistical chance to be classified as false. The “gray zone” is understood as the area around the cutoff numerical results are classified as indeterminate [5]. Further testing is required to a final classification. In this condition, the numerical results have a trinary classification: positive/indeterminate/negative [6]. Adopting the ratio of the screening immunoassays s/co as the expression of the division of the sample raw data (s) by the cutoff (co): negative results are those with an s/co lower than the cutoff (1 s/co) minus the “gray zone”, indeterminate are those in the “gray zone”, and positive those equal or higher than the cutoff. Therefore, it is well recognized that the use of a “gray zone” in the classification of screening immunoassays results reduces the chance of positive samples with low concentrations of measurand to be classified as negative. The numerical results in the “gray zone” are presumed to be sporadic principally in tests with high diagnostic sensitivity. Currently, the use of the “gray zone” in screening immunoassays is not systematically required to classify the numerical results in an ordinal scale in commercial tests. Its use is also not claimed by ISO 15189, intended to the accreditation of medical laboratory tests [7]. Probably, the manufacturer interprets the “gray zone” importance as minor to tests highly reactive to weak concentrations of measurand (antibodies and or antigens tested under stable laboratory conditions), and poorly reactive when the measurand is absent. Nevertheless, the impact of wrong medical decisions has a strong social impact, principally when related to the post-transfusion/posttransplant infection [8]. This text analyses a model to determine the “gray zone” according to the Guide to the expression of uncertainty in measurement (GUM) principles [9]. A case study is presented and discussed to an easier understand of the empirical “gray zone” determination based on short-term and long-term data.

MATERIALS AND METHODS

Pereira et al. proposed an approach for computing the “gray zone” based on the measurement uncertainty of results close to the cutoff covering the whole analytical process [6]. It pools the within-laboratory reproducibility standard deviation s_Rw and the bias uncertainty u_b in the combined standard uncertainty u_c as follows [10, 11]:

S_RW=(S_r²+S_I²)^1/2                             (eq.1)

The s_Rw is the product of the square root of the sum of the squares of the repeatability standard deviation s_r and the intermediate standard deviation s_I. The authors suggest to calculate using one of the following single laboratory empirical approaches by means of method validation data: (a) s_Rw should be determined in a new test according to the Clinical Laboratory and Standards Institute (CLSI) EP15 (short-term data combining reproducibility and repeatability estimates) [12]. Or; (b) it should be calculated using data from between-run variation, as soon as long-term internal quality control results are available. s_Rw is computed pooling the repeatability standard deviation s_r from replicate measurements and the intermediate standard deviation s_I from between runs as follows:

sRW=(S_r²+S_I²)^1/2                              (eq.2)

Bias uncertainty is computed as follows [10, 11]:

u_b=(b²+(s_b/m^1/2)²+u(c_ref))^1/2             (eq.3)

where b is the bias, the mean deviation of results from the corresponding reference value, sb is the standard deviation of the bias measurement, u(cref) is the standard uncertainty of the certified reference value, and m is the number of replicate measurements.

The interlaboratory comparison approach is not suggested since the screening immunoassays are not standardized for what precision data are not available from an Appendix, as it is usually to standardize methods [13]. These data are however not yet available to screening immunoassays. The external quality assessment/proficiency testing model should not be used due to the heterogeneity of the participants of the laboratories’ group contribute significantly to the overestimation of measurement uncertainty/unrealistic indeterminate zone [6].

The “gray zone” is calculated by agreeing to the EURACHEM/ CITAC [14]. This guideline considers two zones: (1) Results are classified as reactive in the “rejection zone,” and (2) classified as negatives in the “acceptance zone.” The intersection between these zones is known as the “decision limit,” computed as follows:

1-1.65uc (eq.4)

where 1 is the cutoff ratio constant, 1.65 is the z-value at the 95% confidence interval, and uc is the combined standard uncertainty. Gaussian distribution and an adequate n are assumed. Further details about this methodology can be found elsewhere [15].  

RESULTS

Human serum or plasma samples are tested on Prism® HIV Ag/Ab Combo (Abbott Diagnostics, Abbott Park, IL, USA) [16]. This test is an in vitro two-step sandwich chemiluminescent immunoassay for the measurement of the concentration of antibodies to the Human Immunodeficiency Virus (HIV) type 1 (HIV-1) and/or type 2 (HIV-2) and/or HIV p24 antigen. Antibodies and antigen could be detected in serum or plasma of infected subjects with HIV. The Prism® HIV Ag/Ab Combo test uses microparticles coated with recombinant HIV-1/HIV-2 antigen and monoclonal HIV p24 antibody as a solid phase which binds possible HIV antibodies and/or antigen present. After incubation and a washing step, acridinium labeled conjugates, HIV-1 synthetic peptide, and HIV p24 antibodies are added. The presence of complexes is measured by the addition of an alkaline hydrogen peroxide solution. The chemiluminescent signal is proportional to the concentration of the anti-HIV-1 and/or anti-HIV-2 antibodies and/or HIV p24 antigen present in the sample. Determining the concentration of the anti-HIV-1/ anti-HIV-2 antibodies and/or HIV p24 antigen is however primarily a screening to classify a blood donor as HIV-infected or not. The test is calibrated using some plasma samples from persons not infected with HIV as negative controls and some plasma samples from patients infected with HIV as positive controls. The reagent manufacturer defines a procedure for computing the “cut-off” value - interpreted as the clinical decision level - for the number of emitted photons between positives and negatives.

This demonstration uses short-term and long-term data scenarios Short-term data This is a new test scenario where long-term data is unavailable. The indeterminate standard deviation is computed using a diluted positive human sample. The precision is calculated according to the CLSI EP15-A3 by one-way analysis of variance method [12]. Five replicates are tested for five days. Each day corresponds to an analytical run. The average is 1.63, the standard deviation is 0.09, and the coefficient of variation percentage is 5.59%. The repeatability standard deviation is 5.48%, lower than the between-run coefficient of variation (1.10%). Note that these estimates are significantly influenced by the replicates variance and by the variance of the average of the replicate, respectively. Bias uncertainty ub is determined between two laboratories of the Portuguese Institute of Blood and Transplantation (IPST) using the batch 116406 of Accurun® 1 Series 2400. Bias is 4.55% calculated with results from March 23, 2010, to June 11, 2011 Usually, (n_Lab1=288, X̄=2.64; n_Lab2=264, X̄=2.76). the certified reference materials are unavailable to screening tests. Thus, the u(c_ref) is zero.

Long-term data This is a routine test scenario where long-term data is already available. The indeterminate standard deviation is computed using internal quality control data from March 23, 2010, to October 25, 2011. OneAccurun®1 Series 2400 MultiMarker Positive Control (Seracare Life Sciences Inc., Milford, MA, USA) batch is tested (116406) [17]. The number of determinations is 348, an average of 2.62, the standard deviation of 0.25, and the coefficient of variation percentage of 9.49%. The repeatability standard deviation is measured from replicate measurements of samples having a ratio from 0.8 to 1.2 taken from the previous sampling (4.62%). The within-lab reproducibility uncertainty is 10.55%. Bias component is the same than in the short-term data case. Bias component is calculated as the one in the short-term data case. The measurement uncertainty is computed with the freeware MUKit v.1.9.5.0 (Finnish Environment Institute (SYKE), Helsinki, Finland) in compliance to [11, 18]. The computed expanded standard uncertainty is 14% on short-term and is 23% with long-term conditions. Therefore, the “decision limit” is equal to the product of 1.65 multiplied by the combined standard uncertainty: 11.69% on short-term and 12.09% with a long-term condition. Consequently and correspondingly, results between 0.88 and one and between 0.81 and one are classified as indeterminate. See (Table 1) for a summarization of the uncertainty components and combined estimate.  

Table 1: Measurement Uncertainty for the Prism® HIV Ag/Ab Combo (Abbott Diagnostics, Abbott Park, IL, USA).

† CLSI EP15-A3 method (one-way analysis of variance)

‡ NordTest TR 537 3.1th ed. method (control sample and routine samples replicates)

†† routine sample replicates

‡‡ control sample; between-run * certified reference material/control sample; the standard uncertainty of the certified concentration is assumed to be zero since it is unknown/unavailability of a certified reference materials material

DISCUSSION

The selection of the model to estimate the measurement uncertainty consider if the test is new in the med lab/a short-term method is used, or the test is implemented, and longterm data is obtainable. Note that the use of short-term model misestimates the measurement uncertainty and the “gray zone.” In some situations, bias remains unknown for what is assumed to be zero, also contributing for misestimating.

The impact of the indeterminate results is an associated issue that should be evaluated. Hypothetically, screening tests with high analytical sensitivity, i.e., high “quotient of the change in an indication and the corresponding change in the value of a quantity being measured”, have a lower chance to have weak s/co from true positive samples in the indeterminate zone [19]. Let consider the plasma samples of blood donors tested from September 27, 2014, to November 11, 2014, in the IPST and the long-term case (worst scenario). The number of blood donors’ samples tested is 10,132. The number of non-reactive results is 10,046 (99.15%), and the number of the reactive results is 14 (0.14%), as predictable in blood donors populations. The number of non-reactive results with indeterminate results is 3 (0.04%). All the samples in this interval are repeatedly indeterminate and negative in the confirmatory scheme. Allegedly, based on this test and sampling, the impact of the “gray zone” on the budget is minor. Otherwise, the impact of a false negative result in a blood bank is critical. The laboratorian could research the chance to occur indeterminate results using the delta-value method; see [20].

The decision limit could be easily placed on the laboratory information management system (LIMS), contributing rationally to an enhancement of the capability to minimize the falsenegative results in the seronegative samples of infected individuals. Pereira et al. proposes a definition to window period considering the effect of “gray zone” on the estimate: “the window period for a test designed to detect a specific disease (particularly an infectious disease) is the time between the first day of infection and the day when the test result cannot reliably rule out the possibility of infection (due to indeterminate results)”

[20]. Pereira suggested an alternative method compute the “gray zone” based on the “Error Approach” (Annex D.5 of [9, 21]) instead “Uncertainty Approach” (Annex D.4 of [9]) using the total analytical error (TAE) [22]. Using the TAE instead of the measurement uncertainty, the uncertainty results are systematically overestimated. When bias is unknown, both outcomes are not significantly different.  

CONCLUSION

It is suggested the use of a “gray zone” primary based on experimental data. Initially, short-term data could be tested. For instance, to a new test. As soon as long-term data is available, the “gray zone” should be retested increasing the reliability of the estimate. The “gray zone” practice decreases the residual risk related principally to post-transfusion infection in the blood, cells, and tissues banks since part of samples in window period have a higher chance not to be classified as negative. Its empirical computation should be recognized as a good laboratory practice.

REFERENCES

1. Clinical and Laboratory Standards Institute. (2006). MM10-A Genotyping for Infectious Diseases: Identification and Characterization, Approved Guideline. Wayne (PA): CLSI.
2. Clinical and Laboratory Standards Institute. (2006). MM12-A Diagnostic Nucleic Acid Microarrays; Approved Guideline. Wayne (PA): CLSI.
3. Pereira P, Westgard JO, Encarnação P, Seghatchian J, et al. (2015). Quality management in European screening laboratories in blood establishments: A view of current approaches and trends. Transfus Apher Sci. 52(2):245-251.
4. Pereira P, Westgard JO, Encarnação P, Seghatchian J, et al. (2015). The role of uncertainty in the results of screening immunoassays in blood establishments. Transfus Apher Sci. 52(2): 252‐255.
5. Kisner HJ. (1998). The gray zone. Clin Lab Manage Rev. 12(4): 277-280.
6. Pereira P, Magnusson B, Theodorsson E, Westgard J, et al. (2016). Measurement uncertainty as a tool for evaluating the “grey-zone” to reduce the false negatives in immunochemical screening of blood donors for infectious diseases. Accred Qual Assur. 21(1): 25-32.
7. International Organization for Standardization. (2012). ISO 15189 Medical laboratories-Requirements for quality and competence.
8. Müller B, Nübling CM, Kress J, Roth WK, et al. (2013). How safe is safe: new human immunodeficiency virus type 1 variants missed by nucleic acid testing. 53(10): 2422-2430..
9. Bureau International des Poids et Mesures. (2008). Evaluation of measurement data - Guide to the expression of uncertainty in measurement. JCGM 100:2008, GUM 1995 with minor corrections. Svres: BIPM.
10. Technical report. (2007). measurement uncertainty revisited: alternative approaches to uncertainty evaluation. EUROLAB.
11. Magnusson B, Näykk T, Hovind H and Krysell M. (2012). NordTest NT TR 537 Handbook for Calculation of Measurement Uncertainty in Environmental Laboratories, 3.1th edn.
12. (2004). Clinical and Laboratory Standards Institute. EP15- A3 User verification of precision and estimation of bias; an Approved guideline, 3rd edn.
13. (1994). International Organization for Standardization. ISO 5725–2 Accuracy (trueness and precision) of measurement methods and results-part 2: a basic method for the determination of repeatability and reproducibility of a standard measurement method.
14. Ellison SLR and Williams A. (2007). Use of uncertainty information in compliance assessment. Eurachem/CITAC.
15. Justice T. (1997). What is a guard band? Solid State Technol. 40(4): 85-86.
16. (2008). Abbott Diagnostics Division. Abbott Prism HIV Ag/ Ab Combo, code: 77-4254/R6. Wiesbaden: Abbott Laboratories.
17.  (2008). ACCURUN 1® Multi-Marker Positive Control Series 2400, code: 11322-02. Milford (MA): Seracare.
18. (2014). Finnish Environmental Institute. MUkit - Measurement Uncertainty Kit. Helsinki: SYKE.
19. (2009). International Organization for Standardization. ISO 15193 In vitro diagnostic medical devices-Measurement of quantities in samples of biological origin-Requirements for content and presentation of reference measurement procedures.
20.  Pereira P, Westgard JO, Encarnação P and Seghatchian J. (2014). Analytical model for calculating indeterminate results interval of screening tests, the effect on seroconversion window period: a brief evaluation of the impact of uncertain results on the blood establishment budget. Transfus Apher Sci. 51(2): 126-131.
21. Eisenhart E. (1963). A realistic evaluation of the precision and accuracy of instrument calibration systems. J Res Nat Bur Stand, Sect C 67C.
22. Pereira P. (2016). Measurement uncertainty in screening immunoassays in blood establishments [Ph.D. thesis]. Porto, Portugal: Faculty of Biotechnology, Catholic University of Portugal: 124.