madman
Super Moderator
ABSTRACT
Objectives
Accuracy of estradiol measurements is important but conventional proficiency testing (PT) cannot assess accuracy when possibly non-commutable samples are used and method peer-group means are the targets. Accuracy-based assessment of estradiol measurements is needed.
Design and Methods
Five serum samples were prepared from single- donors, frozen, and distributed overnight to 76 New York State Department of Health (NYSDOH)-certified laboratories. Participants analyzed samples for estradiol. The biases of group means were assessed against the Centers for Disease Control and Prevention (CDC)-defined targets, and evaluated using the Hormones Standardization Program (HoSt) E2 performance criteria of±12.5%. Each laboratory’s performance was evaluated using total allowable error (acceptance limits) of target ±25% or ±15 pg/mL (55.1 pmol/L) (whichever was greater, NYSDOH), target±30% (Clinical Laboratory Improvement Amendments [CLIA]), and target ±26% (minimal limit based on biological variation [BV]).
Results
The biases (range) were 34% (-17% to 175%), 40% (-33% to 386%), 16% (-45% to193%), 5% (-27% to 117%), and -4% (-31% to 21%), for samples at estradiol of 24.1, 28.4, 61.7,94.1 and 127 pg/mL, or 88.5, 104.3, 226.5, 245.4 and 466.2 pmol/L, respectively. Large positive method/analytical systematic biases were revealed for 9 commonly used method/analytical systems in the United States at low estradiol concentrations. Of the 9 analytical systems, 0, 0, 3,7, and 6 met the HoSt criterion for the samples with estradiol at the five respective concentrations. PT evaluation showed that 59%, 69%, and 87% of laboratories would receive a PT event pass score when the CDC-defined target and a criterion of NYSDOH, CLIA, or BV were used, respectively. However, >95% of laboratories would obtain PT pass scores if method peer group means were used as targets regardless of the criterion used.
Conclusions
Improvement in the accuracy of estradiol measurements is needed, particularly at low estradiol concentrations. Accuracy-based PT provides unambiguous information about the accuracy of analytical/method methods/analytical systems.
INTRODUCTION and OBJECTIVES
Estrogens are responsible for the development of the secondary female sex characteristics and play an important role in female reproductive processes. Estradiol measurements have a wide range of clinical utilities, e.g., diagnosis of fertility disorders, gynecomastia in males, estrogen-producing ovarian and testicular tumors, disorders of sex steroid metabolism, monitoring low-dose female hormone replacement therapy in postmenopausal women, and antiestrogen therapy(1, 2). To meet those clinical needs, accurate measurement of estradiol in patient care at all clinically relevant concentrations is needed; however, information on measurement accuracy is limited. Furthermore, its accurate measurement provides information that can be used to improve the quality of manufacturers’ products, assess the effectiveness of manufacturer standardization, and advance current evaluations performed as part of activities related to meeting regulatory requirements.
Although proficiency testing (PT) is an effective tool for monitoring the quality performance of clinical laboratories and analytical systems, it has limitations (3). Conventional PT often uses non-commutable samples or modified samples whose commutability is unknown, and therefore only evaluates participants’ results using method peer-group mean values as targets (4). Therefore, due to the presumed existence of matrix effects, conventional PT can only assess whether a laboratory’s analysis can meet acceptance limits relative to its peers using the same method. Miller et al. (4) demonstrated that by using commutable materials and a bona fide reference method, it is possible to differentiate calibration bias from artifactual “matrix bias”. However, as it is commonly performed, conventional PT typically cannot differentiate between calibration bias and matrix bias; therefore, it cannot assess whether the results obtained are sufficiently accurate to meet clinical needs (5). In contrast, accuracy-based PT uses authentic, unaltered samples and target values determined by a reference method measurement procedure. Thus, it can assess the proficiency of a laboratory analysis using an analytical system as intended: the accuracy, and reliability of measurement results obtained with the instrument in the context of clinical needs. Because, for many reasons, accuracy-based PT is relatively expensive to perform, it has seen somewhat limited use by external quality assessment programs. New economical approaches using commutable samples are needed.
In 2022, the US Centers for Medicare & Medicaid Services (CMS) finalized changes to the Clinical Laboratory Improvement Amendments of 1988 (CLIA) regulations for proficiency testing PT, including an acceptance limit of ± 30% for scoring estradiol PT results (6). A more stringent evaluation criterion, of target ± 25% or 15 pg/mL (55.1 pmol/L) whichever is greater, was being used by the New York State Department of Health (NYSDOH) PT program at the time this study was done. This is consistent with the estimated “minimal” analytical performance(total error) (26%) based upon measurements of biological variation (BV) for estradiol (7, 8) and using an approach similar to that used by Miller et al. (4). For assessing the accuracy of a method or analytical system, the Centers for Disease Control and Prevention (CDC) Hormone Standardization Program for estradiol (CDC HoSt E2) uses performance criteria derived from epidemiological studies (9) of ±12.5% bias for samples with estradiol of >20 pg/mL (73.4 pmol/L), and ±2.5 pg/mL (9.2 pmol/L) absolute bias for estradiol <20 pg/mL (73.4 pmol/L); in this study all specimens exceeded the 20 pg/mL (73.4 pmol/L) threshold concentration.
Objectives of this study were to assess the accuracy of measurement procedures for total estradiol using an accuracy-based PT and to explore the effect of different possible acceptance limits using either the CDC-defined target or method peer-group mean.
LIMITATIONS
Our findings were limited to one event of five samples, so we cannot make conclusions about accuracy over time. A relatively small number of laboratories participated, but these are representative of the available estradiol assays in use. We were not able to test the five samples for possible endogenous or exogenous steroids or drugs that could have interfered with some assays.
CONCLUSIONS
In summary, we showed that at high (> 60 pg/mL or 220.3 pmol/L) estradiol concentrations in a majority of laboratories were able to meet minimal requirements for accuracy based upon BV. However, we concluded that the immunoassay measurements for estradiol are inaccurate at low concentrations. Applying desirable or optimal criteria would result in more misses and a different overall assessment. Our results are consistent with reports of inaccuracy and variability in estradiol measurements, which could impact patient care, particularly for samples with low concentrations (16, 20 - 21). Efforts to standardize estradiol measurements were made over two decades ago by multiple institutions (2, 20, 22 – 24). This snapshot assessment of the accuracy of estradiol measurements would benefit by following up with similar accuracy-based assessments to determine whether these efforts are continuing to increase accuracy. Only accuracy-based PT, like the approach we describe here, can assess absolute accuracy. Future efforts would be best focused on lower concentrations, where improvements are most needed.
Objectives
Accuracy of estradiol measurements is important but conventional proficiency testing (PT) cannot assess accuracy when possibly non-commutable samples are used and method peer-group means are the targets. Accuracy-based assessment of estradiol measurements is needed.
Design and Methods
Five serum samples were prepared from single- donors, frozen, and distributed overnight to 76 New York State Department of Health (NYSDOH)-certified laboratories. Participants analyzed samples for estradiol. The biases of group means were assessed against the Centers for Disease Control and Prevention (CDC)-defined targets, and evaluated using the Hormones Standardization Program (HoSt) E2 performance criteria of±12.5%. Each laboratory’s performance was evaluated using total allowable error (acceptance limits) of target ±25% or ±15 pg/mL (55.1 pmol/L) (whichever was greater, NYSDOH), target±30% (Clinical Laboratory Improvement Amendments [CLIA]), and target ±26% (minimal limit based on biological variation [BV]).
Results
The biases (range) were 34% (-17% to 175%), 40% (-33% to 386%), 16% (-45% to193%), 5% (-27% to 117%), and -4% (-31% to 21%), for samples at estradiol of 24.1, 28.4, 61.7,94.1 and 127 pg/mL, or 88.5, 104.3, 226.5, 245.4 and 466.2 pmol/L, respectively. Large positive method/analytical systematic biases were revealed for 9 commonly used method/analytical systems in the United States at low estradiol concentrations. Of the 9 analytical systems, 0, 0, 3,7, and 6 met the HoSt criterion for the samples with estradiol at the five respective concentrations. PT evaluation showed that 59%, 69%, and 87% of laboratories would receive a PT event pass score when the CDC-defined target and a criterion of NYSDOH, CLIA, or BV were used, respectively. However, >95% of laboratories would obtain PT pass scores if method peer group means were used as targets regardless of the criterion used.
Conclusions
Improvement in the accuracy of estradiol measurements is needed, particularly at low estradiol concentrations. Accuracy-based PT provides unambiguous information about the accuracy of analytical/method methods/analytical systems.
INTRODUCTION and OBJECTIVES
Estrogens are responsible for the development of the secondary female sex characteristics and play an important role in female reproductive processes. Estradiol measurements have a wide range of clinical utilities, e.g., diagnosis of fertility disorders, gynecomastia in males, estrogen-producing ovarian and testicular tumors, disorders of sex steroid metabolism, monitoring low-dose female hormone replacement therapy in postmenopausal women, and antiestrogen therapy(1, 2). To meet those clinical needs, accurate measurement of estradiol in patient care at all clinically relevant concentrations is needed; however, information on measurement accuracy is limited. Furthermore, its accurate measurement provides information that can be used to improve the quality of manufacturers’ products, assess the effectiveness of manufacturer standardization, and advance current evaluations performed as part of activities related to meeting regulatory requirements.
Although proficiency testing (PT) is an effective tool for monitoring the quality performance of clinical laboratories and analytical systems, it has limitations (3). Conventional PT often uses non-commutable samples or modified samples whose commutability is unknown, and therefore only evaluates participants’ results using method peer-group mean values as targets (4). Therefore, due to the presumed existence of matrix effects, conventional PT can only assess whether a laboratory’s analysis can meet acceptance limits relative to its peers using the same method. Miller et al. (4) demonstrated that by using commutable materials and a bona fide reference method, it is possible to differentiate calibration bias from artifactual “matrix bias”. However, as it is commonly performed, conventional PT typically cannot differentiate between calibration bias and matrix bias; therefore, it cannot assess whether the results obtained are sufficiently accurate to meet clinical needs (5). In contrast, accuracy-based PT uses authentic, unaltered samples and target values determined by a reference method measurement procedure. Thus, it can assess the proficiency of a laboratory analysis using an analytical system as intended: the accuracy, and reliability of measurement results obtained with the instrument in the context of clinical needs. Because, for many reasons, accuracy-based PT is relatively expensive to perform, it has seen somewhat limited use by external quality assessment programs. New economical approaches using commutable samples are needed.
In 2022, the US Centers for Medicare & Medicaid Services (CMS) finalized changes to the Clinical Laboratory Improvement Amendments of 1988 (CLIA) regulations for proficiency testing PT, including an acceptance limit of ± 30% for scoring estradiol PT results (6). A more stringent evaluation criterion, of target ± 25% or 15 pg/mL (55.1 pmol/L) whichever is greater, was being used by the New York State Department of Health (NYSDOH) PT program at the time this study was done. This is consistent with the estimated “minimal” analytical performance(total error) (26%) based upon measurements of biological variation (BV) for estradiol (7, 8) and using an approach similar to that used by Miller et al. (4). For assessing the accuracy of a method or analytical system, the Centers for Disease Control and Prevention (CDC) Hormone Standardization Program for estradiol (CDC HoSt E2) uses performance criteria derived from epidemiological studies (9) of ±12.5% bias for samples with estradiol of >20 pg/mL (73.4 pmol/L), and ±2.5 pg/mL (9.2 pmol/L) absolute bias for estradiol <20 pg/mL (73.4 pmol/L); in this study all specimens exceeded the 20 pg/mL (73.4 pmol/L) threshold concentration.
Objectives of this study were to assess the accuracy of measurement procedures for total estradiol using an accuracy-based PT and to explore the effect of different possible acceptance limits using either the CDC-defined target or method peer-group mean.
LIMITATIONS
Our findings were limited to one event of five samples, so we cannot make conclusions about accuracy over time. A relatively small number of laboratories participated, but these are representative of the available estradiol assays in use. We were not able to test the five samples for possible endogenous or exogenous steroids or drugs that could have interfered with some assays.
CONCLUSIONS
In summary, we showed that at high (> 60 pg/mL or 220.3 pmol/L) estradiol concentrations in a majority of laboratories were able to meet minimal requirements for accuracy based upon BV. However, we concluded that the immunoassay measurements for estradiol are inaccurate at low concentrations. Applying desirable or optimal criteria would result in more misses and a different overall assessment. Our results are consistent with reports of inaccuracy and variability in estradiol measurements, which could impact patient care, particularly for samples with low concentrations (16, 20 - 21). Efforts to standardize estradiol measurements were made over two decades ago by multiple institutions (2, 20, 22 – 24). This snapshot assessment of the accuracy of estradiol measurements would benefit by following up with similar accuracy-based assessments to determine whether these efforts are continuing to increase accuracy. Only accuracy-based PT, like the approach we describe here, can assess absolute accuracy. Future efforts would be best focused on lower concentrations, where improvements are most needed.
