The standards for reporting diagnostic accuracy studies 2015 update: is there a missing link to the triumvirate?

The triumvirate of lab errors

The troika of pre-analytical, analytical and post-analytical testing phases in laboratory medicine is well known to all first year clinical laboratorians as the phases of lab testing that must be controlled in order to produce accurate, valid laboratory results for diagnostic purposes. The need to control these three testing phases is distinct for the evaluation of diagnostic accuracy of laboratory medical tests and its reporting compared to other medical specialties that utilize diagnostic testing. In fact, evidence has established that up to 70% of laboratory errors and failures occur in the pre-analytical phase (1-3), with the post-analytical testing phase is the second most error prone testing phase; by comparison, the analytical phase is relatively low.

The Standards for Reporting of Diagnostic Accuracy (STARD) were first promulgated as a checklist of 25 items in 2003 (4) that aimed to improve the reporting of studies that evaluate the diagnostic accuracy of medical tests. Although some improvement in reporting has been noted in the years since the initiation of STARD, the overall impact of this guidance on the field has been modest to moderate. An updated version of STARD has been developed and published (5) that is intended to (I) facilitate use of the checklist by rearranging and rephrasing items; (II) include new information based on improved understanding of sources of bias and variability and other issues in diagnostic accuracy studies; and (III) improve the consistency with other reporting guidelines such as CONSORT (6).

STARD guidance should be considered valuable, but it is important to note that this guidance was not designed specifically for laboratory application, and its scope does not address the testing phases that are most important susceptible to error in the laboratory medicine field. The following case illustrates this point.

Case presentation

A 50-year-old male presents to the emergency department (ED) with chest pain that radiated to the arms and shoulders. At presentation, a cardiac troponin I (cTnI) was ordered and a serum sample was sent to the central laboratory who produced a result of 0.060 µg/L (99^th percentile cutoff 0.04 µg/L). The electrocardiogram (ECG) showed no specific abnormality from previous tracings. Repeat cTnI testing on blood collected at 4 and 8 h produced a results 0.084 and 0.046 µg/L, respectively. The patient was diagnosed with a non-ST-segment elevation myocardial infarction (NSTEMI) and was admitted to the hospital. The next day, the attending cardiologist ordered a B-type natriuretic peptide (BNP) test from the sample collected at the patient’s presentation to the ED. The cardiologist had just attended a Grand Rounds presentation on the “2014 AHA/ACC Guideline for the Management of Patients With Non-ST-Elevation Acute Coronary Syndrome”, at which it was noted that BNP or amino terminal proBNP (NTproBNP) may be considered to assess risk in patients with suspected acute coronary syndromes (ACS) (Class IIb Recommendation, Level of Evidence B) (7). He recalled reading the original paper (8), finding that patients who have a BNP level >80 µg/L have increased incidence of major adverse cardiac events, whereas ACS patients with BNP <80 µg/L are at relatively low risk.

The clinical laboratory retrieved the sample that was tested for cTnI (collected in heparinized tubes) which had been stored at room temperature and it was tested for BNP. This lab was in the process of evaluating an NTproBNP assay which was intended as a replacement test for BNP. The reported BNP test added onto the EDTA presentation sample was 65 µg/L. Thinking the patient was at low risk for an immediate adverse event, this result led the cardiologist to manage the patient conservatively. It is noteworthy that the NTproBNP run for the lab’s correlation study versus the BNP had a substantially increased result of 2,000 µg/L (normal value: <125 µg/L).

Three weeks later the patient returned to the ED; he again complained of chest pain that radiated to his arms and shoulders. An ECG performed soon after presentation showed an ischemic pattern, and a cardiac cTnI level collected at presentation was 8.5 µg/L. The patient underwent acute cardiac catheterization and angiography procedure that revealed several lesions occluding >90% of his left anterior descending (LAD) coronary artery. These were determined to be the culprit lesions, and the patient had stents placed to restore flow in the LAD. It was determined that he had suffered a large myocardial infarction.

The updated STARD report

The STARD report was updated a dozen years after the original STARD report to include an additional five items onto the checklist (4). This new document provides further clarification of what is a good evidence-based study. The importance of producing quality laboratory studies can have a major impact on clinical laboratory testing is perceived by the public and government agencies, which could be a factor in the viability of the profession (9).

The important pre-analytical error pertinent to this case was the use of an inappropriate blood collection tube and prolonged sample storage conditions for the BNP measurement. When EDTA plasma was stored for 4 h at room temperature, the BNP level decreased significantly, as shown previously (10). Discordant results for BNP versus EDTA plasma have also been reported when heparinized plasma was used (11). In contrast, NTproBNP is stable under these conditions. Given the sub-optimum conditions by which this sample was handled in this case report, it is likely a falsely low BNP result was produced leading to incorrect risk assessment and conservative management by the treating physician.

Pre-analytical variables were not addressed in the original or the current amended STARD report. Physicians are the ultimate customer of the information that the clinical laboratory provides. Unfortunately, errors caused by pre-analytical variables are not appreciated by doctors as a whole. Even if attending physicians are aware of the importance of pre-analytical error, it is not possible for them to determine if such an error had occurred. There is an opportunity to highlight potential pre-analytical variables with each published report on diagnostic accuracy what authors have done to minimize these errors. Table 1 lists some important questions that could and should be asked of all future biomarker studies, and perhaps be included in the next version of STARD. It should be noted that the U.S. Food and Drug Administration (FDA) requires submission of data that document pre-analytical variables when applications are submitted for in vitro product clearance and approval. Reports on clinical diagnostic studies should be held to the same standard.

Table 1 Some pre-analytical variables to be addressed for studies assessing diagnostic accuracy
Full table

Acknowledgements

None.

Footnote

Provenance: This is a Guest Editorial commissioned by Editorial Board Member Prof. Giuseppe Lippi (Section of Clinical Biochemistry, University of Verona, Verona, Italy).

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Bonini P, Plebani M, Ceriotti F, et al. Errors in laboratory medicine. Clin Chem 2002;48:691-8. [PubMed]
2. Hammerling JA. A review of medical errors in laboratory diagnostics and where we are today. Lab Med 2012;43:41-4.
3. Lippi G, Chance JJ, Church S, et al. Preanalytical quality improvement: from dream to reality. Clin Chem Lab Med 2011;49:1113-26. [PubMed]
4. Bossuyt PM, Reitsma JB, Bruns DE, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem 2003;49:7-18. [PubMed]
5. Bossuyt PM, Reitsma JB, Bruns DE, et al. STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies. Radiology 2015;277:826-32. [PubMed]
6. Korevaar DA, Cohen JF. Reporting diagnostic accuracy studies: evaluation 10 years of STARD. Available online: http://www.equator-network.org/2015/03/31/10-years-of-stard/, accessed 12-Dec-2015
7. Amsterdam EA, Wenger NK, Brindis RG, et al. 2014 AHA/ACC guideline for the management of patients with non-ST-elevation acute coronary syndromes: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;130:e344-426. [PubMed]
8. Morrow DA, de Lemos JA, Sabatine MS, et al. Evaluation of B-type natriuretic peptide for risk assessment in unstable angina/non-ST-elevation myocardial infarction: B-type natriuretic peptide and prognosis in TACTICS-TIMI 18. J Am Coll Cardiol 2003;41:1264-72. [PubMed]
9. Lippi G, Plebani M. Improving accuracy of diagnostic studies in a world with limited resources: a road ahead. Ann Transl Med 2016;4:43.
10. Ordonez-Llanos J, Collinson PO, Christenson RH. Amino-terminal pro-B-type natriuretic peptide: analytic considerations. Am J Cardiol 2008;101:9-15. [PubMed]
11. Santos H, Cauliez B, Tron C, et al. Is heparin plasma suitable for the determination of B-type natriuretic peptide on the Beckman-Coulter Access 2? Clin Chem Lab Med 2010;48:399-401. [PubMed]