Pitfalls of statistical analysis and clinical interpretation of the estimates on the example of patients with chronic kidney disease. Part III: Evaluating the informativeness of biomarkers

Physicians do not often correctly interpret the informativeness measures of biomarkers provided in scientific papers. The most common estimates are sensitivity (Se) the probability that the biomarker is positive in a case of disease, specificity (Sp) the probability that the marker is negative for patients who do not have a disease, the positive predictive value of a test result (PPV) the probability of a disease in a marker-positive patient, a negative predictive value (NPV) the probability that a marker-negative patient does not have a disease. Does the very high Se, Sp, PPV or NPV, as well as the revealed a statistically significant association between marker and outcome, mean that the marker is effective? Not always so. The statistical significance of the marker-outcome association is only a necessary condition, but not a sufficient one. The practical use of the marker depends on its frequency of occurrence (PM) and the prevalence of the disease (PD) in the population under study. The fact that some conventional indices of informativeness can be high even in the absence of a real association between marker and outcome (defined in such a case only by PM and PD) can be misleading. This leads us to an important conclusion: conventional indices of biomarker informativeness (Se, Sp, PPV and NPV) must be supplemented by statistics that measure the strength of the relationship between marker and outcome (odds ratio or risk ratio), as well as integral measures of marker informativeness (screening or predictive balance accuracy - SBA or PBA). The quantity of empirically evaluated diagnostic performance measures is directly determined by the study design: the case-control study enables to measure directly Se, Sp, SBA, and OR, the cohort study allows to evaluate PPV, NPV, PBA, OR, and RR. The population-based study provides investigators with any performance indices of diagnostic tests. It also should be kept in mind that Se, Sp and SBA (AUC) characterize only the screening informativeness of the marker. PPV, NPV and PBA characterize only the predictive informativeness of the marker. OR and RR quantify the strength of the association between marker and outcome.

биомаркер, чувствительность, специфичность, прогностическая ценность положительного результата, прогностическая ценность отрицательного результата, распространенность заболевания, статистика, biomarker, sensitivity, specificity, positive predictive value, negative predictive value, disease prevalence, statistics

1. Kidney Disease: Improving Global Outcomes (KDIGO) Glomerulonephritis Work Group. KDIGO Clinical Practice Guideline for Glomerulonephritis. Kidney inter., Suppl. 2012; 2: 139-274. DOI: 10.1038/kisup.2012.9

2. Puhan MA, Steurer J, Bachmann LM, ter Riet G. A randomized trial of ways to describe test accuracy: the effect on physicians' post-test probability estimates. Ann Intern Med. 2005;143(3):184-189. DOI:10.7326/0003-4819-143-3-200508020-00004

3. Leeflang MM, Deeks JJ, Gatsonis C, Bossuyt PM; Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med. 2008;149(12):889-897. DOI:10.7326/0003-4819-149-12-200812160-00008

4. Perneger TV, Agoritsas T. Doctors and patients' susceptibility to framing bias: a randomized trial. J Gen Intern Med. 2011;26(12):1411-1417. Doi:10.1007/s11606-011-1810-x

5. Whiting PF, Davenport C, Jameson C, et al. How well do health professionals interpret diagnostic information? A systematic review. BMJ Open. 2015;5(7):e008155. DOI:10.1136/bmjopen-2015-008155

6. Parker VL, Ritchie JE, Drake TM, Hookham J, Balasubramanian SP. A Randomised Assessment of Trainee Doctors' Understanding and Interpretation of Diagnostic Test Results. World J Surg. 2016;40(1):21-28. DOI:10.1007/s00268-015-3214-2

7. Зулькарнаев А.Б. Подводные камни" статистического анализа и клинической интерпретации полученных оценок на примере пациентов с хронической болезнью почек. Часть I: оценка риска. Нефрология и диализ. 2019. 21(4): 419-429.

8. Alharbi FJ, Baig S, Rambhatla SB, et al. The clinical utility of total concentration of urinary globotriaosylsphingosine plus its analogues in the diagnosis of Fabry disease. Clin Chim Acta. 2020;500:120-127. DOI:10.1016/j.cca.2019.10.005

9. Zhang S, Wu Z, Li J, et al. Evaluation of the clinical relevance of anti-annexin-A5 antibodies in Chinese patients with antiphospholipid syndrome. Clin Rheumatol. 2017;36(2):407-412. DOI:10.1007/s10067-016-3510-8

10. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS international sepsis definitions conference. Crit Care Med 2003;31(4):1250-6. DOI: 10.1007/s00134-003-1662-x

11. Shapiro NI, Trzeciak S, Hollander JE, et al. The diagnostic accuracy of plasma neutrophil gelatinase-associated lipocalin in the prediction of acute kidney injury in emergency department patients with suspected sepsis. Ann Emerg Med. 2010;56(1):52-59.e1. DOI:10.1016/j.annemergmed.2010.02.010

12. Рубанович А.В., Хромов-Борисов Н.Н. Теоретический анализ показателей предсказательной эффективности бинарных генетических тестов. Экологическая генетика. 2013; 11(1): 77-90.

13. Holmer M, Martínez JP, Gil E, Sandberg F, Olde B, Sörnmo L. Detection of ventricular premature beats based on the pressure signals of a hemodialysis machine. Med Eng Phys. 2018;51:49-55. DOI:10.1016/j.medengphy.2017.11.004

14. Luo JC, Qiu XH, Pan C, et al. Increased cardiac index attenuates septic acute kidney injury: a prospective observational study. BMC Anesthesiol. 2015;15:22. DOI:10.1186/s12871-015-0005-0

15. Chang CH, Fu CM, Yang CH, et al. Society of Thoracic Surgeons score predicts kidney injury in patients not undergoing bypass surgery. Ann Thorac Surg. 2015;99(1):123-129. DOI:10.1016/j.athoracsur.2014.07.072

16. Kim S, Kim HJ, Ahn HS, et al. Is plasma neutrophil gelatinase-associated lipocalin a predictive biomarker for acute kidney injury in sepsis patients? A systematic review and meta-analysis. J Crit Care. 2016;33:213-223. DOI:10.1016/j.jcrc.2016.02.014

17. Leeflang MM, Deeks JJ, Rutjes AW, Reitsma JB, Bossuyt PM. Bivariate meta-analysis of predictive values of diagnostic tests can be an alternative to bivariate meta-analysis of sensitivity and specificity. J Clin Epidemiol. 2012;65(10):1088-1097. doi:10.1016/j.jclinepi.2012.03.006