Substantial discussion has emerged on measures to quantify the clinical usefulness of a prediction model. This issue was discussed at a symposium in Cleveland, Sept 29, 2008. A related paper in Epidemiology (Jan 2010) aimed to present a perspective on some traditional and modern performance measures (R code here). It was accompanied by a vivid discussion in an Editorial and response from the authors.
A key element for performance evaluation is a validation graph.Epidemiology 2010, testicular cancer example
Rev Esp Cardiol. 2011, Framingham data
Andrew Vickers (@VickersBiostats) maintains a rich source with information on Net Benefit estimation: software, examples, and interpretation. Marshall Brown <mdbrown@fredhutch.org> also developed an R implementation.
Many references are at the STRATOS TG6 site; some below for a selection.
Literature
Original proposal for DCA
Vickers AJ, Elkin EB.
Decision curve analysis: a novel method for evaluating prediction models.
Med Decis Making. 2006 Nov-Dec;26(6):565-74. doi: 10.1177/0272989X06295361
Net benefit as a concept in other papers
Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD.
Validity of prognostic models: when is a model clinically useful?
Semin Urol Oncol. 2002 May;20(2):96-107. doi: 10.1053/suro.2002.32521
Peirce CS
The numerical measure of the success of predictions
Science, 1884
Explanation and guidance
Vickers AJ, Van Calster B, Steyerberg EW.
Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests.
BMJ. 2016 Jan 25;352:i6. doi: 10.1136/bmj.i6
Vickers AJ, van Calster B, Steyerberg EW.
A simple, step-by-step guide to interpreting decision curve analysis.
Diagn Progn Res. 2019 Oct 4;3:18. doi: 10.1186/s41512-019-0064-7
Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, Roobol MJ, Steyerberg EW.
Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators.
Eur Urol. 2018 Dec;74(6):796-804. doi: 10.1016/j.eururo.2018.08.038
Multiple estimates of Net Benefit can be summarized in a meta-analysis
Wynants L, Riley RD, Timmerman D, Van Calster B.
Random-effects meta-analysis of the clinical utility of tests and prediction models.
Stat Med. 2018 May 30;37(12):2034-2052. doi: 10.1002/sim.7653.
Illustration
Steyerberg EW, Van Calster B, Pencina MJ.
[Performance measures for prediction models and markers: evaluation of predictions and classifications].
Rev Esp Cardiol. 2011 Sep;64(9):788-94. doi: 10.1016/j.recesp.2011.04.017