wally - The Wally Calibration Plot for Risk Prediction Models
A prediction model is calibrated if, roughly, for any
percentage x we can expect that x subjects out of 100
experience the event among all subjects that have a predicted
risk of x%. A calibration plot provides a simple, yet useful,
way of assessing the calibration assumption. The Wally plot
consists of a sequence of usual calibration plots. Among the
plots contained within the sequence, one is the actual
calibration plot which has been obtained from the data and the
others are obtained from similar simulated data under the
calibration assumption. It provides the investigator with a
direct visual understanding of the shape and sampling
variability that are common under the calibration assumption.
The original calibration plot from the data is included
randomly among the simulated calibration plots, similarly to a
police lineup. If the original calibration plot is not easily
identified then the calibration assumption is not contradicted
by the data. The method handles the common situations in which
the data contain censored observations and occurrences of
competing events.
