Grant: Discovery Grants Program - Individual
McGill University
$12,000.00
Statistical methods for assessing diagnostic tests & estimating individualized probabilities of therapeutic benefit
FY 2008-09
Summary
I propose to develop statistical methods to (1) assess the performance of diagnostic tests and prognostic scores
and (2) estimate risk (cumulative incidence) functions and, from them, individualized 'what if' probabilities of
benefit if a specific medical or lifestyle intervention is selected.
In aim (1) the primary focus will be on the 'c' statistic derived from multiple logistic regression. The initial
biostatistical use of c was as the area under the roc curve (auc), to measure the discriminant ability of imaging
tests (interpreted on a rating scale) and laboratory tests (measured on an interval scale). However, the index is
also standard output from the SAS (multiple) LOGISTIC procedure and is increasingly used to assess the
ability of diagnostic and prognostic scoring systems derived from a vector of predictors. If calculated from the
same data from which the model is fitted, the statistic overestimates the true performance of the system.
Indeed, by construction, the c statistic calculated from PROC LOGISTIC cannot be less than the null 0.5.
Several authors (from Cornfield and Lachenbruch several decades ago, to Copas, Rockette, and Pinsky
recently) have studied the factors that determine the magnitude of this bias. I propose to develop a simple
'adjusted-c' statistic, similar to the adjusted-r-squared statistic. I expect that the needed attenuation/shrinkage
will be a function of the numbers of cases and non-cases, and the numbers of useful -- and useless -- candidate
predictor variables.
A secondary focus will be on simplified lower confidence bound for the true auc when - in the simple imaging
or laboratory test situations -- the observed auc is unity. Obuchowshi has provided limits for this situation, but
unfortunately they are too distribution-specific to be of general use. My plan is to draw on the simplicity and
closed-form formulae based on overlapping exponential distributions, and on the insights on the var(auc)
structure in our paper in Academic Radiology in 1997.
Aim (2) has two parts. The first is to develop guidelines for a novel way (developed with my colleague
Miettinen) to fit smooth-in-time hazard functions to survival-type data, where the event E=1 represents an
undesirable outcome. The purpose is to estimate cumulative incidence as a function of a vector of patient
characteristics and lifestyle/medical management options. The approach is based on sampling the
person-moments; the main unknown is the choice of the sampling approach that gives the most stable estimate
of the individualized cumulative incidence. The second aim is to derive an interval estimate for the probability
of benefit within a time horizon T, for an individual with a personal profile vector x, and contemplated
medical/behavioral Action (A=1) or not (A=0), i.e. the difference Prob[E=1 | x, T, A=0] - Prob[E=1 | x, T,
A=1]. The hope is to have the individualized interval be test-based, so that it can be calculated from the
information usually contained in study reports.
We focus on individualized risk differences as a response to the inordinate emphasis on the 'average' patient
and on hazard ratios rather than what matters to an individual: for individuals such as I, with profile x, what is
the difference in the probability of E over a time-horizon T if I choose one action over another? As an example,
consider the 2005 NEJM report on an RCT which documented the extent to which radical prostatectomy
reduces the risk of death from prostate cancer: the 'average' prostate cancer case-fatality rate within 10 years
was 15% for those randomized to watchful waiting and 10% for those randomized to surgery; the hazard ratio
was 0.55. The report contained no useful information for men with a patient / tumour profile (age at diagnosis,
Gleason score, pre-treatment PSA level) that was more/less favourable than the 'average' profile to which the
summary results presumably apply. With methods that are aimed at individualized risk, and that do not rely on
the non-smooth estimates obtained from Cox's proportional hazards model, we plan to change the
contemporary culture of statistical reporting to be more responsive to individual 'clients'.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
8738-2007
Name nom
Hanley, James
Department département
Epidemiology and Biostatistics
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2008
Competition year année de concours
2007
Award amount
12000
Program id
RGPIN
Program nane en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
14
Committee name en
Statistical Sciences
Committee name fr
Statistique
Area of application code
1201
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Life sciences (including biotechnology)
Area of application fr
Sciences de la vie (y compris biotechnologie)
Research subject code
3005
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Biostatistics
Research subject fr
Biostatistiques
Partie
2
Nb partie
5
Application title
Statistical methods for assessing diagnostic tests & estimating individualized probabilities of therapeutic benefit
Keyword
NA
Application summary
I propose to develop statistical methods to (1) assess the performance of diagnostic tests and prognostic scores
and (2) estimate risk (cumulative incidence) functions and, from them, individualized 'what if' probabilities of
benefit if a specific medical or lifestyle intervention is selected.
In aim (1) the primary focus will be on the 'c' statistic derived from multiple logistic regression. The initial
biostatistical use of c was as the area under the roc curve (auc), to measure the discriminant ability of imaging
tests (interpreted on a rating scale) and laboratory tests (measured on an interval scale). However, the index is
also standard output from the SAS (multiple) LOGISTIC procedure and is increasingly used to assess the
ability of diagnostic and prognostic scoring systems derived from a vector of predictors. If calculated from the
same data from which the model is fitted, the statistic overestimates the true performance of the system.
Indeed, by construction, the c statistic calculated from PROC LOGISTIC cannot be less than the null 0.5.
Several authors (from Cornfield and Lachenbruch several decades ago, to Copas, Rockette, and Pinsky
recently) have studied the factors that determine the magnitude of this bias. I propose to develop a simple
'adjusted-c' statistic, similar to the adjusted-r-squared statistic. I expect that the needed attenuation/shrinkage
will be a function of the numbers of cases and non-cases, and the numbers of useful -- and useless -- candidate
predictor variables.
A secondary focus will be on simplified lower confidence bound for the true auc when - in the simple imaging
or laboratory test situations -- the observed auc is unity. Obuchowshi has provided limits for this situation, but
unfortunately they are too distribution-specific to be of general use. My plan is to draw on the simplicity and
closed-form formulae based on overlapping exponential distributions, and on the insights on the var(auc)
structure in our paper in Academic Radiology in 1997.
Aim (2) has two parts. The first is to develop guidelines for a novel way (developed with my colleague
Miettinen) to fit smooth-in-time hazard functions to survival-type data, where the event E=1 represents an
undesirable outcome. The purpose is to estimate cumulative incidence as a function of a vector of patient
characteristics and lifestyle/medical management options. The approach is based on sampling the
person-moments; the main unknown is the choice of the sampling approach that gives the most stable estimate
of the individualized cumulative incidence. The second aim is to derive an interval estimate for the probability
of benefit within a time horizon T, for an individual with a personal profile vector x, and contemplated
medical/behavioral Action (A=1) or not (A=0), i.e. the difference Prob[E=1 | x, T, A=0] - Prob[E=1 | x, T,
A=1]. The hope is to have the individualized interval be test-based, so that it can be calculated from the
information usually contained in study reports.
We focus on individualized risk differences as a response to the inordinate emphasis on the 'average' patient
and on hazard ratios rather than what matters to an individual: for individuals such as I, with profile x, what is
the difference in the probability of E over a time-horizon T if I choose one action over another? As an example,
consider the 2005 NEJM report on an RCT which documented the extent to which radical prostatectomy
reduces the risk of death from prostate cancer: the 'average' prostate cancer case-fatality rate within 10 years
was 15% for those randomized to watchful waiting and 10% for those randomized to surgery; the hazard ratio
was 0.55. The report contained no useful information for men with a patient / tumour profile (age at diagnosis,
Gleason score, pre-treatment PSA level) that was more/less favourable than the 'average' profile to which the
summary results presumably apply. With methods that are aimed at individualized risk, and that do not rely on
the non-smooth estimates obtained from Cox's proportional hazards model, we plan to change the
contemporary culture of statistical reporting to be more responsive to individual 'clients'.