SlideShare una empresa de Scribd logo
1 de 36
Methods for adjusting survival
estimates in the presence of
treatment crossover: a simulation
study
Nicholas Latimer, University of Sheffield
Collaborators: Paul Lambert, Keith Abrams, Michael Crowther, Allan
Wailoo and James Morden
2



Contents

What is the treatment crossover problem
Potential solutions
Simulation study
Results
Conclusions
3



Treatment switching – the problem
 In RCTs often patients are allowed to switch from the control
 treatment to the new intervention after a certain timepoint (eg
 disease progression)

 OS estimates will be confounded

 OS is a key input into the QALY calculation

 Cost effectiveness results will be inaccurate  an ITT analysis is
 likely to underestimate the treatment benefit

 Inconsistent and inappropriate treatment recommendations
 could be made
4



Treatment switching – the problem
 Control Treatment
                                              True OS difference
         PFS                   PPS
 Intervention

                PFS                            PPS

 Control  Intervention                                  RCT OS
                                                        difference
         PFS                         PPS

                           Survival time
Switching is likely to result in an underestimate of the treatment effect
What is usually done to adjust?
 No clear consensus

 Numerous „naive‟ approaches have been taken in NICE appraisals,
 eg:
                                                        Very prone to
    Take no action at all                               selection bias
    Exclude or censor all patients who crossover        – crossover
                                                        isn‟t random
 Occasionally more complex statistical methods have been used, eg:
    Rank Preserving Structural Failure Time Models (RPSFTM)
    Inverse Probability of Censoring Weights (IPCW)

 And others are available from the literature, eg:
    Structural Nested Models (SNM)
6



What are the consequences?
 TA 215, Pazopanib for RCC [51% of control switched]

    ITT:      OS HR (vs IFN) = 1.26  ICER = Dominated
    Censor patients:        HR = 0.80  ICER = £71,648
    Exclude patients:       HR = 0.48  ICER = £26,293
    IPCW:                   HR = 0.80  ICER = £72,274
    RPSFTM:                 HR = 0.63  ICER = £38,925
Potential solutions
RPSFTM (and IPE algorithm)
   Randomisation-based approach, estimates counterfactual survival times
Key assumption: common treatment effect

IPCW
   Observational-based approach, censors xo patients, weights remaining patients
Key assumptions: “no unmeasured confounders”; must model OS and crossover

SNM
   Observational version of RPSFTM
Key assumptions: “no unmeasured confounders”; must model OS and crossover
Simulation study (1)
   None of these methods are perfect
   But we need to know which are likely to produce least bias in different scenarios


Simulation study
   Simulate survival data for two treatment groups, applying crossover that is linked to
   patient characteristics/prognosis
   In some scenarios simulate a treatment effect that changes over time
   In some scenarios simulate a treatment effect that remains constant over time
   Test different %s of crossover, and different treatment effect sizes


How does the bias and coverage associated with each method compare?
9



 Simulation study (2)
Methods assessed
  Naive methods
     ITT
     Exclude crossover patients
     Censor crossover patients
     Treatment as a time-dependent covariate
  Complex methods
     RPSFTM
     IPE algorithm
     IPCW
     SNM
10
     Results: common treatment effect
                         RPSFTM / IPE worked very well

                         IPCW and SNM performed ok when crossover % was lower

                    50.00               ITT        IPCW        RPSFTM    IPE        SNM
                    40.00
     AUC mean bias (%)




                    30.00
                    20.00
                    10.00
                 0.00
19               20                27         28          31        32         39         40   43   44   51
               -10.00
                         IPCW and SNM performed poorly when crossover % was very high

                         Naive methods performed poorly (generally led to higher bias than ITT)
11
 Results: effect 15%                                                      in xo patients
                         RPSFTM / IPE produced higher bias than previous scenarios

                         IPCW and SNM performed similarly to RPSFTM / IPE providing crossover < 90%

                                     ITT   IPCW     RPSFTM      IPE   SNM
                    45.00
     AUC mean bias (%)




                    35.00
                    25.00
                    15.00
                         5.00
                -5.00
17               18                25       26       29        30       37       38         41   42   49
               -15.00

                         IPCW and SNM performed poorly when crossover % was very high

                         Bias not always lower than that associated with the ITT analysis
12
 Results: effect 25%                                                             in xo patients
                         RPSFTM / IPE produced even more bias

                         IPCW and SNM produce less bias than RPSFTM / IPE providing crossover < 90%

                  50.00               ITT    IPCW      RPSFTM       IPE     SNM
     AUC mean bias (%)




                  40.00
                  30.00
                  20.00
                  10.00
               0.00
23             24                   33        34        35        36        45        46        47      48    57
             -10.00
             -20.00

                         No „good‟ options when crossover % is very high

                         Often ITT analysis likely to result in least bias (esp. when trt effect low)
13
Conclusions
  Treatment crossover is an important issue that has come to the fore in HE arena

  Current methods for dealing with treatment crossover are imperfect

  Our study offers evidence on bias in different scenarios (subject to limitations)

  RPSFTM / IPE produce low bias when treatment effect is common
   But are very sensitive to this

  IPCW / SNM are not affected by changes in treatment effect between groups, but in
  (relatively) small trial datasets observational methods are volatile
   Especially when crossover % is very high (leaving low n in control group)

  Very important to assess trial data, crossover mechanism, treatment effect to
  determine which method likely to be most appropriate

 Don’t just pick one!!
14




Back-up Slides




28/09/2012 © The University of Sheffield
15
                                Data Generation (1)
   Used a two-stage Weibull model to generate underlying survival times
   and a time-dependent covariate (called „CEA‟)
   Longitudinal model for CEA (for ith patient at time t):


where
            is the random intercept
            is the slope for a patient in the control arm
                  is the slope for a patient in the treatment arm (all
            is the change in the intercept for a patient with bad prognosis
            compared to a patient without bad prognosis
   Picked parameter values such that CEA increased over time, more
   slowly in the experimental group, and was higher in the badprog group
 28/09/2012 © The University of Sheffield
16


 Data Generation (2)
   The survival hazard function was based upon a Weibull (see Bender
   et al 2005):                1
                              h(t )   t   exp( X )
   In our case,
  X = 1 * trt i + ( * log(t) ) * trt i +  2 badprogi +  (cea(t) )
where  1 is the log hazard ratio (the baseline treatment effect)
        is the time-dependent change in the treatment effect
        2 is the impact of a bad prognosis baseline covariate on
           survival
       is the coefficient of CEA, indicating its effect on survival
  We used this to generate our survival times
 So, CEA has an effect on survival over time, and there is a separate
  time-dependent treatment effect
 28/09/2012 © The University of Sheffield
17



Data Generation (3)
  We applied a reduced treatment effect to crossover patients – this was
  equivalent to the baseline treatment effect multiplied by a factor to
  ensure that this effect was less than (or equal to) the average effect
  received in the experimental group




28/09/2012 © The University of Sheffield
18



       Data Generation (4)
              We then selected parameter values in order that „realistic‟ datasets
              were created:
       1.00




                                                                                                2.5
                                                                                                                                          AF actual
       0.80




                                                                                                                                          AF predicted
                                        trtrand = 0         trtrand = 1
                                                                                                 2
       0.60




                                                                          Acceleration Factor
                                                                                                1.5
       0.40
       0.20




                                                                                                 1
       0.00




                                                                                                0.5
                0       200       400       600       800   1000          1200
                                        analysis time
Number at risk
                                                                                                 0
   trtrand = 0 251      154       86         52       25      9                          0
                                                                                                      0   200   400      600        800   1000             1200
   trtrand = 1 249      186       126        71       42     21                          0
                                                                                                                      Time (days)

         28/09/2012 © The University of Sheffield
19
                                 Data Generation (5)
We made several assumptions about the „crossover mechanism‟:
    1. Crossover could only occur after disease progression (disease
       progression was approximately half of OS, calculated for each patient
       using a beta(5,5) distribution)
    2. Crossover could only occur at 3 „consultations‟ following disease
       progression
                These were set at 21 day intervals
                Probability of crossover highest at initial consultation, then falls in second
                and third
    3. Crossover probability depended on time-dependent covariates:
                CEA value at progression (high value reduced chance of crossover)
                Time to disease progression (high value increased chance of crossover)
                This was altered in scenarios to test a simpler mechanism where probability
                only depended on CEA
 Given all this, CEA was a time-dependent confounder
  28/09/2012 © The University of Sheffield
20
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease        
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect          
Time-dependence of treatment effect Treatment effect received is equal in all crossover   
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions    
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different    
                                    groups become more likely to cross over


      28/09/2012 © The University of Sheffield
21
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease            (2)
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect          
Time-dependence of treatment effect Treatment effect received is equal in all crossover   
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions    
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different    
                                    groups become more likely to cross over


      28/09/2012 © The University of Sheffield
22
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease            (2)
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect              (4)
Time-dependence of treatment effect Treatment effect received is equal in all crossover   
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions    
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different    
                                    groups become more likely to cross over


      28/09/2012 © The University of Sheffield
23
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease            (2)
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect              (4)
Time-dependence of treatment effect Treatment effect received is equal in all crossover       (8)
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in            (12)
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions    
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different    
                                    groups become more likely to cross over


      28/09/2012 © The University of Sheffield
24
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease            (2)
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect              (4)
Time-dependence of treatment effect Treatment effect received is equal in all crossover       (8)
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in            (12)
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions        (24)
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different    
                                    groups become more likely to cross over


      28/09/2012 © The University of Sheffield
25
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease            (2)
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect              (4)
Time-dependence of treatment effect Treatment effect received is equal in all crossover       (8)
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in            (12)
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions        (24)
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different        (72)
                                    groups become more likely to cross over
   This combined to 72 scenarios
      28/09/2012 © The University of Sheffield
26
                                     Scenarios
Variable                                         Value                                    Alternative
Sample size                                      500                                      
Number of prognosis groups (prog)                2                                        
Probability of good prognosis                    0.5                                      
Probability of poor prognosis                    0.5                                      
Maximum follow-up time                           3 years (1095 days)                      
Multiplication of OS survival time due           Log hazard ratio = 0.5                   
to bad prognosis group
Survival time distribution          Alter parameters to test two levels of disease        
                                    severity
Initially assumed treatment effect  Alter to test two levels of treatment effect          
Time-dependence of treatment effect Treatment effect received is equal in all crossover   
                                    patients, and equals baseline treatment effect
                                    multplied by a factor. However set α to zero in
                                    some scenarios. Also include additional treatment
                                    effect decrement in crossover patients in some
                                    scenarios
Probability of switching treatment  Test two levels of treatment crossover proportions    
over time
Prognosis of crossover patients     Test three crossover mechanisms in which different    
                                    groups become more likely to cross over


      28/09/2012 © The University of Sheffield
27



 Estimating AUC
1. ‘Survivor function’ approach
   Apply treatment effect to survivor function (or hazard function)
   estimated for experimental group  calculate AUC
2. ‘Extrapolation’ approach
   Extrapolate counterfactual dataset to required time-point (only
   relevant for RPSFTM/IPE approaches)  calculate AUC
3. ‘Shrinkage’ approach
   Use estimated acceleration factor to „shrink‟ survival times in
   crossover patients in order to obtain an adjusted dataset  calculate
   AUC (only relevent for AF-based approaches)

  28/09/2012 © The University of Sheffield
A topical analogy...                        28




     England                    Spain
                    Vs
  (control group)        (intervention group)




     Kick-off...
A topical analogy...                      29




Halftime: England 0 – 3 Spain
 Spain are statistically significantly
better than England

 For ethical reasons, the
Spanish team are cloned and the
English team are sent home. So
the second half is Spain Vs
Spain
A topical analogy...                               30




Full time: England 2 – 5 Spain
 Spain still win, but not as comfortably as
they would have
Switching is likely to result in an underestimate of
the true supremacy of the intervention
31



Results (4)
While the SNM and IPCW methods appeared to produce similar levels of bias, the
SNM approach was more volatile:

           Relatively often failed to converge (in up to 90% of sims in one scenario)
            Typically when disease severity was high and treatment effect was low
           Only produced lower bias than the ITT analysis in 38% of scenarios
           (compared to 60% for the IPCW method)

           Was more sensitive to increasing crossover %




28/09/2012 © The University of Sheffield
32



 Results (5)
Relationship between bias and treatment crossover %
               140

               120

               100

               80
 Mean % bias




               60

               40
                                                                                ITT
               20

                 0
                     60%         70%            80%             90%      100%
               -20

               -40
                               Crossover proportion (at-risk patients)
 28/09/2012 © The University of Sheffield
33



 Results (5)
Relationship between bias and treatment crossover %
               140

               120

               100

               80
 Mean % bias




               60                                                                  IPE
                                                                                   RPSFTM
               40
                                                                                   ITT
               20

                 0
                     60%      70%                 80%              90%      100%
               -20

               -40
 28/09/2012 © The University ofCrossover
                                Sheffield   proportion (at-risk patients)
34



 Results (5)
Relationship between bias and treatment crossover %
               140

               120

               100

               80
 Mean % bias




                                                                                   IPCW
               60
                                                                                   IPE
               40                                                                  RPSFTM
                                                                                   ITT
               20

                 0
                     60%      70%                 80%              90%      100%
               -20

               -40
 28/09/2012 © The University ofCrossover
                                Sheffield   proportion (at-risk patients)
35



 Results (5)
Relationship between bias and treatment crossover %
               140

               120

               100

               80
                                                                                   IPCW
 Mean % bias




               60                                                                  IPE
                                                                                   RPSFTM
               40
                                                                                   SNM
               20                                                                  ITT

                 0
                     60%      70%                 80%              90%      100%
               -20

               -40
 28/09/2012 © The University ofCrossover
                                Sheffield   proportion (at-risk patients)
36
Conclusions
      Treatment crossover is an important issue that has come to the fore in HE arena

      Current methods for dealing with treatment crossover are imperfect

      Our study offers evidence on bias in different scenarios (subject to limitations)

 1.    RPSFTM / IPE produce low bias when treatment effect is common

 2.    When treatment effect ≈ 15% lower in crossover patients RPSFTM / IPE and IPCW /
       SNM methods produce similar levels of bias (5-10%)
       [provided suitable data available for obs methods and <90% crossover]

 3.    When treatment effect ≈ 25% lower IPCW/SNM produce less bias than RPSFTM/IPE
       [provided suitable data available for obs methods and <90% crossover]
       [and significant bias likely to remain  ITT analysis may offer least bias]

 4.Very important to assess trial data, crossover mechanism, treatment effect to
   determine which method likely to be most appropriate
  Don’t just pick one!!

Más contenido relacionado

Similar a Economic evaluation. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study.

Innovations in coagulation testing
Innovations in coagulation testingInnovations in coagulation testing
Innovations in coagulation testing
derosaMSKCC
 
MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...
MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...
MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...
European School of Oncology
 
Acute Stroke Management Handouts Power Point885
Acute Stroke Management Handouts   Power Point885Acute Stroke Management Handouts   Power Point885
Acute Stroke Management Handouts Power Point885
MedicineAndHealthNeurolog
 
Eugm 2011 pocock - dm cs-and-adaptive-trials
Eugm 2011   pocock - dm cs-and-adaptive-trialsEugm 2011   pocock - dm cs-and-adaptive-trials
Eugm 2011 pocock - dm cs-and-adaptive-trials
Cytel USA
 
Clinical trial bms clinical trials methodology 17012018
Clinical trial bms   clinical trials methodology 17012018Clinical trial bms   clinical trials methodology 17012018
Clinical trial bms clinical trials methodology 17012018
SoM
 
Extending A Trial’s Design Case Studies Of Dealing With Study Design Issues
Extending A Trial’s Design Case Studies Of Dealing With Study Design IssuesExtending A Trial’s Design Case Studies Of Dealing With Study Design Issues
Extending A Trial’s Design Case Studies Of Dealing With Study Design Issues
nQuery
 
Post Mastectomy Radiotherapy
Post Mastectomy RadiotherapyPost Mastectomy Radiotherapy
Post Mastectomy Radiotherapy
fondas vakalis
 

Similar a Economic evaluation. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study. (20)

Random and systematic errors 25.10.12
Random and systematic errors 25.10.12Random and systematic errors 25.10.12
Random and systematic errors 25.10.12
 
Dudas
DudasDudas
Dudas
 
QC test
QC testQC test
QC test
 
Innovations in coagulation testing
Innovations in coagulation testingInnovations in coagulation testing
Innovations in coagulation testing
 
MI-TLIF - effect of Age, BMI & Comorbidity - 2012-10-06 - SA Neurosurgery Con...
MI-TLIF - effect of Age, BMI & Comorbidity - 2012-10-06 - SA Neurosurgery Con...MI-TLIF - effect of Age, BMI & Comorbidity - 2012-10-06 - SA Neurosurgery Con...
MI-TLIF - effect of Age, BMI & Comorbidity - 2012-10-06 - SA Neurosurgery Con...
 
Important Trials of the Day & Basics of Biostatistics | IACTS SCORE 2020
Important Trials of the Day & Basics of Biostatistics | IACTS SCORE 2020Important Trials of the Day & Basics of Biostatistics | IACTS SCORE 2020
Important Trials of the Day & Basics of Biostatistics | IACTS SCORE 2020
 
Intervencionismo en Cardiopatía Isquémica
Intervencionismo en Cardiopatía IsquémicaIntervencionismo en Cardiopatía Isquémica
Intervencionismo en Cardiopatía Isquémica
 
MI-TLIF - effect of age, BMI& comorbidity - 2013-04-06 - global spine congres...
MI-TLIF - effect of age, BMI& comorbidity - 2013-04-06 - global spine congres...MI-TLIF - effect of age, BMI& comorbidity - 2013-04-06 - global spine congres...
MI-TLIF - effect of age, BMI& comorbidity - 2013-04-06 - global spine congres...
 
MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...
MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...
MCO 2011 - Slide 6 - M. Ghielmini - Spotlight session - Haematological diseas...
 
An exploratory analysis of the crash 2 rct
An exploratory analysis of the crash 2 rctAn exploratory analysis of the crash 2 rct
An exploratory analysis of the crash 2 rct
 
Saturday 1050 – tsuchikane – try to stay intimal
Saturday 1050 – tsuchikane – try to stay intimalSaturday 1050 – tsuchikane – try to stay intimal
Saturday 1050 – tsuchikane – try to stay intimal
 
Acute Stroke Management Handouts Power Point885
Acute Stroke Management Handouts   Power Point885Acute Stroke Management Handouts   Power Point885
Acute Stroke Management Handouts Power Point885
 
Eugm 2011 pocock - dm cs-and-adaptive-trials
Eugm 2011   pocock - dm cs-and-adaptive-trialsEugm 2011   pocock - dm cs-and-adaptive-trials
Eugm 2011 pocock - dm cs-and-adaptive-trials
 
Tavi
TaviTavi
Tavi
 
Adjusting for treatment switching in randomised controlled trials
Adjusting for treatment switching in randomised controlled trialsAdjusting for treatment switching in randomised controlled trials
Adjusting for treatment switching in randomised controlled trials
 
Clinical trial bms clinical trials methodology 17012018
Clinical trial bms   clinical trials methodology 17012018Clinical trial bms   clinical trials methodology 17012018
Clinical trial bms clinical trials methodology 17012018
 
Extending A Trial’s Design Case Studies Of Dealing With Study Design Issues
Extending A Trial’s Design Case Studies Of Dealing With Study Design IssuesExtending A Trial’s Design Case Studies Of Dealing With Study Design Issues
Extending A Trial’s Design Case Studies Of Dealing With Study Design Issues
 
2006 terni, workshop interattivo. la terapia ablativa percutanea della fibril...
2006 terni, workshop interattivo. la terapia ablativa percutanea della fibril...2006 terni, workshop interattivo. la terapia ablativa percutanea della fibril...
2006 terni, workshop interattivo. la terapia ablativa percutanea della fibril...
 
Post Mastectomy Radiotherapy
Post Mastectomy RadiotherapyPost Mastectomy Radiotherapy
Post Mastectomy Radiotherapy
 
Nuts & Bolts of Systematic Reviews, Meta-analyses & Network Meta-analyses
Nuts & Bolts of Systematic Reviews, Meta-analyses & Network Meta-analysesNuts & Bolts of Systematic Reviews, Meta-analyses & Network Meta-analyses
Nuts & Bolts of Systematic Reviews, Meta-analyses & Network Meta-analyses
 

Más de HTAi Bilbao 2012

Assessing the International Use
Assessing the International Use Assessing the International Use
Assessing the International Use
HTAi Bilbao 2012
 

Más de HTAi Bilbao 2012 (20)

How Glaucoma Patients Assess Different Aspects of Their Treatment? An Elicit...
How Glaucoma Patients Assess Different Aspects of Their Treatment?  An Elicit...How Glaucoma Patients Assess Different Aspects of Their Treatment?  An Elicit...
How Glaucoma Patients Assess Different Aspects of Their Treatment? An Elicit...
 
Transitional Care for Pediatric Patients with Neuromuscular Diseases: A Healt...
Transitional Care for Pediatric Patients with Neuromuscular Diseases: A Healt...Transitional Care for Pediatric Patients with Neuromuscular Diseases: A Healt...
Transitional Care for Pediatric Patients with Neuromuscular Diseases: A Healt...
 
Health Technology Assessment (HTA) Report: Interventions to increase particip...
Health Technology Assessment (HTA) Report: Interventions to increase particip...Health Technology Assessment (HTA) Report: Interventions to increase particip...
Health Technology Assessment (HTA) Report: Interventions to increase particip...
 
The use of ‘colloquial evidence’ in HTA: the experience of NICE
The use of ‘colloquial evidence’ in HTA:  the experience of NICE The use of ‘colloquial evidence’ in HTA:  the experience of NICE
The use of ‘colloquial evidence’ in HTA: the experience of NICE
 
Social values international programme: integrating research and policy to ens...
Social values international programme: integrating research and policy to ens...Social values international programme: integrating research and policy to ens...
Social values international programme: integrating research and policy to ens...
 
Challenges in commissioning research on what works in integrated care
Challenges in commissioning research on what works in integrated careChallenges in commissioning research on what works in integrated care
Challenges in commissioning research on what works in integrated care
 
Building a portfolio of research findings for use by healthcare managers and ...
Building a portfolio of research findings for use by healthcare managers and ...Building a portfolio of research findings for use by healthcare managers and ...
Building a portfolio of research findings for use by healthcare managers and ...
 
Assessing the International Use
Assessing the International Use Assessing the International Use
Assessing the International Use
 
EVALUATION OF PSYCHOSOCIAL FACTORS INFLUENCING HEALTHCARE PROFESSIONAL ACCEPT...
EVALUATION OF PSYCHOSOCIAL FACTORS INFLUENCING HEALTHCARE PROFESSIONAL ACCEPT...EVALUATION OF PSYCHOSOCIAL FACTORS INFLUENCING HEALTHCARE PROFESSIONAL ACCEPT...
EVALUATION OF PSYCHOSOCIAL FACTORS INFLUENCING HEALTHCARE PROFESSIONAL ACCEPT...
 
On the outside looking in.
On the outside looking in. On the outside looking in.
On the outside looking in.
 
METHODS, MATHEMATICAL MODELS, DATA QUALITY ASSESSMENT AND RESULT INTERPRETATI...
METHODS, MATHEMATICAL MODELS, DATA QUALITY ASSESSMENT AND RESULT INTERPRETATI...METHODS, MATHEMATICAL MODELS, DATA QUALITY ASSESSMENT AND RESULT INTERPRETATI...
METHODS, MATHEMATICAL MODELS, DATA QUALITY ASSESSMENT AND RESULT INTERPRETATI...
 
How to promote the prescription of evidence-based non-pharmacological treatme...
How to promote the prescription of evidence-based non-pharmacological treatme...How to promote the prescription of evidence-based non-pharmacological treatme...
How to promote the prescription of evidence-based non-pharmacological treatme...
 
The eunethta planned and ongoing projects database
The eunethta planned and ongoing projects databaseThe eunethta planned and ongoing projects database
The eunethta planned and ongoing projects database
 
Evaluation of an e health intervention for cancer patients' support
Evaluation of an e health intervention for cancer patients' supportEvaluation of an e health intervention for cancer patients' support
Evaluation of an e health intervention for cancer patients' support
 
Classification of oecd countries reimbursement systems at the individual tech...
Classification of oecd countries reimbursement systems at the individual tech...Classification of oecd countries reimbursement systems at the individual tech...
Classification of oecd countries reimbursement systems at the individual tech...
 
Bevacizumab for neovascular age related macular degeneration
Bevacizumab for neovascular age related macular degenerationBevacizumab for neovascular age related macular degeneration
Bevacizumab for neovascular age related macular degeneration
 
Azienda ospedaliero Universitaria of Udine
Azienda ospedaliero Universitaria of UdineAzienda ospedaliero Universitaria of Udine
Azienda ospedaliero Universitaria of Udine
 
HTAi in hospitals
HTAi in hospitalsHTAi in hospitals
HTAi in hospitals
 
The application of Health Technology Assessment in the field of biologics: an...
The application of Health Technology Assessment in the field of biologics: an...The application of Health Technology Assessment in the field of biologics: an...
The application of Health Technology Assessment in the field of biologics: an...
 
Hospital-based HTA: does it impact on medical technologies’ expenditure and c...
Hospital-based HTA: does it impact on medical technologies’ expenditure and c...Hospital-based HTA: does it impact on medical technologies’ expenditure and c...
Hospital-based HTA: does it impact on medical technologies’ expenditure and c...
 

Último

Último (20)

Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...
Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...
Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...
 
Call Girls Kurnool Just Call 8250077686 Top Class Call Girl Service Available
Call Girls Kurnool Just Call 8250077686 Top Class Call Girl Service AvailableCall Girls Kurnool Just Call 8250077686 Top Class Call Girl Service Available
Call Girls Kurnool Just Call 8250077686 Top Class Call Girl Service Available
 
Call Girls Gwalior Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Gwalior Just Call 8617370543 Top Class Call Girl Service AvailableCall Girls Gwalior Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Gwalior Just Call 8617370543 Top Class Call Girl Service Available
 
Call Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service AvailableCall Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service Available
 
Call Girls Vadodara Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Vadodara Just Call 8617370543 Top Class Call Girl Service AvailableCall Girls Vadodara Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Vadodara Just Call 8617370543 Top Class Call Girl Service Available
 
Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...
Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...
Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...
 
Model Call Girls In Chennai WhatsApp Booking 7427069034 call girl service 24 ...
Model Call Girls In Chennai WhatsApp Booking 7427069034 call girl service 24 ...Model Call Girls In Chennai WhatsApp Booking 7427069034 call girl service 24 ...
Model Call Girls In Chennai WhatsApp Booking 7427069034 call girl service 24 ...
 
Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...
Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...
Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...
 
Premium Call Girls In Jaipur {8445551418} ❤️VVIP SEEMA Call Girl in Jaipur Ra...
Premium Call Girls In Jaipur {8445551418} ❤️VVIP SEEMA Call Girl in Jaipur Ra...Premium Call Girls In Jaipur {8445551418} ❤️VVIP SEEMA Call Girl in Jaipur Ra...
Premium Call Girls In Jaipur {8445551418} ❤️VVIP SEEMA Call Girl in Jaipur Ra...
 
Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any TimeTop Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
 
Manyata Tech Park ( Call Girls ) Bangalore ✔ 6297143586 ✔ Hot Model With Sexy...
Manyata Tech Park ( Call Girls ) Bangalore ✔ 6297143586 ✔ Hot Model With Sexy...Manyata Tech Park ( Call Girls ) Bangalore ✔ 6297143586 ✔ Hot Model With Sexy...
Manyata Tech Park ( Call Girls ) Bangalore ✔ 6297143586 ✔ Hot Model With Sexy...
 
Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...
Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...
Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...
 
Top Rated Bangalore Call Girls Richmond Circle ⟟ 9332606886 ⟟ Call Me For Ge...
Top Rated Bangalore Call Girls Richmond Circle ⟟  9332606886 ⟟ Call Me For Ge...Top Rated Bangalore Call Girls Richmond Circle ⟟  9332606886 ⟟ Call Me For Ge...
Top Rated Bangalore Call Girls Richmond Circle ⟟ 9332606886 ⟟ Call Me For Ge...
 
Top Rated Hyderabad Call Girls Chintal ⟟ 9332606886 ⟟ Call Me For Genuine Se...
Top Rated  Hyderabad Call Girls Chintal ⟟ 9332606886 ⟟ Call Me For Genuine Se...Top Rated  Hyderabad Call Girls Chintal ⟟ 9332606886 ⟟ Call Me For Genuine Se...
Top Rated Hyderabad Call Girls Chintal ⟟ 9332606886 ⟟ Call Me For Genuine Se...
 
Call Girls Service Jaipur {8445551418} ❤️VVIP BHAWNA Call Girl in Jaipur Raja...
Call Girls Service Jaipur {8445551418} ❤️VVIP BHAWNA Call Girl in Jaipur Raja...Call Girls Service Jaipur {8445551418} ❤️VVIP BHAWNA Call Girl in Jaipur Raja...
Call Girls Service Jaipur {8445551418} ❤️VVIP BHAWNA Call Girl in Jaipur Raja...
 
Call Girls Hosur Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Hosur Just Call 9630942363 Top Class Call Girl Service AvailableCall Girls Hosur Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Hosur Just Call 9630942363 Top Class Call Girl Service Available
 
Call Girls Raipur Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Raipur Just Call 9630942363 Top Class Call Girl Service AvailableCall Girls Raipur Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Raipur Just Call 9630942363 Top Class Call Girl Service Available
 
Call Girls Service Jaipur {9521753030} ❤️VVIP RIDDHI Call Girl in Jaipur Raja...
Call Girls Service Jaipur {9521753030} ❤️VVIP RIDDHI Call Girl in Jaipur Raja...Call Girls Service Jaipur {9521753030} ❤️VVIP RIDDHI Call Girl in Jaipur Raja...
Call Girls Service Jaipur {9521753030} ❤️VVIP RIDDHI Call Girl in Jaipur Raja...
 
Pondicherry Call Girls Book Now 9630942363 Top Class Pondicherry Escort Servi...
Pondicherry Call Girls Book Now 9630942363 Top Class Pondicherry Escort Servi...Pondicherry Call Girls Book Now 9630942363 Top Class Pondicherry Escort Servi...
Pondicherry Call Girls Book Now 9630942363 Top Class Pondicherry Escort Servi...
 
Call Girls Rishikesh Just Call 8250077686 Top Class Call Girl Service Available
Call Girls Rishikesh Just Call 8250077686 Top Class Call Girl Service AvailableCall Girls Rishikesh Just Call 8250077686 Top Class Call Girl Service Available
Call Girls Rishikesh Just Call 8250077686 Top Class Call Girl Service Available
 

Economic evaluation. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study.

  • 1. Methods for adjusting survival estimates in the presence of treatment crossover: a simulation study Nicholas Latimer, University of Sheffield Collaborators: Paul Lambert, Keith Abrams, Michael Crowther, Allan Wailoo and James Morden
  • 2. 2 Contents What is the treatment crossover problem Potential solutions Simulation study Results Conclusions
  • 3. 3 Treatment switching – the problem In RCTs often patients are allowed to switch from the control treatment to the new intervention after a certain timepoint (eg disease progression) OS estimates will be confounded OS is a key input into the QALY calculation Cost effectiveness results will be inaccurate  an ITT analysis is likely to underestimate the treatment benefit Inconsistent and inappropriate treatment recommendations could be made
  • 4. 4 Treatment switching – the problem Control Treatment True OS difference PFS PPS Intervention PFS PPS Control  Intervention RCT OS difference PFS PPS Survival time Switching is likely to result in an underestimate of the treatment effect
  • 5. What is usually done to adjust? No clear consensus Numerous „naive‟ approaches have been taken in NICE appraisals, eg: Very prone to Take no action at all selection bias Exclude or censor all patients who crossover – crossover isn‟t random Occasionally more complex statistical methods have been used, eg: Rank Preserving Structural Failure Time Models (RPSFTM) Inverse Probability of Censoring Weights (IPCW) And others are available from the literature, eg: Structural Nested Models (SNM)
  • 6. 6 What are the consequences? TA 215, Pazopanib for RCC [51% of control switched] ITT: OS HR (vs IFN) = 1.26  ICER = Dominated Censor patients: HR = 0.80  ICER = £71,648 Exclude patients: HR = 0.48  ICER = £26,293 IPCW: HR = 0.80  ICER = £72,274 RPSFTM: HR = 0.63  ICER = £38,925
  • 7. Potential solutions RPSFTM (and IPE algorithm) Randomisation-based approach, estimates counterfactual survival times Key assumption: common treatment effect IPCW Observational-based approach, censors xo patients, weights remaining patients Key assumptions: “no unmeasured confounders”; must model OS and crossover SNM Observational version of RPSFTM Key assumptions: “no unmeasured confounders”; must model OS and crossover
  • 8. Simulation study (1) None of these methods are perfect But we need to know which are likely to produce least bias in different scenarios Simulation study Simulate survival data for two treatment groups, applying crossover that is linked to patient characteristics/prognosis In some scenarios simulate a treatment effect that changes over time In some scenarios simulate a treatment effect that remains constant over time Test different %s of crossover, and different treatment effect sizes How does the bias and coverage associated with each method compare?
  • 9. 9 Simulation study (2) Methods assessed Naive methods ITT Exclude crossover patients Censor crossover patients Treatment as a time-dependent covariate Complex methods RPSFTM IPE algorithm IPCW SNM
  • 10. 10 Results: common treatment effect RPSFTM / IPE worked very well IPCW and SNM performed ok when crossover % was lower 50.00 ITT IPCW RPSFTM IPE SNM 40.00 AUC mean bias (%) 30.00 20.00 10.00 0.00 19 20 27 28 31 32 39 40 43 44 51 -10.00 IPCW and SNM performed poorly when crossover % was very high Naive methods performed poorly (generally led to higher bias than ITT)
  • 11. 11 Results: effect 15% in xo patients RPSFTM / IPE produced higher bias than previous scenarios IPCW and SNM performed similarly to RPSFTM / IPE providing crossover < 90% ITT IPCW RPSFTM IPE SNM 45.00 AUC mean bias (%) 35.00 25.00 15.00 5.00 -5.00 17 18 25 26 29 30 37 38 41 42 49 -15.00 IPCW and SNM performed poorly when crossover % was very high Bias not always lower than that associated with the ITT analysis
  • 12. 12 Results: effect 25% in xo patients RPSFTM / IPE produced even more bias IPCW and SNM produce less bias than RPSFTM / IPE providing crossover < 90% 50.00 ITT IPCW RPSFTM IPE SNM AUC mean bias (%) 40.00 30.00 20.00 10.00 0.00 23 24 33 34 35 36 45 46 47 48 57 -10.00 -20.00 No „good‟ options when crossover % is very high Often ITT analysis likely to result in least bias (esp. when trt effect low)
  • 13. 13 Conclusions Treatment crossover is an important issue that has come to the fore in HE arena Current methods for dealing with treatment crossover are imperfect Our study offers evidence on bias in different scenarios (subject to limitations) RPSFTM / IPE produce low bias when treatment effect is common  But are very sensitive to this IPCW / SNM are not affected by changes in treatment effect between groups, but in (relatively) small trial datasets observational methods are volatile  Especially when crossover % is very high (leaving low n in control group) Very important to assess trial data, crossover mechanism, treatment effect to determine which method likely to be most appropriate  Don’t just pick one!!
  • 14. 14 Back-up Slides 28/09/2012 © The University of Sheffield
  • 15. 15 Data Generation (1) Used a two-stage Weibull model to generate underlying survival times and a time-dependent covariate (called „CEA‟) Longitudinal model for CEA (for ith patient at time t): where is the random intercept is the slope for a patient in the control arm is the slope for a patient in the treatment arm (all is the change in the intercept for a patient with bad prognosis compared to a patient without bad prognosis Picked parameter values such that CEA increased over time, more slowly in the experimental group, and was higher in the badprog group 28/09/2012 © The University of Sheffield
  • 16. 16 Data Generation (2) The survival hazard function was based upon a Weibull (see Bender et al 2005):  1 h(t )   t exp( X ) In our case, X = 1 * trt i + ( * log(t) ) * trt i +  2 badprogi +  (cea(t) ) where  1 is the log hazard ratio (the baseline treatment effect)  is the time-dependent change in the treatment effect  2 is the impact of a bad prognosis baseline covariate on survival  is the coefficient of CEA, indicating its effect on survival We used this to generate our survival times  So, CEA has an effect on survival over time, and there is a separate time-dependent treatment effect 28/09/2012 © The University of Sheffield
  • 17. 17 Data Generation (3) We applied a reduced treatment effect to crossover patients – this was equivalent to the baseline treatment effect multiplied by a factor to ensure that this effect was less than (or equal to) the average effect received in the experimental group 28/09/2012 © The University of Sheffield
  • 18. 18 Data Generation (4) We then selected parameter values in order that „realistic‟ datasets were created: 1.00 2.5 AF actual 0.80 AF predicted trtrand = 0 trtrand = 1 2 0.60 Acceleration Factor 1.5 0.40 0.20 1 0.00 0.5 0 200 400 600 800 1000 1200 analysis time Number at risk 0 trtrand = 0 251 154 86 52 25 9 0 0 200 400 600 800 1000 1200 trtrand = 1 249 186 126 71 42 21 0 Time (days) 28/09/2012 © The University of Sheffield
  • 19. 19 Data Generation (5) We made several assumptions about the „crossover mechanism‟: 1. Crossover could only occur after disease progression (disease progression was approximately half of OS, calculated for each patient using a beta(5,5) distribution) 2. Crossover could only occur at 3 „consultations‟ following disease progression These were set at 21 day intervals Probability of crossover highest at initial consultation, then falls in second and third 3. Crossover probability depended on time-dependent covariates: CEA value at progression (high value reduced chance of crossover) Time to disease progression (high value increased chance of crossover) This was altered in scenarios to test a simpler mechanism where probability only depended on CEA  Given all this, CEA was a time-dependent confounder 28/09/2012 © The University of Sheffield
  • 20. 20 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  severity Initially assumed treatment effect Alter to test two levels of treatment effect  Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  over time Prognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
  • 21. 21 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  (2) severity Initially assumed treatment effect Alter to test two levels of treatment effect  Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  over time Prognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
  • 22. 22 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  (2) severity Initially assumed treatment effect Alter to test two levels of treatment effect  (4) Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  over time Prognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
  • 23. 23 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  (2) severity Initially assumed treatment effect Alter to test two levels of treatment effect  (4) Time-dependence of treatment effect Treatment effect received is equal in all crossover  (8) patients, and equals baseline treatment effect multplied by a factor. However set α to zero in (12) some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  over time Prognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
  • 24. 24 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  (2) severity Initially assumed treatment effect Alter to test two levels of treatment effect  (4) Time-dependence of treatment effect Treatment effect received is equal in all crossover  (8) patients, and equals baseline treatment effect multplied by a factor. However set α to zero in (12) some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  (24) over time Prognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
  • 25. 25 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  (2) severity Initially assumed treatment effect Alter to test two levels of treatment effect  (4) Time-dependence of treatment effect Treatment effect received is equal in all crossover  (8) patients, and equals baseline treatment effect multplied by a factor. However set α to zero in (12) some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  (24) over time Prognosis of crossover patients Test three crossover mechanisms in which different  (72) groups become more likely to cross over This combined to 72 scenarios 28/09/2012 © The University of Sheffield
  • 26. 26 Scenarios Variable Value Alternative Sample size 500  Number of prognosis groups (prog) 2  Probability of good prognosis 0.5  Probability of poor prognosis 0.5  Maximum follow-up time 3 years (1095 days)  Multiplication of OS survival time due Log hazard ratio = 0.5  to bad prognosis group Survival time distribution Alter parameters to test two levels of disease  severity Initially assumed treatment effect Alter to test two levels of treatment effect  Time-dependence of treatment effect Treatment effect received is equal in all crossover  patients, and equals baseline treatment effect multplied by a factor. However set α to zero in some scenarios. Also include additional treatment effect decrement in crossover patients in some scenarios Probability of switching treatment Test two levels of treatment crossover proportions  over time Prognosis of crossover patients Test three crossover mechanisms in which different  groups become more likely to cross over 28/09/2012 © The University of Sheffield
  • 27. 27 Estimating AUC 1. ‘Survivor function’ approach Apply treatment effect to survivor function (or hazard function) estimated for experimental group  calculate AUC 2. ‘Extrapolation’ approach Extrapolate counterfactual dataset to required time-point (only relevant for RPSFTM/IPE approaches)  calculate AUC 3. ‘Shrinkage’ approach Use estimated acceleration factor to „shrink‟ survival times in crossover patients in order to obtain an adjusted dataset  calculate AUC (only relevent for AF-based approaches) 28/09/2012 © The University of Sheffield
  • 28. A topical analogy... 28 England Spain Vs (control group) (intervention group) Kick-off...
  • 29. A topical analogy... 29 Halftime: England 0 – 3 Spain  Spain are statistically significantly better than England  For ethical reasons, the Spanish team are cloned and the English team are sent home. So the second half is Spain Vs Spain
  • 30. A topical analogy... 30 Full time: England 2 – 5 Spain  Spain still win, but not as comfortably as they would have Switching is likely to result in an underestimate of the true supremacy of the intervention
  • 31. 31 Results (4) While the SNM and IPCW methods appeared to produce similar levels of bias, the SNM approach was more volatile: Relatively often failed to converge (in up to 90% of sims in one scenario) Typically when disease severity was high and treatment effect was low Only produced lower bias than the ITT analysis in 38% of scenarios (compared to 60% for the IPCW method) Was more sensitive to increasing crossover % 28/09/2012 © The University of Sheffield
  • 32. 32 Results (5) Relationship between bias and treatment crossover % 140 120 100 80 Mean % bias 60 40 ITT 20 0 60% 70% 80% 90% 100% -20 -40 Crossover proportion (at-risk patients) 28/09/2012 © The University of Sheffield
  • 33. 33 Results (5) Relationship between bias and treatment crossover % 140 120 100 80 Mean % bias 60 IPE RPSFTM 40 ITT 20 0 60% 70% 80% 90% 100% -20 -40 28/09/2012 © The University ofCrossover Sheffield proportion (at-risk patients)
  • 34. 34 Results (5) Relationship between bias and treatment crossover % 140 120 100 80 Mean % bias IPCW 60 IPE 40 RPSFTM ITT 20 0 60% 70% 80% 90% 100% -20 -40 28/09/2012 © The University ofCrossover Sheffield proportion (at-risk patients)
  • 35. 35 Results (5) Relationship between bias and treatment crossover % 140 120 100 80 IPCW Mean % bias 60 IPE RPSFTM 40 SNM 20 ITT 0 60% 70% 80% 90% 100% -20 -40 28/09/2012 © The University ofCrossover Sheffield proportion (at-risk patients)
  • 36. 36 Conclusions Treatment crossover is an important issue that has come to the fore in HE arena Current methods for dealing with treatment crossover are imperfect Our study offers evidence on bias in different scenarios (subject to limitations) 1. RPSFTM / IPE produce low bias when treatment effect is common 2. When treatment effect ≈ 15% lower in crossover patients RPSFTM / IPE and IPCW / SNM methods produce similar levels of bias (5-10%) [provided suitable data available for obs methods and <90% crossover] 3. When treatment effect ≈ 25% lower IPCW/SNM produce less bias than RPSFTM/IPE [provided suitable data available for obs methods and <90% crossover] [and significant bias likely to remain  ITT analysis may offer least bias] 4.Very important to assess trial data, crossover mechanism, treatment effect to determine which method likely to be most appropriate  Don’t just pick one!!

Notas del editor

  1. IPCW attempts to control for time-dependent confounders as well as baseline covariatesSNM also tries to adjust for time-dependent confounders as well as baseline covariatesSNM uses the counterfactual survival model to estimate counterfactual survival for each value of psi. It then also models the hazard of the treatment process in order to identify the ‘true’ value of psi. The hazard of the treatment process is modelled as a function of baseline and time-dependent covariates, and counterfactual survival for each value of psi. It is assumed that when all covariates are included the treatment process is not related to counterfactual survival – ie it is random. The value of psi that gives counterfactual survival times that allows this assumption to be borne out is the ‘true’ value of psi.So assume that you can fully explain the treatment decision and the survival process using covariates. Assume that counterfactual survival times are the same for all individuals, controlling for covariates. Once have accounted for all time-dependent confounding the association between exposure and survival can be attributed soley to the treatment effect.No unobserved confounders, model (Cox) for hazard of treatment must be correct, and SNM for counterfactual survival must be correct.Treatment group and disease progression are perfect predictors of receiving treatment and thus would be dropped from a logistic regression modelling the treatment process. Without these (particularly disease progression) our models would be biased as it is an important prognostic covariate. Hence need to think of another method of application  2-stage approach. Including the whole dataset would not work as exp group can’t crossover, and crossover can’t happen before disease progression.Particularly prone to bias with high xo% given way we’ve implemented it in the control group after disease progression – high xo leaves little control ‘controls’.
  2. Say why we need a sim study. With real data we don’t know the truth and so can’t assess how well the different methods do. With a sim study we do know the truth, and we can assess the methods against this.