CHE Economic Evaluation Seminar 8 October 2015 by Jason Madan, University of Warwick

1. Value of Information analysis for trials
of treatments for rare diseases: Early
insights from the InSPiRe Project
Dr Jason Madan, Warwick Medical School.

2. InSPiRE: Innovative methodology for small populations research

3. InSPiRe : Development of novel and methods for the design and
analysis of clinical trials in rare diseases / small populations
WP1 - Research in early phase dose-find
trials in small populations
Sarah Zohar,
INSERM UMR1138, Paris
WP2 - Research in decision-theoretic
designs for clinical trials in small
populations
Nigel Stallard
Warwick Medical School,
Coventry
WP3 - Research in confirmatory trials for
small populations and personalised
medicines
Martin Posch
Medizinische Universität,
Vienna
WP4 - Research in use of evidence synthesis
in the planning and interpretation of clinical
trials in small populations and rare diseases
Tim Friede
University Medical Center
Gottingen
This project has received funding from the European Union's Seventh
Framework Programme for research, technological development and
demonstration under grant agreement no 602144

4. InSPiRe Team Dinner in Vienna, June 2014

5. Small populations - challenges for policy
and research
EU defines a rare disease as affecting <5 in 10000
6000 rare diseases identified, affecting around 30 million EU citizens
Orphan drugs benefit from specific drug development legislation in US
and EU
18% of Orphan drugs cost >£100K per patient per year.
Stratified and personalised medicine approaches can increase the
number of conditions with rare disease status.

6. Is Decision-making in Rare Diseases Different?
Paulden et al review the literature on decision factors and proposed decision-
making frameworks for orphan drugs:
- rule of rescue supports the non-abandonment of patients with severe
diseases if alternatives are not available (irrespective of cost).
- equity principle argues that resource allocation should be based on
need, distribution of health, and magnitude of benefit (no special treatment for
orphan drugs)
- rights-based approach states that individuals have a right to a
minimum level of healthcare (even if their disease is rare).
Cost-effectiveness is the application of these principles, not a separate factor.

7. Paulden et al, Pharmacoeconomics 2015
Proposed framework for aiding coverage decision

8. Effectiveness evidence requirements for orphan
drugs
Some authors that orphan drugs should provide the same level of
evidence.
Proposed decision frameworks often assume high quality evidence
not available.
- Smaller trials
- Intermediate outcomes (e.g. oncology trials that do not look
at long-term survival)

9. Decision – Theoretic Trial Design
• Mainstream statistical literature on trial design follows frequentist paradigm
• Define null hypothesis
• Design trial to control probability of type I / type II error (false positive
/ false negative)
• Desired error rates set by convention (e.g. 5% type I , 10% type II).
• Frequentist hypothesis testing, no synthesis with other data sources
• Decision-theoretic trial design statistical literature:
• Bayesian framework
• Decision involving choice between mutually exclusive alternatives
• Uncertain payoff / utility function to maximise.
Rare diseases can make conventional approaches to trial design unfeasible.

10. Statistical literature on decision-theory and
rare disease trials
• We carried out a systematic review of the literature on decision theoretic
designs for pilot studies and small clinical trials (Hee et al 2015)
• 67 articles identified (up to October 2014)
• 8 explicitly mention Value of Information analysis
• Range of decision-making perspectives – commercial, regulatory, societal
or none.
• Three types of study design – single stage, multi-stage, portfolio / multi-
arm.

11. Key issues from the Statistical Literature on
decision-theoretic trial design
• Computational methods for identifying optima
• Choice and structure of the utility function
• Construction of appropriate priors

12. A rule of thumb for optimal sample size
Let N be the total population for whom two treatments are being
considered
Cheng et al (2003) show that the optimal trial size tends
asymptotically to O(N1/2) where the primary endpoint follows a
Bernouilli distribution
Stallard et al (2015) extend this result to situations where the
primary endpoint follows any distribution from the Exponential
family.
Applicability of this result in small populations limited by
- restrictive assumptions about utility functions
- asymptotic applicability of findings.

13. Characteristics of small population decisions:
- Difficulties in recruiting large numbers of patients
- Multiple trials competing for same patient group
- Multiple regulatory frameworks with different models of interaction
between decision-makers
- Strategic interaction between industry and regulators.
- Regulator and reimbursement roles often separate.
- How to set reimbursement rules to optimise industry-sponsored
trials in rare diseases?
VoI for rare diseases – Structuring the decision
problem

14. Strategic interaction of sponsor and regulator decision-
making
 B

b
Consider a proposed trial of a new treatment T.
Let
B = the true individual incremental net benefit of T relative to standard of care
= prior distribution for B
n = the sample size of the proposed trial
= Required assurance level for net benefit
= Posterior mean for individual net benefit after trial
C(n) = Cost of conducting the trial
N = The total population that are eligible for T.
p = The price paid to the sponsor for each patient treated.

15. ( 0)A p b
U I BN C 
 
 ( 0)R p b
U NI B p 
 
( 0)S p b
U NI p C 
 
Societal, commercial and regulatory objective functions
Societal objective function
Regulatory objective function
Commercial objective function
Need to consider who sets which decision parameters, and in which order, to
arrive at optima.

16. Example: α fixed, regulator chooses p, sponsor chooses n.
If sponsor goes first, n* = p* = 0
If regulator goes first, can identify pareto optima by finding solutions
where = - (assuming convexity).

17. Isoutils for regulator and sponsor, with heavy line depicting pareto optima, and
small circles depicting global optimum for the regulator
 / 0.3E B  
 / 0.1E B  

18. Moving to more realistic decision scenarios
Can use similar approaches to explore more realistic regulatory frameworks for
trials in rare diseases.
E.G :
• Make price a function of the posterior mean and the probability this is zero
• Make total net benefit non-linear in population size
• Introduce asymmetric priors.
• Extend to portfolios of treatments
Value of Information methods in this area need to take the ongoing debate on
decision-making frameworks for rare disease treatments into account.

19. References
http://www.eurordis.org/
Cheng, Y., Fusheng, S., and Berry, D. A. (2003). Choosing sample size for a clinical
trial using decision analysis. Biometrika 90, 923
Hee, S. W., Hamborg, T., Day, S., Miller, F., Madan, J., Posch, M., Zohar, S., and
Stallard, N. (2015). Decision theoretic designs for small trials and pilot studies: a
review. SMMR (online access)
Stallard N, Miller D, Hee SW, Madan J, Zohar S and Posch M (2015). Determination
of the optimal sample size for a clinical trial accounting for the population size.
Submitted to Biometrics.
Paulden, M., Stafinski, T., Menon, D., & McCabe, C. (2015). Value-Based
Reimbursement Decisions for Orphan Drugs: A Scoping Review and Decision
Framework. PharmacoEconomics, 33(3), 255-269.