SlideShare una empresa de Scribd logo
1 de 24
Assessing the Value of Co-dependent Technologies:
How Can Current Methods and Processes Be Improved?
ScHARR Seminar, University of Sheffield
16 April 2013 ● Sheffield, UK
Martina Garau, Office of Health Economics
Adrian Towse (OHE) and the other authors of:
Garau, M., Towse, A., Garrison, L., Housman, L. and Ossa, D.
(2012) Can and should value based pricing be applied to
molecular diagnostics? Personalized Medicine. 10(1), 61-72.
Acknowledgements
• What is the value of co-dependent technologies?
• Framework for assessing value
• How prove value?
• How aggregate value dimensions?
• Proposed institutional processes
• International experience
• Australia
• NICE
• Conclusions
Agenda
• “Technologies that are dependent on another technology
either to achieve their intended effect or to enhance their
intended effect” (www.health.gov.au)
• In particular, a diagnostic test (Dx) can be used to identify
patients most likely to:
• Respond or fail to respond to a drug treatment (Tx)
• Exhibit adverse events
But also to:
• Monitor responses to drugs
• Determine the risk of developing a disease
Definition of co-dependent technologies
1. Key elements of value are:
• Health effects for patients (clinical effectiveness measured by the
QALY)
• Cost offsets (savings to the health care system)
What is the value of co-dependent technologies? (1)
Traditionally, ICERs do not capture benefits beyond health attributes
NHS,PSS Cost of Treatment A - NHS,PSS Cost of Treatment B
ICER =
Health effects of Treatment A - Health effects of Treatment B
The focus is on downstream effects of treatments not recognising the additional
value brought by use of Dx
• Other value dimensions
• Societal preferences giving priority to certain patients or
diseases
• Quality of life aspects not reflected in generic measures
used in CE analyses
• Other effects beyond those to patients and NHS
(productivity gains)
• Health care process related aspects (dignity, time and
location of treatment)
• Information for the patient independent of health
effects
What is the value of co-dependent technologies? (2)
• The value created is a “joint product” and there are no rules
for the attribution of the value to one or the other
• Garrison and Austin (2007) pointed out that how value is
allocated across patients, payers, Dx manufactures and Tx
manufacturers depends on the institutional context
• E.g. whether the Tx was priced before the Dx was available; the
relative strength of intellectual property protection for Dx and Tx
• This will have consequences in terms of incentives for
evidence generation and subsequent innovation
The issue of attribution of value of
co-dependent technologies
Value
1. Reducing
drug adverse
effects
2. Reducing
time delays in
selecting
optimal Tx
3.Increasing
adherence or
willingness to
start Tx
4. Enabling Tx
effective in a
small fraction
to be made
available
5.Reducing
uncertainty
about value
Framework for assessing value of
co-dependent technologies
Value dimensions
derived from:
• Characteristics of
Dx recently
introduced
• Literature review
on the economics
of personalised
medicine and
value of
information of Dx
Availability of Dx can improve average benefit-risk ratio
so, depending on the severity of side effects:
• Tx obtains marketing authorisation, or
• Use of a licensed Tx in clinical practice increases
Example: HLA-B*5701
• Allele associated with hypersensitivity to abacavir for HIV-1
• Identification of the marker has increased prescribing of abacavir, which
now is recommended for HLA-B*5701-negative patients in European and
US guidelines
1. Reducing or avoiding drug adverse effects
Identifying non-responders and switching them to an alternative
treatment regime/care can:
• Improve survival and/or quality of life (particularly in diseases at advanced
stages)
• Avoid or reduce the cost of treating non-responders
• Avoid or reduce inconvenience to patients
Example: BCR-ABL
• Test identifies chronic myelogenous leukemia (CML) patients who are
receiving treatment, but not responding to it
• Can prevent the disease from progressing to blast crisis and death, and
enables stopping first-line treatment when no longer effective
2. Reducing time delays in selecting optimal Tx
• Patients are more motivated if they know (ex-ante) the
intervention is likely to work
• Issue of non-responders who might experience disutility (they
can feel “left-behind”)
Example: PreDx Diabetes Risk test
• Test estimates the patient’s risk for developing Type 2 diabetes
over the next five years
• This can further encourage patients to follow a healthy lifestyle
and take other preventive measures.
3. Increasing adherence or willingness to under-
take Tx or other interventions
A biomarker or other genetic characteristic allowing for
patient stratification can:
1. “Rescue” Tx that otherwise may either not have been licensed or
have been withdrawn
2. Increase the chance of a Tx meeting reimbursement criteria (if
targeting responders improves cost -effectiveness)
3. Accelerate R&D process for Tx (if stratification ascertained at an
early development stage)
4. Enabling Tx effective in a small fraction to be
made available
• Gefitinib for non-small-cell lung cancer (NSCLC) initially licensed, but
withdrawn when Phase III failed to show a survival benefit. With the
identification of EGFR mutations and its association with response rate to
TKIs, gefitinib was approved in the EU and other markets in combination with
the EGFR mutation test.
• NICE recommended trastuzumab for advanced and early-stage breast cancer
in HER2/neu positive patients identified with HER2/neu test. The Dx-Tx cost
per QALY was found to be below the standard threshold.
• Crizotinib targets a small subset of NSCLC patients with an ALK-positive
molecular abnormality. The development of the ALK FISH test has
accelerated the development process and increased the likelihood of
crizotinib delivering health benefits and commercial value.
4. Enabling Tx effective in a small fraction to be
made available – Examples
• Uncertainty around expected health effects and costs; influences the risk of poor
value for money for payers
• Value of information to patients about their medical condition independent of the
health outcome (Ash, et al, 1990)
• “Empowerment” (Payne, et al, 2012)
• Effect of reassurance (measured with EQ-5D?) (Kenen, 1996)
• Lifestyle choices and planning (Lee, et al, 2010)
• Example of Oncotype DX ® and MammaPrint ®
• Multi-gene assays estimating the risk of recurrence in breast cancer patients
following surgery
• Can guide intervention decisions and reduce the risk of dispensing unnecessary
chemotherapy (reduce resource costs to the healthcare system and adverse effect
for the patient)
5. Reducing uncertainty about value
• Low accuracy of Dx will decrease potential net gains to
patients and healthcare system
• False positive and false negative patients will not get most appropriate
therapy
• Tx can be more cost effective when used on its own
• When Dx does not provide binary response, depending on the size of
the subset for which the Dx does not provide clear-cut result and the
Dx cost relative to Tx
• When Dx has low accuracy
Other factors affecting value of
co-dependent technologies
• Barriers to evidence generation
• Cost and feasibility of certain study designs
• Protection of intellectual property rights of Dx
• Regulatory processes for diagnostics
• Assessment of competitive tests with similar clinical use
How prove value of co-dependent technologies?
How is value aggregated? Key issues Key merits
Net benefit As the sum of the benefits,
each assessed in monetary
terms
Challenges estimating the value in
monetary terms of each type of value
Allocating a monetary value to health has
been always one of the mayor criticisms
Arguably, a better grounding in economic theory
Facilitates the comparison of value and value for money across health
and other sectors
Use of monetary value may resonate better with some (private)
payers
MCDA As the sum of the points
assigned to each aspect of
value
The cost -effectiveness threshold would
need to be re-assessed in terms of the
cost per incremental “point”
A pragmatic approach, widely used in the UK public sector.
A more transparent (compared to a weighted QALY, or deliberative
process alone) means of addressing multiple criteria
MCDA is used in local NHS commissioning – potential to develop a
consistent priority-setting framework for both new and existing
health care technologies
Weighted
adjusted
QALYs
1. By QALYs gained, up-rated
or down-rated by one or
multiple weights to represent
the magnitudes of other
aspects of value; or
2. Direct estimation of how
people trade off QALY gains
with other value elements
Assumes that all other sources of value
are proportional to the number of QALYs
gained.
Implications for the threshold. If the value
of new technologies is assessed in terms
of a range of criteria, then opportunity
cost also must be considered in the same
terms, not just QALYs foregone. Even if a
simple social weighting or QALYs is
applied, opportunity cost will change
Is it relevant to state here the classic arguments in favour of the QALY
such as:
- Allows for comparisons across therapeutic areas in the NHS
- “A QALY is a QALY” argument
- Well established in the UK within HTA bodies (and academic
centres)
- Understood by health economics community
Deliberative
process
Weights are assigned by a
committee to each relevant
aspect of value
The weights are often implicit
Are implications for the threshold
Provides an element of flexibility
Is a well-recognised approach taken by HTA bodies around the world.
How aggregate value dimensions?
Source: adapted from Sussex, et al, 2013
• A joint Dx-Tx review of “at launch” technologies; to be done by a drug
committee to exploit synergies across Dx and Tx
• However, there is a need to address the lack of expertise of most drug
committees in the Dx area
• A separate Dx committee to develop Dx-specific expertise and to assess
multiple tests with similar clinical use
• However, there may be a trade-off if there are not enough decisions to justify
a distinct committee
• A comprehensive and consistent approach to assessing value of both Dx
and Tx
Proposed institutional processes for
co-dependent technologies
Proposed institutional processes
New Dx
Dx linked to a Tx
(companion Dx)
Dx-Tx pair
launched
simultaneously
Dx-Tx joint
assessment via
Drug process
Single Dx
launched
separately
Dx assessed via
Diagnostic-
dedicated process
Multiple Dx with
same clinical use
Dx assessed via
Diagnostic-
dedicated process
Dx not linked to a
Tx
Dx assessed via
Diagnostic-
dedicated process
• Until recently, Dx and associated Tx assessed via different
committees (MSAC and PBAC)
• No clear structure for consideration of the interactions and benefits
from joint use
• New coordinated process and decision framework for “co-
dependent technologies”
• “Integrated” applications combining information from Dx and Tx
manufacturers
• Reimbursement decisions are made jointly by PBAC and MSAC to
ensure optimal clinical use (Merlin, et al, 2012)
• The preferred type of evidence to show clinical benefit is a randomised
clinical trial
International experience: Australia
• NICE has dedicated-process for stand-alone Dx that follows
very closely that used for drugs
• Strong preference for measuring health gains with the QALY
• Value dimensions beyond health effects, such as value of information
to patients and process-related benefits, are not explicitly factored in
• “At launch” combinations are appraised via the drug review
programme (TAs)
• No explicit consideration of test-related parameters (accuracy, costs)
• Value dimensions beyond health effects are not explicitly factored in
International experience: NICE in England
and Wales
• The use of Dx-Tx combinations can deliver health gains and cost savings within the
health care system, but also generate broader benefits to patients and society
• To ensure efficient use of limited resources, health decision makers should take
account of the full value generated by health technologies
• Clear incentives are needed to encourage evidence collection
• HTA and other decision making systems need coordinated and consistent approach
to assessing value of Dx and Tx
• NICE is heading in this direction, but does not yet have a comprehensive approach
to assessing the value of Dx or Tx
• In Australia, the common methodology needs to be supported by a realistic view of
evidence development
Conclusions
Ash, D.A., Patton, J.P. and Hershey, J.C. (1990) Knowing for the sake of knowing: The value of prognostic information. Medical
Decision Making. 10(1), 47-57.
Garau, M., Towse, A., Garrison, L., Housman, L. and Ossa, D. (2012) Can and should value based pricing be applied to molecular
diagnostics? Personalized Medicine. 10(1), 61-72.
Garrison, L.P. and Austin, M.J.F. (2007) The economics of personalized medicine: A model of incentives for value creation and
capture. Drug Information Journal. 41(1), 501-509.
Lee, D.W., Neumann, P.J. and Rizzo, J.A. (2010) Understanding the medical and nonmedical value of diagnostics testing. Value in
Health. 13(2), 310-314.
Merlin, T., Farah, C., Schubert, C., Mitchell, A., Hiller, J.E. and Ryan, P. (2012) Assessing personalized medicines in Australia: A
national framework for reviewing codependent technologies. Medical Decision Making. 33(3), 333-342.
Kenen, R.H. (1996) The at-risk health status and technology: A diagnostic invitation and the gift of knowing. Social Science &
Medicine. 42(11), 1545-1553.
Payne, K., McAllister, M. and Davies L. (2012) Valuing the economic benefits of complex interventions: When maximising health is
not sufficient. Health Economics. 22(3), 258-271.
Sussex, J., Towse, A. and Devlin, N. (2013) Operationalising value based pricing of medicines: A taxonomy of approaches.
Pharmacoeconomics. 13(1), 1-10.
References
To enquire about additional information and analyses, please contact
Martina Garau: mgarau@ohe.org
To keep up with the latest news and research, subscribe to our blog, OHE News.
Follow us on Twitter @OHENews, LinkedIn and SlideShare.
Office of Health Economics (OHE)
Southside, 7th Floor
105 Victoria Street
London SW1E 6QT
United Kingdom
+44 20 7747 8850
www.ohe.org
OHE’s publications may be downloaded free of charge for registered users of its website.
©2013 OHE

Más contenido relacionado

La actualidad más candente

ScHARR, ISPOR Presentation, Madrid, November, 2011
ScHARR, ISPOR Presentation, Madrid, November, 2011ScHARR, ISPOR Presentation, Madrid, November, 2011
ScHARR, ISPOR Presentation, Madrid, November, 2011
ScHARR HEDS
 

La actualidad más candente (20)

The importance of_measuring_outcomes
The importance of_measuring_outcomesThe importance of_measuring_outcomes
The importance of_measuring_outcomes
 
Pharmacoeconomic Assessment through Market Approval and Beyond
Pharmacoeconomic Assessment through Market Approval and BeyondPharmacoeconomic Assessment through Market Approval and Beyond
Pharmacoeconomic Assessment through Market Approval and Beyond
 
Poem 2012
Poem 2012Poem 2012
Poem 2012
 
Integrating Laboratory Services
Integrating Laboratory ServicesIntegrating Laboratory Services
Integrating Laboratory Services
 
Structured decision making approaches to the inclusion of multiple criteria i...
Structured decision making approaches to the inclusion of multiple criteria i...Structured decision making approaches to the inclusion of multiple criteria i...
Structured decision making approaches to the inclusion of multiple criteria i...
 
Health Technology Assessment- Overview
Health Technology Assessment- OverviewHealth Technology Assessment- Overview
Health Technology Assessment- Overview
 
Evidence based medicine
Evidence based medicineEvidence based medicine
Evidence based medicine
 
MCDA for orphan_medicines_mg_dec2014
MCDA for orphan_medicines_mg_dec2014MCDA for orphan_medicines_mg_dec2014
MCDA for orphan_medicines_mg_dec2014
 
Getting evidence from economic evaluation into healthcare practice
Getting evidence from economic evaluation into healthcare practiceGetting evidence from economic evaluation into healthcare practice
Getting evidence from economic evaluation into healthcare practice
 
Translational medicine
Translational medicineTranslational medicine
Translational medicine
 
Abpi rd conference_towse_ 20_nov_2014
Abpi rd conference_towse_ 20_nov_2014Abpi rd conference_towse_ 20_nov_2014
Abpi rd conference_towse_ 20_nov_2014
 
Adaptive pathways intro_ispor_second_plenary_nov14
Adaptive pathways intro_ispor_second_plenary_nov14Adaptive pathways intro_ispor_second_plenary_nov14
Adaptive pathways intro_ispor_second_plenary_nov14
 
Ispor workshop value-based_assess_nice_kks_nov2014
Ispor workshop value-based_assess_nice_kks_nov2014Ispor workshop value-based_assess_nice_kks_nov2014
Ispor workshop value-based_assess_nice_kks_nov2014
 
Comparing Cancer-Specific Preference-Based Outcome Measures: The Same but Dif...
Comparing Cancer-Specific Preference-Based Outcome Measures: The Same but Dif...Comparing Cancer-Specific Preference-Based Outcome Measures: The Same but Dif...
Comparing Cancer-Specific Preference-Based Outcome Measures: The Same but Dif...
 
Outcome measures and their importance in physiotherapy practice and research
Outcome measures and their importance in physiotherapy practice and researchOutcome measures and their importance in physiotherapy practice and research
Outcome measures and their importance in physiotherapy practice and research
 
ScHARR, ISPOR Presentation, Madrid, November, 2011
ScHARR, ISPOR Presentation, Madrid, November, 2011ScHARR, ISPOR Presentation, Madrid, November, 2011
ScHARR, ISPOR Presentation, Madrid, November, 2011
 
The EQ-5D and Its Use Internationally
The EQ-5D and Its Use InternationallyThe EQ-5D and Its Use Internationally
The EQ-5D and Its Use Internationally
 
ISPOR Special Task Force on US Value Frameworks: A Non-US Perspective
ISPOR Special Task Force on US Value Frameworks: A Non-US PerspectiveISPOR Special Task Force on US Value Frameworks: A Non-US Perspective
ISPOR Special Task Force on US Value Frameworks: A Non-US Perspective
 
Developing implementing patient reported outcome measures for dietetics pr...
Developing  implementing patient reported outcome measures for dietetics   pr...Developing  implementing patient reported outcome measures for dietetics   pr...
Developing implementing patient reported outcome measures for dietetics pr...
 
Real effectiveness medicine pursuing best effectiveness in the ordinary care ...
Real effectiveness medicine pursuing best effectiveness in the ordinary care ...Real effectiveness medicine pursuing best effectiveness in the ordinary care ...
Real effectiveness medicine pursuing best effectiveness in the ordinary care ...
 

Similar a Improving Methods and Processes for Assessing Codependent Technologies

Implications of Co-Dependent Technologies in Medicine
Implications of Co-Dependent Technologies in MedicineImplications of Co-Dependent Technologies in Medicine
Implications of Co-Dependent Technologies in Medicine
Office of Health Economics
 
How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...
Todd Berner MD
 
Matching the Research Design to the Study Question
Matching the Research Design to the Study QuestionMatching the Research Design to the Study Question
Matching the Research Design to the Study Question
AcademyHealth
 

Similar a Improving Methods and Processes for Assessing Codependent Technologies (20)

Implications of Co-Dependent Technologies in Medicine
Implications of Co-Dependent Technologies in MedicineImplications of Co-Dependent Technologies in Medicine
Implications of Co-Dependent Technologies in Medicine
 
Dimensions of value, assessment, and decision making
Dimensions of value, assessment, and decision making Dimensions of value, assessment, and decision making
Dimensions of value, assessment, and decision making
 
Claire Hulme
Claire HulmeClaire Hulme
Claire Hulme
 
The Value of Knowing and Knowing the Value: Improving the Health Technology A...
The Value of Knowing and Knowing the Value: Improving the Health Technology A...The Value of Knowing and Knowing the Value: Improving the Health Technology A...
The Value of Knowing and Knowing the Value: Improving the Health Technology A...
 
Evaluating the impact of HTA and ‘better decision-making’ on health outcomes
Evaluating the impact of HTA and ‘better decision-making’ on health outcomesEvaluating the impact of HTA and ‘better decision-making’ on health outcomes
Evaluating the impact of HTA and ‘better decision-making’ on health outcomes
 
Policy Implications of Healthcare Associated Infections
Policy Implications of Healthcare Associated InfectionsPolicy Implications of Healthcare Associated Infections
Policy Implications of Healthcare Associated Infections
 
ADAPTIVE PATHWAYS
ADAPTIVE PATHWAYSADAPTIVE PATHWAYS
ADAPTIVE PATHWAYS
 
Towards Patient-Centred Care
Towards Patient-Centred CareTowards Patient-Centred Care
Towards Patient-Centred Care
 
How to Define Effective and Efficient Real World Trials
How to Define Effective and Efficient Real World TrialsHow to Define Effective and Efficient Real World Trials
How to Define Effective and Efficient Real World Trials
 
How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...How to design effective and efficient real world trials TB Evidence 2014 10.2...
How to design effective and efficient real world trials TB Evidence 2014 10.2...
 
1.3.2 Healthcare and Pharmaceuticals
1.3.2 Healthcare and Pharmaceuticals1.3.2 Healthcare and Pharmaceuticals
1.3.2 Healthcare and Pharmaceuticals
 
Improving the Focus of Precision Medicine: the Role of Health Technology Asse...
Improving the Focus of Precision Medicine: the Role of Health Technology Asse...Improving the Focus of Precision Medicine: the Role of Health Technology Asse...
Improving the Focus of Precision Medicine: the Role of Health Technology Asse...
 
2 gf h econ cost effectivness and roi
2 gf h econ cost effectivness and roi2 gf h econ cost effectivness and roi
2 gf h econ cost effectivness and roi
 
Matching the Research Design to the Study Question
Matching the Research Design to the Study QuestionMatching the Research Design to the Study Question
Matching the Research Design to the Study Question
 
Use of real-world data to provide evidence in.pptx
Use of real-world data to provide evidence in.pptxUse of real-world data to provide evidence in.pptx
Use of real-world data to provide evidence in.pptx
 
HTA Collab for Ghana and its usefulnesss
HTA Collab for Ghana and its usefulnesssHTA Collab for Ghana and its usefulnesss
HTA Collab for Ghana and its usefulnesss
 
Placing the Evolution of HTA In Emerging Markets in Context of Health System ...
Placing the Evolution of HTA In Emerging Markets in Context of Health System ...Placing the Evolution of HTA In Emerging Markets in Context of Health System ...
Placing the Evolution of HTA In Emerging Markets in Context of Health System ...
 
An introduction to using cost-effectiveness analysis to inform spending decis...
An introduction to using cost-effectiveness analysis to inform spending decis...An introduction to using cost-effectiveness analysis to inform spending decis...
An introduction to using cost-effectiveness analysis to inform spending decis...
 
ICN Victoria: Gantner on "Translating Research into Practice"
ICN Victoria: Gantner on "Translating Research into Practice"ICN Victoria: Gantner on "Translating Research into Practice"
ICN Victoria: Gantner on "Translating Research into Practice"
 
Factors Influencing Decisions by NICE
Factors Influencing Decisions by NICEFactors Influencing Decisions by NICE
Factors Influencing Decisions by NICE
 

Más de Office of Health Economics

Más de Office of Health Economics (20)

Annual lecture
Annual lecture Annual lecture
Annual lecture
 
Devlin ispor 2020 issues panel 20.05.20
Devlin ispor 2020 issues panel 20.05.20 Devlin ispor 2020 issues panel 20.05.20
Devlin ispor 2020 issues panel 20.05.20
 
Towse 2020 antimicrobials melbourne final
Towse 2020 antimicrobials melbourne finalTowse 2020 antimicrobials melbourne final
Towse 2020 antimicrobials melbourne final
 
Towse cgd price transparency seminar
Towse cgd price transparency seminarTowse cgd price transparency seminar
Towse cgd price transparency seminar
 
OHE presents at G20 AMR-R&D meeting in Paris - Adrian Towse
OHE presents at G20 AMR-R&D meeting in Paris - Adrian TowseOHE presents at G20 AMR-R&D meeting in Paris - Adrian Towse
OHE presents at G20 AMR-R&D meeting in Paris - Adrian Towse
 
Pricing in emerging markets: options to get value for money - Adrian Towse
Pricing in emerging markets: options to get value for money - Adrian TowsePricing in emerging markets: options to get value for money - Adrian Towse
Pricing in emerging markets: options to get value for money - Adrian Towse
 
% GDP spending in UK, G5 countries and OECD upper middle income countries. W...
% GDP spending in UK, G5 countries and OECD upper middle income countries.  W...% GDP spending in UK, G5 countries and OECD upper middle income countries.  W...
% GDP spending in UK, G5 countries and OECD upper middle income countries. W...
 
The role of real world data and evidence in building a sustainable & efficien...
The role of real world data and evidence in building a sustainable & efficien...The role of real world data and evidence in building a sustainable & efficien...
The role of real world data and evidence in building a sustainable & efficien...
 
ISPOR Education Symposium- Go where the money is
ISPOR Education Symposium- Go where the money isISPOR Education Symposium- Go where the money is
ISPOR Education Symposium- Go where the money is
 
Role Substitution, Skill Mix, and Provider Efficiency and Effectiveness : Les...
Role Substitution, Skill Mix, and Provider Efficiency and Effectiveness : Les...Role Substitution, Skill Mix, and Provider Efficiency and Effectiveness : Les...
Role Substitution, Skill Mix, and Provider Efficiency and Effectiveness : Les...
 
IS INDICATION BASED PRICING FEASIBLE AND/OR BENEFICIAL FOR SOCIETY?
 IS INDICATION BASED PRICING FEASIBLE AND/OR BENEFICIAL FOR SOCIETY? IS INDICATION BASED PRICING FEASIBLE AND/OR BENEFICIAL FOR SOCIETY?
IS INDICATION BASED PRICING FEASIBLE AND/OR BENEFICIAL FOR SOCIETY?
 
Ispor 2019 poster - Patricia Cubi-Molla
Ispor 2019 poster - Patricia Cubi-MollaIspor 2019 poster - Patricia Cubi-Molla
Ispor 2019 poster - Patricia Cubi-Molla
 
Understanding what aspects of health and quality of life are important to people
Understanding what aspects of health and quality of life are important to peopleUnderstanding what aspects of health and quality of life are important to people
Understanding what aspects of health and quality of life are important to people
 
Novel approaches for valuing health at the end of life
Novel approaches for valuing health at the end of lifeNovel approaches for valuing health at the end of life
Novel approaches for valuing health at the end of life
 
Assessing the Life-Cycle Value Added of Second Generation Antipsychotics in S...
Assessing the Life-Cycle Value Added of Second Generation Antipsychotics in S...Assessing the Life-Cycle Value Added of Second Generation Antipsychotics in S...
Assessing the Life-Cycle Value Added of Second Generation Antipsychotics in S...
 
HTA and payment mechanisms for new drugs to tackle AMR
HTA and payment mechanisms for new drugs to tackle AMRHTA and payment mechanisms for new drugs to tackle AMR
HTA and payment mechanisms for new drugs to tackle AMR
 
Assessing the Life-cycle Value Added of Second-Generation Antipsychotics in S...
Assessing the Life-cycle Value Added of Second-Generation Antipsychotics in S...Assessing the Life-cycle Value Added of Second-Generation Antipsychotics in S...
Assessing the Life-cycle Value Added of Second-Generation Antipsychotics in S...
 
Pay for Performance for Specialised Care in England
Pay for Performance for Specialised Care in EnglandPay for Performance for Specialised Care in England
Pay for Performance for Specialised Care in England
 
Real option value drugs: is it really an option?
Real option value drugs: is it really an option?Real option value drugs: is it really an option?
Real option value drugs: is it really an option?
 
MCDA OR WEIGHTED CEA BASED ON THE QALY? WHICH IS THE FUTURE FOR HTA DECISION ...
MCDA OR WEIGHTED CEA BASED ON THE QALY? WHICH IS THE FUTURE FOR HTA DECISION ...MCDA OR WEIGHTED CEA BASED ON THE QALY? WHICH IS THE FUTURE FOR HTA DECISION ...
MCDA OR WEIGHTED CEA BASED ON THE QALY? WHICH IS THE FUTURE FOR HTA DECISION ...
 

Último

Último (20)

8980367676 Call Girls In Ahmedabad Escort Service Available 24×7 In Ahmedabad
8980367676 Call Girls In Ahmedabad Escort Service Available 24×7 In Ahmedabad8980367676 Call Girls In Ahmedabad Escort Service Available 24×7 In Ahmedabad
8980367676 Call Girls In Ahmedabad Escort Service Available 24×7 In Ahmedabad
 
Trichy Call Girls Book Now 9630942363 Top Class Trichy Escort Service Available
Trichy Call Girls Book Now 9630942363 Top Class Trichy Escort Service AvailableTrichy Call Girls Book Now 9630942363 Top Class Trichy Escort Service Available
Trichy Call Girls Book Now 9630942363 Top Class Trichy Escort Service Available
 
Call Girls Hosur Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Hosur Just Call 9630942363 Top Class Call Girl Service AvailableCall Girls Hosur Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Hosur Just Call 9630942363 Top Class Call Girl Service Available
 
Best Rate (Guwahati ) Call Girls Guwahati ⟟ 8617370543 ⟟ High Class Call Girl...
Best Rate (Guwahati ) Call Girls Guwahati ⟟ 8617370543 ⟟ High Class Call Girl...Best Rate (Guwahati ) Call Girls Guwahati ⟟ 8617370543 ⟟ High Class Call Girl...
Best Rate (Guwahati ) Call Girls Guwahati ⟟ 8617370543 ⟟ High Class Call Girl...
 
Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...
Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...
Premium Bangalore Call Girls Jigani Dail 6378878445 Escort Service For Hot Ma...
 
Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...
Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...
Jogeshwari ! Call Girls Service Mumbai - 450+ Call Girl Cash Payment 90042684...
 
Call Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service AvailableCall Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service Available
Call Girls Ahmedabad Just Call 9630942363 Top Class Call Girl Service Available
 
Call Girls Guntur Just Call 8250077686 Top Class Call Girl Service Available
Call Girls Guntur  Just Call 8250077686 Top Class Call Girl Service AvailableCall Girls Guntur  Just Call 8250077686 Top Class Call Girl Service Available
Call Girls Guntur Just Call 8250077686 Top Class Call Girl Service Available
 
Call Girls Gwalior Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Gwalior Just Call 8617370543 Top Class Call Girl Service AvailableCall Girls Gwalior Just Call 8617370543 Top Class Call Girl Service Available
Call Girls Gwalior Just Call 8617370543 Top Class Call Girl Service Available
 
Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...
Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...
Independent Call Girls In Jaipur { 8445551418 } ✔ ANIKA MEHTA ✔ Get High Prof...
 
Top Rated Bangalore Call Girls Richmond Circle ⟟ 9332606886 ⟟ Call Me For Ge...
Top Rated Bangalore Call Girls Richmond Circle ⟟  9332606886 ⟟ Call Me For Ge...Top Rated Bangalore Call Girls Richmond Circle ⟟  9332606886 ⟟ Call Me For Ge...
Top Rated Bangalore Call Girls Richmond Circle ⟟ 9332606886 ⟟ Call Me For Ge...
 
Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any TimeTop Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
 
Call Girls in Delhi Triveni Complex Escort Service(🔝))/WhatsApp 97111⇛47426
Call Girls in Delhi Triveni Complex Escort Service(🔝))/WhatsApp 97111⇛47426Call Girls in Delhi Triveni Complex Escort Service(🔝))/WhatsApp 97111⇛47426
Call Girls in Delhi Triveni Complex Escort Service(🔝))/WhatsApp 97111⇛47426
 
Best Rate (Patna ) Call Girls Patna ⟟ 8617370543 ⟟ High Class Call Girl In 5 ...
Best Rate (Patna ) Call Girls Patna ⟟ 8617370543 ⟟ High Class Call Girl In 5 ...Best Rate (Patna ) Call Girls Patna ⟟ 8617370543 ⟟ High Class Call Girl In 5 ...
Best Rate (Patna ) Call Girls Patna ⟟ 8617370543 ⟟ High Class Call Girl In 5 ...
 
Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...
Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...
Call Girls Vasai Virar Just Call 9630942363 Top Class Call Girl Service Avail...
 
Call Girls Visakhapatnam Just Call 8250077686 Top Class Call Girl Service Ava...
Call Girls Visakhapatnam Just Call 8250077686 Top Class Call Girl Service Ava...Call Girls Visakhapatnam Just Call 8250077686 Top Class Call Girl Service Ava...
Call Girls Visakhapatnam Just Call 8250077686 Top Class Call Girl Service Ava...
 
💕SONAM KUMAR💕Premium Call Girls Jaipur ↘️9257276172 ↙️One Night Stand With Lo...
💕SONAM KUMAR💕Premium Call Girls Jaipur ↘️9257276172 ↙️One Night Stand With Lo...💕SONAM KUMAR💕Premium Call Girls Jaipur ↘️9257276172 ↙️One Night Stand With Lo...
💕SONAM KUMAR💕Premium Call Girls Jaipur ↘️9257276172 ↙️One Night Stand With Lo...
 
Top Rated Hyderabad Call Girls Erragadda ⟟ 9332606886 ⟟ Call Me For Genuine ...
Top Rated  Hyderabad Call Girls Erragadda ⟟ 9332606886 ⟟ Call Me For Genuine ...Top Rated  Hyderabad Call Girls Erragadda ⟟ 9332606886 ⟟ Call Me For Genuine ...
Top Rated Hyderabad Call Girls Erragadda ⟟ 9332606886 ⟟ Call Me For Genuine ...
 
Top Rated Bangalore Call Girls Mg Road ⟟ 9332606886 ⟟ Call Me For Genuine S...
Top Rated Bangalore Call Girls Mg Road ⟟   9332606886 ⟟ Call Me For Genuine S...Top Rated Bangalore Call Girls Mg Road ⟟   9332606886 ⟟ Call Me For Genuine S...
Top Rated Bangalore Call Girls Mg Road ⟟ 9332606886 ⟟ Call Me For Genuine S...
 
VIP Hyderabad Call Girls Bahadurpally 7877925207 ₹5000 To 25K With AC Room 💚😋
VIP Hyderabad Call Girls Bahadurpally 7877925207 ₹5000 To 25K With AC Room 💚😋VIP Hyderabad Call Girls Bahadurpally 7877925207 ₹5000 To 25K With AC Room 💚😋
VIP Hyderabad Call Girls Bahadurpally 7877925207 ₹5000 To 25K With AC Room 💚😋
 

Improving Methods and Processes for Assessing Codependent Technologies

  • 1. Assessing the Value of Co-dependent Technologies: How Can Current Methods and Processes Be Improved? ScHARR Seminar, University of Sheffield 16 April 2013 ● Sheffield, UK Martina Garau, Office of Health Economics
  • 2. Adrian Towse (OHE) and the other authors of: Garau, M., Towse, A., Garrison, L., Housman, L. and Ossa, D. (2012) Can and should value based pricing be applied to molecular diagnostics? Personalized Medicine. 10(1), 61-72. Acknowledgements
  • 3. • What is the value of co-dependent technologies? • Framework for assessing value • How prove value? • How aggregate value dimensions? • Proposed institutional processes • International experience • Australia • NICE • Conclusions Agenda
  • 4. • “Technologies that are dependent on another technology either to achieve their intended effect or to enhance their intended effect” (www.health.gov.au) • In particular, a diagnostic test (Dx) can be used to identify patients most likely to: • Respond or fail to respond to a drug treatment (Tx) • Exhibit adverse events But also to: • Monitor responses to drugs • Determine the risk of developing a disease Definition of co-dependent technologies
  • 5. 1. Key elements of value are: • Health effects for patients (clinical effectiveness measured by the QALY) • Cost offsets (savings to the health care system) What is the value of co-dependent technologies? (1) Traditionally, ICERs do not capture benefits beyond health attributes NHS,PSS Cost of Treatment A - NHS,PSS Cost of Treatment B ICER = Health effects of Treatment A - Health effects of Treatment B The focus is on downstream effects of treatments not recognising the additional value brought by use of Dx
  • 6. • Other value dimensions • Societal preferences giving priority to certain patients or diseases • Quality of life aspects not reflected in generic measures used in CE analyses • Other effects beyond those to patients and NHS (productivity gains) • Health care process related aspects (dignity, time and location of treatment) • Information for the patient independent of health effects What is the value of co-dependent technologies? (2)
  • 7. • The value created is a “joint product” and there are no rules for the attribution of the value to one or the other • Garrison and Austin (2007) pointed out that how value is allocated across patients, payers, Dx manufactures and Tx manufacturers depends on the institutional context • E.g. whether the Tx was priced before the Dx was available; the relative strength of intellectual property protection for Dx and Tx • This will have consequences in terms of incentives for evidence generation and subsequent innovation The issue of attribution of value of co-dependent technologies
  • 8. Value 1. Reducing drug adverse effects 2. Reducing time delays in selecting optimal Tx 3.Increasing adherence or willingness to start Tx 4. Enabling Tx effective in a small fraction to be made available 5.Reducing uncertainty about value Framework for assessing value of co-dependent technologies Value dimensions derived from: • Characteristics of Dx recently introduced • Literature review on the economics of personalised medicine and value of information of Dx
  • 9. Availability of Dx can improve average benefit-risk ratio so, depending on the severity of side effects: • Tx obtains marketing authorisation, or • Use of a licensed Tx in clinical practice increases Example: HLA-B*5701 • Allele associated with hypersensitivity to abacavir for HIV-1 • Identification of the marker has increased prescribing of abacavir, which now is recommended for HLA-B*5701-negative patients in European and US guidelines 1. Reducing or avoiding drug adverse effects
  • 10. Identifying non-responders and switching them to an alternative treatment regime/care can: • Improve survival and/or quality of life (particularly in diseases at advanced stages) • Avoid or reduce the cost of treating non-responders • Avoid or reduce inconvenience to patients Example: BCR-ABL • Test identifies chronic myelogenous leukemia (CML) patients who are receiving treatment, but not responding to it • Can prevent the disease from progressing to blast crisis and death, and enables stopping first-line treatment when no longer effective 2. Reducing time delays in selecting optimal Tx
  • 11. • Patients are more motivated if they know (ex-ante) the intervention is likely to work • Issue of non-responders who might experience disutility (they can feel “left-behind”) Example: PreDx Diabetes Risk test • Test estimates the patient’s risk for developing Type 2 diabetes over the next five years • This can further encourage patients to follow a healthy lifestyle and take other preventive measures. 3. Increasing adherence or willingness to under- take Tx or other interventions
  • 12. A biomarker or other genetic characteristic allowing for patient stratification can: 1. “Rescue” Tx that otherwise may either not have been licensed or have been withdrawn 2. Increase the chance of a Tx meeting reimbursement criteria (if targeting responders improves cost -effectiveness) 3. Accelerate R&D process for Tx (if stratification ascertained at an early development stage) 4. Enabling Tx effective in a small fraction to be made available
  • 13. • Gefitinib for non-small-cell lung cancer (NSCLC) initially licensed, but withdrawn when Phase III failed to show a survival benefit. With the identification of EGFR mutations and its association with response rate to TKIs, gefitinib was approved in the EU and other markets in combination with the EGFR mutation test. • NICE recommended trastuzumab for advanced and early-stage breast cancer in HER2/neu positive patients identified with HER2/neu test. The Dx-Tx cost per QALY was found to be below the standard threshold. • Crizotinib targets a small subset of NSCLC patients with an ALK-positive molecular abnormality. The development of the ALK FISH test has accelerated the development process and increased the likelihood of crizotinib delivering health benefits and commercial value. 4. Enabling Tx effective in a small fraction to be made available – Examples
  • 14. • Uncertainty around expected health effects and costs; influences the risk of poor value for money for payers • Value of information to patients about their medical condition independent of the health outcome (Ash, et al, 1990) • “Empowerment” (Payne, et al, 2012) • Effect of reassurance (measured with EQ-5D?) (Kenen, 1996) • Lifestyle choices and planning (Lee, et al, 2010) • Example of Oncotype DX ® and MammaPrint ® • Multi-gene assays estimating the risk of recurrence in breast cancer patients following surgery • Can guide intervention decisions and reduce the risk of dispensing unnecessary chemotherapy (reduce resource costs to the healthcare system and adverse effect for the patient) 5. Reducing uncertainty about value
  • 15. • Low accuracy of Dx will decrease potential net gains to patients and healthcare system • False positive and false negative patients will not get most appropriate therapy • Tx can be more cost effective when used on its own • When Dx does not provide binary response, depending on the size of the subset for which the Dx does not provide clear-cut result and the Dx cost relative to Tx • When Dx has low accuracy Other factors affecting value of co-dependent technologies
  • 16. • Barriers to evidence generation • Cost and feasibility of certain study designs • Protection of intellectual property rights of Dx • Regulatory processes for diagnostics • Assessment of competitive tests with similar clinical use How prove value of co-dependent technologies?
  • 17. How is value aggregated? Key issues Key merits Net benefit As the sum of the benefits, each assessed in monetary terms Challenges estimating the value in monetary terms of each type of value Allocating a monetary value to health has been always one of the mayor criticisms Arguably, a better grounding in economic theory Facilitates the comparison of value and value for money across health and other sectors Use of monetary value may resonate better with some (private) payers MCDA As the sum of the points assigned to each aspect of value The cost -effectiveness threshold would need to be re-assessed in terms of the cost per incremental “point” A pragmatic approach, widely used in the UK public sector. A more transparent (compared to a weighted QALY, or deliberative process alone) means of addressing multiple criteria MCDA is used in local NHS commissioning – potential to develop a consistent priority-setting framework for both new and existing health care technologies Weighted adjusted QALYs 1. By QALYs gained, up-rated or down-rated by one or multiple weights to represent the magnitudes of other aspects of value; or 2. Direct estimation of how people trade off QALY gains with other value elements Assumes that all other sources of value are proportional to the number of QALYs gained. Implications for the threshold. If the value of new technologies is assessed in terms of a range of criteria, then opportunity cost also must be considered in the same terms, not just QALYs foregone. Even if a simple social weighting or QALYs is applied, opportunity cost will change Is it relevant to state here the classic arguments in favour of the QALY such as: - Allows for comparisons across therapeutic areas in the NHS - “A QALY is a QALY” argument - Well established in the UK within HTA bodies (and academic centres) - Understood by health economics community Deliberative process Weights are assigned by a committee to each relevant aspect of value The weights are often implicit Are implications for the threshold Provides an element of flexibility Is a well-recognised approach taken by HTA bodies around the world. How aggregate value dimensions? Source: adapted from Sussex, et al, 2013
  • 18. • A joint Dx-Tx review of “at launch” technologies; to be done by a drug committee to exploit synergies across Dx and Tx • However, there is a need to address the lack of expertise of most drug committees in the Dx area • A separate Dx committee to develop Dx-specific expertise and to assess multiple tests with similar clinical use • However, there may be a trade-off if there are not enough decisions to justify a distinct committee • A comprehensive and consistent approach to assessing value of both Dx and Tx Proposed institutional processes for co-dependent technologies
  • 19. Proposed institutional processes New Dx Dx linked to a Tx (companion Dx) Dx-Tx pair launched simultaneously Dx-Tx joint assessment via Drug process Single Dx launched separately Dx assessed via Diagnostic- dedicated process Multiple Dx with same clinical use Dx assessed via Diagnostic- dedicated process Dx not linked to a Tx Dx assessed via Diagnostic- dedicated process
  • 20. • Until recently, Dx and associated Tx assessed via different committees (MSAC and PBAC) • No clear structure for consideration of the interactions and benefits from joint use • New coordinated process and decision framework for “co- dependent technologies” • “Integrated” applications combining information from Dx and Tx manufacturers • Reimbursement decisions are made jointly by PBAC and MSAC to ensure optimal clinical use (Merlin, et al, 2012) • The preferred type of evidence to show clinical benefit is a randomised clinical trial International experience: Australia
  • 21. • NICE has dedicated-process for stand-alone Dx that follows very closely that used for drugs • Strong preference for measuring health gains with the QALY • Value dimensions beyond health effects, such as value of information to patients and process-related benefits, are not explicitly factored in • “At launch” combinations are appraised via the drug review programme (TAs) • No explicit consideration of test-related parameters (accuracy, costs) • Value dimensions beyond health effects are not explicitly factored in International experience: NICE in England and Wales
  • 22. • The use of Dx-Tx combinations can deliver health gains and cost savings within the health care system, but also generate broader benefits to patients and society • To ensure efficient use of limited resources, health decision makers should take account of the full value generated by health technologies • Clear incentives are needed to encourage evidence collection • HTA and other decision making systems need coordinated and consistent approach to assessing value of Dx and Tx • NICE is heading in this direction, but does not yet have a comprehensive approach to assessing the value of Dx or Tx • In Australia, the common methodology needs to be supported by a realistic view of evidence development Conclusions
  • 23. Ash, D.A., Patton, J.P. and Hershey, J.C. (1990) Knowing for the sake of knowing: The value of prognostic information. Medical Decision Making. 10(1), 47-57. Garau, M., Towse, A., Garrison, L., Housman, L. and Ossa, D. (2012) Can and should value based pricing be applied to molecular diagnostics? Personalized Medicine. 10(1), 61-72. Garrison, L.P. and Austin, M.J.F. (2007) The economics of personalized medicine: A model of incentives for value creation and capture. Drug Information Journal. 41(1), 501-509. Lee, D.W., Neumann, P.J. and Rizzo, J.A. (2010) Understanding the medical and nonmedical value of diagnostics testing. Value in Health. 13(2), 310-314. Merlin, T., Farah, C., Schubert, C., Mitchell, A., Hiller, J.E. and Ryan, P. (2012) Assessing personalized medicines in Australia: A national framework for reviewing codependent technologies. Medical Decision Making. 33(3), 333-342. Kenen, R.H. (1996) The at-risk health status and technology: A diagnostic invitation and the gift of knowing. Social Science & Medicine. 42(11), 1545-1553. Payne, K., McAllister, M. and Davies L. (2012) Valuing the economic benefits of complex interventions: When maximising health is not sufficient. Health Economics. 22(3), 258-271. Sussex, J., Towse, A. and Devlin, N. (2013) Operationalising value based pricing of medicines: A taxonomy of approaches. Pharmacoeconomics. 13(1), 1-10. References
  • 24. To enquire about additional information and analyses, please contact Martina Garau: mgarau@ohe.org To keep up with the latest news and research, subscribe to our blog, OHE News. Follow us on Twitter @OHENews, LinkedIn and SlideShare. Office of Health Economics (OHE) Southside, 7th Floor 105 Victoria Street London SW1E 6QT United Kingdom +44 20 7747 8850 www.ohe.org OHE’s publications may be downloaded free of charge for registered users of its website. ©2013 OHE

Notas del editor

  1. key pathways of value that the use of Dx to inform treatment or intervention decisions
  2. Dx can be available to select patients that are more or less likely to develop adverse effects  a. allow a treatment to receive marketing authorisation by improving the benefit-risk ratio associated with the treatment b. increase adoption of the treatment, in cases where a treatment is licensed, but is not widely used because of its perceived unfavourable average benefit-risk balance when considered across a broad patient population
  3. Avoid trial and error approach and identify the most suitable intervention First two points captured in CE analysis; last one notI will illustrate those points using the exampleit avoids or reduces inconvenience to patients who do not need to experience a long diagnostic process or try different therapies to identify the one most suitable
  4. Patients are more motivated if they know the intervention is likely to work. In the case of companion diagnostics, however, patients found to be non-responders might experience disutility as they can feel ‘left-behind’, and lose hope and even motivation to pursue any other, less effective, but appropriate therapy.
  5. Dx to stratify patientsThree cases where Dx have a positive impact and it is introduced in the market at different stages of the Tx lifecyclepatient stratification in oncology clinical trials could reduce attrition rates in overall clinical development and, in particular, attrition rates from Phase II to Phase III
  6. Measured by EQ5D but may not be captured as patients focus on Tx effects rather than on the overall experience of Dx-Tx
  7. Those were the five pathways through which co-dependent technologies such as Dx and Tx can generate value as compared to a situation where the intervention is used on its own
  8. Only third poikey issue for the assessment of co-dependent technologies is that they are perceived as a joint product so there is no an approach to allocate the value brought by each part.) I then discussed a framework identifying how Dx can bring additional value to Dx-Tx pairs. Here I discuss briefly which the type of process can help ensuring those elements are assessed and considered in the HTA or P&R system. nt
  9. Guide for submission sets a high standard of evidence to demonstrate impact of the test on patient outcomesWhich raises important questions as to how value should be demonstrated and who can generate the evidence