Machine learning can be applied through the development of algorithms that can unravel or "learn" complex associations in large datasets with limited human input. These algorithms are capable of making predictions that go beyond our capabilities as humans and they can process and analyze more possibilities. Machine learning may help us find answers to questions that we didn't even think of in the past, revealing evidence previously hidden among the data. We can use these methods to dig up imperceptible patterns and allow health technologies to be used at the right time and for the right patient population. (A4 Version)

1. MACHINE LEARNING AND
THE VALUE OF HEALTH
TECHNOLOGIES

2. Face-to-face with machine learning
“Because you watched The Crown.” This simple statement gets us to keep
browsing and watching Netflix, despite us often not knowing that the suggested
films and documentaries even existed. Yet what underlies this simple statement
is remarkably complex – it is generated by a sophisticated algorithm trying to
predict what we are likely to enjoy watching based on our previous actions. That’s
the elegance of Netflix’s algorithm. Complexity presented as simplicity, effectively
and with finesse. Netflix represents a relatively new example of what machine
learning can achieve, and may serve as illustration to future generations of how
machine learning entered our daily lives. But Netflix is only one example among
many. Machine learning can estimate road traffic for Google Maps, recognise
our speech, and can even identify the location of a failure in a city’s piping system by
considering water flow data.
Machine learning can be applied through the development of algorithms
that can unravel or ‘learn’ complex associations in large datasets with limited
human input.1,2
These algorithms are capable of making predictions that go beyond
our capabilities as humans as they can process and analyse more
possibilities than we can. And that’s where – perhaps – the greatest
potential of machine learning lies.1,2
The uptake of these state-of-the-art methods in healthcare has not been as quick
as in other fields; still, some examples exist of machine learning algorithms being
used for the prevention, diagnosis and risk stratification of patients in a number
of disease areas.2,3
The application of machine learning for demonstrating the
value of health technologies is at an even earlier stage, although the increase in
research papers and abstracts on this topic in the last few years suggests uptake
may be gathering momentum. The implementation of these methods, however,
will require considerable effort as expert knowledge and particular skillsets need to be
developed. Is this extra effort really worth it?

3. How can machine learning be used to
demonstrate or add value?
New technologies and ever-increasing computational power enable us to gather and
store large amounts of data. We often do not make the most of these data because such
datasets are not easy to interpret, or because they come from different sources and in
various formats. Cleaning and manipulating these large datasets can be a monumental
task. For this reason, it’s more important than ever to provide decision-makers with
tools that can facilitate their work and optimise the allocation of resources in the
healthcare system.2
Treatment pathway optimisation. The likelihood of a health technology being available
to patients will increase if the target population is appropriately defined, i.e., if the
health technology is placed in the treatment pathway where its greatest benefit can be
realised and demonstrated. Machine learning algorithms can help determine patient
subgroups with a significant treatment benefit, thereby establishing where in the
treatment pathway a product will maximise patients’ outcomes. Researchers have
tried, for example, to understand which metastatic melanoma patients can be cured
by a particular treatment.4
This type of information can, according to the researchers,
enable a better assessment of the cost-effectiveness and budget impact of new
technologies, optimising the use of resources available in the healthcare system.4
Other
examples in the recent literature include the prediction of treatment benefit in multiple
myeloma and gastric cancer.5,6
Decode complex associations. With the increasing availability of extensive datasets
containing large numbers of variables, machine learning methods represent our
best choice for deciphering intricate data patterns. This can lead to, for example, the
development of disease models that can be used for cost-effectiveness and budget
impact modelling. Chronic obstructive pulmonary disease (COPD) and systemic lupus
erythematosus (SLE) are two diseases for which health economic models have been
developed based on mathematical equations describing complex relationships between
biochemical factors, clinical indicators and patient outcomes.7,8
In SLE, data from
clinical trials and a comprehensive registry were used to predict and compare patients’
costs and outcomes with belimumab compared with the standard of care.8
This
model structure was accepted by health technology assessment (HTA) agencies such
as the National Institute for Health and Care Excellence (NICE); the manufacturer’s
submission resulted in a recommendation for belimumab in a particular SLE
population.8
So, how can machine
learning help us?

4. Software that strengthens a product’s value proposition. We can use machine
learning algorithms to develop software that can go hand-in-hand with health
technologies, leading to a synergistic increase in the value of the combination.
For example, Zhao et al. explored several machine algorithms to predict the
duration of robot-assisted surgeries (RAS) in a hospital.9
RAS requires a
large investment and, therefore, when a hospital acquires such technology,
hospital managers need to make sure that its use is maximised. In this
case, the authors demonstrated that machine learning algorithms made
better predictions of case durations compared with traditional estimation
procedures (i.e., previous case duration averages and surgeon adjustments).
If implemented in the real world, these algorithms would help optimise the
resources available in the hospital, increasing the value of RAS.9
Shared decision-making. There is an increasing interest and need to promote
patient-centric policies in our healthcare systems. Wouldn’t it be great to
be able to identify the likely outcomes of a particular patient treated with a
particular treatment? For instance, we could try to predict whether a patient
is likely to advance to a progressed cancer stage with a given treatment,
considering a set of baseline clinical and genetic characteristics. This is what
is meant by personalised medicine, and machine learning methods will
undoubtedly help us achieve this.3
Assess the perception of a product. When trying to demonstrate or
understand the value of a health technology, either before or after entering
the market, it is crucial to evaluate different stakeholders’ perceptions and
get answers to questions such as: is this technology providing the benefit
you expected? What do you or don’t you like about the technology? Machine
learning algorithms can be used to analyse social networking sites or
interviews and help improve a product’s value proposition or better define the
product’s target population. This is what is known as sentiment analysis. For
instance, Roccetti et al. extracted and analysed the data posted on Facebook
by Crohn’s disease patients to understand their opinions on the treatment
with infliximab.10
Also, Korkontzelos et al. assessed the value of sentiment
analysis for the identification of adverse events expressed by patients in social
media, highlighting the potential of machine learning in pharmacovigilance.11
These examples show that sentiment analysis could be used to, among other
applications, inform a product’s value proposition and positioning in the
treatment pathway; however, its full potential in healthcare has not yet been
demonstrated.

5. Figure 1. Examples of machine learning algorithms and their applications
Machine learning algorithms have the potential to deal with various data analysis
problems, with many different objectives. Data scientists can choose from a wide
array of algorithms with different levels of complexity, from commonly used
decision trees to cutting-edge neural networks and deep learning, to address the
problem they are considering (Figure 1).1,2
Increasedalgorithmcomplexity
Neural networks
Prediction of the duration of RAS for each particular
patient in a hospital to optimise the use of resources9
(example of software that adds value)
Gradient boosting
Prediction of the probability of being cured with a
particular treatment in metastatic melanoma patients4
(example of treatment pathway optimisation)
Simulated treatment learning
Estimation of benefit in multiple myeloma patients
not treated with the therapy of interest5
(example of
treatment pathway optimisation)
Random forests
Identification of clinical factors contributing to
disease progression in patients with gastric cancer
after nivolumab treatment12
(example of treatment
pathway optimisation)

6. The examples discussed in Figure 1 illustrate the potential of machine learning
in healthcare and, in particular, for demonstrating and adding value to health
technologies. However, the field is advancing rapidly, with new methods and
potential applications coming to light every day.
Machine learning may help us find answers to questions that we
didn’t even think of in the past, revealing evidence previously
hidden among the data. We can use these methods to dig up
imperceptible patterns and allow health technologies to be used at
the right time and in the correct patient population.
There is plenty of room for innovative thinking that might challenge our previous
expectations of how data can help inform decision-making around healthcare
technologies.
What are the challenges with machine
learning?
The application of machine learning is not, however, free of challenges, and
some of them may not be straightforward to handle. Each approach has its
own technical difficulties that expert data scientists can help to handle. Other
challenges include:
Choice of algorithm. There is a wide variety of algorithms available,1
so which one should we use? Due to the relative infancy of machine
learning application in healthcare, and, therefore, a lack of robust
experience, it’s not obvious whether certain algorithms perform
better than others in given situations. That’s why many researchers
implement a range of models and compare their performance in
order to choose the best.9,13
Issues such as over- or under-fitting and
bias also need to be considered when choosing between various well-
performing algorithms.2
Acceptability. Machine learning algorithms often outperform
classic statistical models; but, on the contrary, machine learning
models tend to be less intuitive than classic statistical models.1,13
For this reason, the acceptability of these algorithms among
key stakeholders, such as clinicians, represents a considerable
challenge that must be overcome.3
But we must be clear: the
objective should be to assist experts in making a decision rather
than the algorithm making the decision.

7. Ethics. There are ethical considerations that may come into play.2,3
For
example, what would happen if a model incorrectly predicted that a person
is not sick? Who would be responsible for the error? Challenges related to
data security and privacy may also be prominent. With regard to privacy and
equity, for instance, researchers have shown that insurance companies could
use machine learning methods to identify unprofitable enrolees, highlighting
that vulnerable patients may require special protection from regulators to
avoid being subjected to unfair policies.14
With machine learning becoming increasingly important in our everyday lives, it’s only a
matter of time before its application in healthcare is more commonplace. Now is the time
for open discussion to understand how machine learning can help us make the “right”
healthcare choices. For the patient, the “right” choices allows them access to best available
technologies for their condition and circumstances; for payers, the “right” choices enable
them to deliver the best healthcare to a population given the resources available. Machine
learning might help ease the tensions that exist between these viewpoints by helping us
better understand in which patients and in what application the greatest benefits of a
health technology might be realised. Through this, machine learning has the potential to
improve the healthcare of our societies as a whole, and all of us have the ability to shape it
by embracing what is coming.

8. www.covance.com/marketaccess
Covance Inc., headquartered in Princeton, NJ, USA, is the drug development business of
Laboratory Corporation of America Holdings (LabCorp). COVANCE is a registered trademark and
the marketing name for Covance Inc. and its subsidiaries around the world.
The Americas +1.888.COVANCE (+1.888.268.2623) +1.609.452.4440
Europe / Africa +00.800.2682.2682 +44.1423.500888
Asia Pacific +800.6568.3000 +65.6.5686588
© Copyright 2019 Covance Inc. WPCMA008-0319
References
1. Beam AL et al. JAMA 2018;319(13):1317–1318.
2. Topol EJ. Nat Med 2019;25(1):44-56.
3. Bodas-Sagi DJ et al. IJIMAI 2017;4(7):47-52.
4. Felizzi F et al. Evaluating budget impact of cures. Machine learning methods shed light on who can be cured in
metastatic melanoma. Poster presented at ISPOR Europe 2018, Barcelona, Spain, Nov 10-14, 2018.
5. Ubels J et al. Nat Commun. 2018;9(1):2943.
6. Xu R et al. Prognostic classifier for stage II gastric cancer based by support vector machine. Poster presented at 2011
ASCO Annual Meeting, Chicago, Illinois, US, Jun 3-7, 2011.
7. Exuzides A et al. Med Decis Making 2017;37(4):453–468.
8. National Institute for Health and Care Excellence. 2016 Jun 6. Technology Appraisal Guidance 397. Belimumab for
treating active autoantibody-positive systemic lupus erythematosus. Jun 2016.
9. Zhao B et al. J Med Syst 2019;43(2):32.
10. Roccetti M et al. JMIR Public Health Surveill 2017 Jul-Sep;3(3): e51.
11. Korkontzelos I et al. J Biomed Inform 2016 Aug;62:148-58.
12. Kang YK et al. JCO 2019;37(4):suppl8.
13. Huber M et al. BMC Med Inform Decis Mak 2019;19(1):3.
14. Rose S et al. Biostatistics. 2017 Oct;18(4):682–694.