Machine Learning is the study of how to build systems that can automatically learn and improve with experience similar to humans. Since the early 1980’s there have been significant advances in ML that have affected things such as marketing, banking, manufacturing, household appliances, automobiles, medicine and health care, and most recently the Internet and mobile devices. Machine Learning is poised to extend human mental reach in the virtual world of the 21st century in the same way as flight extended our physical reach in 20th century – it provides the means to filter massive amounts of data, recognize complex patterns, and rapidly make difficult decisions. Danny is a Professor in and the Director of the Jodrey School of Computer Science at Acadia University. His areas of research and development are machine learning, data mining, and adaptive systems. He has published over 60 scientific papers, edited special journal editions, and has been part of the program committee for a number of national and international conferences, seminars and workshops. Most recently he was awarded a Harrison McCain Foundation Award for research into advance machine learningmethods. Since 1993, he has worked on machine learning and data mining projects in the private and public sector providing situation analysis and problem definition, project management and guidance, and predictive analytic services. In 2011, he received the Science Champion Award from the Nova Scotia Discovery Center for his work on youth robotics and the advancement of STEM education.

1. Intelligent Information Technology Research Lab, Acadia University, Canada
1
Daniel L. Silver
Acadia University,
Wolfville, NS, Canada
danny.silver@acadiau.ca

2. Intelligent Information Technology Research Lab, Acadia University, Canada
2

3. Intelligent Information Technology Research Lab, Acadia University, Canada
Key Take Away
 A major challenge in artificial intelligence has
been how to develop common background
knowledge
 Machine learning systems are beginning to
make head-way in this area
 Taking first steps to capture
knowledge that can be used
for future learning, reasoning,
etc.
3

4. Intelligent Information Technology Research Lab, Acadia University, Canada
Outline
 Learning – What is it?
 History of Machine Learning
 Framework and Methods
 ML Application Areas
 Recent and Future Advances
 Challenges and Open Questions
4

5. Intelligent Information Technology Research Lab, Acadia University, Canada
What is Learning?
 Animals and Humans
a. Learn using new experiences and prior
knowledge
b. Retain new knowledge from what is learned
c. Repeat starting at 1.
 Essential to our survival and thriving
5

6. Intelligent Information Technology Research Lab, Acadia University, Canada
What is Learning?
(A little more formally)
 Inductive inference/modeling
 Developing a general model/hypothesis from
examples
 Objective is to achieve good generalization for
making estimates/predictions
 It’s like … Fitting a curve to data
 Also considered modeling the data
 Statistical modeling
7

7. Intelligent Information Technology Research Lab, Acadia University, Canada
What is Learning?
 Generalization through learning is not
possible without an inductive bias
= a heuristic beyond the data

8. Intelligent Information Technology Research Lab, Acadia University, Canada
9
Inductive Bias
ASH ST
THI RDSEC OND
ELM ST
FIR ST
PINE ST
OAK ST
Inductive bias depends upon:
• Having prior knowledge
• Selection of most related
knowledge
Human learners use Inductive Bias

9. Intelligent Information Technology Research Lab, Acadia University, Canada
What is Learning?
 Requires an inductive bias
= a heuristic beyond the data
 Do you know any inductive biases?
 How do you choose which to use?

10. Intelligent Information Technology Research Lab, Acadia University, Canada
Inductive Biases
 Universal heuristics - Occam’s Razor
 Knowledge of intended use – Medical
diagnosis
 Knowledge of the source - Teacher
 Knowledge of the task domain
 Analogy with previously learned tasks
Tom Mitchell, 1980

11. Intelligent Information Technology Research Lab, Acadia University, Canada
What is Machine Learning?
The study of how to build computer
programs that:
 Improve with experience
 Generalize from examples
 Self-program, to some extent

12. Intelligent Information Technology Research Lab, Acadia University, Canada
History of Machine Learning
1950 20001980
PDP Group
Multi-layer
Perceptrons,
New apps
Renaissance
1990
AI Success
Data mining,
Web mining,
User models,
New alg.,
Google
Present
Big Data,
Web Analytics,
Parallel alg.,
Cloud comp.,
Deep learning
Advances
1890
William
James,
Neuronal
learning
Origins
1940
Donald Hebb,
Math models,
The Perceptron
Limited value
Promise
1960
Minsky &
Papert
paper,
Research
wanes
Hiatus
1970
Genetic alg,
Version
spaces,
Decision
Trees
Exploration

13. Intelligent Information Technology Research Lab, Acadia University, Canada
Of Interest to Several Disciplines
 Computer Science – theory of computation, new
algorithms
 Math - advances in statistics, information theory
 Psychology – as models for human learning, knowledge
acquisition and retention
 Biology – how does a nervous system learn
 Physics – analogy to physical systems
 Philosophy – epistemology, knowledge acquisition
 Application Domains – new knowledge extracted from
data, solutions to unsolved problems
17

14. Intelligent Information Technology Research Lab, Acadia University, Canada
Classes of ML Methods
 Supervised – Develops models that predict the value of
one variable from one or more others:
 Artifical Neural Networks, Inductive Decision Trees, Genetic
Algorithms, k-Nearest Neighbour, Bayesian Networks, Support
Vectors Machines
 Unsupervised – Generates groups or clusters of data
that share similar features
 K-Means, Self-organizing Feature Maps
 Reinforcement Learning – Develops models from the
results of a final outcome; eg. win/loss of game
 TD-learning, Q-learning (related to Markov Decision Processes)
 Hybrids – eg. semi-supervised learning

15. Intelligent Information Technology Research Lab, Acadia University, Canada
Focus: Supervised Learning
 Function approximation
(curve fitting)
 Classification (concept learning, pattern
recognition)
x1
x2
A
B
f(x)
x
21

16. Intelligent Information Technology Research Lab, Acadia University, Canada
23
Supervised Machine Learning
Framework
Instance Space
X
Training
Examples
Testing
Examples
(x, f(x))
Model of
Classifier
h
Inductive
Learning System
h(x) ~ f(x)

17. Intelligent Information Technology Research Lab, Acadia University, Canada
Supervised Machine Learning
 Problem: We wish to learn to classifying two people
(A and B) based on their keyboard typing.
 Approach:
 Acquire lots of typing examples from each person
 Extract relevant features - representation!
 M = number of mistakes
 T = typing time
 Transform feature representation as needed
 Use an algorithm to fit a model to the data - search!
 Test the model on an independent set of examples of typing from
each person

18. Intelligent Information Technology Research Lab, Acadia University, Canada
Classification
Mistakes
Typing Speed
A
B
B
B
B
B
B
B
BB
B
B
B
B
B
B
B
B B
B
B
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
B
B
B
B
B
B
B
B
B
Logistic Regression
Y
Y=f(M,T)
0
1
M T
Y

19. Intelligent Information Technology Research Lab, Acadia University, Canada
Classification
A
B
B
B
B
B
B
B
BB
B
B
B
B
B
B
B
B B
B
B
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
B
B
B
B
B
B
B
B
B
Artificial Neural Network
A
Mistakes
Typing Speed
M T
Y
…

20. Intelligent Information Technology Research Lab, Acadia University, Canada
Classification
A
B
B
B
B
B
B
B
BB
B
B
B
B
B
B
B
B B
B
B
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
A
B
B
B
B
B
B
B
B
B
Inductive Decision Tree
A
A
Mistakes
Typing Speed
M?
T? T?
Root
LeafAB
Blood Pressure Example

21. Intelligent Information Technology Research Lab, Acadia University, Canada
Application
Areas
Data Mining:
 Science and medicine: prediction, diagnosis, pattern
recognition, forecasting
 Manufacturing: process modeling and analysis
 Marketing and Sales: targeted marketing, segmentation
 Finance: portfolio trading, investment support
 Banking & Insurance: credit and policy approval
 Security: bomb, iceberg, fraud detection
 Engineering: dynamic load shedding, pattern recognition
31

22. Intelligent Information Technology Research Lab, Acadia University, Canada
Application Areas
 Web mining – information filtering and classification,
social media predictive modeling
 User Modeling – adaptive user interfaces,
speech/gesture recognition
 Intelligent Personal Agents – email spam
filtering, fashion consultant,
 Robotics – image recognition, adaptive control,
autonomous vehicles (space, under-sea)
 Military/Defense – target acquisition and classification,
tactical recommendations, cyber attack detection
32

23. Intelligent Information Technology Research Lab, Acadia University, Canada
Recent and Future Advances
 Robotics
 Neuroprosthetics
 Lifelong Machine Learning
 Deep Learning Architectures
 ML and Growing Computing Power
 NELL – Never-Ending Language Learner
 Cloud-based Machine Learning
33

24. Intelligent Information Technology Research Lab, Acadia University, Canada
OASIS: Onboard Autonomous
Science Investigation System
 Since early 2000’s
 Goal: To evaluate,
and autonomously
act upon, science
data gathered by
spacecraft
 Including planetary
landers and rovers
34

25. Intelligent Information Technology Research Lab, Acadia University, Canada
 Stanford’s Sebastian Thrun holds a $2M check on top of
Stanley, a robotic Volkswagen Touareg R5
 212 km autonomus vehicle race, Nevada
 Stanley completed in 6h 54m
 Four other teams also finished
Source: Associated Press – Saturday, Oct 8, 2005
DARPA Grand
Challenge 2005
35

26. Intelligent Information Technology Research Lab, Acadia University, Canada
The Competition
36

27. Intelligent Information Technology Research Lab, Acadia University, Canada
Autonomous Underwater Vehicles
Arctic Explorer
AUV designed and built by International
Submarine Engineering Ltd. (ISE) of Port
Coquitlam, B.C.
Used to map the sea floor underneath the
Arctic ice shelf in support of Canadian land
claims under the UN Convention on the
Law of the Sea.
Various military uses; e.g. mine detection,
elimination
(Source: ISE, Mae Seto)
37

28. Intelligent Information Technology Research Lab, Acadia University, Canada
Literally Extending Our Reach
– Neuroprosthetic Decoders
 Dec, 2012
 Andy Schwart,
Univ. of Pittsburgh
 Jan Scheuermann,
quadriplegic
 Brain-machine
interface, 96
electrodes
 13 weeks of
training
 High-performance neuroprosthetic
control by an individual with tetraplegia,
The Lancet, v381, p557-654, Feb 2013
39

29. Intelligent Information Technology Research Lab, Acadia University, Canada
40
Lifelong Machine Learning (LML)
 Considers methods of retaining and using
learned knowledge to improve the effectiveness
and efficiency of future learning
 We investigate systems that must learn:
 From impoverished training sets
 For diverse domains of tasks
 Where practice of the same task happens
 Applications:
 Intelligent Agents, Robotics, User Modeling, DM

30. Intelligent Information Technology Research Lab, Acadia University, Canada
41
Supervised Machine Learning
Framework
Instance Space
X
Training
Examples
Testing
Examples
(x, f(x))
Model of
Classifier
h
Inductive
Learning System
h(x) ~ f(x)
After model is developed
and used it is thrown away.

31. Intelligent Information Technology Research Lab, Acadia University, Canada
42
Lifelong Machine Learning
Framework
Instance Space
X
Training
Examples
Testing
Examples
(x, f(x))
Model of
Classifier
h
Inductive
Learning System
short-term memory
h(x) ~ f(x)
Domain
Knowledge
long-term memory
Retention &
Consolidation
Inductive
Bias Selection
Knowledge
Transfer

32. Intelligent Information Technology Research Lab, Acadia University, Canada
43
Lifelong Machine Learning
Framework
Instance Space
X
Training
Examples
Testing
Examples
(x, f(x))
Model of
Classifier
h
Inductive
Learning System
short-term memory
h(x) ~ f(x)
Domain
Knowledge
long-term memory
Retention &
Consolidation
Inductive
Bias Selection
Knowledge
Transfer

33. Intelligent Information Technology Research Lab, Acadia University, Canada
44
Lifelong Machine Learning
One Implementation
Instance Space
X
Training
Examples
Testing
Examples
(x, f(x))
Model of
Classifier
h
h(x) ~ f(x)
Retention &
ConsolidationKnowledge
Transfer
f2(x)
x1 xn
f1(x) f5(x)
Multiple Task
Learning (MTL)
Inductive
Bias Selection
f3(x)f2(x) … f9(x) fk(x)
Consolidated
MTL
Domain
Knowledge
long-term memory

34. Intelligent Information Technology Research Lab, Acadia University, Canada
48
An Environmental Example
Stream flow rate prediction [Lisa Gaudette, 2006]
x = weather data
f(x) = flow rate
11
12
13
14
15
16
0 1 2 3 4 5 6
Years of Data Transfered
MAE(m^3/s)
No Transfer Wilmot Sharpe Sharpe & Wilmot Shubenacadie

35. Intelligent Information Technology Research Lab, Acadia University, Canada
Lifelong Machine Learning
with csMTL
Example:
 Learning to Learn how
to transform images
 Requires methods of
efficiently & effectively
 Retaining transform
model knowledge
 Using this knowledge to
learn new transforms
(Silver and Tu, 2010)
52

36. Intelligent Information Technology Research Lab, Acadia University, Canada
Lifelong Machine Learning
with csMTL
55Demo

37. Intelligent Information Technology Research Lab, Acadia University, Canada
Deep Learning Architectures
 Hinton and Bengio (2007+)
 Learning deep architectures of neural
networks
 Layered networks of unsupervised auto-
encoders efficiently develop hierarchies
of features that capture regularities in
their respective inputs
 Used to develop models for families of tasks
57

38. Intelligent Information Technology Research Lab, Acadia University, Canada
Deep Learning Architectures
 Consider the problem of trying to classify
these hand-written digits.

39. Intelligent Information Technology Research Lab, Acadia University, Canada
Deep Learning Architectures
2000 top-level artificial neurons2000 top-level artificial neurons
00
500 neurons
(higher level features)
500 neurons
(higher level features)
500 neurons
(low level features)
500 neurons
(low level features)
Images of
digits 0-9
(28 x 28 pixels)
Images of
digits 0-9
(28 x 28 pixels)
11 22 33 44
55 66 77 88 99
Neural Network:
- Trained on 40,000 examples
- Learns:
* labels / recognize images
* generate images from labels
- Probabilistic in nature
- Demo
2
3
1

40. Intelligent Information Technology Research Lab, Acadia University, Canada
ML and Computing Power
 Moores Law
 Expected to
accelerate as the
power of computers
move to a log scale
with use of multiple
processing cores
60

41. Intelligent Information Technology Research Lab, Acadia University, Canada
ML and Computing Power
 IBMs Watson – Jeopardy, Feb, 2011:
 Massively parallel data processing system capable
of competing with humans in real-time question-
answer problems
 90 IBM Power-7 servers
 Each with four 8-core processors
 15 TB (220M text pages) of RAM
 Tasks divided into thousands of stand-alone
jobs distributed among 80 teraflops (1 trillion ops/sec)
 Uses a variety of AI approaches including
machine learning
61

42. Intelligent Information Technology Research Lab, Acadia University, Canada
ML and Computing Power
Andrew Ng’s work on Deep
Learning Networks (ICML-2012)
Problem: Learn to recognize human
faces, cats, etc from unlabeled data
Dataset of 10 million images; each
image has 200x200 pixels
9-layered locally connected neural
network (1B connections)
Parallel algorithm; 1,000 machines
(16,000 cores) for three days
62
Building High-level Features Using Large Scale Unsupervised Learning
Quoc V. Le, Marc’Aurelio Ranzato, Rajat Monga, Matthieu Devin, Kai Chen,
Greg S. Corrado, Jeffrey Dean, and Andrew Y. Ng
ICML 2012: 29th International Conference on Machine Learning, Edinburgh,
Scotland, June, 2012.

43. Intelligent Information Technology Research Lab, Acadia University, Canada
ML and Computing Power
Results:
 A face detector that is 81.7%
accurate
 Robust to translation, scaling,
and rotation
Further results:
 15.8% accuracy in recognizing
20,000 object categories from
ImageNet
 70% relative improvement over
the previous state-of-the-art.
63

44. Intelligent Information Technology Research Lab, Acadia University, Canada
Never-Ending Language Learner
 Carlson et al (2010)
 Each day: Extracts information from the
web to populate a growing knowledge
base of language semantics
 Learns to perform this task better than on
previous day
 Uses a MTL approach
in which a large number
of different semantic
functions are trained
together
64

45. Intelligent Information Technology Research Lab, Acadia University, Canada
Cloud-Based ML - Google
69
https://developers.google.com/prediction/

46. Intelligent Information Technology Research Lab, Acadia University, Canada
Machine Flight vs.
Machine Learning
71
Factor Machine Flight Machine Learning
Effectiveness Travel higher, father Learn more things, accurately
To places not reachable Model complex phenomena
Efficiency Travel faster Learn faster
Lower cost Lower cost
Satisfaction Safe travel, beauty Confidence, elegance
Reach the moon,
and beyond
Reach new knowledge,
solve new problems

47. Intelligent Information Technology Research Lab, Acadia University, Canada
72
Thank You!
 danny.silver@acadiau.ca
 http://plato.acadiau.ca/courses/comp/dsilver/
 http://ML3.acadiau.ca