The Rise of Machine-Learned Features

2. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
2
Source: https://www.youtube.com/watch?v=eWSbIXSbMis
AI vs AI
Ryu is
played by AI
that learned
by itself to
play

3. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-
artificial-intelligence-machine-learning-deep-learning-ai/
The AI time line
A broad overview
3

4. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-
artificial-intelligence-machine-learning-deep-learning-ai/
The AI time line
The starting point ...
a series of
if-then-
else rules
4

5. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source:
https://wiki.bath.ac.uk/display/BISAI/Expert+System+in+a+Gaming+En
viroment
Expert system
Example: Arriving
decision to
choose a weapon
at any time by the
expert system
Rule behind
choosing mini gun
Rule behind
choosing missile
Rule behind
choosing defensive
pulse
5

6. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
6
Source: https://www.youtube.com/watch?v=eWSbIXSbMis
AI vs AI
The actions
by E Honda
were
determined
by rules

7. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://yachtharbour.com/news/venus-spotted-in-mallorca-
1887?src=news_view_page_bar
Expert system
Rules to determine ...
a yacht?
7

8. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Expert system
The limitations
•Expensive and time consuming!
Experts are never cheap
•Bad in handling sophisticated
sensory inputs (like signals,
images)
•Possible to make dumb decision
since it just goes through rules;
no common sense in the system
•System not easy to be updated
Source: https://blogs.nvidia.com/blog/2016/07/29/whats-difference-
artificial-intelligence-machine-learning-deep-learning-ai/
8

9. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://verhaert.com/difference-machine-learning-deep-
learning/
Features + classifier
A new solution to the rescue
•Feature: a number or a vector
that describes something about
the input
•Machine learning: classifier
learns the pattern between
features and output
9

10. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://towardsdatascience.com/face-recognition-how-
lbph-works-90ec258c3d6b
Features + classifier
Features?
•Face recognition
10

11. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Features + classifier
Issues?
•We need to design features
manually, through much trial and
error, with luck
•Simple features unlikely to work;
need complicated feature
extraction
•Classifiers used are generic (like
SVM)
11
Source: https://towardsdatascience.com/face-recognition-how-
lbph-works-90ec258c3d6b

12. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://www.guru99.com/machine-learning-vs-deep-
learning.html
Learning the features
Better performance?
•Instead of we deciding the features,
is it possible to get algorithm to
learn to extract the most
appropriate features by itself?
•Series of feature extractors?
•All the way from pixels to classifier,
learn by itself, layer by layer?
•Train all the layers together？
12

13. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: http://cs231n.stanford.edu
Deep in action
Part 1
conv ReLU conv ReLU pool conv ReLU
13

14. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: http://cs231n.stanford.edu
conv ReLU conv ReLU
pool conv ReLU pool
ship
car
airplane
truck
bird
FC
14
Deep in action
Part 2

15. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: https://commons.wikimedia.org/wiki/File:Typical_cnn.png
Learning the features
The idea behind convolutional
neural network (convnet)
Feature Extraction
Classification
15

16. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Source: http://www.image-
net.org/challenges/LSVRC/2012/results.html
From then on …
ILSVRC 2012
Rank Error - 5 Algorithm Team
1 0.153 Deep convolutional neural network University of Toronto
2 0.262 Features + Fisher vectors + linear classifiers ISI
3 0.270 Features + Fisher vectors + SVM Oxford VGG
4 0.271 Not specified XRCE/INRIA
5 0.300
Dense SIFT + colour SIFT + Fisher vectors +
SVM
University of Amsterdam
16
•ILSVRC: ImageNet Large Scale
Visual Recognition Challenge
•Challenge: Correctly classify
150,000 images, 1000
categories

17. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
The progress
So far
17

18. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
18
Source:
http://www.williammalone.com/articles/create-
html5-canvas-javascript-drawing-app/
Application
Three main categories
Source:
https://bitsandatoms.co/tag/reinforcement-
learning/
Source: http://blog.ss8.com
Create
Identify
Act

19. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
19
Volvo in-car cameras
Combat distraction and drunk
driving
•'Pro-active' driver monitoring
system
Source: https://www.caranddriver.com/news/a26893035/volvo-interior-cameras-
distraction-drunk-driving/

20. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
20
Aircraft
Monitoring ground handling
Source: https://assaia.com/tmc/

21. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
21
Aircraft
Monitoring check-in
Source: https://assaia.com/tmc/

22. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
22
Behaviour
identification
by Aipoly
Source: https://www.aipoly.com

23. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
23
Object detection
Auto referee?
•Track ball and players
•Report score based on the events
in scene
Source: https://arxiv.org/abs/2004.09927

24. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
24
To create

25. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
25
Drawing
A sketch
tree
river
rock
water
cloud
sky

26. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
26
Source: https://blogs.nvidia.com/blog/2019/03/18/gaugan-
photorealistic-landscapes-nvidia-research/
Drawing
A sketch

27. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
27
Source: https://blogs.nvidia.com/blog/2019/03/18/gaugan-
photorealistic-landscapes-nvidia-research/
Drawing
GauGAN (inspired by Gauguin)

28. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
28
Practical
usage
fashion
Source: https://qz.com/1090267/artificial-intelligence-can-now-
show-you-how-those-pants-will-fit/
A B C D
E F G H

29. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
29
Generative Adversarial
Networks
Change face

30. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
30
Generative Adversarial
Networks
Change face

31. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
31
Change face
Unlimited possibilities
Source: https://app.generative.photos

32. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
32
How about 3D?
Face?
Source: http://cvl-demos.cs.nott.ac.uk/vrn/index.php

33. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
33
To act

34. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
34
Source: https://towardsdatascience.com/using-deep-q-learning-in-fifa-
18-to-perfect-the-art-of-free-kicks-f2e4e979ee66
Freekick please
Fifa 18

35. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
35
Source: https://towardsdatascience.com/using-deep-q-learning-in-fifa-
18-to-perfect-the-art-of-free-kicks-f2e4e979ee66
Deep Reinforcement
learning
Fifa 18

36. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
36
Source: https://www.youtube.com/watch?v=FCivsotZEjk
Picking strawberries
no human

37. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
37
Source: https://www.tesla.com/autopilotAI
Autonomous
vehicle
Tesla autopilot

38. © 2021 National University of Singapore.
All Rights Reserved.
govtechbrbg01/v1.0
of 38
Conclusion •The AI that uses machine-learned
features (also known as deep
learning) is driving the current AI
boom
•Many applications not possible in
the past are made possible through
deep learning
•There are many uses of deep
learning to be imagined and
explored
38