Presentation slides from my Machine Learning Live talk at the 2012 Clojure Conj

1. Machine Learning Live
Clojure Conj 2012
Mike Anderson
@mikera mike@nuroko.com

2. Machine Learning – A definition
"Field of study that gives computers
the ability to learn without being
explicitly programmed.“
Arthur Samuel, 1959
Source: Good Old Wikipedia 2

3. Learning = Building functions from experience
Task Input Output
Simple mathematical x y = sin(x)
function
Spam filtering Text of an email Probability of email being
message spam (%)
Stockmarket Historical data on Expected price
prediction - Stock prices movements
- Economic indicators
Remembering Thought: Thought:
names “Who was that guy “Oh yes – that was Bob”
who liked windsurfing?
3

4. The state of machine learning
“It works! sort of…. sometimes…. on a good day….”
4

5. nuroko.com
We’re building a toolkit for machine learning that is:
• General purpose – works on any data
• Powerful – advanced algorithms to detect complex patterns
• Scalable – handle unlimited data at internet scale
• Realtime – suitable for online use in real applications
• Pragmatic – designed for solving real problems
5

6. Why Clojure?
Productivity and fun!
Good parts of the JVM
REPL
Interactive experiments
Functional programming
DSLs with composable abstractions
6

7. Some Key Abstractions
Vector 1 0 1 1 0
Efficiently represents information as a
vector of double values
Converts arbitrary data into vectors (and
Coder “Cat” 1 0 1 1 0
back again!)
𝑜𝑢𝑡𝑝𝑢𝑡 = 𝑓 𝑖𝑛𝑝𝑢𝑡 Represents a problem to solve – typically
Task
via provision of training examples
Represents a function
Module - (e.g. a Neural Network)
Algorithm Adjusts parameters in a module to learn a
function from experience / data
- (e.g. back-propagation)
7

8. Neural Networks
Output layer
Direction of
Hidden layer
calculation
Weighted connections
Input layer
Each node’s value is
computed as a function
of the weighted sum of its
inputs:
𝑦𝑖 = 𝑓 𝑤 𝑖𝑗 . 𝑥 𝑗
8

9. How to train a neural network
(BASIC version)
10 Initialise network with some random weights
20 Choose a random training example as input
30 Compute the output
40 Determine error (difference vs. expected output)
50 Adjust the weights very slightly to reduce the error
60 GOTO 20
9

10. Live Demo – Part 1
10

11. A harder problem….
11

12. A trick – compression of data
784 outputs
decompressor
150 units (“bottleneck”)
compressor
784 inputs
12

13. Putting it together
2
10 outputs (one for each digit)
150 units (compressed data)
compressor
784 inputs
13

14. Live Demo – Part 2
14

15. Questions?
15