Fundamental of Machine Learning

Some Basics of
Machine Learning
Sidik Soleman
Prosa.ai
Email: sidik.soleman@prosa.ai

1. Machine Learning
2. Linear Regression
3. Logistic Regression
Agenda
2

Machine Learning in Everyday Life
Search Engine
Machine Translation
Spelling Checker
Spam Detection
Chatbot
4

Definition of Artificial Intelligence (AI)
1. Acting Humanly
○ The turing test approach
○ Computer should to be able to do
■ Natural language processing
■ Knowledge representation
■ Automated reasoning
■ Machine learning (ML)
● To adapt to new circumstances and to detect and extrapolate patterns.
■ Computer vision
■ Robotics
5
Artificial Intelligence - A Modern Approach by Peter Norvig, Stuart J. Russell

Definition of Artificial Intelligent (AI)
2. Thinking Humanly
○ The cognitive approach
○ Understanding human brain by means
■ Introspection—trying to catch our own thoughts as they go by;
■ Psychological experiments—observing a person in action; and
■ Brain imaging—observing the brain in action
6

Definition of Artificial Intelligent (AI)
3. Thinking Rationally
○ The “law of thought” approach
○ Logic → correct inference
4. Acting Rationally
○ The rational agent approach
○ Act rationally is to reason logically to the conclusion and then to act on that.
○ But, correct inference is not all of rationality; in some situations, there is no
provably correct thing to do, but something must still be done
7

Machine Learning Paradigms
1. Supervised Learning
○ Learning based on experience/examples, e.g. classification task
8
Training Data
Machine (Model)
Input
Cat

Machine Learning Paradigms (Cont’d)
2. Unsupervised Learning
○ Finding hidden pattern from data, e.g. clustering task
9
Data
Machine (Model)
Output

Machine Learning Paradigms (Cont’d)
3. Reinforcement Learning
○ Learning based on the interaction with environment to achieve a goal, e.g. to win chess game
○ Decision will be rewarded or punished
10
Machine (Model)
Environmen
t
(action)
(state, reward)

Machine Learning Algorithms
11
Supervised Learning Unsupervised Learning
Reinforcement
Learning
- Linear Regression
- Logistic Regression
- Support Vector
Machine
- Decision Tree
- Random Forest
- Naive Bayes
- Deep Learning
- K-Means Clustering
- K-Nearest Neighbour
- Apriori Algorithm
- Principal Component
Analysis
- Latent Dirichlet
Allocation
- Deep Learning
- Temporal Difference
Learning
- Monte Carlo
- Q-learning
- Policy Gradients
- Deep Reinforcement
Learning

12
2.Linear Regression
Based on machine learning course in Coursera by Andrew Ng
https://www.coursera.org/learn/machine-learning

Linear Regression
● Predicting the price of house
13
What is the price of a 200m2 house?
Finding the best linear function
f(x) g(x)
h(x)

Linear Regression (Cont’d)
14
The best linear function
●
● Finding the best parameters
so that is close to y
General Setup
● m= # of training examples
● x = input variable/features
● y = output variable/target
● (x, y) = one training example
● (xi, yi) = i-th training example
h(x)

Linear Regression (Cont’d)
15
Finding the best parameter
h(x)
cost function

Gradient Descent: Cost Function Derivation
16
h(x)
Repeat until convergence {
}
Simultaneous Update

Derivation Intuition
17
Positive Slope
Negative Slope
Derivation guide the direction of
should be adjusted.
α is the learning rate to control how big the
movement of

Gradient Descent Algorithm
18
h(x)
Repeat until convergence {
}
Gradient Descent Algorithm

The Answer of the Prediction
19
h(x)

Multivariable Linear Regression
20
Previous linear regression function
●
Multivariable linear regression
● m= # of training examples
● x(i) = input variables/features of i-th
example
● y(i) = output variable of i-th example
● x(i)
j = value of feature j in i-th example
repeat {
}
Cost function
Gradient descent

21
3.Logistic Regression
Based on machine learning course in Coursera by Andrew Ng
https://www.coursera.org/learn/machine-learning

Classification Task
22
● Linear regression issues
○ Prediction score can be more or less than 1
○ Not suitable for classification task
● Solution: logistic regression
Parameter and x
as matrix
1
0.5
Score is also probability

Regression Model
23
Training Set
n parameter
Cost
repeat {
}
Gradient descent

Multi-class Classification
● Binary vs multi-class classification
● Handling multi-class classification
○ One vs Rest /One vs All
■ 4 Class Classification → 4 Classification Model
■ Label with maximum score is the answer
24

Machine Learning Problems
● Overfit
○ Hypothesis fit well with the training data → high performance
○ But fail to make generalization → poor performance on test data
○ High variance
○ Too many features
● Underfit
○ High bias → low performance
○ Fail to make generalization → poor performance on test data
25

Some solution for the problem
● Reduce number of features
○ Select features to keep
○ Model selection
● Regularization
○ Keep all features, but reduce magnitude or values of parameters (theta)
○ Works well when having a lot of features
26

Regularization
27
Regularization
Note:
Lambda high → underfit and high bias
Lambda too small → overfit

Take Home Message
● Machine learning is to adapt to new circumstances and to detect and
extrapolate patterns.
● Linear regression for predicting real continuous values
● Logistic regression for predicting class in classification
28

Fundamental of Machine Learning

Recomendados

Recomendados

Más contenido relacionado

Similar a Fundamental of Machine Learning

Similar a Fundamental of Machine Learning (20)

Más de SARCCOM

Más de SARCCOM (20)

Último

Último (20)

Fundamental of Machine Learning