From theory to practice

1. Chatbots and Deep Learning:
From theory to practice

2. Andherson C. Maeda
Developer Consultant @ thoughtworks
$ whoami

3. Chatbots

4. Definition
What's chatbot means?
1

5. “
Program that simulate human
behavior though questions and
answers.

6. Can machines think?
◉ Computing Machinery and Intelligence
(TURING, 1950)
◉ Imitation game

7. In 50 years
An interrogator will not have more than 70%
chance to make the right identification after 5
minutes conversation

8. Chatbots timeline2

9. Chatbots timeline
60's: Eliza
Simulation of
rogerian
psychologist. Use
pieces of input to
build output phrase
(Wallace)
90's: Julia
TinyMUD bot. Based
on rules recognition
and neural networks
(Mauldin)
2000: Alice
Based on rules also,
but with more
features. Inspired by
XML: AIML (Wallace)
Natural Language Processing
Neural Networks

10. What's that?
Natural Language
Processing3

11. “
Field of Computer Science and
Linguistics concerned with
interactions between computers
and human natural languages

12. NLP: example

13. How can I build my own bot?
Chatbots toolbox4

14. Alice: AIML
<category>
<pattern>WHAT ARE YOU</pattern>
<template>
<think><set name="topic">Me</set></think>
I am the latest result in artificial intelligence,
which can reproduce the capabilities of the
human brain
with greater speed and accuracy.
</template>
</category>

15. Platforms, APIs and Libs

16. or not...
Caveats5

17. Caveats (or not)
◉ Handcrafted rules
◉ Unknown questions will generate default
answers

18. Machine Learning
Deep, embeddings, etc.

19. Definition
What's Machine Learning?
6

20. “
"A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P if its
performance at tasks in T, as measured by P, improves with
experience E" (Mitchel, 1997)

21. Machine Learning Algorithms

22. A Neural Conversational Model

23. Word Embeddings
A Neural Conversational Model
Seq2Seq

24. Building...
Building my own neural bot <3
7

25. Place your screenshot here
Dialog corpus selection
Find a large dialog
corpus

26. NLP Tasks
Apply NLP tasks to clean
your dataset like:
- Lowercase
- Tokenization

27. Word Embeddings
Build the Word2Vec
embedding from your
dialog corpus
- 40k most frequent
words could be
good enough

28. Train Seq2Seq Model
Train your model with a
dialog corpus.
- Use some testing
technique
(70%/30%, cross
validation, etc)
- GPU!

29. Demo (portuguese version)
Sequence to Sequence trained with StarWars
corpus:
◉ https://webot.personaltek.com

30. Seq2Seq StarWars Bot

31. or not...
Caveats8

32. Caveats (or not)
◉ Handcrafted rules
◉ Unknown questions will generate default
answers answers!
○ Weird answers, but answers...
◉ Long training time…
◉ Lot of resources needed
◉ Hard to test the model = (

33. Any questions ?
You can find me at
◉ @maedabr
◉ maeda.br@gmail.com
◉ amaeda@thoughtworks.com
Thanks!