https://www.bigdataspain.org/2016/program/thu-from-data-numbers-knowledge-semantic-embeddings.html https://www.youtube.com/watch?v=JjCHvKMgkkY&list=PL6O3g23-p8Tr5eqnIIPdBD_8eE5JBDBik&index=30&t=3s

1. www.iic.uam.es

2. 7 de diciembre de 2016
Data  Numbers  Knowledge
Álvaro Barbero – Chief Data Scientist
www.iic.uam.es

3. www.iic.uam.es 3
How do we see the world?

4. www.iic.uam.es 4
Understanding correlations
The more, the better
Linear correlations

5. www.iic.uam.es 5
Understanding correlations
“Moral virtue is a mean”
Aristotle
Convex correlations

6. www.iic.uam.es 6
Understanding correlations
Few is bad, but slightly more is very good, unless you skip
the sweet spot and then you are doing terribly, but if you
keep going you get better until you do worse again and after
that nothing changes and you do so-so
Nonlinear correlations

7. www.iic.uam.es
THE REAL WORLD IS NON-LINEAR
Shocking
revelations
Big data scientists rampage
the cities after the newly
discovered uselessness of
their linear models.
Lorem ipsum. Ea pro natum
invidunt repudiandae, his et
facilisis vituperatoribus.
Mei eu ubique altera
senserit, consul eripuit
accusata has ne. Ignota
verterem te nam, eu cibo
causae menandri vim. Sit
rebum erant dolorem et, sed
odio error ad.Vel molestie
corrumpit deterruisset ad,
mollis ceteros ad sea.
In libris graecis appetere
mea. At vim odio lorem
omnes, pri id iuvaret
partiendo. Vivendo
menandri et sed. Lorem
volumus blandit cu has.Sit
cu alia porro fuisset.
Ea pro natum invidunt
repudiandae, his et facilisis
vituperatoribus. Mei eu
ubique altera senserit,
consul eripuit accusata has
ne. Ignota verterem te nam,
eu cibo causae menandri
vim. Sit rebum erant
dolorem et, sed odio error
ad.Vel molestie corrumpit
deterruisset ad, mollis
ceteros ad sea.
BIG DATA TIMES
VERY VERY BIG DATA SCIENCE - Since 1802

8. www.iic.uam.es 8
The modelling dilemma

9. www.iic.uam.es 9
The modelling dilemma
Easy to understand
and explain
Linear models
May not faithfully
represent reality
Non-linear models
Accurate
representations
Very difficult
interpretation

10. www.iic.uam.es 10
The
brain trick

11. www.iic.uam.es 11
Brain power
Low level
observations
Multilayered non-linear
processing
Abstract yet linear
concepts

12. www.iic.uam.es 12
Abstraction examples
<<
Art quality

13. www.iic.uam.es 13
Embeddings: abstract non-linearity away
Low level
observations
High level
embedding
Artificial multilayered
non-linear processing
Deep Network

14. www.iic.uam.es 14

15. www.iic.uam.es 15
word2vec
cat chills on a mat
cat chills mushroom a mat
Socher et al (2013)

16. www.iic.uam.es 16
Exploiting the linearity: semantic algebra
King man woman queen
Obama USA Russia Putin
human animal ethics
paella Spain Italy risotto
Cristiano Madrid Barcelona Messi

17. www.iic.uam.es 17
Bilingual word2vec
Word
Space
Wzh
Wen
English
Words
Mandarin
Words
Socher et al (2013)

18. www.iic.uam.es 18
Text embedding models through Recurrent Neural Networks
High dimensional
representation of
a sequence
0.1
0.5
1.0
0.0
2.4
The lazy brown fox
Sutskever et al - Sequence to Sequence Learning with neural networks

19. www.iic.uam.es 19
Example: books embeddings
Cilibrasi, Vitányi - Normalized Web Distance and Word Similarity
Jonathan
Swift
Oscar
Wilde
William
Shakespeare

20. www.iic.uam.es 20
Embedding artistic styles
Gatys et al – A Neural Algorithm of Artistic Style
Bottou et al - Optimization Methods for Large-Scale Machine Learning
Low level
observations
Style
embedding

21. www.iic.uam.es 21
Artistic styles as linear relations
Google Research - https://research.googleblog.com/2016/10/supercharging-style-transfer.html

22. www.iic.uam.es 22
An application: furniture embedding
Bell and Bala - Learning visual similarity for product design with convolutional neural networks

23. www.iic.uam.es
Take home message

24. www.iic.uam.es 24
Take home message
Reality is highly non-
linear
Deep Learning can
abstract complexity
away
Complex relations
become easy
comparisons!

25. www.iic.uam.es

26. www.iic.uam.es
www.iic.uam.es
Álvaro Barbero Jiménez
Chief Data Scientist at
Instituto de Ingeniería del Conocimiento (IIC)
Elementos gráficos de apoyo obtenidos en:
@albarjip
Alvaro Barbero