Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.

1. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
@DocXavi
Deep Learning for Computer Vision
Beyond vision
5 May 2016
Xavier Giró-i-Nieto
Master en
Creació Multimedia

2. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
2
When robots open their eyes...
...is because they have learned to see.
Learning only to see ?

3. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learning only to see ?
Nexi, del MIT Media Lab (Foto: Spencer Lowel)

4. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Big data
Internet of things - IoT
Learning only to see ?

5. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Personal data
Big data
Learning only to see ?

6. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Atlas, de Boston Dynamics
Robust manipulation and motion
Learning only to see ?

7. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Playing with other computers
Learning only to see ?
Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin
Riedmiller. "Playing atari with deep reinforcement learning." arXiv preprint arXiv:1312.5602 (2013).
DeepMind

8. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Playing with humans
AlphaGo (Google DeepMind)
Learning only to see ?

9. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Autonomous Driving
Google Self-driving car
Learning only to see ?

10. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Visual arts
Google Research, “Going deeper into neural networks” - DeepDream (2015)
Learning only to see ?

11. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Google Research, “Going deeper into neural networks” - DeepDream (2015)
Visual arts
Only open their eyes ?

12. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
http://turing.deepart.io/
Visual arts
Learning only to see ?

13. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Music composition
Manuel Araoz, “Training a Recurrent Neural Network to Compose Music” (2016).
Learning only to see ?

14. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Poetry
Ross Goodwin, Neuralsnap (2016).
Learning only to see ?

15. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
“Scripts” (!?)
Darknet
JON
He leaned close and onions, barefoot from his shoulder. "I am not a purple
girl," he said as he stood over him. "The sight of you sell your father with you a
little choice."
"I say to swear up his sea or a boy of stone and heart, down," Lord Tywin
said. "I love your word or her to me."
Learning only to see ?

16. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Public Health
Announcement of Google DeepMind Health (24/02/2016)
Learning only to see ?

17. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Nacho Hernandez, “Why artificial intelligence will democratize
healthcare” (TEDx Talk, 2014)
Public health
Learning only to see ?

18. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Nancy Lublin, “The heartbreaking text that inspired a crisis
helpline” (TED Talk 2015)
Mental health
Learning only to see ?

19. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Psychological support and counseling ?
Learning only to see ?

20. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
20
Affective computing
Rana el Kalioubi, “This app know how you feel, from the look on your
face”, TEDTalks 2015.
Learning only to see ?

21. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
21
Nexi Project,
from MIT Media Lab
(Photos: Spencer
Lowel)
[video]
Affective computing
Learning only to see ?

22. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
“Google’s chairman (Eric Schmidth) thinks artificial intelligence will
let scientists solve some of the world’s "hard problems," like
population growth, climate change, human development, and
education.” (Bloomberg Business, 11/01/2016)
[+info @ MIT Technology Review]
Artificial intelligence

23. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Google’s CEO Sundar Pichai: “Era Of Computers Will End Very Soon,
AI Will Rule” (Fossbytes, 03/05/2016)
Artificial intelligence

24. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Jeremy Howard, “The wonderful and terrifying implications of
computers that can learn”, TEDTalks 2014.
Artificial intelligence

25. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Artificial intelligence
Stephen Hawking, “Artificial intelligence could spell out the
human race.” (2014)

26. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Artificial intelligence

27. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Elon Musk (Tesla), one of OpenAI promoters
Artificial intelligence

28. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Neil Lawrence, OpenAI won’t benefit humanity
without open data sharing (The Guardian,
14/12/2015)
Phd Comics: Who owns your data ? (Hint: it is not you)
Artificial intelligence

29. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
29
Xavier Sala-i-Martin (Columbia University),
“Les conclusions del Fòrum de Davos”
(TV3, 03/02/2016) - in Catalan
Carles Boix (Princeton University),
“La quarta revolució industrial”
(Diari Ara, 08/02/2016) - in Catalan
Artificial intelligence

30. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Artificial intelligence
Open question: in which jobs will robots replace humans first ?

31. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
31
Source: 25 Best jobs in America (Glassdoor)
Data scientist.
The best job in the world ?

32. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
32
The best job in the world ?
Summer internships
for Phd students
related to Data
Analytics.

33. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
33
The Economist, “Million-dollar babies” (02/04/2016)
The best job in the world ?

34. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
34
Nature, “AI talent grab sparks excitement and concern” (26/04/2016)
The best job in the world ?

35. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more with Nat & Lo 20% Google Project:
Learn more

36. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more
● Friendly slides for dissemination (family & friends).
[Available on slideshare.net]

37. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more
● Presentació amigable per a familiar i amics.
[Disponible a slideshare.net]

38. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more
Keras http://keras.io/
Tensor Flow https://www.tensorflow.org/
Caffe http://caffe.berkeleyvision.org/
Torch (Overfeat) http://torch.ch/
Theano http://deeplearning.net/software/theano/
MatconvNet (VLFeat) http://www.vlfeat.org/matconvnet/
CNTK (Mcrosoft) http://www.cntk.ai/
MxNet: https://github.com/dmlc/mxnet

39. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more
Source: @fchollet

40. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
40
?
Learn more

41. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
41
Learn more
http://cs231n.stanford.edu/

42. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
42
Learn more
“Machine learning” sub-Reddit.

43. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
43
Learn more

44. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
44
Learn more
Grup d’estudi de machine learning Barcelona

45. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
45
Learn more
● Reading Group with public listing of videos, slides and papers.

46. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
More details on the
website by
Professor Jordi Torres.
Learn more

47. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more
[Website]

48. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Learn more
[Website]

49. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Conclusions
49

50. Xavier Giró i Nieto, “Deep learning beyond vision”. Master in Multimedia, La Salle URL (May 2016)
Thanks a lot !
50
Slides available at:
https://imatge.upc.edu/web/people/xavier-giro
@DocXavi
/ProfessorXavi