Applying Machine Learning and Artificial Intelligence to Business

1. Applying Machine Learning and AI for Business • www.GoDataScience.io • Peter Morgan – Chief Data Scientist • Russell Miles – CEO • @godatascience • @pmzepto • @russmiles ©GoDataScience 1

2. I. ML - Overview • Definition – “Field of study that gives computers the ability to learn without being explicitly programmed” - Arthur Samuel, 1959 • Used for fitting lines/hyperplanes (regression), finding models, classifying objects, hypothesis testing, etc. • Three main categories of learning • Supervised (labelled data, classifying) • Unsupervised (unlabelled data, clustering) • Reinforcement learning (reward/penalty) ©GoDataScience 2

3. ML Algorithm Classes • Regression, e.g., linear, logistic • Decision Trees, e.g., CART • Ensemble, e.g., Random Forests • Bayesian, e.g., Naïve Bayes • Artificial Neural Networks, e.g., RNN • Instance, e.g., k-Nearest Neighbour (kNN) • Support Vector Machines (SVM) • Evolutionary, e.g., genetic (mimics natural selection) • Dimensionality Reduction, e.g., PCA • Clustering, e.g., K-means • Reinforcement, e.g., Q-learning • List of ML algos https://en.wikipedia.org/wiki/List_of_machine_learning_concepts ©GoDataScience 3

4. ML Applications • Speech recognition • Object recognition and tracking • Spam filtering • Self-driving cars • Recommendation engines • Fraud detection • Search engines, e.g., PageRank • Ad placement • Financial forecasting ©GoDataScience 4

5. Algorithm References • http://en.wikipedia.org/wiki/Machine_learning • http://en.wikipedia.org/wiki/Predictive_analytics • http://en.wikipedia.org/wiki/Pattern_recognition • http://en.wikipedia.org/wiki/Support_vector_machine • http://en.wikipedia.org/wiki/Regression_analysis • http://en.wikipedia.org/wiki/Random_forest • http://en.wikipedia.org/wiki/Non-parametric_statistics • http://en.wikipedia.org/wiki/Decision_tree_learning ©GoDataScience 5

6. Open Source ML Toolkits • Brain https://github.com/harthur/brain • Concurrent Pattern http://www.cascading.org/projects/pattern/ • Convnetjs https://github.com/karpathy/convnetjs • Decider https://github.com/danielsdeleo/Decider • etcML www.etcml.com • Etsy Conjecture https://github.com/etsy/Conjecture • Google Sibyl https://plus.google.com/+ResearchatGoogle/posts/7CqQbKfYKQf • GraphX https://amplab.cs.berkeley.edu/publication/graphx-grades/ • KNIME http://www.knime.org/ • List https://github.com/showcases/machine-learning • ML software http://www.cs.ubc.ca/~murphyk/Software/index.html • MLPNeuralNet https://github.com/nikolaypavlov/MLPNeuralNet ©GoDataScience 6

7. Open Source ML Tookits (cont) • MOA http://moa.cs.waikato.ac.nz/ • Neurokernel http://neurokernel.github.io/ • NuPic https://github.com/numenta/nupic • Orange http://orange.biolab.si/ • RapidMiner http://rapidminer.com • Scikit-learn http://scikit-learn.org/stable/ • Spark http://spark.apache.org/mllib/ • TunedIT http://tunedit.org/ • Vahara https://github.com/thedatachef/varaha • Viv http://viv.ai/ • Vowpal Wabbit https://github.com/JohnLangford/vowpal_wabbit/wiki • Weka http://www.cs.waikato.ac.nz/ml/weka/ ©GoDataScience 7

8. Open Source ML Libraries • Dlib http://dlib.net/ml.html • MADLib http://madlib.net/ • Mahout http://mahout.apache.org/ • MCMLL http://mcmll.sourceforge.net/ • MLC++ http://www.sgi.com/tech/mlc/ • mloss http://mloss.org/software/ • mlpack http://mlpack.org/ • Shogun http://www.shogun-toolbox.org/ • Stan http://mc-stan.org/ ©GoDataScience 8

9. Proprietary ML Toolkits • Ayasdi http://www.ayasdi.com/ • BigML https://bigml.com/ • H2O http://h2o.ai • IBM Watson http://www.ibm.com/smarterplanet/us/en/ibmwatson/ • Matlab http://uk.mathworks.com/solutions/machine-learning/ • Nutonian http://www.nutonian.com/ • Prediction.io http://prediction.io/ • Rocketfuel http://rocketfuel.com/ • Skytree http://www.skytree.net/ • Trifacta http://www.trifacta.com/ • Wolfram Alpha http://www.wolframalpha.com/ • Wise.io http://www.wise.io/ • Yhat https://yhathq.com/ ©GoDataScience 9

10. Other ML Resources • MLaaS (Cloud based) • Microsoft http://azure.microsoft.com/en-us/documentation/services/machine-learning/ • Google https://cloud.google.com/products/prediction-api/ • AWS https://aws.amazon.com/marketplace/search?page=1&searchTerms=machine+learning • Conferences • ICML • NIPS • ML Journals • ML Journal http://www.springer.com/computer/ai/journal/10994 • JMLR http://jmlr.org/papers/ • Pattern Recognition http://www.jprr.org/index.php/jprr • arXiv http://arxiv.org/list/stat.ML/recent • gitXiv http://gitxiv.com ©GoDataScience 10

11. References – Machine Learning • Abu-Mostafa, Yaser et al – Learning from Data, AML, 2012 • Alpaydın, Ethem - Introduction to Machine Learning, 2nd ed, MIT Press, 2009 • Bishop, Christopher - Pattern Recognition and Machine Learning, Springer, 2007 • Domingos, Pedro - The Master Algorithm, Allen Lane, 2015 • Flach, Peter - Machine Learning: The Art and Science of Algorithms that Make Sense of Data, Cambridge University Press, 2012 • Mitchell, Tom – Machine Learning, McGraw-Hill, 1997 • Murphy, Kevin - Machine Learning: A Probabilistic Perspective, MIT Press, 2012 • Rickhert, Willi and Luis Coelho, Building Machine Learning Systems with Python, Packt, 2013 • Witten, Ian et al - Data Mining, Practical Machine Learning Tools and Techniques, 3rd ed, Morgan Kaufman, 2011 ©GoDataScience 11

12. II. Deep Learning •Aims • To understand what Deep Learning is • Look at some of the common toolkits • How is it being used today • Challenges to overcome ©GoDataScience 12

13. Deep Learning Overview • Extract patterns and meaning from data • Modeled after how the human brain processes data • DL methods have gained notable successes in the field of speech and image recognition as well as in cognitive computing • Outperforming other algorithms • They are essentially ANNs • CNN = Convolutional Neural Networks (images) • RNN = Recurrent Neural Networks (speech & text) • LSTM = Long Short Term Memory ©GoDataScience 13

14. Deep Learning Progress Progress in machine classification of images - error rate by year. Red line is the error rate of a trained human ©GoDataScience 14

15. DL Resources • People • Yann LeCun http://yann.lecun.com/ • Geff Hinton http://www.cs.toronto.edu/~hinton/ • Yoshua Bengio http://www.iro.umontreal.ca/~bengioy/yoshua_en/index.html • Andrew Ng http://cs.stanford.edu/people/ang/ • Quoc Le http://cs.stanford.edu/~quocle/ • Jurgen Schmidhuber http://people.idsia.ch/~juergen/ • Blogs & Communities • FastML http://fastml.com/ • Chris Olah http://colah.github.io/ • Andrej Karparthy http://karpathy.github.io • DeepLearning.net http://deeplearning.net/ ©GoDataScience 15

16. DL Open Source Packages • Caffe http://caffe.berkeleyvision.org • CUDA Convnet https://code.google.com/p/cuda-convnet/ • cuDNN https://developer.nvidia.com/cuDNN • Deeplearning4j http://deeplearning4j.org/ • PyBrain http://pybrain.org/ • PyLearn2 http://deeplearning.net/software/pylearn2/ • SINGA http://singa.incubator.apache.org • TensorFlow http://tensorflow.org • Theano http://deeplearning.net/software/theano/ • Torch http://torch.ch/ • In fact, Google, IBM, Samsung, Microsoft and Baidu have open sourced their machine learning frameworks all within the space of the last two weeks ©GoDataScience 16

17. Deep Learning Companies • AlchemyAPI http://www.alchemyapi.com/ • Clarifai http://www.clarifai.com/ • Deepmind www.google.com • Ersatz Labs http://www.ersatzlabs.com/ • Memkite http://memkite.com/ • Nervana http://www.nervanasys.com/ • Numenta http://numenta.org/ • Nvidia https://developer.nvidia.com/deep-learning • Skymind http://www.skymind.io/ • Vicarious http://vicarious.com/ ©GoDataScience 17

18. References – Deep Learning • Bengio, Yoshua et al – Deep Learning, An MIT Press book in preparation http://goodfeli.github.io/dlbook/ • Buduma, Nikhil – Fundamentals of Deep Learning, O’Reilly, 2015 • Gibson, A and J. Patterson - Deep Learning: A Practitioner's Approach, O’Reilly, 2015 • Maters, Timothy, Deep Belief Nets in C++ and CUDA C, Vol. 1, CreateSpace, 2015 • Maters, Timothy, Deep Belief Nets in C++ and CUDA C, Vol. 2, CreateSpace, 2015 ©GoDataScience 18

19. III. Artificial Intelligence • Definition • Overview • History • Applications • Companies • People • Robotics • Opportunities • Threats • Predictions • References ©GoDataScience 19

20. AI - Definition “Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. Machines will solve the kinds of problems now reserved for humans, and improve themselves ”. Dartmouth Summer Research Project on A.I., 1956. ©GoDataScience 20

21. Artificial Intelligence Overview • Agents that learn from and adapt to their environments while achieving goals • Similar to living organisms • Multimodal is goal • AGI - endgame • New software/algorithms • Neural networks • Deep learning • New hardware • GPU’s • Neuromorphic chips • Cloud Enabled • Intelligence in the cloud • IaaS (Watson) • Cloud Robotics ©GoDataScience 21

22. The Bigger Picture Universe Computer Science AI ©GoDataScience 22

23. AI History I • 1940’s – First computers • 1950 – Turing Machine • Turing, A.M., Computing Machinery and Intelligence, Mind 49: 433-460, 1950 • 1951 – Minsky builds SNARC, a neural network at MIT • 1956 - Dartmouth Summer Research Project on A.I. • 1957 – Samuel drafts algos (Prinz) • 1959 - John McCarthy and Marvin Minsky founded the MIT AI Lab. • 1960’s - Ray Solomonoff lays the foundations of a mathematical theory of AI, introducing universal Bayesian methods for inductive inference and prediction ©GoDataScience 23

24. AI History II • 1969 - Shakey the robot at Stanford • 1970s – AI Winter I • 1970s - Natural Language Processing (Symbolic) • 1979 – Music programmes by Kurzweil and Lucas • 1980 – First AAAI conference • 1981 – Connection Machine (parallel AI) • 1980s - Rule Based Expert Systems (Symbolic) • 1985 – Back propagation • 1987 – “The Society of Mind” by Marvin Minsky published • 1990s - AI Winter II (Narrow AI) • 1994 – First self-driving car road test – in Paris • 1997 - Deep Blue beats Gary Kasparov ©GoDataScience 24

25. AI History III • 2004 - DARPA introduces the DARPA Grand Challenge requiring competitors to produce autonomous vehicles for prize money • 2007 - Checkers is solved by a team of researchers at the University of Alberta • 2009 - Google builds self driving car • 2010s - Statistical Machine Learning, algorithms that learn from raw data • 2011 - Watson beats Ken Jennings and Brad Rutter on Jeopardy • 2012+ Deep Learning (DL); Sub-Symbolic • 2013 - E.U. Human Brain Project (model brain by 2023) • 2014 – Human vision surpassed by DL systems at Google, Baidu, Facebook • 2015 – Machine dreaming (Google and Facebook NN’s) ©GoDataScience 25

26. AI Applications • Finance • Asset allocation • Algo trading • Fraud detection • Cybersecurity • eCommerce • Search • Manufacturing • Medicine • Law • Business Analytics • Ad serving • Recommendation engines • Smart homes • Robotics • Industry • Consumer • Space • Military • UAV (cars, drones etc.) • Scientific discovery • Mathematical theorems • Route Planning • Virtual Assistants • Personalisation • Compose music • Write stories ©GoDataScience 26

27. AI Applications (cont) • Computer vision • Speech recognition • NLP • Translation • Call centres • Rescue operations • Policing • Military • Political • National security • Anything a human can do but faster and more accurate – creating, reasoning, decision making, prediction • Google – introduced 60 DL products in last 2 years (Jeff Dean) ©GoDataScience 27

28. AI Applications - Examples • AI can do all these things already today: • Translating an article from Chinese to English • Translating speech from Chinese to English, in real time • Identifying all the chairs/faces in an image • Transcribing a conversation at a party (with background noise) • Folding your laundry (robotics) • Proving new theorems (ATP) • Automatically replying to your email, and scheduling ©GoDataScience 28

29. Learning and doing - from watching videos • Researchers at the University of Maryland, funded by DARPA’s Mathematics of Sensing, Exploitation and Execution (MSEE) program • System that enables robots to process visual data from a series of “how to” cooking videos on YouTube - and then cook a meal ©GoDataScience 29

30. AI Performance evaluation • Optimal: it is not possible to perform better • Checkers, Rubik’s cube, some poker • Strong super-human: performs better than all humans • Chess, scrabble, question-answer • Super-human: performs better than most humans • Backgammon, cars, crosswords • Par-human: performs similarly to most humans • Go, Image recognition, OCR • Sub-human: performs worse than most humans • Translation, speech recognition, handwriting ©GoDataScience 30

31. AI Corporations • IBM Watson • Google Deepmind etc. • Microsoft Project Adam • Facebook • Baidu • Yahoo! ©GoDataScience 31

32. AI Startups • Numenta • OpenCog • Vicarious • Clarafai • Sentient • Nurture • Wit.ai • Cortical.io • Viv.ai Number is growing rapidly (daily?) ©GoDataScience 32

33. Robotics - Embodied AI 1. Industrial Robotics • Manufacturing (Baxter) • Warehousing (Amazon) • Police/Security • Military • Surgery • Drones (UAV’s) • Self-driving cars • Trains • Ships • Planes • Underwater ©GoDataScience 33

34. 2. Consumer Robotics • Robots with friendly user interface that can understand user’s emotions • Visual; facial emotions • Tone of voice • Caretaking • Elderly • Young • EmoSpark, Echo • Education • Home security • Housekeeping • Companionship • Artificial limbs • Exoskeletons ©GoDataScience 34

35. Robots & Robotics Companies • Sawyer (ReThink) • iCub (EU) • Asimo (Honda) • Nao (Aldebaran) • Pepper (Softbank) • Many (Google) • Roomba (iRobot) • Kiva (Amazon) • Many (KUKA) • Jibo (startup) • Milo (Robokind) • Oshbot (Fellows) • Valkyrie (NASA) • DURUS (SRI) ©GoDataScience 35

36. AI & Robotics Websites • Jobs, News, Trade • Robotics Business review • AI Hub • AZoRobotics • Robohub • Robotics News • I-Programmer ©GoDataScience 36

37. Opportunities • Free humans to pursue arts and sciences • The Venus Project • Solve deep challenges (political, economic, scientific, social) • Accelerate new discoveries in science, technology, medicine (illness and aging) • Creation of new types of jobs • Increased efficiencies in every market space • Industry 4.0 (steam, electric, digital, intelligence) • Faster, cheaper, more accurate • Replace mundane, repetitive jobs • Human-Robot collaboration • A smarter planet ©GoDataScience 37

38. Threats • Unemployment due to automation • Replace some jobs but create new ones? • What will these be? • Widen the inequality gap • New economic paradigm needed • Basic Income Guarantee? • Existential risk • AI Safety • FHI/FLI/CSER/MIRI • Legal + Ethical issues • New laws • Machine rights • Personhood ©GoDataScience 38

39. AI Safety - Oversight • BARA = British Automation and Robot Association • EU Robotics • RIA = Robotic Industries Association • IFR = International Federation of Robotics • ISO – Robotics ©GoDataScience 39

40. Organisations - xRisk • FHI = Future of Humanity Institute • Oxford • FLI = Future of Life Institute • MIT • $7million grants awarded in June • MIRI = Machine Intelligence Research Institute • San Francisco • CSER = Center for Science and Existential Risk • Cambridge • AI Safety Facebook Group ©GoDataScience 40

41. Predictions* • More robots (exponential increase) • More automation (everywhere) • Endgame is to automate all work • 50% will be automated by 2035 • Loosely autonomous agents (2015) • Semi-automomous agents (2020) • Fully autonomous agents (2025) • Cyborgs (has started – biohackers, implants) • Singularity (2029?) – smarter than us • Self-aware? (personhood) • Quantum computing • Game changer • Quantum algorithms • Dwave • Advances in science and medicine • Ethics (more debate) • Regulation (safety issues) *Remembering that progress in technology follows various exponentially increasing curves - see “The Singularity is Near”, by Ray Kurzweil.©GoDataScience 41

43. References • Barrat, James, Our Final Invention, St. Martin's Griffin, 2014 • Brynjolfsson, Erik and Andrew McAfee, The Second Machine Age, W.W. Norton & Co., 2014 • Ford, Martin, Rise of the Robots: Technology and the Threat of a Jobless Future, Basic Books, 2015 • Goertzel, Ben et al - Engineering General Intelligence, Part 1, Atlantis Press, 2014 • Hawkins, Jeff – On Intelligence, Owl, 2005 • Kaku, Michio, The Future of the Mind, Doubleday, 2014 • Kaplan, Jerry – Humans Need Not Apply, Yale University Press, 2015 ©GoDataScience 43

44. References (cont) • Kurzweil, Ray, The Singularity is Near, Penguin Books, 2006 • Kurzweil, Ray, How to Create a Mind, Penguin Books, 2013 • Marcus, G. and J. Freeman (eds) - The Future of the Brain: Essays by the World's Leading Neuroscientists, Princeton, 2014 • Markoff, John – Machines of Loving Grace, Ecco Press, 2015 • Nowak, Peter, Humans 3.0: The Upgrading of the Species, Lyons Press, 2015 • Russell and Norvig, Artificial Intelligence, A Modern Approach, Pearson, 2009 • Shanahan, Murray – The Technological Singularity, MIT Press, 2015 ©GoDataScience 44

45. Thanks for listening!! www.godatascience.io @godatascience @pmzepto @russmiles