Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

AI - Externalization of Mind

Cargando en…3

Eche un vistazo a continuación

1 de 114 Anuncio

Más Contenido Relacionado

Similares a AI - Externalization of Mind (20)


Más reciente (20)

AI - Externalization of Mind

  1. 1. 1
  2. 2. This content included for educational purposes. 2 • Lawrence Mills Davis is founder and managing director of Project10X, a research consultancy known for forward-looking industry studies; multi-company innovation and market development programs; and business solution strategy consulting. Mills brings 30 years experience as an industry analyst, business consultant, computer scientist, and entrepreneur. He is the author of more than 50 reports, whitepapers, articles, and industry studies. • Mills researches artificial intelligence technologies and their applications across industries, including cognitive computing, machine learning (ML), deep learning (DL), predictive analytics, symbolic AI reasoning, expert systems (ES), natural language processing (NLP), conversational UI, intelligent assistance (IA), and robotic process automation (RPA), and autonomous multi- agent systems. • For clients seeking to exploit transformative opportunities presented by the rapidly evolving capabilities of artificial intelligence, Mills brings a depth and breadth of expertise to help leaders realize their goals. More than narrow specialization, he brings perspective that combines understanding of business, technology, and creativity. Mills fills roles that include industry research, venture development, and solution envisioning. Lawrence Mills Davis Managing Director Project10X 202-667-6400
  3. 3. This content included for educational purposes. Direct competitors for Publicis.Sapient include digital agencies, consultants, IT services, which are providing AI and cognitive platforms as a basis for custom solutions, products/services, and XaaS offerings to markets addressed by Publicis.Sapient AI encompasses multiple technologies that can be combined to sense, think, and act as well as to learn from experience and adapt over time. SENSE Computer vision, audio and affective processing aim to actively perceive the world around them by acquiring and processing images, sounds, speech, biometrics, and other sensory inputs. One example is identity analytics for facial recognition. Lie detection is another. THINK Natural language processing and inference engines enable AI systems to analyze, interpret, and understand information. One example is speech analytics and language translation of search engine results. Another is interpretation of user intent by virtual assistants.. ACT AI systems take action in digital or physical worlds using machine learning, expert systems and inference engines. Recommendation systems are one example. Another is auto- pilot and assisted-braking capabilities in cars. Cognitive robotics is another. This content included for educational purposes. 3
  4. 4. This content included for educational purposes. 4 AI technologies mimic human abilities to sense, think, and act. Source: Forrester, TechRadar: Artificial Intelligence Technologies, Q1 2017 A I-OPTIMIZED CHIPS Think Learn Sense Act Continuous iteration and feedback HUMAN RECOGNITION Speech, face, and body Sensors (e.g., temperature, chemical, spectral, magnetic) and devices MACHINE RECOGNITION Knowledge representation, rules engines, corporate data, open data, and external data KNOW Virtual agents and natural language generation INTERFACE Machine learning platforms, deep learning platforms, text analytics and NLP, and image and video analysis LEARN Robotic process automation and decision management AUTOMATION Customer, partner, employee, robot, and device This content included for educational purposes.
  5. 5. This content included for educational purposes. 5 Standard Automation Intelligent Automation Systems that do Systems that think Systems that learn Robotic Process Automation Data Collection/ Data Preparation Speech, Video & Image Recognition Predictive APIs Natural Language Processing IT Process Automation Deep Learning Artificial Intelligence IoT & Smart Devices Emotional Recognition Cognitive Computing Machine Learning Autonomic Computing AI systems learn, think, and automate doing
  6. 6. AI: SENSE
  7. 7. This content included for educational purposes. • Pattern recognition • Machine perception • Speech recognition • Computer vision • Affective computing Overview of
 AI: Sense 7
  8. 8. This content included for educational purposes. Pattern recognition 8 PaGern recogniHon involves techniques to dis_nguish signal from noise through sta_s_cal analyses, Bayesian analysis, classifica_on, cluster analysis, and analysis of texture and edges. Pabern recogni_on techniques apply to sensors, data, imagery, sound, speech, language. Automated classificaHon tools dis_nguish, characterize and categorize data based on a set of observed features. For example, one might determine whether a par_cular mushroom is “poisonous” or “edible” based on its color, size, and gill size. Classifiers can be trained automa_cally from a set of examples through supervised learning. Classifica_on rules discriminate between different contents of a document or par__ons of a database based on various abributes within the repository. StaHsHcal learning techniques construct quan_ta_ve models of an en_ty based on surface features drawn from a large corpus of examples. In the domain of natural language, for example, sta_s_cs of language usage (e.g., word trigram frequencies) are compiled from large collec_ons of input documents and are used to categorize or make predic_ons about new text. Sta_s_cal techniques can have high precision within a domain at the cost of generality across domains. Systems trained through sta_s_cal learning do not require human-engineered domain modeling. However, they require access to large corpora of examples and a retraining step for each new domain of interest. Source: Gary Larson Source: Barbara Catania and Anna Maddalena
  9. 9. This content included for educational purposes. 9 Machine perception Enterprises are adopting biometrics based 
 AI solutions to determine: • Identity and authentication — Sensor processing and biometrics for retinal scanning, fingerprint scanning, facial recognition, voice recognition, signature verification. • Sentiment and emotion — sonic analytics, facial expressions, text analytics, gesture analytics and body language • Veracity — biometric sensors (temperature, moisture, heart rate, etc.), sonic analytics, facial expressions, text analytics, gesture analytics and body language. This content included for educational purposes.
  10. 10. This content included for educational purposes. 10 • Cybersecurity is the body of technologies, processes and practices designed to protect networks, computers, programs and data from attack, damage or unauthorized access. • User security is shifting from reliance on usernames, passwords and security questions to incorporate biometric factors including voice recognition, facial recognition, iris recognition, fingerprints and other biometric data. • Biometric security incorporates AI techniques for pattern recognition and anomaly detection. • Facial recognition technology is already a big business; it’s being used to measure the effectiveness of store displays, spot cheaters in casinos, and tailor digital ads to those passing by. • Cognitive security analytics provide capabilities for predicting and assessing threats, recommending best practices for system configuration, automating defenses, and orchestrating resilient response. AI machine perception for user security is incorporating biometric factors.
  11. 11. This content included for educational purposes. Speech recognition The ability to automatically and accurately transcribe human speech plus natural language understanding empowers individuals to interact with enterprise systems using voice commands. Natural language technology processes queries, answers questions, finds information, and connects users with various services to accomplish tasks. Speak, and ye shall find 11
  12. 12. This content included for educational purposes. Evolution of speech technologies 12 Speaker dependent Automatic Speech Recognition Natural Language Understanding Text-to-Speech ASR NLU TTS Speaker independent Rules based grammars Statistical grammars Cloud ASR Deep neural networks Deep neural networksCloud NLUStatistical NLU Format based TTS Concatenated TTS Rules based TTS Cloud TTS Statistical TTS models Deep neural networks Source: Nuance
  13. 13. This content included for educational purposes. 13 Voice application development Text NL Semantics ASR Language Understanding Context Interpretation Grammar and semantic tags Voice application developer
  14. 14. This content included for educational purposes. 14 Computer vision • The ability of computers to iden_fy objects, scenes, and ac_vi_es in unconstrained (that is, naturalis_c) visual environments. • Computer vision has been transformed by the rise of deep learning. • The confluence of large-scale computing, especially on GPUs, the availability of large datasets, especially via the internet, and refinements of neural network algorithms has led to dramatic improvements. • Computers are able to perform some (narrowly defined) visual classification tasks better than people. A current research focus is automatic image and video captioning. This content included for educational purposes.
  15. 15. This content included for educational purposes. Image annotation 
 and captioning using
 deep learning a man riding a motorcycle 
 on a city street a plate of food with
 meat and vegetables 15
  16. 16. This content included for educational purposes. Does My ai really understand what he feels and 
 what he is 
 saying to 
 me? Affective computing • Detecting emotions from videos, audio, text, facial expressions and gestures is a growth market and important part of future cognitive systems. • Audio and video analytics for interpreting sentiment, emotion and veracity 16This content included for educational purposes.
  17. 17. This content included for educational purposes. Six universal facial expressions* * — Anger, happiness, surprise, fear, sadness, disgust 17
  18. 18. This content included for educational purposes. To analyze someone’s facial expressions, body temperature, etc. to determine what that person is feeling, whether they are lying or not, what their gestures and body language are, etc… Who are some of the players and what are their top offerings? 18This content included for educational purposes.
  19. 19. This content included for educational purposes. What can body temperature, heart rate, and other biometrics tell us? 19
  20. 20. This content included for educational purposes. Can you tell when someone is lying by reading their facial expressions? 20
  21. 21. This content included for educational purposes. To analyze someone’s facial expressions, body temperature, etc. to determine what that person is feeling, whether they are lying or not, what their gestures and body language are, etc… Who are some of the players and what are their top offerings? © Copyright Project10x | Confidential 21 This content included for educational purposes.
  22. 22. This content included for educational purposes. Hyper real chatbots and assistants • Customer service chatbots are about to become very realistic. A startup gives chatbots and virtual assistants realistic facial expressions and the ability to read yours. • Would your banking experience be more satisfying if you could gaze into the eyes of the bank’s customer service chatbot and know it sees you frowning at your overdraft fees? Soul Machines made this chatbot for the Australian government to help people get information about disability services. 22This content included for educational purposes.
  23. 23. This content included for educational purposes. Selected vendors* by category of machine perception analytics AI Platforms with APIs for 
 Image & Text • Apple (Emotient) • Facebook • Google • IBM • Microsoft Facial Analytics • Affectiva • Clarifai • CrowdEmotion • Eyeris/EmoVu • Faciometrics • Imotions • Kairos • Noldus • nViso • RealEyes • Sightcorp/ Sonic Analytics • BeyondVerbal • EMO Speech • Nemesysco • NICE • Verint • Vokaturi Gesture Analytics • GRT—Gesture Recognition Toolkit Text 
 Analytics • Clarabridge • Crimson Hexagon • IBM Alchemy API • Indico • Receptiviti Document
 Image Analytics • Cvision • Parascript • Signotec • Topaz Systems 23 * Not included in this research deck.
  24. 24. AI: THINK
  25. 25. This content included for educational purposes. 25 • Machine learning • Deep learning • Natural language processing • Knowledge representation • Reasoning • Cognitive computing • What today’s AI technology can and cannot do Overview of
 AI: Think
  26. 26. This content included for educational purposes. Machine learning • Machine learning is a type of AI that involves using computerized mathema_cal algorithms that can learn from data and can depart from strictly following rule-based, pre-programmed logic. • Machine learning algorithms build a probabilis_c model and then use it to make assump_ons and predic_ons about similar data sets • Machine Learning runs at machine scale: it is data driven and suited to the complexity of dealing with disparate data sources and the huge variety of variables and amounts of data involved. • Unlike for tradi_onal analysis, the more data fed to a machine learning system, the more it can learn, resul_ng in higher quality insights. 26 This content included for educational purposes.
  27. 27. This content included for educational purposes. 27 Machine learning can help solve classification, prediction, and generation problems Source McKinsey Global Institute
  28. 28. This content included for educational purposes. 28 Machine learning has great impact potential across industries and use case types Source McKinsey Global Institute
  29. 29. This content included for educational purposes. 29 Types of machine learning
  30. 30. This content included for educational purposes. Machine learning overview 30
  31. 31. This content included for educational purposes. Types of machine learning and categories of algorithms 31 Type of machine learning Target variable Type of algorithm Sample application
  32. 32. This content included for educational purposes. Clustering 32 Clustering is the process of organizing objects into groups whose members are similar in some way. Clustering is an approach to learning that seeks to place objects into meaningful groups automa_cally based on their similarity. Document clustering techniques iden_fy topics and group documents into meaningful classes. Clustering, unlike classifica_on does not require the categories to be predefined with the hope that the algorithm will determine useful but hidden groupings of data points. The hope in applying clustering algorithms is that they will discover useful but unknown classes of items.
  33. 33. This content included for educational purposes. Outlier Detection • Iden_fying excep_ons or rare events can osen lead to the discovery of unexpected knowledge. 
 Outlier detec_on is used to iden_fy anomalous situa_ons. • Anomalies may be hard-to-find needles in a haystack, but may nonetheless represent high value when they are found (or costs if they are not found). Typical applica_ons include fraud detec_on, iden_fying network intrusion, faults in a manufacturing processes, clinical trials, vo_ng ac_vi_es and criminal ac_vi_es in E-commerce. • Applying machine learning to outlier detec_on problems brings new insight and beber detec_on of outlier events. Machine learning can take into account many disparate sources of data and find correla_ons that are too obscure for human analysis to iden_fy. • Take the example of credit card fraud: with machine learning online behavior (web site browsing history) of the purchaser becomes a part of the fraud detec_on algorithm – rather than simply considering the history of purchases made by the card holder. This involves analyzing huge amounts of data, but it also is a far more robust approach to E-commerce fraud detec_on. 33This content included for educational purposes.
  34. 34. This content included for educational purposes. Machine learning flow — training and prediction 34
  35. 35. This content included for educational purposes. Machine learning: • Supervised— Correct classes of the training data are known. • Unsupervised— Correct classes of the training data are not known • Reinforcement— Machine or software agent learns behavior based on feedback from the environment. This behavior can be learned once and for all or continue to adapt as time goes by. 35
  36. 36. This content included for educational purposes. Deep learning A class of machine learning algorithms that: • Use a cascade of many layers of nonlinear processing units for feature extrac_on and transforma_on. Successive layer use the output from the previous layer as input. Algorithms may be supervised or unsupervised. Applica_ons include pabern analysis (unsupervised) and classifica_on (supervised). • Are based on (unsupervised) learning of mul_ple levels of features or representa_ons of the data. Higher level features are derived from lower level features to form a hierarchical representa_on. • Learn mul_ple levels of representa_ons that correspond to different levels of abstrac_on; the levels form a hierarchy of concepts. 36 This content included for educational purposes.
  37. 37. This content included for educational purposes. Types of Neural Networks How do neural networks work? Information flows through a neural network in two ways. When it's learning (being trained) or operating normally (after being trained), patterns of information are fed into the network via the input units, which trigger the layers of hidden units, and these in turn arrive at the output units. Training is the process of modifying the weights in the connections between network layers so as to achieve the expected output. This is achieved through supervised learning, unsupervised learning, and reinforcement learning. Operations is the process of applying the algorithm to make predictions. Result evaluation feeds back to improve/optimize performance 37
  38. 38. This content included for educational purposes. Master algorithm — towards a synthesis of five approaches to machine learning TRIBE ORIGINS PROBLEM REPRESENTATION EVALUATION OPTIMIZATION MASTER ALGORITHM Symbolists Logic, philosophy Knowledge composition Logic Accuracy Inverse deduction Inductive logic programming Connectionists Neuroscience Credit assignment Neural networks Squared error Gradient descent Back propagation Evolutionaries Evolutionary biology Structure discovery Genetic programs Fitness Genetic search Genetic programming Bayesians Statistics Uncertainty Graphical models Posterior probability Bayesian optimization Probabilistic inference Analogizers Psychology Similarity Support vectors Margin Constrained optimization Kernel machines Source: Pedro Domingos 38
  39. 39. This content included for educational purposes. Natural language processing 39 This content included for educational purposes.
  40. 40. This content included for educational purposes. This research deck précis information from the Forrester Digital Transformation Conference in May 2017. It compiles selected copy and visuals from conference presentations and recent Forrester research reports. Contents are organized into the following sections: • Digital transfor Machine Learning Human CommunicaHon ArHficial Intelligence Natural Language Processing: NLP|NLU|NLG Interac_on: Dialog, gesture,
 emo_on, hap_c Audible Language:
 Speech, sound Visual Language:
 2D/3D/4D Wriben Language:
 Verbal, text Formal
 Processing Symbolic Reasoning Data Deep Learning 40 AI for human communication • Human communication encompasses every way that people exchange ideas. • Artificial intelligence is the theory and development of intelligent machines and software that can sense, learn, plan, act, understand and reason. AI performs tasks that normally require human intelligence. • Natural language processing (NLP) is the confluence of artificial intelligence (AI) and linguistics. - A key focus is the analysis, interpretation, and generation of verbal and written language. - Other language focus areas include audible & visual language, data, and interaction. • Formal programming languages enable computers to process natural language and other types of data. • Symbolic reasoning employs rules and logic to frame arguments, make inferences, and draw conclusions. • Machine learning (ML) is a area of AI and NLP that solves problems using statistical techniques, large data sets and probabilistic reasoning. • Deep learning (DL) is a type of machine learning that uses layered artificial neural networks.
  41. 41. This content included for educational purposes. 41 nat·u·ral lan·guage proc·ess·ing /ˈnaCH(ə)rəl//ˈlaNGɡwij//ˈpräˌsesˌiNG/ Natural language is spoken or wriben speech. English, Chinese, Spanish, and Arabic are examples of natural language. A formal language such as mathema_cs, symbolic logic, or a computer language isn't. Natural language processing recognizes the sequence of words spoken by a person or another computer, understands the syntax or grammar of the words (i.e., does a syntac_cal analysis), and then extracts the meaning of the words. Some meaning can be derived from a sequence of words taken out of context (i.e., by seman_c analysis). Much more of the meaning depends on the context in which the words are spoken (e.g., who spoke them, under what circumstances, with what tone, and what else was said, par_cularly before the words), which requires a pragma_c analysis to extract meaning in context. Natural language technology processes queries, answers questions, finds information, and connects users with various services to accomplish tasks. What is natural language processing? NLP
  42. 42. This content included for educational purposes. Aoccdrnig to a rseearch taem at Cmabrigde Uinervtisy, it deosn't mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer be in the rghit pclae. The rset can be a taotl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe. 42 This content included for educational purposes.
  43. 43. This content included for educational purposes. How natural language interpretation & natural language generation happens 43 This content included for educational purposes.
  44. 44. This content included for educational purposes. Text analytics 44 Text mining is the discovery by computer of new, previously unknown information, by automatically extracting it from different written resources. A key element is the linking together of the extracted information together to form new facts or new hypotheses to be explored further by more conventional means of experimentation. Text analytics is the investigation of concepts, connections, patterns, correlations, and trends discovered in written sources. Text analytics examine linguistic structure and apply statistical, semantic, and machine-learning techniques to discern entities (names, dates, places, terms) and their attributes as well as relationships, concepts, and even sentiments. They extract these 'features' to databases or semantic stores for further analysis, automate classification and processing of source documents, and exploit visualization for exploratory analysis. IM messages, email, call center logs, customer service survey results, claims forms, corporate documents, blogs, message boards, and websites are providing companies with enormous quantities of unstructured data — data that is information-rich but typically difficult to get at in a usable way. Text analytics goes beyond search to turn documents and messages into data. It extends Business Intelligence (BI) and data mining and brings analytical power to content management. Together, these complementary technologies have the potential to turn knowledge management into knowledge analytics.
  45. 45. This content included for educational purposes. Speech I/O vs NLP vs NLU NLP NLU syntactic parsing machine translation named entity recognition (NER) part-of-speech tagging (POS) semantic parsing relation extraction sentiment analysis coreference resolution dialogue agents paraphrase & natural language inference text-to- speech (TTS) summarization automatic speech recognition (ASR) text categorization question answering (QA) Speech I/O 45This content included for educational purposes.
  46. 46. This content included for educational purposes. Natural language understanding (NLU) Natural language understanding (NLU) involves mapping a given natural language input into useful representations, and analyzing different aspects of the language. NLU is critical to making making AI happen. But language is more than words, and NLU involves more than lots of math to facilitate search for matching words. Language understanding requires dealing with ideas, allusions, inferences, with implicit but critical connections to the ongoing goals and plans. To develop models of NLU effectively, we must begin with limited domains in which the range of knowledge needed is well enough understood that natural language can be interpreted within the right context. One example is in mentoring in massively delivered educational systems. If we want to have better educated students we need to offer them hundreds of different experiences to choose from instead of a mandated curriculum. A main obstacle to doing that now is the lack of expert teachers. We can build experiential learning based on simulations and virtual reality enabling student to pursue their own interests and eliminate the “one size fits all curriculum.” To make this happen expertise must be captured and brought in to guide from people at their time of need. A good teacher (and a good parent) can do that, but they cannot always be available. A kid in Kansas who wants to be an aerospace engineer should get to try out designing airplanes. But a mentor would be needed. We can build AI mentors in limited domains so it would be possible for a student anywhere to learn to do anything because the AI mentor would understand what a user was trying to accomplish within the domain and perhaps is struggling with. The student could ask questions and expect good answers tailored to the student’s needs because the AI/NLU mentor would know exactly what the students was trying to do because it has a perfect model of the world in which the student was working, the relevant expertise needed, and the mistakes students often make. NLU gets much easier when there is deep domain knowledge available. Source: Roger C Shank 46
  47. 47. This content included for educational purposes. Machine reading & comprehension AI machine learning is being developed to understand social media, news trends, stock prices and trades, and other data sources that might impact enterprise decisions. 47
  48. 48. This content included for educational purposes. Natural Language GeneraHon Natural language generation (NLG) is the process of producing meaningful phrases and sentences in the form of natural language from some internal representation, and involves: • Text planning − It includes retrieving the relevant content from knowledge base. • Sentence planning − It includes choosing required words, forming meaningful phrases, setting tone of the sentence. • Text realization − It is mapping sentence plan into sentence (or visualization) structure, followed by text-to-speech processing and/or visualization rendering. • The output may be provided in any natural language, such as English, French, Chinese or Tagalog, and may be combined with graphical elements to provide a mul_modal presenta_on. • For example, the log files of technical monitoring devices can be analyzed for unexpected events and transformed into alert-driven messages; or numerical _me-series data from hospital pa_ent monitors can be rendered as hand-over reports describing trends and events for medical staff star_ng a new shis. 48
  49. 49. This content included for educational purposes. Knowledge representation 
 (KR) • Knowledge = theory + information • Knowledge encoding via patterns and language • Spectrum of knowledge representation and reasoning 49 This content included for educational purposes.
  50. 50. This content included for educational purposes. This diagram reflects a philosopher's tradi_onal picture and our acquired defini_ons of knowledge. The scope of knowledge is everything that has ever been thought or ever can be. On the right (in blue) are all observa_ons and measurements of the physical universe, the facts that characterize reality -- past, present, and future. Within its bounds you find every object, every quantum of energy, every _me and event perceived or perceivable by our senses and instrumenta_on. This situa_onal knowledge of physical reality is “informa_on” in the sense of Shannon. It is a world of singular and most “par_cular” things and facts upon which we might figura_vely scratch some serial number or other iden_fying mark. On the other side (in red) are all the “concepts” or ideas ever imagined -- by human's, by animals and plants, by Mickey Mouse, or a can of peas. We can “imagine” a can of peas thinking. It embraces every mode of visualizing and organizing and compelling the direc_on of our thoughts from logic to religion to economics to poli_cs to every reason or ra_onale for making dis_nc_ons or for pu‚ng one thing before another. What is knowledge? “Knowledge is anything that resolves uncertainty. Knowledge is measured mathematically by the amount of uncertainty removed. Knowledge bases are defined by the questions they must answer. Source: R.L. Ballard 50
  51. 51. This content included for educational purposes. As the next internet gains momentum, expect rapid progress towards a universal knowledge technology that provides a full spectrum of informa_on, metadata, seman_c modeling, and advanced reasoning capabili_es for any who want it. Large knowledgebases, complex forms of situa_on assessment, sophis_cated reasoning with uncertainty and values, and autonomic and autonomous system behavior exceed the capabili_es and performance capacity of current descrip_on logic-based approaches. Universal knowledge technology will be based on a physical theory of knowledge that holds that knowledge is anything that decreases uncertainty. The formula is: Knowledge = Theory + InformaMon that reduces uncertainty. Theories are the condi_onal constraints that give meaning to concepts, ideas and thought paberns. Theory asserts answers to “how”, “why” and “what if” ques_ons. For humans, theory is learned through encultura_on, educa_on, and life experience. InformaHon, or data, provides situa_on awareness — who, what, when, where and how-much facts of situa_ons and circumstances. Informa_on requires theory to define its meaning and purpose. Theory persists and always represents the lion’s share of knowledge content — say 85%. Informa_on represents a much smaller por_on of knowledge — perhaps only 15% What will dis_nguish universal knowledge technology is enabling both machines and humans to understand, combine, and reason with any form of knowledge, of any degree of complexity, at any scale. Knowledge = theory + informa_on Knowledge = theory + information that reduces uncertainty 51
  52. 52. This content included for educational purposes. • Knowledge representation is the application of theory, values, logic, and ontology to the task of constructing computable patterns in some domain. • Knowledge is “captured and preserved”, when it is transformed into a perceptible and manipulable system of representation. Systems of knowledge representation differ in their fidelity, intuitiveness, complexity, and rigor. • The computational theory of knowledge predicts that ultimate economies and efficiencies can be achieved through variable-length, n-ary concept coding and pattern reasoning resulting in designs that are linear and proportional to knowledge measure. What is knowledge representation? 52
  53. 53. This content included for educational purposes. Symbolic methods • Declarative languages (Logic) • Imperative languages 
 C, C++, Java, etc. • Hybrid languages (Prolog) • Rules — theorem provers, expert systems • Frames — case-based reasoning, model-based reasoning • Semantic networks, ontologies • Facts, propositions Symbolic methods can find information by inference, can explain answer Non-Symbolic methods • Neural networks — knowledge encoded in the weights of the neural network, for embeddings, thought vectors • Genetic algorithms • graphical models — baysean reasoning • Support vectors Neural KR is mainly about perception, issue is lack of common sense (there is a lot of inference involved in everyday human reasoning Knowledge Representation
 and Reasoning Knowledge representation and reasoning is: • What any agent—human, animal, electronic, mechanical—needs to know to behave intelligently • What computational mechanisms allow this knowledge to be manipulated? 53
  54. 54. This content included for educational purposes. Knowledge encoding Natural language Documents, speech, stories Visual language Tables, graphics, charts, maps, illustrations, images Formal language Models, schema, logic, mathematics, professional and scientific notations Behavior language Software code, declarative specifications, functions, algorithms Sensory language User experience, human-computer interface, haptic, gestic. Humans encode thoughts, represent knowledge, and share meanings using paberns and language. PaNerns are knowledge units. A pabern is a compact and rich in seman_cs representa_on of raw data. Seman_c richness is the knowledge a pabern reveals that is hidden in the huge quan_ty of data it represents. Compactness is the correla_ons among data and the synthe_c, high level descrip_on of data characteris_cs. For example, an image. Language is a system of signs, symbols, gestures, and rules used in communica_ng. Meaning is something that is conveyed or signified. Humans have plenty of experience encoding thoughts and meanings using language in one form or another… Our proficiency varies. We tend to be beber at some kinds of language, and not so good at others. Project teams osen combine different skills and exper_se, e.g. to make a movie; design and construct a building; or coordinate response to an emergency. The table to the right gives examples of five forms of human language: natural, visual, formal, behavioral, and sensory language. 54
  55. 55. This content included for educational purposes. Knowledge encoding: a key limitation of natural language is 
 the inherent ambiguity resulting from overloaded symbol use. "A noun is a sound that has meaning only by conven_on. There is no natural rela_onship between any idea or observa_on and the sound that you uber to describe it." Aristotle — On InterpretaMon 55 Source: Gary Larson, The Far Side.
  56. 56. This content included for educational purposes. Knowledge encoding: visual language consists of words, images and shapes, tightly integrated into communication units 56 Source: Robert Horn Visual language is the _ght integra_on of words, images, and shapes to produce a unified communica_on. It is a tool for crea_ve problems solving, problem analysis, and a way of conveying ideas and communica_ng about the complexi_es of our technology and social ins_tu_ons. Visual language can be displayed on different media and different size communica_on units. Visual language is being created by the merger of vocabularies from many different fields as shown in the diagram to the lower right. As the world increases in complexity, as the speed at which we need to solve business and social problems increases, as it becomes increasingly cri_cal to have the “big picture” as well as mul_ple levels of detail immediately accessible, visual language will become more prevalent in our lives. The internet of subjects, services and things will evolve seman_cally enabled tools for visual language. Computers will cease being mere electronic pencils, and be used to author, manage, and generate visual language as a form of shared executable knowledge.
  57. 57. This content included for educational purposes. Theory is any condi_onal or uncondi_onal asser_on, axiom or constraint 
 used for reasoning about the world. It may be any conjecture, opinion, or specula_on. In this usage, a theory is not necessarily based on facts and may or may not be consistent with verifiable descrip_ons of reality. We use theories to reason about the world. In this sense, theory is a set of interrelated constructs — formulas and inference rules and a rela_onal model (a set of constants and a set of rela_ons defined on the set of constants). "The ontology of a theory consists in the objects theory assumes 
 there to be." 
 -- Quine -- Word and Object, 1960 Theories are accepted or rejected as a whole. If we choose to accept and use a theory for reasoning, then we must commit to all the ideas and rela_onships the theory needs to establish its existence. In science, theory is a proposed ra_onal descrip_on, explana_on, or model of the manner of interac_on of a set of natural phenomena. Scien_fic theory should be capable of predic_ng future occurrences or observa_ons of the same kind, and capable of being tested through experiment or otherwise falsified through empirical observa_on. Values for theory construc_on include that theory should: add to our understanding of observed phenomena by explaining them in the simplest form possible (parsimony); fit cleanly with observed facts and with established principles; be inherently testable and verifiable; and imply further inves_ga_ons and predict new discoveries. Theory Claude Shannon 57 Turing Science Museum
  58. 58. This content included for educational purposes. Structured, semi-structured, and unstructured are types of data representa_ons that seman_c technologies unify. Structured informaHon is informa_on that is understandable by computers. Data structures (or data models) include: relaMonal — tabular formats for data are most prevalent in database systems, and operate best for storage and persistence; hierarchical — tree-like formats (including XML) are most prevalent in document models, and operate best in messaging systems (including SOA); and object — frame systems like Java and C# combine behavior with data encapsula_on, and operate best for compiled sosware programs. Semi-structured informaHon is data that may be irregular or incomplete and have a structure that changes rapidly or unpredictably. The schema (or plan of informa_on contents) is discovered by parsing the data, rather than imposed by the data model, e.g. XML markup of a document. Unstructured informaHon is not readily understandable by machines. Its sense must be discovered and inferred from the implicit structure imposed by rules and conven_ons in language use, e.g. e-mails, lebers, news ar_cles. Data representa_ons Examples of data models. 58
  59. 59. This content included for educational purposes. The fundamental shis in the connected intelligence era is from informa_on-centric to knowledge-centric compu_ng that integrates four innova_on dimensions depicted here: • Intelligent user experience — concerns how I experience things, demands on my aben_on, my personal values. Trend towards exploi_ng higher bandwidth content dimensionality, context sensi_vity, and reasoning power in the user interface. • SemanMc social compuMng — concerns our lived culture, intersubjec_ve shared values, & how we communicate. Trend towards collabora_ve tooling that empowers we humans (and our computers) to co-develop, share, and exploit knowledge in all its forms (e.g., content, models, services, and behaviors). • CogniMve applicaMons, and things — concerns objec_ve things such as product structure & behavior viewed empirically. Trend towards hi-bandwidth, intelligent, autonomic, autopoie_c, and autonomously communica_ng digital products, services, and intellectual property. • CogniMve infrastructure — concerns interobjec_ve network- centric systems and ecosystems. Trend towards, everything self- aware, somewhat intelligent, connected and socially autopoie_c, and capable of solving problems of complexity, scale, security, trust, and change management. Integra_ng knowledge across different domains. Source: Ken Wilber, Integral Institute & Mills Davis, Project10X Concept Computing Information Technology 1st Person (I) Subjective 2nd Person (WE) Social 4th Person (ITS) Systemic 3rd Person (IT) Objective Individual I n t e r i o r E x t e r i o r Collective 59
  60. 60. This content included for educational purposes. 60 Knowledge representation & reasoning:
 from search to knowing More expressive knowledge representation (vertical axis) enables more powerful reasoning (horizontal axis): • Representations from lists to dictionaries, glossaries and lexicons; to taxonomies; to thesauri; to models; to semantic data and models; to ontologies • Reasoning from recovery, to discovery, to intelligence, to question answering, to smart behaviors.
  61. 61. This content included for educational purposes. Dic_onaries, glossaries and lexicons 61 DicHonaries are alphabe_cal lists of terms and their defini_ons that provide variant senses for each term, where applicable. They are more general in scope than a glossary. They may also provide informa_on about the origin of the term, variants (both by spelling and morphology), and mul_ple meanings across disciplines. While a dic_onary may also provide synonyms and through the defini_ons, related terms, there is no explicit hierarchical structure or abempt to group terms by concept. A gazeNeer is a dic_onary of place names. Tradi_onal gazebeers have been published as books or they appear as indexes to atlases. Each entry may also be iden_fied by feature type, such as river, city, or school. Geospa_ally referenced gazebeers provide coordinates for loca_ng the place on the earth’s surface. A glossary is a list of terms, usually with defini_ons. The terms may be from a specific subject field or those used in a par_cular work. The terms are defined within that specific environment and rarely have variant meanings provided. A lexicon is a knowledge base about some subset of words in the vocabulary of a natural language. One component of a lexicon is a terminological ontology whose concept types represent the word senses in the lexicon. The lexicon may also contain addi_onal informa_on about the syntax, spelling, pronuncia_on, and usage of the words. Besides conven_onal dic_onaries, lexicons include large collec_ons of words and word senses, such as WordNet from Princeton University and EDR from the Japan Electronic Dic_onary Research Ins_tute, Ltd. Other examples include classifica_on schemes, such as the Library of Congress subject headings or the Medical Subject Headers (MeSH).
  62. 62. This content included for educational purposes. Taxonomy 62 A taxonomy is a hierarchical or associa_ve ordering of terms represen_ng categories. A taxonomy takes the form of a tree or a graph in the mathema_cal sense. A taxonomy typically has minimal nodes, represen_ng lowest or most specific categories in which no sub-categories are included as well as a top-most or maximal node or la‚ce, represen_ng the maximum or general category. Source: Denise Bedford, World Bank Examples of taxonomy.
  63. 63. This content included for educational purposes. Folk taxonomy 63 A folk taxonomy is a category hierarchy with 5-6 levels that has its most cogni_vely basic categories in the middle. In folk taxonomies, categories are not merely organized in a hierarchy from the most general to the most specific, but are also organized so that the categories that are most cogni_vely basic are “in the middle” of a general- to-specific hierarchy. Generaliza_on proceeds upward from the basic level and specializa_on proceeds down. A basic level category is somewhere in the middle of a hierarchy and is cogni_vely basic. It is the level that is learned earliest. Usually has a short name and is used frequently. It is the highest level at which a single mental image can reflect the category. Also, there is no defini_ve basic level for a hierarchy – it is dependent on the audience. Most of our knowledge is organized around basic level categories. Source: George Lakoff
  64. 64. This content included for educational purposes. Thesaurus 64 A thesaurus is a compendium of synonyms and related terms. It organizes knowledge based on concepts and rela_onships between terms. Rela_onships commonly expressed in a thesaurus include hierarchy, equivalence, and associa_ve (or related). These rela_onships are generally represented by the nota_on BT (broader term), NT (narrower term), SY (synonym), and RT (associa_ve or related). Associa_ve rela_onships may be more granular in some schemes. For example, the Unified Medical Language System (UMLS) from the Na_onal Library of Medicine has defined over 40 rela_onships across more than 80 vocabularies, many of which are associa_ve in nature. Preferred terms for indexing and retrieval are iden_fied. Entry terms (or non- preferred terms) point to the preferred terms that are to be used for each concept.
  65. 65. This content included for educational purposes. Model 65 A model is a representa_on of an actual or conceptual system. It involves mathema_cs, logical expressions, or computer simula_ons that can be used to predict how the system might perform or survive under various condi_ons or in a range of hos_le environments. A simulaHon is a method for implemen_ng a model. It is the process of conduc_ng experiments with a model for the purpose of understanding the behavior of the system modeled under selected condi_ons or of evalua_ng various strategies for the opera_on of the system within the limits imposed by developmental or opera_onal criteria. Simula_on may include the use of analog or digital devices, laboratory models, or “testbed” sites. Examples of models.
  66. 66. This content included for educational purposes. 66This content included for educational purposes.
  67. 67. This content included for educational purposes. Seman_c graph 67 Seman_c networks are ontologies. They are like and unlike other IT models. Like databases, ontologies are used by applica_ons at run _me (queried and reasoned over). Unlike conven_onal databases, rela_onships are first-class constructs. Like object models, ontologies describe classes and abributes (proper_es). Unlike object models, ontologies are set-based and dynamic. Like business rules, seman_c models encode event-based behaviors. Unlike business rules, ontologies organize rules using axioms. Like XML schemas, they are na_ve to the web (and are in fact serialized in XML). Unlike XML schemas, ontologies are graphs not trees, and used for reasoning. People some_mes refer to ontologies as the“O” word, thinking that knowledge models are abstract and scary. Actually, seman_cs is something that every human being already knows very well. We’ve been figuring out what things mean all our lives. Don’t let the nota_on fool you. Any ontology can be expressed clearly in plain English (or other natural language of your choosing).
  68. 68. This content included for educational purposes. Ontology 68 An ontology is a formal explicit specification of a shared conceptualization. An ontology defines the terms and axioms used to describe, represent, and reason about an area of knowledge (subject matter). It is the model (set of concepts) for the meaning of those terms. It defines the vocabulary and the meaning of that vocabulary as well as the assertions, rules, and constraints used in reasoning about this subject matter. An ontology is used by people, databases, and applications that need to share domain information. A domain is a specific subject area or area of knowledge, like medicine, tool manufacturing, real estate, automobile repair, financial management, etc. Ontologies include computer-usable definitions of basic concepts in the domain and the relationships among them. They encode domain knowledge (modular). Knowledge that spans domains (composable). They make knowledge available (reusable). Ontologies are usually expressed in a logic-based language that enables detailed, sound, meaningful distinctions to be made among the classes, properties, & relations as well as inferencing across the knowledge model. Source: Leo Obrst Source: Tom Gruber Source: Andreas Schmidt The diagram above shows that shared ideas and knowledge can be expressed with different degrees of formality.
  69. 69. This content included for educational purposes. REASONING 69 This content included for educational purposes.
  70. 70. This content included for educational purposes. • Reasoning is the derivation of inferences and the warranting of conclusions through application of heuristics, rules, analogies, mathematics, logic, and values. • Reasoning requires knowledge representation. We choose more powerful forms of representation to enable more powerful kinds of reasoning and problem solving. • A broad range of reasoning capabilities exist including pattern detection and machine learning; deep linguistics; ontology and model based inferencing; and reasoning with uncertainties, conflicts, causality, analogies, and values. What is reasoning? 70
  71. 71. This content included for educational purposes. 71 Continuum of machine reasoning and decision making Artificial intelligence describes software that dynamically choses the optimal combination of methods for the solution of a problem, often at very short temporal scales.
  72. 72. This content included for educational purposes. ▪ Statistics is the study of the collection, analysis, interpretation, presentation, and organization of data. ▪ Two main statistical methodologies are used in data analysis: descriptive statistics, which summarizes data from a sample using indexes such as the mean or standard deviation, and inferential statistics, which draws conclusions from data that are subject to random variation (e.g., observational errors, sampling variation). ▪ Descriptive statistics are most often concerned with two sets of properties of a distribution (sample or population): central tendency (or location) seeks to characterize the distribution's central or typical value, while dispersion (or variability) characterizes the extent to which members of the distribution depart from its center and each other. ▪ Inferences on mathematical statistics are made under the framework of probability theory, which deals with the analysis of random phenomena. 72 Statistical inference “He told me I was average.
 I told him he was mean.”
  73. 73. This content included for educational purposes. Similarity Search • Similarity search provides a way to find the objects that are the most similar, in an overall sense, to the object(s) of interest. • A typical example is that of a doctor finding the top 10 past pa_ents who are most similar to the current pa_ent of interest. This could be used for diagnosis, but also adds the human judgement that some other machine learning methods do not necessarily offer. Another example of approximate similarity search is for finding the song in a database corresponding to a given sound sample, or finding the person in a database corresponding to a face photo. • Similarity searches can be thought of as mul_dimensional analogs to SQL queries. SQL queries are composed of condi_ons on individual variables, for example “Find all customers whose age is within a certain range and whose income is greater than a certain amount”, whereas similarity searches are more like “Find all the customers most like this one”. 73
  74. 74. This content included for educational purposes. Analy_cs 74 AnalyHcs studies the cons_tuent parts of something and their rela_on to the whole. Analy_cs seeks to iden_fy and interpret paberns in informa_on derived from mul_ple sources. Seman_c technologies provide the capability to dynamically map together heterogeneous data sets, bridging silos and making interrela_onships explicit and computable. The chart to the right maps different types of analy_cs by purpose and _me horizon. The purpose axis (ver_cal) dis_nguishes "explora_on vs. control" and highlights the difference between analysis and repor_ng. Analysis is about digging deep into data to discover rela_onships, find causa_on, and describe phenomena. Repor_ng, in contrast, is used to track performance and iden_fy varia_on from goals. The temporal axis (horizontal) dis_nguishes “backward looking vs. forward looking vs. real-_me). Most analy_cs is backward looking — in an abempt to understand what has happened, and therefore be equipped to make beber decisions in the future. Alterna_vely, analy_cs can focus explicitly on predic_ng future performance. Increasingly, however, the requirement is to provide informa_on and insight to support opera_onal decisions in real-_me. Source: Zach Gemignani
  75. 75. This content included for educational purposes. Predictive Analytics • PredicHve analyHcs is the science of analyzing current and historical facts/data to make predic_ons about future events. • Unlike tradi_onal business intelligence prac_ces, which are more backward-looking in nature, predic_ve analy_cs is focused on helping companies derive ac_onable intelligence based on past experience. • A typical applica_on is in insurance: predic_ng which policy holders (or poten_al policy holders) will make a claim and how long it will be un_l they make the claim. The more data available on the history of claims and ‘extraneous’ informa_on about the policy holder the more variables a predic_ve analy_cs algorithm can take in to account. • For example, a machine learning algorithm could easily take into account the impact of when a parent has children on claim rates. Iden_fying if such a rela_onship exists (amongst ALL the other possibili_es) is too complex for human analysts. • Another example is a health insurance company using predic_ve analy_cs to iden_fy when pa_ents are likely to have a hospital stay – and to direct health care providers to take preventa_ve ac_ons to avoid the hospital stay. With a growing base of health care data, this sort of data science is set to improve the nature of health care delivery. • Other examples include predic_on of product demand, op_ons prices, or turnover likelihood of sales leads. 75
  76. 76. This content included for educational purposes. • Predic_ve analy_cs is the science of analyzing current and historical facts/data to make predic_ons about future events. • Unlike tradi_onal business intelligence prac_ces, which are more backward-looking in nature, predic_ve analy_cs is focused on helping companies derive ac_onable intelligence based on past experience. • A typical applica_on is in insurance: predic_ng which policy holders (or poten_al policy holders) will make a claim and how long it will be un_l they make the claim. The more data available on the history of claims and ‘extraneous’ informa_on about the policy holder the more variables a predic_ve analy_cs algorithm can take in to account. • For example, a machine learning algorithm could easily take into account the impact of when a parent has children on claim rates. Iden_fying if such a rela_onship exists (amongst ALL the other possibili_es) is too complex for human analysts. • Another example is a health insurance company using predic_ve analy_cs to iden_fy when pa_ents are likely to have a hospital stay – and to direct health care providers to take preventa_ve ac_ons to avoid the hospital stay. With a growing base of health care data, this sort of data science is set to improve the nature of health care delivery. • Other examples include predic_on of product demand, op_ons prices, or turnover likelihood of sales leads. 76This content included for educational purposes.
  77. 77. This content included for educational purposes. Four kinds of reasoning 77 The four methods of reasoning include: (1) DeducHon: deriving implica_ons from premises. (2) InducHon: deriving general principles from examples. (3) AbducHon: Forming a hypothesis that must be tested by induc_on and deduc_on. It involves inferring the best or most plausible explana_on from a given set of facts or data. (4) Analogy: Besides these three types of reasoning there is a fourth, analogy, which combines the characters of the three, yet cannot be adequately represented as composite. Analogy is more primi_ve, but more flexible than logic. The methods of logic are disciplined ways of using analogy. Although deduc_on is important, it is only one of the four methods of reasoning. Induc_on, abduc_on, and analogy are at least as important, and they are necessary for learning or acquiring new knowledge. Current computer systems come close to human ability in deduc_on. But they are far inferior in learning, which depends heavily on the other three methods of reasoning. Source: John Sowa
  78. 78. This content included for educational purposes. Evidence-based reasoning • This evidence-based reasoning framework shows how a premise and data are processed through three distinct steps (analysis, interpretation, and application) to produce a claim as the output. • A claim is a statement about a specific outcome or state phrased as either a prediction of what something will do in the future (e.g., “This box will sink”), an observation of what something has done in the past (e.g., “This box sank”), or a conclusion about what something is in the present (e.g., “This box sinks”). • The premise consists of one or more statements describing the specific circumstances acting as an input that will result in the outcome described by the claim. • Rules link the premise and the claim, asserting a general relationship that justifies how the latter follows from the former. Application is the process by which the rule are brought to bear in the specific circumstances described by the premise. • Evidence consist of statements describing observed relationships. Interpretation of evidence is a process of generalization, grounded in a specific context. • Data are discrete reports of past or present observations, and are collected and related by analysis to produce a statement of evidence. 78 Source: EBR Framework: Assessing Scientific Reasoning,
 Taylor & Francis Group
  79. 79. Axiology, logic & probability • Value is the founda_on of meaning. It is the measure of the worth or desirability (posi_ve or nega_ve) of something, and of how well something conforms to its concept or intension. • Value forma_on and value-based reasoning are fundamental to all areas of human endeavor. Theories embody values. The axiom of value is based on “concept fulfillment.” • Most areas of human reasoning require applica_on of mul_ple theories; resolu_on of conflicts, uncertain_es, compe_ng values; and analysis of trade-offs. For example, ques_ons of guilt or innocence require judgment of far more than logical truth or falsity. • Axiology is integral to the evolu_on of AI. Axiology is the branch of philosophy that studies value and value theory. Things like honesty, truthfulness, objec_veness, novelty, originality, “progress,” people sa_sfac_on, etc. The word ‘axiology’, derived from two Greek roots 'axios’ (worth or value) and ‘logos’ (logic or theory), means the theory of value, and concerns the process of understanding values and valua_on. This content included for educational purposes. 79
  80. 80. This content included for educational purposes. • Most predictive analysis today is done with machine learning and statistical methods, so using this alone is not novel. • Semantic reasoning can be used alongside to guide and validate machine learning methods, catch outliers, and explain how and why predictions were made. • A large class of problems exist where hybrid AI approaches can be applied to improve outcomes provided that means are available to: - Standardize access to multiple AI engines and enable collaboration between them to answer the same query - Compare and combine the results to improve accuracy - Explain the “why” of the results and recommend ways to improve them with different methods and data Neural-symbolic fusion combines statistical correlation and semantic reasoning to deliver unique insight. People Who Will People Who Will Not People Predicted By Semantic Reasoning People Predicted By Statistical Correlation MAXIMIZE Combining statistical and symbolic reasoning 80
  81. 81. This content included for educational purposes. 81 Yann LeCun
 Director, AI Research
 Facebook “Predictive learning is the next frontier for AI” Deep learning has been at the root of significant progress in many application areas, such as computer perception and natural language processing. But almost all of these systems currently use supervised learning with human-curated labels. The challenge of the next several years is to enable machines to learn by themselves about any domain from raw, unlabeled data, such as images, videos and text. The problem is that intelligent systems today do not possess "common sense", which humans and animals acquire by observing the world, acting in it, and understanding the physical constraints of it. Enabling machines to learn predictive models of the world where predictability is only partial, is key to making significant progress in artificial intelligence, and a necessary component of model-based planning and reinforcement learning. Predictive learning is the next frontier for AI.
  82. 82. This content included for educational purposes. 82 /ˈk.ɡnədiv//kəmˈpyo͞odiNG/ noun Cognition is the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses. Cognitive computing is the simulation of human thought processes in a computerized model. It involves self-learning systems that use data mining, pattern recognition, natural language processing, and statistical and symbolic reasoning to mimic the way that humans think and learn. What is 
 cognitive computing?
  83. 83. This content included for educational purposes. This diagram maps cognitive technologies by how autonomously they work, and the tasks they perform. It shows the current state of smart machines—and anticipates how future technologies might unfold. SPRING 2016 MIT SLOAN MANAGEMENT REVIEW WHAT TODAY’S COGNITIVE TECHNOLOGIES CAN — AND CAN’T — DO Mapping cognitive technologies by how autonomously they work and the tasks they perform shows the current state of smart machines — and anticipates how future technologies might unfold. LEVELS OF INTELLIGENCE TASK TYPE SUPPORT FOR HUMANS REPETITIVE TASK AUTOMATION CONTEXT AWARENESS AND LEARNING SELF-AWARENESS T G C Analyze Numbers Business intelligence, data visualization, hypothesis-driven analytics Operational analytics, scoring, model management Machine learning, neural networks Not yet Analyze Words and Images Character and speech recognition Image recognition, machine vision IBM Watson, natural language processing Not yet Perform Digital Tasks Business process management Rules engines, robotic process automation Not yet Not yet Perform Physical Tasks Remote operation of equipment Industrial robotics, collaborative robotics Autonomous robots, vehicles Not yet Source: MIT Sloan Management Review, Spring 2016 What today’s cognitive technologies can and cannot do 83
  84. 84. AI: ACT
  85. 85. This content included for educational purposes. 85 • Building blocks and levels of intelligent action • Levels of intelligent action • Search and question answering • Rules engines • Expert systems • Recommender systems • Automated planning and scheduling systems • Robotic process automation • Autonomic computing • Autonomous systems Overview of
 AI: Act
  86. 86. This content included for educational purposes. Four building blocks for intelligent action Source: HfS Software development toolkit allows non-engineers quickly to create software robots to automate rules-driven business processes. E.g. digitize process of collecting of unpaid invoices, mimicking manual activities in the RPA software, the integration of electronic documents and generation of automated emails to ensure the whole collections, process is run digitally and can be repeated in a hi-throughput, high intensity model. Simulating human thought in a set of processes. It involves self-learning systems, data mining, pattern recognition, and natural language processing to mimic the way the human brain works. E.g. an insurance adjudication system that assesses claims, based on scanned documents and available data from similar claims and evaluates payment awards. Self-learning and self- remediating engines. System makes autonomous decisions, using hi-level policies. Constantly monitors, adapts and optimizes its performance as conditions change and business rules evolve. E.g. a virtual support agent continuously learning to handle queries and creating new rules/exceptions as products evolves and queries change. Intelligent automation systems go beyond routine business and IT process activity to make decisions and orchestrate processes. E.g. an AI system managing a fleet of self-driving cars or drones to deliver goods to clients, manage aftermarket warranties and continuously improve the supply chain. Artificial IntelligenceRobotic Process Automation AutonomicsCognitive Computing 86 This content included for educational purposes.
  87. 87. This content included for educational purposes. What intelligent systems need to possess Source: GRAKN.AI 87This content included for educational purposes.
  88. 88. This content included for educational purposes. Four levels of intelligent action 88 FIXED SEMANTIC AUTONOMIC AUTONOMOUS • Fixed interfaces • Hard-wired design _me stack • Black boxes • H2M interoperability VALUE KNOWLEDGE INTENSIVITYLow Hi LowHi •Model-based, seman_c APIs, dynamic interfaces •Self-declaring, self-defining components •Glass boxes •M2M integra_on and interoperability •Self* (awareness, provisioning, configuring, diagnosing) •Pervasive adap_vity (sense, interpret, respond) •Mobile dynamics, granular security, •M2M performance op_miza_on • Goal-oriented, 
 • Mul_-domain knowledge • Cogni_ve automa_on • Mul_-agent • M2M & predic_ve learning This content included for educational purposes.
  89. 89. This content included for educational purposes. What makes intelligent systems different? Goal-orienta_on Constraint-based 
 inferencing Context
 awareness Adap_vity Self-op_miza_on 89This content included for educational purposes.
  90. 90. This content included for educational purposes. Search 90
  91. 91. This content included for educational purposes. Search engines 91 Search engines look for relevant informa_on based on some criteria. Full-text search is fast, efficient, and simple, but delivers poor relevance in the absence of an exact keyword match. StaMsMcal search mechanisms focus on the frequency of keywords, but provide imperfect results: a keyword may be misspelled in some target documents; it may appear in a plural or conjugated form; it may be replaced by a synonym; it may have different meanings according to context. Osen, sta_s_cally-based searches return results that prove either too voluminous or too restricted to be helpful. Natural language search uses linguis_c analysis, rules, and reference knowledge to improve named en_ty extrac_on, seman_c analysis of word senses, and meaning of texts. SemanMc search expands keyword search by understanding the meaning of concepts and context of the query. It exploits knowledge about the context of the ques_on. It looks at the meaning of full sentences and documents as well as equivalent ways of saying the same thing. It recognizes the gramma_cal role played by words in a sentence (e.g., subject or object), detects the rela_onship between the parts of a sentence (objects, subjects, verbs, abributes, etc.). It exploits reference knowledge about rela_onships between concepts. Seman_c search can be cross- lingual (queries in the user language, answers in all languages). Search technologies Definition Sample vendors Boolean (extended Boole a n ) Retrieves documents based on the number of times the keywords appear in the text . Virtually all search engine s Cluster i n g Dynamically creates “clusters” of documents grouped by similarity, usually based on a statistical analysis . Autonomy, GammaSite, Vivisim o Linguistic analysis (stemming, morphology, synonym- handling, spell- checki n g ) Dissects words using grammatical rules and statistics. Finds roots, alternate tenses, equivalent terms and likely misspellings. Virtually all search engines Natural language processing (named entity extraction, semantic analysi s ) Uses grammatical rules to find and understand words in a particular category. More advanced approaches classify words by parts of speech to interpret their meaning. Albert, Inxight, InQuira Ontology (knowledge representatio n ) Formally describes the terms, concepts and interrelationships in a particular subject area. Endeca, InQuira, iPhrase, Verity Probabilistic (belief networks, inference networks, Naive Baye s ) Calculates the likelihood that the terms in a document refer to the same concept as the query. Autonomy, Recommind, Microsoft Taxonomy (categorization) Establishes the hierarchical relationships between concepts and terms in a particular search area. GammaSite, H5 Technologies, YellowBrix Vector-based (vector support machine) Represents documents and queries as arrows on a multidimensional graph— and determines relevance based on their physical proximity in that space. Convera, Google, Verity Source: Forrester Research
  92. 92. This content included for educational purposes. 92 Search engine parts This diagram looks under the hood of a search engine to iden_fy the component parts: search inputs, query matching algorithms, and types of search outputs that the user sees.
  93. 93. This content included for educational purposes. 93 Question answering Ques_on answering (QA) is more than search, more than discovery, and more than document retrieval. The first step is to analyze the natural language ques_on to determine the informa_on needed and the form that the answer should take, e.g. a factoid for “who discovered oxygen?”, a list for “what countries export oil?”, a defini_on for “what is a quasar?”. Ques_on analysis involves both linguis_c and seman_c processing, and results in targeted informa_on retrieval queries to search engines to return documents that are likely to contain elements of the answer sought. The next step is to extract and aggregate informa_on passages from these sources, then to reason about the content in order to extract or synthesize the answer, and to present it to user. Machine Learning Question Analysis Feature Engineering Ontology Analysis Question & Answer Natural Language Processing Question answering involves analyzing questions, retrieving documents, retrieving passages, and extracting answers.
  94. 94. This content included for educational purposes. Rules engine • A rules engine is a sosware system that executes one or more business rules in a run_me produc_on environment. • A business rule system enables company policies and other opera_onal decisions to be defined, tested, executed and maintained separately from applica_on code. • Rules-based systems use databases of knowledge and if- then rules to automate the process of making inferences about informa_on. 94
  95. 95. This content included for educational purposes. Expert systems KNOWLEDGE BASE Knowledge Models Smart DataRules & Theory Machine Learning & Data Analytics Knowledge Authoring & Curation Question AnsweringKnowledge Management Virtual Assistance INFERENCE
 ENGINE Decisions Actions Explanations Reporting Self-Documentation SMART USER INTERFACE • An expert system (ES) employs knowledge about its application domain and uses inferencing (reasoning) procedures to solve problems that require human competence or expertise. • An expert system contains three subsystems: an inference engine, a knowledge base, and a user interface. • The expert system reasons with the knowledge base. Its reasoning engine interprets directions, answers questions, and executes commands that result in decisions, actions, reporting, explanations, and self-documentation. • Expert systems assist and augment human decision makers. Application areas include classification, diagnosis, monitoring, process control, design, scheduling and planning, and generation of options. 95
  96. 96. This content included for educational purposes. Recommender system • Recommend — to put forward (someone or something) with approval as being suitable for a par_cular purpose or role. • RecommendaHon engines automate the process of making real-_me recommenda_ons to customers. • A simple example: an online customer who is browsing a store for one item (e.g. a power drill), places the item in their shopping cart, and is then recommended to buy a complementary item (e.g., a set of drill bits). This example is trivial. Machine learning can go further, osen uncovering unexpected buying paberns, based on unforeseen rela_onships between different customers and between different products. • Recommender systems take into account where on the site the customer had visited, their history of purchases at the site and even their social network history. It may be that the customer browsed for mortar on the last visit to the site. Perhaps the user also asked friends about selec_ng bathroom _les on Facebook. In this case it might make sense to recommend a mortar mixing abachment – since it is clear the customer is doing a _ling project. For a machine learning algorithm, iden_fying non-explicit rela_onships like this is typical. • A machine learning recommender system improves with _me. It learns from successful, and unsuccessful recommenda_ons. The same underlying technology can be used to provide customers with many other kinds of personalized experiences, based on data of many kinds. 96
  97. 97. This content included for educational purposes. Automated planning 
 and scheduling system AI systems that devise strategies and sequences of actions to meet goals and observe constraints, typically for execution by intelligent agents, autonomous robots and unmanned vehicles. 97
  98. 98. This content included for educational purposes. Robotic process automation (RPA) Captures and interprets existing means for conducting a task, processing a transaction, manipulating data, triggering responses, and communicating with other systems. This may include manual, repetitive tasks, intelligent automation of processes, and augmentation of resources. 98
  99. 99. This content included for educational purposes. Robotics • A robot is a programmable mechanical or software device that can perform tasks and interact with its environment, without the aid of human interaction. • Robotics is embracing cognitive technologies to create robots that can work alongside, interact with, assist, or entertain people. Such robots can perform many different tasks in unpredictable environments, integrating cognitive technologies such as computer vision and automated planning with tiny, high- performance sensors, actuators, and hardware. Current development efforts focus how to train robots to interact with the world in generalizable and predictable ways. • Deep learning is being used in robotics. Advances in machine perception, including computer vision, force, and tactile perception are key enablers to advancing the capabilities of robotics. Reinforcement learning helps obviate the need for large labeled data sets. 99
  100. 100. This content included for educational purposes. 100 au·to·ma·tion /ˌôdəˈmāSH(ə)n/ The use of software and equipment in a system or production process so that it works largely by itself with little or no direct human control. Robotic process automation and intelligent automation are the combination of AI and automation.What is automation?
  101. 101. This content included for educational purposes. 101 • “Automation” today can be defined as including any functional activity that was previously performed manually and is now handled via technology platforms or process automation tools like robotic process automation (RPA) platforms. • With increasing computer processing power, technology has reached a point where its ability to perform human-like tasks has become possible. • There are various names for referring to robotics in service industries such as Rapid Automation (RA), Autonomics, Robotic Process Automation, software bots, Intelligent Process Automation or even plain Artificial Intelligence. • These terms refer to the same concept: letting organizations automate current tasks as if a real person was doing them across applications and systems. • A primary opportunity for robotic process automation in the enterprise is to augment the creative problem-solving capabilities and productivity of human beings and deliver superior business results. Automation: letting organizations automate current tasks as if a real person was doing them across applications and systems.
  102. 102. This content included for educational purposes. Source: HfS - 2016 Evolving landscape of service agents and intelligent automation: • From desktop automation to RPA, to chatbot, to assistant, to virtual agent. • From enhancement of data, to augmentation of human agents, to substitution of digital labor for the human agent. Example vendors: 102
  103. 103. This content included for educational purposes. 103Source: Deloitte Manual process vs robotic process automation
  104. 104. This content included for educational purposes. 104 Robotic Desktop Automation (RDA) • Personal robots for every employee • Call center, retail, branches, back office • 20-50% improvement across large workforce groups • RDA also provides dashboards and UI enhancements Robotic Process Automation (RPA) • Unattended robots replicating 100% of work • Back office, operations, repetitive • 100% improvement across smaller sub-groups • Runs on a virtual server farm (or under your desk) Comparing robotic desktop automation (RDA) and robotic process automation (RPA)
  105. 105. This content included for educational purposes. 105 • Robotic process automation gives humans the potential of attaining new levels of process efficiency, such as improved operational cost, speed, accuracy and throughput volume, and leaving behind the repetitive and time consuming low added-value tasks. • Top drivers for implementing robotic automation beyond cost savings include: - High quality by a reduction of error rates - Time savings via better management of repeatable tasks - Scalability by improving standardization of process workflow - Integration by reducing the reliance on multiple systems/screens to complete a process - Reducing friction (increasing straight-through processing) • For example, back-office tasks do not require direct interaction with customers and can be performed more efficiently and effectively off-site or by robots. It is feasible to re-engineer hundreds of business processes with software robots that are configured to capture and interpret information from systems, recognize patterns, and run business processes across multiple applications to execute activities including data entry and validation, automated formatting, multi-format message creation, text mining, workflow acceleration, reconciliations and currency exchange rate processing among others. Robotic process automation (RPA)
  106. 106. This content included for educational purposes. 106 Intelligent process automation is smart software with machine-learning capabilities: • Unlike RPA, which must be programmed to perform a task, AI can train itself or be trained to automate more complex and subjective work through pattern recognition • Unlike RPA, which requires a human expert to hard code a script or workflow into a system, AI can process natural language and unstructured data • Unlike RPA, AI responds to a change in the environment, adapts and learns the new way Intelligent process automation (IPA)
  107. 107. This content included for educational purposes. Intelligent automation stages Source: Shahim Ahmed, CA Technologies 107
  108. 108. This content included for educational purposes. 108 Trigger based Rules-based dynamic language Rules-based standardized language Structured CHARACTERISTIC OF DATA / INFORMATION Unstructured without patternsUnstructured patterned Data Center Automation: Runbook Scripting Scheduling Job control Workload automation Process orchestration SOA Virtualization Cloud services RPA Cognitive Computing Artificial Intelligence BPM Workflow ERP Autonomics PROCESS CHARACTERISTICS Source: HfS - 2016 Intelligent automation continuum The spectrum of intelligent process automation spans robotic process automation, cognitive computing, autonomics, and artificial intelligence. The direction of travel is 
 along three dimensions. Stages overlap.
  109. 109. This content included for educational purposes. 109 Four aspects of self-management as they are now 
 and as they become with autonomic computing Concept Current computing Autonomic computing Self-configuration Corporate data centers have multiple vendors and platforms. Installing, configuring, and integrating systems is time consuming and error prone. Automated configuration of components and systems follows high-level policies. Rest of system adjusts automatically and seamlessly. Self-optimization Systems have hundreds of manually set, nonlinear tuning parameters, and their number increases with each release. Components and systems continually seek opportunities to improve their own performance and efficiency. Self-healing Problem determination in large, complex systems can take a team of programmers weeks. System automatically detects, diagnoses, and repairs localized software and hardware problems. Self-protection Detection of and recovery from attacks and cascading failures is manual. System automatically defends against malicious attacks or cascading failures. It uses predictive analytics and early warning to anticipate and prevent systemwide failures. Autonomic computing Autonomic computing refers to the self-managing characteristics of AI-based distributed computing resources, adapting to unpredictable changes while hiding intrinsic complexity to operators and users.
  110. 110. This content included for educational purposes. Autonomous vehicles Everyone (Google, Baidu, Apple, NVidia, Uber, Tesla, Volvo, Kamaz, Mercedes- Benz, etc.) is developing their own autonomous car. Automobiles will soon become really auto- mobile. The main restriction here seems to be laws and regulations. 110
  111. 111. This content included for educational purposes. DRONE Autonomous drones Controlling different things now seems efficient using deep learning (e.g., games, game characters, drones, autonomous cars, robotic control.) 111
  112. 112. This content included for educational purposes. Artificial intelligence — issues In order to work on real AI, as opposed to the hype presented by large companies and the media these days, the following problems must be worked on. 1. Knowledge Representation: This has always been the biggest problem in AI but serious work on it stopped on it in the mid 80’s in favor of easy to extract large, shallow libraries of lexical information. 2. Complex Models of Goals and Plans: In order to help and learn, an intelligent system (a dog, a human or a computer) needs to know about goals, and plans to achieve those goals, common mistakes with plans it has tried in the past, and how to explain and learn from those mistakes. 3. Human-Like Models of Memory: Humans update their memory with every interaction. They learn. Every experience changes their world model. In order to build real AI we need to focus on limited domains of knowledge in which the goals and plans of actors are represented and understood so that they can be acted upon or acted against. AI systems must learn from their own experiences, not learn by having information fed into them. 4. Conversational Systems: In practice, this means being able to build a program that can hold up its end of a conversation with you. (unlike Siri or any travel planning program). Such systems, should be linked to a memory of stories (typically no more than 1.5 minutes in length and in video) from the best and brightest people in the world. Those stories should “find” the user when the program knows that they would be helpful. This happens every day in human interaction. One person talks to another person about what they are thinking or working on and the other person reacts with a just-in-time reminding, a story that came to mind because it seemed relevant to tell at the time, a story meant to help the other person think things out. 5. Reminding: A computer in a situation must get reminded of relevant situations it has previously experienced to guide it in its actions. This is real AI. Done on a massive scale, this means capturing the expertise in a any given domain by inputting stories and indexing those stories with respect to what goals and plans and contexts they might pertain so that they can be delivered just in time to a user. We can do this now to some extent, but we need to start working on the real AI problems of automated indexing of knowledge. (Although this may be what machine learning people say they can do, they are talking about words and they are not trying to build an ever increasingly complex world model as humans do through daily life.) Source: Roger C Shank 112
  113. 113. This content included for educational purposes. AI encompasses multiple technologies that can be combined to sense, think, and act as well as to learn from experience and adapt over time. Sense refers to pattern recognition, machine perception, speech recognition, computer vision and affective computing. Think refers to natural language processing, knowledge representation and reasoning, machine learning and deep learning, and cognitive computing. Act refers to search engines and question answering, rules engines, expert systems, recommender systems, automated planning and scheduling, autonomic computing, and autonomous systems. 113
  114. 114. © Copyright Project10x | Confidential