Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

Multiskill Conversational AI

Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Anuncio
Cargando en…3
×

Eche un vistazo a continuación

1 de 20 Anuncio

Más Contenido Relacionado

Más de Daniel Kornev (14)

Más reciente (20)

Anuncio

Multiskill Conversational AI

  1. 1. Multiskill Conversational AI Mikhail Burtsev, PhD, Founder Daniel Kornev, CPO DeepPavlov.ai
  2. 2. DeepPavlov.ai Why Multiskill is important? ▪ Customer experience spans multiple domains • Surveys • Promotions • Campaigns • Customer Service • Technical Support • … ▪ Every domain requires specific skill © Copyright PresentationGO.com Pre-Purchase Post-Purchase Surveys Promotions Campaigns Customer Service Technical Support Product Usage Billing & Payment Account Management Logistics
  3. 3. DeepPavlov.ai AliMe Assist e-commerce assistant ▪ 3 Services with different ML engines • Assistance Service: Slot Filling Engine • Customer Service: Knowledge Graph Engine • Chatting Service: Chat Engine Li, F. L., Qiu, M., Chen, H., Wang, X., Gao, X., Huang, J., ... & Jin, G. (2017, November). Alime assist: An intelligent assistant for creating an innovative e-commerce experience. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (pp. 2495-2498). ACM.
  4. 4. DeepPavlov.ai XiaoIce socialbot ▪ 660 million users worldwide ▪ 5 countries • China • Japan • USA • India • Indonesia ▪ 40 platforms ▪ Average CPS of 23 (better than human to human) Zhou, L., Gao, J., Li, D., & Shum, H. Y. (2020). The design and implementation of xiaoice, an empathetic social chatbot. Computational Linguistics, 46(1), 53-93.
  5. 5. DeepPavlov.ai Conversational AI Stack Multiskill orchestration Conversational skills NLP frameworks ML platforms ▪ Multiskill orchestration • An AI assistant should use many skills to solve different tasks • Long term interaction with a user ▪ Conversational skills • Conversational skill helps user with a specific task • Interaction with a user is limited by one dialog ▪ NLP frameworks • Helps system to understand and generate natural language • Only a current user’s utterance or short history of the dialog is processed ▪ ML platforms • ML models training Proprietary Open Source
  6. 6. DeepPavlov.ai action = request_location system action What we have right now ▪ Modular dialog systems • NLU (Natural Language Undestanding) • Detects Domain, Intent, Slots (entities) • DM (Dialogue Manager) • Updates dialog state • Decides the next action (policy) • NLG (Natural Language Generation) • Generates response in natural language NLU (Natural Language Understanding) • Domain detection • Intent detection • Entities detection DM (Dialogue Manager) • Dialogue state • Script, ML Policy intent = request_movie entities = { genre = ‘comedy’, date = ‘weekend ’ } semantic frame NLG (Natural Language Generation) • Generative models • Templates Are there any comedy movies to see this weekend? Where are you?
  7. 7. DeepPavlov.ai AI Assistant Lifecycle ▪ Business as usual • Product growth requires more and more domains • More domains – adding more features and scriptspolicies • Complexity ceiling ▪ DeepPavlov Vision • Every domain is covered by separate conversational skill (modularity) • Every skill is deployed as a microservice (scalability) • Old skills can be reusedextended in new product (interoperability) • Separate skills might be developed and maintained by separate teams AGENT MVP PRODUCT GROWTH MATURE AI ASSISTANT domain skill DeepPavlov Vision As Usual DM NLU feature script feature script feature DM NLU feature script feature script feature DM NLU feature script feature script feature DM NLU domain skill domain skill default skill default skill default skill AGENT domain skill domain skill domain skill default skill default skill default skill domain skill domain skill feature feature script feature script feature script feature feature feature script feature script
  8. 8. DeepPavlov.ai Agent Models / Components DeepPavlov solution Skill Skill Skill Chit-Chat Factoid Task-Oriented √ √ Named Entity Recognition √ √ Coreference resolution √ √ Intent recognition √ √ Insults detection √ Q&A √ √ Dialogue Policy √ √ Dialog history √ √ √ Lanaguage model … reddit SQuAD DSTC-2 Dataset ▪ NLP pipelines • DeepPavlov Library provides pretrained models and simple declarative approach to build NLP processing pipelines ▪ Conversational Skills • DeepPavlov Dream is a collection of pre-build conversational skills and a default distribution package for Dream AI Assistant ▪ Multiskill Orchestration • DeepPavlov Agent is an engine for conversational skill deployment and orchestration
  9. 9. DeepPavlov.ai DREAM socialbot architecture 1. Multiple Annotators are used to extract information from the user input. 2. Skill Selector defines a subset of active Skills based on the extracted information. 3. Selected Skills propose their response candidates. 4. Finally, Response Selector picks a response to be sent to Response Annotators and, eventually, to the user. All elements of the pipeline are running asynchronously with two points of synchronization: Skill Selector and Response Selector. Dialogue State serves as a shared memory.
  10. 10. DeepPavlov.ai DREAM socialbot infrastructure 1. The core of the DREAM socialbot is implemented with DeepPavlov Agent (DP-Agent) framework. It orchestrates services for Skills, Annotators, Skill Selector and Response Selector, and is located on AWS EC2 instances with Docker Swarm. 2. Dialogue State history is stored on a separate instance with MongoDB. 3. AWS Lambda performs HTTP requests to the DREAM- agent by sending ASR tokens. 4. Testing infrastructure consists of Telegram bots for interacting with the dev version of the socialbot or with selected conversational skill only. 5. Dialogue analytics tool and dashboard were located on a separate EC2 instance. 6. Cluster and application monitoring had configured alerts to email and Slack.
  11. 11. DeepPavlov.ai DeepPavlov Ecosystem ▪ Open repository of NLP models and pipelines • easy to find and reuse NLP components for development of new skills or extension of existing ▪ Open repository of conversational skills • alternative implementations of the most popular skills ▪ Open hub for AI Assistant distributions • general and domainindustry specific distributions of skill sets
  12. 12. DeepPavlov.ai DeepPavlov on NGC ngc.nvidia.com/catalog/containers/partners:deeppavlov
  13. 13. DeepPavlov.ai Conversational AI DeepPavlov Stack Multiskill orchestration Conversa- tionalskills NLP frameworks ML platforms Proprietary Open Source ▪ Multiskill orchestration • DeepPavlov Agent is an engine for conversational skill deployment and orchestration ▪ Conversational skills • DeepPavlov Dream is a collection of pre- build conversational skills and a default distribution package for Dream AI Assistant ▪ NLP frameworks • DeepPavlov Library provides pretrained models and simple declarative approach to build NLP processing pipelines ▪ ML platforms • TensorFlow and PyTorch as backends
  14. 14. DeepPavlov.ai HOW TO BUILD MULTISKILL AI ASSISTANT WITH DEEPPAVLOV LIVING IN PANDEMICS MAKES US LONELY Wouldn’t it be nice to have a friend to care about us?
  15. 15. DeepPavlov.ai MEET GERTY 3000 Can help Sam with problems on the Moon Base Sarang? Can entertain Sam? Yes ✔ Yes ✔ Can we emulate it with DeepPavlov DREAM? Yes ✔ Main Question Functionality Analysis Case: Gerty In the Moon Movie
  16. 16. DeepPavlov.ai Text OR Voice Input TTS (NeMo) Deepy: Example Architecture Spell Checking NeMo ASR Harvesters Status Chit-Chat (AIML) Emotion BUILT-IN SKILL SELECTOR RULE-BASED RESPONSE SELECTOR
  17. 17. DeepPavlov.ai services: agent: [..] depends_on: - mongo harvesters_maintenance_skill: [..] mongo: [..] rule_based_response_selector: [..] nemo: [..] depends_on: - agent emotion_classification: [..] program_y: [..] spell_checking: [..] Deepy: Docker Pipeline Spell Checking Annotators Emotion Classification Harvesters Status Skill Skills Chit-Chat Skill NeMo ASR & TTS Other Services Rule-Based Response Selector
  18. 18. DeepPavlov.ai services: agent: [..] depends_on: - mongo harvesters_maintenance_skill: [..] mongo: [..] rule_based_response_selector: [..] nemo: [..] depends_on: - agent emotion_classification: [..] program_y: [..] clone_tts: [..] Deepy: Part of Agent Pipeline Annotators Services Are in Groups Skills Response Annotators Depend on groups (e.g., “skills”) Services Are Isolated Limited in what they see in dialog Invoke Agent’s State Manager Can run via HTTP or be Python-based "skills": { "harvesters_maintenance_skill": { "connector": { "protocol": "http", "url": "http://harvester s_maintenance_skill:3002/respond" }, "dialog_formatter": "dp_form atters:full_dialog", "response_formatter": "dp_fo rmatters:base_skill_formatter", "state_manager_method": "add _hypothesis", "previous_services": ["annot ators"] }, Response Selectors
  19. 19. DeepPavlov.ai Deepy: Harvesters M. Skill def detect_intent(utterance): [..] return intent def generate_response_from_db(intent, utterance): [..] return response, confidence @app.route("/respond", methods=["POST"]) def respond(): [..] dialogs = request.json["dialogs"] for dialog in dialogs: sentence = dialog['human_utterances'][-1] ['annotations'].get("spelling_preprocessing") [..] intent = detect_intent(sentence) [..] response, confidence = generate_response_from_db (intent, sentence) [..] return jsonify(list(zip(responses, confidences))) What is (all) harvesters’ status? Intents What is harvester status? Prepare rover for a trip Extracts post-processed sentences Dialog Processing Looks up for intents If found, runs, & generates response Returns response + confidence
  20. 20. DeepPavlov.ai Deepy: Chit-Chat Skill (AIML) <?xml version="1.0" encoding="UTF-8"?> <aiml version="2.0"> <category> <pattern>I AM ^ TIRED</pattern> <template> 🙁 <random> <li>Get some sleep<get name="name"/>. You're very tired.</li> <li>Have a rest and be happy! How can I help you?</li> </random> </template> </category> [..] </aiml> </xml> Assistant Profile (Name, Place, etc.) Patterns Greeting scenario Topics Looks up for patterns Dialog Processing Picks random pre-defined response If not sure, confidence is low (0.2) Returns response + confidence

×