Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Chatbots for Brand Representation in Comparison with Traditional Websites

382 visualizaciones

Publicado el

Master thesis presentation, Graz University of Technology, May 2020

Publicado en: Educación
  • Inicia sesión para ver los comentarios

  • Sé el primero en recomendar esto

Chatbots for Brand Representation in Comparison with Traditional Websites

  1. 1. Chatbots for Brand Representation in Comparison with Traditional Websites Johannes Kühnel, BSc May 28, 2020 Institute of Interactive Systems and Data Science Graz University of Technology, Austria
  2. 2. Table of contents 1. Introduction 2. Proof of Concept 3. User Study 4. Conclusion 1
  3. 3. Intro
  4. 4. What are Chatbots? Chatbots are computer programs that interact with users us- ing natural languages. (Shawar and Atwell 2007) Usually chatbots ... • provide a service (e.g. bookings) • use conversational interfaces • feature simple or more complex AI 2
  5. 5. “Bots are the new apps” Satya Nadella, Microso t CEO 2
  6. 6. Motivation Messaging Platform Monthly Active Users WhatsApp 1,500 Facebook Messenger 1,300 Weixin / Wechat 1,083 Table 1: Monthly active users of the top 3 messaging platforms in millions. Sources: Hootsuite and We Are Social (2019) and Nadella (2016) and https://telegram.org/blog/200-million (visited on 2019-02-13). 3
  7. 7. Related Work Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018) • comparison of websites and chatbots of 3 brands • participants perform 1 task per brand and platform • Keypoints: websites offered better user experience, positive emotions with chatbot 4
  8. 8. Related Work Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018) • comparison of websites and chatbots of 3 brands • participants perform 1 task per brand and platform • Keypoints: websites offered better user experience, positive emotions with chatbot Case study by Shawar and Atwell (2015) • Frequently Asked Questions (FAQ) • chatbot vs search engine • Keypoints: more relevant answers, higher preference 4
  9. 9. Proof of Concept
  10. 10. Theodore, a Company Chatbot A chatbot — named Theodore — to represent Graz based so tware developer CodeFlügel was created. 5
  11. 11. Theodore, a Company Chatbot A chatbot — named Theodore — to represent Graz based so tware developer CodeFlügel was created. Goal: chatbot capable of representing and informing about the company (like the existing website) 5
  12. 12. Theodore, a Company Chatbot A chatbot — named Theodore — to represent Graz based so tware developer CodeFlügel was created. Goal: chatbot capable of representing and informing about the company (like the existing website) Theodore should • reproduce most of the website’s features • run on Facebook Messenger and a custom webchat • re-use existing Application Programming Interfaces (APIs) 5
  13. 13. Dialog Design Chatbot has to provide information about • the company itself • products & services • contact information • vacancies • newsletter subscription • social media & blog posts 6
  14. 14. Dialog Design Chatbot has to provide information about • the company itself • products & services • contact information • vacancies • newsletter subscription • social media & blog posts Additional requirements include • help function & fallback mechanism • informal language • aware of being a bot 6
  15. 15. Technologies Backend Languages & Frameworks: Node.js with Express.js and Socket.IO APIs: Facebook, Dialogflow, Mailchimp, Wordpress 1 Webchat Languages & Frameworks: Angular (using TypeScript) with Socket.IO APIs: Backend 2 1connection via REST API / HTTP requests 2connection via WebSockets 7
  16. 16. Message Format Facebook Messenger’s message format (JSON) used for both Messenger and webchat. Support for various components: • Generic Template • List • Button • Media • Quick Replies 8
  17. 17. Natural Language Understanding Natural Language Understanding (NLU): process of “understanding” natural language in computer science Intents: the meaning or purpose of the user input Parameters (entities): terms tied to the intents (e.g. context, amount, time etc.) 9
  18. 18. Natural Language Understanding Natural Language Understanding (NLU): process of “understanding” natural language in computer science Intents: the meaning or purpose of the user input Parameters (entities): terms tied to the intents (e.g. context, amount, time etc.) Figure 1: Dialogflow Basics — Intents Source: https://cloud.google.com/dialogflow/docs/basics 9
  19. 19. Natural Language Understanding cont. • initial design was framework agnostic • looked at several services/frameworks for NLU • Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson Assistant (IBM), Amazon Lex 10
  20. 20. Natural Language Understanding cont. • initial design was framework agnostic • looked at several services/frameworks for NLU • Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson Assistant (IBM), Amazon Lex • final implementation only for Dialogflow • scope of thesis / PoC • differences in framework/service concepts • pricing 10
  21. 21. Bot Frameworks Evaluated bot development frameworks: • Microso t Bot Framework • Botkit • Botmaster.ai • Botpress • BotMan 11
  22. 22. Bot Frameworks Evaluated bot development frameworks: • Microso t Bot Framework • Botkit • Botmaster.ai • Botpress • BotMan Disadvantages include: • reliance on third party services, higher latencies • only one platform per instance • slow development • limited content handling • poor platform support 11
  23. 23. Bot Frameworks Evaluated bot development frameworks: • Microso t Bot Framework • Botkit • Botmaster.ai • Botpress • BotMan Disadvantages include: • reliance on third party services, higher latencies • only one platform per instance • slow development • limited content handling • poor platform support Due to these shortcomings, a custom implementation was chosen. 11
  24. 24. Architecture Figure 2: Request – Response Architecture 12
  25. 25. UI Components Figure 3: Blog post components in the webchat and Facebook Messenger. 13
  26. 26. UI Components cont. Figure 4: Quick Replies in the webchat and Facebook Messenger. 14
  27. 27. UI Components cont. Figure 5: Generic Template components showing products in the webchat and Facebook Messenger. 15
  28. 28. User Study
  29. 29. User Study • comparison of chatbot and website • identify advantages and weaknesses • selection based on relevance to the company • 3 groups • (potential) clients • potential employees • blog readers and everyone else • 20 participants (at least 5 per group) 16
  30. 30. User Study cont. • pre- & post-test questionnaires • users perform 9 typical tasks • e.g. “Find out where CodeFlügel’s office is located and how to call them.” • each with the chatbot and website • interview a terwards Figure 6: Test Setup 17
  31. 31. Results cont. Figure 7: Average time per task 18
  32. 32. Results Figure 8: Average total time and average time for tasks 19
  33. 33. Results cont. Figure 9: What users say about the company’s chatbot and website 20
  34. 34. Results cont. Figure 10: Which platform was more appealing to the users 21
  35. 35. Results cont. Figure 11: 95% would use more chatbots 22
  36. 36. Conclusion
  37. 37. Conclusion • positive feedback about the chatbot • train for whole sentences and keywords • chatbot more entertaining • high acceptance rate • chatbot was faster (specific information) 23
  38. 38. Conclusion • positive feedback about the chatbot • train for whole sentences and keywords • chatbot more entertaining • high acceptance rate • chatbot was faster (specific information) • menus offer quick navigation • minor issues with intent matching • exploration better with website 23
  39. 39. Future Work • improve intent matching • broader audience / more participants • different chatbots and personas • brand perception • conversion rates and other Key Performance Indicators (KPIs) 24
  40. 40. Live Demo URL: https://theodore.kuehnel.co.at 24
  41. 41. Questions? 24
  42. 42. Website Figure 12: Screenshot of the company homepage, taken on 2018-11-04.
  43. 43. Webchat cont. Figure 13: List, contact and typing indicator components in the webchat.
  44. 44. Facebook Messenger cont. Figure 14: List, contact and typing indicator components in Facebook Messenger.
  45. 45. Message Format - Details { "message" : { "attachment" : { "type" : "template" , "payload" : { "template_type" : "generic" , "elements" : [ { "title" : "<TITLE_TEXT>" , "image_url" : "<IMAGE_URL_TO_DISPLAY>" , "subtitle" : "<SUBTITLE_TEXT>" , "default_action" : { "type" : "web_url" , "url" : "<DEFAULT_URL_TO_OPEN>" , "messenger_extensions" : <TRUE | FALSE > , "webview_height_ratio" : "<COMPACT | TALL | FULL>" } , "buttons" : [ < BUTTON_OBJECT > , . . . ] } , . . . ] } } } } Listing 1: A generic template JSON message in Facebook Messenger’s format.
  46. 46. Task List 1. Find out what CodeFlügel does or which services they provide. 2. Find out where CodeFlügel’s office is located and their phone number. 3. Find out which companies CodeFlügel has already implemented projects for. 4. Find and open the latest blog entry. 5. Sign up for the newsletter with the e-mail address <firstname>.<surname>@codefluegel.com. 6. Find out if and which jobs are currently available. 7. Find a way to try Augmented Reality (AR) for yourself. 8. Find at least one Augmented Reality (AR) project created by CodeFlügel. 9. Find out what Augmented Reality (AR) actually is.
  47. 47. Participant Details • age from 20–33, averaging 27.7 • 60% with bachelor degree • 45% studying (bachelor or master’s program) • 70% with technical background (mostly IT) • mostly male
  48. 48. Results cont. Figure 15: What users expect of a company website and chatbot
  49. 49. Results cont. Figure 16: The chart shows how the users communicated with the chatbot
  50. 50. Results cont. Figure 17: Preferred type of speech of the chatbot
  51. 51. Results cont. Figure 18: Total time needed to complete the tasks (per User)
  52. 52. References i Beriault-Poirier, Amélie, Sandrine Prom Tep, and Sylvain Sénécal (Oct. 2018). “Putting Chatbots to the Test: Does the User Experience Score Higher with Chatbots Than Websites?” In: Human Systems Engineering and Design. Springer International Publishing, pp. 204–212. isbn: 978-3-030-02053-8. doi: 10.1007/978-3-030-02053-8_32. Hootsuite and We Are Social (2019). Digital 2019. Global Digital Overview. Tech. rep. Mindbowser. url: https://datareportal.com/reports/digital-2019- global-digital-overview (visited on 02/04/2019). Nadella, Satya (Mar. 25, 2016). Build 2016. Keynote Presentation. Microso t. url: https://channel9.msdn.com/Events/ Build/2016/KEY01#time=1h41m11s (visited on 02/13/2019).
  53. 53. References ii Shawar, Bayan Abu and Eric Atwell (2007). “Chatbots: Are They Really Useful?” In: LDV-Forum 22.1, pp. 29–49. url: https://jlcl.org/content/2-allissues/20-Heft1- 2007/Bayan_Abu-Shawar_and_Eric_Atwell.pdf (visited on 01/24/2019). – (Sept. 2015). “A chatbot as a Question Answering Tool”. In: 2015 International Conference on Advances in So tware, Control and Mechanical Engineering. 2015 International Conference on Advances in So tware, Control and Mechanical Engineering, pp. 1–6. isbn: 978-93-84422-37-0. doi: 10.17758/UR.U0915120.

×