Data scientists should not be content creators. They lack domain knowledge and are no conversational designers. Empowering content creators has been instrumental to scale the N26 chatbot from 1 to 5 languages in less than 6 months. In this talk, Edouard will describe how N26 enables content creators to add, modify and test content autonomously when entering new markets/handling new languages. The session also presents N26's training and deployment pipelines as well as the approach to content management the team took to ensure consistency across markets and languages while allowing for flexibility when necessary. WHAT YOU'LL LEARN Structuring content for scalability and robustness Empowering Content Creators to create custom actions without using Python Training and deploying conversational models across multiple markets and languages. Edouard Malet has been a Data Scientist at N26 for over two years. After working on a variety of projects, he has been the lead developer of N26's customer service Chatbot for the past year. This chatbot is now serving millions of customers in 5 languages across 22 countries.

1. Building Scalable Chatbots by Empowering
Content Creators
Edouard Malet
Data Scientist, N26
Rasa Developer Summit - 2019

2. Edouard Malet 24/09/19
by Empowering Content Creators
Building Scalable Chatbots
1
Rasa Summit

3. Introduction
01
2

4. Markets
N26 Customer
Service at a
Glance
3.5M+
Customers
26
Countries
5
Supported languages
3
3
Different markets (EU/UK/US)

5. How do we respond to a growing customer base?
Why a
chatbot?
4

6. Oct 2014 Dec 2015 Mar 2016 Jan 2017 Sep 2017 Mar 2018 June 2018 Oct 2018 Jan 2019Jan 2019
5
75 000
125 000
250 000
500.000
850.000
1 Million
1,5 Million
2 Million
2.5 Million
3 Million
3,5 Million
Mar 2019 Jun 2019
3.5 Million
Users
June 2019
5

7. How do we respond to a growing customer base?
Scale Customer Service in line with growth
Why a
chatbot?
6
How do we let people ask us questions?

8. 7

9. How do we respond to a growing customer base?
Scale Customer Service in line with growth
Why a
chatbot?
8
How do we let people ask us questions?
Natural Language Querying
How do we support people around the clock?

10. How do we respond to a growing customer base?
Scale Customer Service in line with growth
Why a
chatbot?
9
How do we let people ask us questions?
Natural Language Querying
How do we support people around the clock?
24/7 support

11. How do we respond to a growing customer base?
Scale Customer Service in line with growth
Why a
chatbot? How do we support people around the clock?
24/7 support
How do we let people ask us questions?
Natural Language Querying
10
How do we solve problems in one contact?

12. 11

13. How do we respond to a growing customer base?
Scale Customer Service in line with growth
Why a
chatbot? How do we support people around the clock?
24/7 support
How do we let people ask us questions?
Natural Language Querying
12
How do we solve problems in one contact?
Rule-based routing

14. 02Tech stack
13

15. Technologies
14

16. Training data
format
15
NLU
## intent:want_statement
- hi there, I would like to receive my [balance statement](statement_type)
- can you say where I can find the annual statement for 2018

17. 16
## intent:want_statement
- hi there, I would like to receive my [balance statement](statement_type)
- can you say where I can find the annual statement for 2018

18. Core
Training data
format part I:
Stories
17
## want_statement basic
* greet
- action_welcome
* want_statement{"statement_type": "balance statement"}
- utter_ack
- utter_want_statetement_balance_statement_1
- utter_want_statetement_balance_statement_2
- utter_want_statetement_balance_statement_3
- utter_want_statetement_balance_statement_4

19. 18
## want_statement basic
* greet
- action_welcome
* want_statement{"statement_type": "balance statement"}
- utter_ack
- utter_want_statetement_balance_statement_1
- utter_want_statetement_balance_statement_2
- utter_want_statetement_balance_statement_3
- utter_want_statetement_balance_statement_4

20. slots:
statement_type:
type: unfeaturized
entities:
- statement_type
intents:
- greet
- want_statement
actions:
- action_welcome
- utter_want_statement_balance_statement_1
- utter_want_statement_balance_statement_2
- utter_want_statement_balance_statement_3
- utter_want_statement_balance_statement_4
templates:
utter_want_statement_balance_statement_1:
- text: "You can find your {{balance statements}} in both the web app
and the mobile app."
link_id: STATEMENTS_001
Core
Training data
format part II:
Domain
19

21. 20
slots:
statement_type:
type: unfeaturized
entities:
- statement_type
intents:
- greet
- want_statement
actions:
- action_welcome
- utter_want_statement_balance_statement_1
- utter_want_statement_balance_statement_2
- utter_want_statement_balance_statement_3
- utter_want_statement_balance_statement_4
templates:
utter_want_statement_balance_statement_1:
- text: "You can find your {{balance statements}} in both the web app
and the mobile app."
link_id: STATEMENTS_001

22. Challenges Writing custom actions
How do we allow content creators to create actions
without coding?
Serving multiple markets
How do we ensure consistency across markets?
Serving multilingual content
How do we serve the content in 5 languages?
21
Deploy to different markets
How to make it easy to launch to new markets?

23. Implementation
03
22

24. %
Data Scientists
Content creators
50 %
50
23

25. DATA SCIENCE
Responsibilities
24
Store training data
Train models
Unit/Integration/Model Testing
API Development

26. 25
CONTENT
Responsibilities
Transfer domain knowledge into
Intents, entities and stories
Keep up with Product releases
Add new languages/markets

27. Challenges
Serving multilingual content
How do we serve the content in 5 languages?
26

28. %
Data Scientists
Content creators
50 %
50
27

29. 28
## intent:want_statement
- hi there, I would like to receive my [balance statement](statement_type)
- can you say where I can find the annual statement for 2018

30. NLU
Making entities
language-agnostic
29

31. 30
- id: Q1413339
slot_name: statement_type
description: No description defined
en:
label: balance statement
synonyms:
- balance statements
- transfer statement
fr:
label: relevé de compte
synonyms:
- relevés
- releves
- relevés de compte
- extrait de compte

32. 31
## synonym:Q1413339
- balance statement
- balance statements
- transfer statement

33. 32
{
"text": "Hi, how can I download my balance statement?",
"intent": {
"name": "want_statement",
"confidence": 0.9768562913
},
"entities": [
{
"start": 26,
"end": 43,
"value": "Q1413339",
"entity": "statement_type",
"confidence": 0.9523699849,
"extractor": "CRFEntityExtractor",
"processors": [
"EntitySynonymMapper"
]
}
],
}

34. 33
## want_statement basic
* greet
- action_welcome
* want_statement{"statement_type": "Q1413339"}
- utter_ack
- utter_want_statetement_balance_statement_1
- utter_want_statetement_balance_statement_2
- utter_want_statetement_balance_statement_3
- utter_want_statetement_balance_statement_4

35. Challenges
Serving multiple markets
How do we ensure consistency across markets?
Serving multilingual content
How do we serve the content in 5 languages?
34

36. Consistency
through
sharing
35

37. 36
include:
- want_statement_uk
exclude:
- want_statement

38. Challenges Writing custom actions
How do we allow content creators to create actions
without coding?
Serving multiple markets
How do we ensure consistency across markets?
Serving multilingual content
How do we serve the content in 5 languages?
37

39. PROBLEM
Actions are
Python code
38
"""Helper module for the action.py module."""
from rasa_core_sdk import Action
import requests
from utils.sender_id_converter import sender_id_to_dict
from utils.setup import setup_config
def live_agent_availability(sender_id):
"""Call the endpoint to know about agent availability."""
try:
user_id = sender_id_to_dict(sender_id)["user_id"]
headers = {"x-n26-userid": user_id}
r = requests.get(
setup_config.agent_availability_endpoint, headers=headers, timeout=1
)
if r.status_code == 200:
return r.json()["liveagent"]["status"] == "AVAILABLE"
else:
return False
except requests.exceptions.RequestException:
return False
class ActionWelcome(Action):
"""Welcomes a user depending on agent's availability."""
def name(self):
"""Name."""
return "action_welcome"
def run(self, dispatcher, tracker, domain):
"""Run."""
live_agent_available = live_agent_availability(tracker.sender_id)
if live_agent_available:
dispatcher.utter_template("utter_welcome_1", tracker)
dispatcher.utter_template("utter_welcome_2", tracker)
else:
dispatcher.utter_template("utter_welcome_no_humans_1", tracker)
dispatcher.utter_template("utter_welcome_no_humans_2", tracker)
return []

40. 39
"""Helper module for the action.py module."""
from rasa_core_sdk import Action
import requests
from utils.sender_id_converter import sender_id_to_dict
from utils.setup import setup_config
def live_agent_availability(sender_id):
"""Call the endpoint to know about agent availability."""
try:
user_id = sender_id_to_dict(sender_id)["user_id"]
headers = {"x-n26-userid": user_id}
r = requests.get(
setup_config.agent_availability_endpoint, headers=headers, timeout=1
)
if r.status_code == 200:
return r.json()["liveagent"]["status"] == "AVAILABLE"
else:
return False
except requests.exceptions.RequestException:
return False
class ActionWelcome(Action):
"""Welcomes a user depending on agent's availability."""
def name(self):
"""Name."""
return "action_welcome"
def run(self, dispatcher, tracker, domain):
"""Run."""
live_agent_available = live_agent_availability(tracker.sender_id)
if live_agent_available:
dispatcher.utter_template("utter_welcome_1", tracker)
dispatcher.utter_template("utter_welcome_2", tracker)
else:
dispatcher.utter_template("utter_welcome_no_humans_1", tracker)
dispatcher.utter_template("utter_welcome_no_humans_2", tracker)
return []

41. 40
action_welcome:
- condition:
live_agent_available: true
dispatch:
- utter_welcome_1
- utter_welcome_2
- condition:
live_agent_available: false
dispatch:
- utter_welcome_no_humans_1
- utter_welcome_no_humans_2

42. SOLUTION
Dynamic action
creation
41
action_test:
- condition:
slot_1: True
~slot_2: NULL
slot_3: [value_1, value_2]
live_agent_available: True
intent_detected: True
slot_setting:
slot_4: False
dispatch:
- utter_example

43. Challenges Writing custom actions
How do we allow content creators to create actions
without coding?
Serving multiple markets
How do we ensure consistency across markets?
Serving multilingual content
How do we serve the content in 5 languages?
42
Deploy to different markets
How to make it easy to launch to new markets?

44. 43

45. Question time
44