Smart robot using web resource is proposed as a casual interface in smart house and super smart society, where a robot interactively assists users utilizing existing web service in natural way (e.g., speech, gesture, etc.). In addition, a robot for casual service need to behave like a human beings in daily situation. Humor expression for a service robot is imprinted, where a robot interacts socially with users. We apply the proposed smart robot to the library guidance system. The proposed smart robot is able to realize a casual interface for IT services and IoT services including HEMS applications in smart house. IWIN2017 http://www.infsoc.org/conference/iwin2017/

1. Smart Robot based on Web Resource
with Friendly Communication
and Its Application to Library Guide
Dept. of Home Electronics yamalab.com
◯Yoichi Yamazaki, Takuya Shinkai , and Masao Isshiki
（◯山崎 洋一, 新海 拓也, 一色 正男）
Dept. of Home Electronics
Faculty of Creative Engineering
KAnagawa Institute of Technology
yamazaki@he.kanagawa-it.ac.jp
URL: http://yamalab.com 1/20

2. Dept. of Home Electronics yamalab.com
Super smart society (Society 5.0) is a society that is able to respond precisely to a wide variety
of social needs, and a society in which all kinds of people can readily obtain high-quality
services [The 5th Science and Technology Basic Plan, Government of Japan (2016) ].
Introduction
Super Smart Society : Society 5.0 (Japan)
IoT connects physical space & cyber space, and creates “Smile”!
1st
Hunter-Gatherer
2nd
Agricultural
3rd
Industrial
4th
Information
Society 5.0
Super Smart Society
2/20

3. Smart Living Room and Smart Robot
based on Atmosphere Control for Smile Creation in Daily Life
Elderly Support Robot for ME-BYO Care
Super Human MMA Game: Shin-Ken
神拳 – Shin-Ken -
Casual Service & Atmosphere Control
by Smart Robot
‘Smile’ Living Room sensing Atmosphere
Childcare Living Room
Companion Robot for Mental Care
Gymnastics Support at Home
Atmosphere
Quantification
Health Care Support
Dept. of Home Electronics yamalab.com
(Innovation Japan 2017@ Tokyo Big Sight, 31 Aug.-1 Sep. , 2017)
3/20

4. Dept. of Home Electronics yamalab.com
 Smart robots assist human users on behalf of complex IoT
systems surrounding.
 Definition of Smart Robot:
Concept of Smart Robot
1. Accessible Web resource 2. Casual Communication
Speech recognition
Emotional communication
M. Shirow. Ghost in the Shell. : Dark Horse Comics. 4/20

5. Smart Robot Service in Public Space
User Database/ Internet
Information
terminal
IT Society: User-driven service
User Database/ InternetSmart Robot
Super Smart Society : Things-driven service
Smart robots use Web resource and assist human.
Humans use Web resource through information terminals.
Dept. of Home Electronics yamalab.com
5/20

6. Dept. of Home Electronics yamalab.com
 Smart robot using web resource is proposed as a casual
interface creating super smart society, where a robot
interactively assists users utilizing existing web service in
natural way (i.e., speech and emotional communication).
 In addition, humor expression for a service robot is
implemented for casual communication, where a robot
interacts socially with users.
 We apply the proposed smart robot to the library guidance
system.
Proposal
6/20

7. Companion Robot Nao
Sight（ HD Camera [1220px, 30fps]×2 ）
face detection/tracking
face recognition
object recognition
Hearing （ directional mic×４ ）
Speech Recognition
Sound direction detection
Communication （speaker×2, LED）
19 language
Emotional expression by motion & LEDNAO
（Aldebaran Robotics）
58cm
5.4kg
Dept. of Home Electronics yamalab.com

8. Smart Robot Librarian at Library in KAIT
Search
KAIT Library Web System
Ranking
New books
InformationLibrarian robot
(NAO, Pepper)
TV /Display
Smart Robot Librarian
Request Reply
Smart house network
Or Local network
Internet
• Robot sends requests to the servers according to conversation with users.
• Library Web system replies the searching result.
• Robot replies the result, and TV/display show the result on the web page.
Dept. of Home Electronics yamalab.com
User
8/20

9. Dept. of Home Electronics yamalab.com
 Service Robot need to adopt a good attitude, i.e. friendly
behavior toward the users.
 A humor expression is proposed for communication robots
with speech recognition.
 Humor is fall into following three fundamental classes i.e.
 1) ludic humor that entertains others,
 2) offensive humor that attacks others to obtain a sense of
superiority,
 3) supportive humor that encourages and forgives others
[6].
 Ludic humor and supportive humor are used to decrease
mental stress by error in human-robot communication.
Humor Expression
for Casual Service Communication of Robot
9/20

10. Dept. of Home Electronics yamalab.com
10/20

11. Dept. of Home Electronics yamalab.com
Example of humor expression:
Robot:
Hello. Welcome to the Kanagawa Institute of Technology library.
I am NAO.
Call me plain Nao-chi*!
I want to have an orange in the kotatsu on a cold day. Oh! Where is it
oranged**?
I will show you ‘How to use the service of book,’ ‘Monthly rental
ranking,’ and ‘Latest information’. What do you want to know? …
* ‘-chi’ is one of popular ending of a word as nickname in Japanese.
** A pun using the words ‘orange’ and ‘arrange’.
Humor Expression
for Casual Service Communication of Robot
11/20

12. Dept. of Home Electronics yamalab.com
 The effects of humor expression are evaluated by 30 users
explicitly and implicitly.
 Situation
 The robot guidance is given to users according to request.
 Conditions
 Control: Robot interacts with users without humor expressions.
 Proposed : Robot interacts with humor expressions.
 Evaluations Items:
 2 Subjective Evaluations (explicitly):
 Subjective Assessments Questionnaire
 Stress Level Assessments with Face Scale
 1 Objective Evaluation (implicitly):
 Smile Expression Time
4. Evaluation:
System Evaluation of Smart Robot for Library
12/20

13. Dept. of Home Electronics yamalab.com
 3 points are evaluated by 6 items on a scale of one
to five.
 Fullness in conversation
1. Length of sensation time spent in conversation (‘short’ to ‘long’)
2. Pleasant in conversation (‘unpleasant’ to ‘pleasant’)
 Familiarity of the robot
3. Companionable of the robot (‘uncompanionable’ to ‘companionable’)
4. Humor in conversation (‘unfunny’ to ‘funny’)
 Stress in conversation
5. Response in conversation (‘bad’ to ‘good’)
6. Stress in conversation (‘stressful’ to ‘stress-free’)
4.1.1. Subjective Evaluation 1)
Subjective Assessments Questionnaire for Robot Impression
1 2 4 53 13/20

14. Dept. of Home Electronics yamalab.com
 Stress level is evaluated with a Face scale in the
conversation with the smart robot.
4.1.2. Subjective Evaluation 2)
Stress Level Assessments with Face Scale Measure
very stressful
(score: 10)
very comfortable
(score: 1)
✓
14/20

15. Dept. of Home Electronics yamalab.com
 As objective assessment data of familiarity,
we measure an expression time when users smile
on the face based on the recoded movie.
 𝑆𝑚𝑖𝑙𝑒 𝐸𝑥𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑇𝑖𝑚𝑒 𝑝𝑒𝑟 𝑚𝑖𝑢𝑡𝑒𝑠
=
𝑆𝑚𝑖𝑙𝑒 𝐸𝑥𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑇𝑖𝑚𝑒 𝑖𝑛 𝑖𝑛𝑡𝑒𝑟𝑎𝑐𝑡𝑖𝑜𝑛
𝑇𝑜𝑡𝑎𝑙 𝑇𝑖𝑚𝑒 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑎𝑐𝑡𝑖𝑜𝑛
× 60 (1).
4.1.3. Objective Evaluation 1)
Smile Expression Time
15/20

16. Dept. of Home Electronics yamalab.com
4.3. Results: Subjective Evaluation 1)
Subjective Assessments Questionnaire
2.8
3.3
3.8
3.1
3.5
3.8
2.3
3.6
3.9
3.5
3 3.3
1
2
3
4
5
without Humore (control)
with Humore (proposed)
Score
Positive: Fullness, Familiarity Negative: Stress
16/20

17. Dept. of Home Electronics yamalab.com
4.3. Results: Subjective Evaluation 2)
Stress Level Assessments with Face Scale Measure
7.0 7.1
0.0
2.0
4.0
6.0
8.0
10.0
Stress level
without Humore (control) with Humore (proposed)
Score
17/20

18. Dept. of Home Electronics yamalab.com
 Humor expression has an effect to increase
smile expression time of users.
5.1
18.6
0.0
5.0
10.0
15.0
20.0
Smile Expression Time per minutes
Timeperminute[s]
without Humore (control) with Humore (Proposed)
264.7% increase
4.3. Results: Objective Evaluation 1)
Smile Expression Time per minutes
18/20

19. Dept. of Home Electronics yamalab.com
 Proposed Smart Robot with Humor Expression:
 In the subjective evaluations(explicitly):
 Subjective Questionnaire -> Fullness, Familiarity, Stressful
 Stress Level Assessments with Face Scale
 In the objective Evaluation(implicitly):
 Smile Expression Time -> inclease: Smile Expression Time (264.7%)
 These all results shows:
 the users express simile unconsciously, though they feel a
little stress consciously by the humor expression.
 Human reduce their stress by just smiling on the face in
general [T.L. Kraft, 2012]. Therefore, we conclude the results
shows the proposed expression has possibly positive effect for
the stress by speech recognition error.
Discussion

20. Dept. of Home Electronics yamalab.com
 Smart robot using web resource and humor expression
has been proposed as a casual interface in super smart
society.
 We have applied the proposed smart robot to the library
guidance system.
 The three typed evaluation has been conducted for 30
subjects, and the proposed smart robot shows a 264.7%
increase compared to the situation without humor
expression in the smile expression time.
 The system is able to be widely applied to existing ICT
system in daily life.
Conclusions
20/20