GOR19 Automation of the Real Voice of the Customer - Use of masssive audio and video interaction in online interviews

HTW Berlin · GapFish · pangea labs · insius · Automation of the Real Voice of the Customer. #GOR19 · Cologne 7th
March 2019 · 1
Automation of the Real Voice of the Customer
Use of massive audio and video interaction in online interviews
General Online Research
Cologne, 7th
March 2019
Track: Best Practice Award
Holger Lütters│HTW Berlin
Malte Friedrich-Freksa│GapFish
Dmitrij Feller │pangea labs
Marc Egger│insius u.G.
Mark Wolff │GapFish

March 2019 · 2
Teaming up research and technology competencies for an
example of the future of integrated API research
IDEA & Setup
1

March 2019 · 3
1
Development
questionnaire with
special question types
covering voice and
video formats
Holger Lütters
2
Panel supplier
delivering a massive
sample (9813
contacts) to test the
feasibility in the
existing market
research
infrastructure
Malte Freksa
Mark Wolff
3
Integration of APIs for
text2speech
speech2text and
video2text
inside SaaS survey
tool questfox with
automated
transcription
Dmitrij Feller
The project setup is a blueprint for future research of joint
research of different highly specialized fields of expertise
Teaming up for the future of integrated API research
4
Automated text coding
for open answers with
InMap.
Providing a real time
text analysis API with
sentiment analysis
Marc Egger
Third
Party
APIs

March 2019 · 4
Idea: Speech and Video question formats in research interviews
Welcome to the age of talking to machines
2

March 2019 · 5
Speech is becoming a new interaction standard for digital devices
User Interfaces are changing
Icons made by Freepik from www.flaticon.com
Why not using speech and video answers in
integrated approaches of market research?

March 2019 · 6
Hardware and Software using Digital Speech Assistance is
entering the consumer’s life with voice and video approaches
Speech is everywhere but in research

March 2019 · 7
To use the non-standardized third party hardware a software
development process for each device category is required
Looking for a usable alternative in the world of market research
https://developers.google.com/actions/
Amazon SKILLS Google ACTIONS
https://developer.microsoft.com/en-us/cortana
Microsoft Cortana SKILLS Apple SiriKit
https://developer.apple.com/sirikit/
https://developer.amazon.com/de/alexa-skills-kit

March 2019 · 8
Idea: BYOD allows to use the existing market research infrastructure
Using Text2Speech and Speech2Text in the market research interview with standard devices
BYOD
Bring your own device
Speech2Text (STT)
Receiving Audio Input with
automated transcription
Text2Speech (TTS)
Reading out loud questionnaire
by artificial voice
Standard questionnaire tool
with additional features
Use of existing panel
structures with established
research filter procedures

March 2019 · 9
Empirical approach: Text vs. Speech vs. Video
Large sample study mit 9813 contacts and n=1251 complete interviews
3

March 2019 · 10
Large sample feasibility study with EntscheiderClub from GapFish
with 9 text/voice/video interactions per interview
Classic setting to compare effects of the new opportunities
Text n=437
Control group #A
Standard procedure of typing in text
Voice n=407
Experimental Group #B
Background Speech2Text
transcription procedure
Video n=407
Experimental Group #C
Video Recording with automated
transcription from audio stream
*1,50 € Incentive for questionnaire in german language. 9 Text/Speech/Video Interactions. Median duration 13:20 minutes

March 2019 · 11
A transcript is automatically generated based on the spoken
answer … even with stuttering answers
Transcription success largely depends on the audio quality
Example Male Voice (German)
also ich wäre auf sowas überhaupt nicht gekommen aber
ich frage mich auch wie
aber ich glaube nicht
ich denke nicht
weil
also von
also ich glaube nicht menschen würden sowas machen
ihren partner überwachen lassen
dass es
auch wenn man im streit ist oder irgendwie misstrauen
hat es würde es gibt sie also nicht kann jetzt nur sagen wo
wie wie meine sich so ist aber ich glaube eher nicht
Transcription ProblemImprovement
Automated Transcription Result
Question: "There are fears that people will use these technologies to do inappropriate or even illegal things.
Would you imagine that people might use this to spy after their partner?"

March 2019 · 12
The better the technology used by the participant,
the better the transcription result
Transcription success largely depends on the audio quality
Example Male Video (German)
natürlich birgt es diese gefahren das wird
auch vorkommen so ist der mensch aber das
wird sich nicht verändern ich denke die
vorteile überwiegen je nachdem wie man es
nutzt schwarze schafe gibt es immer und
überall
Transcription ProblemImprovement
Automated Transcription Result

March 2019 · 13
Voice and Video questions outperform in quantity in open answer
questions
Depth in open answers above average
n=437 n=407
ja schon das ist ja naheliegend es gibt ja auch Leute die
die Privatsphäre ihres Partners nicht akzeptieren und
Telefone und Nachrichten und so weiter kontrollieren
E-Mails und Chat Verläufe aber ich denke schon dass
das durchaus naheliegend ist
ja glaube ich definitiv also wir sind im Zeitalter den Medien
und Computer Wirtschaft und ja das ist definitiv ein
einfaches Gerät womit man den Partner überwachen kann
womit man ja auch diverse andere Sachen was illegal
wenn ich sogar höchst kriminell ist
ich denke schon dass es solche Menschen geben wird
die ihren Partner damit überwachen oder auch mich
*Examples from 407 automated transcripts
n=407
sicherlich kann man auch moderne technik illegale sachen
machen aber ja das kann man genauso gut auch ohne die
technik machen also von daher das ist jetzt nicht so der haupt
beweggrund warum ich das ablehnen asos assistentin also das
sicherlich ist das für manche halt noch ein beweggrund mehr
irgendwie zu lästern oder andere menschen sie diffamieren
aber ja aber das ist jetzt nicht so der hauptgrund für mich
ja in der tat ist das wahrscheinlich eine möglichkeit ich bin da
selber glaube ich ganz naiv bzw wenn mir jemand etwas böses
will dann wird er das so oder so schaffen so dann glaube ich
nicht dass die gefahr oder dass die möglichkeit durch diese
dinge wächst aber natürlich kann das auch für menschen die
da sehr empfindlich sind oder sehr ängstlich sind tatsächlich
ein grund sein das abzulehnen ich persönlich glaube das
eigentlich nicht
vogelfutter könnte ich mir gut vorstellen auf menschen
die diesen drang haben andere ständig zu lachen oder zu
wissen was andere tun gibt es das ist glaube ich so dazu
verleitet

March 2019 · 14
The Voice Interview can grow the amount of open answers by
the factor 2,5 in less interview time
Voice and Video outperform in open answers
n=437 n=407 n=407
Amount of
Words
Amount of
Letters
Time used
in seconds
Average amount of letters
per second in interview

March 2019 · 15
The interview experience was rated better in the new interaction
formats
New or just what people are about to expect soon?
Question: Would you recommend the panel provider to use more interviews of that kind?
Net Promoter Score Question

March 2019 · 16
Intense panel effort needed to reach the completion quota.
Video participants more difficult to reach than speech
The approach is already possible in any panel setting, but requires some technical screening
21,7% of starters
-42,6%
-32,7%
-42,3%
-2,2%
16,2% of starters
-51,9%
-41,8%
-39,9%
-3,0%

March 2019 · 17
Analysis of massive audio data today and tomorrow
Analysis of more than 7300 Audio and Video files
4

March 2019 · 18
Massive Audio and Video Data exceeds the possibilities of classic
analysis
9 question blocks as audio or video question, each with n=407 answers = 3663 media files for each route
Anonymized Audio-/Video Files

March 2019 · 19
The detailed text mining approach conducted with insius opens
a brand new API world of text analytics in interviews
The output of massive text answers needs to be better understood

March 2019 · 20
The approach can be combined with additional cognitive service API
ideas to analyze voice and video besides the content
The next step is analytics of the voice pattern and additional information from the video sequence
Voice analytics like computerized
Voice Stress Analysis CVSA
Cognitive Services APIs analyzing
beyond emotional tracking

March 2019 · 21
The Real Voice of the Customer
Critical Reflection
5

March 2019 · 22
The research project shows opportunities for revolutionary new
approaches in interview research
Brand new world of interaction research is not for everyone
● Technical bias / drop-out
● Overrepresentation of
technology-savvy samples
Weaknesses
● Easy access to more text
responses in less time
● Lower barriers for people with
less willingness/possibilities to
express themselves in writing
● More personal information in a
less controlled environment
Strengths
● Human reservations about
new forms of interaction
● Legal questions / data
protection (video as personal
information especially worthy
of protection)
Threats
● New research opportunities
for completely new interaction
designs
● Interviews with illiterate
people
● New ideas for qualitative
interviews
● Speech-Bots in Research
Opportunities
Please do not only evaluate
the new idea based on your
today’s research processes.
Think about how research
could look like tomorrow...

With the approach we are about to build a new game of research
stretching the boundaries of classic market research into new ways of
interaction
Option A
Support established
processes with new
technologies
Option B
Develop completely new
concepts of data collection
and analytics
23

March 2019 · 24
Our joint research is only the start into a new world of market
research in a value chain of expertise in different fields
SpeechBot will replace Chatbots first and human interviewers on the long run
Avatars from Soul Machines https://www.youtube.com/watch?v=rRsBMEwflz8
Example of a research bot using
speech interaction
Next step will be an integration of high-level
avatar interaction with facial coding
questbot is a development of pangea labs based on questfox
www.questbot.de

March 2019 · 25
Authors
HTW Berlin: Prof. Dr. Holger Lütters
Holger.Luetters@htw-berlin.de
GapFish GmbH: Malte Friedrich-Freksa /
Mark Wolff info@gapfish.com
pangea labs GmbH: Ing. Dmitrij Feller
info@pangealabs.com
insius u.G.: Dr. Marc Egger
marc.egger@insius.com
The future of research is probably a future of team research
with expertise in technology and methodology
Thank you for your attention

March 2019 · 26
The idea is to have 4 companies presenting a similar idea.
At GOR19 the topic is: text analytics
Four to the floor @ GOR
18:00
-
19:00
D16: 4-to-the-floor: Text Analytics
Location: Room 248
The audience should learn about the state of the art of an approach in less than one hour.

GOR19 Automation of the Real Voice of the Customer - Use of masssive audio and video interaction in online interviews

Recomendados

Recomendados

Más contenido relacionado

Más de Holger Lütters

Más de Holger Lütters (8)

Último

Último (20)

GOR19 Automation of the Real Voice of the Customer - Use of masssive audio and video interaction in online interviews