Although there are many ways to optimize the speech generated by Amazon Polly's text-to-speech voices, you might find it challenging to apply the most effective enhancements in each situation. Learn how you can control pronunciation, intonation, and timing for text-to-speech voices. In this session, you get a comprehensive overview of the available tools and methods available for modifying Amazon Polly speech output, including SSML tags, lexicons, and punctuation. You also get recommendations for streamlining application of these techniques. Come away with insider tips on the best speech optimization techniques to provide a more natural voice experience.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Amazon Polly Tips and Tricks: How to
Bring Your Text-to-Speech Voices to Life
R e m u s M o i s – A m a z o n T e x t - t o - S p e e c h
A m a z o n P o l l y
N o v e m b e r 3 0 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to expect from this session
• What is Amazon Polly?
• Introducing Seoyeon, Matthew, Takumi, Aditi, and Vicki
• Controlling the output of text-to-speech
• Bringing Amazon Polly voices to life: The Magic Door
• Q&A

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What is Amazon Polly?
• A service that converts text into lifelike speech
• Offers 52 lifelike voices and 25 languages
• Low latency responses enable developers to build
real-time systems
• Developers can store, replay, and distribute
generated speech

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
How to use Amazon Polly
The Amazon Polly service provides API operations for synthesizing high-
quality speech from plain text and Speech Synthesis Markup Language
(SSML), along with managing pronunciation lexicons that enable you to
get the best results for your application domain.

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Text-to-speech pipeline
Text
Text normalization
Grapheme-to-phoneme
conversion
Waveform
generation
Speech
100% Recycled – 8 ½ x 11 inch 20 lb Office Paper – 3,000 count
one hundred percent recycled, eight and a half by eleven inch…
ˈwʌn ˈhʌndrəd pɚˈsɛnt riːˈsaɪkəld ˈeɪt ənd ə ˈhæf ˈɪntʃ…

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Polly TTS converts text into intelligible, accurate, and natural speech
• G2P: rough, though, through.
• Homographs: same spelling, different pronunciations.
I live in Poland
This presentation is broadcasted live from Las Vegas
Context helps 'live' disambiguation. But... I read this book.
• Text normalization: disambiguation of abbreviations, acronyms, units ‘St.’ expanded as
‘street’ or ‘saint’
Example: St. Patrick St.
• Foreign words (déjà vu), proper names (Emmanuel Macron), social media lingo (ASAP, LOL),
and so on.
Main challenges of text-to-speech

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
New Amazon Polly voices
Seoyeon
Matthew
Takumi
Vicki
Aditi

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Speech Synthesis Markup Language (SSML)
• W3C recommendation, XML-based markup language for speech
synthesis applications. Amazon Polly tags are compliant with SSML 1.1
specifications.
• A powerful tool which allows customers to modify certain aspects of the
TTS speech output, such as pronunciation of words, specify expansion of
abbreviations, acronyms. It also enables modifications of pitch, speech
rate, volume, and so on.
Controlling the output of TTS

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The _{tag
In-line aliasing – In many cases we do not want to change all instances of a
certain word.
<speak>
My favorite chemical element is
<sub alias="aluminum">Al},but Al prefers
<sub alias="magnesium">Mg</sub>.
</speak>

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
The <lang> tag
Foreign words and phrases – Foreign phrases are rendered better if they
are enclosed inside the <lang> tag, as in the following example.
German in English
<speak>
<lang xml:lang=”de-DE">Sebastian Kurz,</lang> Austrian
conservative set to become world's youngest leader.
</speak>

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<say-as interpret-as="expletive">
Bleeping undesirable content
<speak>
The longest word in English is <say-as interpret-
as="expletive">pneumonoultramicroscopicsilicovolcanoconiosis</say
-as>
</speak>

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
<say-as interpret-as="spell-out">
Read character by character
<speak>And here is how you spell handkerchief: <prosody
rate="x-slow"><say-as interpret-as="spell-
out">handkerchief</say-as></prosody>.</speak>

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Doing impersonations
<speak>
<prosody rate='-30%'>
<prosody pitch="x-low">
Alfred, did you know there is a city called Batman in Turkey?
</prosody>
</prosody>
</speak>

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Assign custom pronunciation (IPA or X-Sampa alphabets)
<lexeme><grapheme>gif</grapheme><phoneme>"dZIf</phoneme></lexeme>
<lexeme><grapheme>David</grapheme><phoneme>"dA.%vid</phoneme>
</lexeme>
<speak>I like this gif.</speak>
<speak>Here's my friend David.</speak>
Lexicons: <phoneme>

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Fun with SSML
'Can you make your voices sound like an auctioneer?'
<speak><prosody rate='+60%'>I’m at 500 and I want
550<prosody volume='x-loud'>550</prosody></prosody>
<prosody rate='+60%'>bid on 550 I’m at 500 would you go
550 550 for the gentleman in the corner</prosody><prosody
rate="+90%">A big black bug bit a big black bear a big
black bug bit a big black bear.</prosody> Do we get 600?
<prosody rate='+90%'>A big black bug bit a big black
bear.</prosody><prosody rate='+60%'>We got 600 for the
whole herd</prosody><prosody rate='default' volume='x-
loud'>Sold<prosody rate='+60%'>for
600.</prosody></prosody></speak>

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
THE MAGIC DOOR
Andrew Huntwork – Founder and CEO, The Magic Door
B R I N G Y O U R T E X T - T O - S P E E C H V O I C E S T O L I F E

17. The Magic Door
• Interactive storytelling by voice
• 8 million minutes of listening
• 900,000 players
• TTS, human voices, and sound effects
• 20 hours of original stories
• 50 characters voiced by Amazon Polly
• Thousands of handwritten SSML tags

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Talestreamer (https://talestreamer.ai)
• The Magic Door’s audio engine for your voice application
• Combine Amazon Polly voices
• Background sound
• RESTful API
• Interactive editor

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Talestreamer (https://talestreamer.ai)
• The Magic Door’s audio engine for your voice application
• Combine Amazon Polly voices
• Background sound
• RESTful API
• Interactive editor

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Our process
1. Write story XML
2. Add synthesized speech with Amazon Polly
3. Add sound effects
4. Add human speech
5. Launch
6. Iterate

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
An example – Madame Faro Tells Your Fortune

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Making TTS characters sound great
1. Get to know the actors
2. Cast them
3. Direct them (with SSML)

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Our actors
Brian Brian, x-high Hans, slow, low
Amy Brian, x-low Russell, high
Raveena Brian, high Salli, x-low
Emma Amy, low, slow Kimberly, x-high
Russell Emma, fast, high Joey, x-low
Nicole Emma, x-high Ivy, high
Geraint Emma, x-low Raveena, high
Justin Emma, low Raveena, low
Joanna Emma, high Russell, low
Kimberly Geraint, low Justin, slow
Salli Geraint, high Geraint, high, slow
Joey Geraint, low, slow

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Directing Raveena with SSML
<speak>Ah, yes, now you: You have a passion for adventure.
And look at this: You are a good problem solver. Now, let
me see. uh huh. You are clearly very brave. You know, my
profound intuition tells me that you, yes, you two, make a
great team. Together you can do wonderful things and
journey far. And I believe you two would be perfect for a
particular challenge that needs solving. Tell me: Are you
interested in a new challenge?
</speak>

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Directing Raveena with SSML – Punctuation
<speak>Ah, yes. Now You. You have a passion for
adventure. And. look at this. You are a good problem solver.
Now. let me see. uh huh. You are clearly very
brave.</speak>

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Directing Raveena With SSML — Detailed Coaching
<speak>Ah, yes. Now <phoneme ph="%ju">you</phoneme>.
You have a passion for adventure. And. look at this. You are a
good problem solver. Now. let me see. uh huh. You are clearly
very brave.</speak>

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Directing Raveena with SSML – Breaks
<speak>Ah, yes. Now <phoneme ph=“%ju”>you.</phoneme> You have
a passion for adventure. <break time=".5s"/> And. look at this. <break
time=".4s"/> You are a good problem solver. <break time=".3s"/>Now.
let me see. <break time=".4s"/> uh huh. You are clearly very
brave.</speak>

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Directing Raveena with SSML – Prosody
<speak><prosody rate="x-slow">Ah, yes. </prosody><prosody rate="-
50%">Now <phoneme ph=“%ju”>you.</phoneme></prosody> You
have <prosody rate="x-slow"> a passion </prosody> for adventure.
<break time=".5s"/><prosody rate="x-slow">And.</prosody> <prosody
rate="slow">look at this.</prosody><break time=".4s"/> You are
<prosody rate="x-slow"> a good</prosody> <prosody
rate="slow">problem solver. </prosody> <break time=".3s"/>Now.
<prosody rate="x-slow"> let me see. <break time=".4s"/> uh ha.
</prosody><prosody rate="slow"> You are clearly </prosody>
<prosody rate="x-slow"> very brave. </prosody></speak>

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Putting it all together

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
A M A Z O N P O L L Y