At SpeechTEK New York in August 2010, Voxeo's Director of Speech Technologies, Dan Burnett, was part of a panel on "speech mashups". Dan explained what mashups are and demonstrated speech mashups written in Ruby that work on the Tropo service at http://www.tropo.com/ More information about Voxeo's activities at SpeechTEK NY can be found at: http://blogs.voxeo.com/events/speechtek-ny-2010/

1. Speech Mashups
Dan Burnett,Voxeo
SpeechTek 2010

2. What are mashups?
<XML> HTTP
JSON

3. Why VoiceXML?

4. Why not VoiceXML?

5. Tropo scripting
answer/reject/redirect/hangup
call/transfer/conference ask/say/record/log
startCallRecording/stopCallRecording

6. Weather app
• Collect postal code from caller
• Fetch weather from Yahoo using YQL
• Speak weather to the caller

7. Weather app
require 'open-uri'
require 'json'
answer
hangup

8. Weather app
require 'open-uri'
require 'json'
answer
say 'Welcome to the Ruby Yahoo weather reader'
say 'Thats all. Goodbye.'
hangup

9. Weather app
require 'open-uri'
require 'json'
answer
say 'Welcome to the Ruby Yahoo weather reader'
ask 'Enter the ZIP code for a weather check'
say 'Thats all. Goodbye.'
hangup

10. Weather app
require 'open-uri'
require 'json'
answer
say 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]",
:onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI
#Fetch the JSON from the YQL API and convert the resulting
#JSON data to a Ruby hash
#Get the relevant weather channel details and throw them into a hash
#Speak back the results
}
}
ask 'Enter the ZIP code for a weather check', options
say 'Thats all. Goodbye.'
hangup

11. Weather app
require 'open-uri'
require 'json'
answer
say 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]",
:onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI
yahoo_url = 'http://query.yahooapis.com/v1/public/yql?format=json&q='
query = "SELECT * FROM weather.forecast WHERE location = " + choice.value
url = URI.encode(yahoo_url + query)
#Fetch the JSON from the YQL API and convert the resulting
#JSON data to a Ruby hash
weather_data = JSON.parse(open(url).read)
#Get the relevant weather channel details and throw them into a hash
weather_results = weather_data["query"]["results"]["channel"]
#Speak back the results
}
}
ask 'Enter the ZIP code for a weather check', options
say 'Thats all. Goodbye.'
hangup

12. Weather app
require 'open-uri'
require 'json'
answer
say 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]",
:onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI
#Fetch the JSON from the YQL API and convert the resulting
#JSON data to a Ruby hash
#Get the relevant weather channel details and throw them into a hash
#Speak back the results
say weather_results["description"]
say "The wind chill is #{weather_results["wind"]["chill"]} degrees, " +
"the wind speed is #{weather_results["wind"]["speed"]}"
say "The forecast is #{weather_results["item"]["forecast"][0]["text"]}, " +
"with a high of #{weather_results["item"]["forecast"][0]["high"]} degrees, " +
"and a low of #{weather_results["item"]["forecast"][0]["low"]} degrees."
}
}
ask 'Enter the ZIP code for a weather check', options
say 'Thats all. Goodbye.'
hangup

13. Weather app
require 'open-uri'
require 'json'
answer
say 'Welcome to the Ruby Yahoo weather reader'
options = { :choices => "[5 DIGITS]",
:repeat => 3,
:timeout => 7,
:onBadChoice => lambda { say 'Invalid entry, please try again.' },
:onTimeout => lambda { say 'Timeout, please try again.' },
:onChoice => lambda { |choice|
#Set the URI and our YQL select statement, then encode as a URI
#Fetch the JSON from the YQL API and convert the resulting
#JSON data to a Ruby hash
#Get the relevant weather channel details and throw them into a hash
#Speak back the results
}
}
ask 'Enter the ZIP code for a weather check', options
say 'Thats all. Goodbye.'
hangup

14. Demo
• Upload script
• Link to a telephony application name
• Add phone numbers/IM/Jabber/SMS

15. Tropo Web API
tropo
answer/reject/redirect/hangup
call/session/transfer/conference ask/say/record/on/result/log
startCallRecording/stopCallRecording
JSON (JavaScript Object Notation)

16. Weather app
{"tropo": [
{"say": [{"value": "Welcome to the Yahoo weather reader"}]},
{"ask":
{"say": [{"value": "Enter the ZIP code for a weather check"},
{"value": "Invalid entry, please try again.", "event": "nomatch"},
{"value": "Timeout, please try again.", "event": "timeout"}]},
"repeat": 3,
"timeout": 7,
"choices": {"value": "[5 DIGITS]"},
}]
}
----------------- Receive recognition result ------------------------
{"tropo": [
{"say": [{"value": "The wind chill is ..."},
{"value": "The forecast is ..."},
{"value": "Thats all. Goodbye."}]}}
]}

17. Other mashups
• Yahoo Local over phone, SMS, and IM
https://www.tropo.com/docs/scripting/t_php-localsearch.htm
• SFO BART train arrival estimate
https://www.tropo.com/docs/scripting/t_ruby-barteta.htm
• Find location of someone based on telephone number
https://www.tropo.com/docs/scripting/t_groovy-location-mashup.htm
• Look up political campaign contributions
text: [2-letter state id] [name] to +1.240.242.7944 .. ie "ut craig m smith"
(also on jabber at tdata@tropo.im)
• Look up bills in the NY Senate via IM or Twitter
http://www.voiceingov.org/blog/?p=1005
• Volunteer firefighter arrival time at fire station

18. Summary