Multimodal Interaction in Distributed and Ubiquitous Computing presentation

1. ICIW 2010 - The Fifth International Conference on Internet and Web Applications and Services May 9 - 15, 2010 - Barcelona, Spain Titol de la presentació Multimodal Interaction in Distributed and Ubiquitous Computing Marc Pous and Luigi Ceccaroni bdigital Cognom Nom, CàrrecNom de la jornada, lloc

2. SUMMARY Why multimodality? Multimodal distributed services deployment User tests and conclusions

3. In our cities, urban services practically have not changed in a century

4. MOTIVATIONS 1: Location-based services

5. http://www.flickr.com/photos/nnova/4512346900/

6. http://picasaweb.google.com/marc.pous/Xina2007#5102695479944027890

7. http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415374502175450370

8. http://picasaweb.google.com/marcpous.nyc2009/NYC2009#5415371534810675298

9. http://www.flickr.com/photos/loquat73/3385335980/

10. MOTIVATIONS 2: Interactions and multimodality

11. http://www.flickr.com/photos/nnova/4423756547/in/set-72157623598200316/

12. http://www.flickr.com/photos/nnova/4424521014/in/set-72157623598200316/

13. MOTIVATIONS 3: Accessibility

16. http://www.dynamiclanguageblog.com/2009/12/technology-grows-for-hard-of-hearing.html

17. MOTIVATIONS Whatisan ICD? (InteractiveCommunityDisplay)

19. INREDIS 3G = ACCESSIBILITY + UBIQUITY + INTEROPERABILITY http://www.inredis.es

20. INREDIS ICD

21. Management of distributed services that offer the capability of processing and synthesizing multiple modalities of interaction

22. DISTRIBUTED INTERACTION SERVICES Voice services Voice recognition Via microphone, real-time streaming  converting the voice signal into text Voice synthesis Sign language service Spanish sign language recognition Via webcam, real-time streaming  converting the image into text Emotion service Emotion recognition (voice + image) Via webcam and microphone, real-time streaming Avatar service Emotional avatar Sign-language speaker avatar

23. DISTRIBUTED SERVICES SIGN LANGUAGE SERVICES

24. DISTRIBUTED SERVICES VOICE SERVICES

25. How to consume multimodal distributed services offered by an ICD?

26. W3C MMI Architecture W3C MMI Architecture http://www.w3.org/TR/mmi-arch/

27. W3C MMI Framework http://www.w3.org/TR/mmi-framework/

28. Proposed multimodal interactionframework

29. ICD ARCHITECTURE ICD Architecture

30. ICD DESIGN and INTERFACES (1) Interactionidentification (2) Main page - Mapor avatar - Commandbuttons - Content - Scenariooptions (3) Contextualizedmapwithservices (4) Avatar withmap and Street View service

31. USER–CENTERED DESIGN User profiles (Personas) Blind users Deaf users Unable to read People living or visiting a city Scenarios Context-aware informative scenario Emergency scenario

32. IMPLEMENTATION

33. USABILITY TESTING PLI (PeopleLedInnovation) methodology Preliminar usabilitytestswith a limitednumber of users Butthe ICD implementationreceivedgoodfeedback!

34. CONCLUSIONS and FUTURE WORK Implementation based on the W3C MMI Architecture idea Platform able to integrate multimodal interactive distributed services and offer interaction in real-time Web-based applications and device-independent implementation Improvement of the user experience and accessibility of people with special needs What are we working on now? Real-time interaction: delay reduction between interactions Browser-based apps not being able to easily access device hardware Synchronous interaction vs. Asynchronous technologies Improvement of the distributed services orchestration Enhancement of the mobile usability

35. MOBILE PHONES? http://www.slideshare.net/nokiaconversations/younghee-jung-and-jan-chipchase-presentation-for-nokia-design-london-studio-event-london-29th-april-2008

36. Thankyou! Moltesgràcies! ¡Muchas gracias! Marc Pous and Luigi Ceccaroni mpous@bdigital.org and lceccaroni@bdigital.org Titol de la presentació Cognom Nom, CàrrecNom de la jornada, lloc