1. HTML5 APIs
NATIVE MULTIMEDIA SUPPORT AND BEYOND
Patrick H. Lauke / University of Leeds / 5 May 2011

2. “...extending the language to better
support Web applications [...] This puts
HTML in direct competition with other
technologies[...] , in particular Flash and
Silverlight.”
Ian Hickson, Editor of HTML5
http://lists.w3.org/Archives/Public/public-html/2009Jan/0215.html

3. making your site Fonzie compliant...

4. <video>

5. Adobe Flash currently most common
video delivery mechanism

6. <object width="425" height="344">
<param name="movie"
value="http://www.youtube.com/v/9sEI1AUFJKw&hl=en
&fs=1&"></param>
<param name="allowFullScreen"
value="true"></param>
<param name="allowscriptaccess"
value="always"></param>
<embed
src="http://www.youtube.com/v/9sEI1AUFJKw&hl=en&f
s=1&" type="application/x-shockwave-flash"
allowscriptaccess="always" allowfullscreen="true"
width="425" height="344"></embed>
</object>

7. <video src="video.webm"
controls
autoplay
loop
preload="none"
poster="poster.jpg"
width="320" height="240">
<a href="video.webm">Download movie</a>
</video>

8. video as native object
● “plays nice” with rest of the page
●
keyboard accessibility built-in

9. video formats
the big debate

10. MP4 / H.264
ubiquitous, patent encumbered, licensing/royalties

11. Ogg Theora
“free and open”, no licensing/royalties
not many tools for it, not web optimised

12. WebM
open and royalty-free, web optimised

13. www.youtube.com/html5

14. providing multiple sources
<video controls autoplay poster="…" width="…" height="…">
<source src="movie.webm" type="video/webm" />
<source src="movie.ogv" type="video/ogg" />
<source src="movie.mp4" type="video/mp4" />
<!-- fallback content -->
</video>

15. still include fallback for old browsers
http://camendesign.com/code/video_for_everybody

16. <video controls autoplay poster="…" width="…" height="…">
<source src="movie.webm" type="video/webm" />
<source src="movie.ogv" type="video/ogg" />
<source src="movie.mp4" type="video/mp4" />
<object width="…" height="…" type="application/x-
shockwave-flash" data="player.swf">
<param name="movie" value="player.swf" />
<param name="flashvars" value=" … file=movie.mp4" />
<!-- fallback content -->
</object>
</video>

17. powerful (simple) API
to script your own controls

18. icant.co.uk/easy-youtube

19. www.w3.org/TR/html5/video.html#media-elements

20. controlling a <video> element
var v = document.getElementById('player');
v.play();
v.pause();
v.volume = … ;
v.currentTime = … ;
…

21. events fired by <video> element
var v = document.getElementById('player');
v.addEventListener('loadeddata', function() { … }, true)
v.addEventListener('play', function() { … }, true)
v.addEventListener('pause', function() { … }, true)
v.addEventListener('timeupdate', function() { … }, true)
v.addEventListener('ended', function() { … }, true)
…

22. people.opera.com/patrickl/experiments/webm/basic-controls

23. people.opera.com/patrickl/experiments/webm/fancy-controls

24. people.opera.com/patrickl/experiments/webm/fancy-swap

25. <audio>

26. audio exactly the same as video
<audio src=”music.mp3” controls autoplay></audio>
[...]
<audio controls autoplay>
<source src="music.mp3" type="audio/mpeg" />
<source src="music.oga" type="audio/ogg" />
<!-- fallback content -->
</audio>
same format debacle: MP3 vs Ogg Vorbis (vs WAV)

27. <canvas>

28. canvas = “scriptable images”
<canvas width="…" height="…"></canvas>

29. canvas has standard API methods for drawing
ctx = canvas.getContext("2d");
ctx.fillRect(x, y, width, height);
ctx.beginPath();
ctx.moveTo(x, y);
ctx.lineTo(x, y);
ctx.bezierCurveTo(x1, y1, x2, y2, c1, c2);

30. people.opera.com/patrickl/experiments/canvas/particle/3

31. mariuswatz.com/works/abstract01js

32. www.benjoffe.com/code/demos/canvascape

33. canvas drawing ready-made images
ctx = canvas.getContext("2d");
var logo = new Image();
logo.src = 'logo.png';
ctx.drawImage(logo,x1,y1,w1,h1,x2,y2,w2,h2);
or call in an existing image already on the page

34. www.splintered.co.uk/experiments/archives/paranoid_0.3

35. canvas access to image data array
ctx = canvas.getContext("2d");
canvasData = ctx.getImageData(x,y,w,h);
[R,G,B,A,R,G,B,A,R,G,B,A,R,G,B,A, … ]

36. www.splintered.co.uk/experiments/archives/canvas-ambilight

37. github.com/mezzoblue/PaintbrushJS

38. canvas also works with video
ctx = canvas.getContext("2d");
v = document.getElementById('player');
ctx.drawImage(v,x1,y1,w1,h2,x2,y2,w2,h2);
grab currently displayed frame (update as appropriate)

39. html5doctor.com/video-canvas-magic

40. media.chikuyonok.ru/ambilight

41. www.filamentgroup.com/examples/charting

42. canvas accessibility concerns

43. video, audio and canvas on any device
without plugins
(Java / Flash / Silverlight not ubiquitous)

46. HTML5 (and friends) has
lots more APIs for developers
(for powerful client-side apps)

47. isgeolocationpartofhtml5.com

48. geolocation
navigator.geolocation.getCurrentPosition(success, error);
navigator.geolocation.watchCurrentPosition(success, error);
function success(position) {
/* where's Wally? */
var lat = position.coords.latitude;
var long = position.coords.longitude;
...
}

49. owlsnearyou.com

50. offline detection
window.addEventListener('online', function(){ … }, true);
window.addEventListener('offline', function(){ … }, true);
…however, currently unreliable!

51. application cache
<html manifest=”blah.manifest”>
CACHE MANIFEST
# send this with correct text/cache-manifest MIME
images/sprites.png
scripts/common.js
scripts/jquery.js
styles/global.css
NETWORK:
# never cached (apart from normal caching mechanism)
onlineonly.css
FALLBACK:
# pattern matching. fallback file will also be cached
images/ images/not-offline.png

52. localStorage/sessionStorage
like cookies...
document.cookie = 'key=value; expires=Thu, 15 Feb 2010
23:59:59 UTC; path=/'
…
/* convoluted string operations go here … */

53. localStorage/sessionStorage
like cookies...on steroids!
localStorage.setItem(key, value);
localStorage.getItem(key);
localStorage.clear();
localStorage.key = value;
if (localStorage.key == '…') { … }
…

54. WebSQL
relational DB / SQL in browser
var db =
openDatabase(dbName, version, displayName, expectedSize);
db.transaction(function(tx) {
tx.executeSql(sqlStatement, [], function (tx, result) {
/* do something with the results */
});
});
not being developed further (IE/Mozilla prefer IndexedDB)
– nonetheless available right now in Opera+WebKit!

55. is it all safe to use, right now?

56. don't do browser sniffing
http://www.flickr.com/photos/timdorr/2096272747/

58. feature-detection
progressive enhancement, graceful degradation – or use shims
http://diveintohtml5.org/everything.html

59. feature-detection for audio/video
if (!!document.createElement('video').canPlayType;) { … }
if (!!document.createElement('audio').canPlayType;) { … }

60. feature-detection for audio/video codecs
var v = document.createElement('video');
if (!!(v.canPlayType)&&
((v.canPlayType('video/webm;codecs="vp8,vorbis"') == 'probably')
||
(v.canPlayType('video/webm;codecs="vp8, vorbis"') == 'maybe')))
{ … }

61. ready-made HTML5 audio/video players
(for the lazy)

62. sublimevideo.net

63. videojs.com

64. www.happyworm.com/jquery/jplayer

65. HTML5 as Flashkiller?

66. not a question of HTML5 replacing Flash...

67. giving developers a choice!

68. www.opera.com/developer
patrick.lauke@opera.com