In this talk, Chris Mills discusses the historic problems with web apps and which technologies are stepping up to fill the holes. This includes device APIs such as Web Activities, Camera, device orientation and nfc, offline apps (which are finally looking realistic thanks to service workers), installable apps, and high quality games and other immersive high performance experiences using such features as Web audio API, Pointer lock, asm.js and Emscripten.

1. Web versus native
Chris Mills // Mozilla cmills@mozilla.com // @chrisdavidmills

2. don’t worry about taking notes:
๏ These slides are all at slideshare.net/chrisdavidmills
๏ developer.mozilla.org
๏ @chrisdavidmills
๏ cmills@mozilla.com
๏ #mdn irc channel on mozilla irc
๏ dev-mdc@lists.mozilla.org mailing list

3. ๏ heavy metal drummer turned web nerd
๏ tech writer @ mozilla
๏ web tinkerer (HTML, CSS, JS)
๏ accessibility whiner
๏ educator
who am i?
mdn!!

4. what’s the
problem?

5. web versus native, the age-old struggle

6. the usual arguments
native is better?
๏ faster?
๏ offine?
๏ more integrated, consistent experience?
๏ better developer ecosystem/tools?
๏ more secure?

7. the usual arguments
๏ web apps accessing other apps?
๏ web apps accessing
camera
contacts
SMS/MMs
dialler??

8. app ecosystem

9. Firefox os
We have this firefox os thing!
๏ open source project:
everyone can get involved
๏ everything runs on the gecko engine
๏ Gaia: UI and suite of default apps
๏ Gonk: the underlying kernel/haL

10. installable apps
not a new phenomenon, but...
๏ pretty recent concept for web technologies
๏ manifest file defines app (manifest.webapp)
๏ installation controlled by app installation and management apis

11. {
"version": "0.1",
"name": "To-do alarms",
"description": "My awesome open web app",
"launch_path": "/index.html",
"icons": {
"128": "/img/icon-128.png"
},
"developer": {
"name": "Chris Mills",
"url": "http://yourawesomeapp.com"
},
"locales": {
"es": {
"description": "Su nueva aplicación impresionante Open Web",
"developer": {
"url": "http://yourawesomeapp.com"
}
}, manifest example

12. var manifest_url = location.href.substring(0,
location.href.lastIndexOf("/")) + "/manifest.webapp";
function install() {
// install the app
var install = navigator.mozApps.install(manifest_url);
install.onsuccess = function(data) {
// App is installed, do something if you like
};
install.onerror = function() {
// App wasn't installed, info is in
// install.error.name
alert(install.error.name);
};
};
installation example

13. app types
Apps can be:
๏ hosted: just like a normal web site, but with a manifest and install
functionality
๏ packaged: zipped up, hosted somewhere (like the firefox
marketplace), verified

14. app payments
you can charge money for web apps
๏ payments api uses payment providers like bango
๏ receipt verification to make sure payments are completed
๏ in-app payments also available (fxPay, mozpay)

15. web runtime
web rt allows app installation on other platforms
๏ apk factory for android apps, which include native api equivalents
where possible
๏ similar native wrappers for desktop platforms
๏ firefox marketplace/Firefox will generate these

16. developer
experience

17. developer experience
we want to give the web a first class development experience,
alongside native ecosystems:
๏ documentation
๏ developer tools
๏ frameworks, templates, libraries

18. documentation
announce new things, provide useful references, give
recommendations:
๏ hacks blog
๏ mdn (app center)

19. developer tools
developer tools
๏ firefox’s standard toolbox
๏ app manager webide
๏ remote debugging using connect...
๏ you can run gaia on the desktop with firefox mulet
๏ node-firefox

20. webide

21. frameworks and libraries
ready made code to make development easier
๏ mortar app templates
๏ Web components
๏ firefox os boilerplate app
๏ phonegap support for firefox os
๏ and of course, ember, angular, backbone, yadda yadda

22. (device) apis

23. make it work on the web*...

24. *ok, ok, using web technologies!

25. apis!!!
we want to control everything using web technology
๏ apis to handle access to device hardware, system functions, etc.
๏ security handled by permissions, in the manifest

26. api permissions
different apis have different permission levels:
๏ common apis can be accessed by any app
๏ privileged apis can only be used in packaged, verified apps (e.g.
contacts, camera, device storage)
๏ internal (certified) apis can only be used by vendor-installed apps
(e.g. sms, dialer, bluetooth)

27. var pick = new MozActivity({
name: "pick",
data: {
type: ["image/png", "image/jpg", "image/jpeg"]
}
});
web activities (intents)

28. web activities

29. pick.onsuccess = function () {
// Create image and set the returned blob as the src
var img = document.createElement("img");
img.src = window.URL.createObjectURL(this.result.blob);
// Present that image in your app
var imagePresenter = document.querySelector("#image-presenter");
imagePresenter.appendChild(img);
};
pick.onerror = function () {
// If an error occurred or the user canceled the activity
alert("Can't view the image!");
};
web activities

30. var img = '/to-do-notifications/img/icon-128.png';
var text = 'HEY! Your task "' + title + '" is now overdue.';
var notification = new Notification('To do list', { body: text, icon:
img });
notification

31. window.navigator.vibrate(200); // vibrate for 200ms
window.navigator.vibrate([100,30,100,30,100,200,200,30,200,30,200,200,10
0,30,100,30,100]); // Vibrate 'SOS' in Morse.
vibration

32. window.addEventListener('devicelight', function(event) {
var html = document.getElementsByTagName('html')[0];
if (event.value < 50) {
html.classList.add('darklight');
html.classList.remove('brightlight');
} else {
html.classList.add('brightlight');
html.classList.remove('darklight');
}
});
ambient light events

33. ambient light events

34. ambient light events

35. window.addEventListener("deviceorientation", handleOrientation, true);
function handleOrientation(event) {
var alpha = event.alpha;
var beta = event.beta;
var gamma = event.gamma;
// Do stuff with the new orientation data
}
device orientation

36. device orientation
alpha
betagamma

37. getusermedia
var constraints = { audio: true };
var onSuccess = function(stream) {
// do stuff with your media stream
};
var onError = function(err) {
console.log('The following error occurred: ' + err);
}
navigator.getUserMedia(constraints, onSuccess, onError);

38. mediarecorder
var mediaRecorder = new MediaRecorder(stream);
record.onclick = function() {
mediaRecorder.start();
}
stop.onclick = function() {
mediaRecorder.stop();
}
mediaRecorder.ondataavailable = function(e) {
var audio = document.createElement('audio');
audio.setAttribute('controls', '');
var audioURL = window.URL.createObjectURL(e.data);
audio.src = audioURL;
}

39. nfc
function ndefDiscoveredHandler(activity) {
var data = activity.source.data;
var tag = navigator.mozNfc.getNFCTag(data.sessionToken);
console.log(tag instanceof MozNFCTag); // should print true
}
var request = tag.readNDEF();
request.onsuccess = function() {
var ndefRecords = request.result; // ndefRecords should be an array of
MozNDEFRecord.
};

40. nfc
navigator.mozNfc.onpeerready = function (evt) {
var peer = navigator.mozNfc.getNFCPeer(evt.detail);
console.log(peer instanceof MozNFCPeer); // should print true;
};
var request = peer.sendNDEF(ndefRecords);
var request = peer.sendFile(blob);

41. Web speech API
๏ work in progress
๏ doing speech synthesis and recognition in javascript, client-side

42. privileged/certified api examples
๏ CAmERA
๏ FMradio
๏ bluetooth
๏ sms
๏ telephony
๏ Wifimanager

43. advanced communication
sometimes the web model can be a pain
๏ same origin security
๏ cors/systemxhr
๏ broadcast channels/message channels
๏ request response model
๏ web sockets
๏ webrtc

44. broadcast channels
// Connection to a broadcast channel
var bc = new BroadcastChannel(“test_channel");
// Example of sending of a very simple message
bc.postMessage("This is a test message.”);
// Exemple of a simple event handler that only
// logs the event to the console
bc.onmessage = function (e) { console.log(e); }
// Disconnect the channel
bc.close()

45. channel messaging
var channel = new MessageChannel();
var ifr = document.querySelector('iframe');
var otherWindow = ifr.contentWindow;
ifr.addEventListener("load", iframeLoaded, false);
function iframeLoaded() {
otherWindow.postMessage('Hello from the main page!', '*',
[channel.port2]);
}
channel.port1.onmessage = handleMessage;
function handleMessage(e) {
para.innerHTML = e.data;
}

46. offline apps

47. no connection = no experience

48. offline is hard
The main problems are as follows:
๏ offline data storage
๏ offline asset storage
๏ detecting available network and reacting to it

49. offline data
this is not so bad:
๏ indexeddb, localstorage, websql
๏ (datastore api for firefox os)
๏ there’s something available in most browsers
๏ localforage library polyfills across browsers

50. offline apps/assets
offline app assets are a pain
๏ firefox os packaged apps are installed and available offline
๏ this doesn’t help the web at large
๏ we had app cache…

51. …this had promise...
CACHE MANIFEST
# v1
CACHE:
css/app.css
index.html
js/main.js
js/lib/require.js

52. ...but was actually evil.

53. detecting network state
network state is also a pain
๏ Network information API is pretty unreliable
๏ you could check xhr responses, but this isn’t ideal

54. hello service workers!

55. This could be the answer…
service workers are coming
๏ proxy servers that sits between your app and the browser
๏ intercepting network requests and customising responses
๏ does similar things to app cache (plus a lot more)
๏ granular control over actions

56. register service worker
if ('serviceWorker' in navigator) {
navigator.serviceWorker.register('/sw-test/sw.js', {
scope: '/*'
}).then(function(sw) {
// registration worked
console.log('Registration succeeded.');
}).catch(function() {
// registration failed
console.log('Registration failed.');
});
}

57. install service worker
this.addEventListener('install', function(event) {
event.waitUntil(
caches.create('v1').then(function(cache) {
return cache.add(
'/sw-test/',
'/sw-test/index.html',
'/sw-test/style.css',
'/sw-test/app.js',
'/sw-test/image-list.js',
'/sw-test/star-wars-logo.jpg'
// etc.
);
})
);
});

58. custom request responses
this.addEventListener('fetch', function(event) {
event.respondWith(
caches.match(event.request).catch(function() {
return event.default();
return caches.get('v1').then(function(cache) {
cache.add(event.request);
});
}).catch(function() {
return caches.match('/sw-test/gallery/myLittleVader.jpg');
})
);
});

59. performance
enhancers

60. performance enhancers
๏ web workers
๏ web audio api
๏ asm.js
๏ emscripten

61. web workers
web workers
๏ run scripts in a background thread
๏ don’t block the main thread execution

62. web audio api
web audio api
๏ precise control and manipulation of audio
๏ add effects to audio
๏ split and merge audio streams
๏ spatialise audio
๏ create audio visualisations

63. audiochannels
audiochannels api
๏ set different audio to play in different importance hierarchy
channels
๏ react to headphones being plugged in or unplugged
๏ more important audio can continue to play in the background, e.g.
music player audio

64. asm.js
asm.js
๏ a very efficient low-level subset of JS
๏ suitable for ahead-of-time optimizing compilation
๏ Unity3d now has asm.js/WebGL support

65. emscripten
emscripteN
๏ an LLVM to javascript compiler (well, asm.js, specifically)
๏ compile c++ (and others) into JS and run it on the web
๏ = “very fast shit” ™
๏ See emscripten.org

68. resources
๏ MDN: developer.mozilla.org/
๏ demos on github.com/mdn
๏ hacks blog: hacks.mozilla.org
๏ look up localforage - polyfill for indexeddb/websql/localstorage
๏ simplewebrtc.com - simple webrtc library
๏ emscripten.org - try quakejs.com
๏ asmjs.org
๏ mzl.la/openwebapps - give us your feedback on what the web
platform needs!

69. thanks for
listening!!
slideshare.net/chrisdavidmills cmills@mozilla.com // @chrisdavidmills