GPU acceleration on mobile browsers, if it is leveraged correctly, can lead to a smooth and fluid applications, thus improving the user experience. There has been a lot of mentions and best practices of hardware acceleration these days, although so far it has been pretty general and hasn’t provided much technical direction apart from simple magical advice such as “use translate3d”. This talk sheds some more light on browser interactions with the GPU and explain what happens behind the scenes, covering the topic of acceleration of primitive drawing, the use of tiled backing store, and composited layer. Knowing the actual machinery behind hardware acceleration, you will be in the position to plan your strategy to improve the performance of your web application.

1. Wednesday, November 2, 11

2. Wednesday, November 2, 11

3. Hardware Acceleration
on Mobile
Ariya Hidayat & Jarred Nicholls
Sencha
Twitter: @ariyahidayat @jarrednicholls
Wednesday, November 2, 11

4. Challenges
Wednesday, November 2, 11

5. Game vs Web
Wednesday, November 2, 11

6. Game
Text
Textured
objects
Animation
Transformation Physics
Wednesday, November 2, 11

7. Web Pages
Images
Text
Wednesday, November 2, 11

8. How Browser Works
Network stack
Layout engine
JavaScript engine Graphics
User interface
Wednesday, November 2, 11

9. WebKit Components
DOM CSS
WebCore SVG
HTML
rendering
JavaScriptCore
WebKit Library
Wednesday, November 2, 11

10. Platform Abstraction
Network Unicode Clipboard
Graphics Theme Events
Thread Geolocation Timer
Wednesday, November 2, 11

11. Graphics Abstraction
GraphicsContext Mac Chromium Qt Gtk
Skia
CoreGraphics
QPainter
graphics stack
Wednesday, November 2, 11

12. Components of Mobile
Radio
Touch interface
CPU
Power
GPU
Wednesday, November 2, 11

13. Graphics Processor
Divide and conquer
Very specific purpose
Wednesday, November 2, 11

14. Optimized for Games
Fixed geometry
Transformation
Textured triangles
Parallelism
Wednesday, November 2, 11

15. Challenges
Predictable contents (assets) ✔
Mostly text ✔
Mostly images ✔
Expected run-time response immediate
Wednesday, November 2, 11

16. Primitive Drawing
Wednesday, November 2, 11

17. Path is Everything
Triangle Rectangle
Path
Ellipse Polygon
Wednesday, November 2, 11

18. Stroke = Outline
Solid Dashed Dotted Textured
Wednesday, November 2, 11

19. Brush = Fill
None Solid Gradient Textured
Wednesday, November 2, 11

20. Smooth via Antialiasing
Multiple levels of transparency
Perfect for parallelism
Wednesday, November 2, 11

21. Path Approximation
Curves
Polygon
Wednesday, November 2, 11

22. Gradient: Expensive
CPU: sequential, tedious
GPU: parallel, still a lot of work
Wednesday, November 2, 11

23. Blur Shadow: Really Posh
Involved pixel “blending”
GPU: parallel, still tedious
Wednesday, November 2, 11

24. Transformation
Scaling
Perspective
Rotation
Wednesday, November 2, 11

25. Text Rasterization
Wednesday, November 2, 11

26. Font Atlas
Buffer
Bye
Wednesday, November 2, 11

27. Simple to Complex
ello!
H
Simple shape Curves
Solid color Gradient brush
Textured stroke
Blur shadow
Make it as an Serif text
image Rotated
Wednesday, November 2, 11

28. HOW FAST?
Wednesday, November 2, 11

29. Android Drawing
Log file
WebCore
graphics
GraphicsContext
Skia
Wednesday, November 2, 11

30. Example: Bing
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
restorePlatformState
platformDestroy
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
restorePlatformState
platformDestroy
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
fillRect 0,0 800x556 color ff ff ff ff
Wednesday, November 2, 11

31. How Fast?
#include "TimeCounter.h"
bool WebViewCore::drawContent(SkCanvas* canvas, SkColor)
{
uint32_t timestamp = getThreadMsec();
.... painting code ....
DBG_SET_LOGD("% ms", getThreadMsec() - timestamp);
}
external/webkit/WebKit/android/jni/WebViewCore.cpp
Wednesday, November 2, 11

32. Example: Google News
adb logcat -v time | grep drawContent
16:24:04.070 D/webcoreglue( 273): drawContent 11 ms
16:24:04.110 D/webcoreglue( 273): drawContent 13 ms
16:24:04.150 D/webcoreglue( 273): drawContent 13 ms
16:24:04.190 D/webcoreglue( 273): drawContent 10 ms
16:24:04.240 D/webcoreglue( 273): drawContent 10 ms
16:24:04.280 D/webcoreglue( 273): drawContent 13 ms
16:24:04.320 D/webcoreglue( 273): drawContent 13 ms
16:24:04.360 D/webcoreglue( 273): drawContent 13 ms
16:24:06.080 D/webcoreglue( 273): drawContent 12 ms
16:24:06.140 D/webcoreglue( 273): drawContent 10 ms
16:24:06.180 D/webcoreglue( 273): drawContent 13 ms
16:24:06.230 D/webcoreglue( 273): drawContent 14 ms
16:24:06.600 D/webcoreglue( 273): drawContent 26 ms
16:24:06.640 D/webcoreglue( 273): drawContent 13 ms
16:24:06.860 D/webcoreglue( 273): drawContent 33 ms
16:24:06.890 D/webcoreglue( 273): drawContent 12 ms
16:24:06.930 D/webcoreglue( 273): drawContent 13 ms
16:24:06.960 D/webcoreglue( 273): drawContent 13 ms
16:24:07.000 D/webcoreglue( 273): drawContent 13 ms
Wednesday, November 2, 11

33. Backing Store
Wednesday, November 2, 11

34. Maps
Tile
Wednesday, November 2, 11

35. Pinch to Zoom
when you pinch...
Wednesday, November 2, 11

36. Responsive UI
decoupled
Rendering
Processor
User interaction
Wednesday, November 2, 11

37. Rendering vs Interaction
Screen
Backing Store
Rendering
Web Page User interaction
Wednesday, November 2, 11

38. Checkerboard Pattern
Wednesday, November 2, 11

39. Progressive Rendering
Fast but blurry
Crisp but slow
Wednesday, November 2, 11

40. Tiling System
Texture upload
.... ....
CPU GPU
Wednesday, November 2, 11

41. Tile Transformation
Panning = Translation Zooming = Scaling
Wednesday, November 2, 11

42. Backing Store Support
Vector-based
Texture-based
Honeycomb and later
Wednesday, November 2, 11

43. Y U NO
position:fixed
Wednesday, November 2, 11

44. Demo
Wednesday, November 2, 11

45. Layer & Compositing
Wednesday, November 2, 11

46. Elements = Layers
Wednesday, November 2, 11

47. Typical Animation
opacity, movement, scaling, rotation, ...
Wednesday, November 2, 11

48. Immediate Mode
setInterval(function() {
box.draw(x, y);
x += 10;
}, 20);
every animation tick
Wednesday, November 2, 11

49. Scene Graph
box.drawInto(layer);
setInterval(function() {
layer.translate(10, 0); Change transformation
matrix
}, 20);
Wednesday, November 2, 11

50. Logical 3-D
Wednesday, November 2, 11

51. Keep the Textures
Wednesday, November 2, 11

52. Demo
Wednesday, November 2, 11

53. Compositing Support
Honeycomb and later
Wednesday, November 2, 11

54. Well-known Trick
forcing 3-D transform
-webkit-transform: translateZ(0)
Not needed for CSS animation!
Wednesday, November 2, 11

55. Debugging in Safari
defaults write com.apple.Safari IncludeInternalDebugMenu
Wednesday, November 2, 11

56. Compositing Indicators
Texture (number = upload)
Composited layer
No texture
Container layer
Overflow
Wednesday, November 2, 11

57. Debugging in Chrome
about:flags
Wednesday, November 2, 11

58. Avoid Texture Reupload
No reupload Upload
Opacity Everything else!
Position
Size
Animation
“Hardware accelerated CSS”
Wednesday, November 2, 11

59. Avoid Overcapacity
GPU = Limited silicon space
Large layer ➔ broken into “tiles”
Wednesday, November 2, 11

60. Prepare & Reuse
Hide the layer offscreen
Viewport
Wednesday, November 2, 11

61. Wrap Up
Wednesday, November 2, 11

62. Hardware Acceleration
Drawing Backing Layer &
primitives stores compositing
Wednesday, November 2, 11

63. THANK YOU!
Wednesday, November 2, 11

64. QUESTIONS?
ariya @ sencha.com jarred @ sencha.com
ariya.ofilabs.com
ariyahidayat jarrednicholls
Wednesday, November 2, 11