GPU acceleration on mobile browsers, if it is leveraged correctly, can lead to a smooth and fluid applications, thus improving the user experience. There has been a lot of mentions and best practices of hardware acceleration these days, although so far it has been pretty general and hasn’t provided much technical direction apart from simple magical advice such as “use translate3d”. This talk sheds some more light on browser interactions with the GPU and explain what happens behind the scenes, covering the topic of acceleration of primitive drawing, the use of tiled backing store, and composited layer. Knowing the actual machinery behind hardware acceleration, you will be in the position to plan your strategy to improve the performance of your web application.

1. Understanding Hardware
Acceleration on Mobile
Browsers
Ariya Hidayat

3. Magical Advice
Use translate3d for hardware
acceleration

4. Challenges

5. Game vs Web

6. Game
Text
Textured objects
Animation
Transformation Physics

7. Large but Still Bounded

8. Web Page
Images
Text

9. Challenges
Predictable contents (assets) ✔
Mostly text ✔
Mostly images ✔
Initial response immediate

10. Browser Rendering

11. From Wave to Pixels
Network stack
Layout engine
JavaScript engine Graphics
User interface

12. WebKit Components
Render Engine
CSS
DOM SVG
HTML Canvas
WebCore
Client Interface JavaScript Engine (JSC/V8)

13. Platform Abstraction
Network Unicode Clipboard
Graphics Theme Events
Thread Geolocation Timer

14. Graphics Abstraction
GraphicsContext iOS Android
Skia
CoreGraphics
Graphics stack

15. SoC = System-on-a-Chip
CPU GPU

16. GPU Workflow
Matrix
Vertices Rendered Textured

17. Optimized for Games
“Fixed” geometry
Transformation
Textured triangles
Parallelism

18. Powerful Capabilities
http://epicgames.com/technology/epic-citadel http://www.rage.com/

19. http://www.trollquotes.org/619-super-spider-bat-troll-quote/

20. Fundamental Physical Limitations
 Memory
- Can’t store too much stuff
 Speed
- Quad-core CPU can do certain operations better
 Bandwidth
- Bottleneck can be in the data transfer

21. Traffic Congestion

22. Primitive Drawing

23. Path is Everything
Triangle Rectangle
Path
Ellipse Polygon

24. Stroke = Outline
Solid Dashed Dotted Textured

25. Brush = Fill
None Solid Gradient Textured

26. Gradient = Non-uniform Pixel Values

27. Drawing Time: Solid vs Gradient
Only border
Solid color fill
14x slower
(depending on the dimension)
Linear gradient fill
Radial gradient fill

28. Path Approximation
Curves
Polygon

29. Smooth via Antialiasing
Multiple levels of transparency
Perfect for parallelism

30. Shadow
Involved pixel “blending” Often obstructed

31. Drawing Time: Solid vs Blur Shadow
20x slower
Solid shadow (depending on the blur radius)
Blur shadow

32. CSS = Box Model
http://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of Stacking Contexts

33. Shadow Abuse
12,000 pixels
Blur shadow

34. Transformation
Scaling
Perspective
Rotation

35. Text Rasterization

36. Font Atlas
Buffer
Bye

37. From Simple to Complex
ello!
H
Curves
Simple shape Gradient brush
Solid color Textured stroke
Blur shadow
Serif text
Rotated
Make it as an image

38. Library Instrumentation
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
restorePlatformState
platformDestroy
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
restorePlatformState
platformDestroy
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
fillRect 0,0 800x556 color ff ff ff ff

39. Painting Performance
16:24:04.070 D/webcoreglue( 273): drawContent 11 ms
16:24:04.110 D/webcoreglue( 273): drawContent 13 ms
16:24:04.150 D/webcoreglue( 273): drawContent 13 ms
16:24:04.190 D/webcoreglue( 273): drawContent 10 ms
16:24:04.240 D/webcoreglue( 273): drawContent 10 ms
16:24:04.280 D/webcoreglue( 273): drawContent 13 ms
16:24:04.320 D/webcoreglue( 273): drawContent 13 ms
16:24:04.360 D/webcoreglue( 273): drawContent 13 ms
16:24:06.080 D/webcoreglue( 273): drawContent 12 ms
16:24:06.140 D/webcoreglue( 273): drawContent 10 ms
16:24:06.180 D/webcoreglue( 273): drawContent 13 ms
16:24:06.230 D/webcoreglue( 273): drawContent 14 ms
16:24:06.600 D/webcoreglue( 273): drawContent 26 ms
16:24:06.640 D/webcoreglue( 273): drawContent 13 ms
16:24:06.860 D/webcoreglue( 273): drawContent 33 ms
16:24:06.890 D/webcoreglue( 273): drawContent 12 ms
16:24:06.930 D/webcoreglue( 273): drawContent 13 ms
16:24:06.960 D/webcoreglue( 273): drawContent 13 ms
16:24:07.000 D/webcoreglue( 273): drawContent 13 ms

40. Backing Store

41. Maps
Tiles

42. Responsive Interface
Rendering
Processor
User interaction

43. Pinch to Zoom
when you pinch...

44. Rendering vs Interaction
Rendering
Web Page User interaction

45. Checkerboard Pattern
Poor man’s indicator
of performance

46. Progressive Rendering
Fast but blurry
Crisp but slow

47. Perceived Responsiveness
User pinches
Smooth scaled
Blocky (but fast!)

48. Tiling System
.... ....
CPU GPU
Streamed
What happens if the page
background changes color?

49. Tile Transformation
Panning = Translation Zooming = Scaling

50. Y U NO
position:fixed

51. Layer & Compositing

52. Typical Animation
opacity, movement, scaling, rotation, ...

53. Principles of Fluid Animation
At the beginning, push as many resources
1 as possible to the GPU
During the animation, minimize
2 CPU-GPU interaction

54. Immediate vs Retained
At the beginning...
draw the shape onto a buffer
For every animation tick...
draw the shape at (x, y) blit the buffer at (x, y)
x = x + 10 x = x + 10
Off main thread

55. Mechanics of Animation
“Hey, this is good stuff. Cache it as texture #42.”
Initialization
“Apply [operation] to texture #42.”
Animation step

56. Elements = Layer

57. Logical 3-D

58. Compositing By Force
-webkit-transform: translateZ(0)
Not needed for CSS animation!
.someid {
-webkit-animation-name: somekeyframeanimation;
-webkit-animation-duration: 7s;
-webkit-transform: translateZ(0);
}
Don’t do that

59. Debugging in Safari
defaults write com.apple.Safari IncludeInternalDebugMenu 1

60. Compositing Indicators
Texture (number = upload)
Composited layer
No associated texture
Container layer
Overflow

61. Avoid Texture Reupload
No reupload Upload
Opacity
Position Everything else!
Size
Animation
“Hardware accelerated CSS”

62. Examples

63. Color Transition
@-webkit-keyframes box {
0% { -webkit-transform: background-color: blue; }
100% { -webkit-transform: background-color: green; }
}
Need a new texture uploaded to the
GPU for every in-between color

64. Color Transition: The Trick
Blended with

65. Prepare and Reuse
Hide the layer offscreen
Viewport

66. Timer Latency
Depending on
user reaction
•Animation end triggers the JavaScript callback
•JavaScript code kicks the next animation
Prepare both animation and
hide the “wrong” one

67. Avoid Overcapacity
Texture upload
.... ....
Think of the GPU memory like a cache.

68. Wrap-up

69. Hardware Acceleration
Drawing primitives Backing stores Layer & compositing

70. There is no such
Traditional graphics
programming has been
always full of tricks.
It will always be.

71. Thank You
ariya.hidayat@gmail.com Office Hour
Wed 1:45 PM
ariya.ofilabs.com
@AriyaHidayat