9. Rest3d … to the rescue
http://en.wikipedia.org/wiki/Representational_state_transfer
The REST architectural style was developed by W3C Technical Architecture Group (TAG) in parallel with HTTP 1.1,
based on the existing design of HTTP 1.0. The World Wide Web represents the largest implementation of a system
conforming to the REST architectural style
www.rest3d.com
https://groups.google.com/forum/#!forum/3d-rest
discuss the principles of a shared RESTful http API dedicated to 3D media
https://github.com/amd/rest3d/
AMD open-source experimental implementation
Client:
- Workstation style UI in the web browser
- COLLADA and glTF javascript parser
- WebGL renderer
Server:
- node.js http server, server cache, database proxy
- XML native database, Xquery processing
9 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
13. graphics library Transmission Format
COLLADA WG initiative (Khronos)
‒ work-in-progress, not yet a ratified specification
‒ specification and implementation (converter) work at the same time
‒ runtime asset format for GL based APIs
Principles
‒ Streamline rendering: size, speed and ease of implementation
‒ Offload processing from runtime
‒ Built on top of GL Based APIs and Typed Arrays
13 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
14. graphics library Transmission Format
glTF & Open3DGC Integration
‒ First cut, Open3DGC glTF extension is a work-in-progress
‒ geometry: POSITION, NORMAL, multiple sets of TEXCOORD, COLOR and generic attributes
‒ animations: TRS (Translation, Rotation, Scale)
‒ skinning: weights and bone indices
Demo example: Virtual City (www.3drt.com)
geometry
animation
total
raw: 5469 kb
gzip: 1612 kb
COLLADA
glTF
680 kb
1297 kb
raw: 1977 kb
gzip: 878 kb
glTF Open3DGCbinary
198 kb
297 kb
raw: 496 kb
gzip: 429 kb
glTF Open3DGC- ascii
355 kb
533 kb
raw: 888 kb
gzip: 390 kb
14 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
17. OPEN-3DGC: WHAT IS IT?
Efficient Implementation of MPEG 3D Graphics Codecs
‒ Encoders in C++ for fast server side compression
‒ Decoders in javascript and C++ to target mobile and web applications
‒ Available under MIT license https://github.com/amd/rest3d/tree/master/server/o3dgc
Codecs
‒ 3D meshes with multiple attributes
‒ Geometry (e.g., positions, normals, texture coordinates…)
‒ Skinning (e.g., animation weights and joint IDs)
‒ Animations
‒ Bone-based animations
‒ Morphing (coming soon)
17 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
MPEG
18. OPEN-3DGC: HOW EFFICIENT IS IT?
Compression Efficiency
400
‒ Gzip (default level=6)
‒ OpenCTM (default settings)
‒ Open3DGC and Webgl-loader
Size (MBytes)
‒ Positions on 14 bits
‒ Normals and texCoords on 10 bits
300
Gzip
OpenCTM
200
Webgl-loader + Gzip
Open3DGC-ASCII + Gzip
Open3DGC-Binary
100
0
CAD
(3748 models)
3D Scanned
(78 models)
MPEG dataset
(1211 models)
Open3DGC is 5x-9x more efficient than Gzip, 1.3x-2.4x more efficient than OpenCTM and
1.2x-1.5x more efficient than webgl-loader
18 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
19. OPEN 3DGC: HOW FAST IS IT?
Javascript Decoding Speed
‒ Desktop machine
‒
‒
‒
‒
‒ Samsung Galaxy S4
‒ Android 4.2.2
‒ Chrome
Desktop decoding
time (ms)
Smart phone decoding
time (ms)
“Hand”
100K
130
1045
54K
85
768
“Octopus”
‒ Smart phone
Number of
triangles
“Dilo”
Windows® 64-bit
AMD Phenom™ II X4 B95 CPU @ 3.0GHz
8GB RAM
Chrome
34K
65
457
Decoding speed will become even more critical with dense 3D meshes
generated by 3D digitization technologies (e.g., 3D scanners)
19 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
20. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression
‒ Indexed Face Set
‒ Geometry: positions
‒ Connectivity: list of triangles
Requires 192 bits per vertex!
‒ Redundancy
‒
‒
‒
‒
Indexes repeated multiple times
No need to preserve triangles and vertices order
No need for 32-bit precision for positions/attributes
Neighbour vertices exhibit high geometry correlations
Lossy geometry compression and lossless connectivity
encoding reduce the stream size to 10-20 bpv
20 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
V7
Geometry
Connectivity
X1, Y1, Z1
X2, Y2, Z2
X3, Y3, Z3
X4, Y4, Z4
X5, Y5, Z5
X6, Y6, Z6
X7, Y7, Z7
1, 3, 7
1, 7, 6
1, 6, 2
1, 2, 4
1, 4, 5
Reordered
geometry
Reordered
Connectivity
X3, Y3, Z3
X7, Y7, Z7
X6, Y6, Z6
X2, Y2, Z2
1
X4, Y4, Z4
2
X5, Y5, Z5
3
X1, Y1, Z1
4
7, 5, 6
7, 4, 5
7, 3, 4
7, 1, 2
7, 2, 3
5
6
7
V3
V6
V5
V1
V4
V2
21. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression
‒ Algorithm based on TFAN codec *Mammou’09+
‒ Triangular meshes with attributes
‒ Arbitrary topologies (e.g., manifold or not, open/closed, oriented or not, arbitrary genus, holes…)
‒ MPEG-SC3DMC (Scalable Complexity 3D Mesh Coding) published in 2010
Encoder
Connectivity
TFAN-based Connectivity
Encoding
Arithmetic
or ASCII
Encoding
Geometry
21 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
Uniform
Quantization
Compressed
stream
Prediction
*Mammou’09+ K. Mammou, T. Zaharia, F. Prêteux, “TFAN: A low complexity 3D mesh compression algorithm”
Computer Animation and Virtual Worlds, Vol. 20(2-3), pp. 343–354, 2009
22. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression
‒ Algorithm based on TFAN codec *Mammou’09+
‒ Triangular meshes with attributes
‒ Arbitrary topologies (e.g., manifold or not, open, closed, holes…)
‒ MPEG-SC3DMC (Scalable Complexity 3D Mesh Coding) published in 2010
Encoder
Connectivity
TFAN-based Connectivity
Encoding
Arithmetic
or ASCII
Encoding
Geometry
22 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
Uniform
Quantization
Compressed
stream
Prediction
*Mammou’09+ K. Mammou, T. Zaharia, F. Prêteux, “TFAN: A low complexity 3D mesh compression algorithm”
Computer Animation and Virtual Worlds, Vol. 20(2-3), pp. 343–354, 2009
23. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression
‒ Uniform quantization
‒ Map real numbers to integers
‒ Maximum quantization error
Uniform quantization reduces the
number of bits per vertex for positions
from 96 bpv to 24 bpv
q=4
q=6
23 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
q=8
q=10
Original
24. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression
‒ Algorithm based on TFAN codec *Mammou’09+
‒ Triangular meshes with attributes
‒ Arbitrary topologies (e.g., manifold or not, open, closed, holes…)
‒ MPEG-SC3DMC (Scalable Complexity 3D Mesh Coding) published in 2010
Encoder
Connectivity
TFAN-based Connectivity
Encoding
Arithmetic
or ASCII
Encoding
Geometry
24 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
Uniform
Quantization
Compressed
stream
Prediction
*Mammou’09+ K. Mammou, T. Zaharia, F. Prêteux, “TFAN: A low complexity 3D mesh compression algorithm”
Computer Animation and Virtual Worlds, Vol. 20(2-3), pp. 343–354, 2009
25. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ Geometry prediction
‒ Exploit connectivity information
‒ Differential and “parallelogram” prediction
P’
B’
Unprocessed region
A
P
B
C
Processed region
25 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
26. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ Geometry prediction
‒ Exploit connectivity information
‒ Differential and “parallelogram” prediction
‒ Adaptively choose the best predictor
Geometry prediction reduces the
number of bits per vertex for positions
from 24 bpv to 10.3 bpv
P’
B’
Unprocessed region
A
P
B
C
Processed region
26 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
27. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression
‒ Algorithm based on TFAN codec *Mammou’09+
‒ Triangular meshes with attributes
‒ Arbitrary topologies (e.g., manifold or not, open, closed, holes…)
‒ MPEG-SC3DMC (Scalable Complexity 3D Mesh Coding) published in 2010
Encoder
Connectivity
TFAN-based Connectivity
Encoding
Arithmetic
or ASCII
Encoding
Geometry
27 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
Uniform
Quantization
Compressed
stream
Prediction
*Mammou’09+ K. Mammou, T. Zaharia, F. Prêteux, “TFAN: A low complexity 3D mesh compression algorithm”
Computer Animation and Virtual Worlds, Vol. 20(2-3), pp. 343–354, 2009
28. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ Triangle FAN
‒
‒
‒
‒
Each two successive triangles share a common edge
All triangles share a common vertex (i.e., center of the TFAN)
All the triangles have the same orientation
Described by enumerating the vertices in their traversal order
Implicitly encodes adjacency information
(1,3,7,6,2,4,5)
7 indices instead of 15 for IFS
28 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
29. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ TFAN-based connectivity compression
‒ Decompose the mesh into a set of TFANs
‒ Traverse the vertices from neighbour to neighbour
‒ Rename vertices according to the traversal order
‒ For each vertex, group its incident non-visited triangles into TFANs
V5 V’4
V7 V’7
V3 V’10
V6
V’3
V’6 V9
V10 V’8
29 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
V4
V’2
V8 V’9
V2 V’5
V1 V’1
30. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ TFAN-based connectivity compression
‒ Decompose the mesh into a set of TFANs
‒ Traverse the vertices from neighbour to neighbour
‒ Rename vertices according to the traversal order
‒ For each vertex, group its incident non-visited triangles into TFANs
‒ Encode TFANs
‒ Distinguish 10 topological configurations
‒ Use local indices instead of absolute ones
30 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
31. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ TFAN-based connectivity compression
‒ Decompose the mesh into a set of TFANs
‒ Traverse the vertices from neighbour to neighbour
‒ Rename vertices according to the traversal order
‒ For each vertex, group its incident non-visited triangles into TFANs
‒ Encode TFANs
‒ Distinguish 10 topological configurations
‒ Use local indices instead of absolute ones
‒ Entropy encoding
‒ Exploit statistical redundancy
100%
TFAN-based connectivity compression
reduces the number of bits per vertex for
connectivity from 96 bpv to 3.1 bpv
Frequency
80%
60%
40%
20%
0%
1
2
3
4
5
6
TFAN configurations
31 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
7
8
9
10
33. CONCLUSIONS
Content delivery is becoming crucial for Web and mobile 3D applications
Rest3D paves the way toward a unified standard to negotiate and exchange 3D data
glTF provides a efficient out-of-the-box 3D content delivery solution
Open3DGC leverages MPEG codecs to offer a complete set of 3D graphics compression technologies
33 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
36. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Encoding Example
36 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
37. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ Encoding example
Bitstream
10
37 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
Vertex
re-ordering
Neighbors
list
38. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
‒ Encoding example
{}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
Bitstream
10 1,(7,1)
38 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
39. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{V4}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
Bitstream
10 1,(7,1) 2,(4,1)
39 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
40. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{V4, V5}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
V5 → V4
Bitstream
10 1,(7,1) 2,(4,1),(7,2)
40 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
41. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{V5, V2, V9, V7}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
V5 → V4
V2 → V5
V9 → V6
V7 → V7
Bitstream
10 1,(7,1) 2,(4,1),(7,2) 0
41 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
42. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
V5 → V4
V2 → V5
V9 → V6
V7 → V7
Bitstream
10 1,(7,1) 2,(4,1),(7,2) 0 0
42 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
43. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{V9}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
V5 → V4
V2 → V5
V9 → V6
V7 → V7
Bitstream
10 1,(7,1) 2,(4,1),(7,2) 0 0 1,(4, 1)
43 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
44. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{V7,V10}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
V5 → V4
V2 → V5
V9 → V6
V7 → V7
V10 → V8
Bitstream
10 1,(7,1) 2,(4,1),(7,2) 0 0 1,(4, 1)
2,(6,2)
44 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
45. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
‒ Encoding example
Vertex
re-ordering
Neighbors
list
C6
C2
C7
C3
C8
C4
C9
C5
V1 → V1
V6 → V2
C1
C10
{V7,V10,V8}
V4 → V3
V5 → V4
V2 → V5
V9 → V6
V7 → V7
V10 → V8
Bitstream
10 1,(7,1) 2,(4,1),(7,2) 0 0 1,(4, 1)
V8 → V9
2,(6,2),(4,1)
45 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013
46. OPEN-3DGC: WHAT’S UNDER THE HOOD?
3D Mesh Compression Algorithm
Vertex
re-ordering
Neighbors
list
V1 → V1
V6 → V2
‒ Encoding example
{V3}
C1
C6
C2
C7
C3
C8
C4
C9
C5
C10
V4 → V3
V5 → V4
V2 → V5
V9 → V6
V7 → V7
V10 → V8
Bitstream
10 1,(7,1) 2,(4,1),(7,2) 0 0 1,(4, 1)
V8 → V9
V3 → V10
2,(6,2),(4,1) 0
46 | EFFICIENT 3D WEB CONTENT DELIVERY | NOVEMBER 19, 2013