Real time SHVC decoder

1. Real Time SHVC Decoder: Implementation and Complexity Analysis Wassim Hamidouche Research Engineer E-mail: Wassim.Hamidouche@insa-rennes.fr Phone : +33 6 19 14 16 38 October 2014

2. Outline Introduction Real time SHVC decoder Results and analysis Conclusion

4. Introduction (1/3) Scalable High eciency Video Coding (SHVC) standard: Enable higher temporal, spatial, quality, bit depth resolutions and wider color gamut. Use all the powerful of the HEVC standard: quadtree-based block partitioning, large transform and prediction blocks, accurate intra/inter predictions and the in-loop

5. lters.

6. Introduction (2/3) Block diagram of the SHVC encoder with two spatial scalability layers: 4K Downsampling Upsampling scaling MV Entropy coding EL bitstream BL bitstream Original picture SHVC EL encoder HEVC BL encoder Loop filter T /Q T/Q MC/Inter-layer prediction Picture buffer Intra prediction Loop filter T /Q T/Q Entropy coding MC/Inter-layer prediction Picture buffer Intra prediction -1 -1 -1 -1 SHVC bitstream HD

7. Introduction (3/3) Scalable High eciency Video Coding (SHVC) standard: I Uses Inter-layer predictions to improve the coding eciency (15%-30% gain vs independent coding) [1] I Legacy compliant: supports a base layer compliant with the AVC standard I Multi-loop based coding structure: all intermediate layers need to be decoded Access the complexity of a real time SHVC decoder in respect to a simulcast HEVC decoder [1] V. Seregin, T. D. Chuang, Y. He, D. K. Kwon, and F. Le Leannec, AHG Report: SHVC software, in document JCTVC-L0011. Geneva, Switzerland, January 2013.

9. Real time SHVC decoder (1/6) SHVC decoder: I Based on the OpenHEVC decoder [2] I Open source implementation of the HEVC decoder developed in C language on the top of FFmpeg library I Supports the decoding of all HEVC pro

10. les and the HEVC conformance bitstreams I The main decoding operations are heavily optimized in SSE instructions for x86 architecture I Enables all parallel processing solutions adopted in the HEVC standard: tile, slice, wavefront and frame-based [2] Open source HEVC decoder (OpenHEVC) in https:://github.com/OpenHEVC

11. Real time SHVC decoder (2/6) Architecture of the OpenHEVC software: hls_coding_tree Decode a slice hls_sao_filter_ctb Yes hls_deblocking_filter_ctb Is a CTU ? hls_coding_unit hls_prediction_unit hls_transform_tree Yes is a TTU ? hls_transform_unit No Is SAO filter enabled ? No Yes No No Slice decoded Is the CTU decoded ? Yes hls_decode_slice Yes No More CTU in the slice? Yes Is deblocking filter enabled ?

12. Real time SHVC decoder (3/6) Extend the OpenHEVC decoder to decode SHVC enhancement layers (EL): I Parse multi-layers syntax elements at the enhancement layers I New functions to perform the upsampling of the inter-layer reference picture and the upscaling its MVs (8-tap

13. lter for luma and 4-tap

14. lter for chroma) I Manage reference lists by including new inter-layer reference frames Low level optimizations in the SHVC decoder: I Optimize the up-sampling

15. lters in Single Instruction Multiple Data (SIMD) operations (SSE instructions for x86 processor)

16. Real time SHVC decoder (4/6) Multiple instances of the modi

17. ed OpenHEVC decoder, one instance to decode each layer: Demultiplexer SHVC bitstream BL bitstream OpenHEVC decoder OpenHEVC decoder EL bitstream Access to the decoded BL its MVs SHVC decoder

18. Real time SHVC decoder (5/6) Parallelism in the SHVC decoder : I SHVC decoder supports three levels of parallelism: I Wavefront parallelism: the CTB rows of each layer are decoded in parallel for low latency applications I Temporal frame-based parallelism: successive temporal frame are decoded in parallel I Layers parallelism: frames of the SHVC layers are decoded in parallel I Communication control mechanism is implemented to manage wavefront dependencies as well as inter and inter-layer prediction dependencies

19. Real time SHVC decoder (6/6) Hybrid parallel architecture for the SHVC decoder with two layers: B frame Inter prediction B frame P frame P frame No decoded CTBs B frame B frame Inter prediction B frame Decoded CTBs CTB in decoding Wait for the decoding of the referenced PU { { Thread 2 Thread 3 Inter layer prediction I frame I frame Base layer Enhancement layer B frame {{ Thread 0 Thread 1 Thread 5 Thread 6 { { { { Thread 4 Thread 5

21. Results and analysis (1/5) Experimental con

22. guration: I 4 Cores Intel i7 processor running at 2.8 GHz I Common SHVC test conditions: video sequences from class A and B and all QP, 3 scalability con

23. gurations 2x, 1.5x and SNR I Reference Scalable Software Model (SHM) 4.1 is used to encode the test video sequences I Three con

24. gurations of parallelism (n;m) = f(4; 1); (1; 4); (2; 2)g with n: number of threads for wavefront parallelism and m the number of frames decoded in parallel

25. Results and analysis (2/5) Decoding time performance of the SHVC decoder (mono core): Sequences Decoding time (second) HEVC SHVC SSE no SSE SNR 2x 1.5x Kimono 3.53 12.35 5.78 5.06 5.86 ParkScene 4.03 12.08 7.24 5.86 6.92 Cactus 6.52 18.04 12.29 9.51 12.41 BasketBallDrive 8.02 26.56 14.74 11.26 14.27 BQTerrace 9.60 29.53 18.02 12.92 16.56 Trac 4.14 11.27 7.83 6.22 - PeopleonStreet 6.70 15.90 10.42 9.73 - Complexity (%) 0 195 80 43 77

26. Results and analysis (2/5) Decoding time performance of the SHVC decoder (mono core): Sequences Decoding time (second) HEVC SHVC SSE no SSE SNR 2x 1.5x Kimono 3.53 12.35 5.78 5.06 5.86 ParkScene 4.03 12.08 7.24 5.86 6.92 Cactus 6.52 18.04 12.29 9.51 12.41 BasketBallDrive 8.02 26.56 14.74 11.26 14.27 BQTerrace 9.60 29.53 18.02 12.92 16.56 Trac 4.14 11.27 7.83 6.22 - PeopleonStreet 6.70 15.90 10.42 9.73 - Complexity (%) 0 195 80 43 77

27. Results and analysis (3/5) Time repartition in the SHVC decoder (mono core): Transfor m, 2.98 Motion compensa tion, 33.75 Inter-layer prediction , 28.75 Entropy decoding, 10.8 In-loop filters, 16.6 Intra prediction , 0.98 Rest, 6.15 (a) 2x (15.1 seconds) Transfor m, 3.35 Motion compensa tion, 32.45 Inter-layer prediction , 29.55 Entropy decoding, 11.65 In-loop filters, 15.75 Intra prediction , 1.08 Rest, 6.18 (b) 1.5x (16.5 seconds) Transfor m, 4 Motion compensat ion, 45.55 Inter-layer prediction , 7.95 Entropy decoding, 11.05 In-loop filters, 22.2 Intra prediction , 1.45 Rest, 7.8 (c) SNR (16.3 seconds) Transfor m, 4.98 Motion compensat ion, 45.38 , 0 In-loop filters, 22.58 Entropy decoding, 15.93 Intra prediction , 3.45 Rest, 7.7 (d) HEVC (8.5 seconds)

28. Results and analysis (4/5) Parallelism performance in the SHVC decoder: Con

29. gurations Decoding con

30. gurations (1, 1) (4, 1) (1, 4) (2, 2) Speedup 2 1 2.99 2.02 3.24 1:5 1 3.13 2.21 3.36 SNR 1 2.90 2.50 3.34 HEVC 1 3.10 2.64 3.05 Decoding 2 52 156 107 169 Class B (1080p) frame rate 1:5 44 138 99 149 (fps) SNR 41 124 108 138 HEVC 72 226 195 224 Decoding 2 20.54 6.88 24.04 16.08 time per 1:5 24.11 7.76 27.82 19.27 frame (ms) SNR 29.47 10.28 33.36 24.81 HEVC 16.07 5.20 24.05 10.66

31. Results and analysis (5/5) Parallelism performance in the SHVC decoder: Con

32. gurations Decoding con

33. gurations (1, 1) (4, 1) (1, 4) (2, 2) Speedup 2 1 3.6 2.07 3.81 SNR 1 3.49 2.47 3.67 Class A (1600p) HEVC 1 3.49 2.83 3.22 Decoding 2 23 84 47 88 frame rate SNR 17 62 43 65 (fps) HEVC 33 120 96 110 Decoding 2 47.85 13.65 55 32.65 time per SNR 63.92 18.76 73.67 50.95 frame (ms) HEVC 34.56 10.25 48.65 21.90

35. Conclusion and Perspectives (1/2) Conclusion: I First real time and parallel software (open source) implementation of the SHVC decoder I The SHVC decoder decoding two layers introduces 43% and 77% additional complexity in 2x and 1.5x spatial scalability con

36. gurations I Low level optimizations and parallelism are required to reach a real time decoding of 4Kp60 enhancement layer

37. Conclusion and Perspectives (2/2) Perspectives: I Support the decoding of more than two layers I Support the decoding of the base layer in legacy AVC standard I Uni

38. ed software for all HEVC extensions: Design a software decoder that support the decoding of all HEVC extensions including MV-HEVC, RExt, screen content coding, and 3D-HEVC extensions

39. Thank you for your attention

Real time SHVC decoder

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (8)

Similar a Real time SHVC decoder

Similar a Real time SHVC decoder (20)

Último

Último (20)

Real time SHVC decoder