Se ha denunciado esta presentación.
Se está descargando tu SlideShare. ×

Transport methods in 3DTV--A Survey

Próximo SlideShare
Cargando en…3

Eche un vistazo a continuación

1 de 35 Anuncio

Más Contenido Relacionado

Presentaciones para usted (20)

A los espectadores también les gustó (20)


Similares a Transport methods in 3DTV--A Survey (20)

Más reciente (20)


Transport methods in 3DTV--A Survey

  1. 1. Transport Methods in 3DTV—A Survey Tang kai April, 24th, 2011 1
  2. 2. Index • Introduction • 3DTV Broadcast • 3DTV Over IP Networks • Discussion and Conclusion 2
  3. 3. Introduction • Ultimate goal • dynamic holography • Most systems available today • via stereoscopy • Actually, 3DTV systems can be designed to support • fixed-view stereoscopy: only two views • free-view stereoscopy: multiple views 3
  4. 4. Introduction • History of 3-D movie • 1903, first stereoscopic 3-D movie was created • 1922, the first full length stereoscopic movie was shown • in the 1950s, Hollywood started 3-D movie production in big numbers • Consensus: a lasting success • backwards compatible • supports different numbers of users • with affordable 3-D display technologies • requires low additional transport/transmission overhead • perceived quality and viewing comfort is better 4
  5. 5. 3DTV Broadcast • Analog Transmission • US • April 29th, 1953: a trial live broadcast of the series SPACE PATROL was run in Los Angeles • viewers with a pair of special polarization lenses • December 19th, 1980: The first “nonexperimental” 3DTV “Miss Sadie Thompson,” and Three Stooges • 3-D Video Corporation developed a system: anaglyph format • April 10th, 1981: musical classic, “Kiss Me Kate.” • 3-D Video Corporation: perfect in color 5
  6. 6. 3DTV Broadcast • Analog Transmission • European • 1982: Netherlands two popular-scientific 3-D series • a simple red/green anaglyph format • H.-J. Herbst (Hamburg, Germany) and Philips Research Lab • More than 40 million red/green viewing spectacles were sold • “the TV of the future” was disillusioned • 1983 at the International Audio and Video Fair in Berlin • based on a standard PAL channel chain in two-channel mode • For display, two projectors with orthogonal polarization filters were used • so successful that were continued at IAVF in 1985 and 1987 • Unfortunately, transmission system required custom receiver. 6 LIMITED
  7. 7. 3DTV Broadcast • Digital Transmission • Background: ongoing transition from analogue to digital TV services • MPEG developed a new compression technology as part of MPEG-2 • The MPEG-2 multiview profile (MVP) • Left-eye view --- MPEG-2 main profile --- backwards- compatibility • Right-eye enhancement layer using the scalable coding tools • MVP, unfortunately, has not found use in commercially services 7
  8. 8. 3DTV Broadcast • Digital Transmission • Some promising attempts • 1998 Nagano Winter Games in Japan • right-eye and left-eye HDTV images @ 45Mbps • projected onto a large screen. Impressing and Powerful • 2002 FIFA World Cup in Korea/Japan • the right-eye and left-eye HDTV images were compressed in side-by-side format using the MPEG-2 Main Profile 8
  9. 9. 3DTV Broadcast • Digital Transmission • Fixed-view -> flexible 3-D visual data representation formats • Australian DDD company : “video-plus-depth” representation • combination of monoscopic color video and associated per- pixel depth maps • encodes the depth data low bit rate format • transmitted in the “user data” of an MPEG-2 Transport Stream • receiver : rendered by using depth-image-based rendering (DIBR) 9
  10. 10. 3DTV Broadcast • Digital Transmission • European IST project ATTEST • “video-plus-depth” representation • Standard MPEG technologies: H.264/AVC • depth data: 200–300 kbps • overhead for the 3-D visual information is only 10% CMP 2-D 10
  11. 11. 3DTV Broadcast • Digital Transmission • European IST project ATTEST • First demonstration based on ATTEST • Diagram as follows 1st demo of a 3DTV service 3-D programs,the “video-plus-depth” 3-D TS Contained two on each contains basedvideo stream • an MPEG-2 coded color data representation formatcoded depth-image DVB-T transmission. • an H.264/AVC using a real sequence. DTV-Recorder-Generator PC with a PCI DVB-T card 11 real-time replay of an offline- Received MPEG-2 TS was demultiplexed in software generated MPEG-2 TS video bit streams were decoded in real-time
  12. 12. 3DTV Broadcast • Digital Transmission • “video-plus-depth” representation has been standardized within MPEG as a result of work initiated by Philips and Fraunhofer HHI. • The new standard has been published in two parts: • Specification of the depth format itself is called ISO/IEC 23002-3 (MPEG-C) • a method for transmitting “video-plus-depth” within a conventional MPEG-2 TS has become an amendment (Amd. 2) to ISO/IEC 13818-1 (MPEG-2 Systems). 12
  13. 13. 3DTV Over IP Networks • Background • IP is proving to be flexible in accommodating communication services • Classical telephone -> VOIP • Transmission of video over Internet is active in R & D • VoD • 2.5G and 3G offer wireless video service • The IP itself leaves many aspects of the transmission to be defined by other layers of the protocol stack and, • thus, offers flexibility in designing the optimal communications system for various 3-D data 13 representations and encoding schemes.
  14. 14. 3DTV Over IP Networks • General Outline • 3DTV streaming architectures • Server Unicasting • Server Multicasting • P2P Unicasting • P2P Multicasting • Protocol • Current state of the art: RTP/UDP/IP 14 • Next generation: RTP/DCCP/IP
  15. 15. 3DTV Over IP Networks • Streaming Protocols • Most widely used : RTP over UDP • does not contain any congestion control mechanism • lead to congestion collapse when large volumes of video are delivered • Datagram congestion control protocol (DCCP) is designed as a replacement for UDP for media delivery • TCP minus reliability and in-order packet delivery • UDP plus congestion control, connection setup, and acknowledgements 15
  16. 16. 3DTV Over IP Networks • Streaming Protocols • DCCP is a transport protocol that implements bi-directional unicast connections of congestion-controlled, unreliable datagrams. • Despite of the unreliable datagram flow • Reliable handshakes for connection setup/teardown and reliable negotiation of options 16
  17. 17. 3DTV Over IP Networks • Streaming Protocols • DCCP also accommodates two congestion control mechanisms. • TCP-like Congestion Control • TCP-Friendly Rate Control(TFRC) • TCP-like Congestion Control • identified by CongestionCCID2 in DCCP • behaves similar to TCP’s AIMD congestion control • halving the congestion window in response to a packet drop • respond quickly to changes in available bandwidth 17 • must tolerate the abrupt changes in the congestion window size
  18. 18. 3DTV Over IP Networks • Streaming Protocols • TCP-Friendly Rate Control(TFRC) • identified by CCID3 • a form of equation-based flow control that minimizes abrupt changes in the sending rate while maintaining longer-term fairness with TCP • Appropriate for applications that would prefer a rather smooth sending-rate with a small or moderate receiver buffer • streaming media applications 18
  19. 19. 3DTV Over IP Networks • Streaming Protocols • TCP-Friendly Rate Control(TFRC) • Operation: CCID3/TFRC calculates TFRC rate • using the TCP throughput equation • Request gives feedback to sender application • Sender may use this rate information to adjust rate to get better results 19
  20. 20. 3DTV Over IP Networks • Streaming Protocols • (exp)RFC for TCP-Friendly Multicast Congestion Control (TFMC) • compute the TFRC rate in a multicast scenario • each receiver computes own TFRC rate as a function of RTT loss rate • server then selects the minimum of these rates • (limited number clients to prevent feedback explosion) • DCCP is the same way doing this. • Hence, future 3DTV over IP services is expected to employ the DCCP protocol with effective video rate adaptation to 20 match the TFRC rate.
  21. 21. 3DTV Over IP Networks • Multiview Video Encoding and Rate Allocation/Adaptation • Multiview 3-D video can be represented and encoded • Implicitly: “video-plus-depth” representation (discussed) • Explicitly: in raw form • a trade-off between • random access • ease of rate adaptation • compression efficiency • simulcast coding • scalable simulcast coding • multiview coding • scalable multiview coding 21
  22. 22. 3DTV Over IP Networks • Multiview Video Encoding and Rate Allocation/Adaptation • The rate adaptation differs, since rate allocation between views offers new flexibilities. • According to the suppression theory of human visual perception • if the right and left views are transmitted and displayed with unequal spatial, temporal and/or quality resolutions, the overall 3- D video quality is determined by the view with the better resolution • Therefore, rate adaptation may be achieved by • adaptation of the spatial, temporal and/or signal-to-noise (SNR) resolution of one of the views • while encoding/transmitting the other view at full rate. 22
  23. 23. 3DTV Over IP Networks • Multiview Video Encoding and Rate Allocation/Adaptation • Several open loop and closed loop rate adaptation strategies • closed loop strategies • client estimates some function of the received signal and feeds it back to the transmitter • The transmitter determines an optimized rate • open loop strategies • transmitter does not use feedback from the receiver 23
  24. 24. 3DTV Over IP Networks • Multiview Video Encoding and Rate Allocation/Adaptation • open-loop rate adaptation strategies • First paper: content-adaptive video scaling • Rate adaptation has been achieved by • 1) spatial subsampling; • 2) temporal subsampling; • 3) scaling the quantization step-size; • 4) content-adaptive scaling • content-adaptive video scaling approach • Four categories: high/low temporal spatial detail. • Scaling their resolutions 24 • Experiments show that better compression with better quality
  25. 25. 3DTV Over IP Networks • Multiview Video Encoding and Rate Allocation/Adaptation • open-loop rate adaptation strategies • Second paper: adaptive selection of temporal levels and quality layers • video is encoded offline with a predetermined number of spatial, temporal and SNR scalability layers. • Content-aware bit allocation among the views is performed during bit stream extraction by adaptive selection scalability layers • The required bit rate reduction is only applied to one of the views. • Experiments shows that better rate-distortion performance compared to static cases. 25
  26. 26. 3DTV Over IP Networks • Multiview Video Encoding and Rate Allocation/Adaptation • closed-loop rate adaptation strategies • rate adaptation is done at the server side by feedback from the user. • First paper: • The user’s head position is tracked and predicted • The system allocates more bandwidth to the selected views in order to render the current viewing angle. • Make use of MVC and SVC • Second paper: • Each view is streamed to a different IP-multicast address • A viewer’s client joins appropriate multicast groups to only receive the 3-D information relevant to its current viewpoint 26
  27. 27. 3DTV Over IP Networks • Error Correction and Concealment • Sources: packet losses in the wired or wireless IP links • Wired Internet: Congestion -> packet losses • Wireless Internet: capacity limited by bandwidth of radio spectrum • Noise, interference and fading, error bursts(from mobility) • Joint source and channel coding techniques • Error concealment methods (at the decoder) to limit temporal error propagation 27
  28. 28. 3DTV Over IP Networks • Error Correction and Concealment • Common error correction approaches for reliable transmission • ARQ • ACK • Delay, not desirable • FEC • In broadcast and multicast services, channel coding techniques have been widely applied 28
  29. 29. 3DTV Over IP Networks • Error Correction and Concealment • First paper: • Macroblock classification into unequally important slice groups • Using FMO tool of H.264/AVC • LT codes are used for error protection for low complexity and advanced performace 29
  30. 30. 3DTV Over IP Networks • Error Correction and Concealment • Second paper: • Stereoscopic video streaming using FEC techniques • Frames are classified according to their contribution to overall quality • three layers used for UEP • I-frame • P-frame • Left • Right • To find optimum packetization and UEP strategies • Comparative analysis and simulation of Reed–Solomon (RS) and 30 systematic Luby transform (LT) codes
  31. 31. 3DTV Over IP Networks • Error Correction and Concealment • Error concealment algorithm for monoscopic not applicable for stereoscopic. • Based on interpolation -> is not sufficient for not depth info is preserved. • Human perception of errors in 3-D video is different • A small degradation -> significant perceptual distortion • Third paper: an error concealment algorithm • Make full use of characteristic of stereoscopic video • Based on the relativity of prediction mode of right frames -> prediction mode of macroblock • restore the lost macroblock according to the estimated motion 31 vector or disparity vector.
  32. 32. 3DTV Over IP Networks • Error Correction and Concealment • capabilities of error concealment • To increase the quality of the reconstructed block • a stereoscopic movie: the two views are highly correlated(why) • information about the corresponding region is highly useful for the reconstruction of the lost block. • corresponding pixel pairs identified using feature matching and principles of epipolar geometry • robust estimation of the transformation parameters is used to 32 educe the negative effect of outliers
  33. 33. 3DTV Over IP Networks • 3D Video Streaming Demonstrations • end-to-end prototype system for point-to-point streaming of stereoscopic video over UDP supports the 1.over a LAN autostereoscopic Sharp 3-D laptop with no packet losses supports a monocular display to demonstrate 2.employs the backwards protocol stack compatibility RTP/UDP/IP supports an in-house polarized 3-D projection display system that uses a pair projector 33
  34. 34. Discussion and conclusion • A comprehensive survey of the state-of-the art in transport techniques • While the transport solutions must address backwards compatibility issues with the existing digital TV standards and infrastructure • 3DTV flexible • Current and future research issues for 3-D TV transmission • joint transport and coding • Why • determination of the best rate adaptation method • error resilient video encoding and streaming strategies 34
  35. 35. THE END 35