review of H.264 NAL and packetization in RTP

1. H.264 NAL & RTP
Fast Forward Your Development www.dsp-ip.com

2. H264 Layer Structure
Video Coding Layer
Control Data
Macroblock
Data Partitioning
Slice/Partition
Network Abstraction Layer
H.320 H.324 H.323/IP MPEG2
Fast Forward Your Development www.dsp-ip.com

3. H264 & NAL
• Motivation
▫ Many delivery methods are based on packet
based networks
▫ Its better to do the packetization inside the
encoder where all coding information than in
other separate modules
• Architecture: NAL units as the transport entity
▫ NAL units may be mapped into a bit stream
▫ NAL units are self-contained-independently
decodable
▫ The decoding process assumes NAL units are in
decoding order
Fast Forward Your Development www.dsp-ip.com

4. Network Abstraction Layer (NAL)
• H.264 encoder is composed of two layers:
• VCL - Video Coding Layer – unit which translates the
video information into a stream of bits
• NAL - Network Abstraction Layer (NAL). Which maps
and packetize the VCL biststream into unitsprior to
transmission or storage
• Each NAL unit contains:
▫ Payload – RBSP (Raw Byte Sequence Payload), which
contains set of data corresponding to coded video data or
header information
▫ NAL Unit Header – which contains the NAL header
Fast Forward Your Development www.dsp-ip.com

5. NAL
• The coded video sequence is represented by a
sequence of NAL units that can be transmitted
over a packet-based network or a bitstream
transmission link or stored in a file
• There are two NAL Types
▫ VCL Units – NAL units which represents encoded
video data
▫ Non-VCL Units – Parameters sets
Fast Forward Your Development www.dsp-ip.com

6. NAL Unit Header
NAL unit header NAL unit payload
NAL unit header- 1 byte consisting of:
• Forbidden_bit(1 bit) may be used to signal that a NAL unit is
corrupt
• nal_storage_idc(2 bit): signal relative importance, and if the
picture is stored in the reference picture buffer.
• nal_unit_type(5 bit): signal 1 of 10 different NAL unit types:
▫ Coded slice (regular VCL data)
▫ Coded data partition. (DPA,DPB,DPC)
▫ Instantaneous decoder refresh (IDR)
▫ Supplemental enhancement information (SEI)
▫ Sequence and picture parameter sets (SPS,PPS)
▫ Picture delimiter (PD) and filler data (FD)
Fast Forward Your Development www.dsp-ip.com

7. RBSP (NAL Payload) Types
• Parameter Set – global parameters for a sequence
includes: Resolution, video format, macroblock,
allocation map
• Supplemental Enhancement Information
• Picture Delimiter – boundary between video pictures
• Coded slice – header and data for a slice, this unit
contains actual coded video data
• Data Partition A,B or C – Data Partitioned slice layer
data (A – header data for all MBs in the slice, B – intra
coded data, C – inter coded data)
• End of sequence
• End of stream
• Filler data
Fast Forward Your Development www.dsp-ip.com

8. RTP payload format for H.264
• Based on IETF RFC 3984, February 2005
• Described how to use H.264 NAL inside RTP with proper
packetization Employs the native NAL (Network
Abstraction Layer) interface, based on NAL units (NALUs)
• NALU – byte string of variable length that contains syntax
elements of a certain class
• NALU header – defines the information within the NAL
Unit (Corrupted, Type etc)
• There are two basic methods for RTP packetization of NAL
units:
▫ Non fragmented NAL units
▫ Fragmented NAL units
Fast Forward Your Development www.dsp-ip.com

9. RTP Payload for H.264 NAL
• The most common method is to configure the
encoder to output one NAL unit for each RTP
packet. Each NAL unit is ~1.4KB
• Fragment a large NAL unit (Frame) into many
RTP Packets. The difference is in the RTP
Header information
Fast Forward Your Development www.dsp-ip.com

10. RTP and H.264
• RTP Packetization of NAL allows both
aggregation of many NAL units into one RTP
Packet and fragmentation of one NAL units into
many RTP packets
Fast Forward Your Development www.dsp-ip.com

11. DSP-IP Contact information
Download slides at: www.dsp-ip.com
www.dsp-
Course materials & lecture request
Adi Yakov
Training Manager
adi@dsp-
adi@dsp-ip.com
+972-9-8651933
972-
Projects development services:
Alona Ashkenazi
Development Services www.dsp-ip.
www.dsp-ip.com
alona@dsp-
alona@dsp-ip.com Mail : info@dsp-ip.com
info@dsp-ip.
+972-9-8850956
972- Phone: 972- 8850956,
Phone: +972-9-8850956,
Fax : +972-50- 8962910
972-50-
Fast Forward Your Development www.dsp-ip.com