APAN 2014 Bandung HDWG Session presentation Video Encoding for Web and Archive

1. Video encoding for
Web and Archive
High Definition Video Working Group
APAN Bandung 2014
Andrew Howard - The Australian National University

2. Background
•
Significant Digital Humanities media asset library
stored on film, video, audio tapes, portable hard
drives, CD, DVD and Blu-Ray
•
Some assets reaching end of life requiring ongoing
preservation activity
•
Also need to handle media stored on online services
like youtube and vimeo
•
Build on experience with encoding for Digital Lecture
Delivery system and iTunesU

5. Problem Space
•
Media degradation
•
Maintain quality, original format, encapsulation of propriety
playback system/application/OS environment using emulator and/
or virtual machine
•
Encode to an industry standard archive format at high bitrate to
support re-encoding using evolving compression standards
•
Deliver to a range of target playback systems
•
Organise, Identify and Describe assets and the content of assets
•
Cost of ingest, conversion, classification, storage and delivery

6. Some actual collections
Seismology data !
(586 DAT tapes)

7. Media Degradation
•
Digital Media
•
DVD
•
•
Dye process
•
•
Physical damage
Storage
Hard drives
•
Physical damage
•
Magnetic coherence

8. Media Degradation
•
Tape media (Video, Reel to Reel, DAT, Magtape)
•
Storage
•
Physical
•
Replay devices

9. Content Management
•
Organisation, Classification and Description of
assets and the content of the assets
•
E-Culture WG Session on Linked Data

10. Media preparation
Video
•
Video tape
•
Format: PAL,NTSC,SECAM,HDV
•
Aspect ratio: 4:3, 16:9, 3:2, 8:5, Anamorphic
•
Frame type: Interlaced or Progressive
•
Pixel format: Rectangular or Square

11. Media preparation
Video
•
Video tape - General Information
•
Clean the VCR heads regularly
•
Use a video enhancer hardware device like the Canopus to provide additional
signal stabilisation, chroma correction and retiming
•
Adjust VCR tracking
•
Use highest available device resolution for capture
•
Use highest available device connect for capture
•
DV
•
S-Video
•
Composite

12. Media preparation
DVD
•
DVD
•
Format: PAL,NTSC,SECAM,HDV
•
Aspect ratio: 4:3, 16:9, Anamorphic
•
Frame type: Interlaced or Progressive
•
Region coding and DRM

13. DVD Encoding Tools
•
Older tools:
•
(Windows)
•
DVD Decrypter
•
DVD Shrink

14. DVD ingest
•
Experienced problems on both commercial and
user created DVD media from both controlled
and uncontrolled environments
•
Best results using a Blu-Ray drive to read media
which standard DVD drives failed to read

15. Encoding Tools
•
Contemporary tools (OSX & Windows):
•
Handbrake
•
•
•
DVD decoding
DVD and file Encoding into many formats
VLC
•
The “Swiss Army Knife” for media

16. Cataloging, Tagging and
Identification
•
XMP:Description
•
MP3 tags
•
iTunes tags
•
Tools
•
exiftool
•
•
•
read and write asset metadata
mkvinfo
Face and Object recognition with CoreImage and OpenCV

17. Command line
tools
•
ffmpeg
•
VLC
•
vpxenc
•
MKVToolNix

18. ffmpeg recipes
Generate a JPEG poster frame from the video at #SECONDS from start
(15-20) is typical.
!
ffmpeg -i {INPUT}
-y -f mjpeg -vf scale="320:trunc(ow/a/2)*2"
-vframes 1 -ss {#SECONDS}
{OUTPUT}
!

19. ffmpeg recipes
Theora Video @1.2M, Vorbis Audio @128k
!
ffmpeg -i {INPUT}
-y -codec:v libtheora -b:v 1200k -qscale:v 6
-codec:a libvorbis -qscale:a 5 -b:a 128k -ar 22050
{OUTPUT}

20. ffmpeg recipes
H.264 Video @10Mbs, AAC Audio @384k, Lossless, width: preserve
!
ffmpeg -i {INPUT}
-metadata media_type=10 -metadata hd_video=0
-threads 0
-acodec libfaac -ac:a 2 -b:a 384000
-vcodec libx264 -pix_fmt yuv420p -b:v 10240k
-preset veryslow -tune film
-qp 0 -movflags +faststart {OUTPUT}

21. ffmpeg recipes
H.264 Video @1.2Mbs, AAC Audio @128k, scaled to height: 320, width:
matching input ratio
ffmpeg -i {INPUT}
-metadata media_type=10 -metadata hd_video=0
-threads 0
-acodec libfaac -ac:a 2 -b:a 128000
-vcodec libx264 -pix_fmt yuv420p -b:v 1200k
-vf scale="320:trunc(ow/a/2)*2"
-profile:v main -preset medium
-crf 18 -level 3.1 -movflags +faststart {OUTPUT}

22. ffmpeg recipes
WebM (VP8) Video @1.2Mbs, Vorbis Audio @128k Required for
!
multi threading
ffmpeg -i {INPUT} -y -threads 8
-threads 0
-codec:v libvpx -qscale:v 6 -b:v 1.2M
doesn’t work
-codec:a libvorbis -crf 10
-qscale:a 5 -b:a 128k -ar 22050
!
!

23. Playback
•
Target: Web browsers & mobile devices using
HTML5 <VIDEO> and <AUDIO>
Firefox
IE
x
WebM
x
Theora/Vorbis
x
x
FLV
x
x
MP3
x
x
YouTube
x
Safari/Webkit
x
H.264/AAC
Chrome
x
x
html5

24. Web playback HTML5
and flash fallback for H.264
•
Examined a range of open source players
•
•
Projekktor, osmplayer, JWplayer and
MediaElement
Selected MediaElement for quality of API,
documentation, support of SRT subtitles and
plugin support
•
mediaelementjs.com

25. Archive
•
Maintain original format to provide for re-code at
later time
•
Generate a high quality 2-pass H.264 version
•
Generate a high quality DVD version

26. Future Codecs
•
Increased range of macro block forms
•
Larger inter frame comparison
•
Decreased file sizes allow better bandwidth utilisation for
existing assets and the delivery of higher definition and
clarity operating on existing transmission systems
•
H.265/HEVC
•
VP9
•
Jan 2014 code release

27. VP9
•
Google next generation codec
•
libvpx code available
•
Latest VLC and Chrome will play
•
YouTube is a significant market driver

28. VP9
•
Google next generation codec
•
Original video size: 108,887,661 (108.9Mb)
•
•
x264 encode fps:
VP8
•
Single pass ffmpeg encode size: 122,716,927 (122.7Mb) includes Audio
•
vpxenc 2pass size: 24,488,846 bytes (24,5Mb) Video only,
•
encode fps:
•
•
•
Pass 1/2 frame 3857/3858 555552B
1152b/f 27327b/s 131187 ms (29.40 fps)
Pass 2/2 frame 3857/3857 24452079B 50717b/f 1202781b/s 118922 ms (32.43 fps)
VP9
•
Single pass encode vpxenc --codec=vp9 -t 7 -o APAN_demo_nasa.vp9.webm -w 1280 -h 720 --cpuused=4 -p 1 --target-bitrate=1200 —kf-max-dist=360 APAN_demo_nasa.vp8_1.y4m
•
Pass 1/1 frame 3857/3857 24813041B 51465b/f 1220537b/s 998347 ms (3.86 fps)
•
vpxenc 2 pass encode fps:

29. File size comparison
preliminary testing results
Encode
FPS
Video
Sample
MOV
Mb
AVI
Mb
Original
108,887,661
108
591,552,512
591.5
vp8
39,153,628
39.1
444,498,772
444.4
~30
vp9
31,741.303
31.7
216,773,028
216.7
~3-4
x264
97,624,542
97.6
379,207,254
379.2
~450

30. VP8 and VP9 tools
•
vpxdec
•
•
•
Extract a yuv4 uncompressed video
vpxdec --progress --postproc --mfqe -t 7 -o APAN_demo_nasa.vp8.y4m APAN_demo_nasa.vp8.webm
mkvextract tracks
•
•
mkvextract tracks "APAN_demo_nasa.vp8.webm" 1:APAN_demo_nasa.vp8.ogg
mkvmerge

31. HEVC/H.265
•
svn checkout https://hevc.hhi.fraunhofer.de/svn/
svn_HEVCSoftware/tags/HM-1.0/ HM-1.0
•
Still testing encoding

32. HM software: Encoder Version [1.0][Mac OS X][GCC 4.2.1][64 bit]
!
Input
File
Bitstream
File
Reconstruction File
Real
Format
Internal Format
Frame index
Number of Ref. frames (P)
Number of Ref. frames (B_L0)
Number of Ref. frames (B_L1)
Number of Reference frames
CU size / depth
RQT trans. size (min / max)
Max RQT depth inter
Max RQT depth intra
Motion search range
Intra period
QP
GOP size
Rate GOP size
Bit increment
Luma interpolation
Chroma interpolation
Entropy coder
!
!
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
APAN_demo_nasa.vp8_1.y4m
APAN_demo_nasa.vp8_1.bin
APAN_demo_nasa.vp8_1_enc.yuv
1280x720 30Hz
1280x720 30Hz
0 - 8 (9 frames)
1
1
1
1
128 / 5
4 / 32
2
1
128
32
32.00
8
8
4
Samsung 12-tap filter
Bi-linear filter
CABAC
TOOL CFG: ALF:1 IBD:1 HAD:1 SRD:1 RDQ:1 SQP:0 ASR:0 PAD:0 LDC:0 NRF:1 BQP:0 GPB:0 FEN:0 RQT:1 MRG:1
POC
POC
POC
POC
POC
POC
POC
POC
POC
!
0
8
4
2
6
1
3
5
7
(
(
(
(
(
(
(
(
(
I-SLICE,
P-SLICE,
B-SLICE,
B-SLICE,
B-SLICE,
B-SLICE,
B-SLICE,
B-SLICE,
B-SLICE,
QP
QP
QP
QP
QP
QP
QP
QP
QP
32
33
34
35
35
36
36
36
36
)
)
)
)
)
)
)
)
)
928
123656
17248
6576
10208
984
2376
1864
5856
bits
bits
bits
bits
bits
bits
bits
bits
bits
[Y
[Y
[Y
[Y
[Y
[Y
[Y
[Y
[Y
68.0431
39.3042
40.1223
44.9149
37.7147
63.3660
40.5029
37.1593
35.2758
dB
dB
dB
dB
dB
dB
dB
dB
dB
U
U
U
U
U
U
U
U
U
71.3615
43.1461
44.9601
48.0378
42.0799
50.5459
45.3647
42.6453
40.2356
SUMMARY --------------------------------------------------------
Total Frames | Bitrate
Y-PSNR
U-PSNR
V-PSNR
9
a
565.6533
45.1559
47.5974
53.2634
!
!
I Slices--------------------------------------------------------
Total Frames | Bitrate
Y-PSNR
U-PSNR
V-PSNR
1
i
27.8400
68.0431
71.3615
99.9900
!
!
P Slices--------------------------------------------------------
Total Frames | Bitrate
Y-PSNR
U-PSNR
V-PSNR
1
p
3709.6800
39.3042
43.1461
45.9746
!
!
B Slices--------------------------------------------------------
Total Frames | Bitrate
Y-PSNR
U-PSNR
V-PSNR
7
b
193.3371
42.7223
44.8385
47.6294
!
Total Time:
2558.456 sec.
dB
dB
dB
dB
dB
dB
dB
dB
dB
V
V
V
V
V
V
V
V
V
99.9900
45.9746
47.3997
51.2781
45.0517
51.4867
48.8884
45.4117
43.8896
dB]
dB]
dB]
dB]
dB]
dB]
dB]
dB]
dB]
[ET
[ET
[ET
[ET
[ET
[ET
[ET
[ET
[ET
32
77
412
351
479
195
398
322
291
]
]
]
]
]
]
]
]
]
[L0
[L0
[L0
[L0
[L0
[L0
[L0
[L0
[L0
]
0
0
0
4
0
2
4
6
[L1 ]
] [L1
] [L1
] [L1
] [L1
] [L1
] [L1
] [L1
] [L1
]
8
4
8
2
4
6
8
]
]
]
]
]
]
]

33. Next Generation Codecs
•
Trade increased encoding time and cpu for
decreased bandwidth
•
Promise of significant gains in compression
•
Reference code and specifications now available
•
Still tuning for performance
•
Google VP9 developer videos on YouTube

34. VP9 test sequence
•
Convert input video to vp8
•
Encode to vp9
!
•
2 pass
vpxenc --codec=vp9 -t 7 -o APAN_demo_nasa.vp9_2_pass_clang.webm -w 1290 -h 720
bitrate=1200 --kf-max-dist=360 APAN_demo_nasa.vp8_1.y4m
--cpu-used=4 -p 2 --target-

35. Summary
•
Media preparation
•
ffmpeg recipes for media encoding for Web and
Archive
•
Next Generation Codecs