Transcoding 101 (Tutorial) - PBS

Transcoding 101

Introductions
• Who is Rhozet
– Spun out from Canopus in 2004
– Maker of ProCoder and Carbon Coder
– Provides transcoding for Yahoo!, Amazon, Microsoft, Hulu, CBS,
NBC, Turner, BBC, Fox, Discovery, Lifetime, etc.
– Acquired by Harmonic in 2007
• Who is Harmonic
– Leading equipment provider to cable, satellite, and IPTV networks
– Publicly traded (NASDAQ:HLIT)
– 690 people, $365M in revenue

This Transcoding Thing…
• The process of converting one format to another
– Facilitates moving media across production, post-production,
archival, and delivery ecosystems
– Acts as the “glue” between different manufacturers
– Provides future proofing
• Allows repurposing and monetization of assets
– Every destination viewer has different requirements
– Allows for the automated creation of custom assets
(commercials, promos, logos, etc.)
• Enables advanced workflows
– Sending a file to 10 people who all see it differently depending
on their needs (preview, timecode, language…)
– Database integration
• An engine that can be integrated into various applications
and devices

• Codecs
• Profiles
• Containers
• Formats
• Platforms
Transcoding Terminology

Codecs
• Codec = Compressor/Decompressor
– The software or hardware engine that moves uncompressed frames
into the compressed domain (and vice versa)
• Typically “lossy”
– Reduction in information at each encode
• Typically asymetrical
– Decompression is often 10x (or more) faster than compression
• Techniques
– Subsampling
• 4:2:0 vs. 4:2:2 or 4:4:4
• 8 bit vs. 10 bit color resolution
– Transformation and simplification
• Discard high-frequency changes in color
• DCT (MPEG-2)
• Wavelet (JPEG-2000)
• Intra-frame = within a single frame
– Motion analysis and estimation
• Most video frames are similar to the ones around them
• Inter-frame = between multiple frames

Codecs (cont.)
• GOP – Group of Pictures
– Set of frames (max 15 for MPEG-2)
– All P and B references to frames are within the GOP
• I: intra-frame
– Only data within the frame is used
• P: predicted frame
– Variation from previous I or P frame
– ~twice the compression of I frame
• B: bi-directionally predicted frame
– Variation from previous or future frames
– ~twice the compression of P frame
I P B B P B B P B B

Codecs (cont.)
• Codecs using intra-frame compression
– DV, MPEG-2 (IMX), AVC-Intra, JPEG-2000, DNxHD, etc.
– Typically acquisition and editing formats
• Codecs using inter-frame compression
– H.264, MPEG-2 LongGOP, WMV, VP6, etc.
– Typically distribution formats
• Standards like H.264 only specify how to decode the
content
– Allows for compatibility while leaving room for innovation in
compression techniques

Profiles (and Levels)
• A “Profile” defines a specific type of compression for a particular
codec
– Defines the syntax that is supported
– The decoder must match the encoder’s profile support
– A codec vendor does not have to support all possible profiles
– A “Level” defines maximum resolution and data rate
• H.264 Examples
– Baseline Profile (BP): limited computing power required for decode
– High Profile (HiP): primary profile for broadcast and BluRay
– High 4:2:2 Profile (Hi422P): 4:2:2 chroma
• MPEG-2 Examples
– Main Profile@Main Level (MP@ML): standard def at max 15Mb/s
– Main Profile@High Level (HP@HL): up to HD at 80Mb/s
– 4:4:4 Profile@High Level (422P@HL): supports 4:2:2 chroma

Containers
• AKA “wrappers”
• A container can contain multiple types of codecs
• A container can contain more than just video
– Animation, music, speech, text, subtitles, etc.
• A container is used to identify, interleave, and synchronize
the various components
– Critically important for successful playback
– Most of the idiosyncrasies of a particular device or distribution
medium are expressed in the container specifications
– Single biggest source of incompatibility is in containers rather
than codecs
• Example containers
– QuickTime, AVI, ASF, WMV, MXF, M2TS, M2PS, MP4, VOB,
LXF, GXF, WAV, 3GPP

Formats
• The combination of a container and a specified set of
codecs (essence) and metadata
– Example: M2TS with H.264 (HP) video and MPEG-1 Layer 2
audio
• In more detail includes parameters
– M2TS
– H.264 video
• 720x480, 29.97fps, upper field first
• CBR, 3 Mbps data rate,
• High profile, 3.2 Level, ATSC closed-captioning
• …and about 50 other parameters
– MPEG-1 Layer 2 audio
• Stereo, 16-bits per sample, 48Khz sample rate
• 128 Kbps data rate

Platforms
• The device on which a particular format will be played back,
archived, edited, etc.
• Formats can be platform dependent or independent
– MPEG-1 is platform independent
– Flash and WMV are platform dependent (Flash Media Player and
Windows Media Player respectively)
• Just to make things confusing, codecs, containers, formats,
and platforms can all be named similarly
– For example, MPEG-2 is both a codec and a container

The Transcoding Pipeline
Video
Decode
Video
Transform
Video
Encode
DeMultiplex
Multiplex
Audio
Decode
Audio
Transform
Audio
Encode
• Multipex = “wrapping” in the container
• Transform = scale, frame rate, crop, logos, concatenation, filtering, etc.
• Different transcoders can yield very different results even if they use the same codecs

Transformation In The Transcode
Basic Video Operations
– Frame size conversion
– Frame rate conversion
– Color space conversion
– Aspect ratio conversion
– Interlace/De-interlace conversion
– Telecine / inverse telecine
– PAL/NTSC conversion
– SD/HD conversion
– Cropping
Video Processing
– Fade in/out
– Black/white correction
– Blur
– Color correction
– Gamma correction
– NTSC-safe
– Median
– Rotate
– Sharpen
– Temporal noise reduction
Audio Processing
– Normalize
– Fade In/Out
– Low-pass
– Volume
– Dynamic range compressor
Additional Operations
– Timecode imprint
– Subtitle/CC imprint
– XML controllable titler
– Metadata transport and conversion
– Line 21/CC preservation/conversion
– Quality checking
– Logo insertion
– 601/709 color space support
– Video capture board support
– Multiple simultaneous target outputs
– Edit decision list support
– Remote job submission
– Batch processing
– Watch folder automation
– Segment extraction/insertion
– FTP delivery

The Full Transcoding Workflow
Acquire
Transcode
Deliver
- Capture
- FTP
-MAM
- etc.
QA
- Decode
- Transform
- Encode
QA
- DRM
- FTP
- MAM
- etc.
• QA steps can be automated, manual, or skipped
• Transcoding can be controlled manually, with watch folders, or via API’s

Calculating Performance
• Stream-based systems
– Live transformation of video
– Realtime supporting one or more streams
– Typically dedicated hardware
• File-based systems
– Can be slower or faster than real-time
– Typically generic hardware
– Dependency on CPU and network storage performance
– Benchmarks are extremely dependent on target settings
• One pass versus two pass
• Size of GOP
• 70+ parameters in H.264…

Calculating Performance (cont.)
• Estimate number of hours of source content per day
• Perform test encodes from sample source file to approved target
type
– Test on appropriate machine
– Calculate in units of RT (real time)
– 10 minute source transcoding in 5 minutes = 2X RT
– 10 minute source transcoding in 20 minutes = .5X RT
– Check CPU utilization – can do more than one transcode at a time
• Example
– 16 hours per day of incoming source, SD MPEG-2
– Transcode to 4 outputs
• Transcode performance is 1.5RT, 2.2RT, .8RT, and .5RT for the targets
– 16 hrs/1.5 + 16 hrs/2.2 + 16 hrs/.8 + 16 hrs/.5 = 70 hrs transcoding/day
– Each machine does 24 hrs of transcoding/day
– 70/24 = 2.91
– 3 machines to do all the formats

The Great Thing About Standards…
H.264 MXF DPX Flash AAC M2TS
MPEG-2
DPS WMV
VOB
Dolby
DVCPro100
VC-1 MPEG-4 AVC-Intra DV50 M2PS 3GPP
JPEG-2000
DNxHD
3G2
ASF
F4V
OPAtom
DV25 DVCPro AVI HDV
MP4
GXF
OP1a QuickTime LXF WAV
MPEG-1
AC-3 Omneon WAV DivX
AVCHD

Why Can’t We Just Use One Format
• Specific purposes
– Acquisition/editing (highest quality, generational fidelity, direct
frame access)
– Distribution (bandwidth, acceptable quality)
• Hardware restrictions
– Set top boxes
– Cable bandwidth
– Mobile phone processing power
• Money
– Manufacturer “lock-in”
– Platform ownership
– Royalties
– Rhozet needs your business

What’s Next
• Reduction in bitrate for the same quality
• Will there be a codec twice as “good” as H.264
Ghanbari et al, University of Sussex

JPEG-2000 vs. I-frame H.264
• Mathematical “quality”
– PSNR – Peak to peak Signal to Noise Ratio
– Comparing compressed image with pristine source
– H.264 is slightly better at all bitrates
• Blurring
– H.264 better at all bitrates
• Blocking
– JPEG-2000 better at low bit rates since uses transform on
entire image rather than macro-blocks
• At high bit rates, these two formats are indistinguishable
– For archival purposes, consider them identical
– Principal deciding factor is wrappers and workflow

Place Your Bets…
• Acquisition
– H.264 (AVC-Intra)
• Television
– H.264 in M2TS
• Web
– H.264 in MP4/F4V
– WMV/VC-1 in ASF
• Mobile
– H.264 in MP4, 3GPP
• Archiving
– Whatever you acquired in
• Transcoding
– Even if everything is in H.264 you will still be transcoding

Beyond Transcoding
• Watermarking & Fingerprinting
• DRM
• Smooth Streaming
• Royalties
• ROI

Watermarking and Fingerprinting
• Watermarking
– Invisible information embedded into image data, typically
embedding data in color frequency information
– Can be used to track individual assets
– Philips/Teletrax (now Civolution), Thomson NexGuard, Dolby
Cinea, etc.
– Embedder, investigator, manager, database
– Watermarks can “step” on each other
• Fingerprinting
– No information embedded into file
– Audio and video are sampled to create “fingerprint” that can be
searched and matched against central database
– No tracking ability for individual versions of assets
– Civolution, Vobile, Audible Magic, YouTube, etc.

Digital Rights Management (DRM)
• Technology to restrict unauthorized use or distribution of
content
– Most computer-based systems use a combination of a
license server and public key encryption which ties specific
content to a specific machine or device
– Reduces but does not eliminate piracy
– All broadly deployed technologies have been beaten
• Popular Systems
– FairPlay (Apple, iTunes and iPod specific)
– Windows Media DRM (Microsoft, Windows or SilverLight)
– Flash DRM (Adobe, Flash specific)
– MagicGate (Sony, PSP and MemoryStick specific)

Emerging Streaming Technologies
• Adobe Dynamic Streaming
– Available in new Flash Media Server
– True streaming protocol RTMP (Adobe created)
– Switches between mulitple files encoded at different bitrates
• Microsoft Smooth Streaming
– Not really “streaming”, but rather smart download
– Uses HTTP rather than RTSP
– Encodes video at different bitrates in many small “chunks”
– Player requests “chunks” of different bitrates depending on
available connection speed
– Requires Silverlight player and Microsoft IIS server

H.264 Royalties
• H.264 patent pool administered by MPEG-LA
– Four categories: title-by-title, subscription, free TV, free internet
• Title-by-title (includes VOD and Disc)
– No royalty for 1M
• Free TV
– $2,500 one-time per AVC transmission encoder or…
– $2,500 annual per Broadcast Market of 100K to 500K, $5K for 500K to 1M,
$10K > 1M
• Free Internet
– No royalty before 2011
– After that, no more than for “economic equivalent” of free television
– Unclear exactly how that applies

• You have questions…
Q&A

Thank You!
• Demos, white papers, case studies, performance guides,
etc. are all available at www.rhozet .com