Dr. Panos Nasiopoulos EECE 541 - Courses

U N I V E R S I T Y O F B R I T I S H C O L U M B I A
Efficient Transmission of H.264 Video
over Wireless Networks
Dr. Panos Nasiopoulos
EECE 541
March 08 EECE 541 1
March 08
Video Transmission over Wireless Networks
Video Streams may incur significant packet loss, delay and
delay jitter in Wireless Environments.
2
QoS-
Internet
EECE 541
1

March 08
Wireless LAN Access Network Problems
What is different about wireless networks?
Delay, Jitter, Packet Loss and Data Corruption are
considerably higher in wireless networks than in wired
networks
transport service in wireless networks is not consistent or
predictable
WLAN problems may come from two sources
PHY: fading, noise and interference may corrupt packets.
MAC: collision in MAC layer may not be resolved on time, so
packets are dropped. Mobility may cause excessive delay.
Packet loss may occur in any layer
PHY drops the packet if a bit error is detected, or MAC collision
destroys the packet
MAC drops the packet if too many retransmissions happen, and a
fragment of the packet is lost.
RTP/UDP or IP drop the packet if a fragment of the packet is lost
Application drops the packet if it arrives too late
March 08
Why do we need new techniques?
Enhancements may be achieved in
several layers, either separately or in a
cross layer fashion
Application Layer ( H.264 Video )
Error Resiliency Schemes
Scalable Video Coding
MAC Layer
Scheduling to provide Differentiated and
Guaranteed Services
Packet size adjustments to reduce packet loss
PHY Layer
Link Adaptation
3
4
App. Layer:
H.264 Video
EECE 541
RTP/IP
MAC:
Scheduler
PHY:
Link
Adaptation
App. Layer:
H.264 Video
EECE 541
RTP/IP
MAC:
Scheduler
PHY:
Link
Adaptation
2

March 08
NALU – slice1
H.264, Video Slice Coding
When packet sizes exceed the maximum allowable amount for
reliable transmission, we need to slice the frames into smaller data
decodable unit –
the alternative is to let the network fragment the packet.
Slices
Frame Data
Self contained, all information for decoding slice available within slice.
Each slice encapsulated in a NAL (Network Abstraction Layer) Unit.
Size chosen to allow for optimal transport given underlying network MTU (max
transfer unit) size constraints, as well as wireless network BER and
acceptable PER.
Advantages: losing a slice only affects part of a picture, not an entire
frame
Disadvantages: More overhead, less compression efficiency
March 08
NALU – slice2 NALU – slice3 NALU–slice4
5
EECE 541
H.264 Error Resiliency - Flexible Macroblock
Ordering (FMO)
The H.264 provides various error resiliency schemes, including
“Flexible Macroblock Ordering” (FMO)
H.264 allows macroblocks within pictures to use a variety of slice
mapping patterns.These include:
Interleaving
Dispersed
Foreground
Box-out
Raster Scan
Wipe
Explicit
user defined
6
EECE 541
3

March 08
FMO – different configurations
Two typical scenarios of bad channel in Wireless LANs:
BER of 5*10E-4 with MAC layer fragmentation threshold of 340B.
BER of 10E-4 with MAC layer fragmentation threshold of 1000B.
Retransmission limit: MAC layer may retransmit packet 5 times
PSNR & Video Quality Metric (VQM) were averaged over the 120 frames, 10 runs
of the simulation.
Avg PSNR
40
35
30
25
20
15
10
5
0
March 08
Avg PSNR Vs. BER/Packet Size
BER 0.0005/ Frag
Thresh 340
BER/ Packet Size
BER 0.0001/ Frag
Thresh1000
Interleave
Dispersed
Foreground w ith left over
Box-out
Raster scan
Wipe
Explicit
Our Solutions?
7
Avg V QM
9
8
7
6
5
4
3
2
1
0
Avg VQM Vs. BER/Packet Size
BER 0.0005/ Frag
Thresh 340
BER/ Packet Size
Cross Layer Optimization: Use Video information and
optimize the operation of the MAC and PHY layers
Simpler solutions
Frame Size Adjustment: Find the (near) optimal frame size in
MAC and PHY, slice the video accordingly.
More complex cross layer optimized solutions
Intelligent Link Adaptation
BER 0.0001/ Frag
Thresh1000
Use Scalable Video Coding, adjust the configuration of the
multirate PHY, and the traffic control (shaping) module in MAC
and Network layers
Multi-class Fair Scheduling
Guaranteed service provisioning for more important parts of the
video stream
The objective of the algorithm is to provide long term fairness for
all sessions, while providing reduced delay for video traffic.
Specially useful for WiMAX.
8
Interleave
Dispersed
EECE 541
Foreground w ith left over
Box-out
Raster scan
Wipe
Explicit
App. Layer:
Optimal size
Video Slice
Coding
EECE 541
RTP/IP
MAC:
Multi-Class
Fair
Scheduling
PHY:
Intelligent
Link
Adaptation
4

March 08
H.264 Error Resiliency: Data Partitioning
The Compressed Frame is partitioned
into :
Partition A, contains the most important
information such as MB types,
quantization parameters, and motion
vectors.
Partition B (intra partition), contains intra
coded block pattern and intra coefficients.
Partition C (inter partition), contains only
inter coded block pattern and inter
coefficients.
Partitions may be treated differently
by the delivery layer
Unequal Error Protection in PHY and MAC
Prioritized or Guaranteed Services in MAC
March 08
9
Frame/slice
data
RTP Packets
Video Source
VCL: Video Coding Layer
IDR A B A B C
NAL: Network Abstraction Layer
IDR A B A B C
EECE 541
Enhancing H.264 Video
Transmission over 802.11 WLANs
Many enhancements are possible
Enhancements in Application Layer –error resiliency
Cross-layer Enhancements, packet size control (covered
background)
Cross-layer Enhancements, unequal error protection in MAC
Partitioned video or scalable video – different layers – different
priority
10
EECE 541
5

March 08
Enhancing H.264 Video Transmission
over 802.11 WLANs
First, we focus on cross layer enhancements for packet
size control
We determine Optimal size using knowledge of the delivery layer
(MAC & PHY) to achieve lowest PER
Video slicing is done in application layer, instead of
fragmentation in MAC, to achieve optimal packet size.
If Optimal size is larger than Slice Size, Aggregation is done in
application layer (or transport layer) to achieve optimal packet
size.
March 08
11
EECE 541
Effect of Packet (Slice) Size on Packet Loss Ratio
and Video Quality, in 802.11 WLANs
Large packets => high packet loss ratio in PHY due to high PER
Small packets => high Packet loss in MAC due to congestion
There MAY exist an optimum value for packet length
Packet Loss Ratio
0.6
0.5
0.4
0.3
0.2
0.1
0
MAC and PHY (BER 10E-4)
MAC only (BER = 10E-5)
PHY only (BER 10E-4, Retx 5)
250 500 750 1000 1250 1500 1750 2000
Packet Size (Bytes)
12
EECE 541
6

March 08
802.11: Effect of Packet Size on Packet
Loss Ratio
Using large packets:
Advantages
reduce RTP/IP/UDP, and more importantly PHY overhead
reduce collision (MAC operation overhead)
Disadvantages
March 08
Higher PER in PHY layer: PER = 1 – (1-BER)pkt_len ~ BER * pkt_len
Packet Loss Ratio
0.6
0.5
0.4
0.3
0.2
0.1
0
MAC and PHY (BER 10E-4)
MAC only (BER = 10E-5)
PHY only (BER 10E-4, Retx 5)
250 500 750 1000 1250 1500 1750 2000
Packet Size (Bytes)
13
EECE 541
Packet Size Adjustment: MAC vs. App.
Adjusting MAC packet size to optimal value can be done in
MAC layer
Application layer (Cross-Layer optimization)
MAC
Fragmentation
Video Slicing
MAC packet
Video Slice
Video Frame (NAL packet)
MAC packet MAC packet MAC packet MAC packet
Video Slice Video Slice Video Slice Video Slice
MAC packet MAC packet MAC packet MAC packet MAC packet
14
EECE 541
7

Packet Error Rate: fragments or slices of size K, picture of size L,
n fragments, X transmission attempts, e bit error rate.
Assuming an acceptable or optimal packet size, we prefer video
slicing to MAC fragmentation
March 08
H.264
Encoder
March 08
Video Slicing vs. MAC Fragmentation
15
Experiment Test-bed
16
EECE 541
H.264
Decoder
H.264 file, RTP format H.264 file, RTP format
Traffic
Pattern
Applicator
RTP
Packet
Pattern
802.11e
Simulator
Dropped Late
Receiver
Buffer
Simulator
PSNR, Video, …
Pattern for RTP packets
delivered to decoder
Offline
Network
Simulator
EECE 541
8

March 08
Effect of Slice Size on PLR & Video PSNR
Slice Size: 2300B 700B 300B
PSNR
40
35
30
25
20
15
10
5
0
March 08
40
30
20
10
0
Average PSNR
2300 700 300
Fragment or Slice Size (Bytes)
17
EECE 541
Video Slicing vs. MAC Fragmentation
NoSlice 340B Frag
Slice 300B
1 10 19 28 37 46 55 64 73 82 91 100
Frame #
Application Layer frag. (Video Slicing) 18 MAC Fragmentation
30
25
20
15
10
5
0
Average PSNR
MAC Fragmentation
Video Slicing
2300 700 300
Slice or Fragment Size
EECE 541
9

35
30
25
20
15
10
5
0
March 08
Video Slicing at the Optimal Size (700B)
Average PSNR
MAC Fragments
Video Slices
2300 700 300
Fragment or Slice Size (Bytes)
19
PSNR
40
35
30
25
20
15
10
5
2300B Slice
2300B Fragments
700BSlice (2304 frag)
740Fragment
300B Slice
300B Fragments
0
1 11 21 31 41 51 61 71 81 91 101 111
frame #
EECE 541
Transmission of Partitioned H.264 Video
over WLANs
March 08
20
EECE 541
10

March 08
MAC QoS Solutions
Video Communications requires QoS provisioning.
QoS in 802.11 WLAN
802.11e Standard defines two access methods
EDCA (Enhanced Distributed Channel Access)
Prioritized contention access: higher success probability
for video, voice, …
Aggregate QoS
HCCA (HCF Controlled Channel Access)
Scheduled access – requires a user defined scheduler
Per-session QoS
EDCA Based
March 08
21
QoS Support using 802.11e
22
EECE 541
only provides aggregate and prioritized QoS, poor delay and jitter
performance
does not perform well at heavy loads, very sensitive to MAC
parameters
HCCA Based
Scheduled access to the medium is possible through HCCA, thus
service guarantee is feasible
The scheduling scheme is user-defined in HCCA
Challenges of Scheduling in 802.11e
CSMA/CA: channel shared between Uplink/downlink at all times
Mixed contention based & contention free (controlled) access
Multi-rate PHY: Each packet may be sent at a different PHY rate
Our Solution: Controlled Access Phase Scheduling, CAPS
EECE 541
11

Video Source (downstream)
VCL: Video Coding Layer
IDR A B A B C
NAL: Network Abstraction Layer
IDR A B A B C
Other traffic
EDCA
queues
Select access mode (HCCA or EDCA)
EDCA
Contention
Access
March 08
March 08
HCCA
Queues
HCCA
Access
Application Layer
Transport and Network
Layers: RTP/UDP/IP
MAC layer: 802.11e
(CAPS enabled)
CAP (Poll Message)
Indication
Station
Traffic Pattern Information
Physical Layer
Access Point
UPSTREAM
Requests from
stations
Virtual
Packets
HCCA
Queues
23
24
Virtual Packet
Generator (VPG)
Time Stamping
scheduler
Video Source (upstream)
VCL: Video Coding Layer
IDR A B A B C
NAL: Network Abstraction Layer
IDR A B A B C
Actual
Packets
classifier
Select access mode (HCCA or EDCA)
MAC QoS Based Solutions
EDCA
Contention
Access
EECE 541
EECE 541
Application Layer
Transport and Network
Layers
MAC layer (CAPS)
Other traffic
EDCA
queues
Techniques for Transmission of Partitioned H.264 Video over WLANs
TECHNIQUE
1. Single Stream
served by EDCA
2. Single Stream
served by CAPS
3. Partitioned
Video served by
EDCA
4. Partitioned
Video served by
CAPS
5. Partitioned
Video, Partially
Aggregated,
served by CAPS
Supported
H.264
Profiles
baseline,
extended
(all profiles)
baseline,
extended
(all profiles)
extended
extended
extended
Supporte
d 802.11e
WLAN
EDCA
EDCA &
HCCA
with CAPS
EDCA
EDCA &
HCCA
with CAPS
EDCA &
HCCA
with CAPS
Network-Awareness In
The Multimedia Source
Limited: Tagging all
frames with type of
service
Limited: Tagging all
frames with traffic stream
ID
Tagging different
partitions for different
priority levels.
Tagging different
partitions for different
traffic streams.
Tagging different
partitions for different
traffic streams,
aggregating partitions B
and C (or A & B).
Media-Awareness In The WLAN
Serving tagged video in priority
levels (AC2)
Requires video pattern information,
serving tagged video in guaranteed
access traffic session
Serving packet of each partition in
a different priority level (A: AC2, B
& C: AC1)
Requires video pattern information,
serving partition A packets in a
separate guaranteed access traffic
session from partitions B & C
Requires video pattern information,
serving partition A packets in a
separate guaranteed access traffic
session from the aggregated
packets of partitions B & C
12

March 08
March 08
Loss Ratio
Loss Ratio
1
0.8
0.6
0.4
0.2
0
Delivering H.264 Partitions using
EDCA and CAPS
12 14 16 18 20 22 24 26 28
Number of Background Traffic Sources (500 Kbps each)
Delivery of Partitioned Video using CAPS and EDCA
1
0.8
0.6
0.4
0.2
0
Partition A CAPS
Partition B & C - CAPS
Partition A - EDCA
Partition B & C - EDCA
25
26
EECE 541
Effect of Aggregation on
Performance pf EDCA and CAPS
CAPS single stream
EDCA single stream
CAPS Partitioning & Aggreg
CAPS Partitioning-No Aggreg
2 4 6 8 10 12 14 16
Number of Video Sources
Total loss ratio for EDCA, CAPS, and partitioned video with CAPS (aggreg. and no aggreg.)
EECE 541
13

March 08
Loss Ratio
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Delivering H.264 Partitions using
EDCA and CAPS
Prt B & C EDCA
Prt A EDCA
Prt B&C CAPS-Aggreg
Prt A CAPS-Aggreg
CAPS single stream
EDCA single stream
2 4 6 8 10 12 14 16
Num of Video Sources
Results of experiment to determine WLAN capacity to support video sources
EDCA may be used :
March 08
27
EDCA
EECE 541
Conclusion
(Data-Partitioned Video over WLAN)
for delay and jitter tolerant video applications (streaming)
when capacity is not a concern
CAPS based guaranteed service provisioning can provide
the required QoS for real time video
When Data Partitioning (DP) is available
CAPS with DP provides better quality than EDCA with DP
Aggregation of partitions B & C or A & B is necessary in
WLANs
28
EECE 541
14

March 08
APPENDIX
Acronyms
NAL – Network Adaptation Layer
LAN – Local Area Network
VCL – Video Coding Layer
VLC – Variable Length Coding
AIFS[TC]
+SlotTime
March 08
29
EECE 541
802.11e DCF/EDCA Operation
Prioritized Access provided through traffic specific
parameters:
1. Initial Deferment, AIFS
2. Backoff window size
3. Allowed Burst size (TXOP)
Immediate access when
medium is idle >=
AIFS[TC]+SlotTime
Busy medium
AIFS[TC]
DIFS
PIFS
SIFS
Contention Window
from [1,1+CWmin[TC]]
SlotTime
30
Backoff
Window
Defer Access Select Slot and decrement backoff
as long as medium stays idle
Next Frame
EECE 541
15

QAP
(downstream)
…
QSTAs
(upstream)
EDCA
March 08
March 08
802.11e HCCA Operation
Access Point can use PIFS waiting time and generate
a contention free duration (CAP) at almost any time
PIFS
Poll
SIFS
Data
SIFS
Polled TXOP
Ack
31
…
EDCA TXOP
PIFS
Data
Controlled Access Phase (CAP)
Existing Solutions
SIFS
Ack
TXOP obtained by QAP
…
EDCA
EECE 541
Perceptual ARQ in Application Layer (prioritized) [Buc04]
Claims to achieve better performance compared to blind MAC
ARQ in 3GPP
Does not consider MAC collision and congestion problems
Partitioning and assigning a separate priority to each
partition [Cas05] [Ksen06]
Better Performance compared to only using one high priority for
all partitions
Only applicable to extended profile
Does not consider MAC collision and congestion problems
Ignores the effect of PHY errors
32
EECE 541
16

March 08
FMO – WLAN Scenarios
We conducted several experiments to determine which pattern
can provide the best quality of video for a given packet loss
scenario, typical for WLAN environments.
Two typical scenarios of bad channel cases in Wireless LANs
were considered:
The first scenario: a WLAN with BER of 5*10-4 was considered
with MAC layer fragmentation threshold set at 340Bytes.
The second scenario: a WLAN with BER of 10-4 was
considered with MAC layer fragmentation threshold of
1000Bytes.
March 08
The retransmission limit was set to 5 for lost packets.
33
Assumption
34
EECE 541
A higher I-frame frequency increases the video quality
(PSNR)
fI-frames = 1/10 * fframe I-frame
f I-frames = 1/5 * f frame
P-frames
Less time between decoder refreshes (good)
Higher likelihood of I-frame errors (bad)
EECE 541
17

March 08
Bit Errors and Frame Size
I-frame packet size ≠ P-frame packet size
f I-frames = 1/5 * f frame
f I-frames = 1/10 * f frame
March 08
I I
I
9 frames affected
18 frames affected
35
Refreshing with I-Frames
36
I
EECE 541
• Higher I frame frequency usually increases video quality in lossy environments
• However,
- I frames are larger in size -> more error prone since PER ~ length
- Increase in stream bitrate may result in higher packet loss due to congestion.
Assuming availability
of enough BW, we
observe the quality in
different situations
PSNR
32
30
28
26
24
22
PSNR vs I-Frame Interval
20
0 20 40 60 80 100
I-frame Interval
Retry 5; 1E-4 & Frag1000 Retry 5; 5E-4 & Frag1000 Retry 1; 1E-5 & Frag 340
Retry 1; 1E-5 & Frag 1000 Retry 5; 5E-4 & Frag340
EECE 541
18

Conversational
Streaming
March 08
RTCP
March 08
Application Requirements, 3GPP Example
37
Real Time Control Protocol
Responsible for management of real time session
RTCP receiver reports indicate quality of link
Reports can be used to increase error resiliency of:
DPA NAL packets
Duplicate packet transmissions of critical info
MB Coefficients
Redundant representations of the same MBs
MBs coded using different coding parameters
Fine quantization packet
Coarse quantization packet
Increased error coding (FEC)
38
EECE 541
EECE 541
19

March 08
UDP
March 08
39
User Datagram Protocol
Transmission Control Protocol (TCP)
Connection-oriented service
Reliable data transfer
40
EECE 541
Uses Sequence Numbers, Acknowledgements, retransmissions
Creates delays
Not well suited to video streaming
Responds to channel conditions through flow control
UDP provides connection-less service
No reliability, sequence numbers, acknowledgements, retransmissions,
or flow control
Corrupted packet are dropped
Sequence numbering provided by upper layer (RTP)
Has 16 bit length field – MTU size 65,535 bytes (IPv6 jumbograms
supported)
EECE 541
20

IP
Routing protocol
March 08
Internet Protocol & Media Access Control
Not germane to our WLAN environment
MAC
IEEE 802.11b, a, e?
Need to identify which implementation we are using and the maximum
size of a frame in this implementation since this size will determine the
data partitioning, number of packets required per frame (we assume we
are using progressive video since it will be displayed on a computer
screen) and error protection mechanisms.
March 08
41
H.264 profiles
EECE 541
Baseline Profile (BP): Primarily for lower-cost applications demanding less computing resources,
used widely in videoconferencing and mobile applications.
Main Profile (MP): Originally intended as the mainstream consumer profile for broadcast and
storage applications, the importance of this profile faded when the High profile was developed for
those applications.
Extended Profile (XP): Intended as the streaming video profile, this profile has relatively high
compression capability and some extra tricks for robustness to data losses and server stream
switching.
High Profile (HiP): The primary profile for broadcast and disc storage applications, particularly for
high-definition television applications (this is the profile adopted into HD DVD and Blu-ray Disc,
for example).
High 10 Profile (Hi10P): Going beyond today's mainstream consumer product capabilities, this
profile builds on top of the High Profile — adding support for up to 10 bits per sample of decoded
picture precision.
High 4:2:2 Profile (Hi422P): Primarily targeting professional applications that use interlaced video,
this profile builds on top of the High 10 Profile — adding support for the 4:2:2 chroma sampling
format while using up to 10 bits per sample of decoded picture precision.
High 4:4:4 Profile (Hi444P): This profile builds on top of the High 4:2:2 Profile — supporting up
to 4:4:4 chroma sampling, up to 12 bits per sample, and additionally supporting efficient lossless
region coding and an integer residual color transform for coding RGB video while avoiding colorspace
transformation error.
42
EECE 541
21