Predicting Inter-Coded Macroblocks in P-slices
in H.264 video coding standard
Qingyuan Guo
Abhishek Kedia
Video (Image Sequence)
GOP#1
GOP#2
GOP#3
GOP#4
I
B
B
P
...
P
Frame:
Slice#1
Slice#2
Slice#3
Macroblock
..
..
...
(discussed in class)
Group of Pictures:
Macroblocks
16x16 processing unit
16x16 Y
8x8 Cb
8x8 Cr
Luma
Chroma
Macroblocks
Intra-coded
Inter-coded
-
Uses only current slice
-
Uses previously encoded frames
- Reduces spatial redundancy
- Reduces temporal redundancy
- Generally lower compression
- Generally higher compression
How to predict inter macroblocks?
Macroblock in previous frame
+
Motion vector
= Current Macroblock
16x16 Macroblocks are huge!
Single motion vector is not a very accurate representation
Solution
Divide Macroblocks into smaller blocks.
Size of smaller blocks?
Very small blocks
Large Blocks
Very small blocks
Large Blocks
- Need to transmit large number of motion vectors
- Need to transmit only a small number of motion vectors
-1
+1
- Accurate Motion Prediction => small residue to transmit
- Inaccurate Motion Prediction => large residue to transmit
+1
-1
Block size was fixed @ 8x8 in H.261
H.264 uses variable block size
Macroblock partitions: 16x16, 8x16, 16x8, 8x8
Macroblock sub-partitions: 8x8, 4x8, 8x4, 4x4
separate motion vector for each partition /sub-partition
Encoder selects “best” partition size for each part of the frame
Minimizes both residue and # motion vectors
little change between the frames 16x16 partition is chosen
areas of detailed motion, smaller partitions are more efficient.
Optimal sized Block in previous frame
+
Motion vector
Current Block
=
Motion Vector
Current Frame
Reference Frame
MV
Problem
Smallest motion step is a pixel.
Slow moving regions in the image?
In between frames, some blocks can move less then a pixel.
Need to decrease the quantization step in motion vectors
=>
Solution
Sub-pixel accuracy motion vectors
H.264 supports upto 1/4 pixel resolution
But how?
samples don't exist at sub-pixel location!
Interpolation
Generate samples by "averaging" neighboring values
6 tap Finite Impulse Response Filter (FIR)
or
For computing half pixel values
Linear interpolation
For computing quater pixel values
Interpolation of Chroma
Recall : Chroma was sub-sampled by 2 in each direction
So we need to interpolate at every 1/8 interval, instead of 1/4 in luma
Motion Vector Prediction
- Motion vectors also need to be transmitted
- Encoding a motion vector for each partition can cost a significant number of bits.
- Motion vectors for neighboring partitions are often highly correlated
So we predict motion vectors from neighboring values
Finally encode and transmit the MVD (prediction error)
MVD = MV – MVp
Forming MVp
E : Current Partition
A : Partition to immediate left
B : Partition to immediate above
C : Top-Right partition
1. Partition size excluding 16x8 and 8x16 : MVp median of MV for A, B and C.
2. For 16x8 partitions: MVp upper partition = MV B, MVp lower partition = MV A
3. For 8x16 partitions: MVp left partition = MV A, MVp right partition = MV C
Summary
Basic Idea for inter prediction
-
Macroblock partitions.
Variable partition size.
-
Motion Vector
Sub-pixel resolution
Interpolation
Prediction
Bibliography
[1]. "Prediction of Inter Macroblocks in P-slices" - Iain E. G. Richardson
[2]. "Overview of the H. 264/AVC video coding standard" - Thomas Wiegand, Gary J. Sullivan, Gisle Bjøntegaard, and Ajay Luthra [PDF]
[3]. "H.264 and MPEG-4 Video Compression" - Iain E. G. Richardson [PDF]
[4]. "The Emerging H.264/AVC standard" - Ralf Schäfer,
Thomas Wiegand and Heiko Schwarz [PDF]
[5]. NPTEL course on "Digital Voice & Picture Communication" - Prof. S. Sengupta, IIT Kharagpur [link]