MPEG-4 Encoder Structure
Video encoding algorithms like MPEG-4 typically consist of the blocks shown in Figure 2.
Figure 2: typical block decomposition of a video encoder
The two most processing-intensive blocks in the MPEG-4 encoder are Motion Estimation (ME) and Texture Encoding (TE). These blocks have different memory and processing requirements and exhibit different data dependencies.
The ME requires large amounts of local memory in order to hold the search area without stressing the DRAM bandwidth. The ME and TE have roughly comparable processing requirements, but both block processing requirements are data dependent. In the case of the ME, low motion translates into a smaller number of search iterations and fewer computations. The quality of the initial motion vectors used by the ME when processing a MB has a noticeable impact on the efficiency of the ME algorithm. Poor initial motion vectors generally lead to longer iterations and potentially poor final motion vectors. The best initial motion vectors are usually derived from neighboring blocks on which the ME processing has already been applied. As a result, in order to obtain a high-quality ME it is necessary to process groups of MBs in a linear fashion, one at a time. Unfortunately, the larger these groups, the less processing that can be done in parallel since only one processor can be assigned to each of these groups. The size of these groups needs to be selected with care in order to obtain a fast, parallel implementation without compromising the compression efficiency of the codec.
Like the ME, the TE can operate on a single macroblock at a time. However, the TE has quite different characteristics. One of the most noticeable differences is that TE doesn't require as much local memory as the ME. TE includes a prediction mechanism that takes advantage of transform coefficients already computed for neighboring MBs in order to encode more efficiently coefficients belonging to the current MB being processed. This mechanism is commonly referred as AC/DC prediction. In order to take advantage of AC/DC prediction, coefficients from previous encoded blocks need to be kept in memory but they require only a fraction of the size needed to keep the large ME search area. The TE also includes a transform function that can be accelerated significantly when the input data contains a lot of zeros, which happens whenever a MB is predicted efficiently from another MB. Like the ME, the TE has some internal data dependencies: the TE can take advantage of an AC/DC coefficient prediction mechanism, which requires neighboring blocks to have already been processed by the TE.
However, because of the lower memory requirements of the TE, it is possible to run TE in parallel on multiple separate slices of a frame. Each slice is encoded separately and markers are inserted to identify the beginning and end of an encoded slice. The ability to work on separate slices in parallel makes TE a good fit for multi-core architectures, especially those able to support run-time allocation of tasks to processors since each slice may take a variable amount of time to be encoded. Having multiple slices per frame is also useful to recover from data corruption, since an error in the bit stream will not impact the entire frame but only a slice. The size of each slice results in a tradeoff between the amount of memory being consumed (since for each slice a buffer is needed to hold the coefficients from the previous row of MB), the quality of the encoding (since AC/DC prediction cannot be used to encode the first row of each slice), and performance (since the more slices we have, the more work can be done in parallel on multiple processors).
Next: MPEG-4 Implementation on the CT3600
