Algorithm and hardware design of a fast intra-frame mode decision module for h.264/AVC encoders

In the Rate-Distortion Optimization (RDO) technique for video encoding, the process of choosing the best prediction mode is performed through exhaustive executions of the whole encoding process, which increases significantly the encoder computational complexity. Considering H.264/AVC intra-frame prediction there are several modes to encode each macroblock (MB). In order to reduce the number of calculations necessary to determine the best intra-frame mode, this work proposes an algorithm and the hardware design for a fast intra-frame mode decision module for H.264/AVC encoders. The application of the proposed algorithm reduces in more than ten times the number of encoding iterations for choosing the best intra-frame mode when compared with RDO-based decision, at the cost of relatively small bit-rate increase (5% in average) and image quality loss (0.2 dB in PSNR). The architecture takes 36 clock cycles to perform the intra-frame decision for one MB and it achieved an operation frequency of 130 MHz when synthesized for TSMC 0.18μm, being able to process more than 400 HD1080p frames per second. With this approach, we achieved one order-of-magnitude performance improvement compared with RDO-based approaches, which is very important not only from the performance but also from the energy consumption perspective when considering the need to improve the video encoding efficiency for battery operated devices. Compared with the best previous results reported, the implemented architecture achieve a complexity reduction of five times, a processing capability increase of 14 times and a reduction in the number of clock cycles per MB of 11 times.