The Solution for the AIGC Inference Performance Optimization Competition
Sishun Pan, Haonan Xu, Zhonghua Wan, Yang Yang
TL;DR
This paper addresses the challenge of achieving high-throughput, low-latency inference for large Ernie-based generation models in commercial settings. It assembles a multi-faceted optimization pipeline using the Paddle Inference framework, including Faster Transformer, embedding layer pruning, FP16 inference, and multi-process data handling, to maximize throughput. Empirical results show an 8.96-fold speedup over baselines while preserving comparable performance, validating the synergy of model-level and processing-level optimizations. The approach demonstrates practical viability for real-world AIGC deployments on GPUs and large-scale infrastructure.
Abstract
In recent years, the rapid advancement of large-scale pre-trained language models based on transformer architectures has revolutionized natural language processing tasks. Among these, ChatGPT has gained widespread popularity, demonstrating human-level conversational abilities and attracting over 100 million monthly users by late 2022. Concurrently, Baidu's commercial deployment of the Ernie Wenxin model has significantly enhanced marketing effectiveness through AI-driven technologies. This paper focuses on optimizing high-performance inference for Ernie models, emphasizing GPU acceleration and leveraging the Paddle inference framework. We employ techniques such as Faster Transformer for efficient model processing, embedding layer pruning to reduce computational overhead, and FP16 half-precision inference for enhanced computational efficiency. Additionally, our approach integrates efficient data handling strategies using multi-process parallel processing to minimize latency. Experimental results demonstrate that our optimized solution achieves up to an 8.96x improvement in inference speed compared to standard methods, while maintaining competitive performance.