Chinese-Vicuna: A Chinese Instruction-following Llama-based Model

TL;DR

Chinese-Vicuna advances Chinese instruction-following by applying parameter-efficient fine-tuning (LoRA/QLoRA) to LLaMA models, enabling training and deployment on consumer GPUs with 4-bit quantization. It fuses BELLE and Guanaco data to create a large Chinese-centric corpus and demonstrates domain-specific adaptation for medical and legal tasks, alongside a Chinese-chat variant trained on dialogue datasets. The work provides end-to-end tooling for model conversion, CPU inference, and multi-turn interfaces, with detailed evaluations across instruction-following, chat, medical QA, and legal QA. Its open-source, modular design and demonstrated performance on low-resource hardware position Chinese-Vicuna as a scalable foundation for Chinese LLM applications, with future work aimed at reinforcement learning from human feedback and knowledge retrieval.

Abstract

Chinese-Vicuna is an open-source, resource-efficient language model designed to bridge the gap in Chinese instruction-following capabilities by fine-tuning Meta's LLaMA architecture using Low-Rank Adaptation (LoRA). Targeting low-resource environments, it enables cost-effective deployment on consumer GPUs (e.g., RTX-2080Ti for 7B models) and supports domain-specific adaptation in fields like healthcare and law. By integrating hybrid datasets (BELLE and Guanaco) and 4-bit quantization (QLoRA), the model achieves competitive performance in tasks such as translation, code generation, and domain-specific Q\&A. The project provides a comprehensive toolkit for model conversion, CPU inference, and multi-turn dialogue interfaces, emphasizing accessibility for researchers and developers. Evaluations indicate competitive performance across medical tasks, multi-turn dialogue coherence, and real-time legal updates. Chinese-Vicuna's modular design, open-source ecosystem, and community-driven enhancements position it as a versatile foundation for Chinese LLM applications.