Ethnic Festival LLM Via Prompt and LoRA Fine-Tuning

Abstract

 As the core carrier of ethnic culture, the construction of a multi-ethnic festival question-and-answer model is conducive to promoting the digital development of ethnic cultural heritage.Addressing the challenges of constructing large language models and the high costs of full fine-tuning, this paper investigates LoRA fine-tuning methods for ethnic festival large language models. It establishes an ethnic festival question-answering model centered on core content from books such as The Chinese Festival Traditional Culture Reader: Collector's Edition and Ethnic Minority Festivals, alongside ethnic festival introductions from Baidu Baike and ethnic websites. First, we constructed datasets and guided the model to generate knowledge question-answer pairs through prompt design. Subsequently, multiple general-purpose large models underwent LoRA fine-tuning. Finally, the optimal fine-tuned model was selected through evaluation and its effectiveness verified. Experimental results show that the fine-tuned ChatGLM4-9B-Chat model achieved improvements of 16.74, 14.25, 13.48, and 16.00 in BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L metrics respectively compared to the base model. The experiments demonstrate that the selected model can effectively learn and comprehend knowledge related to festival domains, providing accurate answers to user queries. This contributes to promoting the dissemination of ethnic festival culture and advancing the digitalization of festivals.