Deep literature reviews: an application of fine-tuned language models to migration research

TL;DR

This paper presents a hybrid framework for literature reviews that augments traditional bibliometric methods with large language models (LLMs) and demonstrates the potential of fine-tuned LLMs to conduct more efficient, consistent, and insightful literature reviews across disciplines, ultimately accelerating knowledge synthesis and scientific discovery.

Abstract

This paper presents a hybrid framework for literature reviews that augments traditional bibliometric methods with large language models (LLMs). By fine-tuning open-source LLMs, our approach enables scalable extraction of qualitative insights from large volumes of research content, enhancing both the breadth and depth of knowledge synthesis. To improve annotation efficiency and consistency, we introduce an error-focused validation process in which LLMs generate initial labels and human reviewers correct misclassifications. Applying this framework to over 20000 scientific articles about human migration, we demonstrate that a domain-adapted LLM can serve as a "specialist" model - capable of accurately selecting relevant studies, detecting emerging trends, and identifying critical research gaps. Notably, the LLM-assisted review reveals a growing scholarly interest in climate-induced migration. However, existing literature disproportionately centers on a narrow set of environmental hazards (e.g., floods, droughts, sea-level rise, and land degradation), while overlooking others that more directly affect human health and well-being, such as air and water pollution or infectious diseases. This imbalance highlights the need for more comprehensive research that goes beyond physical environmental changes to examine their ecological and societal consequences, particularly in shaping migration as an adaptive response. Overall, our proposed framework demonstrates the potential of fine-tuned LLMs to conduct more efficient, consistent, and insightful literature reviews across disciplines, ultimately accelerating knowledge synthesis and scientific discovery.

Figures (5)

• Figure 1: Framework of deep literature reviews using fine-tuned LLMs
• Figure 2: Model performance measured by accuracy (ACC) for mutually exclusive classifications and by Jaccard Index (JAC) for multi-label classifications. Accuracy is reported for initial labels by the original Llama 3.2 3B model as well as for training and testing labels classified by the fine-tuned Llama mdoel.
• Figure 3: Trends of human migration and mobility studies. Note: the data underlying panel b is a subset of 22,267 articles conditional on articles that is about human migration and mobility.
• Figure 4: Climatic/environmental hazards discussed in migration research. Note: the data underlying this figure is a subset of 22,267 articles conditional on i) the articles are about human migration and mobility; and ii) the articles addressed climate/environment as a migration driver.
• Figure 5: The degree to which climatic/environmental hazards discussed jointly in a given article. Note: the data underlying this figure is a subset of 22,267 articles conditional on i) the articles are about human migration and mobility; and ii) the articles addressed climate/environment as a migration driver.