Free to play: UN Trade and Development's experience with developing its own open-source Retrieval Augmented Generation Large Language Model application
Daniel Hopp
TL;DR
The paper addresses how UNCTAD built an open-source, in-house Retrieval Augmented Generation LLM application to support its work in official statistics, balancing cost, flexibility, and data privacy. It details a fully open stack using components like nlp_pipeline, LlamaIndex, pgvector, and a local_rag_llm pipeline, with a Streamlit front end and Docker deployment to enable scalable, multi-user use on GPU or CPU hardware. The authors argue that in-house development fosters institutional capacity, reduces vendor lock-in, and provides stronger data governance, while acknowledging resource, scalability, and frontier-model access challenges. They advocate a pragmatic, non-binary path that can integrate paid components if needed, and emphasize the importance of institutional space, ongoing optimization, and community contribution to advance AI capabilities for NSOs and developing countries.
Abstract
Generative artificial intelligence (AI), and in particular Large Language Models (LLMs), have exploded in popularity and attention since the release to the public of ChatGPT's Generative Pre-trained Transformer (GPT)-3.5 model in November of 2022. Due to the power of these general purpose models and their ability to communicate in natural language, they can be useful in a range of domains, including the work of official statistics and international organizations. However, with such a novel and seemingly complex technology, it can feel as if generative AI is something that happens to an organization, something that can be talked about but not understood, that can be commented on but not contributed to. Additionally, the costs of adoption and operation of proprietary solutions can be both uncertain and high, a barrier for often cost-constrained international organizations. In the face of these challenges, United Nations Trade and Development (UNCTAD), through its Global Crisis Response Group (GCRG), has explored and developed its own open-source Retrieval Augmented Generation (RAG) LLM application. RAG makes LLMs aware of and more useful for the organization's domain and work. Developing in-house solutions comes with pros and cons, with pros including cost, flexibility, and fostering institutional knowledge. Cons include time and skill investments and gaps and application polish and power. The three libraries developed to produce the app, nlp_pipeline for document processing and statistical analysis, local_rag_llm for running a local RAG LLM, and streamlit_rag for the user interface, are publicly available on PyPI and GitHub with Dockerfiles. A fourth library, local_llm_finetune, is also available for fine-tuning existing LLMs which can then be used in the application.