Decoding Large-Language Models: A Systematic Overview of Socio-Technical Impacts, Constraints, and Emerging Questions
Zeyneb N. Kaya, Souvick Ghosh
TL;DR
This paper presents a systematic review of large language model (LLM) research (2016–2023) to map aims, methodologies, limitations, and societal impacts. By aggregating 61 influential studies across 16 venues, it identifies three main RQ-driven axes: aims (responsible development, performance improvements, investigative understanding), methodologies (datasets/benchmarks, input/output design, training, and understanding), and limitations (performance gaps, study constraints, and societal concerns). The work highlights ethical considerations, data and benchmark development, prompting and tool-use strategies, and the balance between openness and safety in model release. It offers a structured synthesis of current progress and prescribes directions for responsible, scalable, and transparent LLM research and deployment.
Abstract
There have been rapid advancements in the capabilities of large language models (LLMs) in recent years, greatly revolutionizing the field of natural language processing (NLP) and artificial intelligence (AI) to understand and interact with human language. Therefore, in this work, we conduct a systematic investigation of the literature to identify the prominent themes and directions of LLM developments, impacts, and limitations. Our findings illustrate the aims, methodologies, limitations, and future directions of LLM research. It includes responsible development considerations, algorithmic improvements, ethical challenges, and societal implications of LLM development. Overall, this paper provides a rigorous and comprehensive overview of current research in LLM and identifies potential directions for future development. The article highlights the application areas that could have a positive impact on society along with the ethical considerations.