Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives

Desta Haileselassie Hagos, Rick Battle, Danda B. Rawat

TL;DR

This survey analyzes the rapid advancement of Generative AI and Large Language Models, outlining core model families (GANs, VAEs, autoregressive, diffusion, and multimodal models) and transformer-based LLMs. It synthesizes technical foundations, practical applications, and key challenges, including bias, interpretability, privacy, and computational costs, while highlighting methods such as self-supervised pre-training and RLHF as central to progress. The authors provide a holistic view of architectural trends (encoder–decoder vs decoder-only), long-sequence handling, and cross-modal capabilities, culminating in a roadmap for responsible, ethical deployment and future research to address gaps in fairness, memory, and safety. The practical significance lies in guiding researchers, policymakers, and industry practitioners toward robust, scalable, and trustworthy AI systems that can be deployed across diverse sectors while mitigating risks like deepfakes and data leakage.

Abstract

The emergence of Generative Artificial Intelligence (AI) and Large Language Models (LLMs) has marked a new era of Natural Language Processing (NLP), introducing unprecedented capabilities that are revolutionizing various domains. This paper explores the current state of these cutting-edge technologies, demonstrating their remarkable advancements and wide-ranging applications. Our paper contributes to providing a holistic perspective on the technical foundations, practical applications, and emerging challenges within the evolving landscape of Generative AI and LLMs. We believe that understanding the generative capabilities of AI systems and the specific context of LLMs is crucial for researchers, practitioners, and policymakers to collaboratively shape the responsible and ethical integration of these technologies into various domains. Furthermore, we identify and address main research gaps, providing valuable insights to guide future research endeavors within the AI research community.

Recent Advances in Generative AI and Large Language Models: Current Status, Challenges, and Perspectives

TL;DR

This survey analyzes the rapid advancement of Generative AI and Large Language Models, outlining core model families (GANs, VAEs, autoregressive, diffusion, and multimodal models) and transformer-based LLMs. It synthesizes technical foundations, practical applications, and key challenges, including bias, interpretability, privacy, and computational costs, while highlighting methods such as self-supervised pre-training and RLHF as central to progress. The authors provide a holistic view of architectural trends (encoder–decoder vs decoder-only), long-sequence handling, and cross-modal capabilities, culminating in a roadmap for responsible, ethical deployment and future research to address gaps in fairness, memory, and safety. The practical significance lies in guiding researchers, policymakers, and industry practitioners toward robust, scalable, and trustworthy AI systems that can be deployed across diverse sectors while mitigating risks like deepfakes and data leakage.

Abstract

The emergence of Generative Artificial Intelligence (AI) and Large Language Models (LLMs) has marked a new era of Natural Language Processing (NLP), introducing unprecedented capabilities that are revolutionizing various domains. This paper explores the current state of these cutting-edge technologies, demonstrating their remarkable advancements and wide-ranging applications. Our paper contributes to providing a holistic perspective on the technical foundations, practical applications, and emerging challenges within the evolving landscape of Generative AI and LLMs. We believe that understanding the generative capabilities of AI systems and the specific context of LLMs is crucial for researchers, practitioners, and policymakers to collaboratively shape the responsible and ethical integration of these technologies into various domains. Furthermore, we identify and address main research gaps, providing valuable insights to guide future research endeavors within the AI research community.
Paper Structure (21 sections, 27 equations, 1 figure, 3 tables)

This paper contains 21 sections, 27 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Generative AI
  3. GANs
  4. Variational Autoencoder Models
  5. Autoregressive Models
  6. Mixture of Expert Models
  7. Model Merging
  8. Diffusion Models
  9. Multimodal Generative Models
  10. Applications of Generative AI
  11. Language Modeling
  12. Statistical Language Models
  13. Neural Network Language Models
  14. Transformer Language Models
  15. Architecture of Transformer Models
  16. ...and 6 more sections

Figures (1)

  • Figure 1: Timeline and model size of LLMs (M=millions, B=billions).