Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI-University: An LLM-based platform for instructional alignment to scientific classrooms

Mostafa Faghih Shojaei, Rahul Gulati, Benjamin A. Jasperson, Shangshang Wang, Simone Cimolato, Dangli Cao, Willie Neiswanger, Krishna Garikipati

TL;DR

AI-U addresses the need for domain-specific AI tutors in higher education by fine-tuning an open-source LLM with LoRA and a retrieval-augmented generation pipeline to produce instructor-style responses drawn from lecture videos, notes, and textbooks.The framework uses a dedicated expert model, LLaMA-TOMMI-1.0, in conjunction with a RAG-based synthesis module and a provenance-linked web app to ensure traceability to exact course materials and timestamps.Evaluation combines cosine similarity, an LLM-based judge, and domain expert reviews to demonstrate strong alignment with course content and outperformance of the base model.The approach offers a scalable, privacy-preserving path toward broader AI-assisted education within scientific classrooms and beyond FEM.

Abstract

We introduce AI University (AI-U), a flexible framework for AI-driven course content delivery that adapts to instructors' teaching styles. At its core, AI-U fine-tunes a large language model (LLM) with retrieval-augmented generation (RAG) to generate instructor-aligned responses from lecture videos, notes, and textbooks. Using a graduate-level finite-element-method (FEM) course as a case study, we present a scalable pipeline to systematically construct training data, fine-tune an open-source LLM with Low-Rank Adaptation (LoRA), and optimize its responses through RAG-based synthesis. Our evaluation - combining cosine similarity, LLM-based assessment, and expert review - demonstrates strong alignment with course materials. We also have developed a prototype web application, available at https://my-ai-university.com, that enhances traceability by linking AI-generated responses to specific sections of the relevant course material and time-stamped instances of the open-access video lectures. Our expert model is found to have greater cosine similarity with a reference on 86% of test cases. An LLM judge also found our expert model to outperform the base Llama 3.2 model approximately four times out of five. AI-U offers a scalable approach to AI-assisted education, paving the way for broader adoption in higher education. Here, our framework has been presented in the setting of a class on FEM - a subject that is central to training PhD and Master students in engineering science. However, this setting is a particular instance of a broader context: fine-tuning LLMs to research content in science.

AI-University: An LLM-based platform for instructional alignment to scientific classrooms

TL;DR

AI-U addresses the need for domain-specific AI tutors in higher education by fine-tuning an open-source LLM with LoRA and a retrieval-augmented generation pipeline to produce instructor-style responses drawn from lecture videos, notes, and textbooks.The framework uses a dedicated expert model, LLaMA-TOMMI-1.0, in conjunction with a RAG-based synthesis module and a provenance-linked web app to ensure traceability to exact course materials and timestamps.Evaluation combines cosine similarity, an LLM-based judge, and domain expert reviews to demonstrate strong alignment with course content and outperformance of the base model.The approach offers a scalable, privacy-preserving path toward broader AI-assisted education within scientific classrooms and beyond FEM.

Abstract

We introduce AI University (AI-U), a flexible framework for AI-driven course content delivery that adapts to instructors' teaching styles. At its core, AI-U fine-tunes a large language model (LLM) with retrieval-augmented generation (RAG) to generate instructor-aligned responses from lecture videos, notes, and textbooks. Using a graduate-level finite-element-method (FEM) course as a case study, we present a scalable pipeline to systematically construct training data, fine-tune an open-source LLM with Low-Rank Adaptation (LoRA), and optimize its responses through RAG-based synthesis. Our evaluation - combining cosine similarity, LLM-based assessment, and expert review - demonstrates strong alignment with course materials. We also have developed a prototype web application, available at https://my-ai-university.com, that enhances traceability by linking AI-generated responses to specific sections of the relevant course material and time-stamped instances of the open-access video lectures. Our expert model is found to have greater cosine similarity with a reference on 86% of test cases. An LLM judge also found our expert model to outperform the base Llama 3.2 model approximately four times out of five. AI-U offers a scalable approach to AI-assisted education, paving the way for broader adoption in higher education. Here, our framework has been presented in the setting of a class on FEM - a subject that is central to training PhD and Master students in engineering science. However, this setting is a particular instance of a broader context: fine-tuning LLMs to research content in science.

Paper Structure

This paper contains 31 sections, 1 equation, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large language models
  4. Applications in education
  5. Shortcomings and concerns
  6. The Finite Element Method
  7. Methods
  8. Inquiry pipeline
  9. Training data
  10. Fine-tuning Llama-3.2-11B
  11. Hyperparameter optimization
  12. RAG + Synthesis agent
  13. Evaluation
  14. Web application
  15. Experiments
  16. ...and 16 more sections

Figures (3)

  • Figure 1: Overview of the AI-University framework. Sections marked with a dashed line are performed once, at the beginning.
  • Figure 2: Overview of the data generation framework. Sections marked with a dashed line are performed once, at the beginning.
  • Figure 3: Demonstration of the web application, available at https://my-ai-university.com