Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI-VERDE: A Gateway for Egalitarian Access to Large Language Model-Based Resources For Educational Institutions

Paul Mithun, Enrique Noriega-Atala, Nirav Merchant, Edwin Skidmore

TL;DR

The paper addresses the barrier to equitable LLM access in higher education by introducing AI-VERDE, a platform that combines on-premises and cloud-open LLMs with a multi-tenant, RAG-enabled architecture, robust access control, and budget management. It leverages a Kubernetes-based microservices design with vLLM-backed LLM hosting, LiteLLM for unified API access, and a Weaviate vector store for retrieval-augmented generation, all accessible via a federated web UI or API. A pilot at the University of Arizona demonstrates active usage across courses and research projects, with tens of thousands of API calls and millions of tokens processed, illustrating practical adoption. The work contributes a holistic survey of academic adoption barriers, a lower-cost, privacy-preserving alternative to commercial LLM platforms, and a path toward scalable, governance-enabled AI deployment in universities, including future LMS integration and researcher-focused tools.

Abstract

We present AI-VERDE, a unified LLM-as-a-platform service designed to facilitate seamless integration of commercial, cloud-hosted, and on-premise open LLMs in academic settings. AI-VERDE streamlines access management for instructional and research groups by providing features such as robust access control, privacy-preserving mechanisms, native Retrieval-Augmented Generation (RAG) support, budget management for third-party LLM services, and both a conversational web interface and API access. In a pilot deployment at a large public university, AI-VERDE demonstrated significant engagement across diverse educational and research groups, enabling activities that would typically require substantial budgets for commercial LLM services with limited user and team management capabilities. To the best of our knowledge, AI-Verde is the first platform to address both academic and research needs for LLMs within an higher education institutional framework.

AI-VERDE: A Gateway for Egalitarian Access to Large Language Model-Based Resources For Educational Institutions

TL;DR

The paper addresses the barrier to equitable LLM access in higher education by introducing AI-VERDE, a platform that combines on-premises and cloud-open LLMs with a multi-tenant, RAG-enabled architecture, robust access control, and budget management. It leverages a Kubernetes-based microservices design with vLLM-backed LLM hosting, LiteLLM for unified API access, and a Weaviate vector store for retrieval-augmented generation, all accessible via a federated web UI or API. A pilot at the University of Arizona demonstrates active usage across courses and research projects, with tens of thousands of API calls and millions of tokens processed, illustrating practical adoption. The work contributes a holistic survey of academic adoption barriers, a lower-cost, privacy-preserving alternative to commercial LLM platforms, and a path toward scalable, governance-enabled AI deployment in universities, including future LMS integration and researcher-focused tools.

Abstract

We present AI-VERDE, a unified LLM-as-a-platform service designed to facilitate seamless integration of commercial, cloud-hosted, and on-premise open LLMs in academic settings. AI-VERDE streamlines access management for instructional and research groups by providing features such as robust access control, privacy-preserving mechanisms, native Retrieval-Augmented Generation (RAG) support, budget management for third-party LLM services, and both a conversational web interface and API access. In a pilot deployment at a large public university, AI-VERDE demonstrated significant engagement across diverse educational and research groups, enabling activities that would typically require substantial budgets for commercial LLM services with limited user and team management capabilities. To the best of our knowledge, AI-Verde is the first platform to address both academic and research needs for LLMs within an higher education institutional framework.

Paper Structure

This paper contains 36 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. AI-VERDE Architecture components
  3. Backend: LLM serving
  4. Front-end: User facing web interface
  5. Document Intake
  6. Issues with commercial LLM platforms in an academic setting.
  7. Intellectual Property, Privacy and Content Ownership
  8. Limited Access to Specialized Knowledge
  9. Authorization and Authentication
  10. Equity and Resource Constraints
  11. Anonymity and confidentiality of sensitive data
  12. Access to dedicated hardware
  13. Pilot Deployment
  14. Qualitative Analysis
  15. A Quantitative Analysis
  16. ...and 21 more sections

Figures (1)

  • Figure 1: Architecture Diagram of AI-VERDE . The left hand side of the diagram represents the frontend, which consists of the conversational web interface, depicted at the top, and a snippet of code with an example of how to programmatically connect to AI-VERDE using industry-standard python software packages. The right side depicts the backend elements, and illustrates multiple different models running with vLLM, as well as a proxy to commercial models, all exposed to clients through LiteLLM. The backend also contains our managed instance of the Weaviate vector database manager, which houses the different vector databases, corresponding to each course, enabled in AI-VERDE .