Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faculty Perspectives on the Potential of RAG in Computer Science Higher Education

Sagnik Dakshit

TL;DR

This study investigates the potential and limitations of Retrieval Augmented Generation (RAG) in computer science higher education by collecting faculty perspectives on using RAG as virtual teaching assistants and teaching aids. It deploys personalized RAG pipelines based on the Gemini LLM across junior, senior, and graduate CS courses and evaluates two tasks through a faculty-designed questionnaire grounded in the TAM framework. Results show higher acceptance for RAG as a virtual teaching assistant (80%) than as a teaching aid (60%), with all five faculty members approving TA use in some capacity (100% for Task B). Faculty feedback highlights the need for multi-source grounding, expert oversight, improved interfaces, and enhanced capabilities for handling equations and non-text content, signaling concrete directions for safe and effective large-scale deployment. Overall, the work pioneers faculty-facing evaluation of LLM-based RAG in CS education and outlines practical safeguards and research directions to advance responsible digital education integration.

Abstract

The emergence of Large Language Models (LLMs) has significantly impacted the field of Natural Language Processing and has transformed conversational tasks across various domains because of their widespread integration in applications and public access. The discussion surrounding the application of LLMs in education has raised ethical concerns, particularly concerning plagiarism and policy compliance. Despite the prowess of LLMs in conversational tasks, the limitations of reliability and hallucinations exacerbate the need to guardrail conversations, motivating our investigation of RAG in computer science higher education. We developed Retrieval Augmented Generation (RAG) applications for the two tasks of virtual teaching assistants and teaching aids. In our study, we collected the ratings and opinions of faculty members in undergraduate and graduate computer science university courses at various levels, using our personalized RAG systems for each course. This study is the first to gather faculty feedback on the application of LLM-based RAG in education. The investigation revealed that while faculty members acknowledge the potential of RAG systems as virtual teaching assistants and teaching aids, certain barriers and features are suggested for their full-scale deployment. These findings contribute to the ongoing discussion on the integration of advanced language models in educational settings, highlighting the need for careful consideration of ethical implications and the development of appropriate safeguards to ensure responsible and effective implementation.

Faculty Perspectives on the Potential of RAG in Computer Science Higher Education

TL;DR

This study investigates the potential and limitations of Retrieval Augmented Generation (RAG) in computer science higher education by collecting faculty perspectives on using RAG as virtual teaching assistants and teaching aids. It deploys personalized RAG pipelines based on the Gemini LLM across junior, senior, and graduate CS courses and evaluates two tasks through a faculty-designed questionnaire grounded in the TAM framework. Results show higher acceptance for RAG as a virtual teaching assistant (80%) than as a teaching aid (60%), with all five faculty members approving TA use in some capacity (100% for Task B). Faculty feedback highlights the need for multi-source grounding, expert oversight, improved interfaces, and enhanced capabilities for handling equations and non-text content, signaling concrete directions for safe and effective large-scale deployment. Overall, the work pioneers faculty-facing evaluation of LLM-based RAG in CS education and outlines practical safeguards and research directions to advance responsible digital education integration.

Abstract

The emergence of Large Language Models (LLMs) has significantly impacted the field of Natural Language Processing and has transformed conversational tasks across various domains because of their widespread integration in applications and public access. The discussion surrounding the application of LLMs in education has raised ethical concerns, particularly concerning plagiarism and policy compliance. Despite the prowess of LLMs in conversational tasks, the limitations of reliability and hallucinations exacerbate the need to guardrail conversations, motivating our investigation of RAG in computer science higher education. We developed Retrieval Augmented Generation (RAG) applications for the two tasks of virtual teaching assistants and teaching aids. In our study, we collected the ratings and opinions of faculty members in undergraduate and graduate computer science university courses at various levels, using our personalized RAG systems for each course. This study is the first to gather faculty feedback on the application of LLM-based RAG in education. The investigation revealed that while faculty members acknowledge the potential of RAG systems as virtual teaching assistants and teaching aids, certain barriers and features are suggested for their full-scale deployment. These findings contribute to the ongoing discussion on the integration of advanced language models in educational settings, highlighting the need for careful consideration of ethical implications and the development of appropriate safeguards to ensure responsible and effective implementation.
Paper Structure (14 sections, 2 figures, 1 table)

This paper contains 14 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Contribution
  3. Related Works
  4. Methodology
  5. Research Design
  6. RAG Model
  7. Questionnaire
  8. Faculty Perspective: Potential and Limitation Discussions
  9. Task Ratings
  10. RQ 1 and RQ 2 Rating
  11. Faculty Perspectives
  12. RQ3 Interpretation
  13. Discussion and Future Work
  14. Conclusion

Figures (2)

  • Figure 1: Our individual RAG pipeline using Gemini LLM developed on class materials provided by faculty members.
  • Figure 2: Faculty Feedback Rating on 5-Point Likert Scale for 4 questions obtained through Questionnaire: Top Left: Can LLMs aid generate assignment questions?, Top Right: Can LLMs aid in answering questions of students ?, Bottom left: Can LLMs be used as teaching assitant ?,and Bottom Right: Can LLMs be used as teaching aid for faculty in computer science classes ?