Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Jill Watson: A Virtual Teaching Assistant powered by ChatGPT

Karan Taneja, Pratyusha Maiti, Sandeep Kakar, Pranav Guruprasad, Sanjeev Rao, Ashok K. Goel

TL;DR

The paper introduces Jill Watson, a ChatGPT-powered Virtual Teaching Assistant that answers course-content and logistics questions using multiple large documents without fine-tuning. It adopts a XiaoIce-inspired, skill-based architecture with modules for coreference resolution, skill classification, contextual answering, and safety filtering, enabling extensibility via new APIs. Grounding is achieved through dense passage retrieval and textual entailment checks, with citations to document and page numbers. Experiments against legacy Jill Watson and OpenAI Assistants show improved answer quality and reduced harmful outputs, and the system demonstrates real-world classroom deployments.

Abstract

Conversational AI agents often require extensive datasets for training that are not publicly released, are limited to social chit-chat or handling a specific domain, and may not be easily extended to accommodate the latest advances in AI technologies. This paper introduces Jill Watson, a conversational Virtual Teaching Assistant (VTA) leveraging the capabilities of ChatGPT. Jill Watson based on ChatGPT requires no prior training and uses a modular design to allow the integration of new APIs using a skill-based architecture inspired by XiaoIce. Jill Watson is also well-suited for intelligent textbooks as it can process and converse using multiple large documents. We exclusively utilize publicly available resources for reproducibility and extensibility. Comparative analysis shows that our system outperforms the legacy knowledge-based Jill Watson as well as the OpenAI Assistants service. We employ many safety measures that reduce instances of hallucinations and toxicity. The paper also includes real-world examples from a classroom setting that demonstrate different features of Jill Watson and its effectiveness.

Jill Watson: A Virtual Teaching Assistant powered by ChatGPT

TL;DR

The paper introduces Jill Watson, a ChatGPT-powered Virtual Teaching Assistant that answers course-content and logistics questions using multiple large documents without fine-tuning. It adopts a XiaoIce-inspired, skill-based architecture with modules for coreference resolution, skill classification, contextual answering, and safety filtering, enabling extensibility via new APIs. Grounding is achieved through dense passage retrieval and textual entailment checks, with citations to document and page numbers. Experiments against legacy Jill Watson and OpenAI Assistants show improved answer quality and reduced harmful outputs, and the system demonstrates real-world classroom deployments.

Abstract

Conversational AI agents often require extensive datasets for training that are not publicly released, are limited to social chit-chat or handling a specific domain, and may not be easily extended to accommodate the latest advances in AI technologies. This paper introduces Jill Watson, a conversational Virtual Teaching Assistant (VTA) leveraging the capabilities of ChatGPT. Jill Watson based on ChatGPT requires no prior training and uses a modular design to allow the integration of new APIs using a skill-based architecture inspired by XiaoIce. Jill Watson is also well-suited for intelligent textbooks as it can process and converse using multiple large documents. We exclusively utilize publicly available resources for reproducibility and extensibility. Comparative analysis shows that our system outperforms the legacy knowledge-based Jill Watson as well as the OpenAI Assistants service. We employ many safety measures that reduce instances of hallucinations and toxicity. The paper also includes real-world examples from a classroom setting that demonstrate different features of Jill Watson and its effectiveness.
Paper Structure (10 sections, 3 figures, 3 tables, 1 algorithm)

This paper contains 10 sections, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Jill Watson Architecture
  4. Coreference Resolution
  5. Skill Classifier
  6. Contextual Answering
  7. Other Skills
  8. Moderation Filter
  9. Results
  10. Conclusion

Figures (3)

  • Figure 1: Architecture of Jill Watson: After the coreference resolution of an incoming query, the skill classifier is used to find the most appropriate skill for response generation. Jill Watson's skills include Contextual Answering, Greetings, etc. The updated conversation history is used as context for generating responses in the future.
  • Figure 2: Passage representation consists of the original text, heading, clean text, summary text, and context embeddings of both clean and summary texts.
  • Figure 3: Answer generation prompt (left) and an example response with citation (right). The context contains five passages with document name and page numbers.