Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fighting AI with AI: AI-Agent Augmented DNS Blocking of LLM Services during Student Evaluations

Yonas Kassa, James Bonacci, Ping Wang

Abstract

The transformative potential of large language models (LLMs) in education, such as improving accessibility and personalized learning, is being eclipsed by significant challenges. These challenges stem from concerns that LLMs undermine academic assessment by enabling bypassing of critical thinking, leading to increased cognitive offloading. This emerging trend stresses the dual imperative of harnessing AI's educational benefits while safeguarding critical thinking and academic rigor in the evolving AI ecosystem. To this end, we introduce AI-Sinkhole, an AI-agent augmented DNS-based framework that dynamically discovers, semantically classifies, and temporarily network-wide blocks emerging LLM chatbot services during proctored exams. AI-Sinkhole offers explainable classification via quantized LLMs (LLama 3, DeepSeek-R1, Qwen-3) and dynamic DNS blocking with Pi-Hole. We also share our observations in using LLMs as explainable classifiers which achieved robust cross-lingual performance (F1-score > 0.83). To support future research and development in this domain initial codes with a readily deployable 'AI-Sinkhole' blockist is available on https://github.com/AIMLEdu/ai-sinkhole.

Fighting AI with AI: AI-Agent Augmented DNS Blocking of LLM Services during Student Evaluations

Abstract

The transformative potential of large language models (LLMs) in education, such as improving accessibility and personalized learning, is being eclipsed by significant challenges. These challenges stem from concerns that LLMs undermine academic assessment by enabling bypassing of critical thinking, leading to increased cognitive offloading. This emerging trend stresses the dual imperative of harnessing AI's educational benefits while safeguarding critical thinking and academic rigor in the evolving AI ecosystem. To this end, we introduce AI-Sinkhole, an AI-agent augmented DNS-based framework that dynamically discovers, semantically classifies, and temporarily network-wide blocks emerging LLM chatbot services during proctored exams. AI-Sinkhole offers explainable classification via quantized LLMs (LLama 3, DeepSeek-R1, Qwen-3) and dynamic DNS blocking with Pi-Hole. We also share our observations in using LLMs as explainable classifiers which achieved robust cross-lingual performance (F1-score > 0.83). To support future research and development in this domain initial codes with a readily deployable 'AI-Sinkhole' blockist is available on https://github.com/AIMLEdu/ai-sinkhole.

Paper Structure

This paper contains 13 sections, 1 equation, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Architecture and workflow of AI-Sinkhole Framework
  4. Architecture of AI-Sinkhole Framework
  5. Workflow of AI-Sinkhole Framework
  6. Evaluation and Results
  7. Experiment Setup
  8. Dataset
  9. LLM models and implementation details
  10. Results
  11. Observations
  12. Discussion
  13. Conclusion and Future Work

Figures (5)

  • Figure 1: AI-Sinkhole Framework
  • Figure 2: Inference latency of the three quantized LLMs
  • Figure 3: Prompt template used in the experiment with a sample response
  • Figure 4: LLM comparison. Left: Confusion matrices comparing LLM decisions (LLama 3) with the ground truth. Right: hamming distance showing similarity of model decisions.
  • Figure 5: Pi-hole dashboard showing AI-Sinkhole deployment showing 63 AI domains on the blocklist.