Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Let Me Teach You: Pedagogical Foundations of Feedback for Language Models

Beatriz Borges, Niket Tandon, Tanja Käser, Antoine Bosselut

TL;DR

Ideas from pedagogy are compiled to introduce FELT, a feedback framework for LLMs that outlines various characteristics of the feedback space, and a feedback content taxonomy based on these variables, providing a general mapping of the feedback space.

Abstract

Natural Language Feedback (NLF) is an increasingly popular mechanism for aligning Large Language Models (LLMs) to human preferences. Despite the diversity of the information it can convey, NLF methods are often hand-designed and arbitrary, with little systematic grounding. At the same time, research in learning sciences has long established several effective feedback models. In this opinion piece, we compile ideas from pedagogy to introduce FELT, a feedback framework for LLMs that outlines various characteristics of the feedback space, and a feedback content taxonomy based on these variables, providing a general mapping of the feedback space. In addition to streamlining NLF designs, FELT also brings out new, unexplored directions for research in NLF. We make our taxonomy available to the community, providing guides and examples for mapping our categorizations to future research.

Let Me Teach You: Pedagogical Foundations of Feedback for Language Models

TL;DR

Ideas from pedagogy are compiled to introduce FELT, a feedback framework for LLMs that outlines various characteristics of the feedback space, and a feedback content taxonomy based on these variables, providing a general mapping of the feedback space.

Abstract

Natural Language Feedback (NLF) is an increasingly popular mechanism for aligning Large Language Models (LLMs) to human preferences. Despite the diversity of the information it can convey, NLF methods are often hand-designed and arbitrary, with little systematic grounding. At the same time, research in learning sciences has long established several effective feedback models. In this opinion piece, we compile ideas from pedagogy to introduce FELT, a feedback framework for LLMs that outlines various characteristics of the feedback space, and a feedback content taxonomy based on these variables, providing a general mapping of the feedback space. In addition to streamlining NLF designs, FELT also brings out new, unexplored directions for research in NLF. We make our taxonomy available to the community, providing guides and examples for mapping our categorizations to future research.
Paper Structure (71 sections, 3 figures, 2 tables)

This paper contains 71 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Feedback in NLP
  4. Current State of Feedback in NLP
  5. Informativeness of Feedback
  6. Natural Language Feedback for LLMs
  7. How should feedback be provided to LLMs?
  8. What information should feedback content convey?
  9. What is missing?
  10. Feedback in Learning Sciences
  11. What is feedback?
  12. What constitutes effective feedback?
  13. Applicability
  14. Learner Regulation
  15. Personalization
  16. ...and 56 more sections

Figures (3)

  • Figure 1: Connecting feedback research in NLP to foundations of feedback in the Learning Sciences.
  • Figure 2: FELT, the feedback ecosystem adapted for LLMs. The feedback's characteristics, the errors, the task, and the learner all influence both the feedback and its subsequent processing by the learner. Various interactions occur between the task, the learner, and the feedback, such as the task complexity and prior knowledge of the learner affecting the timing of the feedback. Appendix \ref{['app:felt_interactions']} presents a more comprehensive overview of the interactions present in the framework.
  • Figure 3: Visual summary of several future research directions motivated by FELT, for each of its dimensions and across several research axes. The textual counterparts of each bubble are highlighted in their respective color (green, blue, purple, orange) in the main body of Section \ref{['sec:feedback_applications']}.