Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Future of Open Human Feedback

Shachar Don-Yehiya, Ben Burtenshaw, Ramon Fernandez Astudillo, Cailean Osborne, Mimansa Jaiswal, Tzu-Sheng Kuo, Wenting Zhao, Idan Shenfeld, Andi Peng, Mikhail Yurochkin, Atoosa Kasirzadeh, Yangsibo Huang, Tatsunori Hashimoto, Yacine Jernite, Daniel Vila-Suero, Omri Abend, Jennifer Ding, Sara Hooker, Hannah Rose Kirk, Leshem Choshen

TL;DR

An open ecosystem for human feedback on large language models is explored, drawing from peer-production, open-source and citizen-science practices, and addressing key challenges to establish sustainable feedback loops between users and specialized models.

Abstract

Human feedback on conversations with language language models (LLMs) is central to how these systems learn about the world, improve their capabilities, and are steered toward desirable and safe behaviors. However, this feedback is mostly collected by frontier AI labs and kept behind closed doors. In this work, we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for AI. We first look for successful practices in peer production, open source, and citizen science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the center of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholders community of model trainers and feedback providers to support a general open feedback pool.

The Future of Open Human Feedback

TL;DR

An open ecosystem for human feedback on large language models is explored, drawing from peer-production, open-source and citizen-science practices, and addressing key challenges to establish sustainable feedback loops between users and specialized models.

Abstract

Human feedback on conversations with language language models (LLMs) is central to how these systems learn about the world, improve their capabilities, and are steered toward desirable and safe behaviors. However, this feedback is mostly collected by frontier AI labs and kept behind closed doors. In this work, we bring together interdisciplinary experts to assess the opportunities and challenges to realizing an open ecosystem of human feedback for AI. We first look for successful practices in peer production, open source, and citizen science communities. We then characterize the main challenges for open human feedback. For each, we survey current approaches and offer recommendations. We end by envisioning the components needed to underpin a sustainable and open human feedback ecosystem. In the center of this ecosystem are mutually beneficial feedback loops, between users and specialized models, incentivizing a diverse stakeholders community of model trainers and feedback providers to support a general open feedback pool.
Paper Structure (15 sections, 1 figure, 1 table)

This paper contains 15 sections, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Defining Open Human Feedback
  3. Key Lessons from Peer Production and Open Source
  4. Peer Production
  5. Open source software
  6. Challenges and Opportunities of Realizing Open Human Feedback
  7. Incentives
  8. Effort and Involvement
  9. Expert Contributions
  10. Linguistic and Cultural Diversity
  11. Adaptable and Dynamic Feedback
  12. Privacy and Data Protection
  13. Legal and Ownership
  14. Visions of Successful and Sustainable Open Feedback Ecosystems
  15. Conclusion

Figures (1)

  • Figure 1: A sustainable open feedback ecosystem at a glance. Users from diverse domains and expertise chat and give feedback to open Large Language Models within the feedback ecosystem . The conversations and feedback are shared through an open pool of data , which can be utilized by AI builders in the building ecosystem to develop and share the next generation of Large Language Models within the chat and feedback ecosystem.