Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Oracle-Guided Soft Shielding for Safe Move Prediction in Chess

Prajit T Rajendran, Fabio Arnez, Huascar Espinoza, Agnes Delaborde, Chokri Mraidha

TL;DR

This work proposes Oracle-Guided Soft Shielding (OGSS), a simple yet effective framework for safer decision-making, enabling safe exploration by learning a probabilistic safety model from oracle feedback in an imitation learning setting.

Abstract

In high stakes environments, agents relying purely on imitation learning or reinforcement learning often struggle to avoid safety-critical errors during exploration. Existing reinforcement learning approaches for environments such as chess require hundreds of thousands of episodes and substantial computational resources to converge. Imitation learning, on the other hand, is more sample efficient but is brittle under distributional shift and lacks mechanisms for proactive risk avoidance. In this work, we propose Oracle-Guided Soft Shielding (OGSS), a simple yet effective framework for safer decision-making, enabling safe exploration by learning a probabilistic safety model from oracle feedback in an imitation learning setting. Focusing on the domain of chess, we train a model to predict strong moves based on past games, and separately learn a blunder prediction model from Stockfish evaluations to estimate the tactical risk of each move. During inference, the agent first generates a set of candidate moves and then uses the blunder model to determine high-risk options, and uses a utility function combining the predicted move likelihood from the policy model and the blunder probability to select actions that strike a balance between performance and safety. This enables the agent to explore and play competitively while significantly reducing the chance of tactical mistakes. Across hundreds of games against a strong chess engine, we compare our approach with other methods in the literature, such as action pruning, SafeDAgger, and uncertainty-based sampling. Our results demonstrate that OGSS variants maintain a lower blunder rate even as the agent's exploration ratio is increased by several folds, highlighting its ability to support broader exploration without compromising tactical soundness.

Oracle-Guided Soft Shielding for Safe Move Prediction in Chess

TL;DR

This work proposes Oracle-Guided Soft Shielding (OGSS), a simple yet effective framework for safer decision-making, enabling safe exploration by learning a probabilistic safety model from oracle feedback in an imitation learning setting.

Abstract

In high stakes environments, agents relying purely on imitation learning or reinforcement learning often struggle to avoid safety-critical errors during exploration. Existing reinforcement learning approaches for environments such as chess require hundreds of thousands of episodes and substantial computational resources to converge. Imitation learning, on the other hand, is more sample efficient but is brittle under distributional shift and lacks mechanisms for proactive risk avoidance. In this work, we propose Oracle-Guided Soft Shielding (OGSS), a simple yet effective framework for safer decision-making, enabling safe exploration by learning a probabilistic safety model from oracle feedback in an imitation learning setting. Focusing on the domain of chess, we train a model to predict strong moves based on past games, and separately learn a blunder prediction model from Stockfish evaluations to estimate the tactical risk of each move. During inference, the agent first generates a set of candidate moves and then uses the blunder model to determine high-risk options, and uses a utility function combining the predicted move likelihood from the policy model and the blunder probability to select actions that strike a balance between performance and safety. This enables the agent to explore and play competitively while significantly reducing the chance of tactical mistakes. Across hundreds of games against a strong chess engine, we compare our approach with other methods in the literature, such as action pruning, SafeDAgger, and uncertainty-based sampling. Our results demonstrate that OGSS variants maintain a lower blunder rate even as the agent's exploration ratio is increased by several folds, highlighting its ability to support broader exploration without compromising tactical soundness.
Paper Structure (14 sections, 3 figures, 1 table)

This paper contains 14 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Method
  4. Variants of OGSS
  5. Experimental setup
  6. Models & Training Details
  7. Baselines
  8. Evaluation Metrics
  9. Results and discussion
  10. Effectiveness of Blunder Shielding
  11. Safety vs. Exploration Trade-off
  12. Centipawn Drop and Move Quality
  13. Alpha-Value Trade-Off
  14. Conclusion

Figures (3)

  • Figure 1: Overview of the proposed Oracle-Guided Shielding framework. (a) The agent is trained via imitation learning on historical chess games to predict the most likely expert move given any game state. This policy model learns to approximate strong expert play by mapping board positions to high-quality next moves. (b) During the exploratory learning phase, the agent interacts with an environment and receives feedback from an oracle (Stockfish) to learn a blunder prediction model. (c) At inference time, the move prediction model and the blunder model work jointly to generate high-quality, low-risk moves.
  • Figure 2: Plot of exploration ratio as a function of blunder rate. Methods that combine low blunder rates with high exploration ratios are considered optimal for safe exploration.
  • Figure 3: Impact of Alpha Value on Blunder Percentage and Median Centipawn Drop. This plot illustrates the relationship between increasing alpha values and two performance metrics: blunder percentage (trending upwards, shown in red) and median centipawn drop (trending downwards, shown in blue).