Reinforcing Competitive Multi-Agents for Playing 'So Long Sucker'

TL;DR

This work proposes So Long Sucker (SLS) as a negotiation-aware MARL benchmark and provides an open computational framework with GUI support. It formalizes SLS as a sequential zero-sum MARL environment and evaluates classical value-based DRL baselines (DQN, DDQN, Dueling DQN) under centralized training with a shared replay buffer. Results show agents learn the game rules and achieve roughly half of the maximum reward, with convergence around 2000 episodes and occasional illegal moves, highlighting both potential and limitations of standard DRL for complex coalition dynamics. The study motivates future exploration of actor–critic and coalition-aware methods to better capture long-horizon strategic reasoning, deception, and dynamic alliances in multi-agent settings.

Abstract

This paper investigates the strategy game So Long Sucker (SLS) as a novel benchmark for multi-agent reinforcement learning (MARL). Unlike traditional board or video game testbeds, SLS is distinguished by its coalition formation, strategic deception, and dynamic elimination rules, making it a uniquely challenging environment for autonomous agents. We introduce the first publicly available computational framework for SLS, complete with a graphical user interface and benchmarking support for reinforcement learning algorithms. Using classical deep reinforcement learning methods (e.g., DQN, DDQN, and Dueling DQN), we train self-playing agents to learn the rules and basic strategies of SLS. Experimental results demonstrate that, although these agents achieve roughly half of the maximum attainable reward and consistently outperform random baselines, they require long training horizons (~2000 games) and still commit occasional illegal moves, highlighting both the promise and limitations of classical reinforcement learning. Our findings establish SLS as a negotiation-aware benchmark for MARL, opening avenues for future research that integrates game-theoretic reasoning, coalition-aware strategies, and advanced reinforcement learning architectures to better capture the social and adversarial dynamics of complex multi-agent games.

Figures (1)

• Figure 1: Performance comparison of different agents: (a) DQN: reward vs. episodes, (b) DQN: steps vs. episodes, (c) DDQN: reward vs. episodes, (d) DDQN: steps vs. episodes, (e) Dueling DQN: reward vs. episodes, (f) Dueling DQN: steps vs. episodes.