Transformer Guided Coevolution: Improved Team Selection in Multiagent Adversarial Team Games

TL;DR

BERTeam introduces a Transformer-based method for team selection in multiagent adversarial games by reframing team formation as a masked sequence completion task. It is trained alongside coevolutionary deep RL, enabling simultaneous optimization of team composition and individual policies, and can condition on opponent observations. Empirical evaluation in Marine Capture-the-Flag via fixed-policy and coevolutionary setups shows BERTeam learns non-trivial, high-performing teams and outperforms baseline team-selection methods such as MCAA; agent embeddings reveal learned behavioral similarities that support robust generalization against unseen opponents. The work demonstrates a practical, scalable approach to dynamic team formation with strong potential for real-world multiagent coordination and competition.

Abstract

We consider the problem of team selection within multiagent adversarial team games. We propose BERTeam, a novel algorithm that uses a transformer-based deep neural network with Masked Language Model training to select the best team of players from a trained population. We integrate this with coevolutionary deep reinforcement learning, which trains a diverse set of individual players to choose from. We test our algorithm in the multiagent adversarial game Marine Capture-The-Flag, and find that BERTeam learns non-trivial team compositions that perform well against unseen opponents. For this game, we find that BERTeam outperforms MCAA, an algorithm that similarly optimizes team selection.

Figures (6)

• Figure 1: BERTeam's core, a transformer network
• Figure 2: Training of BERTeam alongside coevolutionary RL
• Figure 3: Aquaticus game, and its simulated version
• Figure 4: BERTeam distributions throughout training, sorted by probability (largest on bottom)
• Figure 5: BERTeam learned distribution on trained agents