Mxplainer: Explain and Learn Insights by Imitating Mahjong Agents

TL;DR

Mxplainer targets the interpretability gap in high-performance Mahjong agents by introducing a parameterized search-based framework that can be converted into a differentiable network. By learning the framework’s interpretable parameters from black-box agent data, it can mimic actions with high fidelity while enabling step-by-step explanations of decisions and goal preferences. The approach demonstrates strong top-3 action prediction accuracy for both AI and human data and provides insights into fan and tile preferences, with ablations showing the critical role of draw modeling and goal caps. This work offers a practical pathway to transparent, trust-worthy AI in complex, imperfect-information domains and outlines extensions to broader settings and multimodal explanation modalities.

Abstract

People need to internalize the skills of AI agents to improve their own capabilities. Our paper focuses on Mahjong, a multiplayer game involving imperfect information and requiring effective long-term decision-making amidst randomness and hidden information. Through the efforts of AI researchers, several impressive Mahjong AI agents have already achieved performance levels comparable to those of professional human players; however, these agents are often treated as black boxes from which few insights can be gleaned. This paper introduces Mxplainer, a parameterized search algorithm that can be converted into an equivalent neural network to learn the parameters of black-box agents. Experiments on both human and AI agents demonstrate that Mxplainer achieves a top-three action prediction accuracy of over 92% and 90%, respectively, while providing faithful and interpretable approximations that outperform decision-tree methods (34.8% top-three accuracy). This enables Mxplainer to deliver both strategy-level insights into agent characteristics and actionable, step-by-step explanations for individual decisions.

Figures (6)

• Figure S1: Comparison between current situation of black-box Mahjong agents and Mahjong agents with Mxplainer. (a). Raw action output without explanations. (b). Mxplainer explains possible reasons behind actions.
• Figure S2: Basics of Mahjong. (A). All the Mahjong tiles. There are four identical copies for suited tiles and honored tiles, and one copy for each flower tile. (B). Examples of Chow, Pung, and Kong. Note that only suited tiles are available for Chow. (C). Example of the general winning pattern.
• Figure S3: An overview of Mxplainer. Search-based framework $F$ is a manually engineered, domain-specific, parameterized template. A component of $F$ can be converted into an equivalent network for supervised learning, where parameters of $F$ serve as neurons. Target agents $\Psi$ are black-box agents to be analyzed, and they can be approximated by the converted $F$.
• Figure S4: Components of framework $F$. Search Component $SC$ uses Goal Proposer $P$ for goal search. Calculation Component $CC$ consists of Value Calculator $C$ and Decision Selector $DS$, and it is responsible for action calculations. The least useful tile $t$ can be considered as the tile that appears least frequently among the most probable goals. Consequently, removing this tile has the least impact on the expected value.
• Figure S5: Top-1 and top-3 accuracy of $\hat{\psi_2}$ when the number of goals is 16/32/64/128.

Theorems & Definitions (1)

• Lemma A1