Is decidability of the Submonoid Membership Problem closed under finite extensions?
Doron Shafrir
TL;DR
This work investigates whether the submonoid membership problem ($SMON$) is closed under finite extensions by linking it to rational subset membership ($RAT$) through reductions to products with virtually Abelian groups $H$. The authors develop a general framework of rational sections: any rational subset $R\subseteq G$ can be realized as a section of a submonoid $M\le G\times H$ with $H$ virtually Abelian, using concrete constructions with $H$ (e.g., rotations on a lattice via a semidirect product) and wreath products to encode rational descriptions. They prove a key dichotomy: either the conjectured nonuniformity in how $RAT$ reduces to $SMON$ under finite extensions fails (supporting the conjecture that there exists a nilpotent scenario where $RAT$ remains undecidable even when $SMON$ is decidable on index-2 subgroups), or there exists a group where fixed-rational-subset membership is decidable while the unrestricted rational subset problem is not; the work also highlights limits on uniform reductions, showing that $RAT$ cannot always be captured by a single $H$ independent of $R$, and that such uniformities would imply unlikely decidability consequences for nilpotent groups. Overall, the paper provides a structural bridge between $RAT$ and $SMON$ in finite extensions and maps the boundary between decidability landscapes across nilpotent and virtually abelian extensions.
Abstract
We show that the rational subset membership problem in $G$ can be reduced to the submonoid membership problem in $G{\times}H$ where $H$ is virtually Abelian. We use this to show that there is no algorithm reducing submonoid membership to a finite index subgroup uniformly for all virtually nilpotent groups. We also provide evidence towards the existence of a group $G$ with a subgroup $H<G$ of index 2, such that the submonoid membership problem is decidable in $H$ but not in $G$.