Achieving Optimal Tissue Repair Through MARL with Reward Shaping and Curriculum Learning

TL;DR

Problem: optimize tissue repair using decentralized bioagents. Approach: integrate stochastic reaction-diffusion signaling, neural-like electrochemical communication with Hebbian plasticity, and a biologically informed reward function with curriculum learning, implemented in a MARL framework with a centralized critic. The objective uses a multi-objective reward $R_k(t) = R_{ext}(t) + β_1 r_{chem} + β_2 r_{neu sync}(t) + β_3 r_{robust}(t)$ and a curriculum schedule $\\mathcal{T}(t) = \\mathcal{T}_0 + (\\mathcal{T}_f-\\mathcal{T}_0) \\min(t/n,1)$. In silico experiments reveal emergent repair strategies like pulsatile growth factor secretion and coordinated spatial activity, suggesting potential for intelligent biohybrid regenerative therapies. Limitations include in vitro/vivo validation and extension to 3D scaffolds; future work may address temporal credit assignment and real-time bio-signal integration.

Abstract

In this paper, we present a multi-agent reinforcement learning (MARL) framework for optimizing tissue repair processes using engineered biological agents. Our approach integrates: (1) stochastic reaction-diffusion systems modeling molecular signaling, (2) neural-like electrochemical communication with Hebbian plasticity, and (3) a biologically informed reward function combining chemical gradient tracking, neural synchronization, and robust penalties. A curriculum learning scheme guides the agent through progressively complex repair scenarios. In silico experiments demonstrate emergent repair strategies, including dynamic secretion control and spatial coordination.

Figures (4)

• Figure 1: Top Left: Chemical gradients at $t=1,5,9$ s. Top Right: Probability distribution of agents. Bottom Left: Chemical signal tracking. Bottom Right: Heatmap showing the spatiotemporal evolution of the concentration gradient.
• Figure 2: Collective secretion dynamics evolution by agents.
• Figure 3: Graphs associated with MARL training. Top Left: The standard deviations of the actions taken by the agents over time. Top Middle: The variation of the action entropy and average reward over time. Top Right: Behavior of the reward function \ref{['reward function']} over time. Bottom Left: The frequency of the states visited by the agent. Bottom Middle: Evolution of the neural connection weights over time. Bottom Right: The average maximum action-value function ($Q$-values) over time.
• Figure 4: Curriculum learning progression showing a linear curriculum trajectory. By gradually increasing the difficulty, the agents are better equipped to adapt to complex scenarios, similar to how sentient systems develop and learn.