Cooperation Through Indirect Reciprocity in Child-Robot Interactions

TL;DR

This paper demonstrates that indirect reciprocity can operate in hybrid groups containing children and robots: children adjust their cooperation based on a robot's observed prior cooperative behavior. It combines an empirical LEGO-based Stag Hunt study with a theoretical multi-armed bandit framework to explore how robots can learn to cooperate from human strategies, revealing that learning success is highly contingent on payoff structures and algorithm choice. The findings emphasize the potential and limits of designing social robots that foster prosocial behavior in child-centred settings, while cautioning about generalizability and long-term effects. Overall, the work bridges experimental psychology and reinforcement-learning-inspired models to inform the development of cooperative human-robot systems in education and everyday contexts.

Abstract

Social interactions increasingly involve artificial agents, such as conversational or collaborative bots. Understanding trust and prosociality in these settings is fundamental to improve human-AI teamwork. Research in biology and social sciences has identified mechanisms to sustain cooperation among humans. Indirect reciprocity (IR) is one of them. With IR, helping someone can enhance an individual's reputation, nudging others to reciprocate in the future. Transposing IR to human-AI interactions is however challenging, as differences in human demographics, moral judgements, and agents' learning dynamics can affect how interactions are assessed. To study IR in human-AI groups, we combine laboratory experiments and theoretical modelling. We investigate whether 1) indirect reciprocity can be transposed to children-robot interactions; 2) artificial agents can learn to cooperate given children's strategies; and 3) how differences in learning algorithms impact human-AI cooperation. We find that IR extends to children and robots solving coordination dilemmas. Furthermore, we observe that the strategies revealed by children provide a sufficient signal for multi-armed bandit algorithms to learn cooperative actions. Beyond the experimental scenarios, we observe that cooperating through multi-armed bandit algorithms is highly dependent on the strategies revealed by humans.

Figures (4)

• Figure 1: A Experimental setup: Children and robot play a social dilemma where cooperation is socially beneficial yet risky. If both individuals cooperate, they both receive 2 extra bricks (here used as a unit of payoff), however unilaterally cooperating leads to 0 payoff to the cooperator. The 2 additional bricks received are here referred to as the extra benefit of mutual cooperation ($b=2$). B The chain interaction model stemming from the setup presented in Figure \ref{['fig:overview_experiment']}A, and motivating our simulation experiments. The artificial agent repeatedly plays against an individual, and is observed by another player. After the interaction, the artificial agent plays with the individual previously acting as observer and a new observer joins.
• Figure 2: Empirical results from the Child-Robot experiments. The bars represent the percentage of children that cooperated with the robot after observing a cooperative (left bar) or non-cooperative (right bar) action by the robot. We observe a statistically significant difference between the two conditions, suggesting that children are more likely to cooperate with a robot that was previously cooperative than with a robot that was non-cooperative. Error bars represent the standard error. ** $p<.001$.
• Figure 3: Simulation results: cooperation index, $I$ when varying the extra benefits of mutual cooperation, $b$, for the three algorithms we utilize to simulate the robot learning to play with children. The dashed line, at $b=2$, indicates the benefit of mutual cooperation used in the experimental setting (where 2 additional LEGO pieces are given if mutual cooperation occurs). We observe that while cooperation is common and stable under $\epsilon$-greedy and UCB1 at a moderate $b > 1$, Thompson Sampling requires a greater benefit for cooperation to stabilize. Parameters used: $p =\hat{p} = 0.81$, $q =\hat{q} = 0.36$, $\epsilon = 1/128$, $c = 4$.
• Figure 4: Cooperation index, $I$, varying $p$ and $q$, for the three learning algorithms used. $p$ and $q$ represent, respectively, the probability that an agent cooperates with an opponent, after having observed the same opponent cooperating or defecting with a third-party. The red cross indicates the experimental value found ($p=\hat{p}=0.81$, $q=\hat{q}=0.36$). Labels indicate the pure strategy used by humans at that point: Always Trust, AT ($p=1,q=1$), which always cooperates; Never Trust, NT ($0,0$), which always defects; Trust Cooperators, TC ($1,0$), which only cooperates if the algorithm previously cooperated; and Trust Defectors, TD ($0, 1$), which only cooperates after observing a defection. The vertical line in $\epsilon$-greedy indicates its theoretical threshold for cooperation to dominate defection (see Methods). We observe that each algorithm has a different sensitivity to $p$ and $q$, resulting in distinct regions where each algorithm outperforms others. Parameters used: $b=2$, $\epsilon = 1/128$, $c = 4$.