Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours

Nikhil Churamani, Saksham Checker, Fethiye Irmak Dogan, Hao-Tien Lewis Chiang, Hatice Gunes

TL;DR

This work tackles learning socially appropriate robot behaviours in dynamic, privacy-conscious federated settings by introducing FedRoot, which aggregates only feature-extraction layers $R=f_{\theta_R}(x)$ while keeping task-specific layers $T=h_{\theta_T}(R)$ local. It further extends to Federated Latent Generative Replay (FedLGR), adding a local Generator to perform latent feature rehearsal and mitigate forgetting via $R'$ samples, with a training regime that enforces $\eta_R \ll \eta_T$. Evaluations on the MANNERS-DB living-room dataset show that FedRoot-based methods deliver competitive performance while dramatically reducing compute resources (up to 86% CPU and 92% GPU usage reductions), and FedLGR consistently outperforms FedAvg-based baselines in Federated Continual Learning, especially under higher client counts. The findings underscore the practical value of resource-efficient, privacy-preserving federated approaches for socially intelligent robots operating in households and similar human-centric environments.

Abstract

It is critical for robots to explore Federated Learning (FL) settings where several robots, deployed in parallel, can learn independently while also sharing their learning with each other. This collaborative learning in real-world environments requires social robots to adapt dynamically to changing and unpredictable situations and varying task settings. Our work contributes to addressing these challenges by exploring a simulated living room environment where robots need to learn the social appropriateness of their actions. First, we propose Federated Root (FedRoot) averaging, a novel weight aggregation strategy which disentangles feature learning across clients from individual task-based learning. Second, to adapt to challenging environments, we extend FedRoot to Federated Latent Generative Replay (FedLGR), a novel Federated Continual Learning (FCL) strategy that uses FedRoot-based weight aggregation and embeds each client with a generator model for pseudo-rehearsal of learnt feature embeddings to mitigate forgetting in a resource-efficient manner. Our results show that FedRoot-based methods offer competitive performance while also resulting in a sizeable reduction in resource consumption (up to 86% for CPU usage and up to 72% for GPU usage). Additionally, our results demonstrate that FedRoot-based FCL methods outperform other methods while also offering an efficient solution (up to 84% CPU and 92% GPU usage reduction), with FedLGR providing the best results across evaluations.

Feature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours

TL;DR

This work tackles learning socially appropriate robot behaviours in dynamic, privacy-conscious federated settings by introducing FedRoot, which aggregates only feature-extraction layers while keeping task-specific layers local. It further extends to Federated Latent Generative Replay (FedLGR), adding a local Generator to perform latent feature rehearsal and mitigate forgetting via samples, with a training regime that enforces . Evaluations on the MANNERS-DB living-room dataset show that FedRoot-based methods deliver competitive performance while dramatically reducing compute resources (up to 86% CPU and 92% GPU usage reductions), and FedLGR consistently outperforms FedAvg-based baselines in Federated Continual Learning, especially under higher client counts. The findings underscore the practical value of resource-efficient, privacy-preserving federated approaches for socially intelligent robots operating in households and similar human-centric environments.

Abstract

It is critical for robots to explore Federated Learning (FL) settings where several robots, deployed in parallel, can learn independently while also sharing their learning with each other. This collaborative learning in real-world environments requires social robots to adapt dynamically to changing and unpredictable situations and varying task settings. Our work contributes to addressing these challenges by exploring a simulated living room environment where robots need to learn the social appropriateness of their actions. First, we propose Federated Root (FedRoot) averaging, a novel weight aggregation strategy which disentangles feature learning across clients from individual task-based learning. Second, to adapt to challenging environments, we extend FedRoot to Federated Latent Generative Replay (FedLGR), a novel Federated Continual Learning (FCL) strategy that uses FedRoot-based weight aggregation and embeds each client with a generator model for pseudo-rehearsal of learnt feature embeddings to mitigate forgetting in a resource-efficient manner. Our results show that FedRoot-based methods offer competitive performance while also resulting in a sizeable reduction in resource consumption (up to 86% for CPU usage and up to 72% for GPU usage). Additionally, our results demonstrate that FedRoot-based FCL methods outperform other methods while also offering an efficient solution (up to 84% CPU and 92% GPU usage reduction), with FedLGR providing the best results across evaluations.
Paper Structure (18 sections, 2 equations, 4 figures, 4 tables, 1 algorithm)

This paper contains 18 sections, 2 equations, 4 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Socially Appropriate Robot Behaviours
  4. Federated Learning
  5. Federated Continual Learning
  6. Methodology
  7. FedRoot: Federated Root for Feature Aggregation
  8. Root (R)
  9. Top (T)
  10. FedLGR: Federated Latent Generative Replay
  11. Experiments
  12. MANNERS-DB Dataset
  13. Implementation Details
  14. Evaluation Metrics
  15. Results
  16. ...and 3 more sections

Figures (4)

  • Figure 1: FL (left): Local models are aggregated on the server without sharing data. FCL (right): Individual robots incrementally learn tasks, periodically sharing model updates with each other.
  • Figure 2: MANNERS-DB: A Living Room scenario with a robot tjomsland2022mind.
  • Figure 3: FedRoot: Local model split into (i) Root for feature extraction and (ii) Top for task-based learning. Only model Root is aggregated across clients while Top remains local.
  • Figure 4: FedLGR: Local model split into (i) Root for feature extraction, (ii) Top for task-based learning and (iii) Generator for pseudo-rehearsal. Only Root is aggregated across clients while Top and Generator remain local.