Optimizing Energy and Data Collection in UAV-aided IoT Networks using Attention-based Multi-Objective Reinforcement Learning

TL;DR

This work tackles UAV-enabled data collection in urban IoT settings under a multi-objective trade-off between data yield and energy consumption. It introduces MOSAC-ATT, an attention-based MORL framework that conditions policies on a preference vector and uses a set-based state representation to achieve permutation invariance and strong generalization without channel knowledge. The approach delivers superior multi-objective performance, sample efficiency, and a compact model compared to map-based baselines, aided by interpretable attention maps that reveal how the model focuses on relevant features under different preferences. The results demonstrate robust zero-shot generalization to unseen device counts and configurations, highlighting practical relevance for real-world UAV data harvesting with changing mission parameters. Overall, MOSAC-ATT advances adaptive, explainable MORL for complex, dynamic wireless systems and sets the stage for future extensions to multi-UAV coordination and partial observability.

Abstract

Due to their adaptability and mobility, Unmanned Aerial Vehicles (UAVs) are becoming increasingly essential for wireless network services, particularly for data harvesting tasks. In this context, Artificial Intelligence (AI)-based approaches have gained significant attention for addressing UAV path planning tasks in large and complex environments, bridging the gap with real-world deployments. However, many existing algorithms suffer from limited training data, which hampers their performance in highly dynamic environments. Moreover, they often overlook the inherently multi-objective nature of the task, treating it in an overly simplistic manner. To address these limitations, we propose an attention-based Multi-Objective Reinforcement Learning (MORL) architecture that explicitly handles the trade-off between data collection and energy consumption in urban environments, even without prior knowledge of wireless channel conditions. Our method develops a single model capable of adapting to varying trade-off preferences and dynamic scenario parameters without the need for fine-tuning or retraining. Extensive simulations show that our approach achieves substantial improvements in performance, model compactness, sample efficiency, and most importantly, generalization to previously unseen scenarios, outperforming existing RL solutions.

Figures (7)

• Figure 1: Different approaches in multi-objective learning.
• Figure 2: Map processing on matrix $\boldsymbol{E}$ with $l =10$ and $g = 5$.
• Figure 3: Attention-based DNN architecture used as RL agent. The projection layers $\phi$ are linear maps without positional encoding.
• Figure 4: Multi-objective performance averaged across $3$ seeds over training steps and Pareto fronts of final saved models.
• Figure 5: Examples of MOSAC-ATT trajectories for three different preferences in the RD (a, b, c) and RB (d, e, f) scenarios.