EcoSense: Energy-Efficient Intelligent Sensing for In-Shore Ship Detection through Edge-Cloud Collaboration

TL;DR

This work tackles inshore marine object detection (IMOD) under energy and bandwidth constraints by proposing an edge-cloud collaborative sensing framework that splits localization and fine-grained classification, routing difficult cases to a cloud back-end. The edge front-end performs lightweight localization and a difficulty estimation to decide edge versus cloud classification, while the cloud back-end employs a Swin-Transformer with a Graph Convolutional Network, enhanced by Background Suppression and High-temperature Refinement to improve robustness and multi-scale context. The approach yields substantial improvements in detection accuracy ($mAP@50$) and dramatic reductions in data transmission ($95.43\%$ and $92.83\%$) and energy use ($27.3\%$ and $31.6\%$) compared to centralized baselines, validated on SeaShips and SMD-Plus datasets and through real-world drone experiments. The results indicate practical viability for scalable, energy-efficient coastal surveillance and autonomous maritime operations across diverse embedded platforms.

Abstract

Detecting marine objects inshore presents challenges owing to algorithmic intricacies and complexities in system deployment. We propose a difficulty-aware edge-cloud collaborative sensing system that splits the task into object localization and fine-grained classification. Objects are classified either at the edge or within the cloud, based on their estimated difficulty. The framework comprises a low-power device-tailored front-end model for object localization, classification, and difficulty estimation, along with a transformer-graph convolutional network-based back-end model for fine-grained classification. Our system demonstrates superior performance (mAP@0.5 +4.3%}) on widely used marine object detection datasets, significantly reducing both data transmission volume (by 95.43%) and energy consumption (by 72.7%}) at the system level. We validate the proposed system across various embedded system platforms and in real-world scenarios involving drone deployment.

Figures (6)

• Figure 1: Comparison of different framework design. (a). Centralized manner. The model is deployed on the cloud, leading to high pressure on bandwidth and the cloud. (b). Distributed manner. The model is deployed near each sensor. The inference is implemented locally and only the result is sent out. (c). Our edge-cloud collaboration framework. A powerful classifier is deployed on the cloud. Each edge device has an object localizer and a lightweight classifier. Depending on the difficulty of the ROI, the detected objects are classified on the edge or transmitted to the cloud for classification. The framework makes a good balance between edge device battery life, accuracy, and data transmission volume.
• Figure 2: Front-end model(a). Schematic of front-end model(b). Modified Ghost module. $\Phi(\cdot)$ denotes the linear projection in the original Ghost module. Red arrows show the path of the Squeeze-Excitation operation, and blue arrows show Coordinate Attention ($F_{sq}(\cdot)$ denotes squeeze operation, $F_{ex}(\cdot)$ denotes excitation operation). $\odot$ denotes element-wise multiplication.
• Figure 3: Schematic of attention module. $F_{sq}(\cdot)$ denotes squeeze operation, $F_{ex}(\cdot)$ denotes excitation operation, $\odot$ denotes element-wise multiplication.
• Figure 4: Normalized energy consumption breakdowns of the conventional centralized manner and our system.
• Figure 5: Confusion Matrices of our back-end model. s1: bulk cargo carrier, s2: container ship, s3: fishing boat, s4: general cargo ship, s5: ore carrier, s6: passenger ship, c1: ferry, c2: buoy, c3: vessel ship, c4: boat, c5: kayak, c6: sail boat, c7: others.