Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SEG-JPEG: Simple Visual Semantic Communications for Remote Operation of Automated Vehicles over Unreliable Wireless Networks

Sebastian Donnelly, Ruth Anderson, George Economides, James Broughton, Peter Ball, Alexander Rast, Andrew Bradley

TL;DR

This paper tackles the challenge of remotely operating automated vehicles over unreliable public networks by introducing SEG-JPEG, a semantic layering imagery technique. It combines on-board object detection (YOLO11x-seg) with saliency-driven greyscale JPEG compression, embedding road-user segmentations into limited-color shades and recoloring at the operator end to preserve critical situational cues. The approach achieves up to a 50% reduction in data rate while maintaining low latency, reporting a median glass-to-glass latency of ~198 ms at 500 kbit/s on 4G networks, and demonstrating robust operation where traditional h.264 streaming struggles. The work indicates SEG-JPEG can significantly expand the feasible operational domain for remote vehicle control on public networks and suggests a path toward adaptive, resilient remote operation systems.

Abstract

Remote Operation is touted as being key to the rapid deployment of automated vehicles. Streaming imagery to control connected vehicles remotely currently requires a reliable, high throughput network connection, which can be limited in real-world remote operation deployments relying on public network infrastructure. This paper investigates how the application of computer vision assisted semantic communication can be used to circumvent data loss and corruption associated with traditional image compression techniques. By encoding the segmentations of detected road users into colour coded highlights within low resolution greyscale imagery, the required data rate can be reduced by 50 \% compared with conventional techniques, while maintaining visual clarity. This enables a median glass-to-glass latency of below 200ms even when the network data rate is below 500kbit/s, while clearly outlining salient road users to enhance situational awareness of the remote operator. The approach is demonstrated in an area of variable 4G mobile connectivity using an automated last-mile delivery vehicle. With this technique, the results indicate that large-scale deployment of remotely operated automated vehicles could be possible even on the often constrained public 4G/5G mobile network, providing the potential to expedite the nationwide roll-out of automated vehicles.

SEG-JPEG: Simple Visual Semantic Communications for Remote Operation of Automated Vehicles over Unreliable Wireless Networks

TL;DR

This paper tackles the challenge of remotely operating automated vehicles over unreliable public networks by introducing SEG-JPEG, a semantic layering imagery technique. It combines on-board object detection (YOLO11x-seg) with saliency-driven greyscale JPEG compression, embedding road-user segmentations into limited-color shades and recoloring at the operator end to preserve critical situational cues. The approach achieves up to a 50% reduction in data rate while maintaining low latency, reporting a median glass-to-glass latency of ~198 ms at 500 kbit/s on 4G networks, and demonstrating robust operation where traditional h.264 streaming struggles. The work indicates SEG-JPEG can significantly expand the feasible operational domain for remote vehicle control on public networks and suggests a path toward adaptive, resilient remote operation systems.

Abstract

Remote Operation is touted as being key to the rapid deployment of automated vehicles. Streaming imagery to control connected vehicles remotely currently requires a reliable, high throughput network connection, which can be limited in real-world remote operation deployments relying on public network infrastructure. This paper investigates how the application of computer vision assisted semantic communication can be used to circumvent data loss and corruption associated with traditional image compression techniques. By encoding the segmentations of detected road users into colour coded highlights within low resolution greyscale imagery, the required data rate can be reduced by 50 \% compared with conventional techniques, while maintaining visual clarity. This enables a median glass-to-glass latency of below 200ms even when the network data rate is below 500kbit/s, while clearly outlining salient road users to enhance situational awareness of the remote operator. The approach is demonstrated in an area of variable 4G mobile connectivity using an automated last-mile delivery vehicle. With this technique, the results indicate that large-scale deployment of remotely operated automated vehicles could be possible even on the often constrained public 4G/5G mobile network, providing the potential to expedite the nationwide roll-out of automated vehicles.
Paper Structure (15 sections, 9 figures)

This paper contains 15 sections, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. SEG-JPEG: Semantic Layering Imagery Technique
  5. Object Detection
  6. Salient Compression
  7. Operator Reconstruction
  8. Hardware Setup
  9. Experiments
  10. Results And Evaluation
  11. Network Saturation And Latency
  12. Glass-to-Glass Performance
  13. Imagery Quality
  14. Discussion
  15. Conclusion

Figures (9)

  • Figure 1: Augmented greyscale imagery presented to a remote operator, highlighting, vehicles (blue), humans (red) and vehicles (blue).
  • Figure 2: Frame distortion in h.264 imagery caused by a reduced quality mobile network connection.
  • Figure 3: Augmenting highly compressed imagery with road user segmentations.
  • Figure 4: Highly compressed greyscale JPEG imagery with road users highlighted to enhance situational awareness. Red indicates a person, green a bicycle, blue a road vehicle.
  • Figure 5: Green-Log Mobile Delivery Hub (top), remote operation system components installed in the vehicle (bottom).
  • ...and 4 more figures