CyberCortex.AI: An AI-based Operating System for Autonomous Robotics and Complex Automation

TL;DR

CyberCortex.AI addresses the need for an AI-centric robotics OS capable of coordinating perception-and-control pipelines across heterogeneous robots and HPC cloud resources. It introduces DataBlocks of Filters, a protocol-agnostic DataChannel based on WebRTC, and a Temporal Addressable Memory (TAM) to synchronize timestamped data, with a cloud dojo for training and deploying neural networks. The approach enables end-to-end AI lifecycles—from data acquisition to model deployment—across devices and the cloud, and demonstrates improvements in sampling rate and end-to-end latency in forest-fire prevention and autonomous driving tasks compared with ROS baselines. The work highlights practical benefits for real-time, multi-robot AI applications while outlining scalability and safety considerations for future RTOS and ASIL-enabled deployments.

Abstract

The underlying framework for controlling autonomous robots and complex automation applications are Operating Systems (OS) capable of scheduling perception-and-control tasks, as well as providing real-time data communication to other robotic peers and remote cloud computers. In this paper, we introduce CyberCortex AI, a robotics OS designed to enable heterogeneous AI-based robotics and complex automation applications. CyberCortex AI is a decentralized distributed OS which enables robots to talk to each other, as well as to High Performance Computers (HPC) in the cloud. Sensory and control data from the robots is streamed towards HPC systems with the purpose of training AI algorithms, which are afterwards deployed on the robots. Each functionality of a robot (e.g. sensory data acquisition, path planning, motion control, etc.) is executed within a so-called DataBlock of Filters shared through the internet, where each filter is computed either locally on the robot itself, or remotely on a different robotic system. The data is stored and accessed via a so-called Temporal Addressable Memory (TAM), which acts as a gateway between each filter's input and output. CyberCortex AI has two main components: i) the CyberCortex AI inference system, which is a real-time implementation of the DataBlock running on the robots' embedded hardware, and ii) the CyberCortex AI dojo, which runs on an HPC computer in the cloud, and it is used to design, train and deploy AI algorithms. We present a quantitative and qualitative performance analysis of the proposed approach using two collaborative robotics applications: i) a forest fires prevention system based on an Unitree A1 legged robot and an Anafi Parrot 4K drone, as well as ii) an autonomous driving system which uses CyberCortex AI for collaborative perception and motion control.

Figures (18)

• Figure 1: The CyberCortex.AI operating system software stack, composed of i) a scalable inference platform executing a DataBlock of so-called Filters and ii) a dojo used for designing, training and deploying the Filters and their underling AI algorithms. The Filters from different DataBlocks exchange information via the internet by establishing peer-to-peer communication using a DataChannel and a set of redundant Signaling Servers. The dotted lines are used during development and training.
• Figure 2: Typical distributed robotics application in the field of autonomous surveillance. The autonomous legged robot (UGV) and drone (UAV) can improve their path planners if they exchange perception data between them, as well as by sharing computational resources. A remote Mission Controller is used to pass high level commands, as well as to visualize and store sensory data. AI algorithms are mainly used in the perception, localization and state estimation components. The dotted lines indicate remote communication through the DataChannel.
• Figure 3: Details of the CyberCortex.AI.inference system executed on an autonomous robot. The communication between filters running on different DataBlocks is established using a redundant set of Signaling Servers, which act as a discovery system between the five DataBlocks (legged robot, drone, dojo, smartphone app and web debugger).
• Figure 4: Temporal Addressable Memory (TAM) used for data synchronization and propagation through the AI inference engine. The output of each filter is stored in a finite FIFO cache memory indexed by timestamps. The synchronization of two Filters running at different sampling rates is performed by addressing the TAM with the desired timestamp.
• Figure 5: The AI inference engine organizes its input and output branches as temporal sequences of data from the Temporal Addressable Memory (TAM), indexed through timestamps. The inputs are extracted from the TAM, while the outputs are stored in the TAM.