Optical Computation-in-Communication enables low-latency, high-fidelity perception in telesurgery

TL;DR

An Optical Computation-in-Communication (OCiC) framework is introduced that reduces end-to-end latency significantly by performing AI inference concurrently with optical communication, enabling reliable, low-latency telesurgery across distances up to 10,000 km and opening a new optical frontier for distributed medical intelligence.

Abstract

Artificial intelligence (AI) holds significant promise for enhancing intraoperative perception and decision-making in telesurgery, where physical separation impairs sensory feedback and control. Despite advances in medical AI and surgical robotics, conventional electronic AI architectures remain fundamentally constrained by the compounded latency from serial processing of inference and communication. This limitation is especially critical in latency-sensitive procedures such as endovascular interventions, where delays over 200 ms can compromise real-time AI reliability and patient safety. Here, we introduce an Optical Computation-in-Communication (OCiC) framework that reduces end-to-end latency significantly by performing AI inference concurrently with optical communication. OCiC integrates Optical Remote Computing Units (ORCUs) directly into the optical communication pathway, with each ORCU experimentally achieving up to 69 tera-operations per second per channel through spectrally efficient two-dimensional photonic convolution. The system maintains ultrahigh inference fidelity within 0.1% of CPU/GPU baselines on classification and coronary angiography segmentation, while intrinsically mitigating cumulative error propagation, a longstanding barrier to deep optical network scalability. We validated the robustness of OCiC through outdoor dark fibre deployments, confirming consistent and stable performance across varying environmental conditions. When scaled globally, OCiC transforms long-haul fibre infrastructure into a distributed photonic AI fabric with exascale potential, enabling reliable, low-latency telesurgery across distances up to 10,000 km and opening a new optical frontier for distributed medical intelligence.

Figures (9)

• Figure 1: The OCiC framework enables real-time, AI-enhanced perception for telesurgical procedures. It addresses the lack of local sensory feedback that hampers intraoperative decision-making, while reducing cumulative latency from communication and inference. Panel (b) shows the OCiC architecture, consisting of geographically distributed, cascaded ORCUs that hierarchically implement distinct convolutional layers via fibre-based photonic computation. This design allows OCiC to perform AI inference concurrently with optical data transmission, effectively overlapping computation time with communication delays. As a result, the system supports AI-assisted telesurgery across intercontinental distances, remaining well within the clinically accepted one-way latency threshold of 50 ms. Panel (a) compares this approach to conventional systems, which rely on local computation followed by data transmission, showing that OCiC achieves higher frame rates and lower end-to-end latency (see Supplementary Information C.1 for further details). Beyond latency reduction, OCiC enables dynamic redistribution of surgical capacity from well-resourced centres to underserved regions throughout its deployment network. This helps mitigate surgical delays due to limited specialist availability and restricted access to medical transport, especially in low- and middle-income countries, rural areas, and sparsely populated zones. The framework’s scalability is especially advantageous in disaster relief scenarios, where temporary OCiC branches can be rapidly deployed from nearby nodes, enabling immediate remote surgical support from clinical centres worldwide.
• Figure 1: Illustration of our 2D photonic convolution. (a) In 2D convolution, a kernel (e.g., 3 × 3) is placed at the top-left corner of the input, where it multiplies element-wise with the overlapping region, sums the results, and adds a bias to produce a single output. The kernel then slides across the input from left to right, row by row, until all valid regions are covered. Each result is assigned to the corresponding position in the output feature map. (b) In a conventional optical convolution processor, pixels are fed in column-major order: for each column, typically only the first three pixels are input sequentially from top to bottom before moving to the next column. Such methods enable a one-dimensional photonic convolution processor to handle 2D images, but introduce an inherent stride limitation, reducing the effective throughput to one-third of the nominal rate for 3 × 3 convolutions. As illustrated in the right panel of (b), only outputs at $t_{1}$, $t_{4}$, $t_{7}$, … are valid, while those at $t_{2}$, $t_{3}$, $t_{5}$, $t_{6}$, $t_{8}$, $t_{9}$, … are redundant. (c) In contrast, our design supports 2D photonic convolution with row-by-row pixel feeding into the optical link, thereby achieving nearly 100% computational efficiency.
• Figure 2: ORCU architecture enabling remote optical computation within communication fibres. (a) SMF supports wavelength-division multiplexing, enabling simultaneous optical communication and computation. Communication channels transmit original images and residual data, while dedicated computational channels use two-dimensional structured optical combs to perform photonic convolutions over geographical distances. A representative 3×3 convolutional comb comprises three groups of three spectral modes, with wavelength spacings corresponding to pixel and row intervals. (b) Two-dimensional input images are preprocessed and flattened into row-wise one-dimensional sequences before optical modulation and transmission. (c) Leveraging linear fibre dispersion, computational channel wavelengths are temporally interleaved with precise delays, directly performing two-dimensional convolutions during optical transmission. (d) At the receiver, a fan-out coupler array splits signals into 192 parallel optical paths. OSSM assign kernel weights via spectral shaping; photodiodes convert and simultaneously sum optical signals into electrical outputs. (e) Experimental convolution results show a high degree of consistency with theoretical predictions , illustrated alongside corresponding computed feature maps. Detailed methodological descriptions are provided in \ref{['sec:method']}. More experimental results are provided in Supplementary Information B.
• Figure 2: Experimental arrangement for MNIST classification task
• Figure 3: ORCU achieves CPU-level inference accuracy with superior fidelity on the MNIST classification task. (a) Confusion matrices demonstrate nearly identical classification accuracy between ORCU (96.2%) and CPU-based inference (96.1%). (b) Bubble chart comparing fidelity error (accuracy deviation from electronic inference) and computational speed highlights ORCU’s enhanced fidelity relative to benchmark optical processors feldmann2021parallelxu202111meng2023compactdong2024partialahmed2025universal. (c) Convolutional feature maps generated by four pre-trained kernels confirm that ORCU outputs closely match CPU-computed results at the feature-extraction level. (d) Scatter plot of prediction probabilities from ORCU versus CPU computations exhibits high consistency across all classes (regression correlation, r = 0.96). (e) Precision–Recall (left) and receiver operating characteristic (ROC, right) curves indicate robust classification performance across a wide range of decision thresholds. (f) Bar charts summarising key performance metrics—accuracy, precision, recall, F1 score, AUROC, and average precision—demonstrate that ORCU matches or surpasses CPU-based performance. Together, these results confirm ORCU’s capability to achieve CPU-level inference accuracy while supporting high-throughput parallel optical processing, positioning it as a promising candidate for scalable optical computing architectures.