Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating Sensor Fusion in Neuromorphic Computing: A Case Study on Loihi-2

Murat Isik, Karn Tiwari, Muhammed Burak Eryilmaz, I. Can Dikmen

TL;DR

The paper addresses the need for real-time, energy-efficient sensor fusion in autonomous systems. It applies Intel Loihi-2 neuromorphic hardware and spiking neural networks to accelerate fusion across multimodal datasets using five SNN models. Results show Loihi-2 achieves energy efficiency surpassing CPU by over 100x and GPU by about 30x, with competitive throughput, while maintaining a very low power footprint. The work also outlines a practical framework using Lava-DL, generating HDF5 models and addressing hardware-specific deployment challenges. These findings validate neuromorphic computing as a viable, scalable platform for real-time sensor fusion in autonomous technologies.

Abstract

In our study, we utilized Intel's Loihi-2 neuromorphic chip to enhance sensor fusion in fields like robotics and autonomous systems, focusing on datasets such as AIODrive, Oxford Radar RobotCar, D-Behavior (D-Set), nuScenes by Motional, and Comma2k19. Our research demonstrated that Loihi-2, using spiking neural networks, significantly outperformed traditional computing methods in speed and energy efficiency. Compared to conventional CPUs and GPUs, Loihi-2 showed remarkable energy efficiency, being over 100 times more efficient than a CPU and nearly 30 times more than a GPU. Additionally, our Loihi-2 implementation achieved faster processing speeds on various datasets, marking a substantial advancement over existing state-of-the-art implementations. This paper also discusses the specific challenges encountered during the implementation and optimization processes, providing insights into the architectural innovations of Loihi-2 that contribute to its superior performance.

Accelerating Sensor Fusion in Neuromorphic Computing: A Case Study on Loihi-2

TL;DR

The paper addresses the need for real-time, energy-efficient sensor fusion in autonomous systems. It applies Intel Loihi-2 neuromorphic hardware and spiking neural networks to accelerate fusion across multimodal datasets using five SNN models. Results show Loihi-2 achieves energy efficiency surpassing CPU by over 100x and GPU by about 30x, with competitive throughput, while maintaining a very low power footprint. The work also outlines a practical framework using Lava-DL, generating HDF5 models and addressing hardware-specific deployment challenges. These findings validate neuromorphic computing as a viable, scalable platform for real-time sensor fusion in autonomous technologies.

Abstract

In our study, we utilized Intel's Loihi-2 neuromorphic chip to enhance sensor fusion in fields like robotics and autonomous systems, focusing on datasets such as AIODrive, Oxford Radar RobotCar, D-Behavior (D-Set), nuScenes by Motional, and Comma2k19. Our research demonstrated that Loihi-2, using spiking neural networks, significantly outperformed traditional computing methods in speed and energy efficiency. Compared to conventional CPUs and GPUs, Loihi-2 showed remarkable energy efficiency, being over 100 times more efficient than a CPU and nearly 30 times more than a GPU. Additionally, our Loihi-2 implementation achieved faster processing speeds on various datasets, marking a substantial advancement over existing state-of-the-art implementations. This paper also discusses the specific challenges encountered during the implementation and optimization processes, providing insights into the architectural innovations of Loihi-2 that contribute to its superior performance.
Paper Structure (13 sections, 5 figures, 3 tables)

This paper contains 13 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Sensor Fusion
  4. Neuromorphic Computing
  5. Intel’s Loihi-2 Neuromorphic Chip
  6. Methods
  7. Dataset
  8. Overview of the Proposed Framework
  9. Additional Considerations:
  10. Evaluation
  11. Conclusion
  12. Future Work
  13. Acknowledgment

Figures (5)

  • Figure 1: System architecture integrating the Loihi-2 neuromorphic chip for advanced sensor fusion.
  • Figure 2: Sensory data from cameras, LIDAR, RADAR, GPS, and IMU, processed through a neuromorphic computing unit, exemplified for advanced autonomous driving applications.
  • Figure 3: Overview of a Neuromorphic Computing System Architecture, integrating specialized software framework. The system employs advanced communication protocols and hardware components designed for real-time data processing and interaction with the environment.
  • Figure 4: Block Diagram of Implementation
  • Figure 5: Comparison of energy efficiency, measured as giga-operation per second per billion transistors, against the energy required for one operation (giga-operation per second per watt) across various computing hardware. This plot illustrates the efficiency trade-offs between CPUs, GPUs, and Loihi-2 in processing.