Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TE-PINN: Quaternion-Based Orientation Estimation using Transformer-Enhanced Physics-Informed Neural Networks

Arman Asgharpoor Golroudbari

Abstract

This paper introduces a Transformer-Enhanced Physics-Informed Neural Network (TE-PINN) designed for accurate quaternion-based orientation estimation in high-dynamic environments, particularly within the field of robotics. By integrating transformer networks with physics-informed learning, our approach innovatively captures temporal dependencies in sensor data while enforcing the fundamental physical laws governing rotational motion. TE-PINN leverages a multi-head attention mechanism to handle sequential data from inertial sensors, such as accelerometers and gyroscopes, ensuring temporal consistency. Simultaneously, the model embeds quaternion kinematics and rigid body dynamics into the learning process, aligning the network's predictions with mechanical principles like Euler's laws of motion. The physics-informed loss function incorporates the dynamics of angular velocity and external forces, enhancing the network's ability to generalize in complex scenarios. Our experimental evaluation demonstrates that TE-PINN consistently outperforms traditional methods such as Extended Kalman Filters (EKF) and LSTM-based estimators, particularly in scenarios characterized by high angular velocities and noisy sensor data. The results show a significant reduction in mean quaternion error and improved gyroscope bias estimation compared to the state-of-the-art. An ablation study further isolates the contributions of both the transformer architecture and the physics-informed constraints, highlighting the synergistic effect of both components in improving model performance. The proposed model achieves real-time performance on embedded systems typical of mobile robots, offering a scalable and efficient solution for orientation estimation in autonomous systems.

TE-PINN: Quaternion-Based Orientation Estimation using Transformer-Enhanced Physics-Informed Neural Networks

Abstract

This paper introduces a Transformer-Enhanced Physics-Informed Neural Network (TE-PINN) designed for accurate quaternion-based orientation estimation in high-dynamic environments, particularly within the field of robotics. By integrating transformer networks with physics-informed learning, our approach innovatively captures temporal dependencies in sensor data while enforcing the fundamental physical laws governing rotational motion. TE-PINN leverages a multi-head attention mechanism to handle sequential data from inertial sensors, such as accelerometers and gyroscopes, ensuring temporal consistency. Simultaneously, the model embeds quaternion kinematics and rigid body dynamics into the learning process, aligning the network's predictions with mechanical principles like Euler's laws of motion. The physics-informed loss function incorporates the dynamics of angular velocity and external forces, enhancing the network's ability to generalize in complex scenarios. Our experimental evaluation demonstrates that TE-PINN consistently outperforms traditional methods such as Extended Kalman Filters (EKF) and LSTM-based estimators, particularly in scenarios characterized by high angular velocities and noisy sensor data. The results show a significant reduction in mean quaternion error and improved gyroscope bias estimation compared to the state-of-the-art. An ablation study further isolates the contributions of both the transformer architecture and the physics-informed constraints, highlighting the synergistic effect of both components in improving model performance. The proposed model achieves real-time performance on embedded systems typical of mobile robots, offering a scalable and efficient solution for orientation estimation in autonomous systems.
Paper Structure (35 sections, 12 equations, 7 figures, 1 table)

This paper contains 35 sections, 12 equations, 7 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Deep Learning for Orientation Estimation
  4. PINNs and Transformer Integration for Navigation
  5. Hybrid Architectures with Transformers for Navigation
  6. Methodology
  7. Problem Formulation
  8. Transformer-Enhanced Physics-Informed Neural Network (TE-PINN)
  9. Quaternion-Based Transformer Encoder
  10. Input Representation
  11. Positional Encoding
  12. Transformer Encoder Architecture
  13. Quaternion Output Layer
  14. Attitude Correction Layer
  15. Quaternion to Euler Conversion
  16. ...and 20 more sections

Figures (7)

  • Figure 1: Quaternion Components Estimation. This figure illustrates the estimation of individual quaternion components ($q_w$, $q_x$, $q_y$, $q_z$) over time. The comparison includes ground truth (black), analytical method (red), deep learning model (blue), and TE-PINN (green). The plot highlights TE-PINN's ability to accurately estimate quaternion components, which is crucial for precise orientation representation.
  • Figure 2: Comparative performance across noise levels
  • Figure 3: Performance under dynamic conditions
  • Figure 4: Uncertainty calibration plot
  • Figure 5: Orientation Estimation Results. This figure shows the comparison of estimated Euler angles (Roll, Pitch, Yaw) between the ground truth (black), analytical method (red), deep learning model (blue), and TE-PINN (green) over time. The plot demonstrates the superior accuracy of TE-PINN in tracking orientation across all three axes.
  • ...and 2 more figures