Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Conformal Disentanglement: A Neural Framework for Perspective Synthesis and Differentiation

George A. Kevrekidis, Eleni D. Koronaki, Yannis G. Kevrekidis

TL;DR

The paper tackles extracting common and sensor-specific (uncommon) information from multi-sensor observations by formalizing a conformal disentanglement problem on submanifolds $\mathcal{C}$, $\mathcal{U}$, and $\mathcal{V}$. It introduces a structured autoencoder with separate common and uncommon latent streams and a bi-level optimization that first identifies the common subspace via $\mathcal{L}_\text{reconstruction}$ and $\mathcal{L}_\text{common}$, then disentangles the uncommon components with an orthogonality loss $\mathcal{L}_\text{orthogonality}$ to promote geometric independence. The framework is validated on synthetic dynamical systems and high-dimensional images, demonstrating correct recovery of latent submanifolds, the ability to generate level-set variations, and cross-sensor predictive mappings, including scenarios with time delays. A key contribution is the time-lag (causal) learning demonstration, where a future observation from one sensor can be used to infer the current state of the common system, effectively learning a form of correlational causality across asynchronous observations. The work provides a robust, unsupervised approach to cross-sensor observer construction and level-set exploration, with potential integration with spectral methods and extensions to more sensors.

Abstract

For multiple scientific endeavors it is common to measure a phenomenon of interest in more than one ways. We make observations of objects from several different perspectives in space, at different points in time; we may also measure different properties of a mixture using different types of instruments. After collecting this heterogeneous information, it is necessary to be able to synthesize a complete picture of what is `common' across its sources: the subject we ultimately want to study. However, isolated (`clean') observations of a system are not always possible: observations often contain information about other systems in its environment, or about the measuring instruments themselves. In that sense, each observation may contain information that `does not matter' to the original object of study; this `uncommon' information between sensors observing the same object may still be important, and decoupling it from the main signal(s) useful. We introduce a neural network autoencoder framework capable of both tasks: it is structured to identify `common' variables, and, making use of orthogonality constraints to define geometric independence, to also identify disentangled `uncommon' information originating from the heterogeneous sensors. We demonstrate applications in several computational examples.

Conformal Disentanglement: A Neural Framework for Perspective Synthesis and Differentiation

TL;DR

The paper tackles extracting common and sensor-specific (uncommon) information from multi-sensor observations by formalizing a conformal disentanglement problem on submanifolds , , and . It introduces a structured autoencoder with separate common and uncommon latent streams and a bi-level optimization that first identifies the common subspace via and , then disentangles the uncommon components with an orthogonality loss to promote geometric independence. The framework is validated on synthetic dynamical systems and high-dimensional images, demonstrating correct recovery of latent submanifolds, the ability to generate level-set variations, and cross-sensor predictive mappings, including scenarios with time delays. A key contribution is the time-lag (causal) learning demonstration, where a future observation from one sensor can be used to infer the current state of the common system, effectively learning a form of correlational causality across asynchronous observations. The work provides a robust, unsupervised approach to cross-sensor observer construction and level-set exploration, with potential integration with spectral methods and extensions to more sensors.

Abstract

For multiple scientific endeavors it is common to measure a phenomenon of interest in more than one ways. We make observations of objects from several different perspectives in space, at different points in time; we may also measure different properties of a mixture using different types of instruments. After collecting this heterogeneous information, it is necessary to be able to synthesize a complete picture of what is `common' across its sources: the subject we ultimately want to study. However, isolated (`clean') observations of a system are not always possible: observations often contain information about other systems in its environment, or about the measuring instruments themselves. In that sense, each observation may contain information that `does not matter' to the original object of study; this `uncommon' information between sensors observing the same object may still be important, and decoupling it from the main signal(s) useful. We introduce a neural network autoencoder framework capable of both tasks: it is structured to identify `common' variables, and, making use of orthogonality constraints to define geometric independence, to also identify disentangled `uncommon' information originating from the heterogeneous sensors. We demonstrate applications in several computational examples.
Paper Structure (13 sections, 21 equations, 9 figures, 1 table, 1 algorithm)

This paper contains 13 sections, 21 equations, 9 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Methods
  4. Network Architectures
  5. Bi-Level Optimization Algorithm
  6. Computational Examples
  7. 'Causal' Learning
  8. Discussion
  9. Data and Network Architectures
  10. Data Generation
  11. Architecture Details
  12. Optimization Algorithm
  13. Alternative Architectures

Figures (9)

  • Figure 1: Caricature described in \ref{['exmp:intro']}. Bob (blue, middle) is observed from different perspectives by two cameras. Alice (red, left) is only observed by camera 1, and Carol (green, right) is only observed by camera 2.
  • Figure 2: Sketch of the proposed network architecture outlined in \ref{['sec:architecture']}. The dimensions of the input, output and latent spaces correspond to those of \ref{['exmp:RL']}, while the encoder and decoder networks are just representations (and may have arbitrary width and depth)
  • Figure 3: Latent embeddings produced for a single run of the optimization algorithm applied to \ref{['exmp:Torus']}. (Top Row) Result after successfully terminating the first level optimization (no orthogonality between the common and uncommon components). (Bottom Row) Resulting embedding after the second level of optimization (with orthgonality imposed).
  • Figure 4: Latent embeddings produced for a single run of the optimization algorithm applied to \ref{['exmp:RL']}. The common subspaces are two dimensional while the uncommon are three-dimensional for each system. We show the resulting embedding after imposing the orthogonality constraints in the latent space. Coloring is achieved as in to \ref{['fig:ex_LC_results']}.
  • Figure 5: Bobblehead simulation setup presented in lederman2018. (We use the figures available in sroczynski2024learninglearnheterogeneousobservations). (a) gives a 'bird-eye' view of the setup, with the two cameras observing the bobblehead system; each bobblehead rotates independently with an unknown frequency. (b) shows each camera's point of view at a specific point in time. The bulldog is 'common' between them but observed at different angles, while Yoda and the rabbit are 'uncommon'.
  • ...and 4 more figures

Theorems & Definitions (4)

  • Example 1
  • Example 2: Torus
  • Example 3: Rössler & Lorenz Equations
  • Example 4: Bobbleheads