Semiotics Networks Representing Perceptual Inference

TL;DR

This work introduces Conscious Neural Networks (CONNs), a framework that models object perception and cross-person communication through observed-to-seen cycles in internal representations, enabling interpretable visualization of perceptual processes. It formalizes attractor-based dynamics and bipartite orbit concepts to capture how individuals converge on stable percepts and how dialogue between agents can settle into periodic patterns. The authors build vanilla and stochastic CONN classifiers by embedding perceptual layers based on autoencoders before a baseline classifier, achieving improved performance with limited data while preserving interpretability. The approach generalizes beyond human agents to any system with latent-to-raw representation processing, offering a pathway toward interpretable perceptual inference and robust multi-modal communication in AI systems.

Abstract

Every day, humans perceive objects and communicate these perceptions through various channels. In this paper, we present a computational model designed to track and simulate the perception of objects, as well as their representations as conveyed in communication. We delineate two fundamental components of our internal representation, termed "observed" and "seen", which we correlate with established concepts in computer vision, namely encoding and decoding. These components are integrated into semiotic networks, which simulate perceptual inference of object perception and human communication. Our model of object perception by a person allows us to define object perception by {\em a network}. We demonstrate this with an example of an image baseline classifier by constructing a new network that includes the baseline classifier and an additional layer. This layer produces the images "perceived" by the entire network, transforming it into a perceptualized image classifier. This facilitates visualization of the acquired network. Within our network, the image representations become more efficient for classification tasks when they are assembled and randomized. In our experiments, the perceptualized network outperformed the baseline classifier on MNIST training databases consisting of a restricted number of images. Our model is not limited to persons and can be applied to any system featuring a loop involving the processing from "internal" to "external" representations.

Figures (11)

• Figure 1: Person-to-person CONN. The internal communication loops associated with the persons are comprised of the observed-to-seen transformations and are denoted by rounded rectangles. The persons interchange their seen images, resulting in the internal communication loops, using the external communication loop (denoted by black arrows). The flowchart of the implementation of the CONN using autoencoder operations is shown in Fig. \ref{['fig:scheme_full_b']}
• Figure 2: The figure shows an implementation of the person-to-person CONN from Fig. \ref{['fig:scheme_full_a']}, with the observed-to-seen transformations implemented as the composition of encoder (shown as blue rectangles) and decoder (shown as green rectangles) operations. This implementation is described in Algorithm \ref{['alg:CONN']}. The external communication loop (denoted by black arrows) is represented by step \ref{['nitm:3']}, while the internal communication loops (denoted by rounded rectangles) are represented by step \ref{['nitm:3.15']} of the algorithm
• Figure 3: Partitioning $U$ by segments of $K=3$ and $K=4$
• Figure 4: Illustration of the bipartite convergence of the second type, claimed in Theorem \ref{['theorem_L_A']}. The space $X$ contains 4 basins for $F_{P_1}$ with attractors $x_1, \ldots ,x_4$ depicted as blue triangles. The borders between the basins are denoted by blue lines. Analogously, $X$ contains 3 basins for $F_{P_2}$ with attractors $y_1$, $y_2$, $y_3$, and $y_4$ depicted as green rectangles. The borders between the basins are denoted by dashed green lines. Alternating mappings to the attractors of $F_{P_1}$ and $F_{P_2}$, starting with $Im$, yield sequence $H=Im\rightarrow x_2 \rightarrow y_2 \rightarrow x_3 \rightarrow y_1 \rightarrow x_1 \rightarrow y_2 \rightarrow x_3 \dots$, terminated by cycle $(x_3,y_1,x_1,y_2)$. For a selected proximity, 10 sequential elements $p_1, p_2,\dots, p_{10}$ from a subsequence $(p_i)$ of sequence $U(Im,nsteps_1,nsteps_2)$ (Eq. \ref{['eq:U0']}) are shown. The parameters $nsteps_1$ and $nsteps_2$, and $iter$, the position of $p_1$ in $U$, are chosen sufficiently large, so that the elements $p_i$ fall within the predefined proximity to the respective attractors: $p_1$ is close to $x_2$, $p_2$ to $y_2$, $p_3$ to $x_3$, …, $p_{10}$ to $y_2$, etc.
• Figure 5: Representation of the work of the CONN classifiers as the transformation $F$ of a training set $TR$ (resp. test set $TE$) to a new training set $ATR$ (resp. test set $ATE$) consisting of the images "perceived" by the classifier. For the vanilla classifier, the transformation $F$ denotes the transformation $\widehat{F}$ to the attractor (Eq. \ref{['eq:nakaonetc']}). For the stochastic classifier, $F$ denotes the transformation $F^*$ to the averaged randomized ensemble of attractors (Equation \ref{['eq:ARA']})

Theorems & Definitions (11)

• Definition 1: Bipartite Convergence of the First Type to Orbit
• Remark 2
• Lemma 3
• proof
• Definition 4: Bipartite Convergence of the Second Type to Orbit
• Theorem 5
• proof
• Lemma 6
• Lemma 7
• Theorem 8