Structure from Rank: Rank-Order Coding as a Bridge from Sequence to Structure

TL;DR

A rank-order based neural network inspired by the STG-LIFG-PMC pathway is proposed, modeling the bottom-up transition from acoustic input to abstract rank representation, and the top-down generation from that representation to motor execution, suggesting that rank-order coding not only serve as a compact encoding scheme but also support encoding hierarchical grammar.

Abstract

Understanding how structured sequence information can be represented and generalized in neural systems is key to modeling the transition from acoustic input to emergent structure. In this study, we propose a rank-order based neural network inspired by the STG-LIFG-PMC pathway, modeling the bottom-up transition from acoustic input to abstract rank representation, and the top-down generation from that representation to motor execution. Building on previous work in rank coding, we first demonstrate that this model efficiently compresses input while retaining the capacity to reconstruct full utterances from partial cues, revealing emergent structure-sensitive generation process that reflects context-general representations of sensorimotor states, which are later shaped into context-specific motor plans during speech planning. We then show that the network exhibits global-level novelty detection similar to the P3B novelty wave, replicating the global-sequence-sensitive mechanism. As a supplement, we also compare the model's behavior under local (index-level) and global (rank-level) perturbations, revealing robustness to superficial variation and sensitivity to abstract structural violation, key features associated with proto-syntactic generalization. These results suggest that rank-order coding not only serve as a compact encoding scheme but also support encoding hierarchical grammar.

Figures (10)

• Figure 1: (a) This illustrates the rank-order framework for transforming waveforms into an abstract representation. This illustrates the rank-order framework for transforming waveforms into an abstract representation. Acoustics primitives are categorized, assigned indices by phonetic similarity (e.g., $/AH0/$, $/OW2/$, $/EY1/$ for vowels; $/T/$, $/M/$ for consonants). The indexed elements are then grouped into chunks of three and which are then transformed into rank-order chunks by preserving only their relative ordering. These rank chunks are assigned indices by preserving relative temporal ordering. Finally, 5 rank chunks are integrated into one superordinate chunk, forming a hierarchical structure and yielding a context-general representation. (b) The corresponding dual pathway model, where brain regions are marked with colored borders and pathway activity is color-coded. The pink pathway (STG $\rightarrow$ PMC) represents the first stream for sensorimotor integration that includes direct sound-to-motor mapping. The orange pathway (STG $\rightarrow$ LIFG $\rightarrow$ PMC) performs a bottom-up transition (STG $\rightarrow$ LIFG) from acoustic input (context-specific) to an abstract (context-general) rank representation (also shown by left orange arrow in (a), and a top-down projection (LIFG $\rightarrow$ PMC) from this rank chunk to a concrete motor sequence (context-specific) for articulation (also shown by right orange arrow in (a).
• Figure 2: Brain-inspired neural network diagram. The architecture consists of two primary loops, color-coded to match the pathways in Fig. \ref{['rankanddualpathway']}. Pink Pathway (Sensorimotor): Models the rapid auditory-motor loop from the Superior Temporal Gyrus (STG) to the Premotor Cortex (PMC), enabling elementary level sensorimotor mapping. Orange Pathway (Hierarchical Processing): Supports higher-level processing, where acoustic input are transformed into sensorimotor state index chunks. These are converted into abstract rank codes in the Left Inferior Frontal Gyrus (LIFG) and then recalled into complete, structured motor plans, which tune the PMC output. The model processes inputs through three representational levels in Fig. \ref{['rank_order_chart']}: acoustic (MFCC), index, and rank.
• Figure 3: Neural Network Architecture
• Figure 4: Neuron clusters in STG ($Y_{som}$) (left) and PMC ($Y_{pred}$) (right) layers. (a) illustrates the topology of both layers, with the 20-dimensional neurons of the new pre-trained predictive model reduced to two dimensions. For comparison, Fig. \ref{['chap2_som_cluster_PCA']} shows results from our previous work. The color bar represents neuron indices, with similar indices sharing similar colors. Closely clustered dots of similar colors in the two-dimensional space confirm that the topological structure is preserved. This spatial arrangement validates the alignment between the STG and PMC layers, demonstrating effective topological mapping in the model.
• Figure 5: (a)-(d): Growth of dimensions in terms of MFCCs, unique index chunks, and unique rank chunks across different dataset sizes. (e): Comparison of MFCCs, unique index chunks, and unique rank chunks across different chunk lengths with a dataset size of 16 minutes.