Transforming Hidden States into Binary Semantic Features

TL;DR

Using Independent Component Analysis to overcome some of its challenging aspects, it is shown that large language models represent semantic features in their hidden states.

Abstract

Large language models follow a lineage of many NLP applications that were directly inspired by distributional semantics, but do not seem to be closely related to it anymore. In this paper, we propose to employ the distributional theory of meaning once again. Using Independent Component Analysis to overcome some of its challenging aspects, we show that large language models represent semantic features in their hidden states.

Figures (8)

• Figure 1: This is component number 63 from the ICA-transformed hidden states of the Llama 3 70B model, representing Grammar. The outer circle shows the components that share words with this component. The 10 components that did not fit in the graph are listed in the table bellow the graph (together with top 5 words that combine the listed component and the central component in the graph). See the caption of Figure \ref{['fig:triangle']} for explanation of the graphic symbols.
• Figure 2: Combining components. The blue circle nodes represent the components, the edges represent the connections between them. The labels on the edges show the words that are shared between the components. The words on the triangle in the middle belong to all three components.
• Figure 3: This is component number 37 from the ICA-transformed hidden states of the Llama 3 70B model, representing (musical) Instruments. The outer circle shows the components that share words with this component. The 18 components that did not fit in the graph are listed in the table bellow the graph (together with top 5 words that combine the listed component and the central component in the graph). See the caption of Figure \ref{['fig:triangle']} for explanation of the graphic symbols.
• Figure 4: This is component number 64 from the ICA-transformed hidden states of the Llama 3 70B model, representing Pharmaceutics. The outer circle shows the components that share words with this component. See the caption of Figure \ref{['fig:triangle']} for explanation of the graphic symbols.
• Figure 5: This is component number 191 from the ICA-transformed hidden states of the Llama 3 8B model. The 2 components that did not fit in the graph are listed in the table bellow. See the caption of Figure \ref{['fig:triangle']} for explanation of the graphic symbols.