Construction of a Neural Network with Temperature-Dependent Recall Patterns

TL;DR

It is demonstrated by equilibrium Monte-Carlo simulations that such a temperature-dependent change in recall patterns does occur in this model and that the system undergoes a first-order phase transition when the change in recall patterns occurs.

Abstract

We present a simple model that recalls two different patterns depending on the temperature. To realize a change in recall pattern due to temperature change, we embed two patterns to different graphs: the first pattern into a fully connected graph and the second pattern into a sparse graph. Because a fully connected graph is more resistant to thermal fluctuations than a sparse graph, we can realize a change in recall pattern by tuning relative weights of the two patterns properly. We demonstrate by equilibrium Monte-Carlo simulations that such a temperature-dependent change in recall patterns does occur in our model. Simulation results strongly indicate that the system undergoes a first-order phase transition when the change in recall patterns occurs. It is also demonstrated by annealing simulations that the system fails to recall the pattern embedded in the sparse graph at low temperatures if the free-energy barrier is too high to overcome within the given simulation timescale.

Figures (10)

• Figure 1: The temperature dependences of $\langle m_{\rm F} \rangle_T$ and $\langle m_{\rm S} \rangle_T$ for four samples. The size $L$ is $10$ and the coefficient $C^{\rm F}$ is $0.9$.
• Figure 2: The temperature dependences of $\langle m_{\rm F} \rangle_T$ and $\langle m_{\rm S} \rangle_T$ for four samples. The size $L$ is $30$ and the coefficient $C^{\rm F}$ is $0.9$.
• Figure 3: The temperature dependences of sample-averaged magnetizations. The average was taken over $100$ samples. The size $L$ is $30$. The values of $C^{\rm F}$ are $0.80$, $0.85$, $0.90$, and $0.95$ from top left to right bottom.
• Figure 4: The temperature dependence of the total number of MCS required to perform a variant of the Wang-Landau method. The coefficient $C^{\rm F}$ is $0.90$. The values of $L$ are $10$, $20$, and $30$ from bottom to right. The average was taken over $100$ samples.
• Figure 5: $\beta F_{\rm diff}$ of a sample is plotted as a function of two magnetizations for four different temperatures. $F_{\rm diff}$ is defined by Eq. \ref{['eqn:def_Fdiff']}. The sample is the same as "Sample 1" in Fig. \ref{['fig:SampMag_L030_CMF090']}. The size $L$ is $30$ and the coefficient $C^{\rm F}$ is $0.90$. The locations of $F_{\rm min}$, $S_{\rm min}$, and $B$ are denoted by triangles, inverted triangles, and squares, respectively. See text for the definitions of $F_{\rm min}$, $S_{\rm min}$, and $B$.