Extensional Properties of Recurrent Neural Networks

TL;DR

A version of Rice's theorem for RNNs is proved: any nontrivial extensional property of RNNs is undecidable.

Abstract

A property of a recurrent neural network (RNN) is called \emph{extensional} if, loosely speaking, it is a property of the function computed by the RNN rather than a property of the RNN algorithm. Many properties of interest in RNNs are extensional, for example, robustness against small changes of input or good clustering of inputs. Given an RNN, it is natural to ask whether it has such a property. We give a negative answer to the general question about testing extensional properties of RNNs. Namely, we prove a version of Rice's theorem for RNNs: any nontrivial extensional property of RNNs is undecidable.

Theorems & Definitions (9)

• Theorem 1: Rice's theorem
• Theorem 2: Rice's theorem in general form
• definition 1: function $\psi$
• Lemma 3: computability of $\psi'$
• proof
• Lemma 4: completeness of $\psi'$
• proof
• Theorem 5: Rice's theorem for RNN machines
• proof