Motion Illusions Generated Using Predictive Neural Networks Also Fool Humans

TL;DR

A generative model, the Evolutionary Illusion GENerator (EIGen), that creates new visual motion illusions that supports the hypothesis that illusory motion might be the result of perceiving the brain’s own predictions rather than perceiving raw visual input from the eyes.

Abstract

Why do we sometimes perceive static images as if they were moving? Visual motion illusions enjoy a sustained popularity, yet there is no definitive answer to the question of why they work. Here we present evidence in favor of the hypothesis that illusory motion is a side effect of the predictive abilities of the brain. We present a generative model, the Evolutionary Illusion GENerator (EIGen), that creates new visual motion illusions based on a video predictive neural network. We confirm that the constructed illusions are effective on human participants through a psychometric survey. Our results support the hypothesis that illusory motion might be the consequence of perceiving the brain's own predictions rather than perceiving raw visual input from the eyes. The philosophical motivation of this paper is to call attention to the untapped potential of "motivated failures", ways for artificial systems to fail as biological systems fail, as a worthy outlet for Artificial Intelligence and Artificial Life research.

Figures (16)

• Figure 1: The Artificial Perception Approach. (a) Most AI models, for example here for image recognition, fail in ways that are utterly different from human failures: they do not work the same and therefore do not fail the same. (b) In this paper we use a 4-step method to show that a predictive AI model shares the same transition between failure and success as humans do, suggesting that the two systems also share working principles (in this case, predictive coding): they fail the same, and therefore may work the same. (c) The same approach, used for various tasks, would reveal other AI models that sit on the same failure boundary as humans, and therefore share working principles: cognito-mimetic systems.
• Figure 2: The structure of EIGen. A Predictive Neural Network (Prednet) is trained to predict video frames. It is then used in combination to optical flow calculation to rate the strength of illusory motion in images generated by Compositional Pattern-Producing networks (CPPN). The CPPNs are optimized by the NEAT evolutionary algorithm.
• Figure 3: Examples of greyscale and color illusions generated by EIGen. The smaller image at the bottom of each illusion represents the motion predicted by the network. The vectors origins are marked as yellow dots; the amplitude is multiplied by 60 for easy visualization. These images are handpicked for illustration purposes; the images that were used for the human perception experiment are in the Appendix.
• Figure 4: Generated illusions replicating existing human-designed illusions. The top left image is similar to A. Kitaoka's 'medaka school' illusion (bottom left) medaka, except that Kitaoka's illusion is linear. EIGen invariably converges to this solution when forced to use a binary output (pixels fully black or fully white). The motion predicted by the network is extremely small, and accordingly one of the author does not perceive motion on either the EIGen-generated illusion nor Kitaoka's illusion. The top right image is a close replication of Kitaoka's 2011 iteration (bottom right, kitaokaFraser) on the Fraser-Wilcox illusion fraser1979perception. This output is also frequent. Images reproduced with permission.
• Figure 5: Failed illusions. These images did not produce illusory motion, even when combined through duplicated and mirroring structures as in other figures. The first image has few vectors. The second image has many vectors but their direction and position is unclear. Same for the last image.