Learning Goal-based Movement via Motivational-based Models in Cognitive Mobile Robots

TL;DR

A computationally model a motivation theory proposed by Hull, where the agent is motivated to keep itself in a state of homeostasis and the use of pleasure in the motivational mechanism significantly impacted behavior learning, mainly for slow metabolism agents.

Abstract

Humans have needs motivating their behavior according to intensity and context. However, we also create preferences associated with each action's perceived pleasure, which is susceptible to changes over time. This makes decision-making more complex, requiring learning to balance needs and preferences according to the context. To understand how this process works and enable the development of robots with a motivational-based learning model, we computationally model a motivation theory proposed by Hull. In this model, the agent (an abstraction of a mobile robot) is motivated to keep itself in a state of homeostasis. We added hedonic dimensions to see how preferences affect decision-making, and we employed reinforcement learning to train our motivated-based agents. We run three agents with energy decay rates representing different metabolisms in two different environments to see the impact on their strategy, movement, and behavior. The results show that the agent learned better strategies in the environment that enables choices more adequate according to its metabolism. The use of pleasure in the motivational mechanism significantly impacted behavior learning, mainly for slow metabolism agents. When survival is at risk, the agent ignores pleasure and equilibrium, hinting at how to behave in harsh scenarios.

Figures (11)

• Figure 1: Hull's theory. Adpated from hullCycle.
• Figure 2: Hull's Drive Reduction Theory was adapted to the robotics domain. In this example, an agent is equipped with 3 sensors ($S_1$,$S_2$ and $S_3$). These sensors are used to create a higher-level interpretation of the environment by representing different objects/places. Our agent has one need $N_1$ with its corresponding homeostasis level. The sensors are also used to assess how much this $N_1$ is unsatisfied, represented by its corresponding drive $D_1$. Actions that drive our agent in the environment can be selected by a decision-making mechanism to, for instance, reduce the level of unbalance of the agent.
• Figure 3: Wanting mechanism. Behavior learning occurs from the expected reward resulting from drive reduction.
• Figure 4: Wanting + Liking mechanisms. Behavior learning occurs from the expected reward resulting from drive reduction and hedonic dimension ($L_j$).
• Figure 5: 20x20 environment used in the experiments.