Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inferring Human Intentions from Predicted Action Probabilities

Lei Shi, Paul-Christian Bürkner, Andreas Bulling

TL;DR

This work proposes a two-step approach to human intention prediction: While a DNN predicts the probabilities of the next action, MCMC-based Bayesian inference is used to infer the underlying intention from these predictions.

Abstract

Predicting the next action that a human is most likely to perform is key to human-AI collaboration and has consequently attracted increasing research interests in recent years. An important factor for next action prediction are human intentions: If the AI agent knows the intention it can predict future actions and plan collaboration more effectively. Existing Bayesian methods for this task struggle with complex visual input while deep neural network (DNN) based methods do not provide uncertainty quantifications. In this work we combine both approaches for the first time and show that the predicted next action probabilities contain information that can be used to infer the underlying intention. We propose a two-step approach to human intention prediction: While a DNN predicts the probabilities of the next action, MCMC-based Bayesian inference is used to infer the underlying intention from these predictions. This approach not only allows for independent design of the DNN architecture but also the subsequently fast, design-independent inference of human intentions. We evaluate our method using a series of experiments on the Watch-And-Help (WAH) and a keyboard and mouse interaction dataset. Our results show that our approach can accurately predict human intentions from observed actions and the implicit information contained in next action probabilities. Furthermore, we show that our approach can predict the correct intention even if only few actions have been observed.

Inferring Human Intentions from Predicted Action Probabilities

TL;DR

This work proposes a two-step approach to human intention prediction: While a DNN predicts the probabilities of the next action, MCMC-based Bayesian inference is used to infer the underlying intention from these predictions.

Abstract

Predicting the next action that a human is most likely to perform is key to human-AI collaboration and has consequently attracted increasing research interests in recent years. An important factor for next action prediction are human intentions: If the AI agent knows the intention it can predict future actions and plan collaboration more effectively. Existing Bayesian methods for this task struggle with complex visual input while deep neural network (DNN) based methods do not provide uncertainty quantifications. In this work we combine both approaches for the first time and show that the predicted next action probabilities contain information that can be used to infer the underlying intention. We propose a two-step approach to human intention prediction: While a DNN predicts the probabilities of the next action, MCMC-based Bayesian inference is used to infer the underlying intention from these predictions. This approach not only allows for independent design of the DNN architecture but also the subsequently fast, design-independent inference of human intentions. We evaluate our method using a series of experiments on the Watch-And-Help (WAH) and a keyboard and mouse interaction dataset. Our results show that our approach can accurately predict human intentions from observed actions and the implicit information contained in next action probabilities. Furthermore, we show that our approach can predict the correct intention even if only few actions have been observed.
Paper Structure (14 sections, 3 equations, 5 figures)

This paper contains 14 sections, 3 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Experiment
  5. Datasets
  6. Watch-And-Help Dataset
  7. Keyboard and Mouse Interaction Dataset
  8. Experimental Settings
  9. Results
  10. User Intention Prediction
  11. Different Lengths of Observed Actions in Action Sequences
  12. Discussion
  13. Conclusion
  14. Acknowledgement

Figures (5)

  • Figure 1: Overview of our proposed method to predict human intentions. An agent observes the human actions and tries to infer the human's intention. Deep Neural Networks (DNNs) together with a Bayesian model infers the human intention.
  • Figure 2: Result of intention prediction of users on test set 1 in WAH dataset.
  • Figure 3: Result of intention prediction of users in keyboard and mouse interaction dataset.
  • Figure 4: Posterior mean probabilities and CI bounds when different percentages of observed actions in an action sequence are used for inference. The results on test set 1 in WAH dataset are shown.
  • Figure 5: Posterior mean probabilities and CI bounds when different percentages of observed actions in an action sequence are used for inference. The results in keyboard and mouse interaction dataset are shown.