Fast and Adaptive Questionnaires for Voting Advice Applications

TL;DR

The paper tackles the problem of long Voting Advice Applications (VAA) questionnaires causing user fatigue and incomplete responses. It introduces an adaptive questionnaire framework that embeds voters into a 2D latent space via an encoder/decoder, and uses an information-gain-based selector to choose the next question, evaluated on the Smartvote 2019 Swiss dataset. The IDEAL model for encoding/decoding combined with a PosteriorRMSE question selector yields substantial gains, achieving about $74\%$ accuracy after the same number of questions as the rapid version and reaching high accuracy earlier (e.g., $9$ questions) than fixed approaches. Predicting unanswered questions and using those predictions in the nearest-neighbor matching further improves candidate recommendations, suggesting a practical and transparent enhancement for VAAs with potential applicability to other survey domains. The work provides a benchmark across spatial models and selection methods and demonstrates that adaptive questioning can dramatically increase accuracy while reducing respondent burden.

Abstract

The effectiveness of Voting Advice Applications (VAA) is often compromised by the length of their questionnaires. To address user fatigue and incomplete responses, some applications (such as the Swiss Smartvote) offer a condensed version of their questionnaire. However, these condensed versions can not ensure the accuracy of recommended parties or candidates, which we show to remain below 40%. To tackle these limitations, this work introduces an adaptive questionnaire approach that selects subsequent questions based on users' previous answers, aiming to enhance recommendation accuracy while reducing the number of questions posed to the voters. Our method uses an encoder and decoder module to predict missing values at any completion stage, leveraging a two-dimensional latent space reflective of political science's traditional methods for visualizing political orientations. Additionally, a selector module is proposed to determine the most informative subsequent question based on the voter's current position in the latent space and the remaining unanswered questions. We validated our approach using the Smartvote dataset from the Swiss Federal elections in 2019, testing various spatial models and selection methods to optimize the system's predictive accuracy. Our findings indicate that employing the IDEAL model both as encoder and decoder, combined with a PosteriorRMSE method for question selection, significantly improves the accuracy of recommendations, achieving 74% accuracy after asking the same number of questions as in the condensed version.

Figures (19)

• Figure 1: Latent space of the VAE with a 2-layer decoder. The background color indicates model predictions. ( A) Train-set candidates are colored by their party. The non-linear decoder predicts answers to question 27: "Should same-sex couples have the same rights as heterosexual couples in all areas?" as indicated by the background color. ( B) Train-set candidates are colored by their given answer to question 13: "Should insured persons contribute more to healthcare costs (e.g., by increasing the minimal deductible)?" The answers are 4-Likert scale from "Fully Agree" (yellow) to "Fully Disagree" (dark blue).
• Figure 2: IDEAL embedding of the binary Smartvote dataset. Dots represent candidates of their respective party. ( A) Likelihood function $P(Y|X)$ for question 5: "Should Switzerland terminate the Schengen Agreement with the EU, in order to reintroduce more security checks directly on the border?" ( B) Posterior distribution $P(X|Y_I)$ for test candidate 459 (highlighted by a black circle and arrow) after answering 7 random questions $y\in Y_I$. Other dots represent maximum likelihood estimators for all test candidates with equally many answers given. The dashed white circle corresponds to the standard deviation ellipse of the Gaussian prior P(X).
• Figure 3: W-NOMINATE embedding with 60% missing data for question 6: "Should wealthy individuals contribute more to the funding of the state?" The closer candidates are to the YAY ($\blacktriangle$) and NAY ($\blacktriangledown$) locations, the more likely they vote accordingly. The 50% decision boundary is shown as a dashed line. Candidates are colored by their party.
• Figure 4: Comparison of the candidate recommendation accuracy based on given answers only (dashed lines; Type I) and predicting remaining answers first (solid lines, Type II). Each line corresponds to the mean accuracy of all test users after the respective number of questions, where the uncertainty shading indicates the confidence interval of the mean estimator. The horizontal lines show the final accuracies of the RapidVersion. For clarity, only a selection of methods is shown.
• Figure 5: Comparison of selection methods in an asynchronous setting. Each method can prioritize voter-question pairs where they assume acquired information most useful. In the calculation of the accuracy, only the remaining questions are considered.