Riding the Carousel: The First Extensive Eye Tracking Analysis of Browsing Behavior in Carousel Recommenders

TL;DR

This work provides the first extensive analysis of the eye tracking behavior in carousel recommenders under the free-browsing setting and provides the first extensive empirical results of eye tracked browsing behavior in carousels for improving recommenders.

Abstract

Carousels have become the de-facto standard user interface in online services. However, there is a lack of research in carousels, particularly examining how recommender systems may be designed differently than the traditional single-list interfaces. One of the key elements for understanding how to design a system for a particular interface is understanding how users browse. For carousels, users may browse in a number of different ways due to the added complexity of multiple topic defined-lists and swiping to see more items. Eye tracking is the key to understanding user behavior by providing valuable, direct information on how users see and navigate. In this work, we provide the first extensive analysis of the eye tracking behavior in carousel recommenders under the free-browsing setting. To understand how users browse and model their behavior, we examine the following research questions : 1) where do users start browsing, 2) how do users transition from item to item within the same carousel and across carousels, and 3) how does genre preference impact transitions? This work addresses a gap in the field and provides the first extensive empirical results of eye tracked browsing behavior in carousels for improving recommenders. Taking into account the insights learned from the above questions, our final contribution is to provide takeaways for carousel recommender system designers to better optimize their systems for user browsing behavior. The most important being an improved reordering of the ranked item positions to account for browsing behavior after swiping. These contributions aim not only to help improve current systems, but also to encourage and allow the design of new user models, systems, and metrics that are better suited to the complexity of carousel interfaces.

Figures (8)

• Figure 1: Aggregate first fixation heat map showing initial browsing bias on a to-scale background (blue boxes represent the 5 visible movie posters in each carousel). The first fixations using x,y pixel position from all users and free-browsing screens are aggregated from more to less fixations (red > orange > yellow > green).
• Figure 2: Row fixation transitions matrix for all users and free-browsing screens. It shows the number of instances users examined one row (a carousel genre/list) and afterwards examined a different row. Fixations on a genre label and the swipe button (of that row) as well as the items (swiped/unswiped) in that carousel are considered part of the same row. All fixations on the background were ignored (e.g. Row 1 to background to Row 2 is counted as Row 1 to Row 2). (a), (b), (c) visualize the main results in subareas of the matrix. Blue arrows compare transitions from a given row (to a previous/above row vs. to a subsequent/below row) and white arrows compare inverse transitions (from a given row to the previous one vs. from the previous one to the given row). The arrow points towards a higher value, i.e. it indicates a more prevalent pattern / behavior.
• Figure 3: Column fixation transitions matrix for all users and free-browsing screens. It shows the number of instances users examined an item column (one of the 15 movie rank positions independent of genre) and afterwards examined a different item. All fixations on the background, genre label, and swipe buttons are ignored. Note that Columns 1 through 5 are the first set of movies, Columns 6 through 10 are the second set of swiped movies, and Columns 11 through 15 are the third set. When swiping, the columns shift 5 places (e.g. for a right swipe, Column 1 is replaced by Column 6). Thus, the top left, center, and bottom right areas of the matrix independently visualize a user transitioning without swiping. (a)-(d) visualize the main results in subareas of the matrix.
• Figure 4: Joint row-column fixation transitions for all users and free-browsing screens. Each arrow is a item-item transition through eye movement, colored by the transition conditional probability (given the starting item position). Transitions are aggregated across swiping to be in line with the true browsing path of users. This means that in Genre 1 the arrow from the first item to the second item represents the transitions in Row 1 for item 1-2, 6-7, and 11-12, while the arrow from the first item of Genre 1 to the first item of Genre 2 represents all of the possible transitions from Row 1 item 1, 6, or 11 to Row 2 item 1, 6, or 11.
• Figure 5: Fixation heat map of each movie item for all users and free-browsing screens. We remove all repeat fixations on the same movie and count only fixations transitions to other movies and the initial movie fixation. Only fixations for Movie AOIs are used.