Is it getting harder to make a hit? Evidence from 65 years of US music chart history

Abstract

Since the creation of the Billboard Hot 100 music chart in 1958, the chart has been a window into the music consumption of Americans. Which songs succeed on the chart is decided by consumption volumes, which can be affected by consumer music taste, and other factors such as advertisement budgets, airplay time, the specifics of ranking algorithms, and more. Since its introduction, the chart has documented music consumerism through eras of globalization, economic growth, and the emergence of new technologies for music listening. In recent years, musicians and other hitmakers have voiced their worry that the music world is changing: Many claim that it is getting harder to make a hit but until now, the claims have not been backed using chart data. Here we show that the dynamics of the Billboard Hot 100 chart have changed significantly since the chart's founding in 1958, and in particular in the past 15 years. Whereas most songs spend less time on the chart now than songs did in the past, we show that top-1 songs have tripled their chart lifetime since the 1960s, the highest-ranked songs maintain their positions for far longer than previously, and the lowest-ranked songs are replaced more frequently than ever. At the same time, who occupies the chart has also changed over the years: In recent years, fewer new artists make it into the chart and more positions are occupied by established hit makers. Finally, investigating how song chart trajectories have changed over time, we show that historical song trajectories cluster into clear trajectory archetypes characteristic of the time period they were part of. The results are interesting in the context of collective attention: Whereas recent studies have documented that other cultural products such as books, news, and movies fade in popularity quicker in recent years, music hits seem to last longer now than in the past.

Figures (4)

• Figure 1: Summary of changes in songs' lifetimes. The figure shows statistics on songs' lifetimes and their changes over time. A The average (blue), median (orange), and the curve indicating the 20th percentile (green) of the lifespans of all songs over time. The shaded areas represent the standard errors of the means. The dashed grey lines indicate some of music history’s most important technology launches. All three plotted song summary statistics have been going up until the 2000s and then started to drop. The median and the 20th percentile have fallen below 5, achieving the lowest values in the chart’s history. The drop in the last 2 years can be partially affected by the songs that have just entered the chart, but their lifetime is yet to be established (they last longer than the end of our data frame). B presents the lifetime distributions as violin plots for different decades. The white dot represents the median, and the dark rectangle shows the interquartile range. The colored areas represent the densities of the distributions. As time progresses, the distributions become thinner and get longer tails. One exception is the formation of clear bumps around week 1 and week 20, which indicate songs that left the chart immediately after entering it, and "recurrent songs", which have spent 20 weeks on the Hot 100 and fallen below position number 50 leading to the song's removal from the chart chartlegend. The long tails of the distributions capture the extreme values of the data. In the last decades, the longest-living songs last longer than in previous decades (in the 2010s, reaching four times longer lifetimes than the upper bound of the interquartile range). C The bins represent the average weeks on the chart before the peak (blue) and after the peak (orange). In the first three decades, the blue bars dominated the orange bars, showing that songs quickly disappeared after reaching the peak. Since the 1990s, the bars have been of nearly equal height, except for the top 10, where the average for future weeks is twice higher than for the past weeks. This shows that currently, the best-performing songs linger on for much longer.
• Figure 2: Trajectory and archetype analysis. Key aspects of songs' trajectories and their evolution over time. A The proportion of songs from each proposed archetype over the years. The results were normalized with a half-year rolling window. The groups are brief songs (blue), climbing songs (orange), high-end songs (green), high-start songs (red), and long-lasting (purple). The dashed grey lines indicate the timing of some of the most important technological innovations in music listening history. First, high-end songs dominated until the mid-1970s when the climbing flow took over. Since the 1990s, the climbing songs have become less dominant, now competing with long-lasting, high-start, and the revitalized high-end songs. Over the course of the chart’s history, the prevalence of short songs has steadily declined from 30% to approximately 10%. B Normalized trajectories and the number of songs in each cluster. The trajectories were calculated by averaging the positions of all songs in each week following their first appearance on the chart. The plots summarize the visual aspects of detected archetypes. One thing that stands out is that the clusters are not evenly sized. The brief songs group is almost two times bigger than the second-largest, climbing flow. C Normalized trajectories of songs grouped by top position of the songs. Grouped together are songs whose top position were in top 10 (blue), positions 21-30 (orange), positions 41-50 (green), positions 71-80 (red), and positions 91-100 (purple). Only a few position ranges were chosen for better readability of the figure. The missing intervals lie between the visible ones (e.g., the 11-20 curve lies between the top 1-10 and the top 21-30). The songs that disappear from the charts have the rest of their trajectories filled with 101. As time passes, the average curves widen and start higher at week zero. There is also a visible drop in the 20 weeks, explained in the Billboard Hot 100 as the “recurrent” status chartlegend
• Figure 3: Weekly rank changes over time.A Histograms of weekly position changes over decades. The red dashed lines represent the median of the samples. The grey dashed lines represent the samples’ 2.5th and 97.5th percentiles, respectively. The shaded grey areas show the elements smaller than 2.5% of the data and greater than 97.5% of the data. During the first three decades, the median skewed towards the right, but it ultimately shifted towards zero in later years. Moreover, the distributions got more symmetrical with time, whereas before they were concentrated on positive position differences. B Proportion of small jumps for songs lying in 1-10 (blue), 20-30 (orange), 50-60 (green), and 80-90 (red) songs. The values have been smoothed with a moving average of 3 years. We show these 4 groups of songs for ease of readability and to align with the previous analysis of the best-ranking positions. The overall proportion of small jumps started to drop around the 2000s, but for the top 10, it stabilized at around 80%. For the lowest positions, the proportion drastically dropped to around 30%. C Skewness of yearly position change distributions. The plots have been smoothed with a rolling average of 3 years. The overall skewness has been changing a lot throughout the years. There are fewer fluctuations in the center values. In the first half, the majority of them register values below 0, while in the second half, they record values close to 0. D Heatmap of similarities between yearly position change distributions. The similarities were calculated as Bhattacharyya coefficients between distributions from different years. The colors indicate the similarity between distributions from one year (x-axis) and another year (y-axis). The lightest colors exhibit great similarity, while the darkest do the contrary. The biggest distribution overlap can be spotted between the years in the last three decades (bottom right corner), while the largest disparities are found between the 70s and the period after the 80s, and the 90s and the years before them.
• Figure 4: Artists on the Billboard Hot 100 chart. The figures show various statistics on artists on the chart. A Distribution of the number of songs on the chart per artist. The red arrows point to the outliers with very high song count. The counts have been presented on a logarithmic vertical scale, as the vast majority of artists that get a songs on the Billboard Hot 100 chart achieve this feat for only 1 or 2 songs (80% of all artists that get on the chart). The top 3 artists (Glee Cast, Taylor Swift, and Drake) are far from the rest of the distribution. B Proportion of hitmakers on the charts relative to all artists (blue) and those who achieved the top 10 (orange) over the years. The figure was smoothed with a 3-year moving average. In the 2000s, only 10% of the artists in the top 10 and 40% overall had more than ten songs on the charts. Today, hitmakers account for 60% of all artists in the charts and over 40% of the top 10. C Proportion of features/collaborations over time. The blue curve presents the actual values, and the orange is smoothed with a 5-year window. The ratio was more or less steady till the 1990s, fluctuating around 10%. However, since the late 1990s, it has rapidly increased, achieving around 50% in 2022. D Proportion of new artists (artists appearing for the first time on the weekly chart) over the years. The proportion of new artists without features is decreasing with time. Nowadays it is around 30%. E Proportion of individual artists with different numbers of songs. The different ranges of the number of songs are: $\le 5$ (blue), $5<x\le 15$ (orange), $15<x\le30$ (green), and more than $30$ (red). Plots have been smoothed with a 5-year moving average. The ratio of the smallest number of songs (blue) is naturally very high at the beginning of the chart’s history. It has dropped to around 20% in the last years. The medium values (orange and green) have been fluctuating between 20% and 40%. The red curve, which represents artists with over 30 songs on the Billboard Hot 100, has visibly increased in the last two decades. F Top positions of songs by Madonna and Taylor Swift. The x-axis represents the nth song of the artists, and the y-axis its top position. Many of Madonna's songs reached the top 1 and they rarely achieved worse positions than 50. For Taylor Swift there are more frequent fluctuations in their top positions - some of her songs did not even reach top 80. G Average top position of hitmakers' songs over the years. Their songs performed, on average, slightly better in the 1980s-2000s period, compared to nowadays. In recent years the mean position of hitmakers' songs has dropped to around 50.