Kaggle Chronicles: 15 Years of Competitions, Community and Data Science Innovation
Kevin Bönisch, Leandro Losaria
TL;DR
The paper investigates Kaggle's 15-year trajectory, leveraging Meta Kaggle datasets to map user growth, community dynamics, and technological evolution within the data science ecosystem. It combines longitudinal kernel and discussion analyses with topic modeling to reveal shifts from a competition-centric platform to an open-learning community, including anomaly detection for events and insights into platform governance via leaderboard dynamics. Key findings include sustained Python dominance, increasing diversity of tools and techniques, rapid adoption of transformers and AutoML, and generally robust generalization of models despite occasional public/private leaderboard gaps. The work provides evidence that Kaggle serves as a scalable, empirical benchmark and learning environment that informs both AI practice and platform design, complemented by publicly available datasets and reproducible analyses.
Abstract
Since 2010, Kaggle has been a platform where data scientists from around the world come together to compete, collaborate, and push the boundaries of Data Science. Over these 15 years, it has grown from a purely competition-focused site into a broader ecosystem with forums, notebooks, models, datasets, and more. With the release of the Kaggle Meta Code and Kaggle Meta Datasets, we now have a unique opportunity to explore these competitions, technologies, and real-world applications of Machine Learning and AI. And so in this study, we take a closer look at 15 years of data science on Kaggle - through metadata, shared code, community discussions, and the competitions themselves. We explore Kaggle's growth, its impact on the data science community, uncover hidden technological trends, analyze competition winners, how Kagglers approach problems in general, and more. We do this by analyzing millions of kernels and discussion threads to perform both longitudinal trend analysis and standard exploratory data analysis. Our findings show that Kaggle is a steadily growing platform with increasingly diverse use cases, and that Kagglers are quick to adapt to new trends and apply them to real-world challenges, while producing - on average - models with solid generalization capabilities. We also offer a snapshot of the platform as a whole, highlighting its history and technological evolution. Finally, this study is accompanied by a video (https://www.youtube.com/watch?v=YVOV9bIUNrM) and a Kaggle write-up (https://kaggle.com/competitions/meta-kaggle-hackathon/writeups/kaggle-chronicles-15-years-of-competitions-communi) for your convenience.