An Empirical Study on Common Defects in Modern Web Browsers Using Knowledge Embedding in GPT-4o
Rahul Singh, Yousuf Sultan, Tajmilur Rahman, Sri Vidya Puttareddygari
TL;DR
This paper addresses the problem of understanding common defects in modern web browsers by benchmarking two major engines, Firefox and Chromium, through an empirical study that contrasts LLM-driven bug categorization (GPT-4.o knowledge embedding) with traditional NLP (TF-IDF+K-Means). It constructs a large, linked data ecosystem from repository mining, commits, bug reports, and component mappings, and then uses contextual browser knowledge to classify bugs into meaningful categories. The key finding is that the LLM approach achieves a higher F1 score ($94.63\%$) than NLP ($64.01\%$) and reveals broader defect patterns, including highly defect-prone components and substantial cross-browser similarities and differences. The study provides actionable guidance for developers to prioritize testing and maintenance, highlights Firefox as having more defect-prone components, and suggests avenues for future work such as temporal trend analysis and automated LLM-based bug triaging to enhance browser reliability and resilience.
Abstract
Technology is advancing at an unprecedented pace. With the advent of cutting-edge technologies, keeping up with rapid changes are becoming increasingly challenging. In addition to that, increasing dependencies on the cloud technologies have imposed enormous pressure on modern web browsers leading to adapting new technologies faster and making them more susceptible to defects/bugs. Although, many studies have explored browser bugs, a comparative study among the modern browsers generalizing the bug categories and their nature was still lacking. To fill this gap, we undertook an empirical investigation aimed at gaining insights into the prevalent bugs in Google Chromium and Mozilla Firefox as the representatives of modern web browsers. We used GPT-4.o to identify the defect (bugs) categories and analyze the clusters of the most commonly appeared bugs in the two prominent web browsers. Additionally, we compared our LLM based bug categorization with the traditional NLP based approach using TF-IDF and K-Means clustering. We found that although Google Chromium and Firefox have evolved together since almost around the same time (2006-2008), Firefox suffers from high number of bugs having extremely high defect-prone components compared to Chromium. This exploratory study offers valuable insights on the browser bugs and defect-prone components to the developers, enabling them to craft web browsers and web-applications with enhanced resilience and reduced errors.