Examining DOM Coordinate Effectiveness For Page Segmentation

TL;DR

The paper addresses how to construct effective page segmentation vectors from DOM information and questions the dominance of visual coordinates. It introduces four new DOM coordinates (Tag Depth, Div ID, Tag Group, Data Index), evaluates them against visual cues using OPTICS and HDBSCAN on 44 websites with 23410 DOM nodes, and finds that DOM-informed coordinates generally outperform visual ones, with simple single-coordinate vectors often being the strongest performers. Importantly, the authors demonstrate that per-page matching of the best coordinate vector and clustering algorithm yields a Rand score as high as 0.74, underscoring the benefit of adaptive, DOM-based segmentation strategies. These findings challenge prevailing practices and suggest that diverse, page-specific approaches can significantly enhance automated extraction and reasoning in data-hungry systems like LLMs and search engines.

Abstract

Web pages form a cornerstone of available data for daily human consumption and with the rise of LLM-based search and learning systems a treasure trove of valuable data. The scale of this data and its unstructured format still continue to grow requiring ever more robust automated extraction and retrieval mechanisms. Existing work, leveraging the web pages Document Object Model (DOM), often derives clustering vectors from coordinates informed by the DOM such as visual placement or tree structure. The construction and component value of these vectors often go unexamined. Our work proposes and examines DOM coordinates in a detail to understand their impact on web page segmentation. Our work finds that there is no one-size-fits-all vector, and that visual coordinates under-perform compared to DOM coordinates by about 20-30% on average. This challenges the necessity of including visual coordinates in clustering vectors. Further, our work finds that simple vectors, comprised of single coordinates, fare better than complex vectors constituting 68.2% of the top performing vectors of the pages examined. Finally, we find that if a vector, clustering algorithm, and page are properly matched, one can achieve overall high segmentation accuracy at 74%. This constitutes a 20% improvement over a naive application of vectors. Conclusively, our results challenge the current orthodoxy for segmentation vector creation, opens up the possibility to optimize page segmentation via clustering on DOM coordinates, and highlights the importance of finding mechanisms to match the best approach for web page segmentation.

Figures (7)

• Figure 1: New Coordinates Required to Break Ambiguity
• Figure 2: DOM/text Analysis of Surveyed Web Pages
• Figure 3: Example of Clustering Approach for Annotations
• Figure 4: Clustering Algorithms Performance
• Figure 5: Example of DOM Improved Associations.