Document Classification using File Names

TL;DR

The paper tackles the computational bottleneck of large-scale document classification by showing that many documents can be accurately categorized using only their file names. It introduces a lightweight pipeline combining a universal tokenization scheme, TF-IDF guided keywords added to a trie (and optionally to a BERT tokenizer), and a set of fast classifiers, notably a Random Forest with a trie tokenizer. By leveraging a confidence-based deferral mechanism, the approach defers ambiguous or out-of-scope cases to heavier content-based models, achieving up to 99.78% accuracy at a 0.9 prediction rate on indicative file names and 96.57% on Common Crawl data, with inference times around $1.23\\times 10^{-4}$ seconds per prediction and up to $442\\times$ speedup over heavy models like DiT. The work demonstrates significant practical impact for fast, scalable document classification and introduces two publicly available datasets for further study, while outlining limitations and avenues for future enhancement through metadata integration.

Abstract

Rapid document classification is critical in several time-sensitive applications like digital forensics and large-scale media classification. Traditional approaches that rely on heavy-duty deep learning models fall short due to high inference times over vast input datasets and computational resources associated with analyzing whole documents. In this paper, we present a method using lightweight supervised learning models, combined with a TF-IDF feature extraction-based tokenization method, to accurately and efficiently classify documents based solely on file names, that substantially reduces inference time. Our results indicate that file name classifiers can process more than 90% of in-scope documents with 99.63% and 96.57% accuracy when tested on two datasets, while being 442x faster than more complex models such as DiT. Our method offers a crucial solution to efficiently process vast document datasets in critical scenarios, enabling fast and more reliable document classification.

Figures (5)

• Figure 1: Overview of the file name classification task. An ideal classifier can categorize indicative file names correctly, while also being able to defer predictions on file names that lack enough category information.
• Figure 2: An example of the universal tokenizer tokenizes a fileame with the trie tokenizer and BERT Autotokenizer.
• Figure 3: The confidence score distribution for Random Forest classifier with trie tokenizer on indicative and ambiguous file names from the Common Crawl dataset.
• Figure 4: Overall Accuracy versus Total Inference Time (seconds) per file name at all confidence score threshold between 0 and 1, in increments of 0.001, for a Random Forest classifier with the trie tokenizer predicting on all the file names in the Common Crawl dataset, with a DiT model processing the deferred documents.
• Figure 5: Accuracy of the Random Forest file name classifier with a trie tokenizer with all k values between 0 and 0.3, incremented by 0.001, tested on the Web Search dataset.