EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction

TL;DR

EAVE tackles the efficiency challenge of product attribute value extraction by decoupling encoding into a heavy context/attribute encoder for non-interacting representations and a lightweight encoder for interacting representations, combined via a sparse-layer fusion. The approach enables caching of heavy representations and lightweight, targeted interactions, achieving near-state performance with about an order of magnitude reduction in computation on long contexts and many attributes. Extensive experiments on MAVE and AE-110K, along with ablations, show robust efficiency gains with only small F1 losses and provide insights into fusion strategies and layer mapping. This work offers a practical path to scalable, real-world AVE in large catalogs and supports broader applicability to other context-query tasks.

Abstract

Product attribute value extraction involves identifying the specific values associated with various attributes from a product profile. While existing methods often prioritize the development of effective models to improve extraction performance, there has been limited emphasis on extraction efficiency. However, in real-world scenarios, products are typically associated with multiple attributes, necessitating multiple extractions to obtain all corresponding values. In this work, we propose an Efficient product Attribute Value Extraction (EAVE) approach via lightweight sparse-layer interaction. Specifically, we employ a heavy encoder to separately encode the product context and attribute. The resulting non-interacting heavy representations of the context can be cached and reused for all attributes. Additionally, we introduce a light encoder to jointly encode the context and the attribute, facilitating lightweight interactions between them. To enrich the interaction within the lightweight encoder, we design a sparse-layer interaction module to fuse the non-interacting heavy representation into the lightweight encoder. Comprehensive evaluation on two benchmarks demonstrate that our method achieves significant efficiency gains with neutral or marginal loss in performance when the context is long and number of attributes is large. Our code is available \href{https://anonymous.4open.science/r/EAVE-EA18}{here}.

Figures (5)

• Figure 1: An example of product associated with multiple attributes and their corresponding values extracted from the product context.
• Figure 2: Overview of our EAVE model. (a) There are three key components: 1) Heavy model is used to encode the context and attribute, learning their non-interacting representations. 2) Light model generates the interacting representations of the concatenated context and attribute. 3) Sparse-layer interaction fuses the two representations from heavy and light encoders through sparse layer mapping (detailed in (b)).
• Figure 3: Impact of the sparse-layer interaction by varying the fusion weight $\alpha$ on both dataset.
• Figure 4: Impact of the heavy-light learning rate ratio $\beta$ on the MAVE and AE-110K dataset. For MAVE, $\alpha$ is fixed to 0.7. For AE-110K, $\alpha$ is fixed to 0.05.
• Figure 5: Precision, recall, and F1 gaps from the EAVE to the Transformer baseline on the MAVE dataset. We use a fixed $\alpha=0.7$ and $\beta=1.0$.