Beauty Beyond Words: Explainable Beauty Product Recommendations Using Ingredient-Based Product Attributes

TL;DR

This work presents a system to extract beauty-specific attributes using end-to-end supervised learning based on beauty product ingredients using a novel energy-based implicit model architecture that offers significant benefits in terms of accuracy, explainability, robustness, and flexibility.

Abstract

Accurate attribute extraction is critical for beauty product recommendations and building trust with customers. This remains an open problem, as existing solutions are often unreliable and incomplete. We present a system to extract beauty-specific attributes using end-to-end supervised learning based on beauty product ingredients. A key insight to our system is a novel energy-based implicit model architecture. We show that this implicit model architecture offers significant benefits in terms of accuracy, explainability, robustness, and flexibility. Furthermore, our implicit model can be easily fine-tuned to incorporate additional attributes as they become available, making it more useful in real-world applications. We validate our model on a major e-commerce skincare product catalog dataset and demonstrate its effectiveness. Finally, we showcase how ingredient-based attribute extraction contributes to enhancing the explainability of beauty recommendations.

Figures (6)

• Figure 1: Overview of beauty product extraction workflow and the BT-BERT architecture. Our model is identical to the BERT Transformer bert except in the last layer---the initial N-1 layers remain unmodified. We remove the final MLP from the last layer of the Transformer encoder and directly use the self-attention values to formulate the output probability.
• Figure 2: Skincare recommendation with explainable ingredient for each attribute.
• Figure 3: Difference between implicit and explicit models. Left: In implicit models, the model intakes query attribute together with product ingredients and title. Note that in our case, the output logits come directly from the self-attention values of the last encoder layer. Right: Explicit models represent the standard way of fine-tuning the BERT model, where a classifier is attached to the end of the Transformer.
• Figure 4: Label Distribution across Product Type in our dataset. The height of each bar indicates the number of products associated with the respective attribute. For instance, there are a total of 1809 out of 11580 products for Dry Skin.
• Figure 5: Validation accuracy training on various sizes of dataset