Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Find Matching Faces Based On Face Parameters

Setu A. Bhatt, Harshadkumar B. Prajapati, Vipul K. Dabhi, Ankush Tyagi

TL;DR

This work addresses the need for visual-based matchmaking by letting users specify facial attributes that guide a Text-To-Image generation model to synthesize a face; the generated face is detected and encoded into a $512$-dimensional embedding using FaceNet with InceptionResnetV1, and a vector similarity search against a Jeevansathi dataset stored in a vector database (Qdrant) retrieves the top $5$ matches. The end-to-end pipeline is integrated with a Gradio-based UI, uses MTCNN and Dlib for facial detection and landmarks, and demonstrates cross-model comparisons for image generation quality. The approach enables personalized, visually driven matching and can be adapted to domains like film casting and security, with potential for broader data integration and configurable datasets, while acknowledging ethical considerations and potential limitations in fine-grained facial attributes.

Abstract

This paper presents an innovative approach that enables the user to find matching faces based on the user-selected face parameters. Through gradio-based user interface, the users can interactively select the face parameters they want in their desired partner. These user-selected face parameters are transformed into a text prompt which is used by the Text-To-Image generation model to generate a realistic face image. Further, the generated image along with the images downloaded from the Jeevansathi.com are processed through face detection and feature extraction model, which results in high dimensional vector embedding of 512 dimensions. The vector embeddings generated from the downloaded images are stored into vector database. Now, the similarity search is carried out between the vector embedding of generated image and the stored vector embeddings. As a result, it displays the top five similar faces based on the user-selected face parameters. This contribution holds a significant potential to turn into a high-quality personalized face matching tool.

Find Matching Faces Based On Face Parameters

TL;DR

This work addresses the need for visual-based matchmaking by letting users specify facial attributes that guide a Text-To-Image generation model to synthesize a face; the generated face is detected and encoded into a -dimensional embedding using FaceNet with InceptionResnetV1, and a vector similarity search against a Jeevansathi dataset stored in a vector database (Qdrant) retrieves the top matches. The end-to-end pipeline is integrated with a Gradio-based UI, uses MTCNN and Dlib for facial detection and landmarks, and demonstrates cross-model comparisons for image generation quality. The approach enables personalized, visually driven matching and can be adapted to domains like film casting and security, with potential for broader data integration and configurable datasets, while acknowledging ethical considerations and potential limitations in fine-grained facial attributes.

Abstract

This paper presents an innovative approach that enables the user to find matching faces based on the user-selected face parameters. Through gradio-based user interface, the users can interactively select the face parameters they want in their desired partner. These user-selected face parameters are transformed into a text prompt which is used by the Text-To-Image generation model to generate a realistic face image. Further, the generated image along with the images downloaded from the Jeevansathi.com are processed through face detection and feature extraction model, which results in high dimensional vector embedding of 512 dimensions. The vector embeddings generated from the downloaded images are stored into vector database. Now, the similarity search is carried out between the vector embedding of generated image and the stored vector embeddings. As a result, it displays the top five similar faces based on the user-selected face parameters. This contribution holds a significant potential to turn into a high-quality personalized face matching tool.

Paper Structure

This paper contains 16 sections, 13 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Work
  4. Overview of proposed system architecture
  5. Data Collection and Storage
  6. Defining Face Parameters and Text Prompt Creation
  7. Image Generation Model
  8. Face Detection
  9. Feature Extraction and Vector Representation
  10. Similarity Matcher
  11. Experiment Results
  12. User Feature Selection and Text Prompt Creation
  13. Image Generation
  14. Face Detection and Feature Extraction
  15. Similar Images
  16. ...and 1 more sections

Figures (13)

  • Figure 1: Architecture of the proposed system
  • Figure 2: Gradio user interface for selection of face parameters by the user
  • Figure 3: Text prompt generated for the user-selected face parameters in Fig. \ref{['fig:GradioUI']}
  • Figure 4: Image generated based on the text prompt generated in Fig. 3
  • Figure 5: The result generated by Artples/LAI-ImageGeneration-vSDXL-2 from Hugging Face with compute time of 11.7 seconds.
  • ...and 8 more figures