Fine Grained Image SearchWeb App

Industry: OCR
Region: USA

TECHNOLOGIES

PHPReact Native

SERCVICE USED

  • Team Extension
  • Web Development
  • Mobile Development
  • UX/UI Design

Technology Stack

Introduction

In an increasingly clouded digital world, there is a need to refine searches to narrow down and streamline results. This poses a problem especially when a bulk of images are distinguishable on features that may not seem significant. This project seeks to introduce means and measures to help make that distinction and provide refined and relevant results and recommendations.

Challenge

There was a need for semantic image search using image captioning catering to the fashion domain. The reason for this is that there is not a single AI search engine tailored for Eastern/Pakistani online fashion stores where a user can search semantically similar products, meaning information and results are not as readily available.

Approach

Initially the project demanded a large amount of image data, then specific data representation which would be easy to read for modelling purposes. For Data Collection, clothing images were scrapped from different websites such as Khaadi and Sana Safinaz. For scrapping, different python libraries were used, such as Scrapy, Selenium etc. After this, we managed to gather data from 5 brands and a total of approximately 9,000 images from 20 different categories. 

The first model we used was text to image search. For this we set the train data in a way to save images and their respective text tags to convert the data into embeddings for easier representation. We then trained a model using an unsupervised method (CNN classified) in such a way to pick up an image and then map that image to text. When a user enters a textual description of a clothing item, this text is converted to a vector, matched with the most similar vector from a vector database of images to produce a result.

The second model we use is image to image search. An image encoder needs a vectorized representation of images which can learn the semantics of similar clothing images. We cannot use pretrained models such as Resnet since they are fine tuned for fashion datasets and so need to train a model which can learn fine grained differences between images of clothing products. We trained a model using the CNN method in a way that it takes image embeddings as input and maps it to a similar image class. When a user inputs an image query, we convert the image to a vector representation and find the most similar vectors from a database to return the most similar images.

Outcome

By employing the text to image and image to image search models, we managed to create a semantic image search process that provided refined and distinctive results on semantically similar products in the Pakistani fashion market allowing users access to the specific products that they were looking for.

Launch your product with us

Are you looking to start your own project?

Get Started

Explore More View All

PHPReact Native

Fine Grained Image Search

Semantic Image Search for Refined Results.

Industry: OCR
Region: USA
CSSHTMLJSLessPHPReact Native

Real Time Food Searching & Recommendation

Matching restaurant food items to their respective recipes to gauge nutritional values to users.

Industry: Food
Region: USA
DjangoMySQLPythonReact

Targeted Social Leads

Increasing productivity through a web application.

Industry: E-Commerce
Region: USA
Braden Ericson |

Founder & CEO Sparrow marketing

Datics team was great to work with. They were very proactive and helped me define the requirements for the project and were able to navigate the pivots I threw at them during our development period. They built a database, APIs, and an ML model beyond expectations. The team is actively fixing bugs that are popping up as we test and are a fun team to have weekly syncs with.

Call Us:

Schedule a call with an expert

Email Us:

Let’s start discussing