Sometimes you see something really cool. You capture the moment. Years passes by and one day out of the blue, you just get curious about that thing. You have no idea how to describe that object, but you have the image. What to do? Well, few years back, Google came up with a new tool that lets you find all those memories that you do not have words to describe.
Google’s search engine has been one of the few things that really fascinate me and that is the reason why I am writing this article.
The idea of this write-up is to familiarize you with two most common image search algorithms – the Euclidian distance method and Siamese feature method. We will build “Image search application” or so to say “Image similarity application” using both these algorithms.
Image Search Application Using Euclidian Distance Method
In the first part of the article, we will explore the concept of finding similarity between digital images using the Euclidian Distance Method. It won’t be as good as Google’s search engine (of course) because of the technique we will be using to find similarity between images, but I promise, it will be pretty cool.
A detailed notebook with the presented code is present at the following location : Github Repo
So, let’s get started.
By now you have pretty good idea of what I am going to say.
Yes, right. First thing first, let’s import all the necessary modules.
Now, let’s take an image. What we want to do is we want to find all other similar images from a set of images (call it our dataset).
Here is what our test image looks like:
Now we will find the histogram corresponding to this image, which is basically a 1D array of numbers (Every image can be represented in terms of pixel, where each pixel is a number representing intensity of color).
And here is what it looks like:
Now we will go to the folder where set of images are located (dataset). We will read them one by one and calculate histogram (basically a 1D array of numbers). I have taken 7 images in dataset, so we will have 7 arrays.
Now, all these arrays can be considered as vector and so it is pretty easy to calculate Euclidian distance for each one of these 7 vectors from the original array. Euclidian distance is the square root of the sum of the square of corresponding individual differences between numbers in the arrays.
Now that we have the Euclidian distance for each of the images, the one with the minimum distance will be most similar to our original image.
Why?
Because the array corresponding to the images represent some information about them, when you are calculating some kind of difference, you are comparing this information. Less is the distance, more similar the images.
This is the image with the least distance and corresponding image and histogram look something like this.
Well, both the images are not looking very similar, but both the images are of a cat, are right?
Fun Fact: Our dataset also has images of cows and dog, so this result could be said to be quite satisfactory with the amount of work that we have put. J
No, but jokes apart such a simple algorithm producing such good result is quite a thing.
But as they say- Simplicity is good for simple things. To deal with tough tasks, you have to be tough. The Euclidian distance method fits into this criterion. This algorithm works well in a single dimension, but fails to work with multidimensional data. Since it gives equal importance to all the directions, dimensionality cannot be captured.
In addition to that, with the amount of the dataset that we have, it becomes difficult to calculate and compare individual distance. With the emergence of GPUs and tensor flow, it is easier to apply sophisticated models on a large number of datasets and train them in a very short time period.
Next, we are going to use tensor flow and apply the Siamese feature extraction method to find similarity between images.
Siamese Network Features For Image Matching
The second algorithm that we are going to use is Siamese Feature Network.
What is Siamese Feature Network?
It is a Convolutional Neural Network. It extracts features from a particular image and compares it with the features of the original image to identify similarity. This is calculated using cosine similarity method. However, there is no hard and fast rule as to what similarity method we have to use, as long as we are able to extract good features from the images.
The idea behind Convolutional Neural Network has already been discussed in detail (Reference Code here).
Let’s start
As usual, import all the necessary modules:
Load the dataset. This dataset is from the fashion industry. Similar to an MNIST dataset it is also classified into 10 classes like – shoes, shirts etc. We will divide the dataset into train and test images.
The next step is to define the Siamese Network, which is basically a Convolutional Neural Network. We already have seen how to define it (link of CNN blog). However you can play around with the model if you want to.
Next, we will read the training images and extract features from them using the Siamese Network. Also it is always a good idea to save the model so that we can reuse it later.
Now, we have training features corresponding to all the images in thetraining set. Next, take a random image from test set and calculate feature using the Siamese Network.
Here is the test image we are going to find similarity with.
We are not able to see clearly whether the given 5 images are similar or not with the original, because of the quality of the dataset.
We have presented the Siamese Network to perform image similarity on a given reference image. We achieved this by training a multi-scale Siamese Convolutional Network which is better at finding fine-grained image similarities than traditional CNNs. This model is able to build high quality embedding of images and therefore is better at finding similarity between images.
Now that you are familiar with the two most common algorithms used for identifying similarities between two images, you can start building “Image Search Applications” on your own, using your own dataset.
References:
