ICLR14: J Masci: Sparse similarity-preserving hashing

What is discussed, and when

1. 0:00 – 3:00: Feature space, Similarity-sensitive hashing, Precision, Binary vectors
2. 3:00 – 6:00: Sparsity, Precision, Recall, Hamming radius
3. 6:00 – 9:00: Multi-modal data, Cross-modal learning, Precision, Recall
4. 9:00 – 12:00: Cross SSH, Multi-modal data, Semantic similarity
5. 12:00 – 14:32: Precision, Recall, Collision

Notable quotes from "ICLR14: J Masci: Sparse similarity-preserving hashing"

• Similarity can describe them so given this annotated data what we want to do we want to learn function H which brings us from the feat space to the handing space so we want to…

  about Similarity-sensitive hashing · at 0:59

• Receival system that you would like to have three things efficient search in terms of speed high Precision high but unfortunately these are three sing TRS and these are two plots which have taken from ground Fus which are trying.

  about Precision · at 2:23

• Increase Cod which means that you end up having very low Cod so what we argue here in this work is that we need to AR structure to the space of these codes so that we can limit…

  about Sparsity · at 3:08

• Learn and we can C this we can learn it in a very Manner and as you're interest SP codes what we use is a particular type of architecture which is the Lista Network which…

  about Lista Network · at 4:02

• Neighbors and that's what I'm trying to show here so here I'm showing Precision curves for all the methods when we consider the Heming radius equals to the Cod l in this case…

  about Hamming radius · at 5:08

• Problem for simar cash dealing with multimod data every day I mean we're surrounded by a lot of data which comes in different mity and this is just an example fromia reading…

  about Multi-modal data · at 7:28

• Cross model l which is basically a meta Simon Network and what is nice is that H and G are join to Le so you don't have to resolve these two step optimizations and so on and after the.

  about Meta Siamese Network · at 8:58

• Filter which is an so it makes plausible to assign that image to the category heart and obiously you can do a convert so you can have an image you can f it system and then in…

  about Cross-modal learning · at 10:13

Who and what is mentioned in this episode?

Companies & orgs

• Facebook — Company · Case study · 1 mention · first at 0:14
  Example of a platform with large-scale data storage needs.
  What I mean byar SC problems here are just for a few numbers so as you can see YouTube stores something like 40 million hours of recordings per year Facebook stores millions of…
• YouTube — Company · Case study · 1 mention · first at 0:14
  Example of a platform with large-scale data storage needs.
  What I mean byar SC problems here are just for a few numbers so as you can see YouTube stores something like 40 million hours of recordings per year Facebook stores millions of…

Concepts & topics

• Similarity-sensitive hashing — Concept · Core topic · 2 mentions · first at 0:32
  Primary technique discussed for efficient indexing and retrieval in large-scale vision problems.
• Sparsity — Concept · Framework · 3 mentions · first at 3:13
  Technique used to limit the degrees of freedom in codes for better performance.
• Multi-modal data — Concept · Core topic · 2 mentions · first at 7:30
  Data that comes in different modalities, such as text and images.
• Precision — Concept · Key argument · 5 mentions · first at 2:27
  One of the key metrics discussed for evaluating retrieval systems.
• Recall — Concept · Key argument · 4 mentions · first at 3:04
  One of the key metrics discussed for evaluating retrieval systems.
  Receival system that you would like to have three things efficient search in terms of speed high Precision high but unfortunately these are three sing TRS and these are two plots which have taken from ground Fus which are trying.
• Cross-modal learning — Concept · Strategy · 1 mention · first at 8:58
  Technique for learning joint representations across different modalities.
• Hamming radius — Concept · Strategy · 3 mentions · first at 1:39
  Parameter used in retrieval systems to define the search radius.
• Collision — Concept · Strategy · 3 mentions · first at 1:50
  Desired outcome in hashing for efficient retrieval.
  Radius R which is set of results that I want to consider for my and obviously the best case scenario for this applications is when you have Collision so when you consider…
• Feature space — Concept · Framework · 2 mentions · first at 0:45
  The space where input data is embedded before hashing.
  Similarity can describe them so given this annotated data what we want to do we want to learn function H which brings us from the feat space to the handing space so we want to…
• Binary vectors — Concept · Framework · 2 mentions · first at 1:09
  Compact representation used in hashing for efficient distance computation.
  We want to learn function H which brings us from the feat space to the handing space so we want to go to the space of binary vectors and obviously this is a much more compact…
• Lista Network — Concept · Framework · 1 mention · first at 4:08
  Type of architecture used for learning sparse codes.
• Meta Siamese Network — Concept · Framework · 1 mention · first at 8:58
  Network architecture used for coupling different modalities.
• Semantic similarity — Concept · Key argument · 1 mention · first at 11:07
  Measure of how similar two pieces of data are in meaning.
  Of black but in the black you see lot of annotations which do actually make sense so the system is somehow able to some semantic similarity between text and images which…
• Cross SSH — Concept · Strategy · 2 mentions · first at 9:39
  Linear technique for cross-modal retrieval.
  One location of the given keyword in this case we made up this Vector of people than heart and we compare with cross SSH which is a linear technique was prev it's actually one…
• Sign function — Concept · Framework · 1 mention · first at 4:19
  Function used to obtain binary vectors from continuous values.
  Type of architecture which is the Lista Network which iserable is the follow by B because in the end what we want the Threshold at zero and obtain our binary vectors so this is…

Full transcript with timestamps

0:00 I'm going to talk about hasing which is a very welln technique for large scale vision problems just to fix few ideas of what I mean byar SC problems here are just for a few numbers so as you can see YouTube stores something like 40 40 million hours of recordings per year Facebook stores millions of images per month and even this large collection of data which is fast growing we need methods which are able to index and stre in such large collections in a very efficient manner and this is indeed the similarity sensitive hashing uh go where in setting what we have is some input data here just sh some images and what we assume is that these images are embedded into space and what we also know is that we know a boundary similarity among images which means if we can sample two images and we can assess whether they belong to the same class or they have some underlying similarity can describe them so given

1:02 this annotated data what we want to do we want to learn function H which brings us from the feat space to the handing space so we want to go to the space of binary vectors and obviously this is a much more compact representation which is also the nice proper you have a very efficient distance computation which can be done in reasonable size vectors in one clock side and obviously H can be any any function and as we are interested in Lear problems obviously we cannot resort to exhausted search which means if I want to retrieve some samples I can go and list them all in in my datab base so what I have to do I have to defam of radius R which is set of results that I want to consider for my and obviously the best the best case scenario for this applications is when you have Collision so when you consider radius equals to zero still you obtain a very good performance and this is also because you

1:59 can use table which means I give you vector and then you go in your table and then you have a indes and that's what you have to retrieve so the retrieval time does not depend on the number of data points that you have and in general when you multiply this form of retrial for arit uh re already what you have is the this gr are and what you have in a receival system that you would like to have three things efficient search in terms of speed high Precision high but unfortunately these are three sing TRS and these are two plots which have taken from ground Fus which are trying to explain this problem so if you want High Precision what you do usually increase the Cod and if you increase the Cod L what you see on the left is that this efficient trick for retrieving uh a small ready becomes uh really unable even for small already in the case of the three for example you Resort Force which means that your system is already

3:01 too slow and on the other hand what you see is that the average number of neighbors sharply decreases when increase Cod which means that you end up having very low Cod so what we argue here in this work is that we need to AR structure to the space of these codes so that we can limit the degrees of freedom that these codes have and we do it by adding sparity which is just one of the possibilities in order to this and the L function that we end up minimizing is the classical L function for Dr L which we applied already for learning similar caching so you have the first term which tries to put things which are similar to the same point so it tries to get theion and then the second term which tries to things which are the similar similar points in the handing space and then the last one which is obviously SP and as I said this is efficiently minimized in terms of uh semi architecture and obviously we have our

4:00 model which is the function we want to learn and we can C this we can learn it in a very Manner and as you're interest SP codes what we use is a particular type of architecture which is the Lista Network which iserable is the follow by B because in the end what we want want the Threshold at zero and obtain our binary vectors so this is just a SM approximation of sign function which allows us to do back in traditional fashion and the first result I want to show is the effect of sparcity on so here what I'm showing is a comparison with respect to other previous state of the approaches on in terms of unit codes and this is on 10 so you have 60,000 images and as you see all the standard D methods what they do they try to cover uniform in the space and this result results in a very low number of other neighbors and so low Rec what we have with spity is that we have great or less number of in TAS we have for handing

5:01 radius of 0 one and two much larger number of retreat samples and OB these are just numbers that tell that we have lot of neighbors but we want good neighbors and that's what I'm trying to show here so here I'm showing Precision curves for all the methods when we consider the Heming radius equals to the Cod l in this case 48 ft so this is brute force and obviously we lose a little bit with respect to the ksh and has methods but what we interested in is showing that when we reduce your radius two then than to thanks to constrain the number of how much higher we Precision Al together with respect to the other approaches and this is the same also when you consider R zero so and another interesting plot is this one where ploted Rec and versus Rus and what you want to show is that when you keep the hming radius is small what you have is that is higher for Spar hash and what you would like to do is to have

6:05 this method which has the same performance when you increase number of dimensions and this is what I'm seeing what I'm showing here so here we have 12 Dimensions all the D methods have sharp decrease which is known to happen because they have non constraint and in our case first has has a consider considerably better performance and also if you notice the performance actually increases because at this point 12 dimensions are roughly comparable with the 48 dimensions of d as goes in terms of of freedom and this is another plot which tries to explain why this is beneficial so here I'm plotting precision recod versus time everything is done with look tables so obviously when you have Collision R equals to zero that's the most efficient way you can do the and what is nice to see is that start Ash as a recing Precision which work together better than all the other methods even when they considered ready upud which

7:06 obviously involve a much slower thir time and these are just few examples from C data set you have your image per on left then of the three samples obviously it's not perfect but it's doing a decent job and this brings me to what I believe is the most interesting at least for me problem for simar cash dealing with multimod data every day I mean we're surrounded by a lot of data which comes in different mity and this is just an example fromia reading this page we know that there's some text representation some image representation and for us it's easy to understand text relates with the image and VI Versa but at the end it's not clear how to deal with these two spaces in a common fashion so how to bring them so that we can compare images and text and ultimately compare F in a mutable space which is the finite goal and thanks to the implementation using some networks this was very easy to do in our setting

8:11 so here we have image modality text modality and obviously we want to bring the two head functions to the h space which is joint space for these two modalities so what we assume here is that we have inter model similarities binary as usual so you have similari images similarities for text and then what is important is that you assume that you have some inter model inter model similarity which means that you can simple an image you can simp a text and then you can say whether they're similar or they describe the same concept or not and this allows you to C these two spaces into this joint handing space and obviously the naive way of doing it which is also the most effective and efficient way of do is by coupling the networks so you have two networks and then you coup them with the cross model l which is basically a meta Simon Network and what is nice is that H and G are join to Le so you don't have to

9:07 resolve these two step optimizations and so on and after the system s so scales well BL data sets and so on here I'm just showing some examples for text Bas so the data here is a new data set so there are 270,000 images taken from speaker and the image modity features of 100 dimensions and text representation Dimension Vector we have one location of the given keyword in this case we made up this Vector of people than heart and we compare with cross SSH which is a linear technique was prev it's actually one of the fewest techniques which are trying to this problem and as you see a considerable better performance and what is nice is also in this case where flow and heart and you see that the second image of is face but what is nice is that the uh the image to the image has been applied a filter which is an so it makes plausible to assign that image to the category heart and obiously you can do a convert

10:25 so you can you can have an image you can f it system and then in this joint space you retrieve n's Neighbors in the tax modality so we retrieve in this case the top five neighbors then we get all this uh annotations and we assign them to the image so we try to image annotations in this way and what what you can see is that compared Tosh you have considerably better performance showing the results which are in the ground of the image and what is nice in my opinion is that not all the annotations are right so you see lot of black but in the black you see lot of annotations which do actually make sense so the system is somehow able to some semantic similarity between text and images which sometimes is very hard to capture especially in the data set which is quite I so to conclude what we have presented is a way of modifying the convention problems to a constraint to limit the deg freedom of the codes and

11:31 we have achieved this goal by having sparity which is just the simplest form of simplest form that we can think of to going train these codes and we shown that we can achieve higher code at small ready which means that we can use the efficient tabl and get well because we do have online and thanks to the to the embedding that we learn speci you can extend large number of modalities and thank you for your

12:08 attention so the reason why I look since passion works is not that you know one shot thing good hashing but you have you try multiple times on redundant hashes Each of which has low recall right that's the whole point of vocals of hashing everyone who does learning hashing always seems to optimize like the single one shot how do you think your thing will work if you cramp the recall down but then you're allowed like multiple R yeah in this case okay if you consider the loss function that we minimize obviously this is not something that we are interested in because ideally what we would like to end up is with collision and high call obviously this is all all about this work obviously in that case we should consider different waiting schemes for the for the hashing which means that we would like to as you said L but have hashes which are distributed somehow which goes back to the original problem

13:03 of D hash methods and here is what we at least what we try to do is to Resort through a one shot method so that we we don't have to resort to to Conventional techniques B which are going to have the problems but obviously I mean it's just saying that if you do that you you obtain something which otherwise you couldn't obtain I'm not saying that this is the only way to go so it depends on the system it depends on what you do but in general what we are shown is that you need get much higher without losing in Precision which is usually what you what you're interest so I was wondering um can you give some more details about the is um projections you're using I mean how many iterations do you do with is so at first yeah that's quite funny because weing just two iterations but the thing is that I started with 10 and then I I tried to reduce them obviously trying to find a minimum model which was trying to do

13:59 basically and in this case just going up and obtaining Cod through the B rer is more than enough but obviously the amount of data that we working with and if you consider C 6,000 images 12 dimensions that's that's really so one one iteration in this case work quite fine so we just kep simp small questions okay speak again

Questions about this episode

What is discussed in "ICLR14: J Masci: Sparse similarity-preserving hashing"?

In order: 0:00 Feature space, Similarity-sensitive hashing, Precision, Binary vectors; 3:00 Sparsity, Precision, Recall, Hamming radius; 6:00 Multi-modal data, Cross-modal learning, Precision, Recall; 9:00 Cross SSH, Multi-modal data, Semantic similarity; 12:00 Precision, Recall, Collision.

What is mentioned in this ICLR episode?

17 people, works and ideas are mentioned, including Similarity-sensitive hashing, Sparsity, Multi-modal data, Precision, Recall, Cross-modal learning. Each one on this page links to its own page with every episode that mentions it.

How long is "ICLR14: J Masci: Sparse similarity-preserving hashing"?

The episode runs 15 min; the transcript is about 2,442 words in 15 timestamped sections.

More from ICLR

• All analyzed episodes
• People mentioned on ICLR
• Topics mentioned on ICLR