ICLR14: V Jain: Learned versus Hand-Designed Feature Representations for 3D Agglomeration

What this episode is about

The focus of the discussion is on comparing learned versus hand-designed feature representations for analyzing 3D agglomeration, particularly in the context of neuroscience where techniques are applied to images of brain tissue to extract connectivity information. The presentation highlights challenges in image reconstruction from large datasets and introduces a new model aimed at improving boundary prediction in this complex imaging process.

Key takeaways

• Neural imaging isn't just about pretty pictures; it's a tangled web begging for clarity amidst 3D chaos.
• Months of training deep networks reveal a surprising truth: more data doesn't always equal more clarity.
• Embedding complex structures into algorithms reduces confusion—why not let AI learn the shapes of existence?
• Unsupervised pre-training offers a light in the bottleneck tunnel; sometimes, it pays to 'unsupervise' before you supervise.
• In the world of neural connectivity, the next breakthrough might hinge on a little unsupervised thinking—who knew?

Who speaks in this episode?

• Brian (Host) · 2 min speaking
• V Jain (Guest) · 15 min speaking

What is discussed, and when

1. 0:00 – 3:00: Gary Wang, Jamelia Farm, John Vogovik
2. 3:00 – 6:00: Convolutional Networks, Sparse Coding
3. 6:00 – 9:00: 3D Mesh Limitation
4. 12:00 – 15:00: Dynamic Pooling, Google LLC
5. 15:00 – 16:57: Sparse Coding

Notable quotes from "ICLR14: V Jain: Learned versus Hand-Designed Feature Representations for 3D Agglomeration"

• With john vogovik and gary wang at jamelia farm and i'm actually going to be talking about two different papers that are in icl or iclr this year deep and wide multi-skill recursive networks and then.

  about ICLR · at 0:10

• Obviously help you a lot with segmentation for a number of years we've been using deep networks of various times you know go method 2007 in fact and you know these have been very successful in fact.

  about Convolutional Networks · at 1:55

• Using a lot of unsupervised techniques that are cheaper so for example things like we're thought on actually pursuing sparse coding as opposed to background and that's going to be augmented then by several supervised stages which.

  about Sparse Coding · at 3:56

• Lot of different applications that involve reasoning about 3d objects so for example here's a paper from toronto learning 3d mesh limitation labeling where you have existing…

  about 3D Mesh Limitation · at 8:43

• Of the region we end up pulling over in some different cases and you know once you add in this type of pooling the results are much better so they're in fact start to beat the hand design feature set at least by a little bit so go back to.

  about Dynamic Pooling · at 12:52

Who and what is mentioned in this episode?

People

• Gary Wang — Person · Referenced collaborator (co-author) · 1 mention · first at 0:09
  Named by the speaker as a co-author on the joint work/papers being presented; the transcript provides no further identifying information beyond the stated affiliation.
  All right can you hear me all right my name is brain there's a joint work with john vogovik and gary wang at jamelia farm and i'm actually going to be talking about two different papers that are.
• John Vogovik — Person · Referenced collaborator (co-author) · 1 mention · first at 0:09
  Named by the speaker as a co-author on the joint work/papers being presented, but no additional biographical details are provided in the transcript.
  All right can you hear me all right my name is brain there's a joint work with john vogovik and gary wang at jamelia farm and i'm actually going to be talking about two different papers that are.

Concepts & topics

• Dynamic Pooling — Concept · Key argument · 3 mentions · first at 12:05
  Introduced as a novel approach to improve feature extraction in 3D object representation.
• Convolutional Networks — Concept · Framework · 1 mention · first at 3:15
  Discussed as a traditional approach for boundary prediction in 3D data.
• Sparse Coding — Concept · Strategy · 2 mentions · first at 4:01
  Used as an unsupervised technique for initial layers in the proposed model.
• 3D Mesh Limitation — Concept · Referenced work · 1 mention · first at 8:49
  Cited as an example of 3D modeling applications.

Places & events

• ICLR — Event · Referenced work · 4 mentions
  Mentioned as the conference where the discussed papers were presented.
• Jamelia Farm — Place · Headquarters · 1 mention · first at 0:10
  Mentioned as the affiliation of the speaker.
  All right can you hear me all right my name is brain there's a joint work with john vogovik and gary wang at jamelia farm and i'm actually going to be talking about two different papers that are.

Sponsors

• Google LLC — Company · Sponsor · 1 mention · first at 13:38
  Mentioned in the context of a new effort to solve electronics issues at Google scale.
  Acknowledge a lot of people who returned to the data in this talk and finally if anybody's interested there's sort of a new effort to solve this electronics issue at google scale so steve and if you're interested in just thanks.

Full transcript with timestamps

0:00 presentations for 3d examination and the talk will be given by mirandj all right can you hear me all right my name is brain there's a joint work with john vogovik and gary wang at jamelia farm and i'm actually going to be talking about uh two different papers that are in icl or iclr this year um deep and wide multi-skill recursive networks uh and then um learn versus hand design features uh with review operation and uh the model defined in first page i'll talk about is actually the one we use in the ladder paper uh and also i'm just gonna you know just take one minute to talk about the application that we're studying here um so this is a brain on a dish uh and the application which we're going to be applying the experiments to are data coming from the field of connect clinics which is basically trying to measure neural network structure in actual brains and the approach is to take you know

0:52 actual brains and then slice them up uh image them and then get a very large picture of the frame basically uh and um okay this isn't really how it's done there's a machine and diamond knives and ion beams and stuff like that um but you get the idea right um but actually the real bottleneck is not the image acquisition process it's in the image reconstruction phase so we get these very large three-dimensional uh 8-bit intensity data sets and we want to basically trace the wires in in that data and it turns out this is a reasonably hard problem uh and it's also the entire bottleneck uh in this process of actually acquiring your own connectivity information uh and here's a post of what that data looks like in a little bit more detail so there's a lot going on there's synapses uh so the first one to talk about is a new model for doing boundary prediction in this context um even why multisport

1:47 recursive networks uh so the problem we're trying to solve is taking the type of data i just presented finding the boundaries in the images and if you can do that accurately that'll obviously help you a lot with segmentation um for a number of years we've been using uh deep networks of various times you know go method 2007 in fact and you know these have been very successful in fact we've used such networks to reconstruct about a thousand cells in retina which was published recently but you know a theme which has obviously been discussed quite intensely at this conference other places is that it takes a long time to learn the networks that do this so for us we use 3d filters everywhere in our network so in fact it can take months to train uh the best networks and it keeps getting worse because we want more power in our models so because of that um you know we'd like to somehow have models that are more

2:38 practical uh you know we'd like to put more uh model complexity into them we'd like to explore larger parameter space and cost functions uh and we'd like to quickly retrain based on new label data so we'd like to define a new type of model which is somehow much quicker to train but also has several architectural features so we're going to define three things which i'll talk about in turn the first is a wide feature representation the second is a large field of view and the third is the ability to model statistical structure uh enable space uh and i'll explain what about what i mean by these things um so the first is pretty straightforward uh now versus why future representations so in the convolutional networks i've been using in this context uh you know each layer was pretty narrow so maybe on the order of 20 to 30 feature maps per layer with all connectivity between layers again because of the 3d filters this was

3:31 still quite uh computationally costly to train but that seems sort of you know very narrow considering you know the complexity of the data and so forth so we like to at least be able to test uh approaches that have many many more features potentially thousands and of course that's going to come at some sort of cost and so we're going to make the decision uh instead of using supervisor to learn all of our features we're going to learn at least initial layers using a lot of unsupervised techniques that are cheaper so for example things like we're thought on actually pursuing sparse coding um as opposed to background uh and that's going to be augmented then by uh several supervised stages which are not convolutional so those are just going to be more straightforward mlp layers that that work on top of those features that we learned the second issue of uh field of view so you know what we're doing is taking some

4:21 and by end by end the cube classifying the pixel in the middle of that cube and we'd like to be able to use much more context uh to resolve ambiguities uh in our data but you know in 3d just doubling the linear dimension that's almost an order magnitude more voxels so somehow we need to do this in an efficient way so you know the obvious parameters are of course the number of layers the size of the filters uh we're going to do a few other things we're going to add multi-scale pathways uh in the architecture uh and then we're also going to introduce a notion of uh proteated cooling where different um different areas an image will be represented at different resolutions depending on how far they are and where you're actually trying to classify uh finally we'd like to give the model system structure in enable space so what do we mean by that so here's a video showing some ground truth annotation

5:12 of this data so this is a human that's gone through and basically segmented the data uh and you know what you can tell is that you know it's not uh there's a fairly low dimensional space of possible structures here that are sort of plausible uh it's not like anything can happen uh certainly it's more low-dimensional than natural language so somehow we'd like to take advantage of that in our inference pipeline um so i mean more concretely you know labels for neighboring image locations are correlated so it's really a structural prediction problem um and in most of our approaches we're really making these predictions more or less independent they're only correlated through the adventure so you know an obvious way to go is to find some sort of mrf over the images or the predictions that we're making him up that way we did a lot of work showing that you know there's some issues with doing this

5:59 effectively i'm not going to go into those arguments but just talk about it so we're going to do something different we're going to take our architecture and add a new pathway we're going to take our output and connect it to our input uh and so overall what the entire architecture is going to look like is something like this where we have recursive steps of the overall architecture which is sort of that bottom pipeline here and then blown up is the architecture at any individual step so you see those multi-scale pathways there's the feature extraction stage which is largely supervised and then the supervised stage on top and each recursive step is going to get input from both the previous set of predictions as well as the original image and that will generate the final output here which is something called infinity graph but you can basically just give that as a battery and one nice thing about this is it

6:47 actually scales pretty well so you can distribute across both cpus and gpus we use the cpu cluster to do the unsupervised future extraction stage that paralyzes very cleanly uh and then we store those features and then just train mlp on gpus actually uh we can train very large models so for example the currently the best models that we that we use use the 50 tube fossil input that's 1000 boxes in total there's about 12 or 13 layers but we can train that within it and in the paper that we have in this conference we show that we can actually get better results than at least the previous convolutional limits that we're using pretty quickly using this type of approach okay so i mainly just wanted to set up that model because that's what we're going to be using um in the second part of the talk which is about 3d lower issue so the idea uh here is that you know once you've done your boundary prediction what you really get is this

7:43 ski of fragments so these are basically super pixels or listening super box holes you know distributed throughout your image space and now the task is to piece those fragments together into large coherent holes okay so previously we had actually defined an inference framework for solving this problem it's actually uh it was quite reminiscent of the previous talk it's essentially a deterministic mpp uh that has a state which is your segmentation and various actions like piecing together fragments or asking human computers not sure and then we use techniques from policy control in our rl to to learn to learn how to do this difference process but ultimately for the purposes of this thought what it boils down to is a sequence of binary decisions about whether two fragments are uh part of the same neuron or different neurons on the left is a negative example so those two fragments should not be merged together on the right is a

8:38 positive example those two practices should be merged together uh but more generally there's you know a lot of different applications that involve reasoning about 3d objects so for example here's a paper from toronto learning 3d mesh limitation labeling where you have existing models of 3d objects and one of these segment those models to different semantically meaningful parts um and the way they did this was to define some hand-defined 3d features so various things that measure curvature or other properties of the object and then put that into a crf which will do the actual segmentation um similarly there's been a lot of work on 3d shape retrieval so given some 3d objects finding things that are similar or related and you know especially with the added things like 3d printing and handheld freeze uh modeling uh platforms uh there's a lot more data uh coming online but uh the thing which

9:30 i want to stress here is that all approaches thus far have really been based on hand design teacher representations of this context so people will basically come up with some insight into what properties of 3d shape might be relevant encode that in some feature representation and then do machine learning on top of that so far it's been very difficult to just learn from the data what those features should look like uh so what we're doing is taking an object in 3d space it could be quite large in fact and then trying to come up with some features that are that are useful for in our case agglomeration that you know could be other tasks as well uh and so in this paper we basically compared two different approaches so we take the hand design approach uh where we take a large library of 3d geometry and morphology descriptors and then we also take a purely learning basic graph where we just feed

10:21 various networks the raw data and see if they can learn how to represent the data represent the objects for the prediction task so in terms of the manually designed features uh you know there's a whole literature over the past 20 or 30 years of people experimenting with different types of representation 3d objects uh you know there's like level sets various kinds of ray tracing approaches spherical harmonics i'm not going to go to the details but in our paper we actually went through a lot of effort to implement all the different types of features that people have come up with um at least most of them uh and rigorous to compare their performance so if you're interested in a 3d modeling task i would recommend that you check out this figure in the paper it might save you a lot of work we also note the computational cost as well as the accuracy classification task but we were really interested in seeing

11:10 whether we could just learn the right representations so to feed those objects directly into some sort of network and get good prediction results out of that and doing the naive thing i'm basically just putting those objects through it doesn't work at all so we have this performance figure in the paper the second best line you see there the red line i don't know it's big enough uh that's from using the entire set of hand design features uh that we've decided that we've defined uh and all those other curves that are significantly below that the dotted ones and both those are from basically the naive approach which is shoving 3d objects through some sort of convolutional network or unsupervised features tractor um basically it doesn't work so maybe if we had much more data it would work but certainly with the amount of data that we have so we had to go back and you know see if there was something we had to add to our

12:00 architecture to make it work and it turns out there's something pretty simple you can do and that's based on a notion of dynamic pooling so within our future extraction stage we basically um added uh an alternative notion of cooling as opposed to using something fixed like max cooling or average pooling over fixed spatial windows uh instead what we do is we dynamically cool features based on the shape of the objects that we're trying to represent so for example if we're trying to decide two objects merged we introduced one notion called object pooling which basically will just pull from the union of the two objects and we introduced something else called boundary cooling which basically looks at the area close to the interface of two objects in a particular way and this figure might be a little bit difficult to follow but on the right you can see basically the 3d rendering of the of the region we end up pulling over in

12:54 some different cases and you know once you add in this type of pooling the results are much better so they're in fact start to beat the hand design feature set at least by a little bit so go back to this figure uh the best performing curve there the purple one is using both this dynamic boundary and object point uh in this context uh so there's a lot more work to be done here in terms of improving these models for this past further but uh you know we're excited about uh the first step of at least getting learning to work better than designed features uh so that's it i'm just gonna acknowledge a lot of people who returned to the data uh in this talk and finally um if anybody's interested there's sort of a new effort to solve this electronics issue at google scale so steve uh and if you're interested in just thanks

13:52 question so i should go back to your results uh the purple curve you see it says plus hand design is that oh yes so so that's using um that's right so that's using um the dynamic plus we add in the original features so we don't use any of the original indesign features uh they're pretty much back in that uh yeah oh yeah sorry i probably missed the uh yes the green line is just with the learned features uh and the best results are still combining with the hand side so you don't have plus the dynamic pooling the alerting plus dynamic cooling without the hand design features on the graph uh not on the graph it might be in our supplementary okay but that's neck and neck equipment that's right

14:52 is there a connection between your dynamic pooling and what pedro dominguez was talking about uh in which you somehow optimized where to cool well so our notion of dynamic cooling is coming from sort of the mass of the objects in the input i'm not sure if that's exactly what uh pedro would refer to his notion of invariance whether it was input dependent uh for example

15:52 that's right so uh so learning the unsupervised features is relatively inexpensive because we're doing you know simple things like sparse coding gaming um but extracting the features for each location of the image is expensive because you might be doing it multiple scales and so forth but the nice thing about that is we can just paralyze that easily across the cpu cluster uh and then you store those features on some sort of storage um and then the gpu training is very efficient because uh it's just an mlp that's right uh you know maybe a few layers at most uh and you know we don't have to worry too much about optimization there so as long as you have quick access to the storage of the features so that's that's probably the most subtle infrastructure issue here in terms of ways for the features and whether they're you know accessible for sgd in a relatively efficient way but as long as that's true then it ends

16:44 up being pretty efficient if there's no other questions let's uh

Questions about this episode

Who speaks in "ICLR14: V Jain: Learned versus Hand-Designed Feature Representations for 3D Agglomeration"?

Brian (Host), V Jain (Guest).

What is "ICLR14: V Jain: Learned versus Hand-Designed Feature Representations for 3D Agglomeration" about?

The focus of the discussion is on comparing learned versus hand-designed feature representations for analyzing 3D agglomeration, particularly in the context of neuroscience where techniques are applied to images of brain tissue to extract connectivity information. The presentation highlights challenges in image reconstruction from large datasets and introduces a new model aimed at improving…

What is mentioned in this ICLR episode?

9 people, works and ideas are mentioned, including Dynamic Pooling, ICLR, Convolutional Networks, Sparse Coding, 3D Mesh Limitation, Gary Wang. Each one on this page links to its own page with every episode that mentions it.

How long is "ICLR14: V Jain: Learned versus Hand-Designed Feature Representations for 3D Agglomeration"?

The episode runs 17 min; the transcript is about 3,053 words in 20 timestamped sections.

More from ICLR

• All analyzed episodes
• People mentioned on ICLR
• Topics mentioned on ICLR