ICLR2015-joerg-bornschein

What this episode is about

The discussion centers on the Helmholtz Machine, a framework for fitting generative models to datasets using maximum likelihood training while addressing the challenges of intractable inference in models with many latent variables. It highlights the use of an inference network to facilitate approximate inference and explores recent advancements in the field, including variational methods and neural networks that improve the training of these models by integrating both the inference and generative components into a unified objective function.

Key takeaways

• The Helmholtz Machine: where intractability meets clever approximations, proving that data can indeed have an alter ego.
• Using importance sampling, we trade high variance for insightful updates, transforming ambitious models from dreams to calculations.
• Why wait 20 years for progress? The Helmholtz Machine turns inspired persistence into cutting-edge inference techniques, not just theory.
• An unbiased likelihood estimator is the Holy Grail! One accurate sample could deliver the whole truth—no pressure, right?
• Layer-wise targets in training: like giving each neuron its very own guidebook, ensuring nobody wanders off track.

Who speaks in this episode?

• Joerg Bornschein (Host) · 16 min speaking

What is discussed, and when

1. 0:00 – 3:00: Helmholtz Machine
2. 9:00 – 12:00: Helmholtz Machine
3. 12:00 – 15:00: MNIST

Notable quotes from "ICLR2015-joerg-bornschein"

• And this idea is known as the helmholtz machine and the general idea of the helmholtz machine is that we want to fit a direct generative model to the data set and we want to use maximum likelihood.

  about Helmholtz Machine · at 0:00

• Relatively deep network on a real-world data sets like eminence so for example when we train a 5-6 layer network with on the binarized mnist data set we get a test set log likelihood of roughly 85.5 which is.

  about MNIST · at 12:40

• It's not so obvious to do all these computations because variation auto encoders for example use continuous latent variables but we generally see that for binarized imnis at least this method is very competitive.

  about Variational Autoencoders · at 15:18

Who and what is mentioned in this episode?

Concepts & topics

• Helmholtz Machine — Concept · Core topic · 3 mentions · first at 0:00
  Central theme of the episode, discussing the architecture and training methods of Helmholtz Machines in machine learning.
• MNIST — Concept · Key argument · 1 mention · first at 12:50
  Used as a benchmark dataset to evaluate the performance of Helmholtz Machines and other models discussed.
• Variational Autoencoders — Concept · Related concept · 4 mentions
  Mentioned as one of the modern approaches related to the training of Helmholtz Machines.

Full transcript with timestamps

0:00 and this idea is known as the helmholtz machine and the general idea of the helmholtz machine is that we want to fit a direct generative model to the data set and we want to use maximum likelihood for it but we all know that in general training directed models with many latent variables is a hard problem and in general intractable so the basic idea with the with the helmholtz machine is to use a secondary system inference network that runs from the observed data up to the later variables we call it q that is somehow trained to help us do approximate inference and so the rest of this talk i will use p to denote the generative network that goes from the prior over multiple latent variables down to the observed data and use q also has a multi-layer neural network with latent variables that goes from the observed data up to the blatants so besides of the early wake on early work on wake sleep which was introduced in

1:07 the mid 90s there's recently been a great explosion of work related to training herman's machines and this includes for example the variation variational outer encoder and also the work on stochastic back propagation for approximate inference that both work with continuous latent variable variables and that were introduced in 2014 but this also includes neural variational inference and the darn models that were specifically for discrete binary latent units so the basic theme about all these models is that all of them rely on a variational approximation to essentially obtain an expression that contains both the inference network q and a generative network p into a common joint objective function and so this unfortunately this this common joint objective function cannot be naively trained using samples from q because it turns out that it was this would result in very high variance estimators

2:12 so that is probably one of the reason why it took nearly 20 years to come up with better better models and methods to train hammocks machines so in this work we do not use variational approximation to train our models but we will derive the parameter update equations only using important sampling as kind of the basic mechanism and important template has been used before to evaluate already trained models so in the literature we see different approaches to evaluate a trained model so we are given a generative model p and approximate inference model q and we can interpret the approximate inference model q as a proposal distribution and obtain an estimator that gives us a reasonable good likelihood estimate given a test set data point um so this is the the estimator that we obtained when we just follow the path and so let me let me note a few properties of this estimator so first of all first this is an unbiased estimator

3:19 for the likelihood it's not an unbiased estimator for the log likelihood but it's an unbiased estimator for the likelihood and the variance of this estimator critically depends on the quality of on the on the proposal distribution queue so in january what we need is that the proposal distribution gives us samples that cover the whole the high probability region from the joint from the generative joint of our model what is also interesting is that in the case that so we can minimize the variance of this estimator when our proposal distribution q approximates the true untractable posterior p of h given x from our generative model what is interesting in the extreme case in the unlikely case that we would have an approximate inference model that exactly is exactly equal to the exact posterior p of h given x we actually have a zero variance estimator so in other words in this case if we take a single arbitrary sample from our

4:26 proposal distribution q what we get is a perfect estimate of the likelihood p of x so while this estimator was used before what we here propose is to use the same basic mechanism of important sampling to also derive the parameter updated roots for our model um so if we do this and we write down the parameter update rules for our generative model p using q as a proposal we end up with an um within gradient estimator that's like this and let me maybe expand this a little bit and tell you about some properties of this so this essentially tells us that to get a gradient for our generative model uh given a single training data set x what we should do is we should draw k samples from our proposal distribution and for each of these samples we should calculate the importance weights which essentially look like the ones used on the previous slide and we also calculate for each of these samples we calculate the gradient for

5:38 the generative model and then we use the importance weights to essentially take a weighted average of all these different gradients for the different samples so it's it's important to see here that actually we don't need any kind of back prop through our generative model so we can here train the generative model using layer-wise targets so each layer gets a local gradient because it is kind of a fully observed sample from the generator from the feed forward network cube and i also want to point out that the importance weights are automatically normalized to one so this really essentially gives us that the overall gradient that we should use to train our generative model is just the weighted average over the individual samples that are weighted according to their usefulness of interpreting this data point so while this gave us an intuitively interpretable update a gradient update rule for the

6:46 generative network p there's no signal whatsoever how to train the feed forward proposal network q in this and that's not really surprising because we use the proposal network queue only as a proposal network for our estimators and as such the proposal queue doesn't have any influence on the expected values of our estimators doesn't have any influence of the expected log likelihood and it doesn't have any influence on the expected gradients but it has an influence on the variance of these estimators so a natural choice how we want to train our inference network seems to be we want to minimize the variance of our estimators we want to have good estimators for gradients and for the log likelihood so what we do here is we train q to minimize the variance of our estimators with which essentially means that we train q to approximate the untractable posterior p of h given x which kind of seems

7:48 intuitively so there's still one choice to make and that is um what x should containing and here we have two obvious choices that we could think of there might be more but one is we could train q to minimize the variance for the data points from our training data set or we could train q to minimize the variance for the data points that the generative model could imagine oh that's a generative modular things is true so we will call the one update the wake phase update that uses training data to tranqueue and the one that uses imaginary data from our generative model it's called sleep pace related to the original backsleep so for both of these approaches we can easily derive gradient updates and for the sleep phase queue update this is very simple it actually turns out that we have to sample a single sample from our generative network p and just use the sample as a target for our proposal distribution queue

9:00 for the wake phase queue update we can use the same mechanism that we used before important sampling to derive gradients for our importance first sorry for a proposal distribution queue and it turns out if you do the math that the estimator of the optimal gradient has the very same structure as the same gradient estimator that we already derived for p so this is very convenient and for practical perspective this is very convenient because it essentially means that if we want to train both p and q we take an example from our training set we use q to get proposals we calculate the importance rates for both for we calculate the importance rates for these proposals we then use the same importance rates to train both to average both the q updates and the p updates before showing some empirical results i quickly want to mention the relationship to the original break sleep and also to the other variational

10:07 approximations so it turns out if we look closely at the q update equations on the previous slide we would see that we ascend what we are essentially doing is of what what these these actually show is they are equivalent to minimizing the kl divergence between the true untractable posterior p of h given x and our proposal distribution q while whenever we use a variational approximation to derive a helmholtz machine like chair um we typically at least get update equations for cool for q where the q is a reference distribution so it's a kl divergence between q and p that gets optimized so this was one point the original wake sleep algorithm was criticized for optimizing the kl divergence with a p on as a reference and that was because it was derived or justified using a variational approximation so here we see that if we justify and derive we waited to actually based on important sampling that we get it's a very same

11:16 update rules and here we have kind of a reason why they should look this way another thing i just want to mention is that in the light of the scale divergence that if we take only k equal one sample and if we um only do sleep phase updates that um we essentially get the very same algorithm as a classical worksheet algorithm from the 90. okay some empirical results so first we ran on a variety of test sets so we always use for these experiments we always use binary stochastic units as latent variables and the layers in our p and q networks were sigmoid belief layers and so we noticed across all the data sets that we tried using five to ten samples is usually enough to get significantly better results in the classical website algorithm and to get results that approach state-of-the-art results we also notice that across all data sets and consistently we have the best results if we do sleep learning and

12:25 which means we have two different up tools to update our inference network and we can easily do that and this general strategy actually works to train relatively deep works to train relatively deep network on a real-world data sets like eminence so for example when we train a 5-6 layer network with on the binarized mnist data set we get a test set log likelihood of roughly 85.5 which is competitive at least with state-of-the-art models um yeah we did more experiments to show the the sensitivity to the number of samples i don't think i have enough time to go get into this so we'll just finish

13:27 about i'd be very interested in terry synovsky's i mean this was developed from a machine learning perspective and i think calculating importance rates is something that nature probably doesn't do directly and i cannot make a strong statement about that but so for me the motivation with this was actually to combine a fast forward inference method with a generative network and this general architecture that's an old idea seems to make a lot of sense so it seems a bit sad that about a few years ago there was not so much work going on this helmholtz mission kind of architecture

14:49 so yeah so i think one of the important parts is here that we never train with back crops we always get layer wise gradients so of course if we would use more complex layer models we could use there but that's now a more complicated architecture did you do comparisons to the variational methods that you were talking um yes it's partially a bit it's not so obvious to do all these computations because variation auto encoders for example use continuous latent variables but we generally see that for binarized imnis at least this method is very competitive compared to these other methods in terms of speed as well so we here need to take five to ten samples so that will slow down the process by a factor of five to ten um compared to the methods that i mentioned for binary variables like envel for example neural variational inference and learning these methods need an additional mlp which estimates the baselines

16:01 so in my experiments it was not completely clear how to parameterize these so i cannot really say how this compares up to exact factor but it should be roughly maybe a factor of two slower because we need five samples and there you would need a second pass only but so okay thank you

Questions about this episode

Who speaks in "ICLR2015-joerg-bornschein"?

Joerg Bornschein (Host).

What is "ICLR2015-joerg-bornschein" about?

The discussion centers on the Helmholtz Machine, a framework for fitting generative models to datasets using maximum likelihood training while addressing the challenges of intractable inference in models with many latent variables. It highlights the use of an inference network to facilitate approximate inference and explores recent advancements in the field, including variational methods and…

What is mentioned in this ICLR episode?

3 people, works and ideas are mentioned, including Helmholtz Machine, MNIST, Variational Autoencoders. Each one on this page links to its own page with every episode that mentions it.

How long is "ICLR2015-joerg-bornschein"?

The episode runs 17 min; the transcript is about 2,235 words in 15 timestamped sections.

More from ICLR

• All analyzed episodes
• People mentioned on ICLR
• Topics mentioned on ICLR