Understanding Mixture Density Neural Networks Training Ppt

Slide 1

This slide gives an introduction to Mixture Density Networks. A Mixture Density Network is an interesting model for working on supervised learning issues where a single standard probability distribution cannot represent the target variable. It is created within the basic framework of neural networks and probability theory.

Instructor’s Notes: In essence, an MDN allows you to model a conditional probability distribution p(y|x) as a mixture of distributions, with the individual distributions and mixing coefficients defined by functions of the inputs x.

Slide 2

This slide depicts the two components that a Mixture Density Network is made of. These are a neural network and a mixture model

Instructor’s Notes: 

• Neural Network: Any viable architecture that takes the input X and converts it into a set of learned features is a Neural Network (we can think of it as an encoder or backbone)
• Mixture Model: The Mixture Model is a probability distribution model composed of a weighted sum of simpler distributions

Slide 3

This slide shows how a Mixture Density Network looks like. The Mixture Density Network is a feed-forward neural network that determines the parameters of a mixture density model using its outputs.