Let us begin with the continuous formulation of diffusion models, to develop a good theoretical intuition and to clarify the sturcture behind this generative modelling problem. Afterwards we will turn to the time discretization to implement diffusion models for real.

Consider a stochastic process on with and distributed according to a given distribution which in practical problems pertains to the data being modelled. The process evolves forward in time according to the SDE

where is the n-dimensional Brownian motion, and .

For brevity we omit a rigorous discussion of underlying probability spaces, filterations, well-posedness, existence and uniqueness of strong and weak solutions. Let us assume that has a "sufficiently smooth" density with respect to the Lebesgue measure on .

Let . Under suitable hypotheses, the time reversed process defined on satisfies

where and put

The operation on a matrix results in a vector whose -th component is the divergence of the -th column vector of .

The short attention span heuristic (which is no proof) might go something like: put and it turns out the forward process has infinitesimal generator

which has adjoint

Let be the density of (with respect to Lebesgue measure on ) evolves by the Fokker-Planck equation

and a change of variable in the above leads to and in the reverse SDE. This time reversal problem is pretty fundamental and is well understood in the literature, so we take it for granted here without demanding more rigour.

Inspecting and we find that everything (i.e. , , ) is known when we specify the forward equation, except in we introduce the so called Stein's score function that we don't know.