Variational Bayesian GPLVM Sample Clauses

Variational Bayesian GPLVM. ‌ ( ) = ⋅⋅ ⋅ ( ) ( ) ( ) = N ( ) =∶ N ( )⋅ ⋅ ⋅ ⋅ In the next step, we want to integrate out the inputs X, to provide a latent variable model, where the latent space X is learnt as the solution to the mapping f X Y [19; 86]. f X has a GP prior p X , which we approximate by a variational Xxxx- xxxx xxxxx q X d M d, S dI M, S , where S contains the diagonals S d for each dimension as columns, such that the corresponding variance for the mean dimension M d is S d I . This implies that there is no cross covariance between latent dimensions, the latent points Gaussian variance is aligned to their respec- tive dimensions. In the following, we will omit the inducing inputs Z as much as possible, as it will clutter expressions. We want to integrate over the latent space X and from the standard GP bound (Eq. (2.1)). This is intractable and we need an approximation. Here, we make use of another variational approximation on top of the sparse GP approximation to have the speedup from the sparse GP bound as well as be able to learn the latent space = log J p(Y|X)p(X) q(X) log p(Y) = log J p(Y|X)p(X) dX = log { p(Y|X)p(X)} dX q(X) ≥ {log p(Y|X) − log q(X)} q(X) q(X) (2.19) p(X) q(X) =∶ L3(Y) . ≥ (L2(Y))q(X) − KL (q(X)||p(X)) L When maximizing the above bound 3, we will minimize the (KL) divergence be- tween p and q, while maximizing the sparse GP bound, which will try to find a suitable fit for the latent function F to fit the seen data Y, at the same time as find- ing the right input locations X to provide the most likely latent space under the current function F. This can be confusing in the beginning, we will see the differ- ent contributions for the latent space later on. (L2) = (D log c − 1 tr Yt(βI − KFUβI (KUU + βKUFKFU)−1βIKUF)Y − 1 β tr Λ) Let us focus on the left side of the expression. We will use (2.18) to extract the variational bound for Bayesian treatment of the latent space X. Let, in the following, (⋅) = (⋅)q(X) 2 − 1 tr Yt(βI − β2 (KFU)(KUU + β (KUFKFU))−1 (KUF))Y = − ND log 2π + ND log β + D log |KUU| + 1 log |KUU + β (KUFKFU)| = − ND log 2π + ND log β + D log |KUU| + 1 log |KUU + βΨ2| − β (tr(K )) + β tr(K−1 (K K )) 2 2 − ( − ( + ) ) − β ψ0 + β tr(K−1 Ψ2) , 1 tr Yt βI β2Ψ1 KUU βΨ2 −1Ψt1 Y 2 2 UU = ( )− N |X X−1 K + β−1I |− 1 . Here, we defined with the normalizing factor c 2π 2 the ψ statistics, which have to be computed depending on the covariance function used: [ψ0]i = (k(xi⋅, xi⋅))q(X) ∈ RN [Ψ1]ij = (k(xi⋅, zj⋅)) q(X) ∈ RN ×M [Ψ2]ijk = (k(zj⋅, xi⋅,...