Local mixtures

Global mixture components can be obtained by combining local mixture components. Predicting a time series, for example, one may allow to switch locally (in time) between two or more possible regimes, each corresponding to a different local covariance or template.

The problem which arises when combining local alternatives is the fact that the total number of mixture components grows exponentially in the number local components which have to be combined for a global mixture component.

Consider a local prior mixture model, similar to Eq. (531),

$$p(\phi\vert\theta) = e^{-\int \!dx; \vert\omega(x;\theta(x))\vert^2-\ln Z_\phi(\theta)}$$ (588)

where $\theta(x)$ may be a binary or an integer variable. The local mixture variable $\theta(x)$ labels local alternatives for filtered differences $\omega(x;\theta(x))$ which may differ in their templates $t(x;\theta(x))$ and/or their local filters ${\bf W}(x;\theta(x))$. To avoid infinite products, we choose a discretized $x$ variable (which may include the $y$ variable for general density estimation problems), so that

$$p(\phi) = \sum_\theta p(\theta)e^{-\sum_x \vert\omega(x;\theta(x))\vert^2-\ln Z_\phi(\theta)} ,$$ (589)

where the sum $\sum_\theta$ is over all local integer variables $\theta(x)$, i.e.,

$$\sum_\theta = \sum_{\theta(x_1)} \cdots \sum_{\theta(x_l)} = \left( \prod_x \sum_{\theta(x_1)}\right) .$$ (590)

Only for factorizing hyperprior $p(\theta)$ = $\prod_x p(\theta(x))$ the complete posterior factorizes

$$p(\phi) = \left(\prod_{x^\prime} \sum_{\theta(x^\prime)}\right) \prod_x \le... ...\theta(x))e^{- \vert\omega(x;\theta(x))\vert^2-\ln Z_\phi(x,\theta(x))} \right)$$

$$= \prod_{x} \sum_{\theta(x)} \left( p(\theta(x))e^{- \vert\omega(x;\theta(x))\vert^2-\ln Z_\phi(x,\theta(x))} \right) ,$$ (591)

because

$$Z_\phi = \prod_x \sum_{\theta(x)} \left(e^{-\vert\omega(x;\theta(x))\vert^2} \right) = \prod_x Z_\phi(x,\theta(x)) .$$ (592)

Under that condition the mixture coefficients $a_{\theta}$ of Eq. (552) can be obtained from the equations, local in $\theta(x)$,

$$a_{\theta} = a_{\theta(x_1)\cdots \theta(x_l)} = p(\theta\vert\phi) =\prod_x a_{\theta(x)}$$ (593)

with

$$a_{\theta(x)} = \frac{p(\theta(x))e^{-\vert\omega(x;\theta... ...(x;\theta^\prime(x))\vert^2-\ln Z_\phi(x;\theta^\prime(x))}} .$$ (594)

For equal covariances this is a nonlinear equation within a space of dimension equal to the number of local components. For non-factorizing hyperprior the equations for different $\theta(x)$ cannot be decoupled.