
## 1. Representational Capacity

1. ### Is $RBM_{4,3}$ a universal approximator?

#### Problem 1.1.

[Guido Montufar] Does the closure of $RBM_{4,3}$ fill the simplex $\Delta_{15},$ making it a universal approximator?
Here, $RBM_{4,3}$ indicates the RBM statistical model with 4 visible and 3 hidden units, belonging to the probability simplex $\Delta_{15}.$ It is known that for universal approximation, it is necessary that the number of hidden units be $m \geq 3,$ and sufficient if $m \geq 7.$ Simulations have suggested that $RBM_{4,3}$ fills the simplex.
• ### Determine the maximum divergence of $RBM_{3,1}.$

#### Problem 1.2.

[org.aimpl.user:tmerkh@g.ucla.edu] Determine the maximum divergence of $RBM_{3,1}.$
Currently, the maximum divergence of $RBM_{3,1}$ is unknown. The maximum divergence is defined as ${\cal D}_{RBM_{3,1}} := \text{sup}_{p \in \Delta_7} \text{inf}_{q \in RBM_{3,1}} D(p || q),$ where $D$ can be any measure of divergence between probability distributions, such as the KL-divergence.
• ### What kind of distributions can be represented by an RBM as opposed to directed models?

#### Problem 1.3.

What kind of distributions can be represented by an RBM as opposed to directed models?
• ### Characterize the limiting set of distributions for $\lim_{n\to\infty} RBM_{n,n}.$

#### Problem 1.4.

[Jason Morton] Consider the limit as $n \to \infty$ for $RBM_{n,n}.$ It was suggested that this limiting set of distributions can be seen as a set of measures on the unit interval. Do all possible measures belong to this set? If not, which ones do?

Cite this as: AimPL: Boltzmann Machines, available at http://aimpl.org/boltzmann.