Loading Web-Font TeX/Math/Italic
| Register
\newcommand{\Cat}{{\rm Cat}} \newcommand{\A}{\mathcal A} \newcommand{\freestar}{ \framebox[7pt]{$\star$} }

1. Representational Capacity

    1. Is RBM_{4,3} a universal approximator?

      Problem 1.1.

      [Guido Montufar] Does the closure of RBM_{4,3} fill the simplex \Delta_{15}, making it a universal approximator?
          Here, RBM_{4,3} indicates the RBM statistical model with 4 visible and 3 hidden units, belonging to the probability simplex \Delta_{15}. It is known that for universal approximation, it is necessary that the number of hidden units be m \geq 3, and sufficient if m \geq 7. Simulations have suggested that RBM_{4,3} fills the simplex.
        • Determine the maximum divergence of RBM_{3,1}.

          Problem 1.2.

          [Thomas Merkh] Determine the maximum divergence of RBM_{3,1}.
              Currently, the maximum divergence of RBM_{3,1} is unknown. The maximum divergence is defined as {\cal D}_{RBM_{3,1}} := \text{sup}_{p \in \Delta_7} \text{inf}_{q \in RBM_{3,1}} D(p || q), where D can be any measure of divergence between probability distributions, such as the KL-divergence.
            • What kind of distributions can be represented by an RBM as opposed to directed models?

              Problem 1.3.

              What kind of distributions can be represented by an RBM as opposed to directed models?
                • Characterize the limiting set of distributions for \lim_{n\to\infty} RBM_{n,n}.

                  Problem 1.4.

                  [Jason Morton] Consider the limit as n \to \infty for RBM_{n,n}. It was suggested that this limiting set of distributions can be seen as a set of measures on the unit interval. Do all possible measures belong to this set? If not, which ones do?

                      Cite this as: AimPL: Boltzmann Machines, available at http://aimpl.org/boltzmann.