| Register
\(\newcommand{\Cat}{{\rm Cat}} \) \(\newcommand{\A}{\mathcal A} \) \(\newcommand{\freestar}{ \framebox[7pt]{$\star$} }\)

1. Representational Capacity

    1. Is $RBM_{4,3}$ a universal approximator?


      Add remark

      Problem 1.1.

      [Guido Montufar] Does the closure of $RBM_{4,3}$ fill the simplex $\Delta_{15},$ making it a universal approximator?
          Here, $RBM_{4,3}$ indicates the RBM statistical model with 4 visible and 3 hidden units, belonging to the probability simplex $\Delta_{15}.$ It is known that for universal approximation, it is necessary that the number of hidden units be $m \geq 3,$ and sufficient if $m \geq 7.$ Simulations have suggested that $RBM_{4,3}$ fills the simplex.

        • Determine the maximum divergence of $RBM_{3,1}.$


          Add remark

          Problem 1.2.

          [org.aimpl.user:tmerkh@g.ucla.edu] Determine the maximum divergence of $RBM_{3,1}.$
              Currently, the maximum divergence of $RBM_{3,1}$ is unknown. The maximum divergence is defined as ${\cal D}_{RBM_{3,1}} := \text{sup}_{p \in \Delta_7} \text{inf}_{q \in RBM_{3,1}} D(p || q),$ where $D$ can be any measure of divergence between probability distributions, such as the KL-divergence.

            • What kind of distributions can be represented by an RBM as opposed to directed models?


              Add remark

              Problem 1.3.

              What kind of distributions can be represented by an RBM as opposed to directed models?

                • Characterize the limiting set of distributions for $\lim_{n\to\infty} RBM_{n,n}.$


                  Add remark

                  Problem 1.4.

                  [Jason Morton] Consider the limit as $n \to \infty$ for $RBM_{n,n}.$ It was suggested that this limiting set of distributions can be seen as a set of measures on the unit interval. Do all possible measures belong to this set? If not, which ones do?

                      Cite this as: AimPL: Boltzmann Machines, available at http://aimpl.org/boltzmann.