
3. Effects of Network Connectivity

1. Efficient methods to train RBMs with fixed, sparse connectivity

Problem 3.1.

[Jason Rolfe] What are the most efficient learning algorithms for training sparsely connected RBMs?
Consider an RBM with a sparsely connected network topology. The sparse connectivity can be seen as "pre-breaking" weight space symmetries that would otherwise exist for the fully-connected topology. In general it is unknown if weight space symmetries play a central role in the efficiency of machine learning optimization methods.
• How do RBMs with quantum effects differ from classical RBMs?

Problem 3.2.

How are the following models different from classical RBMs:

1) RBMs with sparse connectivity.

2) RBMs with sparse connectivity, and with quantum effects.

3) RBMs with sparse connectivity, and with quantum effects and quantum training algorithm.
• Given a fixed connectivity structure, how many inference functions can an RBM model compute?

Problem 3.3.

How many inference functions can an RBM model compute, constrained by a fixed connectivity structure?
• Given two RBMs with the same number of units but different connectivities, how much do these statistical models overlap?

Problem 3.4.

How to quantify function approximation given a network topology? In other words, how to measure the proximity between two functions when each are computed by separate networks? Then, can one use this measure to quantify the effects of altering a network’s connectivity structure?
This question was motivated by the observation that biological brains often have fine-scale differences in network topology, but overall have highly similar computational properties.

Cite this as: AimPL: Boltzmann Machines, available at http://aimpl.org/boltzmann.