| Register
\(\newcommand{\Cat}{{\rm Cat}} \) \(\newcommand{\A}{\mathcal A} \) \(\newcommand{\freestar}{ \framebox[7pt]{$\star$} }\)

1. Discrete Entropy Power Inequalities

    1.     The beamsplitter addition $\boxplus_{\eta}$ is a way of combining two random variables supported on $\mathbb{Z}_+$ with a mixing parameter $\eta \in [0, 1]$. Let us first define $\boxplus_{\eta}$ in the notation of [arxiv:1605.07853], [arXiv:1701.07089]:

      Definition: Given a random variable supported on $\mathbb{Z}_+$, define its continuous counterpart $\textbf{X}_c$ (a circularly symmetric random variable supported on the complex plane $\mathbb{C}$), using a map $\mathcal{T}$ with $\textbf{X}_c = \mathcal{T}(X)$ and $X = \mathcal{T}^{-1}(\textbf{X}_c)$ with actions on mass functions and densities given by Equations (14), (15) in [arxiv:1605.07853]:

      \begin{equation} \begin{split} p_{\boldsymbol{X}_c}(\boldsymbol{r}) & = & \sum_{n=0}^\infty p_X[n] \frac{ e^{-|\boldsymbol{r}|^2} | \boldsymbol{r} |^{2n}}{n! \pi}, \\ p_X[n] & = & \frac{1}{\pi} \int_{\mathbb{C}} p_{\boldsymbol{X}_c}(\boldsymbol{r}) \mathbb{L}_n \left( | \boldsymbol{s} |^2 \right) \exp( \boldsymbol{r} \boldsymbol{s}^* - \boldsymbol{r}^* \boldsymbol{s}) d\boldsymbol{r} d\boldsymbol{s}, \end{split} \end{equation}

      where $\mathbb{L}_n$ denotes the $n^{\rm th}$ Laguerre polynomial. As in [arxiv:1605.07853], for $0 \leq \eta \leq 1$, we define the beamsplitter addition operation $\boxplus_{\eta}$ acting on random variables supported on $\mathbb{Z}_+$ by

      \begin{equation} X \boxplus_{\eta} Y = \mathcal{T}^{-1} \left( \sqrt{ \eta}\, \mathcal{T}( X ) + \sqrt{1-\eta}\, \mathcal{T}(Y) \right),\end{equation}

      where ‘$+$’ on the RHS of the above denotes standard addition in $\mathbb{C}$.

      Remark: With $Z \triangleq X \boxplus_\eta Y$, for $\eta = 1$, $Z = X$ and with $\eta = 0$, $Z = Y$. Also, when $Y = 0$, i.e., $p_Y[n] = \delta[0]$, $Z = T_\eta X$ (Renyi thinning). Similarly, with $X=0$ or $p_X[n] = \delta[0]$, $Z = T_{1-\eta}Y$.

      An alternative definition of the $\boxplus_\eta$ addition can be expressed in terms of the following exponential generating function:

      \begin{equation} \widetilde{\phi}_{\boldsymbol{X}_c}(t) := \sum_{m=0}^{\infty} \frac{ t^m}{m!} \left(\frac{\mathbb{E}|\boldsymbol{X}_c|^{2m}}{m!}\right), \end{equation}

      where $\mathbb{E} \left( X \right)_{(m)} := \mathbb{E} X(X-1) \ldots (X-m+1) = \mathbb{E} X!/(X-m)!$ is the falling moment of a random variable on $\mathbb{Z}_+$. We then have:

      Theorem: Given independent random variables $X$ and $Y$ supported on $\mathbb{Z}_+$, and $Z := X \boxplus_{\eta} Y$,

      \begin{equation} \widetilde{H}_Z(t) = \widetilde{H}_{X}( \eta t) \widetilde{H}_{Y}((1-\eta) t). \end{equation}

      Let us define $\cal{E}(\mu) = (1+\mu)\log_2(1+\mu) - \mu \log_2\mu$ as the entropy of the ${\rm Geom}(\mu)$ random variable, and $N(X) = {\cal E}^{-1}(H(X))$ as the discrete entropy power.

      Let us define for i.i.d. $\left\{X_1, \ldots, X_n\right\}$: \begin{equation} U_n = X_1 \boxplus_{1/n} X_2 \boxplus_{1/n} … \boxplus_{1/n} X_n, \end{equation} with $E[X_i] = \mu$.

      Problem 1.1.

      [Saikat Guha] Open problems:

      1. Entropy power inequality

        \begin{equation} N(Z) \geq \eta N(X) + (1-\eta) N(Y). \end{equation}

      2. Monotonicity in the CLT

        \begin{equation} H(U_n) \ge H(U_{n-1}), \, n = 2, 3, … \end{equation}

          Known results so far:

      1. EPI with entropies replacing entropy powers

        \begin{equation} H(Z) \ge \eta H(X) + (1-\eta) H(Y). \end{equation}

      2. Proof of EPI in Open Problem 1 for $Y$ geometric

      3. Monotonicity in the CLT in powers of $2$

        \begin{eqnarray} H(U_2) &\ge& H(U_{1}) \\ H(U_{2^k}) &\ge& H(U_{2^{k-1}}), k = 1, 2, … \end{eqnarray}

      4. Convergence in the CLT

        \begin{equation} U_n \longrightarrow_{d} \text{Geom}(\mu). \end{equation}

          Cite this as: AimPL: Entropy power inequalities, available at http://aimpl.org/entropypower.