When Do Low-Rate Concatenated Codes Approach the Gilbert–Varshamov Bound?

5 A Soft-Decoding Sufficient Condition

In this chapter \(\mathcal{C}_{\mathrm{in}}\) is fixed and \(\tau \)-nice, and we seek a deterministic condition on \(\mathcal{C}_{\mathrm{out}}\) alone guaranteeing that \(\mathcal{C}_{\mathrm{out}}\circ \mathcal{C}_{\mathrm{in}}\) approaches the GV bound. Here \(X_m\) and \(\Omega \) are as in Chapter 3.

Definition 44 Bad message

Fix a constant \(c {\gt} 0\). A message \(m \in \mathbb {F}_q^k\setminus \{ 0\} \) is bad if \(|X_m| \geq c\varepsilon N\). If there are no bad messages then every nonzero codeword of \(\mathcal{C}_{\mathrm{out}}\circ \mathcal{C}_{\mathrm{in}}\) has relative weight at least \(\frac{1-c\varepsilon }{2}\).

Definition 45 Moment counter \(\mathcal{B}_r\)

For \(r \geq 0\) define

\[ \mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}}) \triangleq \frac{q^k}{q^k-1}\cdot \frac{1}{(c\varepsilon N)^r}\sum _{\mathcal{V}\in ([n]\times \Omega )^r} \big(q^k\mathbf{1}[g_\mathcal{V}\in \mathcal{C}_{\mathrm{out}}^\perp ]-1\big). \]
Lemma 46 Bad messages are few, Observation 5.2

For any \(r\), the number of bad messages \(m \in \mathbb {F}_q^k\setminus \{ 0\} \) is at most \(\mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}})\).

Proof

By Lemma 40, \(\mathbb {E}_{m}[X_m^r] = \frac{1}{q^k-1} \sum _\mathcal{V}(q^k\mathbf{1}[g_\mathcal{V}\in \mathcal{C}_{\mathrm{out}}^\perp ]-1)\). For \(r\) even, \(X_m^r = |X_m|^r\), and Markov’s inequality (Lemma 23) gives \(\Pr _m[|X_m|\geq c\varepsilon N] = \Pr _m[X_m^r \geq (c\varepsilon N)^r]\leq \frac{1}{q^k}\mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}})\). The number of bad \(m\) equals \((q^k-1)\Pr _m[|X_m|\geq c\varepsilon N]\leq \frac{q^k-1}{q^k} \mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}})\leq \mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}})\).

Definition 47 Distribution \(\mathcal{D}_r\)

Let \(\mathcal{D}_r\) be the distribution on \(g \in \mathbb {F}_q^n\) obtained by sampling \(\mathcal{V}= ((\alpha _1,b_1),\dots ,(\alpha _r,b_r))\) with each \((\alpha _i,b_i) \in [n]\times \Omega \) independent and uniform, and setting \(g_\alpha = \sum _{b \in \mathcal{V}_\alpha } b\). Rewriting Definition 45 as an expectation gives

\begin{equation} \mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}}) = \frac{q^k}{q^k-1}\Big(\tfrac {1}{c\varepsilon }\Big)^r \big(q^k\Pr _{g\sim \mathcal{D}_r}[g\in \mathcal{C}_{\mathrm{out}}^\perp ]-1\big). \tag {12} \end{equation}
1

Lemma 48 Poissonised decoupling, Claim 5.3

Suppose \(r \sim \operatorname {Poi}(\tilde c\varepsilon ^2 N)\) and \(g \sim \mathcal{D}_r\). Then the joint distribution of the coordinates \(g_\alpha \) is

\[ g_\alpha = \sum _{b\in \Omega }\zeta _{\alpha ,b}\cdot b,\qquad \zeta _{\alpha ,b}\sim \operatorname {Ber}\! \Big(\tfrac {1 - e^{-2\tilde c\varepsilon ^2}}{2}\Big) \text{ i.i.d.} \]

In particular the \(g_\alpha \) are independent and identically distributed across \(\alpha \in [n]\), matching the product distribution \(\mathcal{D}^n\) of Theorem 50.

Proof

View choosing the \((\alpha _i,b_i)\) as dropping \(r\) balls into the \(N = |[n]\times \Omega |\) bins. Since \(r \sim \operatorname {Poi}(\tilde c\varepsilon ^2 N)\), by Poisson splitting (Lemma 30) the occupancy \(Y_{\alpha ,b}\) of each bin \((\alpha ,b)\) is independent and distributed as \(\operatorname {Poi}(\tilde c\varepsilon ^2)\). Since \(Y_{\alpha ,b}\) is the number of times \(b\) appears in \(\mathcal{V}_\alpha \), working over a field of characteristic \(2\) we have \(g_\alpha = \sum _b Y_{\alpha ,b}\, b = \sum _b \zeta _{\alpha ,b}\, b\) with \(\zeta _{\alpha ,b} = Y_{\alpha ,b}\bmod 2\). The \(\zeta _{\alpha ,b}\) are independent, and by Lemma 29, \(\Pr [\zeta _{\alpha ,b} = 1] = \Pr [Y_{\alpha ,b}\text{ odd}] = \frac12(1 - e^{-2\tilde c\varepsilon ^2})\).

Lemma 49 The \(g = 0\) term is small, Claim 5.4

Suppose the constants \(c,\tilde c\) satisfy \(\tilde c \geq 4\ln 2\) and \(c \geq 72\tilde c\), that \(n\) is sufficiently large, \(r \sim \operatorname {Poi}(\tilde c \varepsilon ^2 N)\), and \(n_0 \geq 64/\varepsilon ^4\). Let

\[ (13) \triangleq q^k\Big(1 + \tfrac {1}{q^k-1}\Big) \Big(\tfrac {1}{c\varepsilon }\Big)^r \Pr _{g\sim \mathcal{D}_r}[g = 0]. \]

Then \(\mathbb {E}_r[(13)] \leq 2^{-\varepsilon ^2 N/2}\).

Proof

For fixed \(r\), the event \(g = 0\) is exactly that all rows of \(B(\mathcal{V})\) lie in \(\mathcal{C}_{\mathrm{in}}^\perp \), so by Claim 42 (dividing the bound on \(|W|\) by \(N^r\)),

\[ \Pr _{g\sim \mathcal{D}_r}[g=0] \leq 8^r(r/N)^{r/2} 2^{2N/\sqrt{n_0}+\log _2 N}\big(\max \{ 1,\tfrac {er}{\varepsilon ^2 N}\} \big)^{r/2}. \]

Let \(\lambda = \tilde c\varepsilon ^2 N\) be the Poisson mean. Taking the expectation over \(r \sim \operatorname {Poi}(\lambda )\) and using the Poisson pmf (Lemma 26),

\[ \mathbb {E}_r[q^k(\tfrac {1}{c\varepsilon })^r\Pr _{\mathcal{D}_r}[g=0]] \leq e^{-\lambda }q^k 2^{3N/\sqrt{n_0}}\sum _{r\geq 0}\frac{1}{r!} \Big(\tfrac {8\lambda \sqrt{r/N}}{c\varepsilon }\Big)^r \big(\max \{ 1,\tfrac {er}{\varepsilon ^2 N}\} \big)^{r/2}. \]

Splitting the sum at \(r = \varepsilon ^2 N/e\): the first part (where the maximum is \(1\)) is bounded, using \(r \leq \varepsilon ^2 N/e\), by \(\sum _{r}\frac{1}{r!}\big(\frac{8\lambda \sqrt{\varepsilon ^2/e}}{c\varepsilon }\big)^r = \exp (\frac{8}{c\sqrt e}\lambda )\). The second part (where the maximum is \(\frac{er}{\varepsilon ^2 N}\)), using \(r! \geq (r/e)^r\), is bounded by \(\sum _{r {\gt} \varepsilon ^2 N/e}\big(\frac{8e\sqrt e\, \tilde c}{c}\big)^r \leq 2^{-\varepsilon ^2 N/e} {\lt} 1\) provided \(c \geq 16 e\sqrt e\, \tilde c\). Hence

\[ \mathbb {E}_r[q^k(\tfrac {1}{c\varepsilon })^r\Pr _{\mathcal{D}_r}[g=0]] \leq 2e^{-\lambda }q^k 2^{3N/\sqrt{n_0}}\exp (8\lambda /c) \leq \exp _2\! \Big(-\varepsilon ^2 N\big(1 - \tfrac {4}{\varepsilon ^2\sqrt{n_0}}\big)\Big) \leq 2^{-\varepsilon ^2 N/2}, \]

using \(q^k = 2^{\varepsilon ^2 N}\), \(\lambda = \tilde c\varepsilon ^2 N\), the hypotheses \(\tilde c \geq 4\ln 2\), \(c \geq 16\), and finally \(n_0 \geq 64/\varepsilon ^4\). The factor \(1 + \frac{1}{q^k-1}\leq 2\) only changes the constant.

There are constants \(\tilde c, c {\gt} 0\) so that the following holds. Let \(\varepsilon {\gt} 0\), and let \(\mathcal{C}_{\mathrm{in}}\subseteq \mathbb {F}_2^{n_0}\) be a binary linear code of dimension \(k_0 = \varepsilon n_0\) that is \(\tau \)-nice for \(\tau = 1/\sqrt{n_0}\), with \(n_0 \geq 64/\varepsilon ^4\). Let \(\Omega \subseteq \mathbb {F}_q\) be the set derived from \(\mathcal{C}_{\mathrm{in}}\) (Definition 38). Let \(Y \in \mathbb {F}\) be \(Y = \sum _{b\in \Omega }\zeta _b\cdot b\) where \(\zeta _b \sim \operatorname {Ber}\big(\frac{1-e^{-2\tilde c\varepsilon ^2}}{2}\big)\) are i.i.d., and let \(\mathcal{D}\) be the distribution of \(Y\). Suppose \(\mathcal{C}_{\mathrm{out}}\subseteq \mathbb {F}_q^n\) is a linear code of dimension \(k = \varepsilon n\) with

\begin{equation} \Pr _{x\sim \mathcal{D}^n}[x \in \mathcal{C}_{\mathrm{out}}^\perp \setminus \{ 0\} ] \leq \frac{1}{q^k}(1 + \Delta ),\qquad \Delta \leq \Big(\tfrac {c\varepsilon }{2}\Big)^{\tilde c\varepsilon ^2 N + 100\sqrt{\tilde c\varepsilon ^2 N}}. \tag {11} \end{equation}
2

Then \(\mathcal{C}_{\mathrm{out}}\circ \mathcal{C}_{\mathrm{in}}\) is a binary linear code of rate \(\varepsilon ^2\) with relative distance at least \(\frac{1-c\varepsilon }{2}\).

Proof

The rate is \(\varepsilon ^2\) by construction, so we bound the distance via the number of bad messages. By Observation 46 it suffices to find \(r\) with \(\mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}}) {\lt} 1\), since then there are no bad messages and the relative distance is at least \(\frac{1-c\varepsilon }{2}\) (Definition 44). Choose \(r \sim \operatorname {Poi}(\lambda )\) with \(\lambda = \tilde c\varepsilon ^2 N\). Splitting (12) into the events \(g = 0\) and \(g \neq 0\) gives \(\mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}}) = (13) + (14)\) with \((13)\) as in Claim 49 and

\[ (14) = \frac{q^k}{q^k-1}\Big(\tfrac {1}{c\varepsilon }\Big)^r \big(q^k\Pr _{g\sim \mathcal{D}_r}[g\in \mathcal{C}_{\mathrm{out}}^\perp \setminus \{ 0\} ]-1\big). \]

By Claim 49, \(\mathbb {E}_r[(13)]\leq 2^{-\varepsilon ^2 N/2}\), so by Markov’s inequality, with probability at least \(1 - 2^{-\varepsilon ^2 N/4}\) over \(r\),

\begin{equation} (13)\leq 2^{-\varepsilon ^2 N/4}. \tag {15} \end{equation}
3

For \((14)\): by Claim 48, when \(r \sim \operatorname {Poi}(\lambda )\) the distribution \(\mathcal{D}_r\) equals the product distribution \(\mathcal{D}^n\) of the theorem, so by hypothesis (11), \(q^k\Pr _{r\sim \operatorname {Poi}(\lambda ),g\sim \mathcal{D}_r}[g\in \mathcal{C}_{\mathrm{out}}^\perp \setminus \{ 0\} ] - 1 \leq \Delta \leq (\frac{c\varepsilon }{2})^{\lambda + 100\sqrt\lambda }\). By the Poisson Chernoff bound (Lemma 31), \(\Pr [r \geq \lambda + 100\sqrt\lambda ]\leq \exp (-100^2/2)\). Union bounding over (15) and the event \(r \leq \lambda + 100\sqrt\lambda \), with positive probability over \(r\) we have

\[ \mathcal{B}_r(\mathcal{C}_{\mathrm{out}},\mathcal{C}_{\mathrm{in}})\leq 2^{-\varepsilon ^2 N/4} + \frac{q^k}{q^k-1}\Big(\tfrac {1}{c\varepsilon }\Big)^r \Big(\tfrac {c\varepsilon }{2}\Big)^{\lambda +100\sqrt\lambda } \leq 2^{-\varepsilon ^2 N/4} + 2^{-\tilde c\varepsilon ^2 N} {\lt} 1, \]

using \(r \leq \lambda + 100\sqrt\lambda \) and \(c\varepsilon {\lt} 1\) so that \((\frac{1}{c\varepsilon })^r(\frac{c\varepsilon }{2})^{\lambda +100\sqrt\lambda } \leq 2^{-(\lambda +100\sqrt\lambda )}\leq 2^{-\lambda }\). Thus some \(r\) gives \(\mathcal{B}_r {\lt} 1\), proving the theorem.

Proposition 51 Codes satisfying (11) exist, Remark 8

There exists a linear code \(\mathcal{C}_{\mathrm{out}}\subseteq \mathbb {F}_q^n\) of dimension \(k\) with \(\Pr _{x\sim \mathcal{D}^n}[x\in \mathcal{C}_{\mathrm{out}}^\perp \setminus \{ 0\} ]\leq \frac{1}{q^k}\), satisfying the condition of Theorem 50 with \(\Delta = 0\).

Proof

Let \(\mathcal{C}_{\mathrm{out}}\) be a random linear code of dimension \(k = \varepsilon n\). Then \(\mathcal{C}_{\mathrm{out}}^\perp \) is a random linear code of dimension \(n - k\), and for each fixed nonzero \(x\), \(\Pr _\mathcal{C}_{\mathrm{out}}[x\in \mathcal{C}_{\mathrm{out}}^\perp ] = \frac{q^{n-k}-1}{q^n-1}\leq q^{-k}\). Taking the expectation over \(\mathcal{C}_{\mathrm{out}}\),

\[ \mathbb {E}_\mathcal{C}_{\mathrm{out}}\Big[\Pr _{x\sim \mathcal{D}^n}[x\in \mathcal{C}_{\mathrm{out}}^\perp \setminus \{ 0\} ]\Big] = \sum _{x\neq 0}\mathcal{D}^n(x)\Pr _\mathcal{C}_{\mathrm{out}}[x\in \mathcal{C}_{\mathrm{out}}^\perp ] = q^{-k} - q^{-n} \leq q^{-k}. \]

Hence some fixed \(\mathcal{C}_{\mathrm{out}}\) of dimension \(k\) achieves \(\Pr _{x\sim \mathcal{D}^n}[x\in \mathcal{C}_{\mathrm{out}}^\perp \setminus \{ 0\} ]\leq q^{-k}\), i.e. (11) with \(\Delta = 0\) (the condition only requires this probability not be much larger than \(1/q^k\)).