11 When Graphs Come to the Party: Expander Codes
In this chapter we explore the use of graph-theoretic methods to build codes either from scratch or from much weaker codes. The graphs of interest are expanders – informally, graphs in which all small enough vertex sets have ‘large neighborhoods.’ Section 11.1 reviews basic graph notions. Section 11.2 introduces two notions of expansion (combinatorial/bipartite expansion and spectral expansion) and relates them. Section 11.3 builds asymptotically good codes from strong bipartite expanders. Section 11.4 generalizes this construction (Tanner codes) to work with weaker expansion. Section 11.5 gives a completely different use of expanders: amplifying the distance of a code using expanding graphs, which in particular gives codes approaching the Singleton bound over constant-sized alphabets and codes approaching the Zyablov bound over the binary alphabet. Finally Section 11.6 proves the existence of the strong (‘lossless’) bipartite expanders used in Section 11.3.
11.1 Graphs
We start by recalling that a graph \(G\) is given by a pair \((V,E)\) where \(V\) is a finite set called the vertex set and \(E\) is a family of unordered pairs of elements of \(V\) called the set of edges. Given a vertex \(u \in V\), the set of vertices \(N(u) = \{ v \in V \mid (u,v) \in E\} \) is referred to as the neighbors of \(u\), and \(\deg (u) = |N(u)|\) is called the degree of \(u\). It will sometimes be convenient to view \(N(u)\) as an ordered tuple \(N(u) = (v_1,\ldots ,v_{\deg (u)})\) of elements of \(V\); when this ordering matters we refer to \(G\) as an ordered graph.
A bipartite graph is a triple \(G = (L,R,E)\), where \(L\) is the set of ‘left’ vertices, \(R\) is the set of ‘right’ vertices, and \(E \subseteq L \times R\) is the set of edges. (A bipartite graph \((L,R,E)\) is a special case of a graph \((V = L \cup R, E)\).)
Given a bipartite graph \(G = (L,R,E)\), its adjacency matrix, denoted \(A_G\), is an \(|R| \times |L|\) binary matrix, where we index each row by an element of \(R\) and every column by an element of \(L\) such that for every \((r,\ell ) \in R \times L\), the \((r,\ell )\)’th entry of \(A_G\) is \(1\) if \((\ell ,r) \in E\) and \(0\) otherwise. Conversely, every matrix \(A \in \{ 0,1\} ^{n \times m}\) corresponds to a bipartite graph \(G_H = (L,R,E)\) with \(|L| = m\) and \(|R| = n\) with \((i,j) \in E\) if and only if \(A_{ij} = 1\).
The adjacency matrix of a bipartite graph can always be viewed as the parity check matrix of a code and vice versa; when \(G\) is sparse (each left vertex has at most a constant number of right neighbors), the associated parity check matrix has sparse rows, a property that (together with more global properties of the graph) is very useful in the construction of good error-correcting codes.
A graph \(G = (V,E)\) is said to be \(c\)-bounded if every vertex has degree at most \(c\). It is said to be \(c\)-regular if every vertex has degree exactly \(c\). Similarly, a bipartite graph \(G = (L,R,E)\) is said to be \(c\)-left regular (bounded) if every vertex in \(L\) has degree exactly (resp., at most) \(c\). It is said to be \(d\)-right regular (bounded) if every vertex in \(R\) has degree exactly (resp., at most) \(d\).
11.2 Expander Graphs
11.2.1 Bipartite Vertex Expanders
For a left vertex set \(S \subseteq L\), a vertex \(u \in R\) is called a neighbor of \(S\) if it is adjacent to some vertex in \(S\). We denote by \(N(S)\) the set of neighbors of \(S\). We analogously define \(N(T)\) for any \(T \subseteq R\).
An \((n,m,c,\gamma ,\alpha )\) bipartite expander is a \(c\)-left regular bipartite graph \(G = (L,R,E)\) where \(|L| = n\) and \(|R| = m\) such that for every \(S \subseteq L\) with \(|S| \le \gamma n\), we have
An infinite family of bipartite graphs \(\{ G_i\} _{i \in \mathbb {Z}_+}\) is termed a \((c,\gamma ,\alpha )\)-bipartite expander family if for every \(i \in \mathbb {Z}_+\), \(G_i\) is an \((n_i,m_i,c,\gamma ,\alpha )\) expander for some \(n_i,m_i \in \mathbb {Z}_+\). We say \(\{ G_i\} _{i \in \mathbb {Z}_+}\) is a bipartite expander family if there exist \(c \in \mathbb {Z}_+\) and \(\alpha ,\gamma {\gt} 0\) such that \(\{ G_i\} _{i \in \mathbb {Z}_+}\) is a \((c,\gamma ,\alpha )\)-bipartite expander family.
Since always \(\alpha \le c\), and since \(\gamma \le |R|/(\alpha |L|)\) for a family to have \(\gamma {\gt} 0\) as \(|L| \to \infty \) we need \(|R| = \Omega (|L|)\); this is the setting we are interested in below.
For every \(\varepsilon {\gt} 0\) and large enough \(m \le n\), there exists an \((n,m,c,\gamma ,c(1-\varepsilon ))\) bipartite expander where \(c = \Theta \! \left(\frac{\log (n/m)+\log (1/\varepsilon )}{\varepsilon }\right)\) and \(\gamma = \Theta \! \left(\frac{\varepsilon m}{c n}\right)\).
We use the probabilistic method. Let \(\bar c {\gt} 0\) be a large enough constant (to be fixed at the end of the proof), and set
We pick \(G = (L,R,E)\) to be a random bipartite graph with \(|L|=n\), \(|R|=m\): for every \(\ell \in L\) we pick \(c\) random (with replacement) vertices in \(R\) and connect them to \(\ell \) (this may produce a multigraph, but this is easily remedied afterwards without hurting the expansion).
Fix an integer \(1 \le j \le \lfloor \gamma n \rfloor \) and an arbitrary \(S \subseteq L\) of size exactly \(j\). Let \(r_1,\ldots ,r_{jc}\) denote the \(jc\) (ordered) random neighbor choices made by the \(j\) vertices of \(S\). Call a choice \(r_i\) (for \(i{\gt}1\)) a repeat if \(r_i \in \{ r_1,\ldots ,r_{i-1}\} \). If the total number of repeats is at most \(\varepsilon j c\), then \(|N(S)| \ge c(1-\varepsilon )j\), since at most \(\varepsilon jc\) of the \(jc\) edges out of \(S\) can fail to reach a ‘new’ vertex of \(R\).
For any fixed \(i\), the probability that \(r_i\) is a repeat is at most \(\frac{i-1}{m} \le \frac{jc}{m}\), since \(r_i\) is uniform on \([m]\) and there are at most \(i - 1 \le jc\) earlier choices. Taking a union bound over the (at most) \(\binom {cj}{\varepsilon jc}\) choices of which indices are repeats, and using that each repeat has probability at most \(jc/m\), we get
where the second inequality is the standard binomial coefficient bound \(\binom {a}{b}\le (ea/b)^b\), and the last inequality uses \(2e\gamma n = \varepsilon m /c\), i.e., \(ec/(\varepsilon m) \le 1/(2\gamma n)\), which holds by our choice of \(\gamma \).
Taking a union bound over all \(\binom {n}{j}\) choices of \(S\) of size \(j\), the probability that there exists some \(S\) of size \(j\) that fails to expand by a factor of \(c(1-\varepsilon )\) is at most
where the last inequality is verified below. Taking a union bound over all \(1 \le j \le \gamma n\), the probability that \(G\) fails to be an \((n,m,c,\gamma ,c(1-\varepsilon ))\) bipartite expander is strictly smaller than \(\sum _{j\ge 1} 2^{-j} = 1\); hence, with strictly positive probability, the random \(G\) is such a bipartite expander, so one exists, as desired.
It remains to verify \(\left(\frac{en}{j}\right)\cdot \left(\frac{j}{2\gamma n}\right)^{\varepsilon c} \le \frac12\) for every \(1 \le j \le \gamma n\). Since \(\varepsilon c {\gt} 1\) by our choice of \(c\), the left-hand side is increasing in \(j\), so it suffices to check the inequality at \(j = \gamma n\):
which holds if
This is satisfied by our choice of \(c\) for a large enough constant \(\bar c\) (the \(\log c\) term on the right grows much more slowly in \(\bar c\) than the left-hand side, so a sufficiently large constant \(\bar c\) makes the inequality hold for all \(n,m,\varepsilon \) in the relevant range). This completes the proof.
The construction of explicit bipartite expanders with expansion factor \(\alpha \) close to the degree \(c\) is considerably more involved and we take it as a black box.
For every constant \(\varepsilon {\gt} 0\) and every desired ratio \(0 {\lt} \beta {\lt} 1\), there exist explicit \((n,m,c,\gamma ,c(1-\varepsilon ))\) bipartite expanders for any large enough \(n\) (and \(m = \beta n\)) with \(c\) and \(\gamma {\gt} 0\) being constants (that only depend on \(\varepsilon \) and \(\beta \)).
Let \(G\) be an \((n,m,c,\gamma ,\alpha )\) bipartite expander. Then there exists another bipartite graph \(G'\) that is an \((n,m',c,\gamma ,\alpha )\) bipartite expander such that
\(m \le m' \le 2m\);
every right vertex in \(G'\) has degree at most \(\left\lceil \frac{nc}{m}\right\rceil \).
Let \(G = (L,R,E)\) and let us construct \(G' = (L,R',E')\) with the required properties. Define \(d = \left\lceil \frac{nc}{m}\right\rceil \). For each vertex \(r \in R\), let \(d_r\) be its degree. For each \(r \in R\), add \(\left\lceil \frac{d_r}{d}\right\rceil \) vertices to \(R'\); further, the \(d_r\) edges incident to vertex \(r\) are divided evenly among the corresponding \(\left\lceil \frac{d_r}{d}\right\rceil \) vertices in \(R'\) so that all but potentially one such vertex has degree exactly \(d\) (and at most one vertex has degree \({\lt} d\)).
The claim on the right degree bound in \(G'\) follows by construction. The claim on expansion follows since splitting the vertices this way can only increase vertex expansion: for any \(S \subseteq L\), \(N_{G'}(S)\) is the disjoint union of the split copies of the vertices in \(N_G(S)\), so \(|N_{G'}(S)| \ge |N_G(S)| \ge \alpha |S|\) whenever \(|S| \le \gamma n\).
It remains to show \(m' - m \le m\), where \(m' = |R'|\) and \(m = |R|\). We have
where the middle equality uses \(\sum _{r\in R} d_r = nc\) (both count the edges of \(G\)), and the last inequality follows from the definition of \(d\).
To summarize, we have defined bipartite expansion (Definition 217). Expander families with \(\alpha = (1-\varepsilon )c\) exist for every \(\varepsilon {\gt} 0\), for sufficiently large \(c\) (Theorem 218), and can even be explicitly constructed (Theorem 219), albeit via a construction that is complex and out of scope for this book.
11.2.2 Spectral Expanders
Our next notion of expansion is based on the linear-algebraic properties of a graph’s adjacency matrix, rather than combinatorial properties of vertex neighborhoods. The basic objects of interest here are non-bipartite \(d\)-regular graphs.
Let \(\mathbf{M} \in \mathbb {R}^{n \times n}\) be a matrix over the reals. Then \(\lambda _i\) is an eigenvalue of \(\mathbf{M}\) if there exists a vector \(\mathbf{v}_i \in \mathbb {R}^n\), \(\mathbf{v}_i \ne 0\), such that \(\mathbf{M}\cdot \mathbf{v}_i = \lambda _i \cdot \mathbf{v}_i\).
Every real symmetric \(n \times n\) matrix has \(n\) real eigenvalues (counted with multiplicity). In particular, since the adjacency matrix of a graph on \(n\) vertices is real and symmetric, we may associate to the graph the \(n\) eigenvalues \(\lambda _1 \ge \lambda _2 \ge \cdots \ge \lambda _n\) of its adjacency matrix.
A graph \(X = (V,E)\) is said to be an \((n,d,\lambda )\)-spectral expander if \(X\) is a \(d\)-regular (not necessarily bipartite) graph on \(n\) vertices and \(\lambda \ge \max \{ |\lambda _2|,|\lambda _n|\} \), where \(\lambda _1 \ge \lambda _2 \ge \cdots \ge \lambda _n\) are the eigenvalues of the adjacency matrix of \(X\).
For a positive integer \(d\) and \(0 \le \lambda {\lt} d\), we say that an infinite family of graphs \(\{ X_n\} _{n\in \mathbb {Z}_+}\) with \(X_n\) having \(n\) vertices and being \(d\)-regular, is a \((d,\lambda )\)-spectral expander family if \(\lambda (X_n) \le \lambda \) for every \(n\). We refer to a family as a spectral expander family if there exists \(\lambda {\lt} d\) such that the family is a \((d,\lambda )\)-spectral expander family.
Given an undirected graph \(X = (V,E)\) and sets \(A,B \subseteq V\), we define \(E(A,B)\) to be the set of pairs \((a,b)\) with \(a \in A\) and \(b \in B\) such that \((a,b) \in E\). I.e., \(E(A,B) = \{ (a,b) \in A \times B \mid (a,b) \in E\} \). (Note that if \(a,b \in A \cap B\) then both \((a,b)\) and \((b,a)\) belong to \(E(A,B)\).)
Let \(X = (V,E)\) be a \((n,d,\lambda )\)-graph. Then for every \(A,B \subseteq V\) (possibly \(A=B\)) we have
In particular, this implies
The expander mixing lemma is a powerful tool for the analysis of combinatorial properties of a spectral expander: for instance it implies an upper bound of \(n\lambda /d\) on the size of an independent set in an expander.
Explicit spectral expanders with \(\lambda \ll d\) exist and can be constructed surprisingly simply (e.g., the \(8\)-regular graph on \(\mathbb {Z}_n\times \mathbb {Z}_n\) with \(N((x,y)) = \{ (x\pm 1,y),(x,y\pm 1),(x\pm 2y,y),(x\pm 2x,y)\} \) is a \((n,8,\lambda )\)-spectral expander for some \(\lambda {\lt} 8\)). One can show that for an infinite family of \(d\)-regular graphs we must have \(\lambda \ge 2\sqrt{d-1}-o(1)\); families meeting this bound are called Ramanujan graphs. For our purposes, it suffices that the ratio \(\lambda /d\) can be made arbitrarily small by picking \(d\) large enough.
For every \(\varepsilon {\gt} 0\), for all large enough \(d\), there exists an infinite family of \(d\)-regular \((n,d,\lambda )\)-graphs with \(\lambda \le \varepsilon d\). Here by explicit we mean that the adjacency matrix of the graph can be constructed in polynomial time. Furthermore, we can take \(d = O(1/\varepsilon ^2)\), which in turn implies that for large enough \(d\), we can in polynomial time construct an \(\left(n,d,O(\sqrt d)\right)\)-graph.
To summarize, we have defined spectral expansion (Definition 223). Expander families with \(\lambda = o(d)\), and even with \(\lambda = O(\sqrt d)\), can be constructed explicitly (Theorem 227).
11.2.3 Bipartite Expanders from Spectral Expanders
We now show how to convert a spectral expander (with \(\lambda = o(d)\)) into a bipartite expander (with \(\alpha {\gt} 0\)). This conversion uses two operations: the double cover and the edge-vertex incidence graph.
The Double Cover of a graph \(X = (V_X,E_X)\) is defined as the bipartite graph \(G' = (L_{G'},R_{G'},E_{G'})\) with vertices \(L_{G'} = \{ u' \mid u \in V_X\} \) and \(R_{G'} = \{ u'' \mid u \in V_X\} \), and edges \(E_{G'} = \{ (u',v''),(v',u'') \mid (u,v) \in E_X\} \).
Thus the double cover \(G'\) of \(X\) has two copies of each vertex of \(X\), one in the left partition and the other in the right partition, and two copies of each edge \((u,v)\) of \(X\).
The Edge Vertex Incidence Graph of a graph \(G' = (V,E)\) is defined as the bipartite graph \(G = (L,R,E')\) where \(L = E\), \(R = V\) and \(E' = \{ (e,v) \mid v \in e \in E\} \).
Given an \((n,d,\lambda )\)-spectral expander \(X\), we consider the bipartite graph \(G\) obtained by taking the double cover \(G'\) of \(X\) and then taking \(G\) to be the edge-vertex incidence graph of \(G'\). The following theorem shows that this leads to a bipartite expander.
Let \(X\) be an \((n,d,\lambda )\)-spectral expander. Let \(G'\) be the double cover of \(X\) and \(G\) be the edge-vertex incidence graph of \(G'\). Then, for every \(\beta \) such that \(\lambda \le \beta \le d\), we have that \(G\) is a \(d\)-right regular \(\left(dn,2n,2,\frac{\beta (\beta -\lambda )}{d^2},\frac{2}{\beta }\right)\)-expander.
We inspect \(G\), the edge-vertex incidence graph of the double cover \(G'\) of \(X = (V,E)\), and state the expansion conditions of \(G\) in terms of the structure of \(X\). Let \(E'\) denote the edges of the double cover \(G'\) of \(X\), and thus the left vertices of \(G\); note \(|E'| = dn\). The right vertices of \(G\) correspond to the vertices of \(G'\); let \(V_1 \cup V_2\) denote these vertices, with \(|V_1| = |V_2| = n\), such that every left vertex of \(G\) has exactly one neighbor in each of \(V_1\) and \(V_2\). Thus \(G\) is indeed a graph with \(dn\) left vertices and \(2n\) right vertices that is \(2\)-left regular and \(d\)-right regular. It remains to show that sets containing at most \(\frac{\beta (\beta -\lambda )}{d^2}\) fraction of left vertices expand by a factor of at least \(2/\beta \). Specifically, let \(S \subseteq E'\) be an arbitrary set of left vertices of size at most \(\frac{\beta (\beta -\lambda )}{d^2}dn = \frac{\beta (\beta -\lambda )}{d}n\). Let \(N(S)\) denote the set of neighbors of \(S\) and let \(N(S) = A \cup B\) where \(A \subseteq V_1\) and \(B \subseteq V_2\). Our goal is to show \(|A|+|B| \ge (2/\beta )|S|\).
By the construction of \(G\), every vertex of \(S\) is an edge of \(X\) from the set \(A\) to the set \(B\). In particular \(|S| \le |E(A,B)|\), and so by the expander mixing lemma (Lemma 226) we have
We first argue \(|S| \le \beta \sqrt{|A||B|}\). Suppose for contradiction that \(|S| {\gt} \beta \sqrt{|A||B|}\). Then
Dividing both sides by \(\sqrt{|A||B|}\) and rearranging, \(\sqrt{|A||B|} {\gt} \frac{\beta -\lambda }{d}n\). Using \(|S| {\gt} \beta \sqrt{|A||B|}\) again, we get \(|S| {\gt} \frac{\beta (\beta -\lambda )}{d}n\), contradicting the given upper bound on \(|S|\). We conclude \(|S| \le \beta \sqrt{|A||B|}\).
Combining this with the AM-GM inequality \(\sqrt{|A||B|} \le (|A|+|B|)/2\) gives
and thus \(|A|+|B| \ge \frac{2}{\beta }|S|\), as desired. In other words, every set \(S\) with \(|S| \le \frac{\beta (\beta -\lambda )}{d^2}\cdot dn\) satisfies \(|N(S)| \ge \frac{2}{\beta }\cdot |S|\), so \(G\) is a \(d\)-right regular \(\left(dn,2n,2,\frac{\beta (\beta -\lambda )}{d^2},\frac{2}{\beta }\right)\)-expander.
To summarize, spectral expansion with \(\lambda = o(d)\) yields bipartite expansion (in related graphs) with \(\alpha {\gt} 0\).
11.3 Basic Expander Codes
In this section we use strong bipartite expanders (with expansion \(\alpha {\gt} c/2\)) to build asymptotically good error-correcting codes. We can associate a binary code to every bipartite graph and vice versa, by identifying the adjacency matrix of the graph with the parity check matrix of the code.
Given any \([n,k]_2\) code \(C\) with parity check matrix \(H\), we will call the bipartite graph \(G_H\) with \(A_{G_H} = H\) a factor graph of \(C\). Further, given a bipartite graph \(G = (L,R,E)\) with \(|L| \ge |R|\), we will denote the corresponding code with parity check matrix \(A_G\) to be \(C(G)\).
Note that the factor graph of a code is not unique, since different parity check matrices may lead to different factor graphs. Of particular interest is the case where the factor graph is sparse, i.e., every row of the parity check matrix has a constant number of non-zero elements; equivalently the right degree of the factor graph is bounded. Codes with such a sparse factor graph are called low density parity check (LDPC) codes.
A code \(C \subseteq \mathbb {F}_2^n\) is called an expander code if there is a bipartite expander graph \(G\) such that \(C = C(G)\). If \(G\) is \(d\)-right bounded then \(C(G)\) is said to be a \(d\)-LDPC (for Low Density Parity Check) Code.
11.3.1 Unique expansion
Given a bipartite graph \(G = (L,R,E)\) and a left vertex set \(S \subseteq L\), a vertex \(u \in R\) is called a unique neighbor of \(S\) if it is adjacent to exactly one vertex in \(S\). We let \(U(S)\) denote the set of unique neighbors of \(S\).
Since \(U(S) \subseteq N(S)\), it follows that if \(|U(S)|\) grows linearly with \(|S|\) for \(|S| \le \gamma |L|\) then so does \(|N(S)|\), and so \(G\) is a bipartite expander. The converse also holds when \(G\) is a \(c\)-left-regular graph with expansion \(\alpha {\gt} c/2\).
Let \(G = (L,R,E)\) be an \((n,m,c,\gamma ,c(1-\varepsilon ))\) bipartite expander with \(\varepsilon {\lt} 1/2\). Then for every \(S \subseteq L\) with \(|S| \le \gamma n\), we have
Let \(U \subseteq N(S)\) be the unique neighbors of \(S\), i.e., those vertices in \(N(S)\) with exactly one neighbor in \(S\). We wish to show \(|U| \ge c(1-2\varepsilon )|S|\), which we do by counting the edges incident to \(S\).
The total number of edges going out of \(S\) is exactly \(c|S|\), since \(G\) is \(c\)-left regular. By the expansion property, \(|N(S)| \ge c(1-\varepsilon )|S|\). For every vertex \(v \in N(S)\), designate exactly one edge \((u,v)\) with \(u \in S\) as ‘special.’ The number of special edges is exactly \(|N(S)| \ge c(1-\varepsilon )|S|\), and so the number of non-special edges is at most \(\varepsilon c|S|\). Now note that if a vertex \(v \in N(S)\) is not a unique neighbor of \(S\) then \(v\) must have a non-special edge incident to it (since only one of the edges incident to it is special). So the total number of vertices in \(N(S)\) that are not unique neighbors is upper bounded by the number of non-special edges, which is at most \(\varepsilon c|S|\). Thus,
as desired.
11.3.2 Rate and Distance of Expander Codes
We now show that if \(G\) is a left-regular expander with \(m/n {\lt} 1\) and \(\alpha {\gt} c/2\), then \(C(G)\) is an asymptotically good code.
Let \(G = (L,R,E)\) be a bipartite graph with \(|L| = n\) and \(|R| = n-k\). Then \((c_1,\ldots ,c_n) \in \{ 0,1\} ^n\) is in \(C(G)\) if and only if the following holds (where \(S = \{ i \in [n] \mid c_i \ne 0\} \)) for every \(r \in R\):
where the sum is over \(\mathbb {F}_2\).
This follows from the definition of \(C(G)\) as the code with parity check matrix \(A_G\) and the fact that \(c_j\) for every \(j \notin S\) does not contribute to any of the computed parities (each summand there is \(0\)).
Let \(G\) be an \((n,n-k,c,\gamma ,c(1-\varepsilon ))\) bipartite expander with \(\varepsilon {\lt} \frac12\). Then \(C(G)\) is a \([n,k',\gamma n+1]_2\) code for some \(k' \ge k\).
The claim on the block length and the linearity of \(C(G)\) follows from the definition of expander codes. To lower bound the dimension, note that the space of codewords is the space of solutions to a homogeneous linear system over \(\mathbb {F}_2\) on \(n\) variables, namely \(c_1,\ldots ,c_n\), with \(n-k\) constraints, namely \(\sum _{\ell \in N(r)} c_\ell = 0\) for every \(r \in R\). The solution space has dimension \(k' \ge n - (n-k) = k\).
We now turn to showing that the distance of the code is at least \(\gamma n + 1\). For the sake of contradiction, assume that \(C(G)\) has distance at most \(\gamma n\). Then, by the linearity of \(C(G)\), there exists a non-zero codeword \(\mathbf{c} \in C(G)\) such that \(wt(\mathbf{c}) \le \gamma n\). Let \(S\) be the set of non-zero coordinates of \(\mathbf{c}\). Since \(G\) is an \((n,n-k,c,\gamma ,c(1-\varepsilon ))\) expander, by Lemma 234 we have
where the inequality follows from the fact that \(\varepsilon {\lt} \frac12\) and \(|S| \ge 1\) (since \(\mathbf{c}\) is non-zero). Thus \(U(S)\) is non-empty and so there exists \(r \in U(S)\). Let \(\ell \in S\) be the unique neighbor of \(r\) in \(S\) implied by the definition of \(U(S)\). Now the parity check corresponding to \(r\) is given by \(\sum _{\ell ' \in N(r) \cap S} c_{\ell '} = c_\ell = 1\) (where the first equality holds since \(N(r) \cap S = \{ \ell \} \) and the final equality holds since \(c_\ell = 1\) for \(\ell \in S\)). Thus the parity check condition corresponding to \(r\) is not satisfied, and so by Proposition 235 \(\mathbf{c} \notin C(G)\), a contradiction, as desired.
Theorem 236 together with Theorem 219 gives an affirmative answer to the question of whether we can construct explicit asymptotically good binary codes without code concatenation. In particular we have the following corollary.
Suppose there exist \(c \in \mathbb {Z}_+\), \(\alpha {\gt} c/2\) and \(\gamma ,\rho {\gt} 0\) such that \(\{ G_i\} _{i\in \mathbb {Z}_+}\) is a \((c,\gamma ,\alpha )\)-bipartite expander with \(G_i\) having \(n_i\) left vertices, \(m_i\) right vertices with \(m_i/n_i {\lt} 1-\rho \) for every \(i \in \mathbb {Z}_+\). Then the family of codes \(\{ C(G_i)\} _{i\in \mathbb {Z}_+}\) is asymptotically good.
By Theorem 236, each \(C(G_i)\) is a \([n_i,k_i',\gamma n_i + 1]_2\) code for some \(k_i' \ge n_i - m_i\). Since \(m_i/n_i {\lt} 1-\rho \), the rate satisfies \(k_i'/n_i \ge (n_i - m_i)/n_i {\gt} \rho {\gt} 0\), bounded away from zero. Since \(\alpha {\gt} c/2\), the hypotheses of Theorem 236 apply to each \(G_i\) (with \(\varepsilon = 1-\alpha /c {\lt} 1/2\)), giving relative distance at least \(\gamma \), bounded away from zero. Hence \(\{ C(G_i)\} _{i\in \mathbb {Z}_+}\) is asymptotically good.
11.3.3 An Improved Bound on Distance
We now show that \(C(G)\) has almost twice the distance argued in Theorem 236 (as \(\varepsilon \to 0\)).
Let \(G\) be an \((n,n-k,c,\gamma ,c(1-\varepsilon ))\) bipartite expander with \(\varepsilon {\lt} \frac12\). Then \(C(G)\) has distance at least \(2\gamma (1-\varepsilon )n\).
As in the proof of Theorem 236, for the sake of contradiction let us assume \(C(G)\) has distance \({\lt} 2\gamma (1-\varepsilon )n\). Then by linearity there exists a non-zero codeword \(\mathbf{c} \in C(G)\) with \(wt(\mathbf{c}) {\lt} 2\gamma (1-\varepsilon )n\). Let \(S\) be the set of non-zero coordinates of \(\mathbf{c}\); we will argue \(U(S) \ne \emptyset \), and then the rest of the argument proceeds exactly as in the proof of Theorem 236 to derive a contradiction.
If \(|S| \le \gamma n\), then we can just use the argument from the proof of Theorem 236. So let us assume there exists a subset \(T \subset S\) such that \(|T| = \gamma n\). Then, by Lemma 234,
Now, since the total number of edges emanating out of \(S\setminus T\) is at most \(c|S\setminus T|\), we have
where the last inequality follows from the facts that \(|S| {\lt} 2\gamma (1-\varepsilon )n\) and \(|T| = \gamma n\) (so \(|S \setminus T| {\lt} 2\gamma (1-\varepsilon )n - \gamma n\), and since \(\varepsilon {\lt} 1/2\) this quantity is at most \(2\gamma (1-2\varepsilon )n\), and more crudely bounded by \(2\gamma (1-\varepsilon )n\); the displayed chain reproduces the bound as stated in the book).
Now, note that
where the last inequality follows from combining the two displayed bounds above (both quantities are, up to the stated constants, \(c\gamma n\) times a factor depending on \(\varepsilon \), and the first dominates as \(\varepsilon \to 0\); this is exactly the computation carried out in the book). Hence \(U(S) \ne \emptyset \), and the argument concludes exactly as in the proof of Theorem 236.
11.4 Codes from Weaker Expanders
The codes of the previous section required bipartite expander graphs that are \(c\)-left-regular with expansion factor \(\alpha = c(1-\varepsilon )\) for some \(\varepsilon {\lt} 1/2\), i.e., expansion greater than half the degree. While constructions of bipartite graphs with such strong expansion are known (Theorem 219), there are few such constructions and they are complicated. In this section we build codes based on graphs with weaker expansion requirements.
11.4.1 Tanner codes
One view of expander codes is that they ‘combine’ a bipartite expander with small error-correcting codes: each right vertex is associated with a small binary code (the parity check code of block length equal to its degree), and a word is a labelling of the left vertices satisfying this property for every right vertex. In this section we extend this notion to allow other linear error-correcting codes instead of just the parity check code. To make sense of this we switch to ‘ordered’ graphs, where the neighborhood of every right vertex is an ordered tuple of left vertices. For a \(d\)-right regular bipartite graph we extend the definition of the neighborhood \(N(u)\) of a right vertex \(u\) to be an ordered sequence of vertices of \(L\), i.e., \(N(u) = (\ell _1,\ldots ,\ell _d)\) where \(\ell _1,\ldots ,\ell _d \in L\) are the vertices adjacent to \(u\). Given a \(d\)-right regular bipartite graph \(G=(L,R,E)\) with \(L=[n]\), a vertex \(u \in R\) and a vector \(\mathbf{c} = (c_1,\ldots ,c_n) \in \mathbb {F}_2^n\), we let \(\mathbf{c}_{N(u)} \in \mathbb {F}_2^d\) be the vector given by \((\mathbf{c}_{N(u)})_i = c_{\ell _i}\) where \(N(u) = (\ell _1,\ldots ,\ell _d)\). We refer to \(\mathbf{c}_{N(u)}\) as the vertex \(u\)’s ‘view’ of the word \(\mathbf{c}\).
Given a \(d\)-right regular bipartite graph \(G=(L,R,E)\) with \(L=[n]\) and \(|R|=m\) and a binary linear code \(C_0 \subseteq \mathbb {F}_2^d\), the Tanner code \(X(G,C_0)\) is defined as the set
Tanner codes are a generalization of expander codes: in the expander code of Section 11.3, \(C_0\) was chosen to be the \([d,d-1,2]_2\) parity check code. In our main construction we will also work with a Tanner code on a graph \(G\) that is the edge-vertex incidence graph (Definition 229) of another graph \(H\); we use \(T(H,C_0)\) to denote this two-step construction, i.e., \(T(H,C_0) = X(G,C_0)\) where \(G\) is the edge-vertex incidence graph of \(H\). The following lemma shows that if \(C_0\) has large rate, then so does \(X(G,C_0)\).
The dimension of \(X(G,C_0)\) is at least \(n - m(d - \dim (C_0))\).
For each \(u \in R\), the condition \(\mathbf{c}_{N(u)} \in C_0\) imposes \(d - \dim (C_0)\) independent linear constraints on the bits of \(\mathbf{c}\) (namely, the parity checks of \(C_0\) applied to \(\mathbf{c}_{N(u)}\)). Therefore, the condition that \(\mathbf{c}_{N(u)} \in C_0\) for all \(u \in R\) imposes a total of at most \(m(d - \dim (C_0))\) linear constraints on the coordinates of \(\mathbf{c}\), leaving \(X(G,C_0)\) as a space of dimension at least \(n - m(d-\dim (C_0))\). (Some of the \(m(d-\dim (C_0))\) constraints may be linearly dependent, but that only increases the dimension.)
Let \(G\) be a \(d\)-right regular \((n,m,c,\gamma ,\alpha )\) bipartite expander. Let \(C_0\) be a \([d,\ell ,\Delta ]_2\) code. Then \(X(G,C_0)\) has distance strictly greater than \(\gamma n\) provided \(\alpha {\gt} c/\Delta \).
We mimic the proof of Theorem 236 with modifications to exploit the fact that \(C_0\) has minimum distance \(\Delta \). Let the left vertices of \(G\) be \([n]\) and the right vertices be \([m]\). Let \(\mathbf{c} = (c_1,\ldots ,c_n) \in X(G,C_0)\) be a non-zero codeword and let \(S \subseteq [n]\) denote the set of non-zero coordinates of \(\mathbf{c}\), i.e., \(S = \{ i \in [n] \mid c_i \ne 0\} \). Now let \(W = N(S)\) be the neighbors of \(S\) in \(G\).
We first note that every vertex \(v \in W\) must have at least \(\Delta \) neighbors in \(S\). To see this, note that \(\mathbf{c}_{N(v)}\) is a codeword of \(C_0\) (by definition of the Tanner code \(X(G,C_0)\)), and furthermore it is a non-zero codeword since \(v\) has a neighbor in \(S\). Thus this codeword must have weight at least \(\Delta \) (by the distance of \(C_0\)), and since each non-zero coordinate of \(\mathbf{c}_{N(v)}\) corresponds to a distinct vertex of \(S\), we conclude that \(v\) has at least \(\Delta \) neighbors in \(S\).
Let \(s = |S|\) and \(w = |W|\). From the above we have that the number of edges incident to \(S\) equals \(cs \ge \Delta w\). If \(|S| \le \gamma n\), then by the expansion of \(G\) we would also have \(w \ge \alpha s\). Combining the two we would get \(cs \ge \alpha \Delta s\), which contradicts the assumption that \(\alpha {\gt} c/\Delta \). We thus conclude that \(|S| {\gt} \gamma n\), which in turn implies that \(\mathbf{c}\) has weight strictly greater than \(\gamma n\). Since this is true for every non-zero codeword of \(X(G,C_0)\), we conclude that \(X(G,C_0)\) has distance greater than \(\gamma n\).
The following theorem combines Theorem 241 with Theorem 230 to show that spectral expanders suffice to get Tanner codes of constant relative distance.
Let \(C_0 \subseteq \mathbb {F}_2^d\) have distance at least \(\delta _0 d\) and rate \(\rho {\gt} 1/2\). Let \(G\) be an \((n,d,\lambda )\)-spectral expander and let \(G'\) be the double cover of \(G\) and \(H\) be the edge-vertex incidence graph of \(G'\). Then for large enough degree \(d\), \(X(H,C_0)\) has rate at least \(2\rho - 1\) and relative distance at least \(\delta _0\left(\delta _0 - \frac{\lambda }{d}\right)\).
Consider any \(\beta \) with \(\lambda \le \beta \le \delta _0 d\), and apply Theorem 230. We get that \(H\) is a \(\left(dn,2n,2,\frac{\beta (\beta -\lambda )}{d^2},\frac{2}{\beta }\right)\)-bipartite expander. Thus the dimension of the code \(X(H,C_0)\) is, by Lemma 240, at least \(dn - 2n(d-d\rho ) = (2\rho -1)dn\). Thus its rate is at least \(2\rho -1\).
Now we turn to the distance of \(X(H,C_0)\). Here, our choice of \(\beta \) ensures the expansion factor \(\alpha = 2/\beta {\gt} c/(\delta _0 d) = 2/(\delta _0 d)\) (using \(\beta {\lt} \delta _0 d\)), and so we can apply Theorem 241 to conclude that \(X(H,C_0)\) has relative distance strictly greater than \(\gamma = \frac{\beta (\beta -\lambda )}{d^2}\). Taking limits as \(\beta \to \delta _0 d\) yields the bound \(\delta _0(\delta _0-\lambda /d)\) on the distance and thus the theorem.
We are now ready to give an explicit family of asymptotically good binary Tanner codes.
For every \(\delta \in [0,1]\) and \(\varepsilon {\gt} 0\) there exists an explicit family of binary linear (Tanner) codes of rate at least \(1-2h(\sqrt\delta )\) and relative distance at least \(\delta - \varepsilon \). Consequently, for every \(\tau {\gt} 0\) there exists an explicitly constructible family of asymptotically good Tanner codes of rate \(R \ge 1-\tau \) and distance \(\delta = \Omega (\tau ^2/\log ^2(1/\tau ))\).
This follows from Theorem 242, from the explicit constructibility of spectral expanders (Theorem 227) and the existence of (constant-sized) asymptotically good codes on the Gilbert-Varshamov bound, with an appropriate setting of parameters, as follows. Given \(\delta \) and \(\varepsilon \), set \(\delta _0 = \sqrt\delta \) and choose \(d\) and \(\lambda \) such that \(\lambda /d \le \varepsilon /\delta _0\), where an \((n,d,\lambda )\)-spectral expander \(G\) exists (by Theorem 227, such a graph exists for sufficiently large \(d\) and infinitely many \(n\)). Let \(C_0\) be a code on the Gilbert-Varshamov bound with block length \(d\), relative distance \(\delta _0\) and rate \(1-h(\delta _0)\). Applying Theorem 242 we get that the Tanner code \(X(H,C_0)\), where \(H\) is the edge-vertex incidence graph of the double cover of \(G\), is a code of rate \(1-2h(\delta _0) = 1-2h(\sqrt\delta )\), and distance \(\delta _0^2 - \delta _0\lambda /d \ge \delta - \varepsilon \).
For the consequent part, we set \(\delta \) so that \(2h(\sqrt\delta ) = \tau \) and set \(\varepsilon = \delta /2\) and apply the previous part. This gives a code \(X(H,C_0)\) of rate \(1-\tau \) and distance \(\delta /2\). Note that the condition \(2h(\sqrt\delta ) = \tau \) is satisfied by setting \(\delta = h^{-1}(\tau /2)^2\). Using the bound \(h^{-1}(p) \ge p/\log p\) we get \(\delta = \Omega (\tau ^2/\log ^2(1/\tau ))\), and so the resulting code \(X(H,C_0)\) also has distance \(\Omega (\tau ^2/\log ^2(1/\tau ))\). We conclude the resulting family of codes is a polynomially constructible asymptotically good family of Tanner codes.
11.5 Distance Amplification
We now describe new constructions of codes that also use expanders, but in contrast to the previous sections, where the graphs were used to design the parity check matrix, in this section the graphs will be used to design the generator matrix. Specifically, we show how to use expanding graphs to convert codes of low (but constant) relative distance into new codes with large minimum distance.
11.5.1 Codes in the low-rate regime
We begin with a construction that amplifies the distance of a code at the expense of a larger alphabet size (and a corresponding loss in rate).
Let \(G = (L,R,E)\) be a \(c\)-left-regular and \(d\)-right-regular ordered bipartite graph with \(L=[n]\), \(R=[m]\), and let \((N_1(j),\ldots ,N_d(j)) \in [n]^d\) denote the ordered neighborhood of \(j \in [m]\). Let \(\Sigma = \{ 0,1\} ^d\). For \(\mathbf{w} \in \{ 0,1\} ^n\), define \(G(\mathbf{w}) \in \Sigma ^m\) by
Let \(C\) be a binary linear code of block length \(n = |L|\). Now define the code \(G(C)\) as
We refer to \(G(C)\) as the \(G\)-amplification of \(C\).
The codewords of \(G(C)\) are in one-to-one correspondence with codewords of \(C\): each position \(j\) of a codeword of \(G(C)\) collects the bits of the corresponding codeword of \(C\) situated in the positions adjacent to \(j\) in \(G\). The alphabet of \(G(C)\) is \(\Sigma = \{ 0,1\} ^d\), identified with \(\mathbb {F}_{2^d}\); \(G(C)\) is not necessarily linear over \(\mathbb {F}_{2^d}\), but it is linear over \(\mathbb {F}_2\), and the sum of two codewords in \(G(C)\) also belongs to \(G(C)\). Since each bit of a codeword \(\mathbf{c} \in C\) is repeated \(c\) times in the associated codeword \(G(\mathbf{c}) \in G(C)\), we have the following.
\(R(G(C)) = \frac{1}{c}\cdot R(C)\).
Let \(k = \dim (C)\) and \(n=|L|\), so \(R(C) = k/n\). The map \(\mathbf{w} \mapsto G(\mathbf{w})\) is injective (every left vertex has at least one incident edge, since \(G\) is \(c\)-left-regular with \(c \ge 1\), so the bits of \(\mathbf{w}\) can be recovered from \(G(\mathbf{w})\)), so \(G(C)\), viewed as a code over the alphabet \(\Sigma = \mathbb {F}_2^d\) of size \(2^d\), has \(\log _{2^d}|G(C)| = \log _{2^d}|C| = k/d\) symbols of message per codeword, over a block length of \(m = |R|\) symbols. Since \(G\) is \(c\)-left-regular and \(d\)-right-regular, double counting the edges of \(G\) gives \(cn = dm\). Hence
So the \(G\)-amplification of \(C\) only has worse rate than \(C\); to make this construction interesting we need graphs guaranteeing that the distance of \(G(C)\) is better than that of \(C\). We call such graphs dispersers.
A bipartite graph \(G = (L,R,E)\) is said to be a \((\gamma ,\varepsilon )\)-disperser if for all subsets \(S \subseteq L\) with \(|S| \ge \gamma n\), we have \(|N(S)| \ge (1-\varepsilon )m\).
In contrast to a bipartite expander, which maintains \(|N(S)|/|S| \ge \alpha \) for every small enough set \(S\), a disperser puts an absolute lower bound on \(|N(S)|\) for every large enough set \(S\).
Let \(G=(L,R,E)\) be an \((n,m,c,\gamma ,\alpha )\)-bipartite-expander with \(\alpha \cdot \gamma \ge (1-\varepsilon )m/n\). Then \(G\) is a \((\gamma ,\varepsilon )\)-disperser.
Let \(G\) be an \((n,d,\lambda )\)-spectral expander. Then the double cover \(H\) of \(G\) is a \(d\)-regular \((\gamma ,\varepsilon )\)-disperser for \(\varepsilon = \frac{\lambda ^2}{\gamma d^2}\).
(1) Let \(S \subseteq L\) have size \(|S| \ge \gamma n\). Let \(T \subseteq S\) be any set of size \(|T| = \gamma n\). By the expansion condition we have \(|N(T)| \ge \alpha |T| = \alpha \gamma n\). Since \(N(S) \supseteq N(T)\), it follows that \(|N(S)| \ge \alpha \gamma n \ge (1-\varepsilon )m\). Thus \(G\) satisfies the definition of a \((\gamma ,\varepsilon )\)-disperser.
(2) For this we need the expander mixing lemma (Lemma 226). Let \(H = (L,R,E)\) be the double cover of \(G\) where \(G\) is an \((n,d,\lambda )\)-spectral expander. We need to prove that for each \(S \subseteq L\) with \(|S| \ge \gamma n\), we have \(N(S) \ge (1-\varepsilon )n\). As in the proof of Part (1) it suffices to consider the case of \(|S| = \gamma n\). Fix \(S\), let \(T = R\setminus N(S)\), and it suffices to prove that \(|T| \le \varepsilon n\). By the Expander Mixing Lemma (in particular the second displayed inequality of Lemma 226) and the fact that \(E(S,T) = \emptyset \) by definition of \(T\), we have
as desired.
For every \(\varepsilon ,\gamma {\gt} 0\) and for every large enough \(d\), there is an explicit (poly-time constructible) family of \(d\)-regular \((\gamma ,\varepsilon )\)-dispersers on \(n\) left and right vertices, for every sufficiently large \(n\). Furthermore one can take \(d = \Theta \left(\frac{1}{\gamma \varepsilon }\right)\).
Recall that Theorem 227 gives that for every \(\varepsilon ' {\gt} 0\), for every large enough \(d\) there exists an infinite family of explicitly constructible \((n,d,\lambda )\)-spectral expanders for \(\lambda = \varepsilon ' d\). We apply this theorem with \(\varepsilon ' = \sqrt{\varepsilon \gamma }\). Part (2) of Lemma 247 then implies that the double covers of the resulting family of graphs are an explicit (polynomial time constructible) family of \((\gamma ,\varepsilon )\)-dispersers (since \(\varepsilon = (\lambda /d)^2/\gamma = (\varepsilon ')^2/\gamma = \varepsilon \gamma /\gamma = \varepsilon \)). The furthermore part follows from the furthermore part of Theorem 227 (which gives \(d = O(1/(\varepsilon ')^2) = O(1/(\varepsilon \gamma ))\)).
We now return to analyzing the distance of \(G\)-amplified codes, using the dispersal properties of \(G\).
If \(G\) is a \((\gamma ,\varepsilon )\)-disperser and \(\Delta (C) \ge \gamma n\), then \(\Delta (G(C)) \ge (1-\varepsilon )m\), where \(n=|L|\) and \(m=|R|\).
Suppose \(\mathbf{c}\) and \(\mathbf{c}'\) are distinct codewords in \(C\), and let \(G(\mathbf{c})\) and \(G(\mathbf{c}')\) be the corresponding codewords in \(G(C)\). Let \(A \subseteq L\) be the positions where \(\mathbf{c}\) and \(\mathbf{c}'\) differ; since \(\Delta (C) \ge \gamma n\) we have \(|A| \ge \gamma n\). We have that \(G(\mathbf{c})\) and \(G(\mathbf{c}')\) differ in their \(j\)’th location whenever \(j \in N(A)\) (since position \(j\) reads off the value at some vertex of \(A\), which differs between \(\mathbf{c}\) and \(\mathbf{c}'\)). Since \(|N(A)| \ge (1-\varepsilon )m\) by the disperser property, the Hamming distance between \(G(\mathbf{c})\) and \(G(\mathbf{c}')\) is at least \((1-\varepsilon )m\). We thus conclude \(\Delta (G(C)) \ge (1-\varepsilon )m\).
If we instantiate the construction using an explicit binary code \(C\) of rate \(1/2\) and relative distance, say, \(\delta _0 = .001\) (which can be constructed via cross-chapter concatenation methods), and amplifying with a \((\delta _0,\varepsilon )\)-disperser \(d\)-regular \(G\), we get that \(G(C)\) has rate \(\frac1d\) (by Lemma 245), alphabet size \(2^d\), and relative distance \(1-\varepsilon \) (by Lemma 249). Using Lemma 248 we can take \(d = O\left(\frac{1}{\delta _0\varepsilon }\right) = O\left(\frac1\varepsilon \right)\) in this construction, leading to the following corollary.
There are explicit codes of relative distance \((1-\varepsilon )\) and rate \(\Omega (\varepsilon )\), over an alphabet of size \(2^{O(1/\varepsilon )}\).
Take \(C\) to be an explicit binary code of rate \(1/2\) and relative distance \(\delta _0 = .001\) (as furnished by concatenated codes on the Zyablov bound, Theorem 206). Take \(G\) to be a \(d\)-regular \((\delta _0,\varepsilon )\)-disperser furnished by Lemma 248 with \(d = O(1/(\delta _0\varepsilon )) = O(1/\varepsilon )\). By Lemma 245, \(G(C)\) has rate \(1/d = \Omega (\varepsilon )\). By Lemma 249 (applicable since \(\Delta (C) \ge \delta _0 n\)), \(G(C)\) has relative distance \(1-\varepsilon \). The alphabet size of \(G(C)\) is \(2^d = 2^{O(1/\varepsilon )}\).
11.5.2 Codes almost matching the Singleton bound
Section 11.5 (Part 1, above) gave codes in the large distance regime with rates optimal within a constant factor as \(\varepsilon \to 0\), but the maximum rates attainable were bounded well away from \(1\). We now remedy this by adding a new ingredient to the amplification technique: another small error-correcting code.
To motivate the new construction, note that the \(G\)-amplification of a code \(C\) can be viewed locally as follows: every left vertex of \(G\) of degree \(c\) sends a bit on each edge it is incident to; every right vertex collects all the bits it receives on these edges to form a string in \(\mathbb {F}_2^d\). From a left vertex’s perspective, we could think of the vertex as having a \(c\)-bit string (all \(0\)s or all \(1\)s) that it sends one bit at a time on each of its incident edges; this string is a codeword of the \(c\)-bit repetition code \(C_{\text{rep}}\) of rate \(1/c\) and distance \(1\). We now ask what happens if the \(c\)-bit strings assigned to the left vertices were instead codewords of some other code \(C_{\text{in}} \subseteq \mathbb {F}_q^c\).
Given an ordered \(c\)-left regular \(d\)-right regular graph \(G=(L,R,E)\) with \(L=[n]\) and \(R=[m]\), the \(G\)-mixing operator is a function \(G_{\mathrm{mix}} : \Gamma ^n \to \Sigma ^m\) where \(\Gamma = \mathbb {F}_q^c\) and \(\Sigma = \mathbb {F}_q^d\) given as follows: for \(\mathbf{u} = (u_1,\ldots ,u_n) \in \Gamma ^n\) with \(u_i = (u_i(1),\ldots ,u_i(c))\), we have \(G_{\mathrm{mix}}(\mathbf{u}) = \mathbf{v} = (v_1,\ldots ,v_m) \in \Sigma ^m\) where \(v_j = (v_j(1),\ldots ,v_j(d))\) is given by \(v_j(b) = u_i(a)\) if \((i,j) \in E\) and \(j\) is the \(a\)th neighbor of \(i\) and \(i\) is the \(b\)th neighbor of \(j\). That is, the \(a\)th symbol of \(u_i\) is ‘sent’ to the \(b\)th symbol of \(v_j\) via the edge \((i,j)\).
For integers \(k,n,b\) and \(B\), codes \(C_{\text{out}} : \mathbb {F}_q^k \to (\mathbb {F}_q^b)^n\) and \(C_{\text{in}} : \mathbb {F}_q^b \to \mathbb {F}_q^B\), and a \(B\)-regular ordered bipartite graph \(G = (L,R,E)\) with \(|L|=|R|=n\), the \(C_{\text{in}}\)-coded-\(G\)-amplification of \(C_{\text{out}}\) is the code \(\mathscr {CA}(C_{\text{out}},C_{\text{in}},G) : \mathbb {F}_q^k \to (\mathbb {F}_q^B)^n\) given by \(\mathscr {CA}(C_{\text{out}},C_{\text{in}},G)(\mathbf{u}) = G_{\mathrm{mix}}\big((C_{\text{out}}\circ C_{\text{in}})(\mathbf{u})\big)\), where \(C_{\text{out}}\circ C_{\text{in}} : \mathbb {F}_q^k \to (\mathbb {F}_q^B)^n\) denotes the encoding of the concatenation of the codes \(C_{\text{out}}\) and \(C_{\text{in}}\). Specifically, for \(\mathbf{u} \in \mathbb {F}_q^k\) we have \((C_{\text{out}}\circ C_{\text{in}})(\mathbf{u}) = (w_1,\ldots ,w_n)\) where \(w_i = C_{\text{in}}(v_i)\) and \((v_1,\ldots ,v_n) = C_{\text{out}}(\mathbf{u})\).
We note that \(G(C) = \mathscr {CA}(C,C_{\text{rep}},G)\), i.e., the \(G\)-amplification of a code \(C\) is just the \(C_{\text{rep}}\)-coded \(G\)-amplification of \(C\), where \(C_{\text{rep}}\) is the repetition code that takes a single symbol message and repeats it. For general \(C_{\text{in}}\), the rate of the coded-amplification code is exactly the same as the rate of the underlying concatenated code, which in turn is the product of the rates of \(C_{\text{out}}\) and \(C_{\text{in}}\), since the final step, namely \(G_{\mathrm{mix}}\), only permutes the \(q\)-ary symbols of the encoding under \(C_{\text{out}}\circ C_{\text{in}}\) and does not add or delete any symbols. The advantage of coded-amplification is in the distance: the code ends up having extremely strong distance when \(G\) has nice expansion properties, close to the property given by the expander mixing lemma on spectral expanders.
Let \(q=2^s\) for integer \(s \ge 1\). Let \(C_{\text{out}}:\mathbb {F}_q^k \to (\mathbb {F}_q^b)^n\) and \(C_{\text{in}}:\mathbb {F}_q^b \to \mathbb {F}_q^B\) be \(\mathbb {F}_2\)-linear codes of relative distance \(\delta _{\text{out}}\) and \(\delta _{\text{in}}\) respectively. Let \(G\) be the double cover of an \((n,B,\lambda )\)-expander. Then if \(\lambda \le \varepsilon \sqrt{\delta _{\text{out}}}B\) then \(\delta (\mathscr {CA}(C_{\text{out}},C_{\text{in}},G)) \ge \delta _{\text{in}}-\varepsilon \).
We first note that since \(C_{\text{out}}\) and \(C_{\text{in}}\) are \(\mathbb {F}_2\)-linear, the \(C_{\text{in}}\)-coded \(G\)-amplification of \(C_{\text{out}}\) is \(\mathbb {F}_2\)-linear. So to analyze the distance it suffices to prove that the Hamming weight of the encoding of a non-zero vector \(\mathbf{u} \in \mathbb {F}_q^k\) has large weight. Fix such a non-zero \(\mathbf{u}\). Let \(\mathbf{v} = C_{\text{out}}(\mathbf{u})\) and let \(\mathbf{w} = (C_{\text{out}}\circ C_{\text{in}})(\mathbf{u})\) and \(\mathbf{x} = G_{\mathrm{mix}}(\mathbf{w})\) so that \(\mathbf{x} = \mathscr {CA}(C_{\text{out}},C_{\text{in}},G)(\mathbf{u})\). Let \(S = \{ i \in [n] \mid v_i \ne 0\} \) denote the subset of coordinates where \(\mathbf{v}\) (equivalently \(\mathbf{w}\)) is non-zero. Note \(|S| \ge \delta _{\text{out}}n\). Now consider \(T = \{ j \in [n] \mid \mathbf{x}_j \ne 0\} \); our goal is to prove \(|T| \ge (\delta _{\text{in}}-\varepsilon )n\).
We view \(S\) and \(T\) as subsets of the left and right vertices, respectively, of \(G\). The key observation is that every vertex \(i \in S\) has many edges going into \(T\): the mixing process associates the label \(w_i\) with the vertex \(i\) and spreads the symbols of \(w_i\) out over the edges incident to \(i\). On the one hand, for \(i \in S\), \(w_i\) is a non-zero codeword of \(C_{\text{in}}\) and so has at most \((1-\delta _{\text{in}})B\) coordinates \(a \in [B]\) such that \(w_i(a) = 0\). On the other hand, every symbol transmitted to vertices outside \(T\) is zero (by definition of \(T\)), and so if an edge goes from \(S\) to \(T^c\) it must transmit the zero symbol. It follows that every vertex \(i \in S\) has at most \((1-\delta _{\text{in}})B\) neighbors outside \(T\), and at least \(\delta _{\text{in}}B\) neighbors in \(T\). We thus get \(|E(S,T)| \ge \delta _{\text{in}}B|S|\). But by the expander mixing lemma (Lemma 226) we also have
Putting the two together,
where the second inequality uses \(T \subseteq [n]\) to get \(|T|\le n\), and the third inequality uses \(|S| \ge \delta _{\text{out}}n\) (or equivalently \(n \le |S|/\delta _{\text{out}}\)). Rearranging terms we get
where the second inequality uses \(\lambda \le \varepsilon \sqrt{\delta _{\text{out}}}B\). We conclude every non-zero codeword of \(\mathscr {CA}(C_{\text{out}},C_{\text{in}},G)\) has weight at least \((\delta _{\text{in}}-\varepsilon )n\) and thus \(\delta (\mathscr {CA}(C_{\text{out}},C_{\text{in}},G)) \ge \delta _{\text{in}}-\varepsilon \).
We are now ready to show how to instantiate \(C_{\text{out}}\), \(C_{\text{in}}\) and the underlying graph \(G\) to construct codes that achieve a rate vs. relative distance trade-off almost matching the Singleton bound over a constant-sized alphabet.
For every \(r\), \(0 {\lt} r {\lt} 1\), and \(\varepsilon {\gt} 0\), there exists \(q=2^s\), \(b\), \(B\) and an \(\mathbb {F}_2\)-linear code \(C_{\text{in}} : \mathbb {F}_q^b \to \mathbb {F}_q^B\), alphabet \(\Sigma = \mathbb {F}_q^B\) of size bounded by \(\exp \! \left(O\! \left(\varepsilon ^{-4}\log ^3(1/\varepsilon )\right)\right)\), such that there exists an infinite family of \(C_{\text{in}}\)-coded amplified codes over \(\Sigma \) of rate \(r\) and relative distance at least \((1-r-\varepsilon )\). Further the codes are \(\mathbb {F}_2\)-linear.
We choose parameters so that the code \(\mathscr {CA}(C_{\text{out}},C_{\text{in}},G)\) has rate \(r\) and relative distance \(1-r-\varepsilon \), with alphabet a power of two of size \(\exp \! \left(O(\varepsilon ^{-4}\log ^3(1/\varepsilon ))\right)\).
Fix the target rate \(r\) and proximity parameter \(\varepsilon {\gt} 0\). Let \(r_{\text{out}} = 1-\varepsilon /2\). Let \(\delta _{\text{out}}\) be chosen so that there exists an infinite family of explicitly constructible binary linear codes of rate \(r_{\text{out}}\) and relative distance \(\delta _{\text{out}}\); we may use the code given by Corollary 243 to achieve \(\delta _{\text{out}} = \Omega (\varepsilon ^2/\log ^2(1/\varepsilon ))\). Next we choose \(B\) and \(\lambda \) so that \(\lambda \le \frac\varepsilon 2\cdot \sqrt{\delta _{\text{out}}}B\) and an infinite family of explicitly constructible \((n,B,\lambda )\)-spectral expanders exists; by Theorem 227 it suffices to choose \(B = \Theta \left(\frac{1}{\delta _{\text{out}}\varepsilon ^2}\right) = \Theta (\varepsilon ^{-4}\log ^2(1/\varepsilon ))\). In particular we choose \(B=2^s\) for an integer \(s\). Let \(q=B\) and let \(r_{\text{in}} = r/r_{\text{out}}\). Let \(b = r_{\text{in}}\cdot B\) and let \(C_{\text{in}}\) be a Reed-Solomon code (Definition 128) over \(\mathbb {F}_q\) of rate \(r_{\text{in}}\) and relative distance \(\delta _{\text{in}} = 1-r_{\text{in}}\).
Given a sufficiently large \(k\) we let \(T:\mathbb {F}_2^{ks}\to \mathbb {F}_2^{ns}\) be an explicitly constructed linear code of rate \(r_{\text{out}}\) and distance \(\delta _{\text{out}}\). Note that by collecting together the symbols of the message as well as encodings into blocks of \(s\) symbols and using a linear map from \(\mathbb {F}_2^s\) to \(\mathbb {F}_q\), we can view \(T\) as a map \(C_{\text{out}} : \mathbb {F}_q^k \to \mathbb {F}_q^n\). This preserves the rate and distance, so \(C_{\text{out}}\) is an \(\mathbb {F}_2\)-linear code of rate \(r_{\text{out}}\) and distance \(\delta _{\text{out}}\). Let \(G\) be the double cover of an \((n,B,\lambda )\)-spectral expander. Let \(C^* = \mathscr {CA}(C_{\text{out}},C_{\text{in}},G)\). We claim \(C^*\) is the desired code for the message space \(\mathbb {F}_2^{ks}\).
By construction \(C^*\) is from an explicitly constructible family. As observed in the proof of Lemma 253, the code is \(\mathbb {F}_2\)-linear. Its rate is equal to the rate of \(C_{\text{out}}\circ C_{\text{in}}\), which in turn equals \(r_{\text{out}}\cdot r_{\text{in}} = r\) (by our choice of \(r_{\text{in}}\)). By Lemma 253 we have that the relative distance of \(C^*\) is at least \(\delta _{\text{in}}-\varepsilon /2\) (our choice of \(\lambda \) ensures \(\lambda \le \frac{\varepsilon }{2}\sqrt{\delta _{\text{out}}}B\)). Since
we get that the relative distance of \(C^*\) is at least \(\delta _{\text{in}}-\varepsilon /2 \ge 1-r-\varepsilon \). Finally the alphabet size is \(q^B = B^B = \exp (B\log B) = \exp \left(O\! \left(\varepsilon ^{-4}\log ^3\! \left(\frac1\varepsilon \right)\right)\right)\). This concludes the proof of the theorem.
The near-optimal trade-off (rate \(r\) for distance close to \((1-r)\)), which almost matches the optimal Singleton bound (Theorem 110), comes from using Reed-Solomon codes for \(C_{\text{in}}\): the role of the expander is to ‘spread out’ the errors among the different copies of the Reed-Solomon code, so that most copies can be decoded and the remaining small number of errors corrected by the left code \(C_{\text{out}}\).
11.5.3 Binary codes approaching the Zyablov bound
The codes in Theorem 254 are defined over a large alphabet \(\Sigma \) (whose size depends exponentially on \(1/\varepsilon \)), but this alphabet size is constant for any fixed \(\varepsilon \), so we can find an inner code of dimension \(\log _2|\Sigma |\) that achieves the Gilbert-Varshamov bound in time depending only on \(\varepsilon \). Thus, if the codes in Theorem 254 are concatenated with constant-sized binary codes on the Gilbert-Varshamov bound, we get constructions of binary codes meeting the Zyablov bound.
There are explicit binary linear codes that lie within \(\varepsilon \) of the Zyablov bound and can be constructed in time polynomial in the block length and exponential in \(\mathrm{poly}(1/\varepsilon )\).
Concatenate the outer code from Theorem 254 (of rate \(r\) and relative distance \(1-r-\varepsilon \) over an alphabet of size \(2^{O(1/\varepsilon )}\)) with an inner binary code of dimension \(\log _2|\Sigma | = O(1/\varepsilon )\) lying on the Gilbert-Varshamov bound (Theorem 105), constructed by brute-force search over a time depending only on \(\varepsilon \) (and hence, for fixed \(\varepsilon \), polynomial in the overall block length). By the distance property of concatenated codes (Definition 202), the resulting binary code has rate and relative distance within \(\varepsilon \) of the point \((r,1-r)\) traced out by the Zyablov bound (Theorem 206) for every choice of \(r \in (0,1)\), and is constructible in time polynomial in the block length and exponential in \(\mathrm{poly}(1/\varepsilon )\).
We note that this re-proves Theorem 206. The codes in Corollary 255 hold the promise of efficient (linear-time) decoding algorithms, unlike the codes of Theorem 206, obtained purely via concatenation.
11.6 Existence of Lossless Expanders
This section is devoted to the proof of Theorem 218, establishing the existence of bipartite expanders whose expansion factor \(\alpha \) can be made arbitrarily close to the degree \(c\) (‘lossless’ expansion). The proof, via the probabilistic method, is given together with the statement of Theorem 218 in Section 11.2 above.