Optimality

284 V. J. Ribeiro, R. H. Riedi and R. G. Baraniuk
Lemma 7.1. Given independent random variables A, W, F, define Z and E through
Z := ζA + W and E := ηZ + F where ζ, η are constants. We then have the result
(7.11)
var(A) cov(Z, E)2
cov(A, E) 2· var(Z) = ζ2 + var(W)/var(A)
ζ2 ≥ 1.
Proof. Without loss of generality assume all random variables have zero mean. We
have
(7.12) cov(E, Z) = E(ηZ 2 + FZ) = ηvar(Z),
(7.13) cov(A, E) = E((η(ζA + W) + F)A)ζηvar(A),
and
(7.14) var(Z) = E(ζ 2 A 2 + W 2 + 2ζAW) = ζ 2 var(A) + var(W).
Thus from (7.12), (7.13) and (7.14)
(7.15)
cov(Z, E) 2
var(Z)
var(A)
·
cov(A, E) 2 = η2var(Z) ζ2η2var(A) = ζ2 +var(W)/var(A)
ζ2 ≥1.
Lemma 7.2. Given a positive function zi, i∈Z and constant α > 0 such that
(7.16) ri :=
1
1−αzi
is positive, discrete-concave, and non-decreasing, we have that
(7.17) δi :=
1
1−βzi
is also positive, discrete-concave, and non-decreasing for all β with 0 < β≤ α.
Proof. Define κi := zi− zi−1. Since zi is positive and ri is positive and nondecreasing,
αzi < 1 and zi must increase with i, that is κi≥ 0. This combined with
the fact that βzi≤ αzi < 1 guarantees that δi must be positive and non-decreasing.
It only remains to prove the concavity of δi. From (7.16)
(7.18) ri+1− ri =
α(zi+1− zi)
(1−αzi+1)(1−αzi)
We are given that ri is discrete-concave, that is
(7.19)
0 ≥ (ri+2− ri+1)−(ri+1− ri)
� � �
1−αzi
= αriri+1 κi+2
1−αzi+2
Since ri > 0∀i, we must have
� � �
1−αzi
(7.20)
κi+2
1−αzi+2
Similar to (7.20) we have that
(7.21) (δi+2− δi+1)−(δi+1− δi) = βδiδi+1
− κi+1
�
= ακi+1ri+1ri.
− κi+1
�
≤ 0.
κi+2
�
.
� 1−βzi
1−βzi+2
�
− κi+1
�
.
�