Optimality

Optimal sampling strategies 273
Definition 3.1. A real function ψ defined on the set of integers{0,1, . . . , M} is
discrete-concave if
(3.1) ψ(x + 1)−ψ(x)≥ψ(x + 2)−ψ(x + 1), for x = 0, 1, . . . , M− 2.
The optimization problem we are faced with can be cast as follows. Given integers
P≥ 2, Mk > 0 (k = 1, . . . , P) and n≤ �P k=1 Mk consider the discrete space
�
(3.2) ∆n(M1, . . . , MP) := X = [xk] P P�
�
k=1 : xk = n;xk∈{0, 1, . . . , Mk},∀k .
Given non-decreasing, discrete-concave functions ψk (k = 1, . . . , P) with domains
{0, . . . , Mk} we are interested in
�
P�
�
(3.3) h(n) := max ψk(xk) : X∈ ∆n(M1, . . . , MP) .
k=1
In the context of optimal estimation on a tree, P will play the role of the number of
children that a parent node Vγ has, Mk the total number of leaf node descendants
of the k-th child Vγk, and ψk the reciprocal of the optimal LMMSE of estimating
Vγ given xk leaf nodes in the tree of Vγk. The quantity h(n) corresponds to the
reciprocal of the optimal LMMSE of estimating node Vγ given n leaf nodes in its
tree.
The following iterative procedure solves the optimization problem (3.3). Form
k=1
vectors G (n) = [g (n)
k ]P k=1 , n = 0, . . . ,�k
Mk as follows:
Step (i): Set g (0)
k = 0,∀k.
Step (ii): Set
(3.4) g (n+1)
k
where
(3.5) m∈arg max
k
�
ψk
=
�
g (n)
k
g (n)
k
+ 1, k = m
, k�= m
�
g (n)
� �
k + 1 − ψk
g (n)
k
�
: g (n)
k
< Mk
The procedure described in Steps (i) and (ii) is termed water-filling because it
resembles the solution to the problem of filling buckets with water to maximize the
sum of the heights of the water levels. These buckets are narrow at the bottom
and monotonically widen towards the top. Initially all buckets are empty (compare
Step (i)). At each step we are allowed to pour one unit of water into any one bucket
with the goal of maximizing the sum of water levels. Intuitively at any step we
must pour the water into that bucket which will give the maximum increase in
water level among all the buckets not yet full (compare Step (ii)). Variants of this
water-filling procedure appear as solutions to different information theoretic and
communication problems (Cover and Thomas [4]).
Lemma 3.1. The function h(n) is non-decreasing and discrete-concave. In addition,
(3.6) h(n) = � � �
,
where g (n)
k is defined through water-filling.
k
ψk
g (n)
k
�
.