Optimality

y the P× 1 vector
(3.1) ηn(p) =
Linear prediction after model selection 295
� θ[p] + (X[p] ′ X[p]) −1 X[p] ′ X[¬p]θ[¬p]
(0, . . . ,0) ′
with the conventions that ηn(0) = (0, . . . ,0) ′ ∈ R P and ηn(P) = θ. Furthermore, let
Φn,p(t), t∈R k , denote the cdf of √ nA( ˜ θ(p)−ηn(p)), i.e., Φn,p(t) is the cdf of a centered
Gaussian random vector with covariance matrix σ 2 A[p](X[p] ′ X[p]/n) −1 A[p] ′
in case p > 0, and the cdf of point-mass at zero in R k in case p = 0. If p > 0 and
if the matrix A[p] has rank k, then Φn,p(t) has a density with respect to Lebesgue
measure, and we shall denote this density by φn,p(t). We note that ηn(p) depends
on θ and that Φn,p(t) depends on σ (in case p > 0), although these dependencies
are not shown explicitly in the notation.
For p > 0, the conditional distribution of √ n ˜ θp(p) given √ nA( ˜ θ(p)−ηn(p)) = z
is a Gaussian distribution with mean √ nηn,p(p) + bn,pz and variance σ 2 ζ 2 n,p, where
(3.2) bn,p = C (p)′
n (A[p](X[p] ′ X[p]/n) −1 A[p] ′ ) − , and
(3.3) ζ 2 n,p = ξ 2 n,p− bn,pC (p)
n .
In the displays above, C (p)
n stands for A[p](X[p] ′ X[p]/n) −1ep, with ep denoting
the p-th standard basis vector in Rp , and (A[p](X[p] ′ X[p]/n) −1A[p] ′ ) − denotes a
generalized inverse of the matrix indicated (cf. Note 3(v) in Section 8a.2 of Rao [7]).
Note that, in general, the quantity bn,pz depends on the choice of generalized inverse
in (3.2); however, for z in the column-space of A[p], bn,pz is invariant under the
choice of inverse; cf. Lemma A.2 in Leeb [1]. Since √ nA( ˜ θ(p)−ηn(p)) lies in the
column-space of A[p], the conditional distribution of √ n˜ θp(p) given √ nA( ˜ θ(p)−
ηn(p)) = z is thus well-defined by the above. Observe that the vector of covariances
between A˜ θ(p) and ˜ θp(p) is given by σ2n−1C (p)
n . In particular, note that A˜ θ(p) and
˜θp(p) are uncorrelated if and only if ζ2 n,p = ξ2 n,p (or, equivalently, if and only if
bn,pz = 0 for all z in the column-space of A[p]); again, see Lemma A.2 in Leeb [1].
Finally, for M denoting a univariate Gaussian random variable with zero mean
and variance s2≥ 0, we abbreviate the probability P(|M− a| < b) by ∆s(a, b),
a∈R∪{−∞,∞}, b∈R. Note that ∆s(·,·) is symmetric around zero in its first
argument, and that ∆s(−∞, b) = ∆s(∞, b) = 0 holds. In case s = 0, M is to be
interpreted as being equal to zero, such that ∆0(a, b) equals one if|a| < b and zero
otherwise; i.e., ∆0(a, b) reduces to an indicator function.
3.1. The known-variance case
The cdf G∗ n,θ,σ (t) can be expanded as a weighted sum of conditional cdfs, condi-
tional on the outcome of the model selection step, where the weights are given by
the corresponding model selection probabilities. To this end, let G∗ n,θ,σ (t|p) denote
the conditional cdf of √ nA( ˜ θ∗− θ) given that ˆp ∗ equals p forO≤p≤P; that
is, G∗ n,θ,σ (t|p) = Pn,θ,σ( √ nA( ˜ θ∗− θ)≤t| ˆp ∗ = p), with t∈R k . Moreover, let
π∗ n,θ,σ (p) = Pn,θ,σ(ˆp ∗ = p) denote the corresponding model selection probability.
Then the unconditional cdf G∗ n,θ,σ (t) can be written as
(3.4) G ∗ n,θ,σ(t) =
P�
p=O
G ∗ n,θ,σ(t|p)π ∗ n,θ,σ(p).
�