Measure, Integral, and Conditional Expectation - Functional Analysis

Jan van NeervenMeasure, Integral, andConditional ExpectationHandoutMay 2, 2012

1Measure and IntegralIn this lecture we review elements of the Lebesgue theory of integration.1.1 σ-AlgebrasLet Ω be a set. A σ-algebra in Ω is a collection F of subsets of Ω with thefollowing properties:(1) Ω ∈ F ;(2) A ∈ F implies ∁A ∈ F ;(3) A 1 , A 2 , · · · ∈ F implies ⋃ ∞n=1 A n ∈ F .Here, ∁A = Ω \ A is the complement of A.A measurable space is a pair (Ω, F ), where Ω is a set and F is a σ-algebrain Ω.The above three properties express that F is non-empty, closed undertaking complements, and closed under taking countable unions. From∞⋂n=1( ⋃ ∞A n = ∁n=1∁A n)it follows that F is closed under taking countable intersections. Clearly, F isclosed under finite unions and intersections as well.Example 1.1. In every set Ω there is a minimal σ-algebra, {∅, Ω}, and a maximalσ-algebra, the set P(Ω) of all subsets of Ω.Example 1.2. If A is any collection of subsets of Ω, then there is a smallestσ-algebra, denoted σ(A ), containing A . This σ-algebra is obtained as the intersectionof all σ-algebras containing A and is called the σ-algebra generatedby A .

2 1 Measure and IntegralExample 1.3. Let (Ω 1 , F 1 ), (Ω 2 , F 2 ), . . . be a sequence of measurable spaces.The product∞∏ ∞∏( Ω n , F n )n=1is defined by ∏ ∞n=1 Ω n = Ω 1 ×Ω 2 ×. . . (the cartesian product) and ∏ ∞n=1 F n =F 1 × F 2 × . . . is defined as the σ-algebra generated by all sets of the formn=1A 1 × · · · × A N × Ω N+1 × Ω N+2 × . . .with N = 1, 2, . . . and A n ∈ F n for n = 1, . . . , N. Finite products∏ Nn=1 (Ω n, F n ) are defined similarly; here one takes the σ-algebra in Ω 1 ×· · · × Ω N generated by all sets of the form A 1 × · · · × A N with A n ∈ F n forn = 1, . . . , N.Example 1.4. The Borel σ-algebra of R d , notation B(R d ), is the σ-algebragenerated by all open sets in R d . The sets in this σ-algebra are called theBorel sets of R d . As an exercise, check that(R d , B(R d )) =d∏(R, B(R)).Hint: Every open set in R d is a countable union of rectangles of the form[a 1 , b 1 ) × · · · × [a d , b d ).Example 1.5. Let f : Ω → R be any function. For Borel sets B ∈ B(R) wedefine{f ∈ B} := {ω ∈ Ω : f(ω) ∈ B}.The collectionn=1σ(f) = { {f ∈ B} : B ∈ B(R) }is a σ-algebra in Ω, the σ-algebra generated by f.1.2 Measurable functionsLet (Ω 1 , F 1 ) and (Ω 2 , F 2 ) be measurable spaces. A function f : Ω 1 → Ω 2 issaid to be measurable if {f ∈ A} ∈ F 1 for all A ∈ F 2 . Clearly, compositionsof measurable functions are measurable.Proposition 1.6. If A 2 is a subset of F 2 with the property that σ(A 2 ) = F 2 ,then a function f : Ω 1 → Ω 2 is measurable if and only if{f ∈ A} ∈ F 1 for all A ∈ A 2 .Proof. Just notice that {A ∈ F 2 : {f ∈ A} ∈ F 1 } is a sub-σ-algebra of F 2containing A 2 .⊓⊔

1.3 Measures 3In most applications we are concerned with functions f from a measurablespace (Ω, F ) to (R, B(R)). Such functions are said to be Borel measurable. Inwhat follows we summarise some measurability properties of Borel measurablefunctions (the adjective ‘Borel’ will be omitted when no confusion is likely toarise).Proposition 1.6 implies that a function f : Ω → R is Borel measurableif and only if {f > a} ∈ F for all a ∈ R. From this it follows that linearcombinations, products, and quotients (if defined) of measurable functions aremeasurable. For example, if f and g are measurable, then f + g is measurablesince{f + g > a} = ⋃ q∈Q{f > q} ∩ {g > a − q}.This, in turn, implies that fg is measurable, as we havefg = 1 2 [(f + g)2 − (f 2 + g 2 )].If f = sup n1 f n pointwise and each f n is measurable, then f is measurablesince∞⋃{f > a} = {f n > a}.n=1It follows from inf n1 f n = − sup n1 (−f n ) that the pointwise infimum ofa sequence of measurable functions is measurable as well. From this we getthat the pointwise limes superior and limes inferior of measurable functionsis measurable, since( ) ( )lim sup f n = lim f k = inf f kn→∞ n→∞n1supknsupknand lim inf n→∞ f n = − lim sup n→∞ (−f n ). In particular, the pointwise limitlim n→∞ f n of a sequence of measurable functions is measurable.In these results, it is implicitly understood that the suprema and infimaexist and are finite pointwise. This restriction can be lifted by consideringfunctions f : Ω → [−∞, ∞]. Such a function is said to be measurable if thesets f ∈ B} as well as the sets {f = ±∞} are in F . This extension will alsobe useful later on.1.3 MeasuresLet (Ω, F ) be a measurable space. A measure is a mapping µ : F → [0, ∞]with the following properties:(1) µ(∅) = 0;(2) For all disjoint set A 1 , A 2 , . . . in F we have µ ( ⋃ ∞n=1 A n)=∑ ∞n=1 µ(A n).

4 1 Measure and IntegralA triple (Ω, F , µ), with µ a measure on a measurable space (Ω, F ), iscalled measure space. A measure space (Ω, F , P) is called finite if µ is a finitemeasure, that is, if µ(Ω) < ∞. If µ(Ω) = 1, then µ is called a probabilitymeasure and (Ω, F , µ) is called a probability space. In probability theory, it iscustomary to use the symbol P for a probability measure.The following properties of measures are easily checked:(a) if A 1 ⊆ A 2 in F , then µ(A 1 ) µ(A 2 );(b) if A 1 , A 2 , . . . in F , then( ⋃ ∞µn=1A n)∞∑µ(A n );n=1Hint: Consider the disjoint sets A n+1 \ (A 1 ∪ · · · ∪ A n ).(c) if A 1 ⊆ A 2 ⊆ . . . in F , then( ⋃ ∞µn=1A n)= limn→∞ µ(A n);Hint: Consider the disjoint sets A n+1 \ A n .(d) if A 1 ⊇ A 2 ⊇ . . . in F and µ(A 1 ) < ∞, thenHint: Take complements.( ⋂ ∞µn=1A n)= limn→∞ µ(A n).Note that in c) and d) the limits (in [0, ∞]) exist by monotonicity.Example 1.7. Let (Ω, F ) and ( ˜Ω, ˜ F ) be measurable spaces and let µ be ameasure on (Ω, F ). If f : Ω → ˜Ω is measurable, thenf ∗ (µ)(Ã) := µ{f ∈ Ã}, à ∈ F ˜,defines a measure f ∗ (µ) on ( ˜Ω, ˜ F ). This measure is called the image of µunder f. In the context where (Ω, F , P) is a probability space, the image ofP under f is called the distribution of f.Concerning existence and uniqueness of measures we have the followingtwo fundamental results.Theorem 1.8 (Caratheodory). Let A be a non-empty collection of subsetsof a set Ω that is closed under taking complements and finite unions. If µ :A → [0, ∞] is a mapping such that properties 1.2(1) and 1.2(2) hold, thenthere exists a unique measure ˜µ : σ(A ) → [0, ∞] such that˜µ| A = µ.

1.3 Measures 5The proof is lengthy and rather tedious. The typical situation is that measuresare given, and therefore the proof is omitted. The uniqueness part is aconsequence of the next result, which is useful in many other situations aswell.Theorem 1.9 (Dynkin). Let µ 1 and µ 2 be two finite measures defined on ameasurable space (Ω, F ). Let A ⊆ F be a collection of sets with the followingproperties:(1) A is closed under finite intersections;(2) σ(A ), the σ-algebra generated by A , equals F .If µ 1 (A) = µ 2 (A) for all A ∈ A , then µ 1 = µ 2 .Proof. Let D denote the collection of all sets D ∈ F with µ 1 (D) = µ 2 (D).Then A ⊆ D and D is a Dynkin system, that is,• Ω ∈ D;• if D 1 ⊆ D 2 with D 1 , D 2 ∈ D, then also D 2 \ D 1 ∈ D;• if D 1 ⊆ D 2 ⊆ . . . with all D n ∈ D, then also ⋃ ∞n=1 D n ∈ D.The finiteness of µ 1 and µ 2 is used to prove the second property.By assumption we have D ⊆ F = σ(A ); we will show that σ(A ) ⊆ D. Tothis end let D 0 denote the smallest Dynkin system in F containing A . Wewill show that σ(A ) ⊆ D 0 . In view of D 0 ⊆ D, this will prove the lemma.Let C = {D 0 ∈ D 0 : D 0 ∩ A ∈ D 0 for all A ∈ A }. Then C is a Dynkinsystem and A ⊆ C since A is closed under taking finite intersections. Itfollows that D 0 ⊆ C , since D 0 is the smallest Dynkin system containing A .But obviously, C ⊆ D 0 , and therefore C = D 0 .Now let C ′ = {D 0 ∈ D 0 : D 0 ∩ D ∈ D 0 for all D ∈ D 0 }. Then C ′is a Dynkin system and the fact that C = D 0 implies that A ⊆ C ′ . HenceD 0 ⊆ C ′ , since D 0 is the smallest Dynkin system containing A . But obviously,C ′ ⊆ D 0 , and therefore C ′ = D 0 .It follows that D 0 is closed under taking finite intersections. But a Dynkinsystem with this property is a σ-algebra. Thus, D 0 is a σ-algebra, and nowA ⊆ D 0 implies that also σ(A ) ⊆ D 0 .⊓⊔Example 1.10 (Lebesgue measure). In R d , let A be the collection of all finiteunions of rectangles [a 1 , b 1 ) × · · · × [a d , b d ). Define µ : A → [0, ∞] byµ([a 1 , b 1 ) × · · · × [a d , b d )) :=d∏(b n − a n )and extend this definition to finite disjoint unions by additivity. It can beverified that A and µ satisfy the assumptions of Caratheodory’s theorem.The resulting measure µ on σ(A ) = B(R d ), the Borel σ-algebra of R d , iscalled the Lebesgue measure.n=1

6 1 Measure and IntegralExample 1.11. Let (Ω 1 , F 1 , P 1 ), (Ω 2 , F 2 , P 2 ), . . . be a sequence of probabilityspaces. There exists a unique probability measure P = ∏ ∞∏ n=1 P n on F =∞n=1 F n which satisfiesP(A 1 × · · · × A N × Ω N+1 × Ω N+2 × . . . ) =N∏P n (A n )for all N 1 and A n ∈ F n for n = 1, . . . , N. Finite products of arbitrarymeasures µ 1 , . . . , µ N can be defined similarly.For example, the Lebesgue measure on R d is the product measure of dcopies of the Lebesgue measure on R.As an illustration of the use of Dynkin’s lemma we conclude this sectionwith an elementary result on independence. In probability theory, a measurablefunction f : Ω → R, where (Ω, F , P) is a probability space, is calleda random variable. Random variables f 1 , . . . , f N : Ω → R are said to beindependent if for all B 1 , . . . , B N ∈ B(R) we haveP{f 1 ∈ B 1 , . . . , f N ∈ B N } =n=1N∏P{f 1 ∈ B 1 }.Proposition 1.12. The random variables f 1 , . . . , f N : Ω → R are independentif and only if ν = ∏ Nn=1 ν n, where ν is the distribution of (f 1 , . . . , f N )and ν n is the distribution of f n , n = 1, . . . , N.Proof. By definition, f 1 , . . . , f N are independent if and only if ν(B) =∏ Nn=1 ν n(B n ) for all B = B 1 × · · · × B N with B n ∈ B(R) for all n = 1, . . . , N.Since these sets generate B(R N ), by Dynkin’s lemma the latter impliesν = ∏ Nn=1 ν n. ⊓⊔n=11.4 The Lebesgue integralLet (Ω, F ) be a measurable space. A simple function is a function f : Ω → Rthat can be represented in the formf =N∑c n 1 Anwith coefficients c n ∈ R and disjoint sets A n ∈ F for all n = 1, . . . , N.n=1Proposition 1.13. A function f : Ω → [−∞, ∞] is measurable if and only ifit is the pointwise limit of a sequence of simple functions. If in addition f isnon-negative, we may arrange that 0 f n ↑ f pointwise.Proof. Take, for instance, f n = ∑ 2 2nm=−2 2nm 21 n {f∈[ m2 n , m+12n )}.⊓⊔

1.4 The Lebesgue integral 71.4.1 Integration of simple functionsLet (Ω, F , µ) be a measure space. For a non-negative simple function f =∑ Nn=1 c n1 An we define∫Ωf dµ :=N∑c n µ(A n ).n=1We allow µ(A n ) to be infinite; this causes no problems because the coefficientsc n are non-negative (we use the convention 0 · ∞ = 0). It is easy to checkthat this integral is well-defined, in the sense that it does not depend on theparticular representation of f as a simple function. Also, the integral is linearwith respect to addition and multiplication with non-negative scalars,∫∫ ∫af 1 + bf 2 dµ = a f 1 dµ + b f 2 dµ,Ωand monotone in the sense that if 0 f 1 f 2 pointwise, then∫ ∫f 1 dµ f 2 dµ.1.4.2 Integration of non-negative functionsΩIn what follows, a non-negative function is a function with values in [0, ∞].For a non-negative measurable function f we choose a sequence of simplefunctions 0 f n ↑ f and define∫∫f dµ := lim f n dµ.n→∞ΩThe following lemma implies that this definition does not depend on the approximatingsequence.Lemma 1.14. For a non-negative measurable function f and non-negativesimple functions f n and g such that 0 f n ↑ f and g f pointwise we have∫∫g dµ lim f n dµ.n→∞ΩProof. First consider the case g = 1 A . Fix ε > 0 arbitrary and let A n ={1 A f n 1 − ε}. Then A 1 ⊆ A 2 ⊆ . . . ↑ A and therefore µ(A n ) ↑ µ(A n ). Since1 A f n (1 − ε)1 An ,∫∫limn→∞Ω1 A f n dµ (1 − ε) limn→∞ µ(A n) = (1 − ε)µ(A) = (1 − ε)This proves the lemma for g = 1 A . The general case follows by linearity.ΩΩΩΩΩΩg dµ.⊓⊔

1.4 The Lebesgue integral 9∫∫φ ◦ f dµ = φ df ∗ (µ).Ω 1 Ω 2To prove this, note the this is trivial for simple functions φ = 1 A with A ∈ F 2 .By linearity, the identity extends to non-negative simple functions φ, and bymonotone convergence (using Proposition 1.13) to non-negative measurablefunctions φ.From the monotone convergence theorem we deduce the following usefulcorollary.Theorem 1.17 (Fatou Lemma). Let (f n ) ∞ n=1 be a sequence of non-negativemeasurable functions on (Ω, F , µ). Then∫∫lim inf f n dµ lim inf f n dµ.n→∞ n→∞ΩProof. From inf kn f k f m , m n, we infer∫∫inf f k dµ infkn mnHence, by the monotone convergence theorem,∫∫lim inf f n dµ = lim inf f k dµΩn→∞Ωn→∞ kn∫= lim inf f k dµ limn→∞ knΩΩΩΩinfn→∞ mn1.4.3 Integration of real-valued functionsf m dµ.A measurable function f : Ω → [−∞, ∞] is called integrable if∫|f| dµ < ∞.Ω∫Ω∫f m dµ = lim inf f n dµ.n→∞Ω⊓⊔Clearly, if f and g are measurable and |g| |f| pointwise, then g is integrable iff is integrable. In particular, if f is integrable, then the non-negative functionsf + and f − are integrable, and we define∫ ∫ ∫f dµ := f + dµ − f − dµ.ΩFor a set A ∈ F we writeΩΩ∫ ∫f dµ := 1 A f dµ,AΩnoting that 1 A f is integrable. The monotonicity and additivity properties ofthis integral carry over to this more general situation, provided we assumethat the functions we integrate are integrable.The next result is among the most useful in analysis.

10 1 Measure and IntegralTheorem 1.18 (Dominated Convergence Theorem). Let (f n ) ∞ n=1 be asequence of integrable functions such that lim n→∞ f n = f pointwise. If thereexists an integrable function g such that |f n | |g| for all n 1, then∫In particular,limn→∞Ω|f n − f| dµ = 0.∫ ∫lim f n dµ = f dµ.n→∞ΩΩProof. We make the preliminary observation that if (h n ) ∞ n=1 is a sequence ofnon-negative measurable functions such that lim n→∞ h n = 0 pointwise and his a non-negative integrable function such that h n h for all n 1, then bythe Fatou lemma∫ ∫h dµ = lim infΩΩ lim infn→∞(h − h n) dµn→∞∫∫h − h n dµ =ΩSince ∫ Ω h dµ is finite, it follows that 0 lim sup n→∞therefore∫h n dµ = 0.limn→∞ΩΩ∫h dµ − lim sup h n dµ.n→∞ Ω∫Ω h n dµ 0 andThe theorem follows by applying this to h n = |f n − f| and h = 2|g|.Let (Ω, F , P) be a probability space. The integral of an integrable randomvariable f : Ω → R is called the expectation of f, notation∫Ef = f dP.Proposition 1.19. If f : Ω → R and g : Ω → R are independent integrablerandom variables, then fg : Ω → R is integrable andΩEfg = Ef Eg.Proof. To prove this we first observe that for any non-negative Borel measurableφ : R → R, the random variables φ ◦ f and φ ◦ g are independent. Takingφ(x) = x + and φ(x) = x − , it follows that we may consider the positive andnegative parts of f and g separately. In doing so, we may assume that f andg are non-negative.Denoting the distributions of f, g, and (f, g) by ν f , ν g , and ν (f,g) respectively,by the substitution formula (with φ(x, y) = xy1 [0,∞) 2(x, y)) and thefact that ν (f,g) = ν f × ν g we obtain⊓⊔

1.5 L p -spaces 11∫∫E(fg) = xy dν (f,g) (x, y) = xy d(ν f × dν g )(x, y)[0,∞) 2 [0,∞)∫2 ∫=xy dν f (x)dν g (y) = Ef Eg.[0,∞) [0,∞)This proves the integrability of fg along with the desired identity.⊓⊔1.4.4 The role of null setsWe begin with a simple observation.Proposition 1.20. Let f be a non-negative measurable function.1. If ∫ f dµ < ∞, then µ{f = ∞} = 0.Ω2. If ∫ f dµ = 0, then µ{f ≠ 0} = 0.ΩProof. For all c > 0 we have 0 c1 {f=∞} f and therefore∫0 cµ{f = ∞} f dµ.The first result follows from this by letting c → ∞. For the second, note thatfor all n 1 we have 1 n 1 {f 1 n } f and therefore1n µ{ f 1 ∫} f dµ = 0.n ΩIt follows that µ{f 1 n } = 0. Now note that {f > 0} = ⋃ ∞n=1 {f 1 n}. ⊓⊔Proposition 1.21. If f is integrable and µ{f ≠ 0} = 0, then ∫ f dµ = 0.ΩProof. By considering f + and f − separately we may assume f is non-negative.Choose simple functions 0 f n ↑ f. Then µ{f n > 0} µ{f > 0} = 0 andtherefore ∫ Ω f n dµ = 0 for all n 1. The result follows from this. ⊓⊔What these results show is that in everything that has been said so far, wemay replace pointwise convergence by pointwise convergence µ-almost everywhere,where the latter means that we allow an exceptional set of µ-measurezero. This applies, in particular, to the monotonicity assumption in the monotoneconvergence theorem and the convergence and domination assumptionsin the dominated convergence theorem.Ω1.5 L p -spacesLet µ be a measure on (Ω, F ) and fix 1 p < ∞. We define L p (µ) as theset of all measurable functions f : Ω → R such that

12 1 Measure and Integral∫For such functions we defineΩ‖f‖ p :=|f| p dµ < ∞.( ∫ |f| p dµΩ) 1pThe next result shows that L p (µ) is a vector space:Proposition 1.22 (Minkowski’s inequality). For all f, g ∈ L p (µ) wehave f + g ∈ L p (µ) and‖f + g‖ p ‖f‖ p + ‖g‖ p .Proof. By elementary calculus it is checked that for all non-negative real numbersa and b one has(a + b) p = inft∈(0,1) t1−p a p + (1 − t) 1−p b p .Applying this identity to |f(x)| and |g(x)| and integrating with respect to µ,for all fixed t ∈ (0, 1) we obtain∫∫|f(x) + g(x)| p dµ(x) (|f(x)| + |g(x)|) p dµ(x)ΩΩ∫∫ t 1−p |f(x)| p dµ(x) + (1 − t) 1−p |g(x)| p dµ(x).Stated differently, this says thatΩ‖f + g‖ p p t 1−p ‖f‖ p p + (1 − t) 1−p ‖g‖ p p..ΩTaking the infimum over all t ∈ (0, 1) gives the result.⊓⊔In spite of this result, ‖ · ‖ p is not a norm on L p (µ), because ‖f‖ p = 0only implies that f = 0 µ-almost everywhere. In order to get around thisimperfection, we define an equivalence relation ∼ on L p (µ) byf ∼ g ⇔ f = g µ-almost everywhere.The equivalence class of a function f modulo ∼ is denoted by [f]. On thequotient spaceL p (µ) := L p (µ)/ ∼we define a scalar multiplication and addition in the natural way:c[f] := [cf],[f] + [g] := [f + g].We leave it as an exercise to check that both operators are well defined. Withthese operations, L p (µ) is a normed vector space with respect to the norm

1.5 L p -spaces 13‖[f]‖ p := ‖f‖ p .Following common practice we shall make no distinction between a functionin L p (µ) and its equivalence class in L p (µ).The next result states that L p (µ) is a Banach space (which, by definition,is a complete normed vector space):Theorem 1.23. The normed space L p (µ) is complete.Proof. Let (f n ) ∞ n=1 be a Cauchy sequence with respect to the norm ‖ · ‖ p ofL p (µ). By passing to a subsequence we may assume that‖f n+1 − f n ‖ p 1 , n = 1, 2, . . .2n Define the non-negative measurable functionsg N :=N−1∑n=0|f n+1 − f n |, g :=∞∑|f n+1 − f n |,with the convention that f 0 = 0. By the monotone convergence theorem,∫∫g p dµ = lim g p N dµ.N→∞ΩΩn=0Taking p-th roots and using Minkowski’s inequality we obtainN−1‖g‖ p = lim ‖g ∑N‖ p lim ‖f n+1 −f n ‖ p =N→∞ N→∞n=0∞∑‖f n+1 −f n ‖ p 1+‖f 1 ‖ p .If follows that g is finitely-valued µ-almost everywhere, which means that thesum defining g converges absolutely µ-almost everywhere. As a result, the sumf :=∞∑(f n+1 − f n )n=0n=0converges on the set {g < ∞}. On this set we haveN−1∑f = lim (f n+1 − f n ) = lim f N.N→∞N→∞n=0Defining f to be zero on the null set {g = ∞}, the resulting function f ismeasurable. FromN−1|f N | p ∑= ∣ (f n+1 − f n ) ∣ p n=0( N−1∑n=0) p|f n+1 − f n | |g|pand the dominated convergence theorem we conclude that

14 1 Measure and Integrallim ‖f − f N‖ p = 0.N→∞We have proved that a subsequence of the original Cauchy sequence convergesto f in L p (µ). As is easily verified, this implies that the original Cauchysequence converges to f as well.⊓⊔In the course of the proof we obtained the following result:Corollary 1.24. Every convergence sequence in L p (µ) has a µ-almost everywhereconvergent subsequence.For p = ∞ the above definitions have to be modified a little. We defineL ∞ (µ) to be the vector space of all measurable functions f : Ω → R suchthatsup |f(ω)| < ∞ω∈Ωand define, for such functions,The space‖f‖ ∞ := sup |f(ω)|.ω∈ΩL ∞ (µ) := L ∞ (µ)/ ∼is again a normed vector space in a natural way. The simple proof that L ∞ (µ)is a Banach space is left as an exercise.We continue with an approximation result, the proof of which is an easyapplication of Proposition 1.13 together with the dominated convergence theorem:Proposition 1.25. For 1 p ∞, the simple functions in L p (µ) are densein L p (µ).We conclude with an important inequality, known as Hölder’s inequality.For p = q = 2, it is known as the Cauchy-Schwarz inequality.Theorem 1.26. Let 1 p, q ∞ satisfy 1 p + 1 qg ∈ L q (µ), then fg ∈ L 1 (µ) and= 1. If f ∈ Lp (µ) and‖fg‖ 1 ‖f‖ p ‖g‖ q .Proof. For p = 1, q = ∞ and for p = ∞, q = 1, this follows by a direct estimate.Thus we may assume from now on that 1 < p, q < ∞. The inequality is thenproved in the same way as Minkowski’s inequality, this time using the identity( t p a p )ab = inft>0 p+ bqqt q . ⊓⊔

1.7 Notes 151.6 Fubini’s theoremA measure space (Ω, F , µ) is called σ-finite if we can write Ω = ⋃ ∞n=1 Ω(n)with sets Ω (n) ∈ F satisfying µ(Ω (n) ) < ∞.Theorem 1.27 (Fubini). Let (Ω 1 , F 1 , µ 1 ) and (Ω 1 , F 1 , µ 1 ) be σ-finite measurespaces. For any non-negative measurable function f : Ω 1 × Ω 2 → R thefollowing assertions hold:(1) The non-negative function ω 1 ↦→ ∫ Ω 2f(ω 1 , ω 2 ) dµ 2 (ω 2 ) is measurable;(2) The non-negative function ω 2 ↦→ ∫ Ω 1f(ω 1 , ω 2 ) dµ 1 (ω 1 ) is measurable;(3) We have∫∫∫f d(µ 1 × µ 2 ) = f dµ 1 dµ 2 = f dµ 2 dµ 1 .Ω 1×Ω 2∫Ω 2 Ω 1∫Ω 1 Ω 2We shall not give a detailed proof, but sketch the main steps. First, forf(ω 1 , ω 2 ) = 1 A1 (ω 1 )1 A2 (ω 2 ) with A 1 ∈ F 1 , A 2 ∈ F 2 , the result is clear.Dynkin’s lemma is used to extend the result to functions f = 1 A with A ∈F = F 1 × F 2 . One has to assume first that µ 1 and µ 2 are finite; the σ-finitecase then follows by approximation. By taking linear combinations, the resultextends to simple functions. The result for arbitrary non-negative measurablefunctions then follows by monotone convergence.A variation of the Fubini theorem holds if we replace ‘non-negative measurable’by ‘integrable’. In that case, for µ 1 -almost all ω 1 ∈ Ω 1 the functionω 2 ↦→ f(ω 1 , ω 2 ) is integrable and for µ 2 -almost all ω 2 ∈ Ω 2 the function ω 1 ↦→f(ω 1 , ω 2 ) is integrable. Moreover, the functions ω 1 ↦→ ∫ Ω 2f(ω 1 , ω 2 ) dµ 2 (ω 2 )and ω 2 ↦→ ∫ Ω 1f(ω 1 , ω 2 ) dµ 1 (ω 1 ) are integrable and the above identities hold.1.7 NotesThe proof of the monotone convergence theorem is taken from Kallenberg[1]. The proofs of the Minkowski inequality and Hölder inequality, althoughnot the ones that are usually presented in introductory courses, belong to thefolklore of the subject.

2Conditional expectationsIn this lecture we introduce conditional expectations and study some of theirbasic properties.We begin by recalling some terminology. A random variable is a measurablefunction on a probability space (Ω, F , P). In this context, Ω is often called thesample space and the sets in F the events. For an integrable random variablef : Ω → R we write∫Ef := f dPfor its expectation.Below we fix a sub-σ-algebra G of F . For 1 p ∞ we denote byL p (Ω, G ) the subspace of all f ∈ L p (Ω) having a G -measurable representative.With this notation, L p (Ω) = L p (Ω, F ). We will be interested in the problemof finding good approximations of a random variable f ∈ L p (Ω) by randomvariables in L p (Ω; G ).Lemma 2.1. L p (Ω, G ) is a closed subspace of L p (Ω).Proof. Suppose that (ξ n ) ∞ n=1 is a sequence in L p (Ω, G ) such that lim n→∞ f n =f in L p (Ω). We may assume that the f n are pointwise defined and G -measurable. After passing to a subsequence (when 1 p < ∞) we mayfurthermore assume that lim n→∞ f n = f almost surely. The set C of allω ∈ Ω where the sequence (f n (ω)) ∞ n=1 converges is G -measurable. Put˜f := lim n→∞ 1 C f n , where the limit exists pointwise. The random variable˜f is G -measurable and agrees almost surely with f. This shows that f definesan element of L p (Ω, G ).⊓⊔Our aim is to show that L p (Ω, G ) is the range of a contractive projectionin L p (Ω). For p = 2 this is clear: we have the orthogonal decompositionL 2 (Ω) = L 2 (Ω, G ) ⊕ L 2 (Ω, G ) ⊥Ω

18 2 Conditional expectationsand the projection we have in mind is the orthogonal projection, denotedby P G , onto L 2 (Ω, G ) along this decomposition. Following common usage wewriteE(f|G ) := P G (f), f ∈ L 2 (Ω),and call E(f|G ) the conditional expectation of f with respect to G . Let usemphasise that E(f|G ) is defined as an element of L 2 (Ω, G ), that is, as anequivalence class of random variables.Lemma 2.2. For all f ∈ L 2 (Ω) and G ∈ G we have∫∫E(f|G ) dP = f dP.GAs a consequence, if f 0 almost surely, then E(f|G ) 0 almost surely.Proof. By definition we have f − E(f|G ) ⊥ L 2 (Ω, G ). If G ∈ G , then 1 G ∈L 2 (Ω, G ) and therefore∫1 G (f − E(f|G )) dP = 0.ΩThis gives the desired identity. For the second assertion, choose a G -measurablerepresentative of g := E(f|G ) and apply the identity to the G -measurable set{g < 0}. ⊓⊔Taking G = Ω we obtain the identity E(E(f|G )) = Ef. This will be usedin the lemma, which asserts that the mapping f ↦→ E(f|G ) is L 1 -bounded.Lemma 2.3. For all f ∈ L 2 (Ω) we have E|E(f|G )| E|f|.Proof. It suffices to check that |E(f|G )| E(|f| |G ), since then the lemmafollows from E|E(f|G )| EE(|f| |G ) = E|f|. Splitting f into positive andnegative parts, almost surely we have|E(f|G )| = |E(f + |G ) − E(f − |G )| |E(f + |G )| + |E(f − |G )| = E(f + |G ) + E(f − |G ) = E(|f| |G ).Since L 2 (Ω) is dense in L 1 (Ω) this lemma shows that the condition expectationoperator has a unique extension to a contractive linear operator onL 1 (Ω), which we also denote by E(·|G ). This operator is a projection (thisfollows by noting that it is a projection in L 2 (Ω) and then using that L 2 (Ω)is dense in L 1 (Ω)) and it is positive in the sense that it maps positive randomvariables to positive random variables; this follows from Lemma 2.2 byapproximation.Lemma 2.4 (Conditional Jensen inequality). If φ : R → R is convex,then for all f ∈ L 1 (Ω) such that φ ◦ f ∈ L 1 (Ω) we have, almost surely,Gφ ◦ E(f|G ) E(φ ◦ f|G ).⊓⊔

2 Conditional expectations 19Proof. If a, b ∈ R are such that at + b φ(t) for all t ∈ R, then the positivityof the conditional expectation operator givesaE(f|G ) + b = E(af + b|G ) E(φ ◦ f|G )almost surely. Since φ is convex we can find real sequences (a n ) ∞ n=1 and (b n ) ∞ n=1such that φ(t) = sup n1 (a n t + b n ) for all t ∈ R; we leave the proof of this factas an exercise. Hence almost surely,φ ◦ E(f|G ) = sup a n E(f|G ) + b n E(φ ◦ f|G ).n1⊓⊔Theorem 2.5 (L p -contractivity). For all 1 p ∞ the conditional expectationoperator extends to a contractive positive projection on L p (Ω) withrange L p (Ω, G ). For f ∈ L p (Ω), the random variable E(f|G ) is the uniqueelement of L p (Ω, G ) with the property that for all G ∈ G ,∫∫E(f|G ) dP = f dP. (2.1)GProof. For 1 p < ∞ the L p -contractivity follows from Lemma 2.4 applied tothe convex function φ(t) = |t| p . For p = ∞ we argue as follows. If f ∈ L ∞ (Ω),then 0 |f| ‖f‖ ∞ 1 Ω and therefore 0 E(|f| |G ) ‖f‖ ∞ 1 Ω almost surely.Hence, E(|f| |G ) ∈ L ∞ (Ω) and ‖E(|f| |G )‖ ∞ ‖f‖ ∞ .For 2 p ∞, (2.1) follows from Lemma 2.2. For f ∈ L p (Ω) with1 p < 2 we choose a sequence (f n ) ∞ n=1 in L 2 (Ω) such that lim n→∞ f n = fin L p (Ω). Then lim n→∞ E(f n |G ) = E(f|G ) in L p (Ω) and therefore, for anyG ∈ G ,∫G∫∫ ∫E(f|G ) dP = lim E(f n |G ) dP = lim f n dP = f dP.n→∞Gn→∞GGIf g ∈ L p (Ω, G ) satisfies ∫ G g dP = ∫ G f dP for all G ∈ G , then ∫ ∫G g dP =E(f|G ) dP for all G ∈ G . Since both g and E(f|G ) are G -measurable, asGin the proof of the second part of Lemma 2.2 this implies that g = E(f|G )almost surely.In particular, E(E(f|G )|G ) = E(f|G ) for all f ∈ L p (Ω) and E(f|G ) = ffor all f ∈ L p (Ω, G ). This shows that E(·|G ) is a projection onto L p (Ω, G ).⊓⊔The next two results develop some properties of conditional expectations.Proposition 2.6.(1) If f ∈ L 1 (Ω) and H is a sub-σ-algebra of G , then almost surelyGE(E(f|G )|H ) = E(f|H ).

20 2 Conditional expectations(2) If f ∈ L 1 (Ω) is independent of G (that is, f is independent of 1 G for allG ∈ G ), then almost surelyE(f|G ) = Ef.(3) If f ∈ L p (Ω) and g ∈ L q (Ω, G ) with 1 p, q ∞, 1 p + 1 q= 1, then almostsurelyE(gf|G ) = gE(f|G ).Proof. (1): For all H ∈ H we have ∫ H E(E(f|G )|H ) dP = ∫ ∫E(f|G ) dP =Hf dP by Theorem 2.5, first applied to H and then to G (observe thatHH ∈ G ). Now the result follows from the uniqueness part of the theorem.(2): For all G ∈ G we have ∫ G Ef dP = E1 GEf = E1 G f = ∫ f dP, andGthe result follows from the uniqueness part of Theorem 2.5.(3): For all G, G ′ ∈ G we have ∫ G 1 G ′E(f|G ) dP = ∫ ∫E(f|G ) dP =G∩G ′f dP = ∫ G∩G ′ G f1 G ′ dP. Hence E(f1 G ′|G ) = 1 G ′E(f|G ) by the uniquenesspart of Theorem 2.5. By linearity, this gives the result for simple functions g,and the general case follows by approximation.⊓⊔Example 2.7. Let f : (−r, r) → R be integrable and let G be the σ-algebra ofall symmetric Borel sets in (−r, r) (we call a set B symmetric if B = −B).Then,E(f|G ) = 1 2 (f + ˇf) almost surely,where ˇf(x) = f(−x). This is verified by checking the condition (2.1).Example 2.8. Let f : [0, 1) → R be integrable and let G be the σ-algebragenerated by the sets I n := [ n−1N, n N), n = 1, . . . , N. Then,E(f|G ) =N∑n=1m n 1 [n−1N , n N ) almost surely,wherem n = 1 ∫f(x) dx.|I n | I nAgain this is verified by checking the condition (2.1).If f and g are random variables on (Ω, F , P), we write E(g|f) := E(g|σ(f)),where σ(f) is the σ-algebra generated by f.Example 2.9. Let f 1 , . . . , f N be independent integrable random variables satisfyingE(f 1 ) = · · · = E(f N ) = 0, and put S N := f 1 + · · · + f N . Then, for1 n N,E(S N |f n ) = f n almost surely.This follows from the following almost sure identities:E(S N |f n ) = E(f n |f n ) + ∑ m≠nE(f m |f n ) = f n + ∑ m≠nE(f m ) = f n .

2.1 Notes 21Example 2.10. Let f 1 , . . . , f N be independent and identically distributed integrablerandom variables and put S N := f 1 + · · · + f N . Thenand thereforeE(f 1 |S N ) = · · · = E(f N |S N ) almost surelyE(f n |S N ) = S N /N almost surely, n = 1, . . . , N.It is clear that the second identity follows from the first. To prove the first,let µ denote the (common) distribution of the random variables f n . By independence,the distribution of the R N -valued random variable (f 1 , . . . , f N )equals the N-fold product measure µ N := µ × · · · × µ. Then, for any Borel setB ∈ B(R) we have, using the substitution formula and Fubini’s theorem,∫E(f n |S N ) dP{S N ∈B}∫=∫=={S N ∈B}f n dP{x 1+···+x N ∈B}∫ ∫R N−1which is independent of n.x n dµ N (x)y dµ(y) dµ N−1 (z 1 , . . . , z N−1 ),{y∈B−z 1−···−z N−1 }2.1 NotesWe recommend Williams [2] for an introduction to the subject. The biblefor modern probabilists is Kallenberg [1].The definition of conditional expectations starting from orthogonal projectionsin L 2 (Ω) follows Kallenberg [1]. Many textbooks prefer the shorter(but less elementary and transparent) route via the Radon-Nikodým theorem.In this approach, one notes that if f is a random variable on (Ω, F , P) andG is a sub-σ-algebra of F , then∫Q(G) := f dP, G ∈ G ,Gdefines a probability measure Q on (Ω, G , P| G ) which is absolutely continuouswith respect to P| G . Hence by the Radon-Nikodým, Q has an integrabledensity with respect to P| G . This density equals the conditional expectationE(f|G ) almost surely.

References1. O. Kallenberg, “Foundations of Modern Probability”, second ed., Probabilityand its Applications (New York), Springer-Verlag, New York, 2002.2. D. Williams, “Probability with Martingales”, Cambridge Mathematical Textbooks,Cambridge University Press, Cambridge, 1991.