Subjective probability and statistical physics

signing probabilities. The a priori method was based on the principle of insufficient
reason; and according to Bernoulli its application was effectively restricted to games
of chance. The a posteriori method was based on observed frequencies of the event
in previous trials. Bernoulli argued that one could also form a judgment about the
probability of an event a posteriori, on the basis of these observed frequencies in
similar cases:
There is, however another way that will lead us to what we are looking
for and enable us to determine a posteriori what we cannot determine
a priori, that is, to ascertain it from the results observed in numerous
similar instances. It must be assumed in this connection that, under
similar conditions, the occurrence (or nonoccurrence) of an event in the
future will follow the same pattern as was observed for like events in the
past. For example if we have observed that out of 300 persons of the same
age and with the same constitution as Titius, 200 died within ten years
while the rest survived, we can with reasonable certainty conclude that
there is twice as many chances that he will have to pay his debt to nature
within the ensuing decade as there are chances that he will live beyond
that time. (ibid. p. 1453)
Bernoulli argued that this second method was neither new nor unusual. What he
wanted to add, however, was the claim that by increasing the number of observations,
one could increase the degree certainty to become as close to 1 as one wishes. This, he
claimed was the consequence of a rightly celebrated theorem he proceeded to prove,
nowadays often called the (weak) Law of Large Numbers.
Somewhat ahistorically, using the notation of Kolmogorov’s formalism, the theorem
is as follows:
Law of Large Numbers Let 〈X, A, P 〉 be a probability space and take
any event A ∈ A. Let 〈X n , A n , P n 〉 by the product probability space describing
n independent and identically distributed repetitions of the original
probability space. Let X n ∋ η ↦→ relf A n (η) ∈ {0, 1/n, . . . k/n, . . . n} be
the function that counts the relative frequency of event A in n repetitions,
i.e.
relf A n (η) = 1 n∑
1 A (x i ) η = (x 1 , . . . , x n ) ∈ X n
n
i=1
where 1 A is the characteristic function of A on X. Then:
∀ɛ > 0 :
lim P n ({η ∈ X n : |relf A
n−→∞
n (η) − P (A)| ≥ ɛ}) = 1 (1)
However, Bernoulli’s Law of Large Numbers fails to deliver what Bernoulli might have
hoped it would, i.e. a method for assigning probabilities on the basis of observed data
alone as an alternative to the Principle of Insufficient Reason.
The point is, perhaps, somewhat subtle. The Law of Large Numbers shows that
when the number of repetitions increases, the probability of obtaining a sequence of
events such that the relative frequency of A lies close to the given value of P (A)
becomes arbitrarily close to 1. It does not say that when the number of repetitions
5