seminar of probability and random processes applied in computer

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮SEMINAR OF PROBABILITYAND RANDOM PROCESSESAPPLIED IN COMPUTERSCIENCE AND ENGINEERING◭◮Page 1 of 25II. SOME EXAMPLES: Random graphs,The Quicksort and Find algorithmsPLINIO TRIANA BUSTOSGo BackFull ScreenCloseQuitBook: Probability Models for Computer Science,Sheldon M. Ross. Academic Press, 2002.

1. Some Examples1.1. A Random GraphSome ExamplesThe Quicksort and Find . . .Home PageTitle PageA graph consists of a set of elements V called vertices anda set A of vertices called edges (or arcs). It is usual to representsuch a system graphically by drawing circles for verticesand drawing lines between vertices i and j when (i, j)is an edge. For instance, V = {1, 2, 3, 4, 5, 6} and A ={(1, 2), (1, 4), (1, 5), (2, 3), (2, 5), (3, 5), (5, 6)}◭◭◭◮◮◮A sequence of vertices i, i 1 , i 2 , ..., i k , j, for which(i, i 1 ), (i 1 , i 2 ), ..., (i k−1 , i k ), (i k , j) are distic edges, is calleda path form vertex i, to vertex 6.Page 2 of 25Go BackA graph is said to be connected if there a path beween each pairof vertices.Full ScreenCloseQuit

Some ExamplesThe Quicksort and Find . . .Home PageConsider now the graph with vertex set V = {1, 2, ..., n} andedge set A = {(i, X(i)), i = 1, ..., n}, where X(i) are independentrandom variables such that◭◭Title Page◮◮P {X(i) = j} = P j ,n∑P j = 1j=1◭ ◮Page 3 of 25Go BackFull ScreenIn other words, from each vertex we randomly choose, accordingto the probabilities P j , j = 1, . . . , n another vertex and thenjoin these two vertices by an edge. We call (i, X(i)) the edgeemanating from vertex i.A graph of te type being condidered is commonly called a randomgraph . We are interested in determining the probability thatthis random graph is connected.CloseQuit

Some ExamplesThe Quicksort and Find . . .Choose some vertex, say perhaps vertex 1, and followthe sequence of vertices 1, X(1), X 2 (1), . . .,where X n =X(X n−1 (1)), y define N to equal the first k for wich X k (1)is not a new vertex. That is,Home PageTitle PageN = min(k : X k (1) ∈ {1, X(1), X 2 (1), . . . X k−1 (1))In addition, let◭◭◭◮◮◮W = P 1 +∑N−1i=1P X i (1) = 1Page 4 of 25Go BackFull ScreenCloseQuitN, is the number of vertices reached in the sequence1, X(1), X 2 (1), . . . before a vertex appears twice, and W is thesum of the probabilites of these vertices.

Some ExamplesThe Quicksort and Find . . .Home PageTo obtain the probability that the graph is connected,we condition on N and the sequence of vertices1, X(1), X 2 (1), . . . X N−1 (1). Now, given these values, theN vertices {1, X(1), X 2 (1), . . . X N−1 (1), are connected toeach other, there are one of the other n − N vertices will go intoone of these vertices with probability W .◭◭Title Page◮◮Lemma 1 Consider a random graph consisting of vertices0, 1, . . . , r and edges (i, Y i ), i = 1, . . . , r,where the Y i are independentand such that◭◮Page 5 of 25P {X(i) = j} = Q j , j = 0, . . . , r,r∑Q j = 1j=0Go BackFull ScreenCloseIn other words, this random graph consists of r ordinary verticesand one special vertex; out of each ordinary vertex there is anedge that independetly goes into vertex j with probability Q j ;there is no edge emanating form the special vertex. Then,QuitP {graph is connected} = Q 0

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 6 of 25Go BackFull ScreenCloseProof. By induction on r. As it is clearly true when r = 1,assume it to be true for all values less than r. Now, considerY i , and note the following. If Y i , then, because the additionalr-1 edges are not enough to connect the graph of r + 1 separatevertices, the graph will not be connected. If y 1 = 0, then vertices1 and 0 can be rearded as a single vertex and the situation isthe same as if we had r − 1 ordinary vertices and one specialvertex, wigh each ordinary vertex having an edge that goes intothe special vertex with probability Q 0 + Q 1 . If Y 1 = j ≠ 0, 1,then by regarding vertices 1 and j as a single vertex, the situationis the same as if we had r − 1 ordinary vertices and one specialvertex, with each ordinary vertex having an edge that goes intothe special vertex with probability Q 0 . Hence, form the inductionhypotheisi, we see that0 ifj = 1P{graph is connected|Y 1 = j} = Q 0 + Q 1 if j = 0Q 0 if j ≠ 0, 1Quit

Therefore, conditioning on Y 1 yieldsSome ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 7 of 25Go BackFull ScreenCloseP {graph is connected} =r∑P {graph is connected|Y 1 = j}Q jj=0= (Q 0 + Q 1 )Q 0 + Q 0 (1 − Q 0 − Q 1 )= Q 0 □Returning to the original random graph, it follows, upon regardingthe set of vertcices 1, X(1), X 2 (1), . . . , X N−1 (1) as the specialvertex Lemma 1, thatP {graph is connected|N, 1, . . . , X N−1 (1)} = WFor the rest, we will restrict attention to the special case wherethe edge emanating from each vertex is equally likely to go toany of the n vertices of the graph.P j = 1/n, j = 1, . . . , nQuit

Proposition 1 P {graph is connected} = E [W ]Corollary 1 When P j = 1/nSome ExamplesThe Quicksort and Find . . .Home PageP {graph is connected} =Proof Because W = N/n, we have(n − 1)!n n∑n−1j=0n jj!Title Page◭◭ ◮◮◭ ◮Page 8 of 25Go BackFull ScreenCloseQuitE[W ] = 1 n E [W ]n−1= 1 ∑P {N > i}n= 1 n= 1 n==i=0n−1∑i=0n−1∑i=0(n − 1)!n n(n − 1)!n n(n − 1) . . . (n − i)n i(n − 1)!(n − i − 1)!n in−1∑i=0n−1∑i=0n n−1−i(n − i − 1)!n jj! □

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 9 of 25Go BackTo obtain a simple approximation for the probability that thegraph is connected when n is large, note that if X is a Poissonrandom variable with mean n then∑n−1P {X < n} = e −nHowever, bacause a Poisson a random variable with mean n canbe regarded as being the sum of n independent Poisson randomvariables with mean 1, it follows from teh C.L.T. that such a rondomvariable is less than its mean with probability 1/2, this impliesthati=0n ii!Full ScreenP {X < n} ∼ 1 2CloseQuit

Consequently, for n largeSome ExamplesThe Quicksort and Find . . .Home Page∑n−1i=0n ii! ∼ en2Hence, from Corollary 1, we se that for large n,Title Page◭◭ ◮◮◭ ◮Page 10 of 25Go BackFull ScreenCloseQuit(n − 1)!enP graph is conected ∼ = n!en2n n 2n n+1Therefore, upon using Stirling’s approximation, which states thatn! ∼ n n+1/2 e −n√ 2πwe see that, for n large√2πP graph is conected ∼2 √ 2nTherefore, we have shown the folowing.Theorem 1 For n large, P{graph is connected} ∼ √ π2n

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮A graph is said to consist of r components if its vertices can bepartitioned into r subsets so that each subest is connected and, inaddition, there are no edges between vertices in differents subsets.◭◮Page 11 of 25Go BackFull ScreenCloseQuit

Some ExamplesThe Quicksort and Find . . .Home PageLet C denote the number of components in the random graphbeing considered, and let us compute its expected value. To doso, we will first argue that every component must contain exactlyone cycle, where a cycle is an edge of the form (i, i), or sequenceof edges of the form (i, i 1 ), (i 1 , i 2 ), . . . , (i k , i) for distinc verticesi, i 1 , . . . , i k .Title Page◭◭◮◮◭◮Page 12 of 25Go BackFull ScreenCloseQuitNote that connected graph consisting of the same number ofedges as vertices must contain exactly one cycle. Hence becausein the random graph there is exactly one edge emanating fromeach vertex, it follows that a component consisting of k verticesmust have exactly k edges and thus one cycle.

For S ⊂ {1, . . . , n}, say that S is a cycle if there exists a cyclewhose vertices are the vertices of S, Consequently, if we letSome ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮then1,if S is a cycleI(S) = { 0, otherwiseE[C] = E[number of cycles]= E[ ∑ SI(S)] = ∑ SE[I(S)]◭◮Page 13 of 25Go BackFull ScreenCloseQuitIf S consists of a single vertex, say S = {1}, then S will be acycle if X(1) = 1, thus E(I({1}) = P {X(1) = 1} = 1 nIf S consists of of k > 1 vertices, say S = {1, . . . , k}, then Sconstitute a cycle if 1, X(1), . . . , X k−1 are all distinc values inS, and X k = 1. Therefore,E[I(S)] = k − 1 k − 2n n. . . 1 1n n

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 14 of 25( nk )Consequently, as there are subsets of size k, we see thatn∑ ( nk ) (k − 1)!E[C] =k=1n kGo BackFull ScreenCloseQuit

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 15 of 25Go BackFull Screen2. The Quicksort and Find AlgorithmsSuppose that we want to sort a given set of n distinc values,x 1 , x 2 , . . . , x n . A more efficient algorithm than bubble sort fordoing so is the quicsort algorithm, which is recursively definedas follows. When n = 2, the algorithm compares the two valuesan puts them in the appropriate order. Whem n > 2, one of thevalues is chosen, say it is x i , and then all of the other values arecompared with x i . Those smaller than x i , are put in bracket to theleft of x i , and those larger than x i ,are put in a bracket to the rightof x i . The algorithm the repeats itself in these brackets, continuinguntil all values have been sorted. For instance, suppose thatwe desire to sort the following 10 distinct values:CloseQuit

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 16 of 25Go BackFull Screen5, 9, 3, 10, 11, 14, 8, 4, 17, 6{5, 9, 3, 8, 4, 6,} 10, {11, 14, 17,}{5, 3, 4}, 6, {9, 8,} 10, {11, 14, 17,}{3}, 4, {5}, 6, {9, 8,} 10, {11, 14, 17,}It is intuitively that the worst case occurs when every comparisonvalue chosen is an extreme value. In this worst scenario, thenumber of comparisons need is (n − 1) + (n − 2) + . . . + 1 =n(n − 1)/2.A better indication by determinig the average number of comparisonsneeded when the compararison value are randomly chosen.CloseQuit

Some ExamplesThe Quicksort and Find . . .Let X denote the number of comparisons needed . Let 1 denotethe smallest value, let 2 denote the second value smallest, and soon. Then, for 1≤ i < j ≤ n, let I(i, j) equal 1 if i and j areever directly compared, and let it equal 0 otherwise.Home PageTitle PageX =n∑j=2j−1∑i=1I(i, j)◭◭◮◮which implies that◭ ◮Page 17 of 25Go BackFull ScreenCloseQuitE[X] = E[==n∑j=2j−1∑i=1I(i, j)]j−1 n∑ ∑E[I(i, j)]j=2i=1j−1 n∑ ∑P {i and j are ever compared}j=2i=1

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 18 of 25Go BackTo determine the probability that i and j are ever compared, notethat the values i, i + 1, . . . , j − 1, j will initially be in the samebracket and will remain in the same bracket if the number chosenfor the first comparison is not between i and j. Thus, theprobability that it is i or j is 2/(j − i + 1). Therefore,P {i and j are ever compared} =2j − i + 1Full ScreenCloseQuit

Consequently, we see thatSome ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 19 of 25E[X] == 2= 2= 2n∑ ∑j−1j=2 i=1n∑j=2 k=2n∑2n∑j − 1 + 1 = 2 ( 1 j + 1j − 1 . . . 1 3 + 1 2 )j∑k=2 j=kn∑k=2n∑= 2(n + 1)1k1kn − k + 1kn∑k=2=n∑k=2j=21− 2(n − 1)k2(n + 1)k−n∑k=22kkGo BackFull ScreenCloseFor large nn∑k=21k ∼ log(n)Thus, the quicsort algorithm requieres, on average, approximately 2n log(n)comparison to sort n values.Quit

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 20 of 25Go BackFull Screen2.1. The Find AlgorithmSuppose that we want to find the kthsmallest of a list. The findalgorithm is quite similar to quicksort; Suppose that r − 1 itemsare put in the bracket to left. There are now three possibilities:{2, 5, 4, 3}, 6, { 10, 21, 16, 18}1. r = k then, the algorithm ends.2. r < k then, kth smallest value is the (k − r)th smallest ofthe n − r values in the right bracket3. r > k then, search for the kth smallest of the r − 1 values inthe left bracket.CloseQuit

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 21 of 25Go BackFull ScreenCloseWe have, as in the quicksort analysis,andE[X] =X =n∑ ∑j−1I(i, j)j=2 i=1n∑ ∑j−1P {i and j are ever compared}j=2 i=1To determine the probability that i and j are ever compared, weconsider cases:Case 1 i < j ≤ kIn this case i, j, k will remain together until one of the valuesi, i + 1, . . . , k is chosen as the comparison value.P {i and j are ever compared} =2k − i + 1Quit

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 22 of 25Go BackCase 2: i ≤ k < jCase 3: k < i < jP {i and j are ever compared} =P {i and j are ever compared} =2j − i + 12j − k + 1Full ScreenCloseQuit

It follow from the preceding thatSome ExamplesThe Quicksort and Find . . .1n∑2 E[X] =j−1∑j=2 i=11k − i + 1 +n∑k∑j=k+1 i=11j − i + 1 +n∑j−1∑j=k+2 i=k+11j − i + 1Home PageTitle Page◭◭ ◮◮◭ ◮Page 23 of 25Go BackFull ScreenCloseTo the approximate the preceding when n and k are large, letk = αn for 0 < α < 1. Now,n∑j=2j−1∑i=1k−11k − i + 1 = ∑==k∑i=1 j=i+1∑k−1i=1k∑j=2k − ik − i + 1j − 1j∼ k − log(k)∼ k = αn1k − i + 1Quit

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 24 of 25Go BackFull ScreenCloseQuitn∑k∑j=k+1 i=11n∑j − i + 1 =∼∼∼∼As it similarly follows thatwe see thatn∑j−1∑j=k+2 i=k+1j=k+1n∑j=k+1∫ nk1j − k + 1(1j − k + 1 + . . . , 1 )j(log(j) − log(j − k))log(x)dx −∫ n−k1log(x)dxn log(n) − n − (αn log(αn) − αn)−(n − αn) log(n − αn) + (n − αn)n[−α log(α) − (1 − α) log(1 − α)]= n − k = n(1 − α)E[X] ∼ 2n[1 − α log(α) − (1 − α) log(1 − α)]□Thus, the mean number of comparison needed by the find algorithmis a linear function of ghe number of values.□

Some ExamplesThe Quicksort and Find . . .Home PageTitle Page◭◭ ◮◮◭ ◮Page 25 of 25Go BackFull ScreenReferences[1] G.R. Grimmett and D.R. Stirzaker. Probability and RandomProcess. Oxford Science Publications, 1998.[2] C.M. Grinstead and J.L. Snell. Introduction to Probability.American Mathematical Society, 1997.[3] H.P. Hsu. Probability, Random Variables, and Random Processes.McGraw Hill- Schaum, 1996.[4] F.P. Kelly. Probability. Notes form The Archimedeanshttp://www.cam.ac.uk/societies/archim/notes.html, 1996.CloseQuit