Pushdown Automaton: CFL and the pumping lemma
Pushdown Automaton: CFL and the pumping lemma
Pushdown Automaton: CFL and the pumping lemma
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Unit 9<br />
More <strong>Pushdown</strong> Automata<br />
Context-free Languages<br />
Pumping Lemma for <strong>CFL</strong><br />
Reading: Sipser, chapter 2.3<br />
1
Properties of PDAs<br />
• An NFA can only distinguish between |Q|<br />
different characterizations.<br />
• A PDA can distinguish between an unlimited<br />
number of characterizations.<br />
• A PDA can recognize non-regular languages<br />
because <strong>the</strong> stack can ‘count’.<br />
• A PDA can count ‘more than once’. But it can<br />
not mix counters. Only one active counter can<br />
be used each time.<br />
2
Example: L = {a i b i c j d j |i,j0}<br />
Construct a PDA to recognize:<br />
L = {a i b i c j d j |i,j0}<br />
We will use <strong>the</strong> empty stack model.<br />
The basic idea:<br />
• Push to <strong>the</strong> stack an A for each a, pop an A<br />
for each b.<br />
• Push to <strong>the</strong> stack a C for each c, pop a C for<br />
each d .<br />
3
Properties of PDAs<br />
• We had two ways to describe regular<br />
languages:<br />
Regular-Expressions<br />
syntactic<br />
DFA / NFA<br />
computational<br />
• How about context-free-languages<br />
CFG<br />
syntactic<br />
PDA<br />
computational<br />
4
CFG=PDA<br />
Theorem: A language is context-free iff<br />
some pushdown automaton recognizes it.<br />
Proof:<br />
• <strong>CFL</strong>PDA: we show that if L is <strong>CFL</strong><br />
<strong>the</strong>n a PDA recognizes it.<br />
• PDA <strong>CFL</strong>: we show that if a PDA<br />
recognizes L <strong>the</strong>n L is <strong>CFL</strong>.<br />
5
From CFG to PDA<br />
• Proof idea: Use PDA to simulate leftmost<br />
derivations.<br />
• Leftmost derivation : A derivation of a string is<br />
a leftmost derivation if at every step <strong>the</strong> leftmost<br />
remaining variable is <strong>the</strong> one replaced.<br />
• We use <strong>the</strong> stack to store <strong>the</strong> suffix that has not<br />
been derived so far.<br />
• Any terminal symbols appearing before <strong>the</strong><br />
leftmost variable are matched right away.<br />
6
Different derivations for <strong>the</strong> same<br />
CFG: EEE | E+E<br />
E0 | 1 | 2 | … | 9<br />
E<br />
parse tree<br />
leftmost<br />
derivation<br />
E<br />
E<br />
E<br />
E E E<br />
5 E E<br />
5 3<br />
E<br />
5 3<br />
2<br />
E<br />
E<br />
E<br />
E<br />
rightmost<br />
derivation<br />
E<br />
E<br />
E<br />
E<br />
2<br />
E E 2<br />
E 3<br />
2<br />
<br />
5 3<br />
2<br />
5 + 3 x 2<br />
7
Starting configuration:<br />
control<br />
input:<br />
5 + 3 x 2 E<br />
E<br />
stack:<br />
$<br />
control<br />
E EE<br />
input:<br />
stack:<br />
5 + 3 x 2 E<br />
EE<br />
x<br />
E<br />
$
control<br />
E<br />
E<br />
E<br />
E E E<br />
input:<br />
stack:<br />
5 + 3 x 2 E<br />
E+EE<br />
+<br />
E<br />
x<br />
E<br />
$<br />
control<br />
E<br />
EE<br />
E E E<br />
5<br />
E E<br />
input:<br />
5 + 3 x 2<br />
5+EE<br />
stack:<br />
5<br />
+<br />
E<br />
x<br />
E<br />
$
control<br />
E<br />
EE<br />
E E E<br />
5<br />
E E<br />
input:<br />
stack:<br />
5 + 3 x 2 E<br />
EE<br />
x<br />
E<br />
$<br />
E<br />
<br />
E<br />
E<br />
control<br />
<br />
<br />
<br />
E E E<br />
5 E E<br />
5 3<br />
E<br />
input:<br />
stack:<br />
5 + 3 x 2 3<br />
3E<br />
x<br />
E<br />
$
E<br />
E<br />
E<br />
E E E<br />
control<br />
<br />
5 E E<br />
<br />
5 3<br />
E<br />
input:<br />
stack:<br />
5 + 3 x 2 E<br />
$<br />
E<br />
E<br />
E<br />
E<br />
E E E<br />
control<br />
<br />
<br />
<br />
5 E E<br />
5 3<br />
E<br />
5 3<br />
2<br />
input:<br />
5 + 3 x 2<br />
2<br />
stack:<br />
2<br />
$
E<br />
E<br />
E<br />
E E E<br />
control<br />
<br />
5 E E<br />
<br />
<br />
5 3<br />
E<br />
5 3<br />
2<br />
input:<br />
stack:<br />
5 + 3 x 2 $<br />
The string ‘5 + 3 x 2’ is accepted
From CFG to PDA<br />
Informally:<br />
1. Place <strong>the</strong> marker symbol $ <strong>and</strong> <strong>the</strong> start variable<br />
S on <strong>the</strong> stack.<br />
2. Repeat <strong>the</strong> following steps:<br />
– If <strong>the</strong> top of <strong>the</strong> stack is a variable A:<br />
Choose a rule A→ 1 … k <strong>and</strong> substitute A with 1 … k<br />
– If <strong>the</strong> top of <strong>the</strong> stack is a terminal a:<br />
Read next input symbol <strong>and</strong> compare to a<br />
If <strong>the</strong>y don’t match, reject (die)<br />
– If top of stack is $, go to accept state<br />
13
From CFG to PDA<br />
• For a given CFG G=(V,,S,R),<br />
we construct a PDA P=(Q,,,,q 0 ,F) where:<br />
– Q={q start , q loop , q accpt }<br />
– = V{$}<br />
– q 0 =q start<br />
– F={q accpt }<br />
14
From CFG to PDA<br />
• We define as follows (shorth<strong>and</strong> notation):<br />
– (q start ,,)={(q loop ,S$)}<br />
– (q loop ,,A)={(q loop , 1 … k ) | for each A 1 … k in R}<br />
– (q loop ,a,a)={(q loop ,) | for each a }<br />
– (q loop ,,$)={(q accpt, )}<br />
{,A 1 … k | for rules A 1 … k }<br />
{a,a | for all a}<br />
,S$<br />
,$<br />
q start q loop q accpt<br />
15
Example:<br />
• Construct a PDA for <strong>the</strong> following CFG G:<br />
SaTb | b<br />
TTa | <br />
L(G)= a * b<br />
,SaTb<br />
,Sb<br />
,TTa<br />
,T<br />
a,a<br />
b,b<br />
,S$<br />
,$<br />
q start q loop q accpt<br />
16
From PDA to CFG<br />
context-free grammar<br />
✓<br />
pushdown automaton<br />
• First, we simplify <strong>the</strong> PDA:<br />
– It has a single accept state q f<br />
– $ is always popped exactly before accepting<br />
– Each transition is ei<strong>the</strong>r a push, or a pop, but<br />
not both<br />
17
• single accept state q f :<br />
From PDA to CFG<br />
,<br />
,<br />
18
From PDA to CFG<br />
• $ is always popped exactly before accepting:<br />
{,A | A, A$}<br />
,$<br />
19
From PDA to CFG<br />
• Each transition is ei<strong>the</strong>r a push, or a pop:<br />
,ab ,a ,b<br />
z<br />
, ,z ,z<br />
20
From PDA to CFG<br />
• For any word w accepted by a PDA<br />
P=(Q,,,,q 0 ,q f ) <strong>the</strong> process starts at q 0 with an<br />
empty stack <strong>and</strong> ends at q f with an empty stack.<br />
• Definition: for any two states p,qQ we define<br />
L p,q to be <strong>the</strong> language that if we starts at p with<br />
an empty stack <strong>and</strong> run on wL p,q we end at q<br />
with an empty stack.<br />
• We define for L p,q a variables A p,q s.t.<br />
L p,q = {w | A p,q * w}<br />
• Note, that L(P)=L q0 ,q f<br />
21
From PDA to CFG<br />
• Consider a word wL p,q<br />
• While running w on P, <strong>the</strong> stack is empty at p <strong>and</strong><br />
at q but what happens in <strong>the</strong> middle<br />
• Two possibilities:<br />
– Option 1: The stack also empty in <strong>the</strong> middle<br />
– Option 2: The stack never empty in <strong>the</strong> middle<br />
stack<br />
height<br />
p<br />
r q p q<br />
22
From PDA to CFG<br />
Option 1: The stack also empty in <strong>the</strong> middle<br />
• If <strong>the</strong> stack become empty at some state r <strong>the</strong>n <strong>the</strong><br />
word wL pq can be reconstructed by a<br />
concatenation of a word from L pr <strong>and</strong> a word from<br />
L rq , thus L pr L rq L pq<br />
• In <strong>the</strong> CFG we express this by a rule: A pq A pr A rq<br />
p<br />
r<br />
q<br />
generated by A pr<br />
generated by A rq<br />
23
From PDA to CFG<br />
Option 2: The stack never empty in <strong>the</strong> middle<br />
• The symbol that has been pushed at p is <strong>the</strong><br />
symbol that is popped at q.<br />
• Thus, if at p we read a symbol a <strong>and</strong> moved to r,<br />
while from state s we read a symbol b <strong>and</strong> moved<br />
to q, aL r,s bL p,q <strong>and</strong> in CFG we have A pq aA rs b<br />
r<br />
p<br />
a<br />
s q<br />
b<br />
24<br />
generated by A rs
From PDA to CFG<br />
Let P=(Q, , , , q 0 , q f ) a given PDA.<br />
We construct a <strong>CFL</strong> G=(V,,S,R) as follows*:<br />
• V = {A p,q | p,qQ}<br />
• S=A<br />
q 0 ,q f<br />
• R is a set of rules constructed as follows:<br />
* Proof of correctness <strong>and</strong> fur<strong>the</strong>r reading at <strong>the</strong> supplementary<br />
material in <strong>the</strong> course web page .<br />
25
From PDA to CFG<br />
• Add <strong>the</strong> following rules to R:<br />
1. For each p,q,r,sQ, t, <strong>and</strong> a,b ,<br />
if (r,t)(p,a,) <strong>and</strong> (q,)(s,b,t) add a rule<br />
A pq aA r,s b<br />
p<br />
a,t<br />
r<br />
s<br />
b,t<br />
q<br />
push t<br />
pop t<br />
2. For each p,q,rQ, add a rule A p,q A p,r A r,q<br />
p r q<br />
1. For each pQ, add <strong>the</strong> rule A p,p <br />
p<br />
26
Example:<br />
0,A<br />
,$ #,<br />
q s q 0<br />
1,A<br />
q 1<br />
,$<br />
q 2<br />
0,A<br />
1,A<br />
,$ #,z<br />
q s q 0<br />
q 3<br />
,z<br />
q 1<br />
,$<br />
q 2<br />
L(P)=0 n #1 n<br />
27
Example:<br />
0,A<br />
1,A<br />
,$ #,z<br />
q s q 0<br />
q 3<br />
,z<br />
q 1<br />
,$<br />
q 2<br />
start variable: A S2<br />
productions:<br />
A SS → A SS A SS<br />
A SS → A S0 A 0S<br />
A SS → A S1 A 1S<br />
A SS → A S2 A 2S<br />
A S1 → A SS A S1<br />
A SS → <br />
A 00 → <br />
A 11 → <br />
A 22 → <br />
A 33 → <br />
A S2 → A 01<br />
A 01 → 0 A 01 1<br />
A 01 → #A 33<br />
A S1 → A S0 A 01<br />
A S1 → A S1 A 11<br />
...<br />
28
CFG=PDA<br />
• We have shown that a language is context-free<br />
iff some pushdown automaton recognizes it.<br />
• In particular all regular languages can be<br />
generated by CFGs <strong>and</strong> so can be recognized<br />
by PDA.<br />
• The class of languages accepted by nondeterministic<br />
PDAs is larger than those<br />
accepted by deterministic PDAs.<br />
29
The Context-free Languages<br />
DPDA<br />
<strong>the</strong> regular languages<br />
context-free languages<br />
30
Non context-free Languages<br />
• Consider <strong>the</strong> language L={a i b i c i |i0}.<br />
• When trying to build a push-down automaton<br />
that recognizes L, we can compare <strong>the</strong> number<br />
of a-'s with b-'s or c-'s but not both;<br />
• If we compared <strong>the</strong> number of a-'s to <strong>the</strong><br />
number of b-'s <strong>the</strong>n we can't compare c-'s with<br />
any of <strong>the</strong>m, as at this stage <strong>the</strong> stack (or<br />
counter) is empty.<br />
31
Non context-free Languages<br />
• So some languages seem to be not <strong>CFL</strong>.<br />
• The question is which<br />
• This can be determined using <strong>the</strong> <strong>pumping</strong><br />
<strong>lemma</strong> for context-free languages.<br />
32
The Pumping Lemma - background<br />
• Let L be a <strong>CFL</strong> <strong>and</strong> let G be a simple<br />
grammar (no unit/ rules) generating it.<br />
• Let wL be a long enough word (we will say<br />
later what is long).<br />
• The parsing tree of w contains a long path<br />
from S to some leaf (terminal).<br />
• On this long path some variable R must<br />
repeat (remember, w is long).<br />
33
The Pumping Lemma - background<br />
S<br />
R<br />
• Divide w into uvxyz<br />
according to <strong>the</strong> parse<br />
tree, as in <strong>the</strong> figure.<br />
• Each occurrence of R<br />
has a subtree under it.<br />
R<br />
u v x y z<br />
34
The Pumping Lemma - background<br />
• The upper occurrence of R has a larger subtree<br />
<strong>and</strong> generates vxy.<br />
• The lower occurrence of R has a smaller<br />
subtree <strong>and</strong> generates only x.<br />
• Both subtrees are generated by <strong>the</strong> same<br />
variable R.<br />
• That means if we substitute one for <strong>the</strong> o<strong>the</strong>r we<br />
will still obtain valid parse trees.<br />
S<br />
R<br />
R<br />
u v x y z<br />
35
S<br />
S<br />
R<br />
R<br />
u v y z<br />
u<br />
R<br />
x<br />
z<br />
v<br />
x<br />
y<br />
Replacing <strong>the</strong> smaller by<br />
<strong>the</strong> larger repeatedly<br />
generates <strong>the</strong> string<br />
uv i xy i z at each i>0.<br />
Replacing <strong>the</strong> larger by<br />
<strong>the</strong> smaller generates <strong>the</strong><br />
string uxz or uv i xy i z<br />
where i=0.<br />
Therefore, for all i0, w i = uv i xy i z is also in L<br />
36
The Pumping Length<br />
• That means that every <strong>CFL</strong> has a special<br />
value called <strong>the</strong> <strong>pumping</strong> length such that all<br />
strings longer than <strong>the</strong> <strong>pumping</strong> length can<br />
be "pumped".<br />
• The string can be divided into 5 parts<br />
w=uvxyz.<br />
• The second <strong>and</strong> fourth can be pumped to<br />
produce additional words in L.<br />
• for all k0, w k = uv k xy k z can also be<br />
generated by <strong>the</strong> grammar.<br />
37
Pumping Lemma for <strong>CFL</strong><br />
Lemma: Let L be a context-free language.<br />
There is a positive integer p (<strong>the</strong> <strong>pumping</strong><br />
length) such that for all strings wL with<br />
|w|p, w can be divided into five pieces<br />
w=uvxyz satisfying <strong>the</strong> following conditions:<br />
1. |vy|>0<br />
2. |vxy|p<br />
3. for each i0, uv i xy i zL<br />
38
Proof - value of p<br />
• First we find out <strong>the</strong> value of p.<br />
• Let G be a CFG for <strong>CFL</strong> L.<br />
• Let b be <strong>the</strong> maximum number of symbols in<br />
<strong>the</strong> right side of any rule in G.<br />
• So we know that in any parse tree of G a node<br />
can't have more than b children.<br />
• So if <strong>the</strong> height of a parsing tree for wL is h<br />
<strong>the</strong>n |w|< b h<br />
(h>log b |w|).<br />
A<br />
A 1 2 3 b<br />
<br />
1 2 3 b<br />
39
Proof - value of p<br />
• Let |V| be <strong>the</strong> number of variables in G.<br />
• We set p = b |V|+2 .<br />
(h>log b |p|).<br />
• Then for any string of length p <strong>the</strong> parse tree<br />
requires height at least |V|+2 (Note, b>1 since<br />
<strong>the</strong>re are no unit rules).<br />
• Given a string wL, s.t. |w| p , since G has<br />
only |V| variables, at least one of <strong>the</strong> variables<br />
repeats (height |V|+2 |V|+1 variables +<br />
terminal).<br />
• W.l.o.g. assume this variable is R<br />
40
Proof – condition 1<br />
• To prove condition 1 (|vy|>0) we have to show it<br />
is impossible that both v <strong>and</strong> y are .<br />
• We use a grammar without unit rules.<br />
• But <strong>the</strong> only way to have v=y=, is to have a<br />
rule R R, which is a unit rule. Contradiction.<br />
• So condition 1 is satisfied.<br />
S<br />
R<br />
R<br />
u v x y z<br />
41
Proof – condition 2<br />
• To prove condition 2 (|vxy|p) we will check<br />
<strong>the</strong> height of <strong>the</strong> subtree rooted in first R =<br />
<strong>the</strong> subtree that generates vxy.<br />
• Its height is at most |V|+2 (R was selected as<br />
a variable that has two occurrences within<br />
<strong>the</strong> bottom |V|+1 levels of <strong>the</strong> parsing tree).<br />
• So it can generate a string of length at most<br />
b |V|+2 .<br />
• Since p= b |V|+2 , condition 2 is satisfied.<br />
42
Proof – condition 3<br />
S<br />
S<br />
R<br />
R<br />
R<br />
x<br />
u v R y z<br />
u<br />
z<br />
v<br />
Replacing <strong>the</strong> smaller by<br />
<strong>the</strong> larger repeatedly<br />
generates <strong>the</strong> string<br />
uv i xy i z at each i>0.<br />
x<br />
y<br />
Replacing <strong>the</strong> larger by<br />
<strong>the</strong> smaller generates<br />
<strong>the</strong> string uxz or uv i xy i z<br />
where i=0.<br />
Therefore, for all i0, w i =uv i xy i z is also in L<br />
43
Usage of <strong>the</strong> <strong>lemma</strong><br />
• We use <strong>the</strong> <strong>pumping</strong> <strong>lemma</strong> to prove that a<br />
language is not context-free.<br />
General Structure:<br />
• Assume (by contradiction) that L is contextfree<br />
<strong>and</strong> <strong>the</strong>refore should fulfill <strong>the</strong> <strong>lemma</strong>.<br />
• Let p be <strong>the</strong> <strong>pumping</strong> length for L.<br />
• Select a word wL s.t. |w|>p.<br />
• Show that for any partition of w into five parts<br />
uvxyz such that |vy|>0 <strong>and</strong> |vxy|p, <strong>the</strong>re<br />
exists an i such that w i = uv i xy i z L<br />
• Contradiction!<br />
44
Usage of <strong>the</strong> <strong>lemma</strong> - Example<br />
• We use <strong>the</strong> <strong>pumping</strong> <strong>lemma</strong> to prove that <strong>the</strong><br />
language L={a n b n c n |n0} is not context-free.<br />
Proof:<br />
• Assume, by contradiction, that L is context-free<br />
thus satisfying <strong>the</strong> <strong>lemma</strong>.<br />
• Let p be <strong>the</strong> <strong>pumping</strong> length for L.<br />
• We select <strong>the</strong> string w= a p b p c p . wL <strong>and</strong> |w|>p,<br />
so it can be pumped.<br />
45
L={a n b n c n |n0}<br />
• Divide w into five parts uvxyz such that |vy|>0<br />
<strong>and</strong> |vxy|p.<br />
• There are two cases:<br />
1. u <strong>and</strong> y are both homogeneous.<br />
2. v or y is heterogeneous.<br />
46
L<br />
<br />
n n n<br />
p p p<br />
{ a b c | n 0}. Let w a b c uvxyz.<br />
a p b p c p<br />
Case 1:<br />
v<br />
y<br />
v <strong>and</strong> y are<br />
homogeneous<br />
v<br />
y<br />
v<br />
y<br />
v<br />
y<br />
v<br />
y<br />
Case 2<br />
v or y is<br />
heterogeneous<br />
or<br />
v<br />
y<br />
47
L={a n b n c n |n0}<br />
Case 1: v <strong>and</strong> y contain only one type of<br />
alphabet symbol. By choosing i=2 we get<br />
w 2 =uv 2 xy 2 z in which <strong>the</strong> number of<br />
appearances of one or two symbols<br />
increased while <strong>the</strong> third symbol remain<br />
unchanged. So w 2 cannot contain <strong>the</strong> same<br />
number of a's, b's <strong>and</strong> c's <strong>and</strong> w 2 L.<br />
48
L={a i b i c i |i0}<br />
Case 2: v or y contain two types of alphabet<br />
symbols. (Cannot have three because<br />
condition 2 holds). So if we choose i=2 <strong>the</strong>n<br />
<strong>the</strong> order of <strong>the</strong> symbols in v 2 or y 2 was<br />
destroyed <strong>and</strong> w 2 =uv 2 xy 2 z L.<br />
Conclusion: The assumption that L is contextfree<br />
is false. <strong>and</strong> L={a n b n c n |n0} is not <strong>CFL</strong>.<br />
49
Pumping Lemma – ano<strong>the</strong>r example<br />
Let L={ww| w{0,1}*}<br />
[ 00, 110110L ; 010, 0010L ]<br />
Prove that L is not context-free<br />
Proof: In class.<br />
50
Intersection 1<br />
<strong>CFL</strong> are not closed under intersection.<br />
Proof: by contradiction<br />
L 1 ={a n b n c k |n,k>0}<br />
L 2 ={a k b n c n | n,k>0}<br />
Both L 1 <strong>and</strong> L 2 are <strong>CFL</strong> but<br />
L 1 L 2 = {a j b j c j | j>0} is not a <strong>CFL</strong>.<br />
51
Intersection 2<br />
• <strong>CFL</strong> <strong>and</strong> RL are closed under intersection.<br />
Proof:<br />
Build a product automaton of a PDA <strong>and</strong> a DFA:<br />
Note that <strong>the</strong> resulting automaton is a PDA.<br />
input<br />
DFA<br />
PDA<br />
AND<br />
accept/reject<br />
stack<br />
52
Complement<br />
Theorem: <strong>CFL</strong> are not closed under complement.<br />
Proof: by contradiction.<br />
• Take two <strong>CFL</strong>s L 1 <strong>and</strong> L 2 <strong>and</strong> assume that ~L 1<br />
<strong>and</strong> ~L 2 are <strong>CFL</strong>. (~L denotes complement)<br />
• <strong>CFL</strong> are closed under union operation so<br />
~L 1 ~L 2 is a <strong>CFL</strong>.<br />
• Using our assumption again we get that<br />
~(~L 1 ~L 2 ) is a <strong>CFL</strong> as well.<br />
• But ~(~L 1 ~L 2 ) =L 1 L 2 <strong>and</strong> we already know<br />
that <strong>CFL</strong>s are not closed under intersection.<br />
53