ó
AögWc@s
dZd
Z
ddlZddlZddlmZ
dZiad„Zdef
d„ƒYZ	d	e	f
d
„ƒYZ
de	f
d„ƒYZd
„Zdef
d„ƒYZ
de
f
d„ƒYZde
f
d„ƒYZde
f
d„ƒYZde
f
d„ƒYZd„ZdS(sÌ
Python parse tree definitions.

This is a very concrete parse tree; we need to keep every token and
even the comments and whitespace between tokens.

There's also a pattern matching implementation here.
s#Guido van Rossum <guido@python.org>iÿÿÿÿN(
tStringIOiÿÿÿc
CshtsXd
dl
m}

x?|
jjƒD]+\}}t|ƒ
tkr&|t|<q&q&Wntj||ƒS(Ni
(
tpython_symbols(t_type_reprstpygramR
t__dict__titemsttypetintt
setdefault(ttype_numR
tnametval((s./lib2to3/pytree.pyt	type_reprs


tBasecBsûeZ
dZdZdZdZeZeZ	d
„Z
d„ZdZd„Z
d„Zd„Zd„Zd„Zd„Zd	„Zd
„Zd„Zd„Zd
„Zed„ƒ
Zed„ƒ
Zd„Zd„Zd„Zejdkrùd„Z nRS(sÂ
    Abstract base class for Node and Leaf.

    This provides some default functionality and boilerplate using the
    template pattern.

    A node may be a subnode of at most one parent.
    c
Os
tj
|ƒ
S(
s7Constructor that prevents Base from being instantiated.(tobjectt__new__(tclstargstkwds((s./lib2to3/pytree.pyR2scCs#|j|
jk	rt
S|j|
ƒ
S(
sW
        Compare two nodes for equality.

        This calls the method _eq().
        (t	__class__tNotImplementedt_eq(tselftother((s./lib2to3/pytree.pyt__eq__7s

cCs$|j|
jk	rt
S|j|
ƒ
S(
sY
        Compare two nodes for inequality.

        This calls the method _eq().
        (RRR(RR((s./lib2to3/pytree.pyt__ne__Cs

c
Cs
t‚
d
S(s_

        Compare two nodes for equality.

        This is called by __eq__ and __ne__.  It is only called if the two nodes
        have the same type.  This must be implemented by the concrete subclass.
        Nodes should be considered equal if they have the same structure,
        ignoring the prefix string and other context information.
        N(
tNotImplementedError(RR((s./lib2to3/pytree.pyRMs	c


Cs
t‚
d
S(sr
        Return a cloned (deep) copy of self.

        This must be implemented by the concrete subclass.
        N(
R(
R((s./lib2to3/pytree.pytcloneXsc


Cs
t‚
d
S(sx
        Return a post-order iterator for the tree.

        This must be implemented by the concrete subclass.
        N(
R(
R((s./lib2to3/pytree.pyt
post_order`sc


Cs
t‚
d
S(sw
        Return a pre-order iterator for the tree.

        This must be implemented by the concrete subclass.
        N(
R(
R((s./lib2to3/pytree.pyt	pre_orderhscCs#tj
d
tddƒ

|
|_dS(sv
        Set the prefix for the node (see Leaf class).

        DEPRECATED; use the prefix property directly.
        s3set_prefix() is deprecated; use the prefix propertyt
stackleveliN(twarningstwarntDeprecationWarningtprefix(RR"((s./lib2to3/pytree.pyt
set_prefixps	


c

Cstj
d
tddƒ

|jS(sy
        Return the prefix for the node (see Leaf class).

        DEPRECATED; use the prefix property directly.
        s3get_prefix() is deprecated; use the prefix propertyRi(RR R!R"(
R((s./lib2to3/pytree.pyt
get_prefixzs	


cCs¿t|
t
ƒs|
g
}
ng}t}xR|jjD]D}||krk|
d
k	rb|j|
ƒ

nt}q4|j|ƒ

q4W|jj	ƒ
||j_x|
D]}|j|_qœWd
|_d
S(s/Replace this node with a new one in the parent.N(
t
isinstancetlisttFalsetparenttchildrentNonetextendtTruetappendtchanged(Rtnewt
l_childrentfoundtcht
x((s./lib2to3/pytree.pytreplace„s






	





c
Cs=|}
x-t|
t
ƒs5|
js%d
S|
jd}
q	W|
jS(s9Return the line number which generated the invocant node.Ni(R%tLeafR)tlineno(Rtnode((s./lib2to3/pytree.pyt
get_lineno›s

	


c

Cs&|jr|jj
ƒ
nt|_dS(
N(R(R.R,twas_changed(
R((s./lib2to3/pytree.pyR.¤s
	

c
Csf|jrbxVt
|jjƒ
D]?\}
}||kr|jjƒ
|jj|
=d
|_|
SqWnd
S(s‰
        Remove the node from the tree. Returns the position of the node in its
        parent's children before it was removed.
        N(R(t	enumerateR)R.R*(Rt
iR7((s./lib2to3/pytree.pytremove©s	






	
c
Cst|jdkrdSxZt|jjƒ
D]F\}
}||kr&y|jj|
d
SWqltk
rh


dSXq&q&WdS(sŸ
        The node immediately following the invocant in their parent's children
        list. If the invocant does not have a next sibling, it is None
        i
N(R(R*R:R)t
IndexError(RR;tchild((s./lib2to3/pytree.pytnext_sibling¶s






c
Csh|jdkrdSxNt|jjƒ
D]:\}
}||kr&|
d
krNdS|jj|
dSq&WdS(s¤
        The node immediately preceding the invocant in their parent's children
        list. If the invocant does not have a previous sibling, it is None.
        ii
N(R(R*R:R)(RR;R>((s./lib2to3/pytree.pytprev_siblingÇs




c
cs4x-|jD]"}
x|
j
ƒD]}|V
qWq
WdS(
N(R)tleaves(RR>R3((s./lib2to3/pytree.pyRA×s


c

Cs$|jdkrd
Sd|jjƒS(Nii
(R(R*tdepth(
R((s./lib2to3/pytree.pyRBÜs


c
Cs |j}
|
dkrd
S|
jS(s
        Return the string immediately following the invocant node. This is
        effectively equivalent to node.next_sibling.prefix
        uN(R?R*R"(Rtnext_sib((s./lib2to3/pytree.pyt
get_suffixás	


iic

Cst|ƒ
j
d
ƒ
S(Ntascii(tunicodetencode(
R((s./lib2to3/pytree.pyt__str__ìs
N((ii(!t__name__t
__module__t__doc__R*RR(R)R'R9twas_checkedRRt__hash__RRRRRR#R$R4R8R.R<tpropertyR?R@RARBRDtsystversion_infoRH(((s./lib2to3/pytree.pyR
 s6	



		
	
					
	
					
			

tNodecBsªeZ
dZdddd
„Zd„Zd„ZejdkrHeZ	nd„Z
d„Zd„Zd	„Z
d
„Zd„ZeeeƒZd„Zd
„Zd„ZRS(s+Concrete implementation for interior nodes.cCsm|
|_t
|ƒ
|_x|jD]}||_q"W|d
k	rM||_n|r`||_n	d
|_d
S(sð
        Initializer.

        Takes a type constant (a symbol number >= 256), a sequence of
        child nodes, and an optional context keyword argument.

        As a side effect, the parent pointers of the children are updated.
        N(RR&R)R(R*R"tfixers_applied(RRR)tcontextR"RRR2((s./lib2to3/pytree.pyt__init__ós
	







c

Cs#d
|jj
t|jƒ
|jfS(s)Return a canonical string representation.s
%s(%s, %r)(RRIRRR)(
R((s./lib2to3/pytree.pyt__repr__
s

c

Csd
jt
t|jƒƒ
S(sk
        Return a pretty string representation.

        This reproduces the input source exactly.
        u(tjointmapRFR)(
R((s./lib2to3/pytree.pyt__unicode__
siicCs"|j|j
f|
j|
j
fkS(
sCompare two nodes for equality.(RR)(RR((s./lib2to3/pytree.pyR
sc
Cs5t|j
g|jD]}
|
jƒ^qd
|jƒ
S(s$Return a cloned (deep) copy of self.RR(RQRR)RRR(RR2((s./lib2to3/pytree.pyR!
s+
c
cs9x-|jD]"}
x|
j
ƒD]}|V
qWq
W|V
d
S(s*Return a post-order iterator for the tree.N(R)R(RR>R7((s./lib2to3/pytree.pyR&
s



c
cs9|V
x-|jD]"}
x|
j
ƒD]}|V
q"WqWd
S(s)Return a pre-order iterator for the tree.N(R)R(RR>R7((s./lib2to3/pytree.pyR-
s


c

Cs|js
d
S|jdj
S(sO
        The whitespace and comments preceding this node in the input.
        ti(R)R"(
R((s./lib2to3/pytree.pyt_prefix_getter4
s	

cCs |jr|
|jd
_
ndS(Ni(R)R"(RR"((s./lib2to3/pytree.pyt_prefix_setter<
s
	
cCs4||_d
|j|
_||j|
<|jƒ
d
S(s…
        Equivalent to 'node.children[i] = child'. This method also sets the
        child's parent attribute appropriately.
        N(R(R*R)R.(RR;R>((s./lib2to3/pytree.pyt	set_childB
s	



cCs*||_|j
j|
|ƒ
|jƒ
d
S(s‹
        Equivalent to 'node.children.insert(i, child)'. This method also sets
        the child's parent attribute appropriately.
        N(R(R)tinsertR.(RR;R>((s./lib2to3/pytree.pytinsert_childL
s	

cCs'||
_|j
j|
ƒ

|jƒ
d
S(sˆ
        Equivalent to 'node.children.append(child)'. This method also sets the
        child's parent attribute appropriately.
        N(R(R)R-R.(RR>((s./lib2to3/pytree.pytappend_childU
s	

N(ii(RIRJRKR*RTRURXRORPRHRRRRRZR[RNR"R\R^R_(((s./lib2to3/pytree.pyRQïs$

		
								
		R5cBsªeZ
dZd
ZdZdZddgd„Zd„Zd„Z	e
jdkrZe	Znd„Z
d„Zd	„Zd
„Zd„Zd„Zd
„ZeeeƒZRS(s'Concrete implementation for leaf nodes.RYicCsb|d
k	r*|\|_
\|_|_n|
|_||_|d
k	rT||_
n||_d
S(s—
        Initializer.

        Takes a type constant (a token number < 256), a string value, and an
        optional context keyword argument.
        N(R*t_prefixR6tcolumnRtvalueRR(RRRbRSR"RR((s./lib2to3/pytree.pyRTh
s

	
	


c

Csd
|jj
|j|jfS(s)Return a canonical string representation.s
%s(%r, %r)(RRIRRb(
R((s./lib2to3/pytree.pyRU{
s

c

Cs|jt
|jƒ
S(
sk
        Return a pretty string representation.

        This reproduces the input source exactly.
        (R"RFRb(
R((s./lib2to3/pytree.pyRX
sicCs"|j|j
f|
j|
j
fkS(
sCompare two nodes for equality.(RRb(RR((s./lib2to3/pytree.pyRŒ
sc

Cs4t|j
|j|j|j|jffd
|jƒ
S(s$Return a cloned (deep) copy of self.RR(R5RRbR"R6RaRR(
R((s./lib2to3/pytree.pyR
s

c


cs	|V
dS(
N((
R((s./lib2to3/pytree.pyRA–
s
c


cs	|V
d
S(s*Return a post-order iterator for the tree.N((
R((s./lib2to3/pytree.pyR™
sc


cs	|V
d
S(s)Return a pre-order iterator for the tree.N((
R((s./lib2to3/pytree.pyR
sc


Cs|jS(
sP
        The whitespace and comments preceding this token in the input.
        (
R`(
R((s./lib2to3/pytree.pyRZ¡
scCs|jƒ
|
|_
dS(
N(R.R`(RR"((s./lib2to3/pytree.pyR[§
s


N(ii(RIRJRKR`R6RaR*RTRURXRORPRHRRRARRRZR[RNR"(((s./lib2to3/pytree.pyR5_
s&



		
								cCsk|
\}}}}|s'||jkrTt
|ƒ
d
krA|dSt||d|ƒ
St||d|ƒ
SdS(sõ
    Convert raw node information to a Node or Leaf instance.

    This is passed to the parser driver which calls it whenever a reduction of a
    grammar rule produces a new complete node, so that the tree is build
    strictly bottom-up.
    i
iRSN(t
number2symboltlenRQR5(tgrtraw_nodeRRbRSR)((s./lib2to3/pytree.pytconvert­
s


tBasePatterncBs\eZ
dZdZdZdZd
„Zd„Zd„Z	dd„
Z
dd„
Zd„ZRS(s”

    A pattern is a tree matching pattern.

    It looks for a specific node type (token or symbol), and
    optionally for a specific content.

    This is an abstract base class.  There are three concrete
    subclasses:

    - LeafPattern matches a single leaf node;
    - NodePattern matches a single node (usually non-leaf);
    - WildcardPattern matches a sequence of nodes of variable length.
    c
Os
tj
|ƒ
S(
s>Constructor that prevents BasePattern from being instantiated.(RR(RRR((s./lib2to3/pytree.pyRÕ
sc
Csht|j
ƒ
|j|jg}
x!|
rA|
d
dkrA|
d
=q!Wd|jjdjtt	|
ƒƒ
fS(Niÿÿÿÿs%s(%s)s, (
RRtcontentR
R*RRIRVRWtrepr(RR((s./lib2to3/pytree.pyRUÚ
s



c


Cs|S(
sŒ
        A subclass can define this as a hook for optimizations.

        Returns either self or another node with the same effect.
        ((
R((s./lib2to3/pytree.pytoptimizeà
scCs§|jd
k	r%|
j|jkr%tS|jd
k	r~d
}|d
k	rOi}n|j|
|ƒsetS|r~|j|ƒ

q~n|d
k	r£|jr£|
||j<ntS(s#

        Does this pattern exactly match a node?

        Returns True if it matches, False if not.

        If results is not None, it must be a dict which will be
        updated with the nodes matching named subpatterns.

        Default implementation for non-wildcard patterns.
        N(RR*R'Rit	_submatchtupdateR
R,(RR7tresultst
r((s./lib2to3/pytree.pytmatchè
s!




	






cCs*t|
ƒ
d
krt
S|j|
d|ƒS(s
        Does this pattern exactly match a sequence of nodes?

        Default implementation for non-wildcard patterns.
        i
i(RdR'Rp(RtnodesRn((s./lib2to3/pytree.pyt	match_seq
s

ccs4i}|
r0|j|
d
|ƒr0d|fV
ndS(s}
        Generator yielding all matches for this pattern.

        Default implementation for non-wildcard patterns.
        ii
N(
Rp(RRqRo((s./lib2to3/pytree.pytgenerate_matchess

N(
RIRJRKR*RRiR
RRURkRpRrRs(((s./lib2to3/pytree.pyRhÀ
s

			
tLeafPatterncBs2eZ
dddd„Zdd
„
Zdd„
ZRS(cCs=|
d
k	rn|d
k	rn|
|_
||_||_d
S(sp

        Initializer.  Takes optional type, content, and name.

        The type, if given must be a token type (< 256).  If not given,
        this matches any *leaf* node; the content may still be required.

        The content, if given, must be a string.

        If a name is given, the matching node is stored in the results
        dict under that key.
        N(R*RRiR
(RRRiR
((s./lib2to3/pytree.pyRTs



	
	
cCs&t|
t
ƒstStj||
|ƒS(
s*Override match() to insist on a leaf node.(R%R5R'RhRp(RR7Rn((s./lib2to3/pytree.pyRp,s

cCs|j|
j
kS(
s„

        Match the pattern's content to the node's children.

        This assumes the node type matches and self.content is not None.

        Returns True if it matches, False if not.

        If results is not None, it must be a dict which will be
        updated with the nodes matching named subpatterns.

        When returning False, the results dict may still be updated.
        (RiRb(RR7Rn((s./lib2to3/pytree.pyRl2s
N(RIRJR*RTRpRl(((s./lib2to3/pytree.pyRtstNodePatterncBs,eZ
eZdddd„Zdd
„
ZRS(cCs|
d
k	rn|d
k	rbt
|ƒ
}x8t|ƒ
D]'\}}t|tƒr4t|_q4q4Wn|
|_||_||_	d
S(sd
        Initializer.  Takes optional type, content, and name.

        The type, if given, must be a symbol type (>= 256).  If the
        type is None this matches *any* single node (leaf or not),
        except if content is not None, in which it only matches
        non-leaf nodes that also match the content pattern.

        The content, if not None, must be a sequence of Patterns that
        must match the node's children exactly.  If the content is
        given, the type must not be None.

        If a name is given, the matching node is stored in the results
        dict under that key.
        N(
R*R&R:R%tWildcardPatternR,t	wildcardsRRiR
(RRRiR
R;titem((s./lib2to3/pytree.pyRTFs




	
	
cCsÊ|jrhxXt
|j|
jƒD]A\}}|t|
jƒ
kr|d
k	r\|j|ƒ

ntSqWtSt|jƒ
t|
jƒ
krŠtSx9t	|j|
jƒD]"\}}|j
||ƒs tSq WtS(s„

        Match the pattern's content to the node's children.

        This assumes the node type matches and self.content is not None.

        Returns True if it matches, False if not.

        If results is not None, it must be a dict which will be
        updated with the nodes matching named subpatterns.

        When returning False, the results dict may still be updated.
        N(RwRsRiR)RdR*RmR,R'tzipRp(RR7Rnt
cRot
subpatternR>((s./lib2to3/pytree.pyRlcs
	
"







"


N(RIRJR'RwR*RTRl(((s./lib2to3/pytree.pyRuBsRvcBsheZ
dZd
d
ed
d„Zd„Zd
d„
Zd
d„
Zd„Z	d„Z
d„Zd	„ZRS(s

    A wildcard pattern can match zero or more nodes.

    This has all the flexibility needed to implement patterns like:

    .*      .+      .?      .{m,n}
    (a b c | d e | f)
    (...)*  (...)+  (...)?  (...){m,n}

    except it always uses non-greedy matching.
    icCs]|
d
k	r5t
tt
|
ƒƒ
}
x|
D]}q(Wn|
|_||_||_||_d
S(sÏ
        Initializer.

        Args:
            content: optional sequence of subsequences of patterns;
                     if absent, matches one node;
                     if present, each subsequence is an alternative [*]
            min: optional minimum number of times to match, default 0
            max: optional maximum number of times to match, default HUGE
            name: optional name assigned to this match

        [*] Thus, if content is [[a, b, c], [d, e], [f, g, h]] this is
            equivalent to (a b c | d e | f g h); if content is None,
            this is equivalent to '.' in regular expression terms.
            The min and max parameters work as follows:
                min=0, max=maxint: .*
                min=1, max=maxint: .+
                min=0, max=1: .?
                min=1, max=1: .
            If content is not None, replace the dot with the parenthesized
            list of alternatives, e.g. (a b c | d e | f g h)*
        N(R*ttupleRWRitmintmaxR
(RRiR}R~R
talt((s./lib2to3/pytree.pyRTs



	
	
	
c
Cs/
d}
|j
dk	rWt|j
ƒ
d
krWt|j
dƒ
d
krW|j
dd}
n|jd
kr¿|jd
kr¿|j
dkr”td|jƒ
S|
dk	r¿|j|
jkr¿|
jƒSn|jd
k
r+
t|
t	ƒr+
|
jd
k
r+
|j|
jkr+
t	|
j
|j|
j|j|
j|
jƒS|S(s+Optimize certain stacked wildcard patterns.i
iR
N(
R*RiRdR}R~RuR
RkR%Rv(RR{((s./lib2to3/pytree.pyRk°s 

.








!
	






cCs|j|
g
|ƒS(
s'Does this pattern exactly match a node?(
Rr(RR7Rn((s./lib2to3/pytree.pyRpÃscCsuxn|j|
ƒ
D]]\}}|t
|
ƒ
kr|d
k	ri|j|ƒ

|jrit|
ƒ
||j<qintSqWtS(s4Does this pattern exactly match a sequence of nodes?N(RsRdR*RmR
R&R,R'(RRqRnRzRo((s./lib2to3/pytree.pyRrÇs




	


ccsŒ
|jdkrsxv
t|jd
tt|
ƒ
|jƒƒD]4}i}|jra|
| ||j<n||fV
q8Wn
|jdkr“|j|
ƒ
V
nõtt	dƒrºt	j
}tƒt	_
nz¯yOxH|j|
dƒD]4\}}|jrü|
| ||j<n||fV
qÓWWnYt
k
rg



xI|j|
ƒ
D]4\}}|jrU
|
| ||j<n||fV
q,
Wn
XWdtt	dƒr‡
|t	_
nXdS(s"

        Generator yielding matches for a sequence of nodes.

        Args:
            nodes: sequence of nodes

        Yields:
            (count, results) tuples where:
            count: the match comprises nodes[:count];
            results: dict containing named submatches.
        i
t	bare_nametgetrefcountiN(RiR*txrangeR}RdR~R
t_bare_name_matchesthasattrROtstderrRt_recursive_matchestRuntimeErrort_iterative_matches(RRqtcountRotsave_stderr((s./lib2to3/pytree.pyRsÒs0/

	




	



	




	


ccs`
t|
ƒ
}d
|j
kr)d
ifV
ng}xO|jD]D}x;t||
ƒD]*\}}||fV
|j||fƒ

qOWq9WxØ|r[
g}x¿|D]·\}}	||kr—||jk
r—x|jD]}xvt||
|ƒD]a\}
}|
d
krâi}|j|	ƒ

|j|ƒ

||
|fV
|j||
|fƒ

qâqâWqÈWq—q—W|}q„WdS(s(Helper to iteratively yield the matches.iN(RdR}RiRsR-R~Rm(RRqtnodelenRnRRzRotnew_resultstc0tr0tc1tr1((s./lib2to3/pytree.pyRˆÿs*




	



 







)
cCs›d
}i}t}t
|
ƒ
}x_|r||krt}x?|jD]4}|d
j|
||ƒrD|d7}t}PqDqDWq!W|
| ||j<||fS(s(Special optimized matcher for bare_name.ii
(R'RdR,RiRpR
(RRqR‰RotdoneR~tleaf((s./lib2to3/pytree.pyRƒs












c	csÁ||jkrd
ifV
n||j
kr½xŽ|jD]€}xwt||
ƒD]f\}}xW|j|
||dƒD];\}}i}|j|ƒ

|j|ƒ

|||fV
qsWqLWq6WndS(s(Helper to recursively yield the matches.ii
N(R}R~RiRsR†Rm(	RRqR‰RRRŽRRRo((s./lib2to3/pytree.pyR†,s




'





N(
RIRJRKR*tHUGERTRkRpRrRsRˆRƒR†(((s./lib2to3/pytree.pyRvs#		-		tNegatedPatterncBs/eZ
dd„
Zd
„Zd„Zd„ZRS(cCs|
d
k	rn|
|_
d
S(s

        Initializer.

        The argument is either a pattern or None.  If it is None, this
        only matches an empty sequence (effectively '$' in regex
        lingo).  If it is not None, this matches whenever the argument
        pattern doesn't have any matches.
        N(R*Ri(RRi((s./lib2to3/pytree.pyRT=s	

c
CstS(
N(
R'(RR7((s./lib2to3/pytree.pyRpJscCst|
ƒ
d
kS(Ni(
Rd(RRq((s./lib2to3/pytree.pyRrNsccse|jdkr2t|
ƒ
d
krad
ifV
qan/x!|jj|
ƒ
D]
\}}dSWd
ifV
dS(Ni(RiR*RdRs(RRqRzRo((s./lib2to3/pytree.pyRsRs



N(RIRJR*RTRpRrRs(((s./lib2to3/pytree.pyR”;s
		c	cs´|sd
ifV
nœ|d
|d}}x„|j|
ƒ
D]s\}}|sY||fV
q9xPt||
|ƒD];\}}i}|j
|ƒ

|j
|ƒ

|||fV
qmWq9WdS(sR

    Generator yielding matches for a sequence of patterns and nodes.

    Args:
        patterns: a sequence of patterns
        nodes: a sequence of nodes

    Yields:
        (count, results) tuples where:
        count: the entire sequence of patterns matches nodes[:count];
        results: dict containing named submatches.
        ii
N(RsRm(	tpatternsRqt
ptrestRRŽRRRo((s./lib2to3/pytree.pyRs^s




 





(RKt
__author__RORRR“RRRR
RQR5RgRhRtRuRvR”Rs(((s./lib2to3/pytree.pyt<module>s"


	
ÏpN	V,=¼#