$8Zc@sdZddklZdZddkZddkZddkZddkZddkZddk Z ddk l Z ddk l Z lZlZddklZlZed Zd efd YZd Zd ZdZdZeiddfjo+ddkZeiZdZdZ neZeZeZ dZ!defdYZ"de#fdYZ$defdYZ%de$fdYZ&dS(sRefactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. i(twith_statements#Guido van Rossum N(tchaini(tdriverttokenizettoken(tpytreetpygramcCst|ggdg}tii|i}g}xgtti|D]P}|ido:|ido*|o|d}n|i |d qIqIW|S(sEReturn a sorted list of all available fix names in the given package.t*tfix_s.pyii( t __import__tostpathtdirnamet__file__tsortedtlistdirt startswithtendswithtappend(t fixer_pkgt remove_prefixtpkgt fixer_dirt fix_namestname((s&/usr/lib/python2.6/lib2to3/refactor.pytget_all_fix_namess t _EveryNodecBseZRS((t__name__t __module__(((s&/usr/lib/python2.6/lib2to3/refactor.pyR+scCst|titifo+|idjo tnt|igSt|tio"|i ot |i Stnt|ti oFt}x5|i D]*}x!|D]}|i t |qWqW|St d|dS(sf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. s$Oh no! I don't understand pattern %sN(t isinstanceRt NodePatternt LeafPatternttypetNoneRtsettNegatedPatterntcontentt_get_head_typestWildcardPatterntupdatet Exception(tpattrtptx((s&/usr/lib/python2.6/lib2to3/refactor.pyR%/s"     cCstit}g}x|D]}|io\yt|i}Wn tj o|i|qXxX|D]}||i|qiWq|idj o||ii|q|i|qWx:t t i i i t i iD]}||i|qWt|S(s^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N(t collectionst defaultdicttlisttpatternR%RRt _accept_typeR!RRtpython_grammart symbol2numbert itervaluesttokenstextendtdict(t fixer_listt head_nodesteverytfixertheadst node_type((s&/usr/lib/python2.6/lib2to3/refactor.pyt_get_headnode_dictKs(  cCs0g}t|tD]}||d|q~S(sN Return the fully qualified names for fixers in the package pkg_name. t.(RtFalse(tpkg_namet_[1]tfix_name((s&/usr/lib/python2.6/lib2to3/refactor.pytget_fixers_from_packagedscCs|S(N((tobj((s&/usr/lib/python2.6/lib2to3/refactor.pyt _identityksiicCs|iddS(Nu u (treplace(tinput((s&/usr/lib/python2.6/lib2to3/refactor.pyt_from_system_newlinesrscCs,tidjo|idtiS|SdS(Ns u (R tlinesepRG(RH((s&/usr/lib/python2.6/lib2to3/refactor.pyt_to_system_newlinestscst}titi|ifd}ttititi f}t }yx}t ou|\}}||joq]q]|ti jo|oPnt }q]|ti jo|djo |\}}|ti jp |djoPn|\}}|ti jp |djoPn|\}}|tijo |djo|\}}nxg|ti joQ|i||\}}|tijp |djoPn|\}}qnWq]Pq]WWntj onXt|S(Ncsi}|d|dfS(Nii(tnext(ttok(tgen(s&/usr/lib/python2.6/lib2to3/refactor.pytadvances ufromu __future__uimportu(u,(R@Rtgenerate_tokenstStringIOtreadlinet frozensetRtNEWLINEtNLtCOMMENTR"tTruetSTRINGtNAMEtOPtaddt StopIteration(tsourcethave_docstringROtignoretfeaturesttptvalue((RNs&/usr/lib/python2.6/lib2to3/refactor.pyt_detect_future_featuressH     t FixerErrorcBseZdZRS(sA fixer could not be loaded.(RRt__doc__(((s&/usr/lib/python2.6/lib2to3/refactor.pyRdstRefactoringToolcBseZhed6ZdZdZdddZdZdZ dZ dZ dZ eed Z eed Zd Zeed Zd ZedZdZdZdeddZddZdZdZdZdZdZdZdZdZRS(tprint_functiontFixRcCs||_|pg|_|ii|_|dj o|ii|n|idoti|_ n ti |_ g|_ t i d|_g|_t|_ti|i dtid|i|_|i\|_|_t|i|_t|i|_g|_dS(sInitializer. Args: fixer_names: a list of fixers to import options: an dict with configuration. explicit: a list of fixers to run even if they are explicit. RgRftconverttloggerN(tfixerstexplicitt_default_optionstcopytoptionsR!R'Rt!python_grammar_no_print_statementtgrammarR2terrorstloggingt getLoggerRjt fixer_logR@twroteRtDriverRRit get_fixerst pre_ordert post_orderR>tpre_order_headstpost_order_headstfiles(tselft fixer_namesRoRl((s&/usr/lib/python2.6/lib2to3/refactor.pyt__init__s&       c Csg}g}x|iD]}t|hhdg}|iddd}|i|io|t|i}n|id}|idig}|D]}||i q~} yt || } Wn)t j ot d|| fnX| |i |i} | io7|itj o'||ijo|id|qn|id || id jo|i| q| id jo|i| qt d | iqWtid } |id| |id| ||fS(sInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. RR?iit_tsCan't find %s.%ssSkipping implicit fixer: %ssAdding transformation: %stpretpostsIllegal fixer order: %rt run_ordertkey(RkR trsplitRt FILE_PREFIXtlentsplitt CLASS_PREFIXtjointtitletgetattrtAttributeErrorRdRoRuRlRWt log_messaget log_debugtorderRtoperatort attrgettertsort( R~tpre_order_fixerstpost_order_fixerst fix_mod_pathtmodRCtpartsRBR+t class_namet fix_classR;tkey_func((s&/usr/lib/python2.6/lib2to3/refactor.pyRxs: 7cOsdS(sCalled when an error occurs.N((R~tmsgtargstkwds((s&/usr/lib/python2.6/lib2to3/refactor.pyt log_errorscGs)|o||}n|ii|dS(sHook to log a message.N(Rjtinfo(R~RR((s&/usr/lib/python2.6/lib2to3/refactor.pyRscGs)|o||}n|ii|dS(N(Rjtdebug(R~RR((s&/usr/lib/python2.6/lib2to3/refactor.pyRscCsdS(sTCalled with the old version, new version, and filename of a refactored file.N((R~told_texttnew_texttfilenametequal((s&/usr/lib/python2.6/lib2to3/refactor.pyt print_outputscCsRxK|D]C}tii|o|i|||q|i|||qWdS(s)Refactor a list of files and directories.N(R R tisdirt refactor_dirt refactor_file(R~titemstwritet doctests_onlyt dir_or_file((s&/usr/lib/python2.6/lib2to3/refactor.pytrefactor s c Csxti|D]\}}}|id||i|ixk|D]c}|id oLtii|dido,tii||}|i |||qJqJWg} |D]!} | idp | | qq~ |(qWdS(sDescends down a directory and refactor every Python file found. Python files are assumed to have a .py extension. Files and subdirectories starting with '.' are skipped. sDescending into %sR?itpyN( R twalkRRRR tsplitextRRR( R~tdir_nameRRtdirpathtdirnamest filenamesRtfullnameRBtdn((s&/usr/lib/python2.6/lib2to3/refactor.pyRs   c Csyt|d}Wn)tj o}|id||dSXzti|id}Wd|iXt|dd|i i }z#|~}t |i |fSWdQXdS(sG Do our best to decode a Python source file correctly. trbsCan't open %s: %siNR*tencoding(NN( topentIOErrorRR!Rtdetect_encodingRRtcloset_open_with_encodingt__exit__t __enter__RItread(R~RtfterrRRB((s&/usr/lib/python2.6/lib2to3/refactor.pyt_read_python_source(s ,cCs|i|\}}|djodS|d7}|o`|id||i||}||jo|i|||||q|id|n^|i||}|o4|io*|it|d |d|d|n|id|dS( sRefactors a file.Nu sRefactoring doctests in %ssNo doctest changes in %siRRsNo changes in %s(RR!Rtrefactor_docstringtprocessed_filetrefactor_stringt was_changedtunicode(R~RRRRHRtoutputttree((s&/usr/lib/python2.6/lib2to3/refactor.pyR8s   c Cst|}d|joti|i_nzOy|ii|}Wn2tj o&}|id||ii |dSXWd|i|i_X||_ |i d||i |||S(sFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. RgsCan't parse %s: %s: %sNsRefactoring %s( RcRRpRRqt parse_stringR(Rt __class__Rtfuture_featuresRt refactor_tree(R~tdataRR`RR((s&/usr/lib/python2.6/lib2to3/refactor.pyROs     cCstii}|oT|id|i|d}||jo|i|d|q|idnN|i|d}|o'|io|it|d|n|iddS(NsRefactoring doctests in stdinssNo doctest changes in stdinsNo changes in stdin( tsyststdinRRRRRRR(R~RRHRR((s&/usr/lib/python2.6/lib2to3/refactor.pytrefactor_stdinjs  cCsx-t|i|iD]}|i||qW|i|i|i|i|i|ix-t|i|iD]}|i||qxW|iS(sARefactors a parse tree (modifying the tree in place). Args: tree: a pytree.Node instance representing the root of the tree to be refactored. name: a human-readable name for this tree. Returns: True if the tree was modified, False otherwise. ( RRyRzt start_treet traverse_byR{R|t finish_treeR(R~RRR;((s&/usr/lib/python2.6/lib2to3/refactor.pyRzs cCs|pdSxv|D]n}xe||iD]V}|i|}|o:|i||}|dj o|i||}q}q'q'WqWdS(sTraverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N(R tmatcht transformR!RG(R~Rkt traversaltnodeR;tresultstnew((s&/usr/lib/python2.6/lib2to3/refactor.pyRs   cCs|ii||djo)|i|d}|djodSn||j}|i|||||o|id|dS|o|i||||n|id|dS(sP Called when a file has been refactored, and there are changes. iNsNo changes to %ssNot writing changes to %s(R}RR!RRRt write_file(R~RRRRRR((s&/usr/lib/python2.6/lib2to3/refactor.pyRs    c Csyt|dd|}Wn,tij o}|id||dSXzIy|it|Wn+tij o}|id||nXWd|iX|id|t|_ dS(sWrites a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. twRsCan't create %s: %sNsCan't write %s: %ssWrote changes to %s( RR terrorRRRKRRRWRv(R~RRRRRR((s&/usr/lib/python2.6/lib2to3/refactor.pyRs s>>> s... c Csg}d}d}d}d}x5|itD]$}|d7}|ii|io_|dj o#|i|i||||n|}|g}|i|i} || }q.|dj oF|i||i p|||i i djo|i |q.|dj o#|i|i||||nd}d}|i |q.W|dj o#|i|i||||ndi |S(sRefactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) iiu uN( R!t splitlinesRWtlstripRtPS1R6trefactor_doctesttfindtPS2trstripRR( R~RHRtresulttblockt block_linenotindenttlinenotlineti((s&/usr/lib/python2.6/lib2to3/refactor.pyRs<       c Csy|i|||}Wnytj om}|iitio.x+|D]}|id|idqIWn|id|||i i ||SX|i ||ot |i t}||d ||d} }| dg|djp t| |didp|dcd7>>" (possibly indented), while the remaining lines start with "..." (identically indented). s Source: %su s+Can't parse docstring in %s line %s: %s: %siii(t parse_blockR(tlogt isEnabledForRstDEBUGRRRRRRRRRWtAssertionErrorRRtpopR( R~RRRRRRRRtclippedRB((s&/usr/lib/python2.6/lib2to3/refactor.pyRs(! "8c Cs'|io d}nd}|ip|id|n2|id|x|iD]}|i|qRW|io2|idx"|iD]}|i|qWn|iott|idjo|idn|idt|ix1|iD]"\}}}|i|||qWndS( Ntweres need to besNo files %s modified.sFiles that %s modified:s$Warnings/messages while refactoring:isThere was 1 error:sThere were %d errors:(RvR}RRuRrR(R~RtfiletmessageRRR((s&/usr/lib/python2.6/lib2to3/refactor.pyt summarizes*         cCs1|ii|i|||}t|_|S(sParses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. (Rt parse_tokenst wrap_toksRSR(R~RRRR((s&/usr/lib/python2.6/lib2to3/refactor.pyR1s! c csti|i||i}xe|D]]\}}\}}\} } } ||d7}| |d7} ||||f| | f| fVq%WdS(s;Wraps a tokenize stream to systematically modify start/end.iN(RRPt gen_linesRL( R~RRRR5R Rbtline0tcol0tline1tcol1t line_text((s&/usr/lib/python2.6/lib2to3/refactor.pyR;s !ccs||i}||i}|}xm|D]e}|i|o|t|Vn7||idjo dVntd||f|}q'Wxto dVqWdS(sGenerates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. u sline=%r, prefix=%rRN(RRRRRRRW(R~RRtprefix1tprefix2tprefixR((s&/usr/lib/python2.6/lib2to3/refactor.pyRIs    N(RRR@RmRRR!RRxRRRRRRRRRRRRRRRRRRRRRR(((s&/usr/lib/python2.6/lib2to3/refactor.pyRfs8  (            +   tMultiprocessingUnsupportedcBseZRS((RR(((s&/usr/lib/python2.6/lib2to3/refactor.pyR]stMultiprocessRefactoringToolcBs5eZdZeeddZdZdZRS(cOs/tt|i||d|_d|_dS(N(tsuperRRR!tqueuet output_lock(R~Rtkwargs((s&/usr/lib/python2.6/lib2to3/refactor.pyRcs ic Csv|djott|i|||Syddk}Wntj o tnX|idj otdn|i |_|i |_ g}t |D]}||i d|iq~}z;x|D]} | iqWtt|i|||Wd|iix$t |D]}|iidq"Wx)|D]!} | io| iqCqCWd|_XdS(Niis already doing multiple processesttarget(RRRtmultiprocessingt ImportErrorRR R!t RuntimeErrort JoinableQueuetLockR txrangetProcesst_childtstartRtputtis_alive( R~RRRt num_processesR RBRt processesR+((s&/usr/lib/python2.6/lib2to3/refactor.pyRhs8  /    cCsq|ii}x[|dj oM|\}}ztt|i||Wd|iiX|ii}qWdS(N(R tgetR!RRRt task_done(R~ttaskRR ((s&/usr/lib/python2.6/lib2to3/refactor.pyRs  cOsG|idj o|ii||fntt|i||SdS(N(R R!RRRR(R~RR ((s&/usr/lib/python2.6/lib2to3/refactor.pyRs(RRRR@RRR(((s&/usr/lib/python2.6/lib2to3/refactor.pyRas    ('Ret __future__Rt __author__R RRsRR-RQt itertoolsRtpgen2RRRRRRRWRR(RR%R>RDRFt version_infotcodecsRRRIRKRcRdtobjectRfRR(((s&/usr/lib/python2.6/lib2to3/refactor.pyt s>                (