U `a[O @szdZddlZddlmZddlmZmZmZddlm Z m Z m Z ddgZ GdddeZ iZed ZGd ddeZdS) z pygments.lexers.special ~~~~~~~~~~~~~~~~~~~~~~~ Special lexers. :copyright: Copyright 2006-2017 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. N)Lexer)TokenErrorText)get_choice_opt text_typeBytesIO TextLexer RawTokenLexerc@s:eZdZdZdZdgZdgZdgZdZddZ d d Z d S) r z3 "Null" lexer, doesn't highlight anything. z Text onlytextz*.txtz text/plaing{Gz?ccsdt|fVdS)Nr)r)selfr r 9/usr/lib/python3/dist-packages/pygments/lexers/special.pyget_tokens_unprocessed sz TextLexer.get_tokens_unprocessedcCstjS)N)r priority)r r r r analyse_text#szTextLexer.analyse_textN) __name__ __module__ __qualname____doc__namealiases filenames mimetypesrrrr r r rr ss.*? c@s<eZdZdZdZdgZgZdgZddZddZ d d Z d S) r aq Recreate a token stream formatted with the `RawTokenFormatter`. This lexer raises exceptions during parsing if the token stream in the file is malformed. Additional options accepted: `compress` If set to ``"gz"`` or ``"bz2"``, decompress the token stream with the given compression algorithm before lexing (default: ``""``). zRaw token datarawzapplication/x-pygments-tokenscKs*t|dddddgd|_tj|f|dS)NcompressZnonegzbz2)rrr__init__)r Zoptionsr r rr<s  zRawTokenLexer.__init__ccst|tr|d}|jdkrDddl}|dddt|}|}n|jdkr`ddl}| |}| dd}| |D]\}}}||fVqxdS) Nasciirrrrb r ) isinstancerencodergzipZGzipFilerreadr decompressstripr)r r r&Zgzipfileritvr r r get_tokensAs      zRawTokenLexer.get_tokensc csd}t|D]}z|dd\}}Wn(tk rR|dd}t}YnlXt|}|st }|ddd}|D]&}|r|d stdt ||}q||t|<|dd d }|||fV|t |7}qdS) Nr r replace.zmalformed token namezunicode-escape) line_refinditergroupsplit ValueErrordecoder _ttype_cachegetrisuppergetattrlen) r r ZlengthmatchZttypestrvalZttypeZttypesZttype_r r rrSs&    z$RawTokenLexer.get_tokens_unprocessedN) rrrrrrrrrr-rr r r rr +s )rreZpygments.lexerrZpygments.tokenrrrZ pygments.utilrrr__all__r r:compiler4r r r r rs