관리-도구
편집 파일: universaldetector.cpython-310.pyc
o ƚRe�0 � @ s� d Z ddlZddlZddlZddlmZ ddlmZmZm Z ddl mZ ddlm Z ddlmZ dd lmZ G d d� de�ZdS )a Module containing the UniversalDetector detector class, which is the primary class a user of ``chardet`` should use. :author: Mark Pilgrim (initial port to Python) :author: Shy Shalom (original C code) :author: Dan Blanchard (major refactoring for 3.0) :author: Ian Cordasco � N� )�CharSetGroupProber)� InputState�LanguageFilter�ProbingState)�EscCharSetProber)�Latin1Prober)�MBCSGroupProber)�SBCSGroupProberc @ sn e Zd ZdZdZe�d�Ze�d�Ze�d�Z dddd d ddd d�Z ejfdd�Z dd� Zdd� Zdd� ZdS )�UniversalDetectoraq The ``UniversalDetector`` class underlies the ``chardet.detect`` function and coordinates all of the different charset probers. To get a ``dict`` containing an encoding and its confidence, you can simply run: .. code:: u = UniversalDetector() u.feed(some_bytes) u.close() detected = u.result g�������?s [�-�]s (|~{)s [�-�]zWindows-1252zWindows-1250zWindows-1251zWindows-1256zWindows-1253zWindows-1255zWindows-1254zWindows-1257)z iso-8859-1z iso-8859-2z iso-8859-5z iso-8859-6z iso-8859-7z iso-8859-8z iso-8859-9ziso-8859-13c C sN d | _ g | _d | _d | _d | _d | _d | _|| _t� t �| _d | _| � � d S )N)�_esc_charset_prober�_charset_probers�result�done� _got_data�_input_state� _last_char�lang_filter�logging� getLogger�__name__�logger�_has_win_bytes�reset)�selfr � r ��/builddir/build/BUILDROOT/alt-python310-pip-21.3.1-3.el8.x86_64/opt/alt/python310/lib/python3.10/site-packages/pip/_vendor/chardet/universaldetector.py�__init__Q s zUniversalDetector.__init__c C sV dddd�| _ d| _d| _d| _tj| _d| _| jr| j� � | j D ]}|� � q"dS )z� Reset the UniversalDetector and all of its probers back to their initial states. This is called by ``__init__``, so you only need to call this directly in between analyses of different documents. N� ��encoding� confidence�languageF� )r r r r r � PURE_ASCIIr r r r r )r �proberr r r r ^ s �zUniversalDetector.resetc C s6 | j rdS t|�sdS t|t�st|�}| jso|�tj�r%dddd�| _n;|�tj tj f�r6dddd�| _n*|�d�rCdddd�| _n|�d �rPd ddd�| _n|�tjtjf�r`dddd�| _d| _| jd durod| _ dS | j tjkr�| j�|�r�tj| _ n| j tjkr�| j�| j| �r�tj| _ |dd� | _| j tjkr�| js�t| j�| _| j�|�tjkr�| jj| j�� | jjd�| _d| _ dS dS | j tjk�r| js�t | j�g| _| jt!j"@ r�| j�#t$� � | j�#t%� � | jD ]}|�|�tjk�r |j|�� |jd�| _d| _ nq�| j&�|��rd| _'dS dS dS )a� Takes a chunk of a document and feeds it through all of the relevant charset probers. After calling ``feed``, you can check the value of the ``done`` attribute to see if you need to continue feeding the ``UniversalDetector`` more data, or if it has made a prediction (in the ``result`` attribute). .. note:: You should always call ``close`` when you're done feeding in your document if ``done`` is not already ``True``. Nz UTF-8-SIG� �?� r zUTF-32s �� zX-ISO-10646-UCS-4-3412s ��zX-ISO-10646-UCS-4-2143zUTF-16Tr ���)(r �len� isinstance� bytearrayr � startswith�codecs�BOM_UTF8r �BOM_UTF32_LE�BOM_UTF32_BE�BOM_LE�BOM_BEr r r$ �HIGH_BYTE_DETECTOR�search� HIGH_BYTE�ESC_DETECTORr � ESC_ASCIIr r r �feedr �FOUND_IT�charset_name�get_confidencer"