관리-도구
편집 파일: models.cpython-38.pyc
U *�eh- � @ s� d dl mZ d dlmZ d dlmZ d dlmZmZm Z m Z m Z m Z m Z ddlmZ ddlmZmZmZ G dd � d �ZG d d � d �Ze eef Ze e ZG d d � d �ZdS )� )�aliases)�sha256)�dumps)�Any�Dict�Iterator�List�Optional�Tuple�Union� )�TOO_BIG_SEQUENCE)� iana_name�is_multi_byte_encoding� unicode_rangec @ s� e Zd Zd8eeeedee d�dd�Ze ed�dd�Z e ed�d d �Z e ed �d d ��Z ed �dd�Zed �dd�Zd dd�dd�Ze ed �dd��Ze ee d �dd��Ze ed �dd��Ze ed �dd��Ze ee d �d d ��Ze ed �d d ��Ze ed �d d!��Ze ed �d"d#��Ze ed �d$d%��Ze ed �d&d'��Ze ed �d(d)��Z e ed d �d*d+��Z e ed �d,d-��Z e ee d �d.d/��Z e ee d �d0d1��Z d9eed3�d4d5�Z!e ed �d6d7��Z"dS ):� CharsetMatchN�CoherenceMatches)�payload�guessed_encoding�mean_mess_ratio�has_sig_or_bom� languages�decoded_payloadc C sF || _ || _|| _|| _|| _d | _g | _d| _d | _d | _ || _ d S )N� ) �_payload� _encoding�_mean_mess_ratio� _languages�_has_sig_or_bom�_unicode_ranges�_leavesZ_mean_coherence_ratio�_output_payload�_output_encoding�_string)�selfr r r r r r � r% �I/opt/hc_python/lib64/python3.8/site-packages/charset_normalizer/models.py�__init__ s zCharsetMatch.__init__)�other�returnc C s> t |t�s&td�t|j�t| j����| j|jko<| j|jkS )Nz&__eq__ cannot be invoked on {} and {}.)� isinstancer � TypeError�format�str� __class__�encoding� fingerprint�r$ r( r% r% r& �__eq__$ s ��zCharsetMatch.__eq__c C s� t |t�st�t| j|j �}t| j|j �}|dk rJ|dkrJ| j|jkS |dk r�|dkr�t| j�tkrt| j|jk S | j |j kS | j|jk S )zQ Implemented to make sorted available upon CharsetMatches items. g{�G�z�?g{�G�z�?) r* r � ValueError�abs�chaos� coherence�lenr r �multi_byte_usage)r$ r( Zchaos_differenceZcoherence_differencer% r% r& �__lt__- s zCharsetMatch.__lt__�r) c C s dt t| ��t | j� S )Ng �?)r7 r- �raw�r$ r% r% r& r8 C s z CharsetMatch.multi_byte_usagec C s" | j d kr t| j| jd�| _ | j S )N�strict)r# r- r r r< r% r% r& �__str__G s zCharsetMatch.__str__c C s d� | j| j�S )Nz <CharsetMatch '{}' bytes({})>)r, r/ r0 r< r% r% r& �__repr__M s zCharsetMatch.__repr__c C s8 t |t�r|| kr"td�|j���d |_| j�|� d S )Nz;Unable to add instance <{}> as a submatch of a CharsetMatch)r* r r3 r, r. r# r �appendr1 r% r% r& � add_submatchP s ��zCharsetMatch.add_submatchc C s | j S �N)r r< r% r% r& r/ [ s zCharsetMatch.encodingc C sD g }t �� D ]2\}}| j|kr*|�|� q | j|kr |�|� q |S )z� Encoding name are known by many name, using this could help when searching for IBM855 when it's listed as CP855. )r �itemsr/ r@ )r$ Z also_known_as�u�pr% r% r& �encoding_aliases_ s z CharsetMatch.encoding_aliasesc C s | j S rB �r r< r% r% r& �boml s zCharsetMatch.bomc C s | j S rB rG r< r% r% r& �byte_order_markp s z CharsetMatch.byte_order_markc C s dd� | j D �S )z� Return the complete list of possible languages found in decoded sequence. Usually not really useful. Returned list may be empty even if 'language' property return something != 'Unknown'. c S s g | ] }|d �qS )r r% )�.0�er% r% r& � <listcomp>z s z*CharsetMatch.languages.<locals>.<listcomp>�r r< r% r% r& r t s zCharsetMatch.languagesc C sp | j sbd| jkrdS ddlm}m} t| j�r8|| j�n|| j�}t|�dksVd|krZdS |d S | j d d S )z� Most probable language found in decoded sequence. If none were detected or inferred, the property will return "Unknown". �asciiZEnglishr )�encoding_languages�mb_encoding_languagesz Latin Based�Unknown)r �could_be_from_charsetZcharset_normalizer.cdrO rP r r/ r7 )r$ rO rP r r% r% r& �language| s � �zCharsetMatch.languagec C s | j S rB )r r< r% r% r& r5 � s zCharsetMatch.chaosc C s | j s dS | j d d S )Nr r r rM r< r% r% r& r6 � s zCharsetMatch.coherencec C s t | jd dd�S �N�d � )�ndigits)�roundr5 r< r% r% r& � percent_chaos� s zCharsetMatch.percent_chaosc C s t | jd dd�S rT )rX r6 r< r% r% r& �percent_coherence� s z CharsetMatch.percent_coherencec C s | j S )z+ Original untouched bytes. )r r< r% r% r& r; � s zCharsetMatch.rawc C s | j S rB )r r< r% r% r& �submatch� s zCharsetMatch.submatchc C s t | j�dkS �Nr )r7 r r< r% r% r& � has_submatch� s zCharsetMatch.has_submatchc C s@ | j d k r| j S dd� t| �D �}ttdd� |D ���| _ | j S )Nc S s g | ] }t |��qS r% )r )rJ �charr% r% r& rL � s z*CharsetMatch.alphabets.<locals>.<listcomp>c S s h | ] }|r|�qS r% r% )rJ �rr% r% r& � <setcomp>� s z)CharsetMatch.alphabets.<locals>.<setcomp>)r r- �sorted�list)r$ Zdetected_rangesr% r% r& � alphabets� s �zCharsetMatch.alphabetsc C s | j gdd� | jD � S )z� The complete list of encoding that output the exact SAME str result and therefore could be the originating encoding. This list does include the encoding available in property 'encoding'. c S s g | ] }|j �qS r% )r/ )rJ �mr% r% r& rL � s z6CharsetMatch.could_be_from_charset.<locals>.<listcomp>)r r r< r% r% r& rR � s z"CharsetMatch.could_be_from_charset�utf_8)r/ r) c C s2 | j dks| j |kr,|| _ t| ��|d�| _| jS )z� Method to get re-encoded bytes payload using given target encoding. Default to UTF-8. Any errors will be simply ignored by the encoder NOT replaced. N�replace)r" r- �encoder! )r$ r/ r% r% r& �output� s zCharsetMatch.outputc C s t | �� ��� S )zw Retrieve the unique SHA256 computed using the transformed (re-encoded) payload. Not the original one. )r rh � hexdigestr< r% r% r& r0 � s zCharsetMatch.fingerprint)N)re )#�__name__� __module__� __qualname__�bytesr- �float�boolr r' �objectr2 r9 �propertyr8 r> r? rA r/ r rF rH rI r rS r5 r6 rY rZ r; r[ r] rc rR rh r0 r% r% r% r& r sb �� r c @ s� e Zd ZdZdeee d�dd�Zee d�dd�Z e e e f ed �d d �Z e d�d d �Zed�dd�Zedd �dd�Zed d�dd�Zed d�dd�ZdS )�CharsetMatchesz� Container with every CharsetMatch items ordered by default from most probable to the less one. Act like a list(iterable) but does not implements all related methods. N)�resultsc C s |r t |�ng | _d S rB )ra �_results)r$ rs r% r% r& r' � s zCharsetMatches.__init__r: c c s | j E d H d S rB �rt r< r% r% r& �__iter__� s zCharsetMatches.__iter__)�itemr) c C sN t |t�r| j| S t |t�rFt|d�}| jD ]}||jkr.| S q.t�dS )z� Retrieve a single item either by its position or encoding name (alias may be used here). Raise KeyError upon invalid index or encoding not present in results. FN)r* �intrt r- r rR �KeyError)r$ rw �resultr% r% r& � __getitem__� s zCharsetMatches.__getitem__c C s t | j�S rB �r7 rt r< r% r% r& �__len__� s zCharsetMatches.__len__c C s t | j�dkS r\ r| r< r% r% r& �__bool__� s zCharsetMatches.__bool__c C s| t |t�s td�t|j����t|j�tkr`| j D ],}|j |j kr2|j |j kr2|� |� dS q2| j � |� t| j �| _ dS )z~ Insert a single match. Will be inserted accordingly to preserve sort. Can be inserted as a submatch. z-Cannot append instance '{}' to CharsetMatchesN)r* r r3 r, r- r. r7 r; r rt r0 r5 rA r@ ra )r$ rw �matchr% r% r& r@ s �� zCharsetMatches.appendr c C s | j s dS | j d S )zQ Simply return the first match. Strict equivalent to matches[0]. Nr ru r< r% r% r& �best s zCharsetMatches.bestc C s | � � S )zP Redundant method, call the method best(). Kept for BC reasons. )r� r< r% r% r& �first s zCharsetMatches.first)N)rj rk rl �__doc__r r r r' r rv r rx r- r{ r} ro r~ r@ r� r� r% r% r% r& rr � s rr c @ sj e Zd Zeee ee ee eee eeeee ed� dd�Ze e ee f d�dd��Z ed�dd�Z d S ) �CliDetectionResult� �pathr/ rF �alternative_encodingsrS rc r r5 r6 � unicode_path� is_preferredc C sF || _ | | _|| _|| _|| _|| _|| _|| _|| _| | _ | | _ d S rB ) r� r� r/ rF r� rS rc r r5 r6 r� ) r$ r� r/ rF r� rS rc r r5 r6 r� r� r% r% r& r' ) s zCliDetectionResult.__init__r: c C s2 | j | j| j| j| j| j| j| j| j| j | j d� S )Nr� r� r< r% r% r& �__dict__C s �zCliDetectionResult.__dict__c C s t | jddd�S )NT� )� ensure_ascii�indent)r r� r< r% r% r& �to_jsonS s zCliDetectionResult.to_jsonN)rj rk rl r- r r ro rn r' rq r r r� r� r% r% r% r& r� ( s � r� N)Zencodings.aliasesr �hashlibr �jsonr �typingr r r r r r r Zconstantr �utilsr r r r rr r- rn ZCoherenceMatchr r� r% r% r% r& �<module> s $ XC