관리-도구
편집 파일: storage.cpython-311.pyc
� 9�1g` � �� � d Z ddlZddlmZ ddlmZmZmZmZ ddl Z ddl m Z m Z m Z m Z mZmZ ddlmZmZ e G d� d� � � � Zd ee fd �Zd � Zd eeee f fd �ZdS ) zF Represents storage where ssa data is collected, stored and extracted � N)� dataclass)�List�Iterator�Tuple�Dict)�func�cast�case�literal_column�distinct�text)� session_scope� RequestResultc �D � e Zd ZU eed< ee ed< eed< eed< dS )� DomainData� domain_name�domain_total_reqs�is_a_wordpress_domain� urls_numberN)�__name__� __module__� __qualname__�str�__annotations__r �int�bool� � �J/opt/cloudlinux/venv/lib64/python3.11/site-packages/ssa/modules/storage.pyr r sF � � � � � � ������C�y� � � � � � � ������r r �returnc # �z K � t | � � 5 }|� t j t j dt j � � t j t j � � t j t j � � t j t t j � � � � � � � t j t j dt j � � � � � t j t j dt j � � � � }t! j |d� �� � }|D ]^\ }}t% |� � }d}dgdz }|D ]*\ } } } } } | |t' | � � < t || � � }�+t) ||| |�� � V � �_ ddd� � dS # 1 swxY w Y dS )z7 Iterates data from database domain-by-domain. �%Hc � � | d S �Nr r ��items r �<lambda>z#iter_domains_data.<locals>.<lambda>/ s � �VZ�[\�V]� r )�keyr � )r r r r N)r �queryr �domainr �strftime� created_at�Count�id�max� wordpress�countr �path�group_by�order_by� itertools�groupby�tupler r )�engine�db�results_by_hour�results_by_hour_groupedr �group�domain_results_by_hourr �requests_number_by_hour�_�hour� requests_num� is_wordpress�urlss r �iter_domains_datarE s� � � � � �v� � � !�"� �(�(� � � �M�$� � 8� 9� 9� �J�}�'� (� (� �H�]�,� -� -� �J�x� � 2�3�3� 4� 4� � � �(� � � �M�$� � 8� 9� 9� � � �(� � � �M�$� � 8� 9� 9� � � � #,�"3�O�I]�I]�"^�"^�"^� �"9� � � �K��%*�5�\�\� "��K� ()�c�B�h� #�=S� 5� 5�9��4� �|�T�5A�'��D� � �2� !�+�t� 4� 4� � � � '�"9�&2� '� � � � � � � � �#!� !� !� !� !� !� !� !� !� !� !� !���� !� !� !� !� !� !s �FF0�0F4�7F4c # � K � t | � � 5 }d� |D � � }|� t j t j dt j � � t j t t j t j � � � � � d� � t j t j � � � d� � t j t t j t j � � � � � d� � � � � t j |k � � � t% t j � |� � j � ddi�� � j � � � � � t j t j dt j � � � � � t j t j dt j � � � � }d }d gd z d gd z d gd z } }}|D ]q\ } } } } }|r/|| k r)|t3 |||| � � � fV � d gd z d gd z d gd z } }}| |t5 | � � < | |t5 | � � < || t5 | � � < | }�r| t3 | ||| � � � fV � d d d � � d S # 1 swxY w Y d S ) z6 Iterates urls data from database url-by-url. c �: � g | ]}|� d d� � ��S )�:z\:)�replace)�.0r3 s r � <listcomp>z"iter_urls_data.<locals>.<listcomp>H s& � � L� L� L�$�T�\�\�#�u� 5� 5� L� L� Lr r" �url_throttled_reqs�url_total_reqs� url_slow_reqs� literal_bindsT)�compile_kwargsNr r) )r3 rL rM rN )r r* r r3 r r, r- �Sumr �hitting_limits� sqlalchemy�Integer�labelr. r/ �is_slow_request�filterr+ r �in_� expression�compile�stringr4 r5 �dictr )r9 r � all_pathsr: �all_paths_escaped� urls_data� previous_pathrL rM rN r3 rA �url_throttled_req� url_total_req� url_slow_reqs r �iter_urls_datard C s � � � � �v� � � 2 �"� L� L�)� L� L� L���H�H� � � �M�$� � 8� 9� 9� �H�T� �,�j�.@�� � � ��u�)�*�*� �J� � � � ��e�$�%�%� �H�T� �-�z�/A�C� C� � ��e�O�$�$� � � �&� � �K� /� � � �&� � �#�'�'�(9�:�:�E�M�M�^m�os�]t�M�u�u�|� }� }� � � �(� � �� �d�M�4L� M� M� � � �(� � �� �d�M�4L� M� M� � �% �, � � �C�"�H�q�c�B�h���b�� -:�N�� KT� !� !� F�D�$�)�=�,� � 1� �$�!6�!6�#�T�&�'9�#1�"/� &� &� &� � � � � �C�"�H�q�c�B�h���b�� 5B�N�"� ->� �s�4�y�y� )�(5�N�3�t�9�9� %�'3�M�#�d�)�)� $� �M�M��D�� 1�)�'� � � � � � � �[2 � 2 � 2 � 2 � 2 � 2 � 2 � 2 � 2 � 2 � 2 � 2 ���� 2 � 2 � 2 � 2 � 2 � 2 s �J J<�<K �K c # � K � t | � � 5 }|� t j t j � � � t j |k � � � t j � � }t j t |� � d� � � }|D ]\ }}|d� |D � � fV � � ddd� � dS # 1 swxY w Y dS )zA Get information about durations of requests url-by-url. c � � | d S r$ r r% s r r' z#get_url_durations.<locals>.<lambda>� s � �$�q�'� r c � � g | ]\ }}|��S r r )rJ r@ �durations r rK z%get_url_durations.<locals>.<listcomp>� s � �:�:�:�[�Q���:�:�:r N) r r* r r3 rh rW r+ r5 r6 r7 �list)r9 r r: r_ �durations_by_pathr( r= s r �get_url_durationsrk | s) � � � � �v� � � ;�"��H�H� � � � "� � � �&� � �K� /� � � �(� � � � � � &� -� ��O�O� 1� 1� 3� 3��+� ;� ;�J�C���:�:�E�:�:�:�:� :� :� :� :� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;���� ;� ;� ;� ;� ;� ;s �B$C�C� C)�__doc__r6 � dataclassesr �typingr r r r rS r r r r r r �ssa.dbr r r rE rd r r rk r r r �<module>rp s> ��� � � � � � !� !� !� !� !� !� .� .� .� .� .� .� .� .� .� .� .� .� � � � � G� G� G� G� G� G� G� G� G� G� G� G� G� G� G� G� /� /� /� /� /� /� /� /� �� � � � � � � ���%��*�!5� %� %� %� %�N6 � 6 � 6 �r;�d�3��c� �?�.C� ;� ;� ;� ;� ;� ;r