Reply to thread

Message: <blockquote data-quote="Cupper" data-source="post: 579" data-attributes="member: 22">ARTLASARTLAS is a real-time Apache log analyzer. Based on the top 10 OWASP vulnerabilities, this program detects attempts to exploit your web applications and notifies you or your incident response team via Telegram, Zabbix and Syslog / SIEM.ARTLAS uses regular expressions from the PHP-IDS project to identify exploitation attempts.For details and installation instructions, see the page: <a href="https://kali.tools/?p=4832" target="_blank">https://kali.tools/?p=4832</a>Unfortunately, this program is written in Python 2 and has not been updated for a long time.Analyzing logs with command line tools (Bash)It is very convenient to use a combination of Linux commands for quick analysis of logs. This will help determine, for example, from which IPs the most requests came.In the following commands, replace the access_log file with the name of your Apache log file. You can specify the full path to this file, for example, / var / log / httpd / access_log.If the file is zipped, use zcat instead of cat:Code:zcat site.ru/logs/access_log.1.gzIf the command does not use cat, but the file is zipped, then you can slightly edit the command. For example, the following example processes the access_log file:Code:awk -F \ "'{print $ 6}' access_log | sort | uniq -c | sort -frThe awk program (see Awk Tutorials), like most others, can accept data from standard input, so the same command can be rewritten as follows:Code:cat access_log | awk -F \ "'{print $ 6}' | sort | uniq -c | sort -frAs you can see, it now has cat in it and therefore for compressed files this snippet can be used as follows:Code:zcat access_log.1.gz | awk -F \ "'{print $ 6}' | sort | uniq -c | sort -frSearch by arbitrary stringThe simplest example, search among requests by an arbitrary string (IP address, User-Agent, page address, etc.) using grep:Code:cat access_log | grep 'STRING'To find all lines containing a specific response status, for example 403 (access denied):Code:cat access_log | grep '403'List of all user agents, sorted by the number of times they have appeared:Code:awk -F \ "'{print $ 6}' access_log | sort | uniq -c | sort -frAnalysis of various server responses and requests that triggered them:Code:awk '{print $ 9}' access_log | sort | uniq -c | sortThe output shows how many types of requests your site has received. A "normal" request result is a code of 200, which means that the page or file was requested and delivered. But many other options are also possible The most common answers are:<ul> <li data-xf-list-type="ul">200 - OK</li> <li data-xf-list-type="ul">206 - Partial Content</li> <li data-xf-list-type="ul">301 - Moved Permanently</li> <li data-xf-list-type="ul">302 - Found</li> <li data-xf-list-type="ul">304 - Not Modified</li> <li data-xf-list-type="ul">401 - Unauthorized (password required)</li> <li data-xf-list-type="ul">403 - Forbidden</li> <li data-xf-list-type="ul">404 - Not Found</li> </ul>A 404 error indicates a missing resource. Take a look at the requested URIs that got this error.Code:grep "404" access_log | cut -d '' -f 7 | sort | uniq -c | sort -nrAnother option for displaying the most frequently not found pages on the site:Code:cat access_log | awk '($ 9 ~ / 404 /)' | awk '{print $ 7}' | sort | uniq -c | sort -rn | head -n 25The IP addresses that made the most requests:Code:cat access_log | awk '{print $ 1}' | sort | uniq -c | sort -rn | head -n 25Top 25 IP addresses with the most requests showing their country:Install the required dependencies:Code:sudo apt install geoip-bin geoip-database-extraThe command to display the country of the IP addresses that made the most requests to the server:Code:cat access_log | awk '{print $ 1}' | sort | uniq -c | sort -rn | head -n 25 | awk '{printf ("% 5d \ t% -15s \ t", $ 1, $ 2); system ("geoiplookup" $ 2 "| cut -d \\: -f2")} 'To find sites that insert images of my site (when stealing articles, for example):Code:awk -F\" '($2 ~ /\.(jpg|png|gif)/ && $4 !~ /^https:\/\/(|www\.)hackware\.ru/){print $4}' access_log | sort | uniq -c | sortRemember to edit the domain name in the previous and next commands.To analyze all archives:Code:zcat access_log. * gz | awk -F \ "'($ 2 ~ /\.(jpg|png|gif)/ && $ 4! ~ /^https:\/\/(|www\.)hackware\.ru/) {print $ 4}' | sort | uniq -c | sortEmpty user agentAn empty user agent usually indicates that the request is coming from an automated script. The following command will display a list of IP addresses for these user agents, and based on it, you can decide what to do with them next - block or allow access:Code:awk -F \ "'($ 6 ~ / ^ -? $ /)' access_log | awk '{print $ 1}' | sort | uniqToo much load from one source?When your site is under heavy load, you need to figure out if the load is coming from real users or something else:<ul> <li data-xf-list-type="ul">Setup or system problems</li> <li data-xf-list-type="ul">A custom app or bot is requesting information from your site too quickly</li> </ul>Displaying IP addresses sorted by the number of requests:Code:cat access_log | cut -d '' -f 1 | sort | uniq -c | sort -nr10 most active IPs:Code:cat access_log | awk '{print $ 1; } '| sort | uniq -c | sort -n -r | head -n 10Traffic in kilobytes by status codes:Code:cat access_log | awk '{total [$ 9] + = $ 10} END {for (x in total) {printf "Status code% 3d:% 9.2f Kb \ n", x, total [x] / 1024}}'10 most popular referrers (don't forget to edit your domain name):Code:cat access_log | awk -F \ "'{print $ 4}' | grep -v '-' | grep -v '<a href="https://hackware.ru" target="_blank">https://hackware.ru</a>' | sort | uniq -c | sort -rn | head -n 1010 most popular user agents:Code:cat access_log | awk -F \ "'{print $ 6}' | sort | uniq -c | sort -rn | head -n 10</blockquote>

Name

Verification