Home
Forums
New posts
Search forums
What's new
New posts
New profile posts
Latest activity
Members
Current visitors
New profile posts
Search profile posts
Log in
Register
What's new
Search
Search
Search titles only
By:
New posts
Search forums
Menu
Log in
Register
Install the app
Install
JavaScript is disabled. For a better experience, please enable JavaScript in your browser before proceeding.
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an
alternative browser
.
Reply to thread
Home
Forums
CARDING & HACKING
HOSTING & BOTNET
Apache logs 2023
Message
<blockquote data-quote="Cupper" data-source="post: 574" data-attributes="member: 22"><p><strong>200 (%> s)</strong></p><p></p><p>This is the status code that the server sends back to the client. This information is very valuable because it shows whether the request resulted in a successful response (codes start with 2), a redirect (codes start with 3), an error caused by the client (codes start with 4), or errors on the server (codes start with 5). A complete list of possible status codes can be found in the HTTP specification (RFC2616 section 10).</p><p></p><p><strong>25858 (% b)</strong></p><p></p><p>The last part indicates the size of the object returned to the client, not including the response headers. If content has not been returned to the client, this value will be "-". To write "0" when there is no content, use <strong>% Bed and</strong> .</p><p></p><p><strong>Combined Log Format</strong></p><p>Another commonly used format string is called the Combined Log Format. It can be used as follows.</p><p></p><p>Code:</p><p>LogFormat "% h% l% u% t \"% r \ "%> s% b \"% {Referer} i \ "\"% {User-agent} i \ "" combined</p><p>CustomLog "log / access_log" combined</p><p></p><p>This format is exactly the same as the Common Log Format, with the addition of two more fields. Each of the additional fields uses a% {header} i percentage directive, where header can be any HTTP request header. The access log in this format will look like this:</p><p>Code:</p><p>2a02: 2168: a13: 430b :: 1 - - [18 / Aug / 2019: 09: 38: 53 +0300] "POST / ru /? Act = locatepicture HTTP / 1.1" 200 25627 "<a href="https://suip.biz" target="_blank">https://suip.biz</a> / ru /? act = locatepicture "" Mozilla / 5.0 (Windows NT 6.1; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 76.0.3809.100 Safari / 537.36 "</p><p>Please note that the IP address can also be IPv6 as in the example above.</p><p></p><p>Additional fields are:</p><p><strong>"<a href="https://suip.biz/en/?act=locatepicture" target="_blank">https://suip.biz/en/?act=locatepicture</a>" (\ "% {Referer} i \")</strong></p><p></p><p>This is the HTTP header of the "Referer" request. In this line, the client tells the site from which site and which page he came from (this should be the page on which the link to the requested address is posted, or the page that includes the requested file (for example, an image).</p><p></p><p><strong>"Mozilla / 5.0 (Windows NT 6.1; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 76.0.3809.100 Safari / 537.36" (\ "% {User-agent} i \")</strong></p><p></p><p>User-Agent HTTP request header. This is the identifying information that the client browser communicates about itself.</p><p></p><p><strong>Multiple Access Logs</strong></p><p>Multiple access logs can be created simply by specifying a few CustomLog directives in the configuration file. For example, the following directives will create three access logs. The first contains the basic CLF information, while the second and third contain the referrer and browser information. The last two lines of CustomLog show how to simulate the effects of the ReferLog and AgentLog directives.</p><p>Code:</p><p>LogFormat "% h% l% u% t \"% r \ "%> s% b" common</p><p>CustomLog "logs / access_log" common</p><p>CustomLog "logs / referer_log" "% {Referer} i ->% U"</p><p>CustomLog "logs / agent_log" "% {User-agent} i"</p><p></p><p>This example also shows that there is no need to define an alias using the LogFormat directive. Instead, the log format can be specified directly in the CustomLog directive.</p><p></p><p><strong>Conditional Logs</strong></p><p>There are times when it is convenient to exclude certain entries from the access logs based on the characteristics of the client request. It's easy to do this with environment variables. First, you need to set an environment variable to indicate that the request meets certain conditions. This is usually achieved with SetEnvIf. The <strong>env =</strong> clause of the CustomLog directive is then used to include or exclude requests in which the environment variable is set. Some examples:</p><p>Code:</p><p># Flag requests from the loop-back interface</p><p>SetEnvIf Remote_Addr "127 \ .0 \ .0 \ .1" dontlog</p><p># Flag requests for robots.txt file</p><p>SetEnvIf Request_URI "^ / robots \ .txt $" dontlog</p><p># Write down what's left</p><p>CustomLog "logs / access_log" common env =! Dontlog</p><p></p><p>As another example, consider writing requests from English-speaking users to one log file and non-English speakers to a different log file.</p><p>Code:</p><p>SetEnvIf Accept-Language "en" english</p><p>CustomLog "logs / english_log" common env = english</p><p>CustomLog "logs / non_english_log" common env =! English</p><p></p><p>In a caching scenario, I would like to know about the efficiency of the cache. A very simple way to find out would be:</p><p>Code:</p><p>SetEnv CACHE_MISS 1</p><p>LogFormat "% h% l% u% t"% r "%> s% b% {CACHE_MISS} e" common-cache</p><p>CustomLog "logs / access_log" common-cache</p><p>mod_cache will run before mod_env and, if successful, will deliver content without it. In this case, the cache will lead to the appearance of the entry -, and if there is no cache, then 1 will be written.</p><p></p><p>In addition to the <strong>env =</strong> syntax, LogFormat supports variable registration values depending on the HTTP response code:</p><p>Code:</p><p>LogFormat "% 400,501 {User-agent} i" browserlog</p><p>LogFormat "%! 200,304,302 {Referer} i" refererlog</p><p></p><p>In the first example, the User-agent will be logged if the HTTP status code is 400 or 501. Otherwise, the literal string “-” will be written instead. Likewise, in the second example, the Referer will be logged if the HTTP status code is not 200, 204, or 302. (Note the " <strong>!</strong> " In front of the status codes).</p><p></p><p>While we have just shown that conditional logging is very powerful and flexible, it is not the only way to manage the content of the logs. Log files are more useful when they contain a complete record of server activity. In most cases, it is easier to simply process the complete log files to extract only the data you need from them, or to remove certain information.</p><p></p><p><strong>Rotation of logs</strong></p><p>Even on a moderately busy server, the amount of information stored in the log files is very large. The access log file typically grows 1 MB or more for 10,000 requests. Therefore, it is necessary to periodically rotate the log files by moving or deleting existing logs. This cannot be done while the server is running because Apache httpd will continue to write to the old log file as long as it keeps the file open. Instead, the server must be restarted after moving or deleting log files to open the new log files.</p><p></p><p>By using a graceful restart, the server can be instructed to open new log files without losing existing or pending connections from clients. However, to do this, the server must continue to write to the old log files while it finishes serving old requests. Therefore, you must wait a while after the restart before doing any processing on the log files. A typical scenario that just rotates logs and shrinks old logs to save space:</p><p>Code:</p><p>mv access_log access_log.old</p><p>mv error_log error_log.old</p><p>apachectl graceful</p><p>sleep 600</p><p>gzip access_log.old error_log.old</p><p></p><p>Another way to perform log rotation is by using pipelining, as described in the next section.</p><p></p><p><strong>Pipelining (Piped Logs)</strong></p><p>Apache httpd is capable of writing access and error log files down a pipe (through a pipe) to another process, not directly to a file. This capability greatly improves the flexibility of logging without adding code to the core server. To write logs to a pipe, simply replace the filename with the pipe character "|" followed by the name of the executable file that should receive log entries on its standard input. The server will start the piped-log process on server startup and restart it if it crashes while the server is running (this latter feature allows us to call this technique "reliable pipe logging".)</p><p></p><p></p><p>Pipeline processes are spawned by the parent Apache httpd process and inherit the user ID of that process. This means that pipelined log programs are usually run as root. Therefore, it is very important that the programs are simple and safe.</p><p></p><p>One of the important uses of pipelined logs is to allow log rotation without rebooting the server. The Apache HTTP Server includes a simple rotatelogs program for this purpose. For example, to rotate logs every 24 hours, you can use:</p><p></p><p>CustomLog "| / usr / local / apache / bin / rotatelogs / var / log / access_log 86400" common</p><p>Note that quotes are used to include the entire command that will be invoked for the pipe. While these examples are for the access log, the same method can be used for the error log.</p><p></p><p>As with conditional logging, piped logs are very powerful, but should not be used where a simpler solution such as offline post-processing is available.</p><p></p><p>By default, the piped log process is spawned without invoking the shell. Use "| $" instead of "|" to run with a shell (usually with / bin / sh -c):</p><p></p><p>Code:</p><p># Call "rotatelogs" using the shell</p><p>CustomLog "| $ / usr / local / apache / bin / rotatelogs / var / log / access_log 86400" common</p><p>This was the default behavior for Apache 2.2. Depending on the specifics of the shell, this can lead to an extra shell process for the lifetime of the log pipe program and problems with signal handling on restart. For Apache 2.2 compatibility reasons, the notation "||" also supported and equivalent to using "|".</p></blockquote><p></p>
[QUOTE="Cupper, post: 574, member: 22"] [B]200 (%> s)[/B] This is the status code that the server sends back to the client. This information is very valuable because it shows whether the request resulted in a successful response (codes start with 2), a redirect (codes start with 3), an error caused by the client (codes start with 4), or errors on the server (codes start with 5). A complete list of possible status codes can be found in the HTTP specification (RFC2616 section 10). [B]25858 (% b)[/B] The last part indicates the size of the object returned to the client, not including the response headers. If content has not been returned to the client, this value will be "-". To write "0" when there is no content, use [B]% Bed and[/B] . [B]Combined Log Format[/B] Another commonly used format string is called the Combined Log Format. It can be used as follows. Code: LogFormat "% h% l% u% t \"% r \ "%> s% b \"% {Referer} i \ "\"% {User-agent} i \ "" combined CustomLog "log / access_log" combined This format is exactly the same as the Common Log Format, with the addition of two more fields. Each of the additional fields uses a% {header} i percentage directive, where header can be any HTTP request header. The access log in this format will look like this: Code: 2a02: 2168: a13: 430b :: 1 - - [18 / Aug / 2019: 09: 38: 53 +0300] "POST / ru /? Act = locatepicture HTTP / 1.1" 200 25627 "[URL]https://suip.biz[/URL] / ru /? act = locatepicture "" Mozilla / 5.0 (Windows NT 6.1; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 76.0.3809.100 Safari / 537.36 " Please note that the IP address can also be IPv6 as in the example above. Additional fields are: [B]"[URL]https://suip.biz/en/?act=locatepicture[/URL]" (\ "% {Referer} i \")[/B] This is the HTTP header of the "Referer" request. In this line, the client tells the site from which site and which page he came from (this should be the page on which the link to the requested address is posted, or the page that includes the requested file (for example, an image). [B]"Mozilla / 5.0 (Windows NT 6.1; Win64; x64) AppleWebKit / 537.36 (KHTML, like Gecko) Chrome / 76.0.3809.100 Safari / 537.36" (\ "% {User-agent} i \")[/B] User-Agent HTTP request header. This is the identifying information that the client browser communicates about itself. [B]Multiple Access Logs[/B] Multiple access logs can be created simply by specifying a few CustomLog directives in the configuration file. For example, the following directives will create three access logs. The first contains the basic CLF information, while the second and third contain the referrer and browser information. The last two lines of CustomLog show how to simulate the effects of the ReferLog and AgentLog directives. Code: LogFormat "% h% l% u% t \"% r \ "%> s% b" common CustomLog "logs / access_log" common CustomLog "logs / referer_log" "% {Referer} i ->% U" CustomLog "logs / agent_log" "% {User-agent} i" This example also shows that there is no need to define an alias using the LogFormat directive. Instead, the log format can be specified directly in the CustomLog directive. [B]Conditional Logs[/B] There are times when it is convenient to exclude certain entries from the access logs based on the characteristics of the client request. It's easy to do this with environment variables. First, you need to set an environment variable to indicate that the request meets certain conditions. This is usually achieved with SetEnvIf. The [B]env =[/B] clause of the CustomLog directive is then used to include or exclude requests in which the environment variable is set. Some examples: Code: # Flag requests from the loop-back interface SetEnvIf Remote_Addr "127 \ .0 \ .0 \ .1" dontlog # Flag requests for robots.txt file SetEnvIf Request_URI "^ / robots \ .txt $" dontlog # Write down what's left CustomLog "logs / access_log" common env =! Dontlog As another example, consider writing requests from English-speaking users to one log file and non-English speakers to a different log file. Code: SetEnvIf Accept-Language "en" english CustomLog "logs / english_log" common env = english CustomLog "logs / non_english_log" common env =! English In a caching scenario, I would like to know about the efficiency of the cache. A very simple way to find out would be: Code: SetEnv CACHE_MISS 1 LogFormat "% h% l% u% t"% r "%> s% b% {CACHE_MISS} e" common-cache CustomLog "logs / access_log" common-cache mod_cache will run before mod_env and, if successful, will deliver content without it. In this case, the cache will lead to the appearance of the entry -, and if there is no cache, then 1 will be written. In addition to the [B]env =[/B] syntax, LogFormat supports variable registration values depending on the HTTP response code: Code: LogFormat "% 400,501 {User-agent} i" browserlog LogFormat "%! 200,304,302 {Referer} i" refererlog In the first example, the User-agent will be logged if the HTTP status code is 400 or 501. Otherwise, the literal string “-” will be written instead. Likewise, in the second example, the Referer will be logged if the HTTP status code is not 200, 204, or 302. (Note the " [B]![/B] " In front of the status codes). While we have just shown that conditional logging is very powerful and flexible, it is not the only way to manage the content of the logs. Log files are more useful when they contain a complete record of server activity. In most cases, it is easier to simply process the complete log files to extract only the data you need from them, or to remove certain information. [B]Rotation of logs[/B] Even on a moderately busy server, the amount of information stored in the log files is very large. The access log file typically grows 1 MB or more for 10,000 requests. Therefore, it is necessary to periodically rotate the log files by moving or deleting existing logs. This cannot be done while the server is running because Apache httpd will continue to write to the old log file as long as it keeps the file open. Instead, the server must be restarted after moving or deleting log files to open the new log files. By using a graceful restart, the server can be instructed to open new log files without losing existing or pending connections from clients. However, to do this, the server must continue to write to the old log files while it finishes serving old requests. Therefore, you must wait a while after the restart before doing any processing on the log files. A typical scenario that just rotates logs and shrinks old logs to save space: Code: mv access_log access_log.old mv error_log error_log.old apachectl graceful sleep 600 gzip access_log.old error_log.old Another way to perform log rotation is by using pipelining, as described in the next section. [B]Pipelining (Piped Logs)[/B] Apache httpd is capable of writing access and error log files down a pipe (through a pipe) to another process, not directly to a file. This capability greatly improves the flexibility of logging without adding code to the core server. To write logs to a pipe, simply replace the filename with the pipe character "|" followed by the name of the executable file that should receive log entries on its standard input. The server will start the piped-log process on server startup and restart it if it crashes while the server is running (this latter feature allows us to call this technique "reliable pipe logging".) Pipeline processes are spawned by the parent Apache httpd process and inherit the user ID of that process. This means that pipelined log programs are usually run as root. Therefore, it is very important that the programs are simple and safe. One of the important uses of pipelined logs is to allow log rotation without rebooting the server. The Apache HTTP Server includes a simple rotatelogs program for this purpose. For example, to rotate logs every 24 hours, you can use: CustomLog "| / usr / local / apache / bin / rotatelogs / var / log / access_log 86400" common Note that quotes are used to include the entire command that will be invoked for the pipe. While these examples are for the access log, the same method can be used for the error log. As with conditional logging, piped logs are very powerful, but should not be used where a simpler solution such as offline post-processing is available. By default, the piped log process is spawned without invoking the shell. Use "| $" instead of "|" to run with a shell (usually with / bin / sh -c): Code: # Call "rotatelogs" using the shell CustomLog "| $ / usr / local / apache / bin / rotatelogs / var / log / access_log 86400" common This was the default behavior for Apache 2.2. Depending on the specifics of the shell, this can lead to an extra shell process for the lifetime of the log pipe program and problems with signal handling on restart. For Apache 2.2 compatibility reasons, the notation "||" also supported and equivalent to using "|". [/QUOTE]
Name
Verification
Post reply
Home
Forums
CARDING & HACKING
HOSTING & BOTNET
Apache logs 2023
Top