LinuxLogFiles
Differences between revisions 5 and 6
|
Size: 26396
Comment:
|
Size: 57
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ||<tablestyle="float:right; font-size: 0.9em; width:40%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;">'''Contents'''[[BR]][[TableOfContents(3)]]|| == Introduction == One of the things which makes GNU/Linux a great operating system is that virtually anything, and everything happening on, and to the system may be logged in some manner. This information is invaluable to using the system in an informed manner, and should be one of the first resources used in trouble-shooting system, and application issues. The logs can tell you almost anything you need to know, so long as you have an idea where to look first. Your Ubuntu system provides vital information concerning events, operation, and other functions in various system log files. These log files are typically plain ASCII text in a standard log file format, and the majority of them are located in the traditional system log subdirectory {{{/var/log}}}. Many of these log files are generated by the system log daemon, {{{syslogd}}} on behalf of the system, and certain applications, while some applications generate their own logs by writing directly to log files located in the {{{/var/log}}} subdirectory. This guide discusses several of these system log files, describes their content with examples, and further presents examples on extracting useful information from these system log files with such command-line utilities as {{{grep}}} and {{{less}}}. This guide will also discuss the system logging daemon, {{{syslogd}}}, its configuration, and the concept of log rotation. Additional information will be provided in the '''Resources''' section of this guide. == Target Audience == This guide is for anyone with sufficient experience using the GNU/Linux command-line, and particularly experience in executing command-line utilities, and editing system configuration files with a preferred console-based text editor. == System Logs == This section of the guide addresses so-called system logs, or logs which deal primarily with the functioning of the Ubuntu system, and not necessarily with additional applications added by the System Administrator, or users of the system. Examples of such logs include the logging of authorization mechanisms, running system daemons, system messages, and the all-encompassing system log itself, or ''syslog'' as it is known. (not to be confused with the actual {{{syslogd}}} daemon'' which we'll cover later). === Authorization Log === The Authorization Log tracks usage of authorization systems, that is, all of the Ubuntu mechanisms for authorizing users which prompt for user passwords, such as the Pluggable Authentication Module (PAM) system, the {{{sudo}}} command, remote logins to {{{sshd}}}, and so on. The Authorization Log file may be accessed at {{{/var/log/auth.log}}}. This log is useful for learning about user logins, and usage of the sudo command on your Ubuntu system, for example. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/auth.log}}} Press the '''SPACE BAR''' to advance to the next page, or the '''ENTER''' key to advance one line at-a-time. The '''b''' key will scroll backwards one full page, and the '''q''' quits the {{{less}}} utility. Specific information may be accessed from the Authorization Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Authorization Log pertaining to {{{sshd}}} logins, you might use a command such as the following at a terminal prompt: {{{grep sshd /var/log/auth.log | less}}} === Daemon Log === The daemon log exists at {{{/var/log/daemon.log}}} and contains information specific to running system, and application daemons such as the Gnome Display Manager daemon, ({{{gdm}}}) the Bluetooth HCI daemon, ({{{hcid}}}) or the MySQL Relational Database Management System (RDBMS) daemon, ({{{mysqld}}}). This log is useful for gaining information about running daemons, and for trouble-shooting problems with a particular daemon. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/daemon.log}}} Specific information may be accessed from the Daemon Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Daemon Log pertaining to the MySQL daemon, you might use a command such as the following at a terminal prompt: {{{grep mysqld /var/log/daemon.log | less}}} === Debug Log === The debug log exists at {{{/var/log/debug}}} and provides detailed, debug messages from the Ubuntu system, and applications which log to {{{syslogd}}} at the DEBUG level. These messages are useful for debugging issues with everything from hardware drivers, to server daemons. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/debug}}} Specific information may be accessed from the Debug Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Debug Log pertaining to the Advanced Configuration and Power Interface (ACPI), you might use a command such as the following at a terminal prompt: {{{grep ACPI /var/log/debug | less}}} === Kernel Log === The kernel log at {{{/var/log/kern.log}}} provides a detailed log of messages from the Ubuntu Linux kernel. These messages may prove useful for trouble-shooting a new, or custom-built kernel for example. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/kern.log}}} Specific information may be accessed from the Kernel Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Kernel Log pertaining to the computer's Central Processing Unit (CPU), you might use a command such as the following at a terminal prompt: {{{grep CPU /var/log/kern.log | less}}} === Kernel Ring Buffer === The kernel ring buffer is not really a log file per se, but rather an area in the running kernel which may be queried for kernel bootup messages via the {{{dmesg}}} utility. You may see all kernel bootup messages by using a command such as the following at a terminal prompt: {{{dmesg | less}}} You may also use the {{{dmesg}}} utility to examine specific information from the kernel bootup messages such as Plug and Play (PNP) messages by using a command such as the following at a terminal prompt: {{{dmesg | grep pnp | less}}} In relation to the Kernel Ring Buffer, the default behavior of the {{{/etc/init.d/bootmisc.sh}}} system initialization script is to use the {{{dmesg}}} command to log all bootup messages to the file {{{/var/log/dmesg}}} as well. This file may be used as any other log file for examining Kernel bootup messages via commands {{{grep}}}, {{{less}}}, and others. === Messages Log === The messages log contains informational messages from applications, and system facilities, and is available at {{{/var/log/messages}}}. This log is useful for examining message output from applications, and system facilities which log to the syslog / sysklog daemon at the INFO level. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/messages}}} Specific information may be accessed from the Messages Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Messages Log pertaining to the Gnome Configuration Daemon ({{{gconfd}}}), you might use a command such as the following at a terminal prompt: {{{grep gconfd /var/log/messages | less}}} === System Log === The system log typically contains the greatest deal of information by default about your Ubuntu system. It is located at {{{/var/log/syslog}}}, and may contain information other logs do not. You may wish to consult the System Log when you are unable to locate the desired log information in another log. == Application Logs == In addition to the myriad of system-specific logs available on your Ubuntu system, you may also access individual logs which may be used by certain applications. If you list the contents of your {{{/var/log}}} subdirectory, you will see familiar names of applications you may have installed, such as {{{/var/log/apache2}}} representing the logs for the Apache 2 Hyper Text Transport Protocol (HTTP) server, or {{{/var/log/samba}}}, which contains the logs for the Samba Server Message Block (SMB) server. This section of the guide introduces some specific examples of application logs, and information contained within them. === Apache HTTP Server Logs === The default installation for Apache2 on Ubuntu creates a log subdirectory: {{{/var/log/apache2}}}, and within this subdirectory, are two log files with two distinct purposes: * {{{/var/log/access.log}}} : Contains records of all access to the HTTP server by clients * {{{/var/log/error.log}}} : Contains records of all error conditions reported by the HTTP server With this information in mind, and a command of the tools {{{grep}}}, and {{{less}}} basic information gathering from these logs becomes possible. For example, and in terms of access, if you wished to see log records for every recorded access to your Apache2 server from the client IP address ''82.211.81.166'', and display the results as one page per screen, you would simply use a command such as the following at a terminal prompt: {{{grep "82.211.81.166" /var/log/apache2/access.log | less}}} Or, if you wished to determine if any clients using your Apache2 server were using Mac OS X, a simple command such as the following, typed into a terminal window would suffice: {{{grep "Mac OS X" /var/log/apache2/access.log | less}}} On the other side of the coin, suppose you wished to see information from the {{{/var/log/apache2/error.log}}}. This log can be used to search for instances of the Apache2 server being shut down, by using a command such as the following from a terminal prompt: {{{grep "shutting down" /var/log/apache2/error.log | less}}} You may also see all log entries which are considered errors by Apache2 with a command such as the following entered at a terminal prompt: {{{grep error /var/log/apache2/error.log | less}}} === CUPS Print System Logs === The Common Unix Printing System (CUPS) uses the default log file {{{/var/log/cups/error_log}}} to store informational and error messages. If you need to solve a printing issue in Ubuntu, then this log may be a good place to start. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/cups/error_log}}} Specific information may be accessed from the CUPS Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the CUPS Log pertaining to Full reloads, you might use a command such as the following at a terminal prompt: {{{grep reload /var/log/cups/error_log | less}}} === Rootkit Hunter Log === The Rootkit Hunter utility ({{{rkhunter}}}) checks your Ubuntu system for backdoors, sniffers, and so-called "rootkits", which are all signs of compromise of your system. The log which rkhunter uses is located at {{{/var/log/rkhunter.log}}} You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/rkhunter.log}}} Specific information may be accessed from the Rootkit Hunter Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Rootkit Hunter Log pertaining to Warnings, you might use a command such as the following at a terminal prompt: {{{grep WARNING /var/log/rkhunter.log | less}}} === Samba SMB Server Logs === The Server Message Block Protocol (SMB) server, Samba is popularly used for sharing files from your Ubuntu computer to other computers which support the SMB protocol. Samba keeps three distinct types of logs in the subdirectory {{{/var/log/samba}}}: * {{{log.nmbd}}} : Logs all messages related to Samba's NETBIOS over IP functionality (the network stuff) * {{{log.smbd}}} : Logs all messages related to Samba's SMB/CIFS functionality (the file, print, etc. sharing stuff) * {{{log.[IP_ADDRESS]}}} : Logs messages related to requests for services from the IP address contained in the log file name, for example, {{{log.192.168.1.1}}}. To view all information related to Samba's networking, you could use a command such as the following at a terminal prompt: {{{less /var/log/samba/log.nmbd}}} If you wanted only to see information logged about Master Browsers, then you might use a command like this at a terminal prompt: {{{grep "master browser" /var/log/samba/log.nmbd | less}}} If you would like to see details related to the SMB functionality of Samba, you can view the appropriate log in its entirety with a command similar to the following at a terminal prompt: {{{less /var/log/samba/log.smbd}}} To see only messages related to the start up of the Samba server, a command like the following at a terminal prompt may be used: {{{grep started /var/log/samba/log.smbd | less}}} To view all the details on connections from the client system with IP address ''192.168.99.99'' you could use a command such as the following at a terminal prompt: {{{less /var/log/samba/log.192.168.99.99}}} === X11 Server Log === The default X11 Windowing Server in use with Ubuntu is the Xorg X11 server, and assuming your computer has but one display defined, it stores log messages in the file {{{/var/log/Xorg.0.log}}}. This log is helpful for diagnosing issues with your X11 environment. You may examine the log, from its most recent rotation point, (if applicable) one page per screen, using a command such as the following from a terminal prompt: {{{less /var/log/Xorg.0.log}}} Specific information may be accessed from the Xorg Log by using such commands as {{{grep}}}, and {{{less}}}. For example, to see only information in the Xorg Log pertaining to the freetype font engine, you might use a command such as the following at a terminal prompt: {{{grep freetype /var/log/Xorg.0.log | less}}} == Non-Human-Readable Logs == Some log files found in the {{{/var/log}}} subdirectory are designed to be readable by applications, and not necessarily by humans. Some examples of such log files which appear in {{{/var/log}}} follow. === Login Failures Log === The login failures log located at {{{/var/log/faillog}}} is actually designed to be parsed, and output by the {{{faillog}}} command. For example, to print recent login failures using the {{{faillog}}} command, simply enter the following at a terminal prompt: {{{faillog}}} === Last Logins Log === The last logins log at {{{/var/log/lastlog}}} should not typically be parsed, and examined by humans, but rather should be used in conjunction with the {{{lastlog}}} command. For example to see a listing of logins with the {{{lastlog}}} command, displayed one page per screen with the {{{less}}} command, use the following command at a terminal prompt: {{{lastlog | less}}} === Login Records Log === The file {{{/var/log/wtmp}}} contains login records, but unlike {{{/var/log/lastlog}}} above, {{{/var/log/wtmp}}} is not used to show a list of recent logins, but is instead used by other utilities such as the {{{who}}} command to present a listed of currently logged in users. For example, if you wish to see who is currently logged in on the machine you are currently using, issue the following at a terminal prompt: {{{who}}} == System Logging Daemon (syslogd) == The system logging daemon, or {{{syslogd}}} (also known as {{{sysklogd}}}) is a system daemon, or special application which executes silently, in the background and does good things for your system. Specifically, {{{syslogd}}} awaits logging messages from numerous system, and application sources, and routes the messages to their proper target, be it a standard log file, a First In First Out (FIFO) pipe for use by a log analyzation application, or even across the network to another system's {{{syslogd}}}. Messages logged to {{{syslogd}}} usually contain important common elements, such as system hostnames, and time-stamps in addition to the specific log information from a system source, the Linux kernel, or a user application. === Configuration of syslogd === The detailed configuration of {{{syslogd}}} is beyond the scope of this guide, and the reader is encouraged to seek additional information via the '''Resources''' section of this guide for information on correctly configuring, and modifying the configuration of {{{syslogd}}}. The file which configures the behavior of the {{{syslogd}}} daemon is {{{/etc/syslog.conf}}} and consists primarily of two fields, the selector, and the action. The selector field consists of a facility, to be logged, such as for example the '''auth''' facility which deals with authorization, and a priority, or level to log such information at, such as '''info''', or '''warning''' priorities, which would log all messages at the informational priority and higher, or only at the warning level and higher respectively. The action field consists of a target for the log information, such as a standard log file (i.e. {{{/var/log/syslog}}}), or the hostname of a remote computer to send the log information to (e.g. @myotherubuntu). The configuration file is very flexible, and powerful in nature, allowing a seemingly infinite combination of logging to take place to fit the particular requirements your installation may have. === Echoing Messages to syslogd With Logger === A neat utility exists in the {{{logger}}} tool, which allows one to place messages into the System Log (i.e. {{{/var/log/syslog}}}) arbitrarily. This is a powerful tool, which you may use in Administrative scripts, such as Perl, or shell scripts to provide them with standard logging capabilities, or you may use it just to place things in the system log as needed. For example, assume your user name is {{{buddha}}}, and you would like to enter a message into the syslog about a particularly delicious pizza you're eating, you could use a command such as the following at a terminal prompt: {{{logger This Pizza from Vinnys Gourmet Rocks}}} and you would find a line in the {{{/var/log/syslog}}} file such as this afterward: {{{ Jan 12 23:17:02 localhost buddha: This Pizza from Vinnys Gourmet Rocks }}} Used in a little more professional manner in shell scripts, you can even specify a tag the messages come from, and redirect the output standard error too. This lets you have excellent error logging in a script, such as in this example snippet: {{{ #!/bin/bash # # sample logger error jive # logmsg="/usr/bin/logger -s -t MyScript " # announce what this script is, even to the log $logmsg "Directory Checker FooScript Jive 1.0" # test for the existence of Fred's home dir on this machine if [ -d /home/fred ]; then $logmsg "I. Fred's Home Directory Found" else $logmsg "E. Fred's Home Directory was NOT Found. Boo Hoo." exit 1 fi }}} Executing this script as {{{chkdir.sh}}} on the machine {{{butters}}} where Fred does not have a home directory, {{{/home/fred}}}, gives the following results: {{{ bumpy@butters:~$./chkdir.sh MyScript: Directory Checker FooScript Jive 1.0 MyScript: E. Fred's Home Directory was NOT Found. Boo Hoo. bumpy@butters:~$tail -n 2 /var/log/syslog Jan 12 23:23:11 localhost MyScript: Directory Checker FooScript Jive 1.0 Jan 12 23:23:11 localhost MyScript: E. Fred's Home Directory was NOT Found. Boo Hoo. }}} So, as you can see, we received the messages both via standard error, at the terminal prompt, and they also appear in our syslog! === Log Rotation === When viewing directory listings in {{{/var/log}}} or any of its subdirectories, you may encounter log files with names such as {{{daemon.log.0}}}, {{{daemon.log.1.gz}}}, and so on. What are these log files? They are 'rotated' log files. That is, they have automatically been renamed after a predefined time-frame, and a new original log started. After even more time the log files are compressed with the {{{gzip}}} utility as in the case of the example {{{daemon.log.1.gz}}}. The purpose of log rotation is to archive and compress old logs so that they consume less disk space, but are still available for inspection as needed. What handles this functionality? Why, the {{{logrotate}}} command of course! Typically, logrotate is called from the system-wide cron script {{{/etc/cron.daily/logrotate}}}, and further defined by the configuration file {{{/etc/logrotate.conf}}}. This guide will not cover the myriad of ways logrotate may be configured to handle the automatic rotation of any log file on your Ubuntu system, but rather the reader is encouraged to use the '''Resources''' section of this guide, and study the requisite manual pages to determine how to configure logrotate for a particular log file, and needs. attachment:IconsPage/IconNote.png '''NOTE:''' You may also rotate system log files via the {{{cron.daily}}} script {{{/etc/cron.daily/sysklogd}}} instead of using logrotate. Actually, the utility {{{savelog}}} may produce unexpected results on log rotation which configuring {{{logrotate}}} seems to have no effect on. In those cases, you should check the cron.daily {{{sysklogd}}} script in {{{/etc/cron.daily/sysklogd}}} and read the {{{savelog}}} manual page to see if {{{savelog}}} is not in fact doing the rotation in a way that is not what you are specifying with {{{logrotate}}}. == Additional Tips == Some additional tips for quickly viewing logs manually, (i.e. without a log file analyzer application) which may help you expediently locate the information you require. === Just the Beginning === You may look at just the beginning of any log file by using the {{{head}}} command. by default, {{{head}}} shows the first ten lines of any text file, so for example, if you wished to see the oldest entries in your Authorization Log file, a command such as the following could be used at a terminal prompt: {{{head /var/log/auth.log}}} If ten lines is not enough, and you need to see the first twenty-five (25) lines, then use {{{head}}} with the {{{-n}}} switch as such: {{{head -n 25 /var/log/auth.log}}} === Just the End === The compliment to {{{head}}} of course is none other than the {{{tail}}} command. Can you guess what {{{tail}}} allows you to do? Say you need the last ten lines of the Kernel log for important messages from the kernel of late. A command such as the following entered into a terminal prompt should do: {{{tail /var/log/kern.log}}} Again, and as with {{{head}}}, you may get more than the default ten lines of output with {{{tail}}} by specifying the {{{-n}}} switch as such: {{{tail -n 30 /var/log/kern.log}}} to see the last thirty (30) lines of the Kernel log instead. ==== Real-Time Tail ==== Another neat use for the {{{tail}}} command is to use it for watching a log in 'real-time' by specifying the {{{-f}}} switch. For example, if you wished to watch in real-time as clients access your Apache2 server, issuing a command such as this from a terminal prompt would allow you to do so: {{{tail -f /var/log/apache2/access.log}}} You will see the log spit out, then stop, and as the Apache2 server is accessed, log entries will fly by in real-time! If you have a very busy server, they will fly by too fast for you to read them! You can use the {{{-f}}} switch to view any log file in this manner. == Resources == Additional information on system, and application logs, and syslogd is available via the following resources: === Local System Resources === ||<style="background:#F1F1ED;">{{{man dmesg}}}|| System manual page for the {{{dmesg}}} kernel ring buffer utility|| ||<style="background:#F1F1ED;">{{{man faillog}}}|| System manual page for the {{{faillog}}} command (and also the faillog configuration file via {{{man 5 faillog}}})|| ||<style="background:#F1F1ED;">{{{man grep}}}|| System manual page for the {{{grep}}} pattern searching utility|| ||<style="background:#F1F1ED;">{{{man head}}}|| System manual page for the {{{head}}} utility|| ||<style="background:#F1F1ED;">{{{man klogd}}}|| System manual page for the kernel log daemon ({{{klogd}}})|| ||<style="background:#F1F1ED;">{{{man last}}}|| System manual for the {{{last}}} command which shows last logged in users|| ||<style="background:#F1F1ED;">{{{man less}}}|| System manual page for the {{{less}}} paging utility|| ||<style="background:#F1F1ED;">{{{man logger}}}|| System manual page for the {{{logger}}} command-line interface to syslog utility|| ||<style="background:#F1F1ED;">{{{man logrotate}}}|| System manual page for the the {{{logrotate}}} utility|| ||<style="background:#F1F1ED;">{{{man savelog}}}|| System manual page for the {{{savelog}}} log file saving utility|| ||<style="background:#F1F1ED;">{{{man syslogd}}}|| System manual page for the system log daemon ({{{syslogd}}})|| ||<style="background:#F1F1ED;">{{{man syslog.conf}}}|| System manual page for the {{{syslogd}}} configuration file|| ||<style="background:#F1F1ED;">{{{man tail}}}|| System manual page for the {{{tail}}} utility|| === WWW Resources === [http://www.samag.com/documents/s=1146/sam0109m/0109m.htm Checking Your System Logs with awk] [http://www.linuxvoodoo.com/resources/howtos/syslog/ Syslog - Watching Your Logs] [http://www-128.ibm.com/developerworks/linux/library/l-roadmap5/ Windows-to-Linux roadmap: Part 5. Linux logging (IBM)] [http://www.linuxsecurity.com/content/view/116430/151/ Sawing Linux Logs With Simple Tools] ---- CategoryDocumentation |
#REFRESH 0 http://help.ubuntu.com/community/LinuxLogFiles |
LinuxLogFiles (last edited 2008-08-06 16:32:05 by localhost)