Questions tagged [log-files]

A log file is a file, often in clear text, which is generated by a software process to record some specific details about the workings of this process.

A log file is a file which is generated by a software process to record some specific details (events) about the workings of this process. Most logfiles are plain, human-readable text files, some however are using a binary format to conserve space and need specific treatment for post-processing. (Example: Unix login history in utmp/wtmp format)

A log file can be categorized into several types:

  1. Access log which records information related to the usage of the service provided by this process.
  2. Error log which records error conditions.
  3. Info/Debug log which records more detailed information as compared with other logs.
  4. Transaction log to document a chain of events, this often is required to be immutable to avoid later tampering.
  5. redo logs, containing the complete information to replay a chain of actions, often used where incomplete data transactions would lead to inconsistent data, for example in file systems or databases.

The main two goals of creating and maintaining log files are:

  1. Finding and debugging problems.
  2. Gathering usage information.

Depending on the frequence of events and the level of detail, logfiles may be growing very quickly, and usually require a strategy for managing their growth.

Depending on special reqirements, log-files may go to specialized devices to meet criteria of speed or immutability.

1103 questions
175
votes
7 answers

How can I read pcap files in a friendly format?

a simple cat on the pcap file looks terrible: $cat tcp_dump.pcap ?ò????YVJ? JJ ?@@.?E
Tony
  • 4,213
  • 11
  • 35
  • 29
110
votes
1 answer

InnoDB: Error: log file ./ib_logfile0 is of different size

I just added the following lines in /etc/mysql/my.cnf after I converted one database to use InnoDB engine. innodb_buffer_pool_size = 2560M innodb_log_file_size = 256M innodb_log_buffer_size = 8M innodb_flush_log_at_trx_commit =…
jack
  • 1,725
  • 5
  • 21
  • 25
109
votes
1 answer

What is the purpose of the ".well-known"-folder?

If've found a new error message in our log files and would like to know, for what this .well_known folder is used for. Which application/client would need to access such a folder and which application would create files inside it? Here are some…
Sascha
  • 1,214
  • 2
  • 9
  • 7
95
votes
9 answers

How can I fully log all bash scripts actions?

From my script output I want to capture ALL the logs data with error messages and redirect them all to log file. I have script like below: #!/bin/bash ( echo " `date` : part 1 - start " ssh -f admin@server.com 'bash…
BlueMark
  • 1,089
  • 1
  • 9
  • 8
94
votes
7 answers

My /var/log/btmp file is huge! What should I do?

My /var/log/btmp file is 1.3 GB in size. I've read that the file is "Used to store information about failed login". What does this mean for my server? And can I delete this file?
Juddling
  • 1,075
  • 1
  • 7
  • 11
85
votes
11 answers

Where is my mysql log on OS X?

I checked /var/log and /usr/local/mysql and i can't seem to find the log. I am trying to troubleshoot an error establishing a database connection with a php function.
Tony
  • 4,213
  • 11
  • 35
  • 29
75
votes
16 answers

Alternatives to Splunk?

I'm pretty impressed with Splunk, especially version 4. Pretty graphs, alerting (Enterprise only), and fast, accurate, searching. It's a great product. However, the cost just way too high to consider for full production use for our company. All we…
MichaelGG
  • 1,739
  • 8
  • 25
  • 30
67
votes
4 answers

What is the difference between syslog, rsyslog and syslog-ng?

I am a bit confused in syslog, rsyslog and syslog-ng. From where can I get the source code for syslog()? Is there any difference between rsyslog and rsyslogd?
StackUser
  • 803
  • 1
  • 7
  • 7
60
votes
13 answers

How to monitor a windows log file in real time?

On windows how can I easily monitor a log file and see updates to the file in real time? Basically, same functionality like tail -f log_file on Unix systems Platform: Windows XP/2003/2008 server [Update] this is quite handy for a quick…
dance2die
  • 2,011
  • 7
  • 32
  • 41
43
votes
2 answers

Understanding a Postfix log file entry

We have a Postfix hub and I'm trying to better understand the information in the mail.log file. I use tools like qshape, pflogsumm.pl and amavis-logwatch to summarize the log files, but I have still have questions about some of the elements of the…
KodeTitan
  • 881
  • 2
  • 10
  • 15
39
votes
8 answers

Colorize Monitoring of Logs

I sometimes monitor apache and php error logs using tail under FreeBSD. Is there any way to get colorized output, either using tail or some other command line app? Alternatively, what is your favorite way to monitor the various web-related logs in…
Ian
  • 1,498
  • 4
  • 26
  • 32
35
votes
3 answers

Grep in a huge log file (>14 GB) only the last x GB?

I need to search something in a huge log-file (over 14 GB). I'm pretty sure it's in the last 4 GB or so. Is there a way to skip the first X GB to speed things up?
Roger
  • 800
  • 1
  • 6
  • 17
33
votes
6 answers

How can I display and log PHP errors on IIS7?

We're running PHP 5.2.5 on an IIS 7 Server and we're having problems making PHP errors visible... At the moment whenever we have a PHP error the server sends back a 500 error with the message "The page cannot be displayed because an internal server…
Ben
  • 463
  • 1
  • 6
  • 8
32
votes
1 answer

How to set up fail2ban to read multi log in a jail?

How can I configure multiple logpaths for the same rule? I'm trying to write a syntax like this: [apache-w00tw00t] enabled = true filter = apache-w00tw00t action = iptables-allports logpath = /var/log/apache*/*error.log logpath =…
Max121
  • 343
  • 1
  • 3
  • 6
31
votes
6 answers

Apache in Docker: How do I "access.log"?

I'm just getting started with Docker and richt now I'm trying to figure out how to set up my first dockerized Apache 2 / PHP environment. Up to now I have been using full Linux VMs, where I used log-files being written to /var/log/apache2, then use…
BlaM
  • 3,886
  • 5
  • 27
  • 28
1
2 3
73 74