Helpful log parsing tips

tux_awk1Most programs and services produce logs. When a user visits an apache web server, the service will most likely keep a log of that request, along with the date and requester’s ip address. Other details might be logged as well. Here us an example of some entries in a logfile:

192.168.1.20 - - [21/Sep/2011:11:04:40 +1000] "GET / HTTP/1.0" 200 468
192.168.1.20 - - [21/Sep/2011:11:07:48 +1000] "GET /login.php HTTP/1.0" 200 6433

Log files would usually contain hundreds of such entries, most, if not all of which are important to us. If there is an issue with a service, perhaps there is an entry in the logfile that can tell us why. Another scenario is where management require some statistical information. For example, how many unique IP addresses visited their website in the past hour and what pages did they visit. Or which web pages are the most frequently visited. Read more