Searching through data from within a specific time frame in a log file

Common Log Searching Scenarios (on Linux, AIX, SunOS systems)

 ./autodrgrep.kl.sh <type-of-log> <path-to-logfile> <date-range-or-how-long-ago> '<string1>' '<string2>' 5 10 -show
Example:

Search date range (2016-05-08_18:48:00) to (2016-05-08_19:54:00) for all entries that contain both "INFO" and "a2ensite" on the same line

 ./autodrgrep.kl.sh notchef /tmp/client.log '2016-05-08_18:48:00,2016-05-08_19:54:00' 'INFO' 'a2ensite' 5 10 -show
 
OR
 
Get data written to the log within the last 3 hours and search for all entries that contain both "INFO" and "a2ensite" on the same line

 ./autodrgrep.kl.sh notchef /tmp/client.log 3h 'INFO' 'a2ensite' 5 10 -show
Installation & Sample Runs:

$ cd /tmp/
$
$
$ unzip autodrgrep.kl.sh.verify_Support\@KingLazy.com_1927649707070582148232016.zip
Archive:  autodrgrep.kl.sh.verify_Support@KingLazy.com_1927649707070582148232016.zip
  inflating: KingLazySHIELD/README.lcs 
  inflating: KingLazySHIELD/autodrgrep.kl.sh 
  inflating: KingLazySHIELD/install.sh 
  inflating: KingLazySHIELD/lgrguaransct.ee.eez 
 extracting: KingLazySHIELD/license.lcs 
  inflating: KingLazySHIELD/license.logrobot.autodrgrep.kl.sh 
$ cd KingLazySHIELD/
$
$
$ ./install.sh  /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh  /home/users/jbowman  -force
======================================================================================================
======================================================================================================
SUCCESS: The logXray Directory [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh ] Was Succesfully Created and is now Owned by the User [ root ] - Enjoy!
======================================================================================================
======================================================================================================
SUCCESS: License file [ license.lcs ] was copied successfully to the designated directory [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh ]!
SUCCESS: [ license.logrobot.autodrgrep.kl.sh ] was copied successfully to the designated directory [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh ]!
SUCCESS: [ autodrgrep.kl.sh ] was copied successfully to the designated directory [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh ]!
SUCCESS: Created [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh/engines/.lgrguaransct.ee.eez ] without any issues!
SUCCESS: Created [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh/engines/.lgrguaransct.ee ] without any issues!
======================================================================================================
======================================================================================================
SUCCESS: Created a link from [ /var/tmp/KINGLAZY/SHIELDX-autodrgrep.kl.sh/autodrgrep.kl.sh ] to [ /home/users/jbowman ] without any issues!
======================================================================================================
======================================================================================================
$
$
$ ./autodrgrep.kl.sh notchef /tmp/client.log '2016-05-08_19:12:00,2016-05-08_21:13:00' 'INFO' 'a2ensite' 5 10 -show
 
[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:43:08-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:43:11-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:42:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:42:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:42:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:43:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:43:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 21:12:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 21:12:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 21:12:59-07:00] INFO: execute[a2ensite default] ran successfully

23

2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM
$ ./autodrgrep.kl.sh notchef  /tmp/client.log   '2016-05-08_18:48:00,2016-05-08_19:54:00' 'INFO' 'a2ensite'  5  10  -show       
 
[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
8
1---78600---8---(2016-05-08)-(18:48)---(2016-05-08)-(19:54)---ETWNFILF---(2016-05-08)-(19:12)---(2016-05-08)-(19:43) NAGAV12736a
$ echo $?
1
$ ./autodrgrep.kl.sh notchef  /tmp/client.log   3h  'INFO' 'a2ensite'  5  10  -show                                     
[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:43:08-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:43:11-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:42:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:42:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:42:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:43:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:43:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 21:12:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 21:12:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 21:12:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 21:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 21:13:13-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
25
 
 
2---78660---25---(2016-05-08)-(18:51)---(2016-05-08)-(21:13)---ETWNFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) NAGAV12q736zx
$

About the Tool:

Syntax:

./autodrgrep.kl.sh (type-of-log)  (path-to-logfile)   (date-range-or-how-log-ago)  '(string1)'  '(string2)'  5  10  -show


Usage:

[root@monitor jbowman]#
[root@monitor jbowman]#  ./autodrgrep.kl.sh notchef  /tmp/client.log   '2016-05-08_19:12:00,2016-05-08_21:13:00' 'INFO' 'a2ensite'  5  10  -show

2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM

[root@monitor jbowman]#
[root@monitor jbowman]#


So now lets break this down:

autodrgrep.kl.sh is the tool name.

notchef is an option that is passed to the logrobot tool to tell it what to do. In this particular case, it is telling the tool what type of log file /tmp/client.log is.

/tmp/client.log is of course the log file.

2016-05-08_19:12:00,2016-05-08_21:13:00 is the range of date from within the log that you wish to scan

"INFO" is one of the strings that is in the lines of logs that you're interested in.

"a2ensite" is another string on the same line that you expect to find the "INFO" string on. Specifying these two strings (INFO and a2ensite) isolates and processes the lines you want a lot quicker, particularly if you're dealing with a huge log file.

5 specifies Warning. By specifying 5, you're telling the program to alert as WARNING if there are at least 5 occurrences of the search strings you specified

10 specifies Critical. By specifying 10, you're telling the program to alert as CRITICAL if there are at least 10 occurrences of the search strings you specified.

-show specifies what type of response you'll get. By specifying -shown, you're saying if anything is found that matches the specified patterns, output to screen.

Summarized Explanation:

As you can see, the logrobot tool is monitoring a log file. The arguments that are passed to the tool instructs it to do the following:

Get all entries written to the log between the dates '2016-05-08_19:12:00' AND '2016-05-08_21:13:00'.  If the tool finds less than 5 occurrences of the specified strings in the log file, DO NOT alert. If the tool finds between 5 to 9 occurrences of the specified strings in the log, it'll alert with a WARNING. If the tool discovers 10 or more instances of the strings in the log within the specified date range, it'll alert with a CRITICAL.

Now, let us look at the result of the command:

2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM

There are 6 columns which are separated by 3 hyphens (---). The first column shows the exit code of the command you just ran. 0 means all is well. 1 means WARNING, which means, LOGROBOT discovered conditions that fell under the WARNING specification you provided. 2 means CRITICAL, which means, the worst case scenario has been reached.

In this particular example, here's what the output is telling us: 

You requested to have the /tmp/client.log file scanned for a specific date.

The date range that was scanned was from '2016-05-08_19:12:00' AND '2016-05-08_21:13:00'. After scanning through the records that were written to the log in that time frame, LOGROBOT found 23 lines that contained both strings of "INFO" and "a2ensite".

ATWFILF means that the actual date range or time frame you requested searched was found in the log.  So this is very good.

ETWNFILF means the actual date range or time frame you requested searched was NOT found in the log.  In this case, the closest time to the time you specified will be detected and used instead.

Other common log monitoring scenarios