Linux Log Searching - Search within date range in Linux/Unix logs - Search data in log files from specific range of dates

Searching through data from within a specific time frame in a log file

Common Log Searching Scenarios (on Linux, AIX, SunOS systems)

Syntax for the most common scenarios

./logXray.py <type-of-log> <path-to-logfile> <date-range-or-how-long-ago> '<string1>' '<string2>' 5 10 -show

####   (OR)   ####

./logXray.py <type-of-log> <path-to-logfile> <date-range-or-how-long-ago> '<string1>' '<string2>' 5 10 -stats

Real world case scenario of pulling data from logs from within a specific time period


###############################################################################################################
### Show 'fatal' entries logged in the access.log within the last 4 hours
###############################################################################################################
USAGE-A1: ./logXray accesslog /var/log/apache2/graphite-web_access.log 4h 'fatal' . 5 10 -show 
###############################################################################################################
### Show 'fatal' entries logged in the access.log file within the time specified.
###############################################################################################################
USAGE-A2: ./logXray accesslog /var/log/apache2/graphite-web_access.log 19/Jun/2018:07:04,22/Jun/2018:21:30 'fatal' . 5 10 -show

USAGE-B1: ./logXray cheflog /var/log/chef/client.log 2018-06-26T10:00:28,2018-06-26T11:00:28 'fatal' . 5 10 -show
USAGE-B2: ./logXray cheflog /var/log/chef/client.log 5h 'fatal' . 5 10 -show

USAGE-C1: ./logXray livestatuslog /var/log/nagios/livestatus.log '2018-06-26 10:00:28,2018-06-26 11:00:28' 'fatal' . 5 10 -show
USAGE-C2: ./logXray livestatuslog /var/log/nagios/livestatus.log 5h 'fatal' . 5 10 -show

USAGE-D1: ./logXray nagioslog /var/log/nagios/nagios.log 'Jun-19-2018-07:04:08,Jun-22-2018-21:30:05' 'fatal' . 5 10 -show
USAGE-D2: ./logXray nagioslog /var/log/nagios/nagios.log 5h 'fatal' . 5 10 -stats

USAGE-E1: ./logXray syslog /var/log/messages 'Jun-19-2018-07:04:08,Jun-22-2018:21:30:05' 'fatal' . 5 10 -show
USAGE-E2: ./logXray syslog /var/log/messages 5h 'fatal' . 5 10 -stats

[root@nagios001 EnScryption.com]#

Installing the logXray tool on your system



[root@nagios001 tmp]# unzip logXray.pythonscript.verify_support\@enscryption.com_634669252050760623552018.zip 
Archive:  logXray.pythonscript.verify_support@enscryption.com_634669252050760623552018.zip
  inflating: EnScryption.com/logXray.py  
[root@nagios001 tmp]# 
[root@nagios001 tmp]# 
[root@nagios001 tmp]# cd EnScryption.com/
[root@nagios001 EnScryption.com]# 
[root@nagios001 EnScryption.com]# ls
logXray.py
[root@nagios001 EnScryption.com]# 
[root@nagios001 EnScryption.com]# ./logXray.py 

====================================================================================================================
====================================================================================================================
SUCCESS: The unique EnScryption.com directory [ /var/tmp/EnScryption.com/SHIELDX-logXray.py ] was successfully created!
====================================================================================================================
====================================================================================================================
====================================================================================================================
SUCCESS: Created a link from [ /var/tmp/EnScryption.com/SHIELDX-logXray.py/logXray.py ] to [ ( PWD ) /tmp/EnScryption.com ]!
VERSION: Script [  /var/tmp/EnScryption.com/SHIELDX-logXray.py/logXray.py  ] was encrypted by EnScryption version [  1.18.06.18.0 ].
SUCCESS: Installation of [ protected ] script [  /var/tmp/EnScryption.com/SHIELDX-logXray.py/logXray.py  ] is now complete
====================================================================================================================

[root@nagios001 EnScryption.com]# 
[root@nagios001 EnScryption.com]# ./logXray.py 

###############################################################################################################
### Show 'fatal' entries logged in the access.log within the last 4 hours
###############################################################################################################
USAGE-A1: ./logXray accesslog /var/log/apache2/graphite-web_access.log 4h 'fatal' . 5 10 -show 
###############################################################################################################
### Show 'fatal' entries logged in the access.log file within the time specified.
###############################################################################################################
USAGE-A2: ./logXray accesslog /var/log/apache2/graphite-web_access.log 19/Jun/2018:07:04,22/Jun/2018:21:30 'fatal' . 5 10 -show

USAGE-B1: ./logXray cheflog /var/log/chef/client.log 2018-06-26T10:00:28,2018-06-26T11:00:28 'fatal' . 5 10 -show
USAGE-B2: ./logXray cheflog /var/log/chef/client.log 5h 'fatal' . 5 10 -show

USAGE-C1: ./logXray livestatuslog /var/log/nagios/livestatus.log '2018-06-26 10:00:28,2018-06-26 11:00:28' 'fatal' . 5 10 -show
USAGE-C2: ./logXray livestatuslog /var/log/nagios/livestatus.log 5h 'fatal' . 5 10 -show

USAGE-D1: ./logXray nagioslog /var/log/nagios/nagios.log 'Jun-19-2018-07:04:08,Jun-22-2018-21:30:05' 'fatal' . 5 10 -show
USAGE-D2: ./logXray nagioslog /var/log/nagios/nagios.log 5h 'fatal' . 5 10 -stats

USAGE-E1: ./logXray syslog /var/log/messages 'Jun-19-2018-07:04:08,Jun-22-2018:21:30:05' 'fatal' . 5 10 -show
USAGE-E2: ./logXray syslog /var/log/messages 5h 'fatal' . 5 10 -stats

[root@nagios001 EnScryption.com]#

Searching log data when the user-specified time period is actually FOUND in the log file

$ ./logXray.py livestatus /tmp/client.log '2016-05-08 19:12:00,2016-05-08 21:13:00' 'INFO' 'a2ensite' 5 10 -show
 
[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:43:08-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:43:11-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:42:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:42:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:42:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:43:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:43:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 21:12:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 21:12:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 21:12:59-07:00] INFO: execute[a2ensite default] ran successfully

23

2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM

Searching log data when the user-specified time period is NOT FOUND in the log file

$ ./logXray.py livestatus /tmp/client.log '2016-05-08 18:48:00,2016-05-08 19:54:00' 'INFO' 'a2ensite' 5 10 -show       
 
[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
8
1---78600---8---(2016-05-08)-(18:48)---(2016-05-08)-(19:54)---ETNF---(2016-05-08)-(19:12)---(2016-05-08)-(19:43)
$ echo $?
1

$ ./logXray.py notchef  /tmp/client.log   3h  'INFO' 'a2ensite'  5  10  -show                                     
[2016-05-08 19:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:13:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 19:42:57-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 19:42:57-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 19:42:57-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 19:43:08-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 19:43:11-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:12:58-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:12:58-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:12:58-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:13:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 20:42:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 20:42:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 20:42:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 20:43:09-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 20:43:12-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
[2016-05-08 21:12:59-07:00] INFO: Processing template[/usr/sbin/a2ensite] action create (apache2::default line 90)
[2016-05-08 21:12:59-07:00] INFO: Processing execute[a2ensite default] action run (apache2::default line 24)
[2016-05-08 21:12:59-07:00] INFO: execute[a2ensite default] ran successfully
[2016-05-08 21:13:10-07:00] INFO: Processing execute[a2ensite nagios3.conf] action run (logXrayServer::install line 24)
[2016-05-08 21:13:13-07:00] INFO: execute[a2ensite default] sending restart action to service[apache2] (delayed)
25
 
2---78660---25---(2016-05-08)-(18:51)---(2016-05-08)-(21:13)---ETNF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13)
$

Detailed explanation of this tool and the output it provides


Syntax:

./logXray.py (type-of-log)  (path-to-logfile)   (date-range-or-how-log-ago)  '(string1)'  '(string2)'  5  10  -show

Usage:

[root@jbowman-server]# ./logXray.py livestatuslog /tmp/client.log '2018-06-26 10:00:28,2018-06-26 11:00:28' 'INFO' 'a2ensite' 5 10 -show

2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13) SEAGM

[root@jbowman]#
[root@jbowman]#


So now lets break this down:

logXray.py is the tool name.

livestatus is an option that is passed to the logXray tool to tell it what to do. In this particular case, it is telling the tool what type of log file /tmp/client.log is.

/tmp/client.log is of course the log file.

'2018-06-26 10:00:28,2018-06-26 11:00:28' is the range of date from within the log that you wish to scan

"INFO" is one of the strings that is in the lines of logs that you're interested in.

"a2ensite" is another string on the same line that you expect to find the "INFO" string on. Specifying these two strings (INFO and a2ensite) isolates and processes the lines you want a lot quicker, particularly if you're dealing with a huge log file.

5 specifies Warning. By specifying 5, you're telling the program to alert as WARNING if there are at least 5 occurrences of the search strings you specified

10 specifies Critical. By specifying 10, you're telling the program to alert as CRITICAL if there are at least 10 occurrences of the search strings you specified.

-show specifies what type of response you'll get. By specifying -shown, you're saying if anything is found that matches the specified patterns, output to screen.


Summarized Explanation:

As you can see, the logXray tool is monitoring a log file. The arguments that are passed to the tool instructs it to do the following:

Get all entries written to the log between the dates '2016-05-08_19:12:00' AND '2016-05-08_21:13:00'.  If the tool finds less than 5 occurrences of the specified strings in the log file, DO NOT alert. If the tool finds between 5 to 9 occurrences of the specified strings in the log, it'll alert with a WARNING. If the tool discovers 10 or more instances of the strings in the log within the specified date range, it'll alert with a CRITICAL.

Now, let us look at the result of the command:

2---78720---23---ATWFILF---(2016-05-08)-(19:12)---(2016-05-08)-(21:13)

There are 6 columns which are separated by 3 hyphens (---). The first column shows the exit code of the command you just ran. 0 means all is well. 1 means WARNING, which means, LOGROBOT discovered conditions that fell under the WARNING specification you provided. 2 means CRITICAL, which means, the worst case scenario has been reached.

In this particular example, here's what the output is telling us: 

You requested to have the /tmp/client.log file scanned for a specific date.

The date range that was scanned was from '2016-05-08_19:12:00' AND '2016-05-08_21:13:00'. After scanning through the records that were written to the log in that time frame, LOGROBOT found 23 lines that contained both strings of "INFO" and "a2ensite".

ATWF means that the actual date range or time frame you requested searched was found in the log.  So this is very good.

ETNF means the actual date range or time frame you requested searched was NOT found in the log.  In this case, the closest time to the time you specified will be detected and used instead.

logXray - Date Range Scan

Search log files within a specific time period. Specify range of time to search in when scanning the log.

When the user-specified patterns are found in the time frame provided, choose to see the actual entries from the log, or see numerical statistics.

If the time frame provided is not found in the log, logXray will analyze and identify the closest available times and will print data from that period.

Additionally, if necessary, alert on the output provided by the command..i.e. alert as critical if X number of entries are found within X time period.