hit counter


LoGrobot / logXray

What is LoGrobot/logXray? LoGrobot is a powerful, fully loaded Linux/Unix Log Monitoring, Analysis & Alerting solution. It is designed to simplistically centralize the monitoring of local and remote log files. It is highly versatile and can be used to monitor and alert on all types of logs; application logs, database logs, system logs, event logs and service logs. It can monitor single and multiple log files and alerts when log patterns (single or multiple patterns) are detected. Additionally, it allows for custom log monitoring tailored to specific individual user requirements. logXray on the other hand provides on-demand graphs and automated analysis which can be used to draw out several important statistics, reveal hidden metrics and quickly isolate problematic trends. When Splunk (and similar applications) are overkill for your daily log monitoring needs, you turn to LoGrobot!


LoGrobot: Who needs it?! Download LoGrobot if you wish to...

Monitor/Alert

Get notified when specific strings, patterns or keywords are dumped into your log files

Get notified when expected patterns of strings ARE NOT found within specific log files

Get notified when unfamiliar lines are introduced into your system and/or application logs

Get notified when critical log files stop getting written to after a user-specified period of time

Monitor/Graph

Graph specific metrics about your logs to enable the ability to isolate and predict problems before they occur

Get notified if a log file is moved, deleted or no longer exist where it should

Get notified when a log file lacks the proper permissions that will allow read access

Generate Alerts if log files begins to consume too much disk space - Growth Spurts

Utilize a reliable log monitor that is maintained regularly & used in real production environments

Dissect/Analyze

Scan logs for specific entries and exclude user specified patterns from the result

Monitor any log file (regardless of size) that lacks a consistent date and/or time format

Monitor multiple logs without the nuisance of complex time consuming configurations

Monitor not just log content, but log time stamp, log size, stale logs, a directory of logs and directory file count as well

Reporting/Benefits

Generate quick color coded excel reports on past notification alerts on all monitored log files - avoid digging through archives!

Implement a clean log monitoring solution that does not require the installation of nonnative modules

Avoid having to read endless pages of convoluted documentation or wasting time training staff on new tools

Have a technical support team available to accommodate your log monitoring requirements

Utilize an advanced intuitive log monitor that eliminates the need to maintain complex configuration files



Monitor/Alert

Monitor any log file on any Unix host regardless of format, use with Nagios (or other monitoring applications) for management and scheduling of log checks, Use with Cacti or Graphite (for graphing log metrics), Crontab (for scheduling & emailing of log alerts)

Monitor/Graph

Compatible with other Monitoring applications, Allows for the custom modification of existing features and the addition of new ones to accommodate unique situations

Quick & Clean Automated Install - Requires only an "unzip" - No complicated compilations to deal with, no dubious modules or libraries to download...EVER!

Dissect/Analyze

Consolidate all log monitoring tasks; Use one tool to Alert on the frequency (or lack thereof) of any set of keywords, strings or patterns, Monitor log file size, Log timestamps, Stale logs

Automated Management: Monitor logs on all unix hosts in your environment with just One Tool & One Master Server, Keep an unwavering eye on all aspects of your environment!

Reporting/Benefits

Graph & Analyze a limitless number of logs including HTTP / Apache / Tomcat / JAVA / MySql / Oracle / Postfix / Mail / Weblogic / System & Applicaiton Log Files



















Simplified Log File Monitoring

Linux Log Monitoring ; Monitor, Alert on & Analyze Linux / Unix Log files the easy way ; Application Logs, Database logs, System Logs, Custom Logs, Any log file - Genenerate graphs automatically on all monitored logs - Trend any log file metric you desire - Utilize the versitility of LoGrobot to eliminate the tedious effort often required to configure log checks - Perform all log monitoring tasks with just one tool!

Some of the many labor saving capabilities and benefits of LoGrobot include:

  • Watch a directory of log files, detect exceptions/errors in monitored logs

  • PlugnPlay command-line parameters (avoid dealing with scattered configs)
  • Adapts seamlessly to any custom scenario - Usable as a plugin, service or both
  • Monitors dynamic log files efficiently (log files with changing names / dates)
  • Monitors different patterns in single or different log files, with one check
  • Hot Spot Analysis: Identify times when there is unusually high # of log statements
  • Monitors log file timestamps, log file growth, log file size and directory file count
  • Assignment of different thresholds to patterns in a multi-pattern log check
  • A clean log monitor that does not require installation of nonnative modules
  • Scanning, monitoring and alerting on log files of any format, type or size
  • ON-Demand graphs for insight on the health of your App, DB, Network
  • Can alert based on values in certain columns of specific log entries
  • Monitoring rotated logs automatically - never miss events between logchecks!
  • Scan logs in time frames (i.e. show entries within previous 20 mins, 1 hr...etc)
  • Remote agent included for monitoring of remote logs from ONE master server
  • Alert when expected log events are NOT found within a set period of time
  • Monitor all log files or specific types of log files in a particular directory
    • Point logxray to ANY directory with just one check!

      • Avoid having to define separate checks for each log file

    • Specify the type of files to exclude / include in monitoring

      • Assign different thresholds for each file type

  • Easily integrated with other monitoring apps i.e. Nagios, Zenoss, Zabbix
  • Use one tool to monitor anything and everything about your Unix log files!
  • Request development of custom personalized log monitoring features
    • Allows passing of different thresholds to each pattern being monitored

    • Allows for the filtering of specific log entries to eliminate unnecessary noise

  • Buy / Download


How-To Videos on Common Tasks

How to use logrobot/logxray to monitor single or multiple patterns in single or multiple log files, alert on stale logs (logs that arent growing in size or getting updates), monitor file count in a directory.

Print X Number Around Pattern

Monitor a log for a specific pattern. When that pattern is found, print X number of lines before the pattern and X number oflines after the pattern. If multiple patterns are found in the log, perform these instructions on each one of them.

Watch

Directory Log Monitor

Monitor and alert on patterns, strings or keywords found in all logs in a specific directory. Avoid having to create different/separate logchecks for each file. Easily configure log monitoring checks on a large scale with very little effort.

Watch

Growth Monitoring

Log File Growth Monitoring - Monitor the growth of a log file to ensure the log is getting updates. Alert when the growth and/or growth rate of a monitored log is determined to be too slow, too fast or just stale.

Watch


File Size / Log Size

Monitor file size and alert if file size is greater than user specified thresholds. Generate notification alerts on logs. Monitor the size of log files of any application or database on Unix systems. Trend log file size (feature available). Works on Unix (Linux/AIX/SunOS/HP-UX/MacOS).

Watch

Directory File Count Monitor

Monitor and alert on the number of files in a specific directory. Generate notification alerts when the file count of a directory breaches user-specified thresholds. Tested on Unix (Linux/AIX/SunOS/HP-UX).

Watch

File / Log Timestamp

Timestamp Monitor - Monitor the time stamp of single or multile logs / files on a Unix system. If the age of the file you're monitoring is older than a predetermined number of minutes, hours, days...

Watch






 

Other Specific Features:

  • Scan / Scrape / Monitor log files for any error

  • Monitor all logs in a specific directory

    • Point logrobot to ANY directory with just one check!

      • Avoid having to define separate checks for each log file

    • Specify the type of files to exclude / include in monitoring

  • Automate log checks via Nagios or CRONTAB

  • Get email alerts & notifications on all log checks

  • Monitor Directory File Count***

  • Access documentation directly from the CLI

  • Monitor log files for abnormal behavior/activity

  • Manage log file checks from a central location

    • Eliminate tedious administration

    • Avoid cumbersome maintenances

  • Adapts seamlessly to any custom scenario
     

 


 

Supported Log Files

  • Will all my logs be supported?

    Yes, all log types / log formats are supported. 

    Some of the supported log files are listed below:

    • Tomcat Catalina.out logs

    • Apache Maxclient logs

    • Apache access logs

    • Apache error logs

    • OutOfMemory logs

    • JBoss log files

    • Java log files

    • Weblogic logs

    • Glassfish logs

    • Syslog log monitor

    • Maillog / Postfix / Syslog log files

    • Mysqld / Oracle Alert logs

    • Log4j

    • NEW FEATURE: Monitor any type of log file regardless of format
       

What is a Log File Monitor

A log file monitor is a utility designed and built specifically to monitor and alert on messages produced by computer systems and the applications that run on them.

In UNIX, the monitoring of log files is absolutely necessary, and for good reason. You see, the time of a Unix Professional is valuable. Few, if any, can afford to spend hours each day scouring through the many log files that are generated by systems and network applications. However, if you fail to quickly recognize the abnormal or fatal events chronicled in these log files, entire networks can be abused and/or removed from service....which can cost your company dearly, monetarily speaking.

If you wish to monitor log files, there are basically [ 3 ] options available to you:
  • You can try writing your own log monitoring script and see how far that takes you (this is worth looking into if you only have a couple of logs to monitor)
  • Download any of the FREE log monitoring scripts that are available all over the internet OR
  • Purchase a professional tool that was developed specifically for situations like yours and that can easily accommodate future customizations, if necessary

If you embark on a journey to write your own script, you have to understand that it will be an endeavor that will take years to complete, and that's assuming you're a skilled programmer. Monitoring log files goes far beyond simply watching the contents of files for specific errors. As time goes on, there will be new requirements, changes, and continuous requests for modifications which in the end, if the developer isn't creative, can lead to an unusable script - one that is not user friendly.

If you choose to download the FREE log monitoring scripts that are available on the internet, you will quickly discover how ineffective they all are and how much work is necessary to get them to cooperate. If this is the option you choose to go with, you must ask yourself some very important questions:

  • Will I be able to easily administer the creation and modification of several log checks (from a central location) using this method?
  • Is this method scalable?
    • Can I use this one method to monitor different logs on several hundred servers, or am I going to have to do a lot of configurations, compilations, installations, tweaking etc?

The answers to these questions are usually quite depressing. Proceed with caution.

Characteristics of the Ideal Log Monitor:

When searching for the right utility to use to monitor & alert on log files, what features should the perfect tool have?

The ideal log monitor must be able to scan and monitor log files in a very short period of time, preferably in seconds (no matter how big the log file is). At the very least, the perfect log monitor must be able to:

  • Detect abnormal usage patterns in log files
  • Recognize system or network abuse (through mathematical analysis of data)
  • Detect vulnerability scans (e.g. port scans) through the use of user-specified patterns
  • Detect intruders or attempted intrusions (through the use of user-specified patterns)
  • Detect resource shortages (e.g. slow response times, out-of-memory conditions etc)
  • Detect imminent application and system failures (this is usually in some log file on your system)
  • Scan, monitor & alert on log files of different formats (this is absolutely crucial)

While each feature listed here is important, it is worth noting that above all else, the perfect log monitoring utility must be easy to use. Users SHOULD NEVER have to spend too much time reading documentations before being able to utilize a software. The more complex a utility is, the more likely it is to be used the wrong way or abandoned altogether. Imagine having to re-read the Instruction Guide of your Television remote control each time you wanted to use it. Can you picture the annoyance of that?

When it comes to log monitoring, ease of use is essential. I cannot stress this enough. The developer(s) must focus a great deal of effort into drastically limiting or eliminating the need for configuration files. Also, the syntax of the tool must be easily comprehensible and applicable directly from the command line. This means, if a random user were to run the tool from the command line, there shouldn"t be room for confusion. That user should be able to conveniently obtain whichever end result he/she was expecting WITHOUT having to read several pages of complex documentations or desperately scouring Google for help!

This is where the superiority of LoGrobot comes into play. LoGrobot is a commercial Log Monitoring utility that is very easy to utilize. It is robust, seasoned and efficiently versatile like no other tool. It understands the overriding significance of log alerts and focuses on ensuring only valid notifications are generated for the log files it monitors. Installation wise, LoGrobot does not require the addition of any nonnative modules or libraries to the system. Which means, you can install it freely on production/dev/qa servers without tampering with existing libraries or modules.

LoGrobot has a wide range of capabilities. It isn't limited to just scanning log file contents for errors. It can do virtually anything as long as it falls under the banner of log monitoring. Additionally, LoGrobot has years of real life situations, scenarios, possibilities and conditions built into it, which basically means it is highly unlikely you will come up with a need that hasn't already been thought of and programmed into the tool. In the unlikely event that does happen, chances are, work is already in progress to address it.

When it comes to keeping an unwavering eye on all important log files in your UNIX environment, you need ONE log monitoring tool, and LoGrobot is that tool!





Monitoring logs in time frames (if format is supported)


logrobot autofig (logfile) (time-in-minutes) '(string1)' '(string2)' (warn) (critical) (-foundn)

Basic Usage: 

[root@monitor jbowman]#
[root@monitor jbowman]#
[root@monitor jbowman]# logrobot autofig /var/log/messages 1440 'ntpd' 'stratum' 5 10 -foundn
 
2---240---108---ATWFILF---(Apr/13)-(03:35)---(Apr/14)-(03:35:23)

[root@monitor jbowman]#
[root@monitor jbowman]#

So now lets break this down:

logrobot is the tool name.

autofig is an option that is passed to the logrobot tool to tell it what to do.  In this particular case, autofig is instructing logrobot to "automatically figure out" what type of log file /var/log/messages is, and if the format of the log file is supported, perform the remaining functions.

/var/log/messages is of course the log file.

1440 is the amount of previous minutes you want to search the log file for. 1440 = last 24 hours.

"ntpd" is one of the strings that is in the lines of logs that you're interested in.

"stratum" is another string on the same line that you expect to find the "ntpd" string on. Specifying these two strings (luance and Err1310) isolates and processes the lines you want a lot quicker, particularly if you're dealing with a huge log file.

5 specifies Warning. By specifying 5, you're telling the program to alert as WARNING if there are at least 5 occurrences of the search strings you specified, in the log file within the last 60 minutes.

10 specifies Critical. By specifying 10, you're telling the program to alert as CRITICAL if there are at least 10 occurrences of the search strings you specified, in the log file within the last 60 minutes.

-foundn specifies what type of response you'll get. By specifying -foundn, you're saying if anything is found that matches the specified strings within the 60 minute time frame, then that should be regarded as a problem and outputted out.

Summarized Explanation:

As you can see, the logrobot tool is monitoring a log file. The arguments that are passed to the tool instructs it to do the following:

Within the last 60 minutes, if the tool finds less than 5 occurrences of the specified strings in the log file, DO NOT alert. If the tool finds between 5 to 9 occurrences of the specified strings in the log, it'll alert with a WARNING. If the tool discovers 10 or more instances of the strings in the log within the last 60 minutes, it'll alert with a CRITICAL.

Now, let us look at the result of the command:

2---240---108---ATWFILF---(Apr/13)-(03:35)---(Apr/14)-(03:35:23)

There are 6 columns which are separated by 3 hyphens (---).  The first column shows the exit code of the command you just ran.  0 means all is well. 1 means WARNING, which means, LoGrobot discovered conditions that fell under the WARNING specification you provided.  2 means CRITICAL, which means, the worst case scenario has been reached.

In this particular example, here's what the output is telling us: 

You requested to have the /var/log/messages file scanned as far back as 24 hours ago (1440 minutes).

The timeframe that was scanned was from [ April 13, 03:35 ] to [ April 14, 03:35 ].  After scanning through the records that were written to the log in that time frame, LoGrobot found 108 lines that contained both strings of "ntpd" and "stratum 2".  Also, as an FYI, the last date and time those specific strings were found in the log file was 240 seconds ago.
							

Case Scenario:

Within the last 30 minutes, find out how many lines in the log file [ /var/log/app.log ] contain both entries of "ERROR" and "Client". If any lines are found containing these two strings (ERROR.*Client), take note of that.

From the list of lines found, see if there are any lines that also contain the keywords "error 404" OR "updateNumber".  If there are, remove them from the list.  After removing them, show me what is left.  If the number of lines left is between 5 and 9, alert as WARNING.  If equal to or over 10, alert as CRITICAL.  If below 5, do not alert!

Command:

logrobot autofig /var/log/app.log 30 "ERROR.*Client" '(error 404|updateNumber)' 5 10 -showexcl


Case Scenario:

For instance, within the last 30 minutes, if LoGrobot does not find at least 2 lines containing the words "Success" and "Client"  and "returned 200" OR "update:OK" in the log file, it must alert.  So in other words, the lines to search for MUST contain both words of Success & Client (Success.*Client) AND one or both of the strings returned 200 and update:OK.

Command:

logrobot autofig /var/log/app.log 30 "SUCCESS.*Client" '(returned 200|update:OK)' 2 2 -notfoundn
							

This is particularly helpful in cases where you might want to see the actual lines that contain the patterns you instructed the tool to search for.

Example:

logrobot  autofig  /var/log/app.log  30  "ERROR.*Client"  '(error 404|updateNumber:OK)'  5  10  -show

Example:

logrobot  autofig  /var/log/app.log  30  "SUCCESS.*Client"  '(returned 200|update:OK)'   5  10  -show
							

For instance, to pull out 2 weeks of information from within a large log file and to find out how many lines contain certain strings and patterns, you can run a command similar to this:

Example:

logrobot autofig /var/log/app.log 2w "ERROR|error|panic|fail" "ERROR|error|panic|fail" 5 10 -foundn

Notice the [ 2w ].  And also, notice the strings being searched for.  I repeated the strings "ERROR|error|panic|fail" twice because there is no need to specify different search terms to look for.  You don't have to repeat the first string.  You can just enter a dot in its place for the second string..i.e:

logrobot  autofig  /var/log/app.log  2w  "ERROR|error|panic|fail"  "."  5  10  -foundn

From this specific example, I'm telling LoGrobot that I care about EVERY single line that contains any of the keywords I provided.  The [ 2w ] of course means 2 weeks. 
 
See below for the different ways of specifying the date range:

5m = 5 minutes (changeable to any number of minutes)

10h = 10 hours (changeable to any number of hours)

2d = 2 days (changeable to any number of days)

2w = 2 weeks (changeable to any number of weeks)

3mo = 3 months (changeable to any number of months)
							

Suppose you inherited a Unix environment at your new job and don't know what to search for in the logs, here's an idea; instead of worrying about what to watch for, why not force the logs to reveal their hidden contents?

In the example below, LoGrobot was instructed to search the entire messages file (denoted with the '.').  Then, it is to ignore every line that contains any one of these specific strings: 'nagios-primary nagios' OR 'not responding' OR 'synchronized to'.  Whatever lines are left after these THREE patterns are ignored should be outputted to the screen.  The logic here is; if you can identify which entries in the logs are of NO importance to you, you can exclude them from being monitored.  Therefore, if a log file is stripped of the familiar/unwanted, whatever is left will be unfamiliar, thus requiring investigation.
 
[root@nagios-primary ~]# logrobot sanal /var/log/messages 24h '.' 'nagios-primary nagios|not responding|synchronized to' 1 5 -showexcl

Jun 13 13:40:04 nagios-primary abrt[8269]: saved core dump of pid 8128 (/prod/nagios-core/sbin/status.cgi)
Jun 13 13:40:04 nagios-primary abrtd: Directory 'ccpp-2012-06-13-13:40:04-8128' creation detected
Jun 13 13:40:04 nagios-primary abrtd: Executable '/prod/nagios-core/sbin/status.cgi' doesn't belong to any
Jun 13 13:40:04 nagios-primary abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-06-13-13:40:04
Jun 14 02:20:41 nagios-primary auditd[5813]: Audit daemon rotating log files

2---0---(93)-(41064)-(0.226476%)-(28.4323)-(422.97)---ATWFILF---(Jun/13)-(13:23)---(Jun/14)-(13:23:26) 
							

Instead of forcing users to have to read complex documentations, LoGrobot provides real life examples of its usage right from the command line. Yes, REAL LIFE EXAMPLES! No guessing, no confusion, no scratching of the head. We strongly believe in simplicity and we take the extra steps many utilities refuse to take.

In the below output, let's say you forgot how to use the LoGrobot tool. Instead of having to find the documentation and then having to read it as well, you can just run the the tool from the command line and pass to it the option you're interested in...i.e. autofig (or you can type 'auto' to get more information on other available features).

Example:

[root@nagios-primary ~]#  ./logrobot  autofig

-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
Scan log file for 30 minutes worth of information. Show all lines found containing 'ERROR'
-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------

EXAMPLE:

./logrobot  autofig  /var/log/messages  30m   'ERROR'   '.'   5  10  -show

-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
							

Scan the /var/log/messages log file for 24 hours worth of information.  Exclude all lines that contain 'nagios-primary nagios | not responding, timed out| synchronized to'

[root@nagios-primary ~]# logrobot  sanal  /var/log/messages  24h  '.'  'nagios-primary nagios|not responding, timed out| synchronized to'  1  5  -showexcl


Jun 13 13:40:04 nagios-primary abrt[8269]: saved core dump of pid 8128 (/prod/nagios-core/sbin/status.cgi) to /var/spool/abrt/ccpp-2012-06-13-13:40:04-8128.new/coredump (2490368 bytes)
Jun 13 13:40:04 nagios-primary abrtd: Directory 'ccpp-2012-06-13-13:40:04-8128' creation detected
Jun 13 13:40:04 nagios-primary abrtd: Executable '/prod/nagios-core/sbin/status.cgi' doesn't belong to any package
Jun 13 13:40:04 nagios-primary abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-06-13-13:40:04-8128 (res:2), deleting
Jun 14 02:20:41 nagios-primary auditd[5813]: Audit daemon rotating log files

2---0---(93)-(41064)-(0.226476%)-(28.4323)-(422.97)---ATWFILF---(Jun/13)-(13:23)---(Jun/14)-(13:23:26) ZEAGMK

[root@nagios-primary ~]#
[root@nagios-primary ~]#
[root@nagios-primary ~]#

Scan the /var/log/messages log file for 1 week's worth of information.  Show me all lines that contain the strings: 'nagios-primary abrtd:'

[root@nagios-primary ~]# logrobot sanal /var/log/messages 1w '.' 'nagios-primary abrtd:' 1 5 -show

Jun 10 19:45:34 nagios-primary abrtd: Directory 'ccpp-2012-06-10-19:45:34-19662' creation detected
Jun 10 19:45:35 nagios-primary abrtd: Executable '/prod/nagios-core/sbin/status.cgi' doesn't belong to any package
Jun 10 19:45:35 nagios-primary abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-06-10-19:45:34-19662 (res:2), deleting
Jun 12 07:07:03 nagios-primary abrtd: Directory 'ccpp-2012-06-12-07:07:02-30780' creation detected
Jun 12 07:07:03 nagios-primary abrtd: Executable '/prod/nagios-core/sbin/status.cgi' doesn't belong to any package
Jun 12 07:07:03 nagios-primary abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-06-12-07:07:02-30780 (res:2), deleting
Jun 13 13:40:04 nagios-primary abrtd: Directory 'ccpp-2012-06-13-13:40:04-8128' creation detected
Jun 13 13:40:04 nagios-primary abrtd: Executable '/prod/nagios-core/sbin/status.cgi' doesn't belong to any package
Jun 13 13:40:04 nagios-primary abrtd: Corrupted or bad dump /var/spool/abrt/ccpp-2012-06-13-13:40:04-8128 (res:2), deleting

2---81900---(9)-(176115)-(0.0051103%)-(3)-(0)---(Jun/7)-(13:27)---(Jun/14)-(13:27:26)---ETWNFILF---(Jun/10)-(03:37:03)---(Jun/14)-(13:27:26) NAGCGKiv

[root@nagios-primary ~]#
[root@nagios-primary ~]#
[root@nagios-primary ~]#
[root@nagios-primary ~]#

							

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# logrobot autofig /var/log/kern.log 2h '.' '.' 1 2 -show

Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.160050] usb 5-1: new full-speed USB device number 26 using uhci_hcd
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.388215] hub 5-1:1.0: USB hub found
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.390118] hub 5-1:1.0: 4 ports detected
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.673128] usb 5-1.2: new low-speed USB device number 27 using uhci_hcd
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.831895] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.2/5-1.2:1.0/input/input34
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.832071] logitech 0003:046D:C517.001B: input,hidraw0: USB HID v1.10 Keyboard [Logitech USB Receiver] on usb-0000:00:1d.0-1.2/input0
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.863133] logitech 0003:046D:C517.001C: fixing up Logitech keyboard report descriptor
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.865367] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.2/5-1.2:1.1/input/input35
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.865633] logitech 0003:046D:C517.001C: input,hiddev0,hidraw3: USB HID v1.10 Mouse [Logitech USB Receiver] on usb-0000:00:1d.0-1.2/input1
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.249129] usb 5-1.3: new low-speed USB device number 28 using uhci_hcd
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436287] input: No brand 4 Port KVMSwicther as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.3/5-1.3:1.0/input/input36
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436429] generic-usb 0003:10D5:55A4.001D: input,hidraw4: USB HID v1.10 Keyboard [No brand 4 Port KVMSwicther] on usb-0000:00:1d.0-1.3/input0
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.442165] usbhid 5-1.3:1.1: couldn't find an input interrupt endpoint
     
2---3240---13---(Sep/20)-(16:49)---(Sep/20)-(17:55:08)---ETWNFILF---(Sep/20)-(17:55)---(Sep/20)-(17:55:08) NAGC

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~#


							

Scan through the above output and show ONLY lines that contain the strings "USB HID":
 
root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# logrobot autofig /var/log/kern.log 2h '.' 'USB HID' 1 2 -show

Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.832071] logitech 0003:046D:C517.001B: input,hidraw0: USB HID v1.10 Keyboard [Logitech USB Receiver] on usb-0000:00:1d.0-1.2/input0
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.865633] logitech 0003:046D:C517.001C: input,hiddev0,hidraw3: USB HID v1.10 Mouse [Logitech USB Receiver] on usb-0000:00:1d.0-1.2/input1
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436429] generic-usb 0003:10D5:55A4.001D: input,hidraw4: USB HID v1.10 Keyboard [No brand 4 Port KVMSwicther] on usb-0000:00:1d.0-1.3/input0

2---3420---3---(Sep/20)-(16:52)---(Sep/20)-(17:55:08)---ETWNFILF---(Sep/20)-(17:55)---(Sep/20)-(17:55:08) NAGC

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# 

							


root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# logrobot sanal /var/log/kern.log 8h '.' '.' 1 2 -exceldh

frq=19,zsc=1.41421,asc=[Sep-20-(16)]
frq=13,zsc=-0.707106,asc=[Sep-20-(17)]
frq=13,zsc=-0.707106,asc=[Sep-20-(15)]

root@nagios-primary ~#
root@nagios-primary ~#

Search the [ kern.log ] file once again. Find which MINUTE(S) within the last 8 hours had the most entries logged:

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# logrobot sanal /var/log/kern.log 8h '.' '.' 1 2 -exceldm

frq=13,zsc=0.816496,asc=[Sep-20-(17:55)]
frq=13,zsc=0.816496,asc=[Sep-20-(16:16)]
frq=13,zsc=0.816496,asc=[Sep-20-(15:31)]
frq=3,zsc=-1.22474,asc=[Sep-20-(16:24)]
frq=3,zsc=-1.22474,asc=[Sep-20-(16:15)]

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~#

							

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# logrobot autofig /var/log/kern.log 2h '.' '.' 1 2 -show

Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.160050] usb 5-1: new full-speed USB device number 26 using uhci_hcd
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.388215] hub 5-1:1.0: USB hub found
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.390118] hub 5-1:1.0: 4 ports detected
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.673128] usb 5-1.2: new low-speed USB device number 27 using uhci_hcd
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.831895] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.2/5-1.2:1.0/input/input34
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.832071] logitech 0003:046D:C517.001B: input,hidraw0: USB HID v1.10 Keyboard [Logitech USB Receiver] on usb-0000:00:1d.0-1.2/input0
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.863133] logitech 0003:046D:C517.001C: fixing up Logitech keyboard report descriptor
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.865367] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.2/5-1.2:1.1/input/input35
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.865633] logitech 0003:046D:C517.001C: input,hiddev0,hidraw3: USB HID v1.10 Mouse [Logitech USB Receiver] on usb-0000:00:1d.0-1.2/input1
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.249129] usb 5-1.3: new low-speed USB device number 28 using uhci_hcd
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436287] input: No brand 4 Port KVMSwicther as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.3/5-1.3:1.0/input/input36
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436429] generic-usb 0003:10D5:55A4.001D: input,hidraw4: USB HID v1.10 Keyboard [No brand 4 Port KVMSwicther] on usb-0000:00:1d.0-1.3/input0
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.442165] usbhid 5-1.3:1.1: couldn't find an input interrupt endpoint

2---3960---13---(Sep/20)-(17:01)---(Sep/20)-(17:55:08)---ETWNFILF---(Sep/20)-(17:55)---(Sep/20)-(17:55:08) NAGC

root@nagios-primary ~#
root@nagios-primary ~#

From the above output, exclude all lines that contain 'Logitech' and show me what is left:

root@nagios-primary ~#
root@nagios-primary ~#
root@nagios-primary ~# logrobot sanal /var/log/kern.log 2h '.' 'Logitech' 1 2 -showexcl

Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.160050] usb 5-1: new full-speed USB device number 26 using uhci_hcd
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.388215] hub 5-1:1.0: USB hub found
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.390118] hub 5-1:1.0: 4 ports detected
Sep 20 17:55:06 jake-XPS-M1530 kernel: [87310.673128] usb 5-1.2: new low-speed USB device number 27 using uhci_hcd
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.249129] usb 5-1.3: new low-speed USB device number 28 using uhci_hcd
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436287] input: No brand 4 Port KVMSwicther as /devices/pci0000:00/0000:00:1d.0/usb5/5-1/5-1.3/5-1.3:1.0/input/input36
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.436429] generic-usb 0003:10D5:55A4.001D: input,hidraw4: USB HID v1.10 Keyboard [No brand 4 Port KVMSwicther] on usb-0000:00:1d.0-1.3/input0
Sep 20 17:55:08 jake-XPS-M1530 kernel: [87312.442165] usbhid 5-1.3:1.1: couldn't find an input interrupt endpoint

2---4320---(8)-(13)-(61.5385%)-(8)-(0)-(frq=8,zsc=0,asc=[Sep-20-(17:55)])---(Sep/20)-(17:07)---(Sep/20)-(17:55:08)---ETWNFILF---(Sep/20)-(17:55)---(Sep/20)-(17:55:08) NAGCzzmm

root@nagios-primary ~#
root@nagios-primary ~#


							


Monitoring logs of any format or size (has no limitations!)


./logrobot localhost <default-dir> <feature> <logfile> <age> <str-1> <str-2> <WARNING> <CRITICAL> <tag> <option>

Example:

logrobot  localhost  /tmp/logXray  autonda  /var/log/kern.log  60m  'error'  '.'  1  2  app_err_monitor  -ndfoundn

Explanation of Parameters:

logrobot - This is the tool that does the work for you 

/var/tmp/logXray - This is the designated default directory where logrobot will process its data

autonda - This is the feature that allows logrobot to perform this particular auto-resolve task for you

/var/log/kern.log - This is the log file which is going to be scanned

To scan a directory, simply specify the directory path instead...i.e. /var/log

age - The age the monitored log file must be for it to be monitored

'error' - This is where you specify the string/pattern to look for in the log

Make sure there are no spaces in the patterns you specify.

For instance, to search for the pattern "error found in data", you can specify it this way:

'error.*found.*in.*data'

'.' - This is where you specify an additional pattern you wish to look for on the same line as the previous string

Useful if you want to filter out specific log entries

1 - This is the WARNING number of entries that must be found in the log before an alert is generated.

2 - This is the CRITICAL number of entries that must be found in the log before an alert is generated.

app_err_check - This is the tag name given to this particular log check

The name should describe the application/database or function that's writing to the log - Basically, give this a deserving name

-ndshow - When entries are found in the log, this option will show you those entries

-ndfoundn - When entries are found in the log, this option will NOT them - It will tell you the total count of the newest entries found matching your criteria
Example 1 - (this shows the matching entries found in each log):

Command:

./logrobot localhost /var/tmp/logXray,tail=10 autonda /usr/WebSphere/AppServer_ast_/profiles/paposa_ast_AppServer_ast_/logs/rmcosCluster1-paposa_ast_-node_ast_-server_ast_/SystemOut.log 60m 'Total.*time.*taken' '.' 1 1 testing1 -ndshow

CRITICAL: [/usr/WebSphere/AppServer_ast_/profiles/paposa_ast_AppServer_ast_/logs/rmcosCluster1-paposa_ast_-node_ast_-server_ast_/SystemOut.log][4]
/usr/WebSphere/AppServer2/profiles/paposa01AppServer02/logs/rmcosCluster1-paposa01-node2-server1/SystemOut.log:P=(2)_F=(13s,1s)_R=(39232,39253=21)
/usr/WebSphere/AppServer1/profiles/paposa01AppServer01/logs/rmcosCluster1-paposa01-node1-server2/SystemOut.log:P=(2)_F=(13s,6s)_R=(75789,75811=22)
/usr/WebSphere/AppServer2/profiles/paposa01AppServer02/logs/rmcosCluster1-paposa01-node2-server2/SystemOut.log:P=(2)_F=(13s,0s)_R=(105911,105932=21)

usr_WebSphere_AppServer2_profiles_paposa01AppServer02_logs_rmcosCluster1-paposa01-node2-server2_SystemOut.log:::
[11/16/16 13:48:41:722 PST] 000004e3 SystemOut O TOK : Total time taken to De-Tokenize a number is [12] ms.
[11/16/16 13:48:53:265 PST] 000004b6 SystemOut O TOK : Total time taken to De-Tokenize a number is [15] ms. 2

usr_WebSphere_AppServer2_profiles_paposa01AppServer02_logs_rmcosCluster1-paposa01-node2-server1_SystemOut.log:::
[11/16/16 13:48:43:915 PST] 000004f6 SystemOut O TOK : Total time taken to De-Tokenize a number is [17] ms.
[11/16/16 13:48:52:317 PST] 000004f6 SystemOut O TOK : Total time taken to De-Tokenize a number is [17] ms. 2

usr_WebSphere_AppServer1_profiles_paposa01AppServer01_logs_rmcosCluster1-paposa01-node1-server2_SystemOut.log:::
[11/16/16 13:48:45:693 PST] 000002e3 SystemOut O TOK : Total time taken to De-Tokenize a number is [14] ms.
[11/16/16 13:48:47:873 PST] 000002b2 SystemOut O TOK : Total time taken to De-Tokenize a number is [26] ms. 2

usr_WebSphere_AppServer1_profiles_paposa01AppServer01_logs_rmcosCluster1-paposa01-node1-server1_SystemOut.log::: 0

Example 2 - (this shows the total count of each matching entry in each log)

Command:

./logrobot localhost /var/tmp/logXray,tail=10 autonda /usr/WebSphere/AppServer_ast_/profiles/paposa_ast_AppServer_ast_/logs/rmcosCluster1-paposa_ast_-node_ast_-server_ast_/SystemOut.log 60m 'Total.*time.*taken' '.' 1 1 testing3 -ndfoundmul

CRITICAL: [/usr/WebSphere/AppServer_ast_/profiles/paposa_ast_AppServer_ast_/logs/rmcosCluster1-paposa_ast_-node_ast_-server_ast_/SystemOut.log][4] 

/usr/WebSphere/AppServer1/profiles/paposa01AppServer01/logs/rmcosCluster1-paposa01-node1-server2/SystemOut.log:P=(Total__time__taken=8)_F=(25s)_R=(76970,77031=61)
/usr/WebSphere/AppServer2/profiles/paposa01AppServer02/logs/rmcosCluster1-paposa01-node2-server1/SystemOut.log:P=(Total__time__taken=4)_F=(25s)_R=(40355,40503=148)
/usr/WebSphere/AppServer1/profiles/paposa01AppServer01/logs/rmcosCluster1-paposa01-node1-server1/SystemOut.log:P=(Total__time__taken=3)_F=(25s)_R=(23434,23467=33)
/usr/WebSphere/AppServer2/profiles/paposa01AppServer02/logs/rmcosCluster1-paposa01-node2-server2/SystemOut.log:P=(Total__time__taken=9)_F=(25s)_R=(106908,106997=89)

NOTE:

The '_P_' represents the pipe "|"(OR) symbol.  If using this tool as a log monitoring alert system, specifying "_P_" instead of "|" prevents unnecessary errors.

The default log file age limit is 60 minutes.  That means, the above commands will only scan log files that were modified/created within the last 60 minutes.

To change the age limit, see the full syntax example below...simply replace the 60m with whichever age you prefer

If no entries are found matching the patterns you specified, but you believe there should be, simply add a ".*" to the beginning and end of each pattern...i.e:

'.*error.*_P_.*panic.*_P_.*fail.*_P_.*fault.*'


[root@localhost jserver]# 
[root@localhost jserver]# time ./logrobot localhost /var/tmp/logXray autonda /var/log 60m 'error' '.' 1 2 appmon -ndfoundn
CRITICAL: [/var/log] maillog:P=(25)_F=(107s)_R=(0,281=281) up2date:P=(5)_F=(51s)_R=(0,73=73), Xorg.0.log:P=(1)_F=(197s)_R=(0,659=659) 

real 0m1.571s
user 0m0.694s
sys 0m0.637s

[root@localhost jserver]# 
[root@localhost jserver]# time ./logrobot localhost /var/tmp/logXray autonda /var/log 60m 'error' '.' 1 2 appmon -ndfoundn
OK: [/var/log] up2date:P=(0)_F=(5s)_R=(73,73=0) boot.log:P=(0)_F=(5s)_R=(58,58=0) cron:P=(0)_F=(5s)_R=(214,214=0) messages:P=(0)_F=(5s)_R=(643,643=0) dmesg:P=(0)_F=(5s)_R=(502,502=0) Xorg.0.log:P=(0)_F=(5s)_R=(659,659=0) maillog:P=(0)_F=(5s)_R=(281,281=0) pm-powersave.log:P=(0)_F=(5s)_R=(2,2=0) secure:P=(0)_F=(5s)_R=(13,13=0)

real 0m1.604s
user 0m0.674s
sys 0m0.634s

[root@localhost jserver]# 
[root@localhost jserver]# time ./logrobot localhost /var/tmp/logXray autonda /var/log/messages 60m 'error' '.' 1 2 appmsg -ndfoundn
OK: [/var/log/messages] /var/log/messages:P=(0)_F=(383s)_R=(0,643=643) 

real 0m1.331s
user 0m0.734s
sys 0m0.622s
[root@localhost jserver]#

[root@nagios-primary ~]# ./logrobot localhost /var/tmp/logXray autodoc /wms/prod/jdf/data/log/error 1GB 1.6GB filesize

OK: File [ /wms/prod/jdf/data/log/error ]. Current Size = [ 682.637MB 7 ]. Thresholds: [ W=1GB ] and [ C=1.6GB ].

[root@nagios-primary ~]# ./logrobot localhost /var/tmp/logXray autodoc /var/lib/nagios/retention.dat 80MB 100MB filesize

CRITICAL: File [ /var/lib/nagios/retention.dat ]. Current Size = [ 179.734MB ]. Thresholds: [ W=80MB ] and [ C=100MB ].

# Sending metrics to a graphite/graphing server:

[root@nagios001 ~]# ./logrobot localhost /var/tmp/logXray,graphite,52.88.12.122:2003,typical autonda /var/log/messages 60m 'nothing-to-search-for' '.' 1 2 LogGrowthChk -ndfoundn


The following command will alert if files are found with size greater than zero.

[root@nagios-primary ~]# ./logrobot localhost /var/tmp/logXray autodoc /var/mqm/errors,.FDC,12m 0B 0B filesize

CRITICAL: File [ /var/mqm/errors,.FDC,12m ]. Current Size = [ /var/mqm/errors/AMQ24835.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24834.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24821.0.FDC(repeat),81673(bytes),11 /var/mqm/errors/AMQ24832.0.FDC(repeat),26973(bytes),11 /var/mqm/errors/AMQ24827.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24826.0.FDC(repeat),26973(bytes),11 /var/mqm/errors/AMQ24833.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24828.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24836.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24825.0.FDC(repeat),26973(bytes),11 /var/mqm/errors/AMQ24831.0.FDC(repeat),26973(bytes),11 /var/mqm/errors/AMQ24830.0.FDC(repeat),27053(bytes),11 /var/mqm/errors/AMQ24829.0.FDC(repeat),27053(bytes),11 ]. Thresholds: [ W=0B ] and [ C=0B ].


[root@nagios001 ~]# ./logrobot localhost /var/tmp/logXray autodoc /apps/scope/GAP/wmswave/cbs/logs/cores,1,*,1440m 0B 0B filesize

OK: File [ /apps/scope/GAP/wmswave/cbs/logs/cores,1,*,1440m ]. Current Size = [ no_problem_files_detected ]. Thresholds: [ W=0B ] and [ C=0B ].

[root@nagios001 ~]# ./logrobot localhost /var/tmp/logXray autodoc /apps/scope/GAP/wmswave/cbs/logs/cores,1,*,1440m 0B 0B filesize

CRITICAL: File [ /apps/scope/GAP/wmswave/cbs/logs/cores,1,*,1440m ]. Current Size = [ /apps/scope/GAP/wmswave/cbs/logs/cores/PkShipWaveS/core.10114,533901312(bytes),3m ]. Thresholds: [ W=0B ] and [ C=0B ].

Next time check runs, you'll see the word 'repeat' next to each file that has already been reported/alerted on

CRITICAL: File [ /apps/scope/GAP/wmswave/cbs/logs/cores,1,*,1440m ]. Current Size = [ /apps/scope/GAP/wmswave/cbs/logs/cores/PkShipWaveS/core.12263(repeat),592871424(bytes),7m ]. Thresholds: [ W=0B ] and [ C=0B ].

[root@nagios-primary ~]# ./logrobot localhost /var/tmp/logXray autodoc /opt/apps/iptuibatch/logs/iptconflictCheck.log 1 5 filegrowth

CRITICAL: File [ /opt/apps/iptuibatch/logs/iptconflictCheck.log ]. Size Now = [ 744KB (Wed Dec 30 17:35:56 2015) ]. Size Before = [ 744KB (Wed Dec 30 17:35:55 2015) ].

[root@nagios-primary ~]# ./logrobot localhost /var/tmp/logXray autodoc /opt/apps/iptuibatch/logs/iptconflictCheck.log 1 5 filegrowth

OK: File [ /opt/apps/iptuibatch/logs/iptconflictCheck.log ]. Size Now = [ 752KB (752) (Wed Dec 30 17:37:55 2015) ]. Size Before = [ 744KB (Wed Dec 30 17:35:55 2015) ].

[root@nagios001 ~]# ./logrobot localhost /tmp/logXray,graphite,52.88.12.122:2003,typical autonda /var/log/messages 60m 'nothing-to-search-for' '.'  1 2 LogGrowthChk -ndfoundn

[root@monitor jbowman]#
[root@monitor jbowman]#
[root@monitor jbowman]# ./logxray localhost /var/tmp/logXray autodoc /var/log/syslog 10 20 -timestamp

OK: File = [ /var/log/syslog ]. Timestamp = [ 4s ] = [ 0d, 0h, 0.066m ago ]. Thresholds: [ W=(10m) / C=(20m) ].

[root@monitor jbowman]#
[root@monitor jbowman]#

[root@monitor jbowman]#
[root@monitor jbowman]# ./logxray logrobot001.phx.logrobot.com /var/tmp/logXray autodoc /var/log/syslog 10 20 -timestamp

OK: File = [ /var/log/syslog ]. Timestamp = [ 4s ] = [ 0d, 0h, 0.066m ago ]. Thresholds: [ W=(10m) / C=(20m) ].

[root@monitor jbowman]#


Case Scenario:

Monitor all files that have the pattern "gap_inc" in their names, under the /opt/apache/httpd-2/3/2/htdocs/pkicrlpub directory.

Alert as Warning if the age of any of the discovered file is at least 4 hours old but less than 8 hours.

Alert as Critical when the age of any of the discovered files is at least 8 hours old.

The _ast_ is used to denote "*"

Asterisks have the potential to cause problems, therefore, we allow users to use a predetermined string to reference them.

In other words, when having to specify the path to a log file with asterisks in it, replace the asterisks with "_ast_"

For example,

	This:

		/opt/apache/httpd-2.4.2/htdocs/pkicrlpub/*gap_inc*

	Becomes:

		/opt/apache/httpd-2/3/2/htdocs/pkicrlpub,_ast_gap_inc__ast_

[root@monitor jbowman]#
[root@monitor jbowman]#
[root@monitor jbowman]# ./logxray localhost /var/tmp/logXray autodoc /opt/apache/httpd-2/3/2/htdocs/pkicrlpub,_ast_gap_inc__ast_ 4h 8h timestamp

OK: [ /opt/apache/httpd-2.4.2/htdocs/pkicrlpub/gap_inc_stores_issuing_ca_g1.crl,age=(0d/0h/39.6m ago) /opt/apache/httpd-2.4.2/htdocs/pkicrlpub/gap_inc_corp_root_ca_g1.crl,age=(0d/0h/39.6m ago) /opt/apache/httpd-2.4.2/htdocs/pkicrlpub/gap_inc_corp_issuing_ca_g1.crl,age=(0d/0h/39.6m ago) /opt/apache/httpd-2.4.2/htdocs/pkicrlpub/gap_inc_corp_intermediate_ca_g1.crl,age=(0d/0h/39.6m ago) ].

[root@monitor jbowman]#
[root@monitor jbowman]#

[root@monitor jbowman]#
[root@monitor jbowman]#
[root@monitor jbowman]# ./logxray localhost /var/tmp/logXray autodoc /var/log/syslog 10 20 -timestamp

OK: File = [ /var/log/syslog ]. Timestamp = [ 4s ] = [ 0d, 0h, 0.066m ago ]. Thresholds: [ W=(10m) / C=(20m) ].

[root@monitor jbowman]#
[root@monitor jbowman]#

[root@monitor jbowman]#
[root@monitor jbowman]# ./logxray logrobot001.phx.logrobot.com /var/tmp/logXray autodoc /var/log/syslog 10 20 -timestamp

OK: File = [ /var/log/syslog ]. Timestamp = [ 4s ] = [ 0d, 0h, 0.066m ago ]. Thresholds: [ W=(10m) / C=(20m) ].

[root@monitor jbowman]#


    • Save Labor


      No programming required, no new query languages to learn, no classes to take, No complicated configuration files to deal with, No scary modules or libraries to download or configure - Quick & Clean automated install - Requires only an "unzip"
      • Detect App/DB/System Anomalies via logs
      • Monitor logs with/without time stamps or dates
      • (Single Log) Timestamp Monitor
      • (Multi Log) Timestamp Monitor
      • (Single Log) Growth Monitor
      • (Multi Log) Growth Monitor
      • Automated Graphing/Trending of Log Metrics
      • Monitor HTTP/Apache Status Codes
      • Ability to choose # of entries from log to show in alerts
      • Log Rotation - Reads unread entries from rotated logs
      • Databases - Monitor all DB logs w/ just one check
    • Save Time


      Monitor all log files in your Unix environment with just ONE tool & ONE master server - Manage log checks from a centralized interface; Nagios, Zabbix, Zenoss, Cron...etc - We've done all the complicated, tedious, time consuming work so you NEVER have to write another script again!
      • Stale Logs Alert when log is stagnant / inactive
      • Centralized Monitoring - Monitor Local & Remote Logs
      • Single Log Size Monitor
      • Multi Log Size Monitor
      • Single / Multiple Directory File Count Monitor
      • Single / Multiple Directory Size Monitor
      • Self Healing Auto-Resolve Capabilities
      • Tail log content through time frames
      • Alert on different patterns on different lines
      • Exclude specific log entries to eliminate false alerts
      • Applications - Monitor all App logs w/ just one check

Frequently Asked Questions

To purchase the logrobot tool, please visit our pricing page:

http://www.logrobot.com/pricing-tables.html

The log monitoring capabilities of LoGrobot are vast in nature and therefore, we strongly recommend simply reaching out to us for documented verification (if needed for your peace of mind) that LoGrobot can handle whichever use case scenario(s) you wish to use it for.

When you purchase the LoGrobot tool, you're not just purchasing any monitoring tool. You're buying a tool that was built over the span of 7+ years and is equipped with numerous use case scenarios, scenarios we've had to account for, thanks to the many customization requests from our users.

Here are just some of the tasks LoGrobot does, right out of box:

  1. Monitors and alerts on the contents of system log files (errors, strings, keywords, patterns etc)
  2. Monitors and alerts on the timestamps of log files (verify specific files are being updated regularly)
  3. Monitors several log files at the same time - allows you to monitor all logs of a Database or Application
  4. Graphing the frequency with which user-specified patterns occur in log files
  5. - Or graph for anomalies
  6. Monitoring/Alerting on the size of log files (alerts when a log or file grows past a certain size)
  7. Conditional Monitoring..i.e:
  8. a). Alert if the value(s) in a certain field of specific log entries has a value greater than/less than X
  9. ANALYSIS - Easily identify which minute or hour of the day had the most entries recorded
  10. - Anomaly Detection

Benefits:

  1. Configurable to run either via Zabbix, Zenoss, Nagios or CRONTAB (as a cron entry)
  2. a). Get email alerts & notifications on all log checks b). Does not require the installation of Nagios, Zabbix or Zenoss
  3. Automatically send log metrics to Graphite for historical trending and visualization
  4. - No need for any extra configurations on your part!
  5. Monitor several different patterns in the same log
  6. a). Allows passing of different thresholds to each pattern b). Allows for the filtering of specific lines to avoid unnecessary noise
  7. Manage log file checks from a central location
  8. - Integrate with Nagios, Zenoss, Zabbix, Sensu, Hyperic, New Relic and much more! - Aggregate critical log entries into one central server
  9. Simple, pluggable command-line parameters (no need for any confusing configuration files)
  10. - Eliminates the need to have to re-deploy configs to remote hosts each time a log check is implemented or updated.
  11. Configurable to alert on the size of log files
  12. Example: Alert if the size of /var/app/custom.app.log exceeds 10MB
  13. Configurable to alert on the growth of log files
  14. Example: Alert if the most recent size of /var/log/messages is the same size it was at the time of last check
  15. Monitor all or specific type of logs in a specific directory
  16. a). Point logxray to ANY directory with just one check! i). Avoid having to define separate checks for each log file b). Specify the type of files to exclude / include in monitoring i). Assign different thresholds for each file type
  17. Scan specific logs via time frames (i.e. previous 20 minutes, 60 minutes, 1 day, 1 week etc)
  18. Remote Agent Included to enable monitoring of logs on several hosts FROM ONE master server
  19. a). This is for users who don't have NRPE installed in their environment i). Allows complete control of log checks on all remote hosts / servers
  20. Use ONE tool to automatically monitor any log format - Avoid using several different scripts!
    EXAMPLE 1: Specifying multiple logs to monitor, in addition to specifying a directory in cases where you don't know how deep the log file will be in the directory:

      Note:
      • _ast_ = *
      • _ds_ = $
      • _mulast_ = (multiple asterisks)

    Specifying _ds_ behind a file name (right before the comma), indicates that you only want to monitor that specific log file. No variations of it. For instance, /var/log/chef/client.log_ds_ means scan only client.log, do not scan any other log file that may have “client.log” in their name…i.e. client.log.1, client.log.save etc. If you wish to scan logs with similar names, replace the _ds_ with _ast_.

      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$ ./logrobot localhost /var/tmp/logXray,tail=10 autonda /var/log/messages_ds_,/var/log/chef/client.log_ast_,/opt/oracle/diag/_mulast_/alert_DC4WMMH2.log 60m '.*error.*_P_.*fatal.*_P_Session.*of.*user' '.' 1 2 mylogCheck –ndshow

      CRITICAL: [/var/log/messages_ds_,/var/log/chef/client.log_ast_,/opt/oracle/diag/_mulast_/alert_DC4WMMH2.log][4]

      /var/log/chef/client.log.save:P=(33)_F=(35704356s)_R=(0,5110=5110)

      /var/log/messages:P=(1804)_F=(28s)_R=(0,7284=7284)

      opt_oracle_diag_rdbms_dc4wmmh_DC4WMMH2_trace_alert_DC4WMMH2.log::: 0

      var_log_chef_client.log.save:::
      [2016-01-20T06:51:54-07:00] ERROR: yum_package[nagios-nsca-client] (gapNagios::client_package line 71) had an error: Chef::Exceptions::Exec: yum -d0 -e0 ....
      Transaction check error:
      [2016-01-20T06:57:09-07:00] ERROR: bash[install_oracle] (gapOracleDBA::default line 83) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected proc....
      [2016-01-20T07:21:52-07:00] ERROR: yum_package[nagios-nsca-client] (gapNagios::client_package line 71) had an error: Chef::Exceptions::Exec: yum -d0 -e0....
      Transaction check error:
      [2016-01-20T07:27:06-07:00] ERROR: bash[install_oracle] (gapOracleDBA::default line 83) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected pro....
      [2016-01-20T07:29:04-07:00] ERROR: yum_package[nagios-nsca-client] (gapNagios::client_package line 71) had an error: Chef::Exceptions::Exec: yum -d0 -e0....
      Transaction check error:
      [2016-01-20T07:34:19-07:00] ERROR: bash[install_oracle] (gapOracleDBA::default line 83) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected pro...
      33
      var_log_messages::: Mar 8 15:15:03 tpphxwmmdb002 systemd: Started Session 12915 of user oracle.
      Mar 8 15:15:03 tpphxwmmdb002 systemd: Starting Session 12916 of user oracle.
      Mar 8 15:15:03 tpphxwmmdb002 systemd: Started Session 12916 of user oracle.
      Mar 8 15:20:01 tpphxwmmdb002 systemd: Starting Session 12919 of user root.
      Mar 8 15:20:01 tpphxwmmdb002 systemd: Started Session 12919 of user root.
      Mar 8 15:25:01 tpphxwmmdb002 systemd: Starting Session 12920 of user oracle.
      Mar 8 15:25:01 tpphxwmmdb002 systemd: Started Session 12920 of user oracle.
      Mar 8 15:25:01 tpphxwmmdb002 systemd: Starting Session 12921 of user oracle.
      Mar 8 15:25:01 tpphxwmmdb002 systemd: Started Session 12921 of user oracle.
      1804
      var_log_chef_client.log::: 0
      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$

    EXAMPLE 2 - Specifying just one directory in cases where you dont know how deep the log file you want to monitor, will be:

      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$ ./logrobot localhost /var/tmp/logXray,tail=10 autonda /opt/oracle/diag/_mulast_/alert_DC4WMMH2.log 60m '.*error.*_P_.*fatal.*_P_Session.*of.*user' '.' 1 2 mylogCheck -ndshow

      OK: [/opt/oracle/diag/_mulast_/alert_DC4WMMH2.log][1] /opt/oracle/diag/rdbms/dc4wmmh/DC4WMMH2/trace/alert_DC4WMMH2.log:P=(0)_F=(17s,112s)_R=(1109,1109=0)

      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$ ./logrobot localhost /var/tmp/logXray,tail=10 autonda /opt/oracle/diag/_mulast_/alert_DC4WMMH2.log 60m '.*error.*_P_.*fatal.*_P_Session.*of.*user' '.' 1 2 mylogCheck -ndshow

      OK: [/opt/oracle/diag/_mulast_/alert_DC4WMMH2.log][1] /opt/oracle/diag/rdbms/dc4wmmh/DC4WMMH2/trace/alert_DC4WMMH2.log:P=(0)_F=(117s)_R=(0,1109=1109)

      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$
      [jbowman@tpphxwmmdb002 plugins]$
logXray is used to generate ON-Demand graphs on the recorded statistics of a log file. LoGrobot is a full blown log monitoring plugin that is used to handle all things related to log monitoring...i.e. monitoring log content, log size, log timestamp, log file size, simulteneous monitoring of multiple log files and multiple different patterns at the same time.
Yes. If your list of strings is too long or too many to fit nicely on the command line, you can instruct LoGrobot to use configuration files instead. All you need to put in the config file(s) is the list of patterns (one per line) you want to monitor. Nothing more.
Yes. LoGrobot automatically watches for signs of log rotation and when detected, it proceeds to scan the unread entries from the recently rotated log, in addition to any unread entries from the fresh live log.
Yes. LoGrobot can monitor any log file regardless of format or size.
No. LoGrobot does not rely on any other application in order for it to monitor and alert on logs.

If you wish to visualize your log file activities, you have options.

  1. You can download and install the Graphite Application
    • You may want to also install Grafana if you wish to beautify your graphs
      • - We can help you with the installation of both Graphite and Grafana
    • After Graphite/Grafana is installed, simply add an entry to each log check you create
      • The entry you will need to add will include the graphite server IP and the port
        • Whenever LoGrobot sees a check with a graphite setting, it will automatically send its metrics to the listed IP at the listed port.
          • Example:
            • .. '.*error.*' '.' 1 2 errchk -ndfoundmul graphite,52.88.12.122,2003,typical

  2. Utilize the licensed logXray dashboard
    • With this dashboard, you dont need to install Graphite.
      • All you need is an Apache/HTTP PHP Enabled Webserver
    • You can generate on-demand charts and graphs to show the historical trend of:
      • Application, Database, System & Network errors
      • Volume of entries
      • Compare the latest metrics retrieved from a log check to past metrics
        • Know quickly if the current value is cause for concern
        • i.e.
          • Why is the volume of entries today lower than that of a week ago?
          • Why did the number of errors suddenly triple in size?
          • Why is the volume of entries for the current hour so different from
          • the same hour, yesterday, the day before, 3 days ago, a week ago etc
    • Uncover valuable pieces of information you didnt even know were available!
No. LoGrobot / logXray has years of real life situations built into it. It has been heavily tested in QA, DEV, PrePROD and PROD environments. The tool as it is, is highly versatile and able to handle any log monitoring situation you throw at it.
Yes. There is a 30 Day Money Back Guarantee.
Absolutely! We usually complete custom development requests within 24 to 72 hours of submission. If your request isn't of an urgent nature, please state so in your email. NON-Urgent email requests will be completed within 5 business days. Contact us for more information.

If using NRPE,

  1. Copy the logXray zip file you just purchased to the hosts on which you have log files to monitor
  2. ssh to one of the remote hosts from above step.
  3. Download our free auto installer
  4. Pass the download link of your recently purchased LoGrobot zip to the autoinstaller
    • - Also, pass, as a parameter, the directory to install logrobot into
      • Make sure you specify whichever directory you consider to be your plugins or scripts directory
  5. Define an entry in the nrpe.cfg file on the remote host - Reference the absolute path from above step 4
  6. Restart the nrpe process on the host
    • In other words, perform the following steps on the remote nodes:
    • cd ~
    • wget http://www.LoGrobot.com/klazy ; ls -ld klazy ; chmod 755 klazy ; ls -ld klazy
    • ./klazy http://www.LoGrobot.com/the-logrobot.zip /the/path/you/prefer/to/put/the/executable/for/easy/access
    • i.e.
    • ./klazy http://www.LoGrobot.com/logrobot.verify_your@emailaddress.com...zip /prod/nagios-4.2.4/plugins/logrobot
  7. vi /path/to/your/nrpe.cfg
    • - Add an entry referencing the location of the logrobot tool
  8. Restart nrpe
  9. When the above steps complete successfully, the logrobot tool is now installed and ready to be used.

If using the Custom Monitoring Agent that comes with logXray:

  1. First install the agent on the remote box:
    • su - nagios (or whatever your monitoring user name is)
    • cd ~
    • wget http://www.LoGrobot.com/klazy ; ls -ld klazy ; chmod 755 klazy ; ls -ld klazy
    • ./klazy logXray /var/tmp/logXray 1040 <ip(s)-of-your-master-server(s)>
      • .i.e.
      • ./klazy logXray /var/tmp/logXray 1040 10.20.30.40
      • (OR)
      • ./klazy logXray /var/tmp/logXray 1040 10.20.30.40,50.60.70.80
    • ./klazy logXray status
      • - Verify the logXray remote agent is up and running.

  2. Then install the logrobot tool on the remote box:
    • cd ~
    • ./klazy http://www.LoGrobot.com/the-logrobot.zip /var/tmp/logXray/plugins
    • i.e.
    • ./klazy http://www.LoGrobot.com/logrobot.verify_your@email.....zip /var/tmp/logXray/plugins
      • - Note, on the REMOTE NODES, you MUST specify the directory [ /var/tmp/logXray/plugins ] as the location for the log monitoring plugin.
      • - When the above completes successfully, the logrobot tool is now installed on the remote node.

  3. Finally, test remote log monitoring and confirm all is well:
    • ssh to the master server
    • run the following command
      • ./logrobot <node-fqdn> /var/tmp/logXray autonda /tmp/err.log 60m '.*fatal.*' '.' 1 2 TagErr -ndshow sudo:remote
  1. Simplicity - It does not require an extensive learning process to get used to. Extremely user-friendly!
    • - Unlike our competitors, we built LoGrobot / logXray to cater directly to the everyday needs of the typical:
      • System Administrator - Watch system logs, security logs, mail logs and basically any logs
      • Database Administrator - Monitor multiple different error codes in one log or multiple logs
      • Be able to easily specify exclusion patterns in areas where you wish to eliminate unnecessary noise
      • Monitoring Engineer - Spin up new log monitoring checks very quickly without having to develop them yourself!
      • Developers - Monitor important log files for errors or activity during code testing
  2. Versatility - It can be used either as a plugin or its own standalone monitoring system
    • Usable directly on the command line to perform a wide range of different operations on logs & directories
  3. Compatibility - Easily integrated with your existing monitoring system
    • Nagios
    • Zabbix
    • Zenoss
    • Sensu
    • Tivoli
    • Datadog
    • Crontab / Cron (for sending log alerts in case you dont have any monitoring system in place)
    • ....
  4. Support - All users of LoGrobot receive free support
    • When it comes to the monitoring of log files and the management of alerts on them, we understand there are many different ways things can be done
      • Our users are given the chance to request the development of custom features for a fee
        • These customer specific features will be tailored specifically towards each individual user need
  5. Command line Usability - All necessary parameters are passable directly from the command line - No configs!
  6. Modules - Unlike most tools, LoGrobot does not require the installation of nonnative modules or libraries to the system
    • What that means is, there is nothing complicated for you to configure
  7. Affordable - A very inexpensive log monitoring tool considering the amount of work it will save you
    • No more scripts for you to write!
    • If you need a custom feature, simply reach out us (support@logrobot.com) to develop it for you!
    • Chances are, your custom feature already exist in the LoGrobot arsenal, in which case, we'll just need to show you how to access it.
  8. Maintenance - Constantly updated for added simplicity, building of new features & polishing of the old
    • - Yes, all customers get those updates for FREE for the first year
  9. Speed - Completes scanning of log files in a very short period of time
    • Can monitor multiple logs in a directory in under 1.5 second
    • Requires NO extra system configuration or new package/library acquisition for it to work.
      • - It's ready to go right out of the box!
  10. Alerting - Its main purpose is to monitor log files/directories & alert on their content, size, timestamp and growth