Community
- Command Central
- MVP Program
- Monthly Mission
- Blogs
- Groups
- Events
- Media Vault
Products
- Observability
- Network Management
- Application Management
- IT Security
- IT Service Management
- System Management
- Database Management
Content Exchange
- SolarWinds Platform
- Server & Application Monitor
- Database Performance Analyzer
- Server Configuration Monitor
- Network Performance Monitor
- Network Configuration Manager
- SQL Sentry
- Web Help Desk
Free Tools & Trials
Store

Monitor a log file on a server that when the log shows > 1000 ms in a specific line

I am just getting into PERL scripting within Solarwinds and I had a request come in today that I am unsure about. They want to monitor a log file on a server that when the log shows > 1000 ms in a specific line to send an alert. I have gone round and round with them to come up with a different way of monitoring this but I am stumped. Any help would be appreciated as I don't fully understand PERL scripting. I have used the basic log file monitoring for specific lines but I don't know if something with a variable like this is doable.

This is the script I am using for some other items I am currently monitoring. I know it does not do the same thing but I would figure a little tweak to the script would allow me to do what I am looking for. I just do not understand PERL enough (yet!) to make the changes required.

This monitor uses the following arguments:

perl ${SCRIPT} LogFilePath RegularExpression

where

LogFilePath - This is the path of the target log file on the target server. The path can contain spaces.

RegularExpression - This is used for regular expression searches to find a desired string in the log file. Searches are case sensitive and can contain spaces.

Below is an example using the Command Line field:

perl ${SCRIPT} "/etc/inittab" "init"

Example of the line in the log file (they want to be notified when the etime is > than 1000):

[03/Sep/2019:10:36:58 -0500] SEARCH RES conn=596928 op=1 msgID=2 result=0 nentries=0 etime=10138

Thank you for any help!

Charlie

Find more posts tagged with

Accepted answers

All comments

jrouviere

I don't know enough Perl to add to your script a way to get at it the way that I did in Python, but I wrote a few lines of code that would solve this with Python if that were an option.

The main way I solved it was that you seem to have a way to find the line already, if you transform the line slightly so that you're able to return the etime entry and then the value as an actual statistic for Orion, you can just set up Orion to alert on a threshold over 1000 for that statistic.

If it would help I could share the python that I wrote, but not sure how much it would help you directly.

smalleggo

Share away! The more information about this the better! I am starting down a different path but it's possible that what you've written may be better than my current plan.

Thank you,

KMSigma.SWI

I'm in the same boat as @jrouviere. I could do this in PowerShell in a few minutes, but from the read-along here, I'm feeling like this might be Linux. I think we are probably talking about regex and then maybe either a length comparison (if the token is over 10 characters long ["etime=1000" is 10 characters, so longer would mean larger than 1000]) or converting to an integer and comparing it numerically. Personally, I'd vote for the second because I think the logic might be easier to follow, but I leave that up to people smarter than myself.

jrouviere

My language of choice is Python, so maybe you'd be able to use this, or translate it to Perl for your needs, but what I think would be easiest overall for your case is to turn the "etime =" entry into the statistical entry.

So you may have instances where you see etime = 'XXX' or something else, and then you can set thresholds for when the statistic is greater than 1000.

Now, the challenge becomes getting that to return as a statistic.

So this is my sort of Python method for it:

#You need to identify your line, looks like you've got this squared already above.

line = '[03/Sep/2019:10:36:58 -0500] SEARCH RES conn=596928 op=1 msgID=2 result=0 nentries=0 etime=10138'

for item in line.split():
    if 'etime' in item:
        stat = int(item.split('=')[-1])

Looks harmless enough, but what I'm doing is turning that line into a list, iterating over the list to find our etime term and then finally returning that, but splitting it on '=' to only get the 'number'.

Technically it's a string, but you change it to stat = int(item.split('=')[-1]) which would type it as an int which would match what we would expect.

Either this may be dubiously helpful, or the ramblings of an insane person. How to translate this to perl for you isn't in my wheelhouse at the moment, so hopefully this helps somewhat.

From there you would need to print it out how Orion would expect, could do something simple like "print(f'Statistic: {stat}')" and set your threshold on the template.

jrouviere

Not even sure I know enough powershell at the moment without researching it to see how I'd attempt the same thing, I'm also "just OK" at regex, but take a look and if you have your own ideas maybe we'll all learn something.

KMSigma.SWI

If this is a Windows box and has PowerShell, It would go something like this:

# Anything over $BadTime consider bad$BadTime = 1000#dump the contents of the Log file into a variable$Content = Get-Content -Path "C:\Users\kevin.sparenberg\Documents\SampleLog.txt"For ( $i = 0; $i -lt $Content.Length; $i++ ){    #Split the line at spaces and select the 10th entry (arrays in PoSH start at 0)    # this is the etime=#### segment    $etime = $Content[$i].Split(" ")[9]    #Let's extract out the number itself by splitting this smaller string at the "=" and then selecting the second element (the number)    # To make things easier, let's also convert it to an integer    $etimeNumerical = [int]( $etime.Split("=")[1] )    if ( $etimeNumerical -gt $BadTime )    {        Write-Host $Content[$i] -ForegroundColor Red        Write-Host "Danger Will Robinson - value is $etimeNumerical!" -ForegroundColor Red    }    else    {        Write-Host $Content[$i] -ForegroundColor Green        Write-Host "Everything here is fine - value is $etimeNumerical" -ForegroundColor Green    }}

Note that dumping the entire file into a variable is NOT the idea for very large files. There are other ways to do it.

smalleggo

Thank you everyone who looked at this. We are still working with the end user and attempting PERL scripting. I will let you know what the outcome is.

Charlie