File Count Monitor deficiencies, and how to overcome them?

This thread is very similiar to the one found at Need help getting File Count monitor to work, but different enough (and enough time has lapsed) for me to feel that a new topic was necessary.

I am wanting to use the File Count Monitor to tell me how many *.tmp files are in a particular directory on each of a set of our servers. However, I began to notice yesterday that once the count got somewhere above 28k or so the process would just bomb out entirely. Instead, it would generate an error of "Unexpected error occurred. Quota violation".

This makes me think that there is some sort of upper limit (time to complete, file count number, etc) that File Count Monitor can handle before it bombs out. However, I am at a loss for coming up with a replacement. I had tried a PowerShell Script Monitor with the following code:

Param([string]$Path)$FileType = "*.tmp"$FileCount = [System.IO.Directory]::GetFiles("$Path", "$FileType").Count'Message.FileCount: Number of *.tmp files found.''Statistic.FileCount: ' + $FileCount

When this code is tested outside of Orion, I get the expected results and file count, and it completes within a few seconds. However, when I test it in Orion, I get an output of:

Message.FileCount: Number of *.tmp files found.

Statistic.FileCount: NaN

How can I accurately monitor this directory in Orion, or what am I doing wrong? Such a simple task is giving me nothing but headache.

Find more posts tagged with

file_count

quota_violation

PowerShell

Accepted answers

jangliss

I've had issues with powershell and dealing with large volumes of files in the past. Usually it's been an issue with memory. It's possible that there is a memory cap that SolarWinds is hitting due to the size. I've had better success using WMI to query the file system instead. For example doing the following:

$files = Get-WmiObject CIM_DataFile -Filter "Drive='C:' and Path='\\yourpath\\' and Extension='tmp'"

This will return WMI objects, which can be counted just as you were doing before.

All comments

jangliss

$files = Get-WmiObject CIM_DataFile -Filter "Drive='C:' and Path='\\yourpath\\' and Extension='tmp'"

This will return WMI objects, which can be counted just as you were doing before.

aLTeReGo

The script is indeed good. What you appear to be lacking is good error handling. The "NaN" that you're receiving means "Not a Number". That means whatever is being returned is a string and not an integer. Since your script appears to work without issue on my machine, the problem is most likely with your WinRM configuration. I'll assume you are trying to execute this script against a remote host? If that's the case you need to ensure that WinRM is properly configured on both the Orion server itself, as well as the remote host where you wish the script to run.

temp files.png

jangliss

Re-reading the original issue, and reading aLTeReGo's response made me go back an look at this again. What I saw was that you can successfully run it, up until you hit 28k files, at which point you get a quota violation error. This would explain the NaN message, because the error isn't a number, as was pointed out. How is this being executed? Is it against a local path, or a remote path? Are you using remoting to query a local path on a remote server? I'm wondering if you're hitting a quota limit on the remote memory if you're using remoting. By default, remoting only allows 150MB per shell, and doing a GetFiles on a path with 28k files in might exceed that. Of course, I could be way off the mark, and it could be something simple.

ouberlord

Actually, the Execution Mode has been set to Local Host, as I've always used the same for my other Windows PowerShell Monitors. Its possible that I could enable WinRM on that server to perform a test of Remote Execution, but I was hopeful that I wouldn't have to. Do you think this may be the issue? If so, is it the same issue that affects the File Count Monitor?

ouberlord

That seems likely. So far the Execution Mode has been set to Local Host, as I was hopeful I wouldn't need to enable WinRM. That said, if Local Host isn't an option I don't suppose I have much choice as to otherwise. Still, it seems odd; I wouldn't think that doing a file count would consume all that terribly much RAM.

aLTeReGo

The limitation you are encountering with SAM's File Count Monitor is imposed by Microsoft and WMI. Any query that takes longer than "X", or consumes more than "Y" amount of memory will result in timeouts or hard errors. In this case it looks to be a timeout that is causing the number of WMI connections to exceed the quota limit for the server. Increasing the quota would do nothing unfortunately because the violation of memory or time would still occur. In this case the problems appear similar, but are in fact have different causes.

Without proper error handling it's difficult to say for sure what the exact issue the script is having. It may however be resolved by executing the script locally on the host you are trying to monitor using WinRM. Remote execution is far less efficient and can pose other problems.

ouberlord

Well, I tried a new tactic, and I've got some more info. For testing purposes, I temporarily changed and tested this script in Orion:

Param([string]$Path)$FileType = "*.tmp"$FileCount = [System.IO.Directory]::GetFiles("$Path", "$FileType").Count$FileCount

Interestingly, I get this as a response:

Errors: ==============================================

Exception calling "GetFiles" with "2" argument(s): "Access to the path '\\10.1.42.249\c$\Users\imagerightscheduler\AppData\Local\Temp\' is denied."

At line:4 char:45

+ $FileCount = [System.IO.Directory]::GetFiles <<<< ("$Path", "$FileType").Count

+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException

+ FullyQualifiedErrorId : DotNetMethodException

However, I am running the test with a set of stored credentials that matches my current domain credentials. Using these same credentials to run the same script on PowerShell from my desktop works fine.

Still digging into this, but this just gets odder and odder.