7 Replies Latest reply on Oct 28, 2014 10:19 AM by gfoltz@azgfd.gov

    How to make status lights reflect CURRENT status?

    gfoltz@azgfd.gov

      I want to be able to provide a "dashboard" for my management, based on server and application status in SAM.  However, right now, it looks downright SCARY!  If I fix a problem (or adjust a threshold to reflect my environment), I want to see that red light GO AWAY, but often, it isn't.  Also, there are some alerts (like available system memory) that I know are going to trigger on some of my older servers, because they do have puny amounts of RAM, and I can't fix it.

       

      One of the biggest irritants is the "SQL Server Agent Job Info".  This single component is causing ALL my SQL server instances to show RED--just because I have a maint job that failed on them 2 weeks ago... And I can't disable it because its an "AppInsight" template!

       

      Bottom line, is there a way to show CURRENT status on a node/app?  One with no historical reference?

        • Re: How to make status lights reflect CURRENT status?
          aLTeReGo

          The ability to disable SQL agent jobs in AppInsight for SQL is a feature request we're currently tracking internally under FB342956. To your main point however, all status icons are reflective of the current state of anything it is monitoring. Adjustment of thresholds or other configuration changes and alterations many take effect after the next poll of that object.

            • Re: How to make status lights reflect CURRENT status?
              gfoltz@azgfd.gov

              Ok, now that you mention it, I think I saw another thread comabout being able to disable the SQL Job info.  Good to know I am not the only person having issues with that sucker.

               

              As for the status icon, I must be missing something, because I have gone into a component (e.g. "page reads/sec") and see that according to the last poll, the "current value" is in the acceptable range, but the icon is still RED!  Sure enough, maybe it spiked over an hour ago, or YESTERDAY, and as far as I can tell, that is the only reason it is still red...

                • Re: How to make status lights reflect CURRENT status?
                  aLTeReGo

                  Something is clearly not right if the component status icon is displaying warning or critical when the thresholds are not currently exceeded. Is it possible that you've configured sustained status conditions for that component? The only thing I can think of that would explain the behavior you're describing is if the component status was not being calculated based on a single poll, but instead was being calculated based on "X out of Y polls" with some crazy numbers like 2 out of 100 polls. This means the component would go into a warning or critical state if any two polls out of 100 (sliding window) were above the threshold. I would recommend removing the warning and critical thresholds entirely for this component (leave them blank/empty) save your changes, and force a poll to see if that changes the state of that component.

                   

                  Page Reads.png

                    • Re: How to make status lights reflect CURRENT status?
                      gfoltz@azgfd.gov

                      Well, I am not saying that the lights never change, just that they don't change in any kind of real time--indeed, the term "glacial" might be more fitting, especially for "parent" status lights.  Even forcing the poll, and then refreshing the screen, the lights are out of whack for quite a while.  If I wait a few hours, everything seems to finally sync, but I don't want to wait a couple hours--I want to fix a problem (rebuild a db index, for example), force a poll, see the component turn green, and then go refresh the parent on my "summary" screen, and the error is gone.  But it is NOT working that way...

                       

                      On the "x out of y" scenario, I do have a couple components set that way, but they are not the ones I am talking about.  But you have me concerned, because if it works the way you say it does, then it is COMPLETELY useless to me.  The way I read that was that in order to go into the warning or critical state, it has to hit AND MAINTAIN the "x out of y" ratio.  Anything LESS than X out of Y, and the status is not triggered (or untriggered), resulting in a green.  You seem to be saying that once I go yellow/red, the polling has to complete a full cycle to go back to green?  I am trying to make it so my status does not react to momentary spikes, but you seem to be saying that it will make it stay yellow/red longer!

                        • Re: How to make status lights reflect CURRENT status?
                          aLTeReGo

                          What you describe is "Consecutive Polls", which is different than "X out of Y Polls" as described in this excerpt from my SAM 6.1 blog posting.

                           

                          Server & Application Monitor 6.1 - New Feature Overview and Beta Sign-Up

                           

                          Sustained conditions in SAM 6.1 can be defined independently for both warning and critical thresholds to provide maximum flexibility. Both "X Consecutive Polls" and "X out of Y Polls use a sliding window approach to evaluating thresholds. After each poll, the conditions defined for the threshold are evaluated based on the bounds of the sample size. Put simply, that means that after each poll a new sample is collected and added to the evaluation, while the oldest sample is removed from evaluation. Below, I provide two examples. The first example on the left demonstrates the "X consecutive polls" method. In the left column I show the numerical value collected from the poll (the sample). In the right column I show the status of the component as defined by the sustained condition. The "Sample Size" in this example is "3", meaning that three consecutive polls/samples must exceed the threshold of "80" before the status should change to "Warning".

                           

                          Warning = Greater Than 80 for 3 Consecutive PollsWarning = Greater Than 80 for 3 out of 5 Polls
                          Polled ValueStatus
                          65UP/Green
                          77UP/Green
                          88UP/Green
                          85UP/Green
                          89Warning
                          83Warning
                          46UP/Green
                          81UP/Green
                          22UP/Green
                          Polled ValueStatus
                          65UP/Green
                          82UP/Green
                          34UP/Green
                          95UP/Green
                          88Warning
                          90Warning
                          35Warning
                          25Warning
                          15UP/Green

                           

                          The second example demonstrates the "X out of Y polls" method. While the "Sample Size" for evaluation in this example is "5" polls, any three of those 5 polled samples must exceed "80" before the status of this component would change to "Warning". Using the same sliding window approach as the first example, with each successive poll a new sample is collected, while the 6th sample is dropped from evaluation.

                  • Re: How to make status lights reflect CURRENT status?
                    aLTeReGo

                    SAM 6.2 (currently in beta) provides the ability to disable the SQL Agent Job info component monitor in AppInsight for SQL.

                     

                    SQL Agent Job.png