This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Group Member Added/Removed Frequently??

I've noticed recently that one of our SolarWinds groups keep regularly losing and gaining a member, but I don't know why this is. Is there anyway to ascertain which member(s) are getting added/lost to try and figure out what is happening. This is happening when I'm confident there are no new nodes being added or removed on the system.

Any ideas? Could this be a bug? The behavior started about a week ago (I suspect after I applied the latest HF bundle) but none of our other groups are doing this....

Capture.PNG

  • Hi blueyviii

    How did you define the group?

    Is it a static or dynamic assignment? 

  • There's a dynamic query and a couple of statically added nodes too....

  • Do you know which node(s) is coming and going from the group?

    Is it statically assigned or dynamically?

    If it's statically assigned, then it's sounds very bug like (support call time).

    If it's dynamic what does it look like?

    Is it possible the chosen quality is changing itself, such as a DHCP node and a dynamic IP filter for the group?

    Just trying to get an idea of what could be going wrong (with your group).

  • Is there a way to see which node(s) is getting added/removed?? I'm pretty sure If I can figure that out I can solve the problem...

  • If you were to view the group, in say Manage Nodes and ensure you refresh the screen every 30-45 minutes, it should be obvious.

    You could also create a report to list the group members and run it every 30-45 minutes with a schedule, until you catch it.

    I did a search of my Orion databases events and auditevents tables, and then added and removed a few devices to a group or two, but all I could see was that the group had lost/gained members not actual device details.

  • Thanks - I've just set-up a scheduled report to see if that identifies the culprit emoticons_happy.png

    I was hoping there would be some kind of log that identified the device which keeps getting removed/added....

  • OK, I've now found the server that is repeatedly being removed/added but I can't see any reason for it. The server had been constantly available, and if I check the node it has a 100% availability???

    Any ideas?

  • Good news on finding the culprit, but I have no idea without seeing how the query is built emoticons_wink.png

    Can you screenshot (obscuring anything sensitive) or at least break down in words the Dynamic Query(ies) the group uses ?

    And the Group notifications you are seeing, aren't node down events, they are Audit Events. If the node had gone down, you may well have seen the Group go into a warning state (depending on your status propagation setup), but these would be seen in the Events table, not the Auditing one.

    .

    It is the group definition that define what is in the group, so there must be something in the definition which is not static, on that device.

    At times the node qualifies to be in the group, and at others it doesn't.

    And not to forget, this could still be a bug, have you raised a support call with Customer Support?

    On a side note, here's some SQL if you want to have have look at your database audit table, to get you started.

    SELECT TOP (250) AE.TimeLoggedUtc

          ,AE.AccountID

          ,AAT.ActionTypeDisplayName

          ,AE.ActionTypeID

          ,AE.AuditEventMessage

          ,AE.NetworkNode

          ,AE.NetObjectID

          ,AE.NetObjectType

      FROM AuditingEvents AE

    INNER JOIN AuditingActionTypes AAT ON AAT.ActionTypeID = AE.ActionTypeID

    ORDER BY AE.TimeLoggedUtc DESC

    The ActionTypeIDs you might be interested in are:

       

    ActionTypeIDActionTypeActionTypeDisplayName
    19Orion.GroupAddedGroup added
    20Orion.GroupDeletedGroup deleted
    21Orion.GroupChangedGroup edited
    22Orion.ContainerMemberDefinitionAddedGroup Member Definition added
    23Orion.ContainerMemberDefinitionDeletedGroup Member Definition deleted
    24Orion.ContainerMemberDefinitionChangedGroup Member Definition changed
  • The Node itself is behaving perfectly, but for some reason it's still getting added/removed from the Group. The Dynamic Group query is really simple and consists only of Vendor=Windows and SystemName contains "SRVPROD" There are 49 other members found by the dynamic query but only this 1 particular server is exhibiting this behaviour.

    I deleted the node and the re-added it about 90 mins later. The  dynamic query picked up the server and added to the group within 30 secs. But after around an hour it was removed, and then re-added 15 minutes later!!

    In the end I got so frustrated I just manually assigned this particular node to the group and it hasn't been removed/re-added since!! I'm happy with the workaround for now but would be an issue if it starts happening to more groups/nodes. I've been monitoring other groups and none of them are showing this behaviour...

  • Nicely done on the workaround emoticons_happy.png

    But I'd be inclined to, if you can, raise a support call (Customer Portal Login | SolarWinds)​. The system's Vendor should not be changing and I'm struggling to think of scenario where the system's hostname would be changing.

    Thoughts of load balanced DNS, say through an F5 to two DNS servers with clashing records, but would that change the system name? I doubt it....