This discussion has been locked. The information referenced herein may be inaccurate due to age, software updates, or external references.
You can no longer post new replies to this discussion. If you have a similar question you can start a new discussion in this forum.

Strange results when enabling Apache Access logging

FormerMember
FormerMember

LEM version 5.3 on a Virtual Appliance, Agent version 5.3

I just started installing LEM on our production web platforms and when I enable Apache Access Logging my system gets flooded with disconnected nodes.  These nodes happen to coincide with log entries from the Apache Access log.

When I turn off the Access Log tool the random node creation stops.  If I leave those nodes in the node list the system becomes unstable, randomly loosing connectivity to the agents.  If I delete these newly discovered nodes the system seems to stabilize.

They continue to fill up the node list until my licensed node count reaches it's maximum. 

It did not do this with the test systems on 5.2.

I imagine I should open a ticket but I thought I would also post it here to see if there's a simple setting I'm not using (I turned off remote updates).

 

  • FormerMember
    0 FormerMember

    UPDATE:

    I tested this on multiple nodes with alerts, nDepth and both configured with the same results.  The difference between this server and the test server is it's behind a firewall but all ports needed are open and all other tools seem to work fine from these servers.

  • FormerMember
    0 FormerMember in reply to FormerMember

    Can you give me an example of what the random nodes are named?

    Do you have nDepth full log storage enabled?

    Do you have Apache Error also enabled, or just Access?

    There's a detection engine that tries to extract node names from log data to make sure they appear in Manage > Nodes. It sounds like it may be mis-detecting against Apache Access logs.

  • FormerMember
    0 FormerMember in reply to FormerMember

    Nicole,

    I think that's exactly what's happening.  I'm removing nDepth from all the tools although it does the same regardless how I configure the agent.  I just have a small VM appliance supporting a LEM50 license so I was sending nDepth to localhost but until I get this sorted out I'm disabling that.

    I'm not sure how to enable or disable nDepth, it's in it's default config.

    Apache Error is on and it's not having an issues.

    Here's an example of the nodes, the IP addresses coincide directly with access IP's from the access log.

    Node IP        Node Name    Agent Node
    74.203.107.88,    74.203.107.88,    198.152.214.22

    I have a ticket open, Case #282424

  • FormerMember
    0 FormerMember in reply to FormerMember

    Thanks - development has confirmed what you're seeing and is investigating a resolution.

    nDepth log storage defaults to "off" (you can always search alerts), so if you haven't turned it on, you're okay there. Enabling it in the tools doesn't cause any harm or take up any space until the appliance has it enabled.

    So, what's happening is that the tools themselves (Apache Access, for example) try to detect the originating source IP based on the contents of the log data itself. The regular expression that does this detection is picking up the wrong IP.

    And, we're fixing it! We should have an update that you can install soon. It'll be super easy, it's just a change to the tool. After that, you can delete the non-Agent nodes that are appearing, and they won't reappear.

  • FormerMember
    0 FormerMember in reply to FormerMember

    We have resolved this issue - I asked the owner of your case to contact you with the resolution.

    Anyone else affected should contact support for the tool update associated with this fix. It will be automatically included in the next maintenance release.

  • FormerMember
    0 FormerMember in reply to FormerMember

    Well, I received the message, downloaded the update and applied it, the problem IS NOT RESOLVED.

    After some investigation I discovered both the old and new version of the tool .xml still exists on the appliance so it's not pushing the updated .xml to the agents.

    The new .xml may indeed correct the problem but the update script on the 5.3 appliance isn't replacing the older script, it's just adding another one to the list and confusing the manager.

    I submitted all this information over the weekend and this morning but have not heard back from support.

  • FormerMember
    0 FormerMember in reply to FormerMember

    I think we might have this one nailed down... hopefully you have the update in hand.

    There was a bug introduced when we addressed an issue for a customer with a non-standard logging config. We've separated these two issues and you should be able to use the newer version.

    We're also looking into what caused the two versions issue that support cleaned up.

    Thanks for being patient!

  • FormerMember
    0 FormerMember in reply to FormerMember

    I applied the new version and it seems to be working well.  Thanks for working through the issues and getting it resolved.