8 Replies Latest reply on Nov 3, 2011 7:12 AM by mwhatman

    Strange results when enabling Apache Access logging

    mwhatman

      LEM version 5.3 on a Virtual Appliance, Agent version 5.3

      I just started installing LEM on our production web platforms and when I enable Apache Access Logging my system gets flooded with disconnected nodes.  These nodes happen to coincide with log entries from the Apache Access log.

      When I turn off the Access Log tool the random node creation stops.  If I leave those nodes in the node list the system becomes unstable, randomly loosing connectivity to the agents.  If I delete these newly discovered nodes the system seems to stabilize.

      They continue to fill up the node list until my licensed node count reaches it's maximum. 

      It did not do this with the test systems on 5.2.

      I imagine I should open a ticket but I thought I would also post it here to see if there's a simple setting I'm not using (I turned off remote updates).

       

        • Re: Strange results when enabling Apache Access logging
          mwhatman

          UPDATE:

          I tested this on multiple nodes with alerts, nDepth and both configured with the same results.  The difference between this server and the test server is it's behind a firewall but all ports needed are open and all other tools seem to work fine from these servers.

            • Re: Strange results when enabling Apache Access logging
              nicole pauls

              Can you give me an example of what the random nodes are named?

              Do you have nDepth full log storage enabled?

              Do you have Apache Error also enabled, or just Access?

              There's a detection engine that tries to extract node names from log data to make sure they appear in Manage > Nodes. It sounds like it may be mis-detecting against Apache Access logs.

                • Re: Strange results when enabling Apache Access logging
                  mwhatman

                  Nicole,

                  I think that's exactly what's happening.  I'm removing nDepth from all the tools although it does the same regardless how I configure the agent.  I just have a small VM appliance supporting a LEM50 license so I was sending nDepth to localhost but until I get this sorted out I'm disabling that.

                  I'm not sure how to enable or disable nDepth, it's in it's default config.

                  Apache Error is on and it's not having an issues.

                  Here's an example of the nodes, the IP addresses coincide directly with access IP's from the access log.

                  Node IP        Node Name    Agent Node
                  74.203.107.88,    74.203.107.88,    198.152.214.22

                  I have a ticket open, Case #282424

                    • Re: Strange results when enabling Apache Access logging
                      nicole pauls

                      Thanks - development has confirmed what you're seeing and is investigating a resolution.

                      nDepth log storage defaults to "off" (you can always search alerts), so if you haven't turned it on, you're okay there. Enabling it in the tools doesn't cause any harm or take up any space until the appliance has it enabled.

                      So, what's happening is that the tools themselves (Apache Access, for example) try to detect the originating source IP based on the contents of the log data itself. The regular expression that does this detection is picking up the wrong IP.

                      And, we're fixing it! We should have an update that you can install soon. It'll be super easy, it's just a change to the tool. After that, you can delete the non-Agent nodes that are appearing, and they won't reappear.