11 Replies Latest reply on Mar 22, 2016 2:43 PM by Radioteacher

    Three known security issues in LEM 6.2.1

    Radioteacher

      The issues below are documented in a Solarwinds Case opened on February 18.  I am still waiting for a fix and/or an estimated time to fix.  I did talk to a person in Lehi yesterday that is going to discuss this with development and get back to me with an estimate when the issues below will  be resolved.

       

      The Java Deserialization issue is rated as critical, the Apache Flex BlazeDS XXE Injection is rated as high and Slowloris is rated as low.

       

      I wish these security issues were handled in a consistent manner across development groups. 

       

      Last month, a very similar Java RMI issue in Virtualization Manager was fixed six days after I opened a case.

       

       

      Issue details below.

       

      Java Deserialization Remote Code Executionremotermi10009

      TCP

       

      Vulnerability Details

       

      This asset is running a Java RMI service that utilizes a third party library known as Apache Commons Collections. The version of this library being used by the RMI service contains a Java deserialization vulnerability that can be leveraged to execute code remotely. Serialization is a form of data transformation that makes the data suitable to transfer over the network so that it may be reassembled by the receiver into it's original form. The act of transforming the data to be sent is known as serialization. The converse of this is known as deserialization. In this case, the library in use by the RMI service does not properly validate the data that is being deserialized and ends up executing arbitrary code supplied by the attacker. In order to validate this vulnerability, the scanner sent a serialized payload that will force a vulnerable host to issue ICMP requests to the scanner so that it can know whether or not the asset is executing arbitrary code. If the scanner sees the ICMP requests in response to sending the serialized payload, the host is vulnerable. Impact: This vulnerability can be leveraged by an attacker to execute arbitrary code on the asset which could result in complete compromise of the underlying system running the Java RMI

       

      Solution

       

      If the Java RMI service is not required for business purposes, it should be disabled. If the RMI service is running as part of third party software installed on this asset, please contact the vendor for a patched version of the software or to determine when this issue will be resolved.

       

      Apache Flex BlazeDS XXE Injection - remote - http 443, 80, 8080, and 8443 TCP

      Vulnerability Details 118600

       

      This asset is running software that is utilizing the Apache Flex BlazeDS library to process the Action Message Format (AMF) protocol. This library is vulnerable to an XML external entities attack due to the fact that the XML parser implementation does not disallow the processing of DTDs (Document Type Declarations) in it's default configuration. Impact: A remote, unauthenticated attacker may be able to use this vulnerability to retrieve arbitrary files from the filesystem or create a denial of service condition.

       

      Solution

       

      Upgrade to the latest version of the software to remediate this vulnerability.

       

       

      Slowloris Resource Depletion And Denial Of Service

       

       

      Vulnerability Details

       

      This host is running a web server that appears to utilize a configuration which allows a single remote host to consume all connection resources. The slowloris denial of service is achieved by systematically establishing and maintaining connections through partial header requests and http keepalive messages. The attacker can continue this methodology until all web server connections are tied up, resulting in a complete denial of service to legitimate users. Web servers that utilize threading are more susceptible to a slowloris attack. In contrast to other web server denial of service techniques, this attack only requires a tiny amount of bandwidth and a single host IP address.

       

      Solution

       

      This attack can be mitigated by; 1) limiting the number of inactive concurrent web server connections a single user may be allowed to maintain, or 2) setting a minimum threshold of data per second a web server connection is allowed to maintain without being dropped. Examples of applicable apache modules useful for implementing suggested remediation include: mod_reqtimeout -

      http://httpd.apache.org/docs/2.3/mod/mod_reqtimeout.html mod_qos -

      http://sourceforge.net/projects/mod-qos/ mod_cband -

      http://cband.linux.pl mod_limitpconn -

      http://dominia.org/djao/limitipconn.html mod_evasive -

      http://www.networkdweebs.com/stuff/security.html mod_security -

      http://www.modsecurity.org/ mod_antiloris -

      ftp://ftp.monshouwer.eu/pub/linux/mod_antiloris/

      Additional third party modules may also be available. Please contact the vendor for additional information.

        • Re: Three known security issues in LEM 6.2.1
          rharland2012

          Has there been any movement on this from the Solarwinds side? Barring an actual fix, it would be very useful to at least have mitigations we can leverage in the meantime on the appliance.

          • Re: Three known security issues in LEM 6.2.1
            Radioteacher

            This is not a fix but a possible direction.

             

            The VMan team used a command similar to the code below except the xxxxx was the port number that was a problem.

             

            sudo iptables -A INPUT -p tcp --dport xxxxx -s 127.0.0.0/8 -d 127.0.0.0/8 -j ACCEPT

             

            sudo /etc/init.d/iptables save

             

            RT

            • Re: Three known security issues in LEM 6.2.1
              nicole pauls

              There's another report of the XXE vulnerability here - Apache Flex BlazeDS XXE Injection Risk - so I'm sure the team is indeed looking into it. (You might want to watch that thread in case they follow up.)

               

              For all of these you can mitigate at the bare minimum by restricting access to the LEM console, which is the only thing that uses http/https/rmi. (Use "restrictconsole" from the CMC via SSH or the appliance command line and enter a set of allowed IPs) That will reduce your exposure, though will not entirely eliminate the vulnerabilities if they are valid.

                • Re: Three known security issues in LEM 6.2.1
                  Radioteacher

                  Nicole,

                   

                  I have attended two of your live LEM events and met you in person at Thwack Camp 2015.  I was the lone non-Solarwinds employee in the room at Thwack Camp.  Now I know what the inside of a tornado feels like.

                   

                  Below is what my security scanner is showing me.  As you see, the Java rmi on TCP port 10009 is the worst of the issues.  This is followed by the Apache Flex BlazeDS XXE Injection issue.  Last issue is Slowloris.  The only thing slower then Slowloris is the LEM's development team's ability to patch this seven year old issue. 

                   

                  Slowloris is so old.....How old is it?  Slowloris is so old, if it was a child it would be in second grade.  If they wait a few more years maybe Slowloris would be old enough to code a fix for itself.

                   

                  Next Monday my team will be meeting with a Director of Software Development at Solarwinds to discuss our concerns.

                   

                  For years I have said over and over.  "Design Secure, Build Secure and Maintain Secure."  Security is the first and last thought I have in every task I do at work.  Because of that, security has changed over the years from about 15% of my job to over 85% of my job.

                   

                  For weeks I have been trying to see movement at SW to get these issues.  Over the past few days I think it is happening.

                   

                  I get it, adding new enhancements and widgets is a lot more fun then fixing issues but sooner or later you have to clean things up.

                   

                  Thanks for all you do,

                   

                  Radioteacher

                  Capture.JPG

                    • Re: Three known security issues in LEM 6.2.1
                      nicole pauls

                      Thanks RT.

                       

                      I know we discussed Slowloris before I left SolarWinds, I was surprised I couldn't find a blog post or comment. One mitigating factor to many of the tomcat (and related) attacks are that the LEM console is intended to be used on the internal network, so a low severity attack is somewhat lower. Ultimately it'd be nice to come up clean. We talked about limiting the number of connections to some arbitrary high number around the time of LEM 6.2, but I didn't find it in the release notes so I don't think it was changed.

                       

                      We had also discussed permanently firewalling off port 10009 because it's only used internally, so that change is probably pending as well. Port 10009 is only used for portions of the CMC/appliance management console that need to connect back to perform UI-like operations (kind of like an API), and there is authentication expected on that connection as well (unless it's coming from 127.0.0.1).

                       

                      There's always the chance some of these could be false positives based on what's publicly accessible that can't actually be compromised (e.g. version info but there's a back ported patch)  - do you have any more details on the RMI issue beyond what you posted that might indicate what the scanner was checking for that trigger?

                       

                      I also didn't see wolram comment on this thread when he investigated the other one, maybe he has additional details.

                  • Re: Three known security issues in LEM 6.2.1
                    Radioteacher

                    I am happy to report that the Critical level issue Java Deserialization Remote Code Execution remote rmi 10009 TCP is fixed!!!!  I got a call today from support and they were able to get this one off my issues list.

                     

                    One down, two to go!

                     

                    RT