Skip navigation

PatchZone

3 Posts authored by: arch3angel

Reflecting back to 3rd Party Patching - The Unseen Security Threat we saw a scenario that described an attack vector which is much more prevalent than many believe.  Using 3rd party applications and their internal flaws to gain local privileges, which in turn provides a pivot point for other attacks and access throughout your environment.

 

In reviewing patch management programs I have found many which cover the topic of keeping these aforementioned applications patched, but little to describe the actual process of patching them; only the simple "keep 3rd party applications up to date...".  However, taking this general approach fails to improve the security within your environment.  A number of 3rd party applications leave what many refer to as "patching artifacts" or "software artifacts".  In the situation described in the previous post we have the attacker gaining access through the 3rd party application.  Expanding upon the scenario mentioned in this situation, we find that the patch management process was followed correctly and it was not the update, or lack of updates, that permitted the attackers access, but the artifact that was left behind.  The artifacts are the result of the previous versions either not being removed and/or not completely removed during the installation.  This practice is much more common than one might expect, and is often seen even in default system updates for certain operating systems.

 

Example 1: Oracle Java

Let's look at this from the software developer’s perspective for a moment.  As you can see, Oracle's Java website even stresses the fact that previous versions need to be completely removed prior to the installation of the latest version.

 

On a clean installation of Windows 7, fully patched, but running Java version 6 update 37, we begin a typical update process prompted by Java's own notification bubble.  We complete the update and are now at version 7, update 9 .  So we are good to go right?  Wrong, take a moment to look at not only the Java installation, but also the Java Runtime Environment screenshots, which shows that both of these versions are enabled at the same time.  So all the exploits that were available to an attacker prior to the update are still available and fully functional.

 

So how did the warning on their website, which was not so obviously found, notification bubble, and the update do to protect our systems? Absolutely nothing!

JRE6 Artifact After JRE7 Installation.pngJRE Runtime Settings.png

So maybe this is just a Windows flaw, bug, or feature.  It's not every operating system right? No, below is another screenshot, which came from an Apple OSX 10.8 system, which was recently updated, via Apple's internal patching process.  Apple would certainly remove the previous version, right?

JRE6 Artifact After JRE7 Installation (Apple).png

In reviewing this screenshot we see that not only is Java version 6 update 35 installed from Apple by default, but also we have both the 32 bit and 64 bit versions installed side by side.  So let's do the update shall we, let's update the Java installation to version 7, update 5.  Nope, the same result as the Windows Java update.  Both versions installed and enabled for the attackers pleasure.

 

This of course is just isolated to Oracle and Java right? Negative, this is actually very common if you begin to look around closely at your 3rd party applications.

 

Example 2: VanDyke SecureCRT

Just so that everyone doesn't think I am picking on Oracle here, let's look at a program I have used in the past and actually truly enjoy having installed when I work in a Windows environment. VanDyke's SecureCRT is described on their website as:

 

SecureCRT for Windows, Mac, and Linux provides rock-solid terminal emulation for computing professionals, raising productivity with advanced session management and a host of ways to save time and streamline repetitive tasks. SecureCRT provides secure remote access, file transfer, and data tunneling for everyone in your organization.

 

It's a program developed with Security in mind correct? They would certainly make sure no artifacts were left behind after an update right? Yet again we find ourselves facing the artifacts of patching and updating...

SecureCRT Artifcat After Upgrade.png

In this screenshot we see version 5.5 and version 6 installed side-by-side, again sitting there waiting for an attacker to begin their assault on your systems.

 

Some thoughts on why

So why does Oracle do this with Java, VanDyke do this with SecureCRT, and other software development companies follow suit?  I mean Oracle even warns you on their website not to have both versions installed, to completely remove the previous version.

 

The answer remains unanswered, but the theory is that many large corporations use these applications within their internal application development and they want to provide the opportunity to install the new version while preserving the previous version just in case issues begin to surface.  This is great in theory, however in practice this has become a hidden treasure for attackers.  Little Egyptian artifacts spread across your environment just waiting for the next tomb robber to sneak on in and steal your most vital data.

 

Recommendations

In closing, when you are developing the patch management processes and beginning the deployment of updates to 3rd party applications, let's look at the artifacts being left behind.  Maybe having version 6 and version 7 of Java installed on the same machine just in case you run into an issue isn't such a great idea after all.  Let's go back to the The Patch Management Process: 5 Common Sense Tips and consider building into our process a much needed step two, "Development / Testing Environment", this way you can have your testing completed and all bugs corrected prior to the update rolling out.  This way you can roll out the removal of the previous version first, and then roll out any updates or upgrades you might require.

You’re sitting at your desk, punching out a few keys on the keyboard, and then it happens.  The moment everyone knows is possible, but always thinks it happens to someone else.  Your phone rings and the person on the other end is now informing you that the company has been part of a data breach.  Your company’s databases and access has been found on an underground website where this type of information is sold and traded.  After hanging the phone up you say to yourself, “How?”, “What about our firewall?”, and “Our monitoring and intrusion prevention should have caught this.”.  These are just some of the questions which begin to flow from you and others.  All these questions are important but at this minute the only one to consider is “How?”.  You need to begin Incident Response procedures and find the cause.

 

Patch Applications and the Systems they run on

The cause was covert, unexpected, but used by many.  A 3rd party application was the open door, the way the attacker found access.  When patching systems many people focus on the system itself but overlook the applications actually installed on these systems.  Attackers are still using the systems, so without a doubt this is important, but the initial entrance and pivot point is hardly the system.  Many times the attacker uses a form of Social-Engineering to gain the users trust, allowing the user to give access to the attacker from within the company; bypassing all the external protections put in place.  This type of attack is often made easier by tools such as TrustedSec’s very own Social-Engineer Toolkit (SET) (https://www.trustedsec.com/downloads/social-engineer-toolkit/ ).   When focusing on patching we all need to take a look at the applications themselves.  I would like to say either the system or the applications is most important; however, I believe they are equally important.  The installed applications need to be documented with versions, by system.  This needs to be maintained frequently and the updates pushed out after following the “5 Common Sense Tips.”

 

Application attacks are a real security threat

Attacks on applications are common, and a real security threat.  When the 3rd party applications are ignored as a threat or simply unmaintained you will not be able to know whether the recently exploited application is in your environment.  That single application, installed just on one system, just for one project, and then forgotten about, has now opened the door and allowed an attacker to exploit your company.  This exploited machine was the pivot point for attacks on every other system, in turn exploiting other applications, and finally gaining access to the databases.  Now your company is being traded in the underground like stocks traded on the world’s largest stock markets.  This breach has caused your company financial losses in such areas as intellectual property, reputation, and IT resources to say the least; all because 3rd party applications were overlooked in the overall security posture of the organization.

 

Document and Patch 3rd party applications

When developing your patching process you need to have a procedure in place for documenting 3rd party applications as well as patching them.  The patch cycle for 3rd party applications are not as defined as Microsoft, so staying vigilant with these vendors is a must.  A great starting point with 3rd party applications can be found at the Table of 3rd Party Patches.  Also consider your patch management solution, consider whether or not it includes support for the Common Vulnerabilities and Exposures (CVE) (http://cve.mitre.org/ ) databases.  Also consider becoming part of the mailing lists at the United States Computer Emergency Readiness Team (US-CERT) (http://www.us-cert.gov/) or a CERT in your region.  Maybe try using resources found at the National Vulnerability Database (NVD) (http://nvd.nist.gov/ ) website. 

 

There are many resources, but ultimately it falls back on making sure the procedure is developed and followed.  Everyone will be hacked at some point, which is a given fact.  The variable in the equation is to what extent are the damages and whether or not you were able to catch the attacker before got everything; otherwise you may be the next person getting that call, having no idea you even had an attacker running through your 3rd party applications.

Editor’s note:  PatchZone welcomes Robert Miller who will be writing a series of blog posts on PatchZone.  Robert has over 14 years of experience with network, system, telephony administration and corporate security and is currently a Security & Operations Manager for a privately held company.  Robert is also a respected security researcher.  Check out his personal blog at www.armoredpackets.com.

 

The Patch Management Process: 5 Common Sense Tips

 

When an IT professional says “We need to update the servers this weekend”, what comes to mind?

 

Many of us simply say to ourselves “Great, another weekend spent in the datacenter…”  However, few never really think about what the patching of a system or application really does; or what is the best approach to take when getting ready to deploy these updates.  Just like many things we as IT professionals do, it is not the action that counts, but rather the prep work that makes a successful deployment, or a dreaded failure.

 

So how can you put together a patch management solutions that will not only give back your weekends, be able to provide the security needed for your organization, but also prevent that dreaded failure?

 

Let’s take a look at the key areas to achieve these goals.

 

I. System Scope

II. Development / Testing Environment

III. Change Management / Change Control

IV. Patch / Update Deployment

V. System Monitoring & Vulnerability Scanning

 

One might think the process begins on the update release date.  This is not the case by any means.  The work really begins even before the very first system is built and configured.

 

I. System Scope
The system administrator, manager, director, and chief information officer need to sit down and outline the purpose of the systems.  This will include the system role as well as the baseline the system will take when first built.


What will the role be of the system?
What applications will run on each system?
What systems are mission critical, mission sensitive, business critical, general operations, and testing?


Once the system scope has been developed and documented you can move on to the next step in the process.  Until you have the scope of what the systems are going to be doing and what category that system resides in, you cannot truly have a successful patch management process.

 

II. Development / Testing Environment

When building your test environment consider 3 key characteristics – is the environment flexibility, dynamic, and accurate?  Flexibility refers to the ability of the environment to serve multiple roles.  Dynamic in the sense that the system can be changed to a different role and do so quickly. You have the flexibility of a system that can serve more than one role but if you have to spend days building it then you are not very dynamic.  Most importantly, in my opinion, is the accuracy of the system as it equates to the production system.  If it does not have the same baseline and installed applications you will not be able to test the patches properly to find issues before rolling it out into the production environment.

 

How can we achieve all three requirements?

 

Virtualize your test environment; create a clone of the physical production system.  Then prior to installing the patch take a snapshot; this way if the patch hoses something up you can simply restore from the snapshot and continue working on solutions.  Virtualization has become so vital in a testing environment; the cost verse reward is incredible.

 

III. Change Management / Change Control

Now incorporate change management.  The key is to make sure you have a process in place prior to installing the patch and even more important is the back out plan for that patch if something goes wrong.

 

IV. Patch / Update Deployment

Let’s move to actually installing the patch or application update.  During this step you need to take good notes, note anything that was required as a dependency which was not documented by the vendor.  While taking these notes you need to observe adverse reactions to the patch which may affect the production environment.

 

The rollout is the last piece of the patch deployment process.  Prior to this step you should already have a confirmed maintenance and patching schedule which will be incorporate not only what time and day of week you deploy patches but also the order of the systems based on system and security scope.


However, the process is yet to be completed!

 

V. System Monitoring & Vulnerability Scanning

Finally you need to run the systems and monitor the operation, as well as use a vulnerability scanner such as OpenVAS or Nessus to verify the systems received the patch or update successfully as well as actually fixed the issue the patch or update was supposed to resolve.


This was a brief example on the steps required in order to build a successful program.  However, understand that every organization will be different and so long as you have the methodology and support you can have a successful patch management program.