I recently got involved with a little DevOps integration project, using SolarWinds NPM and Plexxi Control's Data Service Engine.  The goal was to turn SolarWinds monitoring data into something Plexxi Control can use to modify a Plexxi network in true SDN fashion.

 

This integration took on a life of its own and evolved throughout the year and we ended up with a module that can use Twitter! to send out Events and Alarms from SolarWinds.

Before you declare that we are crazy Tweeting our Events this is more of a story of a modern DevOps integration and the use of Twitter is just an example of a mechanism to pub / sub data between data sources.

 

Earlier this year, Plexxi started working with SolarWinds NPM REST API's as we wanted to harvest network node data, but we soon realized that the most complex part of the integration was managing adds/moves and changes. Polling the entire list of nodes every few seconds to check if something changed does not scale. So we created an integration with SolarWinds Event / Alarm feed.

 

Plexxi had previously developed an integration tool named the Data Services Engine that helps us to modularize these integrations, you can find out about it here

The DSE is an in-memory message bus tool that makes data available for products to consume using their standard API interfaces.

 

Using the DSE, we poll the SolarWinds Event / Alarm feed every few seconds to see if there are new Events. We then process any Event that is interesting, and publish the data in a well known format on a well known channel using the DSE.

 

Here is what we published. Its just a bit of JSON, using the SWISS db field names, and you can process this easily in Python.

 

{'SolarWindsEventID':{5629: {'date': '2013-10-30T21:12:22.973', 'eventType': 56, 'message': 'Group xyz was created.', 'netObjectID': 96}}}

 

 

I don't even really need to explain these fields, but essentially this means according to SolarWindsEvent 5629 on 30th Oct 13 at 21:12, a new Group 'xyz' was created, and its group ID is 96.

 

At this point in the integration you can probably see that this is fairly generic SolarWinds data, but a similar format could be applied to any Event monitoring product, its just publishing Event data. So you could easily combine all your Event sources and publish them in a similar way.

 

Now, Publishing data like this is only useful if you have something that will Subscribe to that data, so we created a DSE channel that Plexxi could read and our integration was done (you can see this in action in our SDN central demo with SolarWinds on December 6th, http://t.co/yLZBomgFAt).

 

Later on, prompted by an article by @mbushong (http://bit.ly/184PL2H) talking about events with Twitter (I'm not really sure if he was joking)  I decided as a thought experiment to see how easy it would be using the DSE to get this data published into Twitter!

 

The way of the world for API development these days is that for most modern systems you can find a nice Python library, Twitter has a few of said modules (all open source). I grabbed one at random, and created a little DSE module that talks to Twitter.

 

The base module is literally 35 lines of code, here is a snipit, where we process new events and post an Update to the Twitter channel

 

               # gets a list of new events
               changes = [x for x in newEvents if x not in self.config['events']]
               for change in changes:
                   newEntry = {}
                   newEntry[change] = newEvents[change]
                   # store current event list
                   self.config['events'].update(newEntry)
                   try:
                        # publish on Twiiter
                        status = self.api.PostUpdate(str(newEntry))

 

 

Now what happens is that each time SolarWinds generates an Event, the SolarWinds DSE channel we created reads them, filters them out and publishes the service affecting ones (what is defined as service affecting is housed in the SolarWinds channel we created, and totally customizable).

 

Running in parallel the Twitter channel reads those Events and publishes any new ones into my temp Twitter account ( please don't follow me, its a little verbose).

Screen Shot 2013-11-15 at 12.04.10 PM.png

 

There is a lot of extensibility here also, we started off using just Tweets to our followers, but that means everyone gets everything, so we changed it to use Twitter Direct Messages, so that our subscribers can request certain events. For example a DM with Start:56, would result in our feed doing a DM back to that subscriber each time there was a Type 56 Event.

 

Twitter is fairly secure and you can hide all this in the Direct Messages, its available everywhere, it scales, and its free. I mean this is crazy but we have a highly scalable Event exchange engine that anyone can use.

 

Now in your environment, all you need is to get your application to process the Twitter events from your account (don't use mine), and you have a perfectly workable Dev Ops operational Event model for your organization. If your application has a good Python API module (which it should), then you may find the integration is as easy as I found.