cancel
Showing results for 
Search instead for 
Did you mean: 

Hybrid IT: Complex Simplicity?

Level 13

Raise your hand if you have witnessed firsthand rogue or shadow IT. This is when biz, dev, or marketing goes directly to cloud service providers for infrastructure services instead of going through your IT organization. Let's call this Rogue Wars.

Recently, I was talking to a friend in the industry about just such a situation. They were frustrated with non-IT teams, especially marketing and web operations, procuring services from other people’s servers. These rogue operators were accessing public cloud service providers to obtain infrastructure services for their mobile and web app development teams. My friend's biggest complaint was that his team was still responsible for supporting all aspect of ops, including performance optimization, troubleshooting, and remediation, even though they had zero purviews or access into the rogue IT services.

They were challenged by the cloud’s promise of simplified self-service. The fact that it's readily available, agile, and scalable was killing them softly with complexities that their IT processes were ill prepared for. For example, the non-IT teams did not leverage proper protocol to retire those self-service virtual machines (VMs) and infrastructure resources that form the application stack.That meant that they were paying for resources that no longer did work for the organization. Tickets were also being opened for slow application performance, but the IT teams had zero visibility to the public cloud resources. For this reason, they could only let the developers know that the issue was not within the purview of internal IT. Unfortunately, they were handed the responsibility of resolving the performance issue.

This is how the easy button of cloud services is making IT organizations feel the complex burn. Please share your stories of rogue/shadow IT in the comments below. How did you overcome it, or are you still cleaning up the mess?

31 Comments
Jfrazier
Level 18

raising hand....this actually existed before the term shadow IT or Rogue IT existed.

gfsutherland
Level 14

Both hands raised!!!!

Responsibility for performance issues is only the beginning, then they want total support!

Jfrazier
Level 18

then they are confused when there is a breach or loss of data as a result of their Rogue operations or no provisions for SLA's or performance built into the contract.

ecklerwr1
Level 19

Sounds like a recipe for VM Sprawl for sure!

jhandberg
Level 13

In my last job we had the situation once where a department purchased a cloud service but didn't tell IT until something went wrong.  Then of course we were expected to fix it.  We had no access at all to start with. Then we found out this service was storing non-public (aka personal) student information and we had no idea of the security of that, let alone any impact if it was compromised.  It turns out the database they used was unencrypted and vulnerable but fortunately there was no breach.

The original complaint that got us involved was slow performance they blamed on the network.  Fortunately, our NPM showed it was either external to the campus (IPS level) or slow server.  It turned out it was both, but at different times.  ISP network issues and an under performing server in the cloud service because our IT wasn't involved in helping with the overall service specifications.

It took months to convince upper management that we needed to pull that sensitive data back on campus for security reasons and a bit longer to get the convoluted authentication it require working with our campus system.  Then they abandoned the application about a month after that.  Oh, joy.

tallyrich
Level 15

Yup, I saw that the second that it was announced that we were using virtual machines. For some reason (probably our own fault) people get the impression that virtual means free. And while certain aspects are "free" the maintenance an upkeep certainly are not free. How many times have I heard that dreaded phrase "Just spin up another server." One place I worked had 100 physical servers at the start of the VM migration and ended up with nearly 400 at the end of the project. Yes, every application now had Dev, Test, Model Office and Production environments and they didn't all have them prior, but . . .

shuckyshark
Level 13

Not here...everyone knows my take on clouds...

rschroeder
Level 21

Folks know my opinion of the cloud and bypass me if possible. 

But equally as bad is when a leader or department head is exposed to direct advertising by a vendor specializing in that leader's departmental area.  Someone perceives a need, gets private budget to buy whatever the vendor is selling--WITHOUT GETTING IT APPROVAL/REVIEW/SUPPORT!.

The new technology ends up not working and someone assigns our teams tickets to "fix" what is incompatible with our environment.  For example, some medical device manufacturers built "networked" hardware that has no subnet mask and no gateway settings.  Their "Enterprise-compatible" equipment is good for a flat network only--no VLAN Isolation /Layer 3 routing / WAN service is possible.  And then the vendors have the nerve to say they can't change their systems because they're FDA approved as they are, and would take many years for review and re-approval.  Hmph!

It's enough to get one's goat.

pastedImage_1.png

The Screaming Sheep (Original Upload) - YouTube

jhandberg
Level 13

I ran into that scenario as well.  It was actually fairly common for a department to get a grant for some new toy a vendor exposed them to, but no ongoing budget for training, maintenance, or support.  "Oh, that is IT's problem."

rschroeder
Level 21

pastedImage_0.png

gfsutherland
Level 14

love it!!!! rschroeder

bobmarley
Level 15

Or marketing bloating out your ecom site with feeds from google, facebook, etc that occasionally get crawling slow causing the entire site to slow down.

'Well, that looked nice and pretty, but we didn't make any sales....."

The WPM waterfall chart was good at finding these.

goodzhere
Level 14

Unfortunately, this isn't going away anytime soon

rharland2012
Level 15

It's never going away.

ecklerwr1
Level 19

No subnet mask and gateway?  Geesh that's bad.

shuckyshark
Level 13

a bit creepy...

mtgilmore1
Level 13

Here you fix it -----

Image result for cut cable photo

Jfrazier
Level 18

at least those are color coded....

vinay.by
Level 16

Isnt the case with us

shuckyshark
Level 13

yeah, but what if you're colour blind?

tallyrich
Level 15

I worked for a company that did automatic doors many years ago. We had an opportunity to install some doors at the ATT IC plant in north Orlando Florida. The tech that we sent had done the work for years and could wire a door in about 1/4th the time of any other tech. On install day he struggled for several hours (rather than a couple of minutes) to make the first door work. An engineer came by and asked "You having trouble?" - one of those "here's your sign questions." He then said "these special lights may be causing you problems - and shined a flashlight on the work the tech was doing. Sure enough the special lighting that they use around Clean Rooms was causing all the colors to look different than they do in regular lighting. With that the tech rewired the door - in about 3 minutes - and it worked perfectly fine.

Goes to show sometimes we are color blind - or just blind - when we think we can see clearly. Never be afraid to ask questions, never be afraid to admit you need help and never assume what you see is what actually is.

shuckyshark
Level 13

LoL...

funny, we just (like an hour ago) had someone call service centre - "I can access my email, but nothing else". We had moved some cubicles, and I had our vendor come in to re-test our cables to ensure they meet the specs as per Belden certification.  They had plugged one cable back into itself, creating a loop.

deverts
Level 14

Anyone out there work in a manufacturing environment? How many of your plant engineers think they know IT? "Oh, I'm an engineer, I can do your job!" Then they call a vendor to build a networked solution (ethernet and SCADA), and when the job is done, hand it off and say, "here, maintain this."

So, I have both hands raised! And don't get me started on the sales and marketing teams that go out and buy GoDaddy domains for promotions.

D

rharland2012
Level 15

And let's not forget the 'proactive' sales and marketing teams that buy consumer Dropbox accounts...and then leave the company's proprietary and private data and designs just, you know, sitting out there.

MMMMM...good times.

rschroeder
Level 21

In one job interview, I had to pass a color test to prove I wasn't color blind.  If I had not passed it, I would have been eligible for other positions, but not the one to which I was applying.

Where color recognition is required, folks shouldn't have to fight it when they're color blind, and they shouldn't be made to feel "different" or ostracized.  Similarly, their safety shouldn't be compromised by equipment that is color dependent.

They must be allowed to do other useful and valuable work that doesn't require color sense, or the company must accommodate them by providing different item-recognition that doesn't use colors.  That may be inconvenient, or even extremely expensive, to achieve.

An example:  there are numbers or characters or figures in each of these images.  Successfully identifying every one of them--100%--was required for the job I mentioned:

pastedImage_0.png

tinmann0715
Level 16

My company we have a corporate center and state distribution centers ("House") that run semi-autonomously (because of state liquor laws). However, we have a One-IT whereas a tech in Delaware can help a user in Arizona. Nonetheless, I've been here 7 years now and each House will setup their little own little IT operation and create their own processes and dependencies. This goes directly against the One IT initiative and the investment in the redundant corporate datacenter and infrastructure.

   It's all political, territorial, and messy. Worst of all, it's expensive and dangerous... and I can't get leadership to back me.

deverts
Level 14

Oh, HAHAHAHA, we just started blocking that with content filtering...talk about ANGRY people!

D

shuckyshark
Level 13

I block everything by default...then you have to provide a case to open up access to something.

rharland2012
Level 15

Oh, wow....LOL!

"But that's how I get quotes to my customers!!"

"By using a personal dropbox account tied to your Yahoo! email?"

"......"

rharland2012
Level 15

That's the best practice right there...

mcam
Level 14

Hands in air also

but thankfully - sorry that isn't an application that is supported by our IT organization, however we can forward you to the user in your dept that supports you.

Even better will be when our DLP implementation starts and all those processes that send files offsite suddenly stop working.

About the Author
Mo Bacon Mo Shakin' Mo Money Makin'! vHead Geek. Inventor. So Say SMEs. vExpert. Cisco Champion. Child please. The separation is in the preparation.