1 2 Previous Next

Geek Speak

29 Posts authored by: cxi

Welcome to the last in this series of posts about machine learning, artificial intelligence, and so many other topical points of discussion. If this is your first time here, feel free to check out the previous articles:

 

Does Data Have Ethics? Data Ethics Issues and Machine Learning

Machine Learning and Destroying the Library of Alexandria

Why Businesses Don't Want Machine Learning or Artificial Intelligence

Can Tesla Cure the 10th Leading Cause of Death in the World With Autonomous Vehicles?

 

I know you might be asking yourself, how can I follow those gems up with something you'll be able to ingest, take in, and reflect on thoughtfully?

 

Let's take this opportunity to celebrate. (Thank goodness!) And in that celebration, I'm going to lay some machine artificial intelligence learning (MAIL) on you, so you can get started, play around, and whatever else you’d like.

 

One of the big problems of dealing with the amorphous concept of something like ML or AI is, it's so big, you don't even know where to start.

 

Image: CXI was on vacation recently in Bermuda

 

It's like trying to classify waves and clouds. If you start small, you can get somewhere, but imagine trying to understand everything in this image...

 

So instead of trying to figure out how to build something from scratch, start with the work others have already done, so you can play with things, understand things, and get started without all the brain damage.

 

Let's get started with emotion and sentiment analysis with DeepMoji. (Artificial Emotional Intelligence). It's an interesting project taking words and using AI to convert them into emotions and emojis. (Emojitions?)

 

Image: https://deepmoji.mit.edu/

 

On the surface it may not seem too sophisticated, but it's a start and with the DeepMoji project up on GitHub, you can get started with understanding how it works today.

 

The next one, which lets you immediately see things “working,” is the Move Mirror AI Experiment.

 

Move Mirror AI Experiment

Image: AI Experiments: Move Mirror

 

This fun little project lets you use your own camera and movements to match your body movement to an image of the same movement. I've seen the principle of this used in drastic other ways (to make it look like someone is talking or moving lips when you talk, or other funny and equally scary tactics).

 

Is this project going to be a game-changer for you? Probably not. Is it a little fun? Definitely!

 

And that brings us to the next piece of this puzzle, which is the Experiments with Google page. This is quite cool because they have AI experiments, AR experiments, voice experiments, and so much more.

 

But after you're done playing around and interacting with these game-changing models, samples, and examples, you can start getting your hands dirty.

 

I'd immediately encourage you to look at the TensorFlow Hub (a collection of reusable machine learning modules). It's a great resource to learn, interact with, and ingest work others have done, so you don't have to start from scratch.

 

And after you’re ready to take things seriously and even start competing with others in making machine learning and AI even better, head over to Kaggle.

 

Jupyter Notebooks

 

 

You can also use Jupyter notebooks to get started on building, testing, and so much more.

 

And lastly, if you have some real business things you need to work on, and don't have the time, inclination, or interest in learning quite so much upfront, check out AutoML from Google. They take a lot of the effort out of it (for a small fee of course). [Interesting little fact, some people have competed in Kaggle competitions using AutoML and trained for as little as an hour and got the same or better results of some people that trained for a month or more.]

 

And this is just scratching the surface of some of the resources to dig into. A few other resources I would highly encourage investigating are:

  • YouTube! Yes, YouTube has some great ML/AI-ers who produce some amazing content, some of which is quite ridiculous.
  • Code Bullet
  • On Twitter, if you're not following these folks already, it's imperative for your MAIL future to do so.

@tkasasagi (Who I already mentioned in Machine Learning and Destroying the Library of Alexandria)

@dynamicwebpaige (Amazing, smart, rock star who works at Google AI with TensorFlow)

@fchollet (Creator of Keras, neural networks library, and author of Deep Learning with Python, not to mention a great resource and person)

@suzatweet (MLT Lead, Minds and Machine Learning, and some other awesome things.)

@_aiCarly (Technical powerhouse and working at an AI company.)

@cxi (Hey—that's me! I cannot publicly discuss my own ML/AI projects, but I talk about and share a lot of other people's work.)

 

The list could go on forever, but I have to play favorites, mainly because these people contribute to the community, produce, and help others benefit and learn. Just as this community allows us to grow stronger and better together.

 

I hope you’ve enjoyed this journey pulling back the curtain and scratching the surface of the areas of machine learning and artificial intelligence. I'll continue to stay deep on this (things I can and cannot talk about) and continue to grow and contribute.

 

Don't hesitate to share some of your AI and ML favorites, projects you really enjoy, or projects you're working on.

Are you excited for this post? I certainly know I am!

 

If this is the first article you're seeing of mine and you haven't read my prior article, "Why Businesses Don't Want Machine Learning or Artificial Intelligence," I advise you go back and read it as a primer for some of the things I'm about to share here. You might get “edumicated,” or you might laugh. Either way, welcome.

 

Well, officially hello returning readers. And welcome to those who found their way here due to the power of SEO and the media jumping all over any of the applicable buzzwords in here.

 

The future is now. We’re literally living in the future.

 

Tesla Self-Driving, look ma no hands!

Image: Tesla/YouTube

 

That's the word from the press rags if you've seen the video of Tesla running on full autopilot and doing a complicated series of commute/drives cited in the article, "New Video Shows Tesla's Self-Driving Technology at Work." And you would be right. Indeed, the future IS now. Well, kind of. I mean it's already a few seconds from when I wrote the last few words, so I'm clearly traveling through time...

 

But technology and novelty like driving in traffic are a little more nuanced than this.

 

"But I want this technology right now. Look, it works. Stop your naysaying. You hate Tesla, blah blah blah, and so on."

 

That feels very much like Apple/Android fanboy or fan-hate when someone says anything negative about a thing they want/like/love. Nobody wants this more than me. (Well, except for the robots ready to kill us.)

 

Are There Really Self-Driving Teslas?

 

You might be surprised to know that Tesla has advanced significantly in the past few years. I know, imagine that—so much evolution! But just as we reward our robot overlords for stopping at a stop sign, they're only as good as the information we feed them.

 

We can put the mistakes of 2016 behind us with tragedies like this: "Tesla self-driving car fails to detect truck in a fatal crash."

 

Fortunately, Tesla continues to improve and get better and we'll be ready to be fully autonomous with self-driving vehicles roaming the roads without flaw or problem by 2019. (Swirly lines, and flashback to just a few months into 2019: Tesla didn't fix an Autopilot problem for three years, and now another person is dead.)

 

Is the Tesla Autopilot Safe?

 

Around this time, as I was continuing my study, research, and analysis of this and problems like it, I came across the findings of @greentheonly.

 

Truck's are misunderstood and prefer to be identified as Overpasses

Image: https://twitter.com/greentheonly/status/1105996445251977217

 

And rightly so, we can call this an anomaly. This doesn't happen that frequently. It's not a big deal, except for when it does happen. Not only just when, but the fact that it does happen… whether it's seeing the undercarriage of a truck and interpreting it as an overpass and thus you can "safely pass" under it, shearing the top off of the cab, or seeing a truck to the side of you and interprets the space beneath the truck as a safe “lane” to change into.

 

But hey, that's Autopilot. We're not going to use that anymore until it's solid, refined, and safe. Then the AI and ML can't kill me. I'll be making all the decisions.

 

 

ELDA Taking Control!ELDA may be the death of someone!

Image: https://twitter.com/KevinTesla3/status/1134554915941101569

 

If you recall in the last article, I mentioned the correlation of robots, Jamba Juice, and plasma pumps. Do you ever wonder why Boston Dynamics doesn't have robot police officers like the ED-209 working on major metro streets, providing additional support akin to RoboCop? (I mean, other than the fact that they're barely allowed to use machine learning and computer vision. But I digress.)

 

It’s because they're not ready yet. They don't have things fully baked. They need a better handle on the number of “faults” that can occur, not to mention liability.

 

Is Autonomous Driving Safe?

 

Does this mean, though, that we should stop where we are and no one should use any kind of autonomous driving function? (Well, partially...) The truth is, there are, on average, 1.35 million road traffic deaths each year. Yes, that figure is worldwide, but that figure is also insanely staggering. If we had autonomous vehicles, we could greatly diminish the number of accidents we experience on the roads, which could bring those death tolls down significantly.

 

And someday, we will get there. The vehicles’ intelligence is getting better every day. They make mistakes, sometimes not so bad—"Oh, I didn't detect a goose on the road as an object/living thing and ended up running it over." Or, "The road was damaged in an area, so we didn't detect that was a changing lane/crossing lane/fill-in-the-blank of something else."

 

The greatest strength of autonomous vehicles like Tesla, Waymo, and others is their telemetry. But their greatest weakness is their reliance solely on some points of telemetry.

 

Can Self-Driving Cars Ever Be Safe?

 

In present-day 2019, we rely on vehicles with eight cameras (hey, that's more eyes than us humans have!), some LIDAR data, and a wealth of knowledge of what we should do in conditions on roadways. Some complaints I've shared with various engineers of some of these firms are the limitations of these characteristics, mainly that the cameras are fixed, unlike our eyes.

 

Landslide!

Video: https://youtu.be/SwfKeFA9iEo?t=22

 

So, if we should encounter a rockslide, a landslide, something falling from above (tree, plane, meteorite, car-sized piece of mountain, etc.) we'll be at the will of the vehicle and its ability to identify and detect this. This won't be that big of a deal, though. We'll encounter a few deaths here or there, it'll make the press, and they'll quietly cover it up or claim to fix it in the next bugfix released over the air to the vehicles (unlike the aforementioned problem that went unsolved for three years).

 

The second-biggest problem we face is, just like us, these vehicles are (for the most part) working in a vacuum. A good and proper future of self-driving autonomy will involve the vehicles communicating with each other, street lights, traffic cameras, environmental data, and telemetry from towers, roads, and other sensors. Rerouting traffic around issues will become commonplace. When an ambulance is rushing someone to a hospital, it can clear the roadways in favor of emergency vehicles. Imagine if buses ran on the roads efficiently. The same could be true of vehicles.

 

That's not a 2020 vision. It’s maybe a 2035 or 2050 vision in some cities. But this is a future that can be well seen ahead of us.

 

The Future of Tesla and Self-Driving Vehicles

 

It may seem like I’m critical of Tesla and their Autopilot programs. That’s because I am. I see them jumping before they crawl. I've seen deaths rack up, and I've seen many VERY close calls. It's all in the effort of better learning and training. But it's on the backs of consumers and on the graves of the end users who've been made to believe these vehicles are tomorrow's self-drivers. In reality, they’re in an Alpha state with the sheer limited amount of telemetry available.

 

Will I use Autopilot? Yeah, probably... and definitely in an effort of discovering and troubleshooting problems because I'm the kind of geek who likes to understand things. I don't have a Tesla yet, but that's only a matter of time.

 

So, I obviously cannot tell you what to do, with your space-age vehicle driving you fast forward into the future, but I will advise you to be cautious. I've had to turn ELDA off on my Chevy Bolt as it has been steering me into traffic, and that effectively has little to nothing I would consider "intelligent" in the grand scheme of things.

 

At the start of this article, I asked if you were as excited as I was. I'm not going to ask if you're as terrified as I am! I will ask you to be cautious. Cautious as a driver, cautious as a road-warrior. The future is close, so let's see you live to see it with the rest of us. Because I look forward to a day where the number 10 cause of death worldwide is a thing of the past.

 

Thank you and be safe out there!

Did you come here hoping to read a summary of the past 5+ years of my research on self-driving, autonomous vehicles, Tesla, and TNC businesses?

Well, you’re in luck… that’s my next post.

 

This post helps make that post far more comprehensible. So here we lay the foundation, with fun, humor, and excitement. (Mild disclaimer for those who may have heard me speak on this topic before or listened to this story shared in talks and presentations. I hope you have fun all the same.)

 

I trust you’re all relatively smart and capable humans with a wide knowledge, depth, and breadth of the world, so for the following image, I want you to take a step back from your machine. In fact, I want you to look at it from as far as you possibly can, without putting a wall between you and the following image.

 

THIS IS A HOT DOG

 

OK, have you looked at the image? Do you know what it is? If you're not sure, and perhaps have a child or aging grandparents, I encourage you to ask them.

 

Did you get hot dog? OK, cool. We’re all on the same page here.

 

Now as a mild disclaimer to my prior disclaimer—I’m familiar with the television series Silicon Valley. I don’t watch it, but I know they had a similar use-case in that environment. This is not that story, but just as when I present on computer vision being used for active facial recognition by the Chinese government to profile the Muslim minority in China and people say, "Oh, you mean like Black Mirror..." I mean “like that,” but I mean it's real and not just "TV magic."

 

Last year in April (April 28, 2018), this innocuous meat enclosure was run through the paces on our top four favorite cloud and AI/ML platforms, giving us all deep insight into how machines work, how they think, and what kinds of things we like to eat for lunch on a random day—the 4th day of July, perhaps. These are our findings.

 

I think we can comfortably say Google is the leader in machine learning and artificial intelligence. You can disagree with that, but you'd likely be wrong. Consider one of the leading machine learning platforms Tensor Flow (TF), which is an open-source project by Google. TF was recently released in 2.0 as an Alpha, so you might think "Yeah, immature product is more like it." But when you peel back that onion a bit and realize Google has been using it internally for 20 years to power search and other internal products, you might feel it more appropriate to call it version 25.0, but I digress.

 

What does Google think a hotdog is?

Image: https://twitter.com/cloud_opinion/status/989691222590550016

 

As you can see, if we ask Google, "Hey, what does that look like to you?" it seems pretty spot on. It doesn't come right out and directly say, "that’s a hot dog," but we can't exactly argue with any of its findings. (I'll let you guys argue over eating hot dogs for breakfast.) So, we're in good hands. Google is invited to MY 4th of July party!

 

But seriously, who cares what Google thinks? Who even uses Google anyway? According to the 2018 cloud figures, Microsoft is actually the leader in cloud revenue, so I'm not likely to even use Google. Instead, I'm a Microsoft-y through and through. I bleed Azure! So, forget Google. I only care about what my platform of choice believes in, because that's what I'll be using.

 

What Microsoft KNOWS a hotdog is!

Image: https://twitter.com/cloud_opinion/status/989691222590550016

 

Let’s peel back the chopped white onion and dig into the tags a bit. Microsoft has us pegged pretty heavily with its confidences, and I can't say I agree more. With 0.95 confidence, Microsoft can unequivocally and definitively say, without a doubt, I am certain:

 

This is a carrot.

 

We're done here. We don't need to say any more on this matter. And to think we Americans are only weeks away from celebrating the 4th of July. So, throw some carrots on the BBQ and get ready, because I know how I'll be celebrating!

 

Perhaps that's too much hyperbole. It’s also 0.94 confident that it's "hot" (which brings me back to my Paris Hilton days, and I'm sure those are memories you've all worked so hard to block out). But... 0.66 gives us relative certainty that if it wasn't just a carrot, it's a carrot stick. I mean, obviously it's a nicely cut, shaved, and cleaned-off carrot stick. Throw it on the BBQ!

 

OK, but just as we couldn't figure out what color “the dress” was, Microsoft knows what’s up, at 0.58 confidence. It's orange. Hey, it got something right. I mean, I figure if 0.58 were leveraged as a percentage instead of (1, 0, -1) it would be some kind of derivative of 0.42 and it would think it’s purple. (These are the jokes, people.)

 

But, if I leave you with nothing more than maybe, just MAYBE, if it's not a carrot and definitely NOT a carrot stick... it's probably bread, coming in at 0.41 confidence. (Which is cute and rings very true of The Neural Net Dreams of Sheep. I'm sure the same is true of bread, too. Even the machines know we want carbs.)

 

But who really cares about Microsoft, anyway? I mean, I'm an AWS junkie, so I use Amazon as MY platform of choice. Amazon was the platform of choice of many police departments to leverage its facial recognition technology to profile criminals, acts, and actions to do better policing and protect us better. (There are so many links on this kind of thing being used; picking just one was difficult. Feel free to read up on how police are trying to use this, and how some states and cities are trying to ban it.)

 

Obviously, Amazon is the only choice. But before we dig into that, I want to ask you a question. Do you like smoothies? I do. My favorite drink is the Açai Super Anti-Oxidant from Jamba Juice. So, my fellow THWACKers, I have a business opportunity for you here today.

 

Between you, me, and Amazon, I think we can build a business to put smoothie stores out of business, and we can do that through the power of robots, artificial intelligence, and machine learning powered by Amazon. Who's with me? Are you as excited as I am?

 

Amazon prefers you to DRINK your Hotdogs

Image: https://twitter.com/cloud_opinion/status/989691222590550016

 

The first thing I'll order will be a sweet caramel dessert smoothie. It will be an amazing confectionery that will put your local smoothie shop out of business by the end of the week.

 

At this point, you might be asking yourself, "OK, this has been hilarious, but wasn't the topic something about businesses? Did you bury the lede?”

 

So, I'll often ask this question, usually of engineers: do businesses want machine learning? And they'll often say yes. But the truth is really, WE want machine learning, but businesses want machine knowledge.

 

It may not seem so important in the context of something silly like a hot dog, but, when applied at scale and in diverse situations, things can get very grave. The default dystopian future or “realistic” future I lean towards is plasma pumps in a hospital. Imagine a fully machine controlled and AI-leveraged device like a plasma pump. It picks out your blood type based on your chart or perhaps it pricks your finger, figures out what blood it should give you, and starts to administer it. Not an unrealistic future, to be honest. Now, what if it was accurate 99.99999% of the time? Pretty awesome, right? But let's say it was as accurate as Google was with a hot dog, at 98% confidence. A business can hardly accept the liability that 99.9999999% might give. Drop that down to 98%, or the more realistic 90-95%, and that is literal deaths on our hands. 

 

Yes, businesses don't really want machine learning. They say they do because it's popular and buzzword-y, but when lives are potentially on the line, tiny mistakes come with a cost. And those mistakes add up, which can affect the confidence the market can tolerate. But hey, how much IS a life really worth? If you can give up your life to make a device, system, or database run by machine learning/AI better for the next person, versus, oh I don't know, maybe training your models better and a little longer—is it worth it?

 

There will come a point or many points where there are acceptable tolerances, and we'll see those tolerances and accept them for what they are at a certain point because, "It doesn't affect me. That's someone else's problem." Frankly, the accuracy of machine learning has skyrocketed in the past few years alone. That compounded with better, faster, smarter, and smaller TPUs (tensor processing units) means we truly are in a next-generation era in the kinds of things we can do. 

 

Google Edge TPU on a US Penny

Image: Google unveils tiny new AI chips for on-device machine learning - The Verge

 

Yes, indeed, the future will be here before we know it. But mistakes can come with dire costs, and business mistakes will cost in so many more ways because we "trust" businesses to do the better or the right thing.

 

"Ooh, ooh! You said four cloud providers. Where's IBM Watson?"

 

Yes, you're right. I wasn't going to forget IBM Watson. After all, they're the businessman's business cloud. No one ever got fired for buying IBM, blah blah blah, so on and so forth.

 

I always include this for good measure. No, not because it’s better than Google, Microsoft, or Amazon, but because of how funny it is.

 

IBM is like a real business cloud AI ML!

Image: https://twitter.com/cloud_opinion/status/989762287819935744

 

I’ll give IBM one thing—they're confident in the color of a hot dog, and really, what more can we ask for?

 

Hopefully, this was helpful, informative, and funny (I was really angling on funny). But seriously, you should have a better foundation for the constantly “learning” nature of the technology we work with. Consider that machines have a large knowledge base of information, and what they can learn in such a short time to make determinations about what a particular object is. Then also consider if you have young children. Send them into the kitchen and say, "Bring me a hot dog," and you're far more likely to get a hot dog back than you are to get a carrot stick (or a sweet caramel dessert). The point being, the machines that learn and the "artificial" intelligence we expect them to work with and operate from have the approximate intelligence of a 2- or 3-year-old.

 

They will get better, especially as we throw more resources at them (models, power, TPUs, CPU, and so forth) but they're not there yet. If you liked this, you'll equally enjoy the next article (or be terrified when we get to the actual subject matter).

 

Are you excited? I know I am. Let us know in the comments below what you thought. It's a nice day here in Portland, so I'm going to go enjoy a nice hot dog smoothie. 

Hey THWACKers! Welcome back for week 2 in machine learning (ML). In my last post, Does Data Have Ethics? Data Ethic Issues and Machine Learning, you may have noticed I mentioned "evil" four times, but also mentioned "good" four times. Well, you're in luck. After all that talk about evil and ethics, I want to share with you some good that's been happening in the world.

 

But who can talk about goodness, without mentioning the dark circumstances “the machines” don't want you to know about?

 

For those who aren't familiar, the Library of Alexandria was a place of wonder, a holder of so much knowledge, documentation, and so much more. But what happened? It was DESTROYED.

 

In preparation for this topic, and because I wanted to mention some very specific library destructions over the years, I found this great source on Wikipedia so you can see just how much of our history has been lost.

 

Some notable events were:

  • ALL the artifacts, libraries, and more destroyed by ISIS
  • The 200+ years’ worth of artifacts, documents, and antiquities destroyed in the National Museum of Brazil fire
  • The very recent fire at Notre Dame, where the fires are hardly even out while this topic smolders within me
  • The Comet Disaster that breaks off and destroys this sleepy Japanese town every 1,200 years (OK, so this one’s from an anime movie, but natural disasters are disasters all the same.)

 

 

Image: Screen capture from the movie “Your Name” (Original title: Kimi no na wa) 50:16

https://www.imdb.com/title/tt5311514/

 

But how can machine learning help with this? Because I'm sure you all think “the machines” will cause the next level of catastrophe and destruction, right?

 

I’d like to introduce you to someone I'm honored to know and whose work has inspired growth, change, and not only can be used to preserve the past, but will enlighten the future.

This inspiration is Tkasasagi, who has been setting the ML world on fire with natural language processing and evolutionary changes to the translation of Ancient, Edo era, and cursive Hiragana.

 

To give you a sense of the significance of this, there's a quote from last June, "If all Japanese literature and history researchers in the whole country help transcribing all pre-modern books in Japan, it will only take us 2000 years per persons to finish it."

 

Let's put that into perspective—there are countless tomes of knowledge, learning, information, education, and so much more that documents the history and growth of Japanese culture and nation. An island nation in a region with some of the most active volcanoes and frequent earthquakes in the world. It's only a matter of time before more of this information suffers from life's natural disasters and gets lost to the winds of time. But what can be done about this? How can this be preserved? That's exactly the exciting piece that I'm so happy to share with you.

 

  Here in the first epoch of this transcription project, machine learning does an OK job… but is it a complete job? Not even in the least. But fast forward to a few weeks later, and the results are staggering and impressive (even if nowhere near complete). 

 

Images: https://twitter.com/tkasasagi/status/1036094001101692928

Now some of you may feel (justifiably so) that this is an impressive growth in such a short amount of time, and I would agree.  Not to mention the model is working with >99% accuracy at this point which is impressive in its own right.

Image: https://twitter.com/tkasasagi/status/1115862769612599296

 

But the story doesn't end there—it continues literally day by day. (Feel free to follow Tkasasagi and learn about these adventures in real time.)

 

Every day, every little advancement in technologies like this through natural language processing (NLP), computer vision (CV), and convolutional neural networks (CNN) continue to grow the entire industry as a whole, where you and I, as consumers of this technology, will eventually find our everyday activities to be easier, and one day will just be seen as commonplace. For example, how many of you are using, or have used, the image language translate function of Google Translate to help display another language, or used WeChat's natural conversion of Chinese into English or vice-versa?

 

We are leap-years beyond where we were just a few years ago, and every day, it gets better, and efforts like these just continue to make things better, and better, and better.

 

How was that for using our machines for good and not the darkest of evils? I'm excited—aren't you?

Hello THWACKers long time no chat! Welcome to part one in a five-part series on machine learning and artificial intelligence. I figured what better place to start than in the highly contested world of ethics? You can stop reading now because we’re talking about ethics, and that’s the last thing that anyone ever wants to talk about. But before you go, know this isn’t your standard Governance, Risk, and Compliance (GRC) talk where everything is driven by and modeled by a policy that can be easily policed, defined, dictated, and followed. Why isn’t it? Because if that were true, we wouldn’t have a need for any discussion on the topic of ethics and it would merely be a discussion of policy—and who doesn’t love policy?

 

Let me start by asking you an often overlooked but important question. Does data have ethics? On its own, the simple answer is no. As an example, we have Credit Reporting Agencies (CRAs) who collect our information, like names, birthdays, payment history, and other obscure pieces of information. Independently, that information is data, which doesn’t hold, construe, or leverage ethics in any way. If I had a database loaded with all this information, it would be a largely boring dataset, at least on the surface.

 

Now let’s take the information the CRAs have, and I go to get a loan to buy a house, get car insurance, or rent an apartment. If I pass the credit check and I get the loan, the data is great. Everybody wins. But, if I’m ranked low in their scoring system and I don’t get to rent an apartment, for example, the data is bad and unethical. OK, on the surface, the information may not be unethical per se, but it can be used unethically. Sometimes (read: often) a person's credit, name, age, gender, or ethnicity will be calculated in models to label them as “more creditworthy” or “less creditworthy” in getting loans, mortgages, rent, and so on and so forth.

 

That doesn’t mean the data or the information in the table or model is ethical or unethical, but certainly claims can be made that biases (often human biases) have influenced how that information has been used.

 

This is a deep subject—how can we make sure our information can’t be used inappropriately or for evil? You’re in luck. I have a simple answer to that question: You can’t. I tried this once. I used to sell Ginsu knives and I never had to worry about them being used for evil because I put a handy disclaimer on it. Problem solved.

 

Disclaimer

 

Seems like a straightforward plan, right? That’s what happens when policy, governance, and other aspects of GRC enter into the relationship of “data.” “We can label things so people can’t use them for harm.” Well, we can label them all we want, but unless we enact censorship, we can’t STOP people from using them unethically.

 

So, what do we do about it? The hard, fast, and easy solution for anyone new to machine learning or wanting to work with artificial intelligence is: use your powers for good and not evil. I use my powers for good, but I know that a rock can be used to break a window or hurt someone (evil), but it also can be used to build roads and buildings (good). We’re not going to ban all rocks because they could possibly be used wrongly, just as we’re not going to ban everyone’s names, birthdays, and payment history because they could be misused.

 

We have to make a concerted effort to realize the impacts of our actions and find ways to better the world around us through them. There’s still so much more on this topic to even discuss, but approaching it with an open mind and realizing there is so much good we can do in the world will leave you feeling a lot happier than looking at the darkness of and worry surrounding things you cannot control.

 

Was this too deep? Probably too deep a subject for the first in this series, but it was timely and poignant to a Lightning Talk I was forced (yes, I said forced) to give on machine learning and ethics at the recent ML4ALL Machine Learning Conference.

 

ML4ALL Lightning Talk on Ethics

 

https://youtu.be/WPZd2dz5nfc?t=17238

 

Feel free to enjoy the talk here, and if you found this useful, terrifying, or awkward, let’s talk about it. I find ethics a difficult topic to discuss, mainly because people want to enforce policy on things they cannot control, especially when the bulk of the information is “public.” But the depth of classifying and changing the classification of data is best saved for another day.

Screen Shot 2017-06-20 at 12.47.16 AM.png

 

IT professionals are a hardworking group. We carry a lot of weight on our shoulders, a testament to our past and future successes. Yet, sometimes we have to distribute that weight evenly across the backs of others. No, this is not because we don’t want to do something. I’m sure that any of you, while capable of performing a task, would never ask another person to do something you wouldn’t willingly do yourself. No. Delegating activities to someone else is actually something we all struggle with.

 

Trust is a huge part of delegating. You're not only passing the baton of what needs to be done to someone else, but you’re also trusting that they’ll do it as well as you would, as quickly as you would, and -- this is the hard part -- that they'll actually do it.

 

As the world continues to evolve, transition, and hybridize, we are faced with this challenge more often. I’ve found there are some cases where delegation works REALLY well, and other cases where I’ve found myself banging my head against the wall, desk, spiked mace, etc. You know the drill.

 

One particular success story that comes to mind involves the adoption of Office 365. Wow! My internal support staff jumped for joy the day that was adopted. They went from having to deal with weird, awkward, and ridiculous Exchange or Windows server problems on a regular basis to... crickets. Sure, there were and still are some things that have to be dealt with, but it went from daily activity to monthly activity. Obviously, any full-time Exchange admin doesn't want to be replaced by Robot365, but if it's just a small portion of your administrative burden that regularly overwhelms, it's a good bet that delegating is a good idea. In this particular use-case, trust and delegation led to great success.

 

On the other hand, I’ve seen catastrophes rivaled only by the setting of a forest fire just for the experience of putting it out. I won’t name names, but I've had rather lengthy conversations with executives from several cloud service providers we all know and (possibly) love. Because I’m discussing trust and delegation, let’s briefly talk about what we end up trusting and delegating in clouds.

 

  • I trust that you won’t deprecate the binaries, libraries, and capabilities that you offer me
  • I trust that you won’t just up and change the features that I use and my business depends on
  • I trust that when I call and open a support case, you’ll delegate activities responsibly and provide me with regular updates, especially if the ticket is a P1

 

This is where delegating responsibility and trusting someone to act in your best interest versus the interests of themselves or some greater need beyond you can be eye-opening.

 

I’m not saying that all cloud service providers are actively seeking to ruin our lives, but if you talk to some of the folks I do and hear their stories, THEY might be the one to say that. This frightful tale is less about the fear and doubt of what providers will offer you, and more about being aware and educated about the things that could possibly happen, especially if you aren’t fully aware of the bad things that happen on the regular.

 

In terms of trust and delegation, cloud services should provide you with the following guarantees:

  • Trust that they will do EXACTLY what they say they will do, and nothing less. Make sure you are hearing contractual language around that guarantee versus marketing speak. Marketing messages can change, but contracts last until they expire.
  • Trust that things DO and WILL change, so be aware of any depreciation schedules, downtime activities, impacts, overlaps of changes, and dependencies that may lie within your business.
  • Delegate to cloud services only those tasks and support that may not matter to your production business applications. You want to gauge how well they can perform and conform to an SLA. It’s better to be disappointed early on when things don’t matter than to be in a fire-fight and go looking for support that may never come to fruition.

 

This shouldn't be read as an attack or assault on cloud services. Instead, view this as being more about enlightenment. If we don’t help make them better support organizations, they won’t know to and will not improve. They currently function on a build-it-and-they-will-come support model, and if we don’t demand quality support, they have no incentive to give it to us.

 

Wow! I went from an OMG Happy365 scenario to cloudy downer!

 

But what about you? What kinds of experiences with trust and delegation have you had? Successes? Failures? I’ll offer up some more of my failures in the comments if you’re interested. I would love to hear your stories, especially if you've had contrary experiences with cloud service providers. Have they gone to bat for you, or left you longing for more?

cat-hiding-2872.jpg

 

 

 

 

Hey, guys! This week I’d like to share a very recent experience. I was troubleshooting, and the information I was receiving was great, but it was the context that saved the day! What I want to share is similar to the content in my previous post, Root Cause, When You're Neither the Root nor the Cause, but different enough that I thought I'd pass it along.

 

This tale of woe begins as they all do, with a relatively obscure description of the problem and little foundational evidence. In this particular case it was, “The internet wasn't working on the wireless, but once we rebooted, it worked fine.” How many of us have had to deal with that kind of problem before? Obviously, all answers lead to, “Just reboot and it’ll be fine." While that’s all fine and dandy, it is not acceptable, especially at the enterprise level, because it offers no real solution. Therefore, the digging began.

 

The first step was to figure out if I could reproduce the problem.

 

I had heard that it happened with some arbitrary mobile device, so I set up shop with my MacBook, an iPad, my iPhone and my Surface tablet. Once I was all connected, I started streaming content, particularly the live YouTube stream of The Earth From Space. It had mild audio and continuous video streaming that could not buffer much or for long.

 

The strangest thing happened in this initial wave of troubleshooting. I was able to REPRODUCE THE PROBLEM! That frankly was pretty awesome. I mean, who could ask for more than the ability to reproduce a problem! Though the symptoms were some of the stranger parts, if you want to play along at home, maybe you can try to solve this as I go. Feel free to chime in with something like, “Ha ha! You didn’t know that?" It's okay. I’m all for a resolution.

 

The weirdest part of this resolution was that for devices connecting on lower wireless bands, 802.11A, 802.11N, things were working like a champ, or seemingly working like a champ. They didn’t skip a beat and were working perfectly fine. I was able to reproduce it best with the MacBook connected at 802.11AC with the highest speeds available. But seemingly, when it would transfer from one APS channel to another AP on another channel, poof, I would lose internet access for five minutes. Later, it was proven to be EXACTLY five minutes (hint).

 

At the time though, like any problem in need of troubleshooting, there were other issues I needed to resolve because they could have been symptoms of this problem. Support even noted that these symptoms relate to a particular problem that was all fine and dandy when adjusted in the direction I preferred.  Alas, they didn’t solve my overwhelming problem of, “Sometimes, I lose the internet for EXACTLY five minutes.” Strange, right?

 

So, I tuned up channel overlap, modified how frequent devices will roam to a new access point and find their new neighbor, cleaned up how much interference there was in the area, and got it working like a dream. I could walk through zones transferring from AP to AP over and over again, and life seemed like it was going great. But then, poof, it happened again. The problem would resurface, with its signature registering an EXACT five-minute timeout.

 

This is one of those situations where others might say, “Hey, did you check the logs?” That's the strange part. This problem was not in the logs. This problem transcended mere logs.

 

It wasn’t until I was having a conversation one day and said, “It’s the weirdest thing. The connection with a full wireless signal, with minimal to no interference and nothing erroneous showing in the logs would just die, for exactly five minutes.” My friend chimed in, “I experienced something similar once at an industrial yard. The problem would surface when transferring from one closet-stack to another closet-stack, and the tables for Mac Refresh were set to five minutes. You could shorten the Mac Refresh timeout, or simply tunnel these particular connections back to the controller."

 

That prompted an A-ha moment (not the band) and I realized, "OMG! That is exactly it." And it made sense. In the earlier phases of troubleshooting, I had noted that this was a condition of the problem occurring, but I had not put all of my stock in that because I had other things to resolve that seemed out of place. It’s not like I didn’t lean on first instincts, but it’s like when there’s a leak in a flooded basement. You see the flooding and tackle that because it’s a huge issue. THEN you start cleaning up the leak because the leak is easily a hidden signal within the noise.

 

In the end, not only did I take care of the major flooding damage, but I also took care of the leaks. It felt like a good day!

 

What makes this story particularly helpful is that not all answers are to be found within an organization and their tribal knowledge. Sometimes you need to run ideas past others, engineers within the same industry, and even people outside the industry. I can’t tell you the number of times I've talked through some arbitrary PBX problem with family members. Just talking about it out loud and explaining why I did certain things caused the resolution to suddenly jump to the surface.

 

What about you guys? Do you have any stories of woe, sacrifice, or success that made you reach deep within yourself to find an answer? Have you had the experience of answers bubbling to the surface while talking with others? Maybe you have other issues to share, or cat photos to share. That would be cool, too.

I look forward to reading your stories!

Expectation.png

 

Hey, everybody!  Welcome to this week’s quandary of Root Cause, Correlation Analysis, and having to collaborate across cross-functional teams where you have all the hands but none of the fingers!

 

If that sounds confusing to you, it’s because frankly, it is! I’d like to share a tale of woe and heartbreak driven by frustration in functional and equally dysfunctional IT team dynamics!

 

The story is set in a fairly cross-functional organization. You're probably familiar with the type. While there are clearly defined teams with responsibilities, there are also hard lines in the sand of who does what, where, when, how and why. Honestly, this story rings so true that I’ve seen this story blur with other ones. If that isn’t excitement, I don’t know what is!

 

As the story goes, our team had deployed a series of tools enabling a cross-stack data correlation engine allowing us to identify and truly correlate events as they happen to allow troubleshooting to be better, easier.   The problem was the true burden of responsibility this team had ALL the responsibility of identifying problems, but none of the authority to actually resolve those problems, let alone the authorization to work on them!   What makes this particularly fun is that we were chartered with and burdened by the responsibility of being held accountable for the issues until they were resolved.   If that sounds like some kind of decision made in a government sector… I wouldn’t tell you you’re wrong! J

 

This is where simple technical skills while essential were not good enough.  And frankly, all of the project management skills in the world wouldn’t matter here, because it’s not like a “problem” is a “project” per se.   No, we had to get everyone on board, every stakeholder at the table where egos were strong and stubborn.   Just like we discussed recently in Better Together - Working Together in Silo Organizations and When Being an Expert Isn’t Good Enough: Master of All Trades, Jack of None merely knowing the answer or the cause of the problem wasn’t good enough here.   All parties would reject the issue being theirs, even in light of evidence proving otherwise and would instead resort to finger pointing. Fortunately how we started to navigate these waters was through education of the tools we were using and how it would provide insight into their systems, access to our tools so we weren’t just the messenger they were trying to shoot but a helpful informant in things, and we also offered our guidance as IT Professionals to help them navigate the errors or problems so they could resolve them better.

 

It sounds so simple, it’s something fairly straight-forward but the timing it took and would continue to take whenever new members would join a team, or new problems would surface would take months or longer to reach a sense of team parity.

 

It’s been an interesting element of Systems Operations in the face of having intelligence, and knowledge not meaning much of anything unless you had all parties engaged, and even then that was no guarantee that people would agree, let alone do anything about it.

 

Have you faced a similar issue as well, where you identify a problem which isn’t your problem and the challenges faced in trying to resolve it?  Or perhaps even just having accountability for something which isn’t your responsibility and the woes of trying to get parties to take responsibility?

 

Or really any other story of problem correlation and root cause and how you were able to better or faster resolve it than what we faced!

In my last post WHEN BEING AN EXPERT ISN’T GOOD ENOUGH: MASTER OF ALL TRADES, JACK OF NONE, you all shared some great insight on how you were able to be find ways to be successful as individual SMEs and contributors, and how you could navigate the landscape of an organization.  

 

This week, I’d like to talk about silo organizations and how we’ve found ways to work better together. (You can share your stories, as well!)

 

 

This is the first thing I imagine when I hear that an organization is silo-ed off:

oldsilos.jpg

 

The boundaries are clearly defined, the foundation is well set, it’s very aged and well established. It doesn’t mean any of it is particularly good or bad, but it certainly shows the test of time. Navigating in that landscape requires more than tackling a delicate balance of ego and seniority.

 

Once upon a time, we had a very delicate situation we were trying to tackle. This may sound simple and straightforward, but needless to say, it’ll all make sense on how things were far from easy. We were faced with deploying a syslog server. Things literally do NOT get any easier than that! When I first found out about this (security) initiative, I was told that this was a "work in progress" for over two years, and that no syslog servers had been deployed, yet. Wait. Two years? Syslog server. None deployed?! This can’t be that difficult, can it? Welcome to the silo-ed organization, right?

 

On its surface, it sounds so simple, yet as we started to peel back the onion:

 

Security needed syslog servers deployed.

The storage team would need to provision the capacity for these servers.

The virtualization team would need to deploy the servers.

The networking team would need to provide IP addresses, and the appropriate VLANs, and advertise the VLANs as appropriate if they did not exist.

The virtualization team would then need to configure those VLANs in their networking stack for use.

Once all that was accomplished, the networking and security teams would need to work together to configure devices to send syslog data to these servers.

 

All that is straightforward, and easy to do when everyone works together! The disconnected, non-communicating silos prevented that from happening for years because everyone felt everyone else was responsible for every action and it’s a lot easier to not do things than to work together!

 

Strangely, what probably helped drive this success the most was less the clear separation of silo-by-silo boundary and more the responsibility taken by project managing this as a single project. When things are done within a silo, they’re often done in a bubble and begin and end without notifying others outside of that bubble. It makes sense, like when driving a car we’re all driving on the same road together and our actions may influence each other’s (lane changes, signal changes, and the like), but what music I’m listening to in my car has no influence on any other car.  

 

So, while we all have our own interdependencies that exist within our silos, when we’re working together ACROSS silos on a shared objective, we can be successful together as long as we recognize the big picture.   Whether we recognize that individually, or we do collectively with some dictated charter, we can still be successful. When I started this piece, I was more focused on the effects and influence we can make as individuals within our silos, and the interaction and interoperability with others within silos. But I came to realize that when we each individually manage our responsibilities within a “project,” we become better together. That said, I'm not implying that formal project management is required for any or all multi-silo interactions. It really comes down to accepting responsibility as individuals, and working together on something larger than ourselves and our organization, not just seeing our actions as a transaction with no effect on the bigger whole.

 

Then again, I could be crazy and this story may not resonate with any of you.   

 

Share your input on what you’ve found helps you work better together, whether it be inter-silo, intra-silo, farming silos, you name it!

Has this situation happened to you? You've dedicated your professional career -- and let's be honest -- your life, on a subject, only to find “that's not good enough.” Maybe it comes from having too many irons in the fire, or it could be that there are just too many fires to be chasing.

 

Ericsson (1990) says that it takes 10,000 hours (20 hours for 50 weeks a year for ten years = 10,000) of deliberate practice to become an expert in almost anything.

 

I’m sure you’ve heard that Ericsson figure before, but in any normal field, the expectation is that you will gain and garner that expertise over the course of 10 years. How many of you can attest to spending 20 hours a day for multiple days to even multiple weeks in a row as you tackle whatever catastrophe the business demands, often driven by a lack of planning on their part? (Apparently, a lack of planning IS our emergency when it comes to keeping that paycheck coming in!)

 

I got my start way back in Security and Development (the latter of which I won’t admit if you ask me to code anything :)). As time progressed, the basic underpinnings of security began delving into other spaces. The message became, “If you want to do ANYTHING in security, you need networking skills or you won’t get very far.” To understand the systems you’re working on, you have to have a firm grasp of the underlying Operating Systems and kernels. But if you’re doing that, you better understand the applications. Oh, and in the late 1990s, VMware came out, which made performing most of this significantly easier and more scalable. Meanwhile, understanding what and how people do the things they do only made sense if you understood System Operations. And nearly every task along the way wasn’t a casual few hours here or there, especially if your goal was to immerse yourself in something to truly understand it. Doing so would quickly become a way of life, and before long you'd quickly find yourself striving for and achieving expertise in far too many areas, updating your skill sets along the way.

 

As my career moved on, I found there to be far more overlap of specializations and subject matter expertise, rather than clearly delineated silos. Where this would come to head as a strong positive was when I worked with organizations as a SME in storage, virtualization, networking and security, finding that the larger the organization, the more these groups would refuse to talk to each other. More specifically, if there was a problem, the normal workflow or blame assignment would look something like this picture. Feel free to provide your own version of events that you experience.

 

 

Given this very atypical approach to support by finger-pointing, having expertise in multiple domains would become a strong asset since security people will only talk to other security people. Okay, not always, but also, yes, very much always. And if you understand what they’re saying and where they’re coming from, pointing out, “Hey, do you have a firewall here?” means a lot more coming from someone who understands policy than from one of the other silos, which they seemingly have nothing but disdain for. Often, a simple network question posed by one network person to another could move mountains, because each party respects the ability or premise of the other. Storage and virtualization folks typically take the brunt of the damage because they regularly have to prove that problems aren’t their fault because they’re the easiest point of blame due to storage pool consolidation or hardware pool consolidation. Finally, the application guys simply won’t talk to us half the time, let alone mention that they made countless changes without understanding what WE did wrong to make their application suddenly stop working the way it should. (Spoiler alert: It was an application problem.)

 

Have you found yourself pursuing one or more subject matter domains of expertise, either just get your job done, or to navigate the shark-infested waters of office politics? Share your stories!

Wow, can you believe it? 2016 is almost over, the holidays are here I didn’t even get you anything!   It’s been a bit of a wild rollercoaster of a year through consolidation, commoditization, and collaboration!

 

I’m sure you have some absolute favorite trends or notable things which have occurred here throughout 2016.  Here are some that in particular have been a pretty recurring trend throughout the year.

 

 

  • Companies going private such as Solarwinds (closed in February), DellEMC (closed in September)
  • Companies buying other companies and consolidating industry like Avago buying Broadcom (Closed Q1), Brocade buying Ruckus (Closed Q3), Broadcom buying Brocade (Initiated in October)
  • Or companies divesting of assets like Dell selling off SonicWall and Quest, and Broadcom selling off Brocade’s IP division

 

 

Alright so that’s some of the rollercoaster at least a small snapshot of it, and the impact those decisions will have on practitioners like you and I only time will tell (I promise some of those will be GREAT and some of those, not so much!)

 

But what else, what else?! Some items I’ve very recently discussed include.

 

 

All three of these net-net benefit in the end really means that we will continue to see better technology, with deeper investment and ultimately (potentially) lower costs!

 

On the subject of Flash though if you haven’t been tracking the Density profiles have been insane this year alone and that trend is only continuing with further adoption and better price economics with technology like NVMe.  I particularly love this image as it reflects the shrinking footprint of the data center while reflecting our inevitable need for more.

 

Moores Law of Storage.png

 

 

This is hardly everything that happened in 2016 but these are particular items which are close to my heart and respectively my infrastructure.   I will give a hearty congratulation to this being the 16th official “year of vdi” a title we continue to grant it yet continues to fail to fulfill on its promises.  

 

Though with 2016 closing quickly on our heels there are a few areas you’ll want to be on the watch for in 2017!

 

  • Look for Flash Storage to get even cheaper, and even denser
  • Look to see even more competition in the Cloud space from Microsoft Azure, Amazon AWS and Google GCP
  • Look to Containers to become something you MIGHT actually use on a regular basis and more rationally than the very obscure use-cases promoted within organizations
  • Look to vendors to provide more of their applications and objects as Containers (EMC did this with their ESRS (Secure Remote Support)
  • Obviously 2017 WILL be the Year of VDI… so be sure to bake a cake
  • And strangely with the exception of pricing economics making adoption of 10GigE+ and Wireless wave2 we’ll see a lot more of the same as we saw this year, maybe even some retraction in hardware innovation
  • Oh and don’t forget, more automation, more DevOps, more “better, easier, smarter”

 

But enough about me and my predictions, what were some of your favorite and notable trends of 2016 and what are you looking to see coming forward looking to 2017?

 

And if I don’t get a chance to… Happy Holidays and a Happy New Year to ya’ll!

Well hey everybody, I hope the Thanksgiving holiday was kind to all of you. I had originally planned to discuss more DevOPS with ya’ll this week however a more pressing matter came to mind in my sick and weakened state of stomach flu!

 

Lately we’ve been discussing ransomware but more important, lately I’ve been seeing an even greater incidence of ransomware affecting individuals and businesses, and worse when it would hit a business it would have a lot of collateral damage (akin to encrypting the finance share that only cursory access was allowed to or such)

 

KnowBe4 has a pretty decent Infographic on Ransomware I’m tossing in here and I’m curious what ya’ll have been seeing in this regards.

Do you find this to be true, an increased incidence, a decrease, roughly the same?

 

Ransomware-Threat-Survey.jpg

 

Some real hard and fast takeaways I’ve seen from those who aspire to mitigate ransomware attacks is to Implement:

 

  • Stable and sturdy firewalls
  • Email filtering scanning file contents and blocking attachments
  • Comprehensive antivirus on the workstation
  • Protected Antivirus on the servers

 

Yet all too often I see all of this investment around trying to ‘stop’ it from happening without a whole lot left to handling clean-up should it hit the environment, basically… Having some kind of backup/restore mechanism to restore files SHOULD you be infected.

 

Some of the top ways I’ve personally seen where Ransomware has wrought havoc in an environment have happened in the cases of; 

  • Using a work laptop on an untrusted wireless network
  • Phishing / Ransomware emails which have links instead of files and opening those links
  • Opening a “trusted” file off-net and then having it infect the environment when connected
  • Zero Day Malware through Java/JavaScript/Flash/Wordpress hacks (etc)

 

As IT Practitioners not only do we have to do our daily jobs, and the business to keep the lights on, and focus on innovating the environment, and keeping up with the needs of the business.   Worst of all when things go bad, and few things are as bad as Ransomware attacking and targeting an environment, then we have to deal with that on a massive scale! Maybe we’re lucky and we DO have backups, and we DO have file redirect so we can restore off of a VSS job, and we can detect encryption in flight and stop things from taking effect.   But that’s a lot of “Maybe” from end-to-end in any business and all of the applicable home devices that may be in play.  

 

There was a time when Viruses would break out in a network and require time and effort to cleanup, but at best it was a minor annoyance.  Worms would breakout and so long as we stopped whatever was the zero-day trigger we could stop it from occurring on the regular.   And while APTs and the like are more targeted threats this was less of a common occurrence for us to deal with where it would occupy our days as a whole.   But Ransomware gave thieves a way to monetize their activities, which gives incentives to infiltrate and infect our networks.   I’m sure you’ve seen the Ransomware now offering Helpdesk to assist victims with paying?

 

 

It’s definitely a crazy world we live in, one which leaves us only with more work to do on a daily basis, a constant effort to fend off and fight against.  This is a threat which has been growing at constant pace and is leaking and growing to infect Windows, Mac AND Linux.

 

What about your experiences, do you have any attack vectors for Ransomware you’d like to share, or other ways you were able to fend them off?  

Sitting back at the office getting work done, keeping the ship afloat, living the Ops life of the perception of DevOps, only to have your IT Director, VP or CxO come and demand, “Why aren’t we using Containers! It’s all the rage at FillInTheBlankCon!” And they start spouting off the Container of the week, Kubernetes, Mesos, Docker, CoreOS, Rocket, Photon, Marathon, and another endless Container product, accessory or component of a container.   If it hasn’t happened to you, that may be a future you’ll be looking at.  If it has happened to you, or you’ve already adopted some approach to Containers in your environment even more the better.

 

Screen Shot 2016-11-16 at 8.59.51 PM.png

Just as a VERY brief primer in the infinite world of containers for those of you who are not aware I’ll try to overly simplify it here.  Using the following image as an example and comparing it to Virtualization. Typically, Virtualization is hardware running a hypervisor to which you abstract of the hardware and install an Operating System on the VM and then install your applications into that. Whereas in most container scenarios you have hardware, running some kind of abstraction layer which you present Containers where you install your applications, abstracting out the Operating System.  

 

So quite possibly the most overly simplified version of it because there are MANY moving parts under the covers to make this a reality and make it possible.  However, who cares how it works as much as how you can use it to improve your environment, right?!

 

That’s kind of the key of things, Docker one of the more commonly known Container approaches (albeit technically Kubernetes is used more) has some real cool benefits and features of it. Docker officially has support for running Docker Containers on Microsoft Servers, Azure and AWS, and they also released Docker for Windows clients and OSX!   One particular benefit there that I like as a VMware user ishttp://www.virtuallyghetto.com/2016/10/powercli-core-is-now-available-on-docker-hub.htmlPowerCLI Core is now available on Docker Hub!

But they don’t really care about how you’re going to use it, because all roads lead to DevOps and how you’re supposed to implement things to make their lives better. But in the event that you will be forced down a road of learning a particular Container approach for better or worse it’s probably best to find a way to make it better your life rather than just another piece of infrastructure we’re expected to understand even if we don’t.   I’m not saying that one Container is better than another, I’ll leave that up to you guys to make that particular determination in the comments if you have Container stories to share.   Though I’m particular to Kubernetes when it comes to running cloud services on Google, but then I really like Docker when it comes to running Docker for OSX (because I run OSX )

The applications are endless and continually growing and the solutions are plentiful, some might say far too plentiful depending.   What are some of the experiences you’ve had with containers, the good the bad and the ugly, or is it an entirely new road you’re looking at pursuing but haven’t yet?  We’re definitely in the no judgement zone!

As always I appreciate your insight into how ya’ll use these technologies to better yourselves and your organizations as we all grow together!

 

That’s a good question, what do self-driving vehicles have to do with our infrastructure? Is it that it’s untested, untrusted and unproven and could result in death and fear mongering? That’s certainly a true enough statement, though what is the key difference in the distinction of ‘autonomous’ vs merely self-driving vehicles?

Screen Shot 2016-11-09 at 10.36.24 PM.png

Screen Shot 2016-11-09 at 10.33.07 PM.png

 

 

Telemetry is the watch word

 

Today’s networks and systems are a huge ball of data, big data, helpful, insightful, useless and befuddling endless piles of information.   Left up to their own devices that information lives in its respective bubble waiting for us to ‘discover’ a problem and then start peeling back the covers to figure out what is going on.   The Autonomous Self-Driving example vs simply ‘self-driving’ is that you’re using data from many continuous and constant streams, using that data to correlate events and understand conditions.   In its primitive state it can be fairly effective, in a networked sense; Imagine every vehicle on the road communicating with each other, constantly panning and analyzing everything in front of, behind you and here, there and everywhere.   Compound that data with collected information from external sources such as road sensors, lights and other conditions and you have the power of having traffic management be automated (Slap weather stations into each of these vehicles and we get closer to predicting even more accurate weather patterns)

 

But hey, whoa, What about my network? My systems?!

 

More and more we’re continuing to see solutions which are evolved far beyond simply a point solution. SIEMs don’t just collect security and event information in a bubble. Syslogs aren’t just an endless repository of arbitrary strings of ‘event’ information.  SNMP need not live caught in its own trap.

 

There are tools, solutions, frameworks, and suites of tools which aim to bring your NOC and SOC into the future, a future wholly unknown.   There is no true panacea to tie everything together and be the end-all-be-all solution, though as time goes on evolutions and consolidations of products have been starting to make that possible.   There was a time when I ran a massive enterprise we would have ‘point’ tools, which do an amazing job of keeping up on THEIR data and telemetry though they were independent and not even remotely interdependent. Monitoring VMware with vCOPS, Monitoring the network with Orion and NPM, collecting some event data with ArcSight, while separately collecting Syslog information with Kiwi Syslog server, and yet SNMP traps would flow into SNMPc, oh and lets not forget monitoring Microsoft… That’s where System Center came in. 

 

On the one hand that may seem like an excessive amount of overkill, yet each ‘product’ covered and fulfilled its purpose, doing 80% of what it did well, yet in the remaining 20% unable to cover the rest of the spread. (Slight disclaimer, there were some 50+ more tools, those were just the ‘big’ ones that we’ve all likely heard of J)

 

So each of these solutions as they evolve or other products in the industry continue to evolve they’re taking what has effectively been the ‘cruise control’ button in our cars or even slightly better than cruise control and building the ability to provide real data, real analytics, real telemetry so that the network and our systems can work for us and with us, vs being little unique snowflakes that we need to care and feed for and figure out when things go wrong.

 

So what have you been using or looking at to help drive the next generation of infrastructure systems telemetry?   Are you running any Network Packet Brokers, Sophisticated ‘more than SIEM’ like products, or Solarwinds suites to tie many things together, Or has anyone looked at Intel’s open sourced Open Telemetry Framework, SNAP?

 

Please share your experiences!

Cloud Dollars.png

 

The Cloud! The Cloud! Take us to the Cloud it’s cheaper than on-premises, why? Because someone in marketing told me so!  No, but seriously. Cloud is a great fit for a lot of organizations, a lot of applications, a lot of a lot of things! But just spitting ‘Cloud’ into the wind doesn’t make it happen, nor does it always make it a good idea.   But hey, I’m not here to put Cloud down (I believe that’s called Fog) nor am I going to tout it unless it’s a good fit.   However, I will share some experiences, and hopefully you’ll share your own because this has been a particular area of interest lately, at least with me but I’m weird about things like deep tech and cost benefit models.

 

The example I’ll share is one which is particularly dear to my heart. It’s dear because It’s about a Domain Controller!   Domain Controllers are for all intents and purposes, machines which typically MUST remain on at all times, yet don’t necessarily require a large amount of resources.  So when you compare a domain controller running On-Premises let’s say as a Virtual Machine in your infrastructure it carries with it an arbitrary cost aggregated and then taken as a percentage of the cost of your Infrastructure, Licensing, allocated resources, and O&M Maintenance cost for Power/HVAC and other.   So how much does a Domain Controller running as a Virtual Machine run inside your data center? If you were not to say, “It Depends” I might be inclined not to believe you, unless you do detailed charge back for your customers.

 

Yet, we’ve stood up that very same virtual machine inside of Azure, let’s say a standard Single Core, Minimal memory A1-Standard instance to act as our Domain Controller.   Microsoft Azure pricing for our purposes was pretty much on the button, coming in at around ~$65 per month.   Which isn’t too bad, I always like to look at 3 years at a minimum for the sustainable life of a VM just to contrast it to the cost of on-premises assets and depreciation.   So while $65 a month sounds pretty sweet, or ~$2340 over three years I have to also consider other costs which I might not normally be looking at.  Egress network bandwidth, Cost of backup (Let’s say I use Azure backup, that adds another $10 a month, so what’s another $360 for this one VM)

 

The cost benefits can absolutely be there if I am under or over a particular threshold, or if my workloads are historically more sporadic and less ‘always-on, always-running’ kind of services.

An example of this, is we have a workload which normally takes LOTS of resources and LOTS of cores and runs until it finishes.   We don’t have to run it too often (Quarterly) and allocating those resources, obtaining the assets while great, they’re not used every single day.   So we spin up a bunch of Compute or GPU Optimized jobs and when it might have taken days or weeks in the past we can get it done in hours or days, which means we get results and we release the resources once we get our data dumped out.

 

Certain workloads will tend to be more advantageous to others to be kept on-premises or hosted exclusively in the cloud, whether sporadically or all the time.   That really comes down to what matters to you, your IT and your support organization.

 

This is where I’m hoping you my fellow IT Pros can share your experiences (Good, Bad, Ugly) about workloads you have moved to the Clouds, I’m preferable to an Azure, Google or Amazon as they’ve really driven things down to a commoditized goods and battle amongst themselves, whereas an ATT, RackSpace, and other ‘hosted’ facility type cloud can skew the costs or benefits when contrasted to the “Big Three”

 

So what has worked well for you, what have you loved and hated about it. How much has it cost you? Have you done a full shift taking ALL your workload to a particular cloud or Clouds. Have you said ‘no more!’ and taken workloads OFF the Cloud back On-Premises? Share your experiences so that we may all learn!

 

P.S., We had a set of Workloads hosted Off-Premises in Azure which were brought wholly back in house as the high performance yet persistent always-on nature of the workloads was costing 3x-4x more than if we had simply bought the Infrastructure and hosted it internally. (Not every workload will be a winner )

 

Thanks guys and look forward to hearing your stories!

Filter Blog

By date: By tag:

SolarWinds uses cookies on its websites to make your online experience easier and better. By using our website, you consent to our use of cookies. For more information on cookies, see our cookie policy.