cancel
Showing results for 
Search instead for 
Did you mean: 
Create Post

The only source of knowledge is experience...

Level 11

The title of this post is a quote that might be (or not be) from Albert Einstein. I first thought the title of the post would be something like:” why root cause analysis is your saviour”, but I think knowledge/experience title has more potential, so I’ll stay with Albert…

I’ve seen my fair share of companies collecting loads of data on incidents happening in their database environments, but using this “knowledge” and use it to come out better experienced seems to be a hard thing to establish.

What Albert means is that we need to make sure that the only way to make things better is by learning from our mistakes. Mistakes will happen and so will incidents, there is nothing we can do to prevent that from happening. And to be honest Albert (and me ;)) wouldn’t want you to stop making mistakes because without mistakes innovation would come to an end. And that would mean that this is it, nothing will ever be better than what we have now. No. We need innovation. We need to push further and harder. In the database world, I mean.

Screen Shot 2017-11-30 at 22.38.36.png

To do so, we’ll have incidents. But if we perform root cause analysis and learn from the incidents happening and try to figure out what we can do to prevent the same problem from ruining our day ever again.

The right tool to perform root cause analysis will provide you all the information needed. We all want to end up like the kid in the picture below, right? Although…. Albert and me might say the only way to innovate is by making mistakes, AND learn from them.

In two weeks I’ll write another post in which I’ll look back at the post from the last couple of weeks and what tools I think are essential for a good performing database environment, now and in the future. In the meantime, I love the comments you all provide.  I’ll try to answer as many as possible in due time.

10 Comments

Albert had it right.  When I was little kid, hearing my parents say "don't touch the stove--it's hot!" didn't provide knowledge.  Touching it DID!

That first-hand experience makes the information real instead of theoretical.

I've sat through enough RCA's & PIR's to appreciate honest scrutiny and the value of a work environment and culture that does not point fingers, but identifies areas in which processes can be improved for the future.  The most important take-away is thoroughly understanding the cause of an outage and not repeating it.

Then sharing the information learned so that others can accept your "theoretical" knowledge, and not have to duplicate mistaken processes to make your information "real" to them.

Level 14

An old friend of mine used to say to her kids, "they call them accidents(ie. mistakes, incidents), not purposes!" 

The secret is as my father always told me is "never do the same stupid thing twice!"

RCA's are an excellent tool in the technical world to keep from being stupid twice! (Enter here the old saying about history repeating itself!!!!  I always do....)

Thanks for reminding us of the importance of the proces arjantim​ ! Well done!

I'd agree, "never do the same stupid thing twice" is something I've said to my teams many times.

Level 14

RCA is all well and good but getting people to learn from the results is another matter. 

I've just had a user complaining that she was having inbound e-mails blocked and wasn't receiving messages telling her of the block.  Turned out she had marked inbound e-mails from IT as junk so wasn't seeing the block messages.  We un-junked the IT e-mail address, released the e-mails which were OK and explained it all to her.  The very next day she complained again.  Guess what.  E-mails from IT were again being marked as junk.  Some people are just too stupid to use a computer.

MVP
MVP

Nice write up

MVP
MVP

Who was the coach that encouraged his team to take risks? His quote went something like "the team today that wins will be the team that makes the most mistakes." Taken out of context that sounds terrible, but when you understand what he is saying it is freeing and breeds creativity. Obviously in IT mistakes can be costly, but they are a part of the learning, growing process - without them we are tempted to "That's how we always do it" which, personally, I think is way worse.

Interesting points. Here is our basic RCA template. I would be curious to see others:

Executive Summary

I. Technical Details - Ticket #'s, outage times, etc.

II. What We Did To Fix It

III. What We Are Going To Do To Make Sure It Doesn't Happen Again

IV. Service Improvement Projects (complete with ticket/project #'s)

MVP
MVP

Excellent article ..

Had a manager who used to sum up the gist of the article as "TRIPLE E" - EDUCATION, EXPOSURE & EXPERIENCE.

You may get the knowledge via some education or learning  program, but if you are not exposed to situations that warrant usage of the learning, you will never experience the results

Level 20

With some risk often can come reward but also making mistakes and it's how we learn.

Level 21

RCA's are an important part of life in the tech world and unfortunately they are not done often enough with leads to recurrence of the same issues.

About the Author
In the IT since 1998 and enjoying every last bit of it. The last few years are mainly focused on virtualization and Storage. VMware VCAP-DCA, VCP 4/5, VSP 4/5, VTSP 4/5, MCSA, MCTS, MCP, CCA and CCNA