Showing results for 
Search instead for 
Did you mean: 
Create Post

Project Documentation Redux

Level 9

A couple of weeks ago, I wrote about how I love documentation and how useful it can be in the future, for you or someone else. I also lamented slightly about how it is hard on most projects to get the time to write good reference documentation. This week, I'm going to discuss one possible way to knock out some of the more time-consuming parts.

The key word is: Design. That is--the "Design" phase of the project. In most projects, regardless of the flavor of SDLC utilized, there is a block of time to be utilized for designing the solution. Since this is work that will be happening anyway, getting some good documentation out of it is simply a matter of writing down what gets discussed, decisions that get made (along with the "why"s, of course), and how the solution(s) will be structured. Chances are these things get written down, anyway, but outside the mindset of their possible use as future reference material. Usually, by its nature, designing a project will produce some useful artifacts; things like high-level architecture diagrams or possibly an ERD or two. If it's a data-integration or BI project, and one is into details, source-to-target mappings are likely generated at this point.

All of these items add up to a decent set of notes for the future, explaining the solution and where it came from. This way, even if no dedicated time can be worked into a project for documentation, useful items can be produced.

I think there's another benefit to this course of action. I have a phrase I use to describe my philosophy on this topic: I don't like to think while I'm writing ETL. This sounds kind of bad on the surface, but what I really mean is this: When it comes time to actually sling some code, I want to be able to concentrate on the mechanical details; Things like doing everything right, following best practices, security concerns, and making things as highly-performing as they can be, all right from the get-go. With the data flows already drawn out and the business rules documented and agreed upon, it is easy to focus on those things.

We had a fairly nice conversation about it being hard to get the time to write documentation when I wrote about it before. Would beefing up the efforts during the design phase help? Are ya'll even able to get good time for the design stage in the first place (I've been in predicaments where that gets axed, too)?

Level 12

I try to wedge as much of this writing into the models themselves.  Whether it's an ERD, or a data lineage design mapping, I want those decisions and discussions right in the models so that as the models change, I can update those thoughts.  Plus I don't have to go find some word document or SharePoint site to find some version of the documentation.  In fact, i call this all "modeling" and try not to even call it documentation. It's "just enough" writing stuff down.

This also means that I can generate the documentation more than write it. That's a win-win-win.

Level 17

A wise DBA once said that "great database performance starts with great database design", and I can't argue with such logic. Thanks for the post!

Level 11

Having done a number of large BI projects, docs are the first thing that get cut. Especially when resources get tight, and the ETL invariably changes. The problem with doing it in the design phase is that of changing requirements.

I feel like one of the best approaches (aside from a formal docs team) is to use OneNote as a rolling notes library, with another step being (if using a tool like BIML) documenting within code.

Level 9

I'm afraid if I did that, there'd be more blobs of text on the model than there would be objects!

Level 12

Oh. You're thinking of adding the text to the diagram. Since a data model > that the diagram of the model, I put the text in the notes, comments, properties, meta data, whatever in the model.  Those can be shown on a diagram, or reported upon.

And even better, published to a special portal that hosts all the models.

So many exciting things happening in the modeling world to make writing stuff down much easier and more agile.

There may be other documentation that doesn't have a nice home in the model.  That can go in other notes.

Level 9

Right, that's true...writing things as "design" means that it needs to be maintained as the project goes along, because everything always changes. There isn't, unfortunately, always time for that.

I'm a big proponent of mumbling in OneNote as things go along, whether I'm solo on a project or working on a team. That said, this doesn't always lead to the most organized pile of text, and that means (in my experience), it has an even less chance than normal of getting read

Level 9

I think I need to go back to Modeling School(tm)

Level 12

Somebody should consider my presentations on these topics to be PASS-worthy

Level 11

We used Confluence Wiki at Cabletown, and it was pretty decent. We weren't perfect, but it wasn't bad.

Level 12

Great database design starts with great data modeling.

Level 12

Wikis are wonderful.

In a high-trust, teamwork-happy, professional environment.  Throw a fixed-price, "value" contracting firm in the mix and you end up spending hours a day rolling back changes that were made  just to cut scope and quality. Fortunately, I never work on projects like that.

Level 15

I have been involved with tons of LEAN projects and worked on many small to medium scale projects in my decades of IT career.  I have found it to be the most beneficial to get the design into the documentation early.  During many Kaizen that was the predominate requirement.  Gain an understanding of the project, document the current state, design the future state.  Keeping the documentation accurate as the process evolves is the only way to ensure the documentation gets done.  When the process heads towards completion, the only remaining items is to finalize the documentation.  It is an evil effort but it is a necessary evil.  I presently have ten open projects and are all in partial stages of design, implementation, and fortunately, I have Visio diagrams to show how the projects are supposed to be configured, revisions are included, and my co-workers are on-board.  It takes discipline but the ends justifies the means.

I just found out that a project I designed and documented back in 1991 is still be utilized by customer.  I couldn't believe it but they told me that it was so well designed and documented that they have not had a need to change it.

Level 9

Unfortunately, I think the current process for selecting is making that less-likely. Hopefully I've got it backwards.

Level 9

I...I..I... I got nothin'

Level 12

But I bet you've worked with those guys who think "just enough documentation" means "just go ask the guy who wrote it*".  Of course, the guy who wrote it has been long gone from 5 companies since he worked on your project.

I think the main issue is that people think documentation means sitting down and writing a single 800 page Word document.  No need for it to be that at all. In fact, I'd say those rarely get referenced.

*Scott Ambler says this all the time.  It's sad.

Level 9

You dropped one of the magic words in there: discipline. Something that should be able to go without saying or having to pressure people to maintain, but, well, doesn't always work that way.

I can only hope that in 10 years someone is still using something I built!!

Level 9

I'm pretty guilty of generating great big Word docs when left to my own devices, but I'd like to think that a good table of contents and a well-organized doc makes it possible to roll in and just hit the section that addresses what you're looking for.

Just because I'm willing to do that doesn't mean everyone else is, though.

Level 15

I have been in departments that they require the enormous weight in documentation and then as I have studied more in the LEAN world, that a simple diagram is often the most helpful.  I have been the guy that wrote the documentation (datachick) so I have gotten tons of questions, but if you take a few minutes as the project moves forward and get visual diagrams updated, it is documentation that gets referenced.  The other thing that I have made a habit is, when new team members are on-boarded that they spend some time looking over the documentation and ask any questions, then from that I update the relevant documentation.  They may not understand all the details but if it serves as good reference material it will be used.

As busy as our days are, without good reference documentation, they could be much worse!


datachick‌ - Excellent follow-on to your previous blog.

Level 15

As long as the goal of the documentation is to remove tribal knowledge and at the same time increase the overall knowledge level of the organization, the method of the actual documentation is not as important as much as there is documentation. 

Level 14

Agreed.  Great follow up post.  Thanks!

About the Author
Kerry Tyler started his SQL Server career when he debuted as an accidental DBA in 2005. It wasn't all bad, as he had been hoping for such an opportunity. Seeing Reporting Services 2005 demoed for the first time sealed the deal, and it has been all data ever since, leaving the worlds of networking and systems admin behind. After being a full-time dev/operational DBA with everything since SQL 2000, Kerry is now back to BI, as a Senior BI Engineer/Consultant in Nashville, Tennessee.