Documentation – Your Secret Weapon(anup.io)
anup.io
Documentation – Your Secret Weapon
https://www.anup.io/2019/08/05/documentation-your-secret-weapon/
23 comments
Documentation often turns into wasted effort. Either because it is not very good or because it is never read. Another thing that can easily happen is that the documentation gets scattered in various places and it is difficult to know whether there useful information pertaining to the problem at hand and if so where it would be.
Good documentation needs a judgement about what is important to tell people and how to best present that information. This is not an easy thing to do and is never going to happen without some dedication. Certainly if the developer actually does not want to write anything but is told to 'just write some comments' it is not going to lead to anything helpful.
So, either management has to insist on documentation and actually check that what is written is helpful and is in some sort of searchable structure or it is just a waste of everybody's time.
Good documentation needs a judgement about what is important to tell people and how to best present that information. This is not an easy thing to do and is never going to happen without some dedication. Certainly if the developer actually does not want to write anything but is told to 'just write some comments' it is not going to lead to anything helpful.
So, either management has to insist on documentation and actually check that what is written is helpful and is in some sort of searchable structure or it is just a waste of everybody's time.
Scattered docs is big issue if there are not company-wide policies. I started to put docs in code after wasting so much time pursuing management for consistency.
Sometimes, the mere process of writing documentation (or a specification) can lead the author to a better understanding and helps uncover problems. Writing, in this case, is an aid to thinking. It may not be the best-case scenario, but it's not a total waste of time, either.
Absolutely. If nothing else, it helps the author realize what he doesn't know. I know of no better way to understand how well I understand something, than to try to document it.
This seems like a “cultural” issue you want to investigate before you start working somewhere. The problem is, you need to drill down to technical staff, in private, to have any hope of getting an honest answer, as all managers seem to fervently believe that their staff is going to write documentation.
Personally, I am NOT in the “self documenting” camp. I’m more on the side of what research there is that suggests that summary documentation for modules/routines saves about 10% of the time when you go to reread some code. Also, the “self documenting code” side seems to be under the impression that I actually want to read their code - AT ALL - when I could often avoid having to do so, at least for large swaths.
That said, the developers who prefer not to document should stick to themselves, and away from thos of us who wish that they did. Thus, “cultural issue”, I think both sides will be happier if they stay away from each other. Yeah, this feels like Steve Yeagge’s “software liberal vs conservative” sort of axis.
I’m glad they didn’t teach “self documenting mathematics” back when I was in school :-)
Personally, I am NOT in the “self documenting” camp. I’m more on the side of what research there is that suggests that summary documentation for modules/routines saves about 10% of the time when you go to reread some code. Also, the “self documenting code” side seems to be under the impression that I actually want to read their code - AT ALL - when I could often avoid having to do so, at least for large swaths.
That said, the developers who prefer not to document should stick to themselves, and away from thos of us who wish that they did. Thus, “cultural issue”, I think both sides will be happier if they stay away from each other. Yeah, this feels like Steve Yeagge’s “software liberal vs conservative” sort of axis.
I’m glad they didn’t teach “self documenting mathematics” back when I was in school :-)
Few random observations from my experience:
- The docs and the code should live at different abstraction levels. Docs add most value when they explain things which are not readily expressible in code.
- Docs are often written first, as a mental tool to fully understand the problem. In this context, if the docs and code differ, the bug is usually in the code.
- In rare, but highly valuable cases, docs can be non-textual. Database models (ER diagrams) are a prime example.
- With right tooling, docs become part of IDE experience. A popup shows not just the signature of the method under the cursor, but its doc comments as well. This means less jumping around and better "flow" when trying to comprehend somebody else's (or your own!) code.
- The docs and the code should live at different abstraction levels. Docs add most value when they explain things which are not readily expressible in code.
- Docs are often written first, as a mental tool to fully understand the problem. In this context, if the docs and code differ, the bug is usually in the code.
- In rare, but highly valuable cases, docs can be non-textual. Database models (ER diagrams) are a prime example.
- With right tooling, docs become part of IDE experience. A popup shows not just the signature of the method under the cursor, but its doc comments as well. This means less jumping around and better "flow" when trying to comprehend somebody else's (or your own!) code.
I agree that docs and code comments should complement each other. As a visual learner, I find diagrams showing how classes/objects fit together to be much more helpful than reading the documentation. A picture can replace a thousand words and tools like Vizio/Yed are really helpful in my opinion while at the same time minimizing the amount of documentation.
I'm sort of the opposite. Diagrams just show an abstract to me. At a high level they are great, but I always find myself trying to dust off the diagram and figure out what really is happening underneath. Its always so damn complex!
Two pieces of software I’ve had to work with for the last five years are matlab and labview. Say what you will about matlab, I think the documentation is the best I’ve seen anywhere. The labview documentation is some of the worst I’ve seen.
I’ll generally forgive an open source project for poor docs, because I can read the code(and it’s free, so how much can I really complain??) Closed source that costs thousands? You better have have your docs tight. That said, labview docs are worse than many open source projects. Most of it boils down to useless tautological statements like
“X: the input variable x”
Three things that make the matlab docs so much better:
1. A plethora of examples, in the same page as the function doc. Labview makes you open a special program (no, it’s not available via a web search) and search for an example. The search function sucks.
2. Web searches for matlab docs return the doc for the latest version. Almost always with labview, you get something from like 2012.
3. A real discussion of the algorithm being implemented, with real references. I can’t recall ever seeing a citation in a labview doc.
So there’s some impressions, from a users point of view:)
I’ll generally forgive an open source project for poor docs, because I can read the code(and it’s free, so how much can I really complain??) Closed source that costs thousands? You better have have your docs tight. That said, labview docs are worse than many open source projects. Most of it boils down to useless tautological statements like
“X: the input variable x”
Three things that make the matlab docs so much better:
1. A plethora of examples, in the same page as the function doc. Labview makes you open a special program (no, it’s not available via a web search) and search for an example. The search function sucks.
2. Web searches for matlab docs return the doc for the latest version. Almost always with labview, you get something from like 2012.
3. A real discussion of the algorithm being implemented, with real references. I can’t recall ever seeing a citation in a labview doc.
So there’s some impressions, from a users point of view:)
Clear, readable, up-to-date documentation is also a 'secret weapon' in another scenario: when you want people to adopt your software and use it. It doesn't matter whether you're selling to businesses or promoting an open source project - good quality documentation is the exception not the rule (even from large multi-million dollar companies).
You may think the program operation or the code you wrote is self-explanatory, but there will be dozens, hundreds, maybe even thousands of users who will never email or contact you to tell you they're struggling with your software. They'll just give up and silently go elsewhere.
You may think the program operation or the code you wrote is self-explanatory, but there will be dozens, hundreds, maybe even thousands of users who will never email or contact you to tell you they're struggling with your software. They'll just give up and silently go elsewhere.
Completely agree. Good documentation:
- Makes a system more understandable
- Stops trivial support tickets
- Makes mediocre code seem great
- Lowers the bar to entry for non-trivial projects
- Increases the competency of many users/coders/etc.
Doc is awesome. And under-appreciated. And a duty far too many programmers slough off.
- Makes a system more understandable
- Stops trivial support tickets
- Makes mediocre code seem great
- Lowers the bar to entry for non-trivial projects
- Increases the competency of many users/coders/etc.
Doc is awesome. And under-appreciated. And a duty far too many programmers slough off.
Not the same as a wiki or a tech spec or whatever but detailed, searchable pull requests go a long way towards good documentation. And it's something any dev can match up with code with git blame. Sometimes pull requests are vague because there's a lot of discussion happening in a different channel. That's fine -- just make sure the discussion is happening in a form you can easily link to, like a chat message or mailing list thread or meeting notes or whatever.
I don't think the agile manifesto got it wrong. It doesn't implore us to avoid writing documentation. It's not even an excuse to forego it. It's simply a value statement: we prefer working software to extensive documentation.
That means you should still write documentation.
And it also means that you shouldn't stall your project, delay work, or otherwise blocking developers from shipping working software because the documentation isn't 100%.
That means you should still write documentation.
And it also means that you shouldn't stall your project, delay work, or otherwise blocking developers from shipping working software because the documentation isn't 100%.
The closest I ever had to 'comprehensive documentation' was a Sandcastle site based on the triple /// XML comments above each method describing the parameters and expected return types in a C# codebase for some ticket scanners.
Other places had a 'covert' dokuwiki living at sandiego.companyname.local which if you knew the right email administrator to give you a login would reveal a treasure trove of knowledge written down or copy/pasted from emails by the old guard. (PC LOAD BETTER for any former fellow coworkers, you know who you are)
Setting up a Dokuwiki (on a flash drive or server) is fall off a log easy these days. Same with Onenote (2010 is the prettiest IMO), or if you want to put more work in, hook up your Jenkins build process to include fetching all markdown files from source and posting them to a statically generated site such as MkDocs.
Other places had a 'covert' dokuwiki living at sandiego.companyname.local which if you knew the right email administrator to give you a login would reveal a treasure trove of knowledge written down or copy/pasted from emails by the old guard. (PC LOAD BETTER for any former fellow coworkers, you know who you are)
Setting up a Dokuwiki (on a flash drive or server) is fall off a log easy these days. Same with Onenote (2010 is the prettiest IMO), or if you want to put more work in, hook up your Jenkins build process to include fetching all markdown files from source and posting them to a statically generated site such as MkDocs.
And out of date documentation can be a footgun.
I would rather a Dev spend time reading well written, self-documenting code than "documentation" that has become months or years out of date.
As systems become large and complex - and evolve - the work to keep documentation up to date becomes enormous. I have seen many project that generate reams of documentation at the start, and for almost all of it to never be updated again even after major shifts in design and technology.
And arguing that documentation can compensate for poorly written code as far as maintainability goes is simply absurd. Yet if it really is a "secret sauce" it would do so.
I would MUCH rather a comprehensive unit/regression test suite than copious documentation, and I think reading tests is one of the best ways, if not the best way, to start understanding a code base.
I would rather a Dev spend time reading well written, self-documenting code than "documentation" that has become months or years out of date.
As systems become large and complex - and evolve - the work to keep documentation up to date becomes enormous. I have seen many project that generate reams of documentation at the start, and for almost all of it to never be updated again even after major shifts in design and technology.
And arguing that documentation can compensate for poorly written code as far as maintainability goes is simply absurd. Yet if it really is a "secret sauce" it would do so.
I would MUCH rather a comprehensive unit/regression test suite than copious documentation, and I think reading tests is one of the best ways, if not the best way, to start understanding a code base.
[deleted]
This is a space that could use a good SAAS product. Like a Rap Genius for software with documentation tied to specific Git versions and tied into flow charts, etc. You could add a VSCode extensions on top to easily add new documentation & see when documentation exists. Open source projects use it for free and private companies essentially pay for it forever. Maybe something exists?
Well, I think even more important than the amount and coverage of documentation is its quality.
I have been in projects, where it would have been favorable if there would have been no documentation at all because the existing one was simply wrong and it took the organization weeks until someone noticed it and got a developer to actually look into the code.
I have been in projects, where it would have been favorable if there would have been no documentation at all because the existing one was simply wrong and it took the organization weeks until someone noticed it and got a developer to actually look into the code.
This is one of the reasons I tend to prefer comments over external documentation whenever possible. No, there's no magic force keeping comments in sync with the code they document changes, but you at least have a fighting chance. Programmers have enough state to juggle in their head as it is without expecting them to even necessarily know what all the external documentation it is they should update. Heck, even with comments it can be bad enough knowing whether some change to this module necessitates changes to comments in other modules in the code that you may have just looked at.
It isn't always possible. Very high-level architecture may belong in a format where diagrams are more available, and the high-level design justification for the project itself doesn't have a great place in the code.
It isn't always possible. Very high-level architecture may belong in a format where diagrams are more available, and the high-level design justification for the project itself doesn't have a great place in the code.
Similarly, I try to bake the documentation directly into the code. Eg (from a shell):
OPTION(*line=='[',"[EXPR]","the result of evaluating EXPR as a math expression (see -?m)")
{ ... }
qsh -?s | grep -F '$['
'$[EXPR]': the result of evaluating EXPR as a math expression (see -?m)
Without context, I have no idea what are you trying to imply or document with OPTION?
I can more or less figure out what the code does, but not why it does it. Here you should document why are you grepping for something
I can more or less figure out what the code does, but not why it does it. Here you should document why are you grepping for something
Developers have to be hands-on with documentation. Even if tech writers produce the final copy, devs need to be willing to provide sufficient notes to explain what needs to be documented.
I've worked (as a tech writer) at places where good documentation was part of the developer culture, like RethinkDB, and at places where documentation was seen as either something to get around to later because gosh we're just so busy now or something that tech writers can just magically pull out of their (ahem) hats with no guidance. I've been at one place where the same dev lead who literally walked out of a meeting with me saying that he didn't have time to follow the guidelines I'd written up at his request later berated me for not keeping documentation up to date. You know, documenting changes that I could only learn about if his team followed the guidelines he blew off.
Not that I'm still a little salty about it.
Anyway, tl;dr:
* If you want good documentation for your thing, you need to pitch in.
* Internal documentation matters.
* For developer tools, documentation is basically your UI. Don't skimp.
I've worked (as a tech writer) at places where good documentation was part of the developer culture, like RethinkDB, and at places where documentation was seen as either something to get around to later because gosh we're just so busy now or something that tech writers can just magically pull out of their (ahem) hats with no guidance. I've been at one place where the same dev lead who literally walked out of a meeting with me saying that he didn't have time to follow the guidelines I'd written up at his request later berated me for not keeping documentation up to date. You know, documenting changes that I could only learn about if his team followed the guidelines he blew off.
Not that I'm still a little salty about it.
Anyway, tl;dr:
* If you want good documentation for your thing, you need to pitch in.
* Internal documentation matters.
* For developer tools, documentation is basically your UI. Don't skimp.
For external-facing APIs and libraries, one reason I've found motivating to write documentation first is that you can "playtest" very quickly as a user/dev who comes in and starts wanting to use your thing.
If you (or another teammate) have to write a use-case snippet of code similar to how you want your thing to be used, based on your docs, you often find friction points and inconsistencies that should be streamlined and simplified.
It allows you to start iterating much faster than having to wait until you have a functioning product. It also often lets you avoid sunk cost fallacies where you run into that friction point later but don't want to go back and rewrite/redesign code to streamline the friction point.
If you (or another teammate) have to write a use-case snippet of code similar to how you want your thing to be used, based on your docs, you often find friction points and inconsistencies that should be streamlined and simplified.
It allows you to start iterating much faster than having to wait until you have a functioning product. It also often lets you avoid sunk cost fallacies where you run into that friction point later but don't want to go back and rewrite/redesign code to streamline the friction point.
I have a lot of sympathy with this argument. Ive seen some projects recently that were heavily influenced by the agile manfesto with very little documentation where I particularly miss the high level stuff and justifications for some key decisions. I cant helping thinking that this was not the intention of the authors. I loved Django for the quality of its documentation from the get go.
I worked with a team of 6 remote contractors for a year at my last company. They suffered greatly from a lack of documentation for our project. As it turned out, most people were using saved emails in Outlook as their own personal documentation store. Emails from years ago might describe how to set up a dev environment or deploy code. This left me with an easy solution: I started pasting emails into OneNote and touting them as official documentation.
There was low buy-in from other developers to contribute because many of them already had emails saved. After posting all of my own emails in OneNote, other developers did the same. Some even updated the documents with later threads that less of the organization was included on. If something was incorrect, it was better to have it written down and correct it in text than figure it out and spread the knowledge via word of mouth. It was also better to have poor email documentation than no documentation at all. Quality quickly improved on the poorer pages and the habits of some developers who were referring to old emails for documentation changed when they learned that new emails described better ways of doing things.
There was low buy-in from other developers to contribute because many of them already had emails saved. After posting all of my own emails in OneNote, other developers did the same. Some even updated the documents with later threads that less of the organization was included on. If something was incorrect, it was better to have it written down and correct it in text than figure it out and spread the knowledge via word of mouth. It was also better to have poor email documentation than no documentation at all. Quality quickly improved on the poorer pages and the habits of some developers who were referring to old emails for documentation changed when they learned that new emails described better ways of doing things.
> Emails from years ago might describe how to set up a dev environment or deploy code
In some kind of bizarro world that might be encapsulated in a script in version control of some kind instead of a word document in a document management system. Imagine that!
In some kind of bizarro world that might be encapsulated in a script in version control of some kind instead of a word document in a document management system. Imagine that!
I use zim desktop wiki myself - with the git plugin. Cross platform. Just works. Eventually people want to clone my git repo and read the raw notes I have. It can eventually be imported into a wiki or whatever.
OneNote is amazing.
Sadly they are trying to discontinue the old version that let you use a plain file share to distribute it.
Newer versions can only be used with cloud sync which scares a lot of important people (and for good reason).
Sadly they are trying to discontinue the old version that let you use a plain file share to distribute it.
Newer versions can only be used with cloud sync which scares a lot of important people (and for good reason).
I think reduction of friction can be beneficial here, elixir for example integrates documentation and tests - so you write you examples and tests right there, you can write gnarly tests with regular testing suite, but giving documentation code value I think increases inherent value of writing documentation for developers.
It depends on how you work.
I practically never read documentation. My first question when I start working on a project is often "where are the sources?".
There are exceptions of course, like external public APIs (ex: manpages), protocol definitions (ex: RFCs) and "black boxes". As a subcontractor, specifications are are useful too but mostly for legal reasons.
Maybe I trained myself to work like that after reading so much bad documentation.
I practically never read documentation. My first question when I start working on a project is often "where are the sources?".
There are exceptions of course, like external public APIs (ex: manpages), protocol definitions (ex: RFCs) and "black boxes". As a subcontractor, specifications are are useful too but mostly for legal reasons.
Maybe I trained myself to work like that after reading so much bad documentation.
> Maybe I trained myself to work like that after reading so much bad documentation.
Same experience here. The documentation is always wrong. Sometimes it's wrong in minor, obvious ways, sometimes it's wrong in subtle corner cases you won't hit for years, sometimes it bears no resemblence to reality[0], but it's always wrong.
The only way to be sure is to look at the code.
0: And then there's cases like the C and C++ standards, where it bears no resemblence to reality and people are actively twisting and sabotaging reality to fit, but that's a different problem.
Same experience here. The documentation is always wrong. Sometimes it's wrong in minor, obvious ways, sometimes it's wrong in subtle corner cases you won't hit for years, sometimes it bears no resemblence to reality[0], but it's always wrong.
The only way to be sure is to look at the code.
0: And then there's cases like the C and C++ standards, where it bears no resemblence to reality and people are actively twisting and sabotaging reality to fit, but that's a different problem.
The only documentation I read is the onboarding stuff. After that, I dive into the code
I wrote about this a while back, relating my experiences of spending months documenting an SRE operations team [1].
Since then I've learned that there's an optimum time to document, and it's not early in the project. That led me to look at the Cynefin framework [2], which prods you to classify where you are on the maturity scale of an activity, which in turn gives you a clearer idea of what to do next.
The corollary (and I've experienced this a few times since) is that it's a waste of time to document early in the maturity cycle, as much changes; too fast for your docs to keep pace. It's like imposing an enterprise HR function on a small startup - incongruous and likely to lead to friction.
Picking the right time to start taking documentation seriously is an interesting problem.
[1] https://zwischenzugs.com/2017/04/04/things-i-learned-managin...
[2] https://en.wikipedia.org/wiki/Cynefin_framework
Since then I've learned that there's an optimum time to document, and it's not early in the project. That led me to look at the Cynefin framework [2], which prods you to classify where you are on the maturity scale of an activity, which in turn gives you a clearer idea of what to do next.
The corollary (and I've experienced this a few times since) is that it's a waste of time to document early in the maturity cycle, as much changes; too fast for your docs to keep pace. It's like imposing an enterprise HR function on a small startup - incongruous and likely to lead to friction.
Picking the right time to start taking documentation seriously is an interesting problem.
[1] https://zwischenzugs.com/2017/04/04/things-i-learned-managin...
[2] https://en.wikipedia.org/wiki/Cynefin_framework
The agile manifesto warned against comprehensive documentation. I don't think many of today's developers have ever been in an organization that seriously tried to be comprehensive? In the old days, some computer software came with a shelf of books. Often they were quite badly written, but official.
I really enjoy documentation. Modern wikis make the process so simple that it is almost effortless. Even if nobody else reads the documentation, it acts as a great reference point for me when I need to quickly refresh my memory on something I did a while ago.
Good documentation is must and it must be source code. I use plenty of tools/langs for it - PlantUml, markdown, matjax, mkdocs, sphynx etc. Docs site should be among deliverables along with its CI/CD and automatic tests. One other important benefit of keeping it as a source code is that a team on a feature branch can see diffs on funspec and easily follow the iterations on it instead of looking at one gigantic Word file as usual which is anything but practical for quickly determining what was changed in previous few days.
The main problems appears to be:
- When to start documenting
- How to keep it up to date.
This means that there need to be some processes, some good tools and ideally executable documentation (ansible, jupiter...). For example, I like to keep readme.md in each folder describing the purpose of that folder and what files are expected to be in it, in what format etc. Then during build, I can simply concatenate this into technical docs appendix and/or link directly to appropriate readme when specific section needs details. This means that in technical/functional docs there is only high level picture that rarely changes, and hence is more up to date and stuff that changes often is closer to the source of change.
I also think that keeping ticket numbers inside README.md as a reference is good practice, and keeping tickets itself connected. So I need entry point into PM tool and then I can follow from there if I need even more details.
For architectural decisions that are not obvious from the tickets one can use ADR in repository (architectural decision records) that are great for on-boarding and reminding on why we took certain path.
Documentation is hard, its literally a full time job. Its usual I spend weeks on projects automating documentation. The latest setup I use puts everything in single docker image and I spin it on each new project on CI level to build static site [1]. I also have detailed Gitlab procedures on how to create tickets that export well to the changelog. Visual Studio Code team nailed this IMO.
[1]: https://github.com/majkinetor/mm-docs
The main problems appears to be:
- When to start documenting
- How to keep it up to date.
This means that there need to be some processes, some good tools and ideally executable documentation (ansible, jupiter...). For example, I like to keep readme.md in each folder describing the purpose of that folder and what files are expected to be in it, in what format etc. Then during build, I can simply concatenate this into technical docs appendix and/or link directly to appropriate readme when specific section needs details. This means that in technical/functional docs there is only high level picture that rarely changes, and hence is more up to date and stuff that changes often is closer to the source of change.
I also think that keeping ticket numbers inside README.md as a reference is good practice, and keeping tickets itself connected. So I need entry point into PM tool and then I can follow from there if I need even more details.
For architectural decisions that are not obvious from the tickets one can use ADR in repository (architectural decision records) that are great for on-boarding and reminding on why we took certain path.
Documentation is hard, its literally a full time job. Its usual I spend weeks on projects automating documentation. The latest setup I use puts everything in single docker image and I spin it on each new project on CI level to build static site [1]. I also have detailed Gitlab procedures on how to create tickets that export well to the changelog. Visual Studio Code team nailed this IMO.
[1]: https://github.com/majkinetor/mm-docs
> Docs site should be among deliverables along with its CI/CD and automatic tests.
100000000x yes. I was fortunate enough to have this ingrained in me at my first job out of college. Can't thank my old mentor enough.
100000000x yes. I was fortunate enough to have this ingrained in me at my first job out of college. Can't thank my old mentor enough.
I've also gotten better at writing comments, and write them faster because I have so much practice. If you don't do something that often, you're bound to suck at it, and that's true of people who think they're going to write the world's best comments and documentation as soon as their project is settled. The truth is that most programmers are terrible at comments because they treat them with such disdain and don't take the time to get better at it just as they got better at writing code.