StackOverflow’s announcement to end their documentation beta was both disappointing and yet, not surprising. After a valiant attempt to tap into the wisdom of the crowd, StackOverflow discovered that good documentation is hard to write. As someone who’s read, written, and researched software documentation for a few decades (and with a dose of 20/20 hindsight), it was easy to see this coming in the assumptions they made or implied on their tour page and quoted here.
When we reviewed traditional documentation, two things were clear:
- It had to be based on assumptions It was usually written once, often by someone not even using the technology, so it was a guess at what to focus on.
- It didn’t prioritize good examples People learn best when they can see things demonstrated in actual code.
Let me break this down… (or you can just jump to my suggestions)
Traditional documentation had to be based on assumptions
Of course it did. Documentation written before an initial product release must be based on assumptions. Having written quite a few pages of such documentation, those assumptions, however, are not based on guesses. Initial product documentation is based on the same assumptions (or we could use their real name: research data) that were used to develop the product. Do they always hit their mark? No. But, it’s a chicken and egg situation. Developers/customers would like the whole chicken (all possible documentation) right from the start but, with a new product, having just barely enough documentation (i.e. just the egg) is really all that is necessary to get started.
The post-release documentation problem has been solved
Can initial product documentation be improved upon after it’s released? Definitely! Agile documentation does a pretty good job at that and the success of StackOverflow’s Q&A sites has demonstrated another way to fill in the gaps. Before the fact? It depends. I talk about one way in Audience Market and Product.
The practical reality is that documentation of an initial product release has to be just good enough to enable customers to get started. The real problem this observation is based on occurs when the initial documentation doesn’t change once it’s been published. If (or when) the documentation is abandoned after release is a business decision. However, if there’s a demand, post-release documentation has been traditionally addressed by third-party content, such as books, websites, and StackOverflow. I don’t see how adding another “documentation” site would change this.
Traditional documentation didn’t prioritize good examples
Back to the first point, initial product documentation is lucky to have ANY examples, so building in a prioritization scheme hardly seems important. As examples accumulate, however, such a scheme becomes important—and it seems that StackOverflow has already solved that problem.
Examples are necessary, but not sufficient
The notion that “People learn best when they can see things demonstrated in actual code” only holds in a limited set of learning scenarios. People learn how to do specific tasks by examples, yes. However, examples become limiting (or overwhelming) when you try to make them more general or illustrate more than a few concepts. To address those cases, technical writers use non-programming forms of communication such as words, pictures, schematic diagrams, and other, more abstract, forms of expressing ideas.
We need a better way to identify good content
As far as prioritization goes, if anything, technical documentation has a hard time enabling the identification of good content. Sure, there are rating schemes (stars, thumbs up/down, etc.) but technical documentation feedback schemes are severely inadequate. Often times, technical documentation topic ratings reflect readers’ feelings about the feature as much as their feelings about the documentation of the feature. Finding a way to identify a clear meaning of the (usually rare) feedback will help improve both the product and the documentation. Again, I’m not sure how concentrating documentation in StackOverflow’s site helps here, unless they are providing better feedback to readers, authors, and product developers.
I would imagine that the reason up/down-voting voting works well in Q&A and Example topics is that their use cases and learning goals are very specific—such as, “Did this help you?” This clarity of purpose and reader goal is not uniform across topics—even a topic as simple as an API reference topic has many use cases, of which Q&A is only one. (I would, however, argue that good reference topics are neither simple nor easy to write). Multiple reader goals make it difficult to attribute and understand the meaning of general feedback, such as “thumbs up.” More about readers goals.
Good documentation is hard to write
If anything, StackOverflow’s documentation experiment demonstrates that good documentation is hard to write. Large documentation sets are both hard to write and hard to organize. It’s hard to do when this is your job on which your livelihood depends. It’s no easier when is just something you’re interested in (only with less incentive to write it because your livelihood depends on doing something else).
When I studied open-source software documentation, the quality and content was all over the map. The only common thread I saw in open-source software documentation was that it was as good as it needed to be. In spite of a collection of studies that describe how bad software (API) documentation is and how its miserable quality hinders learning and use, the fact remains it’s usually as good as it needs to be—and no more. If there’s a market for improving the content, either in third-party documentation or answers on a Q&A site, those improvements will (eventually) appear. At the same time, like so many other things, if there’s no market or audience for the content, it won’t be written, or, if written, it won’t be maintained.
How to fix what’s really broken with documentation
I hope they learn from this and try again. StackOverflow has a huge following and could really make a dent in the problem. My suggestions for how Documentation v2.0 can add value to the documentation universe are:
- Look for ways to improve upon and fill in the gaps of existing and product documentation.
Try to identify and promote the good content, whether it’s on a company’s site, in a blog, or in a StackOverflow question. - Find a way to separate product-feature feedback from product-feature-documentation feedback.
This might be less ambiguous for a reader reviewing documentation on a non-product site than a reader on a product site. So maybe it’ll look something like “Yelp for documentation topics.” - Make sure it supports more than example topics
As mentioned in the Tearing Down the Structure of Documentation topic. - Remember that there is no one-size-fits-all solution to documentation.
Every software product is different. Each product solves a different problem, in a different way, for a different audience. V2.0 of the documentation should embrace this.