Still buzzing about StackOverflow documentation

I don’t think I’ve ever seen a technical documentation project get so much attention (be it successful or not). The good news is at least people are talking.

But the talk is still looking at only symptoms. The conversations have been interesting but fall short of the root causes.

The documentation project kick-off post identified these problems with documentation (paraphrased here) that Documentation intended to fix:

  • Documentation is often an afterthought
  • Documentation is lacking in examples
  • Documentation is rarely complete

The kick-off described the problem, fundamentally, as a shortage of documentation and approached the solution by making it easier to add more documentation (articles and examples). Consider each of these points as discussed by some other technical writers.

Documentation is often an afterthought

Beth Aitman’s post agrees that this is a problem. I haven’t seen it that way.

After 20+ years of working on software and documentation, I can say with absolute certainty that sometimes it is, sometimes it’s not. Yet, I have not seen the point in the process where documentation is thought of to correlate to its utility. Nor does the documentation quality correlate to the product’s success. I’ve worked on projects that include documentation in the planning from the very start and seen it fall flat. I’ve worked on projects where documentation was whipped together towards the end of the project and have it work well. When the documentation is thought about isn’t the problem to solve, in and of itself. Being an afterthought isn’t a documentation problem–and it’s not a problem that is solved by more documentation. It’s a product management problem. Adopting Documentation-Driven product design, however, might help, but I don’t know if that’s a problem that StackOverflow wants to take on.

Documentation is lacking in examples

That could be. Again, I’ve seen it both ways. When they are lacking, they definitely should be added, but that decision needs to be made thoughtfully. Examples are not free and they are not a one-size-fits-all solution as Beth describes.

After years of writing a lot of documentation, reviewing a lot of feedback, and reading a lot of studies, I think the ideal situation for the user (in the voice of the customer) is to have examples that do everything they want their app to do.

The ideal scenario for the example-driven developer (in case you want to build it) is:

  1. Enter your app idea
  2. See a code sample or “example” that implements the app idea
  3. Copy/Paste code into your empty project file
  4. Build/Compile/Upload to app store (no need to test, right?)
  5. Watch revenue accumulate in your dev account

The farther your documentation and examples are from this ideal, the more your documentation sucks in the eyes of the Example-Driven Dev. Until this is possible (and, who knows…), do the best you can and don’t be distracted by non-specific claims that there are “not enough examples.”

You will never create enough examples to satisfy everyone.

Documentation is rarely complete

If the documentation is not “complete,” which is a subjective and moving target, the product is not complete. As a developer/customer, if that’s good enough for you (and it might be), go for it! Share what you learn on StackOverflow and help pave the road for those who follow. If it’s not good enough for you, use another product, fork it and fix it, or try to get the owner to realize that their product is incomplete and have them fix it.

This isn’t a documentation problem. It’s a product problem. The problem is that many people will use “partial products” to get a  head start in the marketplace. Enter at your own risk. I submit that that’s a product management decision and the customer’s choice–not a documentation problem; however, it is a problem that could be solved with more documentation.

The problem they were solving was not a documentation problem

The bulk of the problems listed in the documentation project kick-off are product problems that happen to appear (or not appear) in the documentation. They all trace back to product design and business decisions—often made at the very beginning of the project. Trying to fix them with documentation solutions on the end might help some, but it is not going to take you very far.

What does this bode for technical writers?

Tom Johnson looked at this from an interesting angle—that of how this experiment and its outcome might impact technical writers. I think his points on the documentation beta not being better enough than the competition (i.e. official product docs) to attract an audience are largely on the mark. On the other hand, I have a different take on a couple of points.

The crowd; can’t live with them, can’t live without them

I disagree with Tom that it’s evidence against crowdsourcing, rather it reinforces that crowds are not free and take cultivation . I’m certain that Jon Ericson (the community manager who posted the sunset message) gets that.

The existing Example Doc model is okay

I don’t think this announcement condemns or vindicates the location of examples in a topic. This has been experimented with several places and, at best, it depends on the audience and product. However, in most cases, it makes no practical difference to the audience. The important points are that examples must be meaningful, clear, and explained.

How to make it better?

I agree that the Q&A and Example topic format are excellent for the “I need an answer” use case. For the “How does this thing work?” or “What are the limitations, prerequisites, preconditions, etc. of an API?” or “What problem were they trying to do solve when they wrote this?” or “How does this fit in the larger architecture?” questions, it falls short. I like Tom’s chatbot/AI idea—it is just one step short of delivering the whole app as I described above. (Maybe we’re closer than we think?!)

All of this dodges the real bullet

Documentation is part of the customer experience. It used to annoy me as a writer when documentation feedback would contain product feedback, but that’s because I was missing the point.

To the customer, documentation is part of the product.

I think many of the problems StackOverflow’s documentation was trying to solve in the documentation are really reflections of product design decisions. For the documentation to work well, the product has to be designed well. I haven’t used the Stripe or Twilio APIs but their documentation is very clean and reflects an organized system design. (Bonus points for Twilio because I like their documentation feedback prompt, “Did the documentation on this page serve your need?’ which is followed by a graduated series of feedback prompts that culminate in a t-shirt offer, but that’s for another blog post).

Until you fix the underlying software, we’ll be stuck with an ongoing stream of documentation bandages.

Leave a Reply