I had the pleasure of joining the Boulder/Denver WriteTheDocumentarians at their meetup in Denver, last month. I presented a short talk on what I had learned about measuring the value of content that is typically produced by technical writing, which started an enlightening conversation with the group.
I’ve linked the slides and provided a brief narrative to go with them, here. Unfortunately, you had to be there to enjoy the conversation that followed–a good reason to not miss these events!
Measuring content value is a process, not a destination
For some, the idea of measuring the value of technical writing requires a shift in thinking. Measuring the value that content provides is just one step in the process of setting and evaluating content goals (see also Design Thinking). Without getting too philosophical, the first realization to make is that
You can’t measure value until you define what is valuable.
I’m in the process of writing one of several academic articles that my current profession (professor) demands of me. An essential part of the process is indulging in the diversions and distractions necessary to retain some sanity throughout the process. Today’s diversion was updating my global bibliography. Unfortunately, that idea turned out to have some depressing side effects, which I’m here to share with you.
It turns out that there’s a lot of research being done in how to automatically generate API documentation. Having written a lot of it and read a lot more, I can certainly understand the motivations. What I didn’t realize was from how many different directions the problem was being attacked. Someone even patented an idea for it (US 8819629 B2, in case you were wondering).
I published my first API around 1988 for a peripheral to the IBM PC in which the API consisted of software interrupts to MS-DOS. (A software interrupt is similar in function to a procedure call, but used for operating system and device driver functions. I didn’t write the documentation (at least not the published version), but a couple of co-workers and I wrote the interface.
Later, I want to say in 1997-ish, I wrote the API provided by the Performance Data Helper (PDH) dynamic link library (DLL) that shipped in Microsoft Windows NT 4.0 (IIRC). I didn’t write the documentation for this API either as I was still developing software at the time. A super technical writer did the heavy lifting of making sense out of the functions the PDH.DLL provided.
While this isn’t a particularly impressive resume of API development, it shows that APIs and I go way back. They are a very useful tool for solving many problems. Yet, over the past 30+ years that I’ve been using, developing, and documenting APIs, there hasn’t really been much written about their documentation–until recently.
In my informal count (i.e. what I could find on Google while having my morning coffee a few days ago). I came across 17 articles and blog posts on the topic in just the past seven years (full list at the end of the post).
I just launched TC Myths, my site in which I’ve collected all the myths about technical communication that I could find.
It started out as a quick lit. review, but the list grew more quickly than I expected (or imagined). I originally thought the list of myths would result in a duplicate of a similar list at UX Myths and then that would be the end of it. As it turns out, there is very little overlap. It also turns out that TC Myths are (or have been so far) much harder to research, compared to the UX Myths. The myths are also mysterious, which is why I’ve opened it up to comments and contributions.
I’m trying to understand why Technical Communication has (by my last count) over 70 different myths! Do we really need that many?
I’m delighted to see that the style guide includes, albeit below the fold, the advice to:
Remember that everything in this guide is a guideline, not a draconian rule.
Personally, I think this caveat should be on every page as part of the chrome, but at least it’s in the introduction. Unfortunately, my cursory review of the guide shows that the rules provide little context to help readers (in and out of Google) decide when would be good time to break a given rule and what the consequence or effect of that might be. But, this is not new. It has been a deficiency in technical communication guidelines and best practices that I’ve complained about for several years now. Maybe in v2?
The highlights are a good place to start and provide a short cheat sheet of rules that are relatively universal and, while they don’t provide any resources or background for why these are good practices, I’m familiar with research that supports most of the suggestions, but I’m an outlier.
If you’ve been following the preceding posts on measuring content, as the use-cases and customer journey paths start to become less funnel-shaped, this is about the point where whataboutism starts to occur.
In the post on measuring Tutorials, for example, I assert that “the customer’s goal in reading a tutorial is to accomplish something outside of the web,” making detecting and measuring their success difficult to do from within the topic. While the definition of a tutorial might make that seem like a pretty clear goal, that doesn’t make it immune to whataboutism.
Whataboutism can enter the discussion at this point in the form of “What about the people who come to the tutorial topic looking for a code sample to copy and paste? They don’t want to learn anything.” Or, “What about the executive who looks at the tutorial to see if it addresses a particular issue they care about?” Or, what about… You get the idea. From what I’ve seen, it’s easiest for whataboutism to enter the discussion when the goals are broad and vague and the data supporting the goals and their subsequent measurement are scarce.
(Does that sound like a content project to you?)
So, what can you do about the “what about…” cases?
In my previous post, Measuring your technical content – Part 1, I described some content goals and how those might be defined and measured for an introduction topic. I this post, I look at Getting Started topics.
Getting Started topics
If Introduction topics bring people in by telling them how your product can help them, Getting Started topics are where you show them how that works. Readers who come here from the Introduction topic will want to see some credible evidence that backs up the claims made in the Introduction topic and these topics are where you can demonstrate that.
Technical readers will also use this as the entry point into the technology, so there are at least two customer journey paths intersecting here.
One path will come to a conclusion here, moving from the Introduction page to see the value and then the Getting Started topic to see how it works
Another path starts from the Getting Started page (already understanding the value proposition of the product) and moving deeper into the technology to apply it to their specific case.
Because at least one of the customer journeys through the Getting Started topics are less funnel-shaped than for the Introduction topics (some are almost inverted funnels), it’s important to start with the goals and required instrumentation before writing so that you can design your page to provide the information that the customer needs for their goals as well as the data you’ll need to evaluate the page (your goal).
So, in that case, what how might you measure such a topic’s success?
This started out as a single post, but grew, and grew, and… Well, here’s the first installment.
After the last few posts, it would be easy to get the impression that I don’t like Google Analytics.
I just don’t like when it’s treated like it’s the only tool that can collect data about a web site—especially a non-funnel (informational) web site.
In this collection of posts, I’ll look at my favorite topic, API documentation, and how you might analyze it in the context of the customers’ (readers’) journeys. This analysis doesn’t have to apply only to API documentation, because it’s based on customers’ goals, which are more universal and, if you look carefully, you might see a customer goal that matches some of your content.
In my last post, I talk about how Google Analytics isn’t very helpful in providing meaningful data about technical or help content. It can’t answer questions like: Did it help the reader? Did they find it interesting? Could they accomplish their task/goal thanks to my content?What do readers really want? You know, simple questions like those.
While a little disappointing, that’s not what makes me sad.
What’s sad is that the charts on the dashboard have all the makings of dysfunctional communication. For example, the dashboard seems to tell me, “You’re not retaining readers over time.” But, it can’t, or it won’t, tell you why.