I had the pleasure of joining the Boulder/Denver WriteTheDocumentarians at their meetup in Denver, last month. I presented a short talk on what I had learned about measuring the value of content that is typically produced by technical writing, which started an enlightening conversation with the group.
I’ve linked the slides and provided a brief narrative to go with them, here. Unfortunately, you had to be there to enjoy the conversation that followed–a good reason to not miss these events!
Measuring content value is a process, not a destination
For some, the idea of measuring the value of technical writing requires a shift in thinking. Measuring the value that content provides is just one step in the process of setting and evaluating content goals (see also Design Thinking). Without getting too philosophical, the first realization to make is that
You can’t measure value until you define what is valuable.
Value, however, can take many shapes, and different people in your organization will likely define value differently, especially when it comes to content.
Identify and reconcile the different definitions of value
You’ll need to seek them out and make them visible. Invariably different stakeholders will see content’s value differently. You’ll need to find them, unearth them, in some cases, but find them, you must. The importance of this stage can’t be overstated. The peril of missing some values is that it will call into question all the ones you did find—making it momentarily uncomfortable for you to report your content’s value.
Fortunately, it’s a process so don’t worry, you’ll be sure to include the new values in the next iteration. (But, it will still be a bit awkward for a bit.)
Customer journeys can show you the way
Customer is used in a very general sense, here. In this context, customer is used to mean anyone who derives value from the customer. For example, your actual customers are definitely a customer of your content. Also customers (by this definition) are the stakeholders and your organization so consider how they each derive value from the content so that you can understand the content’s value from their perspective.
Most importantly, remember that different content can provide different types of value and its customers (readers) can have different goals. Don’t let this bog you down into all the possible cases. But, definitely consider the documented and likely cases.
Customer journeys can help you…
Think like a customer
While identifying the value, think like a customer. The customer doesn’t get value from your product when they buy it (the salesperson might, of course). The customer gets value from your product when it solves the customer’s need.
For content, especially help content, the customer’s goal is NOT to read your content, that a just means to an end. To the customer, the end is to solve the problem that led them to your content in the first place. That end probably exists outside of your content, and quite possibly might exist outside of the Internet!
Consider where the customers’ goals are and how your content fits into their journey toward achieving those goals. It is only from that perspective that you will be able to understand, and ultimately measure, the value your content provides from a customer’s point-of view.
Operationalize the goals
As the goals come into focus, identify ways to measure them. You might find that a goal, as it’s defined, is hard to measure. For example, “Help content should make people buy our product/service” could be a goal, but it would be very hard to measure, when you consider all the other influences in that process. You could, however, set as a goal, “Customers should have a positive experience with our content,” which identifies the measurable part that content plays in a customer’s journey to make a purchase.
At some point in this process, you should be able to identify the value these goals provide, although they might need some further revision to get to that point. But, if your goal is to measure value, at some point, you’ll need to understand that value.
This process can requires some tough calls. If you can’t identify the value of some part of the content (or the whole thing, in the worst-case), you’ll need to ask the hard question, “Does it really have any value to the organization?” It might, it might not. Be ready to answer this honestly.
Follow your customer’s journey
Unlike the path to filling the shopping cart of your favorite e-commerce site, help content makes up only part of the journey to a purchase decision—much of which takes place elsewhere, in places that might be hard to measure. So, you’ll need to find where that journey intersects places you can instrument. This might mean creative instrumentation or moving that journey, slightly, so that it can be measured (without disturbing it for the customer).
In a tutorial case, the customers’ goal might be to learn how to make the product work for their particular application, so how can you link the two events and then collect feedback? The specifics depend on the situation, but knowing that your goal is to find out how it went, can guide you as you create the tutorial. Remember, you can always ask them how it went.
As help content becomes less of an influencing factor in the customer’s success, it’s influence becomes harder to identify and measure. So, maybe you need to loosen your requirements on this data, (in the “you can’t know everything” sense, or find ways to aggregate information across the content set and collect more general data.
Knowing where the content fits in the customer’s journey to success and the role it plays can help guide you.
Measuring performance
After you’ve identified your goals in a way that can be measured, you can start measuring it. (This is the fun part!). How you measure the answer, depends on the question. Page views might answer the question, “how much content is being read,” but it can’t answer a question like, “How satisfied are they after reading it?” Use the right tool for the job. Don’t be afraid to ask a direct question, if you’re looking for a direct answer. The challenge for many content publishing systems is that feedback beyond that provided by the web server is not supported well. If that’s the case, look for alternatives and options.
Knowing the question you want answered goes a long ways towards finding the best way to ask it.
Now that you’ve identified the value, you know what it’s worth to get the information.
While collecting data, you’ll want to collect your performance data—the data that measures the performance that relate to your goals. You’ll probably also want to also collect some forensic data—data that is not particularly related to a goal, but can help you troubleshoot problems and conduct formative research periodically.
For help content, Google Analytics collects a lot of data that is mostly useful for formative research given how most help content supports goals that exist outside of the web (and are not tracked by Google Analytics).
Reporting performance
Before your data is going to help anyone (especially you, the writer trying to demonstrate value), your data need to be turned into information. This is the process of describing your data in ways that relate directly to the goals and values you identified above. You must make this translation and not rely on the stakeholders’ imaginations.
If you design your goals and measurements well, this process will be easy and the translation, minimal. If this process is challenging, review your goals and measurements to see how they can be adjusted.
Remember, this is a cycle
Think in cycles. If you don’t get it on this one, you’ll get it on the next one. If you’re just getting started, don’t expect it to be perfect after the first iteration.
It’s a process of learning what you don’t know you don’t know.