Best practice…for you?

Last week, I saw a post in LinkedIn about a “new” finding (from 2012) that “New research shows correlation between difficult to read fonts and content recall.” First, kudos for not confusing correlation and causation (although, the study was experimental and did prove a causal relationship), but the source article demonstrates an example of inappropriate generalization. To the point of this post, it also underscores the context-sensitive nature of content and how similar advice and best-practices should be tested in each specific context.

Hard to read, easy to recall?

The LinkedIn post refers to an article in the March 2012 issue of the Harvard Business Review. The HBR article starts out overgeneralizing by summarizing the finding of a small experiment as, “People recall what they’ve read better when it’s printed in smaller, less legible type.” This research was also picked up by Malcolm Gladwell’s David and Goliath, which has the effect of making it almost as true as the law of gravity.

Towards the end of the HBR article, the researcher tries to rein in the overgeneralizations by saying (emphasis mine), “Much of our research was done at a high-performing high school…It’s not clear how generalizable our findings are to low-performing schools or unmotivated students. …or perhaps people who are not even students? Again, kudos for trying. Further complicating the finding stated by the HBR article is that the study’s findings have not been reliably replicated in subsequent studies, other populations, or larger groups. I’m not discounting the researcher’s efforts, in fact, I agree with his observation that the conclusions don’t seem to be generalizable beyond the experiment’s scope.

Context is a high-order bit

All this reinforces the notion that when studying content and communication, context is a high-order bit1. As a high-order bit, ignoring it can have profound implications on the results. Any “best practice” or otherwise generalized advice should not be considered without including its contexts: the context in which it was derived and the context into which it will be applied.

This also reinforces the need to design content for testing–and to then test and analyze it.



1. In binary numbers, a high-order bit influences the result more than any and all of the other lower-order bits put together.

In-flight reading

Photo of in-flight reading material
The collection of in-flight reading material found in the seat back of a recent Alaska Airlines flight

I’m working on a paper for the HCII 2015 conference and thought of the reading material I saw on a recent airline flight. The paper discusses ways to identify the goals of different types of information-focused web content and how to measure how well the content is accomplishing those goals, so now I see this everywhere I look.

This is what occurred to me while I was staring at this collection of literature for several hours.

The Alaska Airlines magazine

It’s goal is to provide the reader with entertainment, er, I mean provide them with advertising, using entertainment as the bait. So how would you measure success? Interaction with the content, maybe? Coupon codes entered, products ordered from the web links, and other interactions traceable to the magazine. Pretty straightforward. Content can be compared to the corresponding advertisement, reader feedback, and the publisher can decide what’s working and what needs work.

The airsick bag

This isn’t really literature, but a good case of “if it’s not easy to use, it’s going to get messy.” I don’t think any amount of documentation could fix a poorly-designed air sickness bag.

The emergency procedures brochure

This is everyone’s favorite reading material, right? It’s goal is to provide important information and provide it in a way that’s easy to understand (quickly, in the worst-case scenario). This is a document that Alaska Airlines (and its passengers) hope to never need, but when it’s needed, it’s value will be immeasurable. How do you track that goal? User feedback? probably not. Survivor stories? Let’s hope not! Maybe usability testing?

The WiFi and the “Meals & Snacks” advertisements

Again, this is purely promotional material whose effectiveness can be tracked by sales. Like the magazine, this is not unfamiliar territory.

What’s this got to do with me?

As a writer of help content, I relate to the emergencies procedures brochure. Sometimes I don’t think anyone reads my content and frequently, Google analytics tends to agree with me. But, I know that in some cases, when those few people need to read it, that’s going to be the most important thing for them to read (if only for that moment). If I’ve done my job right, what I’ve written will save them time and money. I’ll never know that from looking at Google analytics, but, a usability test (even an informal or discount test) will tell me if a topic will be ready to save the day (moment) when it’s called upon to do so.

Back to the paper.

User research to the rescue!

A-Team-Logo As Hannibal of the A-Team would say, “I love it when a plan comes together!

I responded to this post in Quora from someone asking for advice on how to improve their website. There was lots of good, 1st-person, advice when I read it, so I suggested that they go to their target demographic ask them what they thought.

Have you tried usability testing it with some people from your target market? E.g. go to a local university with this [home page image] on a tablet, or even just a photo on a clipboard, and ask people a few questions:

a) What does this page inspire you to do? Tell me more about that. (after each question)
b) Where would you click?
c) What would you expect to find after you clicked?
d) Would you sign up for this? Why/why not?
e) What does “verified” mean to you?
f) Thank you! here’s a gift card for a coffee.

If you started after breakfast, I would imagine you’d have a much better sense of what you need to improve by lunch time.

(Read Quote of Bob Watson’s answer to What things can I improve on this home page? on Quora)

Later,  I read this comment to the post.

Bob, I  actually did usability testing today as you advised and got some real feedback from students. It really helped in understanding what would resonate with students.

I also want to do A/B testing to see which messaging converts better. Do you happen to know any resource (website)?

Thanks a lot for your feedback

Happy to help!