For my PhD dissertation, I ran an unmoderated, online study to see how variations in page design and content of an API reference topic would affect how people found the information in the topic.
For the study, I solicited participants from several software development groups on LinkedIn and a few universities around the country. It’s definitely a convenience sample in that it’s not a statistically random sample, but it’s a pretty diverse one. Is it representative of my audience? I’m working on that. My suspicion, in the mean time, is that I’ve no reason to think it’s not, in that the people who read API documentation include a lot of people. For now, it’s representative enough.
A wide variety of people responded to the 750,000 or so software developers and people interested in software development that I contacted in one way or another. From those invitations (all in English, by the way), 436 people responded and 253 actually filled out enough of the survey to be useful. The 253 participants who completed the demographic survey and at least one of the tasks were from 29 different countries and reported speaking a total of 32 different native languages. Slightly less than half of the participants reported speaking English as their native language. After English, the top five non-English native languages in this group were: Hindi, Tamil, Telegu, Kannada, Spanish.
More than half of the participants didn’t speak English as their native language, but the vast majority of them should have no problem reading and understanding it. Of the 144 who didn’t speak English as their native language, 81% strongly agreed with the statement, “I can read, write, and speak English in a professional capacity or agreed or strongly agreed with the statement, “I can speak English as well as a native speaker.” So, while they are a very international group, the vast majority seem to speak English pretty well. The rest might need to resort to Google translate.
All of this supports the notion that not providing API documentation in any language other than English is not inconveniencing many developers—developers who respond to study invitations in English, at least. An interesting experiment might be to send this same survey to developers in other countries (India, China, Japan, LatAm, for starters) in their native languages to see how the responses vary.
So many studies. So little time.