Your Open-Ended Survey Answers Are Thin Because Typing Is Hard

On a phone, the same open question took 125 seconds to answer instead of 88, and the answer came out 36% shorter. That was the result when Aigul Mavletova randomly assigned Russian panel members to answer by PC or by mobile phone. Thin open-ended survey responses are usually blamed on lazy respondents. We think they are better explained by the cost of typing, and that cost is a design problem you can fix.

Thin answers are a cost problem, not a character flaw

Every open answer costs the respondent something. They have to work out what you are asking, find something to say, and then physically produce the words. Closed questions skip the last two steps, which is why people answer them so readily.

Pew Research Center measures this on its American Trends Panel. Closed questions are skipped about 1% to 2% of the time, while open questions average about 18%Asare-Marfo, Pew Decoded, 14 October 2021. 92 open questions on more than 40 ATP surveys, September 2018 to July 2021.. The same analysis found that questions demanding more thought were skipped far more often than easy ones.

That gives you two levers. You can make the thinking easier, with better questions. And you can make the typing easier, with better input. Most survey advice covers the first and ignores the second. We think the second is where most of the length goes missing.

Phones make every word more expensive

The mechanical part is simple. A thumb on a glass keyboard is slow. Ruan and colleagues at Stanford and the University of Washington measured text entry on an iPhone in a lab. English speech input reached about 153 words per minute, against 52 for the keyboardRuan, Wobbrock, Liou, Ng and Landay, arXiv 1608.07323, 2016. Lab study transcribing short set phrases. English: 153 WPM speech vs 52 WPM keyboard. Mandarin: 123 vs 43.. That was copying short phrases, not composing answers, so treat it as the upper bound of the gap.

The survey experiments show what that slowness does to answers. In Mavletova's 2013 experiment, respondents were randomly assigned to PC or mobile. The question was "What does 'healthy food' mean for you?" Answers averaged 85 characters on a PC and 55 on a phone. Mobile respondents also took longer to write them.

A Spanish experiment points the same way. Revilla, Toninelli and Ochoa randomised Netquest panel members to PC or smartphone across two waves. They found significantly fewer characters typed on smartphones. Item nonresponse to the open questions did not differ. People still answered, they just said less.

Most of your respondents are now on that keyboard

Pew notes that about 60% of respondents in a typical American Trends Panel wave use a mobile deviceAsare-Marfo, Pew Decoded, 2021. Share of respondents on a mobile device in a typical ATP wave.. If you send a survey link by WhatsApp, SMS or a QR code, your share is likely higher. The open question you designed on a laptop is answered on a phone.

The keyboard is not the only cause

Phones also change context. Mavletova found almost 20% of mobile respondents answered while commuting or in cafes, against 3% of PC respondents. Noise and distraction thin answers too. And Pew's comparisons are not randomised, so some of the gap reflects who chooses a phone. The direction holds across studies. The exact size depends on your audience.

Who you lose when typing is hard

Short answers are one cost. Missing answers are another, and they are not spread evenly. Pew's 2023 analysis by Amina Dunn and Vianney Gómez found that median nonresponse to open questions was 15% for tablet or cellphone users and 11% for desktop or laptop usersDunn and Gómez, Pew Decoded, 7 March 2023. 30 open questions from 26 ATP surveys. Median nonresponse 15% tablet or cellphone vs 11% desktop or laptop..

Education showed a similar split. Adults with a high school diploma or less wrote 74 characters on average, against 107 for postgraduates, and were much less likely to answer at all. Women, younger adults, and Hispanic and Black adults also answered less often.

So the people best placed to tell you about friction are often the people the text box screens out. We think this is the strongest argument for treating input cost as a design problem. Thin answers do not just mean less detail. They mean a narrower set of voices. If your respondents include people who read better than they write, see our guide to survey tools for low-literacy respondents.

Six fixes, ranked by how much evidence backs them

Here is what the research supports, strongest first. "Strength" is our judgement of how consistent and direct the evidence is, not a formal grade.

Say why the answer matters

This is the cheapest fix with the best evidence. Anil Kumar Chaudhary and Glenn Israel randomised 1,348 Florida extension clients to 4 versions of one open question about improving the service. Adding a short importance statement raised item response from 57.7% to 76.0%. It also raised average length from 20.6 to 27.2 words.

An earlier series of web experiments by Smyth, Dillman and colleagues, published in Public Opinion Quarterly in 2009, reached the same conclusion. Any introduction that made answer quality salient improved responses, for early and late responders alike. Write one honest sentence: who reads the answers, and what changes because of them.

Size the box for the answer you want

People read the box as an instruction. A one-line box says "a few words". A larger box says "a few sentences". The same Florida experiment varied box size and prompt together.

A bigger box added words but did not change whether people answered. The statement did both. Combined, they gave the longest answers by a clear margin.

Do not overdo it. Zuell, Menold and Körber reported in 2015 that very large boxes increased skipping in a student web survey. We have read only the abstract of that study. Our default is a box of 4 to 6 lines for a "few sentences" question. In Would You Rather, the long text field lets you set between 2 and 12 rows, with 4 as the default.

Ask a narrower question

"What do you think of our service?" asks the respondent to choose a topic, then write. "What was the slowest part of getting set up?" gives them the topic. Pew's 2021 analysis found questions with high cognitive burden had median skip rates about 3 times those of low-burden ones. The worst offender is the generic closing box, which we take apart in why "any other comments?" gets ignored.

Ask open questions earlier

Galesic and Bosnjak randomised the order of question blocks in a 2009 web survey. Answers to questions placed later were faster, shorter and more uniform. If the open question is the one you care about, put it near the start, after one or two easy closed questions on the same topic.

Probe once, where it matters

A follow-up such as "Can you tell us more about that?" can deepen a thin first answer. But Holland and Christian found in 2009 that interactive probes helped mainly respondents already interested in the topic. Nonresponse to the probe was even higher than to the first question. Use one probe on your most important question, not a probe after every answer.

Offer voice, and expect fewer but longer answers

If typing is the cost, the obvious move is to remove the keyboard. Speaking is roughly 3 times faster than phone typing, and people narrate rather than compress when they talk. A 2024 randomised experiment found spoken answers about 2.5 times longer than typed ones. We covered that study and its drop-off problem in our analysis of voice survey answers, so we will not repeat it here.

Newer evidence is more cautious. Camille Landesvatter and Paul Bauer ran a smartphone experiment with 1,461 US respondents from Prolific, published in 2026. Voice answers were significantly longer on 5 of 9 questions, by 2.2 to 4.4 words, while only 47% of the voice group answered each question, against 99% for textLandesvatter and Bauer, Journal of Survey Statistics and Methodology, advance article 17 September 2026. Fielded December 2022 to January 2023. Text n = 800, voice n = 661. Nine open questions.. Longer, slightly more informative, and far fewer of them.

Willingness is split down the middle

Revilla, Couper and Ochoa asked Spanish panel members whether they would use voice input to answer open questions if offered. Their 2018 survey measured stated intent, not behaviour, but the split is instructive.

That split is the whole design argument. Roughly half your respondents would rather talk, and roughly half would rather type. A voice-only question loses the second half. A text-only question taxes the first. We would not run either on a cold audience. For more on how spoken length behaves, see how long voice answers actually run.

What a voice option looks like in practice

In Would You Rather, a voice answer field records in the phone browser, with no app, and you can set a minimum and maximum length in seconds. A switch on the same field lets respondents type instead. It is off by default, so turn it on for open questions where you want reach as well as depth. People can also answer in the language they think in, which we cover in collecting answers in Hindi, Tamil or any language.

Where the evidence is weak or conflicting

Not every study finds a device gap. Revilla and colleagues cite a 2014 Dutch study by Toepoel and Lugtig that found no difference between mobile and PC respondents in characters typed. We have not opened that paper. Devices and keyboards have also changed since Mavletova's experiment, in which 20% of mobile respondents used feature phones.

The Pew device figures are observational. People choose their device, and the ones who choose phones differ in age and education. Some of the length gap is who they are, not what they type on.

The fixes rest on few studies each. The importance statement result comes from one Florida public service survey, backed in direction by student experiments. The box size evidence conflicts on whether large boxes increase skipping. The voice evidence comes from online panels in Germany and the US, and the newest study found a much smaller length gain than the earlier one.

Finally, longer is not automatically better. Pew's authors found shorter answers were often substantively similar, though longer ones gave more clarity. Length is a proxy for detail. Read a sample of answers before you celebrate a word count.

A 20-minute rewrite of your open questions

You do not need a new tool to start. You need one pass through the survey you already send. Here is the order we would work in, cheapest first.

Then compare the next batch with the last on two numbers: the share of people who answered each open question, and the median answer length. If both rose, the typing tax was costing you more than you knew. For which question types cause people to quit, see our breakdown of form drop-off by question type.

References

1. Mavletova, A., Data quality in PC and mobile web surveys, Social Science Computer Review, 2013. Used for randomised PC vs mobile answer length, response time and context.
2. Dunn, A. and Gómez, V., Nonresponse rates on open-ended survey questions vary by demographic group, other factors, Pew Research Center, 2023. Used for answer length and nonresponse by device and education.
3. Asare-Marfo, D., Why do some open-ended survey questions result in higher item nonresponse rates than others?, Pew Research Center, 2021. Used for open vs closed nonresponse, burden effects and mobile share.
4. Revilla, M., Toninelli, D. and Ochoa, C., PCs versus smartphones in answering web surveys: does the device make a difference?, Survey Practice, 2016. Used for the Spanish device experiment and the Toepoel and Lugtig citation.
5. Ruan, S., Wobbrock, J. O., Liou, K., Ng, A. and Landay, J. A., Speech is 3x faster than typing for English and Mandarin text entry on mobile devices, arXiv, 2016. Used for speech vs keyboard speed.
6. Kumar Chaudhary, A. and Israel, G. D., Influence of importance statements and box size on response rate and response quality of open-ended questions, Journal of Rural Social Sciences, 2016. Used for the importance statement and box size results.
7. Landesvatter, C. and Bauer, P. C., Information content in text and voice response formats for open-ended survey questions, Journal of Survey Statistics and Methodology, 2026. Used for voice vs text length and item response.
8. Revilla, M., Couper, M. P. and Ochoa, C., Giving respondents voice? The feasibility of voice input for mobile web surveys, Survey Practice, 2018. Used for stated willingness to use voice input.
9. Smyth, J. D., Dillman, D. A., Christian, L. M. and McBride, M., Open-ended questions in web surveys, Public Opinion Quarterly 73(2), 2009, DOI 10.1093/poq/nfp029. Used for motivating introductions and box size (abstract only).
10. Galesic, M. and Bosnjak, M., Effects of questionnaire length on participation and indicators of response quality in a web survey, Public Opinion Quarterly 73(2), 2009. Used for question position (abstract only).
11. Holland, J. L. and Christian, L. M., The influence of topic interest and interactive probing on responses to open-ended questions in web surveys, Social Science Computer Review 27(2), 2009. Used for probing (abstract only).
12. Zuell, C., Menold, N. and Körber, S., The influence of the answer box size on item nonresponse to open-ended questions in a web survey, Social Science Computer Review 33(1), 2015. Used for large-box skipping (abstract only).

The weakest point in this source base: the randomised device evidence is old and from one Russian panel, and the newer device figures from Pew are observational, so the exact size of the phone penalty today is uncertain.