Blog

June 28, 2016

Why Do Web Surveys Take Longer On Smartphones?

This is the title of a recent article by Mick Couper and Gregg Peterson that is the first to take on this research question with a comprehensive approach. While web surveys are increasingly being administered on mobile devices and researchers have been looking at differences in outcomes such as time of administration, there is very little known about why this occurs. Some reasons may have relatively mundane implications, such as transmission time, but others can imply increased risk of error in survey reports, such as difficulty in using the small screens or multitasking during the survey. Identification of the main factors leading to longer administration times on smartphones can be beneficial in improving survey designs for mobile devices.

Couper and Peterson identified five factors that could lead to differences in web survey administration times on PCs and smartphones:

1. Connection speed/quality. Different internet connections and the use of cell connections could lead to longer transmission times on smartphones, which will be magnified in surveys with numerous pages.
2. Screen size. Not all questions may be fully visible, requiring more scrolling. Question text may be smaller, requiring more effort to read.
3. Input method. Somewhat related to screen size, it may be difficult for some respondents to select or type responses on a small screen without a keyboard and mouse.
4. Prior research has found that mobile users are more likely to report being away from home or office and to report not being alone. If such distractions account for differences in administration time, it could have particularly concerning to survey practitioners.
5. Familiarity or comfort with device. This factor could be of particular interest when researchers decide whether respondents will be provided with a device, asked to use their own device, or given a choice.

They used an innovative approach, leveraging variations of the design used in three consecutive years of the Sustainability Cultural Indicators Program (SCIP) surveys of University of Michigan students, and variability across pages in the survey. They used multilevel linear mixed models to estimate the effect of device, respondent, and page characteristics on page times, and variability in page times.

Despite a design that was not tailored to measure the impact of the five factors, they were able to identify several key findings. Transmission time did not account for much of the differences in time between smartphones and PCs. However, this is the first study to identify scrolling as a major factor contributing to the longer administration times on smartphones. An interesting implication is that this may not necessarily be undesirable. Many software systems optimize for smartphones by turning grids into item-by-item questions, but no speed advantage was found for this approach.