Experimental comparison of Web, electronic and mail survey technologies in operations management

Abstract With the growing acceptance of the Web (Internet) and electronic mail, it is no surprise that researchers are using an increasingly diverse set of survey technologies to gather data from managers. However, the effectiveness of these electronic technologies has not been rigorously assessed, especially for gathering data from establishment‐level surveys (i.e. firm‐ or plant‐level). To that end, a stratified sample of large and small, service and manufacturing firms was constructed, followed by random assignment to one of four survey technologies: mail, fax, PC disk‐by‐mail and Web‐page survey (combined with e‐mail notification). For each treatment, managers are queried about their use of forecasting characteristics, yielding a sample of 118 firms. Unfortunately, only a low percentage (34%) of firms and managers assigned to the Web technology treatment both reported access to e‐mail and were willing provide their e‐mail addresses; they tended to be large firms and from the service sector. Moreover, those that did offer e‐mail addresses were only about half as likely to respond to the Web‐based survey as those targeted by other survey technologies. However, Web, fax and disk‐by‐mail technologies yielded higher item completion rates than mail. Limited statistical evidence indicated that respondents using computer‐based survey technologies (i.e. Web or disk‐by‐mail) generally reported forecasting characteristics that are associated with firms exhibiting best practices. Thus, a multi‐technology survey approach using the Web and fax can yield a strong combination of benefits over a traditional mail survey.