Web-based Learning

The first reports of Web-based medical education appear in 1992,1 building on 30 years of computer-assisted instruction. Proponents claimed computer-assisted instruction is superior to text-based, lecture, and traditional educational methods for reasons that include control by the learner over content, time, and place of learning2; enhancement of learning, reasoning, and efficiency3,4; and cost savings.5 Many studies that reported advantages contained methodologic flaws6 and reported advantages unrelated to computer-specific features.7 Web-based learning (WBL) represents a further evolution of computer-assisted instruction. Technical advantages of WBL include universal accessibility, ease in updating content, and hyperlink functions that permit cross-referencing to other resources.8 These technical advances, specifically hyperlink and searching capabilities, fit the constructivist learning theory, where learners search out and create their own knowledge bases. However, as was evident with computer-assisted instruction, potential advantages may not translate into significant improvements in educational outcomes. With the widespread adoption of WBL, it is critical that medical educators have evidence regarding its performance as a learning medium. This paper reviews the medical, dental, and nursing WBL evaluation literature to: (1) identify which facets of WBL have been evaluated, (2) describe the evaluation strategies used, (3) synthesize the findings, and (4) discuss educational implications and future research directions. Method We searched Medline and ERIC for “Computer-assisted Instruction” and “Internet or World Wide Web” and “Education, continuing or Education, dental, or Education, medical, or Education, nursing” from 1966 to January 2002, limited to English language. Having selected articles from their abstracts, two authors independently applied the criteria in Figure 1 to classify each article as descriptive or evaluative. If the abstract was unavailable or lacked sufficient information, the article was reviewed. When the two authors disagreed, the third author broke the tie.Figure 1: Literature review and selection of articles for review.We categorized the WBL articles by evaluation domains. Where possible, we divided studies into those evaluating features specific to the Web medium versus those specific to educational content. We determined percentages of descriptive and evaluative studies, identified evaluation domains, described evaluation strategies, and synthesized the findings. The small number of studies and heterogeneity of study designs and participants precluded combining the data by meta-analysis. Results and Discussion We identified 206 WBL papers between 1992 and 2001 in Medline and ERIC. As shown in Figure 1, 76 met our criteria for inclusion, 41 (59%) being descriptive and 35 (46%) evaluative. The two authors agreed on this classification for 71 of 76 papers (kappa = 0.869, p < .001). In the five case of disagreement, the third author classified one study as descriptive and four as evaluative. From the 35 evaluation papers, we identified four domains: knowledge gains, learner attitudes, learning efficiency, and program cost. We divided studies of learner attitudes into those evaluating Web-specific and content-specific attributes. We could not similarly divide studies measuring knowledge gains due to multiple confounding educational influences in all but two studies.9,10Table 1 depicts the evaluation domains, keywords used to describe the focus of the investigation, and the number of studies in each category.TABLE 1: Keywords and Numbers of Studies in Evaluation Domains of 31 Web-based Learning StudiesDomain 1: Studies Evaluating Knowledge Gains Twenty studies evaluated knowledge gains with WBL interventions. Eighteen measured changes in multiple-choice test scores; one used a written case analysis; and one, a multiple-choice test plus a standardized patient interview. Research methods included pretest/post-test self-controlled studies, self-selected controlled studies, assigned (non-randomized) controlled studies, assigned crossover trials, and randomized controlled trials. The most common research design was a pretest/posttest self-controlled study, but there were several randomized controlled trials (see Appendix Table A). The pretest/posttest studies provided strong evidence that WBL interventions resulted in knowledge gains in medical students, practicing physicians, and dentists.11,12,13,14,15 However, studies without a control group cannot separate Web-specific from content-specific learning gains.Table: APPENDIX TABLE A. Studies Evaluating Knowledge Gains with WBL Interventions, Classified by Study DesignStudies where a WBL group and a control group received identical educational content allowed identification of learning gains attributable specifically to the Web medium. Studies that met this criterion demonstrated no difference in knowledge gains between groups.16,17,18,19 As the research designs strengthened, results confirmed WBL was comparable but not superior to other educational methods. One assigned crossover study showed no difference in scores between the WBL and slide/tape groups.10 In randomized controlled trials, authors found WBL superior to no educational method20,21 and equivalent to text-based methods.9 Of three studies of traditional courses with and without Web enhancement, one reported enhanced learning and two did not.22,23,24 In summary, WBL improved posttest scores on multiple-choice tests but did not outperform other educational methods. Domain 2: Learners' Attitudes We categorized studies of learners' attitudes into Web-specific and content-specific. Web-specific studies were (1) comparisons between WBL and other methods, (2) predictors of satisfaction with technology, (3) patterns of Web use, and (4) Web-specific enhancements of self-directed learning or learning stimulation (see Appendix Table B). Content-specific studies were evaluations of (1) courses, (2) predictors of satisfaction with courses, and (3) self-reported changes in effectiveness of learning and confidence (see Appendix Table C).Table: APPENDIX TABLE B. Web-based Learning Studies Reporting Evaluation of Media-specific AttributesTable: APPENDIX TABLE B. ContinuedTable: APPENDIX TABLE C. Web-based Learning Studies Reporting Evaluation of Educational Design FeaturesWeb-specific attributes Do learners prefer WBL to other educational methods? Several studies assessed learners' preference for WBL and their desire to use WBL again (see Appendix Table B). Most learners planned to use WBL again12,15,21 and preferred WBL to continuing medical education conferences, lectures, video, audiotapes, journals, or text-books.17,23 One study compared WBL with slide/tape and reported that 71% of students preferred the WBL.10 A randomized controlled trial reported higher satisfaction with WBL compared with print materials, on a scale of 5 to 20 (means: WBL = 17, text = 15, p < .001).9 Many of these studies introduced selection bias by recruiting participants over the Internet. Nonetheless, two strong studies demonstrate preferences for WBL over slide/tape and text-based materials.9,10 What predicts learners' satisfaction with Web technology? Properties specific to WBL include Web-site accessibility, navigation, and attractiveness. Studies that evaluated these features universally reported high learner satisfaction.10,12,16,21,23,26,29 The main predictor of satisfaction with WBL was download speed. One cross-sectional observational study reported that students who perceived fast download time gave above-average course ratings more often than did students who perceived slow download time (OR 4.25), and concluded that download speed was as important to learners' satisfaction as content.30 Four additional articles reported slow download speed as WBL's major disadvantage and most significant barrier to learning for graduate nursing, dental, radiology, and medical students.10,24,27,31 How do learners react to asynchronous interactions with faculty and peers? There are positive and negative reports of the asynchronous interactions with instructors and peers typical of WBL.21,32,33 One study of a Web-based nursing course reported that learners were more likely to work together and felt more comfortable disagreeing with the instructor or asking awkward questions.34 However, a qualitative study reported equal numbers of positive and negative comments regarding asynchronous interaction with instructors.21 There is insufficient evidence to determine when asynchronous communication benefits or adversely affects learning. What is known about patterns of Web use in WBL? Patterns of Web use were determined from learners' self-reports or from server statistics. Information from server statistics indicated higher use immediately before an examination and up to a 20% discrepancy from self-reported use.35 Self-reported use varied extensively between studies, with little clarification or explanation.27,36,37 For example, in one randomized controlled trial, 72% of the Web-based group reported never using the Web materials, citing lack of time as the major barrier.27 Does WBL enhance or stimulate learning or encourage learners to look for answers? There are mixed reports on WBL in these areas. In one study, only 50% of learners believed the Web-based experience enhanced their learning.24 Another study reported increased stimulation to learn in a WBL program versus a traditional course. However, this study was limited by a high crossover between groups—70% of WBL students also chose to attend lectures.18 A third study of nursing students reported that 85% of graduate and 28% of undergraduate students would search for answers personally rather than asking an instructor because of how the course used the Web.34 This suggests that WBL may stimulate learning differently depending on educational level. It is unclear whether WBL outperforms other media by enhancing or stimulating learning or encouraging learners to seek out answers. Content-specific features How do learners rate WBL courses? Many authors reported positive evaluations of WBL programs,13,26,31 but most studies comparing WBL with other methods found similarly high ratings for both modalities.17,27,32 However, one randomized controlled trial reported higher overall evaluations from students in the WBL group compared with a text-based group.9 Which design features predict satisfaction with an educational course? In a correlation model, sound instructional technique was the only variable that contributed to the variance in students' satisfaction with their learning in a WBL course.38 Variables without effect included previous experience with WBL, self-rated competence with technology, frequency of communication between classes, learners' age, remote group size, technology, and course management. The authors concluded that sound pedagogy is more important that technology in determining learners' satisfaction. Do WBL programs improve learners' confidence? Studies that evaluated learners' self-reported confidence or competence reported improvements after any structured educational intervention. WBL programs achieved changes similar to those attained with other methods.11,13,15,21,28,37 There is strong evidence that well-designed WBL programs can improve learners' confidence, similarly to other educational methods. Domain 3: Studies Evaluating Efficiency of Learning Only two studies evaluated changes in learning efficiency. One well-designed randomized controlled trial comparing WBL with text-based learning reported that learners achieved equivalent test scores with shorter study times using WBL materials (27 minutes versus 38.5 minutes). The authors calculated efficiency scores (median improvement in score per hour with 95% confidence interval) of 8.6 (7.1–11.7) for WBL and 6.7 (5.9–8.1) for text-based learning (p = .04).9 However, the second study reported conflicting findings, with only ten of 32 students perceiving that WBL was more efficient than studying a textbook.31 Learning efficiency needs further study in well-designed trials. Domain 4: Studies Evaluating Costs of WBL Programs Only one 1995 retrospective study evaluated the cost of WBL. The authors calculated lower direct and indirect (distribution) costs for WBL compared with text. However, they did not consider the educational design costs for either modality and assumed that hardware, software, and other equipment were available, and that the institution had a commitment to technology.39 Based on one paper, cost saving in the printing and distribution of materials is a potential advantage of WBL, but further study is needed to determine whether lower distribution costs offset the costs of technical support, and whether WBL saves or costs money in terms of faculty time. Limitations This study reviewed only medical, dental, and nursing literature, although there are many publications on WBL in other higher education fields. The authors limited the search to these health professions because they share similar undergraduate, postgraduate, and continuing educational methods as well as many content areas that differ from those in other types of higher education. With the trend towards interdisciplinary education, many future WBL programs will target multiple health professions. Nevertheless, this wide variety of learners at varying educational levels in different health professions might possess different motivations or learning biases that could confound the results of our review. Another potential weakness of the review is missed studies. There is no Medline heading for WBL. Instead, the authors searched using the keywords “Internet” or “World Wide Web,” which may have decreased the number of studies located. Conclusions Despite the rush to embrace WBL, it does not address all the challenges of medical education. It is a valuable addition to our educational armory, but it does not replace traditional methods such as text, lectures, small-group discussion, or problem-based learning. Educators still must define WBL's unique educational contribution. Evaluation of WBL is in its infancy. Although most learners welcome WBL (provided that download speed is fast), and give high satisfaction ratings, there is no evidence that students learn more from Web-based programs than by traditional methods. Students may learn more efficiently, but there is minimal information about the relative costs of WBL programs. Finally, curriculum development and instructional design are no less important for Web-based educational interventions than for other media. Educators must recognize that poorly designed educational programs or materials are not improved by being presented on a Web page. Educational Implications Medical educators must use well-designed curricula regardless of the method of delivery. We did not find Web-based programs to be superior to traditional methods in terms of gains in learning or learners' satisfaction. We recommend that educators tailor their teaching media to learners' needs rather than assume that WBL is intrinsically superior. When designing a Web-based educational program, educators should create materials that load quickly from all types of Internet connections. Directions for Future Research Research into WBL is in the early stages and many research questions remain unanswered. Which learners benefit most from WBL? Can WBL increase learning efficiency? Are there cost savings compared with other educational methods? Does WBL enhance knowledge using other evaluation methods besides multiple-choice tests? How can interactions between learners and instructors be fostered and preserved with WBL? Does WBL as a second modality add to the educational experience? Finally, investigators may wish to design trials that compare the strengths and weaknesses of educational methods rather than establish the superiority of one medium over another. Such trials would clarify the most appropriate uses of WBL in medical education.