Key points are not available for this paper at this time.
Multimedia instruction offers many potential advantages over traditional methods of instruction. Multimedia programs can interact with the learner, use graphic images, sound, and video, and keep track of progress. Students complete programs at their own pace while accessing material both at school and at home. Multimedia instruction can provide an interactive alternative to lectures and textbooks by quizzing the student over concepts as they are presented and requiring that the student think about the material before proceeding. While several studies have found that multimedia instruction can be more efficient by reducing instructor and classroom time, few have been able to show an increase in learning when compared with traditional methods of instruction. 1, 2, 3, 4, 5, 6 Santer and colleagues compared a multimedia textbook with a lecture presentation on the same material and found an increase in the post-test scores of the multimedia group, but no difference when they compared the multimedia group with a group using a printed textbook. 1 Studies comparing multimedia and traditional approaches to learning in the areas of psychology and computer science instruction suggest an improvement in students' performances using the multimedia versions. 7, 8 Thus, there is a need for well-designed studies to determine whether multimedia instruction more effectively facilitates students' learning—including medical students' learning—than do traditional methods. Multimedia instruction is particularly well suited to help students learn physical diagnosis. Sound, pictures, and movies augment the learning of examination skills and diagnosis findings by allowing students to hear heart and lung sounds, watch videos of physical examination procedures, and see more pictures of pathologic findings than can be included in a textbook or lecture. These visual and audio aids should increase students' recognition of these findings when encountered in patients. We wished to test whether a Web-based multimedia program using interactive learning and virtual reality would be more efficient and effective than traditional print-based self-study by medical students. To accomplish this, we designed a course on physical examination of the eye and ear. Using this material, we conducted a controlled study of first-year medical students to determine whether having students use a multimedia version of the course resulted in a change in the time spent with the material and an increase in knowledge gained when compared with having students study a printed version of the same material. Method Participants. All 126 first-year medical students at the University of Nebraska Medical Center College of Medicine were invited to participate in this study in 1999; of these, 121 volunteered. We obtained Institutional Review Board approval and participants gave informed consent. Multimedia Course. We designed two courses, one about the eye and the other about the ear, to help first-year medical students learn physical diagnosis skills and tests. To minimize contamination, the subject matter chosen represented important clinical skills that ordinarily would not be presented at this time and was not part of the regular first-year curriculum. Besides otoscopy and funduscopy, topics included the interpretation of audiograms and tympanograms, as well as recognition of acute otitis media, serous otitis media, papilledema, glaucoma, and diabetic retinopathy. Some of the topics were chosen because we predicted they could be more effectively taught multimedia presentations. We estimate the course took 300 hours to create at a cost of 300 in materials and 3, 000 in student labor. After developing the courses, we created both a multimedia and a printed version of the course materials. The multimedia version emphasized interactivity and included many pictures and Quick-Time virtual reality (QTVR) movies of the funduscopic examinations and otoscopy. These movies showed normal and pathologic views of the retina and tympanic membrane as they would be seen through an ophthalmoscope or otoscope. Students were asked to scan around the entire picture searching for pathology, just as they would when examining actual patients. In addition, the multimedia version led students on a defined path through the program by presenting a concept on each page and then requiring that the student respond to a question about the concept before proceeding. Correct answers allowed the student to proceed while students giving incorrect answers were provided with immediate feedback before continuing. The multimedia version differed from the printed version in that it included interactive questions and contained more color pictures. Study Design. The students took a pre-test before the courses and a post-test afterwards. The pre-test consisted of 20 multiple-choice questions about physical examination skills and findings related to the eye and ear. The class had already been divided into 12 small study sections. In order to provide two groups matched with regard to their knowledge of the subject matter, we divided the existing study sections into two groups matched on average pre-test score. Group A (n = 60) used the printed manual for the ear material and the multimedia program for the eye material. Group B (n = 61) did the reverse. Students could access the materials one week before the post-test. We used three methods to track the time spent. The computer logged access and time spent with the multimedia material. To use the printed version, students had to check it in and out of the library reference desk, creating a log of the time spent. In addition, we asked the students to keep their own records of the time spent. This was reported on a post-study questionnaire. As an incentive, the student with the top score in each of the two groups was awarded a 25 cash prize. The post-test was given in two sections. The first portion consisted of ten multiple-choice questions presented on the computer and included questions regarding virtual-reality simulations of fundoscopy and otoscopy. The second portion of the post-test, given on the following day, consisted of 40 multiple-choice questions, 20 on the eye and 20 on the ear. These questions included general fact-based questions, images, and case studies. All questions and images were different from those used in either the written or the multimedia version. Evaluation and Analysis. The level of the students' acceptance of the program was evaluated with a written survey. The students were asked which instructional method they preferred, how much time they had spent, and how they rated their learning using each method. Post-test scores were analyzed using multiple regression and ANCOVA, controlling for time spent on the multimedia or printed versions, section, pre-test scores, scores on the Medical College Admission Test (MCAT), and college grade-point average (GPA). Results To ensure that the two groups were equivalent, we compared them with regard to their members' MCAT sections' scores, undergraduate GPAs, and mean pre-test scores. The pre-test scores were similar, with group A correctly answering 8. 65 of 20 (43. 3%) and group B correctly answering 8. 52 of 20 (42. 6%). There was no significant difference between the groups' mean Verbal Reasoning MCAT scores, Physical Sciences MCAT scores, Biological Science MCAT scores, total undergraduate GPAs, and pre-test scores. Reliability for the paper-based portion of the post-test was 0. 69 using the Kuder Richardson formula 20. The students who used the multimedia version of the eye or ear course scored higher on the respective post-tests than did the students who used the printed version, when compared using univariate analysis (see Table 1). This higher achievement in the multimedia version persisted when pre-test scores, MCAT scores, and undergraduate GPAs were controlled for by entering those variables into multivariate analysis. Multiple regression analysis showed that both use of the multimedia version and the time spent were predictors of the eye course's post-test score (r2 = 0. 24). Only the use of the multimedia version (and not time spent) predicted the ear course's post-test score (r2 = 0. 11). Interestingly, the students who used the multimedia version did not score higher on the subset of questions dealing specifically with the virtual-reality simulations or on the computer-based subset. TABLE 1: Post-test Scores and Time Spent by Instruction Method, University of Nebraska College of Medicine, 1999*Students using the multimedia version spent more time on the material than did those using the printed version (see Table 1). However, an increase in time could have resulted in an increase in the post-test score. With this possibility in mind, we found that when time and pre-test score were controlled for using ANCOVA, students using the multimedia version still performed better than did those using the printed manual (p <. 001). The only correlation between time and post-test score occurred with the multimedia version of the eye information (r2 = 0. 61, p <. 0001). The results of the written survey showed that 78% of the students preferred the multimedia version to the printed version and were interested in using similar programs for other areas of physical diagnosis. Most students indicated that their learning had been more effective using the multimedia version, and stated that they had enjoyed the virtual-reality movies and interactivity. The students did not report any difficulty in accessing either the computer or the written version of the program. Discussion As described above, we found that students using an interactive multimedia program improved more than did those using a printed manual with the same content. The students spent more time using the multimedia version but also improved more, given the time spent. These findings indicate that the multimedia program was more effective. Which aspects of the program led to this increase in post-test score? Results showed that the increased time accounted for some, but not all, of the gain. The gain in knowledge could have been due to the increased time that students spent on the multimedia version, but this did not appear to be the case, since even after taking time into account, the multimedia program still showed a greater improvement. The virtual-reality movies may have contributed, but our results showed improvement in all parts of the post-test, not just those dealing with virtual reality. The interactive dialog probably played a large role in the program's effectiveness by encouraging the students to work through problems, inducing them to take more time on particular tasks and probably to give more attention to the material. The computer itself could have affected the results, but there was no indication that this was the case, since the multimedia group had a higher post-test score but not a higher score on only the computer-based questions. There are several limitations to the generalization of these results. We have not yet measured the long-term retention of the information, or whether the students who used the multimedia version perform better when examining patients. The latter activity may be best assessed using a performance-based assessment tool, such as an objective structured clinical examination. The multimedia version had more illustrations, but the results suggest this was not a major cause of the differences in achievement, since the scores were not better on the computer part or on the few questions that involved pictures or feature recognition. The ability to include more pictures is an inherent advantage of multimedia programs over textbooks, where the cost of color pictures generally precludes their inclusion. Finally, we studied first-year students at only one medical school, so the generalizability of these results in other settings and other courses should be confirmed. In our study, we selected two specific content areas that we thought were well suited to multimedia. But other areas of physical diagnosis may show a similar benefit when using multimedia as a learning resource, such as having the ability to listen to heart and lung sounds. Studies have already shown this to be of benefit in teaching cardiac ausculatory skills. 3 Is multimedia instruction more efficient in the medical school setting? Multimedia programs can be reused and offer flexible scheduling, but complex programs may take considerably more time to develop. Lyon found that a multimedia program reduced instructor time with no loss of student achievement. 6 In our study, the students spent more time with the multimedia version, and the time spent resulted in greater achievement, but this version required greater development time. Our experience suggests that whether a multimedia program results in gains in efficacy may depend on the nature of the subject and the learning mode it replaces. Conclusion Our study suggests that multimedia learning, incorporating interactivity and virtual reality, is more effective than traditional approaches to teaching the eye and ear sections of the physical examination. Learning was enhanced in all areas, not just those dealing with virtual reality or the multimedia. The implications are that more multimedia courses in physical diagnosis techniques should be developed and evaluated. Further study is needed to determine what aspects of multimedia learning are most effective and how well the results found here will apply to other areas of the curriculum.
GRUNDMAN et al. (Sun,) studied this question.