Effectiveness of Online Textbooks vs. Interactive Web-Native Content

Abstract Efficacy of Static Web Content Plus Tool vs. Interactive Web-Native ContentStatic web content plus tool utilizes text and pictures for the core explanations (like atraditional textbook), then adds a tool, such as a programming environment alongside thecontent. Interactive web-native content utilizes less text and pictures, and instead utilizesanimations, responsive question sets, and interactive exercises for the core explanations.The objective of this study was to determine the efficacy of static web content plus a toolcompared to interactive web-native content.We compared the two content styles with a lesson on basic output in C++. We chose onelesson from Zyante's Programming in C++ (zyante.com) offering as the interactive web-native textbook, and one lesson from Horstmann's Big C++, the leading textbook forC++, as the static web content. The Zyante lesson had 769 words of explanatory text,including code examples, and 7 interactive elements, including 1 animation, 3 embeddedprogramming exercises, and 3 questions sets, as well as 1 three minute video. The BigC++ lesson had 1,236 words for explanatory text, including code examples, and an easy-to-use programming environment alongside the lesson. We compared the concepts taughtin each lesson and removed non-overlapping concepts whenever possible. Also, all quizquestions focused on a single concept that was taught in both lessons.We conducted a study during the first week of the Fall quarter, prior to any lecture, lab,or assignment on C++, with participants who were enrolled in CS1 in C++. Theparticipants performed 5 tasks in order: background survey, pre-lesson quiz (4 questions),lesson in C++, post-lesson quiz (11 questions with 4 from the pre-lesson quiz), thenfollow-up survey. The lesson style was randomly assigned. The participants were notgiven a time limit during the study.An improvement score is a participant's post-lesson minus pre-lesson quiz scores.Overall, the average improvement score was 16% higher (p-value = 0.016) forparticipants given the interactive web-native lesson (6.4 out of 11 improvement score)than the static web lesson (5.5 out of 11 score). Moreover, for the participants who scoredin the lower-quartile of the pre-lesson quiz (least-prepared participants), the averageimprovement score was 64% higher (p-value < 0.001) for the interactive web-nativelesson (7.4 out of 11 score) than the static web lesson plus tool (4.5 out of 11 score).The significant improvement of the least-prepared participants may be cause by thehigher (p-value = 0.055) self-reported engagement with the interactive web-native lesson(5.3 out of 6 engagement score) than the static web lesson plus tool (4.8 out of 6 score).Also, participants chose to spend more time (p-value < 0.001) with the interactive web-native lesson (17.5 minutes) than the static web lesson plus tool (9.4 minutes). Oneparticipant with the interactive web-native lesson wrote: “I can learn by practicing andauto-check the answer right away ... the activities are fun and helpful.”The interactive web-native lesson was significantly more effective than the static weblesson plus tool for improving quiz scores, especially for the least-prepared students.