Computer-Assisted Assessment in Higher Education, by Sally Brown, Phil Race, and Joanna Bull (Eds. ), (London: Kogan Page, 1999, 224 pp. , 29. 95) The issue of assessment is obviously important in all areas of higher educational practice. In instructional technology and distance education it is not only important to the course instruction itself, it is (or should be) an important component in the curriculum content as well. Consequently, texts like this are important to both the delivery and the study of distance education. At one time, computer-assisted assessment was a very special term, pertaining to special circumstances, resources, and practices. Currently, however, it is hard to imagine any assessment activity that does not involve at least some form of computer utilization in developing, delivering, or collecting the assessment materials. This is especially true in distance education formats as the computer must be an integral part of the instructional process for both the instructor and the student. The structure of this text helps readers relate to the potential value and issues associated with the incorporation of the computer-assisted assessment (CAA) content to whatever extent they plan to utilize computer components in their own assessment activities. While assessment is so important in instructional practice, formal measurement and evaluation courses are often not an integral part of today's programs for educators or instructional developers. Consequently, texts such as this will prove quite valuable for faculty in developing their assessment materials and practices. While the text is not focused specifically on distance education, much of the content can be easily applied or adapted to assessment practices in this arena. This text would also be useful as an adjunct resource in instructional theory or resource development classes. The text's introduction highlights the trends and integral incorporation of technologies into today's educational environment by cautioning the reader that many students “have undertaken much of their learning with a computer keyboard, monitor screen and printer” and that it may not be sound educational practice to test these students in a “formal, silent exam room, able to communicate only by hand written means. ” This is a reversal of general earlier cautions that we should not use computer-based assessment materials, as students may not be familiar with such technologies and factors such as “computer anxiety” may have a negative influence on their performance. Technologies are obviously a major component in today's educational practice. Consequently, educators must be aware of the practices and issues presented in this text. The authors summarize the collective contents of the contributor's chapters as illustrations of “reducing the load on hard-pressed lecturers and teachers, ” “providing students with detailed formative feedback, ” and “bringing the assessment culture experienced by students closer to the learning environments with which they are familiar and confident. ” To facilitate comprehension of the content, the chapters are organized into three functional sections: Pragmatics and Practicalities of CAA, Using CAA for Formative Assessment, and Learning from Experience. The contributors consist of educators and instructional designers from a wide variety of academic departments, representing a number of universities and institutes located primarily in the UK. This section, identified by the editors as the major focus of the text, consists of eight chapters that present issues related to planning and implementing computer-assisted assessment. The first chapter, “Pragmatic issues when integrating technology into the assessment of students, ” discusses the relative benefits of CAA compared to paper-based techniques. The authors then present a series of arguments or reasons to resist this approach that are typically offered by faculty, teachers, and administrators, followed by recommendations for overcoming such resistance. The “Technical and security issues” chapter presents such issues in a range of computer-based components and compares them to traditional approaches. Reliability and validity concerns are presented in relation to the processes of test construction, storage, selection, and delivery. The next chapter, “Effective question design, ” could apply to any testing approach. Good testing and assessment practice is necessary in any educational setting, and this chapter provides a classic review of issues related to good question development and refinement. “Automated online tutorials: new formats for assessment on the WWW” reviews aspects of formative and summative tutorials designed primarily for Web-based delivery. The relative merits and resources of such materials are described. The next chapter, “Computer anxiety in students: should computer-based assessment be used at all? ”, cautions us about potential negative aspects of the movement to CAA related to student reactions and factors such as testing bias. Suggestions for overcoming such problems are offered. Again, the interplay between classic assessment issues and special components resulting from CAA (both good and bad) is apparent. “Validating formative and summative assessment” reminds us that not only are we assessing student performance and growth, but also we are evaluating our teaching effectiveness. Classic assessment issues of reliability and validity are presented with interpretation related to special aspects of technology incorporation. “Open-access randomly generated tests: assessment to drive learning” provides a conceptual and technical introduction to managing and utilizing computer-based question banks. Issues of security are discussed, as well as implications for relating questions and tests to the range of desired learning outcomes. A parallel technical treatment is provided in “Automatic test generation from a database. ” This chapter covers topics related to test construction and implementation. Examples of implementation in mathematics classes are provided along with comparisons of the views of both lecturers and students. As the title implies, this section's chapters focus on the use of computer-assisted assessment components and techniques as they apply to formative evaluation of student performance. As pointed out in the lead chapter, CAA is particularly useful in this capacity due to its propensity for instantaneous processing of responses and its ability to supply repeatable, immediate feedback. The benefits of large-scale, anonymous peer review and feedback are discussed in the second chapter. The nature of computer data storage and transfer techniques makes this feature particularly efficient and desirable in many teaching situations. The section begins with “Issues and impacts of using computer-based assessments (CBAs) for formative assessment. ” This chapter is oriented to the changes in higher education, which are resulting in a “shifting of responsibility for learning to students” and the related change in the role of faculty to “provision of opportunity for this learning. ” Formative assessment is presented as part of the learning process to provide feedback to students, guide student efforts, diagnose problems, help direct teaching activities, encourage students, and provide students with experience of assessment methods. A variety of topics related to the mechanics and functionality of testing components (e. g. , repeatability, immediate feedback, reliability, diversity of assessment portfolios) are presented within this framework. The next chapter, “Computer-assisted peer review, ” covers issues and practices related to the “horizontal flow in learning” utilizing multi-reviewer anonymous peer review methods. The rationale for such practices, alternative applied models, and insights to the various technical advantages associated with today's hardware and software, are provided. The use of a specific technical tool is presented in “Increasing learner responsibility and support with the aid of adaptive formative assessment using QM designer software” as an illustration of the use of CAA for formative assessment in an applied setting. The system described is an authoring environment that provides automated delivery and scoring of objective tests. This system has evolved from use as a summative instrument to that of a formative, self-assessment instrument. Examples of the developmental and delivery environments are provided to illustrate the associated issues and application to typical course content. The last chapter in this section, “Computer managed learning as an aid to formative assessment in higher education, ” focuses on the advantages provided by computer managed learning systems that provide assessment and student tracking functions. A brief review of formative and summative assessment is provided, followed by the results of two studies that focus on the influence of such applications on student performance. Alternative interpretations of the positive results in these studies are discussed. The final section presents a series of seven in-depth descriptions presenting a variety of real-world applications of the principles and techniques described in the text. These descriptions, presented by the editors as “how we did it” and “what we found” accounts, are intended to aid readers in fully understanding the principles and applying them in their own teaching situations. The first chapter, “Optical mark reading (OMR) as CAA: experiences at Loughborough University, ” presents an older technology integrated with today's computer technologies to optimize the deliverability of tests and the subsequent utilization of the resulting data in an efficient and meaningful manner. This chapter provides step-by-step guidance in the components and operations involved in such an application. While many universities already provide such resources and functions to faculty, the information presented will help individuals relate better to such operations and their potential incorporation into actual course activities. The utilization of local, departmental assessment and OMR capability would be enhanced, which may make the integration more likely and more adaptable to individual course or instructor needs. “The computer-assisted assessment of practical IT skills” is focused on the benefits of CAA in the teaching of information technology (IT) skills and knowledge. The chapter provides a review of the types of assessment for appropriate IT and describes the model developed by the author. Illustrations of the implementation of the model in teaching are provided with examples of the types of data and reports that are generated. The next chapter, “Assessment in a cognitive apprenticeship-based learning environment: potential and pitfalls, ” overviews the use of a tutoring system for teaching complex content and stresses features of both formative and summative evaluation. Their recommendation to avoid “merely tagging assessment functionality on to a system designed for learning” is well put. While this chapter is in section three, it focuses on a variety of theoretical issues more than on the “how we did it” theme. Their discussion, however, is still very relevant and useful to educators interested in formative/summative assessment or the design of instructional systems. “Online assessment: creating communities and opportunities” presents a series of case studies that illustrate many current online assessment practices. The authors make a strong case for educators to reconceptualize their assessment practices to include the opportunities offered by today's information and communication technologies (ICT) not only in off-campus teaching situations but in on-campus teaching as well. At the same time, they caution educators not to lose the inherent relative advantages of face-to-face, open, and distance learning. They present four main pedagogical principles to be considered in such instructional activities: conferencing, reflection, collaboration, and a global community. A series of applied project examples are used in “Delivering automated innovative assessment in a varied curriculum” to illustrate the author's views of principles and benefits of automated computer-based assessment applications in higher education. Parallels to traditional test and measurement concepts are offered and presented in the context of real-world applications in assessment programs for courses focused on construction education. A strong case is made for relating the assessment techniques to the learning conditions and learning outcomes. Benefits are summarized in relation to the automated innovative assessment strategies enhancement of “student learning, assessment, and feedback” and the maintenance of learning quality while reducing tutor input. “A structured approach to mass implementation of computer-based assessment (CBA) ” begins with a rationale for the development of computer-based assessment systems based in the identified needs of the academic staff. Enhancement of the quality of students' learning experiences is also stressed. Both summative and formative assessment components are included and implementation is illustrated with a wide variety of university courses. The stages of designing and implementing such an assessment program in an applied setting are identified as demonstration of the potential, implementation of a pilot, expansion of the number of departments involved, and moving from a departmental system to a centralized university-wide system. Two specific examples are then provided to illustrate this process. The final illustration, “Student perceptions of the learning benefits of computer-assisted assessment: a case study in electronic engineering, ” presents an applied computer-based assessment project focused more specifically on learning outcome. The project presented utilized generic software for both summative and formative assessment purposes. The content related to the application of mathematical calculation and manipulation interactions and was designed to encourage students to develop and apply their understanding to problems. The author describes the use of an Internet-administered phase for practice assessment exercises to help integrate assessment into the learning process and give students more control over their own learning. While many course structure recommendations related more to learning activities, the assessment strategies were considered to reduce student anxiety, clarify course content and goals, and provide necessary feedback during the learning/assessment processes. Extensive elaboration on these outcomes and student reactions/insights are provided. In relation to the development of instructional designers and developers, this text would be an excellent adjunct to a variety of applied courses in the theory and practice of instruction and assessment in higher education. Although rarely indicated as such, most of the illustrations supplied are much in line with traditional learning and measurement theories. If readers have a strong background in measurement and evaluation, this text will help them extend this knowledge into educational practices that utilize today's technologies for effective assessment of learning outcomes. A more basic introduction, such as a traditional testing and measurement text, is recommended for those not yet grounded in the associated theory and practices followed by texts such as this which provide real world integration and application. On the other hand, this text can also provide all educators with a good introduction to computer-based assessment and help sensitize them to the potential benefits and implementation issues.
Ronald Zellner (Sun,) studied this question.