Role of temporal overlap of visual and auditory material in forming dual media associations.

Abstract : This paper contains some results about forming associations between the visuals and spoken material in a dual media presentation such as film or television. In particular, it reports the role played by the overlap in time of visual and auditory linguistic material. Fourteen groups of college students were shown a 30 min film which introduces an assembly kit, its pieces, their names, and some of their uses. The film's visuals and narration could be presented in synchrony or one could be shifted relative to the other up to 21 sec. Subjects saw the film in one of seven versions: visuals moved relative to narration by -21, -14, -7, 0 (synchrony), 7, 14, or 21 sec. They were tested immediately or after a week for recall of the names, given the pieces. Scores were highest at zero- and 7-day delay for two groups: synchrony, and visuals 7 sec before narration. At 0-delay, each of the other five groups scored about 80% of the highest groups. At 7-day delay, the other five groups scored differently: the 3 narration first groups were about 30% less than the two visuals first groups. The temporal order in which visual and auditory elements are presented differentially influences the formation of dual media associations. When auditory precedes visuals, much of the auditory component is lost. Two possible theoretical interpretations are given, and practical applications are discussed. (Author)