Influence of semiotic resources on peer interactions during collaborative digital multimodal composing

Collaborative digital multimodal composing (CDMC) has recently gained traction in the advent of technology-enhanced language learning, yet scant attention was paid to the influence of semiotic resources on the interaction patterns between learners. The present study attempted to fill the research gap by examining the interactions between English as a Second Language (ESL) learners in an online English for Academic Purpose (EAP) course. Using a multiple case-study design, three pairs of undergraduates completed a collaborative multimodal writing and video-making task for presentation using ZOOM. Influence of semiotic resources can be unveiled through manipulating the mode of presentation. Data of this study includes the verbal exchanges and non-verbal on-screen interactions between the participants while they are working on the tasks. Their interaction patterns were analyzed through conversation analysis and two aspects of collaborations, namely equality and mutuality, were also examined. Their interaction patterns were found to be strikingly different across both tasks, and languaging mainly serves the function of verbalization of content-related issues, such as searching for information and assembling various multimodal elements, with only rare instances of either grammar-based or lexis-based language-related episodes (LRE). Another striking finding is that pairs who are working collaboratively with balanced division and mutual contribution are less susceptible to our manipulation, as compared to those who are demonstrating a dominant-dominant or expert-novice working pattern. Implications of these findings on fostering peer collaborations during CDMC are discussed.