Modes of Meaning: Multimodal Media & 4E+ Cognition in Tech-Enhanced Learning - Smith, James; Aguayo, Claudio (Tuwhera, AUT Library, 2024)
This presentation proposes an approach to designing technology-enhanced learning (TEL) through the strategic integration of diverse multimodal media forms within a framework informed by the 4E+ view of cognition. The 4E+ cognition framework emphasises the embodied, embedded, enactive, and extended nature of cognition, suggesting that cognition is not solely confined to the brain but extends into the environment while involving the body's interactions with that environment (Carney, 2020; Jianhui , 2019; Menary, 2010; Newen, et al., 2018). In this theoretical context, our study explores how the combination of various modes of media, such as immersive technologies, digital interactive elements, real-world analogue creations, audio, sound, images, videos, animations, text, and the surrounding environment can be orchestrated to create sensorially rich, and more meaningful learning experiences (Gilakjani, et al., 2011; Philippe, et al., 2020; Sankey, et al., 2010). For example, mixed reality (XR) learning design combines immersive media forms to support multi-sensory and expanded cognitive learning (Philippe et al., 2020; Rakkolainen et al., 2021; Villalobos & Videla, 2023). Other relevant approaches include gamification and transmedia storytelling methods (Doumanis et al., 2019; Perry, 2020). By leveraging different modalities, educators can design learning materials that engage learners with different sensory activations and presentation methods (Bouchey et al., 2021). This approach can cater to the 4E+ view of cognition, and subsequently enhancing knowledge acquisition and retention. Examples from our own practice and research (such as the Explora: Chile es Mar, Pipi’s World and O-Tū-Kapua XR learning experiences), as well as current educational examples (Bouchey et al., 2021; Philippe, et al., 2020), demonstrate how multimodal media integration facilitates deeper engagement, critical thinking, and a more holistic understanding of complex concepts. Furthermore, we discuss practical strategies for educators to implement these principles in their TEL design, highlighting the potential of aligning multimodal design choices with the 4E+ cognitive framework. Ultimately, we advocate for a shift towards a more inclusive and effective approach to technology-enhanced learning - one that embraces the diversity of human cognitive processes and leverages multimodal media to communicate meaningful knowledge in ways that resonate with learners' cognitive structures and experiences. Multimodal methods, when aligned with the distributed 4E+ view of cognition, can make TEL appeal and resonate on deeper levels to engage across various sensory, environmental and communication modes. This type of approach acknowledges the diversity of ways that humans process and understand phenomena, and how more effective learning can occur when multiple ways of knowing are engaged and communicated to. Furthermore, through this method, inclusivity can be heightened for students with diverse cultural, neurological or other backgrounds (Anis & Khan, 2023; Boivin & CohenMiller, 2022). Emerging research shows the potential of the 4E+ approach to meet the needs of learning in 21st century technological environments (Videla & Veloz, 2023; Villalobos & Videla, 2023). This presentation contributes to the literature by examining TEL design through a multimodal media lens. It highlights how the holistic 4E+ framework can more effectively and meaningfully engage students than computational, monomodal and bimodal uses of technology in educational settings.
Keywords: 3901 Curriculum and pedagogy; 3904 Specialist studies in education