Exploring Mixed-Interaction Mode in a Virtual Cockpit: Controller and Hand Gesture Integration

Abstract

This paper evaluates a new interaction mode for object manipulation tasks in virtual reality (VR) utilizing an aircraft cockpit simulation. Building on prior research, this study examines the effectiveness and user experience of a mixed-interaction mode that involves the combination of handheld controllers with hand gestures. Qualitative interviews with participants provided detailed feedback on the combined input approach. The analysis highlights the strengths and challenges of the mixed-interaction mode, indicating a perceived increase in task completion efficacy and enhanced user experience. As an outcome of the research, design guidelines were developed based on participants’ insights, focusing on the optimal balance of naturalness and precision for mixed interaction in VR that can also be utilized more generally. This study offers practical implications for creating immersive virtual environments and informs future research in VR interaction modes and user experience.