Spatial Attention in VR

Overview

At the Max Planck Institute for Psycholinguistics, I designed and executed experiments that combined eye-tracking and body-tracking to analyze user behavior in immersive VR environments. The research focused on understanding how people allocate spatial attention and produce anticipatory movements during real-time interaction, with implications for adaptive HCI and cognitive interface design.

The Challenge

Immersive environments generate rich, multimodal behavioral data, but making sense of it requires methods that go beyond traditional screen-based UX research. We needed to understand not just where people look, but how their entire body responds to stimuli in 3D space, and what that reveals about anticipatory cognition and interaction intent.

Research Approach

• Designed VR experiments using a Visual World Paradigm to study anticipatory behavior during language comprehension
• Combined simultaneous eye-tracking and body-tracking to capture gaze and movement data in immersive environments
• Processed and visualized spatiotemporal trajectory data to support predictive modeling and interaction pattern analysis
• Used heatmap visualizations to identify attention hotspots and movement patterns
• Collaborated with an interdisciplinary team across cognitive science, linguistics, and interface design

Why these methods?

VR was necessary because we needed to study embodied behavior in 3D space, which screen-based setups can't capture. Eye-tracking alone would miss the motor preparation that happens before explicit action. Body-tracking let us see anticipatory movements that reveal cognitive processing before any conscious decision. The Visual World Paradigm gave us a controlled way to link linguistic input to physical response timing.

Key Findings

• Users produce anticipatory hand movements that align with gaze patterns, revealing how embodied cognition drives interaction in 3D space
• Trajectory data showed predictable patterns that can be leveraged for proactive interface adaptation
• Multimodal tracking (gaze + body) provides significantly richer interaction data than either modality alone

Impact

Translated research findings into design implications for adaptive HCI frameworks and cognitive interface strategies. The work demonstrates how spatiotemporal behavioral data from immersive environments can inform more intuitive, anticipatory interfaces, systems that respond to user intent before explicit input is given.