Multimodal Human-Robot Interaction

Overview

At KUKA, I conducted end-to-end UX research on collaborative robots (cobots), focusing on the commissioning stage, the critical setup phase where operators first configure and interact with the robot. The research explored how multimodal feedback systems (visual, auditory, haptic) could enhance user understanding, efficiency, and trust during this process.

The Challenge

Setting up a cobot in an industrial environment is a high-stakes interaction. Operators need to configure parameters, teach positions, and verify safety zones, often under time pressure. The existing setup experience lacked clear feedback cues, making it difficult for users to build confidence in the system and understand its state. This led to slower onboarding, hesitant interactions, and reduced trust in the technology.

Research Approach

• Executed mixed-methods studies combining usability testing, SUS surveys, semi-structured interviews, and observational analysis
• Observed operators during real commissioning workflows to identify friction points and mental model gaps
• Explored how adding multimodal feedback (visual indicators, audio cues, haptic responses) to the cobot setup affects comprehension and trust
• Analyzed both qualitative data (interview transcripts, behavioral observations) and quantitative metrics (SUS scores, task completion, error rates)

Why these methods?

SUS gave us a standardized, comparable usability benchmark. Interviews uncovered the reasoning behind hesitations that observation alone couldn't explain. Observation was essential because operators often can't articulate what's difficult in the moment; you have to watch them work. The combination meant we could triangulate: quantitative data told us where problems were, qualitative data told us why.

Key Findings

• Multimodal feedback significantly improved operators' understanding of robot state during setup
• Users reported higher confidence and trust when the system communicated intent through multiple channels
• Observational data revealed specific moments in the commissioning flow where users hesitated or made errors due to lack of system feedback

Impact

• Generated actionable insights that informed product and interface improvements for the commissioning workflow
• Collaborated with cross-functional teams (engineering, design, product) to translate findings into practical design recommendations
• Developed UX research training slides to support knowledge sharing and standardize research practices across teams at KUKA