Human-Autonomous System Interaction

Overview

As autonomous systems (self-driving vehicles, drones, robots, etc.) become increasingly prevalent, the design of perception, trust, and interaction between humans and autonomous systems has become a critical challenge. Our lab investigates external Human-Machine Interfaces (eHMI), Take-Over Requests (TOR) in semi-autonomous driving, and collaborative communication with robots, building knowledge to make autonomous systems safer, more intuitive, and more trustworthy.

eHMI: Communication Between Autonomous Vehicles and Pedestrians

We design and evaluate eHMI (External Human-Machine Interfaces) that convey the intent of autonomous vehicles and drones to pedestrians and other road users.

  • eHMI position and vehicle type optimization in VR (Zheng et al., CHI 2024) Constructed a VR environment to systematically evaluate how eHMI display position (front, side, roof) and vehicle type affect pedestrian decision-making.

  • Effects on pedestrian perception (Zheng et al., IEEE ACCESS 2025) A real-environment experiment quantitatively analyzing how eHMI affects pedestrian perception of, trust in, and behavioral choices around autonomous vehicles.

  • Attention-guided eHMI design (Li et al., CHI 2026) Derived eHMI design principles that appropriately guide pedestrian attention using eye-tracking data.

  • Drone-based POV display (Morita et al., VR 2026) Proposed a novel eHMI format using a first-person-view (POV) display mounted on a drone, validated via VR experiment.

Take-Over Requests (TOR) in Semi-Autonomous Driving

In SAE Level 3 semi-autonomous driving, rapid response to Take-Over Requests (TOR) from the system to the driver is safety-critical. We investigate effective TOR presentation methods.

  • Multimodal visual-auditory TOR (Chen et al., HRI 2023) Examined how combining visual and auditory modalities in TOR presentation affects take-over time and driver response quality.

  • Vibrotactile TOR effectiveness (Ubukata et al., ICCE-Asia 2023) Proposed and evaluated vibrotactile feedback via seat and steering wheel as a TOR modality to enhance driver situational awareness.

Collaborative Communication with Robots

We research intent communication and task coordination between humans and robots working in shared physical spaces.

  • Local step vs. global plan communication (Li et al., IEICE 2025) A comparative experiment revealing how communicating only the next action vs. the entire global plan affects human situational awareness, trust, and collaborative efficiency.

  • Overcooked Plus benchmark (Cai et al., ACSOS 2024) Developed “Overcooked Plus,” an extended benchmark environment based on the cooperative cooking game “Overcooked,” and released it publicly as a testbed for evaluating self-adaptive algorithms in human-robot collaboration.

Perceptual Safety and User Preferences

  • Shielding for perceptual safety (Abe et al., ISDA 2023) Integrates a shielding mechanism into robot control systems to guarantee safety even under perceptual uncertainty, enabling safe autonomous behavior in real environments.

  • User-driven adaptation (Zhang et al., CHI 2024) Presented at CHI 2024, this work proposes a method for autonomous systems to dynamically adapt to user preferences and context using complaint-based feedback.

Selected Publications

  • Nianzhao Zheng et al. “Exploring Optimal eHMI Display Location for Various Vehicle Types: A VR User Study.” CHI 2024.
  • Nianzhao Zheng et al. “Exploring the Impact of eHMI Display Location and Vehicle Type on Pedestrian Perceptions: A VR User Study.” IEEE ACCESS, 2025.
  • Jialong Li et al. “See What I See: An Attention-Guiding eHMI Approach for Autonomous Vehicles.” CHI 2026.
  • Shogo Morita et al. “AeroScale: A Resizable Drone-based POV Display for Mobile and Shared Augmented Reality.” VR 2026.
  • Qingxin Chen et al. “Attention-guiding Takeover Requests for Situation Awareness in Semi-autonomous Driving.” HRI 2023.
  • Jialong Li et al. “Global Progress or Local Intent? Exploring Human Perceptions of Communication Strategies in Human-Robot Collaboration.” IEICE, 2025.
  • Jinyu Cai et al. “Overcooked Plus: A Comprehensive Cooking Scenario TestBed for Enhancing the Evaluation of Autonomous Planning Algorithms.” ACSOS 2024.
  • Mingyue Zhang et al. “User-Driven Adaptation: Tailoring Autonomous Driving Systems with Dynamic Preferences.” CHI 2024.
  • Ryotaro Abe et al. “Towards Enhancing Driver's Perceived Safety in Autonomous Driving: A Shield-based Approach.” ISDA 2023.