Real-Time Smartphone Camera–Based Hand Tracking for Controller-Free XR Exposure Therapy

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Exposure therapy is an effective treatment for anxiety-related disorders, but conventional in vivo approaches are often limited by logistical and accessibility barriers. Extended reality (XR) addresses many of these limitations by providing controlled, repeatable exposure in virtual environments. Smartphone-based XR further improves accessibility through a portable, low-cost platform, although many systems rely on low-fidelity interactions that may reduce immersion. This study presents a stereoscopic smartphone-based XR platform integrating computer vision–based hand tracking for controller-free interaction in an exposure-relevant virtual environment. A virtual dog-fetch scenario was developed in Unity using MediaPipe hand landmark detection and custom interaction techniques. In a preliminary evaluation ( n = 5), participants exhibited longer grasp-to-release durations in XR than in the real world (RW) (7.0 vs. 2.47 s), with frequent failed grasps and virtual object slippage. These findings demonstrate the feasibility of controller-free smartphone XR while highlighting grasp interaction as a key area for improving immersive exposure therapy.