Personalized Human-Swarm Interaction

Controlling a single drone with your hands is already a challenge. Controlling a swarm of them — keeping formation, directing collective motion, maintaining spatial awareness — pushes that challenge into a different league. The question was whether the personalized body-machine interface approach could scale from one robot to many.

This work extended the BoMI framework to swarm teleoperation using a Leap Motion optical hand tracker. Rather than mapping individual hand gestures to pre-set swarm commands, the system learns each user's preferred hand motion vocabulary through a short calibration session, then uses that personalized mapping to translate continuous hand poses into swarm formation and velocity commands in real time.

Users could direct multi-drone formations through natural hand movements with minimal training, and the personalized mapping consistently outperformed a generic baseline in both accuracy and perceived ease of control. The work was published in IEEE Robotics and Automation Letters and has been cited over 20 times.