ML2 Avatars System

Presence in mixed reality depends on one thing above everything else: feeling like the person next to you is actually there. For Magic Leap 2, that meant building a full avatar system from the ground up — one that could represent remote collaborators as believable digital humans in shared XR spaces, in real time, on device.

I led a team of five engineers to design and ship this system end-to-end. The stack covered everything from data collection and ML model training to the full Unity/C# runtime: skeletal animation, blendshape facial rigging, networked state synchronisation, and smooth integration with ML2 perception APIs. On the AI side, we developed hand pose classification and action recognition systems that fed directly into avatar expression and gesture playback.

Beyond the core system, the work had a wider footprint: key components were productised into company-level platform capabilities, made available to other teams building on Magic Leap 2. The Avatars initiative was one of the flagship AI/ML efforts on the platform, and I helped define its roadmap and strategic direction alongside leadership — while also mentoring interns and keeping alignment tight across research, product, engineering, and design.