AR Remote Maintenance Platform

Problem

When complex equipment breaks in the field, the person who can fix it and the person who knows how to fix it are often in different places. Flying an expert out is slow and expensive; a phone call can’t point at the right bolt. Our sponsor, NavSea, wanted a way for a remote expert to guide an on-site technician as if they were standing right there.

What I built

I architected and led development of an augmented-reality remote maintenance platform on Microsoft HoloLens. A technician wearing the headset and a remote expert on another device share a live session: two-way audio and video, plus spatial annotations. The expert draws 3D markings, arrows, and icons directly onto the real equipment in the technician’s field of view.

The project earned 3rd place among UCSB engineering capstones.

How it works

• Built in Unity with C# targeting the HoloLens.
• Live audio/video communication between the HoloLens and remote devices.
• Spatial annotations anchored to the real world, so an arrow the expert draws stays on the bolt as the technician moves.
• The hardest part was spatial alignment: overlaying 3D drawings and icons accurately while respecting the HoloLens’s real performance constraints. Getting annotations to stay locked to physical objects without tanking the frame rate drove most of the engineering.

Impact / results

• Demonstrated a working end-to-end “see-what-I-see + annotate” workflow that collapses the distance between a field technician and a remote expert.
• Recognized with a 3rd-place finish among UCSB engineering senior capstone projects.

What I’d do next

• Tighten annotation persistence across sessions and re-localization.
• Add recording/playback so a guided fix becomes reusable training material.
• Explore multi-expert sessions and lower-end AR hardware support.