Final Product
The Process
    Unwilling to miss the opportunity to develop for my lab's HoloLens, I put together a simple surface painter. The Unity project accepts the scanned environment mesh from HoloLens and uses it as a mesh collider. When raycasts from my camera hit that collider, I can add points to a new line renderer on the point of collision. I also render the environment as a transparent but occluding material to prevent seeing drawn lines through scanned objects. 
    At this point I wanted to draw in 3D, but found the close-range depth data from the HoloLens was inadequate for TiltBrush-like drawing. I mounted a LEAP Motion controller onto the HMD, then send the data to the HoloLens via Wifi in order to capture depth. While still imprecise, I'm working to correct the offset and smooth the input.
Credit to Zhengyi Luo for the LEAP Motion networking tool.
Back to Top