Microsoft Paint HoloLens

AR Design, UX/UI 

Role — UX/UI, Motion Graphics
Partner — Kimberly White: UX/UI, Motion graphics, Cinematographer
3 weeks


HoloLens is a visor for AR/MR experiences. It utilizes hand gestures, voice commands, eye gaze, and a hand-held controller to augment everyday reality. Currently, the start up screen features how to instructions and settings options. There is no opportunity on this home screen to surprise and delight users and engage them creatively. 


Take a program that is old and familiar to many Microsoft users—Microsoft Paint and refresh it in the HoloLens environment. The simplicity of the program combines creativity with learning hand gestures.


We identified a missed opportunity in the initial gesture walkthrough of the HoloLens. Our goal is to build a nostalgic and delightful way for the HoloLens user to be creative.
Our research suggests that a user would retain and learn gestures for future use of HoloLens applications. We envision a world where MS paint comes standard as a native app to Microsoft Hololens. Adding MS Paint to the unboxing lineup would allow for instant creativity in AR space and allow users to walk through a series of voice commands and gestures.

Research + Discovery

User Testing

To understand user needs and expectations we set up a free-standing paper prototype equipped with almost any paint and photo editing tool a user could ask for. We asked users to build a toolkit they would like to have for painting a coloring book scene that was visible to them.

Below, we told our users a series of prompts to build the best toolkit and gestures for Microsoft Paint Hololens.

Build a Toolkit

  1. Create a toolkit using 10 Icons below to would want to see use to paint this picture. Put them on the left board.

  2. Add the color picker you think is most successful.

  3. Add the color palate option that you think is most successful.

  4. Use the sticky notes to write down any features that would help you with this task. Put them on the right board.

  5. Walk us through your thought process along the way.

  6. Pretend you are taking a photo, what gesture would you use with your hands?

From our user testing, we were able to distill our assumptions of what the user would want to focus on and what the user actually needed. For example, the camera gesture we created allows for the best range of control for taking photos. Given the user suggested gestures, we brought more than one idea together to create the best solution.

To view our top suggestions from user testing, click here.

User Personas

Brand System

Branding with Microsoft

We read up on the Microsoft Fluent Design System, and referenced their templates for Universal Windows Platform Apps to create our interface.

How It Works

Our final solution was based on brand guidelines and informed by user testing.

For example, we heavily designed our color selection tool based on user testing, currently there is no model for color selection in HoloLens. We created a color picker that was most requested from testing and user friendly. We found that people preferred to have the color selection and the color palette be in the same set up, to consolidate space. Another feature that was requested was to have recent colors available below the color picker. Even though the Microsoft UWP has a color picker, user testing and designing for an AR experience pushed us to create something new.


In AR, the center space is for the most actively used elements, followed by the right, then last the left. In our AR space this is how our panels would appear for initial settings.

Voice and Sound

We recognize that experiencing AR, especially for the first time can be a clumsy interaction. In order to compensate for this we made sure to include voice functionality for all the tools to give the user options while they create. We also included core voice commands for frequently used actions within Microsoft Paint HoloLens. We added auditory cues, by giving sounds for each of the tools that would be used such as a droplet sound effect for the paint bucket or a swishing sound effect for painting.

Gestures Library

We based our gestures off of user-testing and current gestures used for Microsoft HoloLens.

Final Thoughts

I didn’t understand what Augmented Reality and HoloLens was when we first started this project. Kimberly and I did a lot of research, tried using HoloLens and user-tested as much as we could.

We kept joking, if we had another week we’d do x,y,z... But if we did have another week, I would spend more time user-testing our current UI. Since designing for AR is still a new concept, it would be great to see how users react to our current setup and how we could make it more intuitive.

This is a sneak peak into what Microsoft Paint would look like in HoloLens. Our workflow had to be agile because we only had three weeks. We worked well because we collaborated, ideated efficiently and executed accordingly. I’m happy with what we were able to do and that we were able to create something based off our user-testing. 


—fueled by


All rights reserved © Judilee Haider 2020