This article provides recommendations for designing and implementing optical see-through digital eyewear user experiences including helpful resources for both the Microsoft HoloLens and the Magic Leap 2.
App Development Best Practices for Digital Eyewear
Developing for head-mounted displays entails considering specific design principles for interactions and affordances to achieve an enjoyable experience for users. The guidelines and principles depend on the many scenarios you can develop AR experiences for; but when you are using the Vuforia Engine in Unity with digital eyewear, we can recommend some general guidelines are available.
- Spatial mapping. Both the HoloLens and Magic Leap device use spatial mapping to create a digital replication of the real-world environment. This opens allows you to anchor various digital content and artefacts to the environment. This may also be able to remember its position across AR sessions. The ability to save AR sessions is great for developing AR games and apps that are linked to stationary objects. See Persistence in Unity for HoloLens and Content Persistence for Magic Leap to learn more about the subject.
- Floating user interfaces. It is not always apparent to users how to identify which objects and targets will trigger recognition and tracking. Therefore, we advise apps to include Guide Views for Model Targets and viewfinder UI elements to AR apps that are being used by for example by visitors in a museum where many objects could potentially be targets. It can help guide the users to point the camera in the correct direction and trigger a result more quickly.
To Avoid making these guides ‘headlocked’ I.e. locked to the motions of the user’s head. We advise instead to use the user’s gaze and head pose with for example a track position behavior. See Tag-alongs on HoloLens and Head Locked Content for Magic Leap for more information. - Privileges. As it is for every app that is accessing hardware on the device, privileges and permissions needs to be granted by the user. Magic Leap and HoloLens have both provided a way to manage these privileges through the Unity Editor and their Magic Leap Unity Package. Access the links to learn more about their management of HoloLens Capabilities in Unity and Magic Leap Privileges in Unity.
Distance and Motion Considerations
App experiences that rely on digital eyewear differ from those that use handheld devices in two important ways:
- Greater target distances. In handheld device AR experiences, users hold a device directly in front of themselves. The cameras on a digital eyewear device, however, are located on the user's head. Therefore, targets need to be detectable and trackable from greater distances. Also, most devices have near field clipping planes, which will stop rendering if you move too close to a target. To ensure that users can still move closer and witness the full detail of the target. To accommodate, adjust the position of the near and far clipping plane using the Vuforia Engine APIs. In Unity, for each camera instance in the Hierarchy window, set the clipping plane distances in the Inspector window. See Hologram Render Distances and Scale on HoloLens and Field of View for Magic Leap for more information on this design principle.
- Reduced range of motion. Users can manipulate a handheld device with more flexibility than a digital eyewear device. So, design the digital eyewear app experience to accommodate the range of movements comfortable for your user.
Performance Optimizations
Optical see-through experiences generally run on devices with lower power or capabilities than traditional high-end mobile devices or desktop solutions. Be aware of these restrictions to avoid any throttling during the experience. Typically, you can monitor rendering framerate as a good indicator of system performance, for HoloLens and Magic Leap the render framerate should remain stable and close to 60fps.
The Vuforia Engine API has camera modes you can set before starting Vuforia Engine.
- In most scenarios, head-mounted displays benefit from setting the mode to SPEED which is the fast camera mode that reduces the overall resource impact on the system.
- There is also a QUALITY camera mode which runs a higher resolution. This can improve tracking smaller targets such as VuMarks and barcodes at the price of an increased resource impact on the system.
See Device Performance Optimization for more information.
For general information on optimizing the performance of your apps that doesn’t relate to Vuforia Engine, refer to the Optimizing graphics performance and Light troubleshooting and performance articles in the Unity library. There is further information to be found regarding performance for mixed reality here: Understanding Performance for Mixed Reality and Performance Recommendations for Unity
Useful links
https://docs.microsoft.com/en-us/windows/mixed-reality/design
https://creator.magicleap.com/learn/guides/design-principles