Working with the HoloLens Sample in Unity

Use the Vuforia HoloLens Sample to set up a Unity project for Microsoft’s HoloLens.

The Vuforia sample package contains a pre-configured Unity scene and project settings that act as a starting point and reference for your own apps. It also demonstrates how to use Microsoft’s Mixed Reality Toolkit (MRTK) in Vuforia applications e.g. to work with hand gestures on the HoloLens. For additional information on working with HoloLens, refer to the Developing Vuforia Apps for HoloLens article.

First time set up with the Supported IDE and Unity

The following steps are only required if this is the first time starting up a HoloLens development project in Unity. Please note that only Windows supports developing for HoloLens and UWP.

1. Ensure that the minimum-supported Visual Studio and Unity versions are installed as per Microsoft’s Install the Tools page. 
2. NOTES: 
   • For the MRTK and developing for HoloLens 2, Visual Studio 2019 or later is required.
   • When installing Unity, make sure to install the Windows Platform Build Support.  
3. Configure the Unity Editor to use Visual Studio as the default IDE.      
4. Recommended:
   • Install Visual Studios tools for Unity and familiarize yourself with the performance recommendations for mixed reality in Unity.

For general information on setting up your Windows 10 build environment for Vuforia Engine, refer to the Developing for Windows 10 in Unity article.

Download the Mixed Reality Feature Tool

1. Create an empty Unity Project.

1. Download and install the Mixed Reality Feature Tool and follow Microsoft’s guide to discover and import features into a selected Unity project. In short:
   • With the tool, you select your Unity project (for example an empty project) and press Discover Features.
   • Select the latest version of the Mixed Reality Toolkit Foundation in the Mixed Reality Toolkit category and the Mixed Reality OpenXR Plugin in the category Platform Support.
   • Click Get Features.
   • Click Import and Approve to add the selected dependencies to the project packages and manifest in your Unity project.
2. In Unity, press Yes to the message on updating the project to the new input system. Unity will then restart.

NOTE: Steps 5 and 6 are necessary to do for each new project that targets HoloLens development.

Unity Configuration Setup

In the Unity Project:

1. Switch build platform to UWP.
2. Open the Package Manager and select the Unity Registry to show the list of packages. Import the OpenXR Plugin if it was not imported already.

3. In the MixedReality menu, select Project > Apply recommended project settings for HoloLens 2.
4. Go to Project Settings > XR Plug-in Management, and in the UWP tab, OpenXR should show as enabled with a warning sign next to it.

5. Click on the warning sign to open the OpenXR Project Validation window.
6. In the window, click Fix All.

Unity has now been set up with the latest OpenXR plugin and your Unity project can start to take form.

Add Vuforia, HoloLens Sample, and MRTK

MRTK Project Setup

You can optionally Import any other MRTK feature into your Unity project with the Mixed Reality Feature Tool the same way as you added the OpenXR Plugin.

1. In the top bar, select Mixed Reality -> Toolkit -> Utilities -> Configure Project for MRTK and auto-apply the default settings. These can also be set manually in Build Settings -> Player Settings -> Publishing Settings:
   • Under Capabilities ensure that Internet Client, WebCam, Microphone, and SpatialPerception are selected.
     NOTE: SpatialPerception should only be selected if you intend to use the Surface Observer API.
   • Under Supported Device Families, ensure that Holographic is selected.

     

2. In the Publishing Settings, expand the Resolution and Presentation section.
   • Disable Run in Background so that Vuforia pauses when the app is put into the background and can access the camera again when the app is resumed.
   • In the Default Orientation dropdown, ensure that Landscape Left is selected.
3. Close the Project Settings window.

NOTE: The remaining steps 4-6 are only if you are setting up an empty project.

4. Add the MRTK to each scene using Mixed Reality Toolkit > Add to Scene and Configure.
5. In the MixedRealityToolkit GameObject, in each scene, set the configuration profile to the DefaultMixedRealityToolkitConfigurationProfile.
6. Select the Main Camera found as a child of the MixedRealityPlaySpace GameObject and add the VuforiaBehaviour component to it.

For more information on the MRTK profiles, please refer to Microsoft’s MixedRealityToolkit Documentation.

Scene Elements and Configuration

The HoloLens sample uses the Image Target, VuMark, and Model Target features. The sample scenes are located in the SampleResources folder along with other assets and resources in this sample.

The scene 0-ManagerScene shows the basic setup for development on HoloLens, combining the MixedRealityToolkit with Vuforia components.
All other scenes are loaded additively to add target-specific functionality.

Camera Setup

Vuforia’s life cycle and events are controlled by the Vuforia Behaviour script. In the Vuforia HoloLens sample, this script is added to the Main Camera in each scene.

NOTE: When not using the MRTK, a ready-to-use camera object with the Vuforia Behaviour already attached can be added to a scene using the GameObject > Vuforia Engine > AR Camera menu item.

In the Vuforia Behaviour script, the World Center Mode dropdown defines which object in the scene hierarchy serves as the world origin (0,0,0) of the scene's world space. In HoloLens, only the Device option is supported.

Target GameObjects

The scenes that show how to set up Vuforia targets and virtual content for them are loaded additively. The 2-ImageTargets scene shows a basic setup for multiple Image Targets.

You can easily substitute your own content in this scene to create a unique HoloLens app.

The ImageTarget GameObject encapsulates the ImageTargetBehavior and tracking event handling scripts. These are the primary script components used to customize ImageTargets in a HoloLens application.

For the ObserverStatusEventHandler script, the Database dropdown selects the database that contains the Image Target assigned to this Image Target Behaviour.

The Image Target dropdown defines which target in the database to assign to this Image Target Behaviour.

The ObserverStatusEventHandler script is a customized version of Vuforia’s built-in Default Observer Event Handler script. It registers callbacks from the Image Target Behaviour arising from changes in the state of the Image Target, such as when it has been detected and is being tracked.

The ObserverStatusEventHandler script is used to enable and disable rendering and collision detection on digital content that is a child of the target. The script also exposes OnTargetFound() and OnTargetLost() events that can be used to control application-specific behavior. In this sample the events are used to display the target state in the UI:

The 2-ModelTargets and 2-VuMarks scenes demonstrate the setup of Vuforia’s Model Target and VuMark features on HoloLens. They follow the same principle as the targets 2-ImageTarget scene:

• A target GameObject allows configuration of the properties of the target it represents, e.g. its database and physical size.
• Virtual content is parented to the target as child objects.
• An event handling script is attached to the game object to handle events when the target is found or lost.

Build and Try the Sample

1. In the Build Settings window, click Build to generate a Visual Studio project.
2. In the Windows Explorer dialog that appears, create a new folder to hold Unity's build output. Generally, we name the folder App.
3.  Select the newly created folder and click Select Folder.
4. Once Unity has finished building, a Windows Explorer window opens to the project root directory. Navigate into the newly created folder.
5. Open the generated Visual Studio solution file located inside this folder.
   The project opens in Visual Studio.
6. Set the Solution Platforms dropdown menu to ARM64.
7. Select Release in the dropdown menu next to the Solution Platform.

For more information, refer to Microsoft’s Exporting and building a Unity Visual Studio solution and Using Visual Studio to deploy and debug tutorials.

NOTE: In some cases, Unity exports the Visual Studio project with an incorrect toolset setup. If the build fails with the error “error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found.”, in the Project menu select Retarget Solution.
In the Review Solution Actions set the Platform Toolset to Upgrade to v142 and click OK.

Learn More

Developing Vuforia Apps for HoloLens

Digital Eyewear

Best practices for Optical See-Through Digital eyewear Experiences