Working with the HoloLens Sample in Unity

Use the Vuforia HoloLens Sample to set up a Unity project for Microsoft’s HoloLens.

The Vuforia sample package has a pre-configured Unity scene and project settings that act as a starting point and reference for your own apps. It also demonstrates how to use Microsoft’s Mixed Reality Toolkit (MRTK) in Vuforia applications e.g. to work with hand gestures on the HoloLens. For additional information on working with HoloLens, refer to the Developing Vuforia Apps for HoloLens article.

Set up the Supported IDE and Unity

The following steps are only required if this is the first time starting up a HoloLens development project in Unity. Please note that only Windows 10 and 11 supports developing for HoloLens.

1. Ensure that the minimum-supported Visual Studio and Unity versions are installed as per Microsoft’s Install the Tools page.
2. NOTES:
   • For the MRTK and developing for HoloLens, Visual Studio 2022 or later is required.
   • Install the Unity Editor version or newer as stated in the Supported Versions, and make sure to install the Universal Windows Platform Build Support and the Windows Build Support (IL2CPP).
3. Configure the Unity Editor to use Visual Studio as the default IDE.
4. Recommended:
   • Follow the installation checklist from Microsoft to ensure all dependencies and versions are correctly downloaded and installed.
   • Install Visual Studios tools for Unity and familiarize yourself with the performance recommendations for mixed reality in Unity.

For general information on setting up your Windows 10 build environment for Vuforia Engine, refer to the Developing for Windows 10 in Unity article.

Add the Vuforia HoloLens Sample

When you add the Vuforia HoloLens sample from the Unity Assets Store to your Unity project, you will need to configure the project using the MRTK project configurator. The sample includes the Vuforia Engine SDK, sample material, and MRTK assets.

1. In your Unity project, switch the platform to UWP in the Build Settings.
2. Add the Vuforia HoloLens Unity Sample from the Asset Store or from the package manager.
3. Continue with the steps in MRTK and OpenXR Setup.

Only add Vuforia Engine in an Empty Unity Project

Add Vuforia Engine without sample assets to a Unity project.

1. Create an empty Unity Project and set the build platform to UWP.
2. Download and install the Mixed Reality Feature Tool and follow Microsoft’s guide to discover and import features into a selected Unity project. In short:
   1. With the tool, you select your Unity project (for example an empty project) and press Discover Features.
   2. Select the latest version of the Mixed Reality Toolkit Foundation in the Mixed Reality Toolkit category and the Mixed Reality OpenXR Plugin in the category Platform Support.
   3. Click Get Features.
   4. Click Import and Approve to add the selected dependencies to the project packages and manifest in your Unity project.
3. In Unity, press Yes to the message on updating the project to the new input system. Unity will then restart.
   NOTE: Steps 1 to 3 are necessary to do for each new project that targets HoloLens development.
4. Add Vuforia Engine either through the Package Manager if you have the package added to My Assets, or, Download the *.unitypackage for Vuforia Engine here and import it into your project via Assets -> Import Package -> Custom Package..
5. Replace the MainCamera with GameObject -> Vuforia Engine -> ARCamera.
   
6. Select Mixed Reality -> Toolkit -> Add to Scene and Configure ….The ARCamera is now a child of the MixedRealityPlaySpace GameObject.
7. Continue with the steps in MRTK and OpenXR Setup.

MRTK and OpenXR Setup

1. When the sample is imported, a pop-up window is displayed. You can also open it from Mixed Reality Toolkit -> Utilities -> Configure Unity Project. Follow the MRTK steps for configuring your unity project.
   
2. Select the option: Unity OpenXR plugin - Vuforia only works with this setting.
3. The other steps of the MRTK setup can be chosen freely.
4. After completing the MRTK setup guide, go to Project Settings -> XR Plug-in Management.
   1. Enable Initialize XR on Startup.
   2. Enable Open XR and Microsoft HoloLens feature group.

   

5. Under XR Plug-in Management -> OpenXR, apply the following settings:
   1. Render Mode as Single Pass Instanced.
   2. Depth Submission Mode as Depth 16 Bit.
   3. In Interaction Profiles, add Eye Gaze Interaction Profile and Microsoft Hand Interaction Profile.
   4. Enable Microsoft HoloLens under the OpenXR Feature Groups.
   5. Under All Features, enable Hand Tracking and Motion Controller Model.

Scene Elements and Configuration

The HoloLens sample uses the Image Target, VuMark, and Model Target features. The sample scenes are located in the SampleResources folder along with other assets and resources in this sample.

The scene 0-ManagerScene shows the basic setup for development on HoloLens, combining the MixedRealityToolkit with Vuforia components.
All other scenes are loaded additively to add target-specific functionality.

Camera Setup

Vuforia’s life cycle and events are controlled by the Vuforia Behaviour script. In the Vuforia HoloLens sample, this script is added to the Main Camera in each scene.

NOTE: When not using the MRTK, a ready-to-use camera object with the Vuforia Behaviour already attached can be added to a scene using the GameObject > Vuforia Engine > AR Camera menu item.

In the Vuforia Behaviour script, the World Center Mode dropdown defines which object in the scene hierarchy serves as the world origin (0,0,0) of the scene's world space. In HoloLens, only the Device option is supported.

Target GameObjects

The scenes that show how to set up Vuforia targets and virtual content for them are loaded additively. The 2-ImageTargets scene shows a basic setup for multiple Image Targets.

You can easily substitute your own content in this scene to create a unique HoloLens app.

The ImageTarget GameObject encapsulates the ImageTargetBehavior and tracking event handling scripts. These are the primary script components used to customize Image Targets in a HoloLens application.

The Database dropdown selects the database with Image Targets.

The Image Target dropdown defines which target in the database to assign to the Image Target Behaviour.

The DefaultObserverStatusEventHandler script registers callbacks from the Image Target Behaviour arising from changes in the state of the Image Target, such as when it has been detected and is being tracked.

The DefaultObserverStatusEventHandler script can be used to enable and disable rendering and collision detection on digital content when it is a child of the target. The script also exposes OnTargetFound() and OnTargetLost() events that can be used to control application-specific behavior. In this sample, the events are used to display the target state in the UI:

The 2-ModelTargets and 2-VuMarks scenes demonstrate the setup of Vuforia’s Model Target and VuMark features on HoloLens. They follow the same principle as the targets 2-ImageTarget scene:

• A target GameObject allows configuration of the properties of the target it represents, e.g. its database and physical size.
• Virtual content is parented to the target as child objects.
• An event handling script is attached to the game object to handle events when the target is found or lost.

NOTE: Model Targets on HoloLens only support 3D Guide Views. See Model Targets in Unity on how to set up 3D Guide Views.

Test in Play Mode

If you only wish to test the Vuforia HoloLens sample in Play Mode, the XR Plug-in Management options need to be adjusted for the Standalone build target.

1. In the XR Plug-in Management Standalone tab, enable Initialize XR on Startup, OpenXR, and Windows Mixed Reality Feature Group.
2. Under the OpenXR settings, set:
   1. Render Mode as Single Pass Instanced.
   2. Depth Submission Mode as
   3. Play Mode OpenXR Runtime as Windows Mixed Reality.
   4. In Interaction Profiles, add Microsoft Hand Interaction Profile.
   5. Enable Windows Mixed Reality under the OpenXR Feature Groups.
   6. Under All Features, enable Hand Tracking and Motion Controller Model.
      

NOTE: The Unity Profiler should be turned off during Play Mode in Mixed Reality scenes. Otherwise, the MRTK will log errors in the Console.

To allow the project to configure correctly in Play Mode, the Vuforia HoloLens sample must be played from the 0-Manager scene. The scene can be found in SampleResources/Scenes. 

Build and Try the Sample

1. In the Build Settings window, click Build to generate a Visual Studio project.
2. In the Windows Explorer dialog that appears, create a new folder to hold Unity's build output. Generally, we name the folder App.
3. Select the newly created folder and click Select Folder.
4. Once Unity has finished building, a Windows Explorer window opens to the project root directory. Navigate into the newly created folder.
5. Open the generated Visual Studio solution file located inside this folder.
   The project opens in Visual Studio.
6. Set the Solution Platforms dropdown menu to ARM64.
7. Select Release in the dropdown menu next to the Solution Platform.

For more information, refer to Microsoft’s Exporting and building a Unity Visual Studio solution and Using Visual Studio to deploy and debug tutorials.

NOTE: In some cases, Unity exports the Visual Studio project with an incorrect toolset setup. If the build fails with the error “error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found.”, in the Project menu select Retarget Solution.
In the Review Solution Actions set the Platform Toolset to Upgrade to v142 and click OK.

Learn More

Developing Vuforia Apps for HoloLens

Digital Eyewear

Best practices for Optical See-Through Digital eyewear Experiences