Working with the HoloLens Sample in Unity

This article demonstrates how to use the Vuforia HoloLens Sample project to author a Vuforia Engine scene for HoloLens and customize event handling to implement unique app behaviors. The sample project contains pre-configured Unity scenes and project settings that act as a starting point and reference for your own apps. It also demonstrates how to use Microsoft’s Mixed Reality Toolkit (MRTK) in Vuforia applications e.g. to work with hand gestures on the HoloLens 2. For additional background information on working with HoloLens, refer to the Developing Vuforia Apps for HoloLens article.

 

Getting Started with Visual Studio and Unity

  1. Ensure that the minimum-supported Visual Studio and Unity versions are installed as per Microsoft’s Install the Tools page. 
  2. NOTES
    • For the MRTK and developing for HoloLens 2, Visual Studio 2019 is required.
    • When installing Unity, make sure to install the IL2CPP Scripting Backend.  
  3. Configure the Unity Editor to use Visual Studio as the default IDE.      
  4. Recommended:

For more information on setting up your Windows 10 build environment, refer to the Developing for Windows 10 in Unity article. 

Getting Started with the HoloLens Sample

Set up an Empty Project Using the Vuforia HoloLens Unity Sample
  1. Create a new Unity Project. 
  2. Switch build platform to UWP.
  3. Click the Player Settings -> XR Plugin Management, and install the latest XR Management package and enable the Windows Mixed Reality under Plug-in Providers.
  4. Import MRTK and Vuforia 
    1. Download the *.unitypackage for the MRTK here. Select the Microsoft.MixedReality.Toolkit.Unity.Foundation. The other packages are optional.
    2. Download the *.unitypackage for the Vuforia Engine here
  5. Go to Assets -> Import Package -> Custom Package and import the MRTK and Vuforia packages. 
  1. Create a new Unity project.
  2. In Build Settings, switch build platform to UWP.
  3. In the Window menu, select Asset Store.
  4. In the search bar, enter Vuforia HoloLens.
    1. Select Vuforia HoloLens Sample and click Import.

This will import the HoloLens Sample alongside the installation/upgrade of the Vuforia Engine and the MRTK.

  1. Return to the Build Settings and click the Player Settings button.
  2. Select the UWP icon and expand the XR Settings section.
    • Ensure that Virtual Reality Supported is enabled.    
    • Under Virtual Reality SDKs ensure that:
      1. Window Mixed Reality is included in the list and that Enable Depth Buffer Sharing is enabled. 
      2. The Depth Format is set to 16-bit depth.
    • Ensure that the Stereo Rendering Mode is set to Single Pass Instanced.
Vuforia Image

Project Setup

TIP: The MRTK window can optionally be used to auto-apply the default settings listed as below in 1. and 2. Find the window in Mixed Reality Toolkit -> Utilities -> Configure Unity Project.

  1. In the File menu, select Build Settings.
  2. Select Player Settings and expand the Publishing Settings section.
    • Under Capabilities ensure that Internet ClientWebCamMicrophone, and SpatialPerception are selected.
      NOTESpatialPerception should only be selected if you intend to use the Surface Observer API.
    • Under Supported Device Families, ensure that Holographic is selected. 
  3. Expand the Resolution and Presentation section.
    • Disable Run in Background so that Vuforia pauses when the app is put into the background and can access the camera again when the app is resumed.
    • In the Default Orientation dropdown, ensure that Landscape Left is selected.
  4. Close the Project Settings window.

NOTE: The remaining steps 5-7 are only if you are setting up an empty project

  1. Add the MRTK to each scene using Mixed Reality Toolkit > Add to Scene and Configure
  2. In the MixedRealityToolkit GameObject, in each scene, set the configuration profile to the DefaultXRSDKConfigurationProfile.
  3. Select the Main Camera found as a child of the MixedRealityPlaySpace GameObject and add the VuforiaBehaviour component to it.

For more information on the MRTK profiles, please refer to Microsoft’s MixedRealityToolkit Documentation.

Building and Executing the Sample

  1. In the Build Settings window, click Build to generate a Visual Studio project.
  2. In the Windows Explorer dialog that appears, create a new folder to hold Unity's build output. Generally, we name the folder App.
  3.  Select the newly created folder and click Select Folder.
  4. Once Unity has finished building, a Windows Explorer window opens to the project root directory. Navigate into the newly created folder.
  5. Open the generated Visual Studio solution file located inside this folder.
    The project opens in Visual Studio.
  6. In the Solution Platforms dropdown, 
    • x86 if you are building for HoloLens 1
    • ARM64 if you are building for HoloLens 2
  7. Select Release in the dropdown menu next to the Solution Platform.
Vuforia Image

For more information, refer to Microsoft’s Exporting and building a Unity Visual Studio solution and Using Visual Studio to deploy and debug tutorials.

NOTE: In some cases, Unity exports the Visual Studio project with an incorrect toolset setup. If the build fails with the error “error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found.”, in the Project menu select Retarget Solution.
In the Review Solution Actions set the Platform Toolset to Upgrade to v142 and click OK.

Vuforia Image

Scene Elements and Configuration

The HoloLens sample uses the ImageTarget, VuMark, and ModelTarget features. The sample scenes are located in the SampleResources folder along with other assets and resources in this sample.

The scene 0-Base shows the basic setup for development on HoloLens, combining the MixedRealityToolkit with Vuforia components.
All other scenes are loaded additively to add target-specific functionality.

Camera Setup

Vuforia’s life cycle and events are controlled by the Vuforia Behaviour script. In the Vuforia HoloLens sample, this script has been added to the MRTK camera in the 0-Base scene.

Vuforia Image

NOTE: When not using the MRTK, a ready-to-use camera object with the Vuforia Behaviour already attached can be added to a scene using the GameObject > Vuforia Engine > AR Camera menu item.

In the Vuforia Behaviour script, the World Center Mode dropdown defines which object in the scene hierarchy serves as the world origin (0,0,0) of the scene's world space. In HoloLens, only the Device option is supported.

Vuforia Image

Starting with version 8.5, Vuforia Engine automatically detects if an app is running on HoloLens in Unity. It is no longer necessary to configure the Digital Eyewear settings in the Vuforia Configuration window.

Target GameObjects

The scenes that show how to set up Vuforia targets and virtual content for them are loaded additively. The 2-ImageTargets scene shows a basic setup for multiple Image Targets.

You can easily substitute your own content in this scene to create a unique HoloLens app.

Vuforia Image

The ImageTarget GameObject encapsulates the ImageTargetBehavior and tracking event handling scripts. These are the primary script components used to customize ImageTargets in a HoloLens application.

Vuforia Image

For the Image Target Behaviour script, the Database dropdown selects the database that contains the Image Target assigned to this Image Target Behaviour.

The Image Target dropdown defines which target in the database to assign to this Image Target Behaviour.

The Trackable Status Event Handler script is a customized version of Vuforia’s built-in Default Trackable Event Handler script. It registers callbacks from the Image Target Behaviour arising from changes in the state of the Image Target, such as when it has been detected and is then being tracked.

This script is used to enable and disable rendering and collision detection on digital content that is a child of the target. The script also exposes OnTargetFound() and OnTargetLost() events that can be used to control application-specific behavior. In this sample the events are used to display the target state in the UI:

Vuforia Image

The 2-ModelTargets and 2-VuMarks scenes demonstrate the setup of Vuforia’s Model Target and VuMark features on HoloLens. They follow the same principle as the targets Image Target scene:

  • A target game object allows configuration of the properties of the target it represents, e.g. it’s database and physical size.
  • Virtual content is parented to the target as child objects.
  • An event handling script it attached to the game object to handle events when the target is found or lost.
Vuforia Image