Working with the HoloLens Sample in Unity

This article demonstrates how to use the Vuforia HoloLens Sample project to author a Vuforia Engine scene for HoloLens and customize event handling to implement unique app behaviors. The sample project contains pre-configured Unity scenes and project settings that act as a starting point and reference for your own apps. It also demonstrates how to use Microsoft’s Mixed Reality Toolkit (MRTK) in Vuforia applications e.g. to work with hand gestures on the HoloLens 2. For additional background information on working with HoloLens, refer to the Developing Vuforia Apps for HoloLens article.

The supported versions page lists the operating systems, tools, and device versions supported by Vuforia Engine.

Getting Started with Visual Studio and Unity

  1. Ensure that the minimum-supported Visual Studio and Unity versions are installed (refer to the Vuforia Engine Supported Versions page).
    • For the MRTK and developing for HoloLens 2, Visual Studio 2019 is required.
    • When installing Unity, make sure to install the IL2CPP Scripting Backend.  
  2. Configure the Unity Editor to use Visual Studio as the default IDE.      
  3. Recommended:

For more information on setting up your Windows 10 build environment, refer to the Developing for Windows 10 in Unity article. 

Getting Started with the HoloLens Sample

Importing the sample

  1. Select your Unity version:
  2. For... Then…
    Unity 2018.4 LTS Download the Vuforia Engine installer for Unity 2018.4 as well as the HoloLens sample package from the Vuforia developer portal
    Unity 2019.2 or later The sample automatically references the correct Vuforia Engine package available in Unity’s Package Manager.

When using Unity 2019.2 or later:

  1. Create a new Unity project.
  2. In the Window menu, select Asset Store.
  3. In the search bar, enter Vuforia HoloLens.
  4. Select Vuforia HoloLens Sample.
  5. Click Import.
Vuforia Image

Building and executing the sample

  1. In the File menu, select Build Settings.
  2. In the Platform section, select Universal Windows Platform and click Switch Platform.
  3. From the Target Device dropdown, select HoloLens.
  4. From the Build Type dropdown, select D3D Project.
  5. From the Target SDK Version dropdown, select Latest Installed.
  6. From the Visual Studio Version dropdown, select Visual Studio 2019.
Vuforia Image
  1. Click the Player Settings... button.
  2. Select the UWP icon and expand the XR Settings section.
  3. Ensure that Virtual Reality Supported is selected.    
  4. Under Virtual Reality SDKs ensure that:
    • Window Mixed Reality is included in the list and that Enable Depth Buffer Sharing is selected. 
    • The Depth Format is set to 16-bit depth.
  5. Ensure that the Stereo Rendering Mode is set to Single Pass Instanced.
Vuforia Image
  1. Expand the Publishing Settings section.
  2. Under Capabilites ensure that Internet ClientWebCam, Microphone, and SpatialPerception are selected.
    Note: SpatialPerception should only be selected if you intend to use the Surface Observer API.
  3. Under Supported Device Families ensure that Holographic is selected. 
  4. Expand the Resolution and Presentation section.
  5. Disable Run in Background so that Vuforia pauses when the app is put into the background and can access the camera again when the app is resumed.
  6. In the Default Orientation dropdown, ensure that Landscape Left is selected.
  7. Close the Project Settings window.
  8. In the Build Settings window, click Build to generate a Visual Studio project.
  9. In the Windows Explorer dialog that appears, create a new folder to hold Unity's build output. Generally, we name the folder "App".
  10. Select the newly created folder and click Select Folder.
  11. Once Unity has finished building, a Windows Explorer window opens to the project root directory. Navigate into the newly created folder.
  12. Open the generated Visual Studio solution file located inside this folder.
    The project opens in Visual Studio.
  13. In the Solution Platforms dropdown, select:
    •  x86 if you are building for HoloLens 1
    • ARM64 if you are building for HoloLens 2
  14. Select Release in the dropdown menu next to the Solution Platform.
Vuforia Image

For more information, refer to Microsoft’s Exporting and building a Unity Visual Studio solution and Using Visual Studio to deploy and debug tutorials.

NOTE: In some cases, Unity exports the Visual Studio project with an incorrect toolset setup. If the build fails with the error “error MSB8020: The build tools for Visual Studio 2017 (Platform Toolset = 'v141') cannot be found.”, in the Project menu select Retarget Solution.
In the Review Solution Actions set the Platform Toolset to Upgrade to v142 and click OK.

Vuforia Image

Scene Elements and Configuration

The HoloLens sample uses the ImageTarget, VuMark, and ModelTarget features. The sample scenes are located in the SampleResources folder along with other assets and resources in this sample.

The scene 0-Base shows the basic setup for development on HoloLens, combining the MixedRealityToolkit with Vuforia components.
All other scenes are loaded additively to add target-specific functionality.

Camera Setup

Vuforia’s life cycle and events are controlled by the Vuforia Behaviour script. In the Vuforia HoloLens sample, this script has been added to the MRTK camera in the 0-Base scene.

Vuforia Image

NOTE: When not using the MRTK, a ready-to-use camera object with the Vuforia Behaviour already attached can be added to a scene using the GameObject > Vuforia Engine > AR Camera menu item.

In the Vuforia Behaviour script, the World Center Mode dropdown defines which object in the scene hierarchy serves as the world origin (0,0,0) of the scene's world space. In HoloLens, only the Device option is supported.

Vuforia Image

Starting with version 8.5, Vuforia Engine automatically detects if an app is running on HoloLens in Unity. It is no longer necessary to configure the Digital Eyewear settings in the Vuforia Configuration window.

Target GameObjects

The scenes that show how to set up Vuforia targets and virtual content for them are loaded additively. The 2-ImageTargets scene shows a basic setup for multiple Image Targets.

You can easily substitute your own content in this scene to create a unique HoloLens app.

Vuforia Image

The ImageTarget GameObject encapsulates the ImageTargetBehavior and tracking event handling scripts. These are the primary script components used to customize ImageTargets in a HoloLens application.

Vuforia Image

For the Image Target Behaviour script, the Database dropdown selects the database that contains the Image Target assigned to this Image Target Behaviour.

The Image Target dropdown defines which target in the database to assign to this Image Target Behaviour.

The Trackable Status Event Handler script is a customized version of Vuforia’s built-in Default Trackable Event Handler script. It registers callbacks from the Image Target Behaviour arising from changes in the state of the Image Target, such as when it has been detected and is then being tracked.

This script is used to enable and disable rendering and collision detection on digital content that is a child of the target. The script also exposes OnTargetFound() and OnTargetLost() events that can be used to control application-specific behavior. In this sample the events are used to display the target state in the UI:

Vuforia Image

The 2-ModelTargets and 2-VuMarks scenes demonstrate the setup of Vuforia’s Model Target and VuMark features on HoloLens. They follow the same principle as the targets Image Target scene:

  • A target game object allows configuration of the properties of the target it represents, e.g. it’s database and physical size.
  • Virtual content is parented to the target as child objects.
  • An event handling script it attached to the game object to handle events when the target is found or lost.
Vuforia Image