Developing Vuforia Apps for HoloLens

Overview

Vuforia brings an important capability to HoloLens – the power to connect AR experiences to specific images and objects in the environment. You can use this capability to overlay guided step by step instructions on top of machinery or to add digital features to a physical product.
 
Enterprise developers – you can use VuMarks to uniquely identify each piece of machinery on a factory floor – right down to the serial number. VuMarks are scalable into the billions and can be designed to look just like a company logo. They are the ideal solution for adding AR to any product that HoloLens can see.
 
Existing Vuforia apps that were built for phones and tablets can easily be configured in Unity to run on HoloLens. You can even use Vuforia to take your new HoloLens app to Windows 10 tablets such as the Surface Pro 4 and Surface Book.

Supported Versions

Installation and Configuration

You will need to install the above versions of Visual Studio and Unity and then configure Unity to use Visual Studio and the preferred IDE and compiler. You will also need to install Visual Studios tools for Unity, see the article below for a link to this installer.

See:
Getting Started with Visual Studio Tools for Unity - MSDN
Unity - Manual: Visual Studio C# Integration
 
When installing Unity, be sure to install the Windows Store platform, and also the .Net Scripting Backend. Windows Store components can also be installed afterwards from the Build Settings dialog when the Windows Store platform is selected.

Getting Started

Vuforia’s support for HoloLens is implemented in version 6.1 of the Vuforia Unity extension.
See: Developing for Windows 10 in Unity to learn how to set-up your Windows 10 build environment for Unity and Visual Studio 2015.

The best starting point, to understand the structure and organization of a Vuforia HoloLens project in Unity, is with the Unity HoloLens sample.  This provides a complete HoloLens project that includes the Vuforia Unity Extension for Win 10 as well as a pre-configured scene that can be deployed to a HoloLens.

The scene implements a Vuforia to HoloLens camera binding, along with the Build Settings necessary to deploy a HoloLens app. It shows how to use Image Targets and Extended Tracking to recognize an image and augment it with digital content in a HoloLens experience.
You can easily substitute your own content in this scene to begin experimenting with the creation of HoloLens apps that use Vuforia.
To deploy the sample, follow the steps in the Building and Executing the sample section.  When you build a Windows Store app, Unity generates a Visual Studio project and launches the Visual Studio IDE. You can then build a UWP executable and install that on your HoloLens.


Installing the Unity Extension

See: Getting Started with Vuforia for Unity Development 
 

Installing the Unity HoloLens Sample

See: Working with the HoloLens sample in Unity


Configuring a Vuforia App for HoloLens

Developing a Vuforia app for HoloLens is fundamentally the same as developing Vuforia apps for other devices. You’ll simply need to apply a binding between the Vuforia ARCamera and the HoloLens scene camera, and enable Extended Tracking on your targets. You can then apply the build settings described in the Building and Executing a Vuforia app for HoloLens section. That’s all that’s needed to enable Vuforia to work with the HoloLens spatial mapping and positional tracking systems.

  1. Enable Extended Tracking on your targets
  2. Bind the ARCamera to the HoloLens scene camera

The Role of Extended Tracking

Extended Tracking creates a map of the environment to maintain tracking even when a target is no longer in view. It is Vuforia’s counterpart to the Spatial Mapping performed by HoloLens.

When you enable Extended Tracking on a target, you enable the pose of that target to be passed to the HoloLens Spatial Mapping engine. In this way, targets can exist in both the Vuforia and HoloLens spatial coordinate systems, though not simultaneously.



Enabling Extended Tracking on a Target

Vuforia will automatically transform the pose of a target that uses Extended Tracking into the HoloLens spatial coordinate system.  This allows HoloLens to take over tracking, and to integrate any content augmenting the target into the spatial map of the target’s surroundings.
This process occurs between the Vuforia SDK and HoloLens APIs in Unity, and does not require any programming by the developer - it's handled automatically.

  Here is what occurs...

  1. Vuforia’s target Tracker recognizes the target
  2. Target tracking is then initialized
  3. The position and rotation of the target are analyzed to provide a robust pose estimate for HoloLens to use
  4. Vuforia transforms the target's pose into the HoloLens spatial mapping coordinate space
  5. HoloLens takes over tracking and the Vuforia tracker is deactivated 

The developer can control this process, to return control to Vuforia, by disabling Extended Tracking on the TargetBehaviour.

Binding the HoloLens scene camera

Vuforia uses the ARCamera prefab as its scene camera in a Unity project. You’ll need to bind the scene camera used by HoloLens to the ARCamera to enable Vuforia to interact with HoloLens

The Vuforia HoloLens sample shows how to configure the scene camera used by HoloLens. Also see: https://developer.microsoft.com/en-us/windows/holographic/camera_in_unity


The HoloLens scene camera is a standard Unity Camera instance that has been configured for HoloLens apps.

Follow these steps to bind the HoloLens scene camera to the Vuforia ARCamera :

1. Add an ARCamera and HoloLens scene camera to your scene Hierarchy
2. Set up the scene for stereo rendering in the VuforiaConfiguration asset.

  • Select Eyewear type = Optical See-Through

3. Select HoloLens as the device configuration

  • See Through Config = HoloLens

4. Bind the HoloLens scene camera to the ARCamera

  • Drag the HoloLensCamera onto the Central Anchor Point field of ARCamera's VuforiaBehaviour.

 

Building and executing a Vuforia app for HoloLens

1. Add an Eyewear App License Key
2. Apply the recommended Unity Engine Options for Power and Performance
3. Add the sample scenes to Scenes in Build

Note:  You may need to install additional Unity Windows Store components if you had not selected to install these when you had originally installed the Unity Editor – the dialog will direct you to the necessary installer.



4. Set your platform build target for Windows Store in File > Build Settings.
5. Select the following platform build configuration settings

  • SDK = Universal10
  • UWP Build Type = D3D
User-added image Selecting Build for Windows Store will generate a native project archive for those platforms.  You will then need to build an executable from Visual Studio. Use the SDK and UWP Build Type settings shown in this image.

6. Define a unique Product Name, in Player Settings, to serve as the name of the app when installed on the HoloLens.
7. Select Landscape Left as the in Player Settings > Resolution and Presentation
8. Check Virtual Reality Supported + Windows Holographic in Player Settings > Other Settings
User-added image

9. Check the following Capabilities in Player Settings > Publish Settings

  • InternetClient

  • WebCam

  • SpatialPerception - if you intend to use the Surface Observer API

User-added image

10. Select Build to generate a Visual Studio project
11. Build the executable from Visual Studio and install it on your HoloLens.
See: https://developer.microsoft.com/en-us/windows/holographic/exporting_and_building_a_unity_visual_studio_solution

Visual Studio Build Configuration

Be sure to set your build target for x86. Note that the EAP release supports only 32 bit builds.

©2011-2017 PTC Inc. All rights reserved. Nothing in these materials is an offer to sell any of the components or devices referenced herein. PTC and Vuforia are trademarks or registered trademarks of PTC Inc. in the United States and other countries. Other products and brand names referenced may be trademarks or registered trademarks of their respective owners. Vuforia is a product of PTC Inc.