Using the MixedRealityController in Unity

The MixedRealityController is a utility class that supports the development of mixed reality AR/VR apps in Unity. This class enables you to both define the AR/VR mode of your scene and to gracefully transition between modes.

The MixedRealityController supports three digital eyewear modes and three handheld modes. You can use these modes to develop AR/VR apps for both VR viewers and handheld devices, as well as dual mode apps that transition between and handheld and viewer modes.

Mode

Stereoscopy

Camera Video

Rotational/Positional Tracking

Description

VIEWER_AR YES YES NO Stereo AR
VIEWER_VR YES NO YES Using Head Rotational Model
VIEWER_AR_DEVICETRACKER YES YES YES Using Head Rotational Model
HANDHELD_AR NO YES NO Standard AR
HANDHELD_VR NO NO YES Using Handheld Rotational Model
HANDHELD_AR_DEVICETRACKER NO YES YES Using Handheld Rotational Model

Using the Mixed Reality Controller

The MixedRealityController automatically handles most of the tasks necessary for switching between AR/VR modes for digital eyewear and handheld use cases. Its use is demonstrated in the TransitionManager.cs script in the AR/VR Digital Eyewear Sample.

Role:

  • Enable and disable stereo rendering
  • Enable and disable video background rendering
  • Activate and deactivate device tracking for VR cases
  • Set target FPS to optimize the performance of digital eyewear apps
  • Applies a viewer configuration if previously defined

The MixedRealityController class is a singleton that exposes a single method to switch among AR/VR modes:

Example - setting the mode to support stereo rendering and rotational tracking with no camera view

MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_VR);

Each of the mixed reality modes is associated with an enum field in the MixedRealityController.Mode enumeration. The name of each field and behavior of its associated mode is described in the table above.

Switching from Handheld to Viewer Modes

If your mixed reality experience starts in a handheld mode and transitions to a digital eyewear mode using a viewer, you'll need to define which viewer type to use and then set your scene to a viewer based mode using SetMode(). This step is not necessary if a Eyewear Type had already been defined for your app in the Unity Editor within Digital Eyewear Section of the Vuforia Configuration.

MixedRealityController.Instance.SetViewerParameters(viewerParameters);

Viewer parameters for preconfigured viewers can be obtained from the Device singleton by calling GetViewerList().

Device.Instance.GetViewerList();

Querying and setting available viewers:

// find all names in the viewer list:
IEnumerable<IViewerParameters> viewerParameters = Device.Instance.GetViewerList().GetAllViewers();
 
// set Cardboard v1 viewer:
foreach (IViewerParameters vp in viewerParameters)
    if (vp.GetName().Equals("Cardboard v1"))
        MixedRealityController.Instance.SetViewerParameters(vp);
 
// now tell Vuforia that it's being put into the headset:
MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_AR);

Using AR with VR

The MixedRealityController makes it easy to develop immersive VR apps that use AR functionality. You can incorporate real objects into your VR scene by employing Image and Object Targets, or any of the other Vuforia target and marker types. When your app transitions from AR into its VR mode, the pose of your target will be retained in the VR scene and its proxy can then be tracked using rotational tracking.

To support AR with VR, you'll need to use the both of the rotational/positional AR and VR modes. It's recommended that you initiate the experience using the VIEWER_AR_DEVICETRACKER mode to enable the user to locate the target in their setting. Set the World Center Mode for your project to either FIRST_TARGET or SPECIFIC_TARGET.

MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_AR_DEVICETRACKER);

Once the target is detected and is being tracked, you can transition into a rotational VR mode.

MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_VR);

Use the last pose of the target from the AR mode and to position a virtual proxy for it in the same position within the user's field of view in your VR scene. Once positioned, the proxy will continue to be tracked by the Rotational Device Tracker.

Using Rotational and Positional Tracking

For devices that support positional tracking (such as Google Tango), developers can choose to use Positional Tracking by selecting "POSITIONAL" Tracking mode in the Device Tracker section of the Vuforia Configuration panel. This will enable 6DOF (Degrees of Freedom) tracking on the device.

Tips:

  • Be sure that the physical target is positioned with the correct orientation
  • Align the bottom of the target parallel to the floor
  • Use the same camera fields of view for both the AR and VR scenes
  • Use a transition effect of conceal slight differences in the target pose between the AR and VR modes