Using the MixedRealityController in Unity

The MixedRealityController is a utility class that supports the development of mixed reality AR/VR apps in Unity. This class enables you to both define the AR/VR mode of your scene and to gracefully transition between modes.

The MixedRealityController supports three digital eyewear modes and three handheld modes. You can use these modes to develop AR/VR apps for both VR viewers and handheld devices, as well as dual mode apps that transition between and handheld and viewer modes.
 
ModeStereoscopyCamera VideoRotational TrackingDescription
VIEWER_ARYESYESNOStereo AR
VIEWER_VR YESNOYESUsing Head Rotational Model
ROTATIONAL_VIEWER_ARYESYESYESUsing Head Rotational Model
HANDHELD_ARNOYESNOStandard AR
HANDHELD_VRNONOYESUsing Handheld Rotational Model
ROTATIONAL_HANDHELD_ARNOYESYESUsing Handheld Rotational Model

Using the Mixed Reality Controller

The MixedRealityController automatically handles most of the tasks necessary for switching between AR/VR modes for digital eyewear and handheld use cases. Its use is demonstrated in the TransitionManager.cs script in the AR/VR Digital Eyewear Sample. 

Role:
  • Enable and disable stereo rendering
  • Enable and disable video background rendering
  • Activate and deactivate device tracking for VR cases
  • Set target FPS to optimize the performance of digital eyewear apps 
  • Applies a viewer configuration if previously defined
The MixedRealityController class is a singleton that exposes a single method to switch among AR/VR modes:

Example - setting the mode to support stereo rendering and rotational tracking with no camera view
MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_VR);

Each of the mixed reality modes is associated with an enum field in the MixedRealityController.Mode enumeration. The name of each field and behavior of its associated mode is described in the table above.

Switching from Handheld to Viewer Modes

If your mixed reality experience starts in a handheld mode and transitions to a digital eyewear mode using a viewer, you'll need to define which viewer type to use and then set your scene to a viewer based mode using SetMode(). This step is not necessary if a Viewer Type had already been defined for your app in the Unity Editor within Digital Eyewear Behavior of the ARCamera.
MixedRealityController.Instance.SetViewerParameters(viewerParameters);
However, if an app is configured to start in handheld (mono) mode, you need to tell it which viewer to use by calling the following method once:
MixedRealityController.Instance.SetViewerParameters(viewerParameters);
Viewer parameters for preconfigured viewers can be obtained from the Device singleton by calling GetViewerList().
Device.Instance.GetViewerList();

Querying and setting available viewers:

// find all names in the viewer list:
IEnumerable<IViewerParameters> viewerParameters = Device.Instance.GetViewerList().GetAllViewers();
 
// set Cardboard v1 viewer:
foreach (IViewerParameters vp in viewerParameters)
    if (vp.GetName().Equals("Cardboard v1"))
        MixedRealityController.Instance.SetViewerParameters(vp);
 
// now tell Vuforia that it's being put into the headset:
MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_AR);
 

Using AR with VR

The MixedRealityController makes it easy to develop immersive VR apps that use AR functionality. You can incorporate real objects into your VR scene by employing Image and Object Targets, or any of the other Vuforia target and marker types. When your app transitions from AR into its VR mode, the pose of your target will be retained in the VR scene and its proxy can then be tracked using rotational tracking. 

To support AR with VR, you'll need to use the both of the rotational AR and VR modes. It's recommended that you initiate the experience using the ROTATIONAL_VIEWER_AR mode to enable the user to locate the target in their setting. Set the World Center Mode for your project to either FIRST_TARGET or SPECIFIC_TARGET

MixedRealityController.Instance.SetMode(MixedRealityController.Mode.ROTATIONAL_VIEWER_AR);
Once the target is detected and is being tracked, you can transition into a rotational VR mode.
MixedRealityController.Instance.SetMode(MixedRealityController.Mode.VIEWER_VR);
Use the last pose of the target from the AR mode and to position a virtual proxy for it in the same position within the user's field of view in your VR scene. Once positioned, the proxy will continue to be tracked by the Rotational Device Tracker. 

Tips:
  • Be sure that the physical target is positioned with the correct orientation
  • Align the bottom of the target parallel to the floor
  • Use the same camera fields of view for both the AR and VR scenes
  • Use a transition effect of conceal slight differences in the target pose between the AR and VR modes