Vuforia Fusion

Vuforia Fusion is a capability, introduced with Vuforia Engine 7, that is designed to provide the best possible AR experience on a wide range of devices.

Fusion solves the problem of fragmentation in AR-enabling technologies, including cameras, sensors, chipsets, and software frameworks such as ARKit and ARCore. It senses the capabilities of the underlying device and fuses them with Vuforia Engine features, allowing developers to rely on a single Vuforia Engine API for an optimal AR experience. Vuforia Fusion brings advanced Vuforia Engine features to devices that support ARKit and ARCore, in addition to over 100 other Android and iOS device models.

Features

Vuforia Fusion is used by several Vuforia Engine features such as:

Device Tracker - Providing six-degree-of-freedom Device Pose
Ground Plane - Allowing virtual content to be placed on horizontal planes in the environment
Extended Tracking - enabling extended tracking for all Vuforia target types

Fusion Technology Priority

The priority of technology used by Vuforia Fusion has been optimized to leverage the richest set of hardware/software enablers while also providing developers the widest possible device coverage. Refer to the image below for an illustration of how the Vuforia Engine leverages hardware and software enablers.

In practice, what the above diagram shows is that an AR experience created using the Vuforia Engine will attempt to use the top most technology and work it's way downward dependent on what is available on the device during runtime.

For instance, a Vuforia App running on an iPhone X, will automatically leverage ARKit for all features dependent on Vuforia Fusion. That same application running on an older iOS device - which does not support ARKit but has been calibrated by Vuforia - will automatically use VISLAM for dependent features.

The same scenario appies for Android devices. A Vuforia App running on a Google Pixel 2 XL will leverage ARCore. That same application on a device that does not support ARCore will use VISLAM assuming that the device has the correct sensors and has been calibrated by Vuforia.

Due to addition of support for platform enablers, it is important that AR content, targets, and physical objects all have their coordinate scale in meters. Objects that do not have their scale set appropriately may not track well.

Devices

For a list of devices that support ARKit or ARCore, please refer to Apple's ARKit Site or Google's ARCore Page.

As of Vuforia Engine 7.2, UWP (Universal Windows Platform) handheld devices only leverage VIO or SLAM for dependent features. Starting Vuforia Engine 8.0 most of these devices leverage VISLAM, providing a more robust AR experience. For a complete list of devices that support Vuforia VISLAM - please refer to Vuforia Fusion Supported Devices.

Older Devices and Overheating

In an effort to offer developers access to the broadest range of devices, the Vuforia Engine supports various older devices. Developers targeting older devices should be aware that these devices may not always have the hardware specifications required to sustain augmented reality experiences for long periods of time. Therefore, developers targeting such devices are encouraged to design shorter experiences. Running augmented reality experiences on such devices for prolonged periods can cause the device to overheat and/or battery drain.

There are several safeguards in devices to prevent damage to the device or injury to users. One such safeguard is that the speed of the CPU and/or GPU are throttled in order to prevent the heat to cause permanent damage. When the CPU and/or GPU is throttled, augmented reality experiences may not behave as expected. For instance, augmentations may fail to be properly anchored to the environment or objects.

VISLAM Behavior

Visual-Inertial Simultaneous Localization And Mapping (VISLAM) is an algorithm implemented by Vuforia combining the benefits of Visual-Inertial Odometry (VIO) and Simultaneous Localization And Mapping (SLAM). VISLAM has several benefits:

it generally works better in low-feature environments compared to SLAM based tracking,
provides an estimate of the scale of the world (i.e., the DeviceTracker motion estimate will correspond to the real world motion), and
provides robustness to recover when tracking is lost completely over VIO solutions.

When using Ground Plane, Vuforia Engine's VISLAM system will attempt to estimate the distance between the camera and the Ground Plane. To provide a smooth end-user-experience, Vuforia Engine will use the initial hints to scale based on height to the floor provided via developer API. The default of this hint is 1.4 meters - the average height of an adult holding a device in their hands. In some cases you can see the scale-estimation at work in the initialization phase of your app when the augmentation is corrected in apparent size when placed onto the Ground Plane.

Fusion APIs

In some edge-cases developers may want to have tighter control over which Fusion Provider is used by the Vuforia Engine. The Following APIs are available to change the priority order of the Fusion Providers.

The Vuforia::setAllowedFusionProviders (DeviceTracker.FusionMode in Unity) API takes a bitwise combination of the values as described below.

Fully automated provider selection

FUSION_PROVIDER_ALL - Vuforia Engine will select the best Fusion provider automatically. This is the default setting and should be used for most use-cases requiring Positional Device Tracker. Fusion Providers are prioritized as described above (platform enablers first, next is VISLAM then SLAM).

Enforcement of specific provider

FUSION_PROVIDER_PLATFORM_SENSOR_FUSION - Forces only platform enablers such as ARKit and ARCore. If none is available Vuforia::init() will fail.
FUSION_PROVIDER_VUFORIA_SENSOR_FUSION - Forces only Vuforia sensor fusion (VISLAM) technology only. If device was not calibrated by Vuforia, the Vuforia::init() call will fail.
FUSION_PROVIDER_VUFORIA_VISION_ONLY - Forces visual SLAM technology only without the use of sensors. Shall succeed on all devices, performance identical to Extended Tracking behavior in Vuforia Engine versions prior to SDK 7.0. No Anchors are supported with this Fusion provider.

In cases where the parameter flag does not modify the FUSION_PROVIDER_PLATFORM_SENSOR_FUSION bitwise value, the setAllowedFusionProvider can be set after Vuforia Engine initializes but before any trackers are initialized. In cases where the flag does modify FUSION_PROVIDER_PLATFORM_SENSOR_FUSION, the setAllowedFusionProvider must be set before Vuforia initializes.

Query

In the situation where you need to determine what is the current active Fusion provider, use the getActiveFusionProvider()(VuforiaRuntimeUtilities.GetActiveFusionProvider() in Unity) function. This call returns a specific value indicating what Fusion provider is being actively used by the Vuforia Engine. The return value is one of the listed items above in the Enforcement of specific provider section.

Note: The below feature based optimized settings provided earlier have been deprecated in Vuforia Engine 8.0 and will be removed in a future version:

FUSION_OPTIMIZE_MODEL_TARGETS_AND_SMART_TERRAIN - This is the default setting and should be used for most use-cases requiring Positional Device Tracker for Ground Plane and to enable best-in-class Extended Tracking for Model Targets.
FUSION_OPTIMIZE_IMAGE_TARGETS_AND_VUMARKS - This setting is recommended for experiences that require Extended Tracking support for ImageTargets and VuMark-based use cases. This setting prioritizes platform enablers (such as ARKit and ARCore) and visual SLAM to provide stable augmentations with good recovery characteristics.

Relocalization

Relocalization is the capability of a tracking system to recover tracking result after a short failure without the need to restart the experience. In general, it is a method to cover smaller gaps during tracking function.

In Vuforia Engine, relocalization behavior is highly dependent on the active Fusion Provider during a session.

In order to determine if Vuforia Engine is attempting to relocalize, refer to DeviceTracker.getStatus() and DeviceTracker.getStatusInformation() functions.

When tracking is lost when and getActiveFusionProvider() returns FUSION_PROVIDER_VUFORIA_SENSOR_FUSION, augmentations may not appear where expected. In this situation, getStatus() returns LIMITED. To help the system recover, guide the user to redetect a previous seen target or location to continue the experience. If a new target is detected that the system has not seen previously in this session, it will begin tracking this new target and will recover additional targets and anchors as they come into view (unfortunately, mid-air anchors cannot be recovered). For more detailed information on tracking state, please refer to Interpreting Tracking State API Results.