Vuforia Fusion

Vuforia Fusion is a set of technologies designed to provide the best possible AR experience on a wide range of devices.

Fusion solves the problem of fragmentation in AR-enabling technologies, including cameras, sensors, chipsets, and software frameworks such as ARKit, ARCore, Magic Leap, and Windows Holographic for HoloLens 2. It senses the capabilities of the underlying device and fuses them with Vuforia Engine features, allowing developers to rely on a single Vuforia Engine API for an optimal AR experience. Vuforia Fusion brings advanced Vuforia Engine features to devices that support ARKit and ARCore, in addition to other Android and iOS device models.

Features

Vuforia Fusion is used by several Vuforia Engine features, such as:

  • Device Tracking – providing six-degree-of-freedom Device Pose.
  • Image Targets / Cylinder Targets / Multi Targets / VuMark / Barcode – enabling reliable tracking.
  • Model Targets – enabling reliable tracking around large objects.
  • Area Targets – enabling tracking of entire spaces.
  • Ground Plane - Allowing virtual content to be placed on horizontal planes in the environment.

Fusion Technology Priority

The priority of technology used by Vuforia Fusion has been optimized to leverage the latest set of hardware/software enablers while also providing developers with the broadest possible device coverage. The image below illustrates how the Vuforia Engine leverages hardware and software enablers.

The above diagram shows that an AR experience created using the Vuforia Engine will attempt to use the topmost technology and work its way downward depending on what is available on the device during runtime.

For example, a Vuforia App running on an ARKit-enabled device will automatically leverage ARKit for all features dependent on Vuforia Fusion. That same application running on an older iOS device - which does not support ARKit but has been calibrated by Vuforia - will automatically use Vuforia VISLAM for dependent features.

On Android devices, a Vuforia App running on an ARCore-enabled device will leverage ARCore when it is included by the developer. That same application on a device that does not support ARCore will use Vuforia VISLAM if the device has the required sensors and has been calibrated by Vuforia.

Due to the addition of support for platform enablers, AR content, targets, and physical objects must have their coordinate scale in meters. Objects that do not have their scale set appropriately may not track well.

Devices

For a list of devices that support ARKit or ARCore, please refer to Apple's ARKit Site or Google's ARCore Page.

For a list of recommended devices for use with Vuforia Engine, refer to Vuforia Engine Recommended Devices.

Older Devices and Overheating

To offer developers access to the broadest range of devices, the Vuforia Engine supports various older devices. Developers targeting older devices should be aware that these devices may only sometimes have the hardware specifications required to sustain augmented reality experiences for extended periods. Therefore, developers targeting such devices are encouraged to design shorter experiences. Running augmented reality experiences on such devices for prolonged periods can cause the device to overheat and drain the battery.

There are several safeguards in devices to prevent damage to the device or injury to users. One such safeguard is that the CPU and GPU speed are throttled to avoid the heat from causing permanent damage. When the CPU or GPU is throttled, augmented reality experiences may not behave as expected. For instance, augmentations may fail to be adequately anchored to the environment or objects.

Vuforia Engine offers various options for adjusting the video mode for better battery life or performance. See more at Device Performance Optimizations.

VISLAM Behavior

Visual-Inertial Simultaneous Localization And Mapping (VISLAM) is an algorithm implemented by Vuforia combining the benefits of Visual-Inertial Odometry (VIO) and Simultaneous Localization And Mapping (SLAM). Vuforia VISLAM has several benefits:

  • It works better in low-feature environments than SLAM-based tracking.
  • It provides an estimate of the scale of the world (i.e., the Device Tracking motion estimate corresponds to the real-world motion).
  • It provides robustness to recover when tracking is entirely lost over VIO solutions.

When using Ground Plane, Vuforia Engine's VISLAM system will attempt to estimate the distance between the camera and the Ground Plane. To provide a smooth end-user experience, Vuforia Engine will use the initial hints to scale content based on the height from the floor, as provided via the API. The default of this hint is 1.4 meters - the average height of an adult holding a device in their hands. In some cases, you can see the scale estimation at work in the initialization phase of your app when the augmentation is corrected to an apparent size when placed onto the Ground Plane.

When using VISLAM, some environments that only have edgy structures and fine lines and no textures may perform less robustly. In addition, inform users that your app does not allow them to make too fast motions; it can cause augmentations to shift their relative position to the target. Reapproaching the target or resetting Device Tracking will correct the augmentations’ positions.

NOTE: For Vuforia projects that compile for or target Android API level 31 or above, you must add the permission android.permission.HIGH_SAMPLING_RATE_SENSORS to the manifest.xml file to ensure VISLAM can operate correctly. This must be done manually when using the Vuforia Unity extension version 10.7 or lower; for version 10.8 and higher, this permission is included by the extension.

Fusion APIs

Before initializing, you can enable or disable the Fusion Provider-specific configuration data for the Vuforia Engine. Disabling the fusion provider will set Vuforia Engine to select the best choice between VISLAM or SLAM.

If enabled, Vuforia Engine will select the best Fusion provider automatically. The Fusion Provider prioritizes as described above (platform enablers first, VISLAM next, and then SLAM). If the Engine cannot set the platform fusion provider, Vuforia falls back to using a non-platform-based fusion provider.

For Unity

Set the Fusion Provider configuration with VuforiaApplication.Instance.Initialize(FusionProviderOption). The options can be PREFER_PLATFORM_FUSION_PROVIDER and VUFORIA_FUSION_ONLY. See Vuforia Engine lifecycle in Unity for a code snippet that applies the option or the Reference API library for details on the options.

Query

When determining the current active Fusion provider, use the VuforiaRuntimeUtilities.GetActiveFusionProvider(). This returns the FusionProviderType that the Vuforia Engine is actively using. The return values are described in the reference library.

If the query returns PLATFORM_SENSOR_FUSION, you can then set FusionProviderOption to PREFER_PLATFORM_FUSION_PROVIDER to make use of tracking services by the underlying platform, such as ARKit, ARCore, and Windows Holographic.

For Native

Add a Vuforia Fusion Provider configuration to the engine config with vuEngineConfigSetAddFusionProviderConfig by setting VU_TRUE or VU_FALSE in the usePlatformFusionProvider from the VuFusionProviderConfig. The default value is VU_TRUE.

Query

When determining the current active Fusion provider, use the vuPlatformControllerGetFusionProviderType(). This returns the VuFusionProviderType that the Vuforia Engine is actively using. The return values are described in the reference library.

If the query returns VU_FUSION_PROVIDER_TYPE_PLATFORM_SENSOR_FUSION, you can then use vuPlatformControllerGetFusionProviderPlatformType to get the VuFusionProviderPlatformType to make use of tracking services by the underlying platform, such as ARKit, ARCore, and Windows Holographic. See more on platform functionalities in Using the Vuforia Fusion Platform Handle.

Continued AR Experiences

Vuforia Engine offers developers a toolset to manage and create AR sessions sustained over application interruptions and pauses. By using Device Tracking and its reported pose status, you can regain tracking of all Vuforia targets upon resuming the AR session. See the Best Practices for Continued AR Experiences page for detailed information on application continuity and Device Pose Behaviour statuses suitable for the different Vuforia Fusion platforms.

Relocalization

Relocalization allows a tracking system to recover tracking results after a short failure, without restarting the experience, by covering small gaps during the tracking function.

For Unity

To determine whether Vuforia Engine is attempting to relocalize, refer to the TargetStatus() and the values it returns. See Pose Status and Status Info.

In Vuforia Engine, relocalization behavior highly depends on the active Fusion Provider during a session.

When tracking is lost and FusionProviderType() returns VISION_ONLY, the tracking system resets itself if it detects a previously unseen target. In this situation, the system starts tracking the new target and recovers additional targets as they come into view.

When tracking is lost and FusionProviderType() returns SENSOR_FUSION or VISION_ONLY, augmentations may not appear where expected. In this situation, TargetStatus() returns LIMITED. To help the system recover and continue the experience, instruct the user to point the device back to a previously viewed target or location, depending on the nature of the experience.

For Native

To determine whether Vuforia Engine is attempting to relocalize, refer to the vuObservationGetPoseInfo() function and the values it returns. See Pose Status and Status Info.

In Vuforia Engine, relocalization behavior highly depends on the active Fusion Provider during a session.

When tracking is lost and vuPlatformControllerGetFusionProviderType() returns VU_FUSION_PROVIDER_TYPE_VISION_ONLY, the tracking system resets itself if it detects a previously unseen target. In this situation, the system starts tracking the new target and recovers additional targets as they come into view.

When tracking is lost and vuPlatformControllerGetFusionProviderType() returns VU_FUSION_PROVIDER_TYPE_SENSOR_FUSION or VU_FUSION_PROVIDER_TYPE_VISION_ONLY, augmentations may not appear where expected. In this situation, vuObservationGetPoseInfo() returns VU_OBSERVATION_POSE_STATUS_LIMITED. To help the system recover and continue the experience, instruct the user to point the device back to a previously viewed target or location, depending on the nature of the experience.

Can this page be better?
Share your feedback via our issue tracker