Model Targets API Overview

This article describes the native API for configuring Model Targets, Advanced Model Targets, their Guide Views, and the available modes that can help improve Model Target tracking experiences.

For a general introduction to the key concepts of the Vuforia Engine API, please see Vuforia Engine 10 API. For an introduction to Model Targets and Advanced Model Targets, please see the Model Targets main page. We also encourage you to try the samples that you are free to use as a foundation for your own AR application. See Native Samples for more information.

A Model Target is tracked by creating an Observer from a Model Target Database. The setup of Model Targets is similar to other Observers but with some distinct differences:

  • Model Targets are set up with Guide View images that helps the user approach the object from a point where the object can be detected from, and tracking is then started.
  • Advanced Model Targets are set up with Advanced Views and trained so that the object is automatically detected and tracked

General Setup

Please see the Engine Lifecycle for a general introduction to the Vuforia Engine lifecycle. The Engine is required for creating and using Observers.

Loading a Model Target Database

Prepare and create Model Targets from 3D models with the Model Target Generator (MTG) and generate Model Target databases as pairs of .xml and .dat dataset files.

Model Target Database

When loading Model Target databases, consider the following:

  • Multiple Model Target Observers can be created at once, subject to memory usage limits.
  • Only one Model Target Observer may be active at any one time. (This restriction does not apply to tracking alongside other kinds of observer types. For example, you may have multiple active Image Target Observers at the same time as you have an active Model Target Observer).
  • For Advanced Model Target databases with multiple Model Targets, you may configure Model Target Observers for each available target in the Advanced Model Target database. Vuforia Engine will automatically detect the currently visible object and set it as the active Model Target. See the section below Controlling Recognition Behavior for Advanced Model Targets for more information.

Target Size and Origin

The size of a Model Target is associated with the size of the Bounding Box of the (CAD) Model generated from the MTG Tool. The size of the box depends on the CAD Model used as input in the MTG. 

A Model Target's coordinate system has its origin at the origin of the 3D model used to define the target. This is in contrast to other types of Observers available with Vuforia Engine, such as Image Targets (origin is the center of the image). 

Therefore, you need to know and be aware of the origin of the 3D model in order to do any AR augmentation in relation to the physical object, because you cannot change the origin from the API.

Model Target size and origin is defined implicitly from the 3D model used when generating the Model Target.

Target and Target Pose Unit

The units used for Model Targets (and for their pose) are in meters.

Create a Model Target Observer

Detect and track objects by creating a Model Target Observer from a Model Target database. A Model Target Observer is configured with a valid databasePath, targetName, and activeGuideViewName. See the section Model Target Guide View below. By default, the Model Target Observer is activated when created.

// Create an Observer config
VuModelTargetConfig modelTargetConfig = vuModelTargetConfigDefault();
modelTargetConfig.databasePath = "ModelTargetDatabase.xml";
modelTargetConfig.targetName = "ModelTargetName";
// Select a Guide View to be active 
modelTargetConfig.activeGuideViewName = "GuideView_Front";

// Create a Model Target Observer
VuObserver* modelTargetObserver = { NULL };
VuModelTargetCreationError creationError;
vuEngineCreateModelTargetObserver(engine, &modelTargetObserver, &modelTargetConfig, &creationError);

Configuring and creating an Observer with non-default additional optional arguments should be done while other Observers from the same database are deactivated to avoid performance constraints.

Observations

Observations are reported in the state and deliver information on the pose status and status info of a Model Target Observer. Target information for a Model Target Observer is available in the VuModelTargetObservationTargetInfo struct. See the Observer and Observations article for more information.

Creation of Observation list

// Create observation list
VuObservationList* obsList = { NULL };
vuObservationListCreate(&obsList);

// Get and parse model target observations list
vuStateGetModelTargetObservations(state,obsList);

int32_t listSize = 0;
vuObservationListGetSize(obsList, &listSize);

Then, parse the observation list to get info on the target(s).

// Parse the model target list
for (int i = 0; i < listSize; i++)
{
    VuObservation* obs = { NULL };
    vuObservationListGetElement(obsList, i, &obs);

    VuPoseInfo poseInfo;
    vuObservationGetPoseInfo(obs, &poseInfo);

    VuModelTargetObservationTargetInfo targetInfo;
    vuModelTargetObservationGetTargetInfo(obs, &targetInfo);

    if (poseInfo.poseStatus != VU_OBSERVATION_POSE_STATUS_NO_POSE)
    {
        // Do something with poseInfo and targetInfo
    }
}

Destroy Observer and Observation

Destroy objects and processes after usage to free up memory.

Call the following to destroy the Model Target ObservationList and the Model Target Observer:

Destroy ObservationList

// Destroy observation list
vuObservationListDestroy(obsList);

Destroy the Observer

// Destroy the observer
vuObserverDestroy(modelTargetObserver);

Guide Views

Non-Advanced Model Targets require a Guide View to be displayed on the screen to assist users in finding the right position from where the object can be successfully detected. If you are not already familiar with Guide Views, please refer to the Model Target Guide View page for a more detailed introduction to the Guide View concept.

NOTE: Advanced Model Targets do not require Guide Views to be rendered on the screen. In that case, the activeGuideviewName can be set to null.

Accessing Guide Views

Querying the list of available guide views:

vuGuideViewListCreate(VuGuideViewList** list);
vuModelTargetObserverGetGuideViews(const VuObserver* observer, VuGuideViewList* list);
vuGuideViewListGetSize(const VuGuideViewList* list, int32_t* listSize);

Accessing a specific guide view from the list

vuGuideViewListGetElement(const VuGuideViewList* list, int32_t element, VuGuideView** guideView);

Destroy GuideViewList

vuGuideViewListDestroy(VuGuideViewList* list);

Switching between Guide Views

For Model Targets with multiple Guide Views; these can be individually selected by calling get/set to the Model Target Observer.

vuModelTargetObserverSetActiveGuideViewName(VuObserver* observer, const char* name);

Advanced Model Targets will automatically recognize objects without requiring a Guide View to be set and rendered. Setting an active Guide View to an Advanced Model Target will return VU_FAILED.

Custom Guide View Rendering and Positioning

In the API, the VuGuideView exposes the following properties of a Guide View:

NOTE: Guide Views use a Y-up convention. See Frame of Reference for more details.

  • Guide View Intrinsic Parameters: Intrinsics used to render the Guide View. It is represented as a camera calibration data structure and matches the intrinsics of the camera of the current device.
vuGuideViewGetIntrinsics(const VuGuideView* guideView, VuCameraIntrinsics* cameraIntrinsics);
  • Guide View Extrinsic Parameters: Extrinsics representing model view matrix to render the Guide View. It is represented as a Matrix4x4 data structure.
  • Overlay image: a rendering of the object in an abstracted style (edge rendering), as it would appear from the Guide View position in the MTG (i.e. using the initial View Intrinsic Parameters and View Extrinsic Parameters).

If you change the Guide View pose, you can use the MTG to modify the Detection Position and re-export the Model Target database. See How to Create Model Targets for details.

If you want to change the Guide View pose at runtime, you can use the get/set Guide View pose to modify the pose and consequently, the position from which tracking will begin.

vuGuideViewGetPose(const VuGuideView* guideView, VuMatrix44F* pose);
vuGuideViewSetPose(VuGuideView* guideView, const VuMatrix44F* pose);

For Advanced Model Target databases with multiple Model Targets and/or multiple Advanced Views, a recognized object is activated automatically, and tracking starts as soon as the user has aligned the object with the Guide View.

Mesh Observer for Model Targets

The Mesh Observer API provides access to the geometry of the Model Target Observer at runtime. You can make use of the mesh for rendering special effects, such as occlusion, custom 3D guide views or detection animations. See Create Mesh Observer from Model Target for creating Mesh Observers.

Tracking Optimization

Vuforia Engine provides multiple modes that optimize tracking for different scenarios and cases. Set the mode in the Optimize Tracking tab in the MTG or set it at runtime through the VuTrackingOptimization API to one of the following values

  • VU_TRACKING_OPTIMIZATION_DEFAULT
  • VU_TRACKING_OPTIMIZATION_LOW_FEATURE_OBJECTS
  • VU_TRACKING_OPTIMIZATION_AR_CONTROLLER

Use the get/set methods to set the tracking optimization of a Model Target before starting Vuforia Engine.

// Get a Model Target Observer’s tracking optimization mode
vuModelTargetObserverGetTrackingOptimization(VuObserver* observer, VuTrackingOptimization optimization);
// Set a Model Target Observer’s tracking optimization mode
vuModelTargetObserverSetTrackingOptimization(VuObserver* observer, VuTrackingOptimization optimization);

Setting the tracking optimization mode on a tracked Model Target Observer will reset the tracking of the target.

See more information on tracking optimization at Optimizing Model Target Tracking.

Model Target Pose Status and Status Info

A Model Target’s pose status and status info indicates how well an object is tracked. Possible pose status values are _NO_POSE, _TRACKED, _EXTENDED_TRACKED or _LIMITED. See also Pose Status and Status Info for a table with pose status and status info.

We recommend enabling Extended Tracking when using any type of Model Targets to allow for a robust experience, especially with large-scale objects. Extended Tracking is supported through Device Tracking.

The tracking status reported will be _TRACKED when the Model Target tracking is active and robust, and _EXTENDED_TRACKED when the Device Pose Observation is returning the last observed position while the Model Target is out of camera view or occluded. In situations where the Device Pose Observation provides insufficient information to generate an accurate _EXTENDED_TRACKED pose of the Model Target, the tracking status will be set to _LIMITED.

When _LIMITED tracking status is reported, we recommend hiding augmentations of the Model Target until more robust tracking is reported again.

Status Info – Wrong Scale

Use VuModelTargetObservationGetStatusInfo to get additional information on the tracking status info. A _WRONG_SCALE will be reported if the Model Target is determined to be scaled incorrectly and does not match the size of the physical object. A _WRONG_SCALE is reported in three situations.

  • The Model Target is reported as _TRACKED but its status info returns _WRONG_SCALE. The target is tracked reliably, but poses will be reported in a non-metrical coordinate system that matches the configured size of the Model Target.
  • The Model Target is reported as _EXTENDED_TRACKED but its status info returns _WRONG_SCALE. The target is indirectly tracked but poses will be reported in a non-metrical coordinate system that matches the configured size of the Model Target.
  • The Model Target is reported as _LIMITED and status info is _WRONG_SCALE. Automatic correction of the scale is not possible, tracking is limited and with low accuracy.

In both situations, Vuforia Engine has detected scale issues to a degree where you should consider examining the parity of the physical object and the digital model.

If the Model Target scale differs significantly from the physical object, tracking and augmentations can be seen to drift. Resolve scaling issues by regenerating your Model Target with a correct scale. See Best Practices for Scaling Model Targets for additional options.

Standard Model Target Behavior

For standard Model Targets, a Model Target Observation has the following status info collected in the state after activation with

VuModelTargetObservationTargetInfo == VU_OBSERVATION_POSE_STATUS_NO_POSE;

and

VuModelTargetObservationStatusInfo == VU_MODEL_TARGET_OBSERVATION_STATUS_INFO_RECOMMENDING_GUIDANCE;

Once the user has aligned the object with the Guide View, tracking starts, and the target’s status info switches to_TRACKED.

Advanced Model Target Behavior

For Advanced Model Targets with a detection range up to 360°, it is generally not necessary to display a Guide View at runtime. When an Advanced Model Target database is activated, all objects are initialized in the state with the status info

VuModelTargetObservationTargetInfo == VU_OBSERVATION_POSE_STATUS_NO_POSE

and

VuDevicePoseObservationStatusInfo == VU_DEVICE_POSE_OBSERVATION_STATUS_INFO_INITIALIZING;

When one of the Model Targets is detected, tracking starts automatically and its state switches to _EXTENDED_TRACKED.

Controlling Recognition Behavior for Advanced Model Targets with Multiple Models

As long as an Advanced Model Target is _TRACKED, detecting other Model Targets is automatically disabled to save processing power and prevent unwanted activation of other targets. When tracking of a Model Target is changed from _TRACKED to _EXTENDED_TRACKED or _LIMITED, recognition is automatically enabled again to allow detection of other activated Model Targets (or to relocate the current Model Target). Since only a single Model Target can be actively tracked at a time, recognition and activation of another Model Target can lead to complete tracking loss of the previously tracked Model Target.

NOTE: _LIMITED pose status with _WRONG_SCALE status info does not re-enable recognition of other Model Targets.

In scenarios where this behavior is not desired (e.g., to save power or when only a single Model Target is used in a session), this behavior can be disabled by calling

VuBool enable = VU_FALSE;
vuEngineSetModelTargetRecoWhileExtendedTracked(VuEngine* engine, VuBool enable);

Its default value is set to VU_TRUE and it can only be set when the Vuforia Engine is not running. When set to VU_FALSE, the first successfully tracked Model Target will stay active until its associated Observer is deactivated.

Configuring Model Targets

In this section, we present additional common configuration options for Model Targets.

Size

Get the sizes, height, length, and width of a Model Target in meters.

vuModelTargetObserverGetTargetSize(const VuObserver* observer, VuVector3F* size);

Scale

Scale the Model Target with a scale factor to one of the retrieved values from the GetTargetSize().

vuModelTargetObserverSetTargetScale(VuObserver* observer, float scale);

Bounding Box

Use this function to get an axis-aligned bounding box of an image from its respective Observer, and relative to the target’s frame of reference.

vuModelTargetObserverGetAABB(const VuObserver* observer, VuAABB* bbox);

 

Can this page be better?
Share your feedback via our issue tracker