Model Targets API Overview

This article describes the native API for configuring Model Targets, Advanced Model Targets, their Guide Views, and the available hints that can help improve Model Target tracking experiences. 

For a general introduction to the key concepts of the Vuforia Engine API, please see Vuforia Engine 10 API. For an introduction to Model Targets and Advanced Model Targets, please see the Model Targets main page. We also encourage you to try the samples that you are free to use as a foundation for your own AR application. See Native Samples for more information.

A Model Target is tracked by creating an Observer from a Model Target Database. The setup of Model Targets is similar to other Observers but with some distinct differences:

  • Non-Advanced Model Targets are set up with Guide View images that helps the user approach the object from a point where the object can be detected from, and tracking is then started.
  • A Model Target can be configured with a motion hint that optimizes tracking for certain use cases.
  • A Model Target can also be configured with a tracking mode that specifies the type of object a Model Target represents.

General Setup

Please see the Engine Lifecycle for a general introduction to the Vuforia Engine lifecycle. The Engine is required for creating and using Observers.

Loading a Model Target Database

Prepare and create Model Targets from 3D models with the Model Target Generator (MTG) and generate Model Target databases as pairs of .xml and .dat files.

Model Target Database

When loading Model Target databases, consider the following:

  • Multiple Model Target Observers can be created at once, subject to memory usage limits.
  • Only one Model Target Observer may be active at any one time. (This restriction does not apply to tracking alongside other kinds of observer types. For example, you may have multiple active Image Target Observers at the same time as you have an active Model Target Observer).
  • For Advanced Model Target databases with multiple Model Targets, you may configure Model Target Observers for each available target in the Advanced Model Target database. Vuforia Engine will automatically detect the currently visible object and set it as the active Model Target. See the section below Controlling Recognition Behavior for Advanced Model Targets for more information.

Target Size and Origin

The size of a Model Target is associated with the size of the Bounding Box of the (CAD) Model generated from the MTG Tool. This will be dependent on the CAD Model used as input in the MTG. 

A Model Target's coordinate system has its origin at the origin of the 3D model used to define the target. This is in contrast to other types of Observers available with Vuforia Engine, such as Image Targets (origin is the center of the image), or Object Targets (origin is the center of the object). 

Therefore, you need to know and be aware of the origin of the 3D model in order to do any AR augmentation in relation to the physical object, because you cannot change the origin from the API.

Model Target size and origin is defined implicitly from the 3D model used when generating the Model Target.

Target and Target Pose Unit

The units used for Model Targets (and for its pose) is in meters.

Create a Model Target Observer

Detect and track objects by creating a Model Target Observer from a Model Target database. A Model Target Observer is configured with a valid databasePath, targetName, and activeGuideViewName. See the section Model Target Guide View below. By default, the Model Target Observer is activated when created.

// Create an Observer config
VuModelTargetConfig modelTargetConfig = vuModelTargetConfigDefault();
modelTargetConfig.databasePath = "ModelTargetDatabase.xml";
modelTargetConfig.targetName = "ModelTargetName";
// Select a Guide View to be active 
modelTargetConfig.activeGuideViewName = "GuideView_Front";

// Create a Model Target Observer
VuObserver* modelTargetObserver = { NULL };
vuEngineCreateModelTargetObserver(engine, &modelTargetObserver, modelTargetConfig, NULL);

Configuring and creating an Observer with non-default additional optional arguments should be done while other Observers from the same database is deactivated to avoid performance constraints.

Observations

The Model Target Observer produces Observations that are collected in the State. Get the state from the Engine and parse them to the ObservationTargetInfo. See the Observer and Observations article for more information.

Creation of Observation list

// Create observation list
VuObservationList* obsList = { NULL };
vuObservationListCreate(&obsList);

// Get and parse model target observations list
vuStateGetModelTargetObservations(state,obsList);

int32_t listSize = 0;
vuObservationListGetSize(obsList, &listSize);

Then, parse the observation list to get info on the target(s).

// Parse the model target list
for (int i = 0; i < listSize; i++)
{
    VuObservation* obs = { NULL };
    vuObservationListGetElement(obsList, i, &obs);

    VuPoseInfo poseInfo;
    vuObservationGetPoseInfo(obs, &poseInfo);

    VuModelTargetObservationTargetInfo targetInfo;
    vuModelTargetObservationGetTargetInfo(obs, &targetInfo);

    if (poseInfo.poseStatus != OBSERVATION_POSE_STATUS_NO_POSE)
    {
        // Do something with poseInfo and targetInfo
    }
}

Destroy Observer and Observation

Destroy objects and processes after usage to free up memory.

Call the following to destroy the Model Target ObservationList and the Model Target Observer:

Destroy ObservationList

// Destroy observation list
vuObservationListDestroy(obsList);

Destroy the Observer

// Destroy the observer
vuObserverDestroy(modelTargetObserver);

Guide Views

Non-Advanced Model Targets require a Guide View to be displayed on the screen to assist users in finding the right position from where the object can be successfully detected. If you are not already familiar with Guide Views, please refer to the Model Target Guide View page for a more detailed introduction to the Guide View concept.

NOTE: Advanced Model Targets do not require Guide Views to be rendered on the screen. In that case, the activeGuideviewName can be set to null.

Accessing Guide Views

Querying the list of available guide views:

vuGuideViewListCreate(VuGuideViewList** list);
vuModelTargetObserverGetGuideViews(const VuObserver* observer, VuGuideViewList* list);
vuGuideViewListGetSize(const VuGuideViewList* list, int32_t* listSize);

Accessing a specific guide view from the list

vuGuideViewListGetElement(const VuGuideViewList* list, int32_t element, VuGuideView** guideView);

Destroy GuideViewList

vuGuideViewListDestroy(VuGuideViewList* list);

Switching between Guide Views

For Model Targets with multiple Guide Views; these can be individually selected by calling get/set to the Model Target Observer.

vuModelTargetObserverSetActiveGuideViewName(VuObserver* observer, const char* name);

Advanced Model Targets will automatically recognize objects without requiring a Guide View to be set and rendered. Setting an active Guide View to an Advanced Model Target will return VU_FAILED.

Custom Guide View Rendering and Positioning

In the API, the VuGuideView exposes the following properties of a Guide View:

NOTE: Guide Views use a Y-up convention. See Frame of Reference for more details.

  • Guide View Intrinsic Parameters: Intrinsics used to render the guide view. It is represented as a CameraCalibration data structure and matches the intrinsics of the camera of the current device.
vuGuideViewGetIntrinsics(const VuGuideView* guideView, VuCameraIntrinsics* cameraIntrinsics);
  • Guide View Extrinsic Parameters: Extrinsics representing model view matrix to render the guide view. It is represented as a Matrix4x4 data structure.
  • Overlay image: a rendering of the object in an abstracted style (edge rendering), as it would appear from the Guide View position in the MTG (i.e. using the initial View Intrinsic Parameters and View Extrinsic Parameters).

If you change the Guide View pose, you can use the MTG to modify the Detection Position and re-export the Model Target database. See How to Create Model Targets for details.

If you want to change the Guide View pose at runtime, you can use the get/set Guide View pose to modify the pose and consequently, the position from which tracking will begin.

vuGuideViewGetPose(const VuGuideView* guideView, VuMatrix44F* pose);
vuGuideViewSetPose(VuGuideView* guideView, const VuMatrix44F* pose);

For Advanced Model Target databases with multiple Model Targets and/or multiple Advanced Views, a recognized object is activated automatically, and tracking starts as soon as the user has aligned the object with the Guide View.

Motion Hint and Tracking Mode

Vuforia Engine provides a motion hint and an tracking mode to improve tracking of objects in certain scenarios and use cases. Both can be set in the according tab in the MTG or can be set at runtime through the API.

Before rendering the model and the Guide View, set the motion hint to either

  • VU_MOTION_HINT_STATIC
  • VU_MOTION_HINT_ADAPTIVE
  • VU_MOTION_HINT_DYNAMIC
vuModelTargetObserverSetMotionHint(VuObserver* observer, VuMotionHint motionHint);

Similiarly, a Tracking Mode is best assigned to your Model Target in the MTG. A Tracking Mode can be set to

  • VU_MODEL_TARGET_TRACKING_MODE_DEFAULT
  • VU_MODEL_TARGET_TRACKING_MODE_CARS
  • VU_MODEL_TARGET_TRACKING_MODE_SCAN

Only the _DEFAULT and the _CAR mode can be set at runtime.

vuModelTargetObserverSetTrackingMode(VuObserver* observer, VuModelTargetTrackingMode trackingMode);

See more information on motion hint and Tracking Mode for Model Targets in Optimizing Model Target Tracking.

Model Target Status and Status Info

A Model Target’s status and status info are indications of how well an object is tracked. Possible status values are _NO_POSE, _TRACKED, _EXTENDED_TRACKED or _LIMITED.

We recommend enabling Extended Tracking when using any type of Model Targets to allow for a robust experience, especially with large-scale objects. Extended Tracking is supported through Device Tracking.

The tracking status reported will be _TRACKED when the Model Target tracking is active and robust, and _EXTENDED_TRACKED when the Device Pose Observation is returning the last observed position while the Model Target is out of camera view or occluded. In addition, in situations where the Device Pose Observation provides insufficient information to generate an accurate _EXTENDED_TRACKED pose of the Model Target, the tracking status will be set to _LIMITED.

When _LIMITED tracking status is reported, we recommend hiding augmentations of the Model Target until more robust tracking is reported again.

Status Info

Use VuModelTargetObservationStatusInfo to get additional information on the tracking status. In particular, a _WRONG_SCALE will be reported if the Model Target is determined to be scaled incorrectly and does not match the size of the physical object. A _WRONG_SCALE is reported in two situations.

  • The Model Target is reported as _TRACKED but its status info returns _WRONG_SCALE. The target is tracked reliably, but poses will be reported in a non-metrical coordinate system that matches the configured size of the Model Target.
  • The Model Target is reported as _LIMITED and status info is _WRONG_SCALE. Automatic correction of the scale is not possible, tracking is limited and with low accuracy.

In both situations, Vuforia Engine has detected scale issues to a degree where you should consider examining the parity of the physical object and the digital model.

If the Model Target scale differs significantly from the physical object, tracking and augmentations can be seen to drift. Resolve scaling issues by regenerating your Model Target with a correct scale. See Best Practices for Scaling Model Targets for additional options.

Standard Model Target Behavior

For standard Model Targets, a Model Target Observation has the following status info collected in the state after activation with

VuModelTargetObservationTargetInfo == VU_OBSERVATION_POSE_STATUS_NO_POSE;

and

VuModelTargetObservationStatusInfo == VU_MODEL_TARGET_OBSERVATION_STATUS_INFO_RECOMMENDING_GUIDANCE;

Once the user has aligned the object with the Guide View, tracking starts, and the target’s status info switches to_TRACKED.

Advanced Model Target Behavior

For Advanced Model Targets with a detection range up to 360°, it is generally not necessary to display a Guide View at runtime. When an Advanced Model Target database is activated, all objects are initialized in the state with the status info

VuModelTargetObservationTargetInfo == VU_OBSERVATION_POSE_STATUS_NO_POSE

and

VuDevicePoseObservationStatusInfo == VU_DEVICE_POSE_OBSERVATION_STATUS_INFO_INITIALIZING;

When one of the Model Targets is detected, tracking starts automatically and its state switches to _EXTENDED_TRACKED.

Controlling Recognition Behavior for Advanced Model Targets

As long as an Advanced Model Target is _TRACKED, recognition is disabled automatically to save processing power and prevent unwanted activation of other targets. When tracking of a Model Target is lost, i.e. its pose status changes to _EXTENDED_TRACKED or _LIMITED, recognition is automatically enabled again to allow detection of other activated Model Targets (or to relocate the current model target). Since only a single Model Target can be actively tracked at a time, a recognition and activation of another Model Target can lead to complete tracking loss of the previously tracked Model Target.

In scenarios where this behavior is not desired (e.g. to save power or when only a single Model Target is used in a session), this behavior can be disabled by calling

VuBool enable = VU_FALSE;
vuEngineSetModelTargetRecoWhileExtendedTracked(VuEngine* engine, VuBool enable);

Its default value is set to VU_TRUE and it can only be set when the Vuforia Engine is not running. When set to VU_FALSE, the first successfully tracked Model Target will stay active until its associated Observer is deactivated.

Configuring Model Targets

In this section, we present additional common configuration options for Model Targets.

Size

Get the sizes, height, length, and width of a Model Target in meters.

vuModelTargetObserverGetTargetSize(const VuObserver* observer, VuVector3F* size);

Scale

Scale the Model Target with a scale factor to one of the retrieved values rom the GetTargetSize().

vuModelTargetObserverSetTargetScale(VuObserver* observer, float scale);

Bounding Box

Use this function to get an axis-aligned bounding box of an image from its respective Observer, and relative to the target’s frame of reference.

vuModelTargetObserverGetAABB(const VuObserver* observer, VuAABB* bbox);