Model Target Web API

Take advantage of the Vuforia Web API and integrate it into your workflows and automation processes to generate Model Targets and Advanced Model Targets.

The process of generating Model Targets with the Web API is similar to that of the Model Target Generator Desktop Tool, and includes the same options to customize the target for optimal detection and tracking.

Prerequisites

Authentication

The Model Target Web API uses JWT tokens for authentication that can be obtained by providing the Oauth2 Client ID and Client Secret. See Vuforia Web API Authentication on how to create the client credentials.

Model Target Web API Workflow

The API performs all generation and training tasks in the cloud and consists of the following high- level asynchronous flow:

  • Initiate the dataset creation through an HTTP request.
  • Monitor the generation process through the generated UUID.
  • Retrieve the dataset files as a .zip.

The full OpenAPI specification is available here.

Domains

The API is accessible at https://vws.vuforia.com

 

Create a Model Target Dataset

The API allows you to generate Model Targets and Advanced Model Targets with recognition ranges up to 360 degrees. You can additionally train multiple Advanced Model Targets simultaneously.

Model Target Dataset Creation

In this section we walk through a Standard Model Target dataset creation session using a bash shell script and curl to interact with the API.

  1. First, execute the login request to obtain a JWT access token& by supplying the Oauth2 credentials. See Obtain a JWT Token for more information.
  2. Following a successful login, you can create a Standard Model Target by supplying one CAD model and define a Guide View for it. Multiple Guide Views can be added in the views array. For setting the values of the guideViewPosition, please see Guide View Position below.
    body=$(cat <<EOF 
    {
        "name": "dataset-name",
        "targetSdk": "10.5",
        "models": [
            {
                "name": "CAD-model-name",
                "cadDataUrl": "https://YourURL.com/CAD-model.glb",
                "views": [
                    {
                        "name": "viewpoint-name",
                        "layout": "landscape",
                        "guideViewPosition": {
                            "translation": [ 0, 0, 5 ],
                            "rotation": [ 0, 0, 0, 1 ]
                        }
                    }
                ]
            }
        ]
    } EOF )
    curl -XPOST --data "$body" --header 'Content-Type: application/json' --header "Authorization: Bearer $token" " https://vws.vuforia.com/modeltargets/datasets"

Monitor the Progress and Download

The time it takes to create a Model Target Dataset will depend on the configuration and complexity of the uploaded CAD-model.

  1. To monitor the creation progress, you can call the status API route and retrieve information on:
    • The creation status, resulting in either done, processing, or failed.
    • An estimated remaining processing time, when available.
    • Details about failure if the creation process returns as failed.

    Example request to retrieve the creation status:

    curl -XGET --header "Authorization: Bearer $token" "https://vws.vuforia.com/modeltargets/datasets/$uuid/status"

    Example response when status is processing:

    {"status": "processing", "eta": "2020-07-03T13:19:32.693Z"}

    Example response when status is done:

    {"status": "done"}

    Example response when status is failed:

    {"status": "failed", "error": {"code": "ERROR", "message": "..."}}

    When the status is “done”, the zipped dataset can be downloaded:

    curl -XGET --header "Authorization: Bearer $token" --output dataset.zip " https://vws.vuforia.com/modeltargets/datasets/$uuid/dataset" 

Guide View Position

Model Target Datasets can have one or multiple Guide Views which you can let users switch between manually. In the Model Target Web API, Guide Views can be specified via a translation and rotation that represents the position ofthe virtual camera with respect to the object. This virtual camera follows the GLTF Y-up convention with the lens looking towards the negative Z-axis. Depending on your use case, you can change the layout from landscape to portrait. The image below illustrates a  landscape layout.

The transformation matrix 'T * R' represents the "camera matrix" (equal to the "inverse view matrix") in OpenGL terms.

  • The rotation field defines a 3D rotation quaternion [qx, qy, qz, qw].
  • The translation field defines a 3D translational offset in scene units [tx, ty, tz].

NOTE: The camera view must point towards the object to be detected from a perspective that the user of the app shall approach the model from. Consult the Model Target Guide Views documentation for more information on Guide Views.

 

Create an Advanced Model Target Dataset

The creation of Advanced Model Targets follows much of the same process as above, but with a few key distinctions. For an Advanced Model Target, you will need to specify its views with a targetExtent or a targetExtentPreset, and recognitionRanges or a recognitionRangesPreset.

In addition, for Advanced Model Targets, the realisticAppearance field must be set to true or false at the root of theJson request body. Set it to true if the CAD model contains realistic textures.

For more info, refer to the “CAD Model Best Practices” section in the MTG documentation.

Create, get status, and retrieve the Advanced Model Target dataset by using the advancedDatasets routes.

  • POST /modeltargets/advancedDatasets
  • GET /modeltargets/advancedDatasets/:uuid:/status
  • GET /modeltargets/advancedDatasets/:uuid:/dataset

Target Extent and presets

The target extent is used to define a bounding box that identifies which parts of the model will be used in recognizingthe object or discerning between Guide Views.

When the area of interest is the whole object, the FULL_MODEL preset can be used in the view definition:

"targetExtentPreset": "FULL_MODEL"

If only parts of the object are relevant, then the target extent can be specified as an arbitrary transformation as follows:

"targetExtent": {
    "translation": [ 0, 1, 0 ],
    "rotation": [ 0, 0, 0, 1 ],
    "scale": [ 1, 1, 1 ]
}

The translation, rotation, scale transformations are applied to a "unit-cube", that is, a cube centered at the origin with sides of length 2.

The translation, rotation and scale fields follow the GLTF convention.

See also the Target Extent section in the MTG documentation on Guide Views.

Recognition ranges and presets

The recognition ranges are used to define from which angles and orientation a model can be recognized from.

The recognition ranges can be specified as presets DOME or FULL_360:

"recognitionRangesPreset": "DOME | FULL_360"

When using the DOME preset, an upVector must be defined at the model level in the request body. The recognition ranges can also be specified in full as follows:

"recognitionRanges": {
    "rotation": [ 0, 0, 0, 1 ],
    "azimRange": [ -3.14, 3.14 ],
    "elevRange": [ -1, 0 ],
    "rollRange": [ -0.2, 0.2 ]
}

The rotation field can be used to change the frame of reference for the azimuth, elevation, and roll angles. A unit rotation quaternion means that the azimuth, elevation, and roll, identify rotations around the Y, X and Z axisrespectively.

See also the Viewing Angles and Orientation section in Configuring Advanced Guide Views.

Example request body with presets

Define a view with a dome detection range and a full model bounding box:

{
    "name": "advanced-dataset-with-presets",
    "targetSdk": "10.5",
    "models": [
        {
            "name": "cad-model-name",
            "cadDataUrl":
            https: //file.glb,
            "upVector": [ 0, 1, 0 ],
            "views": [
                {
                    "name": "viewpoint-name",
                    "targetExtentPreset": "FULL_MODEL",
                    "recognitionRangesPreset": "DOME"
                }
            ]
        }
    ],
    "realisticAppearance": false
}

Example request body without presets

Define a view with a full 360-degree detection range and a target volume centered at the origin:

{
    "name": "advanced-dataset-without-presets",
    "targetSdk": "10.5",
    "models": [
        {
            "name": "cad-model-name",
            "cadDataUrl": "https://file.glb",
            "views": [
                {
                    "name": "viewpoint-name",
                    "targetExtent": {
                        "translation": [ 0, 0, 0 ],
                        "rotation": [ 0, 0, 0, 1 ],
                        "scale": [ 1, 1, 1 ]
                    },
                    "recognitionRanges": {
                        "rotation": [ 0, 0, 0, 1 ],
                        "azimRange": [ -3.14, 3.14 ],
                        "elevRange": [ -1.57, 1.57 ],
                        "rollRange": [ -0.2, 0.2 ]
                    }
                }
            ]
        }
    ],
    "realisticAppearance": false
}

Advanced Model Target with Multiple Views

There may be use cases in which multiple detection ranges or multiple CAD models are desired. To accomplish this, add additional views and/or models to your dataset creation request body. Just remember that the recognition rangesshould not overlap on the same CAD model.

Example request body for an Advanced Model Target with multiple CAD models:

{
    "name": "advanced-dataset-multi-model-name",
    "targetSdk": "10.5",
    "models": [
        {
            "name": "cad-model-1-name",
            "upVector": [ 0, 1, 0 ],
            "cadDataUrl": "https://file-1.glb",
            "views": [
                {
                    "name": "viewpoint-name",
                    "targetExtentPreset": "FULL_MODEL",
                    "recognitionRangesPreset": "DOME"
                }
            ]
        },
        {
            "name": "cad-model-2-name",
            "upVector": [ 0, 1, 0 ],
            "cadDataUrl": "https://file-2.glb",
            "views": [
                {
                    "name": "viewpoint-name",
                    "targetExtentPreset": "FULL_MODEL",
                    "recognitionRangesPreset": "DOME"
                }
            ]
        }
    ],
    "realisticAppearance": false
}

Uploading CAD Models

CAD models can be provided in 2 different ways: Either by embedding the model data as Base64 encoded string in the cadDataBlob field or by specifying a URL in the cadDataUrl field.

The cadDataBlob field is appropriate for small CAD models (less than 20MB) while the URL can be used for larger models.

The URL must be accessible from the Model Target Web API, so it is recommended to use a signed URL with an expiration date. Signed URLs can be created easily for AWS S3 objects or Azure Storage objects:

Supported CAD Model Formats

The API preferentially supports reading glTF 2.0, provided as a single .glb file or as zipped glTF. In addition, thefollowing file formats are also supported if the cadDataFormat is specified:

File Format

cadDataFormat

glTF 2.0 as .glb file or as zipped glTF

No specification needed.

Creo View (.pvz), Collada (.dae), FBX (.fbx), IGES (.igs, .iges), Wavefront (.obj), STL (.stl,.sla), VRML (.wrl, .vrml).

The cadDataFormat field must be specified using one of the following extensions: DAE, FBX, IGES, OBJ, PVS, STL, VRML.

Additional Options

The targetSdk field must be used to specify the expected (minimum) SDK version used to load the generated dataset.

Based on the target SDK, the API will allow specific features and optimizations such as Draco compression, runtime Guide View rendering, and etc.

Tracking Mode

The trackingMode field can be used to improve tracking on specific object types. Set it to car, scan or leave it as default. For information on the different modes’ benefits, please refer to Optimizing Model Target Tracking.

TIP: The tracking mode can be specified independently for each model.

Motion Hint

The motionHint field (see the Model Target Motion Hint” section in Optimizing Model Target Tracking) can be used to improve tracking for static vs. moving (adaptive) objects.

The motionHint field can be set for each model in the dataset creation request to either static (default) or adaptive.

Up Vector

If the model has a natural up vector, meaning that the object is always expected to be upright (a factory machine, a table-top object, a car, etc.) it is strongly recommended to specify an upVector field; it will greatly improve its tracking performance.

The upVector can be set independently for each model in the request body. The upVector is mandatory when usingthe DOME recognition range preset.

Automatic Coloring

3D Models without photorealistic textures can have improved detection results if the automatic coloring is enabled.

The automaticColoring field can be set for each model in the request and can have values never, always or auto. By default, automatic coloring is disabled (never).

See also Automatic Coloring.

NOTE: The realisticAppearance flag cannot be enabled together with automatic coloring.

Realistic Appearance

3D Models with photorealistic textures can have improved detection results if realisticAppearance is set to true.

Realistic appearance cannot be set per model and must be defined at the root of the Json request object. It is mandatory for advanced datasets.

NOTE: The realisticAppearance field cannot be enabled together with automatic coloring.

Model Scale

The Vuforia Engine SDK requires models in real life scale and the input model units is assumed to be meters.

If necessary to adjust the 3D model to its real-life scale, each model object in the request body can specify a uniformScale parameter that is a scale factor used to preprocess the 3D model.

Simplify

The API allows simplification of CAD models and as part of the dataset creation.  Simplification will produce optimized models for high performance and robust AR object tracking experiences.

The simplify field can be set for each model in the request and can have values never, always or auto. By default, simplification is set to auto.

At the moment, only simplification of a single model per request is supported.

Data Retention

The API servers and the processing servers are hosted in the AWS Cloud eu-west-1 region.

The uploaded CAD models and data are transferred to the cloud using TLS 1.2 encryption and then stored using AES-256 encryption at rest.

All data associated to a Model Target dataset can be removed by issuing a DELETE request for a specific dataset UUID.

For example, a standard Model Target can be deleted as follows using curl:

curl -XDELETE --header "Authorization: Bearer $token" "https://vws.vuforia.com/modeltargets/datasets/$uuid"

Troubleshooting

For all 4xx and 5xx HTTP status responses, the API returns additional information in the response body that could help with troubleshooting the issue.

For example, if the request body of the dataset creation request does not specify a CAD Model, the response will be:

{
    "error": {
        "code": "BAD_REQUEST",
        "message": "model 'cad-model-name' is invalid. One of `cadDataBlob`, `cadDataUrl` need to be provided"
    }
}

Unknown errors caused by issues on the MTG API, will return a generic error response:

{
    "error": {
        "code": "ERROR",
        "message": "Internal Server Error"
    }
}

If such an error persists contact us via the Support Center.

Additionally, when retrieving the status of a dataset, the response can contain detailed errors or warnings that could help with identifying problematic CAD models (such as symmetric ones), objects that cannot be well discriminated, etc.