Take advantage of the Vuforia Web API and integrate it into your workflows and automation processes to generate Model Targets and Advanced Model Targets.
The process of generating Model Targets with the Web API is similar to that of the Model Target Generator Desktop Tool, and includes the same options to customize the target for optimal detection and tracking.
Prerequisites
Authentication
The Model Target Web API uses JWT tokens for authentication that can be obtained by providing the Oauth2 Client ID and Client Secret. See Vuforia Web API Authentication on how to create the client credentials.
Model Target Web API Workflow
The API performs all generation and training tasks in the cloud and consists of the following high- level asynchronous flow:
- Initiate the dataset creation through an HTTP request.
- Monitor the generation process through the generated UUID.
- Retrieve the dataset files as a .zip.
The full OpenAPI specification is available here.
Domains
The API is accessible at https://vws.vuforia.com
Create a Model Target Dataset
The API allows you to generate Model Targets and Advanced Model Targets with recognition ranges up to 360 degrees. You can additionally train multiple Advanced Model Targets simultaneously.
Model Target Dataset Creation
In this section we walk through a Standard Model Target dataset creation session using a bash shell script and curl to interact with the API.
- First, execute the login request to obtain a JWT access token& by supplying the Oauth2 credentials. See Obtain a JWT Token for more information.
- Following a successful login, you can create a Standard Model Target by supplying one CAD model and define a Guide View for it. Multiple Guide Views can be added in the views array. For setting the values of the
guideViewPosition, please see Guide View Position below.body=$(cat <<EOF { "name": "dataset-name", "targetSdk": "10.5", "models": [ { "name": "CAD-model-name", "cadDataUrl": "https://YourURL.com/CAD-model.glb", "views": [ { "name": "viewpoint-name", "layout": "landscape", "guideViewPosition": { "translation": [ 0, 0, 5 ], "rotation": [ 0, 0, 0, 1 ] } } ] } ] } EOF ) curl -XPOST --data "$body" --header 'Content-Type: application/json' --header "Authorization: Bearer $token" " https://vws.vuforia.com/modeltargets/datasets"
Monitor the Progress and Download
The time it takes to create a Model Target Dataset will depend on the configuration and complexity of the uploaded CAD-model.
- To monitor the creation progress, you can call the status API route and retrieve information on:
- The creation status, resulting in either
done,processing, orfailed. - An estimated remaining processing time, when available.
- Details about failure if the creation process returns as
failed.
Example request to retrieve the creation status:
curl -XGET --header "Authorization: Bearer $token" "https://vws.vuforia.com/modeltargets/datasets/$uuid/status"Example response when status is processing:
{"status": "processing", "eta": "2020-07-03T13:19:32.693Z"}Example response when status is done:
{"status": "done"}Example response when status is failed:
{"status": "failed", "error": {"code": "ERROR", "message": "..."}}When the status is “done”, the zipped dataset can be downloaded:
curl -XGET --header "Authorization: Bearer $token" --output dataset.zip " https://vws.vuforia.com/modeltargets/datasets/$uuid/dataset" - The creation status, resulting in either
Guide View Position
Model Target Datasets can have one or multiple Guide Views which you can let users switch between manually. In the Model Target Web API, Guide Views can be specified via a translation and rotation that represents the position ofthe virtual camera with respect to the object. This virtual camera follows the GLTF Y-up convention with the lens looking towards the negative Z-axis. Depending on your use case, you can change the layout from landscape to portrait. The image below illustrates a landscape layout.
The transformation matrix 'T * R' represents the "camera matrix" (equal to the "inverse view matrix") in OpenGL terms.
- The
rotationfield defines a 3D rotation quaternion[qx, qy, qz, qw]. - The
translationfield defines a 3D translational offset in scene units[tx, ty, tz].
NOTE: The camera view must point towards the object to be detected from a perspective that the user of the app shall approach the model from. Consult the Model Target Guide Views documentation for more information on Guide Views.
Create an Advanced Model Target Dataset
The creation of Advanced Model Targets follows much of the same process as above, but with a few key distinctions. For an Advanced Model Target, you will need to specify its views with a targetExtent or a targetExtentPreset, and recognitionRanges or a recognitionRangesPreset.
In addition, for Advanced Model Targets, the realisticAppearance field must be set to true or false at the root of theJson request body. Set it to true if the CAD model contains realistic textures.
For more info, refer to the “CAD Model Best Practices” section in the MTG documentation.
Create, get status, and retrieve the Advanced Model Target dataset by using the advancedDatasets routes.
- POST /modeltargets/advancedDatasets
- GET /modeltargets/advancedDatasets/:uuid:/status
- GET /modeltargets/advancedDatasets/:uuid:/dataset
Target Extent and presets
The target extent is used to define a bounding box that identifies which parts of the model will be used in recognizingthe object or discerning between Guide Views.
When the area of interest is the whole object, the FULL_MODEL preset can be used in the view definition:
"targetExtentPreset": "FULL_MODEL"
If only parts of the object are relevant, then the target extent can be specified as an arbitrary transformation as follows:
"targetExtent": {
"translation": [ 0, 1, 0 ],
"rotation": [ 0, 0, 0, 1 ],
"scale": [ 1, 1, 1 ]
}
The translation, rotation, scale transformations are applied to a "unit-cube", that is, a cube centered at the origin with sides of length 2.
The translation, rotation and scale fields follow the GLTF convention.
See also the Target Extent section in the MTG documentation on Guide Views.
Recognition ranges and presets
The recognition ranges are used to define from which angles and orientation a model can be recognized from.
The recognition ranges can be specified as presets DOME or FULL_360:
"recognitionRangesPreset": "DOME | FULL_360"
When using the DOME preset, an upVector must be defined at the model level in the request body. The recognition ranges can also be specified in full as follows:
"recognitionRanges": {
"rotation": [ 0, 0, 0, 1 ],
"azimRange": [ -3.14, 3.14 ],
"elevRange": [ -1, 0 ],
"rollRange": [ -0.2, 0.2 ]
}
The rotation field can be used to change the frame of reference for the azimuth, elevation, and roll angles. A unit rotation quaternion means that the azimuth, elevation, and roll, identify rotations around the Y, X and Z axisrespectively.
See also the Viewing Angles and Orientation section in Configuring Advanced Guide Views.
Example request body with presets
Define a view with a dome detection range and a full model bounding box:
{
"name": "advanced-dataset-with-presets",
"targetSdk": "10.5",
"models": [
{
"name": "cad-model-name",
"cadDataUrl":
https: //file.glb,
"upVector": [ 0, 1, 0 ],
"views": [
{
"name": "viewpoint-name",
"targetExtentPreset": "FULL_MODEL",
"recognitionRangesPreset": "DOME"
}
]
}
],
"realisticAppearance": false
}
Example request body without presets
Define a view with a full 360-degree detection range and a target volume centered at the origin:
{
"name": "advanced-dataset-without-presets",
"targetSdk": "10.5",
"models": [
{
"name": "cad-model-name",
"cadDataUrl": "https://file.glb",
"views": [
{
"name": "viewpoint-name",
"targetExtent": {
"translation": [ 0, 0, 0 ],
"rotation": [ 0, 0, 0, 1 ],
"scale": [ 1, 1, 1 ]
},
"recognitionRanges": {
"rotation": [ 0, 0, 0, 1 ],
"azimRange": [ -3.14, 3.14 ],
"elevRange": [ -1.57, 1.57 ],
"rollRange": [ -0.2, 0.2 ]
}
}
]
}
],
"realisticAppearance": false
}
Advanced Model Target with Multiple Views
There may be use cases in which multiple detection ranges or multiple CAD models are desired. To accomplish this, add additional views and/or models to your dataset creation request body. Just remember that the recognition rangesshould not overlap on the same CAD model.
Example request body for an Advanced Model Target with multiple CAD models:
{
"name": "advanced-dataset-multi-model-name",
"targetSdk": "10.5",
"models": [
{
"name": "cad-model-1-name",
"upVector": [ 0, 1, 0 ],
"cadDataUrl": "https://file-1.glb",
"views": [
{
"name": "viewpoint-name",
"targetExtentPreset": "FULL_MODEL",
"recognitionRangesPreset": "DOME"
}
]
},
{
"name": "cad-model-2-name",
"upVector": [ 0, 1, 0 ],
"cadDataUrl": "https://file-2.glb",
"views": [
{
"name": "viewpoint-name",
"targetExtentPreset": "FULL_MODEL",
"recognitionRangesPreset": "DOME"
}
]
}
],
"realisticAppearance": false
}
Uploading CAD Models
CAD models can be provided in 2 different ways: Either by embedding the model data as Base64 encoded string in the cadDataBlob field or by specifying a URL in the cadDataUrl field.
The cadDataBlob field is appropriate for small CAD models (less than 20MB) while the URL can be used for larger models.
The URL must be accessible from the Model Target Web API, so it is recommended to use a signed URL with an expiration date. Signed URLs can be created easily for AWS S3 objects or Azure Storage objects:
Supported CAD Model Formats
The API preferentially supports reading glTF 2.0, provided as a single .glb file or as zipped glTF. In addition, thefollowing file formats are also supported if the cadDataFormat is specified:
|
File Format |
cadDataFormat |
|
glTF 2.0 as .glb file or as zipped glTF |
No specification needed. |
|
Creo View (.pvz), Collada (.dae), FBX (.fbx), IGES (.igs, .iges), Wavefront (.obj), STL (.stl,.sla), VRML (.wrl, .vrml). |
The |
Additional Options
The targetSdk field must be used to specify the expected (minimum) SDK version used to load the generated dataset.
Based on the target SDK, the API will allow specific features and optimizations such as Draco compression, runtime Guide View rendering, and etc.
Tracking Mode
The trackingMode field can be used to improve tracking on specific object types. Set it to car, scan or leave it as default. For information on the different modes’ benefits, please refer to Optimizing Model Target Tracking.
TIP: The tracking mode can be specified independently for each model.
Motion Hint
The motionHint field (see the Model Target Motion Hint” section in Optimizing Model Target Tracking) can be used to improve tracking for static vs. moving (adaptive) objects.
The motionHint field can be set for each model in the dataset creation request to either static (default) or adaptive.
Up Vector
If the model has a natural up vector, meaning that the object is always expected to be upright (a factory machine, a table-top object, a car, etc.) it is strongly recommended to specify an upVector field; it will greatly improve its tracking performance.
The upVector can be set independently for each model in the request body. The upVector is mandatory when usingthe DOME recognition range preset.
Automatic Coloring
3D Models without photorealistic textures can have improved detection results if the automatic coloring is enabled.
The automaticColoring field can be set for each model in the request and can have values never, always or auto. By default, automatic coloring is disabled (never).
See also Automatic Coloring.
NOTE: The realisticAppearance flag cannot be enabled together with automatic coloring.
Realistic Appearance
3D Models with photorealistic textures can have improved detection results if realisticAppearance is set to true.
Realistic appearance cannot be set per model and must be defined at the root of the Json request object. It is mandatory for advanced datasets.
NOTE: The realisticAppearance field cannot be enabled together with automatic coloring.
Model Scale
The Vuforia Engine SDK requires models in real life scale and the input model units is assumed to be meters.
If necessary to adjust the 3D model to its real-life scale, each model object in the request body can specify a uniformScale parameter that is a scale factor used to preprocess the 3D model.
Simplify
The API allows simplification of CAD models and as part of the dataset creation. Simplification will produce optimized models for high performance and robust AR object tracking experiences.
The simplify field can be set for each model in the request and can have values never, always or auto. By default, simplification is set to auto.
At the moment, only simplification of a single model per request is supported.
Data Retention
The API servers and the processing servers are hosted in the AWS Cloud eu-west-1 region.
The uploaded CAD models and data are transferred to the cloud using TLS 1.2 encryption and then stored using AES-256 encryption at rest.
All data associated to a Model Target dataset can be removed by issuing a DELETE request for a specific dataset UUID.
For example, a standard Model Target can be deleted as follows using curl:
curl -XDELETE --header "Authorization: Bearer $token" "https://vws.vuforia.com/modeltargets/datasets/$uuid"
Troubleshooting
For all 4xx and 5xx HTTP status responses, the API returns additional information in the response body that could help with troubleshooting the issue.
For example, if the request body of the dataset creation request does not specify a CAD Model, the response will be:
{
"error": {
"code": "BAD_REQUEST",
"message": "model 'cad-model-name' is invalid. One of `cadDataBlob`, `cadDataUrl` need to be provided"
}
}
Unknown errors caused by issues on the MTG API, will return a generic error response:
{
"error": {
"code": "ERROR",
"message": "Internal Server Error"
}
}
If such an error persists contact us via the Support Center.
Additionally, when retrieving the status of a dataset, the response can contain detailed errors or warnings that could help with identifying problematic CAD models (such as symmetric ones), objects that cannot be well discriminated, etc.