Working with Cloud Recognition

This article introduces you to the steps for creating a basic Vuforia Engine Cloud Recognition app in Unity and in native for Android, iOS, and UWP. Use Cloud Recognition in your application to recognize and track as many as millions of Image Targets or if your targets are frequently updated.

To learn more about the benefits, licensing, and use cases of Cloud Recognition, please refer to the Cloud Recognition article.

Sections

Accessing Sample Code

A great way to start is to see the Cloud Recognition feature in action. It is available only for Unity at Vuforia Engine Developer Portal Samples. The Cloud Recognition Books Sample demonstrates how the OnNewSearchResult method retrieves a CloudRecoSearchResult from which the metadata is available in the CloudRecoEventHandler.cs script.

Unity Setup

The following steps explain how to create a cloud recognition app in Unity. The app reacts to a cloud target detection event, retrieves the metadata associated with the target, and displays the metadata text.

Prerequisites 

Instructions

  1. Create a new project in Unity.
  2. Create a license key for your project.
  3. Add the license key to your Vuforia Engine app.
  4. In the Hierarchy window, delete the Main Camera object.
  5. In the GameObject menu, select and add Vuforia Engine -AR Camera
  6. In the GameObject menu, select Vuforia Engine -> Cloud Recognition -Cloud Recognition. A new Cloud Recognition object is created.
  7. In the Inspector window of your Cloud Recognition GameObject, enter your cloud database client keys in the client keys in the Access Key and Secret Key fields. For information about creating a cloud database, refer to the Cloud Databases article. 
  8. Create a C# script named SimpleCloudRecoEventHandler.cs
  9. Attach the SimpleCloudRecoEventHandler script to the Cloud Recognition GameObject.
  10. Implement the SimpleCloudRecoEventHandler script by referencing the following code samples:
  1. Implement the RegisterEventHandlers interface and register it with the CloudRecoBehaviour:
using UnityEngine;
using Vuforia;

public class SimpleCloudRecoEventHandler : MonoBehaviour
{
    CloudRecoBehaviour mCloudRecoBehaviour;
    bool mIsScanning = false;
    string mTargetMetadata = "";

    public ImageTargetBehaviour ImageTargetTemplate;

    // Register cloud reco callbacks
    void Awake()
    {
        mCloudRecoBehaviour = GetComponent<CloudRecoBehaviour>();
        mCloudRecoBehaviour.RegisterOnInitializedEventHandler(OnInitialized);
        mCloudRecoBehaviour.RegisterOnInitErrorEventHandler(OnInitError);
        mCloudRecoBehaviour.RegisterOnUpdateErrorEventHandler(OnUpdateError);
        mCloudRecoBehaviour.RegisterOnStateChangedEventHandler(OnStateChanged);
        mCloudRecoBehaviour.RegisterOnNewSearchResultEventHandler(OnNewSearchResult);
    }
    //Unregister cloud reco callbacks when the handler is destroyed
    void OnDestroy()
    {
        mCloudRecoBehaviour.UnregisterOnInitializedEventHandler(OnInitialized);
        mCloudRecoBehaviour.UnregisterOnInitErrorEventHandler(OnInitError);
        mCloudRecoBehaviour.UnregisterOnUpdateErrorEventHandler(OnUpdateError);
        mCloudRecoBehaviour.UnregisterOnStateChangedEventHandler(OnStateChanged);
        mCloudRecoBehaviour.UnregisterOnNewSearchResultEventHandler(OnNewSearchResult);
    }
  1. Implementing the OnInitialized()OnInitError() and OnUpdateError() methods are optional. These methods provide useful log data in case an error occurs::
    public void OnInitialized(CloudRecoBehaviour cloudRecoBehaviour)
    {
        Debug.Log("Cloud Reco initialized");
    }

    public void OnInitError(CloudRecoBehaviour.InitError initError)
    {
        Debug.Log("Cloud Reco init error " + initError.ToString());
    }

    public void OnUpdateError(CloudRecoBehaviour.QueryError updateError)
    {
        Debug.Log("Cloud Reco update error " + updateError.ToString());

    }
  1. Implement the OnStateChanged() method of the interface to determine whether Vuforia Engine is currently performing a Cloud Reco scan:
    public void OnStateChanged(bool scanning)
    {
        mIsScanning = scanning;

        if (scanning)
        {
            // Clear all known targets
        }
    }
  1. Implement the OnNewSearchResult() method. This code only processes a cloud recognition event without displaying a 3D augmentation for the target. See Add a Cloud Image Target below.
    // Here we handle a cloud target recognition event
    public void OnNewSearchResult(CloudRecoBehaviour.CloudRecoSearchResult cloudRecoSearchResult )
    {
        // Store the target metadata
        mTargetMetadata = cloudRecoSearchResult.MetaData;

        // Stop the scanning by disabling the behaviour
        mCloudRecoBehaviour.enabled = false;
    }
  1. Implement the OnGUI() method to display the current scanning state and the metadata of the last cloud target detected:
  void OnGUI() {
      // Display current 'scanning' status
      GUI.Box (new Rect(100,100,200,50), mIsScanning ? "Scanning" : "Not scanning");
      // Display metadata of latest detected cloud-target
      GUI.Box (new Rect(100,200,200,50), "Metadata: " + mTargetMetadata);
      // If not scanning, show button
      // so that user can restart cloud scanning
      if (!mIsScanning) {
          if (GUI.Button(new Rect(100,300,200,50), "Restart Scanning")) {
          // Reset Behaviour
          mCloudRecoBehaviour.enabled = true;
          mTargetMetadata=””;
          }
      }
  }
  1. Save the current scene.

Add a Cloud Image Target

Perform the following steps to add a 3D augmentation object on top of your cloud target upon detection:

  1. In the GameObject menu, select Vuforia Engine -> Cloud Image -> Cloud Image Target to add a new Image Target object. 
  2. Right-click ImageTarget and select 3D Object -> Cube to create a simple cube object. 
  3. Center and scale the cube object appropriately so that it fits nicely on top of the Image Target plane in the scene.
  4. Optionally, add directional lighting to the scene so that any augmentation appears shaded.
  5. Add a public member variable named ImageTargetTemplate to the SimpleCloudRecoEventHandler.cs script:
public ImageTargetBehaviour ImageTargetTemplate;
  1. Modify the SimpleCloudRecoEventHandler.cs script by adding the following code to the OnNewSearchResult() method. This code programmatically instantiates an Image Target that corresponds to the one detected by the Cloud Recognition engine:
// Build augmentation based on target 
if (ImageTargetTemplate)
{
    /* Enable the new result with the same ImageTargetBehaviour: */
    mCloudRecoBehaviour.EnableTracking(cloudRecoSearch, ImageTargetTemplate.gameObject);
}
  1. Select the CloudRecognition object in the scene.
    The Inspector should display the Image Target Template field in the Simple Cloud Reco Event Handler component.
  2. In the Hierarchy window, drag the  ImageTarget into the Image Target Template field in the Inspector window.
  3. Save the current scene.
  4. Build and run your project or test it with Vuforia Play Mode.

Native Setup for Android, UWP, and iOS

Cloud Recognition is handled by the Engine and through a Cloud Image Target Observer. The Cloud Recognition Service is configured with VuCloudImageTargetConfig and detected recognition events are retrieved with the VuCloudImageTargetObservationTargetInfo.

Differently from other tracking API in the SDK, where you get continuous tracking updates via pull mechanisms from the State, Cloud Recognition does not necessarily get continuous information; you need to register a handle for getting updates. This is done with the vuCloudImageTargetObserverRegisterHandlers.

Initializing the system

To use Cloud Recognition, you first need to initialize the system and set the credentials for your cloud database. The following code starts the Engine, creates the Cloud Image Target Observer, and initializes the handler.

Create a Cloud Image Target Observer

VuObserver* observer = nullptr;

VuCloudImageTargetConfig cloudImageTargetConfig = vuCloudImageTargetConfigDefault();
cloudImageTargetConfig.userAuth = “mAuthenticationInfo”;
cloudImageTargetConfig.secretAuth = “mAuthenticationInfo”;
cloudImageTargetConfig.activate = VU_TRUE;

VuCloudImageTargetCreationError creationError;
vuEngineCreateCloudImageTargetObserver(appController.mEngine, &observer, &cloudImageTargetConfig, &creationError);

Starting and Stopping a Cloud Recognition Search

The following sample code shows what a simple application needs to do to start Cloud Recognition. It registers both the VuCloudImageTargetObservationsHandler and the VuCloudImageTargetQueryErrorHandler.

// Register Cloud Image Target-specific event handlers
// NOTE: We pass in a pointer to our CloudImageTargetFeature instance as user data
vuCloudImageTargetObserverRegisterHandlers(observer, &recoEventHandler, &queryErrorHandler, this);

 Destroy Observer

Destroy the Observer after usage to free up memory.

// Destroy the observer
vuObserverDestroy(observer);

 

Handling Search Results

During the execution, the developer must regularly check for the availability of new Cloud Recognition search results and, if desired, create the Image Target Observer for tracking.

This can be done with vuEngineCreateImageTargetObserverFromCloudObservation method.

Void recoEventHandler(const VuObservationList* observations, void* clientData)
{    
    vuObservationListCreate(VuObservationList** observations);      
    
    int32_t numObservations = 0;
    REQUIRE_SUCCESS(vuObservationListGetSize(observations, &numObservations));
    
    // Iterate through all Cloud Image Target observations received via reco event
    for (int i = 0; i < numObservations; i++)
    {
        VuObservation* observation = nullptr;
        REQUIRE_SUCCESS(vuObservationListGetElement(observations, 0, &observation));
        
        assert(observation);
        assert(vuObservationIsType(observation, VU_OBSERVATION_CLOUD_IMAGE_TARGET_TYPE) == VU_TRUE);
        
        // Log Cloud Image Target information
        VuCloudImageTargetObservationTargetInfo cloudImageTargetInfo;
        vuCloudImageTargetObservationGetTargetInfo(observation, &cloudImageTargetInfo);
        LOG("Cloud Image Target '%s' recognized on server: rating = %d, metadata = '%s', unique ID = '%s'",
            cloudImageTargetInfo.name,
            cloudImageTargetInfo.trackingRating,
            cloudImageTargetInfo.metadata,
            cloudImageTargetInfo.uniqueId);
        
        // Create Image Target observer using Cloud Image Target observation
        VuImageTargetCloudObservationConfig cloudObservationConfig;
        cloudObservationConfig.observation = observation;
        cloudObservationConfig.activate = VU_TRUE; // automatically activate if engine is running
        
        VuObserver* observer = nullptr;
        VuImageTargetCloudObservationCreationError creationError;
        if (vuEngineCreateImageTargetObserverFromCloudObservation(mEngine, &observer, &cloudObservationConfig, &creationError) == VU_SUCCESS)
        {
            assert(observer);
            assert(vuObserverIsType(observer, VU_OBSERVER_IMAGE_TARGET_TYPE) == VU_TRUE);
            
            // Add observer to internal bookkeeping
            cloudImageTargetFeature->addObserver(observer);
        }
    }
}

For more information on counting and handling the Cloud Recognition search results, please see Counting Cloud Recognition Events.

Error Messages

If you expect to exceed the recognition amount with your plan, or if you receive an error message indicating that you have exceeded the allowed number of calls, please contact us.

The following error codes may be returned by the VuCloudImageTargetQueryError. This table lists the codes alongside their meanings.

Example error handling:

void queryErrorHandler(VuCloudImageTargetQueryError queryError, void* clientData)
{
    LOG("Failed to query Cloud Image Target due to the following error: %s (%d)", queryError);
}

Code

Meaning

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_NO_NETWORK_CONNECTION

No network connection

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_SERVICE_NOT_AVAILABLE

Server not found, down, or overloaded

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_AUTHORIZATION_FAILED

Credentials are wrong or outdated

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_PROJECT_SUSPENDED

The specified project was suspended

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_BAD_FRAME_QUALITY

Low frame quality has been continuously observed

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_UPDATE_SDK

SDK version outdated

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_TIMESTAMP_OUT_OF_RANGE

Client/Server clocks too far apart

VU_CLOUD_IMAGE_TARGET_QUERY_ERROR

_REQUEST_TIMEOUT

No response to network request after timeout