Working with Cloud Recognition

This article introduces you to the steps for creating a basic Vuforia Engine Cloud Recognition app in Unity and in native for Android, iOS, and UWP. Use Cloud Recognition in your application to recognize and track as many as millions of Image Targets or if your targets are frequently updated.

To learn more about the benefits, licensing, and use cases of Cloud Recognition, please refer to the Cloud Recognition article.

Sections

• Unity Setup
• Native Setup

Accessing Sample Code

A great way to start is to see the Cloud Recognition feature in action. It is available only for Unity in the Vuforia Core Samples. The sample demonstrates how the OnNewSearchResult method retrieves a CloudRecoSearchResult from which the metadata is available in the CloudRecoEventHandler.cs script.

1. Implement the RegisterEventHandlers interface and register it with the CloudRecoBehaviour:

using UnityEngine;
using Vuforia;

public class SimpleCloudRecoEventHandler : MonoBehaviour
{
    CloudRecoBehaviour mCloudRecoBehaviour;
    bool mIsScanning = false;
    string mTargetMetadata = "";

    public ImageTargetBehaviour ImageTargetTemplate;

    // Register cloud reco callbacks
    void Awake()
    {
        mCloudRecoBehaviour = GetComponent<CloudRecoBehaviour>();
        mCloudRecoBehaviour.RegisterOnInitializedEventHandler(OnInitialized);
        mCloudRecoBehaviour.RegisterOnInitErrorEventHandler(OnInitError);
        mCloudRecoBehaviour.RegisterOnUpdateErrorEventHandler(OnUpdateError);
        mCloudRecoBehaviour.RegisterOnStateChangedEventHandler(OnStateChanged);
        mCloudRecoBehaviour.RegisterOnNewSearchResultEventHandler(OnNewSearchResult);
    }
    //Unregister cloud reco callbacks when the handler is destroyed
    void OnDestroy()
    {
        mCloudRecoBehaviour.UnregisterOnInitializedEventHandler(OnInitialized);
        mCloudRecoBehaviour.UnregisterOnInitErrorEventHandler(OnInitError);
        mCloudRecoBehaviour.UnregisterOnUpdateErrorEventHandler(OnUpdateError);
        mCloudRecoBehaviour.UnregisterOnStateChangedEventHandler(OnStateChanged);
        mCloudRecoBehaviour.UnregisterOnNewSearchResultEventHandler(OnNewSearchResult);
    }

2. Implementing the OnInitialized(), OnInitError() and OnUpdateError() methods are optional. These methods provide useful log data in case an error occurs::

    public void OnInitialized(CloudRecoBehaviour cloudRecoBehaviour)
    {
        Debug.Log("Cloud Reco initialized");
    }

    public void OnInitError(CloudRecoBehaviour.InitError initError)
    {
        Debug.Log("Cloud Reco init error " + initError.ToString());
    }

    public void OnUpdateError(CloudRecoBehaviour.QueryError updateError)
    {
        Debug.Log("Cloud Reco update error " + updateError.ToString());

    }

3. Implement the OnStateChanged() method of the interface to determine whether Vuforia Engine is currently performing a Cloud Reco scan:

    public void OnStateChanged(bool scanning)
    {
        mIsScanning = scanning;

        if (scanning)
        {
            // Clear all known targets
        }
    }

4. Implement the OnNewSearchResult() method. This code only processes a cloud recognition event without displaying a 3D augmentation for the target. See Add a Cloud Image Target below.

    // Here we handle a cloud target recognition event
    public void OnNewSearchResult(CloudRecoBehaviour.CloudRecoSearchResult cloudRecoSearchResult )
    {
        // Store the target metadata
        mTargetMetadata = cloudRecoSearchResult.MetaData;

        // Stop the scanning by disabling the behaviour
        mCloudRecoBehaviour.enabled = false;
    }

5. Implement the OnGUI() method to display the current scanning state and the metadata of the last cloud target detected:

  void OnGUI() {
      // Display current 'scanning' status
      GUI.Box (new Rect(100,100,200,50), mIsScanning ? "Scanning" : "Not scanning");
      // Display metadata of latest detected cloud-target
      GUI.Box (new Rect(100,200,200,50), "Metadata: " + mTargetMetadata);
      // If not scanning, show button
      // so that user can restart cloud scanning
      if (!mIsScanning) {
          if (GUI.Button(new Rect(100,300,200,50), "Restart Scanning")) {
          // Reset Behaviour
          mCloudRecoBehaviour.enabled = true;
          mTargetMetadata="";
          }
      }
  }

13. Save the current scene.

Add a Cloud Image Target

Perform the following steps to add a 3D augmentation object on top of your cloud target upon detection:

1. In the GameObject menu, select Vuforia Engine -> Cloud Image -> Cloud Image Target to add a new Image Target object. 
2. Right-click ImageTarget and select 3D Object -> Cube to create a simple cube object. 
3. Center and scale the cube object appropriately so that it fits nicely on top of the Image Target plane in the scene.
4. Optionally, add directional lighting to the scene so that any augmentation appears shaded.
5. Add the SimpleCloudRecoEventHandler.cs script as a component to the Cloud Image Target GameObject.
6. Modify the SimpleCloudRecoEventHandler.cs script by adding the following code to the OnNewSearchResult() method. This code programmatically instantiates an Image Target that corresponds to the one detected by the Cloud Recognition engine:

// Build augmentation based on target 
if (ImageTargetTemplate)
{
    /* Enable the new result with the same ImageTargetBehaviour: */
    mCloudRecoBehaviour.EnableObservers(cloudRecoSearchResult, ImageTargetTemplate.gameObject);
}

7. Select the CloudRecognition object in the scene.
   The Inspector should display the Image Target Template field in the Simple Cloud Reco Event Handler component.
8. In the Hierarchy window, drag the  ImageTarget into the Image Target Template field in the Inspector window.
9. Save the current scene.
10. Build and run your project or test it with Vuforia Play Mode.

VuObserver* observer = nullptr;

VuCloudImageTargetConfig cloudImageTargetConfig = vuCloudImageTargetConfigDefault();
cloudImageTargetConfig.userAuth = “mAuthenticationInfo”;
cloudImageTargetConfig.secretAuth = “mAuthenticationInfo”;
cloudImageTargetConfig.activate = VU_TRUE;

VuCloudImageTargetCreationError creationError;
vuEngineCreateCloudImageTargetObserver(appController.mEngine, &observer, &cloudImageTargetConfig, &creationError);

// Register Cloud Image Target-specific event handlers
// NOTE: We pass in a pointer to our CloudImageTargetFeature instance as user data
vuCloudImageTargetObserverRegisterHandlers(observer, &recoEventHandler, &queryErrorHandler, this);

 Destroy Observer

Destroy the Observer after usage to free up memory.

// Destroy the observer
vuObserverDestroy(observer);

Handling Search Results

During the execution, the developer must regularly check for the availability of new Cloud Recognition search results and, if desired, create the Image Target Observer for tracking.

This can be done with vuEngineCreateImageTargetObserverFromCloudObservation method.

Void recoEventHandler(const VuObservationList* observations, void* clientData)
{    
    vuObservationListCreate(VuObservationList** observations);      
    
    int32_t numObservations = 0;
    REQUIRE_SUCCESS(vuObservationListGetSize(observations, &numObservations));
    
    // Iterate through all Cloud Image Target observations received via reco event
    for (int i = 0; i < numObservations; i++)
    {
        VuObservation* observation = nullptr;
        REQUIRE_SUCCESS(vuObservationListGetElement(observations, 0, &observation));
        
        assert(observation);
        assert(vuObservationIsType(observation, VU_OBSERVATION_CLOUD_IMAGE_TARGET_TYPE) == VU_TRUE);
        
        // Log Cloud Image Target information
        VuCloudImageTargetObservationTargetInfo cloudImageTargetInfo;
        vuCloudImageTargetObservationGetTargetInfo(observation, &cloudImageTargetInfo);
        LOG("Cloud Image Target '%s' recognized on server: rating = %d, metadata = '%s', unique ID = '%s'",
            cloudImageTargetInfo.name,
            cloudImageTargetInfo.trackingRating,
            cloudImageTargetInfo.metadata,
            cloudImageTargetInfo.uniqueId);
        
        // Create Image Target observer using Cloud Image Target observation
        VuImageTargetCloudObservationConfig cloudObservationConfig;
        cloudObservationConfig.observation = observation;
        cloudObservationConfig.activate = VU_TRUE; // automatically activate if engine is running
        
        VuObserver* observer = nullptr;
        VuImageTargetCloudObservationCreationError creationError;
        if (vuEngineCreateImageTargetObserverFromCloudObservation(mEngine, &observer, &cloudObservationConfig, &creationError) == VU_SUCCESS)
        {
            assert(observer);
            assert(vuObserverIsType(observer, VU_OBSERVER_IMAGE_TARGET_TYPE) == VU_TRUE);
            
            // Add observer to internal bookkeeping
            cloudImageTargetFeature->addObserver(observer);
        }
    }
}

For more information on counting and handling the Cloud Recognition search results, please see Counting Cloud Recognition Events.

Error Messages

If you expect to exceed the recognition amount with your plan, or if you receive an error message indicating that you have exceeded the allowed number of calls, please contact us.

The following error codes may be returned by the VuCloudImageTargetQueryError. This table lists the codes alongside their meanings.

Example error handling:

void queryErrorHandler(VuCloudImageTargetQueryError queryError, void* clientData)
{
    LOG("Failed to query Cloud Image Target due to the following error: %s (%d)", queryError);
}

Learn More

How to Perform an Image Recognition Query

Rendering in Native

Counting Cloud Recognition Events