How To Implement Cloud Reco

This article explains how to create a basic Vuforia Engine cloud recognition app in Unity and the native SDKs for Android, iOS, and UWP.

Sections

Accessing sample code

The following sample code is accessible in the Advanced Topics section of the Vuforia Engine Developer Portal Samples page.

Platform  Description
Unity In the Books sample, refer to the CloudRecoEventHandler.cs script attached to the CloudRecognition prefab. The OnNewSearchResult method shows how to get a targetSearchResult object, from which you can then get the metadata.
Android In the Books-<x>-<y>-<z> sample project, refer to the code in the Books.java file.
iOS In the Books-<x>-<y>-<z> sample project, refer to the code in the BooksEAGLView.mm file.

Implementing in Unity

The following steps explain how to create a cloud recognition app. The app reacts to a cloud target detection event, retrieves the metadata associated with the target, and displays the metadata text.

Prerequisites

  • Unity 2017.2 or later with Vuforia Augmented Reality Support
  • Your favorite code editor (Visual Studio, Mono Develop, etc)
  • Vuforia Engine Developer Account

Instructions

  1. Create a new project in Unity.
  2. Create a license key for your project.
  3. Add the license key to your Vuforia Engine app. 
  4. In the Hierarchy window, select the Main Camera object.
  5. In the Inspector window, deselect the checkbox next to Main Camera
  6. In the Game Object menu, select Vuforia Engine > AR Camera. A new AR Camera object is created. 
  7. In the Game Object menu, select Vuforia Engine  Cloud Image > Cloud Recognition. A new Cloud Recognition object is created. 
  8. In the Hierarchy window, select the CloudRecognition object.
  9. In the Inspector window, enter your cloud database client keys in the Access Key and Secret Key fields. For information about creating a cloud database, refer to the Vuforia Target Manager article. 
  10. Create a C# script named SimpleCloudHandler.cs 
  11. Attach the SimpleCloudHandler script to the CloudRecognition game object.
  12. Implement the SimpleCloudHandler script by referencing the following code samples:
  1. Implement the IObjectRecoEventHandler interface and register it with the CloudRecognitionBehaviour:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;

using Vuforia;


public class SimpleCloudHandler : MonoBehaviour, IObjectRecoEventHandler { 
    private CloudRecoBehaviour mCloudRecoBehaviour; 
    private bool mIsScanning = false; 
    private string mTargetMetadata = ""; 
// Use this for initialization 
    void Start () { 
// register this event handler at the cloud reco behaviour 
        mCloudRecoBehaviour = GetComponent<CloudRecoBehaviour>(); 
  
        if (mCloudRecoBehaviour) 
        { 
            mCloudRecoBehaviour.RegisterEventHandler(this); 
        } 
    }
}
  1. Implement the OnInitialized(), OnInitError() and OnUpdateError() methods of the IObjectRecoEventHandler interface:
     public void OnInitialized(TargetFinder targetFinder) {
         Debug.Log ("Cloud Reco initialized");
     }
     public void OnInitError(TargetFinder.InitState initError) {
         Debug.Log ("Cloud Reco init error " + initError.ToString());
     }
     public void OnUpdateError(TargetFinder.UpdateState updateError) {
         Debug.Log ("Cloud Reco update error " + updateError.ToString());
     }
  
  1. Implement the OnStateChanged() method of the IObjectRecoEventHandler interface to determine whether Vuforia Engine is scanning the cloud:
      public void OnStateChanged(bool scanning) {
      mIsScanning = scanning;
          if (scanning)
          {
          // clear all known trackables
          var tracker = TrackerManager.Instance.GetTracker<ObjectTracker>();
          tracker.GetTargetFinder<ImageTargetFinder>().ClearTrackables(false);
          }
      }
  
  1. Implement the OnNewSearchResult() method of the IObjectRecoEventHandler interface:
      // Here we handle a cloud target recognition event
      public void OnNewSearchResult(TargetFinder.TargetSearchResult targetSearchResult) {
      TargetFinder.CloudRecoSearchResult cloudRecoSearchResult = 
          (TargetFinder.CloudRecoSearchResult)targetSearchResult;
      // do something with the target metadata
      mTargetMetadata = cloudRecoSearchResult.MetaData;
      // stop the target finder (i.e. stop scanning the cloud)
      mCloudRecoBehaviour.CloudRecoEnabled = false;
     }
  
  1. Implement theOnGUI() method to display the current scanning state and the metadata of the last cloud target detected:
  void OnGUI() {
      // Display current 'scanning' status
      GUI.Box (new Rect(100,100,200,50), mIsScanning ? "Scanning" : "Not scanning");
      // Display metadata of latest detected cloud-target
      GUI.Box (new Rect(100,200,200,50), "Metadata: " + mTargetMetadata);
      // If not scanning, show button
      // so that user can restart cloud scanning
      if (!mIsScanning) {
          if (GUI.Button(new Rect(100,300,200,50), "Restart Scanning")) {
          // Restart TargetFinder
          mCloudRecoBehaviour.CloudRecoEnabled = true;
          }
      }
  }
  
  1. Save the current scene.
  2. Build and run the project.

Showing a 3D augmentation object

Perform the following steps to show a 3D augmentation object on top of your cloud target upon detection:

  1. In the GameObject menu, select Vuforia Engine > Cloud Image > Cloud Image Target to add a new Image Target object. 
  2. Right-click ImageTarget and select 3D Object > Cube to create a simple cube object. 
  3. Center and scale the cube object appropriately so that it fits nicely on top of the Image Target plane in the scene.
  4. Add directional lighting to the scene so that any augmentation appears shaded.
  5. Modify the SimpleCloudHandler script by adding the following code to the OnNewSearchResult() method. This code programmatically instantiates an ImageTarget that corresponds to the one detected by the Cloud Recognition engine:
// Build augmentation based on target 
        if (ImageTargetTemplate) { 
            // enable the new result with the same ImageTargetBehaviour: 
            ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker>(); 
            tracker.GetTargetFinder<ImageTargetFinder>().EnableTracking(targetSearchResult, ImageTargetTemplate.gameObject); 
        }
  1. Add a public member variable named ImageTargetTemplate to the SimpleCloudHandler script:
public ImageTargetBehaviour ImageTargetTemplate;
  1. Select the CloudRecognition object in the scene.
    The Inspector should display the Image Target Template field in the Simple Cloud Handler component.
  2. In the Hierarchy window, drag ImageTarget onto the Image Target Template in the Inspector window.
  3. Save the current scene.
  4. Build and run your project.

Implementing for native SDKs

The Cloud Recognition APIs are exposed from the TargetFinder and CloudRecoSearchResult classes. The following sections demonstrates how to implement these classes using Java and C++.

Initializing the system

To use Cloud Recognition you first need to initialize the system and set the credentials for your image database. The following code initializes and de-initializes Cloud Reco:

C++:

   Vuforia::TargetFinder* finder;
   
   const char* const kAccessKey = "Insert access key here";
   const char* const kSecretKey = "Insert secret key here";
   
   onApplicationInit()
   {
    Vuforia::TargetFinder* targetFinder = objectTracker->getTargetFinder   (Vuforia::ObjectTracker::TargetFinderType::CLOUD_RECO);
       assert(targetFinder != NULL);
   
       // Initialize Cloud Reco with project credentials:
       if (targetFinder->startInit(kAccessKey, kSecretKey))
       {
           targetFinder->waitUntilInitFinished();
       }
   }
   
   onApplicationDeinit()
   {
       // Deinitialize Visual Search
       targetFinder->deinit());
   }

Java:

    private static final String kAccessKey = "Insert access key here";
     private static final String kSecretKey = "Insert secret key here";       
    onApplicationInit()
    {
    TargetFinder targetFinder = objectTracker.getTargetFinder(ObjectTracker.TargetFinderType.CLOUD_RECO)    ;
         
        // Initialize Cloud Reco with project credentials:
        if (targetFinder.startInit(kAccessKey, kSecretKey))
        {
            targetFinder.waitUntilInitFinished();
        }
    }
    
    onApplicationDeinit()
    {
        // Deinitialize Cloud Reco
        TargetFinder finder = objectTracker.getTargetFinder(ObjectTracker.TargetFinderType.CLOUD_RECO);
        targetFinder.deinit());
    }

Starting and stopping a Cloud Recognition search

The following sample code shows what a simple application needs to do to start Cloud Recognition when the application is started (or resumed) and to stop it when the application is paused:

C++:

  Vuforia::TargetFinder* finder;
  
  onApplicationResume()
  {
      // Start cloud based recognition
      finder->startRecognition();
  }
  
  onApplicationPause()
  {
      // Stop cloud based recognition
      finder->stop();
  }

Java:

   onApplicationResume()
   {
       // Start cloud based recognition
       TargetFinder finder = objectTracker.getTargetFinder(ObjectTracker.TargetFinderType.CLOUD_RECO);
       finder.startRecognition();
   }
   
   onApplicationPause()
   {
       // Stop cloud based recognition
       TargetFinder finder = objectTracker.getTargetFinder(ObjectTracker.TargetFinderType.CLOUD_RECO);
       finder.stop();
   }

Handling search results

During the execution, the developer must regularly check for the availability of new Cloud Recognition search results and, if desired, enable the target(s) for tracking.

This can be done in the QCAR_onUpdate() method of the UpdateCallback class.

C++:

   Vuforia::TargetFinder* finder;
   
   void MyVuforia_OnUpdate()
   {
       // Check if there are new results available:
   
       TargetFinderQueryResult queryResults = finder->updateQueryResults();
       if (queryResults.status == Vuforia::TargetFinder::UPDATE_RESULTS_AVAILABLE)
       {
           // Iterate through the new results:
           for (const Vuforia::TargetSearchResult* result: queryResults.results) {
           {
            const Vuforia::CloudRecoSearchResult* cloudRecoResult = static_cast<const    Vuforia::CloudRecoSearchResult*>(result);
               // Get the target metadata
               const char* metadata = cloudRecoResult->getMetaData();
   
               // Check if this target is suitable for tracking:
               if (cloudRecoResult->getTrackingRating() > 0)
               {
                   // Enable this target for tracking:
                   finder->enableTracking(cloudRecoResult);
               }
           }
       }
   }

Java:

  void MyVuforia_OnUpdate()
  {
      TargetFinder finder = objectTracker.getTargetFinder(ObjectTracker.TargetFinderType.CLOUD_RECO);
      // Check if there are new results available:
      TargetFinderQueryResult queryResults = finder.updateQueryResults();
      if (queryResults.getStatus() == TargetFinder.UPDATE_RESULTS_AVAILABLE)
      {
          // Iterate through the new results:
          for (int i = 0; i < queryResults.getResults().size(); ++i)
          {
              CloudRecoSearchResult result = (CloudRecoSearchResult)queryResults.getResults().at(i);
              // Get the target metadata
              String metadata = result.getMetaData();
  
              // Check if this target is suitable for tracking:
              if (result.getTrackingRating() > 0)
              {
                  // Enable this target for tracking:
                  finder.enableTracking(result);
              }   
          }
      }
  }