How To Implement Cloud Reco


Unity

This article explains how to create a basic Vuforia Cloud recognition app in Unity.

The app in question simply reacts to a cloud target detection event. Then the app retrieves the metadata associated with the target, and displays the metadata text on screen.

Prerequisites

  • Unity 2017.2 or Later with Vuforia Augmented Reality Support
  • Your Favorite Code Editor (Visual Studio, Mono Develop, etc)
  • Vuforia Developer Account

Instructions

  1. Create a new Unity project.
  2. Remove the Main Camera from your scene.
  3. Create a new AR Camera by using the GameObject>Vuforia>AR Camera menu.
  4. Create a new CloudRecognition by using the GameObject>Vuforia>Cloud Image>Cloud Provider menu.
  5. Select the CloudRecognition object in your scene and look at the Inspector panel.
  6. Within the CloudRecognition inspector, fill in the Access Key and Secret Key fields with the client keys of your cloud database. For information about creating a cloud database, See: Vuforia Target Manager
  7. Create a C# script called SimpleCloudHandler.cs, for example.
  8. Attach the script to the CloudRecognition GameObject in your scene.
  9. Implement the SimpleCloudHandler script as explained in the following steps:
    1. Implement the ICloudRecoEventHandler interface and register it with the CloudRecognitionBehaviour, as illustrated in the following code snippet:

 

using Vuforia;
public class SimpleCloudHandler : MonoBehaviour, ICloudRecoEventHandler {
private CloudRecoBehaviour mCloudRecoBehaviour;
private bool mIsScanning = false;
private string mTargetMetadata = "";
// Use this for initialization
void Start () {
// register this event handler at the cloud reco behaviour
mCloudRecoBehaviour = GetComponent<CloudRecoBehaviour>();

    if (mCloudRecoBehaviour)
    {
    mCloudRecoBehaviour.RegisterEventHandler(this);
    }
}
}
  1. Implement the methods of the OnInitialized(),OnInitError() and OnUpdateError() of the ICloudRecoEventHandler interface, as shown in this example:
public void OnInitialized() {
    Debug.Log ("Cloud Reco initialized");
}
public void OnInitError(TargetFinder.InitState initError) {
    Debug.Log ("Cloud Reco init error " + initError.ToString());
}
public void OnUpdateError(TargetFinder.UpdateState updateError) {
    Debug.Log ("Cloud Reco update error " + updateError.ToString());
}
  1. Implement the OnStateChanged() method of the ICloudRecoEventHandler interface to keep track of the scanning mode. This action lets you know whether Vuforia is scanning the cloud.
public void OnStateChanged(bool scanning) {
mIsScanning = scanning;
    if (scanning)
    {
    // clear all known trackables
    var tracker = TrackerManager.Instance.GetTracker<ObjectTracker>();
    tracker.TargetFinder.ClearTrackables(false);
    }
}
  1. Implement OnNewSearchResult() method of the ICloudRecoEventHandler interface, as shown in this example:
// Here we handle a cloud target recognition event
public void OnNewSearchResult(TargetFinder.TargetSearchResult targetSearchResult) {
    // do something with the target metadata
    mTargetMetadata = targetSearchResult.MetaData;
    // stop the target finder (i.e. stop scanning the cloud)
    mCloudRecoBehaviour.CloudRecoEnabled = false;
}
  1. Implement the OnGUI() method to display the current scanning state and the metadata of the last cloud target that was detected, as shown in this code snippet:
void OnGUI() {
    // Display current 'scanning' status
    GUI.Box (new Rect(100,100,200,50), mIsScanning ? "Scanning" : "Not scanning");
    // Display metadata of latest detected cloud-target
    GUI.Box (new Rect(100,200,200,50), "Metadata: " + mTargetMetadata);
    // If not scanning, show button
    // so that user can restart cloud scanning
    if (!mIsScanning) {
        if (GUI.Button(new Rect(100,300,200,50), "Restart Scanning")) {
        // Restart TargetFinder
        mCloudRecoBehaviour.CloudRecoEnabled = true;
        }
    }
}
  1. Save the current scene, and then build and run your project.
  2. Make sure you have internet connection enabled.
  3. Enjoy the result!

EXAMPLE

Assume that you want to show a 3D augmentation object on top of your cloud target upon detection. Follow these additional steps:

  1. Add a new Image Target using the GameObject>Vuforia>Cloud Image>Cloud Image Target menu.
  2. Add a 3D model as child of the Image Target. For example, create a simple cube object. Then center and scale the object appropriately so that it fits nicely on top of the Image Target plane in the scene.
  3. Add some lighting to the scene (for example, directional light) so that any augmentation appears nicely shaded.
  4. Modify your SimpleCloudHandler script by adding the following code to the OnNewSearchResult() method. This code is meant to programmatically instantiate an ImageTarget that corresponds to the one detected by the Cloud Recognition engine at any time:
// Build augmentation based on target
if (ImageTargetTemplate) {
    // enable the new result with the same ImageTargetBehaviour:
    ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker>();
   ImageTargetBehaviour imageTargetBehaviour =
    (ImageTargetBehaviour)tracker.TargetFinder.EnableTracking(
    targetSearchResult, ImageTargetTemplate.gameObject);
}
  1. Add a public member variable called ImageTargetTemplate to the script using the following code to the top of the SimpleCloudHandler Class:
public ImageTargetBehaviour ImageTargetTemplate;
  1. Back in the Unity scene, select the CloudRecognition object in your scene. The Inspector should now show the Image Target Template field in the Simple Cloud Handler component.
  2. From the scene hierarchy, drag the Image Target onto the Image Target Template in the Inspector.
  3. Save the current scene, and then build and run your project. Make sure you have internet connection enabled.

Native

The Cloud Recognition APIs are exposed from the TargetFinder and TargetSearchResult classes. This article shows you how to implement these classes using Java and C++ to develop apps using the Vuforia native SDKs for Android, iOS, and UWP.

Initialization

To use Cloud Recognition you first need to initialize the system and set the credentials for your image database. The following snippet shows initializing and deinitializing of Cloud Reco:

C++:

QCAR::TargetFinder* finder;

const char* const kAccessKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
const char* const kSecretKey = "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy";

onApplicationInit()
{
           QCAR::TargetFinder* targetFinder = imageTracker->getTargetFinder();
    assert(targetFinder != NULL);

    // Initialize Cloud Reco with project credentials:
    if (targetFinder->startInit(kAccessKey, kSecretKey))
    {
        targetFinder->waitUntilInitFinished();
    }
}

onApplicationDeinit()
{
    // Deinitialize Visual Search
    targetFinder->deinit());
}

Java:

private static final String kAccessKey = "xxxxxx";
 private static final String kSecretKey = "yyyyyy";       
onApplicationInit()
{
           TargetFinder targetFinder = imageTracker.getTargetFinder();
     
    // Initialize Cloud Reco with project credentials:
    if (targetFinder.startInit(kAccessKey, kSecretKey))
    {
        targetFinder.waitUntilInitFinished();
    }
}

onApplicationDeinit()
{
    // Deinitialize Cloud Reco
    TargetFinder finder = imageTracker.getTargetFinder();
    targetFinder.deinit());
}
 

Starting and stopping a Cloud Recognition search

The following snippet of sample code shows what a simple application would need to do to start Cloud Recognition when the application is started (or resumed) and to stop it when the application is paused:

C++:

QCAR::TargetFinder* finder;

onApplicationResume()
{
    // Start cloud based recognition
    finder->startRecognition();
}

onApplicationPause()
{
    // Stop cloud based recognition
    finder->stop();
}

Java:

onApplicationResume()
{
    // Start cloud based recognition
    TargetFinder finder = imageTracker.getTargetFinder();
    finder.startRecognition();
}

onApplicationPause()
{
    // Stop cloud based recognition
    TargetFinder finder = imageTracker.getTargetFinder();
    finder.stop();
}


Handling search results

During the execution of the application, the developer must regularly check for the availability of new Cloud Recognition search results and if desired enable the target(s) for tracking.

This can be done in the QCAR_onUpdate() method of the UpdateCallback class.

C++:

QCAR::TargetFinder* finder;

MyQCAR_OnUpdate()
{
    // Check if there are new results available:
    if (finder->updateSearchResults() == QCAR::TargetFinder::UPDATE_RESULTS_AVAILABLE)
    {
        // Iterate through the new results:
        for (int i = 0; i < finder->getResultCount(); ++i)
        {
            const QCAR::TargetSearchResult* result = finder->getResult(i);

            // Check if this target is suitable for tracking:
            if (result->getTrackingRating() > 0)
            {
                // Enable this target for tracking:
                finder->enableTracking(result);
            }
        }
    }
}
 

Java:

MyQCAR_OnUpdate()
{
    TargetFinder finder = imageTracker.getTargetFinder();
    // Check if there are new results available:
    final int statusCode = finder.updateSearchResults();
    if (statusCode == TargetFinder.UPDATE_RESULTS_AVAILABLE)
    {
        // Iterate through the new results:
        for (int i = 0; i < finder.getResultCount(); ++i)
        {
            TargetSearchResult result = finder.getResult(i);

            // Check if this target is suitable for tracking:
            if (result.getTrackingRating() > 0)
            {
                // Enable this target for tracking:
                finder.enableTracking(result);
            }   
        }
    }
}