How To Use Object Recognition in an Android App

In this article, you will learn how to integrate Object Recognition into your native Android app. We will use the ObjectReco sample app as a reference (code snippets below).

Before you proceed, please make sure to read Getting Started with the Android Native SDK.

Use the Vuforia Object Recognition Sample project structure as a template for your own app. There are 4 steps:

  1. Get a Vuforia Engine license key and include in your app
  2. Load and activate a device database with one or more Object Targets
  3. Configure the Object Target Renderer and handle events arising from the Object Tracker

1. To get a license key and add it to your app:

In com.vuforia.samples.SampleApplication

File SampleApplicationSession.java

Line 336

// An async task to initialize Vuforia asynchronously.
    private class InitVuforiaTask extends AsyncTask<Void, Integer, Boolean>
    {
        // Initialize with invalid value:
        private int mProgressValue = -1;
      
      
        protected Boolean doInBackground(Void... params)
        {
            // Prevent the onDestroy() method to overlap with initialization:
            synchronized (mShutdownLock)
            {
                Vuforia.setInitParameters(mActivity, mVuforiaFlags, "ad57f7d0d29d41369bd421227b27dc8e");
              
                do
                {
                    // Vuforia.init() blocks until an initialization step is
                    // complete, then it proceeds to the next step and reports
                    // progress in percents (0 ... 100%).
                    // If Vuforia.init() returns -1, it indicates an error.
                    // Initialization is done when progress has reached 100%.
                    mProgressValue = Vuforia.init();
                  
                    // Publish the progress value:
                    publishProgress(mProgressValue);
                  
                    // We check whether the task has been canceled in the
                    // meantime (by calling AsyncTask.cancel(true)).
                    // and bail out if it has, thus stopping this thread.
                    // This is necessary as the AsyncTask will run to completion
                    // regardless of the status of the component that
                    // started is.
                } while (!isCancelled() && mProgressValue >= 0
                    && mProgressValue < 100);
 
                return (mProgressValue > 0);
            }
        }

2. To load and activate Device Databases containing Object Targets:

In com.vuforia.samples.VuforiaSamples.app.ObjectRecognition

File ObjectTargets.java

Line 287

// Methods to load and destroy tracking data.
    @Override
    public boolean doLoadTrackersData()
    {
        TrackerManager tManager = TrackerManager.getInstance();
        ObjectTracker objectTracker = (ObjectTracker) tManager
            .getTracker(ObjectTracker.getClassType());
        if (objectTracker == null)
            return false;
      
        if (mCurrentDataset == null)
            mCurrentDataset = objectTracker.createDataSet();
      
        if (mCurrentDataset == null)
            return false;
      
        if (!mCurrentDataset.load("ObjectRecognition/rigidBodyTarget.xml",
            STORAGE_TYPE.STORAGE_APPRESOURCE))
            return false;
      
        if (!objectTracker.activateDataSet(mCurrentDataset))
            return false;
      
        int numTrackables = mCurrentDataset.getNumTrackables();
        for (int count = 0; count < numTrackables; count++)
        {
            Trackable trackable = mCurrentDataset.getTrackable(count);
            if(isExtendedTrackingActive())
            {
                trackable.startExtendedTracking();
            }
           
            String name = "Current Dataset : " + trackable.getName();
            trackable.setUserData(name);
            Log.d(LOGTAG, "UserData:Set the following user data "
                + (String) trackable.getUserData());
        }
      
        return true;
    }

3. To configure the Object Target Renderer and handle events arising from the Object Tracker:

  • Configuring Object Recognition
  • Handling events from the Object Tracker

In com.vuforia.samples.VuforiaSamples.app.ObjectRecognition;

File ObjectTargetRenderer.java

Line 135

// The render function.
    private void renderFrame()
    {
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
      
        State state = mRenderer.begin();
        mRenderer.drawVideoBackground();
      
        GLES20.glEnable(GLES20.GL_DEPTH_TEST);
      
        // handle face culling, we need to detect if we are using reflection
        // to determine the direction of the culling
        //GLES20.glDisable(GLES20.GL_CULL_FACE);
        GLES20.glEnable(GLES20.GL_BLEND);
        GLES20.glBlendFunc(GLES20.GL_SRC_ALPHA, GLES20.GL_ONE_MINUS_SRC_ALPHA);
      
        if (Renderer.getInstance().getVideoBackgroundConfig().getReflection() == VIDEO_BACKGROUND_REFLECTION.VIDEO_BACKGROUND_REFLECTION_ON)
            GLES20.glFrontFace(GLES20.GL_CCW); // Back camera
           
        // did we find any trackables this frame?
        for (int tIdx = 0; tIdx < state.getNumTrackableResults(); tIdx++)
        {
            TrackableResult result = state.getTrackableResult(tIdx);
            Trackable trackable = result.getTrackable();
            printUserData(trackable);
          
            if (!result.isOfType(ObjectTargetResult.getClassType()))
                continue;
          
            ObjectTarget objectTarget = (ObjectTarget) trackable;
           
            Matrix44F modelViewMatrix_Vuforia = Tool
                .convertPose2GLMatrix(result.getPose());
            float[] modelViewMatrix = modelViewMatrix_Vuforia.getData();
          
            // deal with the modelview and projection matrices
            float[] modelViewProjection = new float[16];
          
            float[] objectSize = objectTarget.getSize().getData();
          
            Matrix.translateM(modelViewMatrix, 0, objectSize[0]/2, objectSize[1]/2,
                objectSize[2]/2);
          
            Matrix.scaleM(modelViewMatrix, 0, objectSize[0],
                objectSize[1], objectSize[2]);
          
            Matrix.multiplyMM(modelViewProjection, 0, vuforiaAppSession
                .getProjectionMatrix().getData(), 0, modelViewMatrix, 0);
          
            // activatrigidBodyTarget.xmle the shader program and bind the vertex/normal/tex coords
            GLES20.glUseProgram(shaderProgramID);
          
            GLES20.glVertexAttribPointer(vertexHandle, 3, GLES20.GL_FLOAT,
                false, 0, mCubeObject.getVertices());
            GLES20.glUniform1f(opacityHandle, 0.3f);
            GLES20.glUniform3f(colorHandle, 0.0f, 0.0f, 0.0f);
              
            GLES20.glEnableVertexAttribArray(vertexHandle);
              
            // pass the model view matrix to the shader
            GLES20.glUniformMatrix4fv(mvpMatrixHandle, 1, false,
                modelViewProjection, 0);
             
            // finally draw the teapot
            GLES20.glDrawElements(GLES20.GL_TRIANGLES,
                mCubeObject.getNumObjectIndex(), GLES20.GL_UNSIGNED_SHORT,
                mCubeObject.getIndices());
              
            // disable the enabled arrays
            GLES20.glDisableVertexAttribArray(vertexHandle);
          
            SampleUtils.checkGLError("Render Frame");
          
        }
      
        GLES20.glDisable(GLES20.GL_DEPTH_TEST);
        GLES20.glDisable(GLES20.GL_BLEND);
      
        mRenderer.end();
    }

To modify the core features sample project so that only the Object Recognition feature is built:

Vuforia Engine also provides a sample app with all features integrated the Core Features Sample . To compile this app so that only the Object Recognition feature is built, delete the other sample names from the mActivities array, as below.


In package com.vuforia.samples.VuforiaSamples.ui.ActivityList;
File: ActivityLauncher.java

// This activity starts activities which demonstrate the Vuforia features
public class ActivityLauncher extends ListActivity
{
  
    private String mActivities[] = {"Object Reco"};

...