Working with Vuforia and Unity

How To Dynamically Add Content to Targets in Unity

This article describes how to augment an image target with a custom 3D model that is instantiated at run time, upon target detection, using the Vuforia.

Follow these steps:

  1. Open or create a Unity project, add an ImageTarget GameObject (menu: GameObject> Vuforia> Image)
  2. Assume that at run time we want to dynamically attach a custom 3D model to the Astronaut image target.
  3. In the Project View, under the Assets folder, create a subfolder called Prefabs, if you don’t already have that subfolder.
  4. Add a prefab object to the Prefabs folder. There are many ways to create custom prefabs to represent 3D objects. For instance, you could:
    1. Create a simple Cube game-object in your scene view and then drag it from the scene view into the Prefabs folder in the Project view,
      OR
    2. Import a 3D model in a format supported by Unity (such as FBX, OBJ, DAE, or 3DS). Refer to the Unity website for details on how to create Prefabs from 3D models from various file formats.
  5.  Create a C# script, call it MyPrefabInstantiator, for instance, and attach it to the Astronaut image target object.
  6. Put the following code into the script and then save the script:
using UnityEngine;
using Vuforia;
using System.Collections;
public class MyPrefabInstantiator : MonoBehaviour, ITrackableEventHandler {
  private TrackableBehaviour mTrackableBehaviour;
  public Transform myModelPrefab;
  // Use this for initialization
  void Start ()
  {
    mTrackableBehaviour = GetComponent<TrackableBehaviour>();
    if (mTrackableBehaviour) {
      mTrackableBehaviour.RegisterTrackableEventHandler(this);
    }
  }
  // Update is called once per frame
  void Update ()
  {
  }
  public void OnTrackableStateChanged(
    TrackableBehaviour.Status previousStatus,
    TrackableBehaviour.Status newStatus)
  { 
    if (newStatus == TrackableBehaviour.Status.DETECTED ||
        newStatus == TrackableBehaviour.Status.TRACKED ||
        newStatus == TrackableBehaviour.Status.EXTENDED_TRACKED)
    {
      OnTrackingFound();
    }
  } 
  private void OnTrackingFound()
  {
    if (myModelPrefab != null)
    {
      Transform myModelTrf = GameObject.Instantiate(myModelPrefab) as Transform;
      myModelTrf.parent = mTrackableBehaviour.transform;
      myModelTrf.localPosition = new Vector3(0f, 0f, 0f);
      myModelTrf.localRotation = Quaternion.identity;
      myModelTrf.localScale = new Vector3(0.0005f, 0.0005f, 0.0005f);
      myModelTrf.gameObject.active = true;
    }
  }
}
  • As you can see from the code, the MyPrefabInstantiator class implements the ITrackableEventHandler interface, so that it is notified when the target is detected.
    • At that point, the Prefab 3D model gets instantiated and gets attached under the image target.
    • The position and scale are adjusted so that the model can augment the target.
    • Adjust the values of the position, rotation, and scale to fit your needs for the specific model.
  • The script also contains a public Transform variable called MyModelPrefab.
    • If you now go back to the Unity scene view and you click on the Astronaut image target, in the inspector you will see that the MyPrefabInstantiator script now exposes a field called My Model Prefab.
    • Select your prefab in the Prefabs folder (in the project view) and drag it onto the My Prefab Model field in the MyPrefabInstantiator script component.
  • Save the project, and then build and run the app. As soon as the Astronaut target is detected, your prefab 3D model should appear on top of it.
  • If the model size is either too small or too big, you can go to the script code and adjust the localScale values.

 

How To Dynamically Swap a 3D Model of a Target in Unity

This section explains how, in Unity3D, you can swap the 3D model of an Image Target (or other trackable) with a different 3D model at run time.


Follow these steps:

  1. Open or create a Unity project, add an ImageTarget GameObject (menu: GameObject> Vuforia> Image)
  2. Add a model as a child of the ImageTarget (a Sphere is a good option)
  3. Create a C# script, name it ModelSwapper, for instance. The purpose of this script is to show a button overlaid on top of the AR view that lets you swap one model with another 3D model. This example swaps in a simple cube.
  4. Insert the following code into the script:
using UnityEngine;
using Vuforia;
using System.Collections;
public class ModelSwapper : MonoBehaviour {
    public TrackableBehaviour theTrackable;
    private bool mSwapModel = false;
    // Use this for initialization
    void Start () {
        if (theTrackable == null)
        {
            Debug.Log ("Warning: Trackable not set !!");
        }
    }
    // Update is called once per frame
    void Update () {
        if (mSwapModel && theTrackable != null) {
            SwapModel();
            mSwapModel = false;
        }
    }
    void OnGUI() {
        if (GUI.Button (new Rect(50,50,120,40), "Swap Model")) {
            mSwapModel = true;
        }
    }
    private void SwapModel() {
        GameObject trackableGameObject = theTrackable.gameObject;
        //disable any pre-existing augmentation
        for (int i = 0; i < trackableGameObject.transform.GetChildCount(); i++)
        {
            Transform child = trackableGameObject.transform.GetChild(i);
            child.gameObject.active = false;
        }
        // Create a simple cube object
        GameObject cube = GameObject.CreatePrimitive(PrimitiveType.Cube);
        // Re-parent the cube as child of the trackable gameObject
        cube.transform.parent = theTrackable.transform;
        // Adjust the position and scale
        // so that it fits nicely on the target
        cube.transform.localPosition = new Vector3(0,0.2f,0);
        cube.transform.localRotation = Quaternion.identity;
        cube.transform.localScale = new Vector3(0.1f, 0.1f, 0.1f);
        // Make sure it is active
        cube.active = true;
    }
}
  1. Attach the script to the ARCamera (menu: GameObject> Vuforia> AR Camera) object in your scene view,
  2. In the inspector view, verify that the MyModelSwapper script exposes a field called The Trackable.
  3. Drag one of the Image Targets from your scene view into the The Trackable field of your script.
  4. Save, build, and run the app. A button labeled Swap model should appear. After you click it, the original model of the target should be replaced with a simple cube.

How To Drag an Augmentation using Touch Input

This article provides sample code of a script that lets users drag an AR object on the screen with their finger.

C# Script

Create a new *.cs script with the following code. Place this code on the AR Camera to enable users to touch and drag an object. The target object must have a mesh collider.

using UnityEngine;
using System.Collections;
public class DragObject : MonoBehaviour {
   public GUIText message = null;
   private Transform pickedObject = null;
   private Vector3 lastPlanePoint;
   // Use this for initialization
   void Start () {
   }
   // Update is called once per frame
   void Update () {
        Plane targetPlane = new Plane(transform.up, transform.position);
        //message.text = "";
        foreach (Touch touch in Input.touches) {
            //Gets the ray at position where the screen is touched
            Ray ray = Camera.main.ScreenPointToRay(touch.position);
            //Gets the position of ray along plane
            float dist = 0.0f;
            //Intersects ray with the plane. Sets dist to distance along the ray where intersects
            targetPlane.Raycast(ray, out dist);
            //Returns point dist along the ray.
            Vector3 planePoint = ray.GetPoint(dist);
            //Debug.Log("Point=" + planePoint);
            //True if finger touch began. If ray intersects collider, set pickedObject to transform 
                of collider object
            if (touch.phase == TouchPhase.Began) {
                 //Struct used to get info back from a raycast
                 RaycastHit hit = new RaycastHit();
                 if (Physics.Raycast(ray, out hit, 1000)) { //True when Ray intersects colider. 
                 If true, hit contains additional info about where collider was hit
                     pickedObject = hit.transform;
                     lastPlanePoint = planePoint;
                 } else {
                     pickedObject = null;
                 }
            //Move Object when finger moves after object selected.
            } else if (touch.phase == TouchPhase.Moved) {
                 if (pickedObject != null) {
                     pickedObject.position += planePoint - lastPlanePoint;
                     lastPlanePoint = planePoint;
                 }
            //Set pickedObject to null after touch ends.
            } else if (touch.phase == TouchPhase.Ended) {
                 pickedObject = null;
            }
        }
   }
}

How To Load a DataSet from the SD Card in Unity

Unity - Load DataSet from SD Card

The Vuforia SDK lets you load datasets from various locations. If your device has an SD card, you can store your dataset (.XML and .DAT files) in the external storage (SD Card) of your device. Then you can tell Vuforia to load the dataset from the SD card.

Code to save dataset to SD Card

The following code snippet shows how this approach can be achieved in a C# script. In the following example, assume that the dataset is called Astronaut. Assume also that the relevant files ( Astronaut.xml and Astronaut.dat) have been copied into the SD card path /mnt/sdcard/.

using UnityEngine;
using Vuforia;
using System.Collections;
using System.Collections.Generic;

public class SDCardDataSetLoader : MonoBehaviour {
     private bool mLoaded = false;
     private DataSet mDataset = null;
    // Update is called once per frame
    void Update () {
          if (VuforiaRuntimeUtilities.IsVuforiaEnabled() && !mLoaded) {
          string externalPath = "/mnt/sdcard/Astronaut.xml";

          if (mDataset == null) {
          // First, create the dataset
          ObjectTracker tracker = TrackerManager.Instance.GetTracker<ObjectTracker>();
          mDataset = tracker.CreateDataSet();
          }
         
         if (mDataset.Load(externalPath, VuforiaUnity.StorageType.STORAGE_ABSOLUTE)) {
            mLoaded = true;
         }
         else {
           Debug.LogError ("Failed to load dataset!");
         }
}
}
}

 

SDCard path

The path identifying the SD card location in the above code is /mnt/sdcard/. This path should be accepted on most devices. However, in case that path does not work for you, other alternate paths for the SD card are also located at:

  • /sdcard
  • /storage/sdcard0

Use the Application.persistentDataPath

Another option for an external storage path is to rely on the Unity abstraction, which is accessible using the following code:

externalPath = Application.persistentDataPath;

Note that the above path points to an external (SD Card) location only if you have enabled External Write Access in your Player Settings. That means that you have set Write Access = External (SDCard), as opposed to using the default setting, which is Write Access =Internal Only.

In the case of external write access, the persistent datapath looks like this:

/storage/sdcard0/Android/data/ + <package-name> + /files

For example, if your package name is com.myorg.myapps, you should see:

/storage/sdcard0/Android/data/com.myorg.myapps/files.

So, based on the above information, you should store your dataset files in the above directory. For example, for the Astronaut dataset, you should copy the dataset files into a subdirectory in /files/ or into the following directory:

  /storage/sdcard0/Android/data/com.myorg.myapps/files/Astronaut.xml

  /storage/sdcard0/Android/data/com.myorg.myapps/files/Astronaut.dat

 


How To Use AssetBundles to Package Augmentations


This section describes how to augment an image target with a custom 3D model that was instantiated from an AssetBundle at run time, upon target detection, using the Vuforia Unity extension.

For more information about Unity Asset Bundles, see the AssetBundle guides on the Unity website:

Note: For accurate and up-to-date information, always consult the Unity website (see links above), since the license requirements can change over time.

  1. Open or create a Unity project, add an ImageTarget GameObject (menu: GameObject> Vuforia> Image)
  2. Assume that at run time we want to dynamically attach a custom 3D model to the astronaut image target. Select "VuforiaMars_Images" Database and "Astronaut" Image Target in the ImageTarget Behaviour component
  3. Create a C# script to define a class that implements the ITrackableEventHandler interface. Name the script: AssetBundleAugmenter
  4. Attach the script to your astronaut Image Target. This action allows you to load a model from an AssetBundle at a specific URL and attach it to your astronaut Image Target at run time.

    The following example shows what the code can look like:
public class AssetBundleAugmenter : MonoBehaviour, ITrackableEventHandler {
  public string AssetName;
  public int Version;
  private GameObject mBundleInstance = null;
  private TrackableBehaviour mTrackableBehaviour;
  private bool mAttached = false;
  void Start() {
    StartCoroutine(DownloadAndCache());
    mTrackableBehaviour = GetComponent<TrackableBehaviour>();
    if (mTrackableBehaviour) {
      mTrackableBehaviour.RegisterTrackableEventHandler(this);
    }
  }
  // Update is called once per frame
  IEnumerator DownloadAndCache() {
    while(!Caching.ready)
    yield return null;
    // example URL of file on PC filesystem (Windows)
    // string bundleURL = "file:///D:/Unity/AssetBundles/MyAssetBundle.unity3d";
    // example URL of file on Android device SD-card
    string bundleURL = "file:///mnt/sdcard/AndroidCube.unity3d";
    using (WWW www = WWW .LoadFromCacheOrDownload(bundleURL, Version)) {
    yield return www;
    if (www .error != null)
      throw new UnityException("WWW Download had an error: " + www .error);
    AssetBundle bundle = www .assetBundle;
    if (AssetName == "") {
      mBundleInstance = Instantiate (bundle.mainAsset) as GameObject;
    }
    else {
      mBundleInstance = Instantiate(bundle.LoadAsset(AssetName)) as GameObject;
    }
  }
}
public void OnTrackableStateChanged(
    TrackableBehaviour.Status previousStatus,
    TrackableBehaviour.Status newStatus) {
  if (newStatus == TrackableBehaviour.Status.DETECTED ||
      newStatus == TrackableBehaviour.Status.TRACKED ||
      newStatus == TrackableBehaviour.Status.EXTENDED_TRACKED) {
    if (!mAttached && mBundleInstance) {
      // if bundle has been loaded, let's attach it to this trackable
      mBundleInstance.transform.parent = this.transform;
      mBundleInstance.transform.localScale = new Vector3(0.3f, 0.3f, 0.3f);
      mBundleInstance.transform.localPosition = new Vector3(0.0f, 0.15f, 0.0f);
      mBundleInstance.transform.gameObject.SetActive(true);
      mAttached = true;
    }
  }
}
}



NOTES:

  • The above code example shows how to instantiate an AssetBundle that was previously stored on the device external storage (SD card). This example assumes that you previously uploaded the AssetBundle file (for example, MyAndroidAssetBundle.unity3d) to the SD card of your device.
  • A more typical and useful approach is to store your AssetBundle on a server and then point the AssetBundle URL to that server location, such as http://my.org/assetbundles/MyAndroidAssetBundle.unity3d.
  • AssetBundles can be built for different target platforms, such as Windows, Android, and iOS. Therefore, if you build the AssetBundle by yourself, be sure to select the appropriate BuildTarget option.
  • AssetBundles built with a previous Unity version may not be compatible with newer Unity versions. For more details, see: