Building and Using the File Driver Sample

Overview

The File Driver Sample demonstrates how to implement a Driver which uses the External Camera features that provides frame data to Vuforia Engine. The File Driver sample can be downloaded from the Developer Portal. The sample loads a sequence of pre-recorded camera frames from device storage and feeds the frame data into Vuforia Engine one frame at a time. The package includes two sets of frame sequences. One set with the Stones Image Target and another set with the Astronaut Image Target. Native developers should use the sequence containing the Stones Image Target while Unity developers should use the Astronaut Image Target sequence.

Building the File Driver

Prerequisites

Instructions

  1. Extract the package
  2. To build on run the appropriate command:
  3. Android:

    python build.py android

    UWP:

    python build.py uwp

    iOS:

    Correct the file line of the file “ios.toolchain.cmake” in the directory “cmake” to point to the right location of your Xcode or remove this block completely, so cmake will find it for you.

    set(CONST_DIR_XCODE_DEVELOPER/Applications/Xcode_93.app/Contents/Developer)

    Then run this command:

    python build.py ios

  4. Output files should be placed in the build/bin directory by default. libFileDriver.so for Android, FileDriver.dll for Windows, and FileDriver.framework for iOS.

Using the File Driver

Android

  1. Add libFileDriver.so from build/bin into your Android-app project. libFileDriver.so can be added to your project in one of two ways. Either using Android.mk or via Gradle. Pick the option that fits best with your project.
    1. Using Android.mk

      Add libFileDriver-prebuilt definition to your Android.mk file.:

      include $(CLEAR_VARS)
      LOCAL_MODULE := libFileDriver-prebuilt
      LOCAL_SRC_FILES = [path-in-your-filesystem]/FileDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libFileDriver.so
      include $(PREBUILT_SHARED_LIBRARY)
      

      When defining your local module in the same Android.mk add libFileDriver-prebuilt as a dependency to your LOCAL_SHARED_LIBRARIES:

      LOCAL_SHARED_LIBRARIES := libFileDriver-prebuilt
      
    2. Using Gradle

      This can be done in your app/build.gradle with the following:

      android {
          sourceSets.main {
              jniLibs.srcDirs += '[path-in-your-filesystem]/FileDriver/build/bin/Android/'
          }
      }
      
  2. Add FileDriver.jar from build/bin into your Android-app project. This can be done in your app/build.gradle with the following:
    dependencies {
        implementation files("[path-in-your-filesystem]/FileDriver/build/bin/Android/FileDriver.jar")
    }
    
  3. Add the sample sequence from the data-directory into your Android-app project.This can be done in your app/build.gradle with the following:
  4. android {
        sourceSets.main {
            assets.srcDirs += '[path-in-your-filesystem]/FileDriver/data'
        }
    }
    
  5. Add following call to your source code before calling Vuforia::init();
    1. Java
      Vuforia.setDriverLibrary("libFileDriver.so");
      
    2. C++:

      Vuforia::setDriverLibrary("libFileDriver.so", nullptr);
      

UWP

  1. Add FileDriver.dll from build/bin/uwp into your Visual Studio UWP-app project.
    1. Import the FileDriver.dll to the root of your project. Remember to use a dll that matches your architecture (x86/x64) and build type (Debug/Release) configurations.
    2. Click FileDriver.dll from the project file list and set property "Content" to True.
  2. Add the sample sequence from the data-directory into your Visual Studio UWP-app project.
    1. Import all files from FileDriver/data-directory to the root of your project.
    2. Select all files and set property "Content" to True.
  3. Add following call before calling Vuforia::init();
    Vuforia::setDriverLibrary("FileDriver.dll", nullptr);
    

iOS

  1. In your Xcode application project open the build settings screen and select the General Tab. Find the area listed as “Embedded Binaries” and click on the “+” to add a new binary. From the file selection menu find the iOS File Driver Framework which is in bin/Release-iphones/FileDriver.framework.
Vuforia Image
  1. Once added remove the entry for FileDriver from the “Linked Frameworks and Libraries” section. It should not be listed here and the application should not try to link against it.
Vuforia Image
  1. Add the sample image files into the Xcode project by dragging them onto the Left hand project pane.
Vuforia Image
  1. Add the following call before calling Vuforia::init();
    Vuforia::setDriverLibrary("FileDriver.framework”, nullptr);

Changing the Sequence

Developers looking to load a specific sequence of images, can do so by providing a new path via the FileDriverUserData struct via the second parameter of the setDriverLibrary call. Refer to the example below:

#include 
static FileDriverUserData fdUserData;
fdUserData.sequenceDirectoryAbsolutePath = "/sdcard/sequence/";
bool ret = Vuforia::setDriverLibrary(VUFORIA_DRIVER_FILEDRIVER_LIB, &fdUserData);

Custom Sequences

Developers looking to create their own sequence, should consider the following:

  • The width and height of each frame should remain consistent and should match the information provided intrinsic information provided with each frame.
  • The framerate which frames are provided to Vuforia Engine does not necessarily need to match the rate at which they were captured.
  • The samples are currently using PNG format, but other image formats may be supported dependent on platform - just be sure that the files can be read at the rate which matches the desired framerate. For instance, if the desired framerate is 30 FPS, then the device needs to be able to read 30 files per second.
  • Avoid compression artifacts in the files.
  • Developers are encouraged create/name their file sequences in such a manner which makes it easy to process the sequence in the appropriate order. The sample uses 'filename#####.png' notion, where ##### represents the frame index, padded to 5-digits.

     

Unity

  1. Copy the compiled binaries from build/bin to:
    • Android: Assets/Plugins/Android/
    • UWP: Assets/Plugins/WindowsStoreApps/[x64, x86]
    • iOS: Assets/Plugins/iOS/
Vuforia Image

Be sure the correct plugin platform options are set for the imported libraries. They should match the platform they are intended to be used in as seen above.

  1. Add the sequence to your Unity project. The png images and sequence.txt file can be directly copied into the Assets/StreamingAssets folder to make sure it’s included in the application build:
Vuforia Image
  1. To use the File Driver from unity, the name of the driver library needs to be set before Vuforia Engine is initialized. To do this, check “Delayed initialization” in the Vuforia configuration of your project:
Vuforia Image

The driver library can then be set from any app level script, e.g. in an Awake() method. After that, Vuforia Engine can be initialized:

  VuforiaUnity.SetDriverLibrary("libFileDriver.so"); // Android
  VuforiaRuntime.Instance.InitVuforia();

On Android, you should now see the image sequence being played in a loop instead of the device camera being used. On UWP, the image files stored in Assets/StreamingAssets end up in a different folder of the application build than the file driver library expects, so the path needs to be set via the FileDriverUserData struct:

  struct FileDriverUserData
  {
      public string sequenceDirectoryAbsolutePath;
  };
  
  private static string filePath;
  private static FileDriverUserData userData;
  private static IntPtr userDataPtr = IntPtr.Zero;
  
   void Awake()
  {
    filePath = Application.streamingAssetsPath.Replace("/", "\\"); // get the absolute   path to the StreamingAssets folder
      
      userData = new FileDriverUserData()
      {
          sequenceDirectoryAbsolutePath = filePath
      };
      userDataPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FileDriverUserData)));
      Marshal.StructureToPtr(userData, userDataPtr, false);
      
      VuforiaUnity.SetDriverLibrary("FileDriver.dll", userDataPtr); // UWP
      VuforiaRuntime.Instance.InitVuforia();
  }

 

On iOS, the change required is similar to the other platforms. The only difference is that you need to use “FileDriver.framework” and you don’t need to change absolute path. The full code is listed below:

struct FileDriverUserData
    {
        public string sequenceDirectoryAbsolutePath;
    }

    private static string filePath;
    private static FileDriverUserData userData;
    private static IntPtr userDataPtr = IntPtr.Zero;
    private bool okSetFileDriver = false;

    void Awake()
    {
     
        filePath = Application.streamingAssetsPath; 
   userData = new FileDriverUserData()
        {
            sequenceDirectoryAbsolutePath = filePath
        };

        userDataPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FileDriverUserData)));
        Marshal.StructureToPtr(userData, userDataPtr, false);

        okSetFileDriver = VuforiaUnity.SetDriverLibrary("FileDriver.framework", userDataPtr);
        if (!okSetFileDriver)
        {
            Debug.Log("Failed to set file driver");
        }
        VuforiaRuntime.Instance.InitVuforia();
    } 

 

On iOS, in addition, the Unity-iPhone.xcodeproj should be modified as described in the section above on iOS to include the framework “FileDriver.framework” as an embedded framework. It is important to remove it from the list of linked libraries (see Step 2 from the section on iOS).

Learn More