Building and Using the File Driver Sample

Overview

The File Driver Sample demonstrates how to implement a driver which uses the External Camera and the External Positional Device Tracker APIs to feed Vuforia Engine camera frame and device pose, respectively.

The File Driver sample can be downloaded from the Developer Portal. From device storage, the sample loads a sequence of pre-recorded camera frames and device poses. The pose, followed by the corresponding frame data, are then fed into Vuforia Engine. The package includes two sets of frame and pose sequences: Stones Image Target and Astronaut Image Target. Native developers should use the Stones Image Target while Unity developers should use the Astronaut Image Target sequence.

The FileDriver contains a sample sequence (XML) file which contains camera frame and pose information. For more information, refer to the Format of Camera and Pose XML Sequence File article.

 

Building the File Driver

Prerequisites

Instructions

  1. Extract the File Driver package and the Vuforia SDK package.
  2. Run the appropriate build command:
    Note: The -vh  or -vf  command line options may be omitted if the driver is put in the SDK's samples directory.
  3. Android:

    python build.py android -vh [path to Vuforia SDK]/build/include

    UWP:

    python build.py uwp -vh [path to Vuforia SDK]/build/include

    iOS:

    Correct the file line of the ios.toolchain.cmake file, located in the cmake directory to point to the correct Xcode location. Alternatively, remove this block completely so cmake will find it for you.

    set(CONST_DIR_XCODE_DEVELOPER/Applications/Xcode_93.app/Contents/Developer)

    Then run the following command (the iOS option is -vf and the value is the folder containing Vuforia.framework):

    python build.py ios -vf [path to Vuforia SDK]/build

  4. Output files should be placed in the build/bin directory by default: libFileDriver.so for Android, FileDriver.dll for Windows, and FileDriver.framework for iOS.

Using the File Driver

Android

  1. Add libFileDriver.so from build/bin to your Android app project by using either Android.mk or Gradle. Select the option that best fits your project:
    • Android.mk:

      Add libFileDriver prebuilt definition to your Android.mk file:

      include $(CLEAR_VARS)
      LOCAL_MODULE := libFileDriver-prebuilt
      LOCAL_SRC_FILES = [path-in-your-filesystem]/FileDriver/build/bin/Android/Release/$(TARGET_ARCH_ABI)/libFileDriver.so   
      include $(PREBUILT_SHARED_LIBRARY)
      

      When defining your local module in the same Android.mk add libFileDriver prebuilt as a dependency to your LOCAL_SHARED_LIBRARIES:

      LOCAL_SHARED_LIBRARIES := libFileDriver-prebuilt
      
    • Gradle:

      Use the following code in your app/build.gradle:

      android {
          sourceSets.main {
              jniLibs.srcDirs += '[path-in-your-filesystem]/FileDriver/build/bin/Android/'
          }
      }
      
  2. Add FileDriver.jar from build/bin to your Android-app project. Use the following code in your app/build.gradle:
    dependencies {
        implementation files("[path-in-your-filesystem]/FileDriver/build/bin/Android/Release/FileDriver.jar")
    }
    
  3. Add the sample sequence from the data directory to your Android-app project. Use the following code in your app/build.gradle:
  4. android {
        sourceSets.main {
            assets.srcDirs += '[path-in-your-filesystem]/FileDriver/data/stones'
        }
    }
    
  5. Before calling Vuforia::init(), add the following call to your source code:
     
Java Vuforia.setDriverLibrary("libFileDriver.so");
C++ Vuforia::setDriverLibrary("libFileDriver.so", nullptr);

UWP

  1. Add FileDriver.dll from build/bin/uwp into your Visual Studio UWP app project.
    1. Import the FileDriver.dll to the root of your project. Remember to use a dll that matches your architecture (x86/x64) and build type (Debug/Release) configurations.
    2. Click FileDriver.dll from the project file list and set property Content to True.
  2. Add a sample sequence from the data directory into your Visual Studio UWP app project. For example, to use the “stones” sequence:
    1. Import FileDriver/data/stones/FileDriverSequence.xml into your project.
    2. Import FileDriver/data/stones/Camera/*.png into your project.
      Note: The .png files sit alongside the .xml file when the app is built. The files are not in a Camera folder.
    3. Select all files and set property Content to True.
  3. Add the following call before Vuforia::init():
    Vuforia::setDriverLibrary("FileDriver.dll", nullptr);
    

iOS

  1. In your Xcode project, open the build settings screen and select the General tab. 
  2. In the Embedded Binaries section, click the + icon to add a new binary. 
  3. From the file selection menu, locate the iOS File Driver Framework in bin/Release-iphoneos/FileDriver.framework
Vuforia Image
  1. In the Linked Frameworks and Libraries section, remove the FileDriver entry. The entry should not be listed in this section and the application should not try to link against it.
Vuforia Image
  1. Drag the FileDriverSequence.xml file and the camera image files into the left hand project pane. Package the .png files into a Camera folder in the app bundle with the folder structure as seen below.
Vuforia Image
  1. Add the following call before Vuforia::init():
    Vuforia::setDriverLibrary("FileDriver.framework”, nullptr);

Changing the Sequence

To load a specific sequence of pose and images, provide a new path using the FileDriverUserData struct and the second parameter of the setDriverLibrary call. Refer to the example below:

#include 
static FileDriverUserData fdUserData;
fdUserData.sequenceDirectoryAbsolutePath = "/sdcard/sequence/";
bool ret = Vuforia::setDriverLibrary(VUFORIA_DRIVER_FILEDRIVER_LIB, &fdUserData);

Custom Sequences

Developers looking to create their own sequence, should consider the following:

  • A top level XML file groups a device pose and a matching camera frame under one <Element> tag. This XML file also contains global attributes, such as total number of camera frames, width,  and height, which apply to the entire sequence. For more information, refer to the Format of Camera and Pose XML Sequence File article.
  • The width and height of each frame should remain consistent and match the intrinsic information provided with each frame.
  • The framerate at which frames are provided to Vuforia Engine does not necessarily need to match the rate at which they were captured.
  • The samples currently use PNG format, but other image formats may be supported depending on the platform. Ensure that the files can be read at the rate which matches the desired framerate. For instance, if the desired framerate is 30 FPS, then the device needs to be able to read 30 files per second.
  • Avoid compression artifacts in the files.
  • Developers need to create/name their file sequences to easily process the sequence in the appropriate order. The sample uses 'filename#####.png' notation, where ##### represents the frame index, padded to 5-digits.

     

Unity

  1. Copy the compiled binaries from build/bin to:
    • Android: Assets/Plugins/Android/
    • UWP: Assets/Plugins/WindowsStoreApps/[x64, x86]
    • iOS: Assets/Plugins/iOS/

Note: Ensure the correct plugin platform options are set for the imported libraries. The selection options should match the intended platform (see below).

Vuforia Image

 

  1. Add the sequence to your Unity project. Copy the Camera folder and FileDriverSequence.xml file into the Assets/StreamingAssets folder. This ensures it's included in the application build:
Vuforia Image
  1. In the VuforiaConfiguration window, select Delayed initialization. This sets the driver library name before Vuforia Engine is initialized in order to use the file driver from Unity.
Vuforia Image

The driver library can then be set from any app level script, e.g. in an Awake() method. After that, Vuforia Engine can be initialized:

  VuforiaUnity.SetDriverLibrary("libFileDriver.so"); // Android
  VuforiaRuntime.Instance.InitVuforia();

On Android, the image sequence should play in a loop instead of the device camera being used. On UWP, the image files stored in Assets/StreamingAssets end up in a different application build folder than the file driver library expects. As a result, the path needs to be set via the FileDriverUserData struct:

  struct FileDriverUserData
  {
      public string sequenceDirectoryAbsolutePath;
  };
  
  private static string filePath;
  private static FileDriverUserData userData;
  private static IntPtr userDataPtr = IntPtr.Zero;
  
   void Awake()
  {
    filePath = Application.streamingAssetsPath.Replace("/", "\\"); // get the absolute   path to the StreamingAssets folder
      
      userData = new FileDriverUserData()
      {
          sequenceDirectoryAbsolutePath = filePath
      };
      userDataPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FileDriverUserData)));
      Marshal.StructureToPtr(userData, userDataPtr, false);
      
      VuforiaUnity.SetDriverLibrary("FileDriver.dll", userDataPtr); // UWP
      VuforiaRuntime.Instance.InitVuforia();
  }

 

On iOS, the required changes are similar to the other platforms. However, you need to use FileDriver.framework and changing the absolute path is not required. The Unity-iPhone.xcodeproj should be modified to include the FileDriver.framework as an embedded framework. Remember to remove it from the list of linked libraries (refer to the step relating to linked libraries in the Using the File Driver > iOs section). The following is the full code:

struct FileDriverUserData
    {
        public string sequenceDirectoryAbsolutePath;
    }

    private static string filePath;
    private static FileDriverUserData userData;
    private static IntPtr userDataPtr = IntPtr.Zero;
    private bool okSetFileDriver = false;

    void Awake()
    {
     
        filePath = Application.streamingAssetsPath; 
   userData = new FileDriverUserData()
        {
            sequenceDirectoryAbsolutePath = filePath
        };

        userDataPtr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FileDriverUserData)));
        Marshal.StructureToPtr(userData, userDataPtr, false);

        okSetFileDriver = VuforiaUnity.SetDriverLibrary("FileDriver.framework", userDataPtr);
        if (!okSetFileDriver)
        {
            Debug.Log("Failed to set file driver");
        }
        VuforiaRuntime.Instance.InitVuforia();
    }