Developers using external systems and looking to take advantage of Vuforia Engine can write their own driver and provide camera data to Vuforia Engine for tracking.
A driver is a native C/C++ dynamic library (.so or .dll file) that implements specific calls. During runtime, Vuforia Engine loads the library and uses it as a source for image data, and optionally, device poses and anchors. See Creating a Custom Driver with a Device Tracker for information on providing device poses via a device tracker.
To get started, download the Native SDK package appropriate for your platform from the Developer Portal. We are mainly interested in the Driver.h file. This file defines several functions and classes which must be extended by the custom driver. For more information, refer to the comments in Driver.h file. The following diagram illustrates the high-level integration of custom drivers.
Vuforia Engine requires data input from the Vuforia Driver in the following data formats.
- Frames from the camera must be in formats YUYV, NV12, or NV21.
Some Vuforia Engine features require 6DOF device poses in addition to camera images. See our guide on Creating a Custom Driver with a Device Tracker.
vuforiaDriver_getAPIVersion()– Returns a number containing the Driver Framework version that the library has been built against and conforms to. This constant is defined in Driver.h as
vuforiaDriver_getLibraryVersion()– Returns a string containing the version describing the library version. The driver developer selects a version value that is non-empty and generally under 50 characters long (e.g.,
vuforiaDriver_init()– Constructs and returns an instance of the VuforiaDriver object which defines the custom driver. The memory and the lifetime of this object is owned by the library. Object is expected to remain valid until
vuforiaDriver_deinit()– Destructs the specified instance of the VuforiaDriver object.
createExternalCamera()– Constructs a new ExternalCamera object. The memory and lifetime of this object is owned by the library. Object should remain valid until
destroyExternalCamera()– Destructs the specified instance of the ExternalCamera object
createExternalPositionalDeviceTracker()- Constructs a new ExternalPositionalDeviceTracker object. The memory and lifetime of this object is owned by the library. Object should remain valid until
destroyExternalPositionalDeviceTracker()- Destructs the specified instance of the ExternalPositonalDeviceTracker object.
getCapabilities()- Queries the driver for supported capability. The driver can support camera frames with corresponding device poses or camera frames only. There are two different capabilities supported by Vuforia Driver. Use Vuforia::Driver::Capability::CAMERA_IMAGE for camera only sequence. For camera and pose sequence, use (Vuforia::Driver::Capability::CAMERA_IMAGE | Vuforia::Driver::Capability::CAMERA_POSE)
open()– Allows the Driver to allocate any resources needed. After calling
open()the supported camera modes must be available.
getNumSupportedCameraModes()– Returns the number of supported camera modes.
getSupportedCameraMode(index)– Returns a specific supported camera mode by index.
start(cameraMode, CameraCallback)– Starts providing camera frames to the specified callback class for the specified cameraMode.
stop()– Stops producing new camera frames to the callback.
close()– Closes the camera and release any resources obtained from
The ExternalCamera class also provides access to the following information through various functions (e.g., gets/sets). For more information, refer to the comments in Driver.h:
The driver provides camera frames to Vuforia Engine through a callback class. The driver pushes
CameraFrame structure to Vuforia Engine through the
CameraCallback::onNewCameraFrame function. Each
CameraFrame structure points to a buffer containing the actual frame data. Frame data also includes an accurate timestamp, exposure time and frame width/height. Along with the frame data, camera details must also be provided through the intrinsics parameter of type
CameraIntrinsics. For more details on the
CameraIntrinsics structure, refer to the External Camera Calibration article. The information provided by the
CameraFrame structure must be accurate to achieve optimal performance.
open()– Allows the Driver to allocate any needed resources. After calling
open()the supported positional device tracker is available
start(PoseCallback)– Start providing pose data to the specified callback class
stop()– Stop producing new pose data to the callback
close()– Close the positional device tracker and release any resources obtained from
The driver provides pose data to Vuforia Engine through a callback class. The driver pushes
Pose structure to Vuforia Engine through the
PoseCallback::onNewPose function. Each
Pose structure includes pose data (e.g., an accurate timestamp and translation (vector) along x, y and z axes followed by a 3*3 rotation matrix for the pose). For a pose and camera frame to be associated together, the timestamp in
CameraFrame structures should match exactly. Driver implementation should first feed pose and then the matching camera frame. Pose is optional and in such scenarios, driver implementation can just feed the camera frames.
Vuforia Engine manages the lifetime of the driver by making function calls in the following order:
vuforiaDriver_init()– Creates a new instance of the VuforiaDriver class
VuforiaDriver::getCapabilities()– Queries driver capability.
VuforiaDriver::createExternalPositionalDeviceTracker()– Called if driver capability includes Vuforia::Driver::Capability::CAMERA_POSE to create a new instance of the ExternalPositionalDeviceTracker class.
ExternalPositionalDeviceTracker::open()– Allocates any resources required by the external system. This call is made only if capability includes Vuforia::Driver::Capability::CAMERA_POSE
VuforiaDriver::createExternalCamera()– Called if driver capability includes Vuforia::Driver::Capability::CAMERA_IMAGE to create a new instance of the ExternalCamera class.
ExternalCamera::open()– Allocates any resources required by the external system. This call is made only if capability includes Vuforia::Driver::Capability::CAMERA_IMAGE
ExternalPositionalDeviceTracker::start()– Signals the external system to start providing
Posestructures through the
ExternalCamera::start()– Signals the external system to start providing
CameraFramestructures through the
- Vuforia Engine now consumes
CameraFramestructures provided through the
ExternalPositionalDeviceTracker::stop()– Signals the external system to stop providing pose
ExternalPositionalDeviceTracker:close()– Deallocates any resources that the external system was using
ExternalCamera::stop()– Signals the external system to stop providing camera frames
ExternalCamera:close()– Deallocates any resources that the external system was using
VuforiaDriver:: destroyExternalPositionalDeviceTracker()– Destructs the ExternalPositionalDeviceTracker instance
VuforiaDriver:: destroyExternalCamera()– Destructs the ExternalCamera instance
vuforiaDriver_deinit()– Destructs the VuforiaDriver instance
Using a Driver in Vuforia Engine
After the driver is compiled, it may be shared with other developers who want to use the external system. To use a driver with Vuforia Engine, the application developer must make the following call to include the Driver to the Vuforia Engine configuration:
VuDriverConfig driverConfig = vuDriverConfigDefault(); driverConfig.driverName = "libFileDriver.so"; driverConfig.userData = nullptr; vuEngineConfigSetAddDriverConfig (configSet, &driverConfig)
The call must be made before calling
Driver configurations added to the Engine is set for the lifetime of the Engine instance. If you wish to run operations without the driver, the Engine should first be destroyed and then re-created.