Building and Using the UVC Camera Driver Sample

Building and Using the UVC Camera Driver Sample

Overview

The UVC (USB Video Class) Camera Driver Sample demonstrates how to access a USB camera from within Android. The UVC Camera Driver Sample is an implementation of an External Camera using the Vuforia Engine Driver Framework and has been designed to work on both handheld devices and Qualcomm’s Dragonboard 410C . You can download the UVC Camera Driver Sample from the Vuforia Developer Portal.

The sample plugin uses an open source UVCCamera library to access camera frame data from USB-cameras. This library uses modified versions of libusb and libuvc to handle the USB-cameras. Unmodified versions of libuvc and libub do not work directly on Android because the USB-device handles can't be obtained in the same way as on a standard desktop Linux-system. So Android's USBManager is needed to obtain a device handle and then inject it into libusb through libuvc. The UVCCamera library that this sample uses has implemented the device handle injection.

The current supported cameras profiles are:

  • Logitech HD Webcam c310
  • Microsoft LifeCam Studio

The UVC Camera Driver uses the Java-based Android USBManager API to access camera data. As the Driver needs to be implemented in C++, a pointer to the current JVM-instance needs to be injected through the Vuforia Engine stack into the Driver using the userData pointer passed in with the Vuforia::setExternalProviderLibrary() call. Then inside the Driver a JNI (Java Native Interface) is used to call the USBManager using the injected JVM-pointer. The Android USBManager is used to obtain a device handle for the attached webcam, which is then passed to libuvc and libusb.

Minimum Requirements

The UVC Camera Driver requires phones, tablets or the Dragonboard running Android 6 or later.

Building the UVC Camera Driver

Prerequisites

  • Operating System: Refer to Vuforia Supported Version for supported systems
  • Python 2 or later
  • Bash (required on Windows https://git-scm.com/downloads)
  • CMake 3.6 or greater
  • Ninja build system (https://ninja-build.org)
  • Android NDK 13b
  • git
  • Android SDK with API level 22 support
  • libuvc, libusb, libjpg-turbo

Build the Dependencies

  1. Download Vuforia Engine for Android from the Vuforia Engine Developer Portal
  2. Download the UVCDriver Sample from the Vuforia Engine Developer Portal
  3. Extract the Vuforia Engine for Android package
  4. Extract the UVC Driver Sample package to the VuforiaSDK/Samples directory. The UVCDriver directory should be located in VuforiaSDK/samples/UVCDriver/
  5. Clone the https://github.com/ptc-shunt/UVCCamera repository to VuforiaSDK/Samples/UVCDriver directory. The UVCCamera directory should be located in VuforiaSDK/samples/UVCDriver/UVCCamera
  6. Open terminal and change directory to the jni-build directory: UVCCamera/libuvccamera/src/main/jni
  7. Run ndk-build from NDK 13b:
  8. [path-to-ndk-on-your-machine]/ndk-build (ndk-build.cmd on Windows)

Compile UVC Camera Driver

These instructions assume that the “Compile Dependencies” section above was followed.

  1. Go to the VuforiaSDK/samples/UVCDriver directory
  2. Run python build.py (For more build options, use python build.py --help)
  3. This should compile the plugin and copy all the dependencies into UVCDriver/build/bin. NOTE: It also creates an intermediate directory to ../../build_android.

Using UVC Camera Driver

  1. Add libUVCDriver.so, libuvc, libusb and libjpeg-turbo from build/bin into your Android-app project.

    libUVCDriver.so can be added to an app by two ways depending on your build system: Using Android.mk or using Gradle. Please note that you need to substitute '[path-in-your-filesystem]' with the correct path.

    1. Using Android.mk (e.g. ImageTargetsNative sample app)

      Add library definitions to your Android.mk:

      include $(CLEAR_VARS)
      LOCAL_MODULE := libUVCDriver-prebuilt
      LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libUVCDriver.so
      include $(PREBUILT_SHARED_LIBRARY)
      
      include $(CLEAR_VARS)
      LOCAL_MODULE := libuvc-prebuilt
      LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libuvc.so
      include $(PREBUILT_SHARED_LIBRARY)
      
      include $(CLEAR_VARS)
      LOCAL_MODULE := libusb-prebuilt
      LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libusb100.so
      include $(PREBUILT_SHARED_LIBRARY)
      
      include $(CLEAR_VARS)
      LOCAL_MODULE := libjpeg-turbo-prebuilt
      LOCAL_SRC_FILES = [path-in-your-filesystem]/UVCDriver/build/bin/Android/$(TARGET_ARCH_ABI)/libjpeg-turbo1500.so
      include $(PREBUILT_SHARED_LIBRARY)
      

      When defining your local module in the same Android.mk add prebuilt libraries as dependencies to your LOCAL_SHARED_LIBRARIES:

      LOCAL_SHARED_LIBRARIES := Vuforia-prebuilt libUVCDriver-prebuilt libuvc-prebuilt libusb-prebuilt libjpeg-turbo-prebuilt
      
    2. Using Gradle (e.g. VuforiaSamples sample app)

      This can be done in your app/build.gradle with the following:

      android {
          sourceSets.main {
              jniLibs.srcDirs += '[path-in-your-filesystem]/UVCDriver/build/bin/Android/'
          }
      }
      
  2. Add UVCDriver.jar from build/bin into your Android-app project.

    This can be done in your app/build.gradle with the following:

    dependencies {
        implementation files("[path-in-your-filesystem]/UVCDriver/build/bin/Android/UVCDriver.jar")
    }
    
  3. Add following call to your source code before calling Vuforia::init();
    1. Java:
      Vuforia.setDriverLibrary("libUVCDriver.so");
      
    2. C++:
      Vuforia::setDriverLibrary("libUVCDriver.so", nullptr);
      

Creating a new Camera Profile

Calibration Information

To add additional camera profiles to the UVC Camera Driver, please follow the steps outlined in External Camera Calibration Instructions. Once the calibration data has been produced, create a new CameraDevice-node in the ExternalCameraCalibration.xml file that is in the UVCDriver/src/main/assets directory. A sample node is shown below.

<cameradevice>
    <calibration
      size="640 480"
      principal_point="315.558 227.668"
      focal_length="898.57 813.503"
      distortion_coefficients="-0.043780 0.226497 0.004928 0.000606 0 0 0 0">
</calibration></cameradevice>

Device Information

In addition to the calibration data, the following camera information must also be provided:

  • VendoriID, integer in hexadecimal format
  • ProductID, integer in hexadecimal format

Both of these values can be obtained from a desktop operating system such as Windows or Mac.

Windows

  1. Go to “Control panel” -> “Device manager”
  2. Find your webcam. It will be under “Cameras” or “Imaging Devices”
  3. Right click on your webcam and select “Properties”
  4. Go to the “Details”-page and select “Hardware Ids” from the dropdown menu
Vuforia Image

Mac:

  1. Click the “Apple” Icon on top-left corner of the screen.
  2. Select “About this Mac” -> “System Report”
  3. From the Hardware-list select the Camera and find the correct camera. See Figure 9.
Vuforia Image

Once the device information is obtained, they must be added to the XML file in hexadecimal format. If the values are not already in hexadecimal, they must be converted before updating the XML file. The hexadecimal values should be prefixed by “0x”. The values should be added as attributes of the CameraDevice node. An example of the ExternalCameraCalibration.xml file is shown below (there can be multiple CameraDevice-nodes):

<CalibrationRoot>
  <CameraDevice VID="0x046D" PID="0x081B">
    <Calibration
      size="640 480"
      principal_point="315.558 227.668"
      focal_length="898.57 813.503"
      distortion_coefficients="-0.043780 0.226497 0.004928 0.000606 0 0 0 0"
    />
  </CameraDevice>
</CalibrationRoot> 

Learn More