Session Recorder API Overview

The Session Recorder allows you to Record data from system devices (e.g. camera and sensors) and tracking data (e.g. the device position or pose). The recorded data can be played back using the File Driver either on device or in Unity Play Mode in order to recreate a Vuforia session offline. In some instances, it may also be useful to share the recording with the Vuforia team to aid in reproducing a reported issue.

Prerequisites

The recording frame rate is by default set to VU_FRAME_RATE_AUTO which lets Vuforia Engine choose the frame rate best fitting the performance of the device. Other frame rate options are:

  • _FULL – record frames at the full frame rate of the camera.
  • _HALF – record frames at half the frame rate of the camera.

The camera recording resolution is likewise set to VU_IMAGE_SCALE_AUTO by default to let Vuforia Engine choose the best image scale in accordance with the device’s performance capabilities. Other image scale options are:

  • _FULL – record images at full resolution.
  • _HALF – resample images in half resolution along both dimensions.

Supported cameras 

Currently devices with cameras that capture frames in the following pixel formats are supported:

  • NV21
  • NV12
  • Luminance-only (Grayscale)

Please note that NV12 is internally converted to NV21 before being saved (for the difference between these two pixel formats, see the wiki.videolan.org).

NOTE: Some Android devices provide camera frames in other formats e.g. YV12; these are not currently supported for recording.

Record sessions in one view orientation 

Record a session only in one view (display) orientation (portrait or landscape) and make sure to set the orientation before starting the Vuforia Engine. If the view orientation is not known, vuRecordingStart will not start a recording and return a  ORIENTATION_NOT_SUPPORTED exit status. If the VuViewOrientation is changed during an ongoing recording, the recording will be stopped, and the status will be set to ORIENTATION_NOT_SUPPORTED. See Rendering in Native for more information on setting the orientation.

Unity API Overview

Session Recorder

The SessionRecorder is a Singleton accessible from VuforiaBehaviour.Instance.SessionRecorder. Through this class it is possible to access all the features detailed in the  Native API Overview below

SessionRecorder.Startstarts recording a Vuforia Session and returns a RecordingStatus enum. If the recording started successfully, Start returns RecordingStatus.RECORDING_IN_PROGRESS. Please refer to the reference library for a list of recording statuses.

The Start method accepts one bool input parameter that determines whether the SessionRecorder should also record data from the device’s sensors. This is enabled by default. 
At any time, it is possible to query the state of the SessionRecorder by calling the method SessionRecorder.GetRecordingStatus
Recording can be stopped by calling SessionRecorder.Stop. When a recording has been stopped or there are no recordings in progress, the status is RecordingStatus.RECORDING_NOT_STARTED.

SessionRecorder.GetDeviceCapabilities returns the capabilities supported by the device whose data can be recorded.

It is possible to know the path of the last recorder session by calling the method SessionRecorder.GetRecordingPath. If called while a recording is in progress, it will return the path of the current recording.

Calling the method SessionRecorder.Clean deletes all the existing recorded session. If called while a recording is in progress, the method will return false and will have no effect.

Session Recorder Behaviour

The SessionRecorderBehaviour provides basic state and error management for the Session Recorder.  

The StartRecordingStopRecording and CleanStorage methods encapsulate the Session Recorder functionality and invoke the events OnRecordingStartedOnRecordingStoppedOnStorageCleaned after executing the corresponding SessionRecorder methods.

All the other methods from the Session Recorder are also exposed in the Session Recorder Behaviour component in the Unity Editor Inspector panel 

Native API Overview

There are two main classes for recording sessions. VuRecording that manages the recording session itself, and the VuSessionRecorderController that lets you create, and destroy recording instances.

NOTE: VuRecording also provides a means to remove all previous recordings. This is useful for instance if the device is running low on available space.

Create a Session Recording

Create a new VuRecordingby configuring the sources to record data from. Setting start to true will start the recording immediately.

VuRecordingConfig config = vuRecordingConfigDefault();
config.sources.camera = VU_TRUE;
config.sources.sensors = VU_TRUE;
config.start = VU_TRUE;
config.frameRate = VU_FRAME_RATE_AUTO;
config.scale = VU_IMAGE_SCALE_AUTO;

vuSessionRecorderControllerCreateRecording(mRecorderController, &config, &mCurrentRecording, &mCurrentRecordingStatus);

NOTE: When done using the Session Recorder, all VuRecording’s created so far can be destroyed using the following call.

vuSessionRecorderControllerDestroyRecordings(mRecorderController, VU_TRUE));

Querying Available Sources

The VuRecordingSources data structure provides a means to configure the sources to record from. The vuSessionRecorderControllerGetDeviceCapabilities method can be used to query the available and supported sources on the current platform. The camera source also implies the device pose – i.e. if the device pose is available on the current platform, AND if camera is requested, then device pose is automatically recorded as well. 

Starting and Stopping a Recording

A recording can be started and stopped respectively by the vuRecordingStart and vuRecordingStop methods of VuRecording. Start returns VU_SUCCESS if the recording has been successfully started. Starting a recording requires that a Vuforia Engine is running (including the camera). Calling the vuRecordingStart twice, i.e. a subsequent call after a successful start, has no effect and will fail. This can be reported with ANOTHER_RECORDING_RUNNING (unless the already started recording has been aborted due to an error). 

Querying the Location of a Recording

The vuRecordingGetPath method can be called for querying the location of the current OR last recording. If this method is called after a successful vuRecordingStart, but before a corresponding vuRecordingStop, it returns the path to the current recording. If it is called after vuRecordingStop, but before the next vuRecordingStart , it returns the path to the last recording. It is recommended to make a copy of this path if a list of all recordings made by an app is required: VuRecording does not provide this information!

Pausing an Application

VuRecording does not provide a pause/resume capability. The recording is automatically stopped if vuEngineStop is called. This means that when an application that is using the VuRecording is paused and resumed the current recording session is stopped and stored. On resume, vuRecordingStart should again be called explicitly to start a new recording.

Please note that pausing an application abruptly with vuEngineStop will force-stop the recording and thus after this the recording might end up being in an unusable state

Querying the Recording Status

The recording status can be queried during a recording using the vuRecordingGetStatus method. This will tell the caller if the recording has for instance been aborted due to the device running out of space (Note that VuRecording will abort an ongoing recording when the free space available on the device falls below a pre-defined threshold, i.e. before the device actually runs out of space).

Removing Previous Recordings

The SessionRecorder API provides a vuSessionRecorderControllerCleanRecordedData method for removing all recorded data. This is convenient for instance if the recorded data are not needed any more, and space needs to be freed on the device storage. Please note that cleaning recorded data will work only when no recording is in progress

Output Format

Each recorded session is saved in a directory with a human-readably timestamped name in the private storage space of the app. The directory name has the form VuforiaSession-[DATE]-[TIME], where [DATE] has the form YYYYMMDD (year, month, day) and [TIME] has the form hhmmss (hour, minute, second). Each complete session directory has an .xml file named FileDriverRecording.xml, accompanied by snapshots of the camera scene saved in a sub-directory called Camera. The .xml file stores timestamped descriptors describing the recorded data and is related to the snapshots directory by the snapshot filenames that are included as part of the camera frame descriptors. The camera snapshots are saved as Portable Gray Map (PGM) or custom Extended PGM (called XPGM) image files, respectively for grayscale and color images. Please note that although XPGM files are non-standard, they are still viewable by some tools (notably the Preview application on macOS).