Session Recorder API Overview

The Session Recorder allows you to Record data from system devices (e.g. camera and sensors) and tracking data (e.g. the device position or pose). The recorded data can be played back using the File Driver either on device or in Unity Play Mode in order to recreate a Vuforia session offline. In some instances, it may also be useful to share the recording with the Vuforia team to aid in reproducing a reported issue.

Prerequisites

The recording frame rate is by default set to VU_FRAME_RATE_AUTO which lets Vuforia Engine choose the frame rate best fitting the performance of the device. Other frame rate options are:

• _FULL – record frames at the full frame rate of the camera.
• _HALF – record frames at half the frame rate of the camera.

The camera recording resolution is likewise set to VU_IMAGE_SCALE_AUTO by default to let Vuforia Engine choose the best image scale in accordance with the device’s performance capabilities. Other image scale options are:

• _FULL – record images at full resolution.
• _HALF – resample images in half resolution along both dimensions.

The recording quality is set to _HIGH by default but can be set to a lower setting if longer Session Recordings captured on less powerful devices are needed. The options are:

• _HIGH – record with minimal compression artifacts
• _MEDIUM – record for general purpose recordings with video encoding allowing for longer captures without resulting in too large a file. The recording may not be suitable for playback with AR tracking.

Supported cameras 

Currently devices with cameras that capture frames in the following pixel formats are supported:

• NV21
• NV12
• YUV420P
• YV12

The formats are internally converted to H.264 before being saved.

NOTE: It is required that the device contains a hardware-accelerated H.264 video encoder which supports encoding images in the pixel format delivered by the camera.

Record sessions in one view orientation 

Record a session only in one view (display) orientation (portrait or landscape) and make sure to set the orientation before starting the Vuforia Engine. If the view orientation is not known, vuRecordingStart will not start a recording and return error code UNKNOWN_ORIENTATION. If the VuViewOrientation is changed during an ongoing recording, the recording will be stopped, and the status info will be set to ERROR_ORIENTATION_CHANGED. See Rendering in Native for more information on setting the orientation.

Unity API Overview

Session Recorder

The SessionRecorder is a Singleton accessible from VuforiaBehaviour.Instance.SessionRecorder. Through this class it is possible to access all the features detailed in the  Native API Overview below. 

SessionRecorder.StartRecording starts recording a Vuforia Session and returns a RecordingStartError enum. If the recording started successfully, StartRecording returns RecordingStartError.NONE. Please refer to the reference library for a list of recording error & status codes.

The StartRecording method accepts additional bool input parameters that determines whether the SessionRecorder should also record data from the device’s sensors or audio from the device’s microphone. Sensor recording is enabled by default.
At any time, it is possible to query the state of the SessionRecorder by calling the method SessionRecorder.GetRecordingStatus.
A recording can be stopped by calling SessionRecorder.StopRecording. When a recording is stopped or there are no recordings in progress, the status is RecordingStatus.STOPPED.

Set the encoding quality before starting the recording with  RecordingVideoEncodingQuality.HIGH for high quality or MEDIUM for general purpose and longer session recordings on e.g., HoloLens devices.

SessionRecorder.GetSupportedSources returns the data sources that can be recorded on the current device. Use SessionRecorder.GetDefaultSources to get the sources that are recommended for recording on the current device. Using the recommended set of sources will always result in a recording that can be used for AR sequence playback.

It is possible to retrieve the path of the file created by the last recorder session by calling the method SessionRecorder.GetRecordingPath. If called while a recording is in progress, it will return the path of the current recording.

Calling the method SessionRecorder.Clean deletes all the existing recorded session. If called while a recording is in progress, the method will return false and will have no effect.

Session Recorder Behaviour

The SessionRecorderBehaviour provides basic state and error management for the Session Recorder.  

The StartRecording, StopRecording and CleanStorage methods encapsulate the Session Recorder functionality and invoke the events events OnStartRecording, OnStopRecording, and OnStorageCleaned after executing the corresponding SessionRecorder methods.

All the other methods from the Session Recorder are also exposed in the Session Recorder Behaviour component in the Unity Editor Inspector panel.

Permissions

In order to support audio recording capabilities in the Session Recorder, your app must fulfill audio-related, platform-specific permission and configuration requirements. Set the permission in the Project Settings -> Player and request the permission from the user at runtime.

RequestUserPermission(string permission);

RequestUserAuthorization(UserAuthorization mode);

RequestPermission( string permission, Callbacks callbacks);

If the aforementioned platform-specific permission and configuration requirements are not fulfilled by the app, then:

• The RecordingSource call won't report audio support (i.e., AUDIO will be missing).
• The SessionRecorder.StartRecording call will fail with DATA_FLAGS_NOT_SUPPORTED if RecordingSource is configured to record audio. 

Native API Overview

There are two main classes for recording sessions. VuRecording that manages the recording session itself, and the VuSessionRecorderController that lets you create, and destroy Session Recording instances.

NOTE: VuSessionRecorderController also provides a means to remove all previous Session Recordings. This is useful for instance if the device is running low on available space.

Create a Session Recording

Create a new VuRecordingby configuring the sources to record data from. Setting start to true will start the recording immediately.

VuRecordingConfig config = vuRecordingConfigDefault();
config.start = VU_TRUE;
config.frameRate = VU_FRAME_RATE_AUTO;
config.scale = VU_IMAGE_SCALE_AUTO;
config.videoEncodingQuality = VU_RECORDING_VIDEO_ENCODING_QUALITY_HIGH;

vuSessionRecorderControllerGetDefaultRecordingDataFlags(recorderController, &config.dataFlags);

VuRecordingCreationError creationError {};
VuRecording* recording {};

vuSessionRecorderControllerCreateRecording(recorderController, &config, &recording, &creationError);

Use the optional parameter config.outputDirectory to specify a custom path for storing the recording. 

Optionally, also set the encoding quality. This can be useful if you for example wish to record longer sessions on HoloLens devices.

Destroying a recording instance

When a recording is completed, the VuRecording instance can be cleaned up using vuRecordingDestroy(recording, VU_TRUE). The second argument determines if the recorded data should be deleted from storage as well. If it is set to VU_FALSE, only the instance in memory will be destroyed.

To destroy all currently existing VuRecording instances, call 

vuSessionRecorderControllerDestroyRecordings(recorderController, VU_TRUE);

Again, the second argument determines if the recorded data should be deleted as well.

Permissions

In order to support audio recording capabilities in the Session Recorder Controller, your App must fulfill audio-related, platform-specific permission and configuration requirements. Your App is responsible for executing the following platform-specific actions prior to invoking vuEngineCreate(). See also Engine Permissions for other necessary permissions checks to initialize Vuforia Engine.

<Capabilities>
    <DeviceCapability Name="microphone" />
</Capabilities>

If the aforementioned platform-specific permission and configuration requirements are not fulfilled by the App, then:

• The vuSessionRecorderControllerGetSupportedRecordingDataFlags call won't report audio support (i.e.,VU_RECORDING_DATA_AUDIO_BIT will be missing).
• The vuSessionRecorderControllerCreateRecording call will fail with VU_RECORDING_START_ERROR_DATA_SOURCE if VuRecordingConfig.dataFlags is configured to record audio.

Querying Available Sources

The VuRecordingDataFlags bitmask provides a means to configure the sources to record from. The vuSessionRecorderControllerGetSupportedRecordingDataFlags method can be used to query the available and supported sources on the current device. 

Use vuSessionRecorderControllerGetDefaultRecordingDataFlags to get the sources that are recommended for recording on the current device. Using the recommended set of sources will always result in a recording that can be used for AR sequence playback.

Starting and Stopping a Recording

A recording can be started and stopped respectively by the vuRecordingStart and vuRecordingStop methods of VuRecording. Start returns VU_SUCCESS if the recording has been successfully started. Starting a recording requires that a Vuforia Engine is running (including the camera). Calling the vuRecordingStart twice, i.e. a subsequent call after a successful start, has no effect and will fail with the error code ERROR_ANOTHER_RECORDING_RUNNING.

Querying the Location of a Recording

After successfully starting a recording, vuRecordingGetPath can be called to retrieve the path to the file where the recording is stored, also if you specified an output directory in the configuration. 

Pausing an Application

VuSessionRecorderController does not provide a pause/resume capability. If a recording is running, it is automatically stopped when if vuEngineStop is called. This also means that a recording will be stopped if the application is paused. On resume, vuRecordingStart should again be called explicitly to start a new recording.

Querying the Recording Status

The recording status can be queried during a recording using the vuRecordingGetStatus method and additional information can be retrieved using vuRecordingGetStatusInfo. This will tell the caller if the recording has for instance been aborted due to the device running out of space (Note that VuRecording will abort an ongoing recording when the free space available on the device falls below a pre-defined threshold, i.e. before the device actually runs out of space).

Removing Previous Recordings

The SessionRecorder API provides a vuSessionRecorderControllerCleanRecordedData method for removing all recorded data. This is convenient for instance if the recorded data are not needed any more, and space needs to be freed on the device storage. Please note that cleaning recorded data will work only when no recording is in progress

Output Format

Each Session Recording is saved to an MP4 file in the private storage space of the app. The file name has the form VuforiaSession-[DATE]-[TIME].mp4, where [DATE] has the form YYYYMMDD (year, month, day) and [TIME] has the form HHMMSS (hour, minute, second). The MP4 file contains the recorded video and the captured tracking data (sensors, poses, etc.) in separate tracks.
Most generic video playback applications can be used to view the recorded video. However, it is not possible to access the recorded tracking data using these applications.

NOTE: If the MP4 file is modified by any third-party tool (e.g., video editor, converter, or similar) it will become unusable for offline playback with Vuforia.

Learn More

Playback and Recording

Creating a Custom Driver

Driver Framework

Vuforia Support for Custom Devices 

Vuforia Play Mode