Virtual Buttons

Virtual Buttons add interactivity for your Image Targets, moving on-screen interactions to the real world. Learn from the Virtual Buttons sample on how to implement and configure Virtual Buttons and immerse your end users in to your AR application.

Virtual buttons provide a useful mechanism for making image-based targets interactive. You can define regions on an image target be buttons that trigger events when occluded. Handle the events with OnButtonPressed and OnButtonReleased when the button is visually obstructed from the camera. When creating a Virtual Button, the size and placement must be considered carefully with respect to the user experience. There are several factors that will affect the responsiveness and usability of Virtual buttons.

• The length and width of the button.
• The area of the target that it covers.
• The placement of the button in relation to both the border of the image, and other buttons on the target.
• The underlying area of the button has a high contrast and enough details so that an occlusion can be recognized.

Design and Placement

Sizing Buttons

The rectangle that you define for the area of a Virtual button should be equal to, or greater than, 10% of the overall target area. Button events are triggered when a significant proportion of the features underlying the area of the button are concealed from the camera. This can occur when the user covers the button or otherwise blocks it in the camera view. For this reason, the button should be sized appropriately for the source of the action it is intended to respond to. For example, a button that should be triggered by a user's finger needs to be smaller than one that will be triggered by their entire hand.

Sensitivity Setting

Virtual Buttons can be assigned multiple sensitivities, which define how readily the button's OnButtonPressed will fire.
Buttons with a HIGH sensitivity will fire more easily than those with a LOW sensitivity. The button's sensitivity is a reflection of the proportion of the button area that must be covered, and the coverage time. It's advisable to test the responsiveness of each of your buttons in a real-world setting to verify that they perform as expected.

Place Over Features

Virtual Buttons detect when underlying features of the target image are obscured from the camera view. You will need to place your button over an area of the image that is rich in features in order for it to reliably fire its OnButtonPressed event. To determine where these features are in your image, use the Show Features link on your image in the Target Manager. You will see the available features marked with yellow hatch marks as in the example image below.

Virtual Button Attributes

Attributes of an ideal virtual button are listed in the following table.

State and Sensitivity

Sensitivity Setting

Virtual Buttons can be assigned multiple sensitivities, which define how readily the button's OnButtonPressed will fire. The setting can be set to:

• LOW
• MEDIUM
• HIGH

Buttons with a HIGH sensitivity will fire more immediately than those with a LOW sensitivity. A LOW sensitivity on the other hand will deliver more reliant event firing. The button's sensitivity reflects the proportion of the button area that must be covered, and the coverage time. It's advisable to test the responsiveness of each of your Virtual Buttons to verify that they perform as expected.

Examples

Configure Virtual Buttons in Unity

In Unity, the Virtual Button functionality can be added to a mesh via the Virtual Button Behaviour script or by copying the Virtual Button GameObjects from the sample.

• Choose the button’s sensitivity in the Inspector Window.

• Add the Virtual Button Event Handler to the target.

Configure Virtual Buttons in Native

Virtual buttons are configured and created as an Observer but also require to be associated to an Image Target Observer in the configuration setup. A Virtual Button needs an assigned area within the area of the Image Target. Use vuImageTargetObserverGetTargetSize() to get the size in meters.

Create a Virtual Button Observer

VuVirtualButtonConfig config = vuVirtualButtonConfigDefault();
config.area = area;
config.sensitivity = sensitivity;
config.observer = imageTargetObserver;
        
VuObserver* virtualButtonObserver = nullptr;
vuEngineCreateVirtualButtonObserver(Engine, &virtualButtonObserver, &config, nullptr);

Observations

Get the Virtual Button Observations from the state. Use a callback to get status information from the state.

VuObservationList* observationList = nullptr;
int numObservations = 0;

// Get all VirtualButton observations from the state
vuObservationListCreate(&observationList);
vuStateGetVirtualButtonObservations(&state, observationList);
vuObservationListGetSize(observationList, &numObservations);

You can request the Virtual Button state from active targets in the scene by iterating through the Virtual Button Observations. The following code snipped provides an example of this action.

// Iterate through all Virtual Button observations
for (int i = 0; i < numObservations; i++)
{
    VuObservation* observation = nullptr;
    vuObservationListGetElement(observationList, i, &observation);
              
    VuVirtualButtonObservationInfo observationInfo;
    vuVirtualButtonObservationGetInfo(observation, &observationInfo);
       
    if (observationInfo.state == VU_VIRTUAL_BUTTON_STATE_PRESSED)
        ++mNumButtonsPressed;
        
        // Do something
}

Destroy Observer

Stop and destroy Virtual Button Observers after usage

// Destroy the observer
vuObserverDestroy(virtualButton);

Configuration Options for Virtual Buttons

To configure the area for placing Virtual Buttons on image-based targets, you can use the vuVirtualButtonObserverGetAssociatedObserver to get the Observer associated with the Virtual Button Observer.

vuVirtualButtonObserverGetAssociatedObserver(const VuObserver* observer, VuObserver** associatedObserver);

You can get/set the area of the Virtual Button Observer which is useful for placing it in relation to the associated Observer’s area.

vuVirtualButtonObserverGetArea(const VuObserver* observer, VuRectangle* area);
vuVirtualButtonObserverSetArea(VuObserver* observer, const VuRectangle* area);

Set the sensitivity with get/set functions. The default is set to low sensitivity.

vuVirtualButtonObserverGetSensitivity(const VuObserver* observer, VuVirtualButtonSensitivity* sensitivity);
vuVirtualButtonObserverSetSensitivity(VuObserver* observer, VuVirtualButtonSensitivity sensitivity);

Virtual Button states

A Virtual Button state can be in either VU_VIRTUAL_BUTTON_STATE_UNKNOWN, VU_VIRTUAL_BUTTON_STATE_PRESSED, or VU_VIRTUAL_BUTTON_STATE_RELEASED state. Use the vuVirtualButtonObservationInfo to retrieve the state of the button.

VuVirtualButtonState state;

Learn More

How to Create and Destroy Virtual Buttons at Runtime

How to Modify the Dataset Configuration XML File Outside the Target Manager

How to Use the Feature Exclusion Buffer