Virtual Buttons

Virtual Buttons add interactivity for your Image Targets, moving on-screen interactions to the real world. Learn from the Virtual Buttons sample on how to implement and configure Virtual Buttons and immerse your end users in to your AR application.

Virtual buttons provide a useful mechanism for making image-based targets interactive. You can define regions on an image target be buttons that trigger events when occluded. Handle the events with OnButtonPressed and OnButtonReleased when the button is visually obstructed from the camera. When creating a Virtual Button, the size and placement must be considered carefully with respect to the user experience. There are several factors that will affect the responsiveness and usability of Virtual buttons.

  • The length and width of the button.
  • The area of the target that it covers.
  • The placement of the button in relation to both the border of the image, and other buttons on the target.
  • The underlying area of the button has a high contrast and enough details so that an occlusion can be recognized.

Design and Placement

Sizing Buttons

The rectangle that you define for the area of a Virtual button should be equal to, or greater than, 10% of the overall target area. Button events are triggered when a significant proportion of the features underlying the area of the button are concealed from the camera. This can occur when the user covers the button or otherwise blocks it in the camera view. For this reason, the button should be sized appropriately for the source of the action it is intended to respond to. For example, a button that should be triggered by a user's finger needs to be smaller than one that will be triggered by their entire hand.

Sensitivity Setting

Virtual Buttons can be assigned multiple sensitivities, which define how readily the button's OnButtonPressed will fire.
Buttons with a HIGH sensitivity will fire more easily than those with a LOW sensitivity. The button's sensitivity is a reflection of the proportion of the button area that must be covered, and the coverage time. It's advisable to test the responsiveness of each of your buttons in a real-world setting to verify that they perform as expected.

Place Over Features

Virtual Buttons detect when underlying features of the target image are obscured from the camera view. You will need to place your button over an area of the image that is rich in features in order for it to reliably fire its OnButtonPressed event. To determine where these features are in your image, use the Show Features link on your image in the Target Manager. You will see the available features marked with yellow hatch marks as in the example image below.

Place the Buttons

Virtual buttons should not be placed against the border of the target. Image based targets have a margin, equivalent to ~8% of the target area, at the edge of the target rectangle that is not used for recognition or tracking. For this reason, it is not possible to detect when a user covers this area. Be sure to insert your buttons so that you are able to detect OnButtonPressed events across their entire button area.

See:  How To Use the Feature Exclusion Buffer

Avoid Stacking Buttons

It is recommended that you don't arrange buttons in a column in the direction that the user is facing the target. This is because the user will need to reach over lower buttons to press higher ones, which can result in the lower buttons firing their OnButtonPressed events.

If you do need to stack buttons in an arrangement that may result in this behavior, you should implement app logic that filters these results to determine which button was actually intended to be selected.

Features of an Image Target

 

Virtual Button Attributes

Attributes of an ideal virtual button are listed in the following table.

Attribute

Suggestions

Size Choose areas in the images that have dimensions of approximately 10% of the image target’s size.
Shape Make buttons easily identifiable to stand out from rest of image. Highlight active buttons in the augmentation layer to hint at active regions on the target.
Texture or contrast Avoid defining buttons on low contrast areas of the targets. The underlying target area needs to have sufficient features to be evaluated.
Choose a button design that is different in texture from the object that causes the occlusion.
Arrangement on the target Arrange buttons around the target’s borders with enough space between to avoid losing tracking when the end user presses a button.

State and Sensitivity

Sensitivity Setting

Virtual Buttons can be assigned multiple sensitivities, which define how readily the button's OnButtonPressed will fire. The setting can be set to:

  • LOW
  • MEDIUM
  • HIGH

Buttons with a HIGH sensitivity will fire more immediately than those with a LOW sensitivity. A LOW sensitivity on the other hand will deliver more reliant event firing. The button's sensitivity reflects the proportion of the button area that must be covered, and the coverage time. It's advisable to test the responsiveness of each of your Virtual Buttons to verify that they perform as expected.

Examples

Explore the Virtual Buttons sample from the Unity Asset Store or from Vuforia’s download page to see it in action and get yourself familiar with the feature. Print the Image Targets included in the sample and test the sample in either Unity’s play mode or by deploying the build to your device.

Configure Virtual Buttons in Unity

In Unity, the Virtual Button functionality can be added to a mesh via the Virtual Button Behaviour script or by copying the Virtual Button GameObjects from the sample.

  • Choose the button’s sensitivity in the Inspector Window.

  • Add the Virtual Button Event Handler to the target.

Configure Virtual Buttons in Native

Virtual buttons are configured and created as an Observer but also require to be associated to an Image Target Observer in the configuration setup. A Virtual Button needs an assigned area within the area of the Image Target. Use vuImageTargetObserverGetTargetSize() to get the size in meters.

Create a Virtual Button Observer

VuVirtualButtonConfig config = vuVirtualButtonConfigDefault();
config.area = area;
config.sensitivity = sensitivity;
config.observer = imageTargetObserver;
        
VuObserver* virtualButtonObserver = nullptr;
vuEngineCreateVirtualButtonObserver(Engine, &virtualButtonObserver, &config, nullptr);

Observations

Get the Virtual Button Observations from the state. Use a callback to get status information from the state.

VuObservationList* observationList = nullptr;
int numObservations = 0;

// Get all VirtualButton observations from the state
vuObservationListCreate(&observationList);
vuStateGetVirtualButtonObservations(&state, observationList);
vuObservationListGetSize(observationList, &numObservations);

You can request the Virtual Button state from active targets in the scene by iterating through the Virtual Button Observations. The following code snipped provides an example of this action.

// Iterate through all Virtual Button observations
for (int i = 0; i < numObservations; i++)
{
    VuObservation* observation = nullptr;
    vuObservationListGetElement(observationList, i, &observation);
              
    VuVirtualButtonObservationInfo observationInfo;
    vuVirtualButtonObservationGetInfo(observation, &observationInfo);
       
    if (observationInfo.state == VU_VIRTUAL_BUTTON_STATE_PRESSED)
        ++mNumButtonsPressed;
        
        // Do something
}

Destroy Observer

Stop and destroy Virtual Button Observers after usage

// Destroy the observer
vuObserverDestroy(virtualButton);

 

Configuration Options for Virtual Buttons

To configure the area for placing Virtual Buttons on image-based targets, you can use the vuVirtualButtonObserverGetAssociatedObserver to get the Observer associated with the Virtual Button Observer.

vuVirtualButtonObserverGetAssociatedObserver(const VuObserver* observer, VuObserver** associatedObserver);

You can get/set the area of the Virtual Button Observer which is useful for placing it in relation to the associated Observer’s area.

vuVirtualButtonObserverGetArea(const VuObserver* observer, VuRectangle* area);
vuVirtualButtonObserverSetArea(VuObserver* observer, const VuRectangle* area);

Set the sensitivity with get/set functions. The default is set to low sensitivity.

vuVirtualButtonObserverGetSensitivity(const VuObserver* observer, VuVirtualButtonSensitivity* sensitivity);
vuVirtualButtonObserverSetSensitivity(VuObserver* observer, VuVirtualButtonSensitivity sensitivity);

Virtual Button states

A Virtual Button state can be in either VU_VIRTUAL_BUTTON_STATE_UNKNOWN, VU_VIRTUAL_BUTTON_STATE_PRESSED, or VU_VIRTUAL_BUTTON_STATE_RELEASED state. Use the vuVirtualButtonObservationInfo to retrieve the state of the button.

VuVirtualButtonState state;