How To Implement Virtual Buttons

The Virtual Buttons sample shows developers how to implement and configure Virtual Buttons, and how to handle the OnButtonPressed and OnButtonReleased events arising from interactions with buttons.

Design and Placement

Virtual buttons provide a useful mechanism for making image based targets interactive. The size and placement of Virtual buttons must be considered when designing an experience that uses them. There are several factors that will affect the responsiveness and usability of Virtual buttons.  

  • The length and width of the button
  • The area of the target that it covers
  • The placement of the button in relation to the both the border of the image, and other buttons on the target


Designing Buttons

  • Sizing buttons

The rectangle that you define for the area of a Virtual button should be equal to, or greater than, 10% of the overall target area.   Button events are triggered when a significant proportion of the features underlying the area of the button are concealed from the camera. This can occur when the user covers the button or otherwise blocks it in the camera view. For this reason, the button should be sized appropriately for the source of the action it is intended to respond to. For example, a button that should be triggered by a user's finger needs to be smaller than one that will be triggered by their entire hand. 

  • Evaluate sensitivity

Virtual Buttons can be assigned multiple sensitivities, which define how readily the button's OnButtonPressed will fire.

Buttons with a HIGH sensitivity will fire more easily than those with a LOW sensitivity. The button's sensitivity is a reflection of the proportion of the button area that must be covered, and the coverage time. It's advisable to test the responsiveness of each of your buttons in a a real-world setting to verify that they perform as expected.

Placing Buttons

  • Place over features

Virtual Buttons detect when underlying features of the target image are obscured from the camera view. You will need to place your button over an area of the image that is rich in features in order for it to reliably fire its OnButtonPressed event. To determine where these features are in your image, use the  Show Features link for your image in the Target Manager. You will see the available features marked with yellow hatch marks as below.

  • Inset the buttons

Virtual buttons should not be placed against the border of the target. Image based targets have a margin, equivalent to ~8% of the target area, at the edge of the target rectangle that is not used for recognition or tracking. For this reason, it's not possible to detect when a user covers this area. Be sure to inset your buttons so that you are able to detect OnButtonPressed events across their entire button area.

See:  How To Use the Feature Exclusion Buffer

  • Avoid stacking buttons

It is recommended that that you don't arrange buttons in a column in the direction that the user is facing the target. This is because the user will need to reach over lower buttons to press higher ones, which can result in the lower buttons firing their OnButtonPressed events.

If you do need to stack buttons in an arrangement that may result in this behavior, you should implement app logic that filters these results to determine which button was actually intended to be selected.

This image shows the images features and its
feature exclusion buffer area along the outer borders.

Attributes

Attributes of an ideal virtual button are listed in the following table.

Attribute

Suggestions

Size Choose areas in the images that have dimensions of approximately 10% of the image target’s size.
Shape Make buttons easily identifiable to stand out from rest of image. Highlight active buttons in the augmentation layer to hint at active regions on the target.
Texture or contrast Avoid defining buttons on low contrast areas of the targets. The underlying target area needs to have sufficient features to be evaluated.
Choose a button design that is different in texture from the object that causes the occlusion.
Arrangement on the target Arrange buttons around the target’s borders with enough space between to avoid losing tracking when the end user presses a button.

Virtual buttons are created by defining them in the Dataset Configuration XML file as a property of image targets or by adding and destroying them at application run time through a set of well-defined APIs. In Unity, Virtual Buttons can be added to a target using the Virtual Button prefab.

See:
How To Modify the Dataset Configuration XML File Outside the Target Manager for instructions for creating virtual buttons in the Dataset Configuration XML file.

Vuforia Image

This image shows the images features and its feature exclusion buffer area along the outer borders (notice that there are no features close to the edge of the image)

Attributes

Attributes of an ideal virtual button are listed in the following table.

Attribute

Suggestions

Size Choose areas in the images that have dimensions of approximately 10% of the image target’s size.
Shape Make buttons easily identifiable to stand out from rest of image. Highlight active buttons in the augmentation layer to hint at active regions on the target.
Texture or contrast Avoid defining buttons on low contrast areas of the targets. The underlying target area needs to have sufficient features to be evaluated.
Choose a button design that is different in texture from the object that causes the occlusion.
Arrangement on the target Arrange buttons around the target’s borders with enough space between to avoid losing tracking when the end user presses a button.

Virtual buttons are created by defining them in the Dataset Configuration XML file as a property of image targets or by adding and destroying them at application run time through a set of well-defined APIs. In Unity, Virtual Buttons can be added to a target using the Virtual Button prefab.

See:
How To Modify the Dataset Configuration XML File Outside the Target Manager for instructions for creating virtual buttons in the Dataset Configuration XML file.

Example

Virtual buttons on an image target four colored buttons at the bottom of the target

User-added image * Press or occlude a button to change the augmented object s color
* Buttons are 200x100 pixels in the original image of 1500x1050px. The upper left corner of the 'Red' button is located at [90,850] origin is at the upper-left corner.
* Units for the button area are in 3D scene units - the coordinate system local to the image target.
In this example, size=[247,173] as the image target size in the actual 3D scene - equals the printed size in millimeters. Therefore, all the above sizes and placement coordinates must be scaled accordingly.

Button

Original image coordinates Scaled image target coordinates

Red

( 90, 850)( 290, 925) (-108.68, -53.52)(-75.75, -65.87)

Blue

( 475, 850)( 675, 925) (- 45.28, -53.52)(-12.35, -65.87)

Yellow

( 840, 850)(1040, 925) ( 14.82, -53.52)( 47.75, -65.87)

Green

(1215, 850)(1415, 925) ( 76.57, -53.52)(109.50, -65.87)

Virtual button parameters

Button name/identifier

* A string that uniquely identifies the button within the target
* Maximum string length is 64 characters
* Character set: a-z, A-Z, 0-9,[ - _ .]

Button coordinates

* Target size in scene units measured along the horizontal and vertical axes of the rectangular target

Button sensitivity

* Buttons can be assigned one of three sensitivities, which define how readily the button will trigger based
on how much of the button is occluded and for how long

Values:
  • Low
  • Medium
  • High

Dataset Configuration XML definition

Define virtual buttons in the Database Configuration XML as children of an image target. To add virtual buttons, insert a section similar to the following code:

<ImageTarget size="247 173" name="wood">
  <VirtualButton name="red" rectangle="-108.68 -53.52 -75.75 -65.87" enabled="true" />
  <VirtualButton name="blue" rectangle="-45.28 -53.52 -12.35 -65.87" enabled="true" />
  <VirtualButton name="yellow" rectangle="14.82 -53.52 47.75 -65.87" enabled="true" />
  <VirtualButton name="green" rectangle="76.57 -53.52 109.50 -65.87" enabled="true" />
</ImageTarget>