Text Recognition

Vuforia can detect words that belong to a pre-defined word list. The SDK provides a list of 100,000 commonly used English words that you can incorporate into your Text Recognition apps. You can also define custom word lists and filters that block words from being recognized.

Use Cases

Text Recognition is useful for applications that need to recognize both individual words and series of words. You can use Text Recognition as a stand-alone feature or in combination with targets. Text Recognition is used for children’s educational games, and as a visual input mechanism (for use in dictionaries).

Supported characters

The Vuforia text recognition engine relies on the UTF-8 character encoding standard and is capable of recognizing any character listed in Characters Supported by Text Recognition.

Fonts and styles

The Vuforia text recognition engine can recognize text that is printed or rendered in most of the commonly occurring serif and sans serif fonts. The supported text styles include styles that are plain, bold, and italic. The following image shows examples of text styles.

Fonts and styles

Supported Environments

Text should be viewed under moderately bright and diffuse lighting. The text should be evenly lit.

Working with Text Recognition

A word list can be loaded from a file and may be extended with additional words that are specific to the application use case.

Word lists can also be filtered using filter lists to exclude certain words from being detected (using black-list filters), or to allow only specific words to be detected (using white-list filters).

Available words

Word Lists

Word lists represent the base set of words that Vuforia can use as a reference for matching text elements that appear in the camera field-of-view.

Word lists are loaded from a binary file encoded in the Vuforia Word List (VWL) format. A default VWL file contains over 100,000 high-frequency English words and is distributed in a bundle with the Vuforia SDK.

The default word list can be extended with additional word lists and the word list can be filtered. The combined use of these two mechanisms (additional words and filters) offers the developer the flexibility of customizing the text recognition behavior to cover a variety of use cases.

Use custom word lists

For most text recognition applications, the default word list for English, along with the filter capability, should cover most commonly used words. But in some cases, such as an application that must recognize only medical terminology (not in everyday usage), the default word list will not suffice and it must be replaced with a custom word list.

To replace the provided default word list with a custom set of words (such as domain-specific terminology), contact the Vuforia support team to access a VWL word list file for your custom set of words. Note that using a replacement VWL word list is required only if your application cannot achieve the desired behavior while using the default word list in combination with additional words and filters.

Review the default word list to make sure that the words you need are not already there or that they cannot be added using the provided APIs. The default word list can be found here.

Add words to the word list

Default and custom word lists can be extended with up to 10,000 additional words, words that typically consist of proper names, brand names, or place names. For example, if our base word list was the entire English dictionary, we could add the words Vuforia and PTC. Add words one-by-one programmatically, or import them from a file.

For a file to add a list of words, the file must be a plain text file and must follow these conventions:

  • The filename must end with an .lst extension (for example, my-additional-words.lst).
  • The file content must start with a UTF-8 header.
  • The file must contain one word per line. Specifically, words in the list file must be separated by a line feed character (“\n”, ASCII code: 0x0A). Click to see this example.

Apply filters

The developer can apply filters to the base set of words to exclude certain words from being detected, or conversely, to ensure that only specific words can be detected. Filters can be specified through the use of the following two elements:

  • A filter list
  • A filter mode

A filter list defines a set of words that the filter is looking for, while the filter mode tells Vuforia how to use those words for purposes of filtering. Specifically, Vuforia defines two filter modes:

  • Black-list mode – If this mode is set, Vuforia will detect all the words in the current word list except the ones specified in the filter list.
  • White-list mode – If this mode is set, Vuforia detects only those words that are included at the same time in the current word list and in the filter list.

A filter list can contain a maximum of 10,000 words and can be loaded from a file. A file containing a filter list is a plain text file and must follow the same conventions as the word list:

  • The filename must end with an .lst extension (example: my-filter-list.lst).
  • The file content must start with a UTF-8 header.
  • The file must contain one word per line. Specifically, words in the filter list file must be separated by a line feed character (“\n”, ASCII code: 0x0A). See this example.

For a basic example, assume that a filter list contains the words cat and zebra, while the word list contains only the following five words:

  • cat
  • dog
  • fish
  • lion
  • zebra

By applying a black-list filter mode with such a filter list, Vuforia can detect only the words dog, fish and lion, which are all the words in our initial word list, but the words cat and zebra have been excluded.

If you use the same filter list but with a white-list filter mode applied, you allow Vuforia to detect only the words cat and zebra, which are the only words that appear in both the filter list and in the word list.

Samples

The Text Recognition sample is in the Core Features samples project for each SDK version.


Text Recognition Tips

If you are building a text recognition application, consider the following suggestions:

  • Provide visual feedback for the detection Region of Interest (ROI).
    • Represent the ROI area visually.
    • For example, draw a rectangle around the ROI or darken the area outside to clearly show users which part of the camera view can be used for word detection.
  • Define the ROI according to the expected text layout.
    • For example, a region with small height will usually work well when only a few words in the same line are expected. However, an area with larger height works better when expected words are arranged on multiple lines.
    • The figure below illustrates an example of the ROI for two different text layouts.
Vuforia Image
  • Avoid unnecessarily large detection ROIs.
    • The size of the detection ROI can affect performance.
    • In particular, larger areas can result in longer detection times and a less than satisfactory user experience.
    • Consider using an optimal size that fits the needs of your application.
  • Consider the expected orientation of the device, which is when the user tries to detect a piece of text.
    • Define the up direction of your detection ROI after considering the expected orientation.
    • The figure below illustrates the ROI up direction with respect to device orientation.
Vuforia Image

How To Add Word Lists to an App

For applications that use the text recognition feature for word targets, a developer needs the following tools:

  1. A device database that contains a word list, which is a reference set of words that the application must recognize and track. The SDK comes with a default word list for English that can be loaded at run time.
    • This word list contains more than 100,000 frequently used English words and it will satisfy most application use cases for text recognition.
  2. An optional list of additional words that is stored in a plain text file ending with .lst, which can be added to the default word list.
    • These are words that are not found in the default word list that is included with the SDK but are words required to be recognized by the app.
  3. An optional filter list file for developers who need to restrict recognition to a set of words or exclude certain words from being recognized.

The process for creating an app with text recognition is illustrated by the following diagram.

Vuforia Image

Characters Supported by Text Recognition

 

The Vuforia text recognition engine words and wordlists should utilize the UTF-8 character encoding standard and the following characters.

 

Vuforia Image
Vuforia Image