Smart Terrain Workflow in Unity

Unity Scene Preparation

  1. Delete the default Main Camera in the scene and add an ARCamera instance from the /Vuforia/Prefabs folder.
  2. Configure the ARCamera for the Device Database and Targets you’ll be using.
  3. Add target prefab instances for the target types that you’ll be using. Smart Terrain works with Image Targets, Multi Targets, and Cylinder Targets.
  4. Add a Directional Light to illuminate the scene.
  5. Enable Smart Terrain for your target(s) and select to either use the new Smart Terrain instance created for your target, or an alternate Smart Terrain instance that already exists in the scene. You can use multiple targets to initialize the same Smart Terrain instance.
  6. Add any content triggered by the target’s detection as a child of your target instance so that it’s rendered when the target is recognized.

 


Smart Terrain scene elements and their Hierarchy in the Unity Editor environment.

  1. Set the World Center Mode on your ARCamera to SPECIFIC_TARGET and drag your Primary Surface instance onto the World Center field.

The ARCamera's VuforiaBehaviour component in the Unity Inspector

Scene props and custom assets as children of the Smart Terrain instance in the Unity Hierarchy

Tip: See section below "Integrating NavMeshes to Smart Terrain".

How to Define Occluder Bounds

The Occluder Bounds for a target define the geometric volume of the area surrounding the initializing target that you want to occlude from mesh updates and rendering. It enables the tracker to distinguish the target area from the background area of your stage, to create distinct meshes for these surfaces. This provides more accurate recognition of the significant shape of the target and improves runtime performance by enabling the tracker to ignore occluded mesh regions.

In most cases, you can simply set the Min and Max Occluder Bounds to the targets size, which is the default setting and can also be established by clicking the ‘reset occluder bounds to target size button‘. But there are a few cases where custom min / max occlude bounds settings are useful.

The Occluder Bounds fields in the initialization target's Image Target Behaviour component in the Unity Inspector

Targets with borders

Occluder Bounds for printed targets with a border region (e.g. a white border ) that is not part of the original target image should accommodate this border by defining the Occluder Bounds Min for the area of the target image itself and the Occluder Bounds Max for the larger area of the printed target including the border. If the border area is not accommodated by defining a maximum occluder area, the tracker may stop mesh generation at the edge of the printed target.

Targets inset within larger volumes

When a target image is surrounded by a larger surface volume that should also be occluded, define the Occluder Bounds Min for the area of the target image and the Occluder Bounds Max for the volume of the surrounding object. For example, if a target image is on the cover of a book, the Max occluder volument should be defined as the volume of the book.

How to Define Surface Offsets

Custom surface offsets enable you to accurately define the position and orientation of the target surface relative to the primary surface. This is helpful when using targets that will not be coplanar with the primary surface, such as those printed on thick media ( e.g. foamcore ), and 3D objects. If you don’t use a surface offset in these cases, Smart Terrain will assume that the primary surface is coplanar and attempt to reconstruct the stage surface from the plane of your target, which is likely to produce poor results.

The Surface Offset fields presented when a custom Surface Offset geometry is enabled in the Unity Inspector

You can define the surface offset by its rotation and translation relative to the primary surface in the target’s inspector, or by positioning the offset by hand using a gizmo in the scene view. Both approaches will be visible in the scene view, enabling you to evaluate the accuracy of the offset you’ve defined.

Memory Management

Prop and surface tracking and reconstruction are computationally intensive processes. You should implement logic in your Smart Terrain apps to manage these performance factors in order to provide the best Smart Terrain experience to users.

You can also implement runtime specific memory management by analyzing your app’s FPS, and by stopping mesh updates and prop creation if your app’s FPS approaches a minimum acceptable frame rate.

Smart Terrain Stage Preparation

It’s helpful to regard the real world setting that is augmented by your Smart Terrain scene as a stage. Like a theater stage, the dominant surface of the setting should be flat and establish a visually distinct area where the initializing target and any props will be placed. Targets should be placed away from the border of the stage so that there is a region of the stage surface between the target border and stage border. This will enable the best performance from Smart Terrain by providing a well defined tracking reference over a contained area.

The stage that the user prepares should also be well lit with a surface appearance that is distinct from its surroundings. Visually uniform surfaces (e.g. plain white ) and those with dense features (e.g. wood grain, marble or granite) will work well. Avoid surfaces that present apparent borders, like a reticulated tile surface, and those that share the appearance of surrounding surfaces, such as a white table against a white wall. Smart Terrain needs to be able to detect the stage surface apart from its surroundings.

The Primary Surface mesh in the Unity scene will be mapped to this real-world stage surface. Props that you add to your Unity scene will be associated with the physical props that are placed on the stage based on the app logic that you define. Smart Terrain will regard props that are close together as a single object. You can implement custom Prop associations based on the properties of reconstructed physical objects, such as their dimensions and distance from the scene origin.

The recommended practice for scanning the stage, in order to achieve the most precise stage area and prop reconstruction, is to start with the device camera close to the initializing target, so that it nearly fills the camera image. Once the target is recognized, slowly pull the device back until the entire stage is within the field of view of the camera. From there you can move around the stage to capture all of the perspectives you need to execute you Smart Terrain experience.

Avoid erratic camera motion during the scanning phase as this will negatively affect the ability of the tracker to generate an accurate reconstruction of the stage. It’s best to move the device in a smooth continuous trajectory when you are scanning the stage area.

How To Configure an Initializing Target for Smart Terrain

A Smart Terrain scene can utilize Image Targets, Multi Targets and Cylinder Targets to initialize the Smart Terrain experience, and multiple targets can be used to initialize the same Smart Terrain instance. The role of the target is to provide a predefined scale reference and to establish the coordinate origin of the generated Terrain.

Tips:

  • Constrain the number of props that can be reconstructed – we recommend a maximum of 5
  • Use occluder bounds effectively. See How to Define Occluder Bounds, below
  • Restrict the potential size of the surface mesh using the Maximum Area settings in the Inspector of your Smart Terrain instance

Enabling Smart Terrain



Steps:

  1. Customize the association of Props to objects within the OnPropCreate callback of the DefaultSmartTerrainEventHandling on the Smart Terrain instance, or use a custom event handler that you’ve created that implements the ISmartTerrainEventHandler interface.
  2. Define the Occluder Bounds and scale factor for your target, and optionally the Maximum Area of your terrain in scene units, and the surface offset of your Primary Surface relative to your initialization target.
  3. Build and deploy your scene.

Defining Occluder Bounds

The occluder bounds represent the area of the Terrain that is occluded by the geometry of the target on the stage and are defined in scene units.

Select Reset Occlude Bounds to Target Size to automatically define the bounds to match your target geometry.

Defining custom min and max occluder bounds enables you to utilize targets that are only a subregion of a larger physical object. In these cases, the Occluder Bounds Min volume should be the volume of your target and the Occluder Bounds Max volume should be that of its surrounding object.

Defining accurate occluder bounds enables Smart Terrain to render the scene realistically and aids performance by enabling the tracker to ignore regions of the stage that are obscured by the target.

Defining Scale Factor

The scale factor you define should be equivalent to the size of your physical target in millimeters divided by its size in scene units. So if you have a physical target with a volume of 100 mm and it has a volume in scene units of 10, then the scale factor is 10 ( 100 / 10 = 10 ).

Tip: Defining your target dimensions by their true physical dimenions in millimeters enables you to use a scale factor of 1 and makes it easy to define the maximum area, occluder, and offset geometries in your scenes accurately.

For example, if you have an ImageTarget that is 20 cm across when printed, simply define its width in the Target Manager as 200 ( 20 cm = 200 mm ), and this will enable you to use a scale factor of 1.

Surface Offset Settings

Custom surface offsets enable you to use physical targets for which the target image is not coplanar with the intended primary surface, such as those on thick target media or which are placed on the face of a 3D physical objects.

Using multiple targets in a scene

Multiple targets can be used to initialize a shared Smart Terrain instance by selecting that instance as the Smart Terrain to Initialize for those targets.

Adding Content

Digital content added as a child of the initializing target will only be enabled when that target is in view.

Tip: We’ve provided a CustomSmartTerrainEventHandler in the Penguin project that shows how to maintain rendering of this content when it’s parent target is no longer in view.

Digital content added as a child of the Smart Terrain instance will be enabled while the stage is in view.

Digital content added to prop instances will be enabled while the associated physical prop is in view.

Props can be assigned based on a variety of factors and properties:

  • Order of appearance, i.e. the nth prop is associated with a specific prefab instance
  • Size/height of prop
  • Location on the primary surface
  • User interaction

How To Configure the Smart Terrain Prefab

The Smart Terrain prefab encapsulates the Smart Terrain Behavior and event handling components for Smart Terrain in the Unity scene. Smart Terrain prefab instances can be added to a scene by enabling Smart Terrain on supported target prefab instances from their Trackable Behavior component in the Inspector.

Tip: Create Smart Terrain prefab instances in your scene using the Create New Smart Terrain Instance button on your initializing Image, Cylinder or Multi-Target. This automatically configures your target for that Smart Terrain instance.

Role

The Smart Terrain prefab represents a single instance of the Smart Terrain Tracker in a scene, and enables you to configure the behavior and event handling for that instance. Each Smart Terrain instance will generate a unique terrain and any props that are children of the instance will be exclusive to that Terrain.

Multiple Smart Terrain prefab instances can be employed in a scene by associating them with individual target instances. This enables the creation of independent terrains for each target in the scene. Multiple targets can also share the same Smart Terrain instance, which enables a single terrain to be initialized by a variety of targets.

You can control and customize the behavior of a Smart Terrain instance by extending the Smart Terrain event handler components on the prefab instance.

Script components and their configuration


Reconstruction Behaviour

The Reconstruction Behaviour is responsible for configuring, and managing a Smart Terrain instance in the scene and for issuing callbacks when the terrain is initialized and updated.

Automatic Start – Enabling this option will cause the terrain to begin updating as soon as the initializing target is detected. If this option is not selected, updates will need to be started using the Smart Terrain Tracker API’s StartMeshUpdates method.

Create Nav Mesh - Enabling this option will cause the Primary Surface mesh to be generated as a dynamic Nav Mesh for the A* path finding project. See:

Use Maximum Area – This option enables you to restrict the terrain to within a bounding rectangle defined in scene units.

Maximum Area – If Use Maximum Area is selected, the rectangle centered at x and y, with width W and height H will define the maximum area of the terrain. The terrain will not grow outside of this area.

DefaultSmartTerrainEventHandler - Implements the ISmartTerrainEventHandler to receive callbacks from the Smart Terrain Behaviour and will associate any prop assigned to it prop Template field with physical props found on the stage. You can also write your own custom event handlers by implementing the ISmartTerrainEventHandler interface and adding the script to the Smart Terrain instance as a script component.

Adding content

Digital content added as a child of the initializing target will only be enabled when that target is in view.

Digital content added as a child of the Smart Terrain instance will be enabled while the stage is in view.

Digital content added to prop instances is enabled while the associated physical prop is in view.

Props can be assigned based on a variety of factors and properties:

  • Order of appearance, i.e., the nth prop is associated with a specific prefab instance
  • Size/height of prop
  • Location on the primary surface
  • User interaction

How To Use the Smart Terrain Prop Prefab

The prop prefab is the 3D representation within a Smart Terrain scene of physical props that are found on the stage at runtime. It provides an oriented bounding box with colliders that are sized and positioned to correspond with their real-world counterpart.

Prop instances are programmatically associated with physical props that are discovered by the Smart Terrain tracker at runtime. The same prop instance can be assigned to multiple physical props. These instances can be controlled through their Smart Terrain prop Behaviour API.

Script Components & their configuration

Mesh Filter

This Mesh Filter will be updated by the Smart Terrain tracker to correspond to the size and position of the prop’s physical counterpart. It should be set to a Cube mesh and is defined as the Mesh Filter to Update within the prop Behaviour component.

Mesh Renderer

The Mesh Renderer handles rendering of the prop mesh and can be customized by adding materials to the renderer’s materials array.

Size – increasing the size of the materials array will enable you to add additional materials to be rendered.

Element0..N – add new materials to the Element fields. Be sure to retain the Depth Mask so that your prop supports depth culling.

Prop Behaviour

Filter to Update – defines which Mesh Filter to update for this prop.

Mesh Collider to Update – defines which Mesh Collider to update for this prop.

Box Collider to Update – defines which Box collider to update for this prop.

Tip: Don’t use both a Mesh and Box collider on your Props, set one of these to None. The Mesh Collider is set to None by default because Box collider are more efficient to process. Only use a Mesh collider if you need to exploit a specific feature of Mesh colliders not available with Box colliders.

Automatic mesh and collider updates of the Prop can be toggled using the SetAutomaticUpdatesDisabled(bool disabled) method of the PropBehaviour.

Additional properties of the Prop trackable can be accessed via the PropBehaviour.Prop object and it’s parent class SmartTerrainTrackable:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
/// <summary>
/// A smart terrain trackable which is reconstructed as part of the smart terrain
/// </summary>
public interface SmartTerrainTrackable : Trackable
{
    /// <summary>
    /// Get the mesh revision, which is increased whenever the geometry of the smart terrain trackable is updated.
    /// </summary>
    int MeshRevision { get; }
  
    /// <summary>
    /// Get the mesh of the smart terrain trackable.
    /// </summary>
    Mesh GetMesh();
  
    /// <summary>
    /// Get the local 3D position of the smart terrain trackable relative to the parent
    /// </summary>
    Vector3 LocalPosition { get; }
      
    /// <summary>
    /// The parent trackable in the Smart Terrain scene graph hierarchy.
    /// If this is the root node, Parent is null
    /// </summary>
    SmartTerrainTrackable Parent { get; }
  
    /// <summary>
    /// All child trackables in beneath this node in the Smart Terrain scene graph hierarchy.
    /// </summary>
    IEnumerable<SmartTerrainTrackable> Children { get; }
}
  
/// <summary>
/// A 3D object - prop - which is reconstructed as part of smart terrain
/// </summary>
public interface Prop : SmartTerrainTrackable
{
    /// <summary>
    /// The bounding box of the smart terrain trackable, oriented around the y axis
    /// </summary>
    OrientedBoundingBox3D BoundingBox { get; }
}

Wireframe Behaviour

The Wireframe Behaviour component enables you to customize how the mesh appears to the user.

Show Lines – disabling this option will make the mesh invisible.

Line Color – select the line color you want users to see using the color picker.

Tip: For the best performance on devices, it’s recommended to disable all wireframe behaviours on Props and the Primary Surface before deploying your app. Wireframe rendering will impede your app’s performance with larger meshes.

Wireframe Trackable Event Handler

This event handler implements the ITrackableEventHandler interface to enable and disable the prop’s rendering and collider components in response to changes to the physical prop’s TrackableBehaviour status, such as when the prop has been detected and when it is activitly being tracked. It also toggles the WireframeBehaviour. This class implements the OnTrackableStateChanged callback to trigger OnTrackingFound and OnTrackingLost handlers.

Similar to the DefaultTrackableEventHandler, this TrackableEventHandler can be customized to achieve a different behavior, in particular when the WireframeBehaviour is not used.

Bounding Box Collider

The Bounding Box Collider is a Box Collider added to each prop as a child object of the prop instance. This collider is also defined for the Box Collider to Update in the prop’s prop Behaviour component. The Bounding Box Collider enables the prop instance to respond to Unity physics events, such as by triggering collision events and repelling rigidbody objects.

Adding Content

Digital content added as a child of the initializing target will only be enabled when that target is in view.

Digital content added as a child of the Smart Terrain instance will be enabled while the stage is in view.

Digital content added to prop instances will be enabled while the associated physical prop is in view.

Props can be assigned based on a variety of factors and properties.

  • Order of appearance, i.e. the nth prop is associated with a specific prefab instance
  • Size/height of prop
  • Location on the primary surface
  • User interaction

How To Customize a Smart Terrain Scene

Smart Terrain utilizes an event driven programming paradigm. The lifecycle and management of its core features are governed by behaviours, and events arising from changes to the states of these behaviours are received by event handlers that are registered with these behaviours. The underlying components of the SDK architecture are exposed as singleton instances which provide a single universal reference to access the API for that component.

Obtaining the Smart Terrain Tracker instance

SmartTerrainTracker mTracker = TrackerManager.Instance.GetTracker<SmartTerrainTracker>();

Methods:

1
2
3
4
5
Starting and stopping the Smart Terrain Tracker
 
Start() starts the Smart Terrain Tracker.
 
Stop() stops the Smart Terrain Tracker.

Resetting, Starting and Stopping Reconstructions

Ref: SmartTerrainUIEventHandler.cs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
SmartTerrainTracker      mTracker = TrackerManager.Instance.GetTracker<SmartTerrainTracker>();
ReconstructionBehaviour  mReconstructionBehaviour = (ReconstructionBehaviour)FindObjectOfType(typeof(ReconstructionBehaviour));
 
// Start the Reconstruction
 
        if ((mReconstructionBehaviour != null) && (mReconstructionBehaviour.Reconstruction != null))
        {
           mReconstructionBehaviour.Reconstruction.Start();
        }
 
// Stop the Reconstruction
 
        if ((mReconstructionBehaviour != null) && (mReconstructionBehaviour.Reconstruction != null))
        {
           mReconstructionBehaviour.Reconstruction.Stop();
        }
 
// Reset Tracking and Reconstruction
 
        if ((mReconstructionBehaviour != null) && (mReconstructionBehaviour.Reconstruction != null))
        {
            bool trackerWasActive = mTracker.IsActive;
            // first stop the tracker
            if (trackerWasActive)
                mTracker.Stop();
            // now you can reset...
            mReconstructionBehaviour.Reconstruction.Reset();
            // ... and restart the tracker
            if (trackerWasActive)
            {
                mTracker.Start();
                mReconstructionBehaviour.Reconstruction.Start();
            }
        }

Setting the initialization target programmatically

SetInitializationTarget sets the Trackable that is used to initialize Smart Terrain.

1
2
3
4
5
6
7
8
// For Cylinder Targets
bool SetInitializationTarget(CylinderTarget cylinderTarget, Vector3 occluderMin, Vector3 occluderMax, float scaleToMillimenters);
 
// For Image Targets
bool SetInitializationTarget(ImageTarget imageTarget, Vector3 occluderMin, Vector3 occluderMax, float scaleToMillimenters);
 
// For Multi Targets
bool SetInitializationTarget(MultiTarget multiTarget, Vector3 occluderMin, Vector3 occluderMax, float scaleToMillimenters);

Use occluderMin and occluderMax to specify an occluder box that will serve as a mask for smart terrain plane finding.

scaleToMillimenters is a scale factor that defines how the initialization target occluder dimensions need to be scaled to be in real world millimeters.

Associating props

Custom prop association logic can be implemented in classes that implement ISmartTerrainEventHandler within the OnPropCreated callback.

Example:

1
2
3
4
5
6
7
8
9
10
public void OnPropCreated(Prop prop)
{
        Debug.Log ("---Created Smart Terrain Prop");
        var manager = TrackerManager.Instance.GetStateManager().GetSmartTerrainManager();
                 
        if(prop.LocalPosition.y < 0)
                 manager.AssociateProp(RedPropTemplate, prop);
        else
                 manager. AssociateProp (BluePropTemplate, prop);
}

Smart Terrain Event Handlers

Classes implementing the ISmartTerrainEventHandler interface can be registered with the SmartTerrainBehaviour.

See the DefaultSmartTerrainEventHandler.cs file for sample code demonstrating how a Smart Terrain event handler should be implemented.

Example:

1
2
3
4
5
6
7
8
void Start()
{
    SmartTerrainBehaviour behaviour = GetComponent<SmartTerrainBehaviour>();
    if (behaviour)
    {
        behaviour.RegisterSmartTerrainEventHandler(this);
    }
}

The following callbacks will be invoked on registered event handlers:

1
void OnInitialized(InitializationInfo info)

The Smart Terrain engine has completely initialized

1
void OnPropCreated(Prop newprop)

A physical prop has been detected for the first time. In this callback the prop can be associated with a specific propBehaviour. This can be done by calling the method AssociateProp(string name, Prop newprop) in the SmartTerrainManager. See: DefaultSmartTerrainEventHandler.cs

1
void OnPropUpdated(PropBehaviour updatedpropBehaviour)

The geometry of an prop has been changed.

1
void OnPropDeleted(Prop deletedprop)

A prop has been deleted. This happens if the object was a false detection, or if an existing prop is determined to actually be two objects. The Smart Terrain Tracker will recognize objects that are at least 2cm apart as two distinct props.

1
void OnSurfaceUpdated(SurfaceAbstractBehaviour updatedSurfaceBehaviour)

This callback is invoked when the geometry of the primary surface has been updated.

Smart Terrain Tracker API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
/// <summary>
/// This class handles Text Tracking and defining the detection and tracking area at runtime
/// </summary>
public abstract class SmartTerrainTracker : Tracker
{
    #region  PUBLIC_METHODS
  
    /// <summary>
    /// Set the Trackable that is used to initialize Smart Terrain.
    /// Use occluderMin and occluderMax to specify an occluder box that will serve as a
    /// mask for smart terrain plane finding.
    /// scaleToMillimenters is a scale factor that defines how the initialization
    /// target/occluder dimensions need to be scaled to be in real world millimeters.
    /// </summary>
    public abstract bool SetInitializationTarget(CylinderTarget cylinderTarget, Vector3
        occluderMin, Vector3 occluderMax, float scaleToMillimenters);
  
    /// <summary>
    /// Set the Trackable that is used to initialize Smart Terrain.
    /// Use occluderMin and occluderMax to specify an occluder box that will serve as a mask
    /// for smart terrain plane finding.
    /// scaleToMillimenters is a scale factor that defines how the initialization
    /// target/occluder dimensions need to be scaled to be in real world millimeters.
    /// offsetToOccluderOrigin and rotationToOccluderOrigin define a translational offset and
    /// rotation of the occluder volume to the initialization target.
    /// </summary>
    public abstract bool SetInitializationTarget(CylinderTarget cylinderTarget, Vector3
        occluderMin, Vector3 occluderMax, float scaleToMillimenters, Vector3
        offsetToOccluderOrigin, Quaternion rotationToOccluderOrigin);
  
    /// <summary>
    /// Set the Trackable that is used to initialize Smart Terrain.
    /// Use occluderMin and occluderMax to specify an occluder box that will serve as a mask
    /// for smart terrain plane finding.
    /// scaleToMillimenters is a scale factor that defines how the initialization
    /// target/occluder dimensions need to be scaled to be in real world millimeters.
    /// </summary>
    public abstract bool SetInitializationTarget(ImageTarget imageTarget, Vector3 occluderMin,
        Vector3 occluderMax, float scaleToMillimenters);
  
    /// <summary>
    /// Set the Trackable that is used to initialize Smart Terrain.
    /// Use occluderMin and occluderMax to specify an occluder box that will serve as a mask            
    /// for smart terrain plane finding.
    /// scaleToMillimenters is a scale factor that defines how the initialization
    /// target/occluder dimensions need to be scaled to be in real world millimeters.
    /// offsetToOccluderOrigin and rotationToOccluderOrigin define a translational offset and
    /// rotation of the occluder volume to the initialization target.
    /// </summary>
    public abstract bool SetInitializationTarget(ImageTarget imageTarget, Vector3 occluderMin,
        Vector3 occluderMax, float scaleToMillimenters, Vector3 offsetToOccluderOrigin,
        Quaternion rotationToOccluderOrigin);
  
    /// <summary>
    /// Set the Trackable that is used to initialize Smart Terrain.
    /// Use occluderMin and occluderMax to specify an occluder box that will serve as a mask
    /// for smart terrain plane finding.
    /// scaleToMillimenters is a scale factor that defines how the initialization
    /// target/occluder dimensions need to be scaled to be in real world millimeters.
    /// </summary>
    public abstract bool SetInitializationTarget(MultiTarget multiTarget, Vector3 occluderMin,
        Vector3 occluderMax, float scaleToMillimenters);
  
    /// <summary>
    /// Set the Trackable that is used to initialize Smart Terrain.
    /// Use occluderMin and occluderMax to specify an occluder box that will serve as a mask
    /// for smart terrain plane finding.
    /// scaleToMillimenters is a scale factor that defines how the initialization
    /// target/occluder dimensions need to be scaled to be in real world millimeters.
    /// offsetToOccluderOrigin and rotationToOccluderOrigin define a translational offset and
    /// rotation of the occluder volume to the initialization target.
    /// </summary>
    public abstract bool SetInitializationTarget(MultiTarget multiTarget, Vector3 occluderMin,
        Vector3 occluderMax, float scaleToMillimenters, Vector3 offsetToOccluderOrigin,
        Quaternion rotationToOccluderOrigin);
  
    /// <summary>
    /// Returns the trackable used for initialization plus additional initialization data.
    /// Returns null if no initialization target has been defined.
    /// </summary>
    public abstract Trackable GetInitializationTarget(out Vector3 occluderMin, out Vector3
        occluderMax, out float scaleToMillimenters);
  
    /// <summary>
    /// Returns the trackable used for initialization plus additional initialization data,
    /// including the occluder offset if defined
    /// Returns null if no initialization target has been defined.
    /// </summary>
    public abstract Trackable GetInitializationTarget(out Vector3 occluderMin, out Vector3
        occluderMax, out float scaleToMillimenters,                                                  
        out Vector3 offsetToOccluderOrigin, out Quaternion rotationToOccluderOrigin);
  
    /// <summary>
    /// Set the maximum extent of the smart terrain in scene units. The maximum area is
    /// defined as rectangle around the center of smart terrain.
    /// </summary>
    /// <returns></returns>
    public abstract bool SetMaximumArea(Rect maximumArea);
  
    /// <summary>
    /// Get the maximum extent of the smart terrain in scene units.
    /// Returns false if no maximum extent has been defined.
    /// </summary>
    public abstract bool GetMaximumArea(out Rect rect);
  
    /// <summary>
    /// Stop mesh updates of smart terrain, but continue tracking.
    /// </summary>
    public abstract bool StopMeshUpdates();
  
    /// <summary>
    /// Start mesh updates, i.e. new objects will be created and existing objects will be
    /// refined.
    /// Mesh updates are enabled by default when initializing the smart terrain tracker.
    /// </summary>
    public abstract bool StartMeshUpdates();
  
    /// <summary>
    /// Returns true if the surface and props are being updated.
    /// </summary>
    public abstract bool IsMeshUpdating();
  
    /// <summary>
    /// Define how much the SmartTerrain ground plane mesh is diminished for the NavMesh
    /// representation.
    /// </summary>
    public abstract void SetNavMeshPadding(float padding);
  
    /// <summary>
    /// Returns the Nav mesh padding previously set by SetNavMeshPadding. Defaults to 0.
    /// </summary>
    public abstract float NavMeshPadding { get; }
  
    /// <summary>
    /// Start creating a nav mesh representation on the primary surface.
    /// By default, nav mesh updates are disabled.
    /// </summary>
    public abstract void StartNavMeshUpdates();
  
    /// <summary>
    /// Stop updating the nav mesh, but continue updating the surface and props and keep
    /// tracking.
    /// </summary>
    public abstract void StopNavMeshUpdates();
  
    /// <summary>
    /// Returns true if the nav mesh is being updated.
    /// </summary>
    public abstract bool IsNavMeshUpdating();
  
    /// <summary>
    /// Delete the created meshes for the ground plane and smart terrain objects
    /// </summary>
    public abstract bool Reset();
  
    #endregion // PUBLIC_METHODS
}

Smart Terrain Manager API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
/// <summary>
/// This class is used to manage all smart terrain behaviours, i.e. primary surface and props on top
/// </summary>
public abstract class SmartTerrainManager
{
    #region PUBLIC_METHODS
  
    /// <summary>
    /// Associate a Prop trackable with a behaviour.
    /// The template PropBehaviour duplicated and associated with the given prop
    /// identified with its property SmartTerrainObjectName.
    /// Association will fail if the Prop has already been associated with another behaviour
    /// or if the template is not a child object of the root primary plane of the smart terrain behaviour
    /// </summary>
    /// <returns>null if association was not successful</returns>
    public abstract PropAbstractBehaviour AssociateProp(PropAbstractBehaviour templateBehaviour, Prop
        newProp);
  
    /// <summary>
    /// Returns active smart terrain or null if no smart terrain is active
    /// </summary>
    public abstract SmartTerrainAbstractBehaviour GetActiveSmartTerrain();
  
    /// <summary>
    /// Returns all active Props
    /// </summary>
    public abstract IEnumerable<Prop> GetActiveProps();
  
    /// <summary>
    /// Get the behaviour that is associated with a currently tracked prop
    /// </summary>
    /// <param name="prop">trackable</param>
    /// <param name="behaviour">resulting behaviour, might be null if specified prop is not associated to
    /// a behaviour</param>
    /// <returns>returns true if behaviour exists for specified trackable</returns>
    public abstract bool TryGetPropBehaviour(Prop prop, out PropAbstractBehaviour behaviour);
  
    #endregion // PUBLIC_METHODS
}

How To Use a Navmesh with Smart Terrain

The primary surface mesh of the Smart Terrain can be used to generate a navigation mesh at run time using the A* Pathfinding Project. To integrate A* to a Smart Terrain project, follow these steps:

  1. Set up and configure the Smart Terrain scene.
  2. Select Create Nav Mesh in the Smart Terrain Behaviour component of your Smart Terrain prefab instance in the Inspector.
  3. Import the A* extension into your Vuforia project. (Find the A* extension in the Unity Asset Store).
  4. Add an empty GameObject to the scene and name it A*.
  5. Add the AstarPath component to the A* GameObject (Components, Pathfinding, then Pathfinder).
  6. In the GameObject’s inspector, under Graphs, select Add New Graph.
  7. Make sure the value of the new NavMeshGraph’s Source Mesh is None.
  8. Disable the Default Smart Terrain Event Handler (Script) on the Smart Terrain instance in the Inspector.
  9. Add the following Update Nav Mesh Event Handler Script to your Smart Terrain instance:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
using System;
using System.Diagnostics;
using Pathfinding;
using UnityEngine;
using Vuforia;
using Debug = UnityEngine.Debug;
  
///TITLE: Update Nav Mesh Event Handler script
/// <summary>
///  A custom handler that implements the ITrackerEventHandler interface.
/// </summary>
public class UpdateNavMeshEventHandler : MonoBehaviour, ISmartTerrainEventHandler
{
    #region PUBLIC_MEMBERS
  
    public PropBehaviour PropTemplate;
    // how often the nav mesh should be updated (in seconds)
    // set to 0 to immediately update when new mesh is available
    public float NavMeshUpdateIntervalSec;
  
    #endregion // PUBLIC_MEMBERS
  
    #region PRIVATE_MEMBERS
  
    private NavMeshWireframeRenderer mNavMeshWireframeRenderer;
  
    private AstarPath mAstar;
    private Mesh mNavMeshToUpdate;
    private DateTime mLastNavMeshUpdate;
    private long mLastMeshBuildingTime = -1;
  
    #endregion // PRIVATE_MEMBERS
  
    #region UNTIY_MONOBEHAVIOUR_METHODS
  
    void Start()
    {
        SmartTerrainBehaviour behaviour = GetComponent<SmartTerrainBehaviour>();
        if (behaviour)
        {
            behaviour.RegisterSmartTerrainEventHandler(this);
        }
  
        mNavMeshWireframeRenderer = GetComponentInChildren<NavMeshWireframeRenderer>();
  
        mAstar = FindObjectOfType(typeof(AstarPath)) as AstarPath;
        if (mAstar != null)
            mAstar.gameObject.SetActive(false);
  
  
        mLastNavMeshUpdate = DateTime.Now;
    }
          
void Update()
    {
        if (mNavMeshToUpdate != null)
        {
            // check if it's time to update the nav mesh
            if ((DateTime.Now - mLastNavMeshUpdate).Seconds > NavMeshUpdateIntervalSec)
                UpdateNavMesh();
        }
    }
  
    #endregion // UNTIY_MONOBEHAVIOUR_METHODS
  
  
  
    #region PUBLIC_METHODS
  
    /// <summary>
    /// Implementation of the ITrackerEventHandler function called after
    /// Vuforia has been initialized completely
    /// </summary>
    public void OnInitialized(SmartTerrainInitializationInfo initializationInfo)
    {
        Debug.Log("Finished initializing");
    }
  
    public void OnSurfaceUpdated(SurfaceAbstractBehaviour surfaceBehaviour)
    {
        // remember the nav mesh
        mNavMeshToUpdate = surfaceBehaviour.Surface.GetNavMesh();
        // only update immediately if the interval is set to 0
        if (NavMeshUpdateIntervalSec <= 0f)
            UpdateNavMesh();
    }
  
    public void OnPropCreated(Prop prop)
    {
        Debug.Log("---Created Prop");
        var manager = TrackerManager.Instance.GetStateManager().GetSmartTerrainManager();
  
        manager.AssociateProp(PropTemplate, prop);
    }
  
    public void OnPropUpdated(Prop prop)
    {
        Debug.Log("---Updated Prop");
    }
  
    public void OnPropDeleted(Prop prop)
    {
        Debug.Log("---Deleted Prop");
    }
  
    #endregion // PUBLIC_METHODS
  
  
  
    #region PRIVATE_METHODS
  
    public void UpdateNavMesh()
    {
        if (mNavMeshToUpdate != null)
        {
  
  
            if (mAstar != null)
            {
                // turn astar on now
                if (!mAstar.gameObject.activeSelf)
                    mAstar.gameObject.SetActive(true);
  
                foreach (NavGraph navGraph in mAstar.graphs)
                {
                    if (navGraph is NavMeshGraph)
                    {
                        NavMeshGraph navMeshGraph = navGraph as NavMeshGraph;
                        // set the source mesh
                        navMeshGraph.sourceMesh = mNavMeshToUpdate;
  
                        Debug.Log("Nodes = "+ navMeshGraph.CountNodes() );
                    }
                }
                // tell astar to rescan
                mAstar.Scan();
            }
  
            // update the wireframerenderer
            if (mNavMeshWireframeRenderer != null)
                mNavMeshWireframeRenderer.SetMeshToRender(mNavMeshToUpdate);
  
              
            mNavMeshToUpdate = null;
            mLastNavMeshUpdate = DateTime.Now;
        }
    }
  
    #endregion // PRIVATE_METHODS
}

The Smart Terrain Inspector shows the components in the figure below, with the Default Smart Terrain Event Handler disabled.

Smart terrain instance
  1. Replace the Wireframe Behaviour on the Primary Surface with this script. This will enable you to visualize the Nav Mesh in the Editor. Disable the Show Lines to hide the mesh from users.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
using UnityEngine;
  
/// <summary>
/// This script renders the mesh from the MeshFilter as wireframe.
/// </summary>
public class NavMeshWireframeRenderer : MonoBehaviour
{
    #region PRIVATE_MEMBERS
  
    private Material mLineMaterial;
    private Mesh mMeshToRender;
  
    #endregion // PRIVATE_MEMBERS
  
  
    #region PUBLIC_MEMBERS
  
    public bool ShowLines = true;
    public Color LineColor = Color.green;
    public float VerticalOffset = 0.1f;
          
    #endregion // PUBLIC_MEMBERS
  
  
    #region PRIVATE_METHODS
  
    private void CreateLineMaterial()
    {
        mLineMaterial = new Material("Shader \"Lines/Colored Blended\" {" +
            "SubShader {" +
                "Pass { Color (" + LineColor.r + "," + LineColor.g + "," + LineColor.b + "," + LineColor.a + ") }" +
            "} }" );
        mLineMaterial.hideFlags = HideFlags.HideAndDontSave;
        mLineMaterial.shader.hideFlags = HideFlags.HideAndDontSave;
    }
  
    #endregion // PRIVATE_METHODS
  
  
    #region UNITY_MONOBEHAVIOUR_METHODS
  
    void OnRenderObject ()
    {
        // avoid lines being rendered in Background-camera
        if (Camera.current != VuforiaManager.Instance.ARCamera)
            return;
  
        if (!ShowLines) return;
        if (mMeshToRender == null) return;
  
        if (mLineMaterial == null)
            CreateLineMaterial();
  
        var vertices = mMeshToRender.vertices;
        var triangles = mMeshToRender.triangles;
      
        GL.PushMatrix();
  
        GL.MultMatrix(transform.localToWorldMatrix);
  
        mLineMaterial.SetPass(0);
        GL.Begin(GL.LINES);
        for (int i=0; i<triangles.Length; i+=3)
        {
            Vector3 P0 = new Vector3(vertices[triangles[i + 0]].x, vertices[triangles[i + 0]].y + VerticalOffset, vertices[triangles[i + 0]].z);
            Vector3 P1 = new Vector3(vertices[triangles[i + 1]].x, vertices[triangles[i + 1]].y + VerticalOffset, vertices[triangles[i + 1]].z);
            Vector3 P2 = new Vector3(vertices[triangles[i + 2]].x, vertices[triangles[i + 2]].y + VerticalOffset, vertices[triangles[i + 2]].z);
              
            GL.Vertex(P0);
            GL.Vertex(P1);
            GL.Vertex(P1);
            GL.Vertex(P2);
            GL.Vertex(P2);
            GL.Vertex(P0);
        }
      
        GL.End();
        GL.PopMatrix();
    }
  
    void OnDrawGizmos()
    {
        if (ShowLines && enabled)
        {
            if (mMeshToRender == null) return;
  
            Gizmos.matrix = Matrix4x4.TRS(gameObject.transform.position, gameObject.transform.rotation, gameObject.transform.lossyScale);
            Gizmos.color = LineColor;
  
            var vertices = mMeshToRender.vertices;
            var triangles = mMeshToRender.triangles;
            for (int i = 0; i < triangles.Length; i += 3)
            {
                Vector3 P0 = new Vector3(vertices[triangles[i + 0]].x, vertices[triangles[i + 0]].y + VerticalOffset, vertices[triangles[i + 0]].z);
                Vector3 P1 = new Vector3(vertices[triangles[i + 1]].x, vertices[triangles[i + 1]].y + VerticalOffset, vertices[triangles[i + 1]].z);
                Vector3 P2 = new Vector3(vertices[triangles[i + 2]].x, vertices[triangles[i + 2]].y + VerticalOffset, vertices[triangles[i + 2]].z);
  
                Gizmos.DrawLine(P0, P1);
                Gizmos.DrawLine(P1, P2);
                Gizmos.DrawLine(P2, P0);
            }
        }
    }
  
    #endregion // UNITY_MONOBEHAVIOUR_METHODS
  
  
  
    #region PUBLIC_METHODS
  
    public void SetMeshToRender(Mesh mesh)
    {
        mMeshToRender = mesh;
    }
  
    #endregion // PUBLIC_MEHODS
}

Your Primary Surface Inspector should now look like this…

Primary Surface
  1. Now you can implement A* Path finding for your characters by following the instructions in the A* documentation at http://arongranberg.com/astar/docs/getstarted.php.
  • Note: The UpdateNavMeshEventHandler scripts prompts A* to repeatedly rescan the Primary Surface mesh, based on the defined update interval. An interval of 0 causes the mesh to be rescanned with each update to the Primary Surface. Adjust your mesh updating interval to suit the performance requirements of your application.
  • Note: The Smart Terrain in Vuforia supports both the free and professional versions of A*. If using the professional version, use the RichAI pathfinding script, which automatically smooths and simplifies paths found in the Nav Mesh.
  • Note: For the free version of A*, we recommend using the Funnel Modifier script with the AIPath component. This combination provides more direct routes between points.
  • Note: See the following screenshot of the components used with a Character Controller running the free version of A*.
Character Controller