Create Model Targets from 3D Scans and Images

Model Targets can be created from captures of real-life objects with 3D scanning technology (LiDAR) and/or with photogrammetry using compositions of images. Capture an object in case CAD data of the object is unavailable.

Before heading out to scan and capture objects with your scanning equipment or camera, we would like to highlight different technologies and considerations that we have found produce 3D models that work well for tracking. Note that some solutions require you to manually correct the model’s scale after scanning. Using an incorrectly scaled 3D model will affect the accuracy and tracking robustness of the Model Target.

Apple Object Capture, Metascan, and Polycam have approaches available using images and LiDAR depth data or only the images for reconstruction. Using LiDAR-supported approaches (when possible) is recommendable as it reconstructs the 3D model at accurate scale. All three applications require an iOS device and/or MacOS device.

See the guides here:

  1. Apple Object Capture
  2. Metascan
  3. Polycam

For other platforms, a professional photogrammetry software can be used such as RealityCapture. This software offers many controls throughout the process and delivers good reproduction of the object, just without accurate scale.

See the guide here:

Supported Object Types

Not all objects are suited for being captured in images or scanned with LiDAR support.

In general, objects should be static and rigid. Objects that are reflective, transparent, glossy, smooth, or translucent should be avoided as such surfaces are difficult to reconstruct. Best results for such more challenging objects have been seen with Apple’s Object Capture technology.

If possible, select objects that are matte, with texture, or color. Small objects will be harder to reconstruct, and their scale is more likely to be needing adjustments after processing.

Finally, as a solution, you could cover up, repaint, or remove parts from the object that fail to be captured.

Environment Settings for Capturing Setup

To get the best results when capturing, and if it is possible, select a suitable location and day and time where shadows and sunlight are favorable e.g., a cloudy day and no shadows are cast on the object. If the object can be moved, set up a photo shoot where you can control the lights and position of the object. The object’s surroundings should also be considered as the object should stand out from the background in the images. Note that any shadows, deformities, or visual damages will also be recorded in the 3D model’s mesh and texture.

  • If possible, create a setting with diffuse light (indirect light) but keep the object well-lit.
  • Position the object so that you have some distance between the background and the object.
  • Avoid casting shadows on the object.

Workflow and Best Practices

Capture the object from all sides and at different heights, looping around it or move in arcs over the object. LiDAR scanning will capture an object quicker while image-only approaches require you to take a larger number of photos around the object.

Use then the reconstruction tools to generate a 3D model from the scan. Readying the 3D model for the next step will vary depending on the approach, object, and circumstances it was scanned. See the individual guides for details on model preparation.

Create a Model Target from a Reconstruction

It is common that reconstructed 3D models from photogrammetry are larger than 100MB. Large files directly affect the loading time of a Model Target, and we recommend simplifying the model and reduce the texture resolution to avoid errors during the Model Target setup.

Once a 3D model of the scanned object is processed and freed from any undesirable surrounding mesh, and it realistically resembles the real-life object in scale and texture, it can be used as a Model Target. For a detailed guide on how to use the Model Target Generator (MTG), please refer to the How to Create a Model Target and the Model Target Generator User Guide.

The Model Target Generator automatically detects if the 3D model originates from a 3D scan and will store this information in the dataset files when the Model Target is generated. This helps improve tracking your objects. See Optimizing Model Target Tracking for additional information on use-case specific tracking modes.

Model Targets with textures from a scan

When scanning a physical object, it is strongly recommend to include the texture if it accurately resembles the object. See Model Targets Supported Objects and CAD Model Best Practices for detailed information on using textures. If the model is imported with textures, the Realistic Appearance option is automatically selected.

Using inaccurate textures that introduce shadows or incorrect surface colors can negatively affect tracking. Advanced Model Targets that are trained for i.e., 360-degree detection are affected by inaccurate textures. If you choose to create an Advanced Model Target with the texture, make sure to select Realistic model appearance in the Coloring tab of the MTG. Finish by proceeding to load your Model Target in the Model Target Test App to test its tracking robustness.