Use the Apple’s Object Capture API to create 3D scans of objects that can be used to create Model Targets.
While it is possible reconstruct objects from images of any source, we recommend using the sample app on iOS devices with a Lidar sensor for best results and accurate scale of the object.
Requirements
See the hardware and software requirements necessary to run the software and build tools at Apple’s Object Capture PhotogrammetrySession page.
In addition, it’s required to have MacOS 12.0+ and Xcode 13.0+ to run the RealityKit Object Capture framework.
Supported image formats are PNG, JPEG, or HEIC format.
Installation
Get the Command-line macOS app here.
Get the Object Capture app for iOS devices here. Other camera devices and image sources can also be used; but using this app on an iOS device with Lidar sensor should be preferred.
Image Capturing Guide
Images can be taken with the Object Capture sample app for iOS devices or from other sources such as digital, DLSR or mirrorless camera. If you have an iOS device with LiDAR, you can use the sample Capture app to also capture depth data alongside the images; it will help in generating the 3D model in more accurate scale.
Listed here are a few guidelines and practices for capturing the object. For an in depth guide, please see Capturing Photographs for RealityKit Object Capture:
Object Selection
- Select an object that is static – it should not have movable or deformable parts.
- Avoid objects that are highly reflective, transparent, or translucent. Choose instead objects with varying color and with texture.
Image sequence
- Take images in a sequence that sets the adjacent shots with an overlap of 70% or more.
- Each image should contain the entirety of the object.
- Take a minimum of 100 images.
- Set your camera to the highest resolution and a narrow aperture to keep the object in focus and the background blurred.
Image Processing
Consistent camera settings should be kept for all images. Maintain the same focal length, shutter speed, aperture, and ISO throughout the image sequencing.
Use then the PhotogrammetrySession with the sequence of images as input to reconstruct a 3D model from it.
Use the Photogrammetry Command-Line App and see this guide for configuring the generation process.
The Command-line app has the following parameters:
input_folder: The path to the directory where the object pictures are located.output_folder: The path to the output directory where the reconstructed model will be stored.-d: The level of detail (e.g., "reduced, medium, full").-o: The order of the photos (e.g., "unordered" or "sequential").-f: The feature sensitivity (e.g., "normal, high").
The choice of parameters naturally depends on the type of object, therefore, inspect the output 3D model according to the CAD model best practices. In general, the detail can be set to full and the sensitivity to normal.
Example output command
./HelloPhotogrammetry ./Wooden_bike_photos ./Wooden_bike_model -d full -o sequential -f normal
The reconstructed 3D model is reconstructed as OBJ and USDZ format with accompanying texture images and material file.
Model Preparation
Before the 3D model is imported into the Model Target Generator (MTG), it should be cleaned up from any undesired mesh artifacts and its scale should be validated and corrected if necessary.
NOTE: If you have used images from a camera that also captured depth data, the scale will likely be correct to that of the real object, and rescaling will not be necessary. Smaller objects might have a larger scale offset.
Open the OBJ in a mesh editor and inspect the model, and if necessary, adjust it uniformly so that it corresponds to the object. Save the file in one of the formats that the MTG supports.
See the Create a Model Target from a Reconstruction for the final steps.