A fully classified point cloud can support many geospatial workflows. Ground points can be used for terrain modeling in construction, hydrology, and surface analysis. Vegetation classes support forestry and corridor encroachment analysis. Classified objects can be extracted into 3D objects that are foundational in a geospatial digital twin.
Point cloud classification can be challenging as it often requires several automated tools followed by manual cleanup. Some features, such as power lines or transmission towers, may require a deep learning model. Pretrained point cloud classification models are available in ArcGIS Living Atlas of the World, but training a custom model requires labeled data, time, and AI experience.
When imagery and point clouds cover the same area, several AI tools and models can provide another path. If you can identify features of interest within the imagery, then you can use that information to help classify overlapping point clouds. This is especially powerful when the imagery and point cloud are collected at the same time.
Two complementary approaches can help:
- Use a pretrained foundation model, such as a prompt-based segmentation model (e.g. SAM3), to identify features of interest in the imagery.
- Use the Embedding-Based Analysis toolset to classify imagery by comparing semantic similarity.
The first approach uses prompts to find familiar objects in imagery and is covered in several ArcGIS Blogs, like this one on extracting GIS features with text prompts or doing multiresolution object detection with Text SAM. This article focuses on the second approach: using imagery embeddings to find a feature and transfer the result to a point cloud.
Use imagery embeddings to classify a point cloud
Embeddings are numerical descriptions of patterns in data such as imagery or lidar. Features that look similar often have similar embeddings. This makes it possible to find a feature using only a few examples instead of training a custom model.
For more information on embeddings, refer to this ArcGIS Blog by Catherine David: An Introduction to Embeddings for GIS Analysts.
ArcGIS provides several pretrained models that create embeddings from imagery, all available on the Living Atlas. In this workflow, you will create embeddings and select those that represent the target feature, find similar, and then use the result to classify point cloud data.
This method works best for features with a clear visual pattern, such as rail lines, roads, solar panels, or specific roof types. It can reduce labeling and provide a useful starting point without training a point cloud model.
Example: Extract railroad tracks at the Dangermond Preserve
For this example, Esri worked with Wingtra, UC Santa Barbara, and The Nature Conservancy to collect high resolution imagery and lidar for the Jack and Laura Dangermond Preserve near Santa Barbara, California. We processed these data into 2D and 3D geospatial products using ArcGIS Reality. The preserve includes several different classes: coastal cliffs, vegetation (trees and chaparral), buildings, roads (paved and unpaved), solar panels, and a railroad corridor.
Standard ArcGIS tools can classify many ground, building, and vegetation points. The railroad tracks are harder to identify from point cloud shape alone, but they are easy to see in the true orthophoto. This makes them a good candidate for embedding based analysis.
Workflow
1. Prepare the imagery and point cloud
Add the true orthophoto and overlapping LAS dataset to ArcGIS Pro. Confirm that they cover the same area and use compatible projected coordinate systems. If the point cloud already has useful classes, preserve them so only the target points are updated.
2. Generate imagery embeddings
Use a pretrained imagery model to create embeddings from the true orthophoto. Choose an output resolution that keeps the narrow railroad features visible. Each feature in the embeddings output stores a numerical description of the imagery at that location. This description is used by AI models and tools to efficiently find similar features.
3. Find embeddings that are similar to the railroad tracks
Select a few clear examples of the railroad tracks. Avoid shadows, vegetation, crossings, and nearby surfaces when possible. Use the Embedding-Based Analysis Find Similar interactive tool to test out different areas and parameters.
Review the result in several areas. If it includes unrelated surfaces, adjust the threshold and run the analysis again. The goal is a reliable mask of the railroad corridor, not a perfect result on the first attempt. Once satisfied, you can use the Find Similar Features Using Embeddings geoprocessing tool across full the image.
4. Review and merge the result
Once you have the results completed for the entire railroad corridor, perform a visual assessment of the results. If needed, use GIS Editing tools to help aid with any needed cleanup. Once satisfied with the result, perform a Pairwise Dissolve to create one single feature for the entire railroad corridor. This step is optional but will help improve performance during the point cloud classification.
5. Transfer the class to the point cloud
No matter the approach you used (embeddings or prompt-based segmentation), you can easily transfer the resulting feature to improve the point cloud classification by using the Set LAS Class Codes Using Features geoprocessing tool. Assign the new class code for rail – 10 (based on ASPRS Classification) to the LAS Dataset. If needed, you can even apply a buffer directly within the tool to create a cleaner result. The feature supplies the region to apply the new class code to the point cloud.
Review the results
Display the LAS dataset symbolized by class and compare it with the orthophoto. Check road crossings, vegetation edges, shadows, and hidden track sections. You can correct small errors by using the Manual LAS classification tools or by adjusting your embeddings threshold value and retrying.
Why use this approach?
Embedding based analysis connects the visual detail in imagery with the 3D structure of lidar. It is useful when the feature is easy to see in imagery but difficult to separate using height, elevation, or point cloud return information.
Use ArcGIS and the 3D Analyst tools for classifying point clouds for common classes like ground, vegetation, and buildings, then combine with AI tools and models to use embeddings for visually distinct features, or train your own custom point cloud classification models when you need to repeat the workflow across many datasets.
Commenting is not enabled for this article.