Remote sensing has entered a new era of AI.
For years, deep learning models used in Earth observation were typically initialized using computer vision models pretrained on ImageNet – a dataset of everyday objects such as people, animals, furniture, and vehicles. While effective, these models were never designed to understand satellite imagery.
Remote sensing foundation models represent a significant shift. Instead of learning from photographs of people, animals, and vehicles, these models are pretrained on vast collections of satellite and aerial imagery, allowing them to learn the unique spatial, spectral, and contextual patterns of the Earth’s surface.
A key output of these models is embeddings – compact numerical representations that capture the essential characteristics of imagery. Rather than comparing raw pixels, embeddings summarize meaningful geographic patterns that can be reused across many downstream workflows. Similar landscapes, land cover types, or built environments tend to produce similar embeddings, enabling applications such as similarity search, image retrieval, clustering, and predictive modeling.
Many remote sensing foundation models can also be fine-tuned for supervised tasks, providing a stronger starting point than traditional computer vision models while reducing the amount of labeled training data required.
ArcGIS now integrates open-source remote sensing foundation models, allowing users to generate embeddings and fine-tune supported models directly within familiar geospatial workflows.
Understanding Geospatial Embeddings
Before exploring the models, it is important to understand embeddings.
Imagine identifying a solar farm in satellite imagery and wanting to find similar solar farms across large geospatial datasets. Comparing raw pixels would be slow and unreliable due to differences in sensors, seasons, and lighting conditions.
Instead, foundation models convert imagery into embeddings – compact numerical representations that capture meaningful characteristics of a scene.
These representations encode spatial layout, textures, spectral response, and contextual relationships rather than individual pixel values.
As a result, similar real-world features tend to produce similar embeddings even when they appear different at the pixel level due to acquisition conditions.
This enables similarity search across geospatial data instead of pixel-based comparison.
A useful way to think about embeddings is as a geospatial fingerprint – a compact representation that describes the meaningful characteristics of data in a form that can be efficiently compared and searched. For a deeper explanation, refer to the official documentation:
https://doc.esri.com/en/arcgis-pro/latest/help/analysis/embeddings/embeddings-in-arcgis-pro.html
Two Ways to Use Remote Sensing Foundation Models
ArcGIS supports two primary workflows when working with Remote Sensing foundation models:
Generate Embeddings
Users can these models to generate embeddings from imagery. These embeddings summarize the visual and spatial characteristics of imagery and can support a variety of downstream workflows, including similarity search, image retrieval, clustering, and machine learning.
Because embeddings capture semantic information rather than raw pixel values, similar geographic features tend to produce similar representations even when acquired under different conditions.
Fine-Tune Foundation Models
For supervised learning tasks, users can fine-tune supported foundation models using their own labeled data.
Fine-tuning starts from a model that already understands remote sensing imagery, requiring fewer labeled examples and less training time than building models from ImageNet-trained backbones.
These models can be used with ArcGIS Pro deep learning tools and the arcgis.learn Python API.
Remote Sensing Foundation Models Available in ArcGIS
ArcGIS integrates several leading open-source remote sensing foundation models.
TerraMind
TerraMind is a multimodal Earth observation foundation model trained across multiple sensing modalities, including optical and radar imagery.
By learning relationships across different sensors, TerraMind supports workflows where multiple data sources are available or environmental conditions vary.
DOFA (Large)
Dynamic-one-for-all (DOFA) introduces a wavelength-aware transformer architecture that can process imagery from many different sensors without requiring fixed spectral band configurations.
This flexibility makes it particularly valuable for organizations working with heterogeneous multispectral datasets.
Clay (Large)
Clay is a general-purpose Earth observation foundation model designed to produce strong embeddings across multiple remote sensing sensors.
It serves as an effective default model for representation learning across a broad range of Earth observation workflows.
Prithvi EO 2.0
Prithvi EO 2.0 is IBM and NASA’s Earth observation foundation model trained on Harmonized Landsat Sentinel-2 (HLS) imagery.
ArcGIS supports both the 300-million and 600-million parameter versions.
These models use masked autoencoder pretraining and Vision Transformer architectures and can be fine-tuned for many remote sensing applications.
DINOv2 and DINOv3
Although originally developed as general computer vision foundation models, Meta’s DINOv2 and DINOv3 produce exceptionally strong visual embeddings for RGB aerial and satellite imagery. DINOv3 in particular was also pretrained on high resolution ortho-rectified satellite imagery.
These DINO models are particularly useful for geospatial visual understanding tasks when working with high-resolution RGB imagery.
Integrated with ArcGIS Workflows
Support for these foundation models is integrated directly into ArcGIS Pro and the arcgis.learn API. Several of these remote sensing foundation models can be used as backbones for fine-tuning pixel classification models like DeebLab, as well as for generating embeddings and performing similarity search based on embeddings.
Users can select supported foundation model backbones during deep learning model training, enabling them to build more accurate geospatial models while leveraging the latest advances from the remote sensing research community.
By combining ArcGIS’s deep learning and embeddings-based analysis tools with open-source Earth observation foundation models, organizations gain access to state-of-the-art AI without needing to assemble complex machine learning pipelines themselves.
Looking Ahead
The remote sensing foundation model ecosystem continues to evolve rapidly, with increasingly capable models appearing every year.
By integrating these models into ArcGIS, Esri enables GIS professionals to take advantage of the latest advances in Earth observation AI while continuing to work within familiar geospatial workflows.
Whether generating embeddings for similarity search or fine-tuning models for feature extraction, land cover mapping, or object detection, remote sensing foundation models provide a powerful new starting point for geospatial analysis.
Article Discussion: