ArcGIS Blog

Data Management

ArcGIS

Aug 14, 2026

The Science of Where in the Modern Data Stack: Data Integration Approaches and Methods in ArcGIS

By Sarah Hanson and Nana Dei and Nolan Reichmann

Unlock deeper insights and solve complex business problems with confidence by bringing your data together with the power of ArcGIS. As an enterprise spatial management system, ArcGIS provides a set of robust tools for creating, visualizing, editing, and managing geospatial data. It also offers numerous capabilities that allow users to integrate data from other enterprise systems and external data sources, supporting databases, cloud data warehouses, cloud storage, APIs, real-time feeds, and more.

In this second entry in The Science of Where in the Modern Data Stack blog series, we will detail the common approaches to data integration that can be used to further your knowledge base and inform your design decisions.

Data Integration Method

To jump to a specific method, use these links: 

Manage your authoritative GIS data with enterprise geodatabases  

While this blog is focused on integrating data with ArcGIS, it’s important to highlight the enterprise geodatabase at the start.  

Geodatabases are the native data structure of ArcGIS and provide a foundation for data management, editing, and spatial analysis workflows. Enterprise geodatabases, sometimes referred to as a multiuser geodatabase, offer a comprehensive set of functionalities to ArcGIS Pro and ArcGIS Enterprise users beyond traditional relational databases and include features for modeling, editing, and maintaining GIS datasets. They also support versioning, topology, relationship classes, and other advanced GIS data modeling capabilities. 

They are used by thousands of organizations today as the geospatial system of record for their authoritative GIS datasets, including utility networks, road networks, parcel fabrics, and indoor information models. While the data is stored in the geodatabase, referenced web services can be published from them to ArcGIS Enterprise allowing for web editing, field data collection, and visualization and analytics on the web.  

Enterprise geodatabases deliver specialized functionality for modeling, editing, and maintaining your spatial datasets. What differentiates enterprise geodatabases from file and mobile geodatabases is that they’re designed for multiuser editing workflows, where multiple people can edit the same data simultaneously without interfering with one another, and with any conflicts flagged for review and resolution.  

Data from an enterprise geodatabase can be published as web services to ArcGIS Enterprise either by reference from ArcGIS Pro or using bulk publishing, and be used to support visualization, analytics, and editing via the web and mobile applications.  

While you can publish layers from an enterprise geodatabase to ArcGIS Online, the data is copied during publishing, so data updates will need to be managed through periodic republishing or overwriting of the content in ArcGIS Online. This process can be automated using a Python script, an ArcGIS Data Interoperability flow, or a third-party ETL tool. Publishing data from an enterprise geodatabase as a copy to ArcGIS Enterprise is also possible, but is less common, as data can be shared by reference, maintaining a live connection to the database. 

Best used for: organizations looking for a robust multiuser geodatabase to create, update, and manage their spatial system of record.  

If you are new to ArcGIS and are unfamiliar with what enterprise geodatabases are, the benefits they offer, and how organizations use them to store their GIS system of record, check out the following resources: 

Now that we’ve highlighted integrating data from an enterprise geodatabase, let’s get into the approaches which allow us to integrate data from external sources, that are being maintained outside of ArcGIS, with the platform.

Use query layers to access external databases without moving your data  

Query layers are a layer type in ArcGIS Pro that allows users to define a feature layer or stand-alone table using SQL after a connection to a database has been established. The list of supported databases for query layers includes many commonly used relational databases, cloud data warehouses, and document databases.  

The SQL statement used to define the query layer can filter, aggregate, join, and otherwise shape the data that is returned, leveraging the SQL functions supported by the underlying database. Because the data remains in the source system, updates made to the data are immediately reflected in the layer in ArcGIS Pro.

ArcGIS Enterprise users can also share query layers from ArcGIS Pro as referenced feature layers and map image layers that can be used in web maps, web apps, and other ArcGIS clients. Like query layers in ArcGIS Pro, these web layers maintain a live connection to the underlying database without requiring the data to be moved or copied into ArcGIS.

Screenshot of ArcGIS Pro showing the Share As Web Layer pane, with a world map displaying sea surface temperature data in a hexagon grid overlay.
Query layer in ArcGIS Pro showing average sea surface temperature from a Google BigQuery table with 358 million data points, aggregated into feature bins with the Share as Web Layer pane open.

To share web layers by reference to ArcGIS Enterprise, the source database must first be added to ArcGIS Enterprise as a data store item.

Best used for: visualization and analysis of data stored in external databases and cloud data warehouses. 

Integrate data from: Microsoft SQL Server, Oracle, PostgreSQL, Snowflake, Google BigQuery, Amazon Redshift, SAP HANA, IBM Db2, Teradata, Dameng, Elasticsearch, and OpenSearch.

Learn more about query layers:

In the future, you will be able to create query layers in Map Viewer in ArcGIS Enterprise. We’re also working to expand the list of supported cloud data warehouses in ArcGIS Pro and ArcGIS Enterprise to include Databricks SQL Warehouse.

Note: Query layers that reference data in relational databases can be shared as a copy to ArcGIS Online or ArcGIS Enterprise. However, to share data from a cloud data warehouse or document database as a copy, you must first export it to a geodatabase feature class and then publish the layer.

Connect, transform, and automate data integration with ArcGIS Data Pipelines

ArcGIS Data Pipelines is a web app designed for data preparation and integration that is available in ArcGIS Online and ArcGIS Enterprise. It provides users with a visual diagramming app that makes it fast and easy to build, maintain, and hand off data preparation workflows.

ArcGIS Data Pipeline
A data pipeline workflow that integrates and transforms data from Snowflake to support downstream mapping and analytics.

Connect to input data sources from databases (Snowflake, Google BigQuery, Databricks), cloud storage (Amazon S3 and Microsoft Azure Storage), URLs, files, and ArcGIS feature services. Use tools to build, blend, and transform data, then configure outputs that create or update content for downstream mapping and analytics. Add notes to the canvas to document decisions and enhance usability, and preview data as you engineer it by reviewing the attributes, geometry, and schema of your data.

Because workflows are saved as reusable data pipeline items, they can be shared, updated, and maintained over time, making it easier for teams to standardize recurring integration processes across an organization. 

Leverage built-in scheduling to automate data updates as often as every 15 minutes or run data pipelines as part of broader automation workflows using the ArcGIS REST API, ArcGIS API for Python, or Microsoft Power Automate. 

ArcGIS Data Pipelines is available to all ArcGIS Online organizations and charges credits while in use. In ArcGIS Enterprise, ArcGIS Data Pipelines comes with ArcGIS Enterprise Advanced on Windows and Linux.  

Best used for: integrating data from the cloud and transforming it into an authoritative layer in ArcGIS Online and ArcGIS Enterprise. 

Integrate data from: files and feature layers from your ArcGIS content, Amazon S3, Microsoft Azure Storage, Google BigQuery, Snowflake, Databricks, URLs and APIs, and file shares.  

To learn more about ArcGIS Data Pipelines:

In the future, we’re planning to add connectors to ArcGIS Data Pipelines in ArcGIS Enterprise to Microsoft SQL Server, PostgreSQL, Oracle, and Databricks, and we’re also working to bring ArcGIS Data Pipelines to ArcGIS Enterprise on Kubernetes.

If you’re interested in staying informed about upcoming beta testing opportunities available for ArcGIS Data Pipelines in ArcGIS Enterprise on Kubernetes, please submit your interest via this form.

Handle complex ETL and format conversion with ArcGIS Data Interoperability 

ArcGIS Data Interoperability is an extension for ArcGIS Pro and ArcGIS Enterprise that connects to hundreds of apps and data formats. Built on Safe Software’s FME technology, Data Interoperability extends ArcGIS with hundreds of readers, writers, and transformation tools for spatial and nonspatial data.

Using a visual, no-code authoring environment, users can build reusable workflows that connect to a wide variety of file formats, databases, APIs, cloud platforms, and enterprise systems. Data can be transformed, validated, merged, filtered, enriched, and converted between formats before being loaded into ArcGIS or delivered to downstream systems. 

While ArcGIS Data Pipelines focuses on making common cloud and enterprise integration workflows simple and approachable, ArcGIS Data Interoperability provides the flexibility needed for more specialized requirements such as support for uncommon data formats, sophisticated transformation logic, enterprise data exchange standards, or integrations with systems that are not supported by Data Pipelines. 

FME Workbench shows a complex workflow diagram. Lines connect various boxes and processes, visualizing a data transformation or integration project.
Complex ETL workflow built using the ArcGIS Data Interoperability extension in ArcGIS Pro.

Because of its extensive format support and transformation capabilities, ArcGIS Data Interoperability is often used as the bridge between GIS and enterprise information systems, helping organizations standardize data from multiple sources and deliver it in the format required by downstream users, business systems, and applications. 

Best used for: advanced ETL workflows, format conversion, data migration projects, and integration scenarios involving complex transformation requirements that extend beyond the capabilities of ArcGIS Data Pipelines. 

Integrate data from: hundreds of file formats, databases, APIs, cloud platforms, enterprise systems, and industry-specific data standards. 

To learn more about ArcGIS Data Interoperability:

Integrate real-time data into ArcGIS using ArcGIS Velocity

ArcGIS Velocity is a real-time processing and analytics capability for ArcGIS Online and ArcGIS Enterprise that enables organizations to ingest, visualize, and analyze streaming, sensor, and asset data. By analyzing data as it is received, organizations can monitor operations, detect incidents, and take action immediately using real-time location intelligence.

Using ArcGIS Velocity, organizations can configure real-time feeds to connect to IoT devices, vehicles, equipment, operational systems, and external event streams. Incoming data can be enriched, filtered, and analyzed in real time to detect conditions of interest, trigger alerts, automate workflows, and maintain a common operating picture for situational awareness and decision-making.

Aerial view of flight routes from Charleston on Halloween.
Real-time flight data from FlightAware is streamed into ArcGIS Enterprise using ArcGIS Velocity.

Track and monitor critical assets, power live maps and dashboards with continuously updating information, archive observations for historical analysis, and share real-time insights across the organization. Resulting data can be written to feature and stream layers, or integrated with systems and platforms outside of ArcGIS, such as Kafka, RabbitMQ, Amazon S3, or Microsoft Azure Storage.

Best used for: Organizations seeking a real-time solution for visualization and analytics. 

Integrate data from: vehicles, personnel, IoT sensors, stationary sensors, business systems, event streams, and data providers.

To learn more about ArcGIS Velocity:

In the future, we’re planning to make ArcGIS Velocity available with ArcGIS Enterprise on Kubernetes.

Build custom data integration workflows with ArcGIS Python libraries 

The ArcGIS Python libraries provide developers and GIS professionals with a powerful way to automate data integration, management, and analysis workflows. While tools such as ArcGIS Data Pipelines provide a visual, no-code experience for common integration scenarios, Python offers complete flexibility to create custom workflows, integrate with virtually any system, and orchestrate complex processes across ArcGIS and non-ArcGIS environments. 

Using the ArcGIS API for Python, ArcPy, and third-party Python libraries, organizations can connect to databases, APIs, cloud storage services, business systems, and enterprise applications to extract, transform, validate, and load data into ArcGIS. Python scripts can also automate administrative tasks such as publishing content, updating web layers, managing users, and monitoring system health. 

ArcGIS Notebooks provide a browser-based environment for automating, building, and sharing Python workflows within ArcGIS Online and ArcGIS Enterprise. Notebooks make it easy to maintain a consistent Python environment across teams, reducing setup complexity and making workflows easier to maintain and reproduce.

Code snippets from a notebook. The snippets show how to connect to GIS, add a CSV file, and publish a feature layer.
The ArcGIS API for Python is used to publish a hosted feature layer from a file.

Whether supporting simple scheduled imports or large-scale enterprise integration workflows, Python provides organizations with a highly flexible integration option that can be tailored to their specific requirements. 

Best used for: organizations that need custom automation or integration with systems and processes that are not supported through no-code tools. 

Integrate data from: virtually any source. 

To learn more about the ArcGIS Python libraries:

Extend ArcGIS to virtually any data source with custom data feeds 

With custom data feeds, ArcGIS Enterprise users can connect directly to data sources that aren’t natively supported in ArcGIS otherwise. This is done without migration or duplication and is available to ArcGIS Enterprise deployed on Windows, Linux, and Kubernetes.

Using custom data feeds, a feature service can be created that reads from any source.

Once a custom data provider has been created for the target data source, ArcGIS Enterprise can fetch and read data directly from the external system while the data remains in its original location. A feature service is created in ArcGIS Enterprise that reflects the source data, and depending on how the feed is configured, updates in the source can be surfaced in the service. This feature service is just like any other feature service in ArcGIS and can be used in the client of your choice. This approach unlocks data that would otherwise be inaccessible in ArcGIS workflows.

Custom data feeds are especially valuable when organizations need to integrate data from proprietary systems, internally developed applications, or specialized data platforms that do not have native connectors in ArcGIS. Rather than building custom export processes or duplicating the data, organizations can expose these sources through standard ArcGIS feature services for use throughout the platform. 

Custom data feeds are supported with ArcGIS Enterprise in addition to stand-alone ArcGIS Server sites. To understand how ArcGIS Online and ArcGIS Enterprise or an ArcGIS Server site can be used together, refer to this ArcGIS Blog: ArcGIS Online or ArcGIS Enterprise: You don’t have to choose. 

Best used for: creating a feature service that can access and query data directly from sources not natively supported by ArcGIS. 

Integrate data from: any database, data lake, API, or other location that you’re not able to connect to natively. 

Learn more about custom data feeds:

Bring distributed spatial analytic results into ArcGIS with ArcGIS GeoAnalytics Engine 

ArcGIS GeoAnalytics Engine is a distributed spatial analytics library for Apache Spark that enables organizations to integrate large-scale spatial analysis workflows with ArcGIS. Rather than moving data into ArcGIS before analysis, GeoAnalytics Engine brings ArcGIS spatial capabilities to modern, Spark-based data lake, cloud data warehouse, and big data environments, allowing organizations to perform spatial analysis directly where their data resides. 

Using GeoAnalytics Engine, organizations can read data from cloud object stores, data lakes, cloud data warehouses, distributed file systems, and ArcGIS Online and ArcGIS Enterprise content while taking advantage of Apache Spark’s scalability. Spatial analysis results can then be written directly to ArcGIS Online or ArcGIS Enterprise as hosted feature layers, making them immediately available for mapping, visualization, sharing, and downstream GIS workflows. This allows organizations to operationalize distributed analytics by seamlessly connecting enterprise data platforms with the broader ArcGIS system. 

ArcGIS GeoAnalytics Engine
Run distributed spatial analytics with GeoAnalytics Engine and write results directly to ArcGIS feature services.

GeoAnalytics Engine complements other ArcGIS data integration approaches by extending spatial analysis beyond the GIS itself. Organizations can enrich enterprise data with spatial context, perform distributed spatial joins and aggregations, and publish the resulting datasets back into ArcGIS, where they become authoritative layers that can be shared across applications, dashboards, field workflows, and made accessible to decision-makers. 

Best used for: integrating distributed spatial analytics performed in Apache Spark with connectivity to ArcGIS Online and ArcGIS Enterprise. 

Integrate data from: Apache Spark environments, cloud object stores, data lakes, cloud data warehouses, distributed file systems, and enterprise analytics platforms. 

To learn more about ArcGIS GeoAnalytics Engine:

Every organization has different integration requirements. Some need live access to operational data sources, others need scheduled ETL workflows, real-time streaming data, or highly customized integrations with enterprise systems. ArcGIS provides multiple approaches so organizations can choose the method that best aligns with their architecture, governance requirements, and operational workflows. 

In this second entry in The Science of Where in the Modern Data Stack blog series, we detailed the common approaches available for integrating data with the ArcGIS platform and shared resources for learning more about each of them. These common integration methods can help you whether you’re building a new GIS implementation or modernizing an existing one. Understanding these integration patterns can also help you design a solution that is scalable, maintainable, and aligned with your organization’s broader data strategy. 

If you would like to learn more about integration methods supported by ArcGIS, please visit our Data Management website, join the conversation on the Esri Community, or check out these resources:

If you have any questions, suggestions, or would like to get started on your integration journey with ArcGIS, please reach out to your local Esri representative or connect with us on the Esri Community.

Share this article

data management arcgis data pipelines arcgis velocity data management data stack open platform arcgis arcgis enterprise arcgis geoanalytics engine arcgis online arcgis pro
Sarah Hanson

Sarah is a Product Manager on the ArcGIS Enterprise team, focused on data management. She also manages ArcGIS Data Pipelines, a no-code data engineering app in ArcGIS. In her free time, Sarah loves to explore new places and spend time outdoors.

Nana Dei

Nana is a Product Manager on the ArcGIS Pro team, focusing on the product's data management capabilities. In his free time, he enjoys travelling, playing Scrabble, and shooting pool.

Nolan Reichmann

Nolan Reichmann is a Senior Product Marketing Manager at Esri, focusing on the value of Data Management and ArcGIS. With over a decade of experience in the field of GIS, he is dedicated to driving customer success through the adoption and use of Esri software and technology.

Commenting is not enabled for this article.

Related articles