ArcGIS

Data Flow - ArcGIS Open Platform Series: Part 1

This 4-part series rounds up the latest features related to data interoperability, enterprise integrations, ArcGIS extensibility, and open source analytics. We’ve looked back through the past several months of blogs and videos and picked out the most important announcements highlighting ArcGIS’s open platform architecture. With the distraction of global pandemics and all the other feature goodies, we wanted to make sure you chief data officers, data scientists, developers, educators, and others didn’t miss these new features and how-to’s from across the ArcGIS platform. Part 1 focuses on how data flows in and out of ArcGIS and open, standards-based interoperability. If you want all the recent updates by product instead, see this Q1 ArcGIS Release Roundup.

Data Flow – Bringing Data In and Sending it Out

With ArcGIS, you can bring virtually any type of data into the platform for analysis and visualization, including data from real-time IoT sensors, satellite navigation systems or specialized drone cameras. And data can be exported from the platform into industry and internationally recognized standard formats. This interoperability gives your organization the flexibility and assurance to grow and adapt without worrying if new projects can integrate with the rest of your organization’s location data.

Ingest Real-Time Data from Internet of Things Sensors – New Product!

With ArcGIS Velocity, you can now ingest and process IoT sensor data feeds directly into ArcGIS Online. This new SaaS extension uses distributed cloud processing to scale tasks, enabling you to ingest, visualize, analyze, store, and act upon massive velocities and volumes of IoT sensor data. You can create feeds and extract data from:

Previously, these capabilities were available in ArcGIS Enterprise for hosting in your own organization’s IT or cloud infrastructure. You can still deploy this option if you need even more messaging or API ingest support. Check out the ArcGIS Geoevent Server SDK documentation on how to write an extensions for custom data real-time data feeds. Learn more about the new, simpler SaaS approach.

Add Data from Kafka using ArcGIS Velocity
Add Data from Kafka using ArcGIS Velocity

Driving BIM Interoperability with International 3D Standards

Building & Information Modeling data is becoming increasingly easy to integrate from architectural design and construction management tools with GIS. Now ArcGIS users can stream 3D BIM data on the web with Building Scene Layers using the open Indexed 3D Scene (I3S) specification v1.7. This approach does not require unnecessarily complex data model transformations and management processes. Esri led the development of this Open Geospatial Consortium (OGC) Community Standard in 2017, which was updated earlier this year with performance improvements and support for Point Cloud Scene Layers (e.g. streaming Lidar). Companies and partners can rest easy knowing they can build tools around this highly performant 3D streaming data approach using an open, standard format.

This standards-based approach complements Esri’s enterprise integration approach with BIM software industry leaders like Autodesk (see more in Part 2 of this series). Learn more about streaming BIM data and the 5 Myths and 5 Realities of BIM-GIS Integration. Learn more about the most recent OGC I3S updates.

Houston International Airport in Web AppBuilder
Data shared with us from George HW Bush Houston International Airport (shown here in Web AppBuilder) demonstrates that if BIM data are standardized, typically through drawing validation tools, then it can be consistently brought into GIS.

Supporting New Sensors

Data flows from all kinds of sensors these days. ArcGIS continually adds native support to those most commonly requested and provides ways to customize to your needs.

Sending Data Out of ArcGIS Insights

ArcGIS Insights, Esri’s premier location analytics product, now let’s you export datasets right from the data pane into open standard formats, such as shapefile, CSV, or GeoJSON. Instead of needing to go back into item pages within ArcGIS Enterprise or ArcGIS Online, you can now more easily then share these datasets with others to use in other applications. Learn more about What’s New with ArcGIS Insights.

Support for OGC Standards Receive Updated Certification

The latest releases to our core platform—ArcGIS Enterprise 10.8, ArcMap 10.8, and ArcGIS Pro 2.5— have received new OGC compliance certificates as part of our ongoing support for OGC standards for data access formats and data services. This includes certificates for OGC Web Map Services (WMS), Web Feature Services (WFS), Web Coverage Services (WCS), OGC Simple Features, Keyhole Markup Language (KML), and Geopackage (GPKG). ArcGIS Online uses the same underlying publishing and client technologies as ArcGIS Enterprise and therefore are also compliant. While many OGC standards do not have associated compliance tests, our products are certified in 376 OGC standard implementations across ArcGIS product versions. A full listing of Esri products implementing OGC standards is available at OGC’s website.

OGC Certificates for ArcGIS

Something not working? Try the Data Interoperability Extension.

What is the Data Interoperability Extension? It’s our extension for ArcGIS Desktop and Server that helps you wrangle data, automate data pipelines and integrate systems in a no-code visual studio. Check out this Q&A to find out more. Here’s a preview:

Q: Data Interoperability is ETL, but I see cloud vendors promoting ELT, do you have comments?

A: Extract Transform & Load (ETL) is where the transformation of data is done before sending it to a system of record, Extract Load & Transform (ELT) is where data is first sent (or already exists) in a system of record and is manipulated within that, for example by using SQL or using a view and/or a SQL mimic language in a JSON store technology.  While Data Interoperability has first class manipulation capability, it also has query connectivity within cloud platforms, letting you work on the data where it is and not hauling it up and down.  Don’t fight data gravity, do what works for you, Data Interoperability lets you do this from within ArcGIS as either ETL or ELT.

Up Next

So that rounds out Part 1. Stay tuned for the rest of the series.

 

Learn more about Esri’s Open Vision.

About the author

Adam Martin

Adam Martin is a Senior Product Manager focused on ArcGIS Knowledge and system interoperability. With more than fifteen years of entrepreneurial, government and private sector experience, he applies business and technology strategies as levers to improve government digital services and public decision making. Based in Washington, DC, he focuses on how ArcGIS connects “all the data” into knowledge graphs to find hidden patterns and relationships, provide quick context and accelerate decision making. He is also a proud dad, metadata hobbyist and consummate starter of things.

Connect:

Commenting is not enabled for this article.

Related Content:

Next Article

Basemap Releases Include Over 300 New and Updated Communities

Read this article