It never takes long for a GIS practitioner to realize that data is the foundation of their work. Whether you’re a user who started out by working with mylar map overlays or someone who is just getting started with ArcGIS Pro (I suppose you could be both), data is crucial for successful GIS workflows.
You want to make a map? Where is the data?
You want to figure out the best location for that new school? What data do you need?
You want to know who will be most impacted by a major flooding event?
You get the picture. Whether you’re creating beautiful, printed maps, pushing web apps to emergency workers’ mobile devices, or conducting an in-depth analysis to inform important decisions, you need accurate, current and high-quality data. And we all know how easy that is, right?!
When somebody emails you a .csv that you add to your map containing data from any number of other sources and formats, do you sometimes feel like that person up there in the image?
Enter data engineering in ArcGIS Pro. While data engineering isn’t new (it was introduced in ArcGIS Pro 2.8), we wanted to resurface it for new and old Pro users alike to make sure you’re getting the most out of your data!
What is data engineering?
Data Engineering is a view in ArcGIS Pro that allows you to explore, visualize, clean and prepare your data. The data engineering process is a common first step for many spatial analysis and mapping workflows. The Data Engineering view and ribbon can help you better understand your data and prepare it for GIS workflows.
How can data engineering facilitate my work?
You don’t have to be a data scientist to use data engineering in ArcGIS Pro. In fact, here are some very common questions that can immediately be answered by popping open the data engineering view:
- How many fields are there in this dataset?
Open the Data Engineering view and find the Field Count in the lower left.
- Does this field contain null values?
Calculate statistics for a field or fields. You’ll immediately see the number (and percent) of null values for the field(s). Right-click to select nulls making it easy to continue work in the map and attribute table to populate the values as needed or launch the Fill Missing Values tool.
- What is the sum of values for these selected features?
Another quick statistics calculation will get you that sum.
- What is the average value for all records (or a selection of records)?
You guessed it! The Data Engineering view contains not only the mean, but also median, mode, least common value and a number of other statistics.
Some of my personal favorite functionality is the ability to filter and search fields to surface just what you’re looking for.
That is just a taste of what data engineering in ArcGIS Pro can do for you. See the Additional Resources section below to take a deeper dive.
Data Engineering at the User Conference
If you will be at the User Conference in July and are interested in learning more, plan on attending the Data Engineering in ArcGIS: Exploring, Transforming, and Integrating Data workshop or come visit the dedicated Data Engineering kiosk in the Spatial Analytics showcase area. (Use this map https://uc2023.esri.com/eventmaps/map.html)
For ArcMap Users
The Data Engineering view is where you can accomplish your data exploration and preparation workflows in ArcGIS Pro – and go beyond what was possible in ArcMap. Remember that ArcGIS 10.8.2 is the current release of ArcGIS Desktop and will enter Mature Support in March 2024. There are no plans to release an ArcGIS Desktop 10.9, and it is recommended that you migrate to ArcGIS Pro. See Migrate from ArcMap to ArcGIS Pro for more information.
Additional Resources:
Introduction to Data Engineering
Data Engineering video playlist
Data Engineering in ArcGIS Pro tutorial series
Banner photo by Glenn Hansen on Unsplash
Article Discussion: