ArcGIS Enterprise

Dev Summit 2023: Automate image data management workflows

The ArcGIS system is a platform built for automation. GIS organizations using ArcGIS products, such as ArcGIS Enterprise, have access to various tools and systems that can automate preexisting workflows. In his demonstration, Jie shows how an ArcGIS Enterprise organization with ArcGIS Image Server can leverage the functionalities of ArcPy, ArcGIS API for Python, and ArcGIS Notebooks to automate image data management.

Gathering data for the image service

For this demonstration, Jie created an image service published from mosaic datasets. The mosaic datasets reference live, cloud-based weather forecast data for the continental United States. Specifically, the mosaic datasets reference wind forecast data, which contains information such as wind direction and speed values. These values can be interpolated by ArcGIS Enterprise and then visualized using vector symbology. 

Figure 1: An example of how vector symbology is used to visualize wind speed and direction data in ArcGIS Pro.

Jie is referencing forecast data from the NOAA National Digital Forecast Database (NDFD). The NOAA NDFD uses public AWS S3 cloud storage that contains data files of up to 7 days of weather forecast data. These data files are constantly updating, which requires Jie’s image service to update its references to include the newest data. To accomplish this, Jie created two scripts in ArcGIS Notebooks: one to create an image service that references these data files, and another to update those references whenever new data is available. 

Creating the image service

Jie’s first script has three main steps to create his image service. First, his script creates the cloud storge connection file that points to NOAA NDFD’s public S3 bucket. 

Figure 2: An script in ArcGIS Notebooks. This specific step in the script creates a cloud storage configuration file.

Once a connection has been established, his script creates the mosaic dataset that references the wind forecast data by calling the Create Mosaic Dataset and Add Raster to Mosaic Dataset tools. Because Jie’s script created a cloud storage connection file, this step will only reference the data, rather than copying it to ArcGIS Image Server.

Figure 3: A script in ArcGIS Notebooks. This step of the script creates a mosaic dataset that references public weather forecast data.

Finally, the script creates the image service using the ArcGIS API for Python Create Service method. Since his service is referencing live data, the service has the live data flag enabled. 

Figure 4: A script in ArcGIS Notebooks. This step of the script that creates an image service that has live data.

Jie shows us his result imagery layer, which has the most recent 7-day wind forecast data. Now that Jie’s created his image service, he needs to keep its data references up to date. 

Figure 5: A imagery layer showing the most-recent, 7-day wind forecast data.

Updating the image service

Jie’s second script updates his image service by performing two functions:

These steps refresh every forecast data slice’s timestamp and moves the entire forecast window forward one hour. 

Figure 6: A script in ArcGIS Notebooks. This script updates the references in a mosaic dataset to use the most recent forecast data.

Jie also creates a task to run his notebook every hour, which will keep his image service up to date with the latest forecast data. 

Figure 7: A scheduled notebook task. The task runs the notebook every hour.

Conclusion 

Though Jie’s plenary demonstration focused on weather forecasting data, this workflow is applicable to any imagery data. Especially imagery data that needs to be frequently updated to maintain mission critical applications. We hope this demonstration has inspired you to find new automation solutions for your organization’s image data management. 

Did you miss the other features showcased in today’s plenary? Do you want to revisit the demonstrations from our talented coworkers? You can find the links to other companion blogs in our Dev Summit 2023 overview. 

About the author

Tori O hara

Tori is a technical writer on the ArcGIS Enterprise team.

Subscribe
Notify of
0 Comments
Inline Feedbacks
View all comments

Next Article

Basemap Releases Include Over 180 New and Updated Communities

Read this article