The ArcGIS system is a platform built for automation. GIS organizations using ArcGIS products, such as ArcGIS Enterprise, have access to various tools and systems that can automate preexisting workflows. In his demonstration, Jie shows how an ArcGIS Enterprise organization with ArcGIS Image Server can leverage the functionalities of ArcPy, ArcGIS API for Python, and ArcGIS Notebooks to automate image data management.
Gathering data for the image service
For this demonstration, Jie created an image service published from mosaic datasets. The mosaic datasets reference live, cloud-based weather forecast data for the continental United States. Specifically, the mosaic datasets reference wind forecast data, which contains information such as wind direction and speed values. These values can be interpolated by ArcGIS Enterprise and then visualized using vector symbology.
Jie is referencing forecast data from the NOAA National Digital Forecast Database (NDFD). The NOAA NDFD uses public AWS S3 cloud storage that contains data files of up to 7 days of weather forecast data. These data files are constantly updating, which requires Jie’s image service to update its references to include the newest data. To accomplish this, Jie created two scripts in ArcGIS Notebooks: one to create an image service that references these data files, and another to update those references whenever new data is available.
Creating the image service
Jie’s first script has three main steps to create his image service. First, his script creates the cloud storge connection file that points to NOAA NDFD’s public S3 bucket.
Once a connection has been established, his script creates the mosaic dataset that references the wind forecast data by calling the Create Mosaic Dataset and Add Raster to Mosaic Dataset tools. Because Jie’s script created a cloud storage connection file, this step will only reference the data, rather than copying it to ArcGIS Image Server.
Finally, the script creates the image service using the ArcGIS API for Python Create Service method. Since his service is referencing live data, the service has the live data flag enabled.
Jie shows us his result imagery layer, which has the most recent 7-day wind forecast data. Now that Jie’s created his image service, he needs to keep its data references up to date.
Updating the image service
Jie’s second script updates his image service by performing two functions:
- It removes the references to outdated data
- It adds new references to the updated data files from NOAA NDFD back into the mosaic dataset
These steps refresh every forecast data slice’s timestamp and moves the entire forecast window forward one hour.
Jie also creates a task to run his notebook every hour, which will keep his image service up to date with the latest forecast data.
Conclusion
Though Jie’s plenary demonstration focused on weather forecasting data, this workflow is applicable to any imagery data. Especially imagery data that needs to be frequently updated to maintain mission critical applications. We hope this demonstration has inspired you to find new automation solutions for your organization’s image data management.
Did you miss the other features showcased in today’s plenary? Do you want to revisit the demonstrations from our talented coworkers? You can find the links to other companion blogs in our Dev Summit 2023 overview.
About the author
Article Discussion: