ArcGIS Living Atlas

Using Live Feed Layers in Analytical Workflows

Have you tried to use near real-time GIS data in your analytical tools, such as tracking a hurricane, assessing areas affected by an earthquake, or mapping flooded areas during a storm? ArcGIS Living Atlas of the World contains a collection of near real-time layers known as Live Feeds. Live Feeds display the latest information available such as active hurricanes, recent earthquakes, or river streamflow forecast. Moreover, these layers can be used in analytical tools, connecting Live Feeds to real-time analysis of current conditions, such as interactively mapping flooded areas for first responders in Austin, Texas.

Utilizing near real-time data in analytical tools often requires the automation of tools and workflows that can run with minimal maintenance or supervision, which can sometimes be challenging. The good news is the development of these tools is easily done using Python. In this blog, I’ll show how to consume Live Feed data from Living Atlas through the ArcGIS REST API and the ArcGIS API for Python using two examples: acquiring river streamflow forecast data and plotting changes in arctic sea ice extent.

Retrieving data from Living Atlas through the ArcGIS REST API

ArcGIS Online items, such as the items in Living Atlas, contain a REST service endpoint which can be leveraged by the ArcGIS REST API. The ArcGIS REST API allows us to interact with the items’ service to query, subset, and retrieve Live Feed data. The Live Feed data returned from the interactive request can be seamlessly included in web apps, workflows and more.

Let’s explore how we can retrieve forecasted river streamflow data from the National Water Model.

Access the REST endpoint

Go to the Living Atlas webpage and search for “National Water Model.”

Living Atlas

Select the “National Water Model (10 Day Forecast).”

National Water Model (10 Day Forecast)

On the item’s overview page, scroll to the bottom. The endpoint is in the URL box at the bottom-right corner. Click on “view” to open it in a new tab.

REST endpoint

The item contains only one layer: “Flow Forecast (cfs)”. Click on it to access the REST endpoint. The webpage displays all the relevant metadata such as extent, time info, and field names. At the bottom, click on “query.”

The query webpage contains boxes to interactively construct a request. Alternatively, we can use the endpoint (i.e. url) https://livefeeds2.arcgis.com/arcgis/rest/services/NFIE/NationalWaterModel_Medium/MapServer/0/query to construct the query string and request the data.

Example: Query forecasted river streamflow data for a single reach of the National Hydrography Dataset (NHD)

We’ll plot the forecasted streamflow for Onion Creek in Austin, Texas. The station ID (i.e. COMID) of Onion Creek in the National Hydrography Dataset (NHD) is 5781733. We can query the forecast for this creek by adding the parameter:

where=egdb.dbo.LargeScale_v2.station_id=5781733

Additionally, we can request (1) to include all fields, (2) order the results by time, (3) return a maximum of 80 records, and (4) to return the data in JSON format.

1. outFields=*
2. orderByFields=egdb.dbo.medium_term_current.timevalue
3. resultRecordCount=80
4. f=pjson

The query url is constructed as follows:

https://livefeeds2.arcgis.com/arcgis/rest/services/NFIE/NationalWaterModel_Medium/MapServer/0/query?
where=egdb.dbo.LargeScale_v2.station_id=5781733&outFields=*&
orderByFields=egdb.dbo.medium_term_current.timevalue&
resultRecordCount=80&f=pjson

The requested data is returned in json format.

JSON data

Note: You can load the results in ArcGIS Pro using the JSON to Features tool and create Line Chart.

Forecasted river streamflow for a single reach

Example: Query forecast streamflow data for an area

We’ll subset the National Water Model (NWM) forecast data for South Austin. We provide the following parameters: (1) the subset area geometry, (2) the geometry type, (3) the spatial relationship, and (4) the time stamp 2019-07-22 09:00:00 (i.e. 1563786000000).

1. geometry={xmin:-10889680,ymin:3524030,xmax:-10873880,ymax:3537115}
2. geometryType=esriGeometryEnvelope
3. spatialRel=esriSpatialRelIntersects
4. time=1563786000000

The query url is:

https://livefeeds2.arcgis.com/arcgis/rest/services/NFIE/NationalWaterModel_Medium/MapServer/0/query?
where=&
time=1563786000000&geometry=xmin:-10889680,ymin:3524030,xmax:-10873880,ymax:3537115}&
geometryType=esriGeometryEnvelope&
spatialRel=esriSpatialRelIntersects&
outFields=*&
resultRecordCount=50&
f=pjson

Note: We are requesting data from a Live Feed layer in which the Time Extent changes frequently. The time selected in this blog post won’t be available after the hosted layer is updated. You can select a different time that falls within the current Time Extent. The current Time Extent and Time Interval can be checked in the REST endpoint and also in the Return Updates operation.

Time

The requested results for the forecast time of 2019-07-22 09:00:00 at South Austin are shown next.

NWM South Austin

Note: You can quickly transform time expressions from datetime variables to milliseconds from 1970-01-01 and vice versa as follows:

Time transformations

Note: The ARGIS REST API documentation includes a complete description on how to query a feature service.

Accessing Living Atlas data using the ArcGIS API for Python

Living Atlas items are also available through the ArcGIS API for Python. The layers can be queried, plotted, and used in analytical workflows. In addition, we can leverage the ArcGIS API for Python through ArcGIS Notebooks.

Let’s examine how Arctic Sea Ice Extent has changed over time.

Example: Plot the changes in arctic sea ice extent in September

Open an ArcGIS Notebook in ArcGIS Enterprise or in your desktop.

Load the required modules.

Search for the ‘Arctic Sea Ice Extent’ item.

Select the first layer and query the data as a DataFrame.

Visualize the structure of the data set by plotting the top rows.

Subset the DataFrame by selecting only the months of September for all years.

Plot all months of September in the data set.

The python script used in the example will query all the monthly data in the Arctic Sea Ice Extent feature service. As soon as new monthly values are added to the service, those can be queried and plotted with the same python script.

Summary

The ArcGIS Living Atlas of the World includes Live Feed layers that can be queried, subset, and retrieved systematically in analytical workflows using the ArcGIS REST API and the ArcGIS API for Python.

Related Learn ArcGIS resources

More Information?

Join GeoNet and ask a question to our community of experts.

 

About the author

I’m an expert on the integration of global and large data sets into applied research, especially in the fields of water resources and environmental engineering. I’m part of the Living Atlas of the World team. I’m strongly committed to the improvement of workflows through automation of routines. I have a keen interest in Web GIS, data science, and education.

Connect:

Next Article

StoryMaps Live! Join us for a new webinar series

Read this article