Montgomery County Parks Department Automates Data Integration Workflows with ArcGIS Data Pipelines

Montgomery Parks in Montgomery County, Maryland, is part of the Maryland-National Capital Park and Planning Commission. The staff manages an award-winning parks system that includes 415 parks, over 37,000 acres of parkland, and 280 miles of trails. Montgomery Parks strives to provide residents with a safe place to gather while enjoying the outdoors, by maintaining park infrastructure such as recreational facilities, athletic fields, playgrounds, as well as natural resources like lakes and streams.

The data analytics team at Montgomery Parks tracks these physical assets, environmental resources, and wildlife. They manage the data and web layers that are used in its ArcGIS Online organization to support park management. A web layer is a dataset that can be overlaid on a basemap to display specific information, enhancing staff’s ability to visualize and analyze various aspects of the park’s assets and infrastructure. 

One key web layer the data analytics team is responsible for updating is the park units and assets feature layer. Often, the data analytics team is asked to analyze what political boundaries the parks exist within, among other analyses. Montgomery Parks staff uses the information in this layer to identify what repairs and updates are needed for the parks. Then, they utilize this important dataset to help apply for grants and funding to modernize existing assets and infrastructure. 

Accessing up-to-date political boundaries helps staff identify the best people to contact to get funding assistance, such as county council members, state representatives, or congresspeople. The data analytics team did not have a streamlined process in place to update this layer, and it was previously a lengthy, manual process. As the data analytics team began building a cloud data lakehouse hosted in the Azure cloud for data analysis, the team needed a new solution that would enable them to automate this process and chose ArcGIS Data Pipelines. 

The previous process used by the data analytics team to keep these layers current was time-consuming, explains Carter Hughes, principal data analyst and engineer at Montgomery Parks. When staff requested the latest information, the data analytics team would manually run an analysis workflow to identify the political boundaries that the county parks lie within, which would include local, state, and federal. 

Hughes explains that the process would start with selecting the park units layer, then getting all the political boundaries, running multiple geoprocessing tools in ArcGIS Pro, and then sharing the results. The multistep process took time and was also not automated. Hughes estimates it would take around 1–2 hours with no formal process in place for each request, plus, any method used to update the layers was ad hoc.

In some instances, when the small data analytics team was unable to accommodate requests, staff would perform the analysis themselves and often leverage datasets that had not been updated, affecting the results. 

Hughes began centralizing Montgomery Parks’s enriched GIS datasets in the Azure data lakehouse. Hughes needed a solution to automate the integration of data from its Azure data lakehouse, where its authoritative data from GIS and other systems is stored, with ArcGIS Online, where the organization is familiar with accessing data. Hughes began looking into the capabilities of ArcGIS Data Pipelines, which is a no-code data integration capability in ArcGIS Online that offers a fast and efficient way to ingest, prepare, and maintain data.

Hughes learned more about the new solution during a demo at the 2023 Esri User Conference and knew that it would be a helpful tool. The GIS team deployed the Azure cloud close to a year later for storing critical GIS data, and Hughes says that was when they were ready to implement Data Pipelines. 

“I found out about [Data Pipelines] early, and I knew it was going to be something useful because I knew where we were going. And in a year’s time, we got to that point where we could use it,” says Hughes. 

Hughes liked that Data Pipelines did not require any customization and would be usable right out of the box. He says that having built custom solutions in the past, specifically writing scripts, he knows that managing and maintaining them in the long term is difficult. Also, other features in Data Pipelines would allow them to do more in the future, such as adding more analyses into data preparation workflows. 

According to Hughes, “Data Pipelines [was] the first thing to use, because [I knew] if that works, that can save us a lot of time and effort from building something…ourselves. And I know that Esri is going to keep up with that tool.” 

Hughes adds, “And if I learn [Data Pipelines], I can get what we need in a very easy manner and not have to manage the underlying code.”

Hughes says Data Pipelines is very user-friendly, noting that he got it up and running in about 30 minutes. Prior to creating his first data pipeline, Hughes found some documentation and videos helpful in learning the basics. 

“I really liked how easy it was to use in terms of getting from 0 to 100 very, very quickly,” says Hughes. “I was able to pretty confidently move forward…knowing that…at least the naming of this stuff is pretty familiar, it’s visual, and I can see my data live as I do stuff with it before I save it as a layer.”

For the new workflow, GIS data is pulled in and enriched in the Azure data lakehouse with other business data. Then, Data Pipelines takes outputs from the analysis, processes it further using out-of-the-box data engineering tools, and publishes the GIS-ready data to a feature layer in ArcGIS Online. The data pipelines are scheduled to run overnight twice a week (Monday and Thursday), keeping information up-to-date. 

Chris Weaver, technology coordinator, says, “Individual subject matter experts curate at their levels, and the data pipeline stitches it all together. Gone are the days of endless joins and relates.”

The Montgomery Parks GIS team has been using Data Pipelines since 2024 and currently has three data pipelines scheduled to run weekly, maintaining three important datasets. The team has seen many benefits, including enhanced efficiency and improved data accuracy. Hughes says Data Pipelines has provided them with a lot of value. 

“With Data Pipelines being in the picture, automation has been injected into that workflow so that the challenge of doing that analysis over and over again is not burdensome,” explains Hughes. “And that’s why I really liked using these more automated solutions…because we don’t have to…[sacrifice] a lot of time and resources.”

Keegan Clifford, a GIS coordinator at Montgomery Parks, echoes this sentiment. “The [park units] layer is a game changer for Montgomery Parks GIS,” says Clifford. “It is something that we’ve envisioned for years...the layer provides a wealth of information in just a click of a mouse.”

In total, the data pipelines run in less than an hour overnight at a low-resource rate, saving money and time, according to Hughes. Also, with data being managed automatically, the data analytics team no longer needs to manage as many requests from staff, freeing up time for other tasks. 

“The [park units layer] made possible with Data Pipelines brings value to our team and Montgomery Parks,” says Tiffany Wu, a GIS specialist at Montgomery Parks. “Making these layers available to everyone has not only improved data transparency and data accuracy but saves countless hours of staff time.”

Hughes adds, “Now we have this automation unlocked…I’m focusing much more on the automation aspects and data engineering. All the time it’s saving me is time that I’m reinvesting back into automating more things.” 

Data Pipelines has helped the data analytics team meet its goal of providing better, authoritative data. With updates to feature layers completed weekly, Hughes says he is more confident in the data they’re providing to other departments, and internal staff know data is being updated in a timely manner. 

“They know and have the confidence themselves that if they use that in their maps, it’s going to stay up-to-date…we can trust that it’s going to be accurate,” according to Hughes. “When we make plans to do projects, whether it’s a short-term or a long-term strategic project, they trust that we have the tools to get it done.”

Sharing has greatly improved with Data Pipelines as the Montgomery Parks staff can easily access information in a system they’re already familiar with, ArcGIS Online. Hughes describes it as “another doorway” to getting the data they need. 

Using Data Pipelines has allowed the data analytics team to focus on additional projects for Montgomery Parks. Now that their GIS data integration tasks are automated with Data Pipelines, and authoritative GIS feature layers are more readily available, they can use their time towards creating more maps and apps that can be used to serve the public. They plan to create more data pipelines in the upcoming year. 

“[Data Pipelines has given us] more confidence in using not only GIS but automation and analysis tools because it’s shown what’s possible when we have a long-term vision for what we need to output. Now we’re able to focus on bigger goals [and] we’re thinking proactively, not reactively,” says Hughes.