The Southern California Association of Governments (SCAG) is the largest metropolitan planning organization in the United States in terms of land and population size. Much of the agency’s work supports local jurisdictions throughout Southern California—a region that consists of 191 cities and six counties; encompasses more than 38,000 square miles; and is home to nearly half of the state’s 39 million residents.
SCAG’s main purpose is to produce Connect SoCal, the long-range Regional Transportation Plan and Sustainable Communities Strategy that outlines how to help Southern California grow efficiently and sustainably. To prepare for Connect SoCal, SCAG initiated a local data exchange process to gather the most up-to-date information on topics such as land use and socioeconomic forecasts from local jurisdictions. This data helps SCAG better understand how the region is developing and meeting its targets in terms of mobility, housing, and the environment.
As a longtime Esri customer, SCAG sees the value in taking a geographic approach to tackling planning challenges. To modernize the organization’s current data contribution process and increase efficiency in data collection, SCAG deployed ArcGIS Workflow Manager. Now, SCAG can automate communications and has more flexibility with sharing data, which enables the agency to better manage and track contributions from local jurisdictions.
A Challenging Data Management System
The SCAG Regional Data Platform (RDP) is a system for collaborative data sharing to enable better planning at the city, county, and regional levels. One of the goals of the RDP is to provide the organization’s regionally significant datasets and resources to local jurisdictions to facilitate transparency and collaboration throughout Southern California.
An important part of this initiative is the exchange of data that takes place every four years when SCAG connects with local jurisdictions. Jurisdictions are asked to provide feedback on an estimated 30 data layers in six different categories that primarily deal with land use and socioeconomics. The jurisdictions’ input is then incorporated into Connect SoCal and SCAG’s database.
Due to a historical lack of standardized processes, however, data from each jurisdiction was typically sent to SCAG in different formats, such as marked-up maps or spreadsheets. Most of this feedback came in via mail and email.
According to Tom Vo, principal regional planner at SCAG, it was challenging to manage jurisdictions’ input because each jurisdiction has a different set of land-use data with its own land-use codes. When SCAG received the data, staff had to translate it into standardized SCAG codes.
“Before, in my opinion, the process of collecting data took a little bit longer,” said Vo. “When we received input, we would have staff keep track [of it] and put [it] into the system.”
With large volumes of data going back and forth between SCAG and local jurisdictions, the SCAG team sought a new enterprise solution to standardize the input process.
Establishing a Streamlined Data Exchange
Because SCAG’s RDP was built on Esri infrastructure, Esri Professional Services suggested that the team implement Workflow Manager, Esri’s scalable, GIS-based workflow management system. Staff members on the SCAG planning team are heavy users of ArcGIS Pro, so integrating Workflow Manager into SCAG’s technology stack would be seamless.
“After looking at what planning wanted to do with the RDP, it became obvious that we would have a lot of processes that would require consolidation…and people being able to look back and know what they did,” said Sean Tucker, lead GIS administrator at SCAG. “Workflow Manager was a good choice to have because it works with the other Esri elements, and we would have that collaboration.”
Regional planners, including Vo, outlined what they wanted to get out of Workflow Manager, and then SCAG collaborated with Esri on the initial implementation. Because of the work SCAG does for the region, the team wanted the inaugural setup to be the industry standard.
“I think that was part of the reason why we really leaned on Esri,” Tucker said. “[Esri has] the understanding of…how to collect data, how to collaborate with data, how to make sure that there’s continuity.”
SCAG established its local information services team to provide technical assistance and information to jurisdictions in the region that wanted to learn how to use Workflow Manager, as well as tools and resources from SCAG’s RDP. This has helped streamline the input collection process, according to Vo.
“We get inputs from many different formats,” he said. “Workflow Manager is one of them, and we are trying our best to encourage our jurisdictions to use Workflow Manager integrated in [the] Regional Data Platform because it…has a better management system.”
The Details Get Easier to See
SCAG now receives jurisdictions’ input on data layers via different methods that are powered by Workflow Manager, including through the organization’s Local Data Exchange (LDX) web app and geospatial file uploads. This enables SCAG to have standard data input and review processes, regardless of which method local jurisdictions use to provide their feedback.
The SCAG team currently uses Workflow Manager in ArcGIS Pro to view the details of specific jobs and see the types of edits users have made to existing datasets. For example, if a user adds code, Workflow Manager highlights the addition so that the team can easily see it.
Workflow Manager also offers APIs for extensibility. The team is exploring using Python scripting to develop an alternate way to view job details and review submissions. Recently, the group employed ArcGIS API for Python and ArcGIS REST API to generate job details, contributing to more comprehensive tracking. APIs are also being used to extract the difference between a database version created by a user and the default version of the database.
“This way, it doesn’t bring us the entire database,” explained Mengdi Li, a senior regional planner at SCAG. “The dataset is very compact and easy to handle.”
Workflow Manager helps SCAG with system management as well. Using Python scripting, the team can see where data comes from. Workflow Manager generates a summary table of all data contribution jobs in the system, and that can be converted into statistics, such as the percentage of jurisdictions that have provided input. This helps streamline various internal tracking and reporting tasks.
“Because we have limited resources right now, I saw opportunities [to use] Python to streamline the process. In our GIS team, we only have several people, but we have hundreds of jobs to deal with. So how to batch process jobs is very important to us,” said Li. “We kind of rely on those ArcGIS APIs to help us get access to all the information and do the spatial analysis.”
Enhancing Trust with Local Partners
Implementing Workflow Manager has streamlined the input process for SCAG, leading to improved transparency and efficiency.
Now, when an external user submits input, the SCAG team receives an email notification sent by Workflow Manager. This helps team members keep track of submissions and ensure their accuracy. SCAG staff members can view all contributions in ArcGIS Pro and incorporate the appropriate changes.
The new system also enables SCAG staff to more easily track what’s in the system and how it has been incorporated in the database. Moreover, cities and other jurisdictions that use Workflow Manager can track the status of their inputs.
“This will enhance our communication and trust with local partners because now, when they provide data to us, it’s not a black box anymore. They know exactly where their data is going, and they can keep track of that,” said Vo. “That will also increase engagement and help us build consensus with different partners in the region.”
