Background
ArcGIS Workflow Manager is a scalable workflow management system that acts as the conductor to orchestrate repeatable geographic information system (GIS) and non-GIS operations. It streamlines production processes by automating and simplifying the performance and management of work across environments in an organization. Workflow Manager has been around since the early 2000’s and started as a desktop-based software centered around an enterprise geodatabase to manage system data.
ArcGIS Workflow Manager (Classic) – ArcMap based product
ArcGIS Workflow Manager (Classic) – areas of interest map
A lot has changed since then. We continued to develop our geodatabase-driven product, introducing a server component so users could utilize their workflows on the web. ArcGIS Pro took over as the predominant desktop GIS product for users, and with it, we introduced a Pro Administrator client. However, at 10.8.1, ArcGIS Workflow Manager transitioned to a new, service-driven version in ArcGIS Enterprise, with the geodatabase version becoming known as ArcGIS Workflow Manager (Classic).
Workflow design in the web application
With these latest changes, we’ve made the decision to deprecate the geodatabase-driven product, (Classic). This decision did not come lightly as we realize many of our existing users rely on (Classic) for their daily GIS and non-GIS operations. The goal of this blog is to offer some insight to the major differences between these two versions of Workflow Manager and offer best practices for transitioning to the service-driven product so that you can continue utilizing Workflow Manager in your organization.
Geodatabase driven vs. service driven
What do we mean when we talk about geodatabase driven vs. service driven? In short, we’re talking about the underlying architecture of the product itself. (Classic) was built around an Esri geodatabase. All of the system tables, views, and data were stored in a database management system (DBMS) that was set up and managed by users. Some of the most common examples are SQL Server, Oracle, and PostgreSQL. This setup required users to manage their own databases and handle ongoing maintenance tasks. The geodatabase also required some management by GIS or IT staff as well. On the Workflow Manager side, (Classic) was built to be used in ArcMap with a separate administrator desktop client. Over time, a JavaScript web viewer was created and an ArcGIS Pro version was built, including an administrator client for ArcGIS Pro. However, the underlying architecture has not changed in this time.
Esri products have shifted towards a service first model as users desire the ability to work on the web and with a dispersed workforce. In line with that, Workflow Manager users began asking for functionality in the product that would not be possible in the current (Classic) architecture. So, we began the process of re-thinking how Workflow Manager could better meet the needs of our users moving forward. The end result was our service-driven product, what is now called ArcGIS Workflow Manager. This product was built around the ArcGIS Enterprise ecosystem. Instead of utilizing a 3rd party DBMS and the geodatabase, we utilize the ArcGIS Data Store to host system tables, views, and data. This means less overhead and maintenance for our users on the data side. This data was then made available through hosted feature services and views, meaning that you no longer connected directly to a geodatabase to work, but connected to services instead.
It’s important to note here that both the geodatabase and service-driven products are based on the same underlying concepts: steps are connected to form workflows, which are associated with a job type or template, which is then used to create jobs. However, the seemingly small changes in how you connect to the product have large impacts. Workforces that are dispersed can easily work with the same data; a service first approach means web-based workflows are easy to implement, perform better, and integrate with new and exciting functionality that was not possible before.
Best practices for transitioning to the service-driven product
Now that we’ve briefly covered some of the history and differences between the two products, you’re probably most interested in understanding how to make the transition. We’ve compiled a list of best practices below to help get that process started.
This is a transition, not a migration
You may notice we’re using the word transition here and not migration. Because the underlying architecture between these two products is so different, a direct migration between them would lead to workflows that are not ideal for users. Instead, this new architecture gives our users the ability to rethink their existing organizational workflows in Workflow Manager to make better use of the new technology. Do you have desktop-based geoprocessing tools that could instead be published as geoprocessing services? Do you need to interface with 3rd party systems to update or get information as a workflow progresses? This transition requires the most time to be spent in this part of the process to get the best result.
Think web
This was alluded to in the first bullet but think about how your existing workflows can be changed to better work within a web-based framework. At the same time, consider how your organizational workflows may have changed over time since they were first built in (Classic). If you need to work with 3rd party systems, do they have an API? Could you then use the Send Web Request step in Workflow Manager to call that API to get or send data? Does your organization have processes that require approvals or review from staff that may not be GIS staff or have access to desktop GIS software? Can you utilize a web application instead for them to review and make approvals? These thought processes will go a long way to help your organization make decisions about how and what to transition vs. possibly starting fresh with a new workflow.
Send Web Request step configuration
Integrate desktop workflows
This may seem like a contradiction to the previous think web section, but we haven’t forgotten about your desktop workflows! ArcGIS Pro and the service-driven product work hand-in-hand to allow data editing, running geoprocessing tools, tasks, and more as part of your workflows. Because (Classic) is a desktop centric product, these pieces of your workflows shouldn’t need to change too much moving to the service-driven product. One advantage is the ability to bring in all the new web-based capabilities into Pro and synchronize between Pro and our web application. Desktop editors can run geoprocessing services, send email notifications, and other tasks directly in Pro while other team members can easily track job progress, read comments, and access attachments added in real time in the web application.
Using ArcGIS Workflow Manager in ArcGIS Pro
Consider your backend needs and workflows
Whether you are an existing ArcGIS Enterprise user or are new to ArcGIS Enterprise, you’ll need to consider how Workflow Manager will be set up in your organization. For production environments, we recommend separating Workflow Manager from the hosting server, while test and development environments can be all-in-one, if desired. This helps to prevent users in Workflow Manager from being slowed down by other ArcGIS products in your organization (or vice versa). Do you require high availability so that if one of your Workflow Manager server machines go down, you have another that can pick up the load without interruption? We can support that as well.
Another consideration is the underlying Workflow Manager schema within ArcGIS Data Store. Previously, Geodatabase and Database Administrators worked directly in the database to create database views or modify tables. This will no longer be available. Instead, access to the underlying data is done using API’s and modification of the system schema is not supported.
Custom steps in (Classic)
As a (Classic) user, you may be familiar with the concept of custom steps. These steps offered the ability to do both simple and complex tasks with a range of customizability. While we’ve built functionality in the service-driven product for the majority of these steps, there are some that will require rethinking how they are configured. Take for example the Create Version step. This step exists in the service-driven product with additional functionality not found in (Classic), so this would require little additional work. However, only branch versioned data is supported in the service-driven product so traditional version data would need to be switched to branch versioned.
Create Version step configuration
Another example familiar to (Classic) users is the use of geoprocessing tools. While you can still utilize geoprocessing tools in the service-driven product with our Run GP Tool step in ArcGIS Pro, you also have the option to use our Run GP Service step in either ArcGIS Pro or the web application to run geoprocessing services, something that is not possible in (Classic).
Run GP Service step configuration
On the other side, the Create PDF step in (Classic) does not have a direct replacement in the service-driven product. While there is an Add Attachment step, the first part of the Create PDF step which creates a PDF document of the job’s base map would need to be a custom geoprocessing service instead. While there are some small limitations, the overall end result is that there is a lot of new functionality in the service-driven product that does not exist in the (Classic) custom steps so some time spent doing research in this area will be beneficial. As you start to consider how your existing steps in (Classic) will work in the service-driven product, it may be helpful to refer to the list of steps available in the new product.
Extensibility
If you’ve made use of extensibility options in (Classic), such as the REST API or ArcGIS Maps SDK for JavaScript, you’ll be glad to know we also offer plenty of options in the service-driven product. From the REST and JavaScript APIs to the Python API, you’ll have plenty of options to customize Workflow Manager to your needs. The ArcGIS Pro SDK is also available and will continue to be developed in future releases of ArcGIS Pro. One consideration here is understanding what will and will not need to be customized in the service-driven product. Tasks, such as sending a web request or running a geoprocessing service, can now be done out of the box where they previously required some customization in (Classic). A recommended approach here is to re-think how your organizational workflows would fit in the service-driven product and then look at extensibility options as needed.
Do you use reports?
While reporting is something on the roadmap for the service-driven product, it isn’t something that is currently implemented. Instead, consider the use of ArcGIS Dashboard for your reporting needs. Dashboards offer immense flexibility in design and functionality and can easily been shared with staff across the hall or across the globe.
Custom dashboard with ArcGIS Workflow Manager
Wrapping Up
When it comes to designing your organizational processes and workflows, knowing what happens, when it happens, and who is involved can be the most time-consuming part. The good news for those of you already using (Classic) is that you’ve got a process already designed. Transitioning to the service-driven Workflow Manager will allow you to rebuild your processes and take advantage of all the new web-based benefits it has to offer.
For more information on the service-driven Workflow Manager, check out our resources blog where you’ll find links to our documentation, other blogs, the ArcGIS Workflow Manager fundamentals learning plan, conference presentation recordings and more. We’re excited to see what you’ll accomplish with ArcGIS Workflow Manager!
Article Discussion: