Introduction
Have you ever wanted to be able to seamlessly import ArcGIS Workflow Manager item configurations between different environments, even if they have existing jobs? If you are a seasoned Workflow Manager customer, I bet that answer is a resounding yes! We are excited to say that you can now do this!
Before we go further, I want to note that this blog is a companion to a video on this same topic. If you would prefer to watch that instead of reading this blog (or even in addition to reading this blog), you can access the video here. The blog contains several video clips from the full video to better demonstrate some of the concepts discussed here. Now that we have gotten that out of the way, back to the blog!
We will cover the following topics in this blog. If anything looks interesting and you would like to jump ahead to that section, feel free!
- We will do a quick overview of the functionality that exists now for import and export. (Jump to this section)
- We will talk about what is new and what has changed in these releases. (Jump to this section)
- We will do a deep dive into something called a mapping file and how that works. (Jump to this section)
- We will take a look at how all this looks and functions in the Workflow Manager web application. (Jump to this section)
- We will demonstrate this functionality in Python for automated workflows. (Jump to this section)
- We will do a quick recap of what we have learned in this blog. (Jump to this section)
- And, finally, go over additional resources for Workflow Manager. (Jump to this section)
Let’s jump in!
A Quick Overview of Existing Import and Export Functionality
The ability to export from one Workflow Manager item and import into a different item has been around for a few releases. In its current implementation, you can set up diagrams, job templates, and other settings and export them out of one item and into a different item. This is helpful when, for example, you are setting things up in a test environment and then need to move things to the next level up, such as QA. It can also be helpful if different departments in your organization each use Workflow Manager, and you need to share configurations between them. Lastly, this was also a form of backup and recovery of workflow item configurations. However, for some of our customers, there were limitations with the existing functionality that needed to change for them to fully be able to take advantage of it. They needed the ability to merge workflow updates into an existing workflow item that contained its own set of workflow diagrams and templates.
What’s New
So, what has changed now? Let’s dive into that. There are three big changes for us to cover:
1. You can now import a configuration into a workflow item that has existing jobs.
This will be a game changer for a lot of you. Think about scenarios where you already have a fully functioning production Workflow Manager environment. You now have a need to add a new workflow to meet a new requirement in your business. So, you develop this diagram and job template in a test environment and get everything squared away. Now you are ready to import it into production. Before, this meant that production could not have any existing jobs in that workflow item for the import to succeed. Now, you can export that one diagram and job template out of the test environment and import it into production seamlessly with no impact to your existing jobs in progress.
2. There is a new mapping file that allows you to associate all of the data and groups used in the export environment to the data and groups in the import environment.
What does this mean? Take that example we just went over with exporting from test and importing into production. Each environment may have a different Survey123 survey in it that has the same survey questions between both. Previously, when doing an import, that survey data would have been broken in the Workflow Manager diagram on import because the item IDs for the survey between the two environments were different. With the mapping file, you can set the target ID for the survey ahead of time and so, on import, the diagram now uses the correct item ID and things work as expected. We will go over this in more detail in the next section of the blog.
3. While not specific to merge functionality, we have made a more general update to remove the use of a passphrase for exporting and importing encrypted user defined settings.
This change was made to enhance security in Workflow Manager.
We have covered what is new. Let’s now delve into some of the details and take a look at this in action!
A look at the Mapping File
The mapping file is a key piece of the new merge functionality! Check out the video clip below for more details on what the mapping file is and how to use it.
Web Application Demonstration
Now that we have gotten an understanding of the new mapping file, let’s take a look at how everything works in the Workflow Manager web application. This section is broken out into subsections with video clips for each.
The Export Experience
Learn more about what is new with exporting workflow item configurations in the web application.
The Import Experience
Learn more about what is new with importing workflow item configurations in the web application.
Import a Configuration While Overwriting Data in the Item
This video clip will explain the “Overwrite data in this item” option when importing a configuration.
Import a Configuration Without Overwriting Data in the Item
The video clip will explain what happens when you do not use the “Overwrite data in this item” option when importing a configuration.
Import a Configuration Using a Mapping File
We will now show how an import works when using a mapping file as part of importing a configuration.
Python Demonstration
One of the use cases for taking advantage of the new merge capabilities is the ability to seamlessly implement continuous improvement / continuous development (CI/CD) workflows. That is, the ability to import item configurations between test, QA, and production environments with no downtime and existing work being able to continue as-is.
The introduction of this new merge capability to the Workflow Manager Python API allows you to automate this process to take advantage of scheduled downtimes and maintenance windows for updating environments. The video clip below shows how this works. If you would like to download the sample Jupyter Notebook mentioned in the video, you can access that here.
Recap
We have covered a lot of ground in this blog! We have shown the existing import and export functionality in Workflow Manager, we have gone over the new changes introduced here and how they will enable true merge capability for your organization, giving you an easy way to merge data between different environments. Lastly, we delved into the new mapping file and its components and then went into some demonstrations of the new merge functionality in the web application and Python. This is exciting new functionality and we look forward to hearing your feedback on how it works for you and your organization.
Resources
As always, we welcome you to check out all of our content on workflow manager. From blogs to videos to documentation, you can find it all listed here in our resources blog.
Article Discussion: