One frequently asked question about ArcGIS content is this:
How do I move my published content from one organization to another?
It sounds straightforward, but the answer may not be as simple as it looks, and depends on several factors:
- Is the content moving across tiers?
- What types of content must be moved?
- What types of infrastructure, hardware, or operating systems are involved?
- Is the administrator who is conducting the workflow most comfortable using out-of-the-box tools for the migration, or do they prefer a programmatic workflow?
- Does the administrator who is conducting the workflow have experience using scripting tools to do the migration?
If you’ve explored a migration workflow before, you’ve likely found that some solutions provide only partial migrations for a subset of content, and others are more comprehensive.
But which one is the right one to use for your case? Which one is easiest? Are there caveats that you need to be aware of?
With those questions front and center, let’s dive into the various migration tools and strategies available in ArcGIS: what they are, how to use them, and when each one is most appropriate for the task at hand.
ArcGIS API for Python: clone_items()
ArcGIS API for Python has multiple tools oriented around programmatically transferring content from one organization to another. Perhaps the most used of these tools is clone_items().
Part of the ContentManager class, this function takes in a list of item objects from one ArcGIS organization and re-creates that list elsewhere, in another ArcGIS organization. It has the ability to re-create almost any item type, with the main exception being ArcGIS Hub sites/pages.
The clone_items() function essentially disassembles items internally to determine all their dependencies, and clones the sources for all required items in the new organization, where they are reassembled from the ground up.
The function has numerous features, such as the ability to search a target organization for existent cloned items, the ability to copy service data or rereference the original service, and the option to re-create services from scratch versus having them exported via a file format.
The clone_items() function has been around for a while and, as such, brings some legacy overhead with it. This means that, despite having perhaps the most extensive range of compatible content, there are some incompatibilities that may arise from the evolution of different components and item types in any ArcGIS system.
It is a popular first option for users looking to directly migrate a subset of items, though there are cases where other migration tools may function more effectively. It works for both ArcGIS Online and ArcGIS Enterprise organizations.
- Experience level: Some scripting experience required
- Supported environments: ArcGIS Online and ArcGIS Enterprise
- Accessible via: ArcGIS API for Python, ArcGIS Pro, and ArcGIS Notebooks (through Python)
- Strengths: Great for cloning almost any kind of item, including underlying services and data
- Caveats: Incompatibilities may arise with certain edge cases, often pertaining to feature services
ArcGIS API for Python: OfflineContentManager
Another, much newer migration tool in ArcGIS API for Python is the set of functions found in the OfflineContentManager module.
This module functions similarly to clone_items(), but with some key distinctions. Most notably, it allows you to create offline backups of your migration items, in addition to all their dependencies.
This means that a user trying to migrate items to multiple different organizations can avoid repeated calls to clone_items(), and that users can create backups of their items to use for restoration in the event that something goes wrong.
Another distinction from clone_items() is that OfflineContentManager gives users the option to avoid a rollback on failure. For example, if clone_items() fails in cloning any item in the process, every item from the call will be deleted. However, OfflineContentManager allows users to skip over failed items and carry on migrating the rest of the items.
One final difference is that, by default, this function must copy services by exporting them to a file geodatabase or shapefile and then reuploading and publishing them to the destination organization.
As opposed to clone_items(), the functions in this class are new, so not every item type is covered.
However, these functions do a great job of circumventing some of the pain points of clone_items(), such as incompatibilities when reconstructing services, repeated calls to the same function, and the all-or-nothing nature of clone_items().
This is a great option for users looking for a programmatic solution when clone_items() can’t sufficiently accommodate the workflow. Users who would benefit from this option can use it to create backups and migrate to multiple different new organizations. It is supported for both ArcGIS Online and ArcGIS Enterprise organizations.
- Experience level: Some scripting experience may be required
- Supported environments: ArcGIS Online and ArcGIS Enterprise
- Accessible via: ArcGIS API for Python, ArcGIS Pro, and ArcGIS Notebooks (through Python)
- Strengths: Can create backups; can help when either the clone_items() function or GroupMigrationManager is not compatible
- Caveats: May be time-consuming; not all item types are supported yet
Group Export and Import
Group export and import is another handy option that allows users to export/import offline packages of items, oriented around the migration of a group’s items. This tool is available to migrate content between ArcGIS Enterprise organizations and is extremely user-friendly: Users can use the interface in their ArcGIS Enterprise organization to export and import group content.
As an alternative, this workflow can be done programmatically using ArcGIS REST API endpoints for the export/import functions, in addition to a GroupMigrationManager module in ArcGIS API for Python.
One distinct advantage this tool has over the two previously mentioned functions is the ability to overwrite extant items with the same ID, allowing users to update items based on changes in a different organization without making duplicates in a new organization.
There are some caveats to consider with this option. For example, to migrate content using this method, the staging (import) environment must be on a version of ArcGIS Enterprise that is the same as or later than the development (export) environment.
In addition, item dependencies aren’t automatically handled in the same way that they are with the prior two options. However, if all item dependencies are accounted for, and the migration is between two eligible ArcGIS Enterprise organizations, then export and import through the organization may be the easiest way to migrate content. As mentioned earlier, it’s also great for users looking to overwrite or update their items.
- Experience level: No scripting experience required
- Supported environments: ArcGIS Enterprise
- Accessible via: GUI in ArcGIS Enterprise, ArcGIS REST API, ArcGIS API for Python, ArcGIS Pro, and ArcGIS Notebooks (through Python)
- Strengths: Easily accessible through the organization; can create backups
- Caveats: Not available for ArcGIS Online; not all item types supported
Distributed and Partnered Collaborations
Distributed collaboration and partnered collaboration are two disparate types of content-sharing mechanisms that are very different from those covered thus far. Distributed collaboration enables organizations to share content between ArcGIS Enterprise and ArcGIS Online or between multiple ArcGIS Enterprise deployments. Partnered collaboration, on the other hand, is specific to ArcGIS Online, as it supports the workflow to share content across ArcGIS Online organizations.
Collaborations are not designed to transfer the ownership of content from one organization to another; instead, as the name implies, they allow one organization to maintain ownership of items while sharing them with another organization to view or edit. Collaborations are a great choice to share content in a professional capacity, such as in a provider-client scenario, or to privately share owned apps and data for public consumption.
The terms of a collaboration are defined by the host to specify who the guests are; how many there are; whether guests can send content, receive content, or do both; and more. Collaboration does require a reasonably involved setup process. Administrators can do this through the organization’s settings, or through a script such as one that leverages ArcGIS API for Python.
This workflow is truly a unique way to share, receive, and discover content while preserving ownership in a single organization. It also allows for the opportunity to share private items outside an organization, though the organization must carefully consider what content qualifies for this level of access.
- Experience level: No scripting experience required
- Supported environments: ArcGIS Online and ArcGIS Enterprise
- Accessible via: ArcGIS Online and ArcGIS Enterprise organization settings, ArcGIS REST API, ArcGIS API for Python, ArcGIS Pro, and ArcGIS Notebooks (through Python)
- Strengths: Unique method of sharing and level of accessibility; high level of customization
- Caveats: Geared toward sharing content as opposed to migrating it
WebGIS Disaster Recovery Tool
The WebGIS Disaster Recovery tool, while not specifically designed for content migration or sharing items, is intended as a large-scale option used to protect content within an entire ArcGIS Enterprise organization. This tool enables an administrator to create a backup of an ArcGIS Enterprise organization so that in the unforeseen event of a failure, the backup can be used to restore the organization.
For content migration, this option is different from other technologies covered, though the core concept and goal remains the same: to maximize access and availability of data in ArcGIS. Because this tool was designed to ensure that the data in a specific organization remains available, it is often used to replicate content across environments. This is particularly useful for organizations that want to migrate all organization content to a new operating system, tier, or data center. It may also be used to ensure backups are available for critical apps and data when they are continuously accessed by numerous users or customers.
- Experience level: No scripting experience required
- Supported environments: ArcGIS Enterprise on Windows and Linux
- Available via: Built-in ArcGIS Enterprise tools
- Strengths: Can back up and migrate an entire organization; provides unmatched capability and level of completeness
- Caveats: May require more extensive knowledge of the full ArcGIS Enterprise deployment than other options
Other Tools
The migration tools covered thus far are all oriented around migration of various item types or even entire organizations. However, there are also a handful of assorted item-specific and situation-specific migration tools.
Some of these solve specific problems, while others provide a convenient way to migrate content given a limited scope.
- Export Service and Publish Service Definition: This pair of functions provide a highly convenient way to migrate services while retaining total parity in data and configuration in a new organization. Using the publishing tools in ArcGIS REST API in ArcGIS Enterprise, an administrator can export services as service definition (.sd) files and then republish them in another organization. This maintains the highest level of accuracy with data and configuration relative to any other method of exporting and importing feature services. Note: This feature is available in ArcGIS Enterprise 11.4 and later.
- Migration with ArcGIS Experience Builder (and ArcGIS API for Python): One specific but popular question is: How can I migrate the web experiences from my locally deployed Experience Builder app to my ArcGIS Online or ArcGIS Enterprise organization? ArcGIS API for Python provides a workflow to automate this process. Locally deployed applications may not directly communicate with a portal, so this workflow essentially bridges that gap.
- Sharing API Functions: ArcGIS REST API includes a sharing API to support all portal operations, including exporting, downloading, uploading, and publishing items. These functions work behind the scenes for other migration tools, such as the ArcGIS API for Python methods. These tools provide the most simplified building blocks for migration solutions, for those who wish to build their own custom solution.
Migration Options at a Glance
To help illustrate these options, the flowchart above covers different scenarios and recommendations to migrate your content across ArcGIS Enterprise organizations. For migrations involving an ArcGIS Online organization, ArcGIS API for Python must be used. To follow the options available for ArcGIS Online, start near the center of the flowchart, in the box that says, “Do I want to just move items, or also create backups?” Note that collaborations aren’t included as a workflow in the flowchart, as it is intended to highlight migration options (re-creating content) as opposed to sharing content.
About the authors
Noah Paravicini
