Publish and overwrite web layers in ModelBuilder

ArcGIS Pro

As we continue the fight against COVID-19 and enter the wildfire and hurricane season, web applications like dashboards help people make informed, time-sensitive, and even life-saving decisions. Depending on the events your organization is monitoring, things can change rapidly. You’ll need to efficiently publish your web layers, update your data, and overwrite your web layers to keep your dashboards current and relevant.

ModelBuilder in ArcGIS Pro allows you to automate analyzing and updating the data used in your web layers. You may already have a model you routinely run to prepare your data for publishing. All that remains is the ability to publish web layers or overwrite existing ones in the model. In this article, you’ll learn how to create a script tool in ArcGIS Pro 2.8 that publishes or overwrites a web feature layer. The script tool is built on ArcPy functions and geoprocessing tools, allowing you to run it like any other geoprocessing tool in new or existing ModelBuilder models.

Use a custom publishing script tool in ModelBuilder

You’ll be guided through each of the four steps below:

Write a Python script. (Not too familiar with Python? Don’t worry; a sample script is provided that you can modify to meet your specific needs.)
Create a script tool.
Add the script tool to ModelBuilder.
Run the model.

If you want to view the completed script tool and see how it works in an existing model, download this project to follow along.

Step 1: Write a Python script

The code below publishes a new web feature layer or overwrites an existing web feature layer in ArcGIS Online or ArcGIS Enterprise. Copy and paste the script in a text editor like Notepad and save it as a Python file (.py). This script can be modified to meet your specific needs. You’ll learn how to do this under Next Steps at the end of this article. If you would like to learn more about this publishing script and others like it, see Introduction to arcpy.sharing.

import arcpy
import os
import xml.dom.minidom as DOM

def ScriptTool(map, service, summary, tags, description, overwriteService, enableEditing, enableSync, enableWFS, timezone, share_public, share_organization, share_groups, outdir):
    """ScriptTool function docstring"""

    # Set output file names
    sddraft_filename = service + ".sddraft"
    sddraft_output_filename = os.path.join(outdir, sddraft_filename)

    # Reference map to publish
    aprx = arcpy.mp.ArcGISProject("CURRENT")
    m = aprx.listMaps(map)[0]

    # Create FeatureSharingDraft and set service properties
    sharing_draft = m.getWebLayerSharingDraft("HOSTING_SERVER", "FEATURE", service)
    sharing_draft.summary = summary
    sharing_draft.tags = tags
    sharing_draft.description = description
    sharing_draft.credits = "My Credits"
    sharing_draft.useLimitations = "My Use Limitations"
    sharing_draft.overwriteExistingService = overwriteService

    # Create Service Definition Draft file
    sharing_draft.exportToSDDraft(sddraft_output_filename)
    outsddraft = sddraft_output_filename
    arcpy.AddMessage("Service definition draft created")

    # Modify capabilities
    if enableEditing or enableSync:
        ModifyCapabilities(sddraft_output_filename, enableEditing, enableSync)
    if enableWFS:
        EnableWFS(sddraft_output_filename)
    
    # Set time zone
    if(timezone != ""):
        property_set = [{
            "key": "dateFieldsRespectsDayLightSavingTime",
            "value": "true"
        },
            {
                "key": "dateFieldsTimezoneID",
                "value": timezone
            }]
        SetTimezone(sddraft_output_filename, property_set=property_set)

    # Create Service Definition file
    sd_filename = service + ".sd"
    sd_output_filename = os.path.join(outdir, sd_filename)
    arcpy.StageService_server(sddraft_output_filename, sd_output_filename)
    arcpy.AddMessage("Service definition created")

    # Upload to portal
    output = arcpy.UploadServiceDefinition_server(sd_output_filename, "My Hosted Services", in_override="OVERRIDE_DEFINITION", in_public=share_public, in_organization=share_organization, in_groups=share_groups)
    arcpy.AddMessage("Service published")

    return output[5]

def ModifyCapabilities(sddraft_output_filename, enableEditing, enableSync):
    capabilities = "Query"
    if enableEditing:
        capabilities += ",Create,Delete,Update,Editing"
    if enableSync:
        capabilities += ",Sync"
    # Modify feature layer capabilities to enable Create and Sync
    doc = DOM.parse(sddraft_output_filename)
    typeNames = doc.getElementsByTagName('TypeName')
    for typeName in typeNames:
        if typeName.firstChild.data == "FeatureServer":
            extension = typeName.parentNode
            for extElement in extension.childNodes:
                if extElement.tagName == 'Definition':
                    for propArray in extElement.childNodes:
                        if propArray.tagName == 'Info':
                            for propSet in propArray.childNodes:
                                for prop in propSet.childNodes:
                                    for prop1 in prop.childNodes:
                                        if prop1.tagName == "Key":
                                            if prop1.firstChild.data == 'webCapabilities':
                                                if prop1.nextSibling.hasChildNodes():
                                                    prop1.nextSibling.firstChild.data = capabilities
                                                else:
                                                    txt = doc.createTextNode(capabilities)
                                                    prop1.nextSibling.appendChild(txt)
    # Write to the .sddraft file
    f = open(sddraft_output_filename, 'w')
    doc.writexml(f)
    f.close()
    arcpy.AddMessage("Capabilities updated")
    return

def EnableWFS(sddraft_output_filename):
    doc = DOM.parse(sddraft_output_filename)
    typeNames = doc.getElementsByTagName('TypeName')
    for typeName in typeNames:
        # Get the TypeName to enable
        if typeName.firstChild.data == "EnableWFSServer":
            extension = typeName.parentNode
            for extElement in extension.childNodes:
                # Enable feature access
                if extElement.tagName == 'Enabled':
                    extElement.firstChild.data = 'true'
    # Write to the .sddraft file
    f = open(sddraft_output_filename, 'w')
    doc.writexml(f)
    f.close()
    arcpy.AddMessage("WFS set")
    return

def SetTimezone(sddraftPath, property_set):
    # Read the sddraft xml
    doc = DOM.parse(sddraftPath)

    # Find all elements named TypeName. This is where the server object extension
    # (SOE) names are defined.
    typeNames = doc.getElementsByTagName('TypeName')
    for typeName in typeNames:
        # Get the TypeName to enable
        if typeName.firstChild.data == "MapServer":
            extension = typeName.parentNode
            # prp = extension.childNodes.getElementsByTagNameNS('PropertyArray')
            for extElement in extension.childNodes:
                if extElement.tagName == 'Definition':
                    for definition in extElement.childNodes:
                        if definition.tagName == 'ConfigurationProperties':
                            for config_prop in definition.childNodes:
                                if config_prop.tagName == 'PropertyArray':
                                    for prop in property_set:
                                        prop_set = doc.createElement("PropertySetProperty")
                                        attr = doc.createAttribute("xsi:type")
                                        attr.value = "typens:PropertySetProperty"
                                        prop_set.setAttributeNode(attr)

                                        prop_key = doc.createElement("Key")
                                        txt = doc.createTextNode(prop["key"])
                                        prop_key.appendChild(txt)
                                        prop_set.appendChild(prop_key)

                                        prop_value = doc.createElement("Value")
                                        attr = doc.createAttribute("xsi:type")
                                        attr.value = "xs:string"
                                        prop_value.setAttributeNode(attr)
                                        txt = doc.createTextNode(prop["value"])
                                        prop_value.appendChild(txt)
                                        prop_set.appendChild(prop_value)

                                        config_prop.appendChild(prop_set)

    # Write to the .sddraft file
    f = open(sddraftPath, 'w')
    doc.writexml(f)
    f.close()
    arcpy.AddMessage("Timezone set")
    return

if __name__ == '__main__':
    # ScriptTool parameters
    map = arcpy.GetParameter(0)
    service = arcpy.GetParameterAsText(1)
    summary = arcpy.GetParameterAsText(2)
    tags = arcpy.GetParameterAsText(3)
    description = arcpy.GetParameterAsText(4)
    overwriteService = arcpy.GetParameter(5)
    enableEditing = arcpy.GetParameter(6)
    enableSync = arcpy.GetParameter(7)
    enableWFS = arcpy.GetParameter(8)
    timezone = arcpy.GetParameterAsText(9)
    share_public = arcpy.GetParameterAsText(10)
    share_organization = arcpy.GetParameterAsText(11)
    share_groups = arcpy.GetParameterAsText(12)
    outdir = arcpy.GetParameterAsText(13)
    
    rest_endpoint = ScriptTool(map, service, summary, tags, description, overwriteService, enableEditing, enableSync, enableWFS, timezone, share_public, share_organization, share_groups, outdir)
    arcpy.SetParameterAsText(1, rest_endpoint)
    arcpy.AddMessage(rest_endpoint)

Step 2: Create a script tool

Next, follow all three sections below to create a script tool.

Create a custom toolbox to store your tool

1. Open your ArcGIS Pro project. In the Catalog pane, right-click Toolboxes and click New Toolbox.

2. On the New Toolbox dialog box, provide a name for your toolbox and click Save.

Import the Python script

In the Catalog pane, right-click the new toolbox you just created, and select New > Script.
On the New Script dialog box, under General, provide the following:
Name: PublishWebFeatureLayer
Label: Publish Web Feature Layer

Browse to the location of the Python script file you saved earlier. Select the file and click OK.
Check Import script.
Leave the Tool Properties dialog box open for the next section.

Define the parameters of your script tool

Parameters are variables that serve as inputs for your script tool. When you run the tool, the Python script reads the parameter values before executing the rest of the script.

Under Parameters, add the following parameters with their properties:

Note: These properties can be modified based on your preferences. (For example, you can change the Share With Organization parameter so that it defaults to SHARE_ORGANIZATION instead of NO_SHARE_ORGANIZATION.)

0. Label: Map
Name: in_map
Data Type: Map
Type: Required
Direction: Input

1. Label: Web Layer Name
Name: service
Data Type: String
Type: Required
Direction: Input

2. Label: Summary
Name: summary
Data Type: String
Type: Optional
Direction: Input

3. Label: Tags
Name: tags
Data Type: String
Type: Optional
Direction: Input

4. Label: Description
Name: description
Data Type: String
Type: Optional
Direction: Input

5. Label: Overwrite Existing Web Layer
Name: overwrite_existing_service
Data Type: Boolean
Type: Optional
Direction: Input

6. Label: Enable Editing
Name: enableEditing
Data Type: Boolean
Type: Optional
Direction: Input
Category: Configuration

7. Label: Enable Sync
Name: enableSync
Data Type: Boolean
Type: Optional
Direction: Input
Category: Configuration

8. Label: Enable WFS
Name: enableWFS
Data Type: Boolean
Type: Optional
Direction: Input
Category: Configuration

9. Label: Time zone
Name: timezone
Data Type: String
Type: Optional
Direction: Input
Category: Configuration

10. Label: Share With Everyone
Name: share_public
Data Type: String
Type: Optional
Direction: Input
Category: Share with
Filter: Value List
PUBLIC
PRIVATE
Default: PRIVATE

11. Label: Share With Organization
Name: share_organization
Data Type: String
Type: Optional
Direction: Input
Category: Share with
Filter: Value List
SHARE_ORGANIZATION
NO_SHARE_ORGANIZATION
Default: NO_SHARE_ORGANIZATION

12. Label: Share With Groups
Name: share_groups
Data Type: String
Type: Optional
Direction: Input
Category: Share with

13. Label: Output Directory
Name: sddraft_output_directory
Data Type: Folder
Type: Required
Direction: Input
Category: Output

14. Label: Rest Endpoint
Name: Rest_Endpoint
Data Type: String
Type: Derived
Direction: Output

Now that you’ve imported your script and set parameters for your tool, click OK on the Tool Properties dialog box.

Learn more about setting script tool parameters

Let’s examine the script tool you created. Double-click your script tool to open it.

Publish Web Feature Layer is now a custom geoprocessing tool that can be run like any other system geoprocessing tool. It’s ready to be added to a model.

Optional: Include metadata

When creating a custom geoprocessing tool, it’s recommended to include documentation that helps others use your tool.

Right-click your script and click Edit Metadata.

Enter tags, summary, and usage. Under Syntax, fill out the fields below and click Save.

in_map:

Dialog Explanation: Map to publish.

Scripting Explanation: Map object of the map to publish.

service:

Dialog Explanation: Enter the name of a new or existing web layer. If the name already exists, the web layer will be overwritten. Otherwise, a new web layer will be created.

Scripting Explanation: A string that represents the service name.

summary:

Dialog Explanation: Enter a short summary of the web layer.

Scripting Explanation: A string that represents summary.

tags:

Dialog Explanation: Enter keywords or terms that describe the web layer.

Scripting Explanation: A string that represents tags, with different tags separated by comma.

description:

Dialog Explanation: Enter an in-depth description of the item.

Scripting Explanation: A string that represents a description.

overwrite_existing_service:

Dialog Explanation: If checked and a web layer with the same name exists, it will be overwritten.

Scripting Explanation: A Boolean that determines whether to overwrite an existing web layer.

enableEditing:

Dialog Explanation: Allow users to edit the web layer.

Scripting Explanation: A Boolean that determines whether to enable editing.

enableSync:

Dialog Explanation: Disconnected editing with synchronization.

Scripting Explanation: A Boolean that determines whether to enable sync.

enableWFS:

Dialog Explanation: Include a WFS layer.

Scripting Explanation: A Boolean that determines whether to share a WFS layer as well.

timezone:

Dialog Explanation: Enter the time zone in which your date values are stored.

Scripting Explanation: A string to set the time zone.

share_public:

Dialog Explanation: Share web layer with the public.

Scripting Explanation: A string that determines if the layer is PUBLIC or PRIVATE.

share_organization:

Dialog Explanation: Share web layer with all authenticated users in your organization.

Scripting Explanation: A string that determines whether the layer is shared to the organization.

share_groups:

Dialog Explanation: Share web layer with groups you belong to and its members.

Scripting Explanation: List of existing groups that the layer is to be shared to.

sddraft_output_directory:

Dialog Explanation: The folder on disk where your .sddraft and .sd files will be stored.

Scripting Explanation: Output directory to create .sddraft and .sd files.

Step 3: Add the script tool to ModelBuilder

To use the tool in ModelBuilder, open an existing model or right-click the toolbox and select New > Model to create a new model. Right-click the script in your toolbox and click Add To Model (or drag it into your model).

Model with Publish Web Feature Layer as a custom geoprocessing tool

You can include other geoprocessing tools in your model. For example, you can set other processes as preconditions so that they execute before the web layer is overwritten. To learn more, see Use ModelBuilder.

Step 4: Run the model

Double-click the tool and fill out the fields. To overwrite a web layer, provide the name of the existing web layer and check the Overwrite Existing Web Layer check box. If the name already exists, the web layer will be overwritten. Otherwise, a new web layer will be created. Before running it, validate the model to ensure that it will successfully run by clicking ModelBuilder > Run > Validate. Once the model has been validated, click Run to run the entire model. You can also schedule a model to run repeatedly or at a designated time.

After the model successfully runs, add the web layer to a map in ArcGIS Pro or open it in Map Viewer.

Next steps

In this article, you learned how to publish and overwrite web feature layers in ModelBuilder by creating and running a script tool in a model. One of the advantages of authoring your own script tool is that you can modify it to meet your specific needs. A couple of examples are provided below. If you want to learn more about automating sharing workflows in ArcGIS Pro, see Introduction to arcpy.sharing.

Let’s suppose you want to publish your web feature layer to a portal folder and allow users to export the data. This can be done by adding the portalFolder and allowExporting properties in the FeatureSharingDraft.

Right-click the script tool in your toolbox and click Edit. Modify the Python file as shown below.

Modify the Python file to include portal and export properties

Save the file. Right-click the script tool and click Properties. Add two more script tool parameters (6 and 11) for the new properties.

Add the new parameters to the script tool

Using what you’ve learned in the previous sections, you can create a script tool specific to map image layers. The code below publishes or overwrites a map image layer on ArcGIS Enterprise. Follow the same steps discussed previously: save the code as a new Python file (.py), create a script tool, import the .py file, and define script tool parameters. Add the script tool to ModelBuilder and run the model when you have configured it to meet your needs.

import arcpy
import os
import xml.dom.minidom as DOM

def ScriptTool(map, service, summary, tags, description, overwriteService, portalFolder, timezone, federatedServerUrl, share_public, share_organization, share_groups, outdir):
    """ScriptTool function docstring"""

    # Set output file names
    sddraft_filename = service + ".sddraft"
    sddraft_output_filename = os.path.join(outdir, sddraft_filename)

    # Reference map to publish
    aprx = arcpy.mp.ArcGISProject("CURRENT")
    m = aprx.listMaps(map)[0]

    # Create MapImageSharingDraft and set service properties
    server_type = "FEDERATED_SERVER"
    federated_server_url = federatedServerUrl
    sharing_draft = m.getWebLayerSharingDraft(server_type, "MAP_IMAGE", service)
    sharing_draft.federatedServerUrl = federated_server_url
    sharing_draft.summary = summary
    sharing_draft.tags = tags
    sharing_draft.description = description
    sharing_draft.credits = "My Credits"
    sharing_draft.useLimitations = "My Use Limitations"
    sharing_draft.overwriteExistingService = overwriteService
    sharing_draft.portalFolder = portalFolder

    # Create Service Definition Draft file
    sharing_draft.exportToSDDraft(sddraft_output_filename)
    outsddraft = sddraft_output_filename
    arcpy.AddMessage("Service definition draft created")

    # Set time zone
    if (timezone != ""):
        property_set = [{
            "key": "dateFieldsRespectsDayLightSavingTime",
            "value": "true"
        },
            {
                "key": "dateFieldsTimezoneID",
                "value": timezone
            }]
        SetTimezone(sddraft_output_filename, property_set=property_set)

    # Create Service Definition file
    sd_filename = service + ".sd"
    sd_output_filename = os.path.join(outdir, sd_filename)
    arcpy.StageService_server(sddraft_output_filename, sd_output_filename)
    arcpy.AddMessage("Service definition created")

    # Share to portal
    output = arcpy.UploadServiceDefinition_server(sd_output_filename, federated_server_url,
                                                  in_override="OVERRIDE_DEFINITION", in_public=share_public,
                                                  in_organization=share_organization, in_groups=share_groups)
    arcpy.AddMessage("Service published")

    return output[5]

def SetTimezone(sddraftPath, property_set):
    # Read the sddraft xml.
    doc = DOM.parse(sddraftPath)

    # Find all elements named TypeName. This is where the server object extension
    # (SOE) names are defined.
    typeNames = doc.getElementsByTagName('TypeName')
    for typeName in typeNames:
        # Get the TypeName to enable
        if typeName.firstChild.data == "MapServer":
            extension = typeName.parentNode
            # prp = extension.childNodes.getElementsByTagNameNS('PropertyArray')
            for extElement in extension.childNodes:
                if extElement.tagName == 'Definition':
                    for definition in extElement.childNodes:
                        if definition.tagName == 'ConfigurationProperties':
                            for config_prop in definition.childNodes:
                                if config_prop.tagName == 'PropertyArray':
                                    for prop in property_set:
                                        prop_set = doc.createElement("PropertySetProperty")
                                        attr = doc.createAttribute("xsi:type")
                                        attr.value = "typens:PropertySetProperty"
                                        prop_set.setAttributeNode(attr)

                                        prop_key = doc.createElement("Key")
                                        txt = doc.createTextNode(prop["key"])
                                        prop_key.appendChild(txt)
                                        prop_set.appendChild(prop_key)

                                        prop_value = doc.createElement("Value")
                                        attr = doc.createAttribute("xsi:type")
                                        attr.value = "xs:string"
                                        prop_value.setAttributeNode(attr)
                                        txt = doc.createTextNode(prop["value"])
                                        prop_value.appendChild(txt)
                                        prop_set.appendChild(prop_value)

                                        config_prop.appendChild(prop_set)

    # Write to the .sddraft file
    f = open(sddraftPath, 'w')
    doc.writexml(f)
    f.close()
    arcpy.AddMessage("Time zone set")
    return

if __name__ == '__main__':
    # ScriptTool parameters
    map = arcpy.GetParameter(0)
    service = arcpy.GetParameterAsText(1)
    summary = arcpy.GetParameterAsText(2)
    tags = arcpy.GetParameterAsText(3)
    description = arcpy.GetParameterAsText(4)
    overwriteService = arcpy.GetParameter(5)
    portalFolder = arcpy.GetParameterAsText(6)
    timezone = arcpy.GetParameterAsText(7)
    federatedServerUrl = arcpy.GetParameterAsText(8)
    share_public = arcpy.GetParameterAsText(9)
    share_organization = arcpy.GetParameterAsText(10)
    share_groups = arcpy.GetParameterAsText(11)
    outdir = arcpy.GetParameterAsText(12)

    rest_endpoint = ScriptTool(map, service, summary, tags, description, overwriteService, portalFolder, timezone, federatedServerUrl, share_public, share_organization, share_groups, outdir)
    arcpy.SetParameterAsText(1, rest_endpoint)
    arcpy.AddMessage(rest_endpoint)

The need for readily available information during times of crisis makes your GIS work invaluable. In this article, we’ve discussed how to create and use script tools and ModelBuilder in ArcGIS Pro to efficiently publish and update your web layers. Learn more about how to share your work in ArcGIS Pro.

About the author

Jonah Lay

Jonah is a product engineer and writer on the ArcGIS Pro Sharing Team with a background in environmental science.

Connect:

Article Discussion:

25 Comments

Oldest

Newest

Inline Feedbacks

View all comments

Engaging Volunteers for a Cause

Read this article