Tips and tricks for optimized work sessions with network diagrams

Introduction

As the utility network owner, you are also in charge of configuring and maintaining network diagrams. The purpose of this article is to provide recommendations and guidance for optimizing the work sessions with network diagrams. The tips and tricks shared in this article do not only apply to your own administrative tasks on network diagrams. They also concern best practices that we recommend you teach your end users before putting your utility network into production.

A – Best practices when configuring diagram templates

Database considerations during custom template creation phase

1. Prefer working in a file or mobile geodatabase during the custom template creation phase

During the diagram template creation phase, prefer configuring and testing your custom diagram templates in a file or mobile geodatabase instead of an enterprise instance.

Why? This is really more flexible for testing. You can work with your single-user geodatabase and chain configuration and test operations in the same ArcGIS Pro session.
When working with an enterprise geodatabase, you have to run any diagram template configuration operations using a direct connection to the dataset. Then, before testing your updated settings, you must restart the related service and start a new ArcGIS Pro session with the restarted service. Having to switch from a direct connection to the published service for testing purposes several times is not efficient.

How? You should consider the following workflow steps:

• Create an empty file or mobile geodatabase.
• Connect to the enterprise geodatabase using the database utility network owner credentials.
• Copy the utility network dataset from your enterprise geodatabase and paste it to your file or mobile geodatabase.
• Configure the diagram templates you expect in the file and mobile geodatabase by following the best practices detailed in the following three sections.
• When anything looks good in the file or mobile geodatabase, you can create the same custom templates in your enterprise geodatabase by running the following workflow steps:
  • Run the Export Diagram Template Definitions tool on each custom diagram template set up in the file or mobile geodatabase to export its Diagram Rule and Layout Definition file and Diagram Layer Definition file (*).
  • Run the Add Diagram Template tool to create each custom template in your enterprise geodatabase using the exported Diagram Rule and Layout Definition file and Diagram Layer Definition file (*).

(*): Make sure you already “shared” the diagram layer definition between your templates in the file or mobile geodatabase, as explained in the ‘Configuration of the diagram layer definition on custom templates’ subsection below. Otherwise, consider the question addressed in that section when running the Add Diagram Template tool to avoid importing various layer definitions on your templates in the enterprise geodatabase.

2. Prevent diagram storage if the custom template creation phase is performed in the enterprise geodatabase directly

If, for any reason, you have no choice and must configure and test your custom diagram templates directly in your enterprise instance, think about preventing any users from storing diagrams during this configuration phase.

Why? At the time you create a custom diagram template, your end users who are consuming the related service can see it and might be tempted to play with it. Since diagram storage saves data in versioned diagram feature classes and unnecessarily increases the size of the database until the template creation completes, it is better to block diagram storage capability.

How? To prevent your end users from storing any diagrams, you must run the Alter Diagram Template tool on each created custom template and uncheck the Enable diagram storage option:

Configuration of diagram rules and layouts on custom templates

As an introduction to this section, you can watch the following video: Network Diagram Build Process Demo. It explains the network diagram building process through rules and layouts configured on a sample diagram template. Then, when configuring diagram rules and layouts on your own custom templates, keep in mind the following recommendations.

1. Working environment

Use ModelBuilder to create and edit geoprocessing models when configuring diagram rule and layout for your custom diagram templates.

Why? The geoprocessing models allow for better control and maintenance of the rule and layout sequence set up on each diagram template.

2. Diagram template design questions

When configuring diagram templates, you first need to think about the kind of diagrams your end users expect.

How? This means consider the following questions:

a. What type of diagrams do they want?

• What kind of standard diagrams do they expect?
  – Do they expect diagrams to represent specific parts of the network? Will these diagrams cover network portions that have a logical meaning? For example, subnetworks/circuits, transmission versus distribution network parts, areas that are upstream/downstream to specified input network elements.
  – Do they need a template to create diagrams for quality control and quality assurance tasks?
• Do they really need subnetwork system diagrams?
  Indeed, when you set your domain network tiers using the Set Subnetwork Definition tool, you can set up and assign diagram templates as subnetwork diagram templates. However, you should take the time to evaluate the real benefits of generating and maintaining subnetwork system diagrams each time you update subnetworks.
  Why? Subnetwork system diagrams based on these templates are systematically created and maintained as stored diagrams for each subnetwork when running subnetwork updates. This considerably slows down the update subnetwork process and potentially increase the database size with stored diagrams that could have no real interest.
  Note: To learn more about differences between subnetwork system diagrams and standard diagrams, read the following blog: A tale of two network diagrams.

b. For each identified diagram template, you must then clarify the various junctions, devices, and lines. Which ones do your users want to see in their diagrams and which they not want to see?

Why? Answering each question helps you clarify and implicitly specify how to set up each diagram template.

3. Diagram rule setting recommendations

When setting up diagram rules on your diagram templates, keep the following in mind:

• For standard diagrams, you should use diagram rules that automatically query and retrieve network elements—for example, Trace rule, Expand Container rule, and Spatial Query rule.
  Why? Templates configured with such diagram rules allow you to generate diagrams that automatically reflect relevant changes after updates.
  For example, once updated, these diagrams will show any newly added or removed containment relations, or reflect extended or reduced area after switch status changes, and so on.

• For subnetwork system diagrams, on the contrary, avoid configuring the system diagram templates with these rules.
  Why? A subnetwork system diagram should strictly include and represent network elements related to a particular subnetwork. Trace rule, Expand Container rule, and Spatial Query rule could query and include network elements that are out of the subnetwork, and generate diagrams that don’t exactly fit the subnetworks.

4. Consider the diagram storage capability

Storing a network diagram or not… This is a key question to consider. There is no reason to store every generated network diagrams.

Why? Diagram storage saves data in various versioned diagram feature classes. This means that each time a user applies layouts to a stored diagram, edits its diagram features, or updates the diagram, there are new rows created in these versioned feature classes, and the size of the geodatabase increases. Blocking the storage of diagrams prevents unnecessary increases in the size of the geodatabase.

How? Blocking diagram storage can be configured at the template level. As the utility network owner, you can enable the diagram storage capability on a few specific templates and disable this capability for other templates. This is the best solution to prevent end users from storing diagrams that have no interest.

• Diagrams that should never been stored
  1. Diagrams that users can easily re-create
  2. Diagrams created for QA/QC purposes
• Diagrams that could make sense to be stored generally share the following characteristics:
  1. Diagrams that must be shared with others
  2. Diagrams on which end users will spend editing time
  3. Diagrams in which the contents are stable
     As an example, a diagram that represents a single subnetwork is generally not very stable since it can cover a more reduced area after a switching job to balance the load over another subnetwork. However, some sets of connected subnetworks that are used for load balance operations can be considered as stable logical portions of the network and stored as diagrams for maintenance operations.

After these considerations and analysis, load the Alter Diagram Template tool and decide whether you should check or uncheck the Enable diagram storage option for each template:

5. Other useful tips

Here are some other useful tips when configuring diagram rule and layout on custom templates:

– Save your diagram template geoprocessing models in a dedicated custom toolbox.

– Regroup sequences of template configuration tools that have the same logic and purpose so your colleagues better understand the various diagram building phases.

– Write a summary and description for each template model to provide details about how the template is configured, what it expects as input, and so on.

– Note that such a description is also very helpful for your end users so they have a way to learn more about the template purpose. To provide the template description to your end users, you can run the Alter Diagram Template tool and fill in the Description text box:

This description is available when end users open the Network Diagrams tab:

Configuration of the diagram layer definition on custom templates

When configuring a diagram template, you also have to set up its layer definition. The diagram layer definition is the set of layer and labeling properties specified for each sublayer that composes a network diagram layer.

To learn more about the various workflows to configure the diagram layer definition on a diagram template, you can watch the following video: Defining Nice Layers for Better Network Diagrams.

Diagram layer definition size can be really huge, in particular, when combining subtype group layers with sophisticated symbols and complex label settings for each sublayer. Also, the number of attributes on the network source classes which are joined to the diagram sublayers have an impact on the diagram layer definition size.

As the utility network owner, you should keep these points in mind when you set up the diagram layer definition.

Furthermore, and this is the most important point to remember, you should ensure that you limit the number of different diagram layer definitions to one or two and “share” them across all your templates.

Why? When you set up the same diagram layer definition (.ndld file) on each diagram template and the published service starts on the server, the diagram templates are loaded in memory, the layer definition on each loaded template is systematically analyzed and compared to the layer definitions set on the templates that are already loaded. If the same layer definition already exists in memory, it is automatically “shared” with the template currently at loading. This avoids loading the same layer definition multiple times and vainly increasing the memory footprint on the server.

How? The best could be the following:

1. Set up a diagram layer definition with rich symbology and labels and import this diagram layer definition to each template used to create relevant and smart diagrams that will be shared (switching diagram, interconnected substations, and so on).

2. Set up a second diagram layer definition with more simple symbology and labels and share it on templates that are used to generate diagrams for quality assurance and quality control, for example.

Tip: At the time you run the Create Diagram Layer Definition tool to build these one or two diagram layer definitions, you should pay attention to settings related to a few specific diagram sublayers that could have really no interest for your custom templates. They concern additional sublayers:
– Prefer initializing these sublayers as standard layers instead of subtype group layers; that is, consider the real added value of using subtype group layers when checking the Subtype layer options for each additional layer you need for your custom templates (see 1 in the screenshot below).
– Consider removing sublayers intended to represent network lines or edge objects reduced as point junctions in your diagrams (see 2 in the screenshot below). Such diagram sublayers have a meaning only if you configure Reduce Edge rules on your custom templates. If you have not configured such a rule on any of your custom templates, remove each of these diagram sublayers.

Limitation on the number of diagram templates

As the utility network owner, you should think about the diagram templates that will have a real interest for your end users and so limit the number of diagram templates you provide.

Why? With the current releases, the number of diagram templates has a significant impact on the server startup time.

How? At any time, you can run the Delete Diagram Template tool to delete any template that is not relevant or pertinent.

At least, consider the real value of keeping the three templates installed by default for any utility network at its creation, that is, Basic, ExpandContainers, and CollapseContainers. These templates are very generic to apply to any utility networks, whatever the domain they apply. They may become of very limited interest once you have provided custom templates allowing you to generate the diagrams your end users expect.

Note: Before running the Delete Diagram Template tool, note the recommendations for deleting diagram templates detailed in the ‘C – Best practices for network diagram administration’ section below.

B – Best practices to teach to end users

The following guidelines concern stored diagrams. They have the same objective: avoid unnecessary creation of rows in versioned diagram feature classes and increase the size of the geodatabase.

Even when they are storable, do not store all diagrams you generate

Even when the diagram storage capability is not blocked at template level, storing a diagram may have no interest. Diagrams that could make sense to be stored generally share the following characteristics:

• Diagrams that need to be shared with other users
• Diagrams on which you spent editing time
• Diagrams in which contents are stable

Make sure your diagram fits these characteristics before storing it.

Do not store a diagram until you have finished its layout

When there are real reasons to store a diagram, make sure you have finished its layout before storing it.

Note: Once you create a diagram in ArcGIS Pro, it opens in a new diagram map that becomes the active map in your project. At that time, it is a temporary diagram. If you save and close the project, you will still have access to this diagram map even when re opening the project (*). This allows you to potentially view and complete your edits on this temporary diagram before storing it when the diagram storage makes sense.

(*): The only scenario that makes a temporary diagram no longer visible in a diagram map saved with your project is after the utility network owner has purged temporary diagrams.

Consider access rights when storing a diagram

By default, when storing a diagram, it is stored with the Public access right, allowing any other users to see, edit, and update this diagram. You can consider sharing your diagram with read-only access for other users.

To do so, you have the following options:

• At the time you run the Store Diagram tool, you can select Protected from the Network Diagram Access Right drop-down list.
• For a diagram that is already stored, you can run the Alter Diagram Properties tool at any time and change the diagram access right level to Protected.

Prefer working in a branch version when editing stored diagrams

When you must apply layouts or operate manual edits to stored diagrams, think about creating a branch version and apply these changes in the branch version.

This avoids keeping track of each edit you made to your stored diagrams once you post the version to Default.

Avoid unnecessary updates of stored diagrams when working in a branch version

On the contrary, when you work in a branch version and edit network features or objects, you should avoid updating stored diagrams that become inconsistent regarding the network topology space in that branch version. Indeed, once network edits are posted to Default, you will end with dirty areas in Default anyway and will have to update your diagrams again after posting so they return to a consistent state.

C – Best practices for network diagram administration

Purging temporary diagrams

As the utility network owner, you should think about regularly cleaning non-versioned diagram feature classes.

Why? Keeping temporary diagrams in non-versioned diagram feature classes makes the geodatabase size increase and can degrade performance. Furthermore, since the temporary diagrams remain in ArcGIS Pro projects until their associated maps are closed but are discarded when the map is closed, there is no reason to keep these temporary diagrams in the geodatabase.

How? Run the Purge Temporary Diagrams tool to fully clean up non-versioned diagram feature classes. You can automate this operation at a specific time each week or multiple times per week if needed.

Geodatabase cleanup after deleting diagram templates

Deleting diagram templates is not a very common operation. However, when necessary, it may require post-processing.

When running the Delete Diagram Template tool to delete a diagram template, we process in two phases.
– The first phase applies to non-versioned diagram feature classes and tables where we save diagrams and diagram features related to temporary diagrams. It deletes any rows existing in the six non-versioned diagram feature classes and tables—that is, TMP_Diagrams, TMP_Junctions, TMP_Edges, TMP_Containers, TMP_Aggregations and TMP_Relations—that are related to the deleted template.

– The second phase applies to versioned diagram feature classes and tables where we save diagrams and diagram features related to stored diagrams. For performance reasons, this phase doesn’t run in the same way. In this case, the Delete Diagram Template operation only flags rows related to the deleted template present in the versioned Diagram feature class as ‘is_deleted’. It doesn’t process any row in versioned Junctions, Edges, Containers, Aggregations, and Relations feature classes and tables.

As the database administrator, if you know that the deleted diagram templates concerned a large number of stored diagrams, run a cleanup script to delete remaining unnecessary rows in the versioned diagram feature classes and tables and decrease the geodatabase size.

Such a script will look like the following one written for a SQL Server geodatabase where

– <db_name> is the geodatabase name,

– <db_owner> is the utility network database owner user,

– XXX in UN_XXX prefix for each table corresponds to the dataset ID of the utility network.

Conclusion

This article is dense and includes a lot of advice. It would be a good idea to save it to your favorites and refer to it whenever you need to configure diagram templates.

Go ahead, diagrams no longer hold secrets for you!