Edit 4/26: Added an additional resources section to help with next steps.

The First Steps

Often, the hardest part of every journey is taking the first step. This article will show you how you can take your first tangible steps towards implementing the ArcGIS Utility Network today using these the five key steps we see in every successful implementation:

Look at how you organize your data into networks
Identify the fields you use to manage those networks
Preparing for and performing data cleanup tasks
Considering how you organize or classify your assets
Reviewing your data for unused or unreliable fields

This article is intended for a wide audience, and I hope that everyone from managers and stakeholders to GIS Admins and Technicians will use this guide to understand implementation of GIS systems for utilities and why we make some of the decisions that we do. With that out of the way, let ‘s begin!

1. Organize your data into networks

Every utility manages some type of commodity and the way these commodities are conveyed to customers are via networks. Transmission networks move large volumes of the commodities from where it is being generated or stored to regional distribution hubs. Distribution networks deliver these commodities from distribution hubs into the residential and commercial areas that need them. The ArcGIS Utility Network is capable of modelling systems like stations and plants that haven’t historically been modelled in a GIS. So, when planning you need to consider how much of your network data you want to model and ask the following questions:

Which networks you want to model?
Do you have the data to support these networks?
What are the business drivers and who are your stakeholders for these networks?
What existing business processes and workflows are in place to maintain this information?

Once you’ve identified which networks you will and won’t be modelling, the next step is to determine how you will store this information in your data model.

2. Identify the fields used to manage your network

While its true that every network is different and that every customer does things differently, there are some things that are universal to the way that networks behave. Once you understand how to organize and configure the utility network dataset you can use it represent any kind network! The questions you need to answer to model a network are the following:

How do we organize and name our networks?
Where do resources enter the system?
What kind of equipment can disrupt the flow of resources?
How are resources measured or metered within the network?

Once you have answered these questions, you’ll then look at how your data supports these answers.

Maybe you have fields that record this information on every feature?
Maybe you are only storing these values on your network sources?
Maybe you just maintain a set of boundary polygons that describe the extents of each network?

Knowledge of how you record this information and how complete it is will be key to ensuring that you account for this information correctly in your data model. It will also help you develop a quality assurance plan to validate that your networks are representing the same sets of features after implementation.

If you have concerns about how reliable this information is then I have good news for you! ArcGIS Pro has a built-in set of tools to help you identify where your utility network data is incomplete or inaccurate. In fact, most customers find that they have an improved level of confidence in their data once they completed their implementation. This is not only because they have worked through the inconsistencies of their data, but because they know that going forward the system will catch any mistakes as they are made, before they become committed to the model.

3. Preparing and performing data cleanup

A self-checking system sounds great, but most of us recognize that our data isn’t perfect, and we may even be nervous about how much work its going to take to implement this model. This is where its important that we review our answers from the previous two steps:

What are the business drives for our networks
Do we have the data to support these networks?

If you already have a mature quality assurance program and tools then you can expect to have an easier time implementing this solution. If you don’t have consistent processes for data entry or robust quality assurance practices in place then you should expect some growing pains as these are implemented. As your project progresses your data quality will improve as you develop a robust quality assurance process that use ArcGIS Pro and the utility network dataset.

The utility network dataset requires that there be no bad data present in your network to ensure the system behaves as expected. This of course raises questions like “What exactly is bad data?” and “How do we fix bad data?”. Within this context bad data is data that satisfies one of three conditions:

Errors reported when you validate your topology are called Topology Errors
Errors returned when you update a subnetwork are called Subnetwork Errors
Features that do not participate in a subnetwork, but should, are called Orphaned Features

Resolving errors

Esri’s online help provides a comprehensive description of all the different types of topology and subnetwork errors. The most common resolutions to these errors involve backfilling missing data and correcting inconsistencies within the data. The following are some specific examples of errors and their resolutions:

Devices missing information that allows them to be properly classified in the new model frequently report errors that no rules have been configured to support their connectivity
- You should review your source data to ensure that the features have all the required information populated
Devices with incorrect or missing status (open/closed) often result in networks being improperly connected
- Review internal notes or perform a field check to determine the current status of the equipment
Incomplete information about network flow, network names, or construction status can result in orphan features
- Review field notes, construction drawings, or perform fields checks to ensure that all features are represented on the correct networks

There are other types of data cleanup you will want to perform as part of your implementation, but the priority should always be to correct topology and subnetwork errors. Any networks that contain errors can’t be traced, analyzed, or exported. As a result, many customers have a zero tolerance policy for topology and subnetwork errors to ensure that they can meet the needs of the business and the expectations of their customers.

The good news is once you’ve gotten past these first three steps the remaining two are not anywhere near as technically challenging although they are just as important and can take just as long, and in some cases even longer, to work through.

4. Classifications

Now that you’ve figured out your network model and data cleanup strategy it’s time to focus on organizing and classifying you data in a way that is meaningful to your end users and meets your existing and future requirements. Features in the utility network dataset are classified using three sets of identifiers:

Table
Asset group
Asset type

Any feature that belongs in the network should be placed in one of the following tables based on its role within the network:

Assembly – A location representing a collection of network features that operate as one collective unit
Device* – Network equipment that controls, monitors, or alters flow within the network
Junction – A location where network equipment connects two or more lines
Line* – Paths that commodities can flow within the network.
Structure Boundary – An area representing a collection of network features supporting a network function
Structure Junction* – A location that supports one or more network features
Structure Line* – Path that contains one or more network features

*Note: These tables also have corresponding nonspatial tables

Asset Groups and Types

Asset groups are used to organize common sets of features with the same purpose and similar characteristics. Features in the same asset group appear on the same layer in the application and have a common set of fields and pick lists. Features within each asset group can be further divided into different asset types. Each asset type has a different set of business rules that control how they connect and relate to other network features.

Before you begin trying to come up with your own asset groups and types you should look at one of the industry models provided by Esri’s Solutions team to get your started.

Industry Models

The Solutions team at Esri provides industry specific solutions with each release that include a recommended set of asset groups, asset types, and rules for what is considered a standard model for that industry. Search the ArcGIS Solutions gallery for “Utility Network Foundation” to see all the models they provide for each industry.

These models are designed to be comprehensive and cover a broad domain of generation, transmission, distribution, and service assets. Because most customers only exist within one of these domains they may only use a subset of the available classifications. Customers are expected to add or rename the classifications in these models to suit their needs and preferences.

Now that you’ve determined what your classifications are you now need to review your source data and ensure you can determine the classification of every feature. This task shouldn’t be underestimated as you will likely discover some gaps in your source data. Once you’ve made sure that every feature in your source database has a home in the new model it’s time to make sure that all of the fields in your database are being migrated to the new data model.

5. Fields and Lookups

Like classifications Esri also provides a recommended set of fields and pick lists in their standard model. The next step in the process it to review the data dictionary provided with your industry model and reconcile it with your current model.

I recommend doing this one asset group at a time. Look at the fields and pick lists in the data dictionary and compare them with your existing fields and pick lists for each source table. If you find any columns that are blank, sparsely populated, or contain unreliable information you will want to consider either correcting the data before you migrate or not migrating that column.

You will then take the remaining list of “good” columns and compare them to the target data model and if you find that the industry model has the corresponding field that you will map your field to it. If you can’t find a corresponding field in the industry model then you are free to add fields to the model, although some care should be exercised to ensure that tables don’t become too large or unwieldy for end users to maintain!

Finally, if your source or target field has a pick list associated with it you will also want to make sure that you create a lookup that can help translate values between the two fields. In some cases you will need to add new values to the target pick lists.

Next Steps

Now that you’ve walked through the five steps to migrating to the ArcGIS Utility Network I hope you have a better understanding of how much of an effort this will be for your organization. I also hope that you have an appreciation of the opportunity you have to transform and improve your data on this journey. This process takes a fair amount of time to work through and while it isn’t something that should be rushed it is also something that you should not put off doing until the last minute.

Additional Resources

Here are some resources to keep your journey going.

Learning Paths – For a hands on approach to learning I recommend you use the learning paths at learn.arcgis.com. Learning paths are playlists of content you can use to get familiar with a topic, so if you go to learn.arcgis.com and search for paths with the words “Utility Network” you’ll find several paths to get you on your way.
Utility Network Foundation Models – When you’re ready to start looking at data models and sample data you’ll want to download a Utility Network Foundation for your industry. You can access these by going to the solutions page and searching for “Utility Network Foundation”. Each of these solutions includes a data model, dictionary, maps, and set of instructions to get you up and running with a sample dataset.
Esri Data Loading Tools – If you’re interested in doing your own conversion but aren’t familiar with any existing data conversion tools you can get started using Esri’s Data Loading Tools which are also available on the ArcGIS Solutions page.
Utility Network Package Tools – When you’re ready to start loading data you’ll want to download and familiarize yourself with the Utility Network Package Tools from the ArcGIS Solutions site. These tools describe the “Asset Package” file format used by the Utility Network Foundations and show you how you can create a usable utility network using a local file or enterprise geodatabase. Note, you’ll need to have a fully licensed ArcGIS Enterprise environment if you want to use an enterprise geodatabase.
Utility Data Management Support Tools – Once you’re comfortable with the utility network and are looking for some tools to help with your data conversion, data cleanup, or analysis with the Utility Network you should check out the Utility Data Management Support tools. These tools help automate common tasks like configuring maps, updating configuration, and even automating tracing!