ArcGIS Pro

Maintain data accountability with archiving - Part 2

In this scenario, the Swiss Federal Office of Energy (SFOE) is preparing the country’s wind turbines for an increase in domestic wind energy. In part 1 of this blog series, we showed how the GIS specialists used editor tracking as a solution to enforce quality standards and maintain data accountability between SFOE and the private turbine manufacturing companies.

During the base assessment process performed by the system operators, editor tracking maintained data accountability by tracking and recording the following information:

Editor tracking recorded values

Information recorded by the editor tracking

Your project details

Associated details from the project feature class (for example, the Wind Facilities feature class)

User who created a new feature Manufacturing company that built the wind turbine
Date and time the feature was created Year of construction of the wind turbine
User who last edited a feature System operator who performed the base assessment in the field
Date and time the feature was last edited The timestamp of when the last edit was made for the base assessment

After the base assessment was completed, to prepare for the increase in production, more testing needs to be done to maximize the rated power of each wind turbine. The testing procedures will be conducted in three phases. During the testing period, the system operators will perform more edits on the wind turbine data that will need to be accounted for at the end of each testing phase, to maintain data accountability.

Because editor tracking records only the creation moment of a feature and the last edit of that feature, in the situation where the same features will have different edits made at the end of each testing phase, we will need a tool that records data changes at a more granular level. To address this project need, we will use geodatabase archiving.

Changes recorded by editor tracking (top) and archiving (bottom) are shown.
Changes recorded by editor tracking (top) and archiving (bottom) are shown.

Archiving and data accountability

Archiving is the mechanism for capturing, managing, and analyzing data change over time.

Archiving helps maintain data accountability as following:

Implementing archiving on the Wind Turbine feature class

In ArcGIS Pro, you can enable archiving on your data using the following methods:

Archiving option on the manage tab
Archiving option on the manage tab
Enable Archiving geoprocessing tool
Enable Archiving geoprocessing tool

Note: Archiving can be enabled on both versioned and non-versioned data. Although there are a few differences between working with archived data in these two environments, the overall process is very similar.

For this workflow, you will be working with non-versioned data.

When you enable archiving on non-versioned data, the geodatabase creates additional date attributes in the base table for that dataset. These date attributes are columns in the base table, named gdb_from_date and gdb_to_date, and are used to record the time stamp for the effective lifespan of the archived row. As edits are made to the dataset, these attributes are updated to maintain an historical record over time.

After enabling archiving, the system operators will have six days to conduct the testing as following:

  Phase 1 Phase 2 Phase 3
Duration December 1–2 December 3–5 December 6–7
System operators Gigi, Hans Elena Yves

Fast forward to December 8, and they are ready for the next step.

The end date of the testing phase
The end date of the testing phase

When the testing phases are completed, the GIS specialist will need to review the edits made and ensure the edits are correct.

Interacting with the archive data

To interact with the archive data, you have two options available:

Connecting to a historic version

To connect to a historic version, you must enable the History pane, which is located in the contextual Versioning tab on the ribbon.

1. In the Contents pane, click List By Data Source. Then click the database connection string to enable the Versioning tab on the ribbon.

List by Data Source in the Content pane
List by Data Source in the Content pane

2. On the Versioning tab, select the Historical Moments option to open the History pane.

Historical Moments option on the Versioning tab
Historical Moments option on the Versioning tab

Note: Notice the versioning options are unavailable. This is because the Wind Turbine feature class in not versioned. You might ask, why is the database connection string pointing to a branch transactional version? This is the versioning type of the workspace level and not the feature class level. To learn more about the difference between the versioning type at the workspace level versus the feature class level, check out this blog on versioning enhancements in ArcGIS Pro 3.0 where these terms are explained in depth.

3. In the History pane, there are two ways to connect to a historical moment.

    • You can specify the exact date and time using the calendar.
Specific Date and Time option
Specific Date and Time option
Historical Market option
Historical Market option

To connect to a moment in time using historical markers, you will first need to create some. These are specific for the project needs. For this workflow, you will create one to mark the end of each testing phase to help you quickly navigate between phases.

Notice there is already one created. The Default historical marker was automatically created when archiving was enabled on the feature class, hence the GDB_FROM_DATE is 12/01/2022.  The Default marker displays the current representation of the archive classes, which is equivalent to the class representations in the transactional DEFAULT version. In other words, it shows the current data and how it looks in the present moment.

4. To create a historical marker, click the Edit Markers tab, which will prompt you to create a new marker by adding a name and a date.

When finished, the three new historical markers should look something like this.

Newly created historical markers to match the end of each testing phase
Newly created historical markers to match the end of each testing phase

5. To connect to a historical marker, click one of the markers and click Apply. You can also check the Auto Apply checkbox to maintain a single click navigation between historical moments.

Connecting to the phase 1 historical marker
Connecting to the phase 1 historical marker

Once connected, notice the data source is now pointing to the historical version associated with the End Phase 1 marker.

Review edits between historical moments

Now that you know how to connect to different historical moments, it’s time to look at the data and inspect the edits made by the system operators.

The first thing that stands out is that at the end of phase 2 there were 85 turbine points and at the end of phase 3 there were only 84. This indicates one turbine feature class was deleted during phase 3.

Changed number of features in phase 3 (Open image in a new tab to zoom in)
Changed number of features in phase 3 (Open image in a new tab to zoom in)

Next, let’s have a closer look at the historic records using the archive class.

Working with the archiving class

The archive class is a collection of all edits made since archiving was enabled on the feature class. It is created and maintained by the geodatabase when a dataset or class is enabled for archiving. The archive class cannot be edited, but it can be added to a map for viewing and querying purposes.

1. To add the archive class to the map, select the Wind Turbine layer from the Contents pane.

2. On the ribbon, on the Data tab, click the Add Archive button.

Add Archive option located on the ribbon
Add Archive option located on the ribbon

This adds the archive class as a layer to the map, named Wind Turbines Archive.

Wind Turbines Archive Class added to the map
Wind Turbines Archive Class added to the map

You can zoom in to the area of interest and compare the Wind feature class with the Archive class. From the GDB_T0_DATE field, you confirm that Turnine_41 was indeed deleted on December 7 at 4:47 p.m.

Turbine 41 GDB_TO_DATE field confirming the feature was deleted during phase 3
Turbine 41 GDB_TO_DATE field confirming the feature was deleted during phase 3

Querying the archive history

First, set up a definition query to look at a specific wind feature. In this example, we will look at feature 5 and assess how this feature changed over the testing process.

Definition Query expression returning all records for the Wind Turbine 5
Definition Query expression returning all records for the Wind Turbine 5

Then you open the Wind Turbine Archive attribute table to see the filtered history records.

Wind Turbines Archive Class with records for feature 5
Wind Turbines Archive Class with records for feature 5

Now let’s read the archive records:

First row indicates when archving was enabled on the feature class (GDB_FROM_DATE) and when the first edit was made (GDB_TO_DATE)
First row indicates when archving was enabled on the feature class (GDB_FROM_DATE) and when the first edit was made (GDB_TO_DATE)
The second row indicates the time span of the edit made by Gigi and the attribute change she made
The second row indicates the time span of the edit made by Gigi and the attribute change she made
The final row indicates the last edit made by Yves
The final row indicates the last edit made by Yves

Maintaining the archive

Based on the SFEO privacy policy, the archive class will be kept as an official record for two years starting with the end of the testing process. After that, the GIS specialist will trim the archive history to improve the enterprise geodatabase performance, using the Trim Archive History geoprocessing tool.

Trim Archive History geoprocessing tool
Trim Archive History geoprocessing tool

In this blog series we focused on the importance of data accountability in an organization and how editor tracking, and archiving can be used to maintain it. In the first blog, we showed how the SFOE used editor tracking to get a clear understanding of who created a wind turbine or edited an existing one and when they did so. In the second blog, we continued the user scenario and showed how geodatabase archiving recorded changes to the turbine data at a more granular level by maintaining a transactional history of all edits made to the feature class. We hope this blog series will raise your awareness of the importance of data accountability and equip you with the right information to implement editor tracking and archiving within your workflows.

 

Acknowledgments: Swiss Federal Office of Energy (SFOE) and Esri Suisse for the Wind Energy Plants dataset.

 

Photo by Waldemar Brandt on Unsplash

 

About the author

Product Engineer in the Geodatabase team, passionate about making a difference in people's lives using GIS. Hiker and a true Éclair and Crêpe lover in her free time.

Next Article

Why should you consider adding ArcGIS Image to your organization?

Read this article