ArcGIS Living Atlas

Accuracy vs. Privacy: create web maps to compare Census demonstration products

The United States Census Bureau is implementing a new form of privacy protection for 2020 Census data. The Disclosure Avoidance System (DAS) works by injecting noise into the actual data to prevent the re-identification of individuals. As they fine tune the DAS method, the Census Bureau is asking for your feedback (2020DAS@census.gov) on how the new system will impact your most important workflows.

Follow the steps below to create a web map comparing published 2010 data to three different versions of the DAS demonstration product data, all available from ArcGIS Living Atlas of the World. This example workflow shows you how to build a web map with pop-ups for American Indian Areas and the Total Population: American Indian and Alaska Native Alone field. The same steps can be followed to compare the DAS values for other fields, such as Population 18 Years and Over or Number of Housing Units. You can also use this workflow to compare DAS values for other geographies such as counties, tracts, or block groups.

Click here to see the final map produced by the workflow.

Note: The Census Bureau has identified an error in the 2020-09-17 demonstration product (v3). The result is a marginal, artificial, increase in accuracy for some person-level variables. Only block and block group level data are impacted. The workflow below remains valid, but for block or block group analyses, you’ll want to focus your attention on v1 and v2 data values. This post will be updated as new demonstration products are released.

Create a new project package and download the data

You’ll start by creating a new ArcGIS Pro project and downloading the demonstration data from Living Atlas.

1.  Start ArcGIS Pro. Under Blank Templates, click the Map template.

New map template

2. In the Create a New Project window, for Name, type AIACompare. If necessary, choose a location for the new project (a folder on your computer).

Create a new project

3. Click OK.

The new ArcGIS Pro project opens.

4. Click the Map tab and click Add Data. In the left panel of the Add Data window, click Living Atlas.

Access Living Atlas from ArcGIS Pro

5. Search for Census Demonstration. Select all layers and click OK.

Add content from Living Atlas

The Census Demonstration layers are added to the Contents pane. A layer for each demonstration product, with data for states, counties, subdivisions, congressional districts, tracts, block groups, and American Indian Areas, appears in the Contents pane.

 

Copy the geometry and the SF1 data values

Next, you’ll create a layer with the American Indian Area geometry, identification fields, and the published 2010 data value for the American Indian and Alaska Native Alone field. You’ll start by making sure the History and Geoprocessing panes are open.

1. On the Analysis tab, click History to open the History window. Click Tools to open the Geoprocessing pane.

Access to the History and Geoprocessing pane

For demonstration purposes, you’ll analyze American Indian Areas. You’ll copy that layer.

2. In the Geoprocessing pane, search for and open the Copy Features tool.

Copy AIA geometry and SF1 values

4. Click Run.

5. In the Contents pane, uncheck all layers except the AIANAloneData layer.

6. Right-click the AIANAloneData layer and choose Zoom To Layer.

Next, you’ll remove all the fields except the ones you want to include in your web map.

7. In the Geoprocessing pane, click the Back button.

Back to search button

8. Search for and open the Delete Field tool.

Search for the Delete Field tool

9. For the Input Table parameter, choose AIANAloneData. For the Drop Field parameter click the arrow next to the Drop Field parameter to reveal all the fields.

List all fields

10. Click the Toggle All Checkboxes button to select all the fields.

Toggle to check all fields

11. Uncheck the following fields, then click Add:

Add the checked fields

The SF1 data is the original published 2010 data values.

12. Click Run.

 

Get the demonstration data values

Next, you’ll add the demonstration data values for the American Indian and Alaska Native alone field to the American Indian Area geometry. For each demonstration product, the field name is identical, so after joining the field value for each demonstration product version to your AIANAloneData layer, you’ll update the field name.

1. In the Geoprocessing pane, search for and open the Join Field tool.

2. For Input Table, choose AIANAloneData. For Input Join Field, choose Geographic Identification Code.

3. For Join Table, choose Census Demonstration Products v1 October 2019\American Indian Areas v1. For Join Table Field, choose Geographic Identification Code.

4. For Transfer Field, choose Total Population: American Indian and Alaska Native alone (DP).

Join Field parameters for DP version 1

The DP field has the demonstration product values.

5. Click Run.

Next, you’ll change the field name to reflect that it is version 1 demonstration product data.

6. Search for and open the Alter Field tool.

7. For Input Table, choose AIANAloneData. For Field Name, choose Total Population: American Indian and Alaska Native alone (DP).

You’ll also change the field name and field alias.

8. For New Field Name, type AIANv1DP. For New Field Alias, type Total Population: V1 American Indian and Alaska Native alone (DP).

 

Alter field tool parameters

You’ve added the version 1 data. You’ll run the Join Field and Alter Field tools again to add version 2 and version 3 data.

9. Click the History tab and double-click Join Field.

Re-run a tool from the History pane

The Join Field tool opens in the Geoprocessing pane with the parameters set the way they were the last time the tool ran.

10. Change the Join Table parameter to Census Demonstration Products v2 May 2020\American Indian Areas v2 (version 2 instead of version 1). Click Run.

The field from the version 2 demonstration product is added to your AIANAloneData layer.

11. Click the History tab and double-click Alter Field.

The Alter Field tool opens with the parameters set. You’ll change the field name and field alias so they identify the data as version 2.

12. Change the New Field Name parameter to AIANv2DP. Change the New Field Alias field to Total Population: V2 American Indian and Alaska Native alone (DP). Click Run.

13. Use the Join Field and Alter Field entries in the History tab to add the version 3 data values.

You won’t need the Census Demonstration layers any longer, so you’ll remove them from the Contents.

14. In the Contents pane, select the Census Demonstration Products layers. Right-click and choose Remove.

Remove layers from Contents

15. On the Quick Access Toolbar, click the Save button to save your project.

Button to save the project

 

Calculate the difference between version 3 DP and the SF1 data values

Next, you’ll create fields showing the difference between version 3 of the demonstration product data and the published data.

1. In the Contents pane, right-click the AIANAloneData layer and choose Attribute Table. Scroll right in the table to confirm you have the id fields, the SF1 field, and the V1, V2, and V3 DP fields.

2. Close the table.

3. In the Geoprocessing pane, find and open the Calculate Field tool.

4. For the Input Table parameter, choose AIANAloneData.

By providing a field name that doesn’t already exist, you’ll create a new field. You’ll populate the new field with the difference between the version 3 demonstration product data value and the published 2010 data value.

5. For Field Name, type DiffV3. Click anywhere outside of the Field Name parameter to reveal the Field Type parameter.

6. For Field Type, choose Long.

7. Set the Expression parameter to: !AIANv3DP! – !sf_americanindian! 

Calculate differences

8. Click Run.

Next, you’ll compute the percentage differences. To avoid division by zero, you’ll initialize the new percentage field to 0.0, select the polygons with populations greater than 1, and then perform the division.

9. Search for and re-open the Calculate Field tool (this clears the parameters).

10. For Input Table, choose AIANAloneData. For Field Name, type PercentV3.

11. Click anywhere outside the Field Name parameter to expose the Field Type parameter. For Field Type, choose Float.

12. For Expression, type 0.0.

13. Click Run.

14. Search for and open the Select Layer by Attribute tool.

15. For Input Rows, choose AIANAloneData.

16. Click the New expression button and create the expression: Where Total Population: American Indian and Alaskan Native alone (SF1) is greater than 0.

Select areas with non-zero population

17. Click Run.

The tool selects 596 features.

18. In the History pane, open the most recent Calculate Field entry (near the top of the pane). Change Expression to: (!DiffV3! / !sf_americanindian!) * 100.0

19. Click Run.

For the polygons where the population is zero, the percentage will remain 0.0. For all other polygons, the value will reflect the proportion or percentage the difference represents of the total population. (A difference of 10 for a place with 1000 people, 1.0, is not as severe as a difference of 10 for a place with 2 people, 500.0).

20. At the top of the project, click the Map tab. In the Selection group, click Clear.

 

Clear selected features

21. Save the project.

 

Share the layer

Next, you’ll share the layer to your ArcGIS organizational account.

1. In the Contents pane, right-click the AIANAloneData, point to Sharing, and choose Share As Web Layer.

2. For Name, type AIANAloneData and add your name or initials. Type a Summary (see below).

3. For Tags, type American Indian Areas, Total Population: American Indian and Alaska Native alone (SF1), 2010 Census, Demonstration Product Data, DPv1, DPv2, and DPv3.

4. Set the Layer Type to Feature. If you want the web layer to be shared to a particular folder in your ArcGIS Online account, specify the folder name.

5. Set the Share with parameter as appropriate. If you will be sharing your final web map with Everyone, for example, you will want to share this web layer with Everyone as well.

 

Parameters to share as web layer

6. Click Publish.

 

Create a web map

You’ll sign into your ArcGIS organizational account and open the web layer you just shared. Then, you’ll symbolize the percent differences (or count differences if you prefer).

1. Sign in to your ArcGIS organizational account. In My Content, locate the web layer you shared.

2. Click the hosted feature layer.

List of ArcGIS Online contents

3. Click Open in Map Viewer. Zoom in to see the polygons better.

4. Click Basemap and choose Light Gray Canvas so the polygons will stand out more.

5. In the Contents pane, point to the layer and click the Change Style button.

Change style button

6. Choose PercentV3 for the attribute to show. Choose Counts and Amounts (Color) for the drawing style by clicking Options.

7. For Theme, choose Above and Below.

8. Slide the middle handle to 0 (or click the value next to the middle handle and type 0). Set the top handle to 100 and the bottom handle to -100.

Setting symbology thresholds

You’ll want to select a color ramp with bright colors on the top and bottom and with no transparency for the middle values.

9. Click Symbols. Select the last color ramp in the palette and click the Invert Color Ramp button. Click OK.

Color ramps to choose from

10. Click OK. Click Done.

Most of the map is yellow (meaning the percentage of error is very low). If you zoom in to the western part of Nevada, however, you see some low and high percentages. There are other locations with high and low percentages as well.

Map of percent error

 

Customize the pop-ups

As the final step, you’ll design what the pop-ups look like and will make the web map accessible to everyone.

1. Point to the AIANAloneData layer and click the More Options button (the three dots). Choose Configure Pop-up.

2. Click Configure Attributes.

3. Uncheck {OBJECTID}, {id}, {name}, {Shape_Area}, and {Shape_Length}.

4. Click the Total Population: American Indian and Alaskan Native alone (SF1) field and change Format to 0 decimal places.

 

Set the numeric format

5. Set Format to 0 decimal places for all 3 demonstration product (DP) fields also.

6. Change Field Alias for the {DiffV3} field to DPv3 – SF1.

7. Change Field Alias for the {PercentV3) to Percent Error. Click OK.

8. Click OK to exit the Configure Pop-up pane.

9. Click one of the polygons on the map to see the pop-up.

Configured pop-up

Finally, you’ll save the web map you created and share it with everyone.

10. Close the pop-up, then click the Save button and choose Save As.

Button to save web map

11. In the Save Map pane, for Title, type American Indian Areas, American Indian and Alaskan Native Population. For Tags, type Census, Demonstration Products, and Differential Privacy evaluation. For Summary, type This map compares the three versions of the demonstration products census is creating to protect privacy against published 2010 census data. Use it to determine how their new privacy protection system will impact your most important workflows.

12. Click Save Map.

13. Click the Share button (next to the Save button). Check Everyone.

14. Click OK for the note telling you to use the URL to provide a direct link to the map. Close the Share pane.

15. Just to be sure the permissions are correct, copy the Link to this map path and send it to someone. Ask them to confirm they can view the map and pop-ups.

 

Next steps:

Repeat this workflow for other variables or other geometries. Use both the SF1 and the DP data in your most important workflows and evaluate the differences. Be sure to report your concerns and findings to the team at the Census Bureau working on the Disclosure Avoidance System. (2020DAS@census.gov)

About the author

Dr. Lauren Griffin has more than 34 years’ experience in software development, spatial analysis, and GIS. She currently works at ESRI as a product specialist on the Geoprocessing and Analysis team where she is responsible for software support, education, documentation, and development of analytical workflows using ArcGIS.

Connect:

Leave a Reply

Please Login to comment

Next Article

Steal this Ice Mountain hillshade style for ArcGIS Pro please

Read this article