{"id":2192742,"date":"2023-12-14T10:00:17","date_gmt":"2023-12-14T18:00:17","guid":{"rendered":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&#038;p=2192742"},"modified":"2024-06-11T05:41:46","modified_gmt":"2024-06-11T12:41:46","slug":"end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis","status":"publish","type":"blog","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis","title":{"rendered":"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS"},"author":154341,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","format":"standard","meta":{"_acf_changed":false,"_searchwp_excluded":""},"categories":[23341],"tags":[760452,35661,24341,30241,759592],"industry":[],"product":[36841,36561],"class_list":["post-2192742","blog","type-blog","status-publish","format-standard","hentry","category-analytics","tag-data-engineering","tag-machine-learning","tag-python","tag-r","tag-spatial-data-science","product-api-python","product-arcgis-pro"],"acf":{"authors":[{"ID":154341,"user_firstname":"Nicholas","user_lastname":"Giner","nickname":"Nick Giner","user_nicename":"nginer","display_name":"Nicholas Giner","user_email":"NGiner@esri.com","user_url":"","user_registered":"2021-01-07 14:31:25","user_description":"Nick Giner is a Product Manager for Spatial Analysis and Data Science.  Prior to joining Esri in 2014, he completed Bachelor\u2019s and PhD degrees in Geography from Penn State University and Clark University, respectively. In his spare time, he likes to play guitar, golf, cook, cut the grass, and read\/watch shows about history.","user_avatar":"<img data-del=\"avatar\" src='https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2021\/01\/headshot-e1610030307989-213x200.jpeg' class='avatar pp-user-avatar avatar-96 photo ' height='96' width='96'\/>"}],"short_description":"This is the fourth in a series of blogs that showcase an end-to-end spatial data science workflow for clustering US precipitation regions.","flexible_content":[{"acf_fc_layout":"content","content":"<h2>Introduction<\/h2>\n<p>In the previous blogs in this series, we used a combination of R and Python to access, clean, wrangle, convert, engineer, and otherwise transform a 30-year time series of ~11,000 daily precipitation rasters into one feature class containing 16 different seasonal precipitation-related variables.\u00a0 The final feature class is a gridded point dataset covering the contiguous United States.\u00a0 Each of the 481,631 grid points corresponds to the raster cell centroid of the original 4km by 4km PRISM precipitation dataset.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2195712,"id":2195712,"title":"gridded_dataset","filename":"gridded_dataset-3.jpg","filesize":232983,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/gridded_dataset-4","alt":"","author":"154341","description":"","caption":"Example of the 4km by 4km gridded dataset overlaid on the average precipitation raster from 1981-2010.","name":"gridded_dataset-4","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:40:20","modified":"2023-12-12 19:40:26","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1166,"height":758,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","medium-width":401,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","medium_large-width":768,"medium_large-height":499,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","large-width":1166,"large-height":758,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","1536x1536-width":1166,"1536x1536-height":758,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","2048x2048-width":1166,"2048x2048-height":758,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3-715x465.jpg","card_image-width":715,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/gridded_dataset-3.jpg","wide_image-width":1166,"wide_image-height":758}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Each of these points also contains 16 columns representing the 30-year seasonal averages of the four precipitation variables calculated in Part #2 in the blog series.\u00a0 Using ArcGIS Pro\u2019s <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/help\/analysis\/geoprocessing\/data-engineering\/what-is-data-engineering.htm\">Data Engineering<\/a> view, we can easily see and explore the summary statistics and distributions of each variable in the feature class.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2195732,"id":2195732,"title":"DE_view_1","filename":"DE_view_1-1.jpg","filesize":207542,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/de_view_1-2","alt":"","author":"154341","description":"","caption":"ArcGIS Pro Data Engineering view. The 4km by 4km dataset contains 481,631 locations.","name":"de_view_1-2","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:41:08","modified":"2023-12-12 19:41:14","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1133,"height":565,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","medium-width":464,"medium-height":231,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","medium_large-width":768,"medium_large-height":383,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","large-width":1133,"large-height":565,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","1536x1536-width":1133,"1536x1536-height":565,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","2048x2048-width":1133,"2048x2048-height":565,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1-826x412.jpg","card_image-width":826,"card_image-height":412,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_1-1.jpg","wide_image-width":1133,"wide_image-height":565}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Now to the fun part\u2026spatial analysis!<\/p>\n<h2>Post-processing and spatial aggregation<\/h2>\n<p>If we refer back to the overall workflow graphic shown in Part #1 of this blog series, you\u2019ll note that the third section \u201cData Post-Processing\u201d involves a spatial aggregation step.\u00a0 The idea here is to coarsen the dataset (e.g. reduce the resolution) to remove noise and make it easier to work with later in the workflow.\u00a0 In our case, we need to spatially aggregate the gridded dataset by averaging all 16 precipitation variables from a 4km by 4km grid to a 16km by 16km grid.\u00a0 At the granular level, this means taking the spatial average of 16 grid points and creating a new grid point at their centroid.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2192802,"id":2192802,"title":"aggregation_v3","filename":"aggregation_v3.jpg","filesize":302296,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/aggregation_v3","alt":"","author":"154341","description":"","caption":"","name":"aggregation_v3","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 12:45:07","modified":"2023-12-12 12:45:07","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1888,"height":832,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3.jpg","medium-width":464,"medium-height":204,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3.jpg","medium_large-width":768,"medium_large-height":338,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3.jpg","large-width":1888,"large-height":832,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3-1536x677.jpg","1536x1536-width":1536,"1536x1536-height":677,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3.jpg","2048x2048-width":1888,"2048x2048-height":832,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3-826x364.jpg","card_image-width":826,"card_image-height":364,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_v3.jpg","wide_image-width":1888,"wide_image-height":832}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>In ArcGIS Pro, this spatial aggregation process involves three geoprocessing tools:<\/p>\n<ul>\n<li><strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/tool-reference\/conversion\/point-to-raster.htm\">Point to Raster<\/a> <\/strong>\u2013 converts the 4km grid points to a raster, where we can specify the cell size. In our case, we specify 0.166 to represent a 16km by 16km raster cell.\u00a0 The <em>Value field<\/em> parameter specifies the attribute value to aggregate (e.g. one of the 16 precipitation variables).\u00a0 The <em>Cell assignment type<\/em> parameter specifies which value will be given to each new raster cell based on the underlying points being aggregated, which in our case is the spatial average (mean) of each group of 16 grid points.<\/li>\n<li><strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/tool-reference\/conversion\/raster-to-point.htm\">Raster to Point<\/a> <\/strong>\u2013 converts the 16km by 16km raster dataset back to points, for the given precipitation attribute.<\/li>\n<li><strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/tool-reference\/data-management\/alter-field-properties.htm\">Alter Field<\/a><\/strong> \u2013 used to rename fields in a feature class.<\/li>\n<\/ul>\n"},{"acf_fc_layout":"image","image":{"ID":2196462,"id":2196462,"title":"tool_flow","filename":"tool_flow-1.jpg","filesize":126679,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/tool_flow-2","alt":"","author":"154341","description":"","caption":"","name":"tool_flow-2","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 22:18:26","modified":"2023-12-12 22:18:26","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1866,"height":595,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1.jpg","medium-width":464,"medium-height":148,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1.jpg","medium_large-width":768,"medium_large-height":245,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1.jpg","large-width":1866,"large-height":595,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1-1536x490.jpg","1536x1536-width":1536,"1536x1536-height":490,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1.jpg","2048x2048-width":1866,"2048x2048-height":595,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1-826x263.jpg","card_image-width":826,"card_image-height":263,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_flow-1.jpg","wide_image-width":1866,"wide_image-height":595}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Using the 481,361 row, 4km gridded points dataset as our input, we run the three tools in succession and end up with a 16km gridded points dataset containing 30,665 locations.\u00a0 Easy enough right?<\/p>\n<p>Not so fast! Remember we have 16 precipitation variables to work with, so we need to run the three tools 16 different times each.\u00a0 That equates to 48 individual geoprocessing tool runs, and all the accompanying steps: open the tool, fill out the parameters, run the tool, close the tool, then repeat 47 more times.\u00a0 There has to be a better way, right?<\/p>\n<h2>Automation opportunity #1: ModelBuilder<\/h2>\n<p>Running a series of geoprocessing tools in sequence is a perfect opportunity to use <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/help\/analysis\/geoprocessing\/modelbuilder\/what-is-modelbuilder-.htm\">ModelBuilder<\/a>.\u00a0 To do so, I\u2019ve created a new toolbox in the Catalog pane and within it, a new blank model.\u00a0 In the model view, I\u2019ve dragged and dropped my input layers and the three geoprocessing tools I need, did a bit of renaming and reformatting, and then strung them all together through a series of connections.\u00a0 I\u2019ve also created variables (blue ovals) from several of the tool parameters so I can specify which values I want to use in the tools.\u00a0 Last, I can right-click on each of the model variables to flag them as <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/help\/analysis\/geoprocessing\/modelbuilder\/model-parameters.htm\">model parameters<\/a> so that they are exposed as geoprocessing tool parameters and behave as such after the model is saved.\u00a0 I can then open the model as a tool from within the Geoprocessing pane, interactively input the desired data and parameters, and run it like any other tool.<\/p>\n"},{"acf_fc_layout":"sidebar","content":"<p><strong>Note:<\/strong> ModelBuilder is a super powerful, awesome capability of ArcGIS Pro.\u00a0 This short paragraph and example only scratches the surface of what\u2019s possible.\u00a0 To learn more, check out this 3-minute <a href=\"https:\/\/www.youtube.com\/watch?v=tq7WPZ18jlk&amp;t=1s\">intro video<\/a>, the ModelBuilder <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/help\/analysis\/geoprocessing\/modelbuilder\/what-is-modelbuilder-.htm\">documentation<\/a>, and the brand new <a href=\"https:\/\/modelbuilder-analysis-1.hub.arcgis.com\/\">ModelBuilder Resources<\/a> page.<\/p>\n","image_reference":false,"layout":"standard","image_reference_figure":"","snippet":"","spotlight_name":"","section_title":"","position":"Center","spotlight_image":false},{"acf_fc_layout":"image","image":{"ID":2195772,"id":2195772,"title":"modelbuilder","filename":"modelbuilder-1.jpg","filesize":88902,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/modelbuilder-2","alt":"","author":"154341","description":"","caption":"","name":"modelbuilder-2","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:44:39","modified":"2023-12-12 19:44:39","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1189,"height":542,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","medium-width":464,"medium-height":212,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","medium_large-width":768,"medium_large-height":350,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","large-width":1189,"large-height":542,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","1536x1536-width":1189,"1536x1536-height":542,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","2048x2048-width":1189,"2048x2048-height":542,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1-826x377.jpg","card_image-width":826,"card_image-height":377,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/modelbuilder-1.jpg","wide_image-width":1189,"wide_image-height":542}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"image","image":{"ID":2195822,"id":2195822,"title":"aggregation_tool","filename":"aggregation_tool.jpg","filesize":116975,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/aggregation_tool","alt":"","author":"154341","description":"","caption":"Example of the three geoprocessing tools connected together and parameterized in ModelBuilder, then opened as a geoprocessing model tool.","name":"aggregation_tool","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:49:58","modified":"2023-12-14 01:19:33","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1362,"height":760,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","medium-width":464,"medium-height":259,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","medium_large-width":768,"medium_large-height":429,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","large-width":1362,"large-height":760,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","1536x1536-width":1362,"1536x1536-height":760,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","2048x2048-width":1362,"2048x2048-height":760,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool-826x461.jpg","card_image-width":826,"card_image-height":461,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/aggregation_tool.jpg","wide_image-width":1362,"wide_image-height":760}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>It\u2019s incredibly easy, helpful, and certainly a time saver to run a model as a tool.\u00a0 However, it can only run the three geoprocessing tools on <em>one precipitation variable at a time<\/em>, so that means I need to run it 16 times.\u00a0 Another opportunity to automate!<\/p>\n<h2>Automation opportunity #2: Python<\/h2>\n<p>This time we\u2019ll automate using some fundamental Python coding patterns and techniques.\u00a0 As such, we will transition our work back to the ArcGIS Notebook directly in ArcGIS Pro.<\/p>\n<p>First, let\u2019s get set up by importing our necessary Python libraries.<\/p>\n<ul>\n<li><a href=\"https:\/\/pandas.pydata.org\/\">pandas<\/a> \u2013 working with and manipulating tabular data<\/li>\n<li><a href=\"https:\/\/developers.arcgis.com\/python\/\">arcgis<\/a> \u2013 the ArcGIS API for Python, used to convert ArcGIS data to Pandas DataFrames<\/li>\n<li><a href=\"https:\/\/docs.python.org\/3\/library\/os.html\">os<\/a> \u2013 working with operating system files and file directories<\/li>\n<li><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/arcpy\/get-started\/what-is-arcpy-.htm\">arcpy<\/a> \u2013 the ArcGIS geoprocessing framework. Provides Pythonic access to the ~2,000+ geoprocessing tools, as well as functions for data management, conversion, and map automation<\/li>\n<\/ul>\n"},{"acf_fc_layout":"image","image":{"ID":2195852,"id":2195852,"title":"python_libs_2","filename":"python_libs_2.jpg","filesize":14462,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/python_libs_2","alt":"","author":"154341","description":"","caption":"","name":"python_libs_2","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:53:44","modified":"2023-12-12 19:53:44","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":269,"height":123,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2-213x123.jpg","thumbnail-width":213,"thumbnail-height":123,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","medium-width":269,"medium-height":123,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","medium_large-width":269,"medium_large-height":123,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","large-width":269,"large-height":123,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","1536x1536-width":269,"1536x1536-height":123,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","2048x2048-width":269,"2048x2048-height":123,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","card_image-width":269,"card_image-height":123,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/python_libs_2.jpg","wide_image-width":269,"wide_image-height":123}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Next, we need to access our gridded dataset so that we can work with it in the Notebook.\u00a0 In a notebook cell, we can set the <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/tool-reference\/environment-settings\/current-workspace.htm\">Current Workspace<\/a> to be the geodatabase that contains all of our geoprocessing inputs and outputs.\u00a0 <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/arcpy\/geoprocessing_and_python\/using-environment-settings.htm\">Environment settings <\/a>such as this will now be honored by all subsequently run geoprocessing tools, regardless of their specific functionality.\u00a0 We can also set the<em> overwriteOutput<\/em> environment setting to <strong>True<\/strong> so that we can overwrite all geoprocessing outputs and don\u2019t have to create a new file each time we run a tool.<\/p>\n<p>We can create variables for the file geodatabase and the file name of the gridded dataset, then run the cell.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2195862,"id":2195862,"title":"env_settings","filename":"env_settings.jpg","filesize":70013,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/env_settings","alt":"","author":"154341","description":"","caption":"","name":"env_settings","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:54:31","modified":"2023-12-12 19:54:31","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":712,"height":246,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","medium-width":464,"medium-height":160,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","medium_large-width":712,"medium_large-height":246,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","large-width":712,"large-height":246,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","1536x1536-width":712,"1536x1536-height":246,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","2048x2048-width":712,"2048x2048-height":246,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","card_image-width":712,"card_image-height":246,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/env_settings.jpg","wide_image-width":712,"wide_image-height":246}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Because we need to run our geoprocessing model tool repeatedly (e.g. 16 times, once for each precipitation variable), let\u2019s think of how to do this Pythonically.\u00a0 In other words, how can we use Python code to iterate over a function repeatedly?<\/p>\n<p>One of the most common ways to perform this type of task in Python is using a \u201c<a href=\"https:\/\/wiki.python.org\/moin\/ForLoop\"><em>for<\/em> loop<\/a>\u201d, which is a block of code that is repeated a specified number of times.\u00a0 The number of times is dictated by the elements of an iterable object, e.g. items in a list, characters in a string, key\/value pairs in a dictionary, etc.).\u00a0 Take this quick example from <a href=\"https:\/\/learnpython.com\/blog\/python-list-loop\/\">LearnPython.com<\/a>:<\/p>\n<p>Here we have a list of different fruits.\u00a0 We create a <em>for<\/em> loop to iterate over each item in the list, then run the <a href=\"https:\/\/realpython.com\/python-print\/\"><em><strong>print<\/strong><\/em><\/a> function four times, once on each of the fruits in the list:<\/p>\n<p>&nbsp;<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2195892,"id":2195892,"title":"fruit_loop","filename":"fruit_loop.jpg","filesize":34401,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/fruit_loop","alt":"","author":"154341","description":"","caption":"A fruit loop.","name":"fruit_loop","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:56:04","modified":"2023-12-14 01:27:29","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":524,"height":224,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","medium-width":464,"medium-height":198,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","medium_large-width":524,"medium_large-height":224,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","large-width":524,"large-height":224,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","1536x1536-width":524,"1536x1536-height":224,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","2048x2048-width":524,"2048x2048-height":224,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","card_image-width":524,"card_image-height":224,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/fruit_loop.jpg","wide_image-width":524,"wide_image-height":224}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Now let\u2019s apply this same logic to our automation task.\u00a0 Instead of the list of fruits, however, we\u2019ll use a list of the 16 precipitation variables.\u00a0 Instead of the <strong><em>print<\/em><\/strong>\u00a0function, we\u2019ll use our Aggregation geoprocessing model tool.<\/p>\n<p>First, we\u2019ll use the ArcPy <strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/arcpy\/functions\/listfields.htm\"><em>ListFields<\/em><\/a><\/strong> function to generate a Python list of all the attribute fields in our gridded point dataset.\u00a0 In this example, we are using a Python <a href=\"https:\/\/learnpython.com\/blog\/python-lists-list-comprehension-new-year-resolutions\/\">list comprehension<\/a>, which is essentially a <em>for<\/em> loop in a single line of code.<\/p>\n<p>Because we only want to iterate over the 16 precipitation variables, we\u2019ll create a new list excluding the OBJECTID, geometry, and other location-related fields.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2195912,"id":2195912,"title":"var_list","filename":"var_list.jpg","filesize":78143,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/var_list","alt":"","author":"154341","description":"","caption":"","name":"var_list","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 19:58:08","modified":"2023-12-12 19:58:08","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":776,"height":246,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","medium-width":464,"medium-height":147,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","medium_large-width":768,"medium_large-height":243,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","large-width":776,"large-height":246,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","1536x1536-width":776,"1536x1536-height":246,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","2048x2048-width":776,"2048x2048-height":246,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","card_image-width":776,"card_image-height":246,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/var_list.jpg","wide_image-width":776,"wide_image-height":246}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>We\u2019ll now take advantage of another powerful, yet overlooked functionality in ArcGIS Pro\u2014the ability to copy the Python syntax from a geoprocessing tool.<\/p>\n<p>First, let\u2019s make sure that we have a list that we can iterate over.\u00a0 We can create a quick <em>for <\/em>loop to print out each item in \u201cvar_list\u201d.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2195952,"id":2195952,"title":"quick_loop","filename":"quick_loop-1.jpg","filesize":78577,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/quick_loop-2","alt":"","author":"154341","description":"","caption":"","name":"quick_loop-2","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 20:00:12","modified":"2023-12-12 20:00:12","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":603,"height":459,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","medium-width":343,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","medium_large-width":603,"medium_large-height":459,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","large-width":603,"large-height":459,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","1536x1536-width":603,"1536x1536-height":459,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","2048x2048-width":603,"2048x2048-height":459,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","card_image-width":603,"card_image-height":459,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/quick_loop-1.jpg","wide_image-width":603,"wide_image-height":459}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Looks good! Now, we\u2019ll open our <strong>Aggregation Tool<\/strong> and fill out the parameters as if we were going to run it one time.\u00a0 In the <em>Run<\/em> menu, you can select <a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/help\/analysis\/geoprocessing\/basics\/run-geoprocessing-tools.htm#ESRI_SECTION1_14C98A99F1BA47FB976F252C8FC0BE50\">Copy Python Command<\/a> to copy the syntax of the parameterized tool, then paste it into your Python scripting environment of choice.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196242,"id":2196242,"title":"copy_python","filename":"copy_python.jpg","filesize":67807,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/copy_python","alt":"","author":"154341","description":"","caption":"","name":"copy_python","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:08:51","modified":"2023-12-12 21:08:51","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":572,"height":570,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","medium-width":262,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","medium_large-width":572,"medium_large-height":570,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","large-width":572,"large-height":570,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","1536x1536-width":572,"1536x1536-height":570,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","2048x2048-width":572,"2048x2048-height":570,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python-467x465.jpg","card_image-width":467,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/copy_python.jpg","wide_image-width":572,"wide_image-height":570}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>You can see from the code snippet below that the first line calls ArcPy\u2019s <strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/arcpy\/functions\/importtoolbox.htm\"><em>ImportToolbox<\/em><\/a><\/strong> function, which is a requirement because we are using a custom toolbox.\u00a0 Below this, we can see the call to the <strong>Aggregation Tool<\/strong> in our custom toolbox, complete with all the parameters we filled out.\u00a0 If we were to now run this notebook cell, it would run the <strong>Aggregation Tool<\/strong> one time, in this case on one of the 16 precipitation variables\u2014&#8221;precip_fall\u201d.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196292,"id":2196292,"title":"toolbox_tool","filename":"toolbox_tool.jpg","filesize":28796,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/toolbox_tool","alt":"","author":"154341","description":"","caption":"","name":"toolbox_tool","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:20:36","modified":"2023-12-12 21:20:36","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":436,"height":337,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","medium-width":338,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","medium_large-width":436,"medium_large-height":337,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","large-width":436,"large-height":337,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","1536x1536-width":436,"1536x1536-height":337,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","2048x2048-width":436,"2048x2048-height":337,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","card_image-width":436,"card_image-height":337,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_tool.jpg","wide_image-width":436,"wide_image-height":337}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"image","image":{"ID":2196302,"id":2196302,"title":"toolbox_script","filename":"toolbox_script.jpg","filesize":109259,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/toolbox_script","alt":"","author":"154341","description":"","caption":"","name":"toolbox_script","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:22:38","modified":"2023-12-12 21:22:38","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1072,"height":273,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","medium-width":464,"medium-height":118,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","medium_large-width":768,"medium_large-height":196,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","large-width":1072,"large-height":273,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","1536x1536-width":1072,"1536x1536-height":273,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","2048x2048-width":1072,"2048x2048-height":273,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script-826x210.jpg","card_image-width":826,"card_image-height":210,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/toolbox_script.jpg","wide_image-width":1072,"wide_image-height":273}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>However, we want to use a <em>for<\/em> loop to run the tool on each individual precipitation variable.\u00a0 In a new notebook cell below, copy the contents of the previous cell, then make the following changes:<\/p>\n<ol>\n<li>Insert the %%time <a href=\"https:\/\/ipython.readthedocs.io\/en\/stable\/interactive\/magics.html\">magic command<\/a> at the top of the cell to calculate the runtime of the cell.<\/li>\n<li>Insert comments about importing the custom toolbox and creating a <em>for<\/em> loop (optional).<\/li>\n<li>Above the <strong>Aggregation Tool<\/strong> syntax, insert a <em>for<\/em> loop to iterate over the list of the 16 precipitation variables.<\/li>\n<li>Indent the <strong>Aggregation Tool<\/strong> syntax such that it sits within the <em>for <\/em>loop<em>.<\/em><\/li>\n<li>Replace the <em>Input_gridded_point_dataset__4km_<\/em> parameter with the \u201cfile_name\u201d variable, which is the 4km by 4km gridded dataset.<\/li>\n<li>Replace the hard coded values in the <em>Precipitation_variable<\/em>, <em>Raster_output<\/em>, <em>Aggregated_gridded_point_dataset__16km_<\/em>, <em>New_field_name<\/em>, and <em>New_field_alias<\/em> parameters with \u201cvar\u201d. This will ensure that the <em>for<\/em> loop not only runs the <strong>Aggregation Tool<\/strong> 16 times, once for each precipitation variable in \u201cvar_list\u201d, but that it also makes the corresponding changes to the new rasters and point datasets that are being created.\u00a0 For example, every new precipitation raster that is created will be named with the precipitation variable of interest and appended with underscores and the word \u201c16km_ras_Py\u201d (e.g. \u201cprecip_fall_16km_ras_Py\u201d).\u00a0 The same concept applies to the output point files.<\/li>\n<\/ol>\n"},{"acf_fc_layout":"image","image":{"ID":2197422,"id":2197422,"title":"tool_loop","filename":"tool_loop-1.jpg","filesize":107647,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/tool_loop-2","alt":"","author":"154341","description":"","caption":"","name":"tool_loop-2","status":"inherit","uploaded_to":2192742,"date":"2023-12-13 17:17:41","modified":"2023-12-13 17:17:41","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":751,"height":439,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","medium-width":446,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","medium_large-width":751,"medium_large-height":439,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","large-width":751,"large-height":439,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","1536x1536-width":751,"1536x1536-height":439,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","2048x2048-width":751,"2048x2048-height":439,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","card_image-width":751,"card_image-height":439,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/tool_loop-1.jpg","wide_image-width":751,"wide_image-height":439}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>After running the notebook cell (and stepping away for a few minutes to stretch my legs), the loop completes having run the <strong>Aggregation Tool<\/strong> 16 times.\u00a0 Looking in my geodatabase, I see the output rasters and output point feature classes corresponding to each of the 16 precipitation variables.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196352,"id":2196352,"title":"files","filename":"files.jpg","filesize":212962,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/files-3","alt":"","author":"154341","description":"","caption":"","name":"files-3","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:39:12","modified":"2023-12-12 21:39:12","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1063,"height":923,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","medium-width":301,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","medium_large-width":768,"medium_large-height":667,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","large-width":1063,"large-height":923,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","1536x1536-width":1063,"1536x1536-height":923,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","2048x2048-width":1063,"2048x2048-height":923,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files-536x465.jpg","card_image-width":536,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/files.jpg","wide_image-width":1063,"wide_image-height":923}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Each point feature class is a spatially aggregated version of the original 4km by 4km gridded dataset, representing a single precipitation variable in a 16km by 16km grid. \u00a0The graphic below shows the Data Engineering view of four feature classes from the average precipitation variable, one for each season.\u00a0 Note that the schema is exactly the same in each feature class, including the \u201cpointid\u201d which represents a unique identifier.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196362,"id":2196362,"title":"point_files","filename":"point_files.jpg","filesize":152005,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/point_files","alt":"","author":"154341","description":"","caption":"Example attribute tables from four of the precipitation variables: Average total precipitation (mm) for each season over the 30-year time period.  Note that all four have the same schema.","name":"point_files","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:46:49","modified":"2023-12-13 17:19:19","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1035,"height":861,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","medium-width":314,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","medium_large-width":768,"medium_large-height":639,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","large-width":1035,"large-height":861,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","1536x1536-width":1035,"1536x1536-height":861,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","2048x2048-width":1035,"2048x2048-height":861,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files-559x465.jpg","card_image-width":559,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/point_files.jpg","wide_image-width":1035,"wide_image-height":861}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>To combine all 16 of these feature classes into one, we\u2019ll use one more simple, yet powerful automation technique.<\/p>\n<h2>Automation opportunity #3: Batch geoprocessing<\/h2>\n<p><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/help\/analysis\/geoprocessing\/basics\/batch-geoprocessing.htm\">Batch geoprocessing<\/a> is a way to repeatedly run a geoprocessing tool using either different input datasets or different parameter settings.\u00a0 Batch mode is available in many geoprocessing tools and can be accessed by right-clicking on a tool after you have searched for it or located it within a toolbox.\u00a0 In my case, I will be using the <strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/tool-reference\/data-management\/join-field.htm\">Join Field<\/a> <\/strong>tool to join all 16 feature classes into one based on a common attribute, or \u201ckey\u201d field.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196382,"id":2196382,"title":"batch join","filename":"batch-join.jpg","filesize":53668,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/batch-join","alt":"","author":"154341","description":"","caption":"","name":"batch-join","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:49:20","modified":"2023-12-12 21:49:20","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":426,"height":392,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","medium-width":284,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","medium_large-width":426,"medium_large-height":392,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","large-width":426,"large-height":392,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","1536x1536-width":426,"1536x1536-height":392,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","2048x2048-width":426,"2048x2048-height":392,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","card_image-width":426,"card_image-height":392,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch-join.jpg","wide_image-width":426,"wide_image-height":392}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>First, we\u2019ll choose which parameter we want to batch.\u00a0 In this case, we are joining 15 feature classes to one, so we will choose <em>Join Table<\/em>.\u00a0 If we wanted to run this batch tool again, we do have the choice to save it, but for now we\u2019ll just make a temporary tool.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196392,"id":2196392,"title":"batch_join","filename":"batch_join.jpg","filesize":47193,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/batch_join","alt":"","author":"154341","description":"","caption":"","name":"batch_join","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:51:57","modified":"2023-12-12 21:51:57","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":650,"height":669,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","medium-width":254,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","medium_large-width":650,"medium_large-height":669,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","large-width":650,"large-height":669,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","1536x1536-width":650,"1536x1536-height":669,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","2048x2048-width":650,"2048x2048-height":669,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join-452x465.jpg","card_image-width":452,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join.jpg","wide_image-width":650,"wide_image-height":669}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>In the tool dialog, we\u2019ll choose one of the 16 feature classes as the input table, select \u201cpointid\u201d as the unique identifier field on which the join will occur, choose the remaining 15 feature classes as tables to batch join, then choose \u201cpointid\u201d again as the key field for the join tables.\u00a0 Then click run and let the software do the work for you!<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196402,"id":2196402,"title":"batch_join_field_tool","filename":"batch_join_field_tool.jpg","filesize":97638,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/batch_join_field_tool","alt":"","author":"154341","description":"","caption":"","name":"batch_join_field_tool","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 21:54:02","modified":"2023-12-12 21:54:02","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":427,"height":847,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","medium-width":132,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","medium_large-width":427,"medium_large-height":847,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","large-width":427,"large-height":847,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","1536x1536-width":427,"1536x1536-height":847,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","2048x2048-width":427,"2048x2048-height":847,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool-234x465.jpg","card_image-width":234,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/batch_join_field_tool.jpg","wide_image-width":427,"wide_image-height":847}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Last, we\u2019ll use the <strong><a href=\"https:\/\/pro.arcgis.com\/en\/pro-app\/latest\/tool-reference\/conversion\/export-features.htm\">Export Features<\/a> <\/strong>tool to create a new feature class.\u00a0 Here, we can add and delete fields, rename them, and reorder them, amongst many other options.\u00a0 In our case, we are using this tool not to convert to a different format, but rather to clean up the dataset and get it ready for the upcoming analysis steps.<\/p>\n<p>We can see from the Data Engineering view below that the final 16km by 16km gridded dataset contains the exact same schema as the original 4km by 4km dataset.\u00a0 However, because we have spatially aggregated from 4km to 16km, we&#8217;ve gone from 481,631 locations to 30,665 locations.\u00a0 This will be the input dataset to the upcoming machine learning steps.\u00a0 So let&#8217;s move on to the next blog!<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2196422,"id":2196422,"title":"DE_view_2","filename":"DE_view_2-1.jpg","filesize":205395,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/de_view_2-2","alt":"","author":"154341","description":"","caption":"ArcGIS Pro Data Engineering view. The 16km by 16km dataset contains 30,665 locations.","name":"de_view_2-2","status":"inherit","uploaded_to":2192742,"date":"2023-12-12 22:02:56","modified":"2023-12-12 22:03:19","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1194,"height":577,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","medium-width":464,"medium-height":224,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","medium_large-width":768,"medium_large-height":371,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","large-width":1194,"large-height":577,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","1536x1536-width":1194,"1536x1536-height":577,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","2048x2048-width":1194,"2048x2048-height":577,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1-826x399.jpg","card_image-width":826,"card_image-height":399,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/DE_view_2-1.jpg","wide_image-width":1194,"wide_image-height":577}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"sidebar","content":"<h2 style=\"text-align: left\">Spatial data science with R, Python, and ArcGIS<\/h2>\n<p>Here are the links to all the articles of the series:<\/p>\n<ul>\n<li><a href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-1-clustering-us-precipitation-regions\/\">Part 1<\/a>. Clustering US Precipitation Regions<\/li>\n<li><a href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-2-data-preparation-and-data-engineering-using-r\/\">Part 2<\/a>. Data preparation and data engineering using R<\/li>\n<li><a href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-3-data-preparation-and-data-engineering-using-python\/\">Part 3<\/a>. Data preparation and data engineering using Python<\/li>\n<li><a href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\/\" target=\"_blank\" rel=\"noopener\">Part 4<\/a>. Data preparation using spatial analysis and automation in ArcGIS<\/li>\n<li><a href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis\">Part 5<\/a>. Machine Learning: Cluster analysis using Python and ArcGIS<\/li>\n<\/ul>\n","image_reference":false,"layout":"standard","image_reference_figure":"","snippet":"","spotlight_name":"","section_title":"","position":"Center","spotlight_image":false}],"related_articles":"","card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/cluster_map_resized.jpg","wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/AdobeStock_96810852_fixed-3.png"},"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\" \/>\n<meta property=\"og:site_name\" content=\"ArcGIS Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/esrigis\/\" \/>\n<meta property=\"article:modified_time\" content=\"2024-06-11T12:41:46+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@ESRI\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"14 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":[\"Article\",\"BlogPosting\"],\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\"},\"author\":{\"name\":\"Nicholas Giner\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/2dc4741deea59d3274cfa775e52501b2\"},\"headline\":\"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS\",\"datePublished\":\"2023-12-14T18:00:17+00:00\",\"dateModified\":\"2024-06-11T12:41:46+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\"},\"wordCount\":13,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#organization\"},\"keywords\":[\"Data Engineering\",\"machine learning\",\"python\",\"r\",\"spatial data science\"],\"articleSection\":[\"Analytics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\",\"name\":\"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS\",\"isPartOf\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#website\"},\"datePublished\":\"2023-12-14T18:00:17+00:00\",\"dateModified\":\"2024-06-11T12:41:46+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.esri.com\/arcgis-blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#website\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/\",\"name\":\"ArcGIS Blog\",\"description\":\"Get insider info from Esri product teams\",\"publisher\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.esri.com\/arcgis-blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#organization\",\"name\":\"Esri\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png\",\"contentUrl\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png\",\"width\":400,\"height\":400,\"caption\":\"Esri\"},\"image\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/esrigis\/\",\"https:\/\/x.com\/ESRI\",\"https:\/\/www.linkedin.com\/company\/5311\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/2dc4741deea59d3274cfa775e52501b2\",\"name\":\"Nicholas Giner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2021\/01\/headshot-e1610030307989-213x200.jpeg\",\"contentUrl\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2021\/01\/headshot-e1610030307989-213x200.jpeg\",\"caption\":\"Nicholas Giner\"},\"description\":\"Nick Giner is a Product Manager for Spatial Analysis and Data Science. Prior to joining Esri in 2014, he completed Bachelor\u2019s and PhD degrees in Geography from Penn State University and Clark University, respectively. In his spare time, he likes to play guitar, golf, cook, cut the grass, and read\/watch shows about history.\",\"sameAs\":[\"www.linkedin.com\/in\/nicholas-giner-0282966b\",\"https:\/\/x.com\/NickGiner\"],\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/author\/nginer\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis","og_locale":"en_US","og_type":"article","og_title":"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS","og_url":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis","og_site_name":"ArcGIS Blog","article_publisher":"https:\/\/www.facebook.com\/esrigis\/","article_modified_time":"2024-06-11T12:41:46+00:00","twitter_card":"summary_large_image","twitter_site":"@ESRI","twitter_misc":{"Est. reading time":"14 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":["Article","BlogPosting"],"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#article","isPartOf":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis"},"author":{"name":"Nicholas Giner","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/2dc4741deea59d3274cfa775e52501b2"},"headline":"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS","datePublished":"2023-12-14T18:00:17+00:00","dateModified":"2024-06-11T12:41:46+00:00","mainEntityOfPage":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis"},"wordCount":13,"commentCount":0,"publisher":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#organization"},"keywords":["Data Engineering","machine learning","python","r","spatial data science"],"articleSection":["Analytics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis","url":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis","name":"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS","isPartOf":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#website"},"datePublished":"2023-12-14T18:00:17+00:00","dateModified":"2024-06-11T12:41:46+00:00","breadcrumb":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-pro\/analytics\/end-to-end-spatial-data-science-4-data-preparation-using-spatial-analysis-and-automation-in-arcgis#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.esri.com\/arcgis-blog\/"},{"@type":"ListItem","position":2,"name":"End-to-end spatial data science 4: Data preparation using spatial analysis and automation in ArcGIS"}]},{"@type":"WebSite","@id":"https:\/\/www.esri.com\/arcgis-blog\/#website","url":"https:\/\/www.esri.com\/arcgis-blog\/","name":"ArcGIS Blog","description":"Get insider info from Esri product teams","publisher":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.esri.com\/arcgis-blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.esri.com\/arcgis-blog\/#organization","name":"Esri","url":"https:\/\/www.esri.com\/arcgis-blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png","contentUrl":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png","width":400,"height":400,"caption":"Esri"},"image":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/esrigis\/","https:\/\/x.com\/ESRI","https:\/\/www.linkedin.com\/company\/5311\/"]},{"@type":"Person","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/2dc4741deea59d3274cfa775e52501b2","name":"Nicholas Giner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/image\/","url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2021\/01\/headshot-e1610030307989-213x200.jpeg","contentUrl":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2021\/01\/headshot-e1610030307989-213x200.jpeg","caption":"Nicholas Giner"},"description":"Nick Giner is a Product Manager for Spatial Analysis and Data Science. Prior to joining Esri in 2014, he completed Bachelor\u2019s and PhD degrees in Geography from Penn State University and Clark University, respectively. In his spare time, he likes to play guitar, golf, cook, cut the grass, and read\/watch shows about history.","sameAs":["www.linkedin.com\/in\/nicholas-giner-0282966b","https:\/\/x.com\/NickGiner"],"url":"https:\/\/www.esri.com\/arcgis-blog\/author\/nginer"}]}},"text_date":"December 14, 2023","author_name":"Nicholas Giner","author_page":"https:\/\/www.esri.com\/arcgis-blog\/author\/nginer","custom_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/12\/AdobeStock_96810852_fixed-3.png","primary_product":"ArcGIS Pro","tag_data":[{"term_id":760452,"name":"Data Engineering","slug":"data-engineering","term_group":0,"term_taxonomy_id":760452,"taxonomy":"post_tag","description":"","parent":0,"count":34,"filter":"raw"},{"term_id":35661,"name":"machine learning","slug":"machine-learning","term_group":0,"term_taxonomy_id":35661,"taxonomy":"post_tag","description":"","parent":0,"count":41,"filter":"raw"},{"term_id":24341,"name":"python","slug":"python","term_group":0,"term_taxonomy_id":24341,"taxonomy":"post_tag","description":"","parent":0,"count":171,"filter":"raw"},{"term_id":30241,"name":"r","slug":"r","term_group":0,"term_taxonomy_id":30241,"taxonomy":"post_tag","description":"","parent":0,"count":19,"filter":"raw"},{"term_id":759592,"name":"spatial data science","slug":"spatial-data-science","term_group":0,"term_taxonomy_id":759592,"taxonomy":"post_tag","description":"","parent":0,"count":17,"filter":"raw"}],"category_data":[{"term_id":23341,"name":"Analytics","slug":"analytics","term_group":0,"term_taxonomy_id":23341,"taxonomy":"category","description":"","parent":0,"count":1324,"filter":"raw"}],"product_data":[{"term_id":36841,"name":"ArcGIS API for Python","slug":"api-python","term_group":0,"term_taxonomy_id":36841,"taxonomy":"product","description":"","parent":36601,"count":151,"filter":"raw"},{"term_id":36561,"name":"ArcGIS Pro","slug":"arcgis-pro","term_group":0,"term_taxonomy_id":36561,"taxonomy":"product","description":"","parent":0,"count":2035,"filter":"raw"}],"primary_product_link":"https:\/\/www.esri.com\/arcgis-blog\/?s=#&products=arcgis-pro","_links":{"self":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/blog\/2192742","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/blog"}],"about":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/types\/blog"}],"author":[{"embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/users\/154341"}],"replies":[{"embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/comments?post=2192742"}],"version-history":[{"count":0,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/blog\/2192742\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/media?parent=2192742"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/categories?post=2192742"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/tags?post=2192742"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/industry?post=2192742"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/product?post=2192742"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}