{"id":2361222,"date":"2024-06-11T07:39:16","date_gmt":"2024-06-11T14:39:16","guid":{"rendered":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&p=2361222"},"modified":"2025-01-13T07:09:44","modified_gmt":"2025-01-13T15:09:44","slug":"end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis","status":"publish","type":"blog","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/analytics\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis","title":{"rendered":"End-to-end spatial data science 5: Machine learning: Cluster analysis in Python and ArcGIS"},"author":154341,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","format":"standard","meta":{"_acf_changed":false,"_searchwp_excluded":""},"categories":[23341],"tags":[760452,35661,24341,30241,759592],"industry":[],"product":[36841,36561],"class_list":["post-2361222","blog","type-blog","status-publish","format-standard","hentry","category-analytics","tag-data-engineering","tag-machine-learning","tag-python","tag-r","tag-spatial-data-science","product-api-python","product-arcgis-pro"],"acf":{"authors":[{"ID":154341,"user_firstname":"Nicholas","user_lastname":"Giner","nickname":"Nick Giner","user_nicename":"nginer","display_name":"Nicholas Giner","user_email":"NGiner@esri.com","user_url":"","user_registered":"2021-01-07 14:31:25","user_description":"Nick Giner is a Product Manager for Spatial Analysis and Data Science. Prior to joining Esri in 2014, he completed Bachelor\u2019s and PhD degrees in Geography from Penn State University and Clark University, respectively. In his spare time, he likes to play guitar, golf, cook, cut the grass, and read\/watch shows about history.","user_avatar":""}],"short_description":"This is the fifth in a series of blogs that showcase an end-to-end spatial data science workflow for clustering US precipitation regions.","flexible_content":[{"acf_fc_layout":"content","content":"

Introduction<\/h2>\n

At this point, we are finally ready to do some machine learning.\u00a0 Let this blog series be a reminder of just how much work may be required to prepare your data prior to getting to any of the fancy stuff!<\/p>\n

In the original paper<\/a>, the authors used two machine learning techniques back-to-back to create the final climate region map: Principal Components Analysis (PCA) and cluster analysis.\u00a0 PCA is a technique used to reduce highly dimensional datasets into smaller, uncorrelated dimensions, while attempting to maintain as much of the variation (e.g. information<\/em>) in the original data as possible.\u00a0 Cluster analysis is a technique used to group similar observations together.\u00a0 The objective is to take a dataset of individual data points and create subgroups (or clusters) of data points, where the data points within each cluster are more similar to each other than to the data points in other clusters.<\/p>\n"},{"acf_fc_layout":"sidebar","content":"

Note:<\/strong> It is a common practice to use PCA prior to cluster analysis when your data is highly dimensional.<\/p>\n","image_reference":false,"layout":"standard","image_reference_figure":"","snippet":"","spotlight_name":"","section_title":"","position":"Center","spotlight_image":false},{"acf_fc_layout":"content","content":"

Principal Components Analysis (PCA)<\/h2>\n

Recall that our final gridded precipitation dataset contains 16 precipitation variables (4 precipitation variables x 4 seasons), so we\u2019ll use PCA to reduce the dimensionality of this dataset into fewer components.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2368222,"id":2368222,"title":"dimensions","filename":"dimensions.jpg","filesize":59409,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/analytics\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis\/dimensions-3","alt":"","author":"154341","description":"","caption":"","name":"dimensions-3","status":"inherit","uploaded_to":2361222,"date":"2024-06-07 15:16:54","modified":"2024-06-07 15:16:54","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":982,"height":379,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","medium-width":464,"medium-height":179,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","medium_large-width":768,"medium_large-height":296,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","large-width":982,"large-height":379,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","1536x1536-width":982,"1536x1536-height":379,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","2048x2048-width":982,"2048x2048-height":379,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions-826x319.jpg","card_image-width":826,"card_image-height":319,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/dimensions.jpg","wide_image-width":982,"wide_image-height":379}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

Back in the notebook, we\u2019ll create a variable for our gridded precipitation dataset, then use ArcPy to make sure we have all the necessary fields.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2368232,"id":2368232,"title":"pca_input","filename":"pca_input.jpg","filesize":72124,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/analytics\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis\/pca_input","alt":"","author":"154341","description":"","caption":"","name":"pca_input","status":"inherit","uploaded_to":2361222,"date":"2024-06-07 15:21:42","modified":"2024-06-07 15:21:42","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1198,"height":250,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","medium-width":464,"medium-height":97,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","medium_large-width":768,"medium_large-height":160,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","large-width":1198,"large-height":250,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","1536x1536-width":1198,"1536x1536-height":250,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","2048x2048-width":1198,"2048x2048-height":250,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input-826x172.jpg","card_image-width":826,"card_image-height":172,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_input.jpg","wide_image-width":1198,"wide_image-height":250}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

In ArcGIS Pro, PCA can be performed using the Dimension Reduction<\/a><\/strong> tool.\u00a0 We\u2019ll specify our input and output datasets and choose all 16 of the precipitation variables for the \u201cFields\u201d parameter.\u00a0 The \u201cScale\u201d parameter gives you the option to transform all the input variables to the same scale, which can ensure that all variables contribute equally to the principal components.\u00a0 This is important in a situation like ours, where our precipitation variables have different units such as millimeters of precipitation, number of precipitation days, and Gini\/Lorenz Asymmetry Coefficients that range in value from 0 to just over 1.\u00a0 The tool also has the option to output eigenvalue and eigenvector tables, which can help you interpret the results of the PCA.\u00a0 For more information on how all this works behind the scenes, check out the documentation<\/a>.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2368342,"id":2368342,"title":"pca_tool","filename":"pca_tool.jpg","filesize":88032,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/analytics\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis\/pca_tool","alt":"","author":"154341","description":"","caption":"","name":"pca_tool","status":"inherit","uploaded_to":2361222,"date":"2024-06-07 15:51:48","modified":"2024-06-07 15:51:48","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1132,"height":370,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","medium-width":464,"medium-height":152,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","medium_large-width":768,"medium_large-height":251,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","large-width":1132,"large-height":370,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","1536x1536-width":1132,"1536x1536-height":370,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","2048x2048-width":1132,"2048x2048-height":370,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool-826x270.jpg","card_image-width":826,"card_image-height":270,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_tool.jpg","wide_image-width":1132,"wide_image-height":370}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

When you run a geoprocessing tool from within a notebook in ArcGIS Pro, you will see the tool result displayed as a rich representation directly within the notebook.\u00a0 This makes it easy for you to see all the information that helps you understand and interpret the tool results the same as you would if you ran the tool from the geoprocessing pane.<\/p>\n

Looking at the PCA results in the notebook, we can see that the first three principal components account for over 80% of the variance in the original 16 precipitation variables.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2368452,"id":2368452,"title":"pca_result","filename":"pca_result.jpg","filesize":113533,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/analytics\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis\/pca_result","alt":"","author":"154341","description":"","caption":"","name":"pca_result","status":"inherit","uploaded_to":2361222,"date":"2024-06-07 16:05:43","modified":"2024-06-07 16:05:43","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1032,"height":750,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","medium-width":359,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","medium_large-width":768,"medium_large-height":558,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","large-width":1032,"large-height":750,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","1536x1536-width":1032,"1536x1536-height":750,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","2048x2048-width":1032,"2048x2048-height":750,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result-640x465.jpg","card_image-width":640,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/pca_result.jpg","wide_image-width":1032,"wide_image-height":750}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

While there is no hard and fast rule for deciding how many of the principal components to retain (e.g. how many it takes to adequately represent the variance in the data), there are a few methods and strategies available to help you do so.\u00a0 One example is to examine the scree plot, which is automatically created for you when you run the Dimension Reduction<\/strong> tool and specify the \u201cOutput Eigenvalues Table\u201d parameter.\u00a0 It plots each principal component (x-axis) against the percent of variance explained by that component (y-axis).<\/p>\n

You can use the scree plot to help you decide on how many principal components to retain by locating the point at which the curve begins to flatten out, which in our case is around PC3.\u00a0 This aligns with the authors\u2019 decision from the original paper, where they chose to use the first three principal components as input into the subsequent cluster analyses.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2368502,"id":2368502,"title":"scree_plot","filename":"scree_plot.jpg","filesize":56630,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/analytics\/analytics\/end-to-end-spatial-data-science-5-machine-learning-cluster-analysis-in-python-and-arcgis\/scree_plot","alt":"","author":"154341","description":"","caption":"Scree plot produced by the Dimension Reduction tool.","name":"scree_plot","status":"inherit","uploaded_to":2361222,"date":"2024-06-07 16:20:33","modified":"2024-06-07 16:21:13","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":938,"height":444,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","medium-width":464,"medium-height":220,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","medium_large-width":768,"medium_large-height":364,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","large-width":938,"large-height":444,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","1536x1536-width":938,"1536x1536-height":444,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","2048x2048-width":938,"2048x2048-height":444,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot-826x391.jpg","card_image-width":826,"card_image-height":391,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2024\/06\/scree_plot.jpg","wide_image-width":938,"wide_image-height":444}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

The “Output Eigenvectors Table” parameter may also be helpful for understanding the results of the PCA.\u00a0 It provides a\u00a0graphical representation of the loadings (e.g. weights) for each component, which gives you an indication of how much the original variables are contributing to each component.<\/p>\n

Cluster analysis<\/h2>\n

Broadly speaking, cluster analysis is a technique used to group together individual objects, entities, or data points into meaningful subgroups (e.g. \u201cclusters\u201d).\u00a0 The end goal is for each data point within a cluster to be more similar to the other data points within that cluster than to the data points in any other cluster.\u00a0 In the context of this case study, the aim is to cluster the gridded precipitation dataset into climate regions, where each region has similar historical precipitation characteristics over the 30-year period from 1981-2010.<\/p>\n

In the field of data science, there are dozens of clustering techniques and algorithms to choose from.\u00a0 Some make specific assumptions about the statistical properties of the data points, while others are more data-driven and fall under the category of machine learning. The common thread among all<\/u> clustering techniques and algorithms is:\u00a0 What does it mean for two data points to be \u201csimilar\u201d? (e.g. how do we define \u201csimilarity\u201d between two data points, and how does this inform the creation of clusters?)<\/em><\/p>\n

In the context of cluster analysis, similarity is most often measured using correlational measures, association measures, or distance-based measures\u2014with distance-based being the most common.\u00a0 From a purely data science perspective, the term \u201cdistance\u201d refers to statistical distance<\/em>, which is simply the distance between observations in multivariate attribute space (e.g. the difference in the data values<\/span>).\u00a0 To geographers and GIS professionals, however, distance refers to geographical distance<\/em>, the physical distance between observations or data points on Earth\u2019s surface.\u00a0 As it turns out, there are clustering techniques and algorithms that use one or the other, or blend the two together.<\/p>\n

In ArcGIS, for example, you have access to clustering tools<\/a> that group spatial data based solely on the data attributes (e.g. Multivariate Clustering<\/a><\/strong>), or solely on their locations (e.g. Density-based Clustering<\/a><\/strong>).\u00a0 There are also several tools that perform clustering based on both the attributes and <\/em>locations of the data.\u00a0 For example, Build Balanced Zones<\/a><\/strong> and Spatially Constrained Multivariate Clustering<\/a><\/strong> take a machine learning approach, while the different flavors of Hot Spot Analysis<\/a><\/strong> and Cluster and Outlier Analysis<\/a> <\/strong>take a statistical approach.\u00a0 Additionally, the Time Series Clustering<\/a><\/strong> tool applies many of these concepts and more to time series data.<\/p>\n

As with any other form of analysis, it\u2019s up to you as the analyst to choose the most appropriate clustering technique or algorithm based on the problem you want to solve and the properties of your data.\u00a0<\/strong><\/p>\n"},{"acf_fc_layout":"sidebar","content":"

Note:<\/strong> In the paper that this blog series is based on, the authors choose a form of cluster analysis that focuses on creating clusters based on the attributes in multivariate data space. In the following sections, we\u2019ll use both open-source Python libraries and ArcGIS tools to replicate their results as close as possible, but for simplicity and educational purposes, we have made small adjustments to their exact workflow.<\/p>\n","image_reference":false,"layout":"standard","image_reference_figure":"","snippet":"","spotlight_name":"","section_title":"","position":"Center","spotlight_image":false},{"acf_fc_layout":"content","content":"

Cluster analysis using open-source Python libraries: Hierarchical clustering<\/h3>\n

Following the workflow in the paper, the first cluster analysis technique we\u2019ll use is hierarchical clustering<\/em>.\u00a0 Fundamentally, hierarchical clustering methods operate by iteratively merging or dividing a dataset into a nested hierarchy of clusters.\u00a0 There are two main varieties of hierarchical clustering: agglomerative and divisive.<\/p>\n

Agglomerative hierarchical clustering (e.g. \u201cbottom-up\u201d) begins with each data point as a member of its own individual cluster, then successively merges similar individual clusters together until all data points form one large cluster.\u00a0 Divisive hierarchical clustering (\u201ctop-down\u201d) initially treats the entire dataset as one large cluster, then splits this large cluster into successively smaller clusters based on the dissimilarity between them.<\/p>\n

In the paper, the authors experiment with several different agglomerative clustering algorithms, each of which uses a different \u201clinkage algorithm\u201d.\u00a0 In a nutshell, these linkage algorithms define how the distance (in multivariate attribute space) between clusters is calculated.\u00a0 You can learn more about the different linkage algorithms in the Hair, Jr. et al. (1998) and O\u2019Sullivan and Unwin (2003) textbooks referenced in Part #1<\/a> of this blog series.\u00a0 You can also watch a great explanation by Luc Anselin in his 2021 Spatial Cluster Analysis course<\/a> at the University of Chicago.<\/p>\n

We saw in the previous section that there are many clustering techniques available in ArcGIS, but hierarchical clustering is not<\/em> one of them.\u00a0 Because I am working with Python in an ArcGIS Notebook, however, it\u2019s very easy to hook into the Python ecosystem and take advantage of its powerful open-source data science libraries.\u00a0 In this case, I\u2019m going to call a specific hierarchical clustering algorithm from one of the most popular of these libraries, scikit-learn<\/a>.<\/p>\n

Back in our notebook, we need to import a few more Python libraries.<\/p>\n