{"id":706352,"date":"2020-05-05T05:44:42","date_gmt":"2020-05-05T12:44:42","guid":{"rendered":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&p=706352"},"modified":"2021-12-01T12:03:57","modified_gmt":"2021-12-01T20:03:57","slug":"mapping-large-datasets-insights","status":"publish","type":"blog","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/insights\/mapping\/mapping-large-datasets-insights","title":{"rendered":"Sticking points: Mapping large datasets in ArcGIS Insights"},"author":6771,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","format":"standard","meta":{"_acf_changed":false,"_searchwp_excluded":""},"categories":[23341,22941],"tags":[26401,222602],"industry":[],"product":[36801],"class_list":["post-706352","blog","type-blog","status-publish","format-standard","hentry","category-analytics","category-mapping","tag-classification","tag-spatial-aggregation","product-insights"],"acf":{"short_description":"Large datasets create some unique challenges for spatial analysis. Spatial aggregation can help you understand spatial patterns in your data.","flexible_content":[{"acf_fc_layout":"content","content":"

Large datasets create some unique challenges for spatial analysis<\/a>. It can be difficult to interpret spatial patterns and gain an understanding of your data when your dataset contains hundreds of thousands of unique features (or more!) In this blog series, we will go over some of the options available in ArcGIS Insights<\/a> for visualizing and analyzing large datasets spatially.<\/p>\n

The problem with points<\/h2>\n

Displaying your data as points (called a location map<\/a>) is useful in a lot of situations, like for mapping the locations of hospitals in your city, or distribution centers in your region. However, as your dataset gets larger, your ability to distinguish patterns is reduced.<\/p>\n

For example, the following map shows the occurrences of wildfires across the United States from 2010 to 2015. The dataset itself includes 450,234 features. However, the warning icon in the bottom left corner of the map indicates that Insights is not able to render all the points. In fact, clicking on the icon tells us that a sample of 112,000 points are being rendered (the full dataset will still be used in all analysis). The points that do get rendered are chosen randomly from the dataset, so the distribution of the sampled points should be representative of the full dataset. However, as your datasets get larger and the proportion of points being displayed decreases, your confidence in the distributions on a location map should also decrease.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":706382,"id":706382,"title":"Location map","filename":"1_Points.png","filesize":19931,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/insights\/mapping\/mapping-large-datasets-insights\/1_points","alt":"Locations of wildfires in the United States","author":"6771","description":"","caption":"","name":"1_points","status":"inherit","uploaded_to":706352,"date":"2020-01-17 14:34:52","modified":"2020-01-17 14:35:54","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":401,"height":366,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","medium-width":286,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","medium_large-width":401,"medium_large-height":366,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","large-width":401,"large-height":366,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","1536x1536-width":401,"1536x1536-height":366,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","2048x2048-width":401,"2048x2048-height":366,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","card_image-width":401,"card_image-height":366,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/01\/1_Points.png","wide_image-width":401,"wide_image-height":366}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

On the map of wildfires, Alaska appears to have a lower density of fires than the contiguous United States. Fires seem to be ubiquitous across the contiguous states with the exception of a small area near the Great Lakes region that does not have any fires. However, with only 25% of the data being displayed, I don\u2019t feel confident making any strong conclusions about wildfire distributions. A location map of this size also doesn\u2019t display regional patterns, like changes in point density, which will affect how you interpret your data.<\/p>\n

So if the location map is not ideal, which visualizations can you use? In Insights there are several options; the one you choose will depend on which questions you want to answer about your data. For this blog we will focus on using spatial aggregation, a technique available in Insights and across ArcGIS, to display your data using graduated symbols.<\/p>\n

Spatial aggregation<\/h2>\n

Spatial aggregation<\/a> calculates the count of features (or a statistic calculated with a number field) within specified boundaries and displays the count using graduated symbols<\/a>. Spatial aggregation is most useful when the boundaries you choose have some significance to your analysis and warrant a comparison.<\/p>\n

On the map below, I\u2019ve aggregated the wildfires by state. I can use the graduated symbols map to compare the number of fires between states. For example, the largest symbols on the map are for California and Texas, indicating that those two states had the most wildfires during the period from 2010 to 2015. Hovering over the states displays a pop-up that gives the minimum, maximum, and mean values for the dataset. Based on the pop-up below, Texas had 63,004 wildfires, which was also the maximum count for all states.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":779321,"id":779321,"title":"2_Aggregation","filename":"2_Aggregation.png","filesize":74590,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/insights\/mapping\/mapping-large-datasets-insights\/2_aggregation-2","alt":"","author":"6771","description":"","caption":"","name":"2_aggregation-2","status":"inherit","uploaded_to":706352,"date":"2020-03-23 16:48:04","modified":"2020-03-23 16:48:04","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":1103,"height":446,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","medium-width":464,"medium-height":188,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","medium_large-width":768,"medium_large-height":311,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","large-width":1103,"large-height":446,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","1536x1536-width":1103,"1536x1536-height":446,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","2048x2048-width":1103,"2048x2048-height":446,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation-826x334.png","card_image-width":826,"card_image-height":334,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2020\/02\/2_Aggregation.png","wide_image-width":1103,"wide_image-height":446}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

The graduated symbols map also allows us to analyze patterns in the spatial data. For example, there is a general trend of higher counts of wildfires in states further south in the continental United States. However, there are some outliers; New York has a relatively high count of wildfires compared to its neighboring states.<\/p>\n

Classification<\/h2>\n

Another aspect of a graduated symbols map that affects how you interpret the spatial patterns is the classification<\/a>. The default classification on a graduated symbols map is natural breaks with five classes. Natural breaks classifications provide breaks based on natural groupings in your data. The other classification options are equal interval, quantile, standard deviation, unclassed, and manual.<\/p>\n

Natural breaks \u2013 Data is grouped based on naturally occurring patterns in the data.<\/p>\n