ArcGIS Blog

100,000 points to nowhere: Right-size your data to create usable maps

It’s easy to get bogged down in details when you have tons of data that you’re trying to display on a map. Plotting too many features on a map can lead to frustration and doesn’t provide valuable insight. This series of posts will help you learn techniques for streamlining your maps to achieve better performance and provide a clear picture of your business data.

Here’s a scenario that occurs all too often in business organizations that understand data systems, but are new to GIS and mapping.

Let’s say you’re the manager of a busy department. You’ve just seen a demo of mapping capabilities that opens up a whole new world of possibilities in your business intelligence (BI) system. Using Esri Maps and the underlying ArcGIS platform, you’ll be able to see what matters to you on a map, in the context of the business applications you already have in place. You’ll be able to analyze your sales results and marketing spending visually—on a map—at any level. You’ll be able to see and analyze performance metrics for all your branches, manufacturing facilities, sales reps, customers, and even physical assets like cell towers. And you’ll be able to do all this in the context of valuable regional data, like demographics and spending potentials.

So great! You install and configure the Esri Maps plug-in for your BI system, and your BI group starts building your first map-enabled report.

Too much data makes it hard to see patterns

Soon, you hit what you see as a major stumbling block. Someone on the project has mandated that the application must plot tens of thousands—or even hundreds of thousands—of individual features on a map. These features might represent cell towers, sales outlets, customers, Twitter feeds—any entity that you want to analyze in the context of location.

Now, visualizing data on a map is different from viewing it in charts and lists typically used in BI applications. Those unfamiliar with GIS often feel that a map should be able to display any number of features, with no limitations. But look at it this way: the data used to plot 100,000 features on a map comes from a table that contains 100,000 records. If you were to display all those features in a chart on your dashboard, how would you use that data? How would you derive any useful meaning from it? The sheer volume of information displayed would make it impossible to discern patterns or trends without some type of filter to narrow your search.

The same concept applies to a map. The map below, created using ArcMap, shows approximately 100,000 points across the United States, representing customer purchases styled according to sales revenue.

100,000 point visualization of customer sales

Looking at this map, it’s really difficult to identify any relevant patterns. Although there seems to be a strong concentration of points in the North-East, the points appear so bunched up that you can’t quickly locate the most loyal customers or those who made the most purchases. You can’t even differentiate between the different colors listed in the legend. The map doesn’t impart any useful information.

Just because you may technically be able to show hundreds of thousands of points on a map doesn’t mean that you should.

Too much overhead causes the map to render too slowly

In addition to creating a virtually illegible map, plotting large numbers of points on a map exacts a huge toll on performance. The Esri Maps plug-in is a browser-based application. In a web browser, data for each point on the map must be converted to a format supported by the map; this includes column values for each row, for each record.

For example, a single retailer record might include information about the retailer, such as the store ID, the store name, location information (latitude and longitude coordinates, province or state, and country), sales information such as the total revenue and gross profit for a specific year, and other information about the products.

Sample data table

When you add this data to your map, the information is transformed into a format that the map can read before it can render the points, and each record requires several bytes of memory. The converted data for the single record shown above represents approximately 200 characters, which translates to about 800 bytes. If you multiply that by 100,000 records, you end up with roughly 80 MB of data to render in memory. That’s a lot of data for a browser to display, which means users with standard laptops and memory capacity will spend a lot of time waiting for points to appear on the map. And that doesn’t even cover the amount of memory needed to retrieve map service layers, or memory used by the underlying business application. This all adds up to a whole lot of memory required just to view the map!

Keep in mind that, in addition to the number of points, the number of attributes related to each point also increases the memory requirements. You choose the number of details to include in the pop-ups for feature points by selecting them from the columns in your data. So, even though your data may contain a relatively small number of rows (feature points), if each row contains many columns (attributes) and you include them all when you add your data to the map, performance will be affected.

Right-size your visualization

Thankfully, there are several ways to avoid too-much-data issues, some of which involve very little effort on your part.

In the coming weeks, we’ll explore each of the following approaches in greater detail:

Stay tuned for more! Keep checking for additions to this series.

About the author

Kathie is a documentation product engineer, creating help and resources for a variety of location analytics apps, including ArcGIS for Power BI and ArcGIS for SharePoint.

0 Comments
Inline Feedbacks
View all comments

Next Article

Designing Effective Dashboards - UC 2022 Recap

Read this article