Every time you visit the doctor, a familiar routine takes place. Temperature, heart rate, blood pressure—information collected and retained.
Why do we do this? You might be there for a sore pinkie toe. Surely heart rate doesn’t impact that.
I Am Not A Doctor—but I know these measurements are done in part to establish a baseline, a collection of data points representing the range of values considered healthy or normal for your body.
Variation from this baseline can be a more revealing health indicator than the number on its own.
In December 2020, ArcGIS Online introduced the Feature Data Store Resource Usage Chart to Premium Feature Data Store. This tool displays your Feature Data Store’s key health indicators over time, offering administrators precise insight into its typical range of values. Early awareness of changes in resource use patterns offers administrators the opportunity to take corrective action before problems arise.
In this blog, we’ll first learn how to access the usage chart, and then we’ll discuss how getting to know your Feature Data Store’s baseline will help you keep it in good health!
For more information about Standard and Premium Feature Data Stores, including pricing and capabilities, please visit our ArcGIS Online FAQ.
Accessing the Usage Chart
To begin, log into your organization with an Administrator role, then navigate to your Organization Overview page.
From here, click on the “Feature Data Store” hyperlink to access the Usage Chart.
This opens the view into two key health indicators: Storage and Resource usage.
Note: though viewing storage capacity is available to all organizations, viewing feature data store computation usage is not yet implemented for organizations using Standard Feature Data Store.
Getting to know your Feature Data Store’s Baseline
Our heart rates and blood pressure fluctuate throughout the day, based on exertion, food intake, ambient temperature, and sometimes for no reason we can discern.
Similarly, your data store’s storage and computational use will fluctuate. Healthy systems perform various self-care routines and recycle their processes periodically—these can result in some visible and normal movement in computational resources.
In addition to these minor variations, our organization’s workflows vary. On some days, your data store may be running a marathon and on others, it may be enjoying a lazy Sunday. If 30 fieldworkers return from the field and sync their offline edits simultaneously, we’ll likely observe a corresponding increase in computational load. And at night (or lazy Sundays) our Feature Data Store may register little or even no activity.
The key to managing your resources lies in recognizing what range of data points lie within your Feature Data Store’s normal use patterns, and what to do if you see unusual variation.
Storage includes three metrics: your Feature Data Store’s total, or allocated space is represented by the horizontal line. Used space is shown in green. And remaining storage, or available storage is shown in grey.
In my example above, the Feature Data Store is allocated 1 terabyte (TB) of total storage space. Of that space, there are 430.50 gigabytes (GB) remaining. A glance at the green bar shows my organization has used just over half of its allocated storage. And we can use a little bit of arithmetic to determine our usage more precisely: total allocation 1000 GB minus available storage 430.50 GB equals 569.5 GB used.
This information helps you maintain awareness of your organization’s storage use. If you observe increasing storage consumption, you can run Organization Reports to find large items or items you and your team can identify as being stale or otherwise unnecessary. And if you hear widespread user reports of consistently failing attempts to add new features or upload data, check your storage use–these could be symptoms of running out of space.
But if you and your team validate all of the content is necessary, it may be time to consider upgrading to a larger Feature Data Store.
Upgrade your Standard Feature Data Store to a Premium Feature Data Store online, or move between Premium Feature Data Store levels by contacting your Account Manager. Upgrading and downgrading between ArcGIS Online Feature Data Store levels requires no work from your team, and all content and URLs are unaffected. Upgrading from Standard to Premium or to a higher Premium level can happen at any time with a minimum billing period of 30 days.
The usage chart tracks an aggregate computational metric comprised of: percentage of CPU in use, input/output (I/O) volume, and memory use. Your organization’s consumption of these resources is tracked on the vertical (y) axis, and time is represented along the horizontal (x) axis.
This example has lots of gaps, and some spikes—how can we tell, from this chart whether or not this Data Store is healthy? First, we’ll understand what gaps and spikes are.
The gaps in this chart are time periods during which this Feature Data Store used so little of its computational capacity it did not register on the chart at all. It does not mean the Feature Data Store is off, sleeping, or locked—your Premium Feature Data Store is always running.
This chart also shows a rapid increase in computational demand sometime around 10 a.m., a 20% rise in resource use.
What these spikes and gaps mean and whether or not they require action will vary by organization, just like body temperature, blood pressure, and heart rate will vary by individual.
As this organization’s Administrator, I stay in touch with members of the organization and maintain awareness of how their activities impact our resources. If I see a spike in computation, I touch base with the team and find out what actions those spikes correlate to: onboarding training for a new division? Field workers coming in and synching their changes just before dinnertime? Heavy analytics supporting development of new datasets or analyses? A nightly ETL process? You will become the expert on your organization’s workflows and the resources which support them.
I know this organization supports collaboration between teams, with use increasing around training exercises centered on upcoming functionality.
Pairing this qualitative knowledge with the quantitative data shown above, I can say this organization is utilizing a relatively small amount of its resources. This leads me to wonder: was this simply a slow day? Or is this part of a larger trend? I can toggle the chart’s timeline to see a broader perspective.
With this revised timeline, I see my organization sustains generally low activity, with occasional spikes using up to approximately half of my Premium Feature Data Store’s computational capacity. My confidence in the established baseline increases as the pattern repeats over time.
ArcGIS Online Feature Data Store Resource Management Analysis & Best Practices
Understanding baseline behaviors is key to answering common questions about your Feature Data Store, such as:
Is our Feature Data Store the correct size for our needs?
Based on the example organization shown earlier, I can see our current storage capacity is sufficient, and we have room to grow. If I notice any significant or sudden increases in storage consumption, I’ll connect with the owners of large or numerous data sets to learn the context around their storage use. Here again, ArcGIS Online Organization Reports support the team’s active discovery and evaluation of large, aging, redundant, or forgotten items.
The computational resources, though not under continuous use, comfortably accommodate spikes I know are related to our typical use case: inviting groups to check out and test new capabilities. Looking at the Feature Data Store levels (shown below), I know the next size down offers approximately half my current resources. This would not adequately serve my highest volume use cases, so it seems my organization is properly sized.
What Feature Data Store size will we need next year?
If you administer an organization which serves multiple teams, you’ll want to maintain communication with their leadership to understand plans for the future. My organization is expected to stay as it is; there are no plans for new data such as statewide parcel data or migrating a new division into my organization. And we do not anticipate adding new members or increasing our reliance on analytical workflows. With this knowledge, I can review our past growth and project a similar rate for the upcoming year.
1. Be on the lookout for indications that your organization’s needs may be increasing.
- Is your baseline computation usage increasing over time?
- Is your storage usage increasing over time?
2. Consider the thresholds which require action.
- What are your highest usage activities? Do you have enough capacity to serve them at all times?
- Do you have seasonal usage increases?
3. Conserve energy through adoption of best practices
- Develop a governance plan
- Remove redundant or extraneous items
- Use Feature Tiles for repeatable and scalable queries
- Minimize or eliminate public editing
Take a quick look at your Feature Data Store usage chart each day, and you’ll quickly learn its baseline patterns. With this knowledge, you’re well-equipped to keep it in excellent health!