This is the first of several blog posts discussing offline workflows for developers using ArcGIS Runtime. In this blog, I will share tips and tricks to help you optimize your offline Runtime apps. Code examples are written in Qt C++ but the concepts apply to Runtime APIs.
Taking maps offline is a key workflow for ArcGIS Runtime. If you’re developing an app that needs to work wherever your users are, regardless of network connectivity, read on. You can do many things when your app is offline: visualization, editing, navigation, analysis, running geotriggers, and more. But with so many possible workflows, how can you be sure that you are getting the most out of your offline data?
Introduction
The Building offline applications guide gives an introduction to offline apps so I’m not going to spend too much time covering the basics in this article. Mostly we’ll be talking about the OfflineMapTask and the OfflineMapSyncTask for taking maps offline with Runtime. A lot of advice in this article focuses on the design of your data or tweaking the GenerateOfflineMapParameters or DownloadPreplannedOfflineMapParameters that you use to set up your offline jobs. There are five areas that can really help boost your offline workflows:
- Don’t take data offline.
- Avoid duplicating effort.
- Think about sync.
- Work with services.
- Don’t forget to tidy up.
1. Don’t take data offline
That’s right – if you don’t need to take data offline, don’t do it. The number one issue we hear from users is that their offline data is BIG. And big means slow to download and lots of disk space on your device. But, I hear you say, isn’t this article supposed to be about taking data offline? Let’s look at some strategies to make sure you’re only taking the data you really need.
Leave data online
With data that is not business critical, or changes frequently, let those layers remain online for when you do have network coverage. Think of a layer with real-time weather reports or a super high quality basemap that covers the whole globe. These give you great context for your app, but do you really need them all the time? Rather than making all your layers sync enabled, or skipping layers that can’t be taken offline, you can leave some as online only. These layers can still be included in your offline map by setting the onlineOnlyServicesOption in the GenerateOfflineMapParameters. That way your most important data is available offline, but you can still have access to your extra online-only layers when you have connectivity.
GenerateOfflineMapParameters myParams; myParams.setOnlineOnlyServicesOption(OnlineOnlyServicesOption::Include);
Don't download all data in your offline maps app if you can leave it online
An empty geodatabase is a small geodatabase
Offline maps are often used in data collection scenarios where field crews are collecting huge amounts of new data. If users in the field are just adding new information, then they might not need existing features in their offline map. Set isReturnSchemaOnlyForEditableLayers to true on your GenerateOfflineMapParameters and the offline map will end up with empty geodatabases ready to be populated with new features in the field. The same goes for attachments: if your users don’t need to see existing attachments (which can be large photos, videos, and so on), set the ReturnLayerAttachmentOption for your parameters to None.
Start with an empty geodatabase if your users are just adding new information
Don’t be the font of all knowledge
Esri’s vector basemaps provide excellent context for offline workflows at all scales. But when taken offline they include a huge set of fonts to support character sets across the globe. If you know for a fact that your app won’t require characters sets such as Chinese, Japanese, Thai, Korean, or Georgian, you can set the EsriVectorTilesDownloadOption to UseReducedFontsService. This will switch to a slimmed down set of fonts that greatly reduces the download size of your basemap.
GenerateOfflineMapParameters myParams;
myParams.setEsriVectorTilesDownloadOption(
EsriVectorTilesDownloadOption::UseReducedFontsService);
DownloadPreplannedOfflineMapParameters myParams;
myParams.setEsriVectorTilesDownloadOption(
EsriVectorTilesDownloadOption::UseReducedFontsService);
Save space by only downloading fonts that will be used by your
Ready to level up
When you do need to take a basemap offline, focus on the data you really need, to keep download sizes small. You can cut out unnecessary levels of detail by setting appropriate minScale and maxScale values for GenerateOfflineMapParameters (I always get these the wrong way round – the max scale is the smaller number and controls how zoomed in you are, so a max scale of 1 would be zoomed in to real world scale). If you are specifying the level IDs directly with ExportTileCacheParameters, you can try skipping out intermediate levels to avoid downloading those tiles.
Only download the levels of detail that you need for your basemap
Small is beautiful
For exporting basemaps, you don’t need to stick with boring old envelopes to define your area of interest. If your study area covers a linear feature like a river or a road, you can buffer it to create a targeted polygon and set that as your GenerateOfflineMapParameters areaOfInterest instead. This also works when defining a map area ahead of time using the preplanned workflow (see below). The Define polygon offline map areas for Collector blog describes creating polygon map areas for the preplanned workflow.
Customize the shape of your area of interest to focus just on the data you need
2. Avoid duplicating effort
If you have many users that work in the same geographic areas, or users who make regular repeat visits, you want them to share as much of the heavy lifting for going offline as possible. Rather than endlessly packaging and downloading the same data, here are a few ideas for reusing data.
I already have a perfectly good basemap
If users typically visit the same geographic region (for example, a city or town) again and again, you can pre-load a basemap onto their device and have your offline map refer to it. This means every map that you take offline is smaller and it also allows you to get creative with the basemap – for example you can export a custom .tpk or .vtpk from ArcGIS Pro. Set the referenceBasemapDirectory and referenceBasemapFilename on either GenerateOfflineMapParameters or DownloadPreplannedOfflineMapParameters to use a basemap that’s already on the device.
// Update default parameters to specify use of local basemap.
params.setReferenceBasemapDirectory(dataPath);
params.setReferenceBasemapFilename("naperville_imagery.tpkx");
Set up a basemap on your device that you can reuse again and again
Preplanned workflow
Let’s say you have many users who regularly want to visit the same areas. You can make use of the pre-planned workflow to define map areas and their data ahead of time. That means that any user who wants to take one of those areas offline can just download the bits and pieces that make up the offline map without needing to wait for backend services to prepare the data. You use the DownloadPreplannedOfflineMapParameters to take one of these areas offline, but when you are defining the map area ahead of time, you can still think about the other tips in this article, such as generating schema-only geodatabases.
Plan your offline maps ahead of time to save time and share data
3. Think about sync
When we talk about editing with offline maps, we are really talking about the feature sync workflow, where edits from a mobile geodatabase on your device are uploaded to a feature service and changes from others are downloaded. If you are using a sync workflow it’s important to think about how and when you sync.
I don’t need any changes at all
This is a common offline workflow, for example, read-only maps for reference or as an aid to navigation. Make sure that you don’t set your map up with unused editing capabilities because you are creating extra work (such as a replica on the feature service that needs to be created and eventually unregistered). Set the PreplannedUpdateMode to NoUpdates for a DownloadPreplannedOfflineMapParameters or the GenerateOfflineMapUpdateMode to NoUpdates for a GenerateOfflineMapParameters. Now your offline map contains the data you want, but you don’t need to worry about updating it. If you need an updated map in the future, take a new one offline.
Create zero maintenance snap-shot offline maps when you don't need any updates
I need to upload local edits and download changes from others
What about when your app needs to make changes and see updates from other users? Not a problem, you can do bidirectional sync with both the on-demand and preplanned workflows. The main thing to consider here is the impact of performing a sync on the backing feature service. As a rule, it is cheap to perform an upload sync (sending your changes to the backing feature service) but it can be expensive to perform a download sync (fetching changes from the service) since it requires an exclusive lock on the database. Design your workflows so that users upload their changes regularly to avoid losing work but only download changes when required. Set the syncDirection for your OfflineMapSyncParameters to either Upload or Download rather than sticking with the default of Bidirectional.
Upload changes whenever you need but avoid performing download sync at busy times
I only need to download changes from others (read-only)
If an offline map only needs to receive updates from other users, and it uses the pre-planned workflow, you can opt into an advanced workflow called Scheduled, or Packaged Updates. In this scenario, the preplanned map area attached to the online web map is regularly updated (for example, every week or every day) with changes from the feature service. The changes that were applied are also cached so that any offline maps based off that map area can just download the relevant update packages. This optimization is massively scalable since none of the offline clients need to go directly to the feature service. If the map area supports it, set the PreplannedUpdateMode to DownloadScheduledUpdates for your DownloadPreplannedOfflineMapParameters.
Use scheduled update packages to get others' changes quickly and easily
4. Work with services
Most of the advice in this article looks at using the OfflineMapTask to simplify taking entire maps offline. However, you can get more fine-grained control over how individual layers are taken offline by working directly with services.
- ExportVectorTilesTask – Take vector tiled basemaps offline as .vtpk files.
- ExportTileCacheTask – Take imagery tiled basemaps offline as .tpk or .tpkx files.
- GeodatabaseSyncTask – Create a local replica geodatabase and sync it back to the feature service.
Use the overrides workflow to tweak individual services
You can get the best of both worlds by taking entire maps offline and still treating individual services differently by using the overrides workflow.
You start with the GenerateOfflineMapParameters to get the basics right and then create a set of GenerateOfflineMapParametersOverrides from these.
connect(m_offlineMapTask, &OfflineMapTask::createDefaultGenerateOfflineMapParametersCompleted,
this, [this](QUuid, const GenerateOfflineMapParameters& params)
{
// Use the parameters to create a set of overrides.
m_parameters = params;
m_offlineMapTask->createGenerateOfflineMapParameterOverrides(params);
});
These overrides present a set of low-level parameters for exporting tiles, generating geodatabases, and so on for each service in the map that you can adjust as needed. For example, you can use a buffered area of interest to create a larger offline basemap like so:
LayerListModel* layers = m_map->basemap()->baseLayers(); if (!layers || layers->isEmpty()) return; OfflineMapParametersKey keyForTiledLayer(layers->at(0)); if (keyForTiledLayer.isEmpty() || keyForTiledLayer.type() != OfflineMapParametersType::ExportTileCache) return; // Obtain the dictionary of parameters for taking the basemap offline. QMap dictionary = m_parameterOverrides->exportTileCacheParameters(); if (!dictionary.contains(keyForTiledLayer)) return; // Create a new geometry around the original area of interest. auto bufferGeom = GeometryEngine::buffer(m_parameters.areaOfInterest(), bufferMeters); // Apply the geometry to the ExportTileCacheParameters. ExportTileCacheParameters& exportTileCacheParam = dictionary[keyForTiledLayer]; // Set the parameters back into the dictionary. exportTileCacheParam.setAreaOfInterest(bufferGeom); // Store the dictionary. m_parameterOverrides->setExportTileCacheParameters(dictionary);
See the full sample this code is taken from on the arcgis-runtime-samples-qt repository.
5. Don’t forget to tidy up
Once you are done with your offline map, clean everything up to save space on your device and on the backend services.
Cancel that
If you are running a job to create an offline map (or any Job for that matter) and you suddenly realize you’ve made a mistake (it happens to us all), you can call Job.cancelAsync to clean things up before you start again. This will clean up any half-finished files lon your device and stops the backing services from doing unnecessary work, like creating basemap packages that are no longer needed.
Cancel unwanted jobs to reduce load on back-end services
Unregister your geodatabases
If you have any mobile geodatabases that are set up to sync with a feature-service, make sure to call GeodatabaseSyncTask::unregisterGeodatabase when you are finished with them. That allows the feature service to forget about that specific replica so it can get on with more important things.
When you are done, unregister geodatabases to free up space in the feature service
Please close the map behind you
Finally, when you are finished with an offline map, delete it from the device to save disk space. Sometimes a file-handle may be left over from your app that prevents the folder from being deleted. If that’s the case, you can call MobileMapPackage::close to free things up before you delete.
To save disk space, remove offline files when you are done with them
Summary
I hope that’s given you some ideas of how you can focus your offline workflows with Runtime. Keep the following in mind:
- Minimize the data you need to download as much as possible.
- Share data between maps and users whenever you can.
- Plan your editing workflows to minimize load on feature services.
- Work with underlying services when you need fine-grained control over your map’s layers.
- Dispose of resources such as files on disk or server resources when you are done.
Do you know any other tips for offline work? What problems have you faced trying to make your app offline? Contact us or join the community to share what and how you are using offline apps with Runtime.
In my next article, I’ll look at working with scheduled update packages in more detail. You can subscribe to new information about Runtime workflows with RSS feeds via the Esri Community Blog
and follow us on Twitter.
About the author
Commenting is not enabled for this article.