ArcGIS Enterprise

Dev Summit 2023: Custom data feeds

Organizations require the ability to leverage data from a wide range of data sources, including those not natively supported with ArcGIS.

At this year’s Developer Summit plenary, Shreyas Shinde demonstrates how to query data from two such sources—MongoDB and Yelp—by creating custom data feeds.

To kick off his demonstration, Shreyas shows the data that analysts in his organization are interested in leveraging: the locations of Supplemental Nutrition Assistance Program (SNAP) authorized retailers in Connecticut stored in a MongoDB database and their business metrics such as ratings, reviews, and price levels on Yelp.

MongoDB database containing the locations of Supplemental Nutrition Assistance Program (SNAP) authorized retailers in Connecticut.
MongoDB database containing the locations of Supplemental Nutrition Assistance Program (SNAP) authorized retailers in Connecticut.

The first step in configuring a custom data feed is to create a custom provider. Shreyas makes a custom provider for each of his data sources with the help of a new command line tool in ArcGIS Enterprise SDK called cdf. He runs the tool twice: first using the cdf createapp <app name> command to create a custom data app and then cdf createprovider <custom provider name> to create custom providers.

 

 

 

Shreyas uses the cdf command line tool to create a custom data app.
Shreyas uses the cdf command line tool to create a custom data app.
Shreyas uses the cdf command line tool to create a custom MongoDB data provider.
Shreyas uses the cdf command line tool to create a custom MongoDB data provider.

Using the cdf tool to create a custom provider generates boilerplate code, including a model.js file which encapsulates the logic required to access data in the data source. If you have ever worked with Koop—an open-source geospatial data server—this should look familiar. Shreyas updates the JavaScript file for his MongoDB and Yelp providers with pertinent code.

model.js file containing code that fetches data from the MongoDB database.
model.js file containing code that fetches data from the MongoDB database.

The next step in configuring a custom data feed is to export code and dependencies as a custom data package file (.cdpk) and deploy it to ArcGIS Server. Once the code is in place, Shreyas uses the cdf tool to generate custom data package files for each of his providers by running the cdf export <custom provider name> command. Next, he uploads the files and registers them with ArcGIS Server using ArcGIS Server Administrator Directory.

MongoDB and Yelp custom providers registered with ArcGIS Server.
MongoDB and Yelp custom providers registered with ArcGIS Server.

The last step in configuring a custom data feed is to create a new feature service that references the custom provider registered with ArcGIS Server. To create the feature services, Shreyas submits JSON representations that include information about his data providers.

Shreyas creates a feature service that references the MongoDB custom provider.
Shreyas creates a feature service that references the MongoDB custom provider.

Shreyas’s data feeds are now ready as read-only feature services, and analysts in his organization can leverage the data using the services’ ArcGIS Server Services Directory URL.

About the authors

Aawaj is a product engineer on the ArcGIS Enterprise team.

Mild mannered programmer by day, mild mannered programmer by night. Lover of distributed software systems. Software Engineer on the ArcGIS Enterprise team at Esri.

Connect:

Next Article

Empowering Communities with Open Data

Read this article