Extend the capabilities of custom apps with AI skills

If you’ve been looking to enhance your custom apps with AI to enable features such as speech recognition, human-like text understanding and generation, and image processing to improve user experience and accelerate decision making, then you’re in luck. You’ll be able to do just that and more with the robust set of AI skills Esri is building in its Geospatial AI framework, as well as your own custom skills that you can implement within it.

At this year’s Developer and Technology Summit plenary, Linda Beale offers a glimpse of what can be accomplished with pre-built and custom AI skills in apps built on the Geospatial AI framework.

For her demonstration, Linda uses a custom app powered by the Geospatial AI framework to identify threats in the manufacturing supply chain of an automotive manufacturer in Germany.

Find a factory near Berlin and a store near Palm Springs.

She begins her demonstration by asking the app, in natural human speech, to locate the manufacturer’s factory near Berlin and a store near Palm Springs. The app harnesses ArcGIS Location Platform’s comprehensive location services to find the facilities in seconds.

Show the route with the smallest carbon footprint.

Next, she asks the app to find the transportation route between the factory and the store with the smallest carbon footprint. The app uses a custom AI skill she implemented within the Geospatial AI framework to identify the route. The skill leverages a multi-modal routing service to determine the most carbon-efficient route by integrating multiple modes of transportation and using carbon emissions as the cost factor, rather than time.

Who supplies parts to this factory?

The first step in identifying threats in the manufacturing supply chain is finding suppliers that provide parts to the factory in Berlin. When Linda asks the app to find the suppliers, it taps into an AI skill in the Geospatial AI framework to create an openCypher query and call a knowledge graph service to discover suppliers associated with the factory.

Show the risk in the supply chain.

Finally, she asks the app to identify suppliers that pose a risk to the manufacturing supply chain. The app uses an AI skill to find and highlight the suppliers on the map and read her the risks they pose.

In the second half of her demonstration, Linda shows some key parts of the app’s code. The base configuration file provides all the configurations, settings, and parameters the framework needs. For example, there is a collection of language models, a conversation store to store and use conversations as part of the chat API, and much more.

The manifest JSON file for the custom AI skill defines the skill’s unique ID and the Python file that serves as the entry point for requests made to the skill. The manifest file is what introduces the skill to the Geospatial AI framework.

The custom AI skill employs a multi-agent framework: a director handles intent-based routings, managers and specialists are assigned to specific categories, and workers execute individual functions.

Using pre-built and custom AI skills, Linda demonstrated how a custom app can be enriched with AI to simplify user experience and enhance its power and capabilities.

Excited to use AI skills in your custom apps? Stay tuned for more information and communication about this ongoing project.