{"id":2332,"date":"2026-11-16T18:10:31","date_gmt":"2026-11-16T18:10:31","guid":{"rendered":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/?post_type=blog&#038;p=2332"},"modified":"2026-11-16T17:55:07","modified_gmt":"2026-11-16T17:55:07","slug":"browser-native-ai-agents","status":"publish","type":"blog","link":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents","title":{"rendered":"Inside Browser-Native AI Agents for Mapping"},"content":{"rendered":"\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">In&nbsp;<a href=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/ai-orchestration-in-the-browser\">Part 1<\/a>, we explored why AI orchestration belongs in the browser, close to the map and its live application state. Here, we look inside the agents: how they find relevant data, choose tools, and interact with the map\u2014and the engineering decisions that shaped their design.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Building on the earlier&nbsp;<a href=\"https:\/\/codepen.io\/dhrumil83\/pen\/myEGaYY\" target=\"_blank\" rel=\"noreferrer noopener\">nursing homes example<\/a>, we\u2019ll use a more specific request:<\/p>\n\n\n\n<p class=\"has-text-align-left block-editor-paragraph wp-block-paragraph\"><em>\u201cShow nonprofit nursing homes in Texas and tell me what percentage of all Texas nursing homes they represent.\u201d<\/em><\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">That sounds simple, but a web map can contain dozens of layers and hundreds of fields whose names and values don\u2019t directly match the user\u2019s wording. In this map, the agent must identify the United States Nursing Homes layer and discover that states are stored as two-letter codes in <code>USER_State<\/code>. Nonprofit facilities aren\u2019t simply labeled \u201cnonprofit\u201d\u2014they span multiple values in <code>USER_Ownership_Type<\/code> beginning with <code>Non profit -<\/code>. Answering the request requires discovering the relevant schema and values, calculating the percentage, and highlighting the matching facilities on the map.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"1440\" height=\"645\" data-id=\"2408\" src=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif\" alt=\"Animated demo of an assistant highlighting 115 nonprofit nursing homes in Texas and reporting that they represent 9.8% of the state\u2019s 1,177 nursing homes.\" class=\"wp-image-2408\" \/><\/figure>\n<\/figure>\n\n\n\n<p class=\"has-text-align-center block-editor-paragraph wp-block-paragraph\"><em>Map highlighting nonprofit nursing homes in Texas alongside the agent&#8217;s count and percentage<\/em><\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Agent tools access the live map through the&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/references\/map-components\/components\/arcgis-map\/\" target=\"_blank\" rel=\"noreferrer noopener\"><code>arcgis-map<\/code><\/a>&nbsp;component, reading its layers, extent, and other view state when they execute. For a request such as \u201cWhat\u2019s in my current view?\u201d, a query tool can use the current extent even if the user has just panned the map.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/\">ArcGIS Maps SDK for JavaScript<\/a>&nbsp;supplies the underlying operations, including&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/query-filter\/#querying\" target=\"_blank\" rel=\"noreferrer noopener\">querying features<\/a>,&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/query-filter\/#client-side-filtering\" target=\"_blank\" rel=\"noreferrer noopener\">applying filters<\/a>,&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/spatial-analysis\/intro-geometry-operators\/\" target=\"_blank\" rel=\"noreferrer noopener\">computing geometry<\/a>, and&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/references\/map-components\/components\/arcgis-map\/#goTo\" target=\"_blank\" rel=\"noreferrer noopener\">controlling the viewport<\/a>. Orchestration runs in the browser; individual operations may use client-side data or call services as needed.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">These agents are available through the SDK\u2019s&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/agentic-apps\/ai-introduction\/\" target=\"_blank\" rel=\"noreferrer noopener\">AI components<\/a>. The&nbsp;<a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/references\/ai-components\/components\/arcgis-assistant\/\" target=\"_blank\" rel=\"noreferrer noopener\"><code>arcgis-assistant<\/code><\/a>&nbsp;component provides the conversational interface and orchestrates requests across registered agents.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-from-predefined-workflows-to-skills\">From predefined workflows to skills<\/h3>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">In early 2025, we began designing these agents with a straightforward approach: put the map\u2019s layer and field metadata into the prompt and let the model decide what to use. This worked for smaller maps, but became less reliable as the number of layers and fields grew. The model sometimes selected the wrong fields or generated queries referencing fields that did not exist.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The first major improvement came from combining&nbsp;<a href=\"https:\/\/www.ibm.com\/think\/topics\/embedding\" target=\"_blank\" rel=\"noreferrer noopener\">embeddings<\/a>&nbsp;with workflows built in <a href=\"https:\/\/docs.langchain.com\/oss\/javascript\/langgraph\/overview\" target=\"_blank\" rel=\"noreferrer noopener\">LangGraph<\/a>. Embeddings helped us retrieve the layers and fields relevant to a request, while the graph organized the steps for retrieving context, generating queries, and calling tools. Providing focused context and an explicit execution path made the agents more reliable.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">As we expanded the workflows, however, we found that our predefined paths could also limit recovery. If an agent selected the wrong layer or discovered that it needed more context, the workflow did not always give it a straightforward way to revisit an earlier decision.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-moving-to-skill-guided-tool-use\">Moving to skill-guided tool use<\/h4>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">We introduced&nbsp;<a href=\"https:\/\/agentskills.io\/home\" target=\"_blank\" rel=\"noreferrer noopener\">skills<\/a>&nbsp;into this architecture: instructions that describe how to carry out a task using the available tools. Instead of encoding each decision as a graph branch, we let the agent choose tools, inspect their results, and determine what to do next. Combined with newer models, this has allowed us to simplify the workflows while giving agents more room to adjust when their first approach does not work.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Moving task-specific decision logic from predefined graph branches into skill instructions also substantially reduced the amount of workflow code we needed to maintain.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"881\" src=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/workflow-to-skills-1-1024x881.png\" alt=\"A predefined workflow follows fixed retrieval and query steps. A skill-guided agent chooses tools, inspects results, and repeats as needed before responding.\" class=\"wp-image-2338\" srcset=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/workflow-to-skills-1-1024x881.png 1024w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/workflow-to-skills-1-300x258.png 300w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/workflow-to-skills-1-767x660.png 767w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/workflow-to-skills-1.png 1334w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\"><em>Simplified representations of a predefined workflow and a skill-guided tool loop.<\/em><\/figcaption><\/figure>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-loading-skills-on-demand\">Loading skills on demand<\/h4>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\"><a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/agentic-apps\/ai-custom-agents-skills\/\" target=\"_blank\" rel=\"noreferrer noopener\">An agent can use multiple skills<\/a>, each focused on a particular task. Through <a href=\"https:\/\/agentskills.io\/specification#progressive-disclosure\" target=\"_blank\" rel=\"noreferrer noopener\">progressive disclosure<\/a>, the model initially sees skill names and descriptions. When a skill is loaded, its detailed instructions are added to the context, and only the tools listed in its&nbsp;<code><a href=\"https:\/\/agentskills.io\/specification#allowed-tools-field\" target=\"_blank\" rel=\"noreferrer noopener\">allowed-tools<\/a><\/code>&nbsp;property are made available to the model. This keeps both instructions and tool availability focused on the task at hand.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The following excerpt from the query skill shows how its instructions guide the agent to find relevant layers and fields, query the data, and answer the user\u2019s question.<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-plain\"><code>---\nname: query\ndescription: Use to find, query, and summarize feature-layer data.\nallowed-tools: &gt;-\n listSharedResources findRelevantLayers findRelevantFields\n getAttribute getStatistics getTopFeatures queryFeatures\n---\n\n# Query\n\n&lt;!-- Simplified excerpt: instructions guide tool use, not a fixed graph.\nThe full skill also covers spatial queries, statistics, rankings,\nand sharing query context for map updates. --&gt;\n\n- Find relevant layers with `findRelevantLayers`.\n- Discover fields and values with `findRelevantFields`;\n never invent them.\n- Use `queryFeatures` to count or retrieve matching features.\n- Answer only from tool results; report failures directly.<\/code><\/pre><\/div>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">In our testing, the skill-guided approach improved accuracy and flexibility, but required more model calls and tokens, increasing execution cost. Predefined workflows can be more efficient for predictable tasks, while skill-guided tool use is useful when the agent needs to adapt based on intermediate results. Both approaches remain supported and can be combined.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-vector-search-in-the-browser\">Vector search in the browser<\/h3>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">We introduced embedding-based <a href=\"https:\/\/www.ibm.com\/think\/topics\/vector-search\" target=\"_blank\" rel=\"noreferrer noopener\">vector search<\/a> to retrieve relevant layer and field metadata before query generation. Feature layers are like database tables, and their fields are like columns: the agent needs the relevant schema, not every available schema.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Embeddings represent text as numerical vectors, allowing us to search for layers and fields by meaning rather than exact wording.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Embedding generation is a separate web map preparation step. Embeddings are generated from layer titles and field metadata and stored as a resource on the web map item. When the application loads the map, it downloads those embeddings for browser-side retrieval.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">In the skill-guided agent loop, the model decides when to retrieve context and what search query to use. The retrieval tool embeds that query, searches the stored layer and field embeddings, and returns relevant candidates. The agent can inspect those results and search again when more context is needed. In the nursing homes example, this helps identify the relevant layer and the state and ownership fields before constructing a query.<\/p>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-2 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image\"><img loading=\"lazy\" decoding=\"async\" width=\"940\" height=\"858\" data-id=\"2339\" src=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/vector-search-1.png\" alt=\"Layer and field embeddings are prepared and downloaded on map load. At runtime, an agent invokes retrieval, with similarity ranking and MMR selection running in a browser web worker.\" class=\"wp-image-2339\" srcset=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/vector-search-1.png 940w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/vector-search-1-300x274.png 300w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/vector-search-1-768x701.png 768w\" sizes=\"auto, (max-width: 940px) 100vw, 940px\" \/><\/figure>\n<\/figure>\n\n\n\n<p class=\"has-text-align-center block-editor-paragraph wp-block-paragraph\"><em>Embedding-based retrieval within a skill-guided agent loop.<\/em><\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-improving-relevance-and-responsiveness\">Improving relevance and responsiveness<\/h4>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Our initial search ranked candidates by cosine similarity to the user\u2019s request. This worked well in many cases, but maps with many similar fields\u2014such as&nbsp;<code>population1990<\/code>,&nbsp;<code>population1991<\/code>, and so on\u2014could produce a shortlist dominated by closely related fields, leaving out other useful context.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">We added maximal marginal relevance (<a href=\"https:\/\/www.elastic.co\/search-labs\/blog\/maximum-marginal-relevance-diversify-results\" target=\"_blank\" rel=\"noreferrer noopener\">MMR<\/a>) to balance relevance to the request with diversity among the selected results. This reduced redundancy in the context passed to the model. The additional comparisons increased the retrieval workload, so we moved the search into a web worker to keep it off the main thread.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-evolving-embedding-generation\">Evolving embedding generation<\/h4>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Our early prototypes generated embeddings in the browser using&nbsp;<a href=\"https:\/\/huggingface.co\/Xenova\/all-MiniLM-L6-v2\" target=\"_blank\" rel=\"noreferrer noopener\">all-MiniLM-L6-v2<\/a>. We later experimented with server-hosted embedding models such as&nbsp;<a href=\"https:\/\/developers.openai.com\/api\/docs\/models\/text-embedding-ada-002\" target=\"_blank\" rel=\"noreferrer noopener\">text-embedding-ada-002<\/a>, which improved retrieval quality. At runtime, the retrieval tool calls the server to embed the search query, while vector comparisons and MMR ranking run locally in a browser web worker.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The agents we ship today use embedding-based retrieval, which requires generating, storing, and updating embeddings as map content and metadata change. We are also exploring alternatives such as <a href=\"https:\/\/typesafe.ai\/blog\/introducing-system-one-models-and-jev\" target=\"_blank\" rel=\"noreferrer noopener\">Jev<\/a> and OpenAI\u2019s <a href=\"https:\/\/developers.openai.com\/api\/docs\/guides\/decisions\" target=\"_blank\" rel=\"noreferrer noopener\">Decisions API<\/a> to select relevant layers and fields without maintaining that embedding pipeline.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-the-data-exploration-agent\">The Data Exploration Agent<\/h3>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The <a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/references\/ai-components\/components\/arcgis-assistant-data-exploration-agent\/\" target=\"_blank\" rel=\"noreferrer noopener\">Data Exploration Agent<\/a> answers questions about data in map layers, including counts, summary statistics, feature rankings, and spatial queries.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-tracing-the-nursing-homes-request\">Tracing the nursing homes request<\/h4>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">In one recorded run, the agent handled the request introduced earlier: \u201c<em>Show nonprofit nursing homes in Texas and tell me what percentage of all Texas nursing homes they represent.<\/em>\u201d<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Route the request:<\/strong>&nbsp;The orchestrator selected the Data Exploration Agent.<\/li>\n\n\n\n<li><strong>Load the query skill:<\/strong>&nbsp;<code>load_skill(\"query\")<\/code>&nbsp;loaded instructions for discovering and querying relevant data and made the skill\u2019s tools available to the model.<\/li>\n\n\n\n<li><strong>Find the layer:<\/strong>&nbsp;<code>findRelevantLayers<\/code>&nbsp;returned the United States Nursing Homes layer.<\/li>\n\n\n\n<li><strong>Discover fields and values:<\/strong>&nbsp;<code>findRelevantFields<\/code>&nbsp;returned the state and ownership fields, with unique-value statistics showing&nbsp;<code>TX<\/code>&nbsp;and nonprofit ownership values beginning with&nbsp;<code>Non profit -<\/code>.<\/li>\n\n\n\n<li><strong>Query twice:<\/strong>&nbsp;The agent called&nbsp;<code>queryFeatures<\/code>&nbsp;for nonprofit nursing homes in Texas and for all Texas nursing homes. The calls returned counts of 115 and 1,177, respectively.<\/li>\n\n\n\n<li><strong>Load the filter skill:<\/strong> <code>load_skill(\"filter\")<\/code> loaded instructions for updating the map using the existing query context and made the skill\u2019s tools available to the model.<\/li>\n\n\n\n<li><strong>Emphasize matching facilities:<\/strong>&nbsp;<code>setFeatureEffect<\/code>&nbsp;reused the nonprofit selection criteria, applying a drop shadow to matches while rendering nonmatches in grayscale with reduced opacity and blur.<\/li>\n\n\n\n<li><strong>Return the result:<\/strong> The agent returned a status, summary, and output message reporting 115 of 1,177 facilities, or 9.8%. The orchestrator completed the request without calling another agent.<\/li>\n<\/ol>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The nonprofit query used this&nbsp;<code>where<\/code>&nbsp;clause:<\/p>\n\n\n\n<div class=\"hcb_wrap\"><pre class=\"prism line-numbers lang-plain\"><code>USER_State = &#039;TX&#039;\nAND USER_Ownership_Type LIKE &#039;Non profit - %&#039;<\/code><\/pre><\/div>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The filter skill reused the layer ID and selection criteria established during querying, without repeating layer or field discovery. This kept the statistical answer and map update aligned within the same agent run. For purely statistical questions, the agent can return an answer without changing the map.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The tools determine whether to perform each operation client-side or through the backing feature service. When all the data needed is available in the browser, they operate on it locally. Otherwise, they use the service. This keeps execution close to the data without requiring the model to choose between client-side and server-side execution.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Data Exploration is one of several built-in agents we ship. Another is the <a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/references\/ai-components\/components\/arcgis-assistant-navigation-agent\/\" target=\"_blank\" rel=\"noreferrer noopener\">Navigation Agent<\/a>, which handles zooming, panning, and locating layers, features, places, and addresses using the same skill-guided tool-use pattern.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-custom-agents\">Custom agents<\/h3>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Developers can build <a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/agentic-apps\/ai-custom-agents\/\" target=\"_blank\" rel=\"noreferrer noopener\">custom agents<\/a> and use them alongside the agents we ship, or configure an assistant entirely around custom agents. Each agent advertises a name and a short description of its capabilities. The orchestrator acts primarily as a router, using those descriptions to hand control to an agent suited to the request.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"969\" height=\"791\" src=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/custom-agents-1.png\" alt=\"An orchestrator uses registered names and descriptions to route requests to built-in or custom agents. Optional human input passes through the application UI to pause and resume execution.\" class=\"wp-image-2340\" srcset=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/custom-agents-1.png 969w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/custom-agents-1-300x245.png 300w, https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/custom-agents-1-767x626.png 767w\" sizes=\"auto, (max-width: 969px) 100vw, 969px\" \/><figcaption class=\"wp-element-caption\"><em>Orchestrator routing requests to selected built-in and custom agents.<\/em><\/figcaption><\/figure>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">For requests that require multiple agents, the orchestrator coordinates those handoffs as well. Each agent returns a status, a summary, and an output message for the user. These results help the orchestrator determine whether the request is complete or another agent needs to contribute.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">This applies progressive disclosure at the agent level. The orchestrator does not need every agent\u2019s internal instructions, skills, or tool definitions to make a routing decision\u2014just enough information to know when to call it. The selected agent handles the details of completing its part of the task.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">We support building agents with LangGraph directly, as well as <a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/references\/ai-components\/agent-utils\/BaseAgent\/\" target=\"_blank\" rel=\"noreferrer noopener\">utility functions<\/a> that reduce the LangGraph code for common agent patterns. Both approaches let developers integrate domain-specific workflows into the same orchestration layer.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Several Esri applications, including those without maps, are already using this architecture to build AI-assisted experiences with custom agents tailored to their workflows.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"h-human-in-the-loop-interactions\">Human-in-the-loop interactions<\/h4>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">The orchestration layer passes the agent\u2019s request for input to the application UI and routes the user\u2019s response back to resume the agent. This gives built-in and custom agents a consistent pause-and-resume mechanism without requiring each agent to implement its own UI communication. For example, a custom agent can <a href=\"https:\/\/developers.arcgis.com\/javascript\/latest\/sample-code\/ai-custom-agent-custom-interrupt\/\" target=\"_blank\" rel=\"noreferrer noopener\">pause to ask for confirmation<\/a> before writing a feature to a feature layer or database, then proceed only if the user approves.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"h-closing\">Closing<\/h3>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Live map state, focused context retrieval, and skill-guided tool use give agents a practical foundation for working with the map. Tools can read the application\u2019s current state, agents can gather additional context and revisit decisions, and human-in-the-loop support lets workflows pause when user input is needed.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">In Part 1, the key question was where orchestration should run. In this post, we explored what that architecture enables in the browser: agents that find relevant data, answer questions, and coordinate map interactions, alongside custom agents built for an application\u2019s own workflows.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">Although the examples here focus on mapping, custom agents are not limited to map interactions. Developers can use the same orchestration layer to build workflows that do not interact with a map at all.<\/p>\n\n\n\n<p class=\"undefined block-editor-paragraph wp-block-paragraph\">As models and retrieval approaches evolve, the separation between orchestration, agents, tools and UI components lets us improve those pieces independently. The goal remains the same: help users explore data and carry out tasks through natural language, with the application\u2019s data and state guiding the work.<\/p>\n","protected":false},"author":28,"featured_media":0,"parent":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[62,3],"tags":[124,122,125,127,123,121],"class_list":["post-2332","blog","type-blog","status-publish","format-standard","hentry","category-engineering-design","category-software-development","tag-agent-skills","tag-ai-agents","tag-arcgis-maps-sdk-for-javascript","tag-browser-native","tag-vector-search","tag-webai"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v23.2 (Yoast SEO v25.0) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Inside Browser-Native AI Agents for Mapping - Esri Software Engineering Blog<\/title>\n<meta name=\"description\" content=\"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Inside Browser-Native AI Agents for Mapping\" \/>\n<meta property=\"og:description\" content=\"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents\" \/>\n<meta property=\"og:site_name\" content=\"Esri Software Engineering Blog\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1-1024x459.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"1024\" \/>\n\t<meta property=\"og:image:height\" content=\"459\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents\",\"url\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents\",\"name\":\"Inside Browser-Native AI Agents for Mapping - Esri Software Engineering Blog\",\"isPartOf\":{\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif\",\"datePublished\":\"2026-11-16T18:10:31+00:00\",\"description\":\"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#primaryimage\",\"url\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif\",\"contentUrl\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif\",\"width\":1440,\"height\":645,\"caption\":\"Map highlighting nonprofit nursing homes in Texas alongside the agent\u2019s count and percentage\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Inside Browser-Native AI Agents for Mapping\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/#website\",\"url\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/\",\"name\":\"Esri Software Engineering Blog\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Inside Browser-Native AI Agents for Mapping - Esri Software Engineering Blog","description":"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents","og_locale":"en_US","og_type":"article","og_title":"Inside Browser-Native AI Agents for Mapping","og_description":"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.","og_url":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents","og_site_name":"Esri Software Engineering Blog","og_image":[{"width":1024,"height":459,"url":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1-1024x459.gif","type":"image\/gif"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents","url":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents","name":"Inside Browser-Native AI Agents for Mapping - Esri Software Engineering Blog","isPartOf":{"@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#primaryimage"},"image":{"@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#primaryimage"},"thumbnailUrl":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif","datePublished":"2026-11-16T18:10:31+00:00","description":"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.","breadcrumb":{"@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#primaryimage","url":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif","contentUrl":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/nursing-homes-demo-1.gif","width":1440,"height":645,"caption":"Map highlighting nonprofit nursing homes in Texas alongside the agent\u2019s count and percentage"},{"@type":"BreadcrumbList","@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/articles\/browser-native-ai-agents#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog"},{"@type":"ListItem","position":2,"name":"Inside Browser-Native AI Agents for Mapping"}]},{"@type":"WebSite","@id":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/#website","url":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/","name":"Esri Software Engineering Blog","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"text_date":"November 16, 2026","author_name":"Dhrumil Shah","author_page":false,"custom_image":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/1-hero-wide-1920x1080-1.png","primary_product":false,"tag_data":[{"term_id":124,"name":"agent skills","slug":"agent-skills","term_group":0,"term_taxonomy_id":124,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":122,"name":"AI Agents","slug":"ai-agents","term_group":0,"term_taxonomy_id":122,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":125,"name":"ArcGIS Maps SDK for JavaScript","slug":"arcgis-maps-sdk-for-javascript","term_group":0,"term_taxonomy_id":125,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":127,"name":"browser-native","slug":"browser-native","term_group":0,"term_taxonomy_id":127,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":123,"name":"vector search","slug":"vector-search","term_group":0,"term_taxonomy_id":123,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"},{"term_id":121,"name":"WebAI","slug":"webai","term_group":0,"term_taxonomy_id":121,"taxonomy":"post_tag","description":"","parent":0,"count":1,"filter":"raw"}],"category_data":[{"term_id":62,"name":"Engineering Design","slug":"engineering-design","term_group":0,"term_taxonomy_id":62,"taxonomy":"category","description":"Articles about how we tackle technical and architectural challenges, focusing on algorithms, rendering, and system design decisions","parent":3,"count":4,"filter":"raw"},{"term_id":3,"name":"Software development","slug":"software-development","term_group":0,"term_taxonomy_id":3,"taxonomy":"category","description":"Articles focusing on how we apply coding practices, software design, development methodologies, and tools used while building our products, systems, etc.","parent":0,"count":9,"filter":"raw"}],"product_data":{"errors":{"invalid_taxonomy":["Invalid taxonomy."]},"error_data":[]},"primary_product_link":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/","short_description":"See how browser-native AI agents find relevant data, choose tools, and interact with maps through natural language.","image":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/app\/uploads\/2026\/11\/0-hero-card-826x465-1.png","_links":{"self":[{"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/article\/2332","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/article"}],"about":[{"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/types\/blog"}],"author":[{"embeddable":true,"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/users\/28"}],"version-history":[{"count":0,"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/article\/2332\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/media?parent=2332"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/categories?post=2332"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.esri.com\/en-us\/software-engineering\/blog\/wp-json\/wp\/v2\/tags?post=2332"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}