"},{"ID":263392,"user_firstname":"Albert","user_lastname":"Schelin","nickname":"Albert Schelin","user_nicename":"aschelin","display_name":"Albert Schelin","user_email":"ASchelin@esri.com","user_url":"","user_registered":"2021-08-12 21:11:17","user_description":"Albert is a Product Marketing Manager at Esri, promoting ArcGIS Web Editor and ArcGIS Data Pipelines. When he isn't promoting Esri's products, you can find him enjoying a cup of single origin light roast coffee at a local coffee shop.","user_avatar":"
"},{"ID":325202,"user_firstname":"Sarah","user_lastname":"Hanson","nickname":"Sarah Hanson","user_nicename":"shanson","display_name":"Sarah Hanson","user_email":"shanson@esri.com","user_url":"","user_registered":"2023-01-11 15:54:54","user_description":"Sarah is a Product Manager on the ArcGIS Enterprise team, focused on data management. She also manages ArcGIS Data Pipelines, a native data integration capability in ArcGIS. In her free time, Sarah loves to explore new places and spend time outdoors.","user_avatar":"
"}],"short_description":"What's New in ArcGIS Data Pipelines (June 2025)","flexible_content":[{"acf_fc_layout":"content","content":"If you find yourself in the repetitive loop of preparing data and then publishing the latest copy of it as a feature to ArcGIS Online, you need to familiarize yourself with <\/span>ArcGIS Data Pipelines<\/span><\/a>. Data Pipelines simplifies data preparation workflows, offering a visual, drag-and-drop interface to connect, clean, and transform your data. The results are written out to feature layers in your ArcGIS organization, ready for use in your visualization and analysis workflows. More importantly, Data Pipelines supports built-in task scheduling functionality and other ways to automate data pipelines runs, such a using <\/span>Microsoft Power Automate<\/span><\/a>, helping you to ensure that feature layers are kept up-to-date without any repetitive manual effort or scripting expertise.<\/span>\u00a0<\/span><\/p>\n Before we delve into the new improvements coming to ArcGIS Data Pipelines in the June update, let\u2019s take a minute to recap some of the recent updates to ArcGIS Data Pipelines. The previous update in <\/span>February 2025<\/span><\/a> brought us a new tool, <\/span>Summarize attributes<\/span><\/a>, the ability to <\/span>filter your data with dynamic time queries<\/span><\/a>, the ability to sort and search your notes, and more. Before that, the <\/span>November 2024<\/span><\/a> update introduced data pipeline notes, the new <\/span>Pivot<\/span><\/a> and <\/span>Dissolve<\/span><\/a> tools, and the ability to <\/span>overwrite existing layers<\/span><\/a> as an output method. Now that we\u2019ve seen some of the recent features and enhancements that have come, let\u2019s dive into the new improvements in the June 2025 update.<\/span>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"content","content":" This update to ArcGIS Data Pipelines is ramps up the possibilities for data integration in ArcGIS Online, with two new input data sources, a new data engineering tool, and an enhanced preview experience for complex field types. Each enhancement was driven by user\u2019s feedback, so we encourage you to continue sharing your ideas and feedback with us through <\/span>Esri Community<\/span><\/a>.<\/span>\u00a0<\/span><\/p>\n Let\u2019s get into the details!<\/span>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"content","content":" You\u2019ve asked for it and we\u2019ve listened. You can now connect to secure <\/span>URLs<\/span><\/a> like open data portals that require a password or commercial REST APIs that use token-based access. This new input significantly expands the range of sources you can integrate with ArcGIS Online, without any need for a custom solution. When connecting to your secured URLs, your credentials, secrets, or tokens will be securely stored in a new service connection item. Service connection items enable you to easily and repeatedly connect to your secured data. <\/span>This feature is in beta. <\/span>\u00a0<\/span><\/p>\n Note: This feature is currently in beta, meaning it is still under active development and may be subject to change. Technical support is not available while features are in beta, and feedback should be submitted via this <\/span><\/i>Esri Community Forum<\/span><\/i><\/a> to help guide future improvements or seek support.<\/span><\/i>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2844632,"id":2844632,"title":"adp_service_connection_ui","filename":"adp_service_connection_ui.png","filesize":34115,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-online\/announcements\/whats-new-in-arcgis-data-pipelines-june-2025\/adp_service_connection_ui","alt":"Service connection modal in ArcGIS Data Pipelines","author":"7431","description":"","caption":"","name":"adp_service_connection_ui","status":"inherit","uploaded_to":2803042,"date":"2025-06-20 18:49:02","modified":"2025-06-20 18:49:17","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":838,"height":744,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","medium-width":294,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","medium_large-width":768,"medium_large-height":682,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","large-width":838,"large-height":744,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","1536x1536-width":838,"1536x1536-height":744,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","2048x2048-width":838,"2048x2048-height":744,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui-524x465.png","card_image-width":524,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/adp_service_connection_ui.png","wide_image-width":838,"wide_image-height":744}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":" There\u2019s another new input data source in the June 2025 update that we are excited to share. One of the top requested ideas on Esri Community was the ask for us to support connecting to data in Databricks. In this release, we have added this support, allowing you to bring data from your <\/span>Databricks<\/span><\/a> warehouse into ArcGIS Online via a data pipeline. <\/span>\u00a0<\/span><\/p>\n Note: This feature is currently in beta, meaning it is still under active development and may be subject to change. Technical support is not available while features are in beta, and feedback should be submitted via this <\/span><\/i>Esri Community Forum<\/span><\/i><\/a> to help guide future improvements or seek support.<\/span><\/i>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"content","content":" The <\/span>Clip<\/span><\/a> tool allows you to cut portions of geometries in one dataset using one or more geometries of another dataset, like a cookie cutter. This data processing tool gives you the ability to more easily limit data to a specific area of interest. For example, you can use the Clip tool to extract points that fall within a specific geographic boundary for targeted analysis, such as identifying your customers who live within a specific sales region.<\/span>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2844712,"id":2844712,"title":"clip_800px","filename":"clip_800px.gif","filesize":1711182,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-online\/announcements\/whats-new-in-arcgis-data-pipelines-june-2025\/clip_800px","alt":"The Clip tool in ArcGIS Data Pipelines.","author":"7431","description":"","caption":"","name":"clip_800px","status":"inherit","uploaded_to":2803042,"date":"2025-06-20 19:28:24","modified":"2025-06-20 19:28:46","menu_order":0,"mime_type":"image\/gif","type":"image","subtype":"gif","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":850,"height":660,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px-213x200.gif","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","medium-width":336,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","medium_large-width":768,"medium_large-height":596,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","large-width":850,"large-height":660,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","1536x1536-width":850,"1536x1536-height":660,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","2048x2048-width":850,"2048x2048-height":660,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px-599x465.gif","card_image-width":599,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/clip_800px.gif","wide_image-width":850,"wide_image-height":660}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":" Snowflake announced that they will block single-factor password authentication later this year. When multi-factor authentication (MFA) is enforced on a user\u2019s Snowflake account, connections to Snowflake in ArcGIS Data Pipelines using basic authentication will stop functioning.\u202fTo ensure our users can continue integrating data from Snowflake with ArcGIS Online, we have added support for using key-pair authentication in this release. You will be able to establish new connections to <\/span>Snowflake<\/span><\/a> using key-pair authentication, or you can also update existing data store items created using basic authentication to switch to this authentication type. For more information on Esri\u2019s recommendations to mitigate potential disruptions due to Snowflake’s upcoming multi-factor authentication (MFA) enforcement, please refer to this <\/span>Technical Support article<\/span><\/a>.<\/span>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"content","content":" Data Pipelines allows you to preview data at every step of your data pipeline, with support for previewing the dataset\u2019s table (attributes), geometry (locations on a map), as well as the schema (data structure definition). <\/span>\u00a0<\/span><\/p>\n In this update, we have enhanced preview experience for complex field types, or provide in-depth insight into subfields and subtypes within struct, array, or map fields when previewing the data\u2019s schema. For example, if you have a struct field named This enhancement allows <\/span>you <\/span>to delve into complex data structures, enabling better interpretation and schema design for optimized data processing<\/span>.<\/span><\/span>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"content","content":" Whether you’re just getting started with ArcGIS Data Pipelines or looking to sharpen your skills, we\u2019ve got you covered. Here are some helpful resources to explore:<\/span>\u00a0<\/span><\/p>\n We\u2019re always listening! If you have <\/span>ideas<\/span><\/a>, <\/span>questions<\/span><\/a>, or feedback, drop by the Data Pipelines Community and let us know what you’d like to see next.<\/span>\u00a0<\/span><\/p>\n"}],"related_articles":"","show_article_image":false,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/06\/Data-Pipelines-Q2-Card.png","wide_image":false},"yoast_head":"\nWhat\u2019s New<\/span><\/b>\u00a0<\/span><\/h3>\n
Secured URL support (beta)\u00a0<\/span><\/b>\u00a0<\/span><\/h3>\n
Databricks support (beta)\u00a0<\/span><\/b>\u00a0<\/span><\/h3>\n
Clip tool\u00a0<\/span><\/b>\u00a0<\/span><\/h3>\n
Key-pair authentication for Snowflake<\/span><\/b>\u00a0<\/span><\/h3>\n
Enhanced schema preview for complex fields<\/span><\/b>\u00a0<\/span><\/h3>\n
City<\/code> containing values such as {\"x\":-63.5827, \"y\": 44.6511, \"name\": \"Halifax\"}<\/code>, the schema preview will reveal subfields of x (double), y (double), and name (string) within the City field. <\/span>\u00a0<\/span><\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2844812,"id":2844812,"title":"schema","filename":"schema-1.gif","filesize":599258,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-online\/announcements\/whats-new-in-arcgis-data-pipelines-june-2025\/schema-2","alt":"Complex schema preview experience in ArcGIS Data Pipelines","author":"7431","description":"","caption":"","name":"schema-2","status":"inherit","uploaded_to":2803042,"date":"2025-06-20 19:38:43","modified":"2025-06-20 19:39:00","menu_order":0,"mime_type":"image\/gif","type":"image","subtype":"gif","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":800,"height":428,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1-213x200.gif","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","medium-width":464,"medium-height":248,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","medium_large-width":768,"medium_large-height":411,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","large-width":800,"large-height":428,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","1536x1536-width":800,"1536x1536-height":428,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","2048x2048-width":800,"2048x2048-height":428,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","card_image-width":800,"card_image-height":428,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/05\/schema-1.gif","wide_image-width":800,"wide_image-height":428}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"Additional resources<\/span><\/b>\u00a0<\/span><\/h3>\n
\n
\n
\n
\n
\n
\n