{"id":1851862,"date":"2023-02-23T09:42:35","date_gmt":"2023-02-23T17:42:35","guid":{"rendered":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&#038;p=1851862"},"modified":"2023-03-06T10:06:43","modified_gmt":"2023-03-06T18:06:43","slug":"migrating-data-into-arcgis-hub-part-2","status":"publish","type":"blog","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2","title":{"rendered":"Migrating data into ArcGIS Hub &#8211; Part 2"},"author":308072,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"closed","template":"","format":"standard","meta":{"_acf_changed":false,"_searchwp_excluded":""},"categories":[37131,22851,37511],"tags":[173712,230422,762182,230432,764952],"industry":[],"product":[36841,36811,36551,36561,380802],"class_list":["post-1851862","blog","type-blog","status-publish","format-standard","hentry","category-constituent-engagement","category-national-government","category-sharing-collaboration","tag-arcgis-hub","tag-data","tag-data-migration","tag-engagement","tag-etl","product-api-python","product-arcgis-hub","product-arcgis-online","product-arcgis-pro","product-arcgis-storymaps"],"acf":{"authors":[{"ID":308072,"user_firstname":"Nick","user_lastname":"O'Day","nickname":"Nick O'Day","user_nicename":"noday","display_name":"Nick O'Day","user_email":"noday@esri.com","user_url":"","user_registered":"2022-04-12 16:31:08","user_description":"Nick O'Day is a Senior Consultant on Esri's Professional Services team where he leads complex ArcGIS implementations that drive insight and action through data. He also hosts Esri's \"Engagement Matters\" podcast where he talks to experts that are setting new standards in collaboration and engagement. Before joining Esri in 2022, Nick worked for 18 years in data analytics supporting local government, consulting firms, and startup companies using ArcGIS as a Chief Data Officer and GIS Manager.  When Nick isn't GIS'ing, he's usually cooking, eating, or exploring something new in sunny Los Angeles.","user_avatar":"<img data-del=\"avatar\" src='https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/Profile-Picture-213x200.jpg' class='avatar pp-user-avatar avatar-96 photo ' height='96' width='96'\/>"}],"short_description":"Getting your data from an external, non-ArcGIS system into ArcGIS Hub can be a challenge.  Let's demystify the process...","flexible_content":[{"acf_fc_layout":"content","content":"<p>ArcGIS Hub gets a lot of love, and rightfully so.\u00a0 It\u2019s a tool that makes it easy to show the extent of an issue or progress made towards a goal.\u00a0 It also makes accessing the raw data behind these visualizations simple which increases transparency and builds trust with your community.\u00a0 The challenge then for most people is in getting (and keeping) high quality, actionable, and relevant data into your ArcGIS Hub site.<\/p>\n<p>As we talked about in <a href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub\/\">Part 1 of this blog series<\/a>, there are a variety of ways that you can get data <em>into<\/em> Hub. The tool you choose largely depends upon two things: how often your data updates and your technical skill level.\u00a0 If your data never changes (imagine a dataset that has only building permits from 2018), then it\u2019s likely that you don\u2019t need a formal extract, transfer, and load (ETL) process.\u00a0 On the other hand, if your dataset is continually updating on some regular schedule (imagine a table of E911 calls) then it\u2019s much more likely that you will need an ETL process to keep your data in Hub fresh.<\/p>\n<p>Of the five most common tools that were outlined previously, the one I tend to rely on the most is Data Interoperability (also known as FME Desktop).\u00a0 It offers several key benefits that push it to the top of my list which may or may not apply to you:<\/p>\n<ul>\n<li>No coding required: I don\u2019t know python well enough to code a complex ETL process on my own<\/li>\n<li>Routine can be scheduled: I regularly need an ETL process that can be executed on a schedule with no human intervention<\/li>\n<li>Easy to maintain: Once an ETL process is built, it\u2019s easy for me to share it with someone else on the team (even if they have less technical skills) and for them to maintain it<\/li>\n<li>Clear process: Adding comments or annotations directly in my ETL process is essential so that I can remember what\u2019s happening in each step and communicate that to anyone in the team that ends up maintaining the process.<\/li>\n<\/ul>\n"},{"acf_fc_layout":"content","content":"<p>The Data Interoperability (DI) extension for ArcGIS Pro (again, also called FME Desktop) has a very intuitive interface.\u00a0 If you\u2019ve ever seen a flow chart diagram where there\u2019s a start, some stuff in the middle, and an end result, then you already have a basic understanding of how DI works.\u00a0 For clarity, here are the key terms you\u2019ll need to know:<\/p>\n<ul>\n<li>Reader: A reader is the first block in your process where the software connects to a data source. This can be something as simple as a flat csv file or as complex as data stored in a NoSQL Amazon DynamoDB table.\u00a0 You can connect to more than one data source in a single process (as is the case when you need to join two different datasets together) but you\u2019ll always need at least one reader.<\/li>\n<li>Writer: A writer is the last block in your process where the software saves or writes the result out. Since we are focused on getting data into ArcGIS Hub, the data you have processed would be written either to a feature service in ArcGIS Enterprise or ArcGIS Online for it to be accessible in your Hub site. \u00a0As with a reader, you can have multiple writers in a single process.\u00a0 For example, you could write the result of your process out to a hosted feature layer and a csv.\u00a0 Either way, you\u2019ll always have at least one writer.<\/li>\n<li>Transformer: A transformer is the step in between reading and writing data. You can have no transformers (which is rare), one transformer, or many transformers in your process.\u00a0 It all depends on how much you need to clean, adjust, or enrich your data before you write it to its destination. \u00a0Most commonly, transformers are used to rename or reorder fields, alter values in rows (like changing coded values to actual words), perform enrichment of data (e.g., adding a city council district number to a point when it falls within the council district polygon), and more.<\/li>\n<li>Workbench: The workbench is the \u201ccanvas\u201d that holds all the readers, writers, and transformers in your process. Within your workbench you can add annotations to help future you (or your coworkers) understand the flow of data through the process. The workbench is also a file that you can execute on a routine basis using something like Windows Task Scheduler.<\/li>\n<li>Features:\u00a0 The term \u201cfeatures\u201d can be used interchangeably with \u201crecords.\u201d\u00a0 GIS professionals typically refer to records in a table as features (since \u201cfeatures\u201d are what we call the points, lines, and\/or polygons in a map) and Data Interoperability\/FME Desktop follows this nomenclature.\u00a0 If you\u2019re coming to this technology from an IT background, just know that the two terms mean the same thing.<\/li>\n<\/ul>\n"},{"acf_fc_layout":"content","content":"<h2><strong>Scenario<\/strong><\/h2>\n<p>To help illustrate the usefulness of Data Interoperability, here\u2019s a hypothetical scenario: Imagine you work for a city that wants to share building permit information.\u00a0 The source system overwrites a csv file each night with all the new building permits that were processed that day.\u00a0 Your task is to create an automated process that takes this new file and adds the records within it to a hosted feature layer in your city\u2019s ArcGIS Hub site.\u00a0 The overall process will look like this:<\/p>\n<ol>\n<li>Examine the Data<\/li>\n<li>Build the workbench to create the hosted feature layer for the first time:\u00a0 Read, Transform, and then Write the Data<\/li>\n<li>Confirm that the data written is correct and fill out remaining item information<\/li>\n<li>Alter the workbench so that it appends future records to the (now) existing hosted feature layer:\u00a0 Update the reader csv source to a new csv file and then update the writer settings to append to the existing hosted feature layer<\/li>\n<\/ol>\n<h3><strong>Step 1:\u00a0 Examine the data<\/strong><\/h3>\n<p>The first, and most important step to any ETL process is to examine the dataset.\u00a0 This can take many forms but often it\u2019s easiest to look at a csv file in a spreadsheet program like Microsoft Excel.\u00a0 Look for the values present in the columns.\u00a0 Common things to ask are:<\/p>\n<p style=\"padding-left: 80px\"><em>Standardization<\/em>: Are the values consistent in the fields? This applies most to fields that have<\/p>\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li>Coded value domains: Do all the rows in the table have a value in the list of potential values? Are there some rows with old values or does the source system allow users to override the domain and enter a free text value?<\/li>\n<li>False duplicate values: Are there values that a human would see as duplicates but that a program would see as different values? Example of this could be:\u00a0\u00a0\u201cCity of Happyville\u201d,\u00a0\u201cCITY OF HAPPYVILLE\u201d,\u00a0\u201cCity of Happyville Government&#8221;<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p style=\"padding-left: 80px\"><em>Null or blank values<\/em>: Are there null values where you expect to see information? Similarly, are there blank values where there should really be a null?\u00a0 A blank field could mean missing data whereas a null value typically means no data.\u00a0 Having this consistently in your dataset is important as it impacts the usability of data by users as well as in charts and maps.<\/p>\n<p style=\"padding-left: 80px\"><em>Dates and times<\/em>: Are date fields using a consistent format? Are the values in a particular time zone?\u00a0 Do time fields contain offsets for daylight savings time?\u00a0 Do time fields use a 12- or 24-hour clock?<\/p>\n<p style=\"padding-left: 80px\"><em>Special characters<\/em>: Are there values that contain commas, dashes, ampersands or other characters that could complicate the reading\/writing of data into certain formats?\u00a0 For example, if there are commas in a field, this could create issues when trying to work with the data in a csv (<em>comma<\/em>-separated value file).<\/p>\n<p style=\"padding-left: 80px\"><em>Location<\/em>: Do the records in the table have location information? Is it in a consistent format like an address or is there a mix of addresses, street intersections, and parcel numbers?<\/p>\n<p style=\"padding-left: 80px\"><em>Links to other datasets<\/em>: Is there a foreign key present in the data that can be used to associate it to another dataset (as in a parcel APN number for each associated building permit)?<\/p>\n<p style=\"padding-left: 80px\"><em>Personally identifiable or restricted data<\/em>: Perhaps most importantly, does the table contain any information that is restricted by laws or policies? Does the data expose personally identifiable information (like names, addresses, social security numbers, or victim information) that needs to be removed or obfuscated prior to publishing?<\/p>\n<p>Examining your dataset before you begin building an ETL process is extremely important and will guide the transformers you will need to use in the following steps.<\/p>\n<h3><strong>Step 2:\u00a0 Build the workbench to create the hosted feature layer for the first time<\/strong><\/h3>\n<p>After you have examined the raw dataset and understand its nuances and challenges, we can begin building the workbench in Data Interoperability.\u00a0 We are assuming in this step that we are creating a hosted feature layer (which will be hosted in your organization\u2019s ArcGIS Online account).\u00a0 The same process can be used if you are relying on services hosted in ArcGIS Enterprise in your infrastructure.<\/p>\n<h3><strong>Step 2a:\u00a0 Read the Data<\/strong><\/h3>\n<p>Since our data is accessible through a csv that is updated nightly, our reader will connect to the csv itself.\u00a0 The first step here is to open Data Interoperability and open the \u201cReader\u201d tool.<\/p>\n<p>From within the Reader prompt, we can search for whatever format we need to read (in this case we\u2019re looking for csv) and then start filling in other information like file location.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1851952,"id":1851952,"title":"data-into-hub-part2-reader","filename":"data-into-hub-part2-reader.png","filesize":31277,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-reader","alt":"reader dialog box in FME Desktop","author":"308072","description":"","caption":"","name":"data-into-hub-part2-reader","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:27:39","modified":"2023-02-22 22:28:00","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":376,"height":209,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","medium-width":376,"medium-height":209,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","medium_large-width":376,"medium_large-height":209,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","large-width":376,"large-height":209,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","1536x1536-width":376,"1536x1536-height":209,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","2048x2048-width":376,"2048x2048-height":209,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","card_image-width":376,"card_image-height":209,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader.png","wide_image-width":376,"wide_image-height":209}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Once we save the configuration Data Interoperability will attempt to read in the data from the csv.\u00a0 It\u2019s best to have the \u201cenable feature caching\u201d option turned on when building a workbench for the first time.\u00a0 This option is found in the dropdown next to the Run button.\u00a0 The interface will show us a cached version of the data from the csv that we can use to visually inspect the data.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1851962,"id":1851962,"title":"data-into-hub-part2-reader-records","filename":"data-into-hub-part2-reader-records.png","filesize":61333,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-reader-records","alt":"Records from FME Desktop reader component","author":"308072","description":"","caption":"","name":"data-into-hub-part2-reader-records","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:28:14","modified":"2023-02-22 22:28:35","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":330,"height":369,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","medium-width":233,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","medium_large-width":330,"medium_large-height":369,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","large-width":330,"large-height":369,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","1536x1536-width":330,"1536x1536-height":369,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","2048x2048-width":330,"2048x2048-height":369,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","card_image-width":330,"card_image-height":369,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-reader-records.png","wide_image-width":330,"wide_image-height":369}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>It\u2019s important to note that Data Interoperability can also connect directly to a variety of databases (the \u201csystem of record\u201d) but that process requires a read-only login to the database and understanding the table structure of the database both of which are beyond the scope of this introductory blog.<\/p>\n<h3><strong>Step 2b:\u00a0 Transform the Data<\/strong><\/h3>\n<p>There are a variety of transformers available in Data Interoperability and even more in the community that you can utilize.\u00a0 The most common ones that I use and, coincidentally, what we\u2019ll use in this example are:<\/p>\n<p style=\"padding-left: 80px\"><a href=\"https:\/\/docs.safe.com\/fme\/html\/FME_Desktop_Documentation\/FME_Transformers\/Transformers\/attributemanager.htm\"><em>AttributeManager<\/em><\/a>: This transformer is my go-to, multi-tool that can change field names, adjust values, and much more.\u00a0 We\u2019ll use it to change the field names and remove the underscores in the data (because I can\u2019t stand underscores in data\u2026 I don\u2019t know why)<\/p>\n<p style=\"padding-left: 80px\"><a href=\"https:\/\/docs.safe.com\/fme\/html\/FME_Desktop_Documentation\/FME_Transformers\/Transformers\/datetimeconverter.htm\"><em>DateTimeConverter<\/em><\/a>: Have weird datetime values (or even text strings) in your data that you need to standardize?\u00a0 We do in this dataset so we\u2019ll convert the data in the \u201cdate issued\u201d field into a format that works with ArcGIS Online.<\/p>\n<p style=\"padding-left: 80px\"><a href=\"https:\/\/docs.safe.com\/fme\/html\/FME_Desktop_Documentation\/FME_Transformers\/Transformers\/centerpointreplacer.htm\"><em>CenterPointReplacer<\/em><\/a>: If you need to create a point feature from a polygon (centroid), this transformer will do it with ease. \u00a0We\u2019ll use the CenterPointReplacer to convert a spatial parcel layer (which is polygons) into single points. \u00a0As an aside, the <a href=\"https:\/\/docs.safe.com\/fme\/html\/FME_Desktop_Documentation\/FME_Transformers\/Transformers\/vertexcreator.htm\">VertexCreator<\/a> is a better choice if you have a coordinate pair and need to create a point.<\/p>\n<p style=\"padding-left: 80px\"><a href=\"https:\/\/docs.safe.com\/fme\/html\/FME_Desktop_Documentation\/FME_Transformers\/Transformers\/featuremerger.htm\"><em>FeatureMerger<\/em><\/a>: If you need to perform a join of one dataset to another using a key, this is the tool to use.\u00a0 You can do inner, outer, left and right joins with this transformer.\u00a0 We\u2019ll use it to join the building permit data to a spatial parcel dataset which will:<\/p>\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li>Add value to the building permit data: Users like the flexibility of data that has a spatial component in ArcGIS Hub. Plus, it\u2019s cool to see building permits on a map over time.<\/li>\n<li>Increase the spatial accuracy of the building permit data: Using existing spatial data to geocode the building permit data is a more accurate process than street-based geocoding.<\/li>\n<li>Reduce the amount of geocoding we must do: Geocoders are great but can sometimes be less accurate than other methods of spatializing data.\u00a0 Also, they use credits to run (unless you\u2019re using your own custom geocoder) which can add up over time.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p style=\"padding-left: 80px\"><a href=\"https:\/\/hub.safe.com\/publishers\/safe-lab\/transformers\/arcgisonlinebatchgeocoder\"><em>ArcGISOnlineBatchGeocoder<\/em><\/a>: Sometimes you just need to sling a bunch of records against a geocoder and see what sticks\u2026 this is the transformer for doing just that.\u00a0 In our case, we\u2019ll use it to make any of the remaining building permits that didn\u2019t match with a parcel have a point.<\/p>\n<p>Once all the transformers in the workbench are wired together, it will appear like this:<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1851992,"id":1851992,"title":"data-into-hub-part2-transformers","filename":"data-into-hub-part2-transformers.png","filesize":77596,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-transformers","alt":"transformers in the workbench","author":"308072","description":"","caption":"","name":"data-into-hub-part2-transformers","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:30:28","modified":"2023-02-22 22:30:41","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":610,"height":212,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","medium-width":464,"medium-height":161,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","medium_large-width":610,"medium_large-height":212,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","large-width":610,"large-height":212,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","1536x1536-width":610,"1536x1536-height":212,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","2048x2048-width":610,"2048x2048-height":212,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","card_image-width":610,"card_image-height":212,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-transformers.png","wide_image-width":610,"wide_image-height":212}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>There are a few important things to note about our workbench as it currently exists:<\/p>\n<ul>\n<li>The diagram is designed to be read from left to right and top to bottom. You can rearrange this to suit your needs but I\u2019ve found it\u2019s easiest to arrange the flow as long and linear with lots of annotation.<\/li>\n<li>There is an additional reader branch at the bottom of the diagram (bottom left). This is to read in parcel data that is converted into points and joined with the building permits (based on the parcel ID) to make them spatial without needing to geocode all 40,000+ records.<\/li>\n<li>There is an additional transformer that we haven\u2019t talked about yet: \u00a0 When records are geocoded against the ArcGIS Online geocoder, they are projected in what\u2019s called the \u201cWorld Geodetic System\u201d (translation: latitude and longitude).\u00a0 Local governments in the United States typically use the State Plan Coordinate System (translation: northing and easting).\u00a0 \u00a0Basically, we need to convert the geocoder\u2019s results from one type of coordinates to another so that all our spatial data lines up correctly.<\/li>\n<li>There is no writer in this diagram yet as we\u2019ll add it in the next step.<\/li>\n<\/ul>\n<h3><strong>Step 2c:\u00a0 Write the Data<\/strong><\/h3>\n<p>Now that we\u2019ve successfully read and transformed the data, we need to write it to our ArcGIS Online account.\u00a0 Since this data doesn\u2019t exist as a hosted feature service, this first writer process will create the layer and load the records.\u00a0 Later, we\u2019ll go into ArcGIS Online and update the item\u2019s details including a thumbnail, summary, description, tags, and more so that users will understand what to expect in the dataset when they find it in your ArcGIS Hub site.<\/p>\n<p>To start this process, we launch the \u201cWriter\u201d dialog and then search for \u201cEsri ArcGIS Online (AGOL) Feature Service\u201d as our output format.\u00a0 We\u2019ll also configure the output hosted feature service name, ArcGIS Online account to use, target projection, and more.\u00a0 Inputs with a red exclamation mark (!) or that are highlighted in pink must be filled in and any other inputs are optional.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1852002,"id":1852002,"title":"data-into-hub-part2-writer","filename":"data-into-hub-part2-writer.png","filesize":55907,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-writer","alt":"writer dialog box in the workbench","author":"308072","description":"","caption":"","name":"data-into-hub-part2-writer","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:31:25","modified":"2023-02-22 22:31:49","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":371,"height":446,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","medium-width":217,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","medium_large-width":371,"medium_large-height":446,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","large-width":371,"large-height":446,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","1536x1536-width":371,"1536x1536-height":446,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","2048x2048-width":371,"2048x2048-height":446,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","card_image-width":371,"card_image-height":446,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer.png","wide_image-width":371,"wide_image-height":446}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>Once the configuration is done, we\u2019ll run the entire workbench and publish the data to ArcGIS Online.\u00a0 The workbench now looks like this with all of the readers, transformers, and writer configured:<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1852012,"id":1852012,"title":"data-into-hub-part2-wholeworkbench","filename":"data-into-hub-part2-wholeworkbench.png","filesize":84855,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-wholeworkbench","alt":"completed sample workbench","author":"308072","description":"","caption":"","name":"data-into-hub-part2-wholeworkbench","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:32:03","modified":"2023-02-22 22:32:18","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":709,"height":212,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","medium-width":464,"medium-height":139,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","medium_large-width":709,"medium_large-height":212,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","large-width":709,"large-height":212,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","1536x1536-width":709,"1536x1536-height":212,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","2048x2048-width":709,"2048x2048-height":212,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","card_image-width":709,"card_image-height":212,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-wholeworkbench.png","wide_image-width":709,"wide_image-height":212}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<p>When we run the workbench, the translation log at the bottom of the screen will show messages that give us insight into what\u2019s happening.\u00a0 Messages highlighted in red typically indicate failed processes and warrant a deeper look.<\/p>\n<h3><strong>Step 3:\u00a0 Confirm that the data written is correct and fill out remaining item information<\/strong><\/h3>\n<p>At this point, we need to login to our ArcGIS Online account and make sure that the data has been written correctly to the hosted feature layer.\u00a0 This means that we will look at the data in both a table and a map.\u00a0 We\u2019re looking for general things like improperly written values, unexpectedly null or blank cells where data should be, and even data that appears way outside of the city limits (possibly indicating that either the projection settings were incorrect or poor geocoding results).<\/p>\n<p>Additionally, we need to complete the dataset\u2019s item page details.\u00a0 These details include:<\/p>\n<ul>\n<li>Title: Clear title of the dataset that non-experts will be able to understand<\/li>\n<li>Summary: A brief summary of the what the dataset is and where it comes from<\/li>\n<li>Description: A deeper summary of the dataset with more detail about the source, update cadence, and any other important characteristics of the dataset that we need users to know (without looking at the formal metadata)<\/li>\n<li>Thumbnail: A high quality, standardized thumbnail that visually communicates what the dataset covers and ideally includes branding information from our organization<\/li>\n<li>Terms of Use: Typically, this section includes a description of the data sharing policy (i.e., Creative Commons, custom license) that helps users understand what they can do with the data and what they shouldn\u2019t do with the data<\/li>\n<li>Categories: Categories help group datasets together within your organization and ArcGIS Hub site in order to make searching and finding similar datasets easier for users<\/li>\n<li>Tags: Specific search terms that users might use to search and find this specific dataset<\/li>\n<li>Credits: The originator\u2019s name(s) that users might have to add to the data if they use or republish it elsewhere<\/li>\n<li>Sharing: The groups and visibility level of this specific dataset that governs if the dataset appears in the ArcGIS Hub site and if it is visible to everyone (public), organization (semi-private), or just to the data owner\/publisher (private)<\/li>\n<\/ul>\n<p>Making sure that these components of the item page are filled out helps users of our ArcGIS Hub site understand the provenance of the data at a glance while also helping to power the search and filtering capabilities of the site.<\/p>\n<h3><strong>Step 4:\u00a0 Alter the workbench so that it appends future records to the newly created layer<\/strong><\/h3>\n<p>Now that we have a hosted feature layer that has all the \u201cold\u201d building permit records and contains the critical item page information, we can adjust our workbench to keep our data fresh.\u00a0 There are a few things we need to do to accomplish this.<\/p>\n<h3><strong>Step 4a:\u00a0 Update reader csv source to new csv file<\/strong><\/h3>\n<p>At the beginning of this process, we discovered that our source system that holds all building permit data can create a new csv file nightly that contains only the new building permits processed for that day.\u00a0 To adjust our existing workbench to take advantage of this, all we need to do is change the dataset source to the new file.\u00a0 This assumes that the schema of the file is the same as what we used previously and that the file name won\u2019t change (source system will simply overwrite the same file).<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1852022,"id":1852022,"title":"data-into-hub-part2-writer-reconfig1","filename":"data-into-hub-part2-writer-reconfig1.png","filesize":25493,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-writer-reconfig1","alt":"writer being reconfigured","author":"308072","description":"","caption":"","name":"data-into-hub-part2-writer-reconfig1","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:33:01","modified":"2023-02-22 22:33:13","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":416,"height":186,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1-213x186.png","thumbnail-width":213,"thumbnail-height":186,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","medium-width":416,"medium-height":186,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","medium_large-width":416,"medium_large-height":186,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","large-width":416,"large-height":186,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","1536x1536-width":416,"1536x1536-height":186,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","2048x2048-width":416,"2048x2048-height":186,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","card_image-width":416,"card_image-height":186,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig1.png","wide_image-width":416,"wide_image-height":186}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<h3><strong>Step 4b:\u00a0 Update writer settings to append to existing hosted feature layer<\/strong><\/h3>\n<p>When we originally built the workbench, the writer was configured to \u201cInsert\u201d records and \u201cUse Existing\u201d as shown below.\u00a0 Frequently when I build a workbench, I\u2019ll change the feature type handling to \u201cTruncate Existing\u201d since I\u2019m testing the process and want the workbench to delete all the records from the hosted feature layer before writing new ones.\u00a0 Make sure that you switch these settings back so you append new data rather than wipe out all your old data.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":1852032,"id":1852032,"title":"data-into-hub-part2-writer-reconfig2","filename":"data-into-hub-part2-writer-reconfig2.png","filesize":40333,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\/data-into-hub-part2-writer-reconfig2","alt":"writer reconfigured to update existing layer","author":"308072","description":"","caption":"","name":"data-into-hub-part2-writer-reconfig2","status":"inherit","uploaded_to":1851862,"date":"2023-02-22 22:33:24","modified":"2023-02-22 22:33:41","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":431,"height":323,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","medium-width":348,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","medium_large-width":431,"medium_large-height":323,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","large-width":431,"large-height":323,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","1536x1536-width":431,"1536x1536-height":323,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","2048x2048-width":431,"2048x2048-height":323,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","card_image-width":431,"card_image-height":323,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/data-into-hub-part2-writer-reconfig2.png","wide_image-width":431,"wide_image-height":323}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"<h2><strong>Conclusion<\/strong><\/h2>\n<p>Data Interoperability is a great tool to help you more easily migrate your data into your ArcGIS Hub site.\u00a0 It takes a little bit of time to learn the terms and to figure out how to build a process that works for your data, but the reward is well worth the effort.\u00a0 If your goal with ArcGIS Hub is to drive transparency or engage with your community, the best way to do it is to make fresh data available.\u00a0 Your users will thank you and, more importantly, they\u2019ll trust the data and visualizations that you create.<\/p>\n<p>If you\u2019re interested in learning more about Data Interoperability and the many, <em>many<\/em> things it can do for you, check out the <a href=\"https:\/\/community.esri.com\/t5\/arcgis-data-interoperability-blog\/bg-p\/arcgis-data-interoperability-blog\">ArcGIS Data Interoperability blog<\/a>.<\/p>\n<h2><strong>Explore this Workbench<\/strong><\/h2>\n<p>Click <a href=\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2023\/02\/Demo-BuildingPermits-shared.zip\">here<\/a> to get a copy of the workbench used in this blog.\u00a0 When deploying this on your computer make sure you have a license (even if it\u2019s a trial license) for ArcGIS Data Interoperability or FME Desktop.<\/p>\n<p>Steps to deploy this workbench on your machine:<\/p>\n<ul>\n<li>Download the building permit data from the link below as a csv file and update the path to the csv in the Building Permits reader (top left of workbench).<\/li>\n<li>Configure your ArcGIS Online account credentials by going to Navigator &gt; Web Connections &gt; Esri ArcGIS Online.<\/li>\n<li>Change the name of the hosted feature layer in the Building Permits writer to something in your ArcGIS Online organization.<\/li>\n<\/ul>\n<h3><strong>Acknowledgement<\/strong><\/h3>\n<p>Special thanks to the City of Worcester, MA\u2019s \u201c<a href=\"https:\/\/opendata.worcesterma.gov\/\">Informing Worcester<\/a>\u201d open data site built on ArcGIS Hub and their <a href=\"https:\/\/opendata.worcesterma.gov\/datasets\/building-permits\/explore\">building permits<\/a> and <a href=\"https:\/\/opendata.worcesterma.gov\/datasets\/parcel-polygons\/explore\">parcel<\/a> data used in this blog.<\/p>\n<p>&nbsp;<\/p>\n<h3><strong>Hungry for more ArcGIS Hub content and learning?<\/strong><\/h3>\n<ul>\n<li>Attend\u00a0<a class=\"fui-Link ___1qmgydl f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh ftqa4ok f2hkw1w fhgqx19 f1olyrje f1p93eir f1h8hb77 f1x7u7e9 f10aw75t fsle3fq\" title=\"https:\/\/www.esri.com\/en-us\/landing-page\/product\/2020\/arcgis-hub-webinar-series\" href=\"https:\/\/www.esri.com\/en-us\/landing-page\/product\/2020\/arcgis-hub-webinar-series\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"Link webinars\">webinars<\/a>\u00a0and meet-ups to discover how others are using ArcGIS Hub<\/li>\n<li>View ArcGIS Hub conference presentations, demos, and how-to <a class=\"fui-Link ___1qmgydl f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh ftqa4ok f2hkw1w fhgqx19 f1olyrje f1p93eir f1h8hb77 f1x7u7e9 f10aw75t fsle3fq\" title=\"https:\/\/mediaspace.esri.com\/channel\/arcgis%2bhub\/238049132\" href=\"https:\/\/mediaspace.esri.com\/channel\/ArcGIS%2BHub\/238049132\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"Link videos\">videos<\/a><\/li>\n<li>Sign up for the ArcGIS Hub\u00a0<a class=\"fui-Link ___1qmgydl f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh ftqa4ok f2hkw1w fhgqx19 f1olyrje f1p93eir f1h8hb77 f1x7u7e9 f10aw75t fsle3fq\" title=\"https:\/\/www.esri.com\/en-us\/arcgis\/products\/arcgis-hub\/news-sign-up\" href=\"https:\/\/www.esri.com\/en-us\/arcgis\/products\/arcgis-hub\/news-sign-up\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"Link e-newsletter\">e-newsletter<\/a><\/li>\n<li>Follow us on Twitter\u00a0<a class=\"fui-Link ___1qmgydl f3rmtva f1ewtqcl fyind8e f1k6fduh f1w7gpdv fk6fouc fjoy568 figsok6 f1hu3pq6 f11qmguv f19f4twv f1tyq0we f1g0x7ka fhxju0i f1qch9an f1cnd47f fqv5qza f1vmzxwi f1o700av f13mvf36 f1cmlufx f9n3di6 f1ids18y f1tx3yz7 f1deo86v f1eh06m1 f1iescvh ftqa4ok f2hkw1w fhgqx19 f1olyrje f1p93eir f1h8hb77 f1x7u7e9 f10aw75t fsle3fq\" title=\"https:\/\/twitter.com\/arcgishub\" href=\"https:\/\/twitter.com\/ArcGISHub\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"Link @ArcGISHub\">@ArcGISHub<\/a><\/li>\n<\/ul>\n"}],"related_articles":[{"ID":1562212,"post_author":"308072","post_date":"2022-04-27 09:14:08","post_date_gmt":"2022-04-27 16:14:08","post_content":"","post_title":"Migrating data into ArcGIS Hub - Part 1","post_excerpt":"","post_status":"publish","comment_status":"open","ping_status":"closed","post_password":"","post_name":"migrating-data-into-arcgis-hub","to_ping":"","pinged":"","post_modified":"2023-03-06 10:10:38","post_modified_gmt":"2023-03-06 18:10:38","post_content_filtered":"","post_parent":0,"guid":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&#038;p=1562212","menu_order":0,"post_type":"blog","post_mime_type":"","comment_count":"0","filter":"raw"},{"ID":1562012,"post_author":"308942","post_date":"2022-11-10 12:15:15","post_date_gmt":"2022-11-10 20:15:15","post_content":"","post_title":"Have you heard about Hub discussions?","post_excerpt":"","post_status":"publish","comment_status":"open","ping_status":"closed","post_password":"","post_name":"have-you-heard-about-hub-discussions","to_ping":"","pinged":"","post_modified":"2022-12-12 10:42:44","post_modified_gmt":"2022-12-12 18:42:44","post_content_filtered":"","post_parent":0,"guid":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&#038;p=1562012","menu_order":0,"post_type":"blog","post_mime_type":"","comment_count":"4","filter":"raw"},{"ID":1732932,"post_author":"308942","post_date":"2025-10-28 11:10:12","post_date_gmt":"2025-10-28 18:10:12","post_content":"","post_title":"Tips and timesavers for ArcGIS Hub and Enterprise Sites","post_excerpt":"","post_status":"publish","comment_status":"open","ping_status":"closed","post_password":"","post_name":"tips-timesavers-sites-hub","to_ping":"","pinged":"","post_modified":"2025-12-05 13:47:55","post_modified_gmt":"2025-12-05 21:47:55","post_content_filtered":"","post_parent":0,"guid":"https:\/\/www.esri.com\/arcgis-blog\/?post_type=blog&#038;p=1732932","menu_order":0,"post_type":"blog","post_mime_type":"","comment_count":"0","filter":"raw"}],"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/g2721746-migrating-data-hub-article-card2.jpg","wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/g2721746-migrating-data-hub-article-banners.jpg"},"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Migrating data into ArcGIS Hub - Part 2<\/title>\n<meta name=\"description\" content=\"Data is the core of any successful ArcGIS Hub initiative. Learn how to migrate data from non-ArcGIS systems into ArcGIS Hub.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Migrating data into ArcGIS Hub - Part 2\" \/>\n<meta property=\"og:description\" content=\"Data is the core of any successful ArcGIS Hub initiative. Learn how to migrate data from non-ArcGIS systems into ArcGIS Hub.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\" \/>\n<meta property=\"og:site_name\" content=\"ArcGIS Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/esrigis\/\" \/>\n<meta property=\"article:modified_time\" content=\"2023-03-06T18:06:43+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@ESRI\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":[\"Article\",\"BlogPosting\"],\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\"},\"author\":{\"name\":\"Nick O'Day\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/9d8c95ab1c23afc3b4d5f32ea66228ab\"},\"headline\":\"Migrating data into ArcGIS Hub &#8211; Part 2\",\"datePublished\":\"2023-02-23T17:42:35+00:00\",\"dateModified\":\"2023-03-06T18:06:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\"},\"wordCount\":7,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#organization\"},\"keywords\":[\"ArcGIS Hub\",\"data\",\"Data Migration\",\"engagement\",\"etl\"],\"articleSection\":[\"Constituent Engagement\",\"Government\",\"Sharing and Collaboration\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\",\"name\":\"Migrating data into ArcGIS Hub - Part 2\",\"isPartOf\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#website\"},\"datePublished\":\"2023-02-23T17:42:35+00:00\",\"dateModified\":\"2023-03-06T18:06:43+00:00\",\"description\":\"Data is the core of any successful ArcGIS Hub initiative. Learn how to migrate data from non-ArcGIS systems into ArcGIS Hub.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.esri.com\/arcgis-blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Migrating data into ArcGIS Hub &#8211; Part 2\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#website\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/\",\"name\":\"ArcGIS Blog\",\"description\":\"Get insider info from Esri product teams\",\"publisher\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.esri.com\/arcgis-blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#organization\",\"name\":\"Esri\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png\",\"contentUrl\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png\",\"width\":400,\"height\":400,\"caption\":\"Esri\"},\"image\":{\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/esrigis\/\",\"https:\/\/x.com\/ESRI\",\"https:\/\/www.linkedin.com\/company\/5311\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/9d8c95ab1c23afc3b4d5f32ea66228ab\",\"name\":\"Nick O'Day\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/Profile-Picture-213x200.jpg\",\"contentUrl\":\"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/Profile-Picture-213x200.jpg\",\"caption\":\"Nick O'Day\"},\"description\":\"Nick O'Day is a Senior Consultant on Esri's Professional Services team where he leads complex ArcGIS implementations that drive insight and action through data. He also hosts Esri's \\\"Engagement Matters\\\" podcast where he talks to experts that are setting new standards in collaboration and engagement. Before joining Esri in 2022, Nick worked for 18 years in data analytics supporting local government, consulting firms, and startup companies using ArcGIS as a Chief Data Officer and GIS Manager. When Nick isn't GIS'ing, he's usually cooking, eating, or exploring something new in sunny Los Angeles.\",\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/nick-oday\"],\"knowsLanguage\":[\"English\"],\"jobTitle\":\"Senior Consultant\",\"worksFor\":\"Esri\",\"url\":\"https:\/\/www.esri.com\/arcgis-blog\/author\/noday\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Migrating data into ArcGIS Hub - Part 2","description":"Data is the core of any successful ArcGIS Hub initiative. Learn how to migrate data from non-ArcGIS systems into ArcGIS Hub.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2","og_locale":"en_US","og_type":"article","og_title":"Migrating data into ArcGIS Hub - Part 2","og_description":"Data is the core of any successful ArcGIS Hub initiative. Learn how to migrate data from non-ArcGIS systems into ArcGIS Hub.","og_url":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2","og_site_name":"ArcGIS Blog","article_publisher":"https:\/\/www.facebook.com\/esrigis\/","article_modified_time":"2023-03-06T18:06:43+00:00","twitter_card":"summary_large_image","twitter_site":"@ESRI","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":["Article","BlogPosting"],"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#article","isPartOf":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2"},"author":{"name":"Nick O'Day","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/9d8c95ab1c23afc3b4d5f32ea66228ab"},"headline":"Migrating data into ArcGIS Hub &#8211; Part 2","datePublished":"2023-02-23T17:42:35+00:00","dateModified":"2023-03-06T18:06:43+00:00","mainEntityOfPage":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2"},"wordCount":7,"commentCount":0,"publisher":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#organization"},"keywords":["ArcGIS Hub","data","Data Migration","engagement","etl"],"articleSection":["Constituent Engagement","Government","Sharing and Collaboration"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2","url":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2","name":"Migrating data into ArcGIS Hub - Part 2","isPartOf":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#website"},"datePublished":"2023-02-23T17:42:35+00:00","dateModified":"2023-03-06T18:06:43+00:00","description":"Data is the core of any successful ArcGIS Hub initiative. Learn how to migrate data from non-ArcGIS systems into ArcGIS Hub.","breadcrumb":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-hub\/constituent-engagement\/migrating-data-into-arcgis-hub-part-2#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.esri.com\/arcgis-blog\/"},{"@type":"ListItem","position":2,"name":"Migrating data into ArcGIS Hub &#8211; Part 2"}]},{"@type":"WebSite","@id":"https:\/\/www.esri.com\/arcgis-blog\/#website","url":"https:\/\/www.esri.com\/arcgis-blog\/","name":"ArcGIS Blog","description":"Get insider info from Esri product teams","publisher":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.esri.com\/arcgis-blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.esri.com\/arcgis-blog\/#organization","name":"Esri","url":"https:\/\/www.esri.com\/arcgis-blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png","contentUrl":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2018\/04\/Esri.png","width":400,"height":400,"caption":"Esri"},"image":{"@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/esrigis\/","https:\/\/x.com\/ESRI","https:\/\/www.linkedin.com\/company\/5311\/"]},{"@type":"Person","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/9d8c95ab1c23afc3b4d5f32ea66228ab","name":"Nick O'Day","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/arcgis-blog\/#\/schema\/person\/image\/","url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/Profile-Picture-213x200.jpg","contentUrl":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/Profile-Picture-213x200.jpg","caption":"Nick O'Day"},"description":"Nick O'Day is a Senior Consultant on Esri's Professional Services team where he leads complex ArcGIS implementations that drive insight and action through data. He also hosts Esri's \"Engagement Matters\" podcast where he talks to experts that are setting new standards in collaboration and engagement. Before joining Esri in 2022, Nick worked for 18 years in data analytics supporting local government, consulting firms, and startup companies using ArcGIS as a Chief Data Officer and GIS Manager. When Nick isn't GIS'ing, he's usually cooking, eating, or exploring something new in sunny Los Angeles.","sameAs":["https:\/\/www.linkedin.com\/in\/nick-oday"],"knowsLanguage":["English"],"jobTitle":"Senior Consultant","worksFor":"Esri","url":"https:\/\/www.esri.com\/arcgis-blog\/author\/noday"}]}},"text_date":"February 23, 2023","author_name":"Nick O'Day","author_page":"https:\/\/www.esri.com\/arcgis-blog\/author\/noday","custom_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2022\/04\/g2721746-migrating-data-hub-article-banners.jpg","primary_product":"ArcGIS Hub","tag_data":[{"term_id":173712,"name":"ArcGIS Hub","slug":"arcgis-hub","term_group":0,"term_taxonomy_id":173712,"taxonomy":"post_tag","description":"","parent":0,"count":39,"filter":"raw"},{"term_id":230422,"name":"data","slug":"data","term_group":0,"term_taxonomy_id":230422,"taxonomy":"post_tag","description":"","parent":0,"count":33,"filter":"raw"},{"term_id":762182,"name":"Data Migration","slug":"data-migration","term_group":0,"term_taxonomy_id":762182,"taxonomy":"post_tag","description":"","parent":0,"count":9,"filter":"raw"},{"term_id":230432,"name":"engagement","slug":"engagement","term_group":0,"term_taxonomy_id":230432,"taxonomy":"post_tag","description":"","parent":0,"count":13,"filter":"raw"},{"term_id":764952,"name":"etl","slug":"etl","term_group":0,"term_taxonomy_id":764952,"taxonomy":"post_tag","description":"","parent":0,"count":8,"filter":"raw"}],"category_data":[{"term_id":37131,"name":"Constituent Engagement","slug":"constituent-engagement","term_group":0,"term_taxonomy_id":37131,"taxonomy":"category","description":"","parent":0,"count":392,"filter":"raw"},{"term_id":22851,"name":"Government","slug":"national-government","term_group":0,"term_taxonomy_id":22851,"taxonomy":"category","description":"","parent":0,"count":413,"filter":"raw"},{"term_id":37511,"name":"Sharing and Collaboration","slug":"sharing-collaboration","term_group":0,"term_taxonomy_id":37511,"taxonomy":"category","description":"","parent":0,"count":424,"filter":"raw"}],"product_data":[{"term_id":36841,"name":"ArcGIS API for Python","slug":"api-python","term_group":0,"term_taxonomy_id":36841,"taxonomy":"product","description":"","parent":36601,"count":151,"filter":"raw"},{"term_id":36811,"name":"ArcGIS Hub","slug":"arcgis-hub","term_group":0,"term_taxonomy_id":36811,"taxonomy":"product","description":"","parent":36591,"count":218,"filter":"raw"},{"term_id":36551,"name":"ArcGIS Online","slug":"arcgis-online","term_group":0,"term_taxonomy_id":36551,"taxonomy":"product","description":"","parent":0,"count":2419,"filter":"raw"},{"term_id":36561,"name":"ArcGIS Pro","slug":"arcgis-pro","term_group":0,"term_taxonomy_id":36561,"taxonomy":"product","description":"","parent":0,"count":2035,"filter":"raw"},{"term_id":380802,"name":"ArcGIS StoryMaps","slug":"arcgis-storymaps","term_group":0,"term_taxonomy_id":380802,"taxonomy":"product","description":"","parent":0,"count":323,"filter":"raw"}],"primary_product_link":"https:\/\/www.esri.com\/arcgis-blog\/?s=#&products=arcgis-hub","_links":{"self":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/blog\/1851862","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/blog"}],"about":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/types\/blog"}],"author":[{"embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/users\/308072"}],"replies":[{"embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/comments?post=1851862"}],"version-history":[{"count":0,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/blog\/1851862\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/media?parent=1851862"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/categories?post=1851862"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/tags?post=1851862"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/industry?post=1851862"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/www.esri.com\/arcgis-blog\/wp-json\/wp\/v2\/product?post=1851862"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}