ArcGIS Velocity<\/a>, including the settings available for data retention, how long to keep data for, and how to ingest data from a feature layer archive.<\/p>\nWhat is data retention?<\/h2>\n
This parameter allows you to specify how long you want the data to be actively maintained in the feature layer. For example, if you have a live feed of vehicle positions and you want to store the historical tracks in a feature layer, you can set the data retention policy to maintain a history of vehicle positions for a set time period. Data retention gives you a way to have more control over your data by keeping it relevant and at a manageable size.<\/p>\n
Why is data retention necessary on a feature layer?<\/h3>\n
Consider a transportation company that utilizes a feature layer to track vehicle locations in real-time. Over months of operation, this layer may accumulate hundreds of thousands of records, including historical locations for each vehicle. The feature layer may experience performance reductions as data accumulates, such as slower response time and decreased query speed. To address this, Velocity offers flexible storage solutions, with layers supporting up to one year of storage within your specified limits. This accumulation of data not only complicates data handling but makes it challenging to visualize insights.<\/p>\n
Implementing data retention can directly address this challenge by allowing users to define a period of time to retain data in a feature layer. By removal of older, less relevant data, organizations can ensure that their feature layers remain streamlined and focused, thus enhancing performance while still retaining essential historical information for analysis.<\/p>\n
What is data retention not intended for?<\/h3>\n
Data retention is not intended to limit or filter features to specific time frames. For example, you want a feature layer to show the vehicle positions from the last hour for the most current view of your fleet. Even if you set the data retention period to one hour, vehicle position records may still be retained for two to three hours because the data removal process operates on a periodic schedule. If you need filter data from specific time periods, you must use alternative methods, such as queries in client applications.<\/p>\n
How do I configure data retention in a feature layer?<\/h2>\n
Real-time or big data analytics can write data to a feature layer output, which can be configured with data retention. To create a feature layer output, create or open an existing real-time or big data analytic. Then select Outputs > <\/span>Feature Layer (new)<\/strong> on the left pane within the editor.<\/p>\nNote- <\/em><\/span>You cannot configure data retention in existing feature layers. You will not be presented with data retention options if you add the output labeled Feature Layer (existing)<\/strong><\/span><\/em>.<\/em><\/span><\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2657562,"id":2657562,"title":"Picture1","filename":"Picture1-1.gif","filesize":1555106,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture1-102","alt":"","author":"367722","description":"","caption":"You can configure data retention when adding the Feature Layer (new) output to a real-time or big data analytic.","name":"picture1-102","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 03:30:13","modified":"2025-01-23 03:31:09","menu_order":0,"mime_type":"image\/gif","type":"image","subtype":"gif","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":694,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1-213x200.gif","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","medium-width":331,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","medium_large-width":768,"medium_large-height":606,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","large-width":880,"large-height":694,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","1536x1536-width":880,"1536x1536-height":694,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","2048x2048-width":880,"2048x2048-height":694,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1-590x465.gif","card_image-width":590,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture1-1.gif","wide_image-width":880,"wide_image-height":694}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"image","image":{"ID":2658952,"id":2658952,"title":"Picture2","filename":"Picture2-3.png","filesize":19630,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture2-64","alt":"The default configuration of a feature layer output in a real-time or big data analytic.","author":"367722","description":"","caption":"The default configuration of a feature layer output in a real-time or big data analytic.","name":"picture2-64","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:26:49","modified":"2025-01-23 21:26:59","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":694,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","medium-width":331,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","medium_large-width":768,"medium_large-height":606,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","large-width":880,"large-height":694,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","1536x1536-width":880,"1536x1536-height":694,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","2048x2048-width":880,"2048x2048-height":694,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3-590x465.png","card_image-width":590,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture2-3.png","wide_image-width":880,"wide_image-height":694}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"In the feature layer configuration window, there is a parameter labeled Data retention (time period for retaining data)<\/strong>.<\/b> This parameter is only visible if Store data in a spatiotemporal feature layer<\/strong> is enabled, and under Each time the analytic starts<\/strong>, Keep existing features and schema<\/strong> is selected. These parameters are selected by default. The data retention parameter is required when Keep existing features and schema<\/strong> is selected because once the analytic publishes the feature layer, it will and may indefinitely grow over time.<\/p>\nNote- If you disable Store data in a spatiotemporal feature layer<\/b><\/strong><\/span><\/em>, the analytic will create an ArcGIS Online hosted feature layer, which is incompatible with data retention. Therefore, disabling this will cause the data retention parameter to disappear. <\/em><\/p>\nNote- If you select Replace existing features and schema<\/strong>, the data retention parameter will disappear. This is because every time the analytic is started\/restarted, any data previously stored in the layer will be discarded and overwritten by new features since starting the analytic.<\/em><\/p>\nThe dropdown menu for data retention will display the following options:<\/p>\n
\n- 1 hour<\/li>\n
- 1 day<\/li>\n
- 1 week<\/li>\n
- 1 month<\/li>\n
- 6 months<\/li>\n
- 1 year<\/li>\n<\/ul>\n
The amount of time you want to retain the data is up to you and may depend on your use case and organization\u2019s available storage. We will explore this topic in the section: How long should I keep data in a feature layer?<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658972,"id":2658972,"title":"picture3","filename":"picture3.jpg","filesize":77331,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture3-63","alt":"The dropdown menu for data retention displays following options: 1 hour, 1 day, 1 week, 1 month, 6 months, and 1 year","author":"367722","description":"","caption":"You can select a data retention period of 1 hour, 1 day, 1 week, 1 month, 6 months, and 1 year.","name":"picture3-63","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:30:14","modified":"2025-01-23 21:33:50","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":694,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","medium-width":331,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","medium_large-width":768,"medium_large-height":606,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","large-width":880,"large-height":694,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","1536x1536-width":880,"1536x1536-height":694,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","2048x2048-width":880,"2048x2048-height":694,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3-590x465.jpg","card_image-width":590,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/picture3.jpg","wide_image-width":880,"wide_image-height":694}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"
After selecting the data retention option, another parameter labeled Data export (feature layer archive) <\/strong>will appear. Selecting the dropdown menu will display Do not export data<\/strong> and Export data<\/strong>. Selecting the Export data<\/strong> will enable Velocity to automatically export data older than the specified data retention period to a feature layer archive. Selecting Do not export data <\/strong>means that Velocity will automatically purge data older than the specified retention period. We will go into detail about how to ingest data from a feature layer archive in the later blog section: How do I ingest data from the feature layer archive?<\/strong><\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2659012,"id":2659012,"title":"Picture4","filename":"Picture4-1.jpg","filesize":21657,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture4-64","alt":"The drop down menu for Data export gives the options \"Do not export data\" and \"Export data\".","author":"367722","description":"","caption":"You have the choice to export data that is older than your chosen data retention period to a feature layer archive.","name":"picture4-64","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:38:19","modified":"2025-01-23 21:39:11","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":549,"height":161,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1-213x161.jpg","thumbnail-width":213,"thumbnail-height":161,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","medium-width":464,"medium-height":136,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","medium_large-width":549,"medium_large-height":161,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","large-width":549,"large-height":161,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","1536x1536-width":549,"1536x1536-height":161,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","2048x2048-width":549,"2048x2048-height":161,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","card_image-width":549,"card_image-height":161,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture4-1.jpg","wide_image-width":549,"wide_image-height":161}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"You can also configure data retention when creating a feed that will be published with a history feature layer. In the Save<\/strong> step during feed configuration, there is a parameter labeled Also publish a feature layer which keeps all observations<\/strong>. Enabling this parameter will allow Velocity to publish a feature layer that will store every record received by the feed. Once enabled, the data retention parameter will appear. You will also have the option to export the data after selecting a retention time period.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2657632,"id":2657632,"title":"Picture5","filename":"Picture5.gif","filesize":849341,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture5-49","alt":"","author":"367722","description":"","caption":"If you configure a feed to publish a feature layer that will store observations, you can configure data retention options on that feature layer.","name":"picture5-49","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 03:46:01","modified":"2025-01-23 03:46:06","menu_order":0,"mime_type":"image\/gif","type":"image","subtype":"gif","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":694,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5-213x200.gif","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","medium-width":331,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","medium_large-width":768,"medium_large-height":606,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","large-width":880,"large-height":694,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","1536x1536-width":880,"1536x1536-height":694,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","2048x2048-width":880,"2048x2048-height":694,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5-590x465.gif","card_image-width":590,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture5.gif","wide_image-width":880,"wide_image-height":694}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"How long should I keep data in a feature layer?<\/h2>\n
You can a data retention period of 1 hour, 1 day, 1 week, 1 month, 6 months,<\/strong> and 1 year<\/strong>. When choosing how long to retain data, you should think about how long the data will remain relevant for your use case, and how much storage you\u2019re willing to allocate to the feature layer. Below, we will consider two scenarios for choosing a data retention period.<\/p>\nScenario 1<\/h3>\n
Consider a fleet of 10 vehicles transmitting their real-time position every second. This data is being written to a feature layer, and the records get updated with their new position every minute. At any given time, the feature layer will only have 10 records (unless you add a new vehicle to the fleet). Since the data is constantly being updated, there\u2019s no need to set a long retention period. Setting a retention period of 1 hour or 1 day would be appropriate in this situation.<\/p>\n
Scenario 2<\/h3>\n
Consider the same fleet of 10 vehicles in scenario 1 writing to a feature layer, except each record is added instead of updating existing records. The resulting feature layer would contain the vehicle\u2019s historical track, enabling you to see a vehicle\u2019s status in the past. This would involve selecting Add all new features<\/strong> for the Data storage<\/strong> method when configuring a feature layer output. In this context, retaining a larger amount of historical data for analysis or incident investigation may be beneficial. Therefore, selecting a longer retention period, such as one month, 6 months, or one year would be suitable.<\/p>\nA longer retention period would result in a larger amount of stored data. Therefore, it\u2019s crucial to consider how the feature layer\u2019s size will grow over time. The chart below shows the size of the feature layer for each data retention option at a rate of 10 KB\/s (10 records that are 1 KB each).<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2657642,"id":2657642,"title":"Picture6","filename":"Picture6-1.png","filesize":48067,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture6-48","alt":"","author":"367722","description":"","caption":"","name":"picture6-48","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 03:46:38","modified":"2025-01-23 03:46:38","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":2147,"height":1140,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1.png","medium-width":464,"medium-height":246,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1.png","medium_large-width":768,"medium_large-height":408,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1.png","large-width":1920,"large-height":1019,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1-1536x816.png","1536x1536-width":1536,"1536x1536-height":816,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1-2048x1087.png","2048x2048-width":2048,"2048x2048-height":1087,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1-826x439.png","card_image-width":826,"card_image-height":439,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture6-1-1920x1019.png","wide_image-width":1920,"wide_image-height":1019}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"
You should also consider the overall storage capacity of your subscription and how much storage you want to allocate to the feature layer. The amount of storage included in your subscription will depend on the license level. The table below shows the storage capacity for each license level.<\/p>\n
\n\n\n| <\/td>\n | Standard<\/td>\n | Advanced<\/td>\n | Dedicated<\/td>\n<\/tr>\n |
\n| Storage capacity<\/td>\n | 128 GB<\/td>\n | 1 TB*<\/td>\n | 3 TB and higher*<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n In Velocity, you can check your Storage utilization<\/strong> by looking at the Storage in the Velocity homepage Subscription Utilization <\/strong>group.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658822,"id":2658822,"title":"Picture7","filename":"Picture7.jpg","filesize":33993,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture7-37","alt":"The Subscription Utilization group in the Velocity homepage.","author":"367722","description":"","caption":"The Subscription Utilization group in the Velocity homepage.","name":"picture7-37","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 20:52:52","modified":"2025-01-23 20:52:58","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":762,"height":260,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","medium-width":464,"medium-height":158,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","medium_large-width":762,"medium_large-height":260,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","large-width":762,"large-height":260,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","1536x1536-width":762,"1536x1536-height":260,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","2048x2048-width":762,"2048x2048-height":260,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","card_image-width":762,"card_image-height":260,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture7.jpg","wide_image-width":762,"wide_image-height":260}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"image","image":{"ID":2658812,"id":2658812,"title":"Picture8","filename":"Picture8.jpg","filesize":11801,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture8-33","alt":"Pie chart showing the storage used and total storage for an ArcGIS Velocity subscription.","author":"367722","description":"","caption":"Administrators have the ability to view the Storage utilization page.","name":"picture8-33","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 20:52:06","modified":"2025-01-23 20:52:16","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":327,"height":203,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","medium-width":327,"medium-height":203,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","medium_large-width":327,"medium_large-height":203,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","large-width":327,"large-height":203,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","1536x1536-width":327,"1536x1536-height":203,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","2048x2048-width":327,"2048x2048-height":203,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","card_image-width":327,"card_image-height":203,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture8.jpg","wide_image-width":327,"wide_image-height":203}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"Selecting the appropriate data retention period for your feature layer is crucial for optimizing both relevance and storage efficiency. By evaluating the nature of your data\u2014whether it is real-time updates or historical records\u2014you can make informed choices that align with your use case. As illustrated in the examples, a short retention period may be suitable for dynamic datasets, while longer retention is beneficial for historical analysis. Additionally, it is essential to monitor your storage capacity and utilization, especially in relation to your subscription level, to ensure that you are maximizing your resources effectively.<\/p>\n What is a feature layer archive?<\/h2>\nWhen configuring a feature layer output and setting a data retention period, you are given the option to Export data<\/strong><\/span>. Choosing this option will mean that data older than the retention period will be removed from the feature layer and stored in a feature layer archive. Velocity will maintain the archived data for up to a year.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2657672,"id":2657672,"title":"Picture9","filename":"Picture9.gif","filesize":1285221,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture9-29","alt":"","author":"367722","description":"","caption":"The Data export parameter will appear after selecting a data retention period.","name":"picture9-29","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 03:49:15","modified":"2025-01-23 03:49:18","menu_order":0,"mime_type":"image\/gif","type":"image","subtype":"gif","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":694,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9-213x200.gif","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","medium-width":331,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","medium_large-width":768,"medium_large-height":606,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","large-width":880,"large-height":694,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","1536x1536-width":880,"1536x1536-height":694,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","2048x2048-width":880,"2048x2048-height":694,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9-590x465.gif","card_image-width":590,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture9.gif","wide_image-width":880,"wide_image-height":694}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"The feature layer will be created once the analytic is run. The data export process operates on a periodic schedule and will send data older than the set retention period to the feature layer archive. The feature layer archive is associated with the feature layer it was configured with and will not appear as a separate item in the Velocity and ArcGIS Online user-interface. To use the data exported to the feature layer archive, it must be added as a data source<\/strong> within a real-time or big data analytic. The steps for this will be covered in the next section.<\/p>\nNote- As the data export process operates on a periodic schedule, you may not immediately see records in the feature layer archive even though the retention time period has passed.<\/em><\/p>\nHow do I use data stored in the feature layer archive?<\/h3>\nTo use the data that has been exported to the feature layer archive, the feature layer must be added as a data source in a real-time or big data analytic.<\/p>\n To get started, select Big Data<\/strong> under Analytics <\/strong>and select Create big data analytic<\/strong>. Select See all<\/strong> in the ArcGIS<\/strong> category > Feature Layer<\/strong> > Feature Layer<\/strong> (archive)<\/strong>. From the list, choose the feature layer that was configured with data export.<\/p>\nNote- Only feature layers configured with data export will appear in the list. You will receive an error message if you select a feature layer with no archived data. Either not enough time has passed to exceed the data retention time period, or the periodic data export process has not yet occurred for this feature layer.<\/em><\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2657682,"id":2657682,"title":"Picture10","filename":"Picture10.gif","filesize":2025416,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture10-31","alt":"","author":"367722","description":"","caption":"","name":"picture10-31","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 03:51:59","modified":"2025-01-23 03:51:59","menu_order":0,"mime_type":"image\/gif","type":"image","subtype":"gif","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":628,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10-213x200.gif","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","medium-width":366,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","medium_large-width":768,"medium_large-height":548,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","large-width":880,"large-height":628,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","1536x1536-width":880,"1536x1536-height":628,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","2048x2048-width":880,"2048x2048-height":628,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10-652x465.gif","card_image-width":652,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture10.gif","wide_image-width":880,"wide_image-height":628}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"In the Confirm Schema<\/strong> section, the data will automatically be sampled and identified in Parquet format. Upon reviewing the schema, you may notice that four additional fields were added to the feature layer: objectid<\/strong>, globalid<\/strong>, SHAPE<\/strong>, and DATE<\/strong>. These are system-managed fields that are added when the feature layer is created. If you plan to send the data to an ArcGIS output, such as a feature layer or stream layer, uncheck or rename the SHAPE<\/strong> field. \u201cSHAPE\u201d is a reserved name and cannot be used.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658912,"id":2658912,"title":"Picture11","filename":"Picture11.jpg","filesize":104111,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture11-25","alt":"The sampled data from the feature layer archive.","author":"367722","description":"","caption":"The sampled data from the feature layer archive.","name":"picture11-25","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:16:12","modified":"2025-01-23 21:16:53","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":825,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","medium-width":278,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","medium_large-width":768,"medium_large-height":720,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","large-width":880,"large-height":825,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","1536x1536-width":880,"1536x1536-height":825,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","2048x2048-width":880,"2048x2048-height":825,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11-496x465.jpg","card_image-width":496,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture11.jpg","wide_image-width":880,"wide_image-height":825}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"In the Identify Key Fields<\/strong> section, the Location<\/strong> parameters will automatically be handled by the data source since the data is in Parquet format. Select a field for Start time<\/strong> if your data has a time field, then select a field to assign as the Track ID<\/strong> field. Click complete<\/strong> to create the data source.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658882,"id":2658882,"title":"Picture12","filename":"Picture12.jpg","filesize":77920,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture12-23","alt":"The Location parameter in the Identify Key Fields section states that the location data will be autoatically handled.","author":"367722","description":"","caption":"The location data will be autoatically handled with the feature layer archive.","name":"picture12-23","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:10:31","modified":"2025-01-23 21:11:47","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":694,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","medium-width":331,"medium-height":261,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","medium_large-width":768,"medium_large-height":606,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","large-width":880,"large-height":694,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","1536x1536-width":880,"1536x1536-height":694,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","2048x2048-width":880,"2048x2048-height":694,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12-590x465.jpg","card_image-width":590,"card_image-height":465,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture12.jpg","wide_image-width":880,"wide_image-height":694}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"With the feature layer archive configured as a data source, you can configure various tools to analyze patterns and trends and then send the data to various output types.<\/p>\n As stated earlier in the blog, Velocity maintains the archived data for up to a year. If you \u00a0need to retain archived data indefinitely and have access to Amazon S3<\/strong> or Azure Blob Storage<\/strong>, you can send the data to the associated output type in a big data analytic.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658842,"id":2658842,"title":"Picture13","filename":"Picture13.jpg","filesize":22034,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture13-22","alt":"Amazone S3 and Azure Blob Store are available output types.","author":"367722","description":"","caption":"Amazone S3 and Azure Blob Store are available output types.","name":"picture13-22","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:01:41","modified":"2025-01-23 21:02:15","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":602,"height":187,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13-213x187.jpg","thumbnail-width":213,"thumbnail-height":187,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","medium-width":464,"medium-height":144,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","medium_large-width":602,"medium_large-height":187,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","large-width":602,"large-height":187,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","1536x1536-width":602,"1536x1536-height":187,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","2048x2048-width":602,"2048x2048-height":187,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","card_image-width":602,"card_image-height":187,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture13.jpg","wide_image-width":602,"wide_image-height":187}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"Merge a feature layer with its archive in a big data analytic<\/h3>\nYou can combine the datasets of a feature layer and its associated feature layer archive into a single pipeline for analysis. For example, let\u2019s say our feature layer currently has data from the last week and our feature layer archive contains data from the previous12 months. \u00a0These datasets can be combined using the Merge<\/strong> tool in a big data analytic.<\/p>\nTo get started, select Big Data<\/strong> under Analytics <\/strong>and select Create big data analytic<\/strong>. Select See all<\/strong> in the ArcGIS<\/strong> category ><\/strong> Feature Layer<\/strong> ><\/strong> Feature Layer<\/strong>. Select the feature layer that was configured with data export and go through the rest of the configuration steps to add this data source. Once the feature layer with the current dataset has been added, go to the left pane and select Sources<\/strong> > More data sources > See all<\/strong> in the ArcGIS<\/strong> category ><\/strong> Feature Layer<\/strong> > Feature Layer (archive)<\/strong>. In the list, select the same feature layer from the previous step.<\/p>\nNote- The feature layer and feature layer archive node labels will share the same name, and whichever item is added last will have \u201c-1\u201d appended to its label. For example, if you add the feature layer \u201cTracks\u201d into the analytic, and then add its associated feature layer archive, the feature layer archive\u2019s node label will be displayed as \u201cTracks-1\u201d. You can leave it as is or rename the label to something like \u201cTracks (archive)\u201d. This will help with keeping track of which data source node you\u2019re working with.<\/em><\/p>\nAfter completing the steps, we have two feature layer data source nodes: one containing the current dataset and the other with the archived dataset. We will use the Merge<\/strong> tool to combine the two nodes. To add the tool, go to the left pane and select Manage Data > Merge<\/strong>. Connect the two data source nodes to the Merge tool node. Both datasets will merge into a single pipeline, which can be analyzed using additional tools and then sent to an output.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658862,"id":2658862,"title":"Picture14","filename":"Picture14.jpg","filesize":32181,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture14-19","alt":"The feature layer and feature layer archive nodes connected to the merge tool.","author":"367722","description":"","caption":"","name":"picture14-19","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:03:22","modified":"2025-01-23 21:03:50","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":615,"height":246,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","medium-width":464,"medium-height":186,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","medium_large-width":615,"medium_large-height":246,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","large-width":615,"large-height":246,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","1536x1536-width":615,"1536x1536-height":246,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","2048x2048-width":615,"2048x2048-height":246,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","card_image-width":615,"card_image-height":246,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture14.jpg","wide_image-width":615,"wide_image-height":246}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"You may receive a validation error on the Merge <\/strong>tool, especially if you chose to keep all the fields when configuring both data sources.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2657732,"id":2657732,"title":"Picture15","filename":"Picture15-1.png","filesize":7419,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture15-11","alt":"The validation error on the merge tool reads: \"The tag 'OBJECT_ID' was present on the target schema but not the join schema.\"","author":"367722","description":"","caption":"","name":"picture15-11","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 03:59:55","modified":"2025-01-23 21:04:46","menu_order":0,"mime_type":"image\/png","type":"image","subtype":"png","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":569,"height":243,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1-213x200.png","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","medium-width":464,"medium-height":198,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","medium_large-width":569,"medium_large-height":243,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","large-width":569,"large-height":243,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","1536x1536-width":569,"1536x1536-height":243,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","2048x2048-width":569,"2048x2048-height":243,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","card_image-width":569,"card_image-height":243,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture15-1.png","wide_image-width":569,"wide_image-height":243}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"As stated previously in the blog, objectid<\/strong> is one of the system-managed fields that were added when the feature layer was created. For the feature layer, the \u201cOBJECT_ID\u201d tag is automatically assigned to the objectid field automatically, but the tag does not get assigned to the feature layer archive\u2019s objectid field. The feature layer and feature layer archive fields\u2019 tags must match for a merge to be possible. You can resolve this by removing the objectid<\/strong> field in the Confirm Schema<\/strong> tab when configuring the data source.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658942,"id":2658942,"title":"Picture16","filename":"Picture16.jpg","filesize":69541,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture16-11","alt":"In the Confirm Schema section, the objectid field is unchecked.","author":"367722","description":"","caption":"Uncheck the objectid field to remove it from the schema.","name":"picture16-11","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:22:45","modified":"2025-01-23 21:24:58","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":880,"height":483,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","medium-width":464,"medium-height":255,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","medium_large-width":768,"medium_large-height":422,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","large-width":880,"large-height":483,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","1536x1536-width":880,"1536x1536-height":483,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","2048x2048-width":880,"2048x2048-height":483,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16-826x453.jpg","card_image-width":826,"card_image-height":453,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture16.jpg","wide_image-width":880,"wide_image-height":483}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"Join a feature layer archive to a feed in a real-time analytic<\/h3>\nThe two previous scenarios involved ingesting a feature layer archive in a big data analytic. Real-time analytics are used to process and analyze real-time data coming from a feed. The feature layer archive is considered a static data source and cannot be added to real-time analytics by itself. For this reason, a join needs to be configured between the feed and feature layer archive. Using the Join Features <\/strong>tool, you can configure a spatial, temporal, or attribute-based join.<\/p>\nTo get started, select Real-Time <\/strong>under Analytics <\/strong>and select Create real-time analytic<\/strong>. You can select an existing feed<\/strong> or configure a new feed<\/strong> to which you wish to join the feature layer archive data to. Once the feed has been configured and added to the real-time analytic, go to the left pane and select Sources<\/strong> > More data sources > See all<\/strong> in the ArcGIS<\/strong> category ><\/strong> Feature Layer<\/strong> > Feature Layer (archive)<\/strong>. \u00a0Now that the feed and feature layer archive data source have been added, we can configure the Join Features<\/strong> tool. In the left pane, select Summarize Data > Join Features<\/strong>. You can determine the join operation, relationship, and the rest of the options based on your use case with the data. Once the Join Features<\/strong> tool has been configured, you can add additional tools and then send the data to an output.<\/p>\n"},{"acf_fc_layout":"image","image":{"ID":2658922,"id":2658922,"title":"Picture17","filename":"Picture17.jpg","filesize":34806,"url":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","link":"https:\/\/www.esri.com\/arcgis-blog\/products\/arcgis-velocity\/data-management\/data-retention-in-arcgis-velocity-key-concepts-best-practices\/picture17-8","alt":"The AVL Feed and feature layer archive connected to the join features tool.","author":"367722","description":"","caption":"","name":"picture17-8","status":"inherit","uploaded_to":2657302,"date":"2025-01-23 21:17:38","modified":"2025-01-23 21:18:16","menu_order":0,"mime_type":"image\/jpeg","type":"image","subtype":"jpeg","icon":"https:\/\/www.esri.com\/arcgis-blog\/wp-includes\/images\/media\/default.png","width":637,"height":253,"sizes":{"thumbnail":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17-213x200.jpg","thumbnail-width":213,"thumbnail-height":200,"medium":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","medium-width":464,"medium-height":184,"medium_large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","medium_large-width":637,"medium_large-height":253,"large":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","large-width":637,"large-height":253,"1536x1536":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","1536x1536-width":637,"1536x1536-height":253,"2048x2048":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","2048x2048-width":637,"2048x2048-height":253,"card_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","card_image-width":637,"card_image-height":253,"wide_image":"https:\/\/www.esri.com\/arcgis-blog\/app\/uploads\/2025\/01\/Picture17.jpg","wide_image-width":637,"wide_image-height":253}},"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"Conclusion<\/h2>\nData retention is a valuable solution for organizations that want to maintain feature layers at a manageable size in ArcGIS Velocity. When a data retention period is set for a feature layer, any features older than the specified duration are removed from the underlying dataset. This approach allows organizations to streamline their data storage by keeping only relevant data, thereby reducing storage space requirements. Organizations can still access historical data by exporting older records to a feature layer archive, which can be retrieved when analyzing historical information is needed.<\/p>\n We are always looking to hear back from the community, if you have any additional questions, please reach out to our team on the |
"},{"ID":364242,"user_firstname":"Adhista","user_lastname":"Chapagain","nickname":"Adhista Chapagain","user_nicename":"achapagain","display_name":"Adhista Chapagain","user_email":"achapagain@esri.com","user_url":"","user_registered":"2024-10-31 19:53:14","user_description":"Adhista is a Product Engineer at Esri.","user_avatar":"
"}],"short_description":" This blog explores key concepts and best practices for using data retention in ArcGIS Velocity.","flexible_content":[{"acf_fc_layout":"content","content":"