{"id":176501,"date":"2019-02-01T11:23:21","date_gmt":"2019-02-01T19:23:21","guid":{"rendered":"https:\/\/www.esri.com\/about\/newsroom\/?post_type=arcnews&p=176501"},"modified":"2023-09-05T16:14:56","modified_gmt":"2023-09-05T23:14:56","slug":"breathe-life-into-big-data","status":"publish","type":"arcnews","link":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/breathe-life-into-big-data","title":{"rendered":"Breathe Life into Big Data"},"author":1312,"featured_media":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"sync_status":"","episode_type":"","audio_file":"","podmotor_file_id":"","podmotor_episode_id":"","castos_file_data":"","cover_image":"","cover_image_id":"","duration":"","filesize":"","filesize_raw":"","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":"","_links_to":"","_links_to_target":""},"categories":[260002],"tags":[157432,871,241,209132,276692],"arcnews_issues":[243981],"class_list":["post-176501","arcnews","type-arcnews","status-publish","format-standard","hentry","category-data-management","tag-arcgis","tag-big-data","tag-gis","tag-github","tag-hadoop","arcnews_issues-summer-2013"],"acf":{"short_description":"With data influx reaching exabyte levels, big data can easily become unmanageable and useless without the proper tools to analyze it fast. ","pdf":{"host_remotely":false,"file":"","file_url":""},"flexible_content":[{"acf_fc_layout":"image","image":176601,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"With data influx reaching exabyte levels,\u00a0big data<\/em>\u00a0can easily become unmanageable and useless without the proper tools to analyze it fast. The strength of your data stewardship therefore depends on how much command you have over big data. To bypass the capacity limitations of working with billions of records at a time, Esri created GIS Tools for Hadoop, a toolkit for executing spatial analysis in the Hadoop environment and looping it back into ArcGIS."},{"acf_fc_layout":"image","image":176611,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

Geoenable Big Data in Hadoop<\/h3>\r\nMany data stewards and developers have come to rely on Hadoop's open source framework to handle large data stores. Because Hadoop lacks the native functionality to exploit the location component in big data, GIS Tools for Hadoop was designed to extend the popular data management platform with utilities for spatially operating on billions of records at a time.\r\n\r\nAlthough most data includes spatial parameters, large volumes typically require tedious, sequential processing to do any meaningful work on them. For non-GIS users of Hadoop, the toolkit lets data stewards study big data as a whole unit in Hadoop format. Results from those analyses reveal patterns and relationships that traditionally could only be derived from smaller, more manageable datasets. For ArcGIS users, GIS Tools for Hadoop bypasses traditional geoprocessing workflows by enabling the execution of spatial queries on Hadoop data from ArcGIS.\r\n\r\nThis new ability gives developers and data analysts much-needed control\u2014essentially transforming big data from being something to be dealt with later into an immediately useful resource."},{"acf_fc_layout":"image","image":176621,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"image","image":176631,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"\r\n

Bring Big Data into ArcGIS<\/h3>\r\nAfter conducting spatial analytics on your Hadoop data, the toolkit provides a way of importing big data into the ArcGIS environment. Results from spatial querying and analytics in Hadoop can be moved to ArcGIS for further processing and visualization. Those geoprocessed datasets can then be saved to ArcGIS or exported back into the Hadoop system, thus creating a looping workflow between the ArcGIS platform and the big data environment.\r\n\r\nForget about archiving your large data stores for later use. Command your big data now with GIS Tools for Hadoop.\r\n\r\nThe free, open source toolkit lets you do the following:\r\n