{"id":29,"date":"2012-05-07T15:05:00","date_gmt":"2012-05-07T22:05:00","guid":{"rendered":"\/Info\/blogs\/esri-insider\/archive\/2011\/11\/08\/What_2700_s-the-Big-Deal-about-Big-Data.aspx"},"modified":"2012-05-07T15:05:00","modified_gmt":"2012-05-07T22:05:00","slug":"whats-the-big-deal-about-big-data","status":"publish","type":"insider","link":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data","title":{"rendered":"What&#039;s the Big Deal about Big Data?"},"content":{"rendered":"<p>The technology tides have shifted again and, as the notion of cloud computing is becoming mainstream across most industries, a new buzzword is emerging: <strong>Big Data<\/strong>.  Never heard of it? Simply put, the term refers to the ever-growing mountain of data, generated from myriad sources, that organizations must effectively address.<br \/>\n<figure id=\"attachment_619\" aria-describedby=\"caption-attachment-619\" style=\"width: 150px\" class=\"wp-caption alignright\"><a href=\"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2012\/11\/Caricature_cropped1.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-thumbnail wp-image-619 \" src=\"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2012\/11\/Caricature_cropped1-150x150.jpg\" alt=\"Big Data Caricature\" width=\"150\" height=\"150\" \/><\/a><figcaption id=\"caption-attachment-619\" class=\"wp-caption-text\">Courtesy: Keith Mann, Esri<\/figcaption><\/figure>\nFor instance, according to a <a title=\"The Big Data Gap\" href=\"http:\/\/www.meritalk.com\/pdfs\/big-data\/MeriTalk_Big_Data_Gap_Press_Release.pdf\" target=\"_blank\">recent MeriTalk survey,<\/a> 96% of Federal IT professionals expect their agency\u2019s stored data to grow in the next two years by an average of 64 percent.<br \/>\nBig Data is often described using the Three \u201cV\u201ds:  <a title=\"Velocity, Volume, Variety via Gartner\" href=\"http:\/\/www.gartner.com\/it\/page.jsp?id=1731916\" target=\"_blank\">Velocity, Volume, and Variety<\/a>.  By example, let\u2019s take a few of the real world case studies gathered by IBM and provided by Mike Rhodin, Senior Vice President at IBM Software Solutions:<br \/>\n<!--more--><\/p>\n<ul>\n<ul>Utility companies record 350<\/ul>\n<\/ul>\n<p><em>billion<\/em><\/p>\n<ul>\n<ul>meter readings per year (= Volume); The financial service industry clocks 5,000,000 trade events<\/ul>\n<\/ul>\n<p><em>per<\/em><em>second<\/em><\/p>\n<ul>\n<ul>(= Velocity); and, as we know, the types of data formats that can be generated easily range from structured traditional file formats, to unstructured video, audio, imagery, email, web logs, and pretty much<\/ul>\n<\/ul>\n<p><em>anything you can think of<\/em><\/p>\n<ul>(= Variety).<\/ul>\n<p>As you might expect, the typical C-level individual is not overly concerned with the definition of Big Data.  What really interests her is finding ways to take advantage of Big Data in order to drive better business outcomes, according to Gartner.  So how does one extract value out of data that is terrabytes large (or larger), at-scale, both on a geographical and also a practical level?   How do you store all this data?  And how do you achieve results that are meaningful to your organization and your customers?  These are a few of the challenges of Big Data.<br \/>\nThe good news is that there are a growing number of technologies that allow individuals to store and conduct analyses on the 3V\u2019s of Big Data.  For example, MapReduce is the original set of distributed computing ideas now embodied today as Apache Hadoop.  Other big data-related technologies include Apache Cassandra, Hive, NoSQL, and MongoDB, just to name a few.   Also emerging are applications or methodologies used to perform data mining and analyses on Big Data via pleasing dashboards and an intuitive user experience (UX).<br \/>\n<figure id=\"attachment_568\" aria-describedby=\"caption-attachment-568\" style=\"width: 300px\" class=\"wp-caption alignright\"><a href=\"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2011\/11\/CloudTrigger_MarketingDashboard.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium wp-image-568  \" src=\"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2011\/11\/CloudTrigger_MarketingDashboard-300x188.png\" alt=\"\" width=\"300\" height=\"188\" \/><\/a><figcaption id=\"caption-attachment-568\" class=\"wp-caption-text\">Marketing Workspace Dashboard, CloudTrigger.com<\/figcaption><\/figure>\nA residual effect of the growth of these offerings is the increased demand for workers with blended skill sets to fill the role of a <a title=\"Data Scientist\" href=\"http:\/\/tech.fortune.cnn.com\/2011\/09\/06\/data-scientist-the-hot-new-gig-in-tech\/\" target=\"_blank\">Data Scientist<\/a>: a half-research scientist, half-data analyst.<br \/>\nEnter: Serious business analytics.<br \/>\nTo put the potential for Big Data into perspective, in 2011, <a title=\"5 Real-World Uses of Big Data\" href=\"http:\/\/gigaom.com\/cloud\/5-real-world-uses-of-big-data\/\" target=\"_blank\">GigaOm shared a few interesting examples<\/a> of real-world situations where Big Data problems were solved:<\/p>\n<div>\n<ul>\n<li>A New York University PhD student conducted a comprehensive analysis of several terabytes worth of Wikileaks data to determine key trends around U.S. and coalition troop activity in Afghanistan.<\/li>\n<li>A global non-profit analyzed 80 Million documents to confirm validity of the Guatemalan genocide of the 1990&#8217;s<\/li>\n<li>A California genomics company consumed over 100 Million gene samples to predict markers for coronary artery disease.<\/li>\n<\/ul>\n<p>It\u2019s not surprising and quite a natural progression that the discussion of Big Data arrives on the heels of cloud computing.  The cloud allows organizations and agencies the ability to store a tremendous amount of data in a [hopefully] highly reliable system, in a distributed environment.  Cloud provides the ability to scale dynamically, leverage existing algorithms for analyses, and take advantage of robust data center hardware, cost effectively, without building from the ground-up.<br \/>\nEsri has been testing Amazon Web Services\u2019 Elastic MapReduce product and deploying prototypes on the AWS cloud, as well as exploring and providing<a title=\"MongoDB code example\" href=\"http:\/\/blogs.esri.com\/esri\/arcgis\/2012\/05\/02\/mongodb-example-code-for-adding-a-nosql-plug-in-data-source\/\" target=\"_blank\"> MongoDB examples to plug-in NoSQL<\/a> data sources to ArcGIS.  More visibly is Esri\u2019s geospatial analysis of tweets generated from Twitter and collected through big data partner, <a title=\"Gnip\" href=\"http:\/\/www.gnip.com\" target=\"_blank\">Gnip<\/a>. You can see examples of social media monitoring via the <a title=\"Twitter and Esri\" href=\"https:\/\/dev.twitter.com\/case-studies\/esri-enriches-maps-tweets-and-streaming-api\" target=\"_blank\">public information maps<\/a>, where tweets are captured then displayed across relevant geographies.  Other Esri partners in the big data space currently are Microsoft, IBM, <a title=\"TerraEchos\" href=\"http:\/\/www.esri.com\/industries\/defense\/community\/partner-showcase.html\" target=\"_blank\">TerraEchos<\/a>, and <a title=\"CloudTrigger\" href=\"http:\/\/www.cloudtrigger.com\" target=\"_blank\">CloudTrigger<\/a>.<br \/>\nEvery organization sees their data as core assets that drive business and decision-making. Mining location data from these assets and making sense of them is perhaps one of the biggest challenges we face with Big Data.  Typically this information is randomly collected and then locked away.  Access is limited or the data is archived and forgotten.  A more democratic platform, such as <a title=\"ArcGIS Online\" href=\"http:\/\/www.arcgis.com\" target=\"_blank\">ArcGIS Online<\/a>, can be used to greatly increase the speed of understanding and sharing of location data assets.  As a result, individual users are empowered with the information they need to make the most effective and innovative decisions, affecting the future of <a title=\"Big Data in Government\" href=\"http:\/\/www.fedtechmagazine.com\/article\/2012\/04\/big-data-big-deal-government\" target=\"_blank\">government<\/a> and society, <a title=\"Why Science Needs Big Data\" href=\"http:\/\/news.cnet.com\/8301-11386_3-57406484-76\/why-science-really-needs-big-data\/\" target=\"_blank\">science<\/a> and business.<br \/>\n<strong>That <\/strong>is the big deal about Big Data.<br \/>\nRelated Links:<\/p>\n<ul>\n<li><a title=\"Hurricane Irene &amp; Big Data\" href=\"http:\/\/ctovision.com\/2011\/08\/hurricane-irene-gis-social-media-and-big-data-shine\/\" target=\"_blank\">Hurricane Irene: GIS, Social Media, and Big Data Shine<\/a><\/li>\n<li><a title=\"White House Big Data Announcement\" href=\"http:\/\/www.whitehouse.gov\/blog\/2012\/03\/29\/big-data-big-deal\" target=\"_blank\">White House Announces $200M Big Data R&amp;D Initiative<\/a><\/li>\n<\/ul>\n<\/div>\n","protected":false},"author":1,"featured_media":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"sync_status":"","episode_type":"","audio_file":"","transcript_file":"","podmotor_file_id":"","podmotor_episode_id":"","castos_file_data":"","cover_image":"","cover_image_id":"","duration":"","filesize":"","filesize_raw":"","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[275952],"tags":[871,170502,157412,276682,209772,276692,276702,276712],"class_list":["post-29","insider","type-insider","status-publish","format-standard","hentry","category-technology","tag-big-data","tag-cloud","tag-data","tag-data-scientist","tag-esri","tag-hadoop","tag-ibm","tag-mongodb"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.9 (Yoast SEO v25.9) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>What&#039;s the Big Deal about Big Data?<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What&#039;s the Big Deal about Big Data?\" \/>\n<meta property=\"og:description\" content=\"The technology tides have shifted again and, as the notion of cloud computing is becoming mainstream across most industries, a new buzzword is emerging: Big Data. Never heard of it? Simply put, the term refers to the ever-growing mountain of data, generated from myriad sources, that organizations must effectively address. For instance, according to a [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data\" \/>\n<meta property=\"og:site_name\" content=\"Esri\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/esrigis\/\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg\" \/><meta property=\"og:image\" content=\"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@Esri\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t    \"@context\": \"https:\/\/schema.org\",\n\t    \"@graph\": [\n\t        {\n\t            \"@type\": \"WebPage\",\n\t            \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data\",\n\t            \"url\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data\",\n\t            \"name\": \"What&#039;s the Big Deal about Big Data?\",\n\t            \"isPartOf\": {\n\t                \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#website\"\n\t            },\n\t            \"primaryImageOfPage\": {\n\t                \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#primaryimage\"\n\t            },\n\t            \"image\": {\n\t                \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#primaryimage\"\n\t            },\n\t            \"thumbnailUrl\": \"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2012\/11\/Caricature_cropped1-150x150.jpg\",\n\t            \"datePublished\": \"2012-05-07T22:05:00+00:00\",\n\t            \"breadcrumb\": {\n\t                \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#breadcrumb\"\n\t            },\n\t            \"inLanguage\": \"en-US\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"ReadAction\",\n\t                    \"target\": [\n\t                        \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data\"\n\t                    ]\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"ImageObject\",\n\t            \"inLanguage\": \"en-US\",\n\t            \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#primaryimage\",\n\t            \"url\": \"\",\n\t            \"contentUrl\": \"\"\n\t        },\n\t        {\n\t            \"@type\": \"BreadcrumbList\",\n\t            \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#breadcrumb\",\n\t            \"itemListElement\": [\n\t                {\n\t                    \"@type\": \"ListItem\",\n\t                    \"position\": 1,\n\t                    \"name\": \"Home\",\n\t                    \"item\": \"https:\/\/www.esri.com\/about\/newsroom\"\n\t                },\n\t                {\n\t                    \"@type\": \"ListItem\",\n\t                    \"position\": 2,\n\t                    \"name\": \"What&#039;s the Big Deal about Big Data?\"\n\t                }\n\t            ]\n\t        },\n\t        {\n\t            \"@type\": \"WebSite\",\n\t            \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#website\",\n\t            \"url\": \"https:\/\/www.esri.com\/about\/newsroom\/\",\n\t            \"name\": \"Esri\",\n\t            \"description\": \"Esri Newsroom\",\n\t            \"potentialAction\": [\n\t                {\n\t                    \"@type\": \"SearchAction\",\n\t                    \"target\": {\n\t                        \"@type\": \"EntryPoint\",\n\t                        \"urlTemplate\": \"https:\/\/www.esri.com\/about\/newsroom\/?s={search_term_string}\"\n\t                    },\n\t                    \"query-input\": {\n\t                        \"@type\": \"PropertyValueSpecification\",\n\t                        \"valueRequired\": true,\n\t                        \"valueName\": \"search_term_string\"\n\t                    }\n\t                }\n\t            ],\n\t            \"inLanguage\": \"en-US\"\n\t        }\n\t    ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"What&#039;s the Big Deal about Big Data?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data","og_locale":"en_US","og_type":"article","og_title":"What&#039;s the Big Deal about Big Data?","og_description":"The technology tides have shifted again and, as the notion of cloud computing is becoming mainstream across most industries, a new buzzword is emerging: Big Data. Never heard of it? Simply put, the term refers to the ever-growing mountain of data, generated from myriad sources, that organizations must effectively address. For instance, according to a [&hellip;]","og_url":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data","og_site_name":"Esri","article_publisher":"https:\/\/www.facebook.com\/esrigis\/","og_image":[{"url":"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg","type":"","width":"","height":""},{"url":"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_site":"@Esri","twitter_misc":{"Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data","url":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data","name":"What&#039;s the Big Deal about Big Data?","isPartOf":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#primaryimage"},"image":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#primaryimage"},"thumbnailUrl":"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2012\/11\/Caricature_cropped1-150x150.jpg","datePublished":"2012-05-07T22:05:00+00:00","breadcrumb":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#primaryimage","url":"","contentUrl":""},{"@type":"BreadcrumbList","@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/whats-the-big-deal-about-big-data#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.esri.com\/about\/newsroom"},{"@type":"ListItem","position":2,"name":"What&#039;s the Big Deal about Big Data?"}]},{"@type":"WebSite","@id":"https:\/\/www.esri.com\/about\/newsroom\/#website","url":"https:\/\/www.esri.com\/about\/newsroom\/","name":"Esri","description":"Esri Newsroom","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.esri.com\/about\/newsroom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/insider\/29","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/insider"}],"about":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/types\/insider"}],"author":[{"embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":0,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/insider\/29\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/media?parent=29"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/categories?post=29"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/tags?post=29"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}