{"id":3887,"date":"2014-12-15T08:42:00","date_gmt":"2014-12-15T16:42:00","guid":{"rendered":"http:\/\/www.esri.com\/about\/newsroom\/insider\/?p=3887"},"modified":"2014-12-15T08:42:00","modified_gmt":"2014-12-15T16:42:00","slug":"big-data-keep-it-simple-keep-it-small","status":"publish","type":"insider","link":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small","title":{"rendered":"Big Data: Keep it Simple, Keep it Small"},"content":{"rendered":"

Turning big data into knowledge is all about relevance and context<\/em><\/strong>
\nBig data may be all the rage these days, but it isn\u2019t exactly new. In fact, Esri has been dealing with big data<\/a> since the inception of digital mapping more than three decades ago. When every contour, stream, street, rail line, park, building, or shoreline for the entire world is stored in an intelligent database, data doesn\u2019t get much bigger than that.
\n
\nData as Big, Beautiful, and Living as the Earth<\/strong>
\nBack in 1992, Esri embarked on an ambitious campaign to create the very first seamless digital map and database of the whole world. This project\u2014aptly named the Digital Chart of the World<\/a>\u2014converted paper maps of political boundaries, transportation lines, utilities, cultural landmarks, and more into a digital map product that could be viewed for the first time as something other than a pretty picture. In a world where CDs were still considered new and expensive storage media, and hard drives came in hundreds of megabytes, the 1.7 gigabyte database was not only huge, but it also challenged many computer specification and storage architectures.
\n

\"\"
The Digital Chart of the World was created in 1992 and includes 2,094 map tiles representing the entire globe.<\/figcaption><\/figure>\nFast forward to 2010, and Esri technology was once again to organize and produce another tome of topographic wonder\u2014Earth: Platinum Edition<\/em><\/a>, published by Millennium House, measuring six feet tall by four-and-a-half feet wide and weighing in at a staggering 264 pounds\u2014by far the world\u2019s largest atlas. A team of cartographers around the world worked together to create the book, a process that took two years from data collection to final production.
\n
\"\"
Big book. Bigger data. Earth: Platinum Edition, by Millennium House.<\/figcaption><\/figure>\nNow, let\u2019s come to the present and look at the Landsat<\/a> program, a series of earth-observing satellite missions jointly managed by the National Aeronautics and Space Administration (NASA<\/a>) and the United States Geological Survey (USGS<\/a>). The first Landsat satellite launched in 1972, and the latest satellite in the series, Landsat 8<\/a>, provides continuity, as well as improvements, on important global monitoring of our earth. Every day, USGS staff receive and process approximately 450 new Landsat 8 scenes, and these scenes are available for download at no cost within 24 hours of acquisition. The current archive of Landsat scenes now contains more than four million scenes.
\nNow that<\/em> is big data.
\n
\nKeep it Relevant, Keep it in Context<\/strong>
\nEsri makes sense of this massive amount of data\u2014more than eight terabytes, and growing every day\u2014to ArcGIS Online<\/a> users via a collection of scenes. Esri downloads the latest Landsat 8 scenes daily and adds them to a set of image services that contain the best and most recent 50,000 Landsat 8 scenes, which requires about 60 terabytes for storage.
\n
\"\"
Landsat 8, launched in 2013, includes two sensors that that orbit the earth, returning to the same location every 16 days.<\/figcaption><\/figure>\nHow do you manage such a massive amount of data? First, not all scenes are kept in ArcGIS Online; otherwise the data volumes for storage would continually increase. Only the best and most current scenes are kept.
\nThis points out what I think are a couple of fallacies and misconceptions about big data. First is the idea that big data needs to include everything<\/em>. Keeping all<\/em> of those videos, tweets, and page views ignores the importance of relevance and context. We really only need to identify relevant<\/em> data and then apply it contextually<\/em>.
\nSecond, the assumption is that because we have data<\/em> we have information<\/em>. Worse still is the implication that somehow simply having information<\/em> leads to knowledge<\/em>. In fact, we\u2019ve been a little too slow to think about how to apply and use data<\/em> to turn it into explicit knowledge<\/em>.
\n
\"\"
Unstructured data can be a bear to try and understand. Viewing and filtering by location makes it manageable, understandable, and actionable.<\/figcaption><\/figure>\nWhen data can be discovered and applied in context it becomes valuable. Vast data can become small\u2014and therefore relevant and manageable\u2014when it is applied in context across content layers and themes. And coming back to where we started: geography is one of the most natural, logical, and intuitive ways to discover, slice, sort, and ultimately apply big data to solve problems.
\n
\nKeep it Simple, Keep it Small<\/strong>
\nEsri has done a lot of innovation around this idea of keeping big data simple<\/em> and small<\/em>. We provide tools and API<\/a>s to aggregate and accumulate social media, documents, and other unstructured data into collections that are attached to places. These can be easily overlayed together to discover with statistics, demographics, and a host of other data. And this is all available in ArcGIS Online. Remember those Landsat 8 services? Those are here too.
\n
\"\"
Basemaps for displaying and analyzing all sorts of information are available from Esri\u2019s partners through ArcGIS Online.<\/figcaption><\/figure>\nArcGIS Online helps you sort data, apply filters, and explore and ask questions. You can take massive amounts of potential data and convert it into a smaller subset as needed. If that first data combination doesn\u2019t work, you can easily try some other combination. ArcGIS Online provides access to more than 7,000 data variables for hundreds of thousands of places, making possible billions of combinations of searches and trillions of user defined ways to ask about that data\u2014ring, drive distance, drive time, geographies, and more. You can appended this information to your Excel spreadsheets, or output it as a summary file\/report. Esri location analytics<\/a> solutions such as Business Analyst<\/a> and Esri Maps for Office<\/a> do the same thing, but in a familiar business IT setting.
\n
\"\"
Making sense of every customer and potential customer in the United States? That a lot of data, but we\u2019ve got you covered through the Esri Tapestry Segmentation System.<\/figcaption><\/figure>\nNo matter who you are and what your business, Esri\u2019s technology lets you tap into the big data fire hose and drink from it more manageably with a straw. Your limit? Only your imagination.<\/p>\n","protected":false},"author":1,"featured_media":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"sync_status":"","episode_type":"","audio_file":"","podmotor_file_id":"","podmotor_episode_id":"","castos_file_data":"","cover_image":"","cover_image_id":"","duration":"","filesize":"","filesize_raw":"","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":"","footnotes":"","_links_to":"","_links_to_target":""},"categories":[275952],"tags":[871,276092],"class_list":["post-3887","insider","type-insider","status-publish","format-standard","hentry","category-technology","tag-big-data","tag-location-analytics"],"acf":[],"yoast_head":"\nBig Data: Keep it Simple, Keep it Small<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Big Data: Keep it Simple, Keep it Small\" \/>\n<meta property=\"og:description\" content=\"Turning big data into knowledge is all about relevance and context Big data may be all the rage these days, but it isn\u2019t exactly new. In fact, Esri has been dealing with big data since the inception of digital mapping more than three decades ago. When every contour, stream, street, rail line, park, building, or […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small\" \/>\n<meta property=\"og:site_name\" content=\"Esri\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/esrigis\/\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg\" \/><meta property=\"og:image\" content=\"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@Esri\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t \"@context\": \"https:\/\/schema.org\",\n\t \"@graph\": [\n\t {\n\t \"@type\": \"WebPage\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small\",\n\t \"url\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small\",\n\t \"name\": \"Big Data: Keep it Simple, Keep it Small\",\n\t \"isPartOf\": {\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#website\"\n\t },\n\t \"primaryImageOfPage\": {\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#primaryimage\"\n\t },\n\t \"image\": {\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#primaryimage\"\n\t },\n\t \"thumbnailUrl\": \"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2014\/11\/dcw-1024x746.jpg\",\n\t \"datePublished\": \"2014-12-15T16:42:00+00:00\",\n\t \"breadcrumb\": {\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#breadcrumb\"\n\t },\n\t \"inLanguage\": \"en-US\",\n\t \"potentialAction\": [\n\t {\n\t \"@type\": \"ReadAction\",\n\t \"target\": [\n\t \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small\"\n\t ]\n\t }\n\t ]\n\t },\n\t {\n\t \"@type\": \"ImageObject\",\n\t \"inLanguage\": \"en-US\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#primaryimage\",\n\t \"url\": \"\",\n\t \"contentUrl\": \"\"\n\t },\n\t {\n\t \"@type\": \"BreadcrumbList\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#breadcrumb\",\n\t \"itemListElement\": [\n\t {\n\t \"@type\": \"ListItem\",\n\t \"position\": 1,\n\t \"name\": \"Home\",\n\t \"item\": \"https:\/\/www.esri.com\/about\/newsroom\"\n\t },\n\t {\n\t \"@type\": \"ListItem\",\n\t \"position\": 2,\n\t \"name\": \"Big Data: Keep it Simple, Keep it Small\"\n\t }\n\t ]\n\t },\n\t {\n\t \"@type\": \"WebSite\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#website\",\n\t \"url\": \"https:\/\/www.esri.com\/about\/newsroom\/\",\n\t \"name\": \"Esri\",\n\t \"description\": \"Esri Newsroom\",\n\t \"potentialAction\": [\n\t {\n\t \"@type\": \"SearchAction\",\n\t \"target\": {\n\t \"@type\": \"EntryPoint\",\n\t \"urlTemplate\": \"https:\/\/www.esri.com\/about\/newsroom\/?s={search_term_string}\"\n\t },\n\t \"query-input\": {\n\t \"@type\": \"PropertyValueSpecification\",\n\t \"valueRequired\": true,\n\t \"valueName\": \"search_term_string\"\n\t }\n\t }\n\t ],\n\t \"inLanguage\": \"en-US\"\n\t }\n\t ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Big Data: Keep it Simple, Keep it Small","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small","og_locale":"en_US","og_type":"article","og_title":"Big Data: Keep it Simple, Keep it Small","og_description":"Turning big data into knowledge is all about relevance and context Big data may be all the rage these days, but it isn\u2019t exactly new. In fact, Esri has been dealing with big data since the inception of digital mapping more than three decades ago. When every contour, stream, street, rail line, park, building, or […]","og_url":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small","og_site_name":"Esri","article_publisher":"https:\/\/www.facebook.com\/esrigis\/","og_image":[{"url":"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg","type":"","width":"","height":""},{"url":"https:\/\/www.esri.com\/about\/newsroom\/app\/themes\/esri_newsroom\/placeholders\/Newsroom-Keyart-Wide.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_site":"@Esri","twitter_misc":{"Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small","url":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small","name":"Big Data: Keep it Simple, Keep it Small","isPartOf":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#primaryimage"},"image":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#primaryimage"},"thumbnailUrl":"http:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/files\/2014\/11\/dcw-1024x746.jpg","datePublished":"2014-12-15T16:42:00+00:00","breadcrumb":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#primaryimage","url":"","contentUrl":""},{"@type":"BreadcrumbList","@id":"https:\/\/www.esri.com\/about\/newsroom\/insider\/big-data-keep-it-simple-keep-it-small#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.esri.com\/about\/newsroom"},{"@type":"ListItem","position":2,"name":"Big Data: Keep it Simple, Keep it Small"}]},{"@type":"WebSite","@id":"https:\/\/www.esri.com\/about\/newsroom\/#website","url":"https:\/\/www.esri.com\/about\/newsroom\/","name":"Esri","description":"Esri Newsroom","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.esri.com\/about\/newsroom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/insider\/3887","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/insider"}],"about":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/types\/insider"}],"author":[{"embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/users\/1"}],"version-history":[{"count":0,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/insider\/3887\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/media?parent=3887"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/categories?post=3887"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/tags?post=3887"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}