{"id":56352,"date":"2019-02-01T14:25:43","date_gmt":"2019-02-01T22:25:43","guid":{"rendered":"https:\/\/www.esri.com\/about\/newsroom\/?post_type=arcnews&p=56352"},"modified":"2023-07-31T16:20:32","modified_gmt":"2023-07-31T23:20:32","slug":"mixing-the-real-with-the-virtual","status":"publish","type":"arcnews","link":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual","title":{"rendered":"Mixing the Real with the Virtual"},"author":1312,"featured_media":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"sync_status":"","episode_type":"","audio_file":"","podmotor_file_id":"","podmotor_episode_id":"","castos_file_data":"","cover_image":"","cover_image_id":"","duration":"","filesize":"","filesize_raw":"","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":"","_links_to":"","_links_to_target":""},"categories":[57842],"tags":[114612,137792,166952,166942],"arcnews_issues":[32862],"class_list":["post-56352","arcnews","type-arcnews","status-publish","format-standard","hentry","category-utilities-2","tag-augmented-reality","tag-simulation","tag-utility","tag-virtual-reality","arcnews_issues-winter-2018","arcnews_sections-your-work"],"acf":{"short_description":"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.","pdf":{"host_remotely":false,"file":"","file_url":""},"flexible_content":[{"acf_fc_layout":"content","content":"

Out in the Field, New Jersey Utility Uses Mixed Reality to View Simulated GIS Features<\/h2>\r\nIn March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet. Len Bundra, the IT\/GIS director for the Toms River Municipal Utilities Authority (TRMUA) in Toms River, New Jersey, foresaw a day when this technology would allow field-workers to \u201csee through the ground,\u201d as he put it, and view the utilities beneath their feet without having to rely on vehicle-mounted rugged laptops, which display static 2D GIS features. He wrote a post on LinkedIn outlining his idea to use virtual or augmented reality goggles to project features from ArcGIS onto the real world.\r\n\r\nWhile Bundra\u2019s post was viewed several hundred times, it sat on the social media site for almost three years without any responses. But then, in October 2016, he received a message from Alec Pestov, a developer from Toronto, Canada, who shared Bundra\u2019s vision. Pestov, the founder and CEO of Esri startup partner Meemim, Inc., proposed that, together, they build something entirely new\u2014a mixed reality app that would allow utilities (and eventually other organizations) to collaborate with shared content and data in novel ways."},{"acf_fc_layout":"image","image":56362,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"Mixed reality, often described as a combination of virtual reality (VR) and augmented reality (AR), enables users to see virtual objects that are not visible from a real-world perspective. This is akin to a VR experience, though the real world doesn\u2019t get occluded. With mixed reality, users can view and retrieve information about both the visible, real-world objects surrounding them and the virtual objects being projected by headsets. In Bundra\u2019s opinion, mixed reality was the perfect environment for allowing his fieldworkers to see concealed utilities while remaining situationally aware of their actual surroundings.\r\n\r\nBundra and Pestov immediately set out to turn this idea into reality.\r\n

Getting Support for Simulation<\/h2>\r\nBefore the two innovators could get started on the project, Bundra had to get backing from TRMUA\u2019s executive director and Board of Commissioners. Pestov proposed that Meemim fund the up-front programming costs so long as staff at TRMUA made themselves available for technical consultations and field testing during the app\u2019s development phase. TRMUA would be able to defer additional expenditures until Meemim had completed a demonstrable pilot project."},{"acf_fc_layout":"youtube","youtube_video_url":"https:\/\/www.youtube.com\/watch?v=86UQ0i5MPOA"},{"acf_fc_layout":"content","content":"Fortunately, TRMUA\u2019s board had a history of supporting the early adoption of technology that would allow the utility\u2019s employees to be more efficient and, in turn, save its ratepayers money. So after Meemim created a test case and presented the technology to the board, TRMUA received approval to go forward with the project.\r\n\r\nTo get things going, Pestov\u2019s team chose the Microsoft HoloLens as the smart glasses TRMUA would use. The head-mounted holographic computer was the only viable device that met all TRMUA\u2019s functional requirements. Plus, it would be supported by the largest network of hardware and software technicians and developers at Microsoft. Esri\u2019s longstanding partnership with Microsoft factored in, too, since TRMUA has been an enterprise client of both companies for many years.\r\n\r\nMeemim then had to develop a way to digest TRMUA\u2019s ArcGIS features\u2014surveyed coordinates, data, and projections. Although Bundra had 25 years of experience editing, creating, and exporting GIS features to and from conventional formats, exporting holographic projections to mixed-reality hardware was something completely new. Since several of Meemim\u2019s programmers were already experienced in working with Unity, a development platform used to create 3D worlds for many online games, the startup chose that as the simulation engine. Thus, GIS programmers met Unity programmers, and it worked.\r\n

Turning the Virtual into Reality<\/h2>\r\nMeemim began developing the technology in December 2016. Within three months, the company had created its app, called vGIS, which could process ArcGIS feature services, upload the output to Meemim\u2019s Microsoft Azure cloud service, and communicate directly with HoloLens to create the holographic projection of underground utility features from the perspective of the headset wearer\u2019s location and orientation. Both Microsoft and Esri independently confirmed that Meemim and TRMUA were innovating, since they were not aware of any other project in the world that was doing what the two organizations were trying to do."},{"acf_fc_layout":"image","image":56392,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"By May 2017, it was time to test the technology out in the field. Bundra\u2019s main concern was whether the Microsoft HoloLens could accurately project and display TRMUA\u2019s underground GIS features\u2014which were field surveyed and postprocessed to subdecimeter accuracy\u2014so the wearer would see them aligned with the real world.\r\n\r\n\u201cWe are dealing with five or six utility lines underneath a street,\u201d said Bundra. \u201cWe have to locate our utility lines so that when we dig up a street, we are confidently aware of all other utilities around us. Given a more dynamic situation\u2014a car hits a telephone pole, the pole collapses onto the street\u2014seven separate agencies are on scene, each viewing\u00a0[its]<\/em>\u00a0own GIS utilities on separate devices. What I am trying to do is provide all field personnel with one common interface to see all underground utilities beneath their feet\u2014[so]<\/em>\u00a0one shared vision via a shared interface.\u201d\r\n\r\nOn Tuesday, May 2, Bundra set out to take one small step for the future of GIS, going where no other GIS specialist had ever gone. On a quiet street in Toms River, he stood on the sidewalk, adjusted his HoloLens headset, and launched Meemim\u2019s vGIS app.\r\n\r\nUsing Meemim\u2019s patented geocalibration process, Bundra anchored himself to a nearby visible GIS feature (a sewer manhole) and then provided the HoloLens with a precise headset orientation via a simple two-step process. It took less than a minute for the HoloLens to perform the calibration, and afterward, Bundra could see all the water, gas, electric, and sanitary and storm water sewer utility features begin to project themselves around him.\r\n\r\nThen, using a voice control system that Meemim built into its vGIS interface, Bundra was able to give the HoloLens standard mapping commands, such as \u201czoom in,\u201d \u201cpan left,\u201d \u201cselect,\u201d and \u201cclose.\u201d This allowed Bundra\u2014and would enable other fieldworkers\u2014to see and retrieve information about the utilities around him, hands-free. Meemim had color-coded the utility features by utility layer, according to common conventions; electric utilities were red, water utilities were blue, sewer and drain lines were green, and so on. The startup ensured that the holograms displayed pipe sizes according to their actual diameters, so that a pipe with a two-inch diameter would appear smaller than a pipe with an eight-inch diameter. Additionally, each utility line was represented by two lines\u2014a solid one that showed the utility line at its depth and a dashed one that showed where the utility line would be marked out (spray painted) on the road or sidewalk.\r\n\r\nWhenever Bundra moved his eyes, a small white cursor would follow. If he placed his gaze on a GIS feature, he could say, \u201cselect,\u201d and an attribute information window would appear. If he said, \u201cclose,\u201d the window would disappear.\r\n\r\nBundra walked along a water main. He made a turn at the first water service line and ended his mixed-reality tour at a water meter beneath his feet. Everything coincided both visually and virtually to within inches of each other.\r\n\r\nFor the first field test, Bundra could not have expected better results. What he saw only reinforced his initial vision: that mixed reality and GIS technology would soon converge to provide tangible benefits for all utility field operations<\/a>.\r\n

Game-Changing Impact<\/h2>\r\nTRMUA now uses the app in the field. And anyone who sees it quickly realizes that this technology is most powerful when organizations share their data so that everyone can see what is beneath their feet.\r\n\r\nGiven that TRMUA serves 100,000 citizens across multiple jurisdictions and coordinates its operational activities with seven public and private entities, the app is now bringing the utility closer together with neighboring political entities. So far, the app has helped initiate long-overdue conversations regarding data sharing and GIS mapping standards.\r\n\r\nIdeally, mixed reality can bring the same game-changing impact to the GIS community that it has initiated in the media, entertainment, and gaming worlds.\r\n\r\nFor more information about mixed reality, email Bundra at\u00a0lbundra@tomsrivermua.org<\/a>\u00a0or Pestov at\u00a0alec.pestov@meemim.com<\/a>."}],"references":null},"yoast_head":"\nMixing the Real with the VirtualUtilities | Winter 2018 | ArcNews<\/title>\n<meta name=\"description\" content=\"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Mixing the Real with the VirtualUtilities\" \/>\n<meta property=\"og:description\" content=\"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual\" \/>\n<meta property=\"og:site_name\" content=\"Esri\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/esrigis\/\" \/>\n<meta property=\"article:modified_time\" content=\"2023-07-31T23:20:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/2019\/09\/topography-blue-card.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:title\" content=\"Mixing the Real with the VirtualUtilities\" \/>\n<meta name=\"twitter:description\" content=\"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.\" \/>\n<meta name=\"twitter:site\" content=\"@Esri\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\n\t \"@context\": \"https:\/\/schema.org\",\n\t \"@graph\": [\n\t {\n\t \"@type\": \"WebPage\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual\",\n\t \"url\": \"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual\",\n\t \"name\": \"Mixing the Real with the VirtualUtilities | Winter 2018 | ArcNews\",\n\t \"isPartOf\": {\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#website\"\n\t },\n\t \"datePublished\": \"2019-02-01T22:25:43+00:00\",\n\t \"dateModified\": \"2023-07-31T23:20:32+00:00\",\n\t \"description\": \"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.\",\n\t \"breadcrumb\": {\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual#breadcrumb\"\n\t },\n\t \"inLanguage\": \"en-US\",\n\t \"potentialAction\": [\n\t {\n\t \"@type\": \"ReadAction\",\n\t \"target\": [\n\t \"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual\"\n\t ]\n\t }\n\t ]\n\t },\n\t {\n\t \"@type\": \"BreadcrumbList\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual#breadcrumb\",\n\t \"itemListElement\": [\n\t {\n\t \"@type\": \"ListItem\",\n\t \"position\": 1,\n\t \"name\": \"Home\",\n\t \"item\": \"https:\/\/www.esri.com\/about\/newsroom\"\n\t },\n\t {\n\t \"@type\": \"ListItem\",\n\t \"position\": 2,\n\t \"name\": \"ArcNews Articles\",\n\t \"item\": \"https:\/\/www.esri.com\/about\/newsroom\/arcnews\"\n\t },\n\t {\n\t \"@type\": \"ListItem\",\n\t \"position\": 3,\n\t \"name\": \"Mixing the Real with the Virtual\"\n\t }\n\t ]\n\t },\n\t {\n\t \"@type\": \"WebSite\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#website\",\n\t \"url\": \"https:\/\/www.esri.com\/about\/newsroom\/\",\n\t \"name\": \"Esri\",\n\t \"description\": \"Esri Newsroom\",\n\t \"potentialAction\": [\n\t {\n\t \"@type\": \"SearchAction\",\n\t \"target\": {\n\t \"@type\": \"EntryPoint\",\n\t \"urlTemplate\": \"https:\/\/www.esri.com\/about\/newsroom\/?s={search_term_string}\"\n\t },\n\t \"query-input\": {\n\t \"@type\": \"PropertyValueSpecification\",\n\t \"valueRequired\": true,\n\t \"valueName\": \"search_term_string\"\n\t }\n\t }\n\t ],\n\t \"inLanguage\": \"en-US\"\n\t },\n\t {\n\t \"@type\": \"Person\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#\/schema\/person\/07c9626c724f42455e6bbe3102fd1a31\",\n\t \"name\": \"Tony Beshay\",\n\t \"image\": {\n\t \"@type\": \"ImageObject\",\n\t \"inLanguage\": \"en-US\",\n\t \"@id\": \"https:\/\/www.esri.com\/about\/newsroom\/#\/schema\/person\/image\/\",\n\t \"url\": \"https:\/\/secure.gravatar.com\/avatar\/1ed4c97e865db254f2377bf7343801377bdf3935c4b65b54b3944483c5f89b63?s=96&d=blank&r=g\",\n\t \"contentUrl\": \"https:\/\/secure.gravatar.com\/avatar\/1ed4c97e865db254f2377bf7343801377bdf3935c4b65b54b3944483c5f89b63?s=96&d=blank&r=g\",\n\t \"caption\": \"Tony Beshay\"\n\t },\n\t \"url\": \"\"\n\t }\n\t ]\n\t}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Mixing the Real with the VirtualUtilities | Winter 2018 | ArcNews","description":"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual","og_locale":"en_US","og_type":"article","og_title":"Mixing the Real with the VirtualUtilities","og_description":"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.","og_url":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual","og_site_name":"Esri","article_publisher":"https:\/\/www.facebook.com\/esrigis\/","article_modified_time":"2023-07-31T23:20:32+00:00","og_image":[{"url":"https:\/\/www.esri.com\/about\/newsroom\/app\/uploads\/2019\/09\/topography-blue-card.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_title":"Mixing the Real with the VirtualUtilities","twitter_description":"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.","twitter_site":"@Esri","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual","url":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual","name":"Mixing the Real with the VirtualUtilities | Winter 2018 | ArcNews","isPartOf":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/#website"},"datePublished":"2019-02-01T22:25:43+00:00","dateModified":"2023-07-31T23:20:32+00:00","description":"In March 2014, articles about virtual and augmented reality being the next big thing flooded the Internet.","breadcrumb":{"@id":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.esri.com\/about\/newsroom\/arcnews\/mixing-the-real-with-the-virtual#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.esri.com\/about\/newsroom"},{"@type":"ListItem","position":2,"name":"ArcNews Articles","item":"https:\/\/www.esri.com\/about\/newsroom\/arcnews"},{"@type":"ListItem","position":3,"name":"Mixing the Real with the Virtual"}]},{"@type":"WebSite","@id":"https:\/\/www.esri.com\/about\/newsroom\/#website","url":"https:\/\/www.esri.com\/about\/newsroom\/","name":"Esri","description":"Esri Newsroom","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.esri.com\/about\/newsroom\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.esri.com\/about\/newsroom\/#\/schema\/person\/07c9626c724f42455e6bbe3102fd1a31","name":"Tony Beshay","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.esri.com\/about\/newsroom\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/1ed4c97e865db254f2377bf7343801377bdf3935c4b65b54b3944483c5f89b63?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ed4c97e865db254f2377bf7343801377bdf3935c4b65b54b3944483c5f89b63?s=96&d=blank&r=g","caption":"Tony Beshay"},"url":""}]}},"sort_order":"9","_links":{"self":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/arcnews\/56352","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/arcnews"}],"about":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/types\/arcnews"}],"author":[{"embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/users\/1312"}],"version-history":[{"count":0,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/arcnews\/56352\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/media?parent=56352"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/categories?post=56352"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/tags?post=56352"},{"taxonomy":"arcnews_issues","embeddable":true,"href":"https:\/\/www.esri.com\/about\/newsroom\/wp-json\/wp\/v2\/arcnews_issues?post=56352"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}