{"id":495832,"date":"2022-02-22T19:14:07","date_gmt":"2022-02-23T03:14:07","guid":{"rendered":"https:\/\/www.esri.com\/about\/newsroom\/?post_type=arcuser&p=495832"},"modified":"2022-02-22T19:14:07","modified_gmt":"2022-02-23T03:14:07","slug":"building-video-games-with-arcgis-technology","status":"publish","type":"arcuser","link":"https:\/\/www.esri.com\/about\/newsroom\/arcuser\/building-video-games-with-arcgis-technology","title":{"rendered":"Building Video Games with ArcGIS Technology"},"author":1031,"featured_media":0,"menu_order":0,"template":"","format":"standard","meta":{"_acf_changed":false,"sync_status":"","episode_type":"","audio_file":"","podmotor_file_id":"","podmotor_episode_id":"","castos_file_data":"","cover_image":"","cover_image_id":"","duration":"","filesize":"","filesize_raw":"","date_recorded":"","explicit":"","block":"","itunes_episode_number":"","itunes_title":"","itunes_season_number":"","itunes_episode_type":"","_links_to":"","_links_to_target":""},"categories":[471931,25032],"tags":[477752,472461,23492],"arcuser_issues":[477652],"class_list":["post-495832","arcuser","type-arcuser","status-publish","format-standard","hentry","category-arcgis-cityengine","category-education-2","tag-arcgis-maps-sdk","tag-arcuser","tag-geodev","arcuser_issues-winter-2022"],"acf":{"short_description":"Video game developers are incorporating real-world geospatial data into their games more realistic.","pdf":{"host_remotely":false,"file":495852,"file_url":""},"flexible_content":[{"acf_fc_layout":"blockquote","content":"The intersection of ArcGIS, geospatial data, and video games is becoming more blurred as more video games are based on real-life examples. Re-creating an accurate, realistic virtual world in a video game that provides unique game experiences and interactions is a growing field. Even for entirely fictional landscapes, real-world geospatial data incorporated into a game can make the setting realistic."},{"acf_fc_layout":"content","content":"For example, the action-adventure video game Assassin\u2019s Creed Syndicate is set in 1869 London. Game developers at Ubisoft worked tirelessly to create an accurate, one-to-one scale representation of London. Many players have reported that when they visited London for the first time in real life, they had already learned how to navigate the city from their gaming experiences. Although incorporating real-world geospatial data into video games is promising, many challenges still exist.\r\n\r\n "},{"acf_fc_layout":"image","image":495892,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

Project Background<\/h3>\r\nAt the Rochester Institute of Technology (RIT), the authors of this article worked as a team that explored using ArcGIS Desktop, ArcGIS Online, ArcGIS CityEngine, and ArcGIS Maps SDK for Unity to create a serious game about natural disasters. The purpose of the game was to educate people and influence policies that can impact lives during disasters. Previously, RIT created Project Lily Pad and Project EOC, both focused on events related to Hurricane Harvey in 2017, as part of a National Science Foundation (NSF) Research Experiences for Undergraduate Students (REU) project. [Read more about these games in \u201cSerious Game Play<\/a>,\u201d which appeared in the August 2020 issue of ArcWatch.]<\/em>\r\n\r\nWith new advancements in Esri technology that are specifically focused on game development, the team decided to use Esri technology to make a serious game and simulation called Project Lake Ontario. It is related to the flooding of Lake Ontario, which severely impacted several coastal residential neighborhoods in 2017 and 2019. A goal of Project Lake Ontario was to educate the coastal residents about Plan 2014, a water level regulation policy.\r\n\r\nSpecifically, the team wanted to use Esri technology to create a realistic simulation of how water levels would look under various scenarios. From a GIS education perspective, the team also wanted to use Project Lake Ontario to teach game development and design to students as well as Esri technology and broader GIS concepts. This would be accomplished during a 10-week undergraduate student summer research and development experience.\r\n

Video Games That Use GIS and Geospatial Data<\/h3>\r\nThe team started building Project Lake Ontario using ArcGIS Pro. Team members downloaded lidar data and imagery from the Monroe County, New York, website for the project area to establish accurate ground terrain for the game. One of the first challenges was that the lidar detected the trees and water in addition to the terrain and buildings.\r\n\r\nAfter examining the lidar data by using ArcGIS Pro, the team found it had only the ground classified so that information was extracted for the height map of the land. To get the buildings, the trees needed to be removed. To solve this problem, the lidar points were filtered by selecting only single return data points. This was done to isolate the buildings because trees tend to have multiple returned values.\r\n\r\nWith the mostly treeless lidar, it was much easier to begin to extrude and construct buildings in ArcGIS Pro. Although some of the houses that appeared in the lidar imagery did not match the imagery of the surrounding area due to differences in data collection times, the lidar data was chosen for determining which houses to display and how they would appear.\r\n\r\nAfter constructing land and buildings in ArcGIS Pro as accurately as possible, the next challenge was importing that data into Unity, an industry-leading game development environment that would be used to create Project Lake Ontario. The team decided to evaluate two Esri products: the latest beta release of ArcGIS Maps SDK for Unity and then ArcGIS CityEngine.\r\n\r\n "},{"acf_fc_layout":"image","image":495902,"image_position":"center","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

Using ArcGIS Maps SDK for Unity<\/h3>\r\nThe documentation for the latest beta release of ArcGIS Maps SDK for Unity seemed to show it as an out-of-the-box tool that could easily incorporate geospatial data directly into Unity. Objects and layers were first created in ArcGIS Pro, and the layers were then exported to ArcGIS Online as scene layers. The ArcGIS Online scene layer URLs were fed into ArcGIS Maps SDK for Unity for direct incorporation into Unity. The full process involved a lot of smaller steps such as selecting settings and manually exporting\/importing each time data was brought into Unity. After overcoming some initial challenges and gaining more experience with Esri tools, the landscape for the project area was imported into Unity using ArcGIS Maps SDK for Unity.\r\n\r\nWith this initial success came the next set of challenges that had to be dealt with to meet the broader goals of Project Lake Ontario. Creating the game required two things. First, features in the geospatial layers needed to become GameObjects, which are the base class for all entities in Unity scenes and the core building blocks of video games.\r\n\r\nGameObjects need to respond to player interactions. In video games that have buildings, a player can\u2019t (usually) walk through a building. Colliders prevent this from happening. [Colliders define the shape of a GameObject for the purposes of physical collisions or interactions in the game.] For custom game interactions, GameObjects can have C# scripts attached to them.\r\n\r\nSecond, the geospatial layers must exist in Editor mode so they can be seen when building the game versus appearing in Play mode when the game is being run to test it. The team quickly realized that the ArcGIS Online layers incorporated into the game via ArcGIS Maps SDK for Unity appeared only in Play mode and were visible only in the absence of colliders.\r\n\r\nWithout geographic features from ArcGIS Online available in Editor mode, attaching scripts and colliders was much more difficult. It required that the entire workflow from its start in ArcGIS Pro to publishing layers to ArcGIS Online be redone anytime the underlying data was modified (for example, to move a building).\r\n\r\nThe team noticed another serious issue. Specifically, ArcGIS SDK for Unity had its own camera rendering in the back end, which caused conflicts when multiple cameras were implemented in the game scene. Additionally, the ArcGIS camera would exhibit strange movement if another camera was used or moved.\r\n\r\nAfter some initial success and challenges with building Project Lake Ontario using the latest beta release of ArcGIS Maps SDK for Unity, the team decided to explore a different Esri technology: CityEngine.\r\n\r\n "},{"acf_fc_layout":"image","image":495912,"image_position":"left","orientation":"horizontal","hyperlink":""},{"acf_fc_layout":"content","content":"

Using ArcGIS CityEngine<\/h3>\r\nWith ArcGIS CityEngine, the team used the following general workflow. Layers created in ArcGIS Pro were imported into ArcGIS CityEngine. Those layers were exported from ArcGIS CityEngine as Wavefront OBJ files and opened in Unity. This workflow allowed models to be textured in ArcGIS CityEngine and modified easily without continually republishing to ArcGIS Online. In ArcGIS CityEngine, multipatch feature layers could be divided into individual models for items such as houses. These models could then be imported into Unity and modified in the Unity Editor mode.\r\n\r\nOnce the ArcGIS Pro to ArcGIS CityEngine to Unity workflow proved successful, more advanced functionality was added. For example, interactively changing water levels in the game allowed players to add sandbags to simulate flood control. Real-world objects, such as gas tanks, cars, and trees, were added to enhance the game\u2019s realism.\r\n

Reflections and Future Outlook<\/h3>\r\nCreating Project Lake Ontario proved to be a challenging yet rewarding experience. Despite lacking experience with Esri technology, the team created a viable video game that incorporated advanced game design functionality using real-world geospatial data and Esri technology in the short span of 10 weeks.\r\n\r\nIn its current form, the team found that ArcGIS Maps SDK for Unity is a great tool for exploring geospatial layers and rapid prototyping in a game engine tool like Unity. Given that ArcGIS Maps SDK for Unity was still in beta, the team expects many of the challenges faced in developing Project Lake Ontario will soon be overcome.\r\n\r\nAccording to Rex Hansen, the ArcGIS Maps SDK product manager, \u201cArcGIS Maps SDK is designed to provide developers with direct integration of live and local ArcGIS data within a game engine like Unity. As the Maps SDK moves toward its first production release in 2022, we\u2019ll continue to improve integration with game engines to support use in Editor mode; provide a geoplacement framework for more intuitive placement of cameras and game objects; enable ArcGIS data for use in ray casting or as a collision mesh; and, of course, continue to add support for additional ArcGIS data sources.\u201d [A ray casting algorithm uses lines (rays) in virtual space to represent the path of light from an object to an observer.]<\/em>\r\n\r\nProject Lake Ontario also represented an excellent opportunity to build unique game experiences that can be used to educate and inform the public on pressing challenges facing society. For example, the goals of the Project Lake Ontario game are related to understanding water level regulation policy.\r\n\r\nIn addition, there are great opportunities to train and educate the next generation of game development students in the use of real-world geospatial datasets and Esri technology. Ideally, there will be greater integration of game development and the geospatial field as shown in the creation of Project Lake Ontario.\r\n\r\nFor more information, contact\r\nBrian Tomaszewski, PhD\r\nCenter for Geographic Information Science & Technology\r\nSchool of Interactive Games and Media\r\nRochester Institute of Technology\r\nbmtski@rit.edu\r\n

Resources<\/h3>\r\n