Geospatial digital twins (GDTs) are a cutting-edge area of technological innovation at the intersection of 3D modeling and visualization, geospatial analytics, and real-time interaction. GDTs have all kinds of practical applications, including real-time monitoring of climate or weather, city planning, disaster response, and many others. Additionally, the emergence of generative AI is rapidly creating demand for computer programming skills often required to incorporate real-time data streams into GDTs—all of which makes it crucial to incorporate GDTs into GIS education.
At the Rochester Institute of Technology (RIT), the School of Interactive Games and Media (IGM) is one of the first Game Design and Development programs to fundamentally incorporate ArcGIS technology to teach students how to create geospatial digital twins.
“RIT’s School of Interactive Games and Media is at the forefront of advancing practical knowledge in the creation and impact of living geospatial digital twins,” said Rex Hansen, Esri’s principal product manager for ArcGIS Maps SDKs for Native Apps and Game Engines. “By integrating Esri’s industrial-grade GIS with game engines in its curriculum, RIT empowers students to develop immersive simulations grounded in real-world data and analytics.”
Combining geospatial technology and data with video game engines is not a new idea, but it is becoming easier and more useful over time. The ability to visualize geospatial data in game engines can improve engagement, understanding, and decision-making in almost any industry by incorporating real-time data into virtual geospatial worlds.
“We quickly realized the potential that incorporating spatial reasoning and geospatial technology perspectives into game development had for broadening the possibilities of what is possible with game design,” said David Schwartz, IGM’s director. “It also is helping align our school’s focus to contribute to the growing field of digital twins.”
IGM wanted to reimagine how geospatial technology education could better prepare students for the rapidly changing landscape of GIS technology. Students would learn to create GDTs by integrating ArcGIS, computer programming languages such as Python, and generative AI. To that end, IGM redesigned a course that had been taught for several years called Spatial Algorithms and Problem Solving to use GDTs as an overarching learning outcome.
Although students in the class were from science, technology, engineering, and mathematics (STEM) backgrounds, many were unfamiliar with GIS technology or had only seen certain software products, such as ArcGIS Pro. Some students had extensive knowledge of computer programming but not necessarily in Python. Some students had no programming background at all. None of the students had ever taken a class that tackled such a wide range of technologies. Ultimately, they emerged with a better understanding not only of GDTs and other ArcGIS software, but also of how these technologies can affect outcomes beyond the classroom.
Technology Integration to Teach Geospatial Digital Twins
The technologies students were exposed to in the Spatial Algorithms and Problem Solving course ran the gamut from ArcGIS Pro to generative AI programs.
The students used ArcGIS Pro for spatial analytics and data management for a GDT as well as a general introduction to GIS. They were also instructed on the fundamentals of geospatial 3D data, and revisited ArcGIS Pro throughout the course to practice analytical techniques such as reclassifying rasters in the context of flood zones and spatial statistics like hot spot analysis.
ArcGIS Online, however, was primarily used to publish and host geospatial content that could eventually be brought into the ArcGIS Maps SDK for Unity plugin. Early in the class, students were taught about some of the very specific types of online data layers that are used in the ArcGIS Maps SDK for Unity plugin, such as image tile layers and 3D objects scene layers, which were unfamiliar to many of the students.
Students also used ArcGIS CityEngine for rapid procedural modeling and generation of city environments to form 3D models for a geospatial digital twin. ArcGIS CityEngine is a particularly complicated software without a background in 3D modeling and procedural generation. Therefore, the class took full advantage of free ArcGIS CityEngine tutorials available through the software and online. During the semester the course was taught, Hurricane Helene struck Florida. This provided an opportunity for the class to use ArcGIS CityEngine to simulate floodwater rising in Florida. They also implemented some introductory Computer Generated Architecture (CGA) scripts to see how this technology could work to create a GDT that could be used during a hurricane.
The class used ArcGIS Maps SDK for Unity to take advantage of video game engine interaction and modeling capacities. The course has used this plugin for years, and it becomes more accessible and easier to use each year, especially with students who do not have a background in game development. However, it is still a robust game development and game engine software that can be challenging to learn. For this aspect of the course, students used existing Esri tutorials to help document the basic process for getting geospatial content into a game engine environment using the map Creator UI component.
Finally, students were instructed on the basics of Python and ArcPy for large data processing. They were also introduced to ChatGPT and prompt engineering to develop scripts supporting the creation of a GDT. While a GDT can theoretically be created without programming, incorporating real-time data feeds and advanced functionalities necessitates diverse coding approaches, including CGA scripting in ArcGIS CityEngine for procedural generation, Python scripting in ArcGIS Pro for data analytics, and C# scripting for creating Unity game environments.
Lessons Learned
For the most part, students were able to incorporate all these technologies into one workflow to create a GDT for both a midterm assignment and a final project. For the midterm, they were instructed to procedurally generate a 3D model from an area of interest in Florida after Hurricane Helene, publish the CityEngine 3D model to ArcGIS Online as a 3D scene layer, run a storm surge analysis and visualization of the area of interest in ArcGIS Pro, and incorporate the 3D model as 3D scene layers into Unity using the ArcGIS Maps SDK for Unity plugin.
Students also practiced using ChatGPT to extract subsets of extensive traffic volume data from New York City, generate hotspots using Python and ArcPy, and implement their work in the Visual Studio (VS) Code environment. Some students became frustrated when the AI would return results that were not working exactly as expected, and they could not troubleshoot erroneous Python code produced by the AI. Some students even reported getting in “fights” with the AI when it would not yield the desired results.
However, others were successfully able to generate ArcPy Python code that enabled them to complete the workflow. Many students attempted to create CGA scripts using generative AI and were often left frustrated that they could not get ChatGPT’s CGA code to work in CityEngine.
The final project was a testament to the power of integrating these technologies, and of the work GDTs can do outside the classroom. Students were free to develop a GDT project that would incorporate several of the technologies.
Game Design and Development students Luke Lepkowski and Isaac Settle developed a GDT to visualize real-time and historical weather and cloud conditions, making radar and cloud maps more accessible and intuitive. The system updates hourly using Python workflows to process .grib2 radar and cloud data, along with an archive resource for historical data going back to December 2021. The ArcGIS Maps SDK for Unity plugin with capacities of the High Definition Render Pipeline (HDRP) of Unity integrated the processed data, creating an interactive, real-time weather visualization tool.
“By combining Esri’s ArcGIS Maps SDK for Unity’s user-friendly mapping and coordinate tools with Unity’s volumetrics system and VFX Graph for clouds and rain, we could integrate numerous datasets into a more cohesive and immersive digital twin than we ever imagined,” said Lepkowski.
GDTs offer an exciting opportunity to leverage various geospatial technologies to create engaging and transformative educational experiences. As this field evolves, its integration with generative AI and video game engines will play a pivotal role in addressing critical societal challenges, such as disaster response. By combining geospatial technology with game design and development education, we can foster interdisciplinary collaborations that inspire innovation and shape impactful solutions for the future.
About the author
