Many ArcGIS Pro workloads must remain behind corporate or organizational firewalls. This is due to a variety of reasons, but the bottom line is the processing and data must remain on-premises. Azure Stack Hub allows for the deployment of Azure infrastructure within on-prem environments. This blog will answer questions on how well Azure Stack Hub supports ArcGIS Pro.
Courtesy of the Microsoft Azure team, Esri Performance Engineering was granted access to a hosted Azure Stack Hub environment for testing. During this testing cycle we ran a series of tests to determine the performance of the hosted system as it relates to running ArcGIS Pro on VMs. The VMs were configured to use the NVIDIA T4 Graphics cards, which is a known commodity to this team but also to the Esri user community. This is one the GPUs we have recommended for use with ArcGIS Pro for some time now and has been consistently able to deliver a positive ArcGIS Pro user experience, and support 2D, 3D, as well as support compute intensive spatial analysis.
During this testing the goal was to use the test VMs as much as possible to represent a normal day’s work, but also to in turn gauge the performance with metrics but also to gauge the overall user experience. Provided in the text below is some sampled results of the conducted testing.
Beyond hands-on usability or exploratory testing, these VMs were run through a series of tests to capture metrics and information on the overall performance running with both 2D and 3D datasets.
ArcGIS Pro Testing Method
Both hands-on testing, and scripted testing are used to test functionality of ArcGIS Pro. This helps to get quantitative as well as qualitative metrics. Qualitative testing helps us to home in on areas of performance issues and can help establish an understanding of the user experience that can be expected. The quantitative metrics come from the scripted testing such as those listed tests below.
2D Testing – What is the 2D Denver test? This test is a 2D project that contains utility, road, and other data of the city of Denver. This is a very typical dataset that various county and state entities would work with. This dataset has a pretty large amount of data and the test is using bookmarks to travel to different points on the map, forcing ArcGIS Pro to render quickly prior to the next bookmark move. This test puts a workload on the system that stresses CPU more than the GPU.
Denver 2D Performance Test leveraging the CPU.
3D Testing – The 3D Philly test is a lighter weight Multipatch cityscape that takes data from the file geodatabase to extrude 3D wireframes and create a realistic representation of the data and render a city in 3D. This test puts down a significant workload onto the CPU as well as the required GPU. In the Montreal 3D test, we are building to a more challenging and intense dataset like the prior, but build a more intense test using CPU, GPU and Memory. Overall testing is conducting in the same manner as the prior test however this workload doles out a harder hit to the performance.
Separate 3D testing for the cities of Philadelphia and Montreal. GPU usage supports smooth rendering and transitions.
The goal in testing is to identify virtualized systems that will work and support the needs of GIS users. In some cases, we are presented with items we expect to work, and anticipate a solid user experience. With the introduction and use of the Azure NV series we have a known commodity that Esri users have experienced and adopted for their own use. Conveniently using RDP ArcGIS Pro was presented in a VM backed by a NVIDIA T4 GPU. Based on the known aspects of an NV-series instance running with a known and recommended GPU, it seemed to be a good fit and predictable outcome.
Running both 2D and 3D workloads saw solid test results as well as a good user experience. ArcGIS Pro 2.6 in scripted testing ran through navigation, rendering and ran analysis quickly and efficiently. This provided a smooth and enjoyable user experience in qualitative testing, but also yielded strong results in quantitative scripted testing results.
In 2D testing we saw smooth transitions during navigation and basemap rendering. This test taxed the CPU, being a 2D test we do not get the full advantage of the GPU; however, we are able to really test that CPU and how much work it can handle. Although we hit that CPU hard and managed to use the full capacity, the test was not negatively impacted. The rendering was smooth and test results even hit 91 FPS (Frames Per Second) which beat the compared desktop system during this test.
To test the 3D capacity of the instance we used 2 separate tests (Philly and Montreal) to gauge the GPU usage as well as compare the different basemap rendering and features as part of the tests. These tests are similar in nature and how they test ArcGIS Pro. The difference in the tests is how well, or how much impact the GPU makes. Ideally, the GPU positively impacts the look and feel of map transitions and as well as software functionality and stability. The findings of these tests can be visualized using Performance Monitor. Performance Monitor is a basic tool from the Microsoft Windows OS which allow performance tracking of various items which can be configured as needed. This allows the GPU, Memory, or Processor to be tracked with its usage and how the test is impacting the virtual hardware. When we see a good sinewave that is an indicator of a good user experience. Ideally the sinewave shows peaks and valleys of usage of both CPU and the GPU, and the GPU will often have a positive impact on CPU usage with the peak usually not reaching 100% usage.
The tests which include multipatch rendering ran in an acceptable elapsed rendering time, and at 30 FPS or greater on average. Numbers above 30 FPS are an unofficial goal and typically show a solid user experience keeping the overall user experience close to a local desktop performance.
The Azure Stack Hub offers ArcGIS Pro users an infrastructure option that can deliver ArcGIS Pro enabled VMs. Users can expect to have a strong performance, with frame buffer yielding greater than 35 FPS. This virtualized system offers the ability to run 2D and 3D workloads with a reliable and predictable user experience. For advanced GIS users and workloads, the Esri recommendation is to use 6 vCPU (or more) to support GIS analysts running more intense spatial analysis based on the performance results from the conducted tests. It is worth noting that not all users have the same needs and this configuration may be suitable for some, but not necessarily all. However, the overall user experience with the Azure Stack Hub and its VMs supported by the NVIDIA T4 GPUs is a solid fit that can support ArcGIS Pro GIS users and their workloads.