Round Rock, Texas, Resolves Critical System Outages with Enterprise Monitoring Solution

A donut shop that sells a two-pound donut; Dell Diamond, which showcases the talents of future baseball stars; and America's largest indoor water park are all features of the vibrant city of Round Rock, Texas. This municipality, located 15 miles north of Austin, is one of the fastest-growing towns in America and one of the safest cities with a population of at least 100,000. The city also has an award-winning park system and school district.

The local government of Round Rock is responsible for providing the essential services that make the city run efficiently and keep residents and businesses moving, including wastewater, transportation, public works, and public safety. There are 15 departments of the city offices, and the geospatial services team in the information technology (IT) department provides geographic information system (GIS) services to internal personnel. This includes data management and quality checks, web application development, cartographic services, and maintenance of the geospatial infrastructure that supports these services.

After experiencing frequent system outages in their ArcGIS Enterprise deployment, the geospatial services team members wanted a solution that would enable them to proactively resolve technical issues. The team members deployed ArcGIS Monitor, a tool designed to monitor the health of an enterprise GIS. This has enabled them to not only resolve outages before help desk tickets are even received, but also provide city employees the reliable data they need to successfully do their job. 
 

The enterprise GIS for the City of Round Rock includes one production and one test environment, each with 12 servers. According to Nathan Smith, IT manager of geospatial services for the City of Round Rock, frequent outages and other technical issues were continually occurring within the system. The help desk system the group had in place received tickets from staff to resolve issues—for example, not being able to access web applications. The geospatial services team would have to conduct some root cause analysis to troubleshoot the issue and determine if it was something at the server or database level. As such, the small team of four wouldn't know if part of the enterprise GIS was down until after the problem occurred, which led the organization to be more reactive than proactive. Smith says, "We weren't thrilled with the downtime because we have high standards here."

In addition, Smith says the team members wanted to quantify their success with resolving issues through key performance indicators (KPIs), so they wanted a solution that would give them metrics on system performance. He says that senior leaders at the City of Round Rock may not be able to dig into the finer details of the department's work, so showing them KPIs that indicate the team's successes is crucial. 

"We weren't really able to quantify any system downtime. It's difficult to portray success when you aren't able to quantify or have metrics," says Smith. "Whether it was human error or an automated task that just broke something, we wanted to get a better handle on that."
 

Smith wanted a new solution to help the geospatial services team better manage and address system outages, and he knew exactly what product could help: ArcGIS Monitor. ArcGIS Monitor is designed to collect, analyze, and optimize the health of enterprise GIS implementations throughout its life cycle. Smith, a former Esri employee in support services, was familiar with Monitor and its capabilities and felt its ability to measure availability and uptime could be valuable.

“At Round Rock, we try to be geospatially forward [with our technology]. When I recognized the problem, I didn't have to consider other options," says Smith. "And because of all those frequent run-ins that we had with servers being down, I wanted to be able to quantify those outages."

Smith says deploying the solution was quick and easy and he referenced available documentation to do a lot of the setup himself while also utilizing the Esri Advantage Program for assistance. Smith worked with the assigned Esri consultant to complete the setup. He recommends the program for anyone who may need technical help or doesn't have the time or resources to deploy software.

Smith and the team now receive reports weekly with different system metrics, like peak-hour transactions, using the Excel Report Task add-in. Smith explains that the reports provide a high-level overview of the metrics as well as the option to get more detail.

"This is our number one reporting tool right now for Monitor: the Excel report add-in. My system administrator will review [reports], and she will notify me of anything that really stands out to her," says Smith.

The geospatial services team members receive alerts from Monitor via an email or a ticket in the help desk system, which allows them to immediately investigate the issue. Smith also checks Monitor throughout the day to ensure that the test and production environments are functioning.

"Availability is important. If I see that we're healthy but like maybe there's one little exclamation point [on the Monitor dashboard], I want to know about that and see why something happened. And then, if we do have any active alerts, I'm checking those consistently," says Smith. 
 

The deployment of ArcGIS Monitor has helped the geospatial services team reduce the number of tickets related to outages and better monitor the health of the entire enterprise GIS, which includes around 280 web services. According to Smith, the process of troubleshooting has been streamlined because Monitor does the root-cause analysis and tells the team immediately what's wrong, eliminating the guesswork. Previously, Smith says he'd have to check multiple things, like every tier of the GIS, to determine the cause of a problem.

"I don't have to investigate and troubleshoot for 15 or 30 minutes anymore, looking at logs, trying to decipher all of those. We are definitely more efficient because of it," says Smith. "Its saved a significant amount of time and money when it comes to troubleshooting."

Monitor also enables the team members to quantify their success through KPIs. Smith says the solution shows their average uptime as far as availability and provides data for many areas such as how busy the services are. The team can also set up counters for performance.

In addition to details on RAM, CPU, and data storage, Monitor offers geospatial context to the team's data by giving metrics like how many transactions have occurred on a specific service or the average response time when a call is made. Smith says the previous system was not able to tell him the "nitty-gritty details at the geospatial level."

"So getting metrics on things that are important to us as providers of geospatial services was definitely taking it to the next level for us," says Smith.

The use of Monitor has increased transparency within the City of Round Rock and enables the geospatial services team to better communicate with upper management. Smith says the metrics displayed in an easy-to-read Monitor dashboard have helped make complex geospatial data easily understandable for other staff.

"If you are in a division of IT as a GIS professional or geospatial [services] professional and you need to be able to communicate on their terms, this tool is essential in helping you do so," says Smith. "You don't have to be a geospatial [services] professional to understand the Monitor dashboard.”

Smith and the geospatial services team created a smart parking application—a mobile app with real-time data feeds—incorporating ArcGIS GeoEvent Server. The app integrates the team's smart parking system with GIS, enabling residents to check parking availability in downtown Round Rock. The app is accessible on mobile devices or a desktop computer. The team members monitor the number and rate of transactions and have built an alert to become immediately aware if they're not receiving enough input and output in GeoEvent Server, because that may mean the solution is down.

"This one's been essential for us in understanding whether or not some of the real-time GIS offerings that we have out there are up and running. I don't think we would be able to be as successful with our availability if we didn't have [Monitor] configured," says Smith.

Another useful Monitor add-on the team has taken advantage of is ArcSOC Optimizer, which helps optimize ArcGIS Server service minimum and maximum instances. Smith explains that there are some services that the city hits regularly, and the team leverages the ArcSOC Optimizer add-on to ensure that those services are performant.

Finally, Monitor has helped deliver an improved user experience for city personnel and residents. For example, if the fire hydrant web service is down when public safety officials are trying to locate a hydrant, a person's life could be at risk. This makes the work the team does more meaningful because supporting these services is crucial to running a city and keeping its residents safe.

"I love ArcGIS Monitor because it allows me to help my end users be more successful. We want to make sure that both the city staff and the citizens have access to those solutions that we provide through the ArcGIS system," says Smith. "Monitor provides us a way of measuring that availability, that uptime, and that success."