Houston Public Works Manages Enterprise GIS with Performance and Optimization Solution

Houston, Texas, is the fourth-largest city in America with a diverse population of more than 2.3 million residents that enjoy unique attractions such as the Space Center. To ensure this expansive and populous city keeps running efficiently and residents can enjoy a high quality of life, Houston Public Works offers services ranging from producing and distributing water, collecting and treating wastewater, and permitting and regulation of construction. More than 4,000 dedicated employees help to cover 671 miles of service area.

The geospatial services team at Houston Public Works is responsible for providing geographic information system (GIS) operations support and guidance and maintaining the GIS IT infrastructure that's needed to support its five business units and its daily operations. Within the business units, GIS subject matter experts are responsible for editing and maintaining their core data, while some also produce solutions such as web maps and dashboards. The team also supports the mayor's office, the public works director's office, and emergency operations, with various applications that need to be available at all hours. 

With a complicated infrastructure that includes many servers, dedicated environments for user groups, and a growing number of users and transactions, geospatial services team members wanted a solution that would help them monitor their enterprise system to proactively address any issues and improve reporting. The team partnered with GIS services provider Dymaptic, a longtime collaborator and an Esri partner, to implement ArcGIS Monitor to oversee the health of its enterprise GIS. The use of this solution has improved efficiency and troubleshooting to enable the team to better support Houston Public Works operations and services.

The large IT infrastructure of Houston Public Works is composed of an enterprise portal and a separate system of two GIS servers that includes a federated portal of three servers and regular map and feature services. According to Anthony Powell, GIS manager for Houston Public Works, the enterprise GIS environment has grown significantly in the past five years with the addition of more critical servers to support the department's efforts.

Houston Public Works staff edit data and build web apps to consume web services as well as enable the public to access some of their data and apps. Powell notes that before he took on the GIS manager role in 2015, the GIS was in a very bad state due to how things were structured and the lack of efficient organization in the environment. Users across the department complained that they were unable to access data, and processes like saving their data were very slow.

"When I started looking at the environment, everything was competing against each other. We had GIS data there, but you also had data from these other systems. So, there were no optimizations," explains Powell. "From an editor versus just someone that was needing to access and read the data, they're all coming from the same environment and competing for resources . . . that were not balanced properly."

The previous enterprise GIS setup also created difficulties for Jane Chen, enterprise geodatabase administrator at Houston Public Works, when trying to monitor and keep up with changes to the database. For example, the department's stormwater team manages a large network of pipes, inlets, etc., to handle stormwater runoff. Powell says editors on this team make edits daily, so there are a large number of transactions to monitor. 

The geospatial services team has a replica database for internal use and a sharing database for production applications. Powell notes that as the department expanded, Chen continued to build replicas, which meant more data was being added into the environment for users to edit.

"Each year, more data is being added, [and] more users are being added to maintain that data. So that just increased the complexity of Chen's role and her responsibility as a database administrator. So then we started saying we need some tools to help her monitor the performance of those activities," says Powell.

When Powell began his work, he wanted to start with a clean slate for the new enterprise GIS environment and decided to move away from an Oracle on-premises hosted environment to an SQL AWS cloud-hosted environment. Chen and Powell created separate environments for users, including a dedicated editing database environment for the editors, one for viewing, and one for publishing. However, they needed a solution to monitor and optimize each system. 

The City of Houston has an existing enterprise agreement with Esri, so Powell began searching for available Esri tools to do the job, which led him to select ArcGIS Monitor. ArcGIS Monitor is a tool designed to monitor system health and provide awareness of system usage and performance. To support the implementation of this solution, Powell decided to bring in Dymaptic and its expert knowledge of implementing ArcGIS Monitor. In addition, Dymaptic's support role was expanded across Houston Public Works with everything from application development to server management and database administration. 

Dymaptic team members, who frequently use Monitor with their own clients, like the fact that this solution is part of the Esri portfolio. Mara Stoica, CEO of Dymaptic, says because the solution is made by Esri, it's assured that it will keep up with Esri technology. 

"If Esri comes up with a new product, the Monitor team will make sure that the new product is being monitored also. Other third-party applications have a harder time keeping up with Esri's invention of new technology," says Stoica.

Kevin Sadrak, a senior GIS developer at Dymaptic, adds, "Monitor has got the support and the usability that makes it easy for us to help a customer to get it [set up and] working." 

After the initial setup, Powell says Monitor was initially connected to a few of the department's emergency operation solutions for monitoring, but the GIS team had some challenges with implementation and later brought in Dymaptic to help. The Dymaptic team began by talking with Powell and Chen about the state of their databases and how they are running and what they'd like to improve. Sadrak said it was important to get a holistic view of the enterprise GIS system. 

"When Kevin came on board, he was able to open up the hood and identify where the disconnects possibly were and to get things configured to where we could then start fully taking advantage of [Monitor]," says Powell. "He got some readings on performance that have helped [Chen] to tweak and make adjustments to her processes."

Monitoring capabilities were established for the enterprise public works environment, including reports to monitor performance. Sadrak says they got the reports running out of the box with only slight adjustments to refine them. He used the Monitor API to customize the alerts, so Powell and the team received only the critical alerts they wanted to see. In addition to email notifications, alerts are added directly to a Microsoft Teams channel.

The deployment of ArcGIS Monitor has resulted in improvements in performance and the overall user experience, enhanced troubleshooting, and a fourfold increase in usage. Sadrak says the user experience has improved because ticket requests have decreased, and users now benefit from a generally improved system with faster performance and fewer hiccups.

Chen echoes this sentiment and says after Monitor was fully operational, it provided a wealth of data. At the beginning of the day, Chen checks how many transaction counts and failed data counts there are. She also checks to see which server has services that may have stopped.

"I really like Monitor, so I start to use it frequently. It's a mirror for me each morning [when I check it]," says Chen. "I don't want to sound like ArcGIS Monitor is just for our database. It's not. It's for our whole GIS system. Now, we can see the whole database server, performance, status, activities, and have very detailed information."

Powell believes Monitor has enabled geospatial services team members to be proactive because they are leveraging it to inform them of critical issues ahead of time. Sadrak agrees and notes that the team can look at past reports to identify trends such as when usage is going up or disk space is starting to climb.

"To me, having that historical record is what's most important because that's where you can start establishing, 'Okay, we've got X usage. But that's normal.' We're able to see what is really happening easily," says Sadrak.

He adds that they can do maintenance tasks more proficiently because they can spend less time troubleshooting with reports.

"It's helping us work more efficiently, and we're able to discover these things so that our response time to issues actually shortens, which is a good thing overall for ourselves and for the customers that are having a problem," says Sadrak.

The improved troubleshooting has helped them better determine the source of performance issues. Sadrak says the challenge before was diagnosing the more than 100 services at Houston Public Works, but with ArcGIS Monitor, they can get information and more easily determine what the problems are. Now, when Monitor issues a performance report, there is enough information to make decisions quickly.

"When you're just hunting for a solution because someone said it was slow, you're kind of feeling around in the dark. Is it the database? Is it the RAM? Some tools can help you with that, but what Monitor does is kind of put it in one place," says Christopher Moravec, CTO for Dymaptic.

For example, with the rollout of new services recently, the system was low on resources, but with the reports from Monitor, they had the data displayed in visuals like charts. These visuals helped justify the requests for additional resources to support the GIS team's activities.

"The data was authoritative, it was right out of the system, and we had that factual information. So it helped support the cause," says Sadrak. "We were able to make that chart and say, 'Look at where we were. This was our usage. And we need you to give us some resources.' It made that conversation with IT and management easy."

The ability to customize components for Monitor and add extensions allowed Dymaptic to provide more capabilities to the Houston Public Works team. Sadrak says he was able to add an extension to watch scheduled tasks such as database loading.

The Houston Public Works geospatial services team has started migrating the utilities, water, wastewater, and storm networks into the new utility network model environment, which will require the use of Monitor. Powell says it will be a big shift from the department's traditional way of managing the software development environment, but with Monitor, he says team members can easily plug it in and apply it to this new environment.

"I feel ArcGIS Monitor can help us [go] from being reactive . . . to [being] proactive. That's a big benefit. We don't want to wait until users are [reporting an issue] to try to find it," says Chen. "We monitor with Monitor frequently, and we find the issues and fix them, hopefully before users find them. That's great."