Improving Trash Monitoring with Drone Imagery, Artificial Intelligence, Mapping

One of California’s premier aquatic and ecosystem science institutes, the San Francisco Estuary Institute (SFEI), worked with partners in the state to develop an automated workflow for validating current trash monitoring methods used by California cities and municipalities.

Improving waste management offers many environmental benefits and is one of the most significant ways to help preserve the planet. Federal and statewide regulations, guidelines, and initiatives actively encourage local agencies to begin reducing the amount of trash and pollutants entering the local environment and waterways.

In 2015, the California State Water Resources Control Board adopted amendments to the Water Quality Control Plan for Ocean Waters of California to Control Trash and Part 1 Trash Provision of the Water Quality Control Plan for Inland Surface Waters, Enclosed Bays, and Estuaries (ISWEBE Plan). The amendments were designed to provide statewide consistency for the water board’s regulatory approach to protecting aquatic life and public health and reducing environmental issues associated with trash in state waters.

City governments and agencies must create ways to monitor trash to measure whether they are reaching the net zero trash generation rate goal. As difficult as this goal is to achieve, progress toward this goal is even harder to measure. Under current regulations, local governments must actively monitor garbage in public areas to ensure that trash-reducing measures are effective.

To validate the current trash monitoring methods in use by California cities and municipalities, SFEI was brought in. The institute’s more than 60 scientists, technologists, and designers strive to improve environmental planning, management, and policy development.

According to Tony Hale, PhD, director of the environmental informatics program at SFEI, “Our organization is designed to produce visionary science that can empower communities to be better stewards over their natural resources. We measure the quality of our work by the decisions that they help to inform.”

To reduce financial costs to municipalities and improve efficiency in the field, the SFEI team set out to implement a new solution to cover survey areas and monitor trash. They used drones to gather imagery from the air and deployed cloud-based drone mapping software to streamline collection and analysis and share results with partnering agencies.

Standardizing Trash Monitoring

With every organization and jurisdiction left to establish its own assessment method, it was difficult to determine whether the trash situation in local communities was improving or worsening and how it compared to other communities, according to Hale.

Trash items detected in a land fill and marked by machine-learning with a turquoise rectangle stating “unknown” with a percentage
The trash items detected by SFEI’s machine learning-based trash detection algorithm are indicated.

The California Ocean Protection Council (OPC) approached SFEI and the Southern California Coastal Water Research Project (SCCWRP) to partner with them to evaluate different trash monitoring methods currently in use across the state and assess their level of accuracy and precision. Under the California Natural Resources Agency, OPC coordinates activities of ocean-related state agencies to improve the effectiveness of state efforts to protect ocean resources. SCCWRP is an aquatic sciences research institute in Southern California.

This collaboration between SFEI, OPC, and SCCWRP sought to develop a trash monitoring playbook that would identify which methods are best suited for monitoring trash and informing local decision-making. Beyond merely monitoring current practices, SFEI and SCCWRP wanted to explore new methods and leverage new technology because other methods previously considered had been inefficient or cost prohibitive, noted Hale. SFEI was specifically interested in developing a novel method using drones and machine learning to create a cost-effective approach to trash surveys.

The traditional method of monitoring trash was time-consuming and could be costly. It involved sending a five-person crew into the field and manually counting each piece of trash. This process could take five hours or more, depending on the site.

Implementing technology without proper planning could also result in significant expense. Manually flying a drone to capture imagery, then downloading the imagery to a local computer to assemble might produce useful results, but it could also incur outsize costs.

According to Pete Kauhanen, GIS manager at SFEI, “If you’re getting your own drone and sensor, trying to integrate it yourself, and then getting your own mission planner, and then the software to process the imagery, and then worrying about where to store all the different images and files…there are a lot of pain points and points where the system can break down. All that can really lead to higher costs and inefficiencies.”

Focusing on Results

SFEI wanted to use drones to fly over survey areas and capture high-quality imagery. Survey areas included public rights-of-way in creeks and rivers and excluded private property. The group also set out to develop a machine learning algorithm to automatically detect trash in the drone-captured imagery.

According to Hale, SFEI wanted to concentrate on the actual products the drones would deliver—imagery, maps, and data—rather than planning, storage, and the mechanics of obtaining those products.

Drone imagery of a road, grass, and dirt, and green markers indicating where trash is located
Captured imagery was annotated to provide training data for SFEI’s machine learning-based trash detection algorithm.

“If it can all just be more or less automatic and handled by another service, then…we could concentrate our attention on what we thought was most critical, which are the maps that the drones generate, the imagery that we could derive from that, and the insights that we could get as far as identifying the trash in the imagery,” said Hale.

The SFEI team chose Site Scan for ArcGIS to help streamline drone flights. Site Scan for ArcGIS is cloud-based drone mapping software designed to transform imagery collection, processing, and analysis.

When he first began exploring Site Scan, Kauhanen was trying to see how he could use it for environmental issues SFEI works on, like wetland ecology. He eventually realized Site Scan would be a great fit for the trash monitoring project because of its ability to manage drone assets and process imagery. SFEI wanted to be lean and not have a lot of overhead to worry about. Kauhanen is the only drone pilot at SFEI, so it made a lot of sense to allow Site Scan to handle the processes. The product support provided would handle any issues.

SFEI created a website, cd3 Contaminant Data Display & Download , that leverages the sampling SFEI did as well as work done by the Bay Area Stormwater Management Agencies Association (BASMAA) and the Southern California Stormwater Monitoring Coalition (SMC). BASMAA is a consortium of nine San Francisco Bay Area municipal stormwater programs and SMC serves a similar collaborative purpose for Southern California public agencies. This website offers a view of all the different sites where trash monitoring was conducted.

When a site displayed on the website is selected, the source of the data is shown as well as the trash level (low or high), the sample date, and whether the site was a targeted (i.e., selected for its likely high trash load) or a probabilistic site (i.e., selected according to randomized sampling design). All data is downloadable. Users can easily explore the map. The map’s symbology indicates the methods SFEI used.

Monitoring More Places, More Frequently

From more automated workflows to increased drone flights in the field, the use of Site Scan has enhanced processes for SFEI. Kauhanen explained that it only takes 10 to 15 minutes and just one or two people to complete a flight. This will let municipalities conduct their own sampling and monitoring in the future.

“This approach using Site Scan allows us to go and monitor many more places and to go out much more frequently so you can get a much better synoptic view of what’s happening in an environment,” said Kauhanen. Instead of visiting a site twice in a season, it is now feasible to visit that site between 10 and 15 times during the same period.

Pete Kauhanen, GIS manager and UAS pilot at SFEI, wearing a bright yellow and orange safety jacket, flying a drone
Pete Kauhanen, GIS manager and UAS pilot at SFEI, gathers UAS training data for trash detection algorithm development. (Photo courtesy of SFEI)

Currently, cities monitor trash by going out to a handful of sites twice a year (at most). This makes it difficult to get an accurate assessment of how much garbage is being generated by a city. Monitoring trash manually in certain locations is completely impractical.

According to Hale, Site Scan allows SFEI to cover a broader geographic range and capture the total view of a survey area. “Not only is it now more efficient—meaning it takes less time—but you’re also able to potentially cover a much broader geography than you could otherwise,” said Hale.

With Site Scan, SFEI can develop a more automated workflow with an integrated solution. Kauhanen said Site Scan was a great option because the software integrates information about the vehicle and sensor into the mission planning, processing, and storage processes, simplifying operations in the field and building an efficient system for data management.

“We’re definitely benefiting…from being able to do volumetric calculations on the fly or being able to now add markers for pieces of trash and being able to export that. Those are all features that are really helpful [to have] in one place. We’re not having to have 5 or 10 different pieces of software to do the work we’re trying to do,” said Kauhanen.

“It’s great that I can go out, basically draw the polygon on the iPad, and hit go. It already lays out the flight plan for [the drone] and…then I can just hit upload to the server. The whole process is automated in a way…where I know it all works together.”

Kauhanen also likes how easy it is to share results with other agencies. With just a hyperlink to a map or summary report, others can view the most current data. “Just sharing a link is really useful versus having to download the imagery and then try and send it as an attachment, which wouldn’t work because it’s too big of a file size,” said Kauhanen. “So, it saves us a lot of time and headaches in terms of being able to use drones as a solution for any of these projects.”

After presenting the results of the trash monitoring project with Site Scan, groups like the Northern California Recycling Association have shown interest in the tool. Hale said people are interested in the ways it can be used for an array of purposes.

“We’re doing more with less. We’re able to have fewer people involved but are being just as effective as if you had someone who programmed it themselves,” said Hale. “We are excited about what we can do in the future.”