November 13, 2018 |
October 25, 2022
Many students have some experience with robotics, such as flying a drone or taking part in BattleBots competitions, a popular after-school activity. However, when they reach college or grad school, few have used robots to gather the scientific data needed for their research.
That’s the gap Jim Blanchard set out to fill with the Land.Air.Sea Robotics (LASR) program at Johns Hopkins University. Blanchard teaches students how to use robots in geospatial decision-making as part of the university’s Master of Science in Geographic Information Systems program.
In his 40-year career as a scientist, software developer, roboticist, educator, and pilot, a key lesson Blanchard has learned is that each robot should be configured to the mission.
“We’re driven by what it is we want to measure, what instruments have that capacity, and then we build a robot to collect the data we need,” Blanchard said.
For the LASR program, robots act like autonomous surveyors, scanning an area of interest with sensors tuned to find missing data. Students with more advanced skills work from testing centers on the Chesapeake Bay from April through November and then in Florida during the winter months. They deploy watercraft, land robots, and aerial drones to survey nearshore environments most susceptible to climate change. The research has provided a greater understanding of the sea-land interface and the role of vegetation—including seagrass in the Chesapeake and mangroves in Florida—in mitigating sea level rise and filtering pollution.
The students also develop their research agenda, using robots to capture data and geographic information system (GIS) maps to visualize and analyze it. They are answering scientific questions, such as whether a species is present in a particular area, how changing climate conditions impact wildlife, and how to deal with an invasive species.
One student, Caitlin Eversmier, used air and sea robots to collect DNA samples from offshore locations too deep for her to venture into only wearing waders or paddling her kayak. She was looking for a specific fish species in an area where there was runoff from a farm. Imagery from aerial drones helped correlate where to look for the fish species. DNA samples confirmed the presence of the species and farm animals. Using GIS to map the data collected via robotics enabled a spatially accurate sample set and a repeatable experiment that could reliably monitor the area over time.
In an ongoing project, LASR course instructors captured videos of ospreys to understand their behavior during the mating and nesting seasons. Faculty have compiled a record of the feeding habits and the species and size of fish that adult ospreys brought back to the nest. Using GIS tools, the extent of food availability is being analyzed to correlate nesting locations with seasonal fish migrations and spawning in the Chesapeake Bay and Charlotte Harbor, Florida. Researchers say spatially based connections “pop out” when using GIS tools to study environmental data.
Another student, Thomas Blair, used aerial and waterborne robotics in a Virginia lake to assess the presence of snakehead—a large invasive fish that can grow up to three feet long; has snake-like skin; and feeds on frogs, other fish, and small birds. Anglers who catch a snakehead must kill it because of the harm it poses to ecosystems. Snakeheads have no natural predators, evidenced by their growing territory and frequency of catch numbers in Maryland and Virginia. In pursuit of this pest, Blair enlisted robots to collect water samples, including DNA from silt-laden shallows that could not be accessed by wading. He will repeat the survey to build a pattern library for the lake—data that can be analyzed using GIS to fight the invasive species.
Blanchard sees science as a never-ending cycle of data, information, and knowledge. One of the tenets of the LASR program is that students learn to be rigorous and persistent with this cycle.
Though much of the coursework is completed online, students participate in one of six annual field expeditions to gain hands-on experience and troubleshooting skills every scientist must learn.
When it comes to working with robotics, Blanchard notes, operator competence requires training. “These are actual skills—how to calibrate instruments and plan a GPS mission—that you gain when you put instructors and student LASR pilots together.”
Among the tools that the LASR program teaches is Site Scan for ArcGIS, the cloud-based software used to plan air robot missions and process the data collected. The tool stitches images together, extracts key measurements, and streamlines image import to the GIS where the data can be visualized and analyzed.
“Many of my students don’t know how to build a robot, but they don’t have to,” Blanchard said. The larger objective, he explained, “is to teach them how to use robots as a tool for applying the scientific method.”
Blanchard uses the analogy of the layers in a sandwich to teach about data collection related to a three-dimensional space. The bottom layer in this case comes from sea robotics (whether a boat or a submarine) to record surface and underwater measurements. The filling is made up of measurements that the researchers gather with instruments and robots on the ground. Toppings are provided by referencing existing data sources that are geotagged—such as automated sensors, security cameras, buoys, or weather sensors. The top layer is made up of images and readings from aerial drones and satellites.
“GIS ties it all together,” Blanchard said. “In GIS, you can access hundreds of layers of measurements for that time and place that you may want to consider.”
The LASR curriculum looks at sensors and robotics as a way to remove the influence of human variabilities. For example, a researcher wading into the water stirs up sediment. And human eyesight has its limitations, such as being dependent on weather conditions. Robots and sensors add so much more to observations, filling in key gaps in understanding.
Increasingly, the LASR courses rely on artificial intelligence (AI) and machine learning to extract information from large volumes of robot-collected data because it would take too much time for a human to sort through it. AI tools are being used to create point clouds from sonar data for comparative analysis, species identification in imagery, and area calculations of subaquatic vegetation.
While autonomous airborne and waterborne robots have been around for nearly 20 years, it’s still early to test different types of sensors to solve the challenging issues of data processing. Reflection and refraction from water for instance can skew sensor readings and will require more advanced image processing tools to fix.
Conor Church, a graduate of the program and a law enforcement officer, is working on methods to reduce and correct the spectral effects in imagery. His use case is the recording of vehicle accidents to produce an accurate point cloud. One of the more challenging issues he addresses is the reflection of surrounding objects in the car’s paint. It’s not as easy as removing a person or object from the background of a photo on your smartphone, and that’s an important lesson. To make GIS tools useful for the masses, some level of preprocessing of imagery and data is needed.
Blanchard says his program gives students practical experience and teaches skills that are in high demand. The data collected by LASR students can be a force multiplier—a large quantity of data gathered in a short amount of time with consideration for spatial and temporal accuracy. Although many questions can be asked of the data, he says, “the answers may prove elusive without on-the-ground scientific observations to validate results.” For those using robotics in earth observations, the current challenge is in the correlation of LASR analysis with proven scientific and engineering methods.
As the LASR program moves research methods forward, Blanchard hopes he’s demystifying the use of robotics and showing students how technology can be practically applied.
“Robots aren’t the silver bullet that solves everything,” Blanchard said. “It’s just a technique for getting instruments out to where you need them, get the data collected, and then it’s back to good old data analysis.”
Learn more about how GIS and imagery are used in higher education.
November 13, 2018 |
September 13, 2022 | Multiple Authors |
October 11, 2021 |