Tracking locations and populations around the world is becoming increasingly important as whales face harm from ship strikes, entanglement of fishing nets and redistribution of prey due to changes in ocean temperature. To accelerate these efforts, the second year Chinmay Govind and the third year Nihar Ballamudi dedicated the summer Penn's Undergraduate Research Mentoring Program (PURM) A project that combines mathematics, signal processing, animal behavior, and machine learning.
Their goal: harness whale sound data and artificial intelligence to map whale locations and determine how many people live in a particular target area.
In this work, Govind and Ballamudi use National Oceanic and Atmospheric Administration (NOAA) data from a sound receiver north of Cape Cod Bay, but the study applies everywhere.
The outcome of this effort could help the duo “get better data on the number of whales in a whale region or distribution. artificial intelligence and Computer Engineering in School of Engineering and Applied Sciences Originally from Mechanicsburg, Pennsylvania. “The results of our study are not just whales, [also] Other sea animals. ”
Provided by Purm, Undergraduate Research & Fellowship CenterStudents who complete their first or second year with pen with 10 weeks of summer research experience under the supervision of the faculty will be immersed.
Ballamudi and Govind are taught John SpiesburgerVisitors of Department of Global Environmental Sciencealong with her son Ali Spiesberger, a recent Penn alumnus, is an alumnus with expertise in machine learning models. Both students are interested in whale surveillance projects. Joseph Krollprofessor of Bureau of Physics and Astronomy And John Spiesberger's colleagues in 50 years are due to the interdisciplinary, problem-solving nature and the specific impact it has on whale conservation efforts around the world.
“Mathematical studies are not actually used. Mathematics The major University of Arts and Sciences and Computer Science Minor From Madison, Wisconsin. “It's really cool to be able to work on a project. [uses math to] If you can have a census, help influence what the policy looks like. ”
Use sound to find whales
Each student led part of the Purm project. Govind focused on the whale's location, and Ballamudi worked on censoring them. In this context, placement involves tracking and counting individual whales. Meanwhile, census involves approximating the size and distribution of whale populations to more effectively monitor movements.
To find whales, Govind utilized acoustic data from NOAA receivers (essentially underwater microphones) to estimate the origin of whales' call. Each receiver detects sound waves from unique whale call at different times. Govind sends recorded audio data to a machine learning model to estimate the “arrival time difference.” This is used to calculate whale coordinates, similar to how mobile phones use GPS to derive locations.
“Time differences separate the sound into a specific curve,” explains Govind. “If you have more receivers (using five), you'll have enough data to locate the whale and generate confidence intervals about where the whale is.”
However, there are several barriers to finding the time difference in the arrival of whales. Ocean noise distorts receiver data, making it difficult to identify where the whale call starts and stops, or whether multiple whales are in close proximity to produce the sound. There is also “multipath” (a standard sound resonating from the seabed and surface).
“This data can be very messy, so we decided to apply machine learning,” Govind says.
Using AI to optimize and refine the acoustic data, Govind was able to record the origin point of the whale call with a “median error of 20 milliseconds”. This slight uncertainty says it is more than enough to estimate the location of the whale.
Building a census of whales' locations
At the same time, Ballamudi used machine learning models and NOAA sound data to simulate marine environments and census whale populations. This AI-driven approach is more effective than relying on data from physical receivers, taking into account the obstacles brought about by ocean noise and multipathing.
“We sampled the actual ocean noise and generated the signal according to the literature on what whale signals usually look like,” says Balamdi. “We can use that information to generate as much data as we need.”
This strategy allows Govind and Ballamudi to innovate in learning about individual and group whale behavior.
During the PURM project, Ballamudi has accurately predicted the number and distribution of whales between 90 and 95% of the time.
Deep mentorship dive
Spiesberger focuses much of his guidance on showing Govind and Ballamudi “the wonders of what a realistic simulation would look like in this case, and what variations should be placed on the simulation.” He also coached the duo to improve their scientific communication skills, especially given the implications of environmental policies in their work.
“We practice talking to people for a period of time without using terminology,” Spiesberger says.
Ari Spiesberger used AI models as Penn Alum, Training Govind and Ballamudi, to build realistic simulations and pay it beforehand to prime the software to predict unknown variables.
The AI model used in this Purm project also demonstrates the promise of future steps that students can take in this study, continually optimizing and improving accuracy.
As part of Purm's experience, Chinmay Govind and Nihar Ballamudi contributed to the work of Penn's broader research team, responsible for multiple student-visiting academic John Spiesberger, focusing on whale location and investigation. [From left to right: Christian Stuit; Nihar Ballamudi; Katherine Zhang; Sydney Fitzgerald; Justin Duong; Chinmay Govind; John Spiesberger; Mason Liu]
(Image: Courtesy of John Spiesberger)
“It would be great if you could get a model that recognizes multiple sources at the same time and allows you to see them all in one shot,” Govind said.
Once the pair can record the exact number of whales in their target range, Balamdi says that they can use that data to retrospectively identify the exact location of each whale.
“We want to see if this approach works no matter what, not just in well-controlled software, but also in a world where there are far more confounding variables than the software can explain,” Ballamudi says.
The Tangible Power of Purm
Towards the end of Purm's experience, Spiesberger invited Govind and Ballamudi to present the results to US Navy sponsors. It highlights the impact on policymakers working to protect whales. And we share the opportunity to expand this research.
“It's important for the Navy to know where the whale sounds came from, and this Purm project will help solve that problem,” says Spiesberger, who aims to secure grants for Govind and Ballamudi. “We hope they will get funding to support future research.”
