The U.S. Coast Guard (USCG) icebreaker Healy is on a cross-Arctic voyage this summer, capturing images of the Arctic to advance studies of this rapidly changing region. Lincoln Lab researchers installed a camera system on the Healy while it was docked at the port of Seattle before leaving on July 11 for a three-month science mission. The resulting dataset, one of the first of its kind, will be used to develop artificial intelligence tools that can analyze images of the Arctic.
“Not only will this dataset help seafarers navigate safer and operate more efficiently, but it will also help protect our nation by increasing our awareness of critical maritime areas and our understanding of how AI analytics can be leveraged in this challenging and unique environment,” said Joe Culker, a researcher in the AI Software Architecture and Algorithms Group at the Lincoln Institute who led the project.
As the planet warms and sea ice melts, Arctic shipping lanes are opening up to both warships and illegal fishing vessels. Such a move could pose a national security challenge for the United States. Questions also remain about how the climate, wildlife and geography of the Arctic are changing.
Currently, few Arctic image datasets exist in which these changes can be studied. Overhead imagery from satellites and aircraft provides limited information about the environment. An exterior camera attached to the ship allows you to capture details of the settings and various angles of other ships and other objects in the scene. These types of imagery can be used to train AI computer vision tools, helping the USCG plan naval missions and automate analysis. Kulker said his USCG assets in the Arctic are thinly dispersed and could greatly benefit from AI tools that act as force augmenters.
Healy is the USCG’s largest and most technologically advanced icebreaker. Given its current mission, it was a good candidate for onboard new sensors to collect this dataset. The Institute’s research team worked with the USCG Research and Development Center to determine sensor requirements. Together they developed the Cold Region Imaging Surveillance Platform (CRISP).
“Lincoln Labs has excellent relationships with the Coast Guard, particularly the Research and Development Center. For more than a decade, we have had relationships that enable the deployment of CRISP systems,” said Amna Greaves, CRISP project leader and assistant leader of the AI Software Architecture and Algorithms Group. “We have a strong bond not only because of the USCG veterans working in our laboratories and groups, but also because our technological missions are complementary. Today we deployed infrared sensing in the Arctic, and tomorrow we may operate quadrupedal robotic dogs with fast-response cutters.”
The CRISP system consists of a Teledyne FLIR (Forward Infrared) longwave infrared camera designed for harsh marine environments. The camera is stable in rough seas and can capture images in complete darkness, fog and glare. Combined with a GPS-enabled time-synchronized clock and network video recorder, it records both video and still images along with GPS location data.
The camera is mounted at the front of the ship’s flybridge, and the electronics are housed in a sturdy rack on the bridge. The system can be operated manually from the bridge or set to autonomous surveillance mode, slowly panning back and forth recording 15 minutes of video every 3 hours and one still image every 15 seconds.
“Installing the equipment was a unique and enjoyable experience. As with any good project, our expectations for the installation did not match reality,” says Michael Emily, the project’s IT systems manager, who traveled to Seattle for the installation. The expected access point proved inaccessible, so the research team had to work with the ship’s crew to quickly coordinate the route of the cable from the camera to the observation station. “For this kind of scenario, he had 100 feet of cable for this project, which was nice because we only had a few inches to spare,” says Emily.
The CRISP project team plans to release the dataset, expected to be about 4 terabytes in size, to the public when the USCG’s science mission concludes in the fall.
The purpose of publishing the dataset is to enable the broader research community to develop better tools for those working in the Arctic, especially as navigation in the region becomes easier. “Collecting and publishing data will allow us to make faster and greater progress than we could have achieved on our own,” added Kruker. “It also enables labs to work on more advanced AI applications, and allows other labs to use the datasets to make further progress.”
In addition to providing the dataset, the research team plans to provide a baseline object detection model from which others can develop their own models. More advanced AI applications planned for development are classifiers for specific objects in a scene and the ability to identify and track objects across images.
Paul Metzger, who heads the AI Software Architecture and Algorithms Group, said the project could not only support the USCG’s mission, but could also create impactful datasets for researchers looking to apply AI to Arctic data to help fight climate change.
Metzger added that the group is honored to be part of the project and excited to see the progress that can be made by applying AI to the new challenges facing the United States. “From helping U.S. and European forces in anticipating COVID-19 outbreaks and assisting Ukraine, to adopting AI in the Arctic for maritime awareness, we are very proud that our group is applying AI to our country’s top priorities.”
As datasets become available, they will be available for free download from the Lincoln Laboratory Datasets website.
