Khan Ozbey, along with researchers at New York University’s Tandon School of Engineering’s C2SMART Center and Center for Robotics and Embodied Intelligence, developed an artificial intelligence system to automate traffic safety analysis. The system, called SeeUnsafe, combines verbal reasoning and visual intelligence to identify collisions and near misses within existing traffic video footage. Published in Accident analysis and preventionthis research won New York City’s Vision Zero Research Award and demonstrates new applications of multimodal large-scale language models. Tested on the Toyota Woven Traffic Safety dataset, SeeUnsafe correctly classified traffic events with 76.71% accuracy, providing a proactive approach to identifying and mitigating dangerous road conditions.
AI system identifies traffic accidents and near misses
Researchers at NYU Tandon have developed SeeUnsafe, an AI system that automatically identifies traffic collisions and near misses from existing traffic video footage. The system combines verbal reasoning and visual intelligence and offers a way to improve road safety without requiring major new investments from transit agencies. Winner of New York City’s Vision Zero Research Award, SeeUnsafe leverages pre-trained AI models to analyze long-form videos to proactively identify dangerous intersections and situations.
SeeUnsafe performed well when tested on the Toyota Woven Traffic Safety dataset, correctly classifying videos 76.71% of the time. The system can also identify road users involved in critical events with up to 87.5% accuracy. SeeUnsafe doesn’t just detect accidents, it also generates “road safety reports” that explain decisions that include factors such as weather and traffic, making them understandable to government agencies. why Something happened.
This technology allows for proactive intervention, rather than just reacting after an incident occurs. By analyzing near-misses, such as those with pedestrians, agencies can take preventative measures, such as improving signage and signal timing. The researchers suggest that this approach could also be extended to in-vehicle dash cameras to enable real-time risk assessment of drivers.
SeeUnsafe system outperforms existing models
Researchers at NYU Tandon have developed SeeUnsafe, an AI system that automatically identifies collisions and near misses in traffic videos by combining verbal reasoning and visual intelligence. The system performs better than existing models, accurately classifying videos as collisions, near misses, or normal traffic 76.71% of the time. It can also identify relevant road users with a success rate of up to 87.5%, offering a way to improve road safety without major new investments in resources and infrastructure.
SeeUnsafe leverages pre-trained AI models to understand both images and text and represents a new application of multimodal large-scale language models to long-form traffic videos. Importantly, government agencies do not need to become computer vision experts or collect/label their own data, making it easier to adopt the technology. The system generates a “traffic safety report” containing natural language explanations explaining the factors that lead to accidents, enabling proactive intervention at dangerous intersections.
By analyzing near-miss patterns, SeeUnsafe enables preventive measures such as improving signage and optimizing signal timing. in front Accidents happen. This is in contrast to traditional methods, which address safety issues only after an accident has occurred. The system, tested on the Toyota Woven Traffic Safety dataset, establishes the foundation for using AI to understand road safety situations from extensive traffic footage and has the potential to be extended to vehicle-mounted dash cameras for real-time risk assessment.
Agencies don’t need to be computer vision experts. You can use this technology without having to collect and label your own data to train AI-based video analytics models.
Chen Feng
C2SMART research improves New York City transportation system
C2SMART’s research is improving New York City’s transportation system through a new AI system called SeeUnsafe. The system leverages both verbal reasoning and visual intelligence to automatically identify collisions and near misses in existing traffic video footage. Winner of New York City’s Vision Zero Research Award, SeeUnsafe helps government agencies pinpoint dangerous intersections and road conditions in front There is no need to thoroughly review thousands of hours of footage manually, which can lead to mishaps.
The SeeUnsafe system showed high accuracy in classifying traffic videos, correctly identifying collisions, near misses, or normal traffic 76.71% of the time. It can also identify relevant road users with a success rate of up to 87.5%. By analyzing patterns of near-misses such as unsafe maneuvers, the system enables proactive safety interventions such as improved signage and signal timing, moving from reactive to preventive measures.
This research builds on more C2SMART work, including projects studying the impact of electric trucks, analyzing the effects of speed cameras, developing a “digital twin” to reduce FDNY response times, and monitoring the Brooklyn-Queens Expressway. The system will be more useful to city authorities as it will generate a “road safety report” that explains decisions by taking factors such as weather and traffic into account.
