As the development of deep neural networks (DNNs) continues to grow in relevance, Illinois Professor of Computer Science Gagandeep Singh’s research focus is on bridging formal logic, machine learning, and systems research. We are perfectly positioned to provide the next step towards , and proven robust deep learning. ”
Researchers in his field have been grappling with this very premise for years, but have found no solution.
However, Singh’s work is promising, and he received a Google Research Scholar Award for this proposal. As Singh explains, researchers working with industry partners have struggled enough to balance his two of these three trade-offs simultaneously. Taking the next step represents a whole new way of thinking.
In a summary of his proposal, he states, “Training a DNN to achieve all these objectives simultaneously is a difficult optimization problem, one that seeks only to balance trade-offs between accuracy and efficiency, or accuracy and robustness. It cannot be realized by the method of
The answers he seeks to develop were formed out of necessity.
“If you want to build a practical system, especially in safety-critical areas, you have to meet all three priorities at the same time. This is a difficult question because it implies a three-way trade-off between mutually contradictory aspects,” Singh said.
“The way our project views this as an optimization problem is that we have proposed a general framework based on new formulations, concepts, algorithms and architectural designs.”
Singh is also particularly excited that this particular project is tied to Google.
Singh was honored to receive the Research Encouragement Award, highlighting Google’s interest in becoming an industry leader while genuinely supporting and engaging in academic research. This combination fulfills an unmet need for his industry collaborators while further enhancing his own work potential.
The result could be a positive change for many real-time systems, he said.
“For example, systems that operate at the edge, such as robots, drones, and self-driving cars, do not only need good accuracy, they also need good response time and robustness. Hopefully, we can actually have a transformative impact on real-world systems,” Singh said.
The potential results, he said, will build on the research that about half of the accredited automation and learning (FOCAL) labs are already doing on verification technology.
The next steps, while unprecedented, are well-applicable to lab specialties.
As DNNs evolve into increasingly large entities, the problem of energy driving the system grows.
The current “electricity-hungry” model has a significant impact on the carbon footprint. Singh said students are excited about the potential to produce smaller models that meet their needs for accuracy and robustness while adding a whole new level of energy efficiency.
“The question now is, can we do something better with a smaller model? That’s where we’re going,” Singh said. “We hope these students will gain knowledge and expertise in this mindset and provide it to our industry partners to help develop a more sustainable pipeline to further develop these machine learning models. hoping.”
Singh believes that given the project’s potential, its impact extends beyond its designed results.
“So far, I think the relevant research communities have worked somewhat separately on this issue. We realized a trade-off between accuracy and robustness,” Singh says. “I see the value of this project as a new kind of research direction, where these separate ideas are brought together to form a long-sought guarantee of accuracy, robustness and efficiency.” will be
