summary: New research shows that integrating artificial intelligence with advanced proximity and pressure sensors allows commercially available bionic hands to grasp objects in a natural and intuitive way, reducing cognitive effort for amputees. By training an artificial neural network on grip posture, each finger independently “sees” the object and automatically moves to the correct position, improving grip safety and accuracy.
Participants performed everyday tasks such as lifting a cup or picking up small items with significantly reduced mental strain and without extensive training. A shared control system balanced human intent with machine assistance, making the prosthetic arm effortless and lifelike.
important facts
- Natural control: AI-enabled fingers used proximity and pressure sensors for a stable and intuitive grasp.
- Reduced cognitive load: Participants performed the task with less mental effort and greater accuracy.
- Shared autonomy: The system combined user control and AI assistance to avoid collisions and enhance dexterity.
sauce: University of Utah
When you reach for a mug, a pencil, or someone's hand, you don't have to consciously tell where each finger should go to get a proper grip.
Losing that natural ability is one of the many challenges faced by prosthetic hands and people with them. Even with state-of-the-art robotic prostheses, these daily activities involve an additional cognitive burden as users intentionally open and close their fingers around the target.
Researchers at the University of Utah are now using artificial intelligence to solve this problem. By integrating proximity and pressure sensors into a commercially available bionic hand and training an artificial neural network in grasping postures, researchers have developed an autonomous approach that approximates the natural, intuitive way we grasp objects. When working in conjunction with artificial intelligence, study participants demonstrated greater grip safety, greater grip accuracy, and less mental effort.
Importantly, participants were able to perform a variety of everyday tasks, such as picking up small objects and using different grasping techniques, such as raising a cup, all without extensive training or practice.
The study was led by engineering professor Jacob A. George and Marshall Trout, a postdoctoral fellow at Utah's Neurorobotics Institute, and will be published Tuesday in the journal Science. Nature Communications.
“Bionic arms are becoming more realistic, but their control is still not easy or intuitive,” Trout says. “Nearly half of all users abandon their prosthesis, often due to poor control and cognitive burden.”
One problem is that most commercially available bionic arms and hands have no way to reproduce the sense of touch that typically provides an intuitive and reflexive way to grasp objects. But dexterity is not just a matter of sensory feedback. Our brains also have subconscious models that simulate and predict interactions between our hands and objects. Even in a “smart” move, you have to take the time to learn these automatic responses.
Researchers in Utah addressed the first problem by equipping a prosthetic hand made by TASKA Prosthetics with custom fingertips. In addition to pressure detection, these fingertips are equipped with optical proximity sensors designed to reproduce the best touch sensation. For example, a finger might sense that a virtually weightless cotton ball is dropped onto the finger.
For the second problem, we trained an artificial neural network model based on proximity data to naturally move the finger to the exact distance needed to fully grasp the object. Each finger has its own sensor in front of which it can “see”, allowing each finger to work in parallel to grasp any object with complete stability.
But one problem still remained. What if the user didn't intend to grasp the object with such precision? For example, what if they wanted to open their hand and drop the object? To address this final piece of the puzzle, the researchers created a bio-inspired approach that shares control between the user and the AI agent. The success of this approach depends on finding the right balance between human and machine control.
“What we don't want is for the user to fight with the machine for control. In contrast, here the machine made the task easier while improving accuracy for the user,” Trout said. “Essentially, machines have enhanced humans' natural control, allowing them to complete tasks without thinking.”
The researchers also conducted a study with four participants who had amputations between the elbow and wrist. In addition to improving performance on standardized tasks, we also challenged multiple daily activities that require fine motor control. Simple tasks such as drinking water from a plastic cup can be extremely difficult for amputees. If it's too weak, you'll drop it, and if it's too strong, it'll break.
“By adding artificial intelligence, we were able to leave this grasping aspect to the prosthesis itself,” George said. “The end result is more intuitive and more dexterous control, making simple tasks simple again.”
George is the Saltzbacher Chen Endowed Professor in the Department of Electrical and Computer Engineering at the John and Marcia Price Institute of Technology and the Department of Physical Therapy and Rehabilitation at the Spencer Fox Eccles School of Medicine.
This research is part of the Utah Neurorobotics Institute's larger vision to improve the quality of life for amputee patients.
“The research team is also investigating implanted neural interfaces that would allow individuals to control the prosthesis with their minds and receive tactile sensations back from it,” George said. “As a next step, the team plans to fuse these technologies together, allowing enhanced sensors to improve tactile capabilities and intelligent prosthetics to seamlessly blend with thought-based control.”
The study was published online on December 9th. nature communications The paper was titled “Shared human-machine control of an intelligent bionic hand improves grasping power and reduces cognitive burden in radial amputees.”
Co-authors include NeuroRobotics Lab members Fredi Mino, Connor Olsen, and Taylor Hansen, as well as Research Assistant Professor Masaru Teramoto of the School of Medicine's Department of Physical Therapy and Rehabilitation, David Warren, Associate Professor Emeritus of the Department of Biomedical Engineering, and Jacob Segil of the University of Colorado Boulder.
Funding: Funding was provided by the National Institutes of Health and the National Science Foundation.
Answers to key questions:
answer: The system uses proximity and pressure sensors in the fingertips, as well as a trained neural network, to automatically position each finger for a stable and natural grasp.
answer: No, a shared control framework blends human intent with machine assistance, prevents conflicts, and maintains user agency.
answer: Current prostheses require a high degree of cognitive effort. The new AI-driven system restores an intuitive, low-effort grasp similar to natural hand function.
Editorial note:
- This article was edited by the editors of Neuroscience News.
- Journal articles were reviewed in full text.
- Additional context added by staff.
About this neurotechnology and robotics research news
author: Evan Lerner
sauce: University of Utah
contact: Evan Lerner – University of Utah
image: Image credited to Neuroscience News
Original research: Open access.
“Shared human-machine control of an intelligent bionic hand improves grip strength and reduces cognitive burden in radial amputee patients.” Jacob A. George et al. nature communications
abstract
Shared human-machine control of an intelligent bionic hand improves grasping power and reduces cognitive burden for patients with radial amputation.
Although bionic hands can reproduce many movements of the human hand, our ability to intuitively control these bionic hands is limited. Human manual dexterity is due in part to control loops driven by sensory feedback.
Here, we describe how proximity and pressure sensors can be integrated into a commercially available prosthesis to enable autonomous grasping, and we show that continuous sharing of control between an autonomous hand and the user improves dexterity and user experience.
Artificial intelligence moved each finger to the contact point, and the user controlled the grasp using surface electromyography. Inspired by biology, dynamically weighted summation integrated machine and user intent. Shared control improved grip safety, improved grip accuracy, and reduced cognitive load.
The demonstration will involve intact and amputee participants using a modified prosthetic leg to perform real-world tasks with different grip patterns. Therefore, granting some degree of autonomy to the bionic hand provides a translatable and generalizable approach towards more dexterous and intuitive control.
