Symbiotic wearables sound like a plot from a Marvel movie. But that's exactly what Ling Tan and Despina Papadopoulous aim to achieve in a embodied dating project. We experiment with the interaction of gesture recognition and human data using hand-woven fabrics with conductive threads.
Prototype of small creatures
The fabric, called “The Little Creature,” is connected to an Arduino Nano 33 BLE board running machine learning models that were trained to use. Tensorflow Lite. The Arduino board is a “classic” gesture recognition device with multiple sensors including the APDS9960 for gesture, proximity, light and color detection. It detects gestures and sends signals to heat different parts of the small creature based on the recognized gestures.
The fabric creates a rhythm of heat, ranging from murmurs to a fiery, almost aggressive sensation in the skin. Warmth mimics the sense of touch, making small creatures appear alive.
Essentialized dating is built on previous wearable companions. Style Freea scarf that uses breathing and heating patterns to bring to life as the wearer interacts with it. With the embodied relationship, Tan says he wants to explore the cybernetic relationship between “technology, wearables (medium), and their wearers.” Using a quadratic feedback loop and machine learning, the project aims to “embodied the machine learning process and shed light on the algorithmic black box.”
Tan is also working on it Supergestures, It is Manchester's participatory performance using gesture sensing wearables, and she has identified several challenges in working on a wearable project.
For one, gesture recognition is extremely complicated, and much more complicated than speech recognition. Simple gestures like swipes are easy to identify and I use them every day. However, subtle, slow gestures are difficult to recognize as sensors “listen” to important movements while generating low-intensity signals.
To classify the input, try to match the gesture recognition system with the preset category. Gesture detection systems usually return one of the learned gestures, regardless of the input. However, depending on the context, the lack of a gesture will become the gesture itself. This “negative space” is easily understood by people, but it presents the challenges of gesture recognition algorithms.
Supergestures (📷: Robin Hill)
Tan says the concept of recognizing “nothing” gestures for wearable and body-based work has not been widely explored, like other machine learning fields such as computer vision. She argues that proper error detection and filtering is not enough, and that “nothing” gestures should be considered on a case-by-case basis.
There are other nuances to consider. Although some are universal, gestures tend to differ from culture to culture. Gestures are honest in some cultures and offensive in others. How can I train my machine to understand the context and nuance of gestures in a technical environment where computers cannot reliably identify skin tone?
Essentialized dating is still a work in progress. This demonstrates the possibility of human computer interactions and quantified self-project gesture recognition. Although it is not a true sympathy yet, the little creatures envision a future in which humans and technology interact in dynamic and mutually beneficial ways.
A concrete relationship The Page contains details about the project.
