Lightweight Neural Networks Achieve Real-Time, Real-Time Rendering of Weaving Fabrics

Recent advances in the fields of artificial intelligence (AI) and computing have enabled the development of new tools for creating highly realistic media, virtual reality (VR) environments, and video games. Many of these tools are now widely used by graphic designers, animation filmmakers, and video game developers around the world.

One of the trickiest elements to recreate realistically in a virtual or digital environment is fabric. While a variety of computational tools already exist for digitally designing realistic fabric-based items (scarves, blankets, pillows, clothing, etc.), creating and editing lifelike renderings of these fabrics in real time can be challenging.

Researchers from Shandong University and Nanjing University recently presented a new lightweight artificial neural network for real-time rendering of textiles. The proposed network: Special Research Group Conference Paper on Computer Graphics and Interactive Technology '24It works by encoding fabric patterns and parameters as small latent vectors that are later interpreted by a decoder to generate realistic representations of different fabrics.

“Our paper [desire] “Cloth is widely used to improve the realism of real-time virtual worlds, and due to its impact on realism, a way to render real cloth in real time is a must. We found that modern surface-based cloth models are lightweight yet realistic and can serve as a basis for real-time cloth rendering, so we set out to propose a way to make it happen,” Xiang Chen, co-author of the paper, told Tech Xplore.

Textile patterns (i.e. the patterns found in woven fabric formed by woven threads) tend to be regular and repetitive, and Chen and his colleagues set out to harness that repetitive nature to enable realistic rendering in real time.

The algorithm they developed first encodes the regular pattern of a fabric into a small latent vector, which is then run through a small decoder, which interprets it and uses the encoded information to generate a realistic representation of a particular fabric.

“We propose a neural network with an encoder-decoder structure,” Chen explained.

“By encoding the weave material into a latent vector, our network is able to represent multiple weave types after training. Furthermore, we found that the target distribution is complex but separable, so we split it into four simpler parts that can be represented by a lightweight decoder.”

Despite the small size and light weight of the network they developed, they found that it was able to effectively and quickly create realistic replicas of many types of fabrics, and in contrast to other computational methods introduced previously, their network also allows the rendering and subsequent editing of the rendered fabrics to be performed in real time.

In initial tests, Chen and his colleagues showed that their algorithm enables the rendering and editing of textiles at an astounding 60 frames per second on an NVIDIA RTX 3090 graphics card. Remarkably, it also produces high-quality renderings with no visible noise or discernible defects.

In the future, the team's neural network could be integrated into graphic design platforms to help designers improve the realism of the video games and animations they create. Meanwhile, Chen and his colleagues plan to extend the algorithm's capabilities to realistically recreate non-woven fibers as well.

“By introducing a fabric representation model into real-time rendering, our approach could further improve the realism of a variety of real-world applications, such as video games,” Chen added.

“Currently, our method only supports woven fabrics, but we hope to extend it to other fabric types, such as knitted fabrics, in the near future. Furthermore, we plan to explore the representation of more complex fabrics.”

© 2024 Science X Network