- GS-LRM: Large-scale reconstruction model for 3D Gaussian splatting
Authors: Kai Zhang, Sai Bi, Hao Tan, Yuanbo Xiangli, Nanxuan Zhao, Kalyan Sunkavalli, Zexiang Xu
Abstract: We propose GS-LRM, a scalable large-scale reconstruction model that can predict high-quality 3D Gaussian primitives from 2-4 pose sparse images in 0.23 seconds on a single A100 GPU. Masu. Our model features a very simple transformer-based architecture. We patch the input pose image, pass the concatenated multi-view image tokens to a set of transformer blocks, and directly decode the final per-pixel Gaussian parameters from these tokens for differentiable rendering. In contrast to previous LRMs that can only reconstruct objects by predicting a pixel-by-pixel Gaussian distribution, GS-LRM naturally handles scenes that vary widely in scale and complexity. We show that it works for both object and scene capture by training the model on Objaverse and RealEstate10K, respectively. In both scenarios, the model significantly outperforms the state-of-the-art baseline. We also demonstrate the application of the model in downstream 3D generation tasks. Our project web page is available at https://sai-bi.github.io/project/gs-lrm/.
2. SAGS: Structure-Aware 3D Gaussian Splatting
Authors: Evangelos Berberas, Rolandos Alexandros Potamias, Jifei Song, Jiangkang Deng, Stefanos Zafeiliu
Abstract: Following the advent of NeRF, 3D Gaussian splatting (3D-GS) has paved the way to real-time neural rendering that overcomes the computational burden of volumetric methods. Following the pioneering efforts of 3D-GS, several methods have been attempted to achieve compressible, high-fidelity performance alternatives. However, by adopting geometry-independent optimization schemes, these methods ignore the inherent 3D structure of the scene, which limits their expressive power and representation quality, resulting in various floating point and artifacts. It happens. In this study, we propose a structure-aware Gaussian splatting method (SAGS) that implicitly encodes the scene geometry. This is reflected in state-of-the-art rendering performance and reduced storage requirements on the benchmark Novelview synthetic dataset. SAGS is based on a local/global graph representation that facilitates the learning of complex scenes and enforces meaningful point displacements that preserve the scene geometry. Furthermore, we introduce a lightweight version of his SAGS using a simple and effective waypoint interpolation scheme. It provides a compact representation of the scene with up to 24x size reduction independent of compression strategies. Extensive experiments across multiple benchmark datasets demonstrate the superiority of his SAGS compared to state-of-the-art 3D-GS methods in both rendering quality and model size. Furthermore, we demonstrate that our structure recognition method can effectively reduce the stray artifacts and irregular distortions of previous methods while obtaining accurate depth maps. Project page https://eververas.github.io/SAGS/.
