3D SMoE Splatting for Edge-aware Realtime Radiance Field Rendering
TU Berlin, Mid Sweden University, HTW Berlin
TL;DR
Abstract
Steered Mixtures-of-Experts (SMoE) is an existing regression framework that has previously been applied for modeling and compression of 2D images and higher-dimensional imagery, including compression of light fields and light-field video. SMoE models are sparse, edge-aware representations that allow rendering of imagery with few Gaussians with excellent quality. In this paper a novel, edge-aware "3D SMoE Splatting" (3DSMoES) framework for 3D rendering is introduced, adopted to fit into the existing "3D Gaussian Splatting" (3DGS) CUDA optimization pipeline. Here, SMoE regression serves as a "plug-and-play" solution that replaces the established 3DGS regression as a novel workhorse. 3DSMoES achieves significant visual quality gains with drastically fewer Gaussian kernels compared to 3DGS. We observe up to approximately 4dB improvement in PSNR on individual scenes with kernel reductions between 20 to 50 percent. The sparse models are significantly faster to train and allow up to 30-50 percent improved rendering speeds.
Bibtex
If you use our work in your research, please cite our publication:
@inproceedings{10.1145/3757377.3763899,
author = {Li, Yi-Hsin and Sikora, Thomas and Knorr, Sebastian and Sj\"{o}str\"{o}m, M\r{a}rten},
title = {3D SMoE Splatting for Edge-aware Realtime Radiance Field Rendering},
year = {2025},
isbn = {9798400721373},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3757377.3763899},
doi = {10.1145/3757377.3763899},
abstract = {Steered Mixtures-of-Experts (SMoE) is an existing regression framework that has previously been applied for modeling and compression of 2D images and higher-dimensional imagery, including compression of light fields and light-field video. SMoE models are sparse, edge-aware representations that allow rendering of imagery with few Gaussians with excellent quality. In this paper a novel, edge-aware "3D SMoE Splatting" (3DSMoES) framework for 3D rendering is introduced, adopted to fit into the existing "3D Gaussian Splatting" (3DGS) CUDA optimization pipeline. Here, SMoE regression serves as a "plug-and-play" solution that replaces the established 3DGS regression as a novel workhorse. 3DSMoES achieves significant visual quality gains with drastically fewer Gaussian kernels compared to 3DGS. We observe up to approximately 4dB improvement in PSNR on individual scenes with kernel reductions between 20 to 50 percent. The sparse models are significantly faster to train and allow up to 30-50 percent improved rendering speeds. Code for this paper is at https://github.com/yihsinli/3D-SMoE-Splatting.},
booktitle = {Proceedings of the SIGGRAPH Asia 2025 Conference Papers},
articleno = {137},
numpages = {11},
keywords = {Gaussian Splatting, Compression, Steered Mixture of Expert},
location = {},
series = {SA Conference Papers '25}
}