Steered-Mixture-of-Experts (SMoE)
The three guiding research questions of the PhD project are:
- How can we design an initialization scheme that yields a compact SMoE model while drastically reducing training time?
- Can the SMoE architecture itself be restructured to accelerate convergence by an order of magnitude?
- Can SMoE be extended to represent 3D radiance fields, enabling real-time rendering without trading off reconstruction quality?
Abstract
This PhD project investigates an explicit regression model aimed at reducing the number of computational primitives in both 2D image and 3D radiance field domains. The scope of this work includes developing methods to reduce computational primitives, investigating explicit regression for transparency and scalability, and evaluating these methods across 2D image and 3D radiance field domains. This thesis is subject to specific delimitations. First, traditional compression techniques, such as quantization, entropy coding, or transform-based representations, are excluded. These methods are useful for reducing memory either during training or after, but they do not address the structural modeling inefficiencies or change the main architecture. The focus of this thesis is to evaluate the effectiveness of the redesigned architecture. Second, implicit neural representations and deep networks, such as NeRF variants, transformers, diffusion models, or convolutional autoencoders, are not used as baselines or design foundations. These architectures often hide the modeling structure behind layers of nonlinearity and offer limited control over parameter allocation. This thesis instead emphasizes transparent, interpretable models with modular behavior. Third, the thesis works with 3D radiance fields. It omits alternative 3D representations such as point clouds, meshes, ray tracing, or light-field rendering. These modalities introduce separate concerns and would weaken the focus on the architecture and regression method.
3D SMoE Splatting for Edge-aware Realtime Radiance Field Rendering
We introduce a novel, edge-aware "3D SMoE Splatting" (3DSMoES) framework for 3D rendering is introduced, adopted to fit into the existing "3D Gaussian Splatting" (3DGS) CUDA optimization pipeline. Here, SMoE regression serves as a "plug-and-play" solution that replaces the established 3DGS regression as a novel workhorse. 3DSMoES achieves significant visual quality gains with drastically fewer Gaussian kernels compared to 3DGS.
Comments