Quantized 2D Gaussian Splatting (Q2DGS)

This repository contains the implementation of the quantized 2D Gaussian splatting Q2DGS. A modified version of the 2DGS developed by me for my master's thesis. The new model is able to obtain lighter reconstructions in terms of memory, making it easier to use the reconstructed scenes.

The process of memory reduction is based on three key elements:

• A vector quantization of Gaussian parameters based on K-means during the optimization
• A regularization function that acts only on opacity to reduce the number of Gaussians with a low opacity value, considering a threshold 𝜏𝜎
• A modificaton to the CUDA kernel code executed in the rasterizationphase and densification step for the removal of points and consequently Gaussians

Installation

# download
git clone https://github.com/Deb0r0h/Q2DGS_Quantized_2D_Gaussian_Splatting.git --recursive
conda env create --file environment.yml
conda activate q2dgs

Training

To train a scene, simply use

python train.py -s <path to COLMAP or NeRF Synthetic dataset>
# or
python trainRunner.py

Commandline arguments for regularizations

--lambda_normal  # hyperparameter for normal consistency
--lambda_distortion # hyperparameter for depth distortion
--depth_ratio # 0 for mean depth and 1 for median depth, 0 works for most cases
--opacity_regularization # hyperparameter for opacity regularization

Tips for adjusting the parameters on your own dataset:

• For unbounded/large scenes, we suggest using mean depth, i.e., depth_ratio=0, for less "disk-aliasing" artifacts.

Rendering

Bounded Mesh Extraction

To export a mesh within a bounded volume, simply use

python render.py -m <path to pre-trained model> -s <path to COLMAP dataset>
# or
python renderRunner.py

Commandline arguments you should adjust accordingly for meshing for bounded TSDF fusion, use

--depth_ratio # 0 for mean depth and 1 for median depth
--voxel_size # voxel size
--depth_trunc # depth truncation
--load_quantization # to apply quantization on render

If these arguments are not specified, the script will automatically estimate them using the camera information.

Unbounded Mesh Extraction

To export a mesh with an arbitrary size, we devised an unbounded TSDF fusion with space contraction and adaptive truncation.

python render.py -m <path to pre-trained model> -s <path to COLMAP dataset> --mesh_res 1024

Quick Examples

Assuming you have downloaded MipNeRF360, simply use

python train.py -s <path to m360>/<garden> -m output/m360/garden
# use our unbounded mesh extraction!!
python render.py -s <path to m360>/<garden> -m output/m360/garden --unbounded --skip_test --skip_train --mesh_res 1024
# or use the bounded mesh extraction if you focus on foreground
python render.py -s <path to m360>/<garden> -m output/m360/garden --skip_test --skip_train --mesh_res 1024

If you have downloaded the DTU dataset, you can use

python train.py -s <path to dtu>/<scan105> -m output/date/scan105 -r 2 --depth_ratio 1
python render.py -r 2 --depth_ratio 1 --skip_test --skip_train

Custom Dataset: We use the same COLMAP loader as 3DGS, you can prepare your data following here.

Evaluation

To compute chamfer and F1

python scripts/dtu_eval.py --dtu dataset/DTU --DTU_Official dataset/stl --skip_training --skip_rendering --output_path ./output/date

To compute PSNR, SSIM, LIPPS

python metricsRunner.py # test for NVS, train for 3DR

Acknowledgements

The project is based on the splendid work done in

• 2DGS
• Compact3D