3DGS · Integrating LVMs with 3D Gaussian Splatting

Generative Augmented Volumetric Reconstruction

Role: Lead Architect & Machine Learning Engineer

A fault-tolerant 3D reconstruction pipeline integrating Large Vision Models (LVMs) with 3D Gaussian Splatting (3DGS). This system acts as an intelligent pre-processing bridge, solving the classic geometric collapse in traditional photogrammetry caused by sensor degradation or extreme data sparsity.

The "CoDiffusion" Architecture

To overcome heuristic matching failures, I developed a custom pipeline composed of four specialized macro-blocks:

• Algorithmic Assessment: An automated image quality screener designed to dynamically route input data based on specific radiometric failures.
• Generative Engine: Utilizes dual-network latent diffusion to restore noise and optical blur, alongside reference attention mechanisms to mathematically hallucinate missing multi-view camera angles from a single sparse image.
• Graph Neural Network Mapping: Replaces exhaustive $O(N^2)$ heuristic matching with fast visual topological pairing, mapping non-linear AI-generated camera trajectories seamlessly.
• Continuous Volumetric Rasterization: Projects AI-injected sparse point clouds into 3D space, utilizing adaptive density control for photorealistic, real-time rendering.

Key Engineering Achievements

• Resolved the "Two-World Problem": Engineered automated background extraction protocols to prevent AI feature extractors from mapping synthetic environmental voids.
• Overcame Broken Source Data: Successfully outputted geometrically accurate, high-fidelity 3D models from fundamentally broken, sparse, and radiometrically corrupted 2D images.

Tech Stack: Python, PyTorch, CUDA, 3DGS, Graph Neural Networks, Latent Diffusion