Multi-view Generative Augmented 3D Reconstruction Pipeline
B.Eng final-year project: a three-stage pipeline that reconstructs high-quality 3D Gaussian Splatting scenes from just 12 sparse images — recovering 51.3% of the quality gap versus a 301-image baseline, a 96% reduction in data acquisition.
Multi-view Generative Augmented 3D Reconstruction Pipeline
B.Eng Final Year Project — David Ogunmola
This project investigates the reconstruction of photorealistic 3D scenes from sparse, degraded image inputs using 3D Gaussian Splatting (3DGS) as the core rendering framework. The central problem addressed is that real-world 3D capture is rarely ideal — photographers cannot always provide the dense, uniformly distributed, high-quality imagery that classical Structure-from-Motion (SfM) and neural rendering pipelines assume.
The pipeline operates in three stages:
- Quality Screening — A five-pillar radiometric screener (MUSIQ + saturation, exposure, contrast, colour cast) evaluates each input image and routes it to the appropriate augmentation path across three scene-adaptive modes (natural, indoor, synthetic).
- Generative Augmentation — Three diffusion-based modes handle different scene types: Zero123++ (Mode A) synthesises geometrically consistent novel views for object-centric scenes; ControlNet Tile (Mode B) restores degraded images; ViewCrafter (Mode C) interpolates between sparse viewpoints for real-world scenes using DUSt3R point cloud conditioning.
- AI-Driven Pose Estimation — SuperPoint + LightGlue exhaustive feature matching replaces classical SIFT for COLMAP-based camera pose estimation, enabling registration in cases where SIFT fails entirely.
Key Result
The proposed pipeline recovers 51.3% of the quality gap between a sparse no-augmentation baseline (PSNR 9.60 dB) and the full 301-image upper bound (PSNR 21.99 dB), using only 12 sparse input images — a 96% reduction in acquisition effort.
| Condition | Images | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|
| Baseline B — Sparse + SIFT | 12 (FAILED) | N/A | N/A | N/A |
| Ablation — Sparse + AI matching | 12 | 9.60 dB | 0.2674 | 0.5628 |
| Proposed — ViewCrafter + AI matching | 12 | 15.96 dB | 0.5456 | 0.3809 |
| Baseline A — Full dataset + SIFT | 301 | 21.99 dB | 0.8069 | 0.2084 |
Repository Structure
process_file.py— Five-pillar quality screener (MUSIQ-based, 3 modes)convert_ai.py— SuperPoint + LightGlue → COLMAP pose estimation (replacesconvert.pyfrom gaussian-splatting)diffusion_script_v0.py— Generative augmentation engine (Modes A / B)corrupt_data.py— Dataset corruption simulator (blur, exposure, noise)Pipeline_files/unified_pipeline.ipynb— Main Colab notebook — all three pipeline modes
Tech Stack: Python, 3D Gaussian Splatting, Zero123++, ControlNet Tile, ViewCrafter, SuperPoint, LightGlue, MUSIQ, DUSt3R