Lidar3D — Streaming 3D Reconstruction Lab
An early research lab for feed-forward streaming 3D reconstruction: an ordered RGB or LiDAR stream becomes a camera trajectory, dense metric depth and a fused colored point cloud. The heavy engine runs offline on a GPU; the site replays the baked cloud across four point-cloud renderers under one unified transform — with a strict renderer-honesty discipline as the point.
Business Context
3D reconstruction and point-cloud visualization matter across surveying, mining and robotics, and the recurring failure is trust: a beautiful render that quietly misrepresents the underlying data. Lidar3D's value is the opposite habit — a place to compare reconstruction engines and renderers on the same footing, where the discrepancies are found and shown (a coordinate-mirror bug fixed, an approximation labelled) instead of polished away. It is a research instrument for what reconstruction actually delivers, and for keeping the viewer honest.
Strategic Value
Lidar3D demonstrates a full streaming-reconstruction pipeline built and evaluated honestly — a from-scratch depth-and-pose network with a real held-out trajectory error, a properly vendored SOTA model kept clearly labelled as vendored, and a four-renderer viewer where parity is enforced under one transform and every approximation is disclosed. It is deliberately framed as an early lab: it replays baked artifacts (it is not real-time and not SLAM by default), it does not claim to beat the state of the art, and it leads with a negative-results ledger. That is the reusable pattern — reconstruction and rendering you can actually trust because it tells you where it doesn't.
The Challenge
Turning a stream of images or LiDAR into a coherent 3D scene is hard, and the demos are usually the least honest part: a polished point cloud that hides where the pose drifted, which numbers had no ground truth, or that different viewers are quietly showing different data. The interesting, honest questions — how well does a from-scratch feed-forward reconstructor actually do, and does what you see in the browser faithfully match the reconstructed cloud — rarely get shown.
Our Approach
Lidar3D is a lab, not a product claim. A heavy engine runs offline on a local GPU and does feed-forward (no per-scene optimization) reconstruction: an ordered RGB/LiDAR stream becomes a camera trajectory, dense metric depth and a fused colored point cloud. Two reconstruction engines are actually wired — a from-scratch depth-and-pose network (ResNet-18 backbone, a Siamese SE(3) pose head, aleatoric depth; ~0.28 m held-out trajectory error on TUM, reconstructing eight real scenes from TUM/7-Scenes/ICL) and a genuinely vendored 2026 model for four outdoor scenes — plus Open3D ICP LiDAR odometry and a CPU synthetic engine. The public site is a static SPA that replays the baked artifacts across four point-cloud renderers (three.js, deck.gl, surfels, Potree LOD), all drawing the same cloud under one unified coordinate transform. The discipline is the headline: metrics with no ground truth are shown as "none" with a reason, and where a renderer physically cannot do per-frame replay, the app approximates and says so in the UI rather than faking it.
Key Performance Indicators
| KPI | Baseline | Result | Impact |
|---|---|---|---|
| Reconstruction method | Per-scene optimization | Feed-forward depth + pose (own net, ~0.28 m ATE on TUM) | A measured, from-scratch reconstructor |
| Renderer honesty | Different viewers, different-looking clouds | 4 renderers, one baked cloud, one unified transform | What you see is the same data |
| No faked metrics | Numbers shown without ground truth | Metrics without GT display "none" + a reason; approximations labelled in-UI | The demo does not oversell |
Architecture
lidar3d renderers
Reconstruct honestly, render honestly
Lidar3D is an early research lab for feed-forward streaming 3D reconstruction: an ordered RGB or LiDAR stream becomes a camera trajectory, dense metric depth and a fused colored point cloud, with no per-scene optimization. The heavy engine runs offline on a GPU; the public site is a static SPA that replays the baked cloud across four point-cloud renderers. Live at lidar3d.fasl-work.com.
What is actually wired
Two reconstruction engines: a from-scratch depth-and-pose network (ResNet-18 backbone, a Siamese SE(3) pose head, aleatoric depth) with a real ~0.28 m held-out trajectory error on TUM, reconstructing eight real indoor scenes (TUM / 7-Scenes / ICL); and a genuinely vendored 2026 model (kept clearly labelled as vendored) for four outdoor scenes. Plus Open3D ICP LiDAR odometry and a CPU synthetic engine.
Renderer honesty is the headline
Four renderers — three.js, deck.gl, surfels and Potree LOD — all draw the same baked cloud under one unified (x, −y, −z) transform (a deck.gl mirror bug was found and fixed). Where a renderer physically cannot do per-frame replay, the app approximates and says so in the UI rather than faking it, and any metric without ground truth is shown as “none” with a reason.
Honest scope
This is a lab, framed as one: it replays baked artifacts (it is not real-time and not SLAM by default — loop closure and global bundle adjustment are opt-in), it does not claim to beat the state of the art, the outdoor model is vendored not ours, and there is no textured mesh or Gaussian-splat output. The value is the discipline and the negative-results ledger, not a fidelity claim.
Technology Stack
Visual assets for this project are not publicly available.