Lidar3D, Streaming 3D Reconstruction Lab
An early research lab for feed-forward streaming 3D reconstruction: an ordered RGB or LiDAR stream becomes a camera trajectory, dense metric depth and a fused colored point cloud. The heavy engine runs offline on a GPU; the site replays the baked cloud across four point-cloud renderers under one unified transform, with a strict renderer-honesty discipline as the point.
Business Context
3D reconstruction and point-cloud visualization matter across surveying, mining and robotics, and the recurring failure is trust: a beautiful render that quietly misrepresents the underlying data. Lidar3D's value is the opposite habit: a place to compare reconstruction engines and renderers on the same footing, where the discrepancies are found and shown (a coordinate-mirror bug fixed, an approximation labelled) instead of polished away. It is a research instrument for what reconstruction actually delivers, and for keeping the viewer honest.
Strategic Value
Lidar3D demonstrates a full streaming-reconstruction pipeline built and evaluated honestly, a from-scratch depth-and-pose network with a real held-out trajectory error, a properly vendored SOTA model kept clearly labelled as vendored, and a four-renderer viewer where parity is enforced under one transform and every approximation is disclosed. It is deliberately framed as an early lab: it replays baked artifacts (it is not real-time and not SLAM by default), it does not claim to beat the state of the art, and it leads with a negative-results ledger. That is the reusable pattern, reconstruction and rendering you can actually trust because it tells you where it doesn't.
The Challenge
Turning a stream of images or LiDAR into a coherent 3D scene is hard, and the demos are usually the least honest part: a polished point cloud that hides where the pose drifted, which numbers had no ground truth, or that different viewers are quietly showing different data. The interesting, honest questions (how well does a from-scratch feed-forward reconstructor actually do, and does what you see in the browser faithfully match the reconstructed cloud) rarely get shown.
Our Approach
Lidar3D is a lab, not a product claim. A heavy engine runs offline on a local GPU and does feed-forward (no per-scene optimization) reconstruction: an ordered RGB/LiDAR stream becomes a camera trajectory, dense metric depth and a fused colored point cloud. Two reconstruction engines are actually wired, a from-scratch depth-and-pose network (ResNet-18 backbone, a Siamese SE(3) pose head, aleatoric depth; ~0.28 m held-out trajectory error on TUM, reconstructing eight real scenes from TUM/7-Scenes/ICL) and a genuinely vendored 2026 model for four outdoor scenes, plus Open3D ICP LiDAR odometry and a CPU synthetic engine. The public site is a static SPA that replays the baked artifacts across four point-cloud renderers (three.js, deck.gl, surfels, Potree LOD), all drawing the same cloud under one unified coordinate transform. The discipline is the headline: metrics with no ground truth are shown as "none" with a reason, and where a renderer physically cannot do per-frame replay, the app approximates and says so in the UI rather than faking it.
Key Performance Indicators
| KPI | Baseline | Result | Impact |
|---|---|---|---|
| Reconstruction method | Per-scene optimization | Feed-forward depth + pose (own net, ~0.28 m ATE on TUM) | A measured, from-scratch reconstructor |
| Renderer honesty | Different viewers, different-looking clouds | 4 renderers, one baked cloud, one unified transform | What you see is the same data |
| No faked metrics | Numbers shown without ground truth | Metrics without GT display "none" + a reason; approximations labelled in-UI | The demo does not oversell |
Architecture
lidar3d renderers
Reconstruct honestly, render honestly
Lidar3D is an early research lab for feed-forward streaming 3D reconstruction: an ordered RGB or LiDAR stream becomes a camera trajectory, dense metric depth and a fused colored point cloud, with no per-scene optimization. The heavy engine runs offline on a GPU; the public site is a static SPA that replays the baked cloud across four point-cloud renderers. Live at lidar3d.fasl-work.com.
What is actually wired
Two reconstruction engines: a from-scratch depth-and-pose network (ResNet-18 backbone, a Siamese SE(3) pose head, aleatoric depth) with a real ~0.28 m held-out trajectory error on TUM, reconstructing eight real indoor scenes (TUM / 7-Scenes / ICL); and a genuinely vendored 2026 model (kept clearly labelled as vendored) for four outdoor scenes. Plus Open3D ICP LiDAR odometry and a CPU synthetic engine.
Renderer honesty is the headline
Four renderers (three.js, deck.gl, surfels and Potree LOD) all draw the same baked cloud under one unified (x, −y, −z) transform (a deck.gl mirror bug was found and fixed). Where a renderer physically cannot do per-frame replay, the app approximates and says so in the UI rather than faking it, and any metric without ground truth is shown as “none” with a reason.
Honest scope
This is a lab, framed as one: it replays baked artifacts (it is not real-time and not SLAM by default: loop closure and global bundle adjustment are opt-in), it does not claim to beat the state of the art, the outdoor model is vendored not ours, and there is no textured mesh or Gaussian-splat output. The value is the discipline and the negative-results ledger, not a fidelity claim.
Technology Stack
In action
A short tour of the live app: the real interface, recorded from the deployed site.
Application Screenshots
