Croquis — On-Device 3D Reconstruction of Real Spaces on Android
An Android app that turns a walk-through (camera, IMU and GPS, no lidar) into a metric, consistency-gated point-cloud reconstruction computed fully on the phone and stored on the phone as a scene library you can view, measure and export, with an honest uncertainty budget. Imagery never leaves the device. Sensors own the trajectory and scale (ARCore VIO); the neural depth model contributes dense geometry, never the trajectory. In development (v0), Apache-2.0. The on-phone companion to the Lidar3D reconstruction line, with a local-GPU reprocessor (Croquis Station) above it.
Business Context
On-device, private 3D capture has clear uses in real-estate, insurance, construction and field survey, where taking a measurement from a phone walk-through is valuable but sending a customer premises to a cloud is a liability. The honesty pillar is the differentiator: a reconstruction that states its own uncertainty budget lets someone measure a volume and know the error bar, instead of a clean-looking mesh that hides how much it drifted.
Strategic Value
Croquis is the mobile end of the same reconstruction line as Lidar3D: sensors own the metric trajectory, the learned model is scoped to geometry it can be trusted for, and the output carries an uncertainty budget rather than a false-clean surface. It is an honestly-scoped, privacy-first capture product for general Android hardware, with a two-tier phone-plus-local-GPU pipeline (Croquis and Croquis Station) under versioned data contracts. It is in development (v0), and the page says so.
The Challenge
Turning a phone walk-through into a metric 3D reconstruction is usually either a cloud service (imagery leaves the device) or a flagship-only lidar feature. Doing it on a general Android phone, from camera and inertial sensors alone, honestly and privately, is the harder and more useful problem: the trajectory must stay metric, the geometry must not drift, and the result has to state how sure it is before anyone measures a volume off it.
Our Approach
Croquis separates responsibilities: the sensors (ARCore visual-inertial odometry plus GPS geo-anchoring) own the camera trajectory and the metric scale, and a neural depth model contributes dense geometry only, never the trajectory. Keyframe depth is fused only where it agrees with the sensor trajectory and neighbouring keyframes; disagreeing or far-field geometry is not deleted but kept in a low-confidence context tier so the user can see what is trustworthy. It targets general devices, not flagships, with thermal duty-cycling and adaptive keyframe rates. The scene is the product: reconstructions persist on-device as tiled point and voxel stores with confidence, geo-anchor and uncertainty metadata, behind versioned data contracts. The app shell is Expo (React Native, TypeScript, EN/ES); the capture core is native (Kotlin and C++) running ARCore, on-device depth inference, fusion and GL rendering; a separate local-GPU tool, Croquis Station, reprocesses the same session at settings no phone sustains, anchored to the phone VIO metric scale so the output is in true metres.
Key Performance Indicators
| KPI | Baseline | Result | Impact |
|---|---|---|---|
| Trajectory source | Learned pose (drifts) | ARCore VIO + GPS | Metric scale owned by sensors, not the net |
| Where imagery goes | Cloud upload | Stays on device | Private by construction |
| Low-agreement geometry | Deleted or hidden | Kept in a low-confidence tier | Honest uncertainty budget |
Technology Stack
Visual assets for this project are not publicly available.