Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization

summary

Video file (mp4)

The gist

The Hilti-Trimble-Oxford Dataset provides a construction-focused visual-inertial state estimation benchmark with prior 2D maps, which matters because it offers realistic data for benchmarking SLAM

In short

This paper introduces a construction-focused dataset featuring thirty visual-inertial sequences captured across seven floors over eight months at an active site. It pairs this dynamic data with corresponding 2D floor plans, providing a realistic benchmark for testing Visual Simultaneous Localization and Mapping (SLAM) and floor plan localization systems in complex environments.

Key concepts

Visual-Inertial State Estimation
This involves using both visual data from cameras and inertial measurements from an IMU to determine the precise position of a device over time. It combines the visual system's ability to see features with the IMU's high-frequency motion tracking, which is crucial for accurate tracking in moving construction settings.
SLAM (Simultaneous Localization and Mapping)
SLAM is a process where a robot or camera system builds a map of an unknown environment while simultaneously figuring out its own location within that map. The dataset tests how well different systems can perform this task, especially when dealing with challenging conditions like moving workers and repetitive structures.
Floor Plan Localization
This is the task of accurately placing a system's estimated position relative to pre-existing 2D blueprints of the construction site. The dataset includes binary floor plans, allowing researchers to evaluate how well systems can align their real-world estimates with these structural maps.
Ground Truth Generation
The true, accurate positions of the data are established using a high-quality LiDAR-inertial SLAM system rigidly attached to the 360 camera. This system uses a LiDAR sensor and an IMU, synchronized precisely to create highly reliable trajectories that serve as the benchmark for evaluating other localization algorithms.

Terminology used across episodes

This episode discusses

The paper

Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization · Read on arXiv

Samuele Centanni, Yuhao Zhang, Yifu Tao, Julien Kindle, Frank Neuhaus, Tilman Koßsöh, Aryaman Patel, Michael Helmberger, Şmilia Szymanska ´1, ŞTorben Grไber

Hilti AG · University of Oxford · ETH Zurich · Vision & Robotics GmbH · Trimble Inc. · inria

Automated progress monitoring on construction sites is an active area of research and development. Robot and human-carried mapping systems have been developed to build 3D maps of building and infrastructure projects. While LiDAR-based mapping systems achieve high accuracy, the cost of LiDAR can be prohibitive. Consumer-grade cameras with wide field of view ("360 cameras") combined with embedded inertial measurement units (IMUs) provide a cost-effective alternative. To support change detection and progress monitoring, highly accurate visual simultaneous localization and mapping (SLAM) and floor plan localization systems are required. In this paper, we present a high-quality dataset collected at an active construction site, which captures realistic challenges such as variable lighting conditions, moving workers, fast motions, and repetitive structures. The dataset offers thirty visual-inertial sequences recorded across ten floors over an eight-month period of the construction project. Ground truth trajectories are collected using a high quality LiDAR-inertial SLAM system rigidly attached to the 360 camera. Additionally, we report the results of an open research challenge evaluating the best visual SLAM and localization systems from around the world. The Challenge attracted substantially higher participation in SLAM, with 62 teams compared to 22 in floor plan localization, which suggests that this task receives less attention from researchers and motivates further exploration through proposed dataset and challenge, available at: https://hilti-trimble-challenge.com/dataset-2026.

Transcript

Introduction to the show: ident: Robotics Radio. Generated commentary on the latest robotics and control papers.

Rosa: We're looking at the paper: "Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization".

Dev: Here's the gist of the paper. The Hilti-Trimble-Oxford Dataset provides a construction-focused visual-inertial state estimation benchmark with prior 2D maps, which matters because it offers realistic data for benchmarking SLAM and localization systems in dynamic construction environments.

Rosa: The gist The Hilti-Trimble-Oxford Dataset provides a construction focused visual inertial state estimation benchmark with prior 2D maps, which matters because it offers realistic data for benchmarking SLAM and localization systems in dynamic construction environments<ref:2607.06464#pg2> Dataset Composition The dataset consists of "thirty visual-inertial sequences recorded across seven floors over an eight-month period of the construction project" <ref:2607.06464#pg2>.

Dev: These sequences capture "variable lighting conditions, moving workers, fast motions, and repetitive structures" <ref:2607.06464#pg2>.

Rosa: The data includes "thirty visual-inertial sequences with synchronized IMU measurements, captured over a span of 8 months at the same construction site" <ref:2607.06464#pg2>.

Paper summary: Rosa: So we're looking at this paper today, "Hilti-Trimble-Oxford Dataset: three hundred sixty Visual-Inertial Benchmark with Floor Plan Priors for SLAM and Localization <ref:2607.06464#pg1,Hilti-Trimble-Oxford Dataset: 360 Visual-Inertial Benchmark with Floor Plan Priors>." It’s about building a dataset for progress monitoring on construction sites.

Dev: Yeah, it tackles the problem of using visual systems instead of expensive LiDAR for mapping huge infrastructure projects. The core idea is that consumer-grade three hundred sixty cameras with IMUs can be a cost-effective alternative to LiDAR <ref:2607.06464#pg1,a cost-effective alternative to>.

Rosa: Exactly, and this paper presents thirty visual-inertial sequences collected over eight months at one construction site. They focused on capturing what happens in real life, like variable lighting and fast movements from workers.

Dev: And they paired those sequences with corresponding 2D floor plans as a prior for localization <ref:2607.06464#pg2>. That’s the main claim: this dataset is designed to benchmark visual-inertial SLAM and localization against 2D maps in dynamic environments <ref:2607.06464#pg2>.

Rosa: It matters because it gives researchers realistic data to test their systems outside of controlled lab settings, which is where most of the current testing happens.

Dev: And they generated ground truth trajectories using a high-quality LiDAR-inertial SLAM system rigidly attached to the three hundred sixty camera, which includes a Hesai XT32M2X LiDAR and an IMU <ref:2607.06464#pg1>.

Rosa: That ground truth setup involved temporal alignment by maximizing crosscorrelation between their inertial motion signals, and then geometric optimization using an online MC2SLAM algorithm followed by a large scale non linear optimization step.

Dev: The evaluation protocol is set up to test two categories: visual-inertial SLAM in an arbitrary reference frame, and localization in the floor plan’s reference frame.

Rosa: And they scored submissions based on achieving at least ninety-nine percent coverage of those ground truth poses, with an exponentially decaying position score over the trajectory.

Dev: The results from the Hilti x Trimble Challenge two thousand twenty-six showed significantly higher participation in the SLAM category, with sixty-two teams compared to twenty-two in floor plan localization <ref:2607.06464#pg2,Hilti x Trimble Challenge 2026>.

Rosa: That suggests that visual SLAM methods are getting more attention from researchers than localization specifically tied to floor plans right now.

Dev: The top submissions for SLAM generally relied on visual-inertial odometry using both the cameras and the IMU, followed by a subsequent optimization step called BA.

Rosa: And for the localization track, they found that semantic information extraction played an important role in map alignment when matching against those 2D floor plans <ref:2607.06464#pg2>.

Dev: So what this all means is that this Hilti-Trimble-Oxford Dataset provides a construction focused visual-inertial state estimation benchmark with prior 2D maps, which matters because it offers realistic data for benchmarking SLAM and localization systems in dynamic construction environments <ref:2607.06464#pg2>.

Rosa: It gives you a solid way to test how well your system handles the messy reality of a construction site where things are constantly moving and changing.

Dev: And the structure they created—with the dataset, ground truth, and evaluation protocol—is really useful for anyone trying to develop robust systems for monitoring progress on projects.

Rosa: But we have to keep in mind one limitation they pointed out: the bitmap floor plans were derived from a single version of architectural floor plans.

Dev: That means the challenge required localization to a similar but different scene, with only some of the features usable for matching, which adds complexity to those localization tests.

Rosa: Plus, they found a few deviations or errors when comparing their floor plans to their own LiDAR-based ground-truth point clouds.

Dev: So while it’s a high-quality dataset for this specific application, you have to account for the fact that the input priors might not be perfectly aligned with the ground truth geometry.

Rosa: It shows how important it is to build benchmarks that aren't just clean data sets but ones that reflect the actual challenges of deployment.

Dev: And this whole setup, which we call the Hilti-Trimble-Oxford Dataset and the Hilti x Trimble Challenge two thousand twenty-six gives us a concrete framework for testing state estimation in these challenging settings <ref:2607.06464#pg2,the Hilti-Trimble-Oxford Dataset and the>.

Rosa: It moves SLAM and localization research away from perfect environments toward practical applications like construction site monitoring.

Dev: So for anyone working on real-time loops or developing autonomous systems, this benchmark offers a specific test case involving variable lighting and fast motions you’ll encounter daily on the job.

Conclusion: Rosa: So we're wrapping up this discussion on the Hilti-Trimble-Oxford Dataset which is about visual inertial state estimation benchmarks for construction sites.

Dev: Yeah, it's basically a dataset designed to test SLAM and localization systems using three hundred sixty cameras and IMUs in a real world context, not just clean lab data.

Rosa: The authors are building this because LiDAR mapping is accurate but really expensive for big infrastructure projects, so they want something more practical.

Dev: They collected thirty visual-inertial sequences over eight months at one construction site to capture those messy conditions like moving workers and bad lighting.

Taro: From my side, I'm interested in how the system handles the fast motions and repetitive structures that construction sites are famous for throwing at them when you try to track progress.

Rosa: The ground truth they generated used a high-quality LiDAR-inertial SLAM system attached right to the three hundred sixty camera with a Hesai XT32M2X sensor.

Dev: That setup involved a specific temporal alignment step using crosscorrelation between the LiDAR and camera clocks, and then geometric optimization with an online MC2SLAM algorithm.

Taro: So what this means for us is that we're testing the systems against data that’s already been rigorously verified by a very good system.

Rosa: The evaluation has two parts, SLAM in an arbitrary frame and localization within the floor plan's frame using 2D images as a prior <ref:2607.06464#pg2>.

Dev: And they found that the Hilti x Trimble Challenge showed much higher participation in the SLAM category, with sixty-two teams compared to only twenty-two for floor plan localization.

Rosa: That suggests visual inertial odometry is seeing more attention right now than just matching against those 2D floor plans <ref:2607.06464#pg2>.

Taro: But there's a catch they mentioned, that the bitmap floor plans were based on only one version of the architectural plans, which means you have to localize to something similar but different.

Dev: They also noted some deviations in their own LiDAR ground truth when compared against those 2D floor plans, so you can't just assume perfect alignment <ref:2607.06464#pg2>.

Rosa: Overall, this dataset provides a specific benchmark for state estimation in dynamic construction environments where things move fast and the conditions are unpredictable.

Taro: It’s a solid test case for seeing how well your system recovers when the world misbehaves on an active site.

More episodes

← Home