February 9, 2021

RoboSense400x275China’s RoboSense has released the latest version of its ground truth data system and evaluation tool chain RS-Reference 2.1, used for lidar and multi-sensor fusion systems performance evaluation. The original RS-Reference version was launched in 2016, when the automotive-grade MEMS solid-state lidar RS-LIDAR-M1 project was established. Used by global OEMs and Tier 1 manufacturers, the system has been improved and upgraded with more efficient and useful evaluation function modules and software tool chains, RoboSense said.

The company compared the evaluation function to an exam, while the ground truth data is the “answer” for the evaluation of the perception system. “Therefore, the accuracy of ground truth data must be significantly higher than the device under test (DuT) in all aspects, including detection performance and geometric error,” RoboSense said. Usually stored in the PB-Level, ground truth data includes dynamic information such as obstacle types, speeds, and locations, and static information such as lane lines and road boundaries.

The company said the reference system includes several new updates, including:

  • A set of ground truth data generation and evaluation solutions, which outputs detection performance and geometric error indicators with a labeling efficiency close to 1:1. RoboSense said this is significantly more accurate than real-time perception, manual labeling and traditional labeling tools.
  • Includes the RoboSense 128-beam lidar RS-Ruby, Leopard camera, Continental 408 millimeter-wave radar, GI-6695 RTK, and two added RoboSense RS-Bpearl lidar for near-field blind spots in the 2.1 version.
  • Detached roof-mounted deployment without vehicle body modification. THe RS-Reference system adapts to different vehicle sizes, does not occupy the sensor installation position of the DuTs, and directly evaluates the intelligent driving system consistent with the sensor sets of commercial vehicles.
  • A full-stack evaluation tool chain, which includes data collection tools, sensor calibration, visualization, manual verification and evaluation tools. The 2.1 version upgrades the data management platform and adds the scene semantic labeling function to serve every step of the evaluation process.

Additional details are available at the RoboSense website.