watonomous.github.io

[ Software Division : Frustum Clustering ]

Created by [ Anita Hu], last modified on Jan 17, 2020

The main purpose of frustum clustering is to associate 3D bounding boxes with 2D detections. 3D bounding boxes are derived from LiDAR point clouds, and 2D detections are from running neural networks on camera images. This node takes in the merged LiDAR point cloud, 2D detections and outputs 3D bounding boxes for traffic signs, pedestrians and cyclists. The main idea is to project LiDAR points onto the camera image and based on where 3D points land on the image we create a segment of the point cloud (called frustum) for each 2D bounding box. Then we do euclidean clustering on each frustum which produces many 3D bounding boxes. The 3D bounding box is selected based on the IOU between the projection of the 3D bounding box onto the image plane and the original 2D detection.

Calibration:

For this algorithm to work well, we need to calibrate the LiDAR and camera each time the sensors are moved.

How to do that: Lidar To Camera Calibration: Projection Matrix

[Current code on Gitlab:]

https://git.uwaterloo.ca/WATonomous/perception-year-2/tree/master/frustum_object_detection

Demo:

Your browser does not support the HTML5 video element

Attachments:

video0.mov (video/quicktime)
video0-2.mov (video/quicktime)\

Document generated by Confluence on Dec 10, 2021 04:02

Atlassian