Adaptive active fusion of camera and single-point LiDAR for depth estimation
Tran, Dang M. Ahlgren, Nate Depcik, Christopher David He, Hongsheng
Tran, Dang M.
Ahlgren, Nate
Depcik, Christopher David
He, Hongsheng
Citations
Altmetric:
Other Names
Location
Time Period
Advisors
Original Date
Digitization Date
Issue Date
2023-06-08
Type
Article
Genre
Keywords
Cameras,Cost-effective LiDAR,Deep learning,Depth completion,Estimation,Image edge detection,Laser radar,Real-time systems,Sensor fusion,Sensors,Single-point LiDAR,Visualization
Subjects (LCSH)
Citation
D. M. Tran, N. Ahlgren, C. Depcik and H. He, "Adaptive Active Fusion of Camera and Single-Point LiDAR for Depth Estimation," in IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1-9, 2023, Art no. 5018509, doi: 10.1109/TIM.2023.3284129.
Abstract
Depth sensing is an important problem in many applications, such as autonomous driving, robotics, and automation. This article presents an adaptive active fusion method
for scene depth estimation by using a camera and a singlepoint light detection and ranging (LiDAR) sensor. An active scanning mechanism is proposed to guide laser scanning based
on critical visual and saliency features, and the convolutional spatial propagation network (CSPN) is designed to generate and refine full depth map from the sparse depth scans. The
active scanning mechanism generates a depth mask by using log-spectrum saliency detection, Canny edge detection, and uniform sampling, which indicate critical regions that require a high
resolution of laser scanning. To reconstruct a full depth map, the designed CSPN network extracts affinity matrices from the sparse depth scans, while reserving global spatial information in
the images. The performance of proposed method was evaluated and compared with the state-of-the-art methods on the NYU depth dataset v2 (NYUv2) and the experiment demonstrated its
outperformance in reconstruction accuracy and robustness to measurement noise. The proposed method was also evaluated in real-world scenarios.
Table of Contents
Description
Click on the DOI to access this article (may not be free).
Publisher
IEEE
Journal
Book Title
Series
IEEE Transactions on Instrumentation and Measurement
Volume 72
Volume 72
Digital Collection
Finding Aid URL
Use and Reproduction
Archival Collection
PubMed ID
DOI
ISSN
1557-9662