Deep Neural Network for Handcrafted Cost-based Multi-view Stereo

초록

In the last decades, depth estimation from multi-view has been treated as an ill-posed problem. This problem becomes severe with limited data, such as sparse-view cases. However, with the availability of convolutional neural network (CNN), recent learning-based depth estimation methods have become effective on occluded and texture-less areas, whereas prior works still suffer when handling such issues. They utilize features from the CNN layer to construct cost volume and regress the input volume with a regression network. To overcome those concerns, we introduce a unique approach by combining hand-crafted and learning-based strategies. Specifically, we utilize the normalized cross-correlation (NCC) cost volume, which is more robust to noise than simple L1 and L2 costs, to improve the photo-consistency between local patches. The entire construction pipeline is implemented by PyOpenCL to speed up the processing time. Finally, we employ the network that estimates depth by regressing the handcrafted cost-based plane sweeping volume.

키워드