Integrated RGBD Perception for Clamp-Type Autonomous Forklifts: Patch-Median Depth Sampling, AND-Gate Grouping, and PTZ-Based Semantic Verification

초록

Clamp-type robots grasp boxes by pressing their lateral surfaces without pallets, requiring the perception system to simultaneously deliver six control-ready outputs before every pick-and-place cycle: metric dimensions (P1), 3D position and standoff distance (P2), yaw angle (P3), stacking tier count and product homogeneity (P4), group composition (P5), and placement safety (P6). No existing method jointly provides all six. This paper introduces an integrated RGB-D perception framework that resolves P1-P6 in a single pipeline. Contribution C1 proposes a mask-aware patch-median depth strategy. Because clamp operations keep the box front face nearly orthogonal to the camera, depth across the face is observed as an approximately constant plane, enabling robust depth estimation without RANSAC plane fitting. Inward retry and vertical-kernel fallback mechanisms address the boundary depth bleeding and missing-value problems inherent in time-of-flight sensors. All metric geometry (P1-P3) is derived from a shared set of front-face mask primitives, guaranteeing geometric consistency, and a four-condition AND-gate grouping algorithm infers stacking structure (P4-P5) while diagnosing failures through four explicit rejection codes. Contribution C2 combines RANSAC-based floor modeling with row-wise depth profiling for placement-safety assessment (P6), and employs a PTZ active-alignment loop to perform barcode decoding, OCR with Levenshtein correction, and handling-mark recognition at distances beyond the reach of static cameras. Experiments on three box types at operating distances of 2.0-3.0 m yield an overall mean relative dimension error (rMAE) of 0.85-1.18% and a yaw mean absolute error (MAE) of 1.21 degrees. Placement safety assessment achieves 98.46% accuracy over 65 cases; ablation studies, parameter sensitivity analysis across six thresholds, and environmental robustness evaluation under varying illumination confirm the indispensability and stable operating range of each pipeline component. Performance evaluation on the target edge computer confirms sustained operation at 7-10 FPS, satisfying the perception update rate required in the operational environment of clamp-type forklifts. The scope of this work is the feasibility validation of the integrated framework under controlled laboratory conditions; analysis under in-situ warehouse dynamics is left to follow-up work.

키워드