Augment with Teacher and Distill with Student: A Two-Stage Teacher-Student Network Training Scheme for 3D Human Segmentation

Human segmentation using point clouds requires clustering of points belonging to the same human body part. In the supervised learning scenario, previous studies can segment the human body parts to some extent. However, segmentation easily fails for complex postures, especially for the parts with a wide range of motion (e.g., parts from the hand to the upper arm). To alleviate this problem, first, the Random Vertex Displacement (RVD) filter is applied to an existing human body point clouds dataset to augment the training data. Specifically, the RVD filter creates a sphere with a given radius centered on each point that constitutes the human point cloud. The point is randomly shifted within the sphere for augmentation. The model trained with the RVD augmented data is treated as the teacher network. Second, we train a student network from scratch to generate the same intermediate representation to mimic the teacher network. In the experiment, the teacher network improves the average IoU by around 2%, and to our surprise, the student network further outperforms the teacher by another 2%, which well validates the effectiveness of the proposed two-stage scheme for human segmentation.