PicassoPlus: Advancing Geometric Deep Learning on 3D Triangular Meshes

PicassoPlus for geometric deep learning over heterogeneous 3D meshes. We formulate the continuous filters for mesh convolution using spherical harmonics as orthonormal basis. Geometric feature learning for 3-D surfaces is critical for many applications in computer graphics and 3-D vision. However, deep learning currently lags in hierarchical modeling of 3-D surfaces due to the lack of required operations and/or their efficient implementations. This journal work is a sigificant extension of our original work presented in CVPR 2021. Together, we provide PicassoPlus for deep learning over heterogeneous 3D meshes. We propose a series of modular operations for effective geometric feature learning from 3-D triangle meshes. These operations include novel mesh convolutions, efficient mesh decimation, and associated mesh (un)poolings. Our mesh convolutions exploit spherical harmonics as orthonormal bases to create continuous convolutional filters. The mesh decimation module is GPU-accelerated and able to process batched meshes on-the-fly, while the (un)pooling operations compute features for upsampled/downsampled meshes. Leveraging the modular operations of PicassoPlus, we further contribute a neural network, PicassoNet++, for 3-D surface parsing. It achieves highly competitive performance for shape analysis and scene segmentation on prominent 3-D benchmarks.