PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learning

Transformer已经彻底改变了点云学习任务，但是二次复杂度限制了其在长序列上的扩展，并给有限的计算资源带来了负担。最近出现的RWKV是一种新的深度序列模型，已经在NLP任务中展现了巨大的潜力。本文提出了PointRWKV，这是一种线性复杂度模型，是从NLP领域的RWKV模型中派生出来的，并对点云学习任务进行了必要的修改。具体来说，我们将嵌入的点补丁作为输入，首先提出使用修改后的多头矩阵值状态和动态注意力递归机制，在PointRWKV块内探索全局处理能力。为了同时提取局部几何特征，我们设计了一个并行分支，使用图稳定器在固定半径的近邻图中高效编码点云。此外，我们将PointRWKV设计为多尺度框架，用于层次化地学习3D点云特征，以促进各种下游任务。对不同的点云学习任务进行广泛的实验表明，我们提出的PointRWKV优于基于Transformer和Mamba的对应模型，同时节省了约46％的FLOPs，证明了构建基础3D模型的潜在选择。

Point cloud learning tasks suffer from quadratic complexity with current transformer models, limiting their extension to long sequences and burdening limited computational resources. The paper proposes a linear complexity model, PointRWKV, derived from the RWKV model in NLP tasks, with necessary modifications for point cloud learning tasks.

PointRWKV is designed as a multi-scale framework for hierarchical feature learning of 3D point clouds. It explores global processing capabilities within PointRWKV blocks using modified multi-headed matrix-valued states and a dynamic attention recurrence mechanism, while extracting local geometric features simultaneously with a parallel branch encoding the point cloud efficiently in a fixed radius near-neighbors graph with a graph stabilizer.

The proposed PointRWKV outperforms transformer- and mamba-based counterparts on different point cloud learning tasks while saving about 46% FLOPs. The experiments use various datasets, and the paper provides open-source code. This model demonstrates potential for constructing foundational 3D models and is worth further research.

Recent related studies in this field include 'Point Transformer' and 'MambaNet: A Compact 3D DenseNet for Point Cloud Analysis'.