Considering the insufficient feature extraction in hyperspectral remote sensing image classification under limited training samples, a multi-scale 3D capsule network is proposed to improve hyperspectral image classification. Compared with the traditional convolutional neural network, the proposed network is equivariant, and its input and output forms are neurons in the form of vectors rather than scalar values in the convolutional neural network. It is conducive to obtaining the spatial relationship between objects and the correlation between features and can avoid problems such as overfitting under limited training samples. Specifically, the network extracts the features of an input image through the convolution kernel operation on three scales to obtain the features of different scales. Then, the three branches are connected to different 3D capsule networks to obtain the correlation between spatial spectrum features. Finally, the results of the three branches are fused, and the classification results are obtained by the local connection and margin loss function. The experimental results reveal that this method has good generalization performance on the open-source hyperspectral remote sensing data set and has higher classification accuracy than other advanced hyperspectral remote sensing image classification methods.