Graph convolutional network (GCN) excels in skeleton-based action recognition, but the recognition accuracy is limited by large inter-joint distances and the restricted feature aggregation capability of existing methods. To tackle this issue, this study proposes a dual-dimension topology-optimized hypergraph convolution network (DDTO-HGCN) for skeleton action recognition. The method extends traditional graph structures using hypergraph theory, leveraging hyperedges to extract local and global information while capturing high-order dependencies. A temporal topology-optimized hypergraph convolution (TTO-HGC) and a joint topology-optimized hypergraph convolution (JTO-HGC) are proposed to optimize the topological structure along the temporal and joint dimensions respectively, thereby enhancing feature representation capabilities. Additionally, a multi-scale temporal convolution network (MS-TCN) is incorporated to enrich temporal feature expression, and a four-stream ensemble method which utilizes relative vectors of the skeleton center as supplementary data streams is introduced, improving recognition performance. Experimental results on the NTU RGB+D and NTU RGB+D 120 datasets demonstrate that the proposed method significantly outperforms some state-of-the-art methods.