随着天文观测技术的飞速发展，获取的天体数据呈现出爆炸式的增长，这些数据不仅数量庞大，而且种类繁多，自动化机器学习方法是处理这些数据的关键。尽管基于单一光谱模态的恒星分类研究已经取得了显著进展，但为了进一步深化对恒星的理解，需要跨越不同数据模态之间的界限，实现多模态数据的有效融合与分析。在这一背景下，多模态恒星分类的重要性日益凸显。

多模态恒星分类旨在通过综合利用多种观测数据，提高恒星识别的准确性和鲁棒性，从而深化我们对恒星特性及宇宙演化的理解。这需要做到：1）对多模态信息有效表征；2）多模态特征精准对齐；3）多模态特征高效融合；4）多模态权重自适应调整。针对这四个问题，本文先提出了一种基于光谱与图像特征融合的恒星分类方法，用以提取光谱与图像的特征并促进不同模态之间的有效融合。然后设计了一种基于对比学习的恒星分类方法，来解决不同模态间特征细粒度对齐的问题。具体而言，本文的研究内容如下：

针对单一光谱模态无法全面地反映恒星特征的问题，提出了一种基于光谱与图像特征融合的恒星分类网络。该网络结合恒星光谱与测光图像数据，利用特征提取网络表征多模态信息，进一步结合残差交叉注意力实现多模态特征融合，并通过动态加权计算出每种模态的贡献度。在自建的恒星数据集上的实验结果表明，该多模态网络在分类任务上的表现超越了传统方法，验证了综合运用多模态数据进行恒星分类策略的有效性和优越性。

针对同一恒星在不同模态下的表示不一致的问题，提出了一种基于对比学习的多模态恒星分类方法。该方法基于对比学习原理，通过策略性地拉近同一恒星的光谱与图像特征表示，同时应用跨模态注意力机制对特征进行加权，有效地克服了特征对齐不精确和模态信息不平衡的问题。通过在自建数据集上进行的一系列实验，证明该方法优于其他常规分类方法，展现了其在恒星分类任务中的潜力和优势。

With the rapid development of astronomical observation technology, there is an explosive growth of acquired celestial data, which are massive in quantity and diverse in variety, automated machine learning methods have become the key to processing these data. Although significant progress has been made in the study of stellar classification based on a single spectral modality, to further deepen the understanding of stars, it is necessary to cross the boundaries between different data modalities and realize the effective fusion and analysis of multimodal data. In this context, the importance of multimodal stellar classification has become increasingly prominent.

Multimodal stellar classification aims to deepen our understanding of stellar properties and cosmic evolution by improving the accuracy and robustness of stellar identification through the integrated use of multiple observational data. This requires: 1) Effective characterization of multimodal information; 2) Precise alignment of multimodal features; 3) Efficient fusion of multimodal features; 4) Adaptive adjustment of multimodal weights. To address these four issues, this thesis first proposes a stellar classification method based on the fusion of spectral and image features, which is used to extract spectral and image features and promote effective fusion between different modalities. Then, a stellar classification method based on contrastive learning is designed to solve the problem of fine-grained alignment of features between different modalities. Specifically, the research of this thesis is as follows:

A stellar classification network based on the fusion of spectral and image features is proposed to address the problem that a single spectral modality cannot fully reflect stellar features. The network combines stellar spectral and photometric image data, characterizes the multimodal information by using a feature extraction network, further combines the ResNet CrossAttention to realize the multimodal feature fusion, and calculates the contribution degree of each modality by dynamic weighting. Experimental results on a self-built stellar dataset show that this multimodal network outperforms traditional methods on the classification task, verifying the effectiveness and superiority of the comprehensive use of multimodal data for star classification strategies.

(2) A multimodal stellar classification method based on contrastive learning is proposed to address the problem of inconsistent representation of the same star in different modes. This method is based on the principle of contrastive learning, which strategically brings the spectral and image feature representations of the same star closer and applies a cross-modal attention mechanism to weigh the features, effectively overcoming the problems of inaccurate feature alignment and imbalanced modal information. Through a series of experiments on a self-built dataset, the method is demonstrated to outperform other conventional classification methods, showing its potential and advantages in the stellar classification task.