自然语言处理（NLP）旨在让计算机理解自然语言。为此，词向量的提出将自然语言符号信息转化为数值向量形式。最初词的one-hot表示存在无法表示相似性等问题，因此词的分布式表示被提出。神经网络语言模型（NNLM）、循环神经网络语言模型（RNNLM）、对数线性模型（Word2vec模型）等相继被提出。其中在Word2vec模型中最初发现线性词类比现象，例如：。这一发现为获得理解自然语言的高质量词向量提供了启示。但以上现象产生的理论机理尚未明晰。

本文综述已有的对该现象的理论研究，并进行拓展研究。其中Levy等人提出了负采样的Skip-gram模型（SGNS模型）等价于隐式地分解一个词-上下文矩阵。本文将其拓展到负采样的CBOW模型与GloVe模型。Allen等人为SGNS模型中线性词类比的存在性提供了基本理论。我们由此启发，推广到GloVe模型中线性词类比的存在性。并且本文证明了上述理论与Ethayarajh等人的理论等价，并提出在SGNS模型和GloVe模型线性词类比存在的充要条件。

本文旨在为语言模型学习概念表示和实现知识存储的可解释性提供理论基础。

Natural Language Processing (NLP) aims to enable computers to understand natural language. To achieve this, the introduction of word vectors transforms symbolic information into numerical vector forms. Initially, the one-hot representation of words posed issues such as inability to express similarity, hence the proposal of distributed representations of words. Neural Network Language Model (NNLM), Recurrent Neural Network Language Model (RNNLM), New Log-linear Models (Word2vec Model), among others, have been successively proposed. The Word2vec Model initially discovered the phenomenon of Linear Word Analogies, exemplified by  . This discovery provides insights for obtaining high-quality word vectors to understand natural language. However, the theoretical mechanisms behind the phenomenon mentioned above remain unclear.

This paper reviews existing theoretical research on this phenomenon and conducts further exploration. Levy et al. proposed that the Skip-gram Model with negative sampling (SGNS Model) is equivalent to implicitly decomposing a word-context matrix. This paper extends it to the CBOW Model with negative sampling and the GloVe Model. Allen et al. provided fundamental theoretical support for the existence of the phenomenon in the SGNS Model. Inspired by this, we generalize the existence to the GloVe Model and demonstrate the equivalence of the aforementioned theory with Ethayarajh et al.'s theory, proposing sufficient and necessary conditions for the existence of Linear Word Analogies in the SGNS Model and GloVe Models.

This paper aims to provide a theoretical foundation for the interpretability of language models in learning concept representations and achieving knowledge storage.