统计力学所关心的核心问题之一在于系统配分函数或自由能、内能等特 性函数的求解，这涉及理解和描述大量粒子之间的相互作用及其在特定条件 下的行为模式。然而，即使对于 Ising 模型等高度简化的统计物理模型，遍历 构型空间也难以实现。在这个背景下，机器学习和统计物理之间的交叉应用 变得愈加密切。本文尝试将生成模型中的 Transformer 架构与变分自回归网络 （VAN）方法相结合来解决平衡态统计力学问题，充分发挥 VAN 在表述能力 和采样上的优势，同时利用 Transformer 在长序列建模上的独特能力。本文对 Transformer 的解码器部分进行了适当改造，引入 ReZero 方法以改善层与层 之间的信息流动和连接。将此模型用于求解 Sherrington-Kirkpatrick（SK）模 型和 Ising 模型，并与基于 MADE 等简单自回归神经网络的 VAN 进行了对比 分析。通过这一系列的研究工作，本文旨在为统计物理中多粒子系统的自由 能和热力学量计算提供新的视角和方法。

How to accurately calculate the partition function or the characteristic functions is one of the core problems in statistical physics. These issues involve understanding and describing the interactions among a large number of particles and their behavior under given conditions. However, even for highly simplified statistical physics models such as the Ising model, traversing configuration space remains a significant challenge. In this context, the intersection between machine learning and statistical physics becomes increasingly close. this paper attempts to combine Transformer architecture from generative models with the Variational Autoregressive Networks (VAN) method to address classical equilibrium statistical mechanics problems. Leveraging the powerful expressive capabilities of VAN and efficient sampling techniques, while harnessing the Transformer’s unique prowess in modeling long sequences, we appropriately modify the decoder part of the Transformer, introducing the ReZero method and layer normalization for connections between layers. This model is applied to solve the Sherrington-Kirkpatrick (SK) model and Ising model, juxtaposed against VAN based on simpler auto-regressive neural networks like MADE. Through this series of research efforts, this paper aims to provide new perspectives and methods for calculating the free energy and thermodynamic quantities of multi-particle systems in statistical physics