双语神经表征是双语研究中的一个核心问题，脑成像为研究该问题提供了强有力的技术手段。早期的研究围绕双语的神经表征是“共享还是分离”来展开，主要从 “静态”和“单脑”的角度来研究双语神经表征，集中在双语语音、词汇或单个句子水平的加工，通过考察不同语言加工所激活的脑功能激活模式或功能连接模式来揭示双语神经表征的关系。这些研究缺乏对自然情境下的双语语言加工过程（如故事理解）的深入考察，难以揭示自然情境中的双语加工所具有的系统性、层级性和动态性，也未能有效利用大脑信号的时空动态信息。本论文以汉-英双语者为研究对象，聚焦自然情境下的口语故事理解过程，综合采用多种脑功能数据分析方法和数据挖掘技术，包括基于多变量分析的表征相似性分析方法(Representational similarity analysis, RSA)，自然语言处理(Natural language processing,
NLP)中的语义计算技术等，将传统的“单脑”范式和“听-说”交互的“双脑”范式相结合，对处在“交流”状态的讲述者和倾听者的大脑活动进行深入考察，通过四个研究系统探究了双语者的双语神经表征机制。

研究一采用脑功能激活、功能连接、脑网络属性以及被试间一致性等方法，系统考察了双语完整故事理解中的神经表征及其语言效应。研究发现，两种语言的口语故事理解在双语者中均诱发了广泛的神经响应和功能连接，且在一些脑区的活动强度、以及脑区之间的功能连接强度存在显著的语言间差异，但全脑功能网络拓扑属性并不存在显著组间差异。进一步结合被试间一致性(ISC)分析和基于ISC特征的功能分区方法，发现两种语言的口语故事在双侧颞叶、额叶、内侧楔前叶等脑区表现出较强的被试间一致性；ISC效应从颞叶初级皮层出发，随着ISC效应向颞叶其他区域以及顶叶、额叶等脑区扩散，其被试间一致性更弱，被试间变异性更强。总体而言，无论是在大脑神经活动的空间维度还是时间维度，两种语言的故事神经表征在全局模式上表现为很强的相似性，而在特定脑区表现出一定的语言特异性。

研究二采用音频包络提取技术、自然语言处理中的语义表征计算技术和表征相似性分析方法，探究在双语故事理解中的语音加工和语义加工的神经表征及其语言效应。结果发现在语音加工过程中，两种语言的故事理解在额叶、颞叶、顶叶的大部分脑区存在和音频特征的正相关，而在枕叶和内侧皮质的大部分区域存在和音频特征的负相关。在语义加工中，两种语言的故事理解在颞上回和颞中回前部存在和语义特征的显著相关。语音加工的语言间对比中，汉语条件在左侧额下回和左侧颞上回中部存在更强的语音效应，英语条件更强的语音效应出现在了双侧的脑岛。语义加工的语言间对比中，在宽松阈限下有一定的语言间差异，而在严格阈限下语言间的差异消失。说明从语音加工到语义加工，随着信息整合程度的加深，两种语言神经表征的差异逐渐减小。

研究三采用层级故事理解实验，结合表征相似性分析和故事跨层级识别方法，探究口语故事理解中情境模型构建的神经表征及其语言效应。结果发现，汉语句子打乱条件情境下在右侧额中回、左侧颞下回、左侧的枕下回等脑区的BOLD信号能显著跨层级识别故事内容。英语句子打乱条件情境在左侧额下回、脑岛等脑区的BOLD信号能显著跨层级识别故事内容。上述脑区在故事倒放条件下的活动信号均无法准确识别故事内容。研究揭示了跨层级情境模型构建的神经表征机制及其语言间差异。

研究四考采用“听-说”交互中的神经耦合效应及时间滞后性分析，探究双语故事表征构建的动态特征及其语言效应。研究对两种语言故事条件下，在五种不同的滞后时间（0秒、2秒、4秒、6秒和8秒）下的神经耦合强度进行了语言间对比，发现随延迟时间增加，倾听者-讲述者神经耦合反应的脑区从听觉区向语义、社会认知加工脑区扩展。同一延迟时间窗下的神经活动中，汉语的神经耦合信号更强，英语的神经耦合信号更弱；随着故事理解加工进程的增加，英语故事理解的神经耦合效应比汉语故事理解更早衰退。故事理解中的表征构建的动态特征显示，与英语故事理解相比，汉语故事理解中构建的神经表征更为丰富，神经活动持续时间也更久。

综上，本论文在具有较高的生态学效度的自然情境下，系统考察了汉英双语者在口语故事理解中的双语神经表征，揭示其在空间维度上既高度共享、又局部特异，在时间维度上既总体一致、又局部分化的综合特性，这反映了双语神经表征所具有的复杂性和异质性。对双语故事神经表征的层级性和动态性考察显示，两种语言在基础语音加工层面的神经表征存在较为显著的语言间差异，在意义整合和情境模型构建层面，两种语言的神经表征差异逐渐减小。“听-说”交互过程中的时间耦合特性和时间滞后特性揭示了双语故事神经表征的动态特征，其动态建构过程在不同时间延迟下存在显著的语言间差异，汉语故事相较于英语故事具有更丰富和更持久的神经表征建构特性。论文为解码大脑双语神经表征提供了新的思路，也拓展和丰富了双语神经表征问题的研究内涵。

Bilingual neural representation is a key problem in bilingual research, and brain imaging provides a powerful technical means for the study of this problem. Early studies focused on whether bilingual neural representation is “shared or separated”, mainly from the “static” and “single-brain” point of view to study bilingual neural representation, focusing on bilingual phonological, lexical or single sentence level processing. These studies aim to explore the relationship between bilingual neural representations by examining the brain functional activation patterns or functional connectivity patterns activated by different language processes. However, prior studies have rarely examined bilingual processing in natural contexts (such as story comprehension) , and failed to reveal the systematic, hierarchical and dynamic nature of bilingual processing in natural contexts. Besides, few bilingual studies have effectively exploited the temporal and spatial dynamic information of brain signals. This thesis focuses on bilingual story comprehension in natural contexts, and adopts a variety of brain function data analysis methods, including Representational similarity analysis (RSA) based on multivariate analysis, and Natural language processing (NLP) based on semantic computing technology. Using the “two-brain” paradigm and various data analysis techniques, this paper investigates the brain activities of the speakers and listeners in the state of “communication”, and explores the mechanism of bilingual neural representation through four studies.

In the first study, the neural representation and language effects in bilingual story comprehension were systematically investigated by means of brain activation, functional connectivity, brain network attributes and inter-subject consistency. The results showed that oral stories in both languages induced extensive and similar patterns of brain activation and functional connectivity. And there were significant differences between languages in the intensity of activity and the strength of functional connections between brain regions in some of the language areas, including the middle frontal gyrus and the inferior frontal gyrus, the superior temporal gyrus and the middle temporal gyrus, the anterior cingulate gyrus and the medial prefrontal cortex. However, there was no significant difference between the first and second languages in the topological properties of the whole brain functional network. Further combining ISC analysis and ISC feature-based functional partitioning method, it was found that there was a strong consistency between the two languages in the bilateral temporal lobe, frontal lobe and medial cuneiform prefrontal lobe. The ISC effect started from the primary temporal cortex, as it spread to other areas of the temporal lobe, parietal lobe, frontal lobe and other brain regions, the consistency between the subjects was weaker, and the variability between the subjects was stronger. Overall, both spatial and temporal dimensions of brain neural activity shows strong similarities between the two languages in the global patterns of story neural representation. However, there are some language specificities in certain brain regions.

In the second study, audio envelope extraction, semantic representation computation in Natural language processing (NLP) and Representational similarity analysis (RSA) were used. This study explores the neural representations of phonological processing and semantic processing and their linguistic differences in the process of bilingual story comprehension. The results showed that in the process of speech processing, the story comprehension of two languages was positively correlated with audio features in most brain regions of frontal lobe, temporal lobe and parietal lobe. Whereas, in most areas of the occipital lobe and medial cortex, there was a negative correlation with audio features. In semantic processing, in the superior temporal gyrus and the anterior part of the middle temporal gyrus, story comprehension are significantly correlated with semantic features. In the inter-language contrast of speech processing, the Chinese condition had stronger speech effects in the left inferior frontal gyrus and the left middle superior temporal gyrus, while the English condition had stronger speech effects in the bilateral insula. In the inter-language contrast of semantic processing, there are some inter-language differences under the loose threshold, but the differences would disappear under the strict threshold. With the deepening of information integration, from phonological processing to semantic processing, the difference of neural representation between the two languages gradually decreases.

In the third study, by using hierarchical story comprehension experiment, representation similarity analysis and story cross-level recognition method, the neural representation and language effects of situation model construction in oral story comprehension were investigated. The results showed that the Chinese sentence scrambled story could identify the content of stories across different levels based on the distributed activity patterns of the right middle frontal gyrus, the left inferior temporal gyrus and the left inferior occipital gyrus area. In the left inferior frontal gyrus and insula, English sentence scrambled story could significantly identify story content across different levels. The activity signals of the above-mentioned brain areas could not accurately recognize the content of the backward story. The study reveals the linguistic differences of neural representation constructed by cross-level situation models.

The fourth study used the neural coupling effect and time lag analysis to examine the neural representation in the speaking-listening “interaction” of bilingual stories. Neural coupling and time-lag analysis were used to analyze the difference of neural coupling patterns between listener and target speaker under different time-lags (0 s, 2 s, 4 s, 6 s and 8 s). The neural coupling response of the listener-speaker extends from the auditory region to the semantic and social cognitive processing regions. In the same delay time window, the neural coupling signals were stronger in Chinese than in English. The neural coupling effect of English story comprehension declines earlier than in Chinese. The dynamic features of neural representation construction in Chinese story comprehension is more abundant and the neural activity lasts longer than that in English.

To sum up, this thesis systematically examines the Chinese-English bilingual neural representation in oral story comprehension in natural contexts with high ecological validity. It reveals that the features of bilingual neural representation are highly shared and locally specific in spatial dimension and globally consistent and locally differentiated in temporal dimension, which reflects the complexity and heterogeneity of bilingual neural representation. The hierarchical and dynamic study of Chinese-English bilingual story neural representation shows that there are significant differences between the two languages in neural representation at the level of basic phonological processing. Whereas, at the level of meaning integration and situation model construction, the differences in neural representation between the two languages gradually decreased. The time-coupled and time-lagged features of the speaking-listening interaction reveal the dynamic features of the neural representation of bilingual stories. Compared with the second language, native stories have richer and more lasting features of story neural representation construction. This thesis provides a new way to decode the bilingual neural representation, and also expands the research angle on bilingual neural representation.