协作问题解决（Collaborative Problem Solving, CPS）能力是21世纪全球公民必备的一项关键能力，培养学生此项能力成为全球教育共同诉求，开展协作问题解决能力评估具有重要的教育意义和实践价值。然而，协作问题解决能力评估工作面临各种挑战：首先，协作问题解决能力本质复杂，难以清晰界定评估的具体内容；其次，该能力的评估依赖多样化协作问题情境的创设，已有评估工具难以支持教师开展常态化评估活动，不利于培养学生协作问题解决能力；第三，该能力的评估依赖学生的过程行为表现，已有的基于计算机的测评工具较少实现自动判分与实时反馈，尤其缺少对会话数据质性特征的分析，不仅降低评估效率和准确性，而且严重滞后干预的最佳时机。因此，迫切需要开展计算机技术支持下的协作问题解决能力评估相关的研究工作，借助智能技术积极探索协作问题解决能力自动化评估方法和开发支持日常教学活动的评估工具。
为破解关键问题，本研究从协作问题解决能力本质出发，依据社会建构主义理论、群体认知理论等构建了协作问题解决能力评估框架，设计并开发了数据驱动的协作问题解决能力评估系统，从系统评价和结果验证两方面确保评估的科学性和有效性，同时基于过程行为数据的事后分析探究了影响协作问题解决能力评估过程的因素，为后续类似评估项目的实施提供借鉴。本研究主要开展了以下三个方面工作：
（1）采用自上而下和自下而上相结合的方法构建了面向“个体-小组”的协作问题解决能力评估框架。在个体层面重新解构了协作问题解决能力，从认知、社会和元认知技能 三维视角定义个体在协作问题解决情境中的一系列外显表现，以此为证据推断个体的能力水平；在小组层面探寻了协作问题解决过程中的交互机制，从态度、行为和认知三个方面刻画小组协作问题解决过程中不断变化发展的涌现性状态，实现对小组整体表现综合评估；
（2）设计并开发了数据驱动的协作问题解决能力评估系统，实现面向“个体-小组”协作问题解决能力自动化评估流程，该系统提供了自定义的交互场景模板，支持教师自主创设评估任务。该系统在评估证据方面增加了会话内容质性特征，并提供一种可视化报告形式展示能力评估结果：在个体层面，基于MGRM模型以多维形式输出个体在认知、社会和元认知三个子维度上的能力值；在小组层面，从联结性、协调性、适应性、积极性和群智性五个子维度利用聚类分析算法确定小组协作问题解决能力等级；
（3）利用数据驱动的协作问题解决能力自动化评估系统开展了两轮准实验，设计了面向初中学生的评估任务，采用定量和定性相结合的混合研究方法，从系统的有用性、易用性、满意度、评估有效性等多个方面评价系统各项功能和验证评估有效性，并对实验采集到的数据进行事后分析，进一步探究机制与规律。研究结果表明，评估框架具有良好的结构效度和聚合效度，评估结果能够反映出个体和群体的真实水平，学生对于该评估系统的科技接受度比较高，通过个体和小组的可视化评估报告能够帮助学生更全面地理解个体和小组的能力水平，掌握自己的优势与不足，有利于培养学生的协作问题解决能力。
通过开展上述研究工作，本研究在以下三个方面取得了一定的创新：
（1）依据协作问题解决能力的多维性和交互性，基于理论演绎和数据分析构建了面向“个体-小组”的协作问题解决能力评估框架，拓宽了评估视角，为理解群体认知发展过程提供了参考依据；在个体评估框架中增加了元认知维度，突破了传统二维结构框架的限制，实现了评估框架理论的创新；
（2）探索基于过程行为数据的协作问题解决能力自动化评估方法，开发了集数据采集、存储、编码、能力推断和输出评估报告一体化流程的评估工具，提高了评估的效率，而且构建了会话数据的自动编码模型，丰富了评估证据，提升了评估的准确性，为后续开展基于过程行为表现的关键能力和核心素养的测评工作提供了可借鉴的操作模板；
（3）基于交互场景设计了通用型的数据采集模块，面向评估框架设计了数据编码规则，建立了底层行为数据与评估证据之间的关联，因此在保证评估有效性的前提下支持教师自主创设丰富的协作问题情境和开发评估任务，能够有效支持常态化评估活动的开展。

Collaborative Problem Solving (CPS) ability has been widely accepted as one of the necessary 21st century skills, thus cultivating such ability has sparked global interests, which highlights the need for the CPS ability assessment. However, the CPS assessment is facing a lot of challenges: first, given the construct of CPS skills is very complex, it is difficult to define the measurement framework; second, as the assessment should rely on the multiple CPS context, existing measurement tools could not support CPS ability assessment in daily teaching work; third, because the CPS ability assessment must rely on the student's process behavior data, previous measurement tools has not yet achieved automated scoring and immediate feedback, which not only lower evaluation efficiency and accuracy, but also seriously lag the best time for intervention. As a result, it is very urgent to carry out the research on the computer-supported CPS ability assessment, explore the automated scoring methods with intelligent technology and develop daily teaching-supported measurement tools.
In order to solve these problems, rooted in the nature of CPS skills, social constructivism theory, group cognition theory and so on, this study constructed CPS ability assessment framework, designed and developed a CPS ability automated evaluation system based on process-streaming data, conducted quasi-experiments to check the technological acceptance, reliability and validity of the evaluation system. Furthermore, this thesis identified the interaction mechanism and impact factors with post-analysis of process behavior data, which could provide more experience for the similar evaluation projects in the future. This research carried out the following three aspects:
(1) Constructing an "individual-group" oriented CPS ability assessment framework with a top-down and bottom-up approach. On the individual level, the construct of CPS skills was comprised of cognitive, metacognitive and social dimensions, with a series of explicit behaviors in CPS situations, which were the evidence to infer the individual's CPS ability. On the group level, in order to achieve a comprehensive evaluation of the group performance, the interaction mechanism of CPS was explored, and the emergent states during CPS term were represented in three aspects, namely attention, behavior and cognition.
(2) Designing and developing a data-driven CPS ability assessment sysytem, which achieved an "individual-group" oriented automated assessing process and provided an customized interactive scene templates to support teachers to creat assessment tasks independently; this system successfully offered automatic coding solutions to conduct qualitative analysis of the discourse data; after assessment there would be visualized reports immediately: for the individuals, CPS ability was interred with MGRM in terms of the cognitive, social, and metacognitive dimensions respectively; for the groups, CPS performance were identified by the clustering analysis algorithm with five sub-dimensions (positivity, connectedness, coordination, adaptability and group cognition).
(3) Implementing two rounds of quasi experiments with the data-driven CPS ability automated assessment system, designing four evaluation tasks for junior middle school students and checking the technological acceptance, reliability and validity with quantitative and qualitative research methods; besides, the data collected through the quasi-experiments were analyzed to further explore the mechanism and patterns. The results indicated that the students were satisfied with the evaluation system and the CPS ability assessment had good construct validity and convergent validity, and both the individual and group visualized feedback could help students to understand group performance and their own advantages and disadvantages, which would be beneficial for their CPS ability improvement.
Based on the above research work, this study has made some achievements in the innovations:
(1) Constructed "individual-group" CPS ability assessment framework with theory deduction and practical research based on the multidimensional and interactive nature of CPS,which expanded the evaluation perspectives, and provided the reference basis for people to understand the group cognition development; the metacognitive dimension was added to the individual assessment framework, which had broken through the traditional two-dimensional framework and achieved the innovation of the assessment framework.
(2) Explored the automated CPS ability assessment method based on process-streaming data and developed a CPS ability assessment tool integrating all the phrases, including data collecting, storing, encoding, scoring and report, which improved the evaluation efficiency. Besides, this tool achieved the automated analysis method of discourse data, which enriched the evidence and increased the evaluation accuracy. Thus, this research could provide a reference for the follow-up assessment of the critical 21st century skills based on process behavior data.
(3) Designed a general data-collecting module based on interactive CPS scenarios and data-encoding rules according to the "individual-group" CPS ability assessment framework, which mapping the process data to the evaluation evidence. Therefore, this module could enable teachers to formulate CPS ability assessment tasks independently and hold regular CPS ability assessments under the premise of ensuring the validity of evaluation.