中文题名: | 教学评估交互式测验编制及其过程数据分析:以抽样分布为例 |
姓名: | |
保密级别: | 公开 |
论文语种: | chi |
学科代码: | 04020005 |
学科专业: | |
学生类型: | 硕士 |
学位: | 教育学硕士 |
学位类型: | |
学位年度: | 2024 |
校区: | |
学院: | |
研究方向: | 过程数据分析 |
第一导师姓名: | |
第一导师单位: | |
提交日期: | 2024-05-31 |
答辩日期: | 2024-05-25 |
外文题名: | Development of Interactive Test for Teaching Evaluation and Analysis of Its Process Data: Taking Sampling Distribution as an Example |
中文关键词: | |
外文关键词: | sampling distribution ; process data ; interactive test ; teaching evaluation ; capability assessment |
中文摘要: |
统计教学评估对于培养满足社会需求的统计学人才尤为关键,随着计算机化测验的发展,教学评估也需要从传统考试分数的形式转化为一种改善学习的工具,提供关于学生学习方法和认知过程的更多信息。抽样分布的学习具有根本的统计学重要性,由于其涉及到抽象概念和抽样过程,在教学过程中需要结合计算机模拟的形式进行演示和评估,但是目前缺少合适的抽样分布模拟交互式测验,学生的过程数据也没有被充分地结合到评估结果中。基于计算机的模拟交互式测验在一些国际大型测验中已经得到了使用,但是由于模拟形式的过程数据存在操作状态难以穷尽、没有标准参考序列等特点,常用的过程数据分析方法较少地适用于这类数据,从而使得模拟过程数据所包含的信息没有被充分挖掘。 因此本研究从测验开发和数据分析方法应用两个角度出发,基于一个抽样分布模拟实验平台编制用于测量学生抽样分布能力的交互式测验,弥补课程教学中交互式测验的缺失。然后收集测验的结果数据和过程数据,选择合适的数据表征形式和过程数据分析方法对学生能力、类别特点进行评估,探索并验证适用于课程教学中模拟过程数据分析方法,拓展过程数据分析的数据类型和应用场景的边界。 研究一进行抽样分布交互式测验编制和信效度分析。基于抽样分布知识点和布鲁姆的认知层级理论设计初版题目,根据学生访谈和专家意见进行修改确认最终10道测验题目。对五所高校的心理学专业学生进行施测,得到388份有效数据。经过测量指标分析和信效度验证,删除了第3题和第7题,最终的8 道题测验各项指标良好,能够测量学生的抽样分布能力并用于后续分析。 研究二构建以过程信息为先验的能力评估模型。用变量+变量数值的形式表征每一次操作,再用多维尺度法对每道题的操作序列进行降维并保留3个特征变量作为过程信息。构建有过程先验的单维广义分部评分模型(GPCM)和无过程信息先验的GPCM模型,两个模型的能力估计值相关为0.985,有先验模型的能力估计值分布范围最广,估计更精确。两模型与考试成绩、动机、态度的相关没有显著差异,有先验模型的能力估计值与操作相似性、有效性指标的相关更高。证明以先验的形式加入过程信息能够为能力评估提供有限额外的操作信息。 研究三为基于过程信息的类别特点分析,主要为了详细探索过程中包含的信息含义。计算每个人操作序列之间的相似度矩阵,采用K-means聚类得到4个类别。比较4类学生的各项指标发现,4类学生的抽样分布能力估计值有显著差异,从趋势上来看逐类降低。通过对比单题得分、操作指标、动机和态度等变量,用抽样分布能力、探索意愿两个维度定义4类学生,针对性提出了一些原因分析和教学指导。说明模拟的过程数据包含了学生的能力和探索意愿等信息,并且可以通过相似度矩阵聚类的方法进行提取和分析。 本研究成功以抽样分布为例开发了教学评估交互式测验,并且有效利用过程数据评估了学生能力和类别,为教师提供了教学指导,为统计学高等教育常规教学评估方式的测验形式和数据分析拓展提供了实证研究的支持。 |
外文摘要: |
The evaluation of statistical teaching is particularly crucial for cultivating statistical talents that meet social needs. With the development of computer-based assessment, teaching evaluation also needs to transform from the traditional form of exam scores into a tool for improving learning, providing more information about students' learning methods and cognitive processes. The study of sampling distribution is of fundamental statistical importance, and due to its involvement of abstract concepts and sampling processes, it requires the integration of computer simulation in teaching demonstrations and evaluations. However, there is currently a lack of suitable interactive tests for sampling distribution simulations, and students' process data has not been fully incorporated into the evaluation results. Computer-based simulation interactive tests have been used in some international large-scale tests, but due to the characteristics of simulation-based process data, such as the difficulty of exhaustively capturing operational states and the absence of standard reference sequences, commonly used process data analysis methods are less applicable to such data, thus preventing the full exploration of the information contained in simulation process data. Therefore, this research approaches from the perspectives of test development and data analysis method application. An interactive test is designed based on a sampling distribution simulation experimental platform to measure students' ability in sampling distribution, compensating for the lack of computerized testing in course teaching. Then, the result data and process data of the test are collected, and appropriate data representation forms and process data analysis methods are selected to evaluate students' abilities and category characteristics, exploring and validating process data analysis methods suitable for course teaching, expanding the boundaries of data types and application scenarios for process data analysis. In Study 1, the development and reliability and validity analysis of the interactive test on sampling distribution were conducted. Initial questions were designed based on sampling distribution knowledge points and Bloom's cognitive taxonomy theory, and finalized through student interviews and expert opinions, resulting in 10 items. Psychology students from five universities were tested, yielding 388 valid data. After analyzing measurement indicators and verifying reliability and validity, item 3 and item 7 were removed, and the final 8 items showed good performance on various indicators, able to measure students' sampling distribution ability and used for subsequent analysis. In Study 2, an ability evaluation model with process information as a prior was constructed. Each operation was represented in the form of variable + variable value, and then the operation sequence of each question was reduced in dimensionality using multidimensional scaling, retaining 3 characteristic variables as process information. A single-dimensional generalized partial credit model (GPCM) with process priors and a GPCM model without process priors were constructed. The correlation between the ability estimates of the two models was 0.985, and the prior model had the widest range and more precise estimates. There were no significant differences in the correlation between the two models and test scores, motivation, and attitudes, but the ability estimates of the prior model had a higher correlation with operational similarity and effectiveness indicators. This demonstrates that incorporating process information in the form of priors can provide limited additional operational information for ability assessment. In Study 3, an analysis of category characteristics based on process information was conducted, aiming to explore in detail the information contained in the process. A similarity matrix was calculated between each individual's operation sequences, and K-means clustering was used to obtain 4 categories. Comparing the indicators of the four student groups revealed significant differences in their sampling distribution ability estimates, with a decreasing trend from one category to the next. By comparing variables such as individual question scores, operational indicators, motivation, and attitudes, the four student groups were defined in terms of sampling distribution ability and willingness to explore, and some targeted reasons and teaching guidance were proposed. This shows that the simulated process data contains information about students' abilities and willingness to explore, and this information can be extracted and analyzed using the method of similarity matrix clustering. This study successfully developed an interactive test for teaching evaluation using sampling distribution as an example, and effectively utilized process data to evaluate students' abilities and categories, providing teaching guidance for teachers and empirical research support for the development of test formats and data analysis in routine teaching evaluation methods for statistical higher education. |
参考文献总数: | 77 |
馆藏号: | 硕040200-05/24001 |
开放日期: | 2025-06-01 |