中文题名: | 引入时间信息的贝叶斯认知诊断建模:对作答时间和作答精度数据的联合分析 |
姓名: | |
保密级别: | 公开 |
论文语种: | 中文 |
学科代码: | 040202 |
学科专业: | |
学生类型: | 博士 |
学位: | 教育学博士 |
学位类型: | |
学位年度: | 2018 |
校区: | |
学院: | |
研究方向: | 认知与学习评价 |
第一导师姓名: | |
第一导师单位: | |
提交日期: | 2018-06-01 |
答辩日期: | 2018-06-01 |
外文题名: | Bayesian cognitive diagnosis modeling incorporating time information: Joint analysis of response times and response accuracy data |
中文关键词: | 认知诊断 ; 项目反应理论 ; 作答时间模型 ; 高阶潜在结构 ; 题目内特征依赖性 ; 属性层级 ; 多维潜在速度 ; 认知诊断模型 |
中文摘要: |
随着心理与教育测量技术的发展,多维化和计算机(网络)化的测评方式逐渐成为了关注焦点和现实需求。基于这两个发展趋势,已经出现了数据的多源化和多维化,进而心理与教育测评也开始踏入大数据时代。多维心理测量模型已成为当前主要的研究对象和应用方法,其中,适用于诊断性评估的认知诊断模型(cognitive diagnosis models, CDMs)受到了相对更多的关注。然而,限于传统建模理念和纸笔测验的数据收集方法,当前几乎所有CDMs只关注题目作答精度(response accuracy, RA)这种单一且传统的数据源,忽略了另一个相对重要的数据源:题目作答时间(response time, RT)。随着计算机成本的降低以及网络化程度的提高,在“互联网+测评”的时代里对RT的收集已成为了一种新常态。RT作为一个重要的过程数据,它能够最直接地反映被试的作答(工作)速度。合理地利用RT这一特性,往往能够帮助我们分析和判断一些仅通过RA难以获得的被试作答行为。因此,如何在认知诊断评估中同时利用RT和RA数据来对被试做出更为精细化更为全面的诊断是一个既具有理论意义又具有实践意义的议题。
全文围绕如何将RT引入认知诊断建模这一核心议题,共包括四个研究,研究一:联合认知诊断建模——对作答时间和作答精度数据的联合分析,研究二:引入题目内特征依赖性的认知诊断建模,研究三:贯序高阶潜在结构模型——对有高阶结构的层级属性的分析,和研究四:多维对数正态作答时间模型——对潜在速度多维性的探究。其中又以研究一为主要研究,之后延伸出研究二、三和四这三个并列研究作为研究一在不同角度的拓广。
研究一探究了如何将RT引入认知诊断建模这一核心议题,新提出了联合认知诊断建模方法。根据研究结果发现(1)新方法够同时对RT和RA数据进行有效的分析,为实证研究提供了理论基础和方法学指导;(2)认知诊断评估中引入RT信息能够提高参数估计的精确性,有助于更准确地给予被试诊断反馈。此外,还能够为诊断出被试的潜在作答速度,用于反映被试在解决问题速度方面的行为特征。
研究二探究了认知诊断建模时引入题目内特征依赖性(WICD)的必要性和益处。根据研究结果发现(1)认知诊断建模时引入WICD有助于进一步提高参数估计的精确性和效率;(2)在边际条件下,引入WICD的建模方法仍能提供与常规建模方法相媲美的参数估计返真性;(3)引入WICD的建模方法具有一定的参数估计稳健性,即当测验存在潜在的题目块时,其参数估计结果仍然较为精确和稳健。
研究三探究了如何在高阶潜在结构模型中处理属性层级结构,并新提出了贯序高阶潜在结构模型。根据研究结果发现(1)贯序高阶潜在结构模型能够实现在高阶潜在结构中处理有层级结构的潜在属性;(2)传统高阶潜在结构模型可视为贯序高阶潜在结构模型的特例,即当潜在属性之间不存在属性层级结构时两者等价;(3)对传统高阶潜在结构模型引入贯序过程对题目参数的估计几乎没有影响;(4)贯序高阶潜在结构模型的相对优势会随着属性层级结构松散度的增加而减小。
研究四探究了如何处理潜在速度多维性的问题,并新提出了多维对数正态RT模型。根据研究结果发现(1)多维对数正态RT模型能够实现对多维潜在速度的有效分析,为实证研究提供了理论基础和方法学指导;(2)在多维测验中,多维对数正态RT模型比传统单维对数正态RT模型对数据的拟合更好;(3)采用多维对数正态RT模型和单维对数正态RT模型分析数据会得到几乎一致的题目时间强度参数,但后者会低估题目时间峰度参数。
综上所述,本文通过对四个研究问题的详细探究,较为完整地探讨了“如何将RT引入认知诊断建模”这一核心议题。模拟研究结果表明新提出的方法具有良好的心理测量学性能;同时,实证数据分析结果表明新方法具有较高的实用性和充足的普适性。
﹀
|
外文摘要: |
In recent years, advances in psychometrics have focused on measuring multidimensional latent construct to provide more detailed and refined feedback to respondents. Cognitive diagnosis has received great attention. One of the main objectives of cognitive diagnosis is to evaluate respondents’ mastery status of latent skills or attributes (e.g., mastery or non-mastery) and then provide diagnostic reports to teachers or clinicians to help them make decisions regarding remedial instruction or targeted interventions.
Despite that numerous cognitive diagnosis models (CDMs) have been developed, all these models only utilize information in item response accuracy (RA). An important source of information about respondent’s behavior, item response time (RT) to items, is often ignored. CDMs are usually applied to response collected in power tests without time limit, where it is assumed that only latent attributes account for item performance. However, pure power tests (unlimited time) are rarely observed in practice. In addition, ignoring RT may be partially due to the difficulty in collecting RT data in paper-and-pencil tests at the individual item level. Nowadays, with advances in computer-based tests, in the age of “Internet + Measurement”, RT collection has become a routine activity in many large-scale and small-scale tests. RT reveals the working speed information of each respondent. For example, when respondents are not motivated in a low-stakes test, they may respond to items in a speeded manner, or respondents with prior item knowledge might have shorter RT. All these responding behaviors may not be easily identified only based on RA. Information embedded in RTs could be collateral information and be utilized jointly with RA to obtain more refined and accurate diagnosis in cognitive classification.
In this dissertation, four studies were conducted to figure out creative ways to incorporate RT into cognitive diagnosis modeling. In study 1, a joint cognitive diagnosis modeling approach for jointly analyze RT and RA was proposed; In study 2, a modified cognitive diagnosis modeling approach incorporating within-item characteristic dependency was proposed. In study 3, a sequential higher-order latent structural model for hierarchical attributes was proposed. In study 4, a multidimensional lognormal RT model for multidimensional latent speed was proposed. The first study is the main study in this dissertation, and other three studies can be seen as extensions of study 1 from different perspectives.
In study 1, to provide more refined diagnostic feedback with collateral information in RT, the joint modeling of attributes and response speed using RT and RA simultaneously for cognitive diagnosis was proposed. For illustration, an extended deterministic input, noisy “and” gate (DINA) model was proposed for joint modeling of RT and RA. Model parameter estimation was explored using the Bayesian Markov chain Monte Carlo (MCMC) method. The PISA 2012 computer-based mathematics data were analyzed first. These real data estimates were treated as true values in a subsequent simulation study. A follow-up simulation study with ideal testing conditions was conducted as well to further evaluate model parameter recovery. The results indicated that model parameters could be well recovered using the MCMC approach. Further, incorporating RTs into the DINA model would improve attribute and profile correct classification rates and result in more accurate and precise estimation of the model parameters.
The proposed joint CDM in study 1 assumes that item parameters follow a multivariate normal distribution. In other words, it assumes that dependencies exist among different types of item characteristics/parameters within an item, which is referenced as the within-item characteristic dependency (WICD). As an uncommon method, the necessity and advantage of incorporating WICD need further study. To explicitly modeling WICD, study 2 proposed a modified Bayesian DINA modeling approach where a bivariate normal distribution was employed as a joint prior distribution for correlated item parameters (i.e., guessing and slip parameters). Simulation results indicated that explicitly modeling WICD improved model parameter estimation accuracy, precision, and efficiency. Additionally, when potential item blocks existed, the proposed modeling approach still demonstrated good performance and high robustness. Further, the fraction subtraction data was analyzed to illustrate the application and advantage of the proposed modeling approach.
The higher-order latent structural model (HO-LSM) was employed in the proposed joint CDM in study 1. Regular HO-LSM assumes that attributes are structural independent. However, cognitive research suggests that cognitive skills should not be investigated in isolation. The attribute hierarchy specifies a network of interrelated attribute mastery process. To our knowledge, it is still impossible to integrate the regular HO-LSM and attribute hierarchy simultaneously. To address this issue, study 3 proposed a sequential HO-LSM by incorporating various hierarchical structures into higher-order models. The feasibility of the proposed HO-LSM was examined using simulated data. Results showed that with hierarchical attributes, the DINA model using the sequential HO-LSM produced considerable improvement in person classification accuracy compared with that using the traditional HO-LSM. An empirical example was presented as well to illustrate the applications of the proposed HO-LSM.
In educational and psychological measurements, latent speed can be defined as a rate of the amount of labor performed on the items with respect to time. As different dimensions may require different kinds of labors, latent speed may also be multidimensional. However, only a unidimensional RT model was used in the proposed joint CDM in study 1. Because of the lack of multidimensional models for RTs to take account of the potential multidimensionality of latent speed, only the relationship between multiple latent attributes and one single latent speed can be evaluated. To capture the multidimensionality of latent speed, study 4 proposed a multidimensional lognormal RT model. The PISA 2012 computer-based mathematics data were analyzed at first to illustrate the implications and applications of the proposed RT model. The results confirmed multidimensionality in the latent speed. A brief simulation study was conducted to evaluate the parameter recovery of the proposed RT model and the consequences of ignoring the multidimensionality.
﹀
|
参考文献总数: | 0 |
作者简介: | 詹沛达,博士,毕业于北京师范大学中国基础教育质量监测协同创新中心,读博期间赴美国University of Maryland, College Park的Department of Human Development and Quantitative Methodology联合培养。目前的主要研究方向是认知与学习评价,研究内容包括:心理与教育测量的理论和方法(e.g., 项目反应理论、认知诊断评估、作答时间建模、计算机化自适应测验、学业质量评价、大规模基础教育质量监测理论与技术等)和心理统计与应用(结构方程模型及其应用、儿童青少年社会性发展等)。目前已在British Journal of Mathematical and Statistical Psychology、Applied Psychological Measurement、Frontiers in Psychology、《心理学报》和《心理科学》等国内外主流权威期刊发表论文10余篇;参与编写英文学术著作2部;并担任Applied Psychological Measurement、Journal of Educational Measurement、《心理学报》和《心理科学》等期刊的审稿专家。 |
馆藏地: | 图书馆学位论文阅览区(主馆南区三层BC区) |
馆藏号: | 博040202/18002 |
开放日期: | 2019-07-09 |