- 无标题文档
查看论文信息

中文题名:

 开放式教师胜任力情境判断测验的开发与自动评分研究    

姓名:

 徐静    

保密级别:

 公开    

论文语种:

 中文    

学科代码:

 045400    

学科专业:

 应用心理    

学生类型:

 硕士    

学位:

 应用心理硕士    

学位类型:

 专业学位    

学位年度:

 2020    

校区:

 北京校区培养    

学院:

 心理学部    

第一导师姓名:

 骆方    

第一导师单位:

 北京师范大学心理学部    

提交日期:

 2020-06-25    

答辩日期:

 2020-06-08    

外文题名:

 A STUDY ON THE DEVELOPMENT OF SITUATIONAL JUDGMENT TEST OF TEACHER COMPETENCY BASED ON AUTOMATIC SCORING SUBJECTIVE RESPONSES    

中文关键词:

 教师胜任特征 ; 情境判断测验 ; 主观题 ; 自动评分    

中文摘要:

教师胜任力研究对于教师队伍的招聘、绩效管理、职业发展等方面具有重要意义,本研究基于McClelland对胜任力的定义以及Spencer的冰山模型理论,运用行为事件访谈法以及文献分析法对教师胜任力核心特征进行建构,对12名一线教师进行深度访谈,广泛收集教学中的典型问题情境和解决办法,确立学生导向、问题解决、情绪智力、学习力、成就动机五大教师胜任力一级维度以及十三个二级特征,在此基础上开发了20道教师胜任力情境判断测验。为了降低猜测效应和称许性、给予作答者更自由的作答空间、获得更丰富的作答信息,测验采取开放式的自由反应形式,对作答文本分别进行人工评分和机器自动化评分,并比较二者的一致性。经统计分析,题目整体难度系数为0.63,各题的区分度在0.4500.631之间,区分度较好。测验总体内部一致性信度为0.877,五大一级维度内部信度为0.3720.728之间,使用Mplus作验证性因素分析,发现测验的CFITLI值分别为0.9610.954RMSEA值为0.032SRMR值为0.041,各项指标拟合情况较好,各项目在各因子上的因子载荷值为0.4200.623之间。对效标关联效度进行检验,发现测验与教学诊断环节中的教学设计、课堂评价、学生作业存在显著相关。

研究二部分主要进行了自动评分方法的探索,分别比较基于文段的多标签分类方法和句子层面的文本多分类方法对个体分数预测的效果,分析不同分类模型在分类任务上的实验性能,对比了CNNRNNLSTMC-LSTMRNN+attentionLSTM+attention等多个深度模型的实验结果,最终采用卷积神经网络CNN对数据集进行分类预测。该模型在二十道题上的分数预测准确率在70%88%之间,准确性较高,人机评分的总分一致性达到0.951,等级评价分数人机评分完全吻合的人数占比79.79%。在学生导向、问题解决、情绪智力、成就动机四个维度上人机评分的相关系数都在0.9左右,在学习力这一维度上相关略低(r =0.78 , p.01)。接下来,把机器在327份文本上的预测分数进行统计分析,结果表明,所有题目的内部一致性信度为0.906,五个维度间内部一致性系数为0.873,验证性因素分析结果显示,因子间结构清晰,CFI值为0.960TLI值为0.953,模型各项指标拟合情况较好。选取教师平时的绩效指标“优秀班主任”作为效标来检验效标关联标度,结果表明,普通教师与优秀教师在本测验上的表现存在显著差异。

外文摘要:

Situational judgment test is an effective test form of teacher competency assessment. This paper based on McClelland’s definition of competence and Spencer’s iceberg model theory, used behavior event interview and literature analysis to construct the core characteristics of teacher's competence, conduct in-depth interviews with 12 teachers, and collect the typical problem situation and solutions in teaching extensively. This study developed twenty teacher competence situational judgment tests with the five first-level dimensions and 13 sub-dimensions , first-level dimensions include student orientation, problem solving, emotional intelligence, learning ability and achievement motivation. In order to reduce the guessing effect and social desirability, give respondent more space to answer and obtain more information, the test take an open free response form. The response text was scored by manual and machine respectively and compare their consistency. Through the analysis of the results, it is found that the difficulty of the test is 0.63, the degree of discrimination of questions in the test are between 0.450 and 0.631.It’s also found that the internal consistency reliability of the test is 0.877, the reliability of the subscale is between 0.372 and 0.728. The structural equation analysis using Mplus reveals that the CFI and TLI value is 0.961 and 0.954, RMSEA value is 0.032, SRMR value is 0.041, the indicators fit the model qualified. There is a significant correlation between test and teaching design, class evaluation and homework in teaching diagnosis part.

The second study mainly explored the automatic scoring method, compared the effect of the multi-label classification method based on the text segment and the text multi-classification method based on sentence level for individual score prediction, analyzed the experimental performance of different classification models on classification tasks, compared the experimental results of CNN, RNN, LSTM, C-LSTM, RNN-attention, LSTM-focus and other models. Finally, selected CNN to classify the text prediction, the score prediction accuracy are between 70% and 88% on 20 questions. The results show that the similarity between the machine and the coder is 0.951. The proportion of man-machine score in the rating score is 79.79%. In the four dimensions of student orientation, problem solving, emotional intelligence and achievement motivation, the correlation of man-machine score is about 0.9, and slightly lower in the dimension of learning ability (r=0.78, p <.01). Next, the automatic score of the machine on 327 texts is analyzed statistically, the results showed that the internal consistency coefficient of all the questions was 0.917, the reliability of the subscale is 0.873. The structural equation analysis using Mplus reveals that the CFI and TLI value is 0.960 and 0.953, the indicators fit the model qualified. Selected the teacher's usual performance indicators "excellent class teacher" as the criterion, the results showed that there are significant differences between the competence performance of ordinary teachers and excellent teachers in this test.

 


馆藏号:

 硕045400/20181    

开放日期:

 2021-06-25    

无标题文档

   建议浏览器: 谷歌 360请用极速模式,双核浏览器请用极速模式