Emotion Classification and Achievement of Students in Distance Learning Based on the Knowledge State Model

By inergency On Jan 28, 2023

Emotion Classification and Achievement of Students in Distance Learning Based on the Knowledge State Model

1. Introduction

The COVID-19 outbreak that began in 2019 has greatly affected the economic life of society. According to current teaching arrangements, students preview classes by watching a course video, participate in the teacher’s courses through remote check-in to form attendance records, complete exercises to consolidate knowledge, and r collaborate to build knowledge through interactive communication such as discussion, “likes”, and evaluation. However, long-term online learning faces difficulties, which include low continuous participation and poor interaction among students. In the context of digital teaching, related research on educational data mining and intelligent teaching assistance systems has become a hot issue in the field of educational technology in recent years [1]. Accurate perception of students’ emotional states in online learning is crucial to personalized learning and is a prerequisite for cracking the “emotional deficit” in current online education [2]. In recent years, the continuous improvement of Internet of Things technology, sensor technology, and big data computing power has provided technical support for learning behavior analysis. Learning emotion detection based on intelligent technology has also attracted widespread attention from scholars [3]. Expression is not only the external manifestation of the activities of the inner world but also a physiological response to the external world [4]. In the actual classroom-style teaching environment, teachers can observe the subtle changes in students’ faces to infer the students’ inner world. For instance pouting their mouths may express disgust for the teacher’s teaching content [5], closing their eyes may indicate that the learners are in a state of fatigue [6], and so on. In classroom teaching, teachers and students can keep abreast of students’ learning status through face-to-face communication. However, in distance teaching, due to the separation of time and space, teachers know nothing about students’ learning emotions and cannot provide students with targeted teaching strategies. Therefore, it is critical to add an emotional cognition function to the network teaching system [7].

In recent years, the effectiveness of computer vision-based emotion recognition methods has been extensively verified. Fatahi et al. developed a self-assessment system with emotional cognition, which can analyze learners’ various emotional states and cognitive situations during the learning process [8]. Sloep et al. established a personalized network learning system by studying the learning behavior, learning emotion, and learning state of learners in the network teaching system [9]. Professor Noori F designed a network teaching model that considers learners’ personality characteristics and emotions [10]. BITS at the University of Regina in Canada has developed an intelligent teaching system that provides intelligent tutoring for primary learners using Bayesian technology [11]. Boban et al. proposed a personalized teaching model based on learning style recognition and a hybrid recommendation teaching strategy, which can provide learners with personalized teaching strategies [12].

To sum up, there have been significant advances in the automatic perception of emotion in distance learning. The learning state of learners can be predicted based on the real situation of each facial feature. Finally, according to the predicted results, an adaptive learning system is adopted to automatically adjust learning strategies, recommend learning resources, and provide early warnings. However, there are still many problems in the research on emotion classification of distance-learning students based on knowledge state model. For example, emotion recognition models are less accurate in identifying emotions such as pleasure, concentration, confusion, and boredom. Facial video-based emotion recognition methods have a lot of room for improvement. The use of single mode instead of fusion multi-mode signals leads to weak generalization ability of the model, and the knowledge state model is not widely used. In this study, a learner knowledge state model integrating learning emotions under the background of digital teaching is proposed, and a multi-dimensional online classification model of students’ emotions is constructed based on ResNet 18 neural network. Two modules of feature embedding and feature aggregation are added to identify students’ cognitive emotional states in distance learning. Finally, this study analyzes the correlation between the emotion classification of distance learning students and their grades, and verifies the validity of the emotion classification model in distance learning.

2. Related Works

Psychologists’ research shows that emotions greatly affect perceptual choices, memory, and thinking activities in the human cognitive process [13]. Cognitive activities, such as causal reasoning, goal evaluation, and planning processes, all accompany emotion [14]. Therefore, sensing the emotions of learners and providing timely positive help can effectively improve the cognitive ability of learners [15].

Pekrun first proposed the concept of learning emotion in 2002 and defined it as an emotion that is directly related to learning, teaching, academic achievement, etc. [16]. Pekrun also proposed the control value theory to analyze the relationship between achievement and learning emotion [17]. Efklides encodes seven learned emotions, including pleasure, curiosity, confusion, anxiety, depression, boredom, and surprise [18]. Critcher believes that learning is an “attention-emotion” model that combines cognitive and emotional experience, and that emotion has a direct impact on the human learning process and other functions [19]. Kort proposes a comprehensive four-quadrant model that explicitly links learning and affective states. This theory divides academic emotions into four types with valence as the horizontal axis and learning degree as the vertical axis. A typical learning experience includes a series of emotional experiences, that is, with the deepening of learning, emotions should fluctuate [20]. Ryan studied the frequency, persistence, and impact of boredom, frustration, confusion, preoccupation, happiness, and surprise among students with different populations, different research methods, and different learning tasks as variables [21].

In recent years, educational researchers have measured the emotional state of students based on data mining, computer vision, and other methods, and the effectiveness of computer vision-based emotion recognition methods has been extensively verified. Myers built a convolutional neural network to detect the three emotional states of boredom, engagement, and neutrality of students [22]. Based on facial expressions and heart rate information in videos, Chen detected learners’ emotions during the writing process and achieved significant improvements in robustness and accuracy [23]. It can be seen that learners experience many different types of emotions in the learning process. The differences in the valence and learning level of these emotions and the emotional state is not fixed and changes with the learning process. The influence of emotion on learning has also been widely noted, and Rowe et al. have done some empirical studies to confirm that learning emotion is related to differences in the performance of learners in the short term [24].

In addition to this, earlier studies attempted to explain human behavior during learning by understanding learners’ emotional and cognitive processes, proposing the concept of a knowledge state model. For example, Alepis proposed a multi-criteria theory combined with bimodal emotion recognition and applied it to the mobile education system [25]. Yang proposed an accurate definition of the knowledge state model. He believed that the dimensions of the knowledge state model should correspond to various aspects of the students in the real learning environment, and the attributes of the knowledge state model should represent the characteristics of the students in the real learning environment [26]. Pavlik believed that the knowledge state model represented the characteristics and corresponding levels of students, including knowledge skills, cognitive behavior, emotional experience, etc. [27]. Bontcheva proposed that knowledge modeling is inseparable from knowledge features, such as the definition of knowledge components contained in knowledge points, the mapping between projects and these knowledge components, and project difficulty, etc. [28]. Abyaa believed that those constructing a knowledge state model should first identify and select the appropriate characteristics that affect learners’ learning, then consider the learner’s mental state in the learning process, and finally select the appropriate modeling technology to simulate the optimal state of each feature [29].

The above studies contain some innovations in content and perspective, but they represent preliminary exploratory research, in terms of a knowledge state model that integrates learning emotions, and there are still many deficiencies. Due to the complexity of learning emotions and the diversity of learner models, this study focuses on the following three issues:

(1): How to effectively classify students’ emotions according to facial images in distance learning?
(2): How to build a knowledge state model integrating learning emotion?
(3): How to explore the characteristics of knowledge state model integrating learning emotion?

4. Knowledge State Model with Learning Emotion

This study establishes a learner knowledge state model that takes learners’ learning emotions into consideration, which can effectively simulate real learning situations, be closer to learning reality, make more accurate predictions of learners’ knowledge state, reflect students’ current learning state, provide a more accurate basis for action for real educators to conduct academic early warnings and adjust teaching strategies, promote the realization of personalized teaching methods, and provide future education big data. The development of an intelligent teaching assistant system provides new modeling methods and practical reference.

Assume that the logical function of the correct probability of the learner’s nth attempt is:

$p_{i j} = \frac{1}{1 + e^{- s_{i j}}} \times \frac{1}{1 + e^{- z_{i j}}}$

(7)

where,

$z_{i j} = θ_{i} + \sum_{k ϵ K C s} q_{j k} (β_{k} + γ_{k} \times o p p (k, i))$

(8)

Emotions are expressed as:

$S_{i j} = w_{0} + \sum_{k ϵ K C s} w_{k} x_{k}$

(9)

In this model, students’ emotion classification modeling in distance learning is used as a weight variable for learning results.

X_{k}

represents one of the emotions of pleasure, focus, confusion, and boredom, and

w_{k}

represents the weight of the corresponding emotion. Emotion

X

uses Boolean representation to indicate the possible influence of emotion on learning results in the learning process. For example, the learners’ confusion emotion

X

(1 = confusion; 0 = no confusion) in the learning process is expressed as

S_{i j} = w_{0} + w_{1} x_{1} + \dots + w_{k} x_{k}

in the learner model. W means that the possible influence of learners’ confusion on knowledge point learning will change with the changes in learners’ individual differences, number of learning items, and learning state.

Worcester Polytechnic Institute. This data set contains data on learning behavior and learning emotion. This data set was used to test the effectiveness of the proposed model. Students’ behavior data and emotional data in the learning process were refined, and five learning characteristics were selected.

The names and corresponding descriptions of the learning characteristics are as follows:

Analytical view: this feature indicates that learners actively click during the learning process;

Correct or incorrect first attempt: this feature indicates whether the learner’s first attempt at solving problems is correct or not;

Number of learning knowledge points: this feature represents the number of different knowledge points learned by learners;

Total attempts: this feature represents the total attempts of learners in the learning process;

Confusion emotion: this indicates the average probability of each confusion emotion.

After defining the above learning characteristics, this study compiled statistics on the frequency of the above five characteristics and learners’ learning time respectively, and finally obtained a data set containing information such as student number, five learning characteristics, and learning duration.

When the the samples with missing learning data of a single subject at the same knowledge point are removed, we obtain the data description of students’ learning behavior, as shown in Table 1.

Through the analysis of 543 pieces of data, we can see that students felt confusion 197 times in the process of answering questions. The average number of perplexities of a single student was 7.04, and the average number of correct answers was 9.04. These two values are relatively low. This may be because the students do not fully understand the knowledge points investigated, do not feel confused, or do not choose the correct answer in the process of answering. However, this does not rule out the possibility that that the students are in a negative state in the process of learning.

This experiment set a specific data sample loading method. Each sample contained T frames. For video sampling, T was set to 3 by default during training. In this study, cross entropy loss was selected as the loss function of model training, and random gradient descent algorithm was used to optimize. Momentum was set to 0.9 and weight attenuation was 1 × 10⁻⁴. The principle of random gradient descent algorithm can be expressed as follows:

$w_{t + 1} = w_{t} - η \frac{1}{n} \sum \nabla l (x, w_{t})$

(10)

n

is the batch size,

η

is the learning rate. It can be seen that learning rate and batch size are two very important parameters that affect gradient descent. In order to get the best model, this study conducted parameter adjustment experiments, and trained 100 epochs for each parameter setting. In order to save the best model in the training, each epoch was trained, and the verification set was used to verify once. When the accuracy of the verification set increased, the model was saved, and the best model obtained from the training was selected for evaluation on the test set. For learning rate, 1 × 10⁻⁴, 4 × 10⁻⁴, and 1 × 10⁻⁵ were set successively. For batch size, 8, 32, and 64 were set. The result of parameter adjustment is shown in Figure 3.

The abscissa represents different learning rates, and the ordinate represents different batch sample sizes. It can be seen that when the learning rate is set to 1 × 10⁻⁵ and the batch size is 32, the training model has the highest accuracy. After parameter adjustment, the best model is saved.

On the basis of the classical knowledge state model, this study proposes a knowledge state model that integrates learning emotions, adjusts the parameter composition in the algorithm, and uses the linear combination of parameters to represent the characteristics of learning emotions, making up for the lack of consideration of the basic knowledge state model in the learners’ learning emotions.

6. Conclusions

Under the background of digital teaching, the rapid development of data mining technology has laid a good technical foundation for personalized teaching. This study constructs an online classification model of students’ multi-dimensional emotions based on ResNet 18 neural network, innovatively adds two modules of feature embedding and feature aggregation, and uses frame attention network to extract frame level features closely related to students’ emotions. The average recognition accuracy of four cognitive emotions, namely pleasure, focus, confusion and weariness, reaches 88.76%. This study uses a mathematical modeling method to add emotional factors as parameters to the modeling of learners’ knowledge state, analyzes the correlation between students’ emotional classification and their scores in distance learning, verifies the effectiveness of students’ emotional classification model in the application of distance learning, and finds that there is a significant correlation between focus, confusion, boredom and learning results. In addition, this paper puts forward a new direction of thinking about teaching intervention. The study provides technical support for distance learning emotion classification and early warning, which is of great significance in helping teachers understand students’ emotional state in distance learning promptly, and in promoting students’ in-depth participation in the distance learning process.

In order to improve the research findings, future work will include:

(1): Further analysis and discussion of the integration of other learning emotions and learner models, and finding more good emotional features that can predict learning results in the learning process;
(2): Learning more methods to improve models, realizing the potential of models that comprehensively consider multiple learning emotions, enhancing the completeness of models that are more suitable for online learning in secondary schools, improving the reliability of learner models that integrate learning emotions, and developing learner knowledge.