Measuring and improving students’ engagement

by Stephanie Chasteen on November 2, 2014

I’ve been working over the last year or so to better understand how to promote student buy-in to interactive techniques such as clickers and group work.  That work resulted in a set of resources on how to “frame” students’ roles in the class, especially in the first week.

Now I’ve been delving deeper into this project — having gotten a little bit of money from our Chancellor for this research — and I’m finding a lot of interesting ideas in the educational psychology and learning sciences literature that is very informative.  I’m reframing my topic as “productive engagement” — rather than buy-in.   I don’t just want students to not be resistant;  I also want them to actively engage with the techniques and understand their value.

I’m eventually planning to pull all this together into:

  • A survey so instructors can assess the level of student engagement in their course (and if there are problems, where those problems are)
  • A set of recommendations for instructors to consider if they do have particular problems with student engagement, based on the literature and on actual activities that I’ve collected from master teachers

I recently wrote up some of my thoughts on how this topic relates to existing literature, so I thought I’d share it here.  Comments and recommendations welcome!  If you want to see a copy of my pilot survey, please let me know.

——

 

Defining productive engagement

Productive engagement describes a positive attitude and behavior in a classroom such that students

  1. participate in activities,
  2.  value these activities, and
  3. are emotionally engaged in these activities.

Productive engagement is related to, but not equivalent to:   (a) motivation (this is an antecedent to engagement), (b) satisfaction with the class (this is a broader idea than engagement), (c) comfort and safety (some active learning techniques create frustration or discomfort, even among students who are actively engaged).

How did I arrive at these three dimensions?   “School engagement” has been defined in the K12 school literature as having behavioral, emotional, and cognitive components.[1] While “school engagement” is broader than the “productive engagement” construct (including aspects such as whether the students follow the rules of the classroom, or engage in extra-curricular activities), it is still a very informative area with a broad research base. A review article by Fredericks [1] describes the current state of the engagement literature, and I draw from that work here, which strongly influenced my construct map.

Behavioral engagement is equivalent to participation – students’ conduct, adherence to norms, absence of disruptive behaviors, effort, persistence, concentration, and contributions to class discussion. Behavioral engagement is typically measured through a scale of conduct, persistence, and/or participation.

Emotional engagement refers to students’ emotional reaction to school and the teacher: interest, boredom, happiness, sadness, anxiety, feeling of belonging, value and success. Fredericks points out that the literature has not focused on what the source of the emotional engagement might be – is it situational or personal interest? Fredericks indicates that ideas of value – interest, identity, importance of the activity, and cost of engaging – are typically included within emotional engagement. Emotional engagement is typically measured through identifying students’ emotions, work orientation, and interest.

Lastly, cognitive engagement focuses on students’ investment in learning, and has strong overlap with various constructs in the motivational literature, such as intrinsic/extrinsic motivation, and self-regulation. Cognitive engagement is typically measured by studying students’ ability to use flexible problem-solving techniques, their preference for hard work or for independent work styles, and their ability to manage their effort.

Another area that has informed my thinking is that of Productive Disciplinary Engagement (PDE)[2]. Productive disciplinary engagement is described as students participating in an activity, responding to one another, using goal-oriented activities that help them demonstrate the skills and understanding of a discipline. Admittedly, my exposure to PDE came late in my definition of my construct, and further exploration of this literature will be part of the continuation of my project.

The antecedents of productive engagement

One particular challenge in creating this construct map was to clearly separate the components of productive engagement from its’ antecedents. There is much advice in the educational psychology literature on how to create motivating, engaging classroom environments[2,3]. This literature on motivation has such a strong overlap with the literature on engagement that it has sometimes been difficulty to identify whether I’m measuring something different from motivation. Motivation is defined as the “psychological processes that direct and sustain students’ behavior toward learning.” (Moreno, pp 328-329). Given that definition, I would argue that motivation is the antecedent to engagement, rather than engagement itself.

However, I have found the motivation literature to be critical in helping me to define engagement, as motivation defines the inputs that result in the outcomes that I am interested in. Thus, I review this literature briefly here, as it has informed my construct map.

Students feel motivated when they feel capable, when they understand what actions will lead to success, when they understand the purpose of the learning activity, when they see the activity as having value and interest, when they have positive emotions about the activity, and when they deal effectively with obstacles (see Boekaerts [3] for a useful review). Textbook treatments of motivation in school (e.g., Moreno) break motivation into behaviorist, cognitive, and sociocultural components. Behaviorist theories recognize that we seek rewards and avoid punishments. Cognitive theories focus on our thoughts, beliefs, expectations and attitudes: This includes interest, goal, and self-determination theories. Sociocognitive theories combine the cognitive and behaviorist approaches: Students’ thoughts and attitudes combine with the learning environment to give rise to their motivational stance: This includes expectancy/value theory, attribution theory, and self-efficacy. Additionally, students are more motivated to engage when they experience positive emotions towards learning activities (and conversely, less likely to attend to learning when they experience negative emotions)[2] : Positive emotions are often described as fulfilling the psychological needs of competence, autonomy, and relatedness.

The Productive Disciplinary Engagement literature has also focused on the antecedents to creating PDE in the classroom, and the seminal work in this area7 postulates four main principles for fostering PDE: Problematizing subject matter (encouraging students’ intellectual contributions), giving students authority to address such problems, holding students accountable to others and to shared disciplinary norms, and providing students with relevant resources. These suggestions have clear connection to the sociocognitive motivational literature, such as self-efficacy and creating expectancies among students.

The final construct

Thus, I would like to justify the three dimensions of my construct, with this literature in mind.

  1. Participation is equivalent to the behavioral engagement in the School Engagement literature, and is an observable outcome of both PDE and motivation.   The two remaining components of my construct both fall within the area of “emotional engagement” as described in the School Engagement literature[1]
  2. Value is an aspect of engagement identified within the engagement literature, and is generated through thoughts, beliefs and expectations as described by cognitive theories of motivation9. Thus, it is an emotional aspect of engagement, with largely cognitive antecedents.
  3. Emotional engagement are the other aspects of engagement that relate to the students’ experience with the task and with the teacher. Thus, this is related to the non-value aspects of emotional engagement defined in the school engagement literature[1], and the whether the psychological needs of the students are being met. Emotional engagement is largely impacted by factors identified in the sociocognitive theories of motivation. I note that this third catetegory is perhaps the least well-defined, and may be mixing antecedents of engagement with actual engagement. Results of the pilot study will help discern the utility of this measure.

I did not include measures of cognitive engagement (as defined in Fredericks) – intrinsic/extrinsic motivation and self-regulation – as these are person-side characteristics that are less likely to be affected by a single classroom experience. The survey also does not use the sociocultural or sociocognitive theories of learning in great depth – mostly in an attempt to restrict the scope of the project.

Construct Map

Here I present the final construct map.

PARTICIPATION

Participation is a measure of behavioral engagement, which is impacted by various motivational factors. It ranges from the most enthusiastic, participatory students who attend deeply to the tasks, to students who are more passive and give up easily, to those who are actively resistant and disruptive.

 High engagement

Respondents
  • Actively participate, or attempt to actively participate
  • Are animated and engaged, attentive
  • Participate readily
  • Are persistent
  • Are passive during activities
  • May require prompting to participate
  • Are not persistent
  •  Avoid participation (e.g., engage in other activities, such as texting)
  • May be disruptive to peers
  • Do not respond to prompting
  • Don’t even try

High resistance

 

 

VALUE

Value is a measure of the degree to which the student understands and accepts the rationale for the activity, and feels that they are useful to his/her learning. This is mainly impacted by cognitive factors, and is what people usually mean by “buy-in” (the original intent of this study). The construct ranges from those who can fully articulate the value of the activities and think that the activities help their learning, to those are either conflicted or neutral (“whatever”) about the value of the activities, down to those who are fairly resistant to doing such activities because they are not seen as valuable. This area of engagement is likely to be strongly impacted by the quality of the task itself, as well as the effectiveness of the instructor’s facilitation.

Respondents
  • Recognize the value of the activities
  • Feel the time used for activities is beneficial
  •  Are not sure that they recognize the value of the activity
  • Are unsure whether the time on the activities is beneficial at all
  •  Disagree with the rationale for the activity
  • Feel that the time spent on the activities could be better spent doing other things (e.g., lecture)

 

EMOTIONAL ENGAGEMENT

Emotional engagement is the degree to which the student feels positively about the task, instructor, and classroom environment. Is the student experiencing positive emotions, feeling connected to classmates, building their confidence, and feeling in control of their learning? Students may be relatively neutral on such measures, or more negative and resistant.

Respondents
  • Show positive affect (autonomy, competence, relatedness)
  • Feel positively towards the instructor and classroom environment
  •  Show neutral affect
  • Feel neutral towards the instructor and classroom environment
  •  Show negative affect during activities
  • Feel negatively towards the instructor and classroom environment

 

References

[1] J.A. Fredericks, P. C. Blumenfeld and Alison H. Paris, “School Engagement: Potential of the Concept, State of the Evidence,” Review of Educational Research, 74:59 (2004).

[2] R. A. Engle and F. R. Conant, “Guiding Principles for Fostering Productive Disciplinary Engagement: Explaining an Emergent Argument in a Community of Learners Classroom,” Cognition and Instruction, 20 (4), 399-483 (2002).

[3] Boekaerts, M., “The crucial role of motivation and emotion in classroom learning,” in H. Dumont, D. Istance and F. Benavides (eds.), “The Nature of Learning: Using Research to Inspire Practice,” Organisation for Economic Co-Operation and Development (2010).

[4] R. Moreno, “Educational Psychology,” J. Wiley & Sons, Inc.: New Jersey (2010).

 

{ 1 comment }

Bill Goffe November 3, 2014 at 2:06 am

I wonder how much engagement might vary over the semester? At least in my experience engagement can very a lot by topic. In some cases, I’ve got activities that students find engaging and others times not so much. There seems to be a fairly narrow window of difficulty to questions that students find engaging and it can be hard to hit it every day (especially in a discipline like mine where there aren’t a plethora of active learning activities). I recall Dan Willingham making this point that engagement is hard to hit and that the window of interest in an activity can be narrow. It was in his book (in an early chapter?).

If engagement does vary over the semester, and you’re only measuring at semester’s end, I recall that Daniel Kahneman claims that when recalling something that occurs over some time humans tend to recall the peak and the end. I would think this must be in his book “Thinking, Fast and Slow.”

Comments on this entry are closed.

Previous post:

Next post: