This blog post is co-written with Dr. Tim Archie, CU Boulder, Ethnography and Evaluation Research, Tim.Archie@Colorado.EDU, and this is work done by him and Dr. Sandra Laursen, Devan Daly, Chuck Hayward, CU Boulder. This effort is funded by collaborative NSF grants DUE-1525077, DUE 1525058.
---
“A lever is a simple machine used to move an object at one location by applying a force somewhere else. When we try something and see that it is working, we have gained leverage on the problem.” -Levers for Change, AAAS Link
In this piece, we look at one of the key levers for change, professional development workshops in higher education. We'll start with our theory of change, and then share results from our research team. Specifically the data is based on participants, who attended intensive 4-day IBL workshops. These workshops provided intensive training to more than 500 college math instructors, impacting hundreds of courses and thousands of students per year. Peer-reviewed articles will be published separately, and if you have questions about technical aspects of this work, please contact Tim. In this piece, we focus on the broad story about how professional development can change instructor behavior in the classroom.
Our Theory of Change is that instructors need professional development to make the switch from passive instructional methods to creating active, student-centered, inquiry-based learning environments. This professional development must address instructors’ knowledge and skills to carry out IBL in their own classrooms, but also their beliefs about instruction, and must mesh with their own identities as teachers and understanding of their own students and teaching contexts. Achieving substantive change in teaching can be addressed via intensive summer workshops that provide these resources and a strong, steady support system that follows each instructor through the evolution of their professional career. Yet we also need to find ways to build the pipeline of instructors who are aware of and receptive to IBL methods. To widely offer these varied forms of professional development will require a larger, more flexible group of skilled professional developers than is now available. Building capacity to design and deliver effective professional development is necessary for increasing uptake in college mathematics.
College math instructors are not fully trained in teaching. We may receive some training as TAs, and then we are off to start our teaching careers. Active, student-centered teaching methods, such as IBL, require skills and practices for maximizing effectiveness. While some instructors are able to learn and develop these skills on their own or by attending conferences, professional development workshops can provide a big boost to help instructors new (or newish) to IBL teaching.
The IBL workshop is a 4-strand model. The fours strands are (1) analyzing video of IBL classes, (2) discussing the nuts and bolts of running an IBL class, (3) developing a target IBL course, and (4) studying articles from the mathematics education literature. The four strands were designed to address specific obstacles instructors face in their classrooms.
How do we know a workshop works? To answer this question, we need to define two terms, IBL capacity, and IBL intensity.
IBL capacity is an aggregate of several variables. It includes a participant’s beliefs about the effectiveness of IBL, knowledge of IBL, and skill in using IBL. What the evaluation team did is ask participants to self-rate their capacity before the workshop, immediately after the workshop and one year after the workshop.
Our data show in the figure below that workshops increase IBL capacity. We see a statistically significant increase in capacity measures from pre to post workshop and that these gains are sustained through to the one year follow-up.
(The slight drop from post-workshop to follow-up is not statistically significant.)
IBL intensity is a way to measure how much or how intensely an instructor is using IBL. We know the workshops are effective because a large percentage of participants, 94%, reported using IBL methods after attending the workshop. We also asked participants about the frequency of use of a range of teaching practices (e.g. group work, student presentations, lectures, etc.) before they attended the workshop and again one year after they attended the workshop. Higher scores indicate more intensive use of IBL and lower scores indicate lower IBL use. Based on their responses, we created an IBL intensity scale shown on the y-axis of the chart below. We see a statistically significant increase in IBL intensity from pre-workshop to follow-up.
Crucially there is more to the story on top of the increases to IBL capacity and intensity. The evaluation team also conducted additional analysis to explain more precisely how the workshops work, using the theory of planned behavior (Ajzen 1991). The theory of planned behavior path model explains how the IBL workshops are influential in changing teaching practices and the relationships between IBL capacity, IBL intensity, and some other key factors.
Let's go down the left side of the model. Belief that IBL is effective is a factor that influences teaching. Simply put, the stronger your belief is in the effectiveness of IBL, the more likely you intend to implement IBL, which in turn affects IBL intensity. The IBL workshop has a strand that provides an opportunity to learn about the research supporting the use of IBL, see successful examples of IBL, and learn about real-world stories of student success.
Subjective norms include factors such as department or institutional support and supportive colleagues. For instance, the more supportive your department chair is, the higher your intent to implement is, which then affects IBL intensity. Generally, the more supportive the environment is, the more instructors use IBL. Subjective norms can also be a barrier. If you lack support, then you are less likely to implement. Norms matter, because teaching is a cultural activity, and local environments play a role.
The last factor on the left side is perceived behavior control. This is the perception of how much you control your behavior, which is highly dependent on IBL capacity. If you feel confident in your skills and knowledge how to implement IBL, then this directly impacts both intent and IBL use. This is where IBL workshops play a direct role. Skills and practices are central topics of the workshops, and as shown we have data supporting participant gains in IBL capacity.
There are other contextual factors that play a role. These factors are prior IBL experience, whether a course is coordinated (in a way that is supportive of IBL), and class size. All of these factors can potentially be barriers, such as coordination that steers instructors away from IBL or large class sizes.
Factors such as gender, career stage, rank and position, and institutional type do not influence IBL capacity or IBL intensity (i.e. were controlled for). This is a striking result, because what this means is that anyone can teach via IBL anywhere! While individuals do face different challenges in the field due to their identity, group differences are not detected in the data. Of course identity matters in teaching, but what we know from experience and from data that no matter what group you are in, it is possible to teach via IBL in whatever setting. This means that the challenges we face are surmountable, and the focus should be on finding ways to support instructors.
Technical Question: What does r = 0.18 mean in the relationship between attitudes that IBL works and intent to use IBL? These are standardized regression coefficients which describe the relative strength of association between an independent and dependent variable. Standardized coefficients have standard deviations as their units, making the coefficients comparable when variables have different levels of measurement. Please also see “An Effect Size Primer: A Guide for Clinicians and Researchers” https://doi.org/10.1037/14805-020
Summary of key findings
- The theory of planned behavior model explains 21% of the variability of behavior change (IBL intensity). (Note: R^2 equal to 21% is a significant result for social science. For more see this Link)
- Workshops increase IBL capacity.
- Workshops increase IBL intensity.
- Department norms and institutional support matter for better or worse.
- Other factors like class size, course coordination, environment, course coordination, beliefs, play a role influencing IBL intensity.
- Anyone can teach IBL anywhere (under the right conditions)
Consequently, we can confidently say that IBL workshops work. Instructors change practices in meaningful ways. We can measure IBL capacity and intensity, and further we can see more clearly how the pieces fit together in a coherent story. Some policy implications include increased and sustained investment in professional development workshops, providing more support for faculty to access training, conducting outreach by individuals and professional societies, targeted efforts to inform and train department chairs and course coordinators, and investing in addressing institutional barriers to adoption of IBL methods (e.g reducing class size, mitigating the effects of large class sizes).
References
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211.
Archie, T., Laursen, S., Hayward, C. N., Yoshinobu, S., & Daly, D. (2020, November 5-7). Findings from 10 years of math instructor teaching professional development [Link to Poster]. This Changes Everything, AAC&U Virtual Conference on Transforming STEM Higher Education.
Ferguson, C. J. (2016). An effect size primer: A guide for clinicians and researchers. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (p. 301–310). American Psychological Association. https://doi.org/10.1037/14805-020
