Here's the context. 1000 students are in first-year Linear Algebra, split into 7 lecture sections with 7 different instructors, and 14 TAs, who teach dozens of tutorials/recitations. That's a lot of people!
We started the term online due to the omicron wave in winter 2022, and then taught the second half of the term with a mix of in-person and online. At the beginning of the term, we did not know when or if we would return to in-person learning, and had to setup the course in early January with the uncertainties of the pandemic. This post focuses on the assessments for the course and some initial thoughts.
TL;DR You can implement grading for growth even in large, coordinated courses.
Here the assessment setup:
- First a major constraint... An in-person final is mandatory and "owned" by the Faculty of Arts and Sciences, and has to be at least 35% of the grade. The other 65% of the grade was based on the items below. Also note that in Canada, 80% is an A-, 70% is a B-, and so on. So the weight of the final is not as immense as it would be in the U.S. In the U.S. 25% is a rough conversion.
- I gave a two-part final. Part 1 tests core standards worth 25% of the course grade. Part 2 of the final had challenging problems worth 10% intended for students who want to improve their grade to an A or A+.
- In lieu of midterms (which would have been online for at least one of them), students submitted 4 graded group reports. (Two additional assignments were reflective writing assignments for a total of 6 reports.) Group size was set at 2-3 students, and some groups were allowed to grow to 4 due to special circumstances (e.g. adding a student to a group).
- Group reports (30%) were submitted online (Gradescope) and the TAs and instructors graded 2 or 3 of the 4 or 5 problems. The ungraded problems were checked for completeness. Problems that were graded, were graded with a rubric for mathematical correctness and presentation. The entire assignment was out of 10 points, and written feedback was given to students.
- Students could resubmit group reports at least once. For the early group assignments, we had the capacity to accept up to 3 resubmissions. The last group assignment, which was due near the end of the term, allowed us to accept one resubmission.
- Online homework (20%) was assigned on MathMatize, and the due date for all assignments was set for the end of term. Students were allowed to redo problems as many times as needed, and were given suggested completion dates that matched the pace of the course.
- Because the course was a flipped, IBL course, students were required to do reading assignments (15%) before class. Reading assignments were done on Perusall, where they were graded using "threshold" grading with instant feedback. If students made 3 or more comments they would get credit for the assignment. Reading assignments had a hard due date, because we expected students to read the sections before we would do activities in lectures. The 4 lowest scores were dropped, which allows students some flexibility.
Lectures were centered on activities to support student learning of the core ideas. Tutorials were a mix of activities, practicing basics, and preparing students for their group reports. I won't go into further details about how classes were organized, since the focus of this post is grading for growth.
Students could pass the course if they did all of groups reports, online homework, and reading assignments. Students would need to perform well enough on the final exam to earn an A or B.
The Whys?
I wanted to accomplish a few things. One is to reset incentives towards learning and intrinsic values. Another is to center honest, hard working students who want to learn, and reduce incentives for cheating. A third is to avoid using creepy proctoring software (where students have to ask a
proctor for permission to move if they need to vomit), which also use
biased algorithms.
One aspect of grading for growth that I appreciate is that the honest students, who do their own work and submit their mistakes are not penalized or behind, compared to people who lookup answers or pay for services that give them the answers. Students who make mistakes receive feedback, and grow from the process. These students appreciated being able to update their reports and fix issues. Their grades aren't being negatively affected by those who cheat. The students who cheat will learn less and be less prepared for the final, future courses, their lives, and careers. Online cheating is a reality at the University of Toronto and sadly almost everywhere, when things are setup the old way with timed, rigid, high-stakes (online) tests as the bulk of the grade.
The pandemic is a major factor still (and will be next year too, imo), and impacts students and their families. The gradient of risk also skews heavily towards the more vulnerable and marginalized. Grading for growth with opportunities to resubmit work without penalty gives students more time to learn the material during the semester and crucially creates a more level playing field. If students get sick or have to deal with a family emergency, flexibility is built into the course to help students get their work done during the term. It should not matter, if a student learned something in week 8 vs. week 10.
Students who don't invest in the learning will not do as well on the final exam or in their future work (or life). The final exam is one of the ways that students are held accountable during the term. More broadly, students need to learn the course material as well as learn how to learn, and the course philosophy is talked about with students. Students will need both the content knowledge and the improved thinking in their lives, and cheating/looking up answers won't help them become better and smarter.
Group reports are focused on why questions or having students explain why things work the way they do. Sample questions on group reports:
Give examples of a plane in $\mathbb{R}^3$, using vector form, normal form, and standard/cartesian form. Explain the advantages and disadvantages of each representation.
The setting for this problem is $\mathbb{R}^3$. Suppose you have a plane $P$ and two vectors $\vec a$ and $\vec b$ in $P$. The task is about the general question, ``If you add two vectors in a plane, is the result still in the plane?'' More specifically, using examples, diagrams, and sentences, find characteristics of planes, $P$, such that $\vec a + \vec b \in P$. Additionally, find characteristics of planes, $P$, such that $\vec a + \vec b \notin P$.
Some things I'd like to change The reason why we have to have group reports vs. individual reports is due to TA hour limitations. Without constraints I would have students submit individual reports and have all problems graded. But that is way beyond the budget for TA time.
Practically speaking, reducing the number of group reports to 2 per term could allow for individual reports, with 1 rewrite each. The pros would be that there would be more individual feedback, and less incentive for students to divvy up group report problems and focus on fewer problems. The downside of going down to 2 reports is that you have fewer topics covered and higher stakes per report. There are other options such as 3 reports done in pairs or 3 reports done individually. I'll have to sort this out this summer. One takeaway here is that there are options and tradeoffs.
Reading assignments and online assignments generally work as they are intended. They focus on basic skills and fundamental concepts. The one issue that is specific to the University of Toronto is regarding Perusall and reading assignments. There are local tutoring services in Toronto that sell Perusall comments that customers can copy-paste into the system. Some of these get flagged as "plagiarism" by the Perusall system, but students can make slight edits and work around the issue. One way to get around this is to switch to reflective writing assignments submitted via Canvas and grade these for completeness.
Tweaking the final into more sections to make clear what the standards are and what students are expected to know for the final is another area that will be worked on. One idea is to have three parts to the final with specific themes.
- Part 1: 10% of course grade is based on core skills (e.g. computing determinants, determining if a set of vectors is linearly independent.)
- Part 2: 10% of course grade on demonstrating conceptual understanding of core concepts (e.g. answering concept questions via short answer or sentences.)
- Part 3: 15% of course grade on applying ideas and skills to solve more challenging problems. (Prove why a given matrix is/is not diagonalizable.)
Students will be given a final exam guide with the details, sample problems, and a list of standards that will be covered on the exam. Students who do all the term work would go into the final with 65% of their course grade in hand (or a course grade equal to a C). Getting 80% of parts 1 and 2, will net 16% or a total score of 81% in the course, which is an A-. Students who want an A or A+ will need to solve some or all of the Part 3 problems (or get 100% on parts 1 and 2 to get an A).
Setting aside the details of the scheme above, the main takeaway is that instructors can set percentages for the term work and final exam parts in ways to fit the assumptions and values of their institution. What I did was try my best to think of something that would work and then I'll adjust as I learn and get feedback.
Places to start A couple easy places to start with grading for growth is to make homework online with infinite attempts (WebWork, MathMatize, or whatever is bundled with your textbook) and setting up a standards-based final exam using. I am unable to implement a standards-based (formerly called mastery-based) final at UofT due to policy restrictions.
With standards-based finals what I did in the past is to write a Part 1 of the final with the core standards, where students need to earn 90% on it in order to keep their grade going into the final OR earn a C- (if the incoming grade is below a C-). Students scoring less than 90% on Part 1 could have their grade go down on a sliding scale up to a whole letter grade. Part 1 has core standards, such as basic skills and computations. The theme of Part 1 is "If you pass this class with a C-, you should know these things." (What is on Part 1 needs to be transparent to students with ample opportunities to practice.)
Part 2 of a standards-based final are challenging problems that are opportunities for students to demonstrate that they learned the material deeply and can raise their grade up to an A. Part 2 problems can be proofs, explanations, or more challenging problem-solving questions.
Again I could not implement this due to policy constraints, but standards-based finals are a way to start without having to change everything. Keeping all the other parts the same, and using a standards-based final is a reasonable starting place. Once you get that down, then you can move onto other parts of the assessment scheme.
Final thoughts I used grading for growth in small classes (enrollment 25-35) for many years, so the idea wasn't new to me. Transitioning to coordinating large courses meant focusing on things like group reports, a "tiered" final exam, and then thinking about how to make things work within the TA hours constraints. The smaller the class, the more options you have.
One advantage of having a TA hours budget is that you have to think about what would work without spending all your time on grading. It's not ideal or "excellent," whatever that means, but it's better. And better is good. More TA hours would also be good :)
If you are teaching a small course and have no TAs, one idea is to think of your own budget in time. Set aside a number of hours you would spend marking per week or per term, and then figure out what could work in that time budget.
I know that for many it is big step to use alternative grading, but there are major benefits to switching that needs to be emphasized again and again. When you align assessment with learning and implement IBL or active learning, it's a much better experience for students and makes the entire course more aligned with the goals of education. It brings us closer to our vision of humanistic math education. Thus, it is worth the effort to go down this route.
Resource Check out the
Grading Conference group, their slack channel, and work with a community of educators working on this grading for growth. They are a fun, friendly group, and will help you get started.
