Standards-Based Grading Example in an IBL Course

I'm sharing an outline of standards-based grading I've recently used in a course for future elementary school teachers, although much of this is generally applicable to other courses. I'll list the main features and then get into some of the details below.  Also this is just one example, so do not assume that what I am sharing is representative. It's really a form that works for the specific course.

Here are the main features:

1. Gateway exams
2. Reading assignments
3. Homework assignments
4. Productive failure
5. Class contributions and participation
6. Final project

How these fit together is that if a student earns a passing grade on all items, they earn a B in the course. Students can raise their grade to an A/A-/B+ with an excellent grade on the final project. Students earning one or more non-passing scores in any of the categories will earn a grade lower than a B, with specific grades reductions based on the nature and quantity of the unsatisfactory grades.

1. Gateway exams: These exams are based on the IBL units we work on regularly in class, and are based on the math being learned in the course. Students are required to pass all of the problems on the gateway exams (i.e. get the correct). For any problem that was not successfully passed, a student must retake that problem on the retake exam.  The retake exam is given about two weeks after the initial exam. Problem done correctly do not have to be retaken.  The first retake is done in class. Subsequent retakes are completed in office hours or alternatively completed in writing and submitted for review.  This past term, I gave 2 exams, and was limited by the quarter system (10 week terms) in how many retakes can be given in class.

Retakes can become a logistical challenge for large classes or in courses where there is a significant amount of material to cover. One has to weigh the costs and benefits of this and plan accordingly. The strategy I've taken is to start with a course that I thought would be relatively easier to manage, and then work my way to other courses where I feel I would be better off with more experience.

2 and 3. Reading assignments and homework assignments are graded for process and completeness. Accuracy feedback is given, however, the goal of these assignments are for students to think and reflect on math and math knowledge for teaching. Points are not taken off for mistakes or incorrect answers, and instead feedback is given when necessary and points are awarded for good process. For example, if a student gets a problem wrong, but writes questions or explains what they did and what they still need to work on, then they earn full credit for the problem. 

4. Each student is required to present one productive failure (i.e. #PF) per term (in a 10-week quarter) about a mistake or something the student was stuck on. The format is to discuss (1) the mistake or issue, and (2) to share what they learned from the process.  (In some courses the number of #PF presentations is 2.)

5. Student contributions to the class discourse is another component. Students work in groups and are expected to show up to every class, contribute to discussions, be effective group mates (i.e. be good at listening, supporting, and sharing), and present math ideas sometimes. More or less this is participation grade, but with stipulations about expected behavior. 

6. In lieu of a final exam, students must submit a final report. The report is based on 4 tracks related to mathematics teaching in the elementary school and the course content (in this case fractions for teachers).  Each track has a lead source (article or book). Students are required to do library research, branching out from the lead source, to find learning challenges (for children) established in the Math Ed literature. Lastly, students are required to create rich mathematical tasks that address the identified challenges that build from starter problems to middle problems to goal problems. 

I get asked if creating math tasks is pedagogy.  The answer is no. Creating math tasks to address specific math learning goals is a teaching specific math activity. Identifying the main math ideas, ordering and sequencing math problems, and building up from first principals is doing a math (applied to teaching children). 

---

Some general comments.

Gateway exams require students to learn all the standards of the course. There's no partial credit for problems, and students are required to demonstrate they know the math they need as teachers. Students have multiple chances to make sure they get problem completely correct. The retakes can be logistically challenging, if you are not organized. Overall, the workload is about the same, because retakes eliminates the time needed to determine partial credit, and there's a tradeoff that more or less washes out (for me).

Further, the overall assessment structure aligns the class to the mathematical work of teachers and the philosophy of IBL. The focus is on learning, and guiding students to what they know well and what they need to work on further. What students need to work on is clear, and this I find one of the main benefits of standards-based grading.

The reading assignments, homework, productive failure, and class contributions, as an ensemble focuses on process and prospective teacher beliefs. The shift is away from "answer-getting" without deep understanding.

Final projects or final exams can be implemented in ways that work with standards-based grading. In this specific case, I decided on final projects, since it gives future teachers the opportunity to connect they math they are learning, the research literature, and connect that to the classroom. In other courses, I have used standards-based final exams. 

If you're thinking about trying standards-based grading, I highly recommend giving it a go.  If you have been using standards-based grading, please share what you do! 

Edit: Dr. Kate Owens published A Beginner's Guide to Standards-Based Grading the AMS Blog.

Dealing with Student Attitudes Through the "Teaching System Lens"

One issue that doesn't get enough air-time is students' attitudes and beliefs about Mathematics.  It is documented in the Math Education literature that students often have beliefs about Mathematics and how to learn Mathematics that is either not helpful or hurtful for their own development.

I also add the caveat that no one intentionally wants the following outcomes.  They are unintended consequences, and they are consequences we should know about.  Here's a partial list of negative attitudes or beliefs that has been documented.  I've mentioned these before.

1. Memorizing facts and formulas and practicing procedures are sufficient to learn mathematics.
2. Mathematics textbook problems can only be solved using the methods described in the textbook.
3. Teachers and textbooks are the mathematical authorities.
4. School mathematics is driven by rules and memorization, and is driven by procedures rather than concepts.
5. If a problem takes longer than 5-10 minutes, then there is something wrong with the student or the problem.
6. The goal of mathematics is to obtain one correct answer and do it quickly.
7. The teacher is the only source of determining whether an answer is correct or incorrect.
8. Students’ role in the classroom is to receive knowledge by paying attention in class and to demonstrate it has been received by producing right answers.
9. The teacher’s role is to transmit knowledge and verify that it has been transmitted.
10. Only geniuses have what it takes to be good at mathematics.
11. Students prefer to have only one way of solving a problem, because it is less to memorize.
12. The processes of formal mathematics have little or nothing to do with discovery or invention.
13. Students who understand mathematics can solve assigned problems in 5 minutes or less.
14. One succeeds in school mathematics by performing the tasks, to the letter, as described by the teacher.
15. The various components of mathematics are unrelated.

So when an instructor teaches via IBL and works with students who have never had an IBL experience, there usually exists a host of default expectations that one has to work through with the students.   

Let's say the students in your class are reluctant to buy into the IBL system.  Then looking at teaching as a system can provide a broader perspective to address the issues, which could lead to a more coordinated effort to get students to become active participants in their own education.

Attack this teaching challenge through content, assessment, and pedagogy (i.e. through a system approach).

Content

Course content is one of the key components.  If tasks are too hard and only the very best students are successful at solving them, then students who are more likely to have some of the negative beliefs and attitudes above will have "evidence" to reinforce them.  "I'm not going to be successful anyways, so I might as well give up after 5 minutes."

Consequently, the tasks presented to students should be matched to their levels, and having a handful of very accessible problems is almost never a mistake.  At worst, the students mow them down, and you've been able to get more students to participate.

If students are having trouble starting up or you have a subgroup that is passive and not engaged, then one tactic available is to breakdown a problem set (or a part of one) into more manageable pieces.  This method can help students get going on problems.  One thing to keep in mind is to keep problems that are more challenging to keep everyone engaged at an appropriate level.

Assessment

If assessment is setup only along traditional lines (homework, midterm, final), and assuming each of these components is implemented in the usual way, then the ethos of the IBL course is mismatched to the course assessment.  If homework is graded based on accuracy, and there are no venues for safe exploration, then students are being told implicitly that "mistakes count against you."  This can inhibit risk taking and undermine the course.  While our viewpoint from the instructor side is that we expect the exploration to take place on scratch paper, this may not be a concept and workflow practiced by students.  It's a curious thing -- students expect to write down the answer in one shot.  This is absurd in art, science, math, etc. when thinking about it.  So how did we get here?

The pristine nature of clear lectures can be beguiling.  The imagery is one of a the brilliant expert doing it right the first time. Every time.   Instructors are never seen struggling, and this is an impossible standard to achieve as a learner, especially for the ones who need the most help. The unintended consequence is that in our effort to make the best possible presentations, the notions that working hard, being willing to explore, learning from mistakes, and having the predilection to work through our own personal learning obstacles can be undermined.

Homework is an opportunity to portray doing math the way mathematicians do it. One option is to provide feedback but not grade the homework in the usual way with numerical scores.  The explicit message that should be given to students is that their effort and the quality of their exploration is being checked, and feedback will be given to them to assist them with the learning process.  I'm not saying that this is how it should always be done, but this is an idea worth considering and adapting for your own courses.   If your goal is to bring about change in student perceptions in math, then grading homework for process and effort is one of the available tools.

Grading presentations is valuable.  In courses where presentations are used regularly, presentation grades should be reflected in final course grades.  Students are being asked to share their ideas, a noble thing indeed, and the quality of this work should be reflected in their course grades.  A level around 25% works well in lower-level courses. As courses become more proof oriented, then raising the presentation grade as a percentage of the overall grade makes sense.  (Grading presentations and some sample rubrics will be discussed in future posts.)

Mastery-based finals are used by some IBL instructors with success.  Such exams are split into two parts.  Part 1 consists of material that anyone who passes the course should know.  Part 1 is contains the fundamentals and essential basics.  Students should be able to complete the entirety of part 1.   Part 2 is an opportunity for students to raise their grades.  Part 2 contains problems that require applying knowledge to problems novel to the students.  (Parts 1 and 2 are given to students at the usual examination time.)  A final exam structured like this allows students opportunities to demonstrate that they have learned the material and can demonstrate that they can use what they learned to solve problems new (to them).  Before the final exam, students are given a (non-final) course grade going into the final exam.  Passing part 1 means students keep their grade OR get at least a C- in the course.  Success on part 2 only improves the final grade and does not count against students (i.e. non-negative help).

Why do mastery-based final exams make sense?  If we value learning, applying ideas, and self-improvement, then mastery-based finals are a form of assessment of mathematics aligned to these values.  Further it provides an incentive for students to work hard through the end of the course, because they have a chance to succeed right up to the very end.   It is also noted that this does not mean one lowers standards.  The idea here is to keep the standards at the usual level, but provides a structure and incentives for students to achieve these standards.

The three main examples (effort/process based homework, assessing presentations, and mastery-based finals) are ways we can align assessment of the usual assessment items to IBL classes.  Additionally instructors can use portfolios, reading assignments, and reflective writing (journaling) as additional forms of assessment.  These assignments do not need to be large part of the final course grade, and are valuable tools for gathering data about how your students think.  Gathering formative assessment is significantly valuable, and all instructors are encouraged to use one or more of these additional strategies to help put together the full picture.

A specific example I want to share is journal assignments.  Specifically this example is about using "The 5 Elements of Effective Thinking," by Burger and Starbird as a vehicle for students to think about their own thinking (metacognition).  When working with a group of students who do not like mathematics (e.g. Math for Liberal Arts), then having students read and write about healthy ways of doing math is a successful strategy for combating math anxiety and buy-in.  Math autobiographies are normally the first assignment, and the rest of the assignments are based on chapters from the book.  A typical assignment asks students to (a) read the assigned chapters, (b) write about 2-3 things they learned, and (c) write a personal reflection about how they are doing math.

For students in education courses, such as "Math for Elementary Teaching," I use a variety of articles from Teaching Children Mathematics, and books like "What's the Point of School?" by Guy Claxton, and "What's Math Got to Do With It?" by Jo Boaler.   Students in math major courses could read autobiographies of famous mathematicians, books like "Fermat's Enigma" by Simon Singh, or other related books.  A vast array of possibilities exists with journal assignments, and you are encouraged to find (and share!) strategies that work for you.

One last facet of assessment is that it needs to be set before the term starts.  Assessment is the least flexible component, once the term gets rolling.  Thus, it is worth thinking about assessment early in the planning phase of a course well before you meet classes for the first time.

Coaching

As mentioned in an earlier post (Link), coaching is an critical facet of teaching.  When students get stuck, then your role as an IBL instructor is to manage the classroom so that students continue to learn, as opposed to giving up.  If the default attitude or belief of your students is to shutdown when stuck, then coaching students through this phase is part of the job.

• Coaching can be verbal encouragement.  "It's okay to get stuck.  Let's see if we can figure this one out together.  Don't give up... Being stuck is a natural, regular part of doing mathematics."
• Coaching can be directive. "Let's try figuring this related example/strategy/definition..." or "Let's pair up and try to see if you can think of a useful idea or draft a plan you can take home and try."
• Coaching can be done through content. "Let's look at this supplemental handout I wrote yesterday after class, when I saw that we were all stuck on number 19.  I think these problems will be really helpful..."

A couple things to keep in mind.  Frustration levels have to be kept below "redline," and students may need to learn specific strategies and habits of mind.  IBL instructors should work to stay attuned with their student and setup a friendly, safe environment, where students are encouraged to ask questions when they are stuck.  If students are quite, then asking them what they are working on is one way to check in without asking for an answer or question.

In summary, a "system approach" to dealing with a complex issue (students' attitudes about Math) provides a richer and likely a more successful approach.  Teaching and learning is complex, and there are many things that affect the day-to-day activities of our course.  At times it can be a daunting challenge, but as my colleague Professor Dylan Retsek says, "Chop wood, carry water."  In this case, we break down our response to content, assessment, and coaching.

Upward and onward!

John Dewey + Moneyball = A Key Insight to Change

A quote from John Dewey

"I may have exaggerated somewhat in order to make plain the typical points of the old education: its passivity of attitude, its mechanical massing of children, its uniformity of curriculum and method. It may be summed up by stating that the centre of gravity is outside the child. It is in the teacher, the textbook, anywhere and everywhere you please except in the immediate instincts and activities of the child himself. On that basis there is not much to be said about the life of the child.  A good deal might be said about the studying of the child, but the school is not the place where the child lives. Now the change which is coming into our education is the shifting of the centre of gravity. It is a change, a revolution, not unlike that introduced by Copernicus when the astronomical centre shifted from the earth to the sun.  In this case the child becomes the sun about which the appliances of education revolve; he is the centre about which they are organized."  http://bit.ly/JvR3cG

This passage is from "The School and Society," originally published more than 100 years ago (in 1899).  It is relevant today, sadly.  Are students at the center of instruction in math classes?  Mostly no.  Math teachers predominantly lecture at students, and the great change that Dewey saw has not yet come about, though many teachers have made the shift.  A question one can ask is "Why have things not changed significantly in all these years?"  Sure the books have colors, and we have technology beyond our grandparents' wildest dreams.  But when you look beyond mere surface beauty, you can see that the heart of it is still the teacher telling, and the students following.

One major issue behind the lack of change is data.  More specifically an issue that persists is in assessing teaching and learning, and more pointedly how this data might change our fundamental beliefs (or axioms) about teaching and learning.  What we assess and how we assess it determines our evaluation of student ability and achievement.  Herein lies one of our fundamental issues.  Lack of good assessments can lead us to continue doing what we have been doing.

To get some insights, let's look outside of education to provide a backdrop for analyzing our own system.  One of the unique aspects of baseball is the wealth of statistical information that has been available for generations upon generations of players. It's one of the reasons why baseball is such a wonderfully interesting sport to be a fan of.

Earned Run Average (or ERA) is one of the traditional measures of a pitcher's ability.   A lower ERA is considered better, since the pitcher gives up fewer runs per 9 innings.  The problem with ERA is that it is a noisy and flawed measurement system of pitching effectiveness.  It depends on factors not under control of the pitcher, such as the quality of the defense supporting the pitcher and the effects of stadiums on balls batted in play.  Some pitchers are overvalued and some are undervalued in terms of their contributions to team wins, if ERA is weighted too heavily as a measure of ability.  Voros McCraken conducted some groundbreaking analysis, establishing the concept itself and subsequently methods to measure pitchers that are "defense independent."  This story among others is chronicled in "Moneyball" by Michael Lewis.  But Voros didn't expect baseball teams to rejoice when learning about his findings.  He knew better.

"The problem with major league baseball... is that it is a self-populating institution.  Knowledge is institutionalized.  The people involved with baseball who aren't players are ex-players... They aren't equipped to evaluate their own systems.  They don't have mechanisms to let in the good and get rid of the bad." (Voros McCraken)

This is a striking insight!  Voros essentially identifies why baseball resisted modern statistical methods that could help.  Baseball is not set up as an institution to evaluate how it evaluates players.  Baseball people normally did not have the knowledge, ability or willingness to entertain ideas developed by people like Voros, who is a baseball outsider.

What is the implication for us in the teaching profession?  It should be stated that major league baseball and education are not very similar as institutions.  That said, we have some similarities and we can draw conclusions about our shortcomings from baseball's own struggles.  Indeed teaching is also a self-populating institution. Students who do well in the current system are the ones who end up become teachers or professors.  Some future math teachers state things like, "The reason why I like math is because there's always one right answer, and there's a simple, straightforward structure to all problems."  They are good at memorizing rote skills, the rote skills appear on tests, they get good grades, they are labeled as good in math (which may or may not be true), and then they model themselves after their favorite teacher.  Thus the cycle perpetuates.

Colleges and universities have in their mission the goal of seeking truth and knowledge.  When it comes to teaching, however, discussions among faculty often are about style, "what my students like...," and about delivery of information.  The focus is usually not on learning and what students are doing.   A major point is that we do not use the scientific method to evaluate teaching, just as major league baseball didn't use any scientific methods to validate their player valuation systems.

Consequently we have several metric problems.  Are the usual metrics like skills-based tests and student evaluations the right ones?  Clearly the answer is no.  Let's consider the typical calculus sequence with a thousand-plus page texts.  In the typical chapter on optimization in calculus books, the authors usually highlight in a colored box the steps for how to find relative extrema.  What this tells many (but not all) students is that they should memorize the recipe and regurgitate it on an exam.  That's how one can get a good grade after all.   These students will not walk away with a conceptual understanding of the subject, and probably will forget what they have memorized once the term is over.  In short, their education is unintentionally of a lower quality than what we want.  Mathematics is reduced to applying recipes that many students do not understand or even care to understand.

If you don't believe this can happen, here's data from Physics by Professor Eric Mazur, Harvard University, presenting at the University of Waterloo.   (It's 1 hour long, but worth it!)  At Harvard, 40% of the students in freshmen physics who did well on the procedures had inadequate understanding of basic concepts.

The result of traditional assessments is that many students who are traditionally given good grades have major gaps in understanding of basic concepts.  Students think they know it, but maintain "Aristotelean understanding of Physics" rather than a Newtonian one.  Their education amounts to very little, even at a sublime places like Harvard.

Now let's consider traditional teaching assessments (i.e. student evaluations), and consider data from Physics, based on the Force Concept Inventory (FCI).  All of the red data points are from traditional instructors who lecture.  Represented in these data points are teachers who are highly rated and lowly rated on student evaluations -- the red dots contain some star teachers and the teachers on the "oh bummer" list.  And they all do about the same on the FCI within statistical significance!  Students gain on average about 23% of what is possible in the pre-post test design.  Student evaluations are like ERA.  They are a noisy, flawed metric.  Actually student evaluations are worse than ERA.  ERA has some value in aggregate (whole team ERA), and outliers tend to have outlier ERAs.   In contrast, the highly-rated, award winning instructors are doing no better than Dr. Boring or Professor Snoozer.

The green data points represent faculty who use Interactive Engagement in their classrooms.  One of the main trends is that there is very little overlap between the reds and the greens.   The average green gain is double compared to traditional instruction.  Thus a better way to measure if an instructor is effective is to know what skills and practice he or she utilizes in the classroom.  While crude and incomplete, it at least it tells you whether the instructor is on the red or green distribution.  But these qualities and practices are not usually assessed or measured in teaching evaluations, so there does not exist sufficient data or incentive for the system to embrace change.  We keep on doing the usual, while the traditional assessments tell us things are okay.  And the results keep staying in the "red zone" above.  Education has a bunch of Voros McCrakens, so there is hope.  Baseball has changed, and I believe education will continue to improve for the better.

What about Math?  Calculus Concept Inventory has been rolled out and studies are underway.  Thus there is hope that we will embrace new assessments that tell us what is going on.  Preliminary results suggest similar outcomes to the FCI.  Interactive Engagement and Traditional instruction are different distributions.  I look forward to seeing the published results.  Moreover, a growing body of evidence in research in undergraduate education also suggests that students in traditional courses are not learning what we want them to learn.  (More on this in a future post.)

The MAA's Calculus Study indicates that 80% of college calculus courses are taught in sections of 40 students or less.  Additionally, very few institutions have large lectures for upper-level courses.  Ample opportunities exist for IBL methods to be deployed courses across the nation.

What can an individual instructor do personally?  Looking at data can be demoralizing at times, but one should be optimistic.  In particular, one can turn assessments into valuable tools that guides students and instructors in the right direction.

Assessment is more than grading stuff so that you can assign course grades.  Assessments should be utilized in ways that provide students with regular feedback (formative), instructors with information about their students (formative), and to evaluate demonstrated achievement (summative).  Assessments should provide incentives for the qualities we actually value, including creativity, clarity, exploration, problem-solving ability, and communication.

Ideas for what to assess:

• Student presentations and/or small group work
• Reading or journal assignment
• Math portfolios
• Exams
• Homework

The items above are not revolutionary.  What matters is what we put into them.  Exams can be rote skill based or they can also test for conceptual understanding, application of ideas, and problem solving.  Homework can be made more interesting.

Student presentations and/or small groups are a wonderful way to assess understanding.  When students present their proofs or solutions, it often a rich experience and rich source of information.  You see it all in IBL classes: great ideas, small ideas, half-baked ideas, insightful questions, victories and defeats.  It's a slice of real life, and it's really great.  It's very easy to detect where students are at, and then take action.

Reading assignments can be used to offload (i.e. flip a class) basics to homework, leaving time in class for the harder tasks, where inquiry is useful.  Portfolios are like a CV, and can be used to demonstrate what a student has been able to prove on his or her own.  Additionally portfolios can be used to create a record of the theorems proved by the class. (I'll write more about portfolios in future posts.) 

In IBL courses, one has continuous formative assessment.  Instructors are always analyzing whether students understand an idea or not, by giving students meaningful tasks and then working with them to overcome learning challenges.  If students are stuck, then there is another question or problem that can be posed, and then students are off on another mathematical adventure.  Students are continuously engaged, are monitored, are self-monitoring, get feedback, and so on.  This rich, integrated assessment system of actual learning is a core advantage in IBL teaching.

Returning to Dewey...  If we continue to teach and assess teaching in traditional ways, we will not gather the data and information necessary that support change for individuals and systemwide.  Gathering good data about our students' thinking, which is also a core part of effective teaching, is a key to the way out.  Engage your students, collect good data, share it, publish it.

Upward and onward!

"It ain't what you don't know that gets you into trouble.  It's what you know for sure that just ain't so." - Mark Twain