Thursday, June 2, 2016

10+ Videos on Productive Failure (Playlist)

Productive failure is increasingly becoming an important aspect in teaching, in light of the growth mindset research that have been published recently.  Below is a short list of videos I find useful to share with students.

1. Michael Jordan "Failure" Commercial

2. Sal Khan interviews Carol Dweck on Growth Mindset

3. John Legend, Musician: "Success through Effort"

4. IBL Instructors discuss the importance of failure

5. Growth Mindset Animation

6. Mike Starbird: Study Skills and Making Mistakes

7. Study Skills: Learning From Mistakes (Jo Boaler)

8. Diana Laufenberg: How to Learn from Mistakes

9. Uri Alon: Why Truly Innovative Science Demands a Leap into the Unknown

10. Astro Teller: The Unexpected Benefit of Celebrating Failure


11. One more suggested by Bret Benesh: Ira Glass

12. Thanks to Jane Cushman for sending me this:  Karen Schultz, On Being Wrong

Thursday, May 26, 2016

For Parents of K-12 Students (U.S.)

Dear Parents,
Every weekday morning I drop off my son at school.  Every weekday afternoon I pick him up. I have a vested interest in the success of schools both personally and professionally. When I talk to parents about their children’s education, I have noticed, however, that most parents have major gaps in their understanding of how our education system works, Common Core, active learning, and the point of education. In this post, I hope to nudge you to learn more about the issue for the sake of our children.

Basics about Education
Before we can talk about the main issues, we need to be clear on the basics. Some parents I talk to do not understand the “ingredients" of education. Just like cooking, you need to get the right ingredients, and on an even more basic level be able to recognize what those ingredients are. 

Basic ingredients of Education (in the US):
  • The state standards
  •  Curriculum (books and textbooks)
  • Teaching (Instruction) and learning environment
  • Student Learning
  • Assessment
  • Other (e.g. counseling, sports, clubs, facilities, etc.)

Looking at a list of ingredients doesn’t convey what the final product is. If you get recipe and only look at the list of ingredients, it doesn't tell you what the final dish will be when served.  Education (as a system) is much more complex than cooking a meal.  Hence, we need a way to organize the information to help us make sense of what we perceive at our schools.

Models let us see better how the pieces fit together, by organizing them in a structure.  The simple model below shows more or less how the basic ingredients fit together. It's not meant to be a definitive model covering all aspects of education. We're going to use the model to illustrate key points.
Your state (not the federal government) sets the standards. School standards then eventually result in textbooks that cover the standards. From there, teachers must take these pieces (and other resources) and incorporate them into their teaching system, and design classroom activities. Teachers must also customize the learning experiences to the actual students they have in the classroom. Student experiences and mindsets vary significantly, and day-to-day instruction adapts according to student learning needs.

An Example of How Some Parents Can Blame the Wrong Thing
I hear a ton of Common Core bashing, and what I hear is a lack of understanding of what education is, what standards are, and what standards are not. I noticed a particular example of this recently. Let's first set this up.

In grades K-6, research on homework strongly suggests that there's no learning gains for homework. It does perhaps cause students to dislike subjects or learning in some cases, so in sum it's a bad idea. High performing nations like Finland essentially do away with homework.

So homework in K-6 doesn't do anything for some students, and for others it's a net negative. The policy that should be adopted is to eliminate homework or reduce it significantly in elementary grades. Despite this research, parent often ask for homework, because that is what they grew up with.

Where things go wrong is when a teacher (in the US) is implementing math, such as in Common Core, where in addition to learning skills, students are also asked to think, explain, experiment, problem solve. This type of curriculum needs a carefully designed class experience and parents who understand that doing math well means doing math like mathematicians. That is, making mistakes, experimenting, and taking time to think deeply about the concepts. This is math that is far beyond what most parents experienced. In contrast, in order to placate parents who want the usual homework assignments, teachers send students home with math homework that often has good problems, but these are problems that take longer and require thought. Getting stuck is likely.  So here we have (a) teaching methodology that isn't going to work, based on research evidence, and (b) homework that is challenging that parents, who may have math anxiety will get into a homework-frustration struggle with their kids (at the end of the day, with dinner to cook...) This is a classic "conflating implementation struggles in the early years" with "the standards are a bad idea."

An added layer is that most people think doing math faster means smarter, and doing math slower means dumber. Misconceptions about Math and the nature of learning feed anxieties further, and then the blame game starts. Something has to be wrong, if Johnny can't add!

The final act of this tragedy is that people then assign blame to the wrong thing. They blame the standards, not the environment of parental pressure and the asking for homework in elementary grades. The standards aren't the problem here. It's instruction and how our society doesn't fully let teachers do their job and apply evidence-based practices. If anything, we (parents) make it harder for teachers to switch to better teaching methods.
If the goal is to improve education, then it's important to focus attention and effort on the correct thing. Helping and supporting teachers implement their curriculum as they were intended is what should be the focus. Instead, people want to tear down the standards (which will not improve education), and likely see us return to methods and curricula that have been shown to be significantly flawed and problematic.

Conflating Assessment and Standards
Here's another example. One of the other large pressures putting teachers and staff in a tough place is assessment. In the U.S. we assess too much, and those assessments use huge resources, and are then tied to job security. This creates unintended, "perverse" incentives that make schools teach toward narrow tests, at the expense of a fuller, holistic education. The arts, music, dance, etc. get nixed, only making it harder for children to find their element. Their passions.

Assessment should be done in scientifically sound ways to see how our schools are doing. The reality is that testing is far too large a force, and actually dominates education choices. Parents sometimes see this. They see too much assessment as a problem, where test results are used to punish or threaten teachers or administrators.  Instead of pushing back at the testing regime (and the massive testing industry behind it), some parents blame the state standards.
The issue here is quite clear again. The wrong ingredient has been identified as the problem. When you go in for knee surgery, you want to make sure the surgeon works on the correct one. Well, this is the same sort of thing. The torn ACL is on the left, but we've done surgery on the right. Most people would agree that it taking a step back, and not solving the real problem.

Understand the Problem First
What can parents do? One of the ideas we try to teach students at all levels it to understand what the problem is saying first, before making conclusions. We want students to ask pertinent questions, and understand the components of the question and the context surrounding it.  I think this lesson applies broadly. Parents in particular can learn more about education and what the actual issues are, BEFORE forming an opinion. Otherwise, you're committing a basic intellectual mistake or sin, called intellectual indulgence. (Intellectual indulgence is when you believe something to be true, because you like how it sounds, and not because you have any good evidence to support it.)

Ignorance is not a virtue, especially when it comes to decisions and policies about education. In fact, ignorance is damaging. Our society's collective, group ignorance prevents us from achieving far more than what we have. People have strong opinions about education and can prescribe or criticize, even when they do not have even a very basic understanding of what education is, how it's constructed, and what it's for. I hope you can see the deep and tragic irony with ignorance about education.

Parents can start with the books and videos listed below to learn more about growth mindset, what math education is like currently, how it could improve in the US, and how international comparisons shed light on what we can do better. Parents obviously matter a great deal, and an informed group of parents with a positive, constructive attitude can be a powerful force in supporting and shaping our education system.

I have seen it time and again. When teachers are supported by their community and get it right, I see students transform from passive, disengaged people to eager, vibrant learners. Parents come up to me and say things like, "My daughter likes math this year!" There is good reason to be optimistic today, as we have learned much about how to effectively teach. The problem and challenge of improving education is a tractable problem. Parents can choose today to be a positive contributor to this process. You can say to your kids, "Hey, I'm going to show you how I learn about something by doing my own research..." And what a wonderful thing that would be to teach your children!

References for Parents
  1. Mindset, by Carol Dweck
  2. What's Math Got to Do With It, Boaler
  3. Mathematical Mindsets, Boaler
  4. The Teaching Gap, Hiebert and Stigler

1. Carol Dweck on Mindset

2. Jo Boaler on Common Core

3. Khan Academy interview Dweck

4. Dan Meyer on Teaching Mathematics

5. Ken Robinson on Education

Thursday, May 19, 2016

New AIBL Website

A quick post to say the AIBL Website has been updated with a new look! Check out

Monday, April 25, 2016

A Practical Solution to "What We Say/What They Hear"

“In art intentions are not sufficient and, as we say in Spanish, love must be proved by deeds and not by reasons. What one does is what counts and not what one had the intention of doing.”  -- Pablo Picasso

It's my belief that teachers at all levels have good intentions and genuinely want their students to learn.  What this post is about is the notion that instructor intentions are not sufficient, and getting students to do what is needed for authentic learning is what counts.

The starting point is for the ideas I want to get across from a the Launchings Blog of the MAA.  David Bressoud does a wonderful job of describing a recent paper by Kristen Lew, Tim Fukawa-Connelly, Juan Pablo Mejia-Ramos, and Keith Weber.

Below are links to Bressoud's twin posts.  They are worth the time!
"What We Say/What They Hear I"
"What We Say/What They Hear II"

The short version is presented in this diagram

Despite multiple passes through the proof and explanations, students in the study have a difficult time pulling out the instructor's intended messages in the proof and the instructor's spoken comments.  At first glance this makes sense, as the "information transfer" model doesn't work so well when the goal is "developing critical thinking."

As mentioned by Bressoud, Annie and John Selden and others have documented the difficulties students have with analyzing, proving, generalizing, packing/unpacking statements, etc. Learning higher mathematics is challenging, and most math majors struggle with learning proof.

Bressoud suggests options like flipped classrooms and clickers can work, but he notes that such methods have high initial investment in time and requisite knowledge and skill.  While these methods are learnable, many instructors may be in situations, where implementing them is not practicably feasible.  Other options are needed.

Another Option Exists!
Now to the main point of this post. There exists an "easy entry, high upside" method to help students come away with the intended messages.  Put simply, instructors can take their list of intended messages and turn them into math tasks. These tasks can be deployed via small group work, homework, etc.

Let's take a closer look. The intended messages of the instructor in the study are
  1. Cauchy sequences can be thought of as sequences that “bunch up”
  2. One can prove a sequence with an unknown limit converges by showing it is Cauchy
  3. This proof shows how one sets up a proof that a sequence is Cauchy
  4. The triangle inequality is useful in proving series in absolute value formulae are small
  5. The geometric series formula is part of the mathematical toolbox that can be used to keep some desired quantities small
      Each of these five points can be turned into "after the proof" problems.  They can be reformulated as
      1. Explain using sentences and diagrams why Cauchy sequences "bunch up."
      2. True or False and Explain:  One can prove a sequence with an unknown limit converges by showing it is Cauchy.
      3. In the proof, find the part that proves the sequence is Cauchy.
      4. In the proof, find the part where the triangle inequality is used, and then identify a general strategy based on this specific instance that you can put in your mathematical toolbox.
      5. Explain how the geometric series is part of a mathematical toolbox to keep some desired quantities small.
      One way to deploy these in the classroom is to present the proof on the board or pass out a handout with the proof, and then have students work through some of the tasks in pairs and then share (i.e. Think-Pair-Share).  If time is an issue, doing one or two and assign the rest for homework.  Alternatively instructors could ask students to read the proof before class, and the instructor could highlight the proof and spend more time on student-centered tasks in class.

      Several advantages exist with this option compared to flipped classes or using clickers.  The first advantage is that it requires the least experience and least amount of pre-class planning or classroom equipment.  One could be in classrooms like I sometimes teach in with no technology, old boards, and desks built 50 years ago.

      Another advantage is that it does not require deep knowledge of common misconceptions.  Instead, the instructor asks students to explain their thinking (in one way or another) and that's how the instructor gets insights into student thinking.  That is, use an activity and gather formative assessment.

      A third advantage of this method is that it does not deviate from the conventional class prep process used by most instructors. Preparing presentations of a proof with explanations is something that instructors have done many times. The method presented here tweaks the process by transforming the intended messages that instructors would normally say to students into concrete mathematical tasks for students to work on. This change is practically feasible and doesn't require a significant alteration of an instructor's workflow.  It's also a step towards active, student-centered teaching and can be built upon over time into forms of teaching that more deeply engages students, such as IBL.

      Instructors have good intentions and intended messages. I claim that it is how the intended messages are deployed in class that can be addressed in practical and effective ways. We can turn those amazing insights into amazing learning experiences, and "let problems do the talking!"

      Friday, April 15, 2016

      IBL Workshop Model and Real-World Results (Wonkish)

      This post is about professional development, and is intended for those interested in faculty professional development.  There may be some applicability to K-12 PD, but I'm not promising that. Presented here is a model that I argue can be used outside of Mathematics in higher ed. My experience in K-12 PD tells me that changes can be made for K-12, but that there are other aspects (such as working with school districts, parents, testing, etc.) that add more layers that need to be carefully and thoughtfully handled (that may require much more resources).

      This post is also from the perspective of people on the front lines. Our team works with teachers, and we are teachers.  Our work is grounded in the hard efforts of teachers who have thought carefully about the issues.

      The evaluators for the project I've been working on are Dr. Sandra Laursen and Chuck Hayward, at Ethnography and Evaluation Research, CU Boulder.   We have run a set of IBL workshops (funded by NSF SPIGOT) in 2013-2015, and have some results to share.

      The main results:
      • Total number of participants is 138
      • ~ 75% implementation rate
      • 61% of the participants are in the first 5 years of their teaching careers.
      • In just the first year after the workshop, participants taught 180+ classes to 4600+ students (in the real world)!
      • Ongoing support via Email mentoring is a critical feature
      • An inclusive or "Big Tent" definition of IBL is critical for allowing participants to find their own way, suitable for their situation.
      The short version is that uptake is happening at a high rate!  This is an encouraging sign, as it provides evidence that effective PD can make a difference.

      How we got to these results is a very, very large undertaking spread over about a decade of work by a team of people.  I'll briefly highlight the main facets in the rest of this post.  An embedded slideshow appears farther down this post with more details and diagrams.

      First, we identified major obstacles that faculty face when learning to implement IBL methods.   If an instructor has not had much or any firsthand experience with IBL, it is hard to convey through discussions what an IBL class is like and how to pull it off. Additionally, there are skills, practices, curricula, and assessment to consider, all of which are different or changes.   Hence the need for a highly specialized professional development experience that directly addresses the challenges and needs new IBL instructors deal with.

      Obstacles or challenges include lack of experience with IBL,  the need for instructors to learn IBL teaching skills, the Math Ed Knowledge Gap, IBL course materials, assessment in IBL courses, and organizing the structure (i.e. syllabus) of an IBL course. With a list of obstacles in hand, much effort went into designing workshop components to direct address and reduce the identified barriers.

      It it emphasized that the workshop is designed as a single composition, where all the parts of the workshop are interconnected and the main goal is getting participants to a point where they can teach an IBL class successfully.  It is definitely NOT about trotting out our favorite activities.  Indeed, teaching is a system, and teaching is a cultural activity. Hence, the workshop is about providing a broad IBL framework that participants can adapt to their situation. The main focal points are (a) IBL teaching skills and practices, (b) understanding the evidence for IBL and how students learn, (c) linked teaching choices (assessment, problem posing, and content), and (d) building a practically feasible (to the participant) course.

      Another important feature of our work is data driven decision making. The staff used evaluation data and research to inform decisions about all aspects of running and designing workshops, from recruitment, building workshops materials, adapting sessions to better meet participant needs, and also to main components that are successful.  It's through this iterative, self-assessment process that the workshop model has improved over time.  Sandra Laursen and Chuck Hayward head up the evaluation and research efforts for our projects, and their regular insights helps use make small and longer-term positive changes.

      Our current NSF funded project, PRODUCT, has the main goal of expanding the profession's capacity to offer a version of the 4-Day IBL Workshop and developing short workshops that can be "sent" around the country and increase awareness of IBL.  The goal is to build the PD network up to a point where a much larger capacity PD exists and a larger variety of workshops (weeklong and short workshops).

      More details are in the slides below.

      OR click this link to Slides About the IBL Workshop Model

      Edit: Click here to link to the evaluation report.

      Tuesday, March 29, 2016

      Engaging More Students Framework

      One of the common struggles for instructors is engaging a larger portion of students.  A typical scenario instructors find themselves in is that a small group of students do the bulk of the talking.  The Usual Suspects, The Fab Five, The Fantastic Four... they chime all the time.  There's another, larger group that sits quietly.  What we really want is for the quieter students to talk more and the Fab Five to listen more and pick their spots.  Instead of a polarized class of talkers and non-talkers, we would like to push the poles towards the center, where everyone is engaged.  How can we accomplish this?

      Enter the engaing more students framework:
      • Pairs or small group discussions to get people talking
      • Instructor visits
      • Not asking for volunteers
      • Calling on groups to "Share what you discussed."
      Let's say a student or group just presented or shared an idea or solution to the whole class.  Instead of asking the class for questions right away, the instructor can prompt students to discuss what was just shared in pairs or small groups. They should be instructed to come up with at least one comment, question, or (substantive) compliment.

      Groups can be assigned to be the initial commenters.  I sometimes ask two groups to start the class discussion portion of the presentation.  I let them know ahead of time so they are primed and ready (and not cold called), and I rotate these duties around the class to spread opportunities around the class evenly.

      Next, while groups are discussing, I make instructor visits to listen in and check in.   If there are 10 groups in a class, I plan on visiting 2-4 per presentations, and remember where I left off.  In the next round I pick up where I left off.  The purpose of visiting is to check in to see how students are thinking so I get a sense for where people are at.  I can also help groups sharpen their questions or comments, and get groups used to talking about the topic so that it is easier for them to chime in.  I also make it a point to talk to the quieter students in my classes in a friendly way, usually 1-1.  "Hey Pat, what did your group talk about?..."

      After a short period of time, it's time for a whole class discussion.  One of the main things to avoid is asking, "Are there any questions?"  Instead, I have a plan to call on the pre-selected groups or ask specific groups to comment, where I spread the opportunities evenly over time.  An easy way to do this is to have a seating chart or diagram of the groups and go through the list in order and then repeat. Another simple organizational strategy is to call on the groups who you visited first, and then open the discussion to anyone else.

      A key idea here is that you are not asking people to opt-in, by asking for volunteers.  Opt-ins allow students to do nothing to opt-out.  Passive students can stay passive to opt-out, by not responding to "Are there any questions?"  That is, it takes zero effort for students to do the thing you don't want them to do.  Instead by using this framework, all students are prepared to say something. Calling on students in a way that is not stressful and spreads the work evenly ensures equal opportunity for students to participate. Small group discussions also encourages students to engage every round.  (Hence, creating an environment that supports learner agency.)

      One of the positive effects of this framework is that the quieter students engage more and the "Fab Five" are not dominating class discussions.  All students can be engaged, and this offers a greater sense of the class working as a team to learn together.   Class discussions are more useful, and more questions can be asked and addressed.

      Flexible Framework:  The example above can be adjusted in many ways.  I'll just mention a couple ideas, and let readers take it from there. Instead of a student presentation, the instructor could present a problem or ask students to work on an example or review what has just been presented.  The framework can then be used at that point.  It's also worth pointing out that the framework can also be used by instructors who primarily lecture, and and can be a launching point for instructors to start the journey towards active, student-centered instruction.

      Details: some important details include using quality math tasks, avoiding instructor questions like "Are there any questions?", using good, professional technique when doing instructor visits, student buy-in, and setting up a class culture where students feel comfortable discussing and sharing their ideas.

      Contrast: Let's look at the opt-in model, where we ask, "Are there any questions?" In this case, students can passively opt-out by doing nothing, and we get the common split where only a small group of students talks and the rest sit back.
      The final takeaway is that something useful and doable can be done in essentially any setting to get more students or even all students involved in class daily.  All instructors need is to start with the engaging more students framework and find ways that work for them to implement some form of it.  


      Friday, March 11, 2016

      More on Productive Failure (#PF)

      Another term, another round of productive failure (or #PF) implemented in my classes.  Last term I used #PF in Calculus 1 with freshman.  Previously I included it in a course taken by math majors in the teaching option (i.e. preservice secondary math teachers).   This term (Winter Quarter 2016), productive failure was implemented in a course for future elementary teachers.

      This term, students were asked to share two instances where they learned from being stuck or making a mistake.  Sharing would be done via short presentations to the whole class.  Learning is what is emphasized in productive failure presentations, and students were asked to share what they were stuck on and more importantly what they gained from it.

      The purpose of highlighting productive failure is to de-stigmatize mistakes.  Most all students in the class commented in their reflective writing assignments that they were taught that mistakes were bad. Mistakes were to be avoided. The goal was to be "perfect." The problems with this type of mindset is clear.  Students who are afraid to make mistakes are not fully engaged in their learning process. They experiment less, are reluctant to try something new, and focus on getting right answers instead of primarily focusing on why things work the way they do.  This is all the more important for future elementary school teachers.  If young children are taught at an early age that memorizing (and only memorizing) is the key to success in math, then they will learn far less than they are capable of. It's a crippled way of doing mathematics really.

      Productive failure ties in with Dweck's notion of a growth mindset.  Early in the term, I share a short video of Carol Dweck interviewed by the Khan Academy to help set the stage.

      Additionally students are given reading assignments (outside of class) based on Jo Boaler's book, What's Math Got to Do With It?   This book is about major issues in K-12 math education in the U.S., what doesn't work, and some research-based claims of what works.  Boaler's book adds another layer of support for #PF, offering more reasons why productive failure is an important aspect in successful learning.

      As the course progressed, more students shared their experiences of being stuck, and after each presentation, they earned a sticker.  Stickers are not necessary, but they add a fun way for students to keep track of how many #PFs they have presented.  Stickers also served as a model for how they could implement #PF in elementary school classes.


      More progress!

      Productive failure is fundamentally about unlocking a robust learning process.  When math classes, especially in K-12, are overly concerned about answer-getting, then students (children) learn to focus almost exclusively on memorizing steps and worrying about not being wrong.  And that's when seeds of math anxiety are planted! Education is about learning to become a powerful learner.  Far beyond memorized algorithms or basic facts lies a much more valuable and useful education.  Society needs independent, critical thinkers, who are filled with curiosity, creativity, and tenacity.  You can't develop these traits, if the predominant concern is not making a mistake.