In a Rush to Judge

The current fad in education news in mainstream media is the Value Added Model or VAM.  It goes like this.  There are a bunch of bad teachers at the cause of our woes in education, and we need to find them.  So they are evaluating teachers, and applying business model methods like VAM.  There exists a group of people in the United State who presuppose that the problem with education is a bunch of "bad teachers."  The purpose of my post is to redirect this discussion into one that is based on reality and one that would actually help us solve real problems.  The current black-and-white debate about education today misses many important issues and ultimately, in my opinion, will be counterproductive, leading to inadequate responses.

I point out that essentially every person in our country wants to improve education and help children become good thinkers.  We all can agree on this.  The problem with our debate is that we are focusing far too much on teacher evaluation, while missing the larger picture.  In this post I explain what I mean by this.

First, pointing fingers at one group or another is counterproductive.  The current media attention has been to place blame on teachers (and teacher unions) and to promote the use of VAM-based evaluations of teachers.  Then using this data, some districts have publicly humiliated teachers who perform poorly by publishing this data.  Even business leaders know that public humiliation is not the way to bring about positive changes, as Bill Gates writes.

Second, VAM appears to have dubious intellectual or scientific merit for teacher evaluation.  Jesse Rothstein (UC Berkeley) published a paper that indicating the using VAM is not suitable for measuring teacher effectiveness.  He states, "My results indicate that policies based on these VAMs will reward or punish teachers who do not deserve it and fail to reward or punish teachers who do."  The essence here is that certain assumptions being made are actually false.  VAM might work if we randomly assigned students to teachers.  In the U.S. we track students.   If I know who your child's teacher is in 5th grade, I can infer information about how you did in 4th grade.  In order for VAM to work, it has to be the case that know a child's 5th grade teacher has no influence on how the child did in the 4th grade.  This condition is false.

To put it simply, we may be using a hammer to paint our walls.  Wrong tool. Wrong job.  We could realize in the future that we fired a bunch of teachers erroneously.  It would be deeply unethical to use a system without knowing it actually works, and then to make life and career altering decisions based on it.

Third, it is assumed that teachers have adequate control and power, and hence should be held accountable.  Rhetoric based on this premise sounds good from the podium, but unfortunately is incomplete.  Richard Ingersoll (University of Pennsylvania) portrays the climate teachers work in in his article "Short on Power, Long on Responsibility."  With great responsibility placed upon then, they are given far too little control or input on issues that would allow them to do their jobs better.

• teachers generally have little input or influence on overall school curriculum
• teachers have little input on course assignments or class size
• teachers generally have little input into schoolwide behavioral and disciplinary policies
• teachers rarely have the authority to have disruptive students removed from their classrooms
• teachers have little input into hiring, firing, budgetary decisions, and whether to hold back or promote students based on their academic performances
• teachers have little input on the content and form of their own on-the-job development and inservice programs

Ingersoll writes "Teachers are akin to men or women in the middle.  A useful analogy is that of supervisors, or foremen, caught between the contradictory demands and needs to two groups: their superordinates -- school administrators -- and their subordinates -- students.  Teachers are not part of management and they are not workers... although [teaching] involves much responsibility, it involves little real power."  If we are to hold someone accountable for a job, then they have to have commensurate power and control over this job.

Fourth, perverse incentives may come into play that are unforeseen at this time.  No one can think through every detail of a system that involves millions of people.  So here is a hypothetical situation:

Suppose Mrs. Jones has two students with learning disabilities.  These students' disabilities have been undiagnosed until Mrs. Jones takes the time and effort to figure this out and help these children deal with and overcome their disabilities. These students succeed, learn, and are set on path that will lead them to a productive and fulfilling life.  This is a major victory for education, for Mrs. Jones, and especially for her students.  BUT because these students were not able to learn some topics in previous years, due to their disabilities, they do not score well on state tests.  Thus Mrs. Jones is labeled a "bad" teacher.  If instead, Mrs. Jones ignored the evidence of a disability, and had these students removed from her class somehow (i.e. failed them), she would then be labeled a "good" teacher, but at great cost to the two children, their families and to society.

We know how perverse incentives can lead to terrible outcomes.  The financial crisis of 2008 is a prime example.  The numerous cases of school districts cheating on high stakes testing is another.  If your job depends on a misguided set of incentives, then you will get tragic results.  We must carefully think through the evaluation system to make sure it works without creating incentives to cheat or to make unethical choices.

Fifth, it is assumed that test scores measure something of great intrinsic value.  VAM is based on test scores.  Standardized testing is a long separate discussion in itself, so I will not describe the tests and how they are inadequate in great detail.  Standardized tests in the U.S. focus primarily on rote skill types of tasks.  These are the kinds of tasks that do not involve high-level thinking or require deep conceptual knowledge.  Moreover, these tests do not measure effectively problem-solving ability or other higher-level, critical reasoning abilities.  It's not that the tests are useless, but there is some question about what they actually tell us.

I'll argue now by way of analogy.  Free throw percentage is a measure of a particular skill in basketball.  In math, it could be something like solving linear equations or adding fractions.  Here is a ranking of NBA players by Career Free Throw Percentage:

1. Player 1, 88.9%
2. Player 2, 87.7%
3. Player 3, 85.4%
4. Player 4, 84.8%
5. Player 5, 83.5%

If I asked you, who is the best player in the list above, you could say player 1.  Or if you thought about it a bit, you would say that you don't have enough data.  Here are the names of the players:

1. Scott Skiles, 88.9%
2. Jeff Hornacek, 87.7%
3. Mario Elie, 85.4%
4. Spud Webb, 84.8%
5. Michael Jordan 83.5%

With no disrespect to any of the players on the list, but I'd choose Michael Jordan to be on my team, even though he's the lowest in rank.  In fact, let's consider Shaquille O'Neil.  Shaq's career free throw percentage is 52.7%.  He's terrible by this measure, yet, in his prime he'd be one of the first players you'd pick for your team.

Let's relate this back to teaching and learning.  As free throw percentage is an incomplete measure of basketball ability, so are today's standardized tests an incomplete measure of student mathematical ability.  Today we can test for achievement of basic skills, but we do not test for problem-solving ability, proof writing, argumentation, the ability to experiment and explore, and so on.   No kid thinks shooting free throws is the end all, be all of basketball.  No kid thinks bubbling in scantron forms is the heart of mathematics.

The example goes on.  It would not be fair then to compute the team free throw percentage and determine if a coach is good or bad.  Let's look once again at the 1992-1993 NBA season.  The Chicago Bulls were ranked 22nd out of 27 teams that year in team free throw percentage.  By this standard, Phil Jackson should be have been fired as a coach after the end of the season, his name shamefully listed on the internet for all to see with disdain.

What happened that season? The Chicago Bulls won the NBA title over the Phoenix Suns.  The Bulls didn't "test" well, but they sure ripped apart the competition.  Clearly we didn't use the right evaluation tool here.

Thus, while I agree that we need to ensure high teacher quality, this doesn't excuse using the wrong tools or using a tool prematurely before we are sure it's the right one.  I propose that we evaluate teachers in a smart way, and develop a trustworthy evaluation system.  This is a big job, and will require a ton of resources.  Administrators and teachers must come to an agreement on a fair system that uses scientifically proven methods as well as classroom visitation and other forms of data (such as student portfolios).

Sixth, we have lost sight of the whole system.  What we are doing is the proverbial rearranging the deck chairs on the Titanic.  If we continue to teach rote skills in passive learning environments, no matter how well we test our students or evaluate teachers, the probability to transform the system is zero.  The reason why I believe this to be the case is that focusing only on teacher evaluation misses big pieces of the system like curriculum, learning environments, developing critical thinking, community and school support of academics, switching to student-centered instruction (e.g. IBL) that support higher-level thinking, etc.  Teacher evaluation is but one (small) piece.

What should we do?
We should rethink our basic assumptions about education.  Now that we live in the google era, it isn't enough to just teach the three R's (reading, 'riting, 'rithmetic).  We need superbly critical and creative thinkers.  Guy Claxton does a wonderful job of describing what we should do in "What's the Point of School?"  Claxton states that the point of school is the great eight qualities.  They are:

• Curiosity
• Courage
• Exploration
• Experimentation
• Imagination
• Reasoning
• Sociability
• Reflection

These qualities are the kinds of big objectives that supporters of student-center or inquiry-based learning methods agree with and pursue daily in our classes.  We have seen immense growth in our students in these qualities, when allowed the time and space to give our students challenging math problems to solve and discussion collaboratively.

Consequently, what we should be doing is working as a team or community to help teachers create class environments that support the learning of these great eight qualities.  Professional development efforts are the best bang for the buck.  Help teachers use effectively the best teaching practices we have available today with better, more modern curricula.  Other countries are more supportive of their teachers, especially high performing, modern nations like Singapore, Japan, Finland, and South Korea.  Some of these countries view their teachers as "nation builders."  Teachers are supported, respected, and given working conditions that allow them to be more effective in the classroom.

Pointing fingers at teachers is counterproductive, and in fact, it is potentially destructive, since we are ignoring more pressing issues.  If hypothetically a foreign power wanted to dismantle our educational system, what strategy would they employ?  Would they like to see us working together with civility towards a common goal or pointing fingers at one another?

In our rush to judge teachers, we have lost sight of some of the overarching issues that need to be addressed.  It's time to shift the discussion to the great eight qualities and how we can pull that off in our classrooms.  Teachers are the ones who will implement this, and they need our support and our respect.   I encourage teachers, parents, administrators, students, and policy makers to rethink basic assumptions about education, and consider how we can work together, without pointing fingers, to upgrade our system to better educate our youth. 

Upward and onward!

Thinking About Thinking

One of the main reasons why IBL produces superior learning outcomes compared to non-IBL teaching methods is metacognition.  Metacognition described simplistically is thinking about one's own thinking.  Professional academics do this as a habit of mind.  We ask ourselves, "What is my approach to this?"  or "The way I'm thinking about this is..."  It's one of the reasons why we are peculiar.

Students sometimes (often) do not think about their thinking.  Most have not had experiences in school that support this.  This is easy to think about.  If all you do is follow rules and procedures to compute algorithms you don't have to think about your own thinking.  All you need to do is follow the thinking of someone else.  My intuition about why passive learning fails for most people is that unless you are predisposed to independent thinking, there is little in traditional education that can transform one towards independent, critical thinking.   Monitoring one's own thinking is part of the sophisticated set of thinking processes that distinguish experts and novices.  And it can be trained!

Where this comes into play in IBL math classes boils down to this:  students in IBL math classes must explain their reasoning on a regular basis.  I claim that the process of justifying answers engages the metacognitive process.  It goes something like this:

Student: "I believe the statement is false."
Instructor: "Can you tell us why?
Student: "Well... I looked at these examples and then I thought that this one here doesn't satisfy the second condition."
Instructor: "Very interesting.  What do the rest of you think?..." <discussion ensues>

The support of metacognition in IBL classes is much richer than what is presented in the vignette above.  Students are stuck on problems.  They are proving theorems from first principles, and are asked to write proofs outside of class.  Moreoever, students are required to present their proofs to the entire class for peer review.  Students cannot get through class without having to think about their thinking.

Some questions to get you to think about your students' thinking about their thinking:

• Do your students think about their thinking and how do you know?
• If you are not sure students are thinking deeply about their thinking, what can you do in class and via assignments to encourage this?
• What mathematical tasks and class setup could you use to support thinking about thinking?

"Wow, that's amazing!"

Calculus is one of the greatest intellectual achievements in the history of mankind!  Very few other achievements rise to the level of Calculus.  Indeed, Uri Treisman said in 1992,

"The subject drips with power and beauty.  It rendered thousand-year-old questions immediately transparent.  Calculus is truly amazing.  But, how many students who take the course as freshmen look up and say, `Wow! That's amazing!'?  How often, math faculty members, have your students had that experience?"

These questions, 20 years later are still relevant.  When I talk to students about calculus, some of them like it, many are afraid of calculus, and very few have a grasp of the subject that goes beyond mere surface computations.

I emphasize emphatically that no one wants this to be the outcome.  This isn't something that people intend to happen.  But it is our reality.  Making our courses into "Wow!" experiences on an intellectual level is a challenge that is presented to the mathematics profession.

So here we are with this tremendous gift of having one of the greatest set of ideas of all time in our hands to teach to hundreds of thousands of freshmen every year.  Should we not feel in our hearts, "What a great opportunity!"?  If calculus is just a job, then there's room for improvement.

IBL Levels (with apologies to Van Hiele)

In this post I present the IBL levels, a la the Van Hiele levels, which sincere apologies to Van Hiele. These are based on my observations of IBL instructors over the past decade.

A very important point is that the IBL levels presented here are simplistic (reductionist) and are intended as a form of guidance to instructors.  The IBL levels indicate the direction in which you ought to go and provide a framework for evaluating your own teaching.

Level 0: non-IBL, teacher-centered instruction.


Level 1: some student engagement via occasional "active lecture" methods like
Think-Pair-Share, concept questions, or students working out examples.  The instructor and textbook remain as the predominant mathematical authorities, where students seek confirmation from the instructor or textbook to check if their answers are correct.


Level 2: a significant percentage of class time (50% or more) is used to engage students in solving problems, discussing solutions, and peer reviewing work.  The problem solving activities are focused on problems that students do not know the answer to or are not shown the strategy or solution method.  Students are supported and encouraged to make their own conclusions, think for themselves, and share their insights.  Students have a predilection for determining the correctness of solutions without always seeking external validation from an instructor or book.  Students have frequent opportunities to engage in rich mathematical task.


Level 3: full IBL.

While level 3 is a lofty goal to set for oneself for teaching all courses, level 2 is more easily attainable, especially for courses like freshman calculus.  Levels 2 and 3 are then goals that can be achieved by math instructors.  In particular, level 2 does not require wholesale changes in curriculum, and is thus appropriate for various contexts and environments.  It should be noted, however, that level 2 does not have the same potential as level 3 for transformative experiences.  There are always tradeoffs.

For novice IBL instructors, you do not have to wait for the "right course" to begin your journey in IBL instruction.  Level 1 can be done in any class, and allows you to test the waters and build up your own IBL teaching skills and understanding of IBL methodology.  Making the transition to level 2 would be much easier, and it sets the foundation for making the transition to full IBL instruction.

If you have been hesitating, start with level one and try out Think Pair Share!

Another look at the 'Coverage Issue'

One of the most common complaints or worries that instructors make is that there is so much material to cover.  I agree.  Many people agree.  Courses are jammed packed with far too many topics.  In some cases, this is warranted, and I am the first to agree that there exists a vast amount that students need to learn.  No one in their right might wants to teach a course with very little content.  The debate then is about the mix of course content.  This mix should be balanced with concepts, problem solving, opportunities for real growth, and of course contain an appropriate number of topics.

The coverage issue become problematic when we sacrifice understanding for the sake of getting through the long list of topics.  I'm going to use the words of college students to convey the core ideas. Keep in mind that these students were some of the best in their high schools, have taken at least precalculus, and most of them have had calculus.  Several have had some upper level courses.

• "I feel as if this was the first math class ever that I actually took the time to understand what exactly a graph was saying.  Before graphs were just lines, or parabolas, that I plotted..."
• "I'm astounded by the fact that I only did 5 actual math problems in my middle and high school careers.  I drilled procedures, and didn't really learn anything from it."
• "The idea of teaching students how to spit out answers to problems is not effective in learning.  As a learner, I would like to understand why we do math a certain way and not that it is 'just done this way.'"
• "I had never known why we 'completed the square.' I understood that completing the square produced an equivalent expression or function because whatever was added was also subtracted, but it always seemed so complicated.  For the first time [using algebra tiles] I really had a clear understanding of what was going on when I completed the square."

I re-emphasize that these are college students, who are eager to learn.  They have done algebra for years, and have not ever been allowed to explore why very basic things make sense.  They do not report learning as much as they wanted to or could have from earlier courses.  We know those courses are all "jam packed" with content from top to bottom, and that there is "no time to do anything else." Yet students are not reporting that they learned what they need to learn, which begs the question, "What is this all for then?"

Instructors often do not have much control over how much material a course must cover.  In this way, there exists a systemic issue or crisis.  Our system measures coverage in antiquated ways (i.e. a list of topics) and does not include covering "practice standards," such as problem-solving ability, communication, understanding concepts, and being able explain ideas.

To IBL instructors this poses a Teaching Minimax problem -- Minimize as much as possible unnecessary content and maximize the amount of time available for students to explore why.

More on this in future posts...

"Do You Understand?"

Perhaps the question, "Do you understand?" is the most commonly asked question by teachers.  All instructors have uttered this, but there are better ways to get at the issue.   In IBL classes students must demonstrate their understanding through careful explanations of their solutions to the whole class (and sometimes in small groups).

Asking someone if they understand can be unreliable.  One reason is that students may give the answer that they think the teacher wants to hear rather than say the through.  Another reason is because students may not be able to self-evaluate on the spot the level of their understanding.  Rather than ask if they understand, it might be more useful to ask students to complete another task that would allow them to demonstrate their level of understanding.

Of course asking "Do you understand what the homework assignment is?" is different than "Do you understand convergence?"  The point here is that if you want know if students understand a mathematical concept, then it is best to ask them to demonstrate it to you rather than just ask if they get it.

Some alternatives:

• "Now that Joe solved problem N, let's see if we can take it another step.  Here's a related problem... (write problem on the board)."
• "I would like to know if you understand this concept.  Here is a question that I'd like you to work on and answer in the next few minutes..."
• "Can you come up with conjectures that extend the problem just presented?  What else might be true (or false)?"
• "We have just started a new section, and there have been a couple of presentations on problems related to definition 25.  Here's a question about this definition I'd like you all to answer..."
• "Janis just solved problem 35.  Using a similar strategy to Janis, how would you solve this type of problem...?"

From the Mouths of Mathematicians

"The first things i learned about mathematics was that there is a hell of a difference between, on the one hand, listening to math being talked about by someone else and thinking that you are understanding, and, on the other, thinking about math and understanding it yourself and talking about it to someone else." Sarah Flannery

Reuben Hirsch from his book What Is Mathematics Really?  "Mathematics is learned by computing, by solving problems, and by conversing, more than by reading and listening."

It is fair to say that mathematicians enjoy doing mathematics.  We enjoy solving puzzles, working on ideas, making connections, and all that.  It is the so called "heart of the matter," and one of our goals as instructors is to make this happen for all our students, at whatever level they are at.