Don’t Teach TO the Test, Teach BY the Test(ing)

This final post in a short series based on the book Make it Stick: The Science of Successful Learning, by Peter Brown, Henry Roediger III, and Mark McDaniel, focuses on how teachers can use new developments in the field of educational psychology to improve their teaching and their students’ learning. If you haven’t read the previous posts, go back and do that now — I’m not repeating the introductory material here.

Use Testing as a Teaching Tool, Not Just an Evaluation Tool
The most important insight from Make it Stick for teachers is that testing is a method of teaching as much as a method of evaluation. When I was in education classes, we learned all about the importance of having “formative” evaluations throughout a unit of study, rather than just “summative” evaluations (like an essay or unit test) at the end. The goal was to help teachers know what their students don’t know so they can revise upcoming lessons and review material that the students have misunderstood. As it turns out, those formative evaluations were as important for the students as they were for the teacher — by testing the material along the way, teachers actually help students learn the content. Testing strengthens the connections between ideas in the brain, so when we test students on what they need to know, it both clearly reveals what they’ve missed and helps the students mentally organize the material.

An additional insight is that the testing need not even wait for the initial teaching: asking students to take a “pretest” on course content helps them cement the facts, terms and concepts before they’ve initially learned them — and it works even if the students do no better than chance on the initial assessment. (For an overview of the pretesting method, see Benedict Carey’s article in the New York Times Magazine). Simply priming the students to think about the important ideas in the course (by giving them a test that will resemble a unit test or final exam) also helps them to differentiate between the “big ideas” and the “filler information,” thus helping even less well-prepared students succeed more fully.

Write to Learn
Teachers can also use the “write to learn” method: at the end of a lecture or class period, the teacher asks the students to take out a piece of paper and summarize the new material in their own words. Students are asked to define key terms, connect new content to previous chapters or units, and come up with additional examples of things discussed in class. This can be done as either a class activity or homework assignment, and it’s a very high-payoff activity: in one study, students scored half a letter grade higher on questions related to concepts that they had written about (89-90).

Give Honest Feedback
Another important idea here is that teachers need to make feedback continuous and honest: we do our students no good by hiding their failures from them. If a student’s initial answers are incorrect or incomplete, we need to make that absolutely clear. The authors suspect that one of the reasons that students so consistently overestimate their competence and comprehension is that they do not get clear, unambiguous feedback before it’s too late. Teachers might think that grades and comments are clear, but if the students are not getting the message, we owe it to them to make sure they understand not only the course material, but also their own understanding of the course material.

Teach Students How to Study
Finally, the authors suggest that teachers explicitly teach their students how to study. Teachers of high school and college aged students might think their students learned to study years ago, but surveys of students indicate that they spend most of their time on the least efficient studying methods (like rereading) and most do not employ testing and other high-value study methods. By teaching our students both how and why to study, we do them a huge service that goes far beyond our individual course.

Study Better, Study Less

In my last post, I reviewed and summarized some of the myths of learning that Peter Brown, Henry Roediger III, and Mark McDaniel offer up in their book Make It Stick: The Science of Successful Learning. They explain that students study inefficiently, by rereading huge chunks of text, and ineffectively, by mistaking fluency with the material for knowledge of the material.

This is borne out in my experience. When I surveyed a group of 150 high school students about what they would do to study for a major test, I got answers like:

  • “I look over my textbook and really focus on the material.”
  • “I look over notes and attempt to jog my memory by reviewing previous lessons. Reading over the textbook also helps if I have time.”
  • “I generally re-read relevant chapters in the books I use to study and review notes.”
  • “Skim the book and notes.”
  • “Review notes, read a textbook, read more textbooks, play video games.”

That last method certainly might not sound very effective, but the critical insight that Brown et al. offer is that none of these methods works. They all trade massive amounts of study time for little in test performance.

For students, the obvious response is, “Okay, I get it, I’m studying wrong. But what should I do instead?”

Thankfully, Make It Stick’s authors don’t leave us stranded, contemplating our failures. They offer a number of scientifically proven strategies for both learning and retaining course material. Students who switch to these forms of studying are likely to spend less time studying, while increasing their overall performance (sure sounds like win-win to me!). These are only a few of their many recommendations — if you want to know more, buy the book!

Strategy #1 — Embrace the Power of Testing
Instead of rereading, Brown et al. identify testing as the critical method of learning and studying. By “testing,” they don’t mean high-stakes tests given in school. Instead, they recommend that learners embrace the power of forcing your brain to recall things it does know and clearly identify things it does not. Fascinatingly, the harder the material is to remember, the more benefit gained from trying to remember it. Another name for testing is “retrieval practice,” and it’s exactly what students do when they make flashcards. The authors explain that the benefits of testing are twofold: “One, it tells you what you know and don’t know, and therefore where to focus further study to improve the areas where you’re weak. Two, recalling what you have learned causes your brain to reconsolidate the memory, which strengthens its connections to what you already know and makes it easier for you to recall in the future” (20).

Remember those myths of learning? Retrieval practice counters a number of them — it is difficult (which helps the memory stick), it helps you focus on the most important parts of the material (rather than a whole chapter or book) and it identifies exactly what you do and don’t know, eliminating the fluency effect.

Even better, it works even if you get the answer wrong or can’t remember the answer at all. The authors’ research indicates that “Every time you work hard to recall a memory, you actually strengthen it. If you restudy something after failing to recall it, you actually learn it better than if you had not tried to recall it” (203) [Emphasis added]. That’s astonishing. Even when you get things entirely wrong, the mere process of trying to remember them correctly helps you in the future.

Students who are looking to take advantage of this method should immediately switch from rereading to retrieval practice: flash cards, study guides with blanks for the terms or definitions, and quizzing by a parent or friend are all easy ways to take advantage of the power of this method. Students should also practice summarizing in their own words what they just read or learned in a class. It’s an easy way to start to mentally consolidate the material. Taking two minutes while waiting for the next class to start to write down what you learned in the last class pays huge dividends down the road when it comes time to take the test.

Strategy #2 — Space Out Your Practice
The second key strategy or technique for improving learning is also relatively simple — space out practice or studying. Big blocks of singly focused time is called “massed” practice, and while it feels like it’s working at the time, the gains don’t last. Spacing out practice feels more frustrating—when you start up again, the material is hard to recall—but that feeling of struggling to remember is actually your brain making new connections to the material. As a bonus, multiple short sessions are better than one long session even if the total time is less—and it’s certainly easier to schedule lots of short sessions around other schoolwork and activities than it is to find one long uninterrupted block of time.  So: study in smaller chunks, and you can both learn the material better and spend less time on it!

Strategy #3 — Mix Up Your Practice
Another way to get the benefits that come from a little struggling is to switch back and forth between different subjects during learning. This is called “interleaving,” and it’s a good way to make your brain more flexible and increase long-term retention. If you have two tests to study for, don’t do all of the studying for the first test and then all of the studying for the second — switching back and forth before you have the material down cold is the best way to keep your brain on its toes. Brown et al. explain that used together, spaced practice and interleaved practice produces vastly better results: “Practice that’s spaced out, interleaved with other learning, and varied produces better mastery, longer retention, and more versatility. But these benefits come at a price: when practice is spaced, interleaved, and varied, it requires more effort. You feel the increased effort, but not the benefits the effort produces. Learning feels slower” (47).

The big message here is that learning that feels difficult is more productive than learning that feels easy. When you make your brain work harder during the learning process, you are setting yourself up to succeed in the recall process (aka “the test”).

Strategy #4 — Generate a Solution Before You Know the Correct Answer
A hot new technique in education is pretesting — giving students a test on the material before they’ve learned it. (Benedict Carey offers a short introduction to the pretesting theory in his recent article in the New York Times Magazine). The idea is that when you are forced to think about what the answer might be before you know for sure, your brain pays more attention to the actual answer. It’s like solving a puzzle: getting the questions in advance piques your brain’s curiosity, so it attunes more carefully when the material is finally introduced.

Students, it’s unlikely that you can convince all of your teachers to give you the final exam on the first day of class, but you can still use this strategy: when you start to read a chapter, look at the title of the chapter and any section headings before you start reading. Predict what the chapter will be about, what key terms might be included. If you have a textbook that has questions at the end of each chapter, this is even easier: try answering the questions before you read the chapter. You will remember the chapter better even if you get the questions 100% wrong. Pretty cool.

These are a few of the most useful strategies for current students, but they’re by no means the full set of content in the book. When I initially generated the list of key strategies from the book, I came up with fourteen specific, easy-to-implement ideas for improving learning and studying for students. Students who are interested in being more efficient and more effective (and who isn’t?) should check out Make it Stick.

The final post in this series will consider how teachers can apply the same types of strategies in their classrooms to improve student learning.

Learning: We’re All Doing It Wrong

One of the best books I read this summer is Make It Stick: The Science of Successful Learning by Peter Brown, Henry Roediger III, and Mark McDaniel. It’s a fascinating contribution to the emerging set of “how humans learn” books written for non-scientists, and I highly recommend it to students, parents, teachers and learners of all ages. I desperately wish this book existed when I was a high school or college student, because I discovered that… I did it all wrong. Diligently re-reading the course material before the test? Wrong. Painstakingly highlighting terms and definitions? Wrong. Sitting down, sans distractions, to “really focus” on the content in hopes of burning it into memory? Also wrong.

It’s not just me. When I surveyed students at both Woodward and a summer program I worked at, a few themes emerged: good students invest tons of time into studying where they reread everything they think will be on the test, often multiple times, and sometimes employ additional strategies like highlighting, making a study guide, taking notes on the text, or reviewing notes from class.

We can do better! This post offers a breakdown of where learning goes awry, by addressing some of the most common myths of teaching and learning. The next post will consider what students of any age should do differently, and then the final post will address the same set of concepts from the teaching perspective — what can we, as teachers, do to help our students more effectively learn and retain the material we teach. I suggest reading them in order because each delves into the methods that Brown, Roediger and McDaniel recommend.
The book is set up as an antidote to common poor study habits. In eight chapters, the authors first demonstrate that the average learner holds a number of mistaken beliefs about learning, and then systematically offer alternatives to those common methods.

What are these mistaken beliefs? I’ve broken them down into five major components:

Myth #1: Fast, Easy Learning is the Best
The authors explain that most people (and especially teachers!) believe that when learning is fast and easy, it’s because it’s going well — if a teacher can just make the learning smooth and comfortable, it will be memorable to the students. Unfortunately, this is not only incorrect, but backwards: “Learning is deeper and more durable when it’s effortful. Learning that’s easy is like writing in sand, here today and gone tomorrow” (3). When students are breezing along, feeling no stress or effort, the material is either too easy (they already knew it) or not being learned.

Myth #2: Rereading is a Good Way to Study
Rereading has the benefit of feeling productive: it “looks like” studying to a curious parent, and how could you possibly be unprepared if you’ve read the book over and over? Easily, as it turns out. A study from Washington University tested just how much we learn when we reread. Unfortunately, the results weren’t uplifting: “Multiple readings in close succession did not prove to be a potent study method for either group, at either school, in any of the conditions tested… The researchers found no rereading benefit at all” (15). Equally importantly, rereading doesn’t just waste a little time, it wastes a lot of time. Most of the more effective methods are both better and require less seat time.

Myth #3: Feeling Like You Know Something Means You Know It
We’ve all had the experience — you go into a test, thinking you’re ready for it. You studied plenty. Maybe you even made a study guide! You sit down, open the test and… no. The words in the questions seem vaguely familiar, but the answers? You don’t have them. This is called the fluency illusion, and it’s one of the most painful discoveries of teaching and testing. Because you read the words in the textbook or class notes, often over and over, you start to believe you know the material. Instead, you’re just familiar with the words or concepts, but when push comes to shove (or exam time comes), you don’t.

Myth #4: We Know What We Don’t Know
This is the flipside of Myth #3. Not only don’t we know what we think we know, but we also don’t know what we don’t know. (Confused? I’ll explain). Students, especially those who are new to a concept or field of study, consistently overestimate what they know. When a professor or textbook explains something clearly, students mistake the clarity of the explanation for the clarity of the concept. They then believe that the reason it was clear is because they already knew it and thus don’t need to study (17). These mistaken beliefs of knowledge are also rarely challenged before the final exam: whether it’s an actual lack of negative feedback or misunderstanding what the feedback means, students often do not get the message.

In debate class, I call this the “Sophomore Problem.” Students who join the team as clueless freshmen return sophomore year feeling like they know everything. They learned so much that first year that they no longer have a concept of just how far they have to go — and now that the debate jargon isn’t totally intelligible to them, the things that their teachers and coaches say start to seem really obvious, even if they aren’t.

Myth #5: “Just Sit Down and Study!” Long Blocks of Single Focused Study Are Effective
I can still hear my mom’s voice: “Maggie, just sit down and study!” I used to block off huge chunks of time to review for a single exam and believe that if I just did what I was supposed to do, I would be ready. And certainly during that study session, I did feel like I was getting better. (But remember, that feeling shouldn’t be trusted!) Brown et al. explain that “Most of us believe that learning is better when you go at something with single-minded purpose: the practice-practice-practice that’s supposed to burn a skill into memory… Researchers call this kind of practice ‘massed,’ and our faith rests in large part on the simple fact that when we do it, we can see it making a difference. Nevertheless, despite what our eyes tell us, this faith is misplaced” (47).

As a teacher and coach, I made this mistake all the time. I thought that if I taught a skill until everyone could do it, then the students would be good to go. Unfortunately, it doesn’t work — those improvements that we see just don’t last. A third grader who can’t recite her times tables today might not be able to tomorrow. Sixth graders who can throw a football with a perfect spiral today will be just as hopeless at it by next week. The 10th graders who understand stoichiometry on the worksheet won’t be able to do it on Friday’s quiz. And adults who sit down to learn basic Italian won’t be able to order a pizza by the time the plane touches down in Rome.

Okay, so we’re all studying wrong. What should we do differently? Check back soon for Part II, the student’s perspective, and Part III, how to teach so students will learn. In the meantime, go order or download Make It Stick!

RIP3 Reflection #1

This week involves looking at reflection, using Rhonda Mitchell’s RIP3 protocol.  The idea here is to reflect more fully on learning experiences, moving beyond simply recalling the experience.  Trinity seems to be using this effectively with very young students — today’s the day for adult-me to try it with one of my learning experiences.

Recently I’ve been doing more professional reading.  One of my personal goals is to read 45 books in 2013, with a significant percent of them being books on thinking, teaching, learning, and coaching.  I’m well on the way to accomplishing that goal (38 of 45 done as of today).  I’ve decided to reflect on one of the books I’ve read and lectured about recently as a way of both practicing the protocol and distilling what I learned.

Remember – Describe what you did.
I read Ashley Bronson and Po Merryman’s most recent book Top Dog.  It’s a look at how competition influences learning and performance, both positively and negatively.  While the subtitle is “The Science of Winning and Losing,” I found that the book went beyond that to talk not only about winning and losing, but also attitude and life-long success.

After reading the book, I put together a lecture for the students at summer debate institute. While the lecture was substantially drawn from the book, it also brought together a number of disparate threads on the growth mindset, coaching, teamwork, and attitude from lots of other sources.

I gave the lecture three times this summer — twice to relatively homogenous groups (one sophomore class and one senior class) and once to a mixed-level group as an elective lecture.  Each time I modified the lecture to focus on the learning in that group.

Identify the Important Part – What was important to you about this learning experience?
I was able to share something that I enjoyed reading with several groups of students.  I also had the time and focus to bring together many of the things I’ve been learning over the last six months.

Put it Together – What does this remind you of? What kind of connection can you make between this learning experience and something else you know?
This reminds me of other lectures I’ve written, but I enjoyed this one more than the others because I was able to write it far in advance and then modify it for a particular group a day or so before that iteration.  Too often I’m writing summer lectures only a few days before I need to give them.  By starting the lecture writing process in May, I had more time and attention to bring together lots of ideas instead of simply parroting a single source.

Pick it Apart –What are the things you have learned from this experience? What do you now know?
In the literal sense, I learned a lot of interesting things about winning, losing and competition.  I’m better able to understand how and why students respond differently to the same competitive pressures.  More broadly, I also learned a more effective (for me!) method of preparing for lectures and a little about why that might be the case.

Plan to Use – How will you use what you learned from this experience in the future?
I’ll (try to!) stop worrying that I’m preparing too far in advance — it seems like while some people like the pressure of working on deadline, I do better and deeper thinking without it.  After reading Top Dog, this doesn’t surprise me at all — while some people substantially benefit from pressure, I’ve never enjoyed working that way.  It turns out there may be some interesting biological bases for some of those differences.  (Want to know more?  It’s a great book!)

Checking Back In: Reflection as Assessment

We’ve been using debate logs with the students for almost 2 months now, so it’s time to take stock of how it’s working as a reflection and self-assessment tool.

Overall use has been somewhat uneven (this is for an extracurricular team, not a grade, though we do check in with debaters who aren’t logging regularly).  Several students log their activities and habits every day and are definitely using the log to see what they need to continue to work on.  It’s great for the coaches to be able to see what they’re working on and reach out if they’re struggling.

Others require more encouragement.  One of the older debaters sent an email yesterday to remind everyone to update.  Immediately after he sent the email, I got notifications that many debaters had written.  It’s great to see the students taking on leadership roles and helping each other out.

They also use the logs as opportunities to notice and praise others.  Here’s a selection from a student’s log from today:

“Warmed up with one of [Student]’s 1NCs. I love that kid right now for all the work he does.

I should start getting ready for [upcoming tournament]. So I did. Here’s what I got done in class:

  • Timed how long it took to read through the entire thing—10 minutes and 18 seconds…

  • Spent the rest of class highlighting down the file. There isn’t time to try and re-read it now, but I’ll do it tonight. That’s a promise!

  • Note: I need to just look at the Aff blocks that [Other Student] wrote, to use as a template for writing my own.”

I love it that the students are considering not only what they are doing (and can do better) but are also looking to each other as a model of what they can improve!

Learning and Success

Jay McTighe offers thoughts on learning a new skill even at an old age.  For McTighe, the skill was surfing in his 50s.  For our students, the skills are reading, writing, math, science — the list goes on and on.

Many of the students who join the debate team in 8th or 9th grade do so because they think they will be good at it right away.  And they will be good at it — but not right away.  One of the biggest challenges is helping students (many of whom aren’t used to struggling with academics) learn an activity that requires new skills, new vocabulary, and new ways of thinking.

Many students get frustrated when they can’t put it all together immediately.  They have all sorts of great ideas, but they struggle to translate them into competitive debates.  The job of the coach, then, is to remind them exactly what McTighe had to remind himself: “Good learners persist… Good learners are strategic.”  Learning how to debate requires both persistence and a strategic approach — no student can learn everything immediately, and they will experience failure, both competitive and personal, before success.  At every tournament, each student has 4-10 individual debates, and it’s very rare for a student to win them all.  That means that debaters encounter failure both regularly and persistently.

In an Op Ed in the New York Times, best-selling author Ashley Merryman writes, provocatively, that “Losing is good for you.”  She explains,

“Po Bronson and I have spent years reporting on the effects of praise and rewards on kids. The science is clear. Awards can be powerful motivators, but nonstop recognition does not inspire children to succeed. Instead, it can cause them to underachieve.” [Emphasis added.]

We need to teach students to persist when they fail, and that anything worth doing is hard.  Instant success not only doesn’t help inspire long term dedication, it can produce the opposite.  Constantly trying new things ourselves, even at ages when some people have stopped, is critical to both understanding our students and being able to continue to inspire them.