One scientific insight for curriculum design

I’ve made the case before that our curriculum and assessment isn’t designed with memory in mind. Here’s what I spoke about at ResearchEd York: what we can do to improve how much our pupils remember of what they’ve learned.

 

There’s a mismatch between what science suggests and what schools do on this.

 

A century of scientific study converges on a key insight for our design of curriculum and assessment: an insight that can be put work immediately, widely, at no cost, and to great effect.

 Image

In the scientific literature there are hundreds (if not thousands!) of studies on this, some from as early as 1907, and the research in the last decade is particularly prolific:

 Image

In 2013, five cognitive scientists (Dunlosky, Rawson, Marsh, Nathan, Willingham 2013) collated hundreds such studies and showed that practice testing has a higher utility for retention and learning than other techniques:

Image 

Cognitive psychologists (Roediger, Putnam and Smith 2011) have summarised from this research the many benefits of low-stakes or no-stakes testing:

 Image

The scientists’ review of the vast literature shows that these benefits are enduring (Dunlosky, et al, 2013):

Image

The questions to ask of this research as a teacher are: what’s the optimal format and frequency of low-stakes testing or retrieval practice? For simplicity, I’ll refer to this as quizzing, as Roediger, McDaniel and others do in their work.

Applied research suggests multiple-choice questions are as effective as short-answer questions. The latest research study is as recent as March 2014, so this is a fast-evolving field, and one to keep an eye on.

Image

There’s also a question of whether regular frequency or longer spacing is more important:

 Image

It’s encouraging that the research evidence resonates with the work of successful school leaders like Paul Bambrick-Santoyo, who works in over 1,000 schools with 90% poverty rates and presided over dramatic academic achievement:

Image

Now, over the last decade, eleven cognitive psychologists have taken over a hundred years of laboratory research on the testing effect and applied it to classrooms and subject curricula. Here’s what they recommend:

  • Use frequent quizzing: testing interrupts forgetting
  • Roll forward into each successive quiz questions on work from the previous term.
  • Design quizzing to reach back to concepts and learning covered earlier in the term, so retrieval practice continues and learning is cumulative.
  • Frequent low-stakes quizzes in class helps the instructor verify that students are in fact learning as well as they appear to be and reveal the areas where extra attention is needed. Cumulative quizzing is powerful for consolidating learning and concepts from one stage of a course into new material encountered later.
  • Simply including one test retrieval practice in a class yields a large improvement in final exam scores, and gains continue to increase as the frequency of testing increases.
  • Effortful retrieval makes for stronger learning and retention. The greater the effort to retrieve learning, *provided that you succeed*, the more learning is strengthened by retrieval.
  • In virtually all areas of learning, you build better mastery when you use testing as a tool
  • One of the best habits to instill in a learner is regular self-quizzing.

 Image

As far as I know, very few schools are applying these insights of cognitive psychology in their curriculum and assessment systems. Very few design systematically cumulative tests.

 Image

 

Here’s a very rough ‘before and after’ snapshot of what a Year 7 curriculum and assessment system might look like when changed to put frequent, cumulative retrieval practice at its heart:

 Image

In summary:

  • Repeated retrieval improves long-term memory retention
  • Practice testing outperforms often-used techniques like highlighting and summarising
  • Quizzing has several benefits for memory and motivation
  • The benefits of quizzing for memory retention are long-lasting
  • The scientific research chimes with expert experience
  • There’s a mismatch between what schools do and what the science suggests
  • Cumulative revisiting can improve our curriculum design
  • Recent research is exploring the optimal format and frequency of quizzes

 

I would like to do a much more extensive literature review on this, but for now here are twelve of the papers that I think are useful:

Improving Students’ Learning with Effective Learning Techniques: Promising Directions from Cognitive and Educational Psychology. Dunlosky, Rawson, Marsh, Nathan & Willingham 2011.  

Repeated Retrieval Is the Key to Long-Term Retention. Karpicke & Roediger 2007

Retrieval practice is critical in long-term retention. Roediger & Butler 2010

Test-enhanced learning: Taking memory tests improves long-term retention. Roediger & Karpicke 2006

Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and PlacementMcDaniel, Agarwal, Huelser, McDermott & Roediger 2011

Test-Enhanced Learning in the Classroom: Long-Term Improvements From QuizzingRoediger et al 2011

Generalising test-enhanced learning from the laboratory to the classroom. McDaniel et al 2011

Quizzing promotes transfer in science examsMcDaniel 2011

The power of testing memory: Basic research and implications for educational practice. Roediger & Karpicke 2006

Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. McDermott 2014

Retrieval practice with short-answer, multiple-choice, and hybrid tests. Smith & Karpicke 2013

Repeated Testing Produces Superior Transfer of Learning Relative to Repeated Studying Butler, 2010

About these ads

About Joe Kirby

English teacher, education blogger
This entry was posted in System. Bookmark the permalink.

20 Responses to One scientific insight for curriculum design

  1. grantwiggins says:

    This is a good reminder of what the research says about the movement from short-term to long-term memory, but you don’t draw the other half of the conclusion: the curriculum has to be more than a single march through content; it has to spiral through multiple ‘looks’ at the same ideas and complex tasks. Without moving to an iterative-idea-and-performance curriculum, the material recalled is only about the facts or discrete sub-skills – which is necessary but not sufficient. That is why the use of essential questions that recur with rich content is such a vital piece of design.

  2. marvinsuggs says:

    This is music to my ears! In particular for Science, the amount of content that needs to be at the forefront of their minds is, I believe, extraordinary when you consider how it’s taught. Looking forward to planning new sow for yr 9 and 7 with growing emphasis on this and repetition. Nice work Joe

  3. This was such a thoughtful presentation. Its implications, as we move forward to curriculum redesign, have been buzzing round my head. I want to think through more of the applications for English and read again some of the stuff on short vs multiple choice answers.
    Thanks, Joe.

  4. bt0558 says:

    An interesting post, and as grantwiggins says above “a good reminder”. These ideas are not new and indeed formed a part of my initial teacher training.

    I wonder to what extent schools do not design learning in this way?

    One can achieve frequent retrieval in a number of ways, and quizzing in my experience is as reasonable and effective as some others.

    I am interested to know Joe, whether you are suggesting that quizzing is a better approach than others or just reminding people that quizzing is useful. I am not absoultely sure what you mean by quizzing but I have a rough idea i think.

    I tend to agree with Grantwiggins that multiple looks are highly desirable and whether that is best achieved by quizzing or other means is I think an interesting discussion.

  5. dodiscimus says:

    I think these ideas about frequent testing are important and it’s great that you are promoting this research. I have only read a couple of the papers you are referencing so if the answer to the question that follows comes from some of the others, that would be really helpful (and maybe I should look at the SA v MCQ papers). The research designs I have come across are based on recall of things that are simple to understand e.g. foreign vocabularly, lists of nouns, that sort of thing. Are you aware of research into the effect of quizzing on knowledge that is more difficult, or incompletely understood? I’m not clear whether there is such a thing as testing before students are ready, particularly in physics where students often come with misconceptions that are very hard to shift. Also, I guess it’s obvious, but it’s really important that incorrect answers to MCQ are strongly corrected otherwise there is some evidence that the wrong answers can be learned – that’s possibly an advantage to SAQs.

    • Alx w says:

      In my experience with maths misconceptions, constant re-enforcement is required and can be built into multiple choice questions when thought about. For example why is the statement x (a common misconception wrong?) or x is wrong, which is the correct answer. Etc etc

  6. Reblogged this on HPE&things and commented:
    Frequent quizzing improves learning…who knew!

  7. Pedro says:

    Reblogged this on From experience to meaning… and commented:
    So true!

  8. Alx w says:

    I’m going to use this principle with redesigning as maths curriculum next year. Will trial it with 3 classes and i’ll let you know the results next year!

  9. Pingback: Increasing bandwidth – Planning a revision session. | @mrocallaghan_edu

  10. Pingback: A guide to this blog | Pragmatic Education

  11. Pingback: Dereliction! A History, Geography, Art collaborative enquiry | #hackthecurriculum

  12. Pingback: 25 practical blogposts for the English teacher | Reflecting English

  13. Pingback: Using cognitive science to inform curriculum design | Belmont Teach

  14. Pingback: Planning for Progress over Time – Mastery Learning | The View From the Maths Bunker

  15. Pingback: Walking the Walk: Authentic Tests for GCSE classes | mrbunkeredu

  16. Pingback: Make History Stick Part 1: principles | Clio et cetera

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s