Sunday, January 23, 2011

To Learn, Take a Test ... Really?!?!


I was just sent a New York Times article from one of our teachers entitled, To Really Learn, Quit Studying and Take a Test. The article, in general, says that the results of their study indicate that students learn better by taking tests rather than repeatedly reading information or by drawing concept maps (also referred to as mind maps). I have skimmed through the full research article and have a few concerns about the research.
  1. The students who did better on the test, already wrote a test on the same content, thus received an extra practice round. this also leads to my next point.
  2. These students are university undergraduates, presumably trained to do well in testing or exam writing (since that is generally the main assessment tool used to determine grades for university admissions). I don't think I'm going too far out an a limb here to say that many high school classrooms train students to write exams, by practicing exams. In this experiment, the students are asked to "study" using a method (concept mapping) that is foreign to them.
  3. Tests generally, evaluate one's ability to regurgitate information back. The questioning needs to be of a higher order to assess for understanding. Difficult to do on a test. And isn't understanding more important?
I'm not convinced, but I welcome any thoughts.


Mark S said...

I, like you, will hesitate to jump on board with applying their findings to alter teaching strategies. My bias is that my PhD thesis was on concept-mapping (which actually is not a mind map, though often they are mixed up ). I think that I'll write about concept-mapping in my next blog post.

tanyanizam said...

Hi Ted,

Most people I say this to don't believe me, but I am absolutely certain that it is the examinations that I sat all my life that consolidated my core learning. And I still find that meeting someone else's assessment criteria forces me to find ways to create my own forms of expression in ways that I dont necessarily do on my own.

No doubt when someone bites back at me for this I will see it as another exam and it will enable me to find ways to get an 'A' with my answer.

Yes, certain concepts may have eluded me at a deep level as I was taught the old fashioned way - very much like the final group described in the article. Learn and write it. Then write it again.I still think that it wasn't just nurture that got me to prefer this way of learning. It meant I could learn as a loner and show off without talking. (Perhaps I should be a blogger then:-))

Having said all that, there is no way that I would use that as my main pedagogical approach as a high school teacher. Because, guess what - that system of learning was designed by people who learn like me to create people who are like me. And people who, like you rightly pointed out, are like those Uni students.
Sir Ken Robinson describes this very amusingly:


Brendan said...

Hi Ted
Here is what a member of a science teachers forum had to say about the article;

Retrieval Practice Produces More Learning than Elaborative Studying with Concept
Jeffrey D. Karpicke* and Janell R. Blunt; Department of Psychological Sciences, Purdue University
20 January 2011

"I had a chance to read this article thoroughly now. Despite the hype, it does not seem (to me) to be about whether old-fashioned test-taking is better than concept mapping at solidifying learning. It is about something called "retrieval practices" --- in a nutshell, activities that require students to go back into the material and retrieve information, either because they could not answer a test question or because their answers to problems were wrong or incomplete.

In one sense, it picks up on a couple of older threads in educational research --- (a) just having activity does not necessarily reinforce the learning that we want; (b) that students who construct their own knowledge need some factual material to work with and need to know that factual material well; and (c) students will need to pass examinations in school (and elsewhere), so they will need to be able to apply this learning in these situations (at least for academic success, but also in some careers).

However, these seems (to me) to be more of an indictment of how we use concept mapping, than of the practice itself. In a nutshell, if students draw their own concept maps, but never have to test them against any standard, then various errors can be maintained and perpetuated (think of some of the research that Craig Nelson sent a long a few weeks ago). So, one thing that gives ME pause about the study is that in one study condition (the so-called "retrieval practices" option), students read the passage, took a test, then had to go back into the study material to "retrieve" the material they did not know well before taking another test. In the concept-mapping condition, there was no retrieval (the authors decided to use a measure equivalent to "time on task" --- how much time students spent in reading the material and then demonstrating their levels of mastery). So, in reality, IMHO, this was a test of using retrieval practices vs NOT using retrieval practices, and not a test of whether one way of exhibiting one's understanding is better than another for helping students learn material. Indeed, one of the conditions of the experiment was the RO-TO-NO (read once, test once, next one) approach we often see used with scholastic material, which, as expected, performed pretty badly.

Now, the result showing the greater added effect of "retrieval practices" IS groundbreaking, because so often we simply accept a student's concept map if there are no egregious errors without requiring that student to go back into the material and test that map against available information or some real-world condition***. It seems to me that THIS is the big idea here: the map must be verified by the student and somehow tested for inconsistencies or inaccuracies, etc.

In the end, of course, the assessment was made on students' test-taking abilities. There are questions about whether using, say authentic assessments or other types of evaluations might produce a different result. However, in theory (and what I have always argued), if students are successful in learning the material they are supposed to, THEN they ought to be successful in ANY legitimate assessment of their mastery of the material.

I am going to look into ALL my course materials this spring to see to what extent it entrains retireval practices."
Andrew Petto.

Susi said...

Hi Ken,
I read the NY Times article, but did not have access to the actual research. It seems the article is saying that testing is a 'practice' point for applying information--great, practice makes for better connections with the brain, and retrieval helps to strengthen these connections.

If I were to learn content for example, lets say the parts of a plant cell, would a test on the parts help me learn? Why not?! I would be applying what I learned. The real trick is HOW I learned about plant cell parts.... There is nothing wrong with tests, until there is a grade put on them in my opinion. Tests could be used as formative pieces of assessment, and students can reflect on what they did well.

The article also states, "Some tests are just not learning opportunities"--here lay the key--if tests are not learning opportunities, then they are moot pieces of evidence for enhancing student achievement.

A well thought out test is an application of skills and applies reasoning that can transfer to other content areas. A good test also shows the learner their areas for growth. Further, a GREAT test is one that the students create themselves for their peers.

Thanks for sharing your post....good thinking!
One question for you Ken--how and why do you use tests at your school?


Ted Cowan said...

@Mark S Thanks for the comment, I'll check out your blog, so I'm aware of the differences.

@tanyanizam I don't doubt you. We all learn in different ways, which is why differentiated lessons and assessments are so importnant to learning. But, universities still seem to be stuck in traditional methods. I love that video of Ken Ronison's too. One of my favourites.

@Brendan I like what Andrew is saying. It sounds we have similar views on the subject. He has obviously studied the ideas presented in more detail than myself.

@Susi I agree with your points about testing. There needs to be a purpose and clear learning goals. I would say we still use tests here in the traditional sense, but we are definitely moving towards having more diverse ways for the students to demonstrate their learning. That said, we also run the IB Dilpoma Program which has standard tests at the end of the two year program.

Thanks everyone for the comments.