The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

This is the hub of the site and the place to post queries, start discussions and join in the conversation!
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

This is a really important paper. It is a 'must' read for educators and parents - and for politicians under whose noses the illiteracy tragedy continues to play out. Throwing huge amounts of money at a flawed intervention and failing to understand or acknowledge the full implications of studies of Reading Recovery is simply not acceptable - it is illogical, unscientific and immoral.

Is the Reading Recovery intervention programme so well entrenched in educational establishments that it will always be perpetuated no matter what the weight of scientific evidence and human tragedy?
The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Pamela Cook
Deborah R. Rodes
Kay L. Lipsitz

Reading Recovery, a meaning-based reading program designed for young children at risk of reading failure, is widely implemented across the United States. We discuss the recent Reading Recovery $45 million four-year i3-funded scale- up study that was designed to “cover the expansion of Reading Recovery around the U.S.” (May, Sirinides, Gray, & Goldsworthy, 2016, p. 1). While one of the two goals of the study was to determine the long-term impact of Reading Recovery, this study, described by its authors as “highly successful” (p. 4), found a “not significant” long-term effect on students’ reading skills. Subsequent Reading Recovery publications have failed to mention this “not significant” effect. With the exception of year one of the study, there are no publicly available test score data for the students when they were in Grades 2 or 3. Further, it appears that the actual lowest achieving students (special education students, students retained in first grade, and others) were systematically excluded from Reading Recovery instruction. Overall, there is very limited evidence of Reading Recovery’s efficacy as an effective long-term reading intervention. We discuss the limitations of the Reading Recovery approach, how Reading Recovery can be improved, and strongly recommend that schools do not adopt this program unless it incorporates all components of evidence-based reading instruction.
http://js.sagamorepub.com/ldmj/article/view/8391/6079

Here is the summary:
Where We Stand as Educators, Parents, and Tax-Paying Citizens

In an increasingly complex and technological world, students must become literate and informed citizens in order to adapt to accelerating changes in the workplace and society as a whole. It begins with reading.

The recommendations in the 2016 What Works Clearinghouse Foundational Skills Practice Guide (Foorman et al., 2016) and its companion practice guide on Reading Comprehension (Shanahan et al., 2010) with the emphasis on a complete reading program, apply to all K-3 students. If all K-3 students were taught with evidence-based methodology from their first days in school, there would be far fewer students who would need to be retained in first grade or need special education. Costs to school districts would be substantially reduced (Farrall, n.d.). Discipline referrals would decline (McIntosh, Sadler, & Brown, 2012). And, best of all, the self-esteem of struggling readers would rise and they would indeed be “college and career ready” upon graduation.

If advocates for Reading Recovery cannot accept the overwhelming scientific evidence regarding the need for strong foundational components of early reading instruction and evidence-based training of teachers in these skills, appropriate student and program evaluation measures, sustainable positive long-term outcomes and reasonable costs, then we, as educators, parents of children with reading disabilities, and taxpayers, strongly recommend that schools do not adopt the Reading Recovery program.
I added the bold and the red of the final paragraph.

Please circulate this message.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Here is the link to the IFERI thread featuring the Chapman and Tunmer critique of the i3 Scale Up Study of Reading Recovery:

http://www.iferi.org/iferi_forum/viewto ... 1607#p1607
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Here is video footage, via Facebook, of Pamela Cook, co-author of The Reading Wars and Reading Recovery: What Educators, Families and Taxpayers Should Know:
Pam Cook talks about the article “The Reading Wars and Reading Recovery: What Educators, Families and Taxpayers Should Know” she co-wrote with Deborah Rodes and Kay Lipsitz in Vol. 22 No. 2 (2017) of Learning Disabilities: A Multidisciplinary Journal (LDMJ).
https://www.facebook.com/sagamorepublis ... 601576290/

Please do share with your Facebook networks - thank you.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Susan Godsland flagged this paper up on Twitter, courtesy of John Bald:

http://johnbald.typepad.com/files/readi ... t-work.pdf
Reading Recovery®: Does It Work?

by James W. Chapman and William E. Tunmer

Two recent reports on Reading Recovery® (RR), released by What Works Clearinghouse (WWC, 2007, 2008), claim that RR is an effective intervention even though only 5 out of 106 research papers met the organization’s evidence standards (WWC, 2008). As Reynolds, Wheldall, and Madelaine (2009) noted, this number is small given the extent of the program’s implementation and the amount of funding allocated to RR over 25 years.
Summary:

Note the date of 2003, Chapman and Tunmer have been describing their concerns for years:
The WWC reports fall short in conveying important informa- tion relating to the effectiveness of the RR program. Therefore, we have argued (Tunmer & Chapman, 2003) that fundamental changes to the program, based on contemporary research, should occur and are very likely to improve the effectiveness of the program, both in terms of outcomes and cost.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Still on the same theme, James Murphy writes about:
A Convergence of Interests?
https://horatiospeaks.wordpress.com/201 ... rests/amp/
We can look further back. In the TES, on 28 August 2009, an article claimed that ‘Britain’s Reading Recovery initiative is one of the world’s best programmes for struggling readers, an international study has found’. That study, by the Institute for Evidence in Education, supposedly found that ‘Reading Recovery, which involves specially trained teachers working one-to-one with pupils using a mix of methods, is one of three for which there is strong evidence of effectiveness’. (The note on ‘mix of methods’ is a classic warning signal. It means: we have added a phonics component, because we know we have to be credible, but we’re not changing our foundations). The director of the institute, Professor Robert Slavin, said the two (that’s two) UK studies in the report were very positive. The TES article described the findings as ‘a boost for the literacy scheme, which is at the heart of the government’s Every Child A Reader programme yet has been criticised by some academics who question its value for money’.*

However, in the body of the report, entitled Educators Guide: What Works for Struggling Readers, Reading Recovery is deemed to have an effect size of just 0.23 across eight studies. The authors write: ‘although the outcomes for Reading Recovery were positive, they were less so than might have been expected’. Once again, there is a mismatch between the media presentation and the details of the evidence.

Last year, Kevin Wheldall (Emeritus Professor of Macquarie University) wrote this blogpost, about a report on Reading Recovery’s long-term effectiveness. The author of the report, Jane Hurry, is based at the Institute of Education in London, which Wheldall describes as ‘the British home of Reading Recovery’. The report is published by the Institute for Evidence in Education. Hurry makes broad claims of a similar nature to those above: ‘the programme is known to have impressive effects in the short term’ and ‘substantial gains’ are continued to the end of primary school. Neither of these claims is substantiated by the report.

Wheldall highlights how the headlines in the study did not match the detail and were open to a quite different interpretation, concluding: ‘even the significant differences between the two groups in the Reading Recovery Schools and the non-Reading Recovery school are accompanied by only small effect sizes, all of which are below Hattie’s hinge value of 0.4’. Once again, the substance is different from the marketing.
Do read the whole piece.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Teacher-blogger Greg Ashman writes a post in light of the peer-reviewed report published in the US, The Reading Wars and Reading Recovery: What Educators, Families and Taxpayers Should Know:
No, Reading Recovery doesn’t work in America
https://gregashman.wordpress.com/2017/0 ... n-america/
A couple of years ago, I reported on a large randomised controlled trial in the U.S. of Reading Recovery. I pointed out that, as with other studies of Reading Recovery, it was impossible to tell whether the instructional procedures used were responsible for any effect. Instead, any gains may have been due to the one-to-one tuition format of the intervention. After all, one-to-one tuition has been held up as a ideal form of instruction by none other than Benjamin Bloom.
As always, Greg's post includes plenty of interesting links.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

I have been alerted to the huge interest in the paper The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know (Pamela Cook, Deborah R. Rodes, Kay L. Lipsitz).

Here are the statistics for the first two weeks:
Sagamore's engagement data from August 22 - September 5, 2017

https://www.facebook.com/sagamorepublis ... 601576290/:

Post engagement:

13,288 People Reached

4,514 Video Views

237 Reactions, Comments & Shares (78 shares)

Likes were predominantly on shares

Comments were predominantly on shares

1,002 Post Clicks:

Majority (over 800) were clicks not on the content of the post, but on page title clicks, or clicks to "See more."

6 Hide post

Video engagement:

90% women, 9% men

Top locations:

Pennsylvania (12.9%), New Jersey (10.1%), Georgia (5.89%), California (5.53%), Florida (4.45%), New York (3.94%), Texas (3.11%), Maryland (2.96%), England (2.9%), Illinois (2.77%).

LDMJ website stats Aug 22 - September 4, 2017

Abstract page (http://js.sagamorepub.com/ldmj/article/view/8391): 1,267 pageviews

Article page (http://js.sagamorepub.com/ldmj/article/view/8391/6079): 642 pageviews

These numbers are clearly nowhere near the average!

This is a phenomenal response!
The big question is, after decades of academics and teachers critiquing Reading Recovery and its underpinning methodology and describing their worries in detail, will those in authority still promoting and funding it take responsibility and change course?

They need to 'do the right thing'.

Or is Reading Recovery so entrenched and institutionalised that it will prove to be an unstoppable juggernaut?
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Feedback to Pam:
Pam,

Just a note – I have never seen such engagement with one of our video posts in the first 24 hours.

Reach of 1,372, with 390 video views and ~150 interactions. Also, 16 shares – 5 in just the last 15 minutes.

Also, England and Australia have appeared in the top ten countries viewing a Sagamore video post for the first time ever.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

Thank you to Steven Dykstra for his permission to share his critique of the Reading Recovery i3 Study. I hope to provide an electronic link to it shortly:

A Criticism of the Reading Recovery i3 Study

Steven P. Dykstra, PhD


Any study of treatment effectiveness is inherently limited to those conclusions made possible by the design of the study. A comparison of Acetaminophen to Ibuprofen, for instance, cannot be used to say either is more effective than aspirin since aspirin was not part of the study. Likewise, the nature of the experimental and control groups also limits the possible conclusions. It is possible to design a study which compares different dosages of the same treatment, where the control group simply receives a lower dosage. Such a study can say whether more of the treatment is also more effective, but it is not a comparison to a different treatment. In fact, any number of alternative treatments could be far more effective.

All research (treatment studies in particular) should be considered skeptically, paying attention to details and noting what isn’t in the study as much as what is.

These are the standards for any treatment study. They are not unique to education or reading recovery.

If we apply these standards dispassionately to the i3 study of reading recovery, we arrive at certain observations and conclusions, each of which will be discussed and explained separately:

1) The i3 study was not designed to demonstrate that Reading Recovery was better or worse than any other intervention. It cannot be used to argue that Reading Recovery is superior to any other early intervention, only that Reading Recovery was more effective than doing nothing. The control group in the study received services ranging from nothing, to some. Children who got a single, unplanned lesson are listed as receiving some intervention. None of the interventions are named or described, and, in fact, control group subjects were on a wait list to get Reading Recovery when the study was complete, so it is no surprise they got nothing worth describing during the study. Their intervention was waiting for them when the study was over.

Despite efforts to describe the control group as something other than untreated, the study can only be read as a comparison of Reading Recovery to no treatment at all. With 5 million dollars to spend on the study, investigators could have compared Reading Recovery to one or more Scientifically based Interventions. At a minimum, they could have reported out greater detail about control group subjects and compared performance between those who got some intervention and those who got none. They chose not to do any of those things, and those choices greatly limit the conclusions that can be drawn from this study.

2) Because of the way schools were selected for the study, the subjects were limited to those already receiving what is known as Whole Language or Balanced Literacy Instruction. Reading Recovery is the pinnacle of such instruction and schools which choose to use Reading Recovery, and make the vast investment of time and resources it requires, always teach a regular curriculum consistent with Whole Language and Balanced Literacy.

This is not a criticism, just an important observation. No one expected the investigators to mandate the core curriculum or require schools to implement Reading Recovery because they were randomly selected to be in the study. But the reader must be aware of this limitation. The study is an experiment to show whether children failing to read with Whole Language Reading instruction do better if provided intensive Whole Language tutoring in Reading Recovery. It is not a study to show if students failing in a wide range of core curricula benefit from Reading Recovery, just those failing in Whole Language.

To understand this issue better, it helps to consider a different treatment issue. Consider a study investigating the impact of the SuperFit program on overall fitness. SuperFit is a more intensive version of GetFit, which is a group fitness program relying heavily on self-guided diet and exercise. SuperFit includes a personal trainer and more guidance. All the subjects in the study were doing GetFit, and those who were not responding well were assigned to SuperFit.

The study shows that subjects who get Superfit after failing at GetFit do better than those who continue with GetFit alone. However, when we look closer, we see that GetFit is based on a poor diet and ineffective exercises that are hard to do alone. GetFit is unusually ineffective compared to other programs, and while adding SuperFit improves outcomes, we are left to wonder if another approach entirely isn’t a better idea.

Such is the case with the i3 study, where Reading Recovery was effective at remediating the failures of Whole Language Instruction, compared to Whole Language instruction alone. Taken together with the lack of other treatments in the study, the utility of the i3 study for selecting possible interventions is reduced to nearly zero. It is probably true that if a district is determined to follow a Whole Language approach to reading instruction, they should probably go to the time and expense of adding Reading Recovery. However, the scientific literature, including major reviews of the literature, strongly suggest a different model of reading instruction as a better alternative.


3) The final criticism is the most severe. In what can only be called a catastrophic breach of standard research design, teachers not only knew who the treatment and control subjects were, but the same reading recovery teachers with a personal investment in the success of the program and the outcome of the study had access to control group subjects as well as influence over their instruction and intervention. The study protocol began by providing Reading Recovery Teachers with the names of students and instructing them who should receive Reading Recovery and who should be delayed by assignment to the control group.

Studies which make no effort to prevent bias, invite it. In this case, teachers who had invested their careers in Reading Recovery knew who the control group subjects were and were given unfettered access to influence their instruction. Human beings are so prone to bias that they will engage in it without knowing. That’s why studies must employ measures to prevent bias, particularly when doing so is so simple and obvious.

While a fully blind study, where no one knew who was receiving treatment and who was not is impossible in educational studies, the failure in this study is remarkable. It would have been simple to provide schools with a list of students to enroll in reading recovery without telling them which students were in the treatment group. Letting teachers know which reading recovery students where in the treatment group was a critical failure. When combined with the stunning choice to let teachers both know the identity of control subjects and have influence over their instruction, the error becomes inexcusable and suggests either deliberate bias, or blatant ineptitude.

Remarkably, investigators acknowledged the risk of bias during the final assessment and touted their efforts to prevent it. The Iowa Test of Basic Skills was administered by a teacher other than the student Reading Recovery Teacher in order to prevent the possibility of the teacher influencing the test outcome. Investigators took this step despite the fact the ITBS is a standardized test with little opportunity to introduce bias in the results. They were aware of the risk of bias, took steps to prevent it during the assessment when the risk was very low, but did nothing to prevent it during the course instruction when the risk was constant, and very real.

Reading Recovery teachers with a deep, personal investment in the outcome of the study knew who was in the experimental group, who was in the control group, and had control over the instruction of both groups. Instruction to the control group was not standardized. Teachers were free to offer less, to neglect problems, or even provide damaging instruction. Students in the treatment group could get more minutes of instruction and extra attention throughout the day from teachers determined to help the study along.

I am not suggesting that teachers consciously manipulated the outcome of the study. I am saying the likelihood of unintended bias was so high that protecting against it was essential. That investigators protected against the minor risk of bias during the ITBS but ignored the profound risk during classroom instruction is incredible. It is a methodological failure that would receive a failing grade in the most basic undergraduate course. In most fields of human subject research, this study could not pass basic peer review.


Researchers had 5 million dollars to spend investigating the benefits of Reading Recovery. They could have compared Reading Recovery to Scientifically based tutoring of equal intensity and duration. They did not. They could have included subjects receiving classroom instruction within a scientific model. They did not, choosing instead to limit the study to schools teaching within a whole language framework. Those choices greatly limit the impact of the study. If it was just those limitations, we could accept the study as evidence that Reading Recovery benefits struggling readers in Whole Language classrooms. But the failure to follow the most basic procedures for managing bias in the treatment protocol disqualifies the study entirely and begs the question of what the investigators were trying to accomplish with the 5 million dollars at their disposal.

Any other conclusions require the reader to abandon not only the required skepticism, but any sense of reason, or respect for science.
User avatar
Debbie_Hepplewhite
Posts: 2498
Joined: Sat May 23, 2015 4:42 pm

Re: The Reading Wars and Reading Recovery: What Educators, Families, and Taxpayers Should Know

Post by Debbie_Hepplewhite »

The Reading Recovery programme is still alive and thriving in England.

How can this be when it is statutory to provide 'systematic synthetic phonics' and no multi-cueing word-guessing.

Reading Recovery should not still be available for (mis)training teachers who will misguidedly teach children who are struggling with reading.

Reading Recovery is not in line with official guidance in England - and that official guidance is in line with the findings of a body of research that advises that phonics must be taught systematically and warns about multi-cueing word-guessing which can give children bad reading habits which can be very damaging for the short and long term reading profile.

Reading Recovery is still based at the Institute of Education in England.

This demonstrates that teachers are trained in contradictory guidance. This is surely not acceptable. What children receive for reading instruction should not be based on the chance of the contradictory training that teachers receive.
Post Reply