Examining the evidence on the effectiveness of synthetic phonics teaching: the Ehri et al (2001) and C.Torgerson et al (2006) meta-analyses
Rhona Johnston, Emeritus Professor of Psychology, University of Hull

Introduction

In a recent article, Castles et al (2018) have concluded that there is insufficient evidence as yet to determine whether the synthetic phonics approach is superior to the analytic phonics approach, citing the meta-analyses of Ehri et al ( 2001) and C.Torgerson et al (2006).

Studies of synthetic versus analytic phonics teaching

The experiments comparing the effectiveness of synthetic versus analytic phonics teaching by Johnston and Watson (2004) were carried out on children in their first year of school (equivalent to kindergarten in the U.S.). Johnston and Watson’s (2004) synthetic phonics approach was taken from the method used in Austria, as described by Feitelson (1988), where sounding and blending was introduced at the start of reading tuition. Children learnt to read and spell very much better when taught by the synthetic phonics method compared with the typical analytic phonics method used in Scotland.

When the UK Education Select Committee recommended to the government that it consider introducing the synthetic phonics method into schools in England, it referred to it as ‘phonics first and fast’. A later introduction of sounding and blending means that other methods are used first, approaches which often undermine the synthetic phonics approach. Therefore, in order to examine the evidence supporting the effectiveness of the method, only studies where the synthetic phonics method was introduced in kindergarten should be examined.

Meta-analyses of phonics teaching

a) Ehri et al (2001)

It should be noted that Ehri et al’s (2001) analysis compared synthetic phonics programmes with those containing unsystematic phonics or no phonics (page 400), not analytic phonics, which is systematic. They did compare the effect sizes for synthetic phonics (d=0.45) and larger-subunit programmes (d=0.34), which was not statistically significant. However, in the UK analytic phonics is taught at the grapheme to phoneme level, after establishing an initial sight word vocabulary, so it is not a larger subunit approach in the early stages. In Scotland, the approach traditionally progressed to teaching sounding and blending at the end of the first year at school. In our research, the defining feature of synthetic phonics was that sounding and blending should be taught right from the start.

An examination is made here of all of the kindergarten studies using synthetic phonics included in Ehri et al (2001), with the exception of one study that did not measure word identification so cannot be directly compared to Johnston and Watson’s (2004) experiments (see Table 1). It should be noted that two of these studies carried out phonological awareness training prior to teaching participants to read via synthetic phonics (Blachman et al, 1999; J.Torgesen et al, 1999, see Notes 1 & 2). These studies have very low effect sizes at the kindergarten post-test. When these data are included (see column 3) the mean effect size is low (0.28). A different picture emerges at the next post-test, after synthetic phonics teaching had commenced, see column 4. When these data points from these two studies (see column 5) are used, a mean effect size of 0.49 is found.

The two studies by Johnston and Watson (2004) comparing synthetic phonics teaching with the analytic phonics approach used in Scotland yielded large effect sizes (see Table 2); these studies were not included in the Ehri et al (2001) meta-analysis. In the Clackmannanshire study (Experiment 1), over a 7 year period the effect sizes for word identification versus chronological age increased year after year (Johnston, McGeown and Watson, 2012).

Table 1 Effect sizes for word identification scores from kindergarten synthetic phonics studies included in Ehri et al (2001); synthetic phonics versus little or no phonics teaching

Table 2 Effect sizes for word identification scores from Johnston and Watson’s (2004) studies comparing synthetic and analytic phonics teaching

b) C.Torgerson et al (2006)

This meta-analysis was designed to compare synthetic with analytic phonics teaching, so included only 3 studies; it also deselected any studies which were not randomised controlled trial studies (unlike Ehri et al, 2001). They included Experiment 2 from Johnston and Watson (2004).

However, one of the studies included was an unpublished one that had an incorrect implementation of the synthetic phonics method (50% of the taught words were silent ‘e’ words, which cannot be accurately blended by initial readers). Furthermore, C.Torgerson et al (2006) put into the analysis the post-test data from the trained items, where there was a difference favouring analytic phonics, not the data from the untrained items, where there was no difference.

C.Torgerson et al (2006) also included the J.Torgesen et al (1999) study, but put in the kindergarten data from when the children were largely being taught phonological awareness. At this stage, the embedded (not analytic) phonics control condition included a lot of word reading, whereas the PASP method did not. It can be seen that when the children were exposed to the synthetic phonic element of the programme, the effect size in favour of the method was much greater (see Table 1).

We have published our critique of the C.Torgerson et al (2006) meta-analysis in our book Teaching Synthetic Phonics, and the relevant section can be read online, page 14:
https://play.google.com/books/reader?id=WOOICwAAQBAJ&printsec=frontcover&source=gbs_atb&pg=GBS.PA14

Conclusions

The US National Reading Panel meta-analysis reported in Ehri et al (2001) included studies with a late implementation of synthetic phonics teaching, an approach which is not compatible with the method used by Johnston and Watson (2004). Even when the studies of synthetic phonics that started in kindergarten were examined, two of them were found to have a late introduction of sounding and blending for reading as phonological awareness was taught first. In both cases, the introduction of synthetic phonics later on led to a very large increase in effect size. No study in this sub-analysis self-identified itself as having an analytic phonics control condition. Furthermore, the comparison in the meta-analysis between synthetic and large subunit phonics does not encompass the analytic phonics approach used in the UK for the initial teaching of reading.

A close examination of the C.Torgerson et al (2006) meta-analysis shows evidence of multiple errors in the selection of the studies to be included and in the selection of the data entered into the analysis. The number of studies included was also too small to allow for a meaningful meta-analysis.

It cannot be concluded that these two meta-analyses showed evidence against the superiority of the synthetic over the analytic phonics method.

The synthetic phonics method as implemented in our studies involved, right from the start of school, children learning a small number of letter sounds and using that knowledge right away to sound and blend the letters to find out how to pronounce unfamiliar words. They
then rapidly learnt more letter sounds and continued to use the strategy. We found that these children had much better reading and phonological awareness skills than those taught either by analytic phonics, or by analytic phonics plus phonological awareness (Johnston and Watson, 2004, Experiment 1, the Clackmannanshire Study). Unlike broad-based meta-analytic comparisons, there was strict control of the new printed words used to teach all of the groups compared in our studies, so issues of pace of print exposure between studies do not arise. Furthermore, we have demonstrated that the early rigorous start of synthetic phonics teaching led to the children making increasing gains in reading ability compared with age long after the intervention ended (Johnston et al, 2012).

References

Blachman, B., Tangel, D., Ball, E., Black, R., & McGraw, D. (1999). Developing phonological awareness and word recognition skills: A two-year intervention with low-income, inner-city children. Reading and Writing: An Interdisciplinary Journal, 11, 239–273.

Castles, A., Rastle, K, and Nation, K (2018) Ending the Reading Wars: Reading Acquisition From Novice to Expert. Psychological Science in the Public Interest, Vol. 19(1) 5–51.

Ehri et al (2001) Systematic Phonics Instruction Helps Students Learn to Read: Evidence from the National Reading Panel’s Meta-Analysis. Review of Educational Research,71, No. 3, pp. 393–44.

Feitelson, D (1988) Facts and fads in beginning reading: A cross-language perspective. Norwood, NJ: Ablex.

Johnston, R.S and Watson, J. (2004) Accelerating the development of reading, spelling and phonemic awareness. Reading and Writing, 17 (4), 327-357.

Johnston, R.S, McGeown, S, and Watson, J. (2012) Long-term effects of synthetic versus analytic phonics teaching on the reading and spelling ability of 10 year old boys and girls. Reading and Writing, 25, part 6, 1365-1384.

Johnston, R. and Watson, J. (2014) Teaching Synthetic Phonics, 2nd edition. Sage (Learning Matters): London.

Torgesen, J., Wagner, R., Rashotte, C., Rose, E., Lindamood, P., Conway, T., & Garvan, C. (1999). Preventing reading failure in young children with phonological processing disabilities: Group and individual responses to instruction. Journal of Educational Psychology, 91, 579–593.

Torgerson, C., Brooks, G., & Hall, J. (2006). A systematic review of the research literature on the use of phonics in the teaching of reading and spelling (Research Report RR711). U.K. Department for Education and Skills. Retrieved from http://dera.ioe.ac.uk/14791/1/RR711_.pdf

Notes

1Blachman et al (1999) ‘From February to May (Semester 2) of kindergarten, treatment children participated in 41, 15 to 20 minute phonological awareness lessons (adapted from the shorter, 28 lesson program used in Ball and Blachman, 1991), delivered over an 11 week period to small, heterogeneous groups of four or five children. Classroom teachers and their
teaching assistants were responsible for teaching all of the lessons. Each lesson followed the same daily format: (1) a phoneme segmentation activity (called say-it-and-move-it) in which children learned to move disks to represent the sounds in one-, two-, and three phoneme
words spoken by the teacher, (2) a segmentation-related activity, such as grouping words on the basis of shared sounds (e.g., hat and hot go together because they share the same initial sound), and (3) one of a variety of activities to teach the letter names and sounds of eight letters (a, m, t, i, s, r, f, b).

Grade 1, for the treatment children ‘Instruction in phoneme analysis and blending skills….Children were taught to pronounce as a single unit a consonant (continuant) followed by a vowel. To begin, the teacher represented this strategy on the board as follows:
The teacher pointed to the first letter, and the child was taught to produce that letter’s sound and hold the sound until the teacher’s finger reached the second letter. When her finger touched the second letter, the second sound was produced and held. With each successive practice opportunity, the length of time between sounds was decreased until the two sounds were pronounced as a single unit. By adding final consonants (initially, stop consonants) and
pronouncing the whole word, a set of real words was built (e.g., sat, sam). Words containing new short vowels were also introduced in this manner.’

2J.Torgesen et al (1999) PASP (Phonological awareness plus Synthetic Phonics) condition. Started in Semester 2 of Kindergarten. ‘This group received the Auditory Discrimination
in Depth Program as developed and outlined by Patricia and Charles Lindamood (1984). The program provided explicit instruction in phonemic awareness by leading children to discover and label the articulatory gestures associated with each phoneme. This discovery work was followed by activities to build skills in tracking the phonemes in words using mouth-form pictures, colored blocks, and letters to represent the phonemes in words