Special Issue:
Direct Instruction Reading Programs: Examining Effectiveness
For At-Risk Students in Urban Settings

Guest Editors' Introduction
Martha Abele Mac Iver and Elizabeth A. Kemper

This special issue of JESPAR is devoted to recent studies of the Direct Instruction (DI) reading program and was inspired by several presentations on DI at the Fort Worth Reading Symposium, sponsored by the Fort Worth Independent School District in August 2000. As a result of that conference, a dialog began with many of the authors whose research is presented in this volume. Popularized by recent media reports, including the PBS documentary "The Battle of City Springs," as well as articles in Education Week (Manzo, 1998; Viadero, 1999), the National Review (Nadler, 1998), and Policy Review (Palmaffy, 1998), DI has enjoyed renewed prominence over the past several years. DI as a whole-school reform initiative grew out of the earlier reading instruction research of Sigfried Engelmann and his associates, who developed the DISTAR program more than 30 years ago. Numerous experiments focused on how students learn most effectively shaped the many technical details of the program, which requires teachers to follow carefully scripted lessons in a specific sequence. Kameenui, Simmons, Chard, and Dickson (1997) offered a particularly useful and thorough overview of the model and related research, which we can summarize only briefly in the following introduction.

DI received particular attention as one of the most effective programs involved in Project Follow Through, a federal compensatory education program beginning in 1967 for low-income students in kindergarten through third grade. Because many educators perceived DI as rigid, it was not eagerly embraced by the educational mainstream (Viadero, 1999). Longitudinal research conducted by developers of a competing preschool curricular model (and subsequently challenged on methodological grounds) also associated DI with higher rates of emotional problems and felony arrests when its students reached late adolescence and early adulthood (Bereiter, 1986; Gersten, 1986; Schweinhart, Weikart, & Larner, 1986a; Schweinhart, Weikart, & Larner, 1986b; Schweinhart & Weikart, 1997). Plagued by negative publicity like this, DI continued as a sort of underground reform movement until the later 1990s, when it was named in a study by the American Institutes for Research as one of only three school reform programs to have a "strong" record of evidence of positive effects on student achievement (Herman et. al, 1999). Other groups, including the American Federation of Teachers (AFT), have also identified it as a promising whole-school reform model (AFT, 1998).

Much of the research documenting positive achievement outcomes for DI was based, however, on special education student populations, as Slavin and Fashola (1998) have noted. In fact, of the studies included in a recent meta-analysis of the effects of DI (Adams & Engelmann, 1996), more than half used a special education student sample. Recognizing the important, documented effects of DI for learning-disabled and other special education students (e.g., Gersten, 1985; White, 1988), as well as the impact of DI on achievement in subjects such as mathematics, language, and spelling (see studies cited in Adams & Engelmann, 1996; Hermann et al., 1999), we decided to focus this special issue more specifically on studies of DI in reading among regular education populations. As the following review of relevant literature on DI reading demonstrates, much of the research was published more than a decade ago, and was often based on student test scores from the early 1970s. It is essential for educators and policymakers to have access to more recent studies of DI, and this special issue seeks to address that need by gathering several studies from the late 1990s together in one place. After reviewing the previous research base for DI reading, we present a brief summary of each of the research articles, case studies, and commentaries included in this issue.

The best evidence for an effect of DI on reading achievement for regular education students comes from several reanalyses of data from Project Follow Through (Stebbins, St. Pierre, Proper, Anderson, & Cerva, 1977). As a "planned variation approach," Project Follow Through was designed so that the relative effectiveness of the different models could be evaluated. Beginning in the late 1960s, Follow Through programs were implemented in roughly 4000 classrooms distributed over more than 170 school districts and continued for more than a decade in some cases. The original Follow Through evaluation (Stebbins et al., 1977) concluded that basic skills models (including DI) were the most effective of the programs studied. Although there has been considerable debate about the methodology of the Follow Through study (Anderson, St. Pierre, Proper, & Stebbins, 1978; Bereiter & Kurland, 1981-82; Gersten, 1984; Guthrie, 1977; House, Glass, McLean, & Walker, 1978; House, 1979; Wisler, Burns, & Iwamoto, 1978), it still provides a useful database for analyzing the impact of DI as a school reform program.

Becker and Carnine (1980) presented evidence that students in DI schools outperformed students in other Follow Through reform models on the Metropolitan Achievement Tests (Reading, Math, Spelling, and Language) as well as the Wide Range Achievement Test (WRAT). By the end of third grade, after 4 years of DI, students in the Follow Through sites were also performing at approximately grade level (50th percentile) in all areas except MAT reading (which Becker & Carnine, 1980, defined as testing primarily reading comprehension skills). On the MAT reading subtest DI students were, on average, at the 40th percentile. Four of the nine cohorts of third grade students at a New York City DI site (1973-81) were achieving at grade level or above (higher than 45th percentile) on the MAT or SAT reading test after 4 years of DI (Meyer, Gersten, & Gutkin, 1983). The Meyer, Gersten and Gutkin (1983) report of the Abt evaluation summary of effects on Cohorts 2 and 3 indicated that the reading effect was significant only when a pooled comparison group was used, and only in one of the two cohorts under study (p. 247). Meyer, Gersten, and Gutkin also presented evidence that reading achievement at this New York City DI site was significantly higher than the district as a whole.1 A subsequent study (Gersten, Darch, & Gleason, 1988) noted that participation in a DI kindergarten appeared to be crucial: Students with 4 years of DI (including kindergarten) significantly outperformed control students on the MAT reading test at the end of third grade, while there was no difference between control students and those students who had experienced just 3 years (first through third grade) of DI.

Several other studies built on the early Follow Through studies by examining later achievement effects on children who participated in the DI reform. Becker and Gersten (1982) analyzed fifth and sixth grade achievement effects for students from five different sites who all experienced 4 years of DI. They found strong, significant effects of DI on the WRAT reading test (a measure of decoding) for both fifth and sixth graders (as well as effects in spelling and math problem solving). The effect of DI on MAT reading (comprehension-focused) and word knowledge was not as pronounced, in that most comparisons were not significant, but none favored the comparison group. Gersten, Darch, and Gleason (1988) reported significantly higher ninth grade reading scores on the California Achievement Test (CAT)2 for East St. Louis DI students (compared to control students)3 6 years after students had received the 4-year intervention. The effects were significant for students without DI kindergarten (effect size=+0.31, p < .05) as well as for those who attended a DI kindergarten (effect size=+0.49, p < .01). Similarly, Gersten and Keating (1987) reported significantly higher ninth grade achievement scores on the CAT in reading and mathematics for DI cohorts from an anonymous city (called Finley), as well as a nearly significant effect of early DI on ninth grade reading scores in Flint, Michigan. DI students from Flint were also significantly less likely than their comparison group to have an attendance problem (10 or more absences per year) in high school or to be retained in grade (Gersten & Keating, 1987). Meyer (1984) followed the Follow Through DI students in New York City through high school to ascertain the long-term academic effects of the intervention. Compared to control groups, significantly more of the DI students graduated from high school (59.5% vs. 37.6%), applied to college (34% vs. 18.5%), and were accepted to college (34% vs. 17%). Significantly more of the control students were retained (32.6% vs. 21.4%) or dropped out of school (46% vs. 27.7%). DI students scored significantly higher on ninth grade reading and math tests as well.

Apart from the studies based on the original Project Follow Through data, the only other longitudinal study (examining students who have experienced a DI intervention for at least 3 years) appears to be the study of the Alliance of Quality Schools in Broward County, Florida (Varela-Russo, Blasik, & Ligas, 1997; Varela-Russo, Blasik, & Ligas, 1998).4 DI is one component of a larger reform strategy in a group of 32 schoolwide Title I elementary schools within that district. More than 80% of students in the schools implementing this reform program were eligible for free or reduced-price lunch. Analysis of nearly a thousand students who experienced DI for at least 3 years, compared to a demographically similar group of control students, indicated that DI in reading was able to nearly close the reading achievement gap between program and control students in third and fourth grades (though both groups were achieving at just the 37th percentile on the reading comprehension subtest of the SAT8). But DI students' reading scores decreased, on average, in fifth grade, falling below the comparison group and reopening the gap. One of the recommendations in the 1997 report was to "enhance the elements of the Alliance program to strengthen reading and math achievement in Grade 5" (Varela-Russo et al., 1997, p. 19).

Though Camden, NJ was not part of the original Follow Through study, it began implementing DI in at least one school in 1978, and received a Follow Through grant in 1988 to implement DI in one elementary school. Several published studies of DI in Camden5 yielded mixed results regarding the impact of 1 or 2 years of DI on reading achievement for second graders. A 1-year experimental study of DI in two second-grade classrooms (with randomly selected children and teachers) yielded a significant positive effect of DI on CTBS reading vocabulary scores (DiObilda & Brent, 1985-86). A small sample size (n=47 in each group) may have prevented the positive effect of DI on comprehension scores from reaching statistical significance. Brent, DiObilda, and Gavin (1986) also reported a positive effect of 2 years of DI instruction by experienced teachers on second grade reading scores, while students with an inexperienced DI teacher performed no better than control students. In a larger study of Camden second graders who had experienced 2 years of DI, Brent and DiObilda (1993) found no effects of DI on CTBS reading scores, concluding the program was "as effective as traditional programs that are aligned with a specific standardized test" (p. 337).
Other short-term studies of the impact of DI on reading achievement were similarly mixed. In a 1-year study in which 24 second-grade students were randomly assigned either to DISTAR reading or "Johnny Right to Read" programs, Summerell and Brannigan (1977) found significantly greater gains in reading comprehension (measured by the paragraph meaning subtest of the Stanford Achievement Test) for the DISTAR group, but no significant differences on the word meaning subtest. A study of a 15-week remedial reading program (averaging a total of 45 hours of instruction) for second through sixth graders, in which a total of 72 students were randomly assigned either to DISTAR or another structured reading program, found that both groups made equivalent gains (Richardson et al., 1978). One-year quasi-experimental studies of the impact of DI in reading on first graders found evidence of an effect on language ability (measured by the Slosson Intelligence Test) (Sexton, 1989), spelling and word analysis (measured on the Iowa Test of Basic Skills; Snider, 1990), and reading vocabulary (measured by the Gates McGinitie Reading Test) (Bowers, 1972). But Bowers (1972) did not find evidence of a significant effect of DISTAR on reading comprehension,6 and Snider (1990) found no difference between DISTAR and control groups on reading and vocabulary subtests of the Iowa Test of Basic Skills. Gersten, Carnine, Zoref, and Cronin (1986) presented evidence of considerable growth in reading achievement among first-graders in a DISTAR program (from the 18th percentile on the CTBS Level A reading readiness test in spring of the kindergarten year to the 46th percentile on the CTBS Level B total reading score in spring of the first grade year), but there was no control group in that particular study. Many short-term experimental studies of DI techniques, generally with small groups of remedial students, have indicated significant effects of DI (e.g., Kaiser et al., 1989).

While there is considerable evidence that DI has a significant effect on decoding skills in reading as well as on vocabulary skills, there is much less evidence of an impact of the program on reading comprehension. It is possible that the impact of Direct Instruction on reading comprehension might be more pronounced in sites where implementation is further advanced, given that it often takes several years for a reform effort to take root sufficiently to produce positive results (Fullan, 1999). Several studies of DI have reported a high correlation between achievement results and fidelity of implementation (Gersten & Carnine, 1980; Gersten et al., 1986) or levels of DI experience among teachers (Brent, DiObilda, & Gavin, 1986). It may simply take time for teachers to gain sufficient experience and implementation to be solidified before an effect emerges. Given this consideration, the dearth of longitudinal studies other than the Follow Through study of DI is unfortunate.7 There may indeed be a reading comprehension effect that simply has not been uncovered yet in the studies to date.

DI advocates have directly addressed the issue of their difficulties in achieving an effect on comprehension scores. Becker (1977) argued that standardized tests of reading comprehension assume a working vocabulary that gives children from middle class homes (where they develop such a vocabulary) an advantage over children whose families do not similarly contribute to such vocabulary development. Because of this, he outlined a specific plan to build vocabulary development into the program. In addition, DI advocates have outlined the importance of teaching reading comprehension skills directly (Carnine et al., 1997; Gersten & Carnine, 1986). Short-term experimental studies have shown convincingly the effectiveness of DI in strategies for summarizing passages and drawing inferences (Adams, Carnine, & Gersten, 1982; Carnine, Kameenui, & Woolfson, 1982; Carnine & Kinder, 1985; Carnine, Stevens, & Clements, 1982; Patching, Kameenui, Carnine, Gersten, & Colvin, 1983; Ross & Carnine, 1982).

Since most of the published research on DI is more than 10 years old, and questions remain about its effectiveness in improving reading comprehension, we believe the studies included in this special issue represent a particularly important contribution to the literature on DI and whole-school reform efforts. Four longitudinal research studies from the late 1990s examine the impact of DI on reading achievement in Broward County (Florida), Houston, Fort Worth, and Baltimore. Maria Ligas analyzes the impact of the 5-year Alliance of Quality Schools project in Broward County, Florida, which utilized DI Reading Mastery as the major component of reading instruction. Growth scores in reading comprehension for Alliance school students in Grades 3 through 8 were compared to district averages, adjusted so that program and comparison students had the same demographic characteristics. Coleen Carlson and David Francis present results of their external evaluation of the Rodeo Institute for Teacher Excellence (RITE) program, in which the DI Reading Mastery curriculum was implemented over a 4-year period in kindergarten, first, and second grade classrooms in a total of 20 schools. Their longitudinal study of experimental and comparison school cohorts analyzes the impact of DI reading instruction on several standardized measures of reading achievement. O'Brien and Ware provide a detailed evaluation of how both DI and Open Court reading programs were implemented in a total of 61 schools in the Fort Worth Independent School District in 1998-99 and 1999-2000. They analyze the impact of these programs on reading achievement for kindergarten, first, and second graders. Mac Iver and Kemper examine the effects of DI reading in a 4-year longitudinal study of kindergarten and second grade cohorts at six experimental and six comparison schools in Baltimore. The study focuses on reading achievement for these cohorts, primarily in third and fifth grades respectively after 4 years, as well as on 1-year effects of DI reading for all third and fifth grade students at the study schools in 1999-2000.

In this special issue we also include two case studies focused on particular schools, written by advocates of DI. Muriel Berkeley, who has served as Executive Director of the Baltimore Curriculum Project, presents a case study of City Springs Elementary, one of the schools included in the Mac Iver and Kemper research study. She describes the transformation of City Springs (which was also the subject of the PBS documentary "The Battle of City Springs") from a chaotic school into an oasis of order. Her article documents the dramatic growth in student achievement at the school over the past 5 years since implementation of DI in reading began. She also analyzes the systemic factors that impede full implementation of DI and may limit the achievement effects of the program elsewhere in the district. Bonita Grossen, who is currently leading implementation of a "new generation" of DI programs in secondary schools throughout the country, describes components of the BIG Accommodation model and analyzes its effects on reading and mathematics achievement over a 2-year period in a Sacramento middle school. She also discusses developments in the DI teacher-training model and electronic progress monitoring of student achievement.

The commentary by Jerry Silbert of the University of Oregon provides an insightful look at these studies from the DI developers' point of view. Barak Rosenshine, well known for his many publications on school effectiveness, reading instruction, and explicit teaching, has followed DI research closely over the past three decades and offers a thoughtful critique of these studies.
We believe this set of articles, published together in this special issue of JESPAR, represent a significant contribution to the current national discussion about reading instruction and the utility of whole-school reform models in seeking to raise the achievement of students placed at risk of academic failure. With updates expected for at least a couple of these studies in the next year, we hope to begin a continuing dialogue among researchers regarding Direct Instruction.

References

Adams, A., Carnine, D., & Gersten, R. (1982). Instructional strategies for studying content area texts in the intermediate grades. Reading Research Quarterly, 18, 27-55.

Adams, G. L., & Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond DISTAR.

American Federation of Teachers (AFT). (1998). Six promising schoolwide programs for raising student achievement [on-line]. Available: www.aft.org/edissues/whatworks/wwreading.htm.

Anderson, R. B., St. Pierre, R. G., Proper, E. C., & Stebbins, L. B. (1978). Pardon us, but what was the question again? A response to the critique of the Follow Through evaluation. Harvard Educational Review, 48, 161-170.

Becker, W. C. (1977). Teaching reading and language to the disadvantaged—what we have learned from field research. Harvard Educational Review, 47, 518-543.

Becker, W. C., & Carnine, D.W. (1980). Direct instruction: An effective approach to educational intervention with the disadvantaged and low performers. In B. B. Lahey & A. E. Kazdin (Eds.), Advances in clinical child psychology (vol. 3, pp. 429-473). New York: Plenum.

Becker, W. C., & Gersten, R. (1982). A follow-up of Follow Through: The later effects of the direct instruction model on children in fifth and sixth grades. American Educational Research Journal, 19, 75-92.

Bereiter, C. (1986). Does direct instruction cause delinquency? Early Childhood Research Quarterly, 1, 289-292.

Bereiter, C., & Kurland, M. (1981-1982). A constructive look at Follow Through results. Interchange, 31, 1-22.

Brent, G., & DiObilda, N. (1993). Effects of curriculum alignment versus Direct Instruction on urban children. Journal of Educational Research, 86, 333-338.

Brent, G, DiObilda, N., & Gavin, F. (1986). Camden Direct Instruction Project 1984-1985. Urban Education, 21, 138-148.

Bowers, W. M. (1972). Evaluation of a pilot program in reading for culturally disadvantaged first grade students. Unpublished doctoral dissertation, University of Tulsa, Tulsa, OK.

Carnine, D., Kameenui, E., & Woolfson, N. (1982). Training of textual dimension related to text-based reference inference. Journal of Reading Behavior, 14, 335-340.

Carnine, D., & Kinder, D. (1985). Teaching low-performing students to apply generative and schema strategies to narrative and expository material. Remedial & Special Education, 6, 20-30.

Carnine, D., Stevens, C., & Clements, J. (1982). Effects of facilitative questions and practice on intermediate students' understanding of character movies. Journal of Reading Behavior, 14, 179-190.

Carnine, D., Silbert, J., & Kameenui, E. (1997). Direct Instruction reading (3rd ed.). Upper Saddle River, NJ: Prentice Hall.

DiObilda, N., & Brent, G. (1985-86). Direct instruction in an urban school system. Reading Instruction Journal, 29, 2-5.

Fullan, M.G. (1999). Change forces: The sequel. Philadelphia, PA: Falmer.

Gersten, R. (1984). Follow Through revisited: Reflections on the site variability issue. Educational
Evaluation and Policy Analysis, 6, 109-121.

Gersten, R. M. (1985). Direct instruction with special education students: A review of evaluation research. The Journal of Special Education 19, 41-58.

Gersten, R. (1986). Response to "Consequences of three preschool curriculum models through age 15." Early Childhood Research Quarterly, 1, 293-302.

Gersten, R., & Carnine, D. (1980). Measuring implementation of a structured educational model in an urban school district: An observational approach. Educational Evaluation Policy Analysis, 4, 67-69.

Gersten, R., & Carnine, D. (1984) Direct Instruction mathematics: A longitudinal evaluation of low-income elementary school students. The Elementary School Journal, 84, 395-407.

Gersten, R., & Carnine, D. (1986). Direct Instruction in reading comprehension. Educational Leadership, 44, 69-78.

Gersten, R., Carnine, D., Zoref, L., & Cronin, D. (1986). A multifaceted study of change in seven inner-city schools. The Elementary School Journal, 86, 257-276.

Gersten, R., Darch, C., & Gleason, M. (1988). Effectiveness of a Direct Instruction academic kindergarten for low-income students. The Elementary School Journal, 89, 227-240.

Gersten, R., & Keating, T. (1987). Long-term benefits from Direct Instruction. Educational Leadership, 44, 28-31.

Guthrie, J. T. (1977). Follow Through: A compensatory education experiment. Reading Teacher, 31, 240-244.

Herman, R., Aladjem, D., McMahon, P., Masem, E., Mulligan, I., O'Malley, A., Quinones, S., Reeve, A., & Woodruff, D. (1999). An educators' guide to schoolwide reform. Washington, DC: American Institutes for Research.

House, E. R. (1979). The objectivity, fairness, and justice of federal evaluation policy as reflected in the Follow Through Evaluation. Educational Evaluation and Policy Analysis, 1, 28-42.

House, E. R., Glass, G. V., McLean, L.D., & Walker, D. F. (1978). No simple answer: Critique of the Follow Through Evaluation. Harvard Educational Review, 48, 128-160.

Kaiser, S., Palumbo, K., Bialozor, R.C., & McLaughlin, T.F. (1989). The effects of Direct Instruction with rural remedial students: A brief report. Reading Improvement, 26, 88-93.

Kameenui, E. J., Simmons, D. C., Chard, D., & Dickson, S. (1997). Direct Instruction reading. In S. A. Stahl & D. A.Hayes (Eds.), Instruction models in reading (pp. 59-84). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Manzo, K. K. (1998, June 10). Drilling in Texas. Education Week, 17, 32-37.

Meyer, L. (1984). Long-term academic effects of the Direct Instruction Project Follow Through. The Elementary School Journal, 84, 380-394.

Meyer, L. A., Gersten, R., & Gutkin, J. (1983). Direct Instruction: A Project Follow Through success story in an inner-city school. The Elementary School Journal, 84, 241-252.

Nadler, R. (1998). Failing grade. National Review, 50, 38-39.

On track: Fifteen years of student improvement. Wesley Elementary School, Houston, Texas. (1992, Winter) Direct Instruction News.

Palmaffy, T. (1998). No excuses: Houston educator Thaddeus Lott puts failing schools to shame. Policy Review, 87, 18-23.

Patching, W., Kameenui E., Carnine, D., Gersten, R, & Colvin, G. (1983). Direct instruction in critical reading skills. Reading Research Quarterly, 18, 406-418.

Richardson, E., DiBenedetto, B., Christ, A., Press, M., & Winsberg, B. (1978). An assessment of two methods for remediating reading deficiencies. Reading Improvement, 15, 82-95.

Ross, D., & Carnine, D. (1982). Analytic assistance: Effects of example selection, students' age, and syntactic complexity. Journal of Educational Research, 75, 294-298.

Schweinhart, L. J., & Weikart, D. P. (1997). The High/Scope preschool curriculum comparison study through age 23. Early Childhood Research Quarterly, 12, 117-143.

Schweinhart, L. J., Weikart, D. P., & Larner, M. B. (1986a). Consequences of three preschool curriculum models through age 15. Early Childhood Research Quarterly, 1, 15-45.

Schweinhart, L. J., Weikart, D. P., & Larner, M. B. (1986b). Child-initiated activities in early childhood programs may help prevent delinquency. Early Childhood Research Quarterly, 1, 303-312.

Stebbins, L. B., St Pierre, R. G., Proper, E. C., Anderson, R. B., & Cerva, T. R. (1977). Education as experimentation: A planned variation model (Vol IV-A). Cambridge, MA: Abt Associates.

Sexton, C. W. (1989). Effectiveness of the DISTAR Reading I Program in developing first graders' language skills. Journal of Educational Research, 82(5), 289-93.

Slavin, R., & Fashola, O. (1998). Show me the evidence! Thousand Oaks, CA: Corwin.

Snider, V. E. (1990). Direct Instruction reading with average first graders. Reading Improvement, 27, 143-148.

Summerell, S., & Brannigan, G. (1977). Comparison of reading programs for children with low levels of reading readiness. Perceptual and Motor Skills, 44, 743-746.

Varela-Russo, C., Blasik, K. & Ligas, M. R. (1997, July). Alliance of Quality Schools Evaluation Report. Broward County, FL: The School Board of Broward County.

Varela-Russo, C., Blasik, K., & Ligas, M. R. (1998, September). Alliance of Quality Schools Evaluation Report. Broward County, FL: The School Board of Broward County.

Viadero, D. (1999, March 17). A direct challenge. Education Week, 18, 41-43.

White, W. A. T. (1988). A meta-analysis of the effects of Direct Instruction in special education. Education and Treatment of Children, 11, 364-374.

Wisler, C. E., Burns, G. P., Jr., & Iwamoto, D. (1978). Follow Through redux: A response to the critique by House, Glass, McLean, and Walker. Harvard Education Review, 48, 171-185.

Acknowledgements

The guest editors wish to thank research assistants Kathleen Romig and Naomi Graeff for their help in bibliographic research and compiling the literature review, and Rafeeq Hasan for help in manuscript and table preparation for this special issue. We are also deeply indebted to JESPAR assistant editor Sarah Heneghan, who shepherded the editing process under a very tight production schedule.

Footnotes

1. In one analysis, they compare average reading scores for the NYC DI cohorts for each year 1973 to 1981 against pooled comparison group data from two cities over the time period 1973-74. Another analysis relies on aggregate average grade equivalent scores, since neither NCE scores nor individual level data were available.

2. We assume (though the authors do not directly specify) that the reported average grade equivalents and percentiles for reading for DI and control groups refer to a "Total Reading" score on the CAT (a combination of vocabulary and comprehension).

3. Except for asserting that comparison students are "demographically similar students who received the district's typical curriculum in the early grades," the article does not report how the comparison groups for this ninth grade analysis were selected. Since the sample sizes for the ninth grade control groups are nearly 3 times larger than those for the control group at the end of third grade, the study does not appear to follow the same control students over time.

4. A revised and updated version of these reports appears in Maria Ligas' contribution to this special issue. Despite numerous popular articles about the effectiveness of DI at Wesley Elementary School in Houston, Texas over the past two decades (e.g., Palmaffy, 1998; Manzo, 1998), there do not appear to be any published reports (with the exception of "On Track: Fifteen Years of Student Improvement. Wesley Elementary School, Houston, Texas," 1992) that systematically analyzed the impact of the reform on achievement measures for students at the school.

5. The school where DI was implemented was typical of the district, in which 60% of children lived in poverty and over half of families received welfare.

6. Our reading of Bowers differs from that of Adams and Engelmann (1996, p. 53).

7. The absence of a peer-reviewed research study on Wesley Elementary is particularly disappointing in this regard.