First Outcomes from the National Summer Learning Study

The Wallace Foundation is funding a five-year demonstration project in five urban school districts in Boston, Dallas, Duval County (Florida), Pittsburgh, and Rochester (New York). These districts have been pioneers in offering full-day voluntary programs for five to six weeks free of charge to large numbers of struggling elementary students, not just those facing grade retention. The districts all provide at least three hours of academic instruction in math and reading by certified teachers, along with a range of enrichment activities (e.g., art, music, tennis, swimming), many of which are provided by community-based organizations that partner with the district. 

This study, commissioned by The Wallace Foundation, is the first randomized controlled trial to test the effectiveness of large-scale, district-run, voluntary summer learning programs. In a series of reports, RAND researchers will describe whether such programs benefit low-income elementary students and what program features are associated with positive outcomes. The first results, presented in Ready for Fall? Near-Term Effects of Voluntary Summer Learning Programs on Low-Income Students' Learning Opportunities and Outcomes, describe the effects of the 2013 summer programs in the autumn after the programs ended.

Early Outcomes
 
Wallace found that there was strong demand for these programs among low-income children and their families, and that these programs appeared to provide opportunities that these children would not have had otherwise.
 
The programs had a significant positive effect on students' mathematics achievement when compared to students in the control group.
 
The programs had a significant positive effect on students' mathematics achievement. The average effect size across the five school districts was 11 percent of 1 standard deviation. This number reflects the spread in scores between students who participated in the program and those who did not, and not the growth in learning from the beginning of the summer to the end for either group. The effect is reasonably large for a five-to-six-week program. To set it in context, the average growth in mathematics achievement between the spring of third grade and the spring of fourth grade is about 52 percent of 1 standard deviation.
 
The researchers found no similar effect for reading skills: The difference was just 1 percent of 1 standard deviation, which was not statistically significant. This finding is somewhat surprising, given the improvements in math, but one explanation may be that it is more difficult to improve reading comprehension skills in a short program.
 
Students in the program also showed no difference in social-emotional competencies between the treatment group and the control group. Although some district leaders hypothesized that their programs might have a positive effect in this area, only one district explicitly designed a program with this outcome in mind, and while it did have a higher effect size than the other districts, the estimate was not statistically significant.
 
Five factors that may help improve outcomes
 
Another focus of the study is how to best implement programs of this kind. This non-experimental analysis examined a number of program features to see if they were related to positive student outcomes. Of seven factors examined, five had a statistically significant association with mathematics or reading outcomes.

 

Learn more and read the full report at: http://www.rand.org/pubs/research_briefs/RB9819.html