A recent federal evaluation found that using a voucher to attend a private school in D.C. reduced student math test scores by 10-percentile points and had no statistically significant effect on reading test scores. However, the same private school students were over 35 percent more likely than their public school peers to report that they were in a “very safe” school. These findings have led some education policy analysts to call for additional standardized testing regulations of school voucher programs. After all, we could just force all schools to be “high quality,” right?
Unfortunately, it’s not that easy. Especially when regulators use shaky measures of quality.
New rigorous evidence from education researchers Diether W. Beuermann and C. Kirabo Jackson suggests that parents know what’s best for their own children in the long-run. But whatever that is, it isn’t captured by standardized test scores.
The study uses data from over 40,000 students in Barbados. Families in the Caribbean island are able to rank-order their nine preferred schools for their children after 6th grade. And because the Barbados education system uses test score cutoffs to determine which students ultimately get access to their most preferred schools, the researchers are able to use a rigorous econometric methodology known as regression discontinuity design. This research method simply means that the researchers can compare students that barely get their preferred choice to those that barely miss the test score cutoff. And because it is essentially random chance that students fall on either side of the test score cut point, this methodology allows the researchers to establish a causal relationship between students getting their preferred schools and students’ outcomes.
The study found that attending a preferred school did not improve secondary school exit exam scores overall and reduced the number of tests passed for male students. But while getting a preferred school did not lead to better test scores, it led to significant long-term benefits for students, especially for females.
For instance, by the time respondents were around 25 to 40 years old, females that were able to attend their preferred secondary school were 17.4 percentage points more likely to have a university degree, 59 percent less likely to have a teenage birth, and had 42 percent higher monthly wages. Attending a preferred school increased the likelihood of attending a gym at least once a week by 12.5 percentage points, increased the likelihood of being within a normal BMI by 16.7 percentage points, and increased the chance that males were in the labor market by 11.4 percentage points.
These large long-run benefits are all meaningful. And very few reasonable people would argue that performance on a standardized test in secondary school is more important than employment, health, and earnings later on in life. The problem is that many technocratic education policy analysts still call to regulate reforms – like the D.C. voucher program – based on subpar standardized test score performance in the short-run.
But just imagine what would have happened to the students in Barbados if their preferred schools were closed down due to the fact that they did not improve standardized tests – or if parents lost the freedom to select schools based on evidence that they chose educational institutions that did not focus on shaping test scores. Their children might have instead ended up in schools that did better with improving test scores, but did not do a good job at building the character skills – such as hard work, timeliness, and strong communication – that are necessary for true long-run success. For similar reasons, it would be unwise to restrict the educational choices of parents using the D.C. voucher program based on faulty measures.
This new study adds to the mounting scientific evidence suggesting that standardized test scores are not strong proxies for the long-term outcomes that society actually cares about. In other words, education regulators ought to realize that the tools that they have to attempt to control the quality of schools are far from perfect. And they ought to realize that families already know what’s best for their own kids.