It’s been nearly 13 years since Hurricane Katrina decimated the city and its school system. And a lot has changed since then. Now the city is the first, large school district in the nation where nearly all students attend charter schools. But the reforms are controversial, and have left many wondering, did they work?
To help answer that question WWNO’s Jess Clark spoke with Tulane University professor Douglas Harris. Harris is the co-author of a new report from Tulane’s Education Research Alliance. The report measures the long-term impact of the school reforms.
Below is a Q&A based on their conversation. It has been lightly edited for clarity.
Q: You looked at how chartering schools has affected test scores, graduation rates, college access rates, from pre-Katrina to 2014. What are the main takeaways?
A: I think the big takeaway is that we find positive effects on every available measure - from test scores, to high school graduation, college entry, college persistence, and college graduation. Test score effects would be the equivalent of going from the 50th percentile to the 66th percentile, which is a pretty substantial improvement for an entire city to move that far. With high school graduation, it was a increase of 3-9 percentage points from a starting point of about 54 percent. The biggest percentage increase was actually in terms of college entry. There was a 67 percent increase in college attendance right after high school. So again, all positive, and these are all really big effects.
Q: Did you find that the reforms had any negative effects?
A: Not on these outcomes. I think there is some debate about whether we're measuring the right things. We're studying what we have and what we can measure. And I think they're all outcomes that people think are important. But I also think there's legitimate debate about whether we're capturing everything we want students to know. I think there's interest in whether students are learning about their local communities, whether they're connecting to their own cultures and the histories and backgrounds of their people.
Q: There has been a lot of criticism that past studies on education outcomes in New Orleans have failed to account for the fact that the population of New Orleans public schools is different than it was before the storm. Is that something you all accounted for in your study?
A: We did. We looked at this many different ways, and it doesn't change our results, no matter how we looked at it.
Q: The study seems to show that at first, the reforms were very effective, but that in the last couple years you looked at (2012-2014), they haven’t been working as well. What can you say about that?
A: I think basically what happened is that the outcomes improved steadily since the reforms started, and then we hit a plateau. In a few cases, the effects started to drop a little. The effects are still positive even in those most recent years, they're just not quite as positive as they were before. And so I think it's an interesting question about why that happened.
Q: Do we have any sense of how the reforms impacted the achievement gap - that is the gap in test scores between advantaged groups of students and disadvantaged groups of students?
A: We measured achievement gaps by looking at both race and income. When we look at the disadvantaged groups - black and low-income students - we see positive effects, and also reductions in the achievement gap when we look at high school graduation and college-going rates. In test scores, in the initial years the achievement gap seemed to grow between black and low-income students and their white and wealthier counterparts. Even though the disadvantaged students were improving, they just weren't improving as fast as advantaged students. In the most recent year though, it looks like the gap has returned to where it was before the reforms.
Q: New Orleans schools took in millions of federal and philanthropic dollars as a result of Hurricane Katrina. How do we know the improvements aren’t just the result of the schools having more money to work with?
A: We looked at this a couple years ago and found that spending in New Orleans increased faster than it had in other districts in the state - by about $1,300 per pupil. So this is a substantial increase in funding - about 10-15 percent. And prior research suggests that money matters. It's hard to determine how much of this is funding and how much of it's not. However, if you look at previous school reforms in other places where there have been increases in funding of similar amounts, you don't see these kinds of effects. So some of it is probabably driven by the reforms and some of it is about how the money was used by these new schools. But it's hard to know exactly how much of it is driven by funding.
Q: New Orleans is often talked about nationally as a model for education reform. Does the success of schools here suggest that going to a majority-charter district is something that should be replicated in other places?
A: I think it suggests that it's a possiblity. And I think communities need to make these decisions for themeselves. I also would say that New Orleans is different and had unique circumstances, not only in terms of the hurricane and how that led to the creation of the reforms, but also some advantages in making the reforms work. One of those was that lots of people wanted to come to New Orleans to be part of this reform effort. And so when you have lots of people who want to come, it builds an energy and brings in a lot of talent to help make things work. The second thing is that the district was doing really poor to begin with. So whenever you're performing really poorly, it's easier to improve on that. So in other districts that are performing better already, we might not see the same kinds of effects.
Q: It seems like this reports points to almost all positive outcomes, at least in terms of what you measured. Is there anything that should give us pause about the impact of these reforms?
A: Not so much in terms of the outcomes, but the process. I think that was what was most concerning - the way it was put in place after the tragedy of Katrina, and without a lot of local input and local support. I think that's the real challenge here. So it's a question of what kind of schools do we want? And now as a city, we get to decide that, and need to decide that. Whereas, it was really the state that was deciding that before.
Q: What does this report tell us about the challenges that lie ahead for the Orleans Parish School Board, now that all the schools have come back under its control?
A: I think there are a couple of challenges. I think one is building a stronger base of community support around the reforms. To this point the state has been really the driving force behind the reforms and the level of local input has been really modest. Now, I think the question is: is there going to be enough support to continue the system as it is now?
I think a second challenge is, when we look at the reasons why we see these effects, one of the factors seems to be closing and taking over low-performing schools. So I think there's also a question about how the district is going to handle that going forward. Those are difficult decisions to make.
Support for WWNO’s education reporting comes from Entergy Corporation.