To make any reasonably legitimate claim about whether charter schools have been worth it or not, two basic questions need to be answered.
First – how well have these students done compared to how well they would have done if they hadn’t been in charter schools?
Second – what’s the impact been on other schools in the areas where the charters opened?
If we only answer the first part of question one (how well they have done) without any comparison to how well they may have done otherwise, we’re not getting useful information. Sure some of them might get good results at NCEA or in their National Standards, but without knowing (with some certainty) how these students would have done if they hadn’t been at a charter school, it’s meaningless.
How could an evaluation work this out? Well, there are quite a few different methods that could be useful. One is to look at how comparable cohorts of students do at regular schools. The crucial word here is ‘comparable’. I’ve no doubt that the MoE is already comparing the charter school results to data from other schools with similar profiles of students in terms of ethnic make- up, and the national and regional data for the groups of students in the charter schools. But is this really a comparable group of students? For one thing, these students and their parents have chosen a charter school. Perhaps then a comparison with students in other ‘schools of choice’ such as state integrated or special character schools would be fairer. Ideally, the best ‘control’ group to compare to would be students who applied to the charter schools but didn’t get in on a ballot – this won’t happen because so far the charter schools have been able to take everyone who has applied.
The second question, about impact on other schools matters because this is supposed to be a policy about addressing student underachievement as a whole. If the charter schools are getting great results, but the schools that the charter students are leaving are seeing their results slide (perhaps as a result of losing the more motivated students, or the impact of falling rolls meaning they can’t offer the educational options they could before) then it can’t be said to be an effective policy.
Anyway, neither of these questions will be answered by the charter school evaluation that the Ministry has contracted, nor by the reports that the Ministry and the schools themselves release trumpeting their successes.
The evaluation will no doubt provide interesting insights into how they operate, which is mostly what it is designed to do, but based on the proposal the Ministry put out for the research, is going to go nowhere near these difficult questions.
Finally, a question that won’t be answered by evaluation or research, that certainly is worth considering in any broad and honest appraisal of the policy is, could we have got something better if the money had been spent another way?
We know that programmes that cost less than charter schools ($15 mil in 2015), like Te Kotahitanga, can make a significant difference to large numbers of students. So what is the opportunity cost of the charter school policy? And on one side of the ledger has to sit the damage that this policy has done to the relationship between teachers and the government.