Educational research – a form of lying?
PPTA News, October 2009, p. 5
Education research in New Zealand could be about as credible as research commissioned by drug and tobacco companies into their products, a Massey University professor says.
Professor of teacher education John O’Neill gave a presentation “New Zealand Research Says Teachers Matter Most: Truth or Truespeak?” to PPTA’s Wellington regional conference in August.
Seeking research to support policy - a vested interest
In it he pointed out there was little completely independent or disinterested research done in New Zealand.
He questioned the independence, credibility and truthfulness of educational research, when the Ministry of Education (MoE) is all but the monopoly commissioner of it – particularly when research in New Zealand tends to be closely tied to policy in which the funder had a vested interest, he said.
Professor O’Neill is critical of research that underpins claims quality of teaching is the biggest factor in student underachievement.
It has been widely asserted by the MoE that “evidence” shows up to 59% of variance in student performance can be attributed to teachers.
Leading questions get the 'right' answer
Professor O’Neill disputes this figure – questioning the credibility of research that relied on ministry funded, competitively tendered, short-term contracts. MoE tenders often specified the questions, methods and even samples researchers use, professor O’Neill said.
“This raises questions about its independence and truthfulness, just as with much of the ‘research’ commissioned by drug and tobacco manufacturers.
Variance - what is the supporting data?
“Do most studies show 59% variance, one study only, or is this an average variance across all studies? The assertion is meaningless unless we know about the data that were used to support it."
Context - looking at the whole picture
Fifty nine percent nevertheless became a political benchmark, irrespective of context. However, at the same time the MoE released another review which said “taken as a whole, family and community resources, processes and characteristics are the most important influence on educational outcomes for children in early childhood and schooling”.
Both statements together say something considerably more meaningful and truthful to uninformed readers than either does alone, Professor O’Neill said.
The whole picture shows family and community are the most important in terms of successful educational outcomes over all, but teachers have the greatest influence within the school.
Picking up the pace and class size
Issues relating to class size have been another casualty of this cavalier approach to research, Professor O’Neill said.
He used the example of a large scale MoE schooling improvement initiative in South Auckland called Picking up the Pace. Research data reported a significant increase in literacy achievement. However, a much less widely reported aspect of the study was that class sizes were kept unusually low so teachers could complete the additional assessment, teaching and record keeping.
“To not give the class size variable equal weighting is like reporting an observational study which showed that light and heat are the requirements for successful plant growth, while omitting the fact that the plants were also watered each day.
“No reputable scientist would do so but for some reason it appears to be regarded as perfectly acceptable to be selective in the dissemination of New Zealand educational research. “It’s the overriding message that counts, not the simple facts,” he said. “This is marketing, not science.”
Fact or opinion?
Another factor of concern for Professor O’Neil is published research that doesn’t clearly define what is fact and what is opinion.
He uses John Hattie’s book, Visible Learning to illustrate this. Along with his controversial claim that class size is unimportant, the media picked up on Hattie’s personal beliefs about performance pay, which professor O’Neill says have no basis in the evidence.
“It is difficult to imagine a report of a viticulture science researcher’s findings on grape vine quality being accompanied by her opinions on the hourly wage rates paid to seasonal fruit pickers in Marlborough vineyards,” he said.
It should be up to the researcher to clarify these misrepresentations, but in Hattie’s case this did not happen.
Truthful presentation is a researcher responsibility
Educational researchers need to ensure, in so far as they are able, that their research is presented in the most truthful manner, Professor O’Neill said.
“These statements do not necessarily falsify the facts, but they omit essential information and, as such, may be regarded as a form of lying.”
 New Zealand research says teachers matter most: truth or truespeak (John O'Neill)
PPTA News October 2009
 |
Subscribe to this comment's feed
Show/hide comment form