Not too long ago, the word meta-analysis was ever-present in the education world. John Hattie and his meta-meta-analysis was the big news. Actually, to be more precise, the german translation came out only two years ago, and it was around that time when I first heard about it. I was in the middle of my studies and our teacher training didn't really include any in-depth discussions about educational research. Although I became somewhat familiar with the topic in his book, I didn't really get closer to research and the theory behind it.
I graduated and co-wrote my bachelor thesis with a good friend of mine. We did some basic research, but we had a great supply of data from the research project we were involved with. Additionally, it is basically a teaching certificate, so in its essence, we only really acquire the competency for teaching, not as much continuing on with research.
Either way, I became involved in educational research in my Masters program right from the get go. It is kind of funny, because based on a couple of forum messages and questions from other students, it seemed like this particular MA program really did presume that everyone enrolled is (fully) competent when it comes to research methods and theories, or at least have statistical and mathematical knowledge. I wouldn't say I was surprised that they assumed everyone has those basic principles down, but I also think about the many different degrees in the german-speaking countries. Considering they are a distance-education university, one would think they are aware that their students come from varying backgrounds. In addition to that, they accept various degrees and the Austrian degree we attained, is focused more on going straight into teaching and like I mentioned before, not like other "teaching" degrees from universities.
Anyway, it was not easy getting into statistics again. Since that is pretty much the basis for educational research and studies, there was no way around it, we had to work through it. It was really interesting in the beginning, because the tasks we got, got us started on the right foot. We had a couple of example meta-analysis we could choose from and analyze. As the term went on we got more and more comfortable in discussing meta-analysis, despite all the statistics and maths.
We delved into meta-analysis, research and numbers, and a couple interesting philosophical ideas and thoughts came up. For one, it started with having read a couple of postings about the aforementioned meta-meta-study (here is one and two.) Secondly, our task was to pick a certain topic for the end of the term and pretty much do a "theoretical" meta-analysis about it. Obviously the focal point was not doing the actual research, but getting us to understand the process.
Although I have limited knowledge and experiences, I do feel like educational research is really, really tough. It is tough because people can be very different. Of course one could take on neuroscience and psychology in order to work out "how" exactly people work, but even that is one of the more challenging scientific research fields.
Secondly, a more 'hands-on' thought if you will, concerned the sample size. Obviously the sample size is something that is not only relevant to educational research, it is vital for medical research as well. I think it is interesting to discuss and think about the threshold of sample sizes in educational research. What kind of threshold would you feel comfortable with? Is it simply the class size? The number of students at a specific university? The children attending schools in a city? I think herein is one of the toughest parts. If you decide to research the effects of primary school children using tablets in mathematics classes, and you find out that 12 out of 20 show improvements in their learning, does it make sense to introduce the same exact method in another random primary class (and expect the similar results)?
Would it make sense to try this method on the whole school? What if it still shows a positive effect (depending on your method of statistical-choice and your definition of 'positive')? Can we just take that method and put it to use in the whole district? Although the sample size becomes bigger, the results might become less encouraging. Now if the sample sizes are big enough and you get a somewhat 'positive' effect, and you look at it straight on, you have to think about the actual number of children that improved, and weigh it against the number of children that yielded the same results as before. Did your opinion of the method change?
Now the most obvious point is regarding variables. It kind of ties in with the first point I made (and could shift the attention to more individualization in the classroom). Not only are there many people who are all different, but there are even more variables around those people. How old are they, where are they from, what previous experiences do they have and so on. So even if you have a reasonable sample size, is it even possible to include enough variables?
The field of educational research is fascinating. It might not be for all people, because it involves way more statistics and mathematics than regular humans would like to see. Also, we will probably not get any "100%-positive-effect-guaranteed" study results any time soon. So what is its purpose?
For me, most of it is a great thought experiment. For example, if a study performed in Europe about the effects of tablet use in the classroom and the results are positive, does it mean that I can stick tablets into African schools and see the children sprout with knowledge? If the answer is no, then it gets interesting, because then I can think about the differences and variables that would change that and so on.
Educational research can also point to various signs or function as an indicator. Just think about when the use of computers started in the educational world, and how it shaped its own development and changed along the way. So it is certainly trend-setting in some way. It can point to new directions or ideas and it can lead to the development of new theories, and those theories can someday turn into studies, and the cycle can repeat again.
