Listening to a ytube talk of brokenscience.org, I was directed to a peer-reviewed article of Trafimow (2021). A wonderful share, as this paper helps you to critique generalizations of results. Powerful. How often are you at a loss as to "what to write about" or "where research gap?". Thankfully, journal articles (for the most part) provide a rich vein of active thinking (critique) tips and ticks. In this context: Consideration of generalisation of results. Most interesting to reflect on was that results are not the only element of a research study that can be generalised. Trafimow (2021) points out that a research aim may be to generalise the theory guiding the study design, rather than the study findings per se. To determine what is being generalised requires considerations of sets of assumptions. Researcher bias
Table summarising Trafimow's assumption types which influence findings generalisation
The first part of Trafimow's (2021) paper unpacks these assumptions, with detailed examples. For example, a theory may be imbued with auxiliary assumptions (educated [or not] guesses which are not part of the theory per se). As an empirical hypothesis contains concepts, there can be a number of ways the researcher measures any given concept. The statistical hypothesis provides a concrete representation of the concept. Such as operationalising the concept 'higher' to be the 'mean score on a scale above the number x). Alternatively, the researcher could operationalise the concept to be the 'median score on a scale above the number x). Choices. How a researcher chooses to specify the statistical representation of a concept is arbitrary. This is helpful, as more than one statistical hypothesis can be generated to test the empirical hypothesis.
Trafimow (2021) identifies a gap that perhaps you, like me, have not considered. How does a researcher make the choice to bridge empirical and statistical hypotheses? Assumptions must be made, and again researcher bias comes into play. This is not 'bad', simply something to think about when evaluating a generalisation of a research finding.
The second part of the article unpacks inferential hypotheses and accompanying assumptions. A good argument is made for ensuing a p-value hypothesis test. However, as this is a topic I am still learning to wrap my mind around, I'll leave it for a future post.
Anyways, I thought to myself, what a useful article for developing my and others active critical thinking skills for a journal critique, an essay, a research report, or thesis implications section.
It was not an easy read at times. Though written in a simple and straightforward manner, with a visual model included ... it took me a few days to dip into parts, and re-reread others. Well worth the effort though.
Light & Life~ Charmayne
References Trafimow, D. (2021). Generalizing across auxiliary, statistical, and inferential assumptions. Journal for the Theory of Social Behaviour, 52(1), 37–48 https://doi.org/10.1111/jtsb.12296
