In defence of preregistration

This post is a response to “Pre-Registration of Analysis of Experiments is Dangerous for Science” by Mel Slater (2016). Preregistration is stating what you’re going to do and how you’re going to do it before you collect data (for more detail, read this). Slater gives a few examples of hypothetical (but highly plausible) experiments and explains why preregistering the analyses of the studies (not preregistration of the studies themselves) would not have worked. I will reply to his comments and attempt to show why he is wrong. read more

Podcast list

I’ve recently discovered podcasts and they are awesome. They’re a great way to learn interesting new things, especially when you’re travelling. So this post is a collection of fantastic podcasts that I listen to and would recommend you pick up. Any suggestions are welcome so please let me know if there are any you like. (*= my favourites).

Social and Life sciences:

*Everything Hertz: Discussions about biological psychiatry, psychology, and the process of science with heavy sarcasm. (iTunes) (Soundcloud) read more

Video games cause violence

For almost as long as there have been video games, there have been people arguing that they are bad for you. There also seems to be a wealth of experimental evidence behind it (Hasan et al., 2013, to name just one of many). But there have been suggestions that these negative outcomes are oversold.


Problems with the literature:

One of the strongest pieces of evidence for the negative effects of video games is a meta-analysis by Anderson et al. (2010). They found strong evidence that “exposure to violent video games is a causal risk factor for increased aggressive behaviour, aggressive cognition, and aggressive affect and for decreased empathy and prosocial behaviour”. However, there were immediate questions about the methodology in this meta-analysis. Ferguson & Kilburn (2010) commented that many studies do not relate well to aggression and the authors do not consider the impact of unstandardised aggression measures (differences between studies in how they measured aggressive behaviour), among other things. They comment that the studies analysed in Anderson et al. (2010) only show weak evidence for their conclusion. A more recent reanalysis by Hilgard, Engelhardt, and Rouder (2016) used more advanced tools to adjust for research bias and found that the short-term effects of game play on aggressive feelings and behaviour were badly overestimated by bias. The adjustments recommended by Hilgard et al. (2016) were mostly substantially lower than those performed by Anderson et al. (2010), with some being smaller adjustments. In some studies, the result was adjusted to zero e.g. aggressive affect. This does not completely eliminate the original findings but I feel we should adjust our estimate of the strength of the causal association downwards. read more

Collection of criticisms of Adam Perkins’ ‘The Welfare Trait’

In late 2015, Dr Adam Perkins published his book called ‘The Welfare Trait’. The main crux of his argument was that each generation who is supported by the welfare state becomes more work-shy. He also argued that the welfare state increased the number of children born to households where neither parent works. His solution is to change the welfare state to limit the number of children that each non-working household has.

His book caused quite a storm when it was first released. Some people argued that it was crudely-disguised eugenics, others argued that those who were dismissing it were refusing to face the facts. Over time, more and more criticisms of and problems with Perkins’ work have come to light (e.g. basic statistical errors and incorrect conclusions from papers). Below is a collection of some (but not all) of the criticisms levelled at Perkins’ book. read more

Stereotype threat

Don’t you just love being wrong? Of course you don’t, no one does. But there is a grim satisfaction in no longer believing something that there isn’t good enough evidence for. This is what I experienced after examining the phenomenon known as ‘stereotype threat’. In short, it’s the idea that groups with negative stereotypes about them feel anxiety when these stereotypes are made salient (and are therefore more likely to confirm those stereotypes) e.g. women being inferior than men at maths. read more

How biased are you? The role of intelligence in protecting you from thinking biases.

People generally like to believe they are rational (Greenberg, 2015). Unfortunately, this isn’t usually the case (Tversky & Kahneman, 1974). People very easily fall prey to thinking biases which stops them from making a purely rational judgement (whether always making a rational judgement is a good thing is a discussion for another time). These are flaws in thinking e.g. the availability bias, where you judge the likelihood of an event or the frequency of a class by how easily you can recall an example of that event (Tversky & Kahneman, 1974). So after seeing a shark attack in the news, people think the probability of a shark attack is much higher than it is (because they can easily recall an example of one). read more

The benefits of single-sex schooling

Many people claim that single-sex (SS) education is better for students than co-educational (CE) e.g. Jackson (2016). There have been criticisms of this idea e.g. Halpern et a. (2011) but generally it is believed to be beneficial. But what does the evidence suggest? A large-scale meta-analysis by Pahlke et al. (2014), involving 184 studies and 1,663,662 students, compared them on a variety of variables (mathematics performance; mathematics attitudes; science performance; science attitudes; attitudes about school; gender stereotyping; self-concept; interpersonal relations; aggression; victimisation; and body-image) to see if attending a SS school benefited males, females, or both. read more

How views about willpower affect you and your grades

There has been a lot of research into how self-control (defined as “restraint exercised over one’s own impulses, emotions and desires” Merriam-Webster, 2015) is affected by performing tasks that require self-control. One hypothesis with a large amount of experimental evidence to support it is the strength model of self-control (Baumeister, Bratslavsky, Muraven, & Tice, 1998; Baumeister, Vohs, & Tice, 2007). This holds that people’s self-control is a limited resource and that once used up, people will be less able to exert self-control later and will therefore be less likely to restrain themselves (Hagger, Wood, Stiff, & Chatzisarantis, 2010). This loss of your self-control resource is called “ego depletion”. Believing this then supposedly allows you to allocate you resources more efficiently and thus improve self-regulation (Vohs, Baumeister, & Schmeichel, 2012).

However there have been several studies that suggest ego depletion itself is not the cause of reduced self-control at a later time; it’s the person’s beliefs about whether their self-control resources is depleted or not that results in lapses of self-control (Job, Dweck, & Walton, 2010). So it’s not that your self-control resources is actually depleted; it’s that you believe they have and you will therefore be less likely to put in the effort required to maintain self-control. This is contrasted with those who believe in a nonlimited theory of willpower who do not experience a decrease in self-control across demanding tasks (Miller, Walton, Dweck, Job, Trzesniewski, & McClure, 2012).

A study by Job, Walton, Bernecker, & Dweck (2015) looked at the effect different beliefs about willpower had on everyday self-regulation (e.g.procrastination, consumption of unhealthy foods, poor time management, excess spending, and failure to control emotions). The participants had to say when they had experienced “self-regulation failures” in the past week. The data was therefore based on self-report, which comes with a host of problems (social desirability bias, lying), some time after the event occurred (so the participants may have forgotten). They were also  required to predict how many demands they would face in the coming week (academic tasks e.g. “tests to take”, and social stressors e.g. “experience of social exclusion”). Their natural self-control ability was also calculated (through a questionnaire).

They found no significant difference in anticipated demands between students with different theories about willpower. When students experienced/reported high demand, those with a limited resource-theory reported a greater number of self-regulation failures on procrastination. The other measures either didn’t reach significance or only just reached it (so I’m not going to focus on those). There was no significant difference between theories of willpower when demands were low. The possibility that students who endorsed the limited resource-theory were simply worse at self-regulating behaviour was controlled for and they still found a significant effect of different theories of willpower on self-regulatory failures (during high demand). This implies their beliefs about willpower affected their reported self-regulatory failures, as opposed to their natural ability to control themselves being the only causal factor.

The next step was examining whether beliefs about willpower affected an objective measure (in this case, GPA or grade point average for us non-Americans). Even when controlling for prior GPA, the students who agreed with the limited resource-theory scored lower on their GPA (though this variable only just reached significance). They also found that students who believed in the limited resource theory (and were on a course with a high work-load) scored significantly lower GPA’s than students who endorsed a nonlimited view of willpower on the same course (this last result was found even when the participant’s natural self-control was controlled for).

This is an interesting study as it suggests students who hold willpower is a limited resource are more likely to procrastinate and thus achieve a lower grade in their final tests. I feel running this as a longitudinal study and using better methods of recording self-regulatory failures would be a good next step.

Baumeister, R.F.; Bratslavsky, E.; Muraven, M.; & Tice, D.M. (1998). Ego Depletion: Is the Active Self a Limited Resource? Journal of Personality and Social Psychology, 74 (5), 1252-1265.
Baumeister, R.F.; Vohs, K.D.; & Tice, D.M. (2007). The Strength Model of Self-Control. Current Directions in Psychological Science, 16 (6), 351-355.
Hagger, M.S.; Stiff, C.; & Chatzisarantis, N.L.D. (2010). Ego Depletion and the Strength Model of Self-Control: A Meta-Analysis. Psychological Bulletin, 136 (4), 495-525.
Job, V.; Dweck, C.S.; & Walton, G.M. (2010). Ego Depletion- Is It All In Your Head? Implicit Theories About Will-Power Affect Self-Regulation. Psychological Science, 21 (11), 1686-1693.
Job, V.; Walton, G.M.; Bernecker, K.; & Dweck, C.S. (2015). Implicit Theories About Willpower Predict Self-Regulation and Grades in Everyday Life. Journal of Personality and Social Psychology, 108 (4), 637-647.
Merriam-Webster. (2015). Self-control. Available at: Last accessed 07/04/2015.
Miller, E.M.; Walton, G.M.; Dweck, C.S.; Job, V.; Trzesniewski, K.H.; & McClure, S.M. (2012). Theories of Willpower Affect Sustained Learning. PLoS ONE, 7 (6), e38680, doi:10.1371/journal.pone.0038680.
Vohs, K.D.; Baumeister, R.F.; & Schmeichel, B.J. (2012). Motivation, personal beliefs, and limited resources all contribute to self-control. Journal of Experimental Social Psychology, 48 (4), 943-947. (function(i,s,o,g,r,a,m){i[‘GoogleAnalyticsObject’]=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,’script’,’//’,’ga’); ga(‘create’, ‘UA-63654510-1’, ‘auto’); ga(‘send’, ‘pageview’); read more

So what are the best learning techniques?

We’ve looked at (some of) the evidence for different learning techniques, but which ones are the most effective? The clear winners are active retrieval and distributed practice (click on each technique for the evidence and a more in-depth analysis), so definitely try and make them a part of your learning/revision schedule. After that, it seems that summarisation has limited utility if you know how to do it properly (if you don’t it appears to be a bit of a waste of time). Self explanation may be useful, but there’s not really enough evidence to conclusively say that either way. Rereading and highlighting are very ineffective ways of learning so I would encourage you not to use them, especially if it stops you from using far more effective techniques. They may also install a false sense of understanding because you are more familiar with them, rather than actually understanding the material.

So if you want to be as effective as possible with your learning and give yourself the best chance of doing well, do these things: test yourself on the material; spread out your revision sessions and don’t just highlight or reread your notes!

PsychologyBrief. (2014). Active Retrieval. Available: Last accessed 19/02/2015.
PsychologyBrief. (2014). Highlighting. Available: Last accessed 19/02/2015.
PsychologyBrief. (2014). Rereading. Available: Last accessed 19/02/2015.
PsychologyBrief. (2014). Self-explanation. Available: Last accessed 19/02/2015.
PsychologyBrief. (2014). Summarising. Available: Last accessed 19/02/2015.
PsychologyBrief. (2015). Distributed Practice. Available: Last accessed 19/02/2015. (function(i,s,o,g,r,a,m){i[‘GoogleAnalyticsObject’]=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,’script’,’//’,’ga’); ga(‘create’, ‘UA-63654510-1’, ‘auto’); ga(‘send’, ‘pageview’); read more

Active Retrieval

Active retrieval is the method of looking at some information and then testing yourself (either through free-recall or guided recall) to see how much you can remember. But the key aspect of this method is that it isn’t used just to see how much someone has learnt or if there are any gaps in their knowledge. Rather it is to actually help them learn and remember the information (by consolidating it in their mind).

The utility of active retrieval is well documented. They have found that students who used practice testing showed greater retention of information compared to: creating mind maps (Karpicke & Blunt, 2011); increased exposure to the materials or restudying (Roediger & Karpicke, 2006; Pyc & Rawson, 2010).

This effect is also seen across age groups, from kindergartners (Fritz, Morris, Nolan & Singleton, 2007) to university medical students (Kromann, Jensen & Ringsted, 2009).

There is evidence to suggest that testing can help transfer information; when tested in a different format to the the one they had learnt the original information in, participants remembered more and performed better if they had employed active retrieval learning techniques (Carpenter, 2012 for a review).

After a thorough review, Dunlosky et al. (2013) rated practice retrieval’s utility as a learning technique as “high” and it is easy to see why, given how effective it is and how easy it is to implement.

Carpenter, S.K. (2012). Testing Enhances the Transfer of Learning. Current Directions in Psychological Science, 21 (5), 279-283.
Dunlosky, J.; Rawson, K.; Marsh, E.; Nathan, M. & Willingham, D. (2013). Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychological Science in the Public Interest. 14 (1), 4-58.
Fritz, C.O.; Morris, P.E.; Nolan, D. & Singleton, J. (2007). Expanding Retrieval Practice: an effective aid to preschool children’s learning. The Quarterly Journal of Educational Psychology, 60 (7), 991-1004.
Karpicke, J.D. & Blunt, J.R. (2011). Retrieval Practice produces more learning than Elaborative Studying with Concept Mapping. Sciece, 331 (6018), 772-775.
Kromann, C.B.; Jensen, M.L. & Ringstead, C. (2009). The effect of testing on skills learning. Medical Education, 43, 21-27.
Pyc, M.A. & Rawson, K.A. (2010). Why Testing Improves Memory: Mediator Effectiveness Hypothesis. Science, 330 (6002), 335.
Roediger, H.L. & Karpicke, J.D. (2006). Test Enhanced Learning: Taking Memory Tests Improves Long-term Retention. Association for Psychological Science, 17 (3), 249-255. (function(i,s,o,g,r,a,m){i[‘GoogleAnalyticsObject’]=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,’script’,’//’,’ga’); ga(‘create’, ‘UA-63654510-1’, ‘auto’); ga(‘send’, ‘pageview’); read more