The replication crisis, context sensitivity, and the Simpson’s (Paradox)

The Reproducibility Project: Psychology:

The Reproducibility Project: Psychology (OSC, 2015) was a huge effort by many different psychologists across the world to try and assess whether the effects of a selection of papers could be replicated. This was in response to the growing concern about the (lack of) reproducibility of many psychological findings with some high profile failed replications being reported (Hagger & Chatzisarantis, 2016 for ego-depletion and Ranehill, Dreber, Johannesson, Leiberg, Sul, & Weber, 2015 for power-posing). They reported that of the 100 replication attempts, only ~35 were successful. This provoked a strong reaction not only in the psychological literature but also in the popular press, with many news outlets reporting on it. read more

Notes on Paul Meehl’s “Philosophical Psychology Session” #03

These are the notes I made whilst watching the video recording of Paul Meehl’s philosophy of science lectures. This is the third episode (a list of all the videos can he found here). Please note that these posts are not designed to replace or be used instead of the actual videos (I highly recommend you watch them). They are to be read alongside to help you understand what was said. I also do not include everything that he said (just the main/most complex points).

Descriptive discourse: what is. read more

Brexit in graphs

This is a collection of graphs showing how people voted and other interesting statistics. Some you might not have seen and others you definitely will have. I’m not going to include every graph, especially the most common ones, as you will almost certainly have seen them. Please remember to take all the polls with a pinch of salt (only a small sample of people can be asked and it may not be representative, people may have given socially desirable answers, lied, etc.). If there are any graphs you feel I have missed, please comment below and I will add them. read more

Notes on Paul Meehl’s “Philosophical Psychology Session” #02

These are the notes I made whilst watching the video recording of Paul Meehl’s philosophy of science lectures. This is the second episode (a list of all the videos can he found here). Please note that these posts are not designed to replace or be used instead of the actual videos (I highly recommend you watch them). They are to be read alongside to help you understand what was said. I also do not include everything that he said (just the main/most complex points).

Popper did not accept the verifiable criterion of meaning. Popper never said falsifiability was a criterion of meaning. read more

Notes on Paul Meehl’s “Philosophical Psychology Session” #01

These are the notes I made whilst watching the video recording of Paul Meehl’s philosophy of science lectures. This is the first episode (a list of all the videos can be found here). Please note that these posts are not designed to replace or be used instead of the actual videos (I highly recommend you watch them). They are to be read alongside to help you understand what was said. I also do not include everything that was discussed (just the main/most complex points).

  • Power of hard sciences doesn’t come from operational verbal definitions but from the tools of measurements & the mathematics.
  • A subset of the concepts must be operationally defined otherwise it doesn’t connect with the facts.
  • Methodological remarks= remark in the meta language (statements that occur in science and the relations between them, properties of statements and between statements, relations between beliefs and evidence e.g. true, false, rational, unknown, confirmed by data, fallacious, deducible, valid, probable) rather than object language (language that speaks about subject matter e.g. protons, libido, atom, g, reinforce),
  • Hans Reichenbach was wrong about induction
  • Pure observations are infected by theory (FALSE for psychology). If protocol you record is infected by theory, bad scientist e.g. Choosing to look at 1 thing rather than another just because of a theory OR falsifying data just to fit your theory.
  • Watson’s theory that learning took place in muscles (from proprioception feedback) was falsified by rats being able to negotiate a maze almost as quickly after having neural pathway that controlled proprioception feedback severed or when the maze was flooded.
  • Operationalism (we only know a concept if we can measure it & all necessary steps for demonstrating meaning or truth must be specified) sparked psychologies’ obsession with operationalising our terms (even though the harder sciences we are trying to emulate are not as rigorous with it) but Carnap suggests it is folly.
  • Logical positivism- taking things that could not be doubted by any sane person and building up from there a justification for science, and with the math and logic on top of the protocols you “coerce them” into believing in science. Urge for certainty.
  • Analyse science and rationally reconstruct (justify) it, show why a rational person should believe in science. Negative aim: liquidation of metaphysics (by creating meaning criterion).
  • A statement is cognitively meaningless if you don’t know how to verify it (either empirically or logically)- Criterion of meaning. The meaning of a sentence is the method of it’s verification, statement of affirmative meaning. A sentence’s meaning is derived from the evidence that supports it (“the meaning of a sentence is to be found entirely in the conditions under which it could be verified by some possible experience”*). Rejected because the sentence “Caesar crossed the Rubicon” means COMPLETELY different things to us and to a Centurion at Caesar’s side because we have different evidence.
  • Lots of our information comes from “authorities” (even though it’s a logical fallacy). We have to calibrate the authority and often we presume someone has done it for us so we trust it.

*http://hume.ucdavis.edu/mattey/phi156/schlickslides_ho.pdf read more

In defence of preregistration

This post is a response to “Pre-Registration of Analysis of Experiments is Dangerous for Science” by Mel Slater (2016). Preregistration is stating what you’re going to do and how you’re going to do it before you collect data (for more detail, read this). Slater gives a few examples of hypothetical (but highly plausible) experiments and explains why preregistering the analyses of the studies (not preregistration of the studies themselves) would not have worked. I will reply to his comments and attempt to show why he is wrong. read more

Podcast list

I’ve recently discovered podcasts and they are awesome. They’re a great way to learn interesting new things, especially when you’re travelling. So this post is a collection of fantastic podcasts that I listen to and would recommend you pick up. Any suggestions are welcome so please let me know if there are any you like. (*= my favourites).

Social and Life sciences:

*Everything Hertz: Discussions about biological psychiatry, psychology, and the process of science with heavy sarcasm. (iTunes) (Soundcloud) read more

Video games cause violence

For almost as long as there have been video games, there have been people arguing that they are bad for you. There also seems to be a wealth of experimental evidence behind it (Hasan et al., 2013, to name just one of many). But there have been suggestions that these negative outcomes are oversold.

2DJimmyN/DeviantArt

Problems with the literature:

One of the strongest pieces of evidence for the negative effects of video games is a meta-analysis by Anderson et al. (2010). They found strong evidence that “exposure to violent video games is a causal risk factor for increased aggressive behaviour, aggressive cognition, and aggressive affect and for decreased empathy and prosocial behaviour”. However, there were immediate questions about the methodology in this meta-analysis. Ferguson & Kilburn (2010) commented that many studies do not relate well to aggression and the authors do not consider the impact of unstandardised aggression measures (differences between studies in how they measured aggressive behaviour), among other things. They comment that the studies analysed in Anderson et al. (2010) only show weak evidence for their conclusion. A more recent reanalysis by Hilgard, Engelhardt, and Rouder (2016) used more advanced tools to adjust for research bias and found that the short-term effects of game play on aggressive feelings and behaviour were badly overestimated by bias. The adjustments recommended by Hilgard et al. (2016) were mostly substantially lower than those performed by Anderson et al. (2010), with some being smaller adjustments. In some studies, the result was adjusted to zero e.g. aggressive affect. This does not completely eliminate the original findings but I feel we should adjust our estimate of the strength of the causal association downwards. read more

Collection of criticisms of Adam Perkins’ ‘The Welfare Trait’

In late 2015, Dr Adam Perkins published his book called ‘The Welfare Trait’. The main crux of his argument was that each generation who is supported by the welfare state becomes more work-shy. He also argued that the welfare state increased the number of children born to households where neither parent works. His solution is to change the welfare state to limit the number of children that each non-working household has.

His book caused quite a storm when it was first released. Some people argued that it was crudely-disguised eugenics, others argued that those who were dismissing it were refusing to face the facts. Over time, more and more criticisms of and problems with Perkins’ work have come to light (e.g. basic statistical errors and incorrect conclusions from papers). Below is a collection of some (but not all) of the criticisms levelled at Perkins’ book. read more

Stereotype threat

Don’t you just love being wrong? Of course you don’t, no one does. But there is a grim satisfaction in no longer believing something that there isn’t good enough evidence for. This is what I experienced after examining the phenomenon known as ‘stereotype threat’. In short, it’s the idea that groups with negative stereotypes about them feel anxiety when these stereotypes are made salient (and are therefore more likely to confirm those stereotypes) e.g. women being inferior than men at maths. read more