Category: Metascience
-
The Death of the Academic Author: why we should focus less on the writer
A recent article exploring the relationship between exercise and mental health (Chekroud et al., 2018) has stirred a lot debate. It was an interesting study and gained a lot of traction, with over 2500 retweets of the original summary thread by Adam Chekroud and multiple articles by various news outlets e.g. the BBC, Time. There was…
-
Failed replications, questioning authority, and outdated mentalities
Researchers face a tricky situation when another scientist fails to replicate their work. Do you double down on your original findings, admit your work was likely a false positive, or something between these poles? And how do you respond when a colleagues work fails to replicate? All these questions were raised again after a series…
-
The positive predictive value/false discovery rate are fundamentally flawed
The positive predictive value (PPV) was brought to a wider audience by John Ioannidis in his famous 2005 paper ‘Why Most Published Research Findings Are False’. The related concept false discovery rate (FDR) was popularised by Colquhoun (2014), though the idea it’s based on dates back to Jeffreys (1939). They have become very popular in recent…
-
Detaching my worth from my work: should criticism be taken personally?
Honest criticism of scientific work is obviously essential. It’s generally agreed criticism should focus on the work in question, not on the skill or motives of the researcher[note]For a good set of guidelines on how to give constructive criticism, I recommend reading Rapoport’s rules[/note]. Some times criticism can go too far and there is a…
-
How I keep track of what I’ve read
To be a scientist, you need to read a lot. Not just field relevant work, but other topics you need to be familiar with in order to conduct research like statistics, philosophy of science, etc. Unless you have an eidetic memory, you need to have a system of storing what you’ve read. I’ve wasted hours…
-
Replication and Reproducibility Event II: Moving Psychological Science Forward
On Friday 26th of January at the Royal Society there was a series of talk on how psychology could progress as a science, with an emphasis on replication and reproducibility. I’m going to summarise the key points from the individual talks and make a few comments. A collection of all the individual videos of the…
-
Why do people leave academia?- The results
On social media, there are often discussions about why psychologists leave academia. Some argue that the new culture of criticism (where overly harsh[note]In their view[/note] criticism is leveled against those who make errors and accusations of malfeasance are rife) make academics, especially younger ones, change profession. They give examples of high profile cases where researchers have…
-
I’m a non-methodologist, does it matter if my p-value definition is wrong?
A few weeks ago, Nature published an article summarising the various measures and counter-measures suggested to improve statistical inferences and science as a whole (Chawla, 2017). It detailed the initial call to lower the significance threshold to 0.005 from 0.05 (Benjamin et al., 2017) and the paper published in response (Lakens et al., 2017). It…
-
Assessing the validity of labs as teaching methods and controlling for confounds
Anyone who has taken one of the harder sciences at university or knows someone who has will know what “labs” are. You are given practical assignments to complete that are meant to consolidate what you’ve learnt in the lecture/seminar. They are almost ubiquitous in physics after becoming widespread by the beginning of the 20th century…
-
Why do psychologists leave academia?
Every once in a while in the psychology sphere of social media there’s a discussion about why people leave academia. This talking point often comes up in the context of “the open science movement” and whether more academics leave because of the cultural of criticism or because of the lack of replicability of some findings.…
PsychBrief
PsychBrief 