These are the notes I made whilst watching the video recording of Paul Meehl’s philosophy of science lectures. This is the seventh episode (a list of all the videos can he found here). Please note that these posts are not designed to replace or be used instead of the actual videos (I highly recommend you watch them). They are to be read alongside to help you understand what was said. I also do not include everything that he said (just the main/most complex points).

10 obfuscating factors (contd)

Example for Lyken’s crud factor:

T is literally true, two auxiliary theories (A1 and A2) both of which have a .9 probability of being true, cp clause has a .9 probability of being true, and the conditions have a .9 probability of being true. What’s the probability (given the above) that o1⊃o2? .9⁴=.66. Chances of getting that result because of theory being true is 2/3 even if you had perfect power. With 80% the probability is .52 of the observation coming from the theory.

On a normal distribution graph, one standard deviation is 8/10ths from the mean.

Mean deviation roughly equals .85 of a standard deviation.

When the theory is false (with a crud factor of .3) you have a .32 probability of getting the correct result. These numbers (probability of getting the observed results) are based on the principle that researchers are just as likely to submit a non-significant result as a significant result (obviously false). Not totally illogical: when you have moderate n and power and you reject H0 it tells you something, but having moderate n and power and failing to reject H0 tells you very little.

Causal arrow from state of the world to the statistical state of the world (state of nature), another causal arrow from causal parameter to sample, inferential arrow running in opposite direction for both. We want to infer parameter from the statistic and given our inference as to the parameter to the substantive causal theory. Attenuating power means your ability to differentiate between a true theory and a crud factor theory is attenuated.

Can only reduce type I AND type II errors simultaneously by increasing the degrees of freedom (increasing n), improve logical structure of design, increase sensitivity of design, reliability of measures. Can change alpha levels and power to suit our needs/reduce the error which we believe is more pernicious.

7- Pilot studies. Reasons to do a pilot study: see if an effect exists. If it looks like an effect exists, you choose a sample size to help you get past the alpha. If an effect doesn’t find an effect, then it’s unlikely to be published. This means there is a filter for negative pilot studies, undermining the verisimilitude of the literature as people won’t know about these non-significant results. If an effect is found when the theory is false and the crud factor is poor you will then up the n for future studies; if an effect is found when the theory is false and the crud factor is stronger you won’t have that corresponding increase in n. You will have an increase in n that is negatively correlated with crud factor size. Sieving effect in literature as areas with lower crud factor bump their degrees of freedom but those with a higher one don’t. This will result in different sieving effects between different areas as for some areas it is much easier to increase n.

8- selective bias in submitting reports refuting null.

9- selective bias by referee and editors.

10- detached (logic detachment: once you’ve deductively proved a → b, you don’t have to keep on proving it later in the paper; you can detach it. Only works for deductive, not inductive) validation claim for psychometric devices. In inductive logic, there is a probabilistic element to the relationship. Can’t therefore detach it. In papers, researchers will often argue the measure they are using is the most valid based on previous literature. But won’t then reference how valid or the reliability established by the previous literature. Will just assume it’s totally valid.

First five will (generally) make good theories look poor, last five will (generally) make poor theories look good.

References

Yonce, J. L., 2016. Philosophical Psychology Seminar (1989) Videos & Audio, [online] (Last updated 05/25/2016) Available at: http://meehl.umn.edu/video [Accessed on: 06/06/2016]