Statistical Inferences Test

The data is surprising if we assume there is effect; the null hypothesis is likely to be false.

The null hypothesis is likely to be false; the alternative hypothesis is true.

The data favors the the alternative hypothesis; the null hypothesis is likely to be false.

The data is surprising assuming the alternative hypothesis is true; there is a true effect.

Both intervals give you estimates for the most probable values of the true mean.

The confidence interval gives you the values you believe are most likely based only on the data, the credible interval gives you the most probable values given the data and your prior.

Both procedures give intervals which contain the true value 95% of the time, but credible intervals are more accurate.

The confidence interval contains the true value 95% of the time, the credible interval contains the 95% most plausible values.

Lower Type 1 error rates, lower Type II error rates, same accuracy of estimates.

Lower Type 1 error rates, lower Type II error rates, higher accuracy of estimates.

Same Type 1 error rates, lower Type II error rates, higher accuracy of estimates.

Same Type 1 error rates, lower Type II error rates, same accuracy of estimates.

This distribution is typical for when there is no true effect.

This distribution is typical for when there is a true effect.

This distribution suggests the presence of p-hacking.

This distribution indicates there is publication bias.

We can’t be sure there is publication bias; because of the Type 1 error rate we will eventually get a non-significant result.

It’s impossible that 91.5% of findings in psychology confirm the hypothesis; the type 1 error rate makes that unlikely in the long run.

Researchers have admitted to not submitting all their results for publication; it is impossible that all hypotheses a researcher examines are true.

It’s unlikely that the observed drop in the p-value distribution for p-values higher than 0.05 in the published literature occurs by chance; If all four studies examine a true effect the probability of only significant results is power*power*power*power.

A progressive research line; affirming the consequent

A falsifiable prediction; denying the consequent

A degenerative research line; denying the consequent

A falsifiable prediction; affirming the consequent

Because the researcher performed two analyses, the alpha level should be divided by two for each test to bring the overall error rate back to 5%.

Because the second hypothesis was not predicted, the overall error rate is inflated by an unknown amount.

The predicted finding is confirmatory, the unexpected finding is exploratory. The error rate for each finding is maintained at 5%.

The exploratory finding has increased the Type 1 error rate to 10%.

The prior probability of model A is 10 times larger than that of model B

Model A is 10 times more likely to be true than model B

The data favor model A 10 times more than model B, given the prior

We can have a strong personal belief in the absence of a true effect.

The likelihood of H0 being true, and the likelihood of H1 being true; it is not unlikely to have some significant effects, even if H0 is false

The likelihood of the data, assuming H0 is true; it is unlikely to get 4 significant results out of 6

How much more or less likely H0 is, compared to H1; it is likely to have some non-significant effects, even if H1 is true

If 4 significant studies out of 6 is likely to happen if H0 is true; it is unlikely to get 4 significant results out of 6, assuming H0 is false

Minutes; a-priori power analysis

Cohen's d; comparing the effect size across studies using different measures

Meters; calculating less biased effect size estimates

Eta-Squared; controlling Type 1 error rates