The third lecture of this module introduced the framework of Null Hypothesis Significance Testing and, on an example of arm span in elite climbers vs non-climbers, illustrated how it can be used to make an informed guess about what kind of world it is that we live in. In the lecture, we likened the competing hypotheses (the alternative hypothesis \(H_1\) and the null hypothesis \(H_0\)) to parallel universes or alternative realities. In the universe of \(H_0\), there is no effect, no relationship, no difference of interest, while in the universe \(H_1\), these effects exist.

Let’s stretch this fanciful theme further and imagine a scenario where we can do some real-life NHST…

So, you’re in a pub.

A year into the COVID-19 pandemic¹, this is as fantastic a scenario as we can fathom!

So, you’re in a pub with your friends, there’s no need for face masks, people are clicking their glasses, and talking to each other at a high volume and short distance.

You’re sipping on your beverage of choice when, suddenly, you find yourself transfixed at the scene playing out in the corner of the room. Two middle-aged guys, their polo shirts heaving to contain their respectable beer bellies, are playing a game of darts.

Honestly, that’s the kind of pub you go to after a year of confinement?!

In any case, here you are. In an old pub with a sticky carpet, watching dudes throw darts. You have no idea why you find the sight so mesmerising, you’ve never really had any interest in darts; frankly, you don’t even know what the goal of the game is! But there’s just something about it…

After a while, you think to yourself that these chaps must either not be very good or they’ve had a few more than just a few pints, because they seem to be missing the centre rather a lot. But then you think, maybe they’re not trying to hit the centre! Maybe the optimal strategy for the game requires that you aim for something else than the bullseye.

You are intrigued so you decide to investigate further. For a brief moment, you consider walking up to them and asking about the rules of the game but quickly dismiss the idea to actually talk to people. Oh no, you’re in this on your own and you ain’t need no men!

But you are not afraid. You put on your quantitative researcher hat and boldly go where seemingly only middle-aged men have gone before. (Other people have squeezed past on their way to the loo but that’s beside the point! Stay focused!)

Formulate a hypothesis

Conceptual

As a researcher, you know that the first thing to do is to come up with a hypothesis. After pondering your options for a moment, you settle on:

“The optimal strategy for this game – whatever the game is – is not to aim at the centre of the dart board.”

You know this is a good hypothesis: It’s specific, simple, and, in principle, testable. However, it doesn’t really comment on where a player should be aiming. And of course it doesn’t! Without a prior knowledge of the game, there’s really no point guessing. It does occur to you to check out the rules of the game but where’s the fun in that?!

So yes, it’s a good hypothesis but it’s not obvious what kind of data/observation would support it best. You quickly realise that a way out of this problem is to find yourself a null hypothesis to test.

Operational

Though you have your hypotheses figured out, you know that’s not the end of it just yet. In order to be able to gather data to test your hypothesis, you first need to operationalise your variables.

Task 3

You take a few moments to think about how you could operationalise “aiming for the centre” in measurable terms.

Task 4

Then, you restate your conceptual hypotheses as operational ones.

Statistical

You are close and you can feel it. Only one step till you have hypotheses fit for some stats love!

At last, you are ready to start testing your hypothesis!

Before you do, though, you ponder the two parallel worlds of your two hypotheses, focusing on what a dartboard full of darts would look like if either of the hypotheses was true:

Inspecting this mental image, you realise that you’ll need statistics to figure out whether or not the guys are aiming for the bullseye. After all, there is always some uncertainty and so even if they are trying to hit the middle, maybe in the short window that you’ll be observing them, they’ll go through an unlucky streak and the average distance from the middle in the sample of dart throws will not be representative of its population!

Setting an \(\alpha\)

You’ve now firmly resolved to use a statistical test of the null hypothesis and so you know you need to settle on a significance level (\(\alpha\)) by which you are going to judge your results. If you find that the probability of your data assuming that you live in the world of \(H_0\) is smaller than your chosen \(\alpha\) level, you’ll reject the null. If not, you will retain it.

Collecting data

You’re now ready to start taking some sweet, sweet measurements. You find a cosy spot not far from the dart action with a vantage point that lets you see the entire dart board, unobstructed by the robust physiques of the athletes. Devising a way of accurately measuring the distance of each dart from the bullseye takes some ingenuity but, armed with a thick layer of plot armour, you are unhindered by this task.

After each player’s turn, you snap a sneaky pic with your phone and, after 10 rounds, you think you have enough data. Excited, you pack your things and, merely as an after-though, call out to your friends on your way out: “Gotta run, have data to analyse. Byee!” Everyone accepts this without as much as the slightest hint of bafflement. That’s just what you be like…

When you get home, you download the picture on your computer and overlay them one on top of another. This is what you see:

“Good thing I got that optical scanning software the other day,” you think to yourself. And a good thing indeed! Here’s the data for the vertical distance, just as you intended:

darts <- tibble::tibble(
  round = rep(1:10, each = 6),
  player = rep(rep(c("Colin", "Stu"), each = 3), 10),
  dart = rep(1:3, 20),
  distance = c(
    3.5, 4.5, 7.2, 3, 5.5, 8, 1, 0.2, 7.5, 9, 9.2, 9.5, 0.5, 4.8, 5, -0.2,
    -2.2, 8.2, -2, 10, 6.2, 6.5, 3.8, -3.8, 4.5, 5.8, 4, 6.8, -4.2, 3, -1,
    2.5, 7.2, 5.8, 7.2, -0.8, 0, -1.8, -0.8, 0.5, 9.2, 4.5, 9, 6.2, -1.2,
    8.5, 5.2, 0.8, 5.2, 5.2, 3.2, 9.8, 0.5, 6.5, -1.2, 4, 3.2, 3, 6.5, 3.2)
)

You have to admit that, at face value, it seems like you might be onto something here but you know better than to rely on eye-balling. After all, you are the world’s foremost dartologist and accuracy is the name of the game! Well, the name of the game is darts but accuracy is its important component, anyway… Dart Vader is your name and precision is your game! Your favourite musketeer is Dartagnan. Since you were small, you’ve always wanted to be a darta scientist.

Once you’ve stopped simultaneously cackling with self-satisfaction and shuddering with self-loathing at those awful jokes, you are ready to analyse your… observations.

First, you need your test statistic.

Of course, what else!?

It must be mean distance because your \(H_0\) talks about mean population distance from the bullseye. You’re moving swiftly now.

mean(darts$distance)

[1] 3.915

Hmm… it does look like the punters were aiming above the middle of the dart board after all…

Sampling distribution of the test statistic

However, you know very well that in order to assess the probability of your result or an even more extreme result, you need to know what the sampling distribution that your statistic comes from looks like.

One parameter down; one more to go.

Unfortunately, you know that the standard deviation of the sampling is tricky. You simply don’t know what it is. But you do know its relationship with population standard deviation in human dart-throwing accuracy.

After a quick internet search, you come across a paper reporting that the variance in people’s accuracy in a game of darts is 210.937 cm². The study was conducted on a huge sample and so you take this as a trustworthy estimate of the population variance, \(\sigma^2\).

For a moment, you wonder if anyone has ever researched the rules of 501 darts but you quickly snap out of your daydream. There’s no time for escapist fancy!

Get the p-value

Your heart is racing now. You are so tantalisingly close to the Truth^TM! Only one thing left to do: calculate the probability of obtaining a test statistic of 3.915 or even a more extreme result, if you actually live in the wold of \(H_0\)!

n <- 60
variance <- 210.9375
se <- sqrt(variance/n)
test_stat <- mean(darts$distance)
# times two because we had a 2-tailed hypothesis
# it could have turned out that people aim BELOW the centre
p_value <- pnorm(test_stat, 0, se, lower.tail = FALSE) * 2
p_value %>% round(3)

[1] 0.037

Decision

A blissful smile is beaming in your face as you relax back into your chair, exhausted but happy. Now you know… You’ve been through it all, formulated your hypotheses, gathered your data, calculated your stats, found your results.

Your hunch was correct, they were not aiming for the bullseye!

Well, it may have been correct. You are well-aware that no single piece of research can ever answer a research question definitively. Of course you know that! But now it’s not the time for raining on your own parade.

Now it’s the time to watch some darts.

Well deserved!