A Sample Size of One

From a weekly email to Praxis participants and alumni.

How many data points do you need to prove something?

Sometimes just one.

This is important to understand, especially if you’re ever put in the position of defending deviation from the status quo. Some of the most important arguments don’t gain anything from big studies or large samplesizes.

The other day, Derek Magill was in a Twitter conversation with someone who made the common claim that a person can’t get a job that says “degree required” unless they have a degree. Derek disagreed. His interlocutor demanded to see some studies that would verify Derek’s claim.

It’d be easy to feel pressure to produce a study or be thought a fraud. But you don’t need a study. A single example is sufficient to prove Derek’s point and disprove the other.

The guy made a binary claim; “You can’t get a degree-required job without a degree”. That’s not a probabilistic statement. It’s either true or false. He claimed getting the job is impossible. The quickest way to show something is possible is to show that it already exists. A single instance of a non-degreed person getting a job that said “degree required” refutes the claim. If we can find one, the claim is false. And that’s easy to do.

Someone might respond with, “He was just being loose with language. He didn’t actually mean it was impossible, just rare and difficult.” Good. Now we’re getting somewhere. “It’s rare and difficult to do X” is a lot different than “You can’t do X”. Once it is conceded that it’s possible (which is undeniable if it already exists), we can have a really interesting conversation about just how hard, and how rare it is, and why, and for whom? That’s when the “Go get a degree” narrative really starts to reveal itself for its mindless blindness.

If one person did it, how? Does it mean employers actually mean something else when they say degree required? Could what that person did be emulated? How? Are rare and difficult things usually better to pursue than common easy things? Is getting that same job with a degree actually easy?

These questions are incredibly valuable, not just for making a point in a debate, but asking them alone provides useful career information. All of that critical thinking is shut down if we accept “It’s not possible.”

We live in a culture that gives ceaseless lip-service to “Science”. Everyone claims to want data, studies, and research. That’s all well and good where it makes sense, but many claims are binary claims about what is and is not possible. Those don’t need more than a single sample. Once we move conversations away from possible/impossible, we start to get to probabilistic thinking, which is usually more valuable for navigating real life.

Look for opportunities to prove the possible by showing it already exists, and thus move the conversation to questions of how something can be done, rather than whether it can. This is why success stories are useful. Not because they give a blueprint we can all follow, but because even a single instance proves something possible, and moves us into examining how to increase the probability in our own lives.

%d bloggers like this: