Survivorship Bias in Science

Let’s imagine for a moment that uncertain job prospects and too much caffeine pushes me over the edge and I gather up every monkey in the world and shut them in a room with a bunch of computers. Sometime later, I return to a lot of flung poo and, among all the random strings of letters typed by the unfortunate (and now cannibalistic) monkeys, I discover that one capuchin has typed the sentence: “HELLO KAT”.

This is a version of the Infinite Monkey Theorem, which basically states that a monkey hammering on a keyboard for an infinite amount of time will eventually type out the complete works of William Shakespeare. It’s all about probabilities.

Give a million monkeys ten years, and the probability that one of them will type ‘HELLO KAT’ entirely by chance is 1 in 2. The same as guessing the outcome of a coin toss*. Throw in all the other 9 character sentences that can be made from the letters on a keyboard, and the likelihood of one of the monkeys NOT typing something meaningful by chance is practically zero.

But what happens if I now take that one, single monkey, and I publish a paper saying that I have found the world’s first literate capuchin? Disregarding all the random sentences typed by all the other monkeys, I proclaim that there was only a 1 in 1.8x10⁷ chance that my monkey could have randomly typed ‘HELLO KAT’. Those odds are so slim that surely this particular monkey must have intentionally hit those particular keys?

This is an example of Survivorship Bias, in which only focussing on the successes while ignoring the failures can lead you to make incorrect conclusions.

The same thing happens when it comes to careers. I can’t count the number of times I’ve listened to a leading scientist explain how they made it to the top using their formula of:

(Being smart) x (Choosing the right field) ^{(Hard Work)} + (Networking) = Success

The thing is, this doesn’t take into account all the people who are plugging the exact same numbers into the exact same formula and coming up with entirely different results. When something is heavily dependent on chance and luck, you can’t make conclusions based only on the survivors—you need to check the graveyard too. The road to permanent scientific positions is littered with the tombstones of postdocs who have fallen along the way and I can’t believe I didn’t notice them until the point at which I was down on my hands and knees, scrabbling around in the dirt.

The stupid thing is that others did try to warn me when I started out, but I didn’t want to listen. Looking back, I wish I hadn’t been so quick to disregard the experiences of older scientists finding themselves in the same position I am now in. It was all too easy to presume that they’d done something wrong; that they hadn’t tried hard enough or they simply weren’t very good at science. Understanding the role that luck plays in a scientific career wouldn’t have stopped me from becoming a scientist, but it might have made me less of a dick.

Now that I am picking myself back up and heading off for pastures new, I am experiencing yet another example of Survivorship Bias. People who know that I write science fiction novels in my spare time keep sending me articles about self-publishing success stories. Why are you trying to find a traditional publisher when E. L. James self-published 50 Shades of Grey and look at her now! What they don’t realise is that, for every wannabe author who becomes famous from self-publishing, there are hundreds of thousands who fail miserably.

When it is a scientist who tries to tell me how to be successful as a writer, I ask them if they would self-publish a scientific paper that had been rejected by a few dozen journals. It’s not the same, they say, science isn’t subjective like writing. You either do it right or wrong. Then I sit back and wait for them to do everything right and find that it still isn’t quite enough.


*Let’s just say there are 50 keys on my keyboard. So the probability of that monkey hitting the first ‘H’ is 1/50. The probability that the ‘H’ will be followed with ‘E’ is (1/50) X (1/50) and so on. I worked it out, and the overall probability is 1 in 1.9x10¹⁵, which in the grand scheme of things is extremely close to zero. But let’s say that a monkey can type at a speed of 200 characters a minute and it manages to type around 100 million strings of 9 characters over one year. If we work out the probability that the monkey will type ‘HELLO KAT’ at some point over the year, it works out at 1 in 1.8x10⁷ – still very, very unlikely. But what if we give a million monkeys ten years? Now the probability that one will type ‘HELLO KAT’ entirely by chance is up to 1 in 2.3. Entirely doable.

Scientific Stockholm Syndrome

Science embodied as a person would be a rubbish date. You’d be so dazzled by Science's awesome that you’d not only end up paying for dinner, but you’d find yourself promising them your undying loyalty. Then, before you know it, you're feeling guilty for not spending all of your time with Science and Ohhh that Kool-Aid looks really tasty*.

Misplaced loyalty to a career undoubtedly isn’t unique to scientists, but it does sometimes seem to be worryingly common in my profession. How many non-science people do you know who’d continue to work when they’re no longer being paid? Not many, yet it is all too common for end-of-PhD students and sometimes even postdocs who need to do that last experiment for the paper. And don’t get me started on the long hours and weekend work that seem to be the norm in most research laboratories.

We tell ourselves that we’re doing it for our own benefit—because we love what we do and want to give ourselves an edge in a very competitive environment. But lab heads and universities happily take advantage of this devotion to our careers and there comes a point where they are benefiting far more than the temporary scientisit. Like the vampires inexplicably romanticised by young adult fiction, of course employers aren't going to say no to willing victims eager to be sucked dry of their intellectual creativity*. But maybe they should.

Sure, less PhDs would get funded because it would cost a hell of a lot to keep paying every student until the moment they submit their thesis. Some papers wouldn’t get finished if universities couldn’t find extra money to keep on postdocs at the end of a grant. And the scientists would be the first ones to complain and defend their right to be exploited. 

With four months left in the lab, I’m not sure what scares me more—coming to the end of my contract and not having a new career to move on to, or finding a new position with time to spare and having to leave my project unfinished. Come December 31st, neither my current boss nor my research career is going to be buying my New Year’s Eve beers, so why do I feel like I would be letting both down if I don’t stick it out until the very last chime of Big Ben?

I’m sure that there are few lab heads out there who, if offered the professorship of their dreams, would turn it down out of loyalty to their postdocs and PhD students. So why do some temporary staff like me feel so guilty at the prospect of jeopardising a lab’s future grants and papers by making a selfish decision that would be in our best interest? It's like I have to keep reminding myself that my contract with the university is for a three year postdoc and not my soul. 

My relationship with Science has reached the point where I’m sat comfortably on the sofa in jogging bottoms, with barbeque sauce smeared around my face. Science is out there being all sciencey and cool, and here I am, clinging on to the memories of all our happy times together*. I keep telling myself that loyalty is only worth as much as the rewards it yields, that I could be so much happier in a new relationship, but it is so hard to not feel guilty about leaving. 

*I blame impending unemployment for all this melodrama. If any potential employers are reading this, I really am entirely sane. Please give me a job. 

Swan (Dive) Song

On leaving laboratory science and why it’s going to be awesome (but first a rant)

A few years ago, I went on this residential course for postdocs whose years’ experience was greater than their output of Nature/Science/Cell papers. We made paper bridges for hamsters and drew our innermost feelings on giant shields for a reason I can’t quite fathom. Later, when we’d all got to know each other through the medium of unrelenting pessimism and beer, we went around the room and stuck post-it note ideas for alternative careers on everyone else’s shields.

My alternative career suggestions? Science fiction novelist, science journalist, or primary school art teacher.

Today, with five months left of my contract and the decision made that it is time to move on to a new career, I find myself looking back on those suggestions and thinking how absolutely, ridiculously naïve they were. We all know that there are more postdocs than there are permanent research jobs, and that most of us will have to pack up our ‘transferrable skills’ in a little knotted handkerchief and venture out into the big wide world. But this enthusiastic ‘You can do anything you want with a PhD!’-mentality doesn’t help anyone. It’s right up there with patting a five-year-old on the head and telling her that of course she can grow up to be an astronaut if she Just Dreams Big Enough.

It’s the science journalist suggestion that bugs me the most because I hear this from students. All. The. Time.

“What are you going to do after your PhD?”

“Oh, you know, I’ll probably just go into science journalism.”

No experience, no training, no particular interest in science communication. But, for some reason, there seems to be a prevailing attitude among a worrying proportion of scientists that having Dr in front of our name somehow qualifies us to hop out of the lab and easily pinch someone else’s hard-won career. It might be our plan B, but it will do. And I worry that it makes the rest of us look like dicks by association.

This is my big problem with PhD training.

Universities are churning out all these slightly entitled 25-year-olds with no idea of how the real world works. Students are paying more and more for their undergraduate courses, and the teaching is becoming increasingly structured with teaching fellows taking the lectures instead of researchers. To me, it feels a bit like we’re spoon-feeding people who should be able to learn independently by this point in their life. Then some start a PhD and a small minority never pause to consider that maybe they should stop thinking of themselves as a student and start acting like an adult.

No one really fails a PhD—I've seen far too many poor students saunter through their vivas with no problems after spending 3 or 4 years treating their PhD like a hobby rather than a professional job. And this devalues the PhD for everyone else. With so many of us wanting to use our skills in other careers, I'd kind of like it to represent the pinnacle of scientific education and not become an esoteric qualification unworthy of respect.

It's hard enough for the best and the brightest scientists to secure fellowships or lectureships, so why are we letting people continue wasting their time and money on a pointless PhD that won't help them become a scientist and hasn't taught them anything they couldn't have learnt better in the workplace?

There is no grading for a PhD but maybe there should be. Or maybe the standards just need to be higher and more consistent. Would I have passed if this was the case, or even got a project in the first place? I like to think I'd have risen to the challenge but we will never know. 

But what does this bitter little rant have to do with me leaving science?

At the end of the day, it’s not the poor job prospects and uncertainty that got me (although that didn't help). No, it’s being part of a system that spews out more and more PhDs despite knowing that there just aren’t enough jobs, then tells us that we can do anything we want with our little qualification and starts again with the next batch of naive wannabe scientists. Throw enough people at Science and a few will stick. Everyone else? Transferable skills!

It lets everyone down—the students who don’t have a clue and the postdocs who become demoralised at the thankless task of mothering adults who don't know why they’re doing a PhD in the first place. Science and scientists are complicit in a system that screws over postdocs in more than one way and it's shit.

So I'm going to take all my 'transferable skills' and find a job that makes me happy instead of frustrated; challenged instead of used; that respects me for the things I am good at instead of treating me like a disposable scientific thinker, broken equipment tinkerer and exhausted nursemaid.

I’ve always felt like leaving science and starting something new would feel like I’d failed. And I guess this is part of the reason why I’m jumping before I am pushed. But, now that I’ve told my boss that this postdoc is it for me, I feel inexplicably happy. I have no idea what I will be doing after Christmas, and it’s going to be awesome finding out.