In 1968, researchers created utopia. Sealed away from the outside world by impenetrable walls and perfectly climate controlled to their preferences, the settlers of the new universe lived in a world of abundance. Food and drink were available on demand, disease known only by its absence, housing of the best so freely provided that generations of growth couldn’t hope to fill it. 

Freed from care and want, the lucky inhabitants went forth and multiplied. At first, the population grew exponentially. Why wouldn’t they? The basic urge to reproduce existed, and in paradise they had no need to care about resources. After a time, though, something odd began to happen. As the world became more crowded — although never full — population growth began to slow. Male members of society who were unable to win social status began to withdraw from society, sitting inactive in large groups, marked only by brief outbursts of violence towards members of their own who disrupted their calm. 

Whatever you think about human intelligence, it probably does exist

The females began to reject the idea of raising their offspring, who in turn learned to behave in a society which had broken down. The delicate bonds of social behaviour broken, these children would never learn courtship or maternality. Young males would ignore females of their age, obsessing instead over their own appearance. With society in collapse, birth rates plummeted. By the time the end was in sight over half of the youngest generations were marked by these patterns of behaviour. They would have no offspring of their own; the colony would die

This is not a cautionary tale. It really happened. Admittedly, it happened to mice, and the lessons we can draw from it are therefore naturally limited. Whatever you think about human intelligence, it probably does exist. If the human population rose, it’s not like we’d fall for an obvious trap like clustering together in densely packed cities, obsessing over our appearance and having no children. Comforted by that thought, I put the article down. And then I booted up a chat with an AI, and realised humanity is doomed.

Researchers into the safety of humanlike artificial intelligence envision a wide and interesting range of ways it might kill you. Perhaps it will fail the critical test of outer alignment: told to maximise production of paperclips, an AI will do just that. What starts with “making the factory more efficient”, without appropriate guardrails, can easily escalate to “seizing the world’s metal supplies, and eliminating these irritating squishy humans that try to stop me from making more paperclips”. Alternatively, it could go wrong in even more subtle ways. While we try to train it to do one thing (make paperclips), it may learn to do something else entirely (bend pieces of metal) and then, as an intelligent, optimising agent, lie about it until it’s sure we can’t stop it. 

There are at least as many AI apocalypse scenarios as there are people thinking about AI safety, and many more they haven’t thought up yet. A fundamental problem with creating a new sort of intelligence is that humans run on totally different hardware and software. There is nothing that says our patterns of thinking will carry over to a totally alien mind, and certainly nothing that says it will work in our interest. 

But this is getting ahead of ourselves; there are plenty of risks from artificial intelligences far below human level. Just the other day a Google engineer was persuaded by a chatbot spitting out basic text that it was sentient, acted accordingly, and was suspended for his actions. The machine passed the Turing test — persuading a human they were talking to a humanlike intelligence — despite the engineer knowing perfectly well the results were generated by linear algebra. It’s not that hard to imagine a scenario in which a machine learning model trained on the internet — and the reams of rationalist text written on AI safety — learns to spit out responses which drive people to surprisingly extreme forms of behaviour.

People are surprisingly bad at finding ‘purpose’ outside of religion

With that said, what persuaded me humanity is doomed wasn’t the intelligence of the chatbot, or the idea that it might go rogue. It was something far simpler: what if it all goes horribly right?

Imagine, briefly, a world where we succeed in making human-level AI, which in turn produces superhuman intelligence. In a very short period, human creative and intellectual endeavour becomes obsolete; whatever you thought to create, the machine can do better. Whatever scientific puzzle you set out to solve will be solved by the machine. Whatever job you held is soon performed by the machine. 

Liberated from manual labour, intellectual effort and all the other concerns of the world, humans would finally live in the universe of the rats: pampered and cared for pets, with no need to exert themselves. If the modern world has shown us anything, it’s that people are surprisingly bad at finding “purpose” outside of religion. Without that purpose we are prone to falling into the sort of demographic spiral which characterised the death of the mouse colony.

This, I appreciate, is a very high level concern. Most people are not artists, writers or intellectuals. They are outcompeted in these domains by their fellow humans, rather than by machines, and get by just fine consuming their output and living their own lives. Perhaps the adjustment of the intelligent will simply be a brief period of humbling followed by sinking with relief into the life of the normal human. But remember also, without work or reason to engage with the outside world, those humans will be increasingly tempted to sink into the delights of endlessly personalised content generated on demand by machines: whatever book, film, game, pornographic fantasy they could possibly wish for, in endless abundance. Taking time away from this to go through the tedious business of engaging with other people (rather than their idealised artificial counterparts) and to raise a family, could prove a niche pursuit. 

It’s not as if we haven’t already seen enough evidence that we’re capable of creating ideas and distractions so consuming as to sublimate the basic biological urge to reproduce into other, more immediately rewarding, outcomes. With the arrival of the machine gods, superabundance could find us finally reduced to the level of the mice.