In 2019 UK Athletics introduced a new non-binary category for road running, following the lead of international races like the New York City Marathon. Although the International Olympic Committee (IOC) recommends that sports bodies pursue an evidence-based approach, UK Athletics made this change despite a total absence of research on the performance of non-binary athletes. In addition, there has been a lack of any meaningful consultation: UK Athletics consulted with only one running club in developing its policy, the LGBTQ running club London Frontrunners.

This is a familiar pattern. In 2015 the IOC introduced a policy that allowed males to compete in the female category so long as they reduced their testosterone to 10nmol/L (the normal range for females 0.5-2.4 nmol/L). This was done without consulting female athletes and despite there being no convincing scientific evidence that this was fair.

The same pattern can also be seen outside of sports. The theory has become ubiquitous in policy making that gender identity is a more important factor in people’s lives than sex. This has resulted in males being allowed into women’s prisons, girl’s schools, women’s hospital wards, women’s toilets and onto all-women shortlists. Yet there is very little empirical evidence to support the theory that gender identity rather than sex influences outcomes.

Indeed, there seems to be a reluctance to gather evidence that might support, or perhaps refute, this theory. The UK Office of National Statistics required a judicial review before it was willing to gather accurate data on sex in the England and Wales Census. Meanwhile in Scotland, the Chief Statistician recommends avoiding collecting data on sex except in exceptional circumstances.

Because the trans and non-binary populations are very small, standard social science data sets are not large enough to find statistically significant results about gender identity. To study these populations, it is usually necessary to create specially designed studies that target these populations, but that can lead to biases.

This makes the non-binary road-race category a very exciting source of data. With tens of thousands of runners competing in mass participation marathon races, road race data provides a rare opportunity to test the validity of gender-identity theory and to compare it with the alternative gender-critical theory that sex matters.

We found no evidence that sex differences in performance were lower for non-binary athletes

In a study published this week in BMJ Open Sports and Exercise Medicine, we used data from the New York Road Runners race database to test these two theories.

Because New York Road Runners only record whether a runner is male, female or non-binary we had to find an alternative way of finding the sex of non-binary runners. For most runners this is easy: many runners have run other races where they reveal their sex. For other runners what we could do instead was estimate the probability that they were male or female based on their first name, using a database of baby names and sex to calculate these probabilities. We validated this probability model in the case where the runner’s sex was known, and it gave excellent results.

First, we tested the gender-critical theory. It was supported by the data. We found evidence that sex makes a big difference to the performance of non-binary athletes at a high level of statistical significance.

Next, we tested the gender-identity theory and found it was not supported by the data. We found no evidence that sex differences in performance were lower for non-binary athletes than other athletes.

For some people, our results will be surprising. A recent Scientific American article claimed, “The inequity between male and female athletes is a result not of inherent biological differences between the sexes,” and the American Anthropology Association believes “anthropologists and others have long shown sex and gender to be historically and geographically contextual, deeply entangled, and dynamically mutable categories”.

These are extreme views, but our findings may also surprise those who hold the much more reasonable view that sex is real, but that gender identity has a significant role to play in social inequalities. Since it is generally accepted that social factors play a role in sports performance, it is interesting that we did not find any reduction in the sex gap amongst non-binary athletes.

Our finding has obvious implications for sports policy. There is a strong case for replacing the male category in running with an open category; this would mean that anyone who wants to keep their sex private can still compete. However, in the light of our evidence a separate non-binary category seems hard to justify.

If the sex gap is the same amongst non-binary athletes as amongst other athletes, then having only one non-binary category for males and females discriminates against females. As there is no robust evidence of any impact of being non-binary on race performance, the existence of a non-binary category also appears to discriminate based on trans status. One might argue that the purpose of the non-binary category is to celebrate non-binary performance, but why single out non-binary identities? Is it fair to award $5,000 to the winning non-binary athlete when there are no prizes for runners in Para categories?

However, our research was not motivated solely by sports. Whilst our study only tests gender identity theory in the case of mass participation running performance, it raises the question of whether gender identity theory is valid in other contexts. For example, based on gender identity theory, one may hypothesise that transwomen would devote more time to childcare than other males. It would be interesting to see if this was true. Similarly, one could study whether transwomen experience a gender pay gap. Currently we know very little about such questions and so are making policy blind.

The central point of our study was to demonstrate that gender identity theory can, and should, be subjected to scientific scrutiny just like any other theory. Policy should be based on evidence, not wishful thinking.