This week: What the rise and rise of large scale models means for the future of AI; entrepreneurship is contagious, but you don’t catch it from everyone equally; the long-term impact of living at the frontier; and more…

Welcome new readers! Thoughts in Between is a newsletter (mainly) about how technology is changing politics, culture and society and how we might do it better.

Forwarded this email? Subscribe here. Enjoy this email? Forward it to a friend.

We’ve talked a lot about the impact on AI of models like GPT-3 that are enormous in scale and power (and, it seems, relatively generalisable; see this new paper on using language models to solve maths problems). In a further sign that this is becoming a - perhaps the - major focus of AI research, more or less the entire Stanford AI department has released a book-length paper that lays out the opportunities and risks that such models, which they term “foundation models”, represent. There’s a good non-technical write up here.

One of the core ideas is that foundation models tend to be so expensive to train that they can’t easily be replicated (a point we’ve discussed before - see TiB 142), which means that much future AI work will build directly on top of them - and inherit any weaknesses and biases they have. More academic attention to these models is therefore welcome - but as Michael Nielsen points out (worth reading the replies too), it’s precisely because foundation models require so much computational firepower that universities, which tend to be richer in talent than capital, may struggle to have an impact.

One part of the solution, as we discussed in TiB 159, is for rich countries to fund their own National Research Clouds. This seems to me one of the first tangible frontiers in AI nationalism: does your public sector have the capability to interrogate and critique the models on which an increasingly broad swathe of critical applications will be run? As I’ve said before, the capital demands of even the largest machine learning models are small relative to government budgets. It’s a good time to get ahead of what is likely to be an exponential curve.

Bonus 1: It starts slowly, but this demo of OpenAI’s Codex is pretty amazing

Bonus 2: AI safety-themed short story that illustrates the potential pitfalls of model inheritance

One of the big ideas I took away from my podcast episode with Anton Howes earlier this year is that innovation is not instinctive: people who become innovators have usually been exposed previously to what Anton calls “the improving mentality”. Matt Clancy (see TiB 117 and 160 for previous coverage) cites Anton’s work in a fascinating new post* that reviews the evidence and concludes that not only is entrepreneurship contagious, but you’re more likely to “catch” it from someone who is like you.

Gender is one big dimension. Clancy cites evidence from this paper that children who are adopted by entrepreneurs are more likely to become entrepreneurs themselves - but that the effect is greatly mediated by gender. An entrepreneurial father has twice the impact on the career choice of his sons as an entrepreneurial mother, and almost no impact on those of his daughters (whereas an entrepreneurial mother has a strong effect on daughters). There are similar effects among women who work in startups led by female founders.

Clancy sums up the evidence for contagion as follows:

This strongly suggests, as we discussed in TiB 149, that if we want more entrepreneurs, we need more role models, both in real life and in fiction - and, particularly, if we want more founders from underrepresented groups, we need to be telling the stories of those who succeed loudly and often.

An infrequent but favourite TiB topic is the surprising persistence of historical experience, sometimes over decades or centuries (see, e.g., TiB 152 on the Reformation or TiB 99 on the Cultural Revolution). Last week I came across this paper which looks at the long-run impact of the “frontier experience” on American responses to COVID. The authors construct an index of “Total Frontier Experience” that measures how long a given US county was at the frontier during the period of American expansion. They find that greater historical frontier experiences predicts less social distancing and mask use and fewer lockdowns.

The paper argues that frontier experience creates a long-run culture of “rugged individualism” that makes individuals more suspicious of the state and less inclined to collective action. This builds on earlier work by the same authors that suggests frontier experience is a distinct phenomenon from many of the other geo-demographic variables, such as north/south and urban/rural divides, often used to explain political behaviour. They argue this isn’t a mere selection effect; there is causal impact too. This is not a new theory - see Frederick Turner’s 1893(!) essay on the same topic - but it’s interesting to see it quantified.

There are few physical frontiers left on earth, but it’s tempting to speculate about whether exposure to frontier-like parts of the internet might have a similar impact. I suspect so. Today most of the internet is relatively corporate and sanitised, but it wasn’t always so (and arguably crypto represents a frontier-like space today). Perhaps a hundred years from now historians will seek explanations for contemporary behaviour in the Usenet archives of the 1990s.

Thanks for reading Thoughts in Between. Please do share it with a friend.

I’m always happy to hear from readers - feel free to hit reply with comments, questions or recommendations.

Until next week,

Matt Clifford