TiB 179: Big AI; contagious entrepreneurship; frontiers and history; and more...

Welcome new readers! Thoughts in Between is a newsletter (mainly) about how technology is changing politics, culture and society and how we might do it better.

It goes out every Tuesday to thousands of entrepreneurs, investors, policy makers, politicians and others. It’s free.

Forwarded this email? Subscribe here. Enjoy this email? Forward it to a friend.

Big models and the future of AI

We’ve talked a lot about the impact on AI of models like GPT-3 that are enormous in scale and power (and, it seems, relatively generalisable; see this new paper on using language models to solve maths problems). In a further sign that this is becoming a - perhaps the - major focus of AI research, more or less the entire Stanford AI department has released a book-length paper that lays out the opportunities and risks that such models, which they term “foundation models”, represent. There’s a good non-technical write up here.

One of the core ideas is that foundation models tend to be so expensive to train that they can’t easily be replicated (a point we’ve discussed before - see TiB 142), which means that much future AI work will build directly on top of them - and inherit any weaknesses and biases they have. More academic attention to these models is therefore welcome - but as Michael Nielsen points out (worth reading the replies too), it’s precisely because foundation models require so much computational firepower that universities, which tend to be richer in talent than capital, may struggle to have an impact.

One part of the solution, as we discussed in TiB 159, is for rich countries to fund their own National Research Clouds. This seems to me one of the first tangible frontiers in AI nationalism: does your public sector have the capability to interrogate and critique the models on which an increasingly broad swathe of critical applications will be run? As I’ve said before, the capital demands of even the largest machine learning models are small relative to government budgets. It’s a good time to get ahead of what is likely to be an exponential curve.

Bonus 1: It starts slowly, but this demo of OpenAI's Codex is pretty amazing

Bonus 2: AI safety-themed short story that illustrates the potential pitfalls of model inheritance

You catch entrepreneurship from someone like you

One of the big ideas I took away from my podcast episode with Anton Howes earlier this year is that innovation is not instinctive: people who become innovators have usually been exposed previously to what Anton calls “the improving mentality”. Matt Clancy (see TiB 117 and 160 for previous coverage) cites Anton’s work in a fascinating new post* that reviews the evidence and concludes that not only is entrepreneurship contagious, but you’re more likely to “catch” it from someone who is like you.

Gender is one big dimension. Clancy cites evidence from this paper that children who are adopted by entrepreneurs are more likely to become entrepreneurs themselves - but that the effect is greatly mediated by gender. An entrepreneurial father has twice the impact on the career choice of his sons as an entrepreneurial mother, and almost no impact on those of his daughters (whereas an entrepreneurial mother has a strong effect on daughters). There are similar effects among women who work in startups led by female founders.

Clancy sums up the evidence for contagion as follows:

Entrepreneurs are often found in social clusters (workplaces, neighborhoods)

Quasi-random exposure to entrepreneurs increases the probability of becoming an entrepreneur

Entrepreneurial influence seems stronger when entrepreneurial peers occupy a more similar social position

The effect of exposure to entrepreneurs is much weaker for the people most likely to already be considering a career in entrepreneurship

This strongly suggests, as we discussed in TiB 149, that if we want more entrepreneurs, we need more role models, both in real life and in fiction - and, particularly, if we want more founders from underrepresented groups, we need to be telling the stories of those who succeed loudly and often.

*Thanks Michael for the link

Being at the frontier changes behaviour, maybe forever

An infrequent but favourite TiB topic is the surprising persistence of historical experience, sometimes over decades or centuries (see, e.g., TiB 152 on the Reformation or TiB 99 on the Cultural Revolution). Last week I came across this paper which looks at the long-run impact of the “frontier experience” on American responses to COVID. The authors construct an index of “Total Frontier Experience” that measures how long a given US county was at the frontier during the period of American expansion. They find that greater historical frontier experiences predicts less social distancing and mask use and fewer lockdowns.

The paper argues that frontier experience creates a long-run culture of “rugged individualism” that makes individuals more suspicious of the state and less inclined to collective action. This builds on earlier work by the same authors that suggests frontier experience is a distinct phenomenon from many of the other geo-demographic variables, such as north/south and urban/rural divides, often used to explain political behaviour. They argue this isn’t a mere selection effect; there is causal impact too. This is not a new theory - see Frederick Turner’s 1893(!) essay on the same topic - but it’s interesting to see it quantified.

There are few physical frontiers left on earth, but it’s tempting to speculate about whether exposure to frontier-like parts of the internet might have a similar impact. I suspect so. Today most of the internet is relatively corporate and sanitised, but it wasn’t always so (and arguably crypto represents a frontier-like space today). Perhaps a hundred years from now historians will seek explanations for contemporary behaviour in the Usenet archives of the 1990s.

Quick links

Afghanistan round-up. Lots of responses to last week's segments, so a collection of the best things I read this week. The best withdrawal-skeptic piece (Thanks Tom). The best withdrawal-sympathetic piece. More from Tanner Greer. Superb Afghan history podcast. How much is a (US soldier's) life worth?
Easy in retrospect. Excellent Q&A thread on the topic "What is a technology prediction in which you had strong conviction that turned out to be wrong?", including answers from some prominent people.
Long Zuck. The bull case for Mark Zuckerberg (I think by Gwern)
Make it quick. Fantastic long list of short story recommendations.
The internet is good. What does fast internet do to employment in Africa?

Help me grow

Thanks for reading Thoughts in Between. Please do share it with a friend.

I'm always happy to hear from readers - feel free to hit reply with comments, questions or recommendations.

Until next week,

Matt Clifford

PS: Lots of newsletters get stuck in Gmail’s Promotions tab. If you find it in there, please help train the algorithm by dragging it to Primary. It makes a big difference.