TiB 160: Nuclear weapons and AI; anti-portfolios in philanthropy; the second order benefits of science; and more...

Welcome new readers! Thoughts in Between is a newsletter (mainly) about how technology is changing politics, culture and society and how we might do it better.

Forwarded this email? Subscribe here. Enjoy this email? Forward it to a friend.

Oxford University’s Future of Humanity Institute (FHI) has an excellent new paper that looks what we can learn from the early days of nuclear weapons about how to govern potentially destabilising future technologies, such as AI. I’m a big fan of this historical approach. We previously discussed what the history of general purpose technologies can tell us about the future of AI with Jade Leung (formerly of FHI) on the TiB podcast and in TiB 149.

The paper identifies several key themes: the window of opportunity for radical solutions; the contrasting roles of secret and open discourse; and the need for technical experts to be politically savvy, among others. Particularly interesting is the importance of “political entrepreneurs” - politicians who can react to an exogenous shock that reveals the inadequacy of existing intellectual frameworks and introduce new ones into the political sphere (see this piece for more; the canonical examples are Reagan/Thatcher as political entrepreneurs for monetarism).

It’s interesting to think about how this might apply to AI. First, what event or milestone in its development might constitute a sufficient “shock” to change the political or ideological landscape, domestically or internationally? Second, who stands ready to play the role of political entrepreneur? My sense is that today we lack both leaders with sufficient interest in the space and a platform of policy ideas for them to adopt. Both feel like important gaps if you believe that AI’s impact is only going to accelerate.

One useful idea in venture capital is the “anti-portfolio” - the set of now-successful companies that a venture firm could have invested in, but passed on. Bessemer Venture Partners famously post theirs - which includes Apple, Google and Airbnb - complete with self-deprecating commentary on their web site (Admittedly it is easier to do this when you’ve invested in 130 startups that have IPO’d…)

This week the always-interesting Michael Nielsen suggested that a similar concept might be useful in research funding and philanthropy. This was prompted by the remarkable story of Kati Kariko, whose work on mRNA proved critical to the development of the Pfizer and Moderna COVID vaccines, but was shunned and failed to attract funding for decades. The idea is that funders who go through the exercise might identify weaknesses in their processes (Apparently the Gates Foundation does just this).

As Alexander Berger points out, funders have different incentives from VCs: VCs care who gets the upside from a big company; philanthropists (should) care only whether the world gets the benefit - and by definition a philanthropic anti-portfolio contains only projects that someone funded. But I suspect the exercise is still useful: the factors that cause a funder to miss an important project are likely similar across those that do and don’t find money elsewhere. We discussed in TiB 127 the challenges of applying VC to philanthropy, but this feels like low hanging fruit.

We’ve talked before about the challenge of bad incentives in science. In particular, it seems likely that scientists are incentivised to pursue lower risk, incremental research, rather than riskier but more fundamental work (see TiB 103 for more on this). This might be especially damaging if such improvements in fundamental science have indirect but compounding benefits in terms of the other science and technology they enable.

One model for thinking about this is Jerry Neumann’s excellent argument, which we discussed in TiB 113, that “radical” and “normal” science are in fact the same process - it’s just that the former is more “upstream” and so its ripples are felt further. Matt Clancy - who is an absolute must-follow if you’re interested in the economics of innovation - has an excellent new post on this topic, which provides some empirical evidence for this idea.

Clancy cites this interesting paper, which uses chains of citations from patents to identify upstream (i.e. more fundamental) and downstream technologies. The authors find that patent growth in upstream fields predicts future patent activity in downstream fields. Another study shows that the technologies most directly influenced by new science are more likely to be these upstream fields. Taking these findings together, we can trace a causal chain from scientific progress to more innovation. Clancy concludes that these indirect benefits are at least as valuable as the direct ones (which he discusses in another very good post) - which underlines the importance of finding ways to fix the incentive problem.

BONUS: Thanks to the many of you who spotted that I missed a link last week. Here’s the correct link:

Thanks for reading all the way to the end. If you enjoyed it, please take 30 seconds to share with a friend who might like it.

And, as ever, just hit reply if you have questions, comments, suggestions or feedback.

Until next week,

Matt Clifford