TiB 133: Why the best ideas don't win; in praise of brain drain; how to preserve political norms; and more...

This week: how the “hardware lottery” shapes the development of machine learning; why brain drain can benefit a startup ecosystem; how to preserve constitutional norms when one side doesn’t want to play ball; and more…

Welcome new readers! Forwarded this email? Subscribe here. Enjoy this email? Forward it to a friend.

Which ideas end up succeeding? One (perhaps comforting) take is that the best ideas win out because of their superiority. But perhaps that’s not true. Sara Hooker of Google Brain has a fantastic paper, one of the most thought provoking I’ve read this year (thanks to a loyal reader for the link) that argues that machine learning research ideas need not only to be good, but also to win what she calls “the hardware lottery”. That is, they have to be compatible with - or, better, optimised for - the current hardware paradigm.

It’s worth looking at what Hooker calls the “bigger is better” race in deep learning (which we previously discussed in TiB 119 and 128) in this context. Is building larger and larger models the right path to AGI? Or is that just what today’s hardware is ever more optimised for? As Hooker notes, success within a particular hardware paradigm promotes further investment in that hardware, which “makes it even more costly to stray off the beaten path of research ideas”. We can think of this as “algorithm-hardware co-evolution”.

As a metaphor this has applications far outside machine learning. It reminds me of my favourite book of 2019, Joe Henrich’s The Secret of Our Success, which I wrote about in TiB 79. Henrich argues that you can’t isolate genetic evolution and cultural development: it’s “culture-gene co-evolution” that has made humans what we are. Just as Henrich reminds us that evolution doesn’t guarantee global maxima, so Hooker says we shouldn’t forget that there’s noise in the “marketplace of ideas”. Developments that reduce the cost of looking beyond the beaten track are often extremely valuable. 

How much should the rest of the world worry about losing its best talent to Silicon Valley (or anywhere else, if you believe the Bay Area has peaked)? One potential concern is “brain drain”. If you’re a policy maker who aspires to build a local technology hub, you might worry that your efforts will be undermined if your leading technologists emigrate. There have long been counterarguments - see this 2015 Marginal Revolution post on high-skill migration and global public goods - but an interesting new(ish) paper argues that brain drain can actually stimulate a startup ecosystem.

The authors argue that immigrant entrepreneurs act as a bridge for US venture capital firms to invest in their home countries. According to their model, each additional investment a US firm makes in an Indian emigrant entrepreneur is associated with almost 10% more investments in startups in India itself. Their explanation is that the investment in the emigrant entrepreneur gives the VC better networks in and knowledge of the home country.

I’ve been thinking a lot about how the pandemic might change venture capital. Does growing comfort with remote investing globalise early stage investing (especially given the persistent possibility of geographic valuation arbitrage)? If so, one interpretation of this paper is that such activity could rapidly snowball. As VCs gain networks in and knowledge of countries in which they’d otherwise never deploy capital, suddenly many more opportunities are in scope. That seems an exciting prospect. 

It’s a good week to revisit “asymmetric constitutional hardball”, last discussed in TiB 126. In the UK, Boris Johnson’s government is threatening to break international law to “get Brexit done” (again). In the US, the death of Ruth Bader Ginsburg (superb profile here) has reignited fierce debate about Mitch McConnell’s, err, innovative and flexible, beliefs about the proper time for the Senate to consider judicial nominations.

Let’s take at face value the idea that norm breaking in the pursuit of desired policy outcomes really is asymmetric (I noted my skepticism last time). How should liberals on both sides of the Atlantic respond? One view is that it’s crucial to uphold norms, even when the other side doesn’t. But Henry Farrell (whose work we discussed in TiB 42, 50, 93 and 123), has a great piece that makes the opposite case: without the willingness to defect from norms, a party will struggle to enforce them. This is a variant of the “tit-for-tat” argument that is crucial in classic work on the evolution of cooperation.

What does that mean in practice? One view is that the Democrats should threaten to pack the courts. Farrell notes that the threat of court packing was crucial to FDR’s political success. And, in an excellent piece, Matt Karp points out that Abraham Lincoln had to go to war politically with the Supreme Court to end slavery. Others, like David Pozen, think there might be room for a popular politics of “anti-hardball”. Whatever the answer, it’s important to get it right. Liberal democracies are rare, precious and more fragile than we’d like to think. 

If you enjoy Thoughts in Between, the easiest way to help is to forward it to a friend (and the best way is to share the URL on social media).

I’m always happy to hear from you. Feel free to reply directly or message me on Twitter (though I’m on paternity leave this week and next, so may be a little slow).

Until next week,

Matt Clifford