Matt's Thoughts In Between - Issue #30

Forwarded this email? Subscribe here.

Machine learning in the wild

We've discussed progress in machine learning before in TiB and noted its extraordinary potential, but also the challenges of making it work "in the wild". This week seems a good week to revisit the topic, as there have been three major stories in the last couple of weeks that suggest very different paths for ML's real-world impact in the medium term.

First, OpenAI, a big budget not-for-profit, ran a high profile competition pitting its AI against professional video gamers. The AI lost, but only just. The Verge's write up is very good and touches on a number of important issues, including the sheer engineering (as opposed to research) efforts required for these kinds of feats. Second, Google announced that it has now given over control of some of its data centres to a machine learning algorithm. This was announced as an experiment before (to some skepticism), but it now appears to be working properly and delivering a 40% energy saving.

So far, so good - but about a week ago The Information published a big report (summarised here by the author - worth a read) that says that Google's much heralded self-driving car initiative, Waymo, can't get its tech (i.e. AI) to work. Will it ever? The indispensable Ben Evans is skeptical.

What can we draw from these stories? Perhaps one lesson is that the fewer humans an AI has to deal with, the better it performs - which shows how far we still have to go...

How gambling might save academia

The field of psychology has been in the grip of a crisis for a number of years: many of its most important findings can't be replicated and may not be true. This week Ed Yong has a superb piece in the Atlantic on fascinating new work by the Social Sciences Replication Project. In short, if you give people a prediction market in which they bet real money on which studies will replicate, the results are stunningly accurate.

That is, not only can much of the field not be replicated, but which studies don't replicate is so far from random that individuals with the right incentives can guess them very accurately. How? One participant shares her thoughts (and another). I'm fascinated by the fact that "newsworthy" results are particularly unlikely to be replicable; it perhaps suggests that, consciously or otherwise, scientists are least likely to be rigorous when there's the opportunity to discover something marketable.

The fact that prediction markets do such a good job here is encouraging if we want to increase confidence in (future) published results. Ben Mathes has an interesting proposal along these lines for "replication bounties". Not that this is new. Robin Hanson (who else?) actually proposed something similar 28 years ago. Perhaps it's time to give it a go.

Do we need another Moore's Law?

I'm reading a lot about transhumanism at the moment and so was listening to this Econtalk episode from a couple of years ago with Richard Jones, author of Against Transhumanism. It's an interesting discussion in its own right, but I was particularly struck by the discussion of Moore's Law (the relevant section starts around 14:30 in).

Jones makes the argument that Moore's Law is, of course, not a law at all, but a social construction that became a self-fulfilling prophecy by creating a focal point for coordination across a wide range of sectors and companies. In particular, creating a known, but constantly moving technical target allowed companies to justify large capital investments (see here for a good reminder this week of just how large these can be in semiconductors) that would be irrational if you couldn't reliably assume that your suppliers and customers would be making complementary investments to the same timetable.

In this respect, Moore's Law is an extremely useful social construction - and I wonder if we need more like it. It feels like there are no shortage of areas where we want to see technological acceleration, but achieving it requires extraordinarily broad coordination. One option is central planning, but perhaps another is collective belief in monotonic progress. Another case for definite optimism, perhaps?

Quick links

Un-prosperity gospel? Chart of the relationship between GDP and prayer (note the outlier)

More definite optimism. High speed trains in China (amazing photo; compare Crossrail...)

Das Zweikaiserproblem. John von Neumann's favourite joke.

Everyone's best 10 minutes. Good short thread on intellectual compounding in great leaders.

Amazon's unlikely legacy. The strange rebirth of the indie bookstore (and, indeed the brewery - animation)

Your feedback

As always, thanks for reading. If you have thoughts or links to share, just hit reply. And if you like this, forward to a friend or two and encourage them to subscribe.

Until next week,

Matt