Book: Superintelligence

NON-FICTION
Superintelligence: Paths, Dangers, Strategies
By Nick Bostrom
Oxford University Press (2014)
RRP $38.95

Superintelligence might be the most important book you’ll read this year. Indeed, perhaps even the most significant book of your lifetime. If we are greeted by a superintelligent artificial intelligence (SAI) within this century, as Oxford University philosopher Nick Bostrom suggests is highly likely, we may look upon this book either as an alarm that helped avert disaster or as a prescient insight into the makings of our doom.

This is because superintelligence is serious business. No matter what form an SAI might take – and Bostrom describes a few likely alternatives – and no matter what its values might be (even if its penchant happens to be to calculate pi out to a googolplex decimal points), it is likely to have interests that rub up against ours – such as a desire for resources or energy. Furthermore, it is likely to be better than we are at strategic thinking, and thus at getting what it wants.

This could lead to what Bostrom calls the “treacherous turn” by an SAI, which is likely to take us by surprise, and happen far too late for us to do anything about it. Bostrom also thinks the various countermeasures we might use, such as stunting the SAI, or “boxing” it in by confining it to a walled garden remote from the rest of the world’s computers, are all unlikely to work. After all, if we can think of a countermeasure, so can the SAI. And it can probably think it through more rapidly and formulate counter-countermeasures faster than we can respond.

As such, we must act to resolve the issues of how to control an SAI well before such an entity springs into being. Given that Bostrom considers the emergence of an SAI to be more or less inevitable, that means we had best not delay. He recommends we grapple now with how to develop “good” AI, how to spot a “bad” one, and urges the adoption of best practices by all AI researchers worldwide. 

The stakes are high. As Bostrom puts it: “Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb … We have little idea when the detonation will occur, though if we hold the device to our ear we can hear a faint ticking sound.”

Besides its rallying call, one of the most impressive features of Superintelligence is the absence of hyperbole. Bostrom’s style can probably best be described as achingly clear – if a little dry – and level-headed to a fault. That is a virtue in a field populated by many breathless hand-wavers on one side and giddy doomsayers on the other.