It seems likely to me that long before we’re playing pinochle with some smart box

It seems likely to me that long before we’re playing pinochle with some smart box over the fate of our livers, an Al-enabled weapons system of sort will come ripping through our world. This need not be a fully-escaped McGuyver system making pipe- bombs from our cars; even existing technology tools when salted with AI can be slipped into an accidental gear - particularly when they begin interacting with one another. Such AI weapons systems will be trained to operate and move along the most invisible elements of our topologies, sometimes pulling violently at life support cords for currency or logistics or trade but also - perhaps more dangeroulsly - we will find them insinuated into cognition systems we will come to depend upon, whispering into our ears or tapping us on the shoulder “Look that way!” when in fact we should be gazing at some other gaping hole. Of course the problems of how Al-enabled machines are permitted to touch our commerce or our brains or our health have to be considered. Allowed: “You should rehydrate.” Not allowed: “You should have a Coke. It would make people like you.” But these “civilian” problems will be solved, somehow, | think. We haven't yet figured that the culmination of network attack and defense is racing at us and will emerge in the form of smartened weapons. The project of developing a national security or arms control doctrines or treaty frames in these fields has not even begun. Really this means, since we've no hope of honestly controlling every AI that could be possibly written: How do we design the topologies on which Als operate??”° Can we protect ourelves? In the rooms where AI systems “values” are being carefully poked and limited, it’s vitally important that the lessons of history and war have a first place at the table. Sucha conversation, informed by all the popping Seventh Sense warnings we've seen in this book and by a catalog of specificly sharp dangers of diplomacy and security, must happen in cold blood. It will be impossible to tackle these problems cleanly in the heat of an emergency. In our jack-filling enthusiasm for the new, we’d be wise to also gate ourselves and these Al-fired dangers as best we can. For as long as possible. Which, unfortunately, will not be forever. At the start of this book, I explained how the future will unspool: First, there will be a struggle between those who have the Seventh Sense and those who don’t. This is playing out around us today. In the end, the people without the Seventh Sense will lose, because people who fight the future always lose. Then there will be a battle between different groups who have the Seventh Sense, each wired for different aims and instincts. Networks of terror taking on networks of bots. Gene adjusting health protocols competing to become the platform of choice. This battle for the topological high-ground, where unimaginable profit, power and security linger, awaits us. If we're lucky, it will unfold in a co-evolutionary way. Everyone will be better off. But then, finally, there will be a contest between the winners of final topological mastery and the system itself. The Boxers against the Box. The AI machines will have the Seventh Sense, too. Just as computers can see better than us, hear better, or remember longer so the device webs of our future will own this new, essential sense with unimpeachable fidelity. They will glow with it, honed to a sensitive sharpness more acute than any human will ever achieve. What do we do then? We are already 270 Really this means: Kaj Sotala and Roman V. Yampolskiy, “Responses to catastrophic AGI risk: a survey”, Physica Scripta 90 (2015) 197 HOUSE_OVERSIGHT_018429