1.12 Key Claims of the Book 13

1.12 Key Claims of the Book 13 The best way to deal with this second “bad AGI” problem is to build ethics into your AGI architecture — and we have done this with CogPrime, via creating a goal structure that explicitly supports ethics-directed behavior, and via creating an overall architecture that supports “ethical synergy” along with cognitive synergy. In short, the notion of ethical synergy is that there are different kinds of ethical thinking associated with the different kinds of memory and you want to be sure your AGI has all of them, and that it uses them together effectively. In order to create AGI that is not only intelligent but beneficial to other sentient beings, ethics has got to be part of the design and the roadmap. As we teach our AGI systems, we need to lead them through a series of instructional and evaluative tasks that move from a primitive level to the mature human level — in intelligence, but also in ethical judgment. 1.11 Structure of the Book The book is divided into two parts. The technical particulars of CogPrime are discussed in Part 2; what we deal with in Part 1 are important preliminary and related matters such as: e The nature of real-world general intelligence, both conceptually and from the perspective of formal modeling (Section I). e The nature of cognitive and ethical development for humans and AGIs (Section IIT). e The high-level properties of CogPrime, including the overall architecture and the various sorts of memory involved (Section IV). e What kind of path may viably lead us from here to AGI, with focus laid on preschooL-type environments that easily foster humanlike cognitive development. Various advanced aspects of AGI systems, such as the network and algebraic structures that may emerge from them, the ways in which they may selfmodify, and the degree to which their initial design may constrain or guide their future state even after long periods of radical self-improvement (Section V). One point made repeatedly throughout Part 1, which is worth emphasizing here, is the current lack of a really rigorous and thorough general technical theory of general intelligence. Such a theory, if complete, would be incredibly helpful for understanding complex AGI architectures like CogPrime. Lacking such a theory, we must work on CogPrime and other such systems using a combination of theory, experiment and intuition. This is not a bad thing, but it will be very helpful if the theory and practice of AGI are able to grow collaboratively together. 1.12 Key Claims of the Book We will wrap up this Introduction with a systematic list of some of the key claims to be argued for in these pages. Not all the terms and ideas in these claims have been mentioned in the preceding portions of this Introduction, but we hope they will be reasonably clear to the reader anyway, at least in a general sense. This list of claims will be revisited in Chapter 49 near the end of Part 2, where we will look back at the ideas and arguments that have been put forth in favor of them, in the intervening chapters. HOUSE_OVERSIGHT_012929