Anca Dragan is an assistant professor in the Department of Electrical Engineering and

PUTTING THE HUMAN INTO THE AI EQUATION Anca Dragan Anca Dragan is an assistant professor in the Department of Electrical Engineering and Computer Sciences at UC Berkeley. She co-founded and serves on the steering committee for the Berkeley AI Research (BAIR) Lab and is a co-principal investigator in Berkeley’s Center for Human-Compatible Al. At the core of artificial intelligence is our mathematical definition of what an AI agent (a robot) is. When we define a robot, we define states, actions, and rewards. Think of a delivery robot, for instance. States are locations in the world, and actions are motions that the robot makes to get from one position to a nearby one. To enable the robot to decide on which actions to take, we define a reward function—a mapping from states and actions to scores indicating how good that action was in that state—and have the robot choose actions that accumulate the most “reward.” The robot gets a high reward when it reaches its destination, and it incurs a small cost every time it moves; this reward function incentivizes the robot to get to the destination as quickly as possible. Similarly, an autonomous car might get a reward for making progress on its route and incur a cost for getting too close to other cars. Given these definitions, a robot’s job is to figure out what actions it should take in order to get the highest cumulative reward. We’ve been working hard in AI on enabling robots to do just that. Implicitly, we’ve assumed that if we’re successful—if robots can take any problem definition and turn into a policy for how to act—we will get robots that are useful to people and to society. We haven’t been too wrong so far. If you want an AI that classifies cells as either cancerous or benign, or a robot that vacuums the living room rug while you’re at work, we've got you covered. Some real-world problems can indeed be defined in isolation, with clear-cut states, actions, and rewards. But with increasing AI capability, the problems we want to tackle don’t fit neatly into this framework. We can no longer cut off a tiny piece of the world, put it in a box, and give it to a robot. Helping people is starting to mean working in the real world, where you have to actually interact with people and reason about them. “People” will have to formally enter the AI problem definition somewhere. Autonomous cars are already being developed. They will need to share the road with human-driven vehicles and pedestrians and learn to make the trade-off between getting us home as fast as possible and being considerate of other drivers. Personal assistants will need to figure out when and how much help we really want and what types of tasks we prefer to do on our own versus what we can relinquish control over. A DSS (Decision Support System) or a medical diagnostic system will need to explain its recommendations to us so we can understand and verify them. Automated tutors will need to determine what examples are informative or illustrative—not to their fellow machines but to us humans. Looking further into the future, if we want highly capable Als to be compatible with people, we can’t create them in isolation from people and then try to make them compatible afterward; rather, we’ll have to define “human-compatible” AI from the get- go. People can’t be an afterthought. 98 HOUSE_OVERSIGHT_016318