Every indexed document

but people will need good mental models of robots.

to what role we each take when we collaborate on preparing breakfast

enabling them to take part in the negotiations people

too. It is important for robots to account for this when deciding

that influences what actions the human starts taking.

for instance.) What the intent-prediction approach misses

we’re probably going to enter the next room

you often reach around other objects and stop when you

so we might include the physical location of the person in the

it’s tempting to abstract people away. A

we’ll have to define “human-compatible” AI from the getgo.

we can’t create them in isolation from people and then try to make them

if we want highly capable AIs to be compatible

where you have to actually interact with

or a robot that vacuums the living room rug while you’re at work

we’ve assumed that if we’re successful—if robots can

a robot’s job is to figure out what actions it should take in

and it incurs a small cost every time it moves; this reward function

we define a reward function—a mapping from states and

for instance. States are locations in the world