Origins February 24 — 26, 2017

Origins February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop ARIZONA STATE UNIVERSITY Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes 1) FINANCIAL MARKETS Al, Trading Systems, and Market Manipulation (Incorporating contributions by Michael Wellman and others) There has been advances in the realm of trading in financial markets with the use of autonomous decision systems. Financial markets now operate almost entirely electronically, over networks with relatively well-scoped and well-defined interfaces. Markets generate large quantities of data at high velocity, which require algorithms to digest and assess state. The dynamism of markets means that timely responses to information are critical, providing a strong incentive to take slow humans out of the decision loop. Finally, the rewards available for effective trading decisions are large, enabling a commensurate devotion of resources toward talent and effort to develop and analyze technically sophisticated strategies. The rewards and pervasive automation are a tempting target for market manipulation. Thus there are potential incentives to employ deceptive tactics designed to mislead counterparties about market conditions or world state, toward the goal of exploiting misled participants for profit. “Manual” market manipulation—from spoofing to outright fraud—is prevalent in financial markets today. Al can amplify the magnitude and effectiveness of manipulative behavior, degrading market efficiency or even subverting the essential economic functions of global capital markets. For example, automation can enable more rapid and massive simultaneous attacks on electronic markets, and adaptive capabilities may persistently evade known detection methods. DISCUSSION What are key costly scenarios that we might come to expect and their time frames? What might be done to counter this direction and help to keep markets efficient and functioning well? How might adversaries and incentives lead to a thwarting of such attempts? POTENTIAL GOALS Identify key challenges ahead, including very costly outcomes. Identify key directions with best practices, mechanism design, monitoring and regulatory activity to help to thwart poor outcomes. REFERENCES R. Harris. The Fear Index, Hutchinson, 2011. Summary: https://en.wikipedia.org/wiki/The_Fear_Index M.P. Wellman and U. Rajan. Ethical issues for autonomous trading agents. Minds & Machines, 2017. doi:10.1007/s11023-017-9419-4 http://strategicreasoning.org/publications/2016-2-2/minds-machines-wr/ HOUSE_OVERSIGHT_014698