Esa Origins 7 February 24 — 26, 2017

Esa Origins 7 February 24 — 26, 2017 PROJECT An Origins Project Scientific Workshop Challenges of Artificial Intelligence: Envisioning and Addressing Adverse Outcomes ARIZONA STATE UNIVERSITY overall software security remains vulnerable: “vulnerabilities are dense” in production code, incentives for securing loT systems are low, key vulnerabilities are stockpiled rather than globally patched. Using machine learning the techniques for vulnerability detection are increasingly sophisticated but opaque. At some point adaptive cyber defense/offense systems become scalable so they can take over vulnerable systems. More aggressive actors combine these systems with botnet functionality and retaliatory responses (e.g. counter-hacking or DDoS attacks) to protect themselves. Since vulnerability discovery is scalable, as they spread and acquire more resources they become more effective. At this point an external cause (e.g. cyberattacks due to an international conflict) or just chance cause aggressive systems to begin large-scale cyberwarfare. This triggers other systems to join in. Some attacks disrupt command-and-control links, producing self-replicating independent systems. All together this leads to a massive degradation of the functionality of the Internet and modern society. Defeating the evolving cyberwarfare systems is hard without taking essential parts of society offline for an extended time - made doubly difficult due to the international stresses unleashed by the outbreak, which in some cases spill over into real-world conflicts and economic crashes. But without a decisive way of cleaning systems the problem will be persistent until entirely new secure infrastructure can be built at a great cost. HUMAN DIMENSION OF CYBERSECURITY: Al FOR SOCIAL ENGINEERING Beyond direct effects on computing systems, rising concerns include the use of Al methods for social engineering to gain access to system authentication information. For example, recent work demonstrated the use of an iterative machine learning and optimization loop for spear phishing on Twitter. There are concerns with Al leveraging one of the weakest links in cybersecurity: people and their actions. DISCUSSION What are key threats ahead and how might they be addressed with new designs? How might we thwart the risk of Al for guiding “social engineering” of attacks and release of information? What are concrete proposals for best practices for thwarting Al for cyberattacks, including highlighting of areas where more research is needed? REFERENCES Singer and Friedman. 2014. Cybersecurity and Cyberwar: What Everyone Needs to Know Flashpoint, 2016. “Ransomware as a Service: Inside an Organized Russian Ransomware Campaign,” (registration required for download), available from Flashpoint library at https://www.flashpoint- intel.com/library/ Seymour, J. and Tully, P. 2016. “Weaponizing data science for social engineering: Automated E2E spear phishing on Twitter,” available at https://www.blackhat.com/docs/us-16/materials/us-16- 14 HOUSE_OVERSIGHT_011297