Andrew Critch and David Kreuger wrote “AI Research Considerations for Human Existential Safety (ARCHES).”

30 May 2020

Andrew Critch, CHAI’s Research Scientist, and David Kreuger wrote “AI Research Considerations for Human Existential Safety (ARCHES).”

This report attempts to explain the relevance of twenty-nine AI research directions to existential risk, and how they interrelate. It also introduces the concept of prepotence, a property weaker than superintelligence, which is more likely to occur before super intelligence, and sufficient to pose a substantial (arguably inevitable) existential threat.