Imagine that someone points a gun to your head and threatens to pull the trigger. How would you assess the overall risk of your situation? One possibility is to examine the gun: to determine its various properties, how powerful it is, the speed at which bullets emerge from the barrel, and so on. This is what many existential risk scholars have focused on with respect to existential risks: the range of technologies that could be used for harmful ends.
But focusing only on the technologies is a potentially catastrophic mistake, if the aim is survival. To properly evaluate your overall risk in the scenario outlined above, you’d also want to know something about the individual holding the gun: what are his motives, how psychologically unstable is he, what might trigger him to shoot, and so on. Similarly, it’s absolutely paramount that existential risk scholars spend at least as much time studying the various agents who might use future tools to damage civilization as they already do studying the various tools.
Superintelligence is one such agent that has received considerable attention. But there are other agents that ought to be examined with academic rigor in the specific context of existential risks as well. Once the properties of these agents are sufficiently understood, we can then begin to devised strategies—from moral bioenhancement to space colonization—that could neutralize them. This paper is a first step toward this end.