Despite being a transhumanist who wants to transcend my boundaries, I agree strongly with the need for limits and constraints as we move towards increasingly transformative technologies. For some, “no limits, yay!” is the rallying call, but I look at the situation from a thermodynamic, not political, perspective.
In Nick Bostrom’s “Dignity and Enhancement”, commissioned for The President’s Council on Bioethics, he says:
Let us make a leap into an imaginary future posthuman world, in which technology has reached its logical limits. The superintelligent inhabitants of this world are autopotent, meaning that they have complete power over and operational understanding of themselves, so that they are able to remold themselves at will and assume any internal state they choose. An autopotent being could, for example, easily transform itself into the shape of a woman, a man, or a tree. Such a being could also easily enter any subjective state it wants to be in, such as state of pleasure or indignation, or a state of experiencing the visual and tactile sensations of a dolphin swimming in the sea. We can also assume that these posthumans have thorough control over their environment, so that they can make molecularly exact copies of objects and implement any physical design for which they have conceived of a detailed blueprint. They could make a forest of redwood trees disappear, and then recreate an exactly similar forest somewhere else; and they could populate it with dinosaurs or dragons – they would have the same kind of control of physical reality as programmers and designers today have over virtual reality, but with the ability to imagine and create much more detailed (e.g. biologically realistic) structures. We might say that the autopotent superintelligences are living in a plastic world, not because everything is literally made out of plastic but because they can easily remold their environment exactly as they see fit.
Now, it might be that in any technological utopia which we have any real chance of creating, all individuals will remain constrained in important ways. In addition to the challenges of the physical frontiers, which might at this stage be receding into deep space as the posthuman civilization expands beyond its native planet, there are the challenges created by the existence of other posthumans, that is, the challenges of the social realm. Resources even in Plastic World would soon become scarce if population growth is exponential, but aside from material constraints, individual agents would face the constraints imposed on them by the choices and actions of other agents. Insofar as our goals are irreducibly social – for example to be loved, respected, given special attention or admiration, or to be allowed to spend time or to form exclusive bonds with the people we choose, or to have a say in what other people do – we would still be limited in our ability to achieve our goals. Thus, a being in Plastic World may be very far from omnipotent. Nevertheless, we may suppose that a large portion of the constraints we currently face have been lifted and that both our internal states and the world around us have become much more malleable to our wishes and desires.
In Plastic World, many of the moral imperatives with which we are currently struggling are easily satisfiable. As the loud values fall silent, the quiet values become more audible. With most externally imposed constraints eliminated by technological progress, the constraints which we choose to impose on ourselves become paramount.
In this setting, Dignity as a Quality could be an organizing idea. While inanimate objects cannot possess Human Dignity, they can be endowed with a kind of Dignity as a Quality. The autopotent inhabitants of Plastic World could choose to cultivate their sensibility for Dignity as a Quality and the other quiet values. By choosing to recognize these values and to treat the world accordingly, they would be accepting some constraints on their actions. It is by accepting such constraints that they could build, or rather cultivate their Plastic World into something that has greater value than a daydream. It is also by accepting such constraints – and only by doing so – that it would be possible for them to preserve their own Dignity as a Quality. This dignity would not consist in resisting or defying the world. Rather, theirs would be a dignity of the strong, which would consist in self-restraint and the positive nurturance of both internal and external values.
Think about the number of possible configurations the atoms making up your body could be in. It’s an exponentially large space. But the vast majority of that space is a lifeless pile of protein sludge. Limits and constraints are what makes any form of order and structure possible. Human beings laughing, making love, creating things, and enjoying life are all in a tiny thermodynamic neighborhood in a tiny city in a miniscule nation that’s a blemish on a Betelgeuse-sized superplanet.
Systems theorists point out that interesting behavior takes place on the boundary between order and chaos. They make it seem as if our bodies are made up of half order, half chaos. But there is so much more order than chaos. Every cell is governed by organelles that behave in a remarkably orderly way. Consider the process of embryogenesis, where a single cell unfolds into a huge organism with trillions of moving parts - getting the majority of them right every time, as long as the process is not compromised.
We are used to things being orderly by default, so when we think we’re introducing “disorder” into the system, say by downloading mp3s from a P2P service, it’s easy to get all excited by the ethos of entropy and disorder. What’s easy to ignore is the massive base of orderly complexity underlying our ability to make a minor tweak to the system we proudly regard as our own little rebellion. The MP3 is a discrete string of bits, that stays exactly the same as it is downloaded from a peer to our own computers. Our computer has to interpret the string in a precise way, converting it into electric signals and then audio on our speakers. The air vibrates in a precise pattern dictated by the laws of wave propagation, where it then reaches our ear, and is relayed and processed by our audio cortex in a way that has been finely tuned over the course of millions of years of complexity accrual. The “disorderly” act we committed takes place in a context of order so huge and consistent that we don’t even know it’s there - our orderly brains are programmed to take it for granted, and only focus on the tiny surface variations.
In a Plastic World, where superintelligences roam with technology that can modify the environment utterly at will, all the order that we take for granted could be knocked right out from under us. I don’t want that to happen. We are tremendously ordered structures, and order-indifferent superintelligences, unless they care about human beings specifically, could easily perturb our structure like a painter irreverently mixing together pigments on his palette. Just like a precise hue, tint, tone, and value is required to paint a masterpiece, precise initial conditions will be required to build Artificial Intelligences that don’t remove the foundations of order from our universe.
To think that we can just sit back and hope for the best, and everything will turn out okay, is the height of folly. We’re creating a new species here, something likely to happen within our lifetimes, if not the next couple decades. Careful planning and implementation will be necessary. Human-safe AI will not spontaneously emerge as gremlins in the Net, no matter how many cyberpunk writers would like to fantasize otherwise. The way that a mere mention of Asimov’s laws causes people to stop thinking seriously about the deep structure of AI, despite its status as a transparent plot device, is very worrisome. We need to think in more detail, and ask ourselves, “how do we retain the types of order we want in a world where it is deeply threatened?” Not through political discussion. This is a technical problem. There are preliminary attempts at an answer, but they’re not nearly enough. We must keep in mind Nick Bostrom’s Maxipok principle: instead of arguing endlessly about exactly how things should be, we must think of strategies to maximize the probability of an okay outcome. In a post-Singularity world, the pie is large enough that everyone will be able to have a slice. The challenge is making sure that we all make it there intact.
IEET Blog