---

Tools, the ancestor of technology, predated humans. Gavin Kendall, citing Hirst and Woolley, notes:

The emergence of Homo sapiens from earlier forms was a result of a pre-human-culture couple....it is the interaction between the pre-human animal and the technology which leads to the birth of something we call human....Homo sapiens are a direct result of a human-nonhuman network.

While we tend to focus on how we create technology, an alternate possibility exists. That possibility is that it is not we who create technology, but rather the interaction between both we and technology that create one another.

The technology of the past and present is inanimate, non-sentient, inert, but still it shapes us, molds us, changes us. It was the roads that enabled the Roman Empire to flourish.

As we look ahead to AGI+ and AGI++ there is anxiety among some that the technology will turn violent or be malevolent. That is an externalized fear; it asks: "How will the technology behave toward us?"

But what has an equal potential of happening is that AGI will become pervasive in culture through business. It will begin to shape our culture and view of human ideals. Here the question is an internalized fear: "What will technology do to us?"

At the Singularity Summit in San Francisco this past summer, Shane Legg in his talk on "Measures of Machine Intelligence" used the terms external behavior ("if it looks like a duck...") and internal properties (e.g. thought, emotions, understanding, imagination, etc.) to describe different goals for machine intelligence.

In the short and near term, AGI is likely to use external behavior. What will be missing are the intangible, subjective aspects of human intelligence. Yet it could be argued that it is those very internal properties that make up the human experience. When we witness a child being born or an animal dying, our reactions are visceral, deep, and often quite subjective.

When we seek to demonstrate ROI, when we seek to demonstrate value, when we seek to evaluate success, we will turn increasingly toward external behavior that can readily be measured, and away from internal properties that cannot. This is already the trend, is it not?

AGI may exacerbate our reliance on the purely quantifiable. Eventually, we may cease to think of it as a tool. We may think of AGI as better, we may seek to be like it, we may seek to emulate it. Remember, technology creates humanity as much as humanity creates technology.

Slowly, the droplets of reason will erode us. We and the machine will converge, not simply through a transhuman vision of body and machine merging, but through a cultural mindset that denies that love and joy and ecstasy have equal value. How quickly will it take for internal properties to become viewed as liabilities?

Marcelo Rinesi pointed out in an essay last year that not all of humanity is on the same technological curve. For those who are not technology-dependent the essence of the internal properties will likely continue to define their perception of the world.

The technologically-privileged and the minimally-connected will be trying to coexist, but the technologically-privileged will likely to have the money, power, and influence. What is compassion when based only on objective reasoning; is it the same as justice? Will we become the chickens who could be genetically modified for blindness so as to reduce our anxiety about being cooped up too closely with others?

Paul Thompson, a philosophy professor at Purdue University, has posed this philosophical conundrum:

There's a strain of chickens that are blind, and this was not produced through biotechnology. It was actually an accident that got developed into a particular strain of chickens. Now blind chickens, it turns out, don't mind being crowded together so much as normal chickens do. And so one suggestion is that, "Well, we ought to shift over to all blind chickens as a solution to our animal welfare problems that are associated with crowding in the poultry industry."

Will we become blind chickens? To avoid that, what counterbalances will we create? What, if anything, is essential to our humanity? What value will there be in art and poetry? Will some of what we prehend from Michelangelo, perhaps all of what we seek from Rumi, dissipate as steam?

"Some say the world will end with fire, Some say ice." - Robert Frost