Institute for Ethics and Emerging Technologies


The IEET is a 501(c)3 non-profit, tax-exempt organization registered in the State of Connecticut in the United States. Please give as you are able, and help support our work for a brighter future.


Search the IEET
Subscribe and Contribute to:


Technoprogressive? BioConservative? Huh?
Quick overview of biopolitical points of view




whats new at ieet

Rituals Improve Life According to Ancient Chinese Philosophers

The Future of PR in Emotionally Intelligent Technology

Optimize Brain Health by Balancing Social Life with Downtime

Les membres bioniques seront-ils un jour à la mode ?

Multitasking Is a Myth, and to Attempt It Comes at a Neurobiological Cost

Faithfulness—The Key to Living in the Zone


ieet books

Philosophical Ethics: Theory and Practice
Author
John G Messerly


comments

almostvoid on 'Optimize Brain Health by Balancing Social Life with Downtime' (May 23, 2016)

instamatic on 'Faithfulness--The Key to Living in the Zone' (May 22, 2016)

R Wordsworth Holt on 'These Are the Most Serious Catastrophic Threats Faced by Humanity' (May 22, 2016)

Giulio Prisco on 'Faithfulness--The Key to Living in the Zone' (May 22, 2016)

Giulio Prisco on 'Faithfulness--The Key to Living in the Zone' (May 22, 2016)

spud100 on 'Guaranteed Mirage Income?' (May 22, 2016)

John G Mess on 'Ethicists Generally Agree: The Pro-Life Arguments Are Worthless' (May 21, 2016)







Subscribe to IEET News Lists

Daily News Feed

Longevity Dividend List

Catastrophic Risks List

Biopolitics of Popular Culture List

Technoprogressive List

Trans-Spirit List



JET

Enframing the Flesh: Heidegger, Transhumanism, and the Body as “Standing Reserve”

Moral Enhancement and Political Realism

Intelligent Technologies and Lost Life

Hottest Articles of the Last Month


Ethicists Generally Agree: The Pro-Life Arguments Are Worthless
May 17, 2016
(4019) Hits
(10) Comments

Artificial Intelligence in the UK: Risks and Rewards
May 12, 2016
(3133) Hits
(0) Comments

Nicotine Gum for Depression and Anxiety
May 5, 2016
(2962) Hits
(0) Comments

3D Virtual Reality Is the Best Storytelling Technology We’ve Ever Had
May 5, 2016
(2798) Hits
(1) Comments



IEET > Vision > Artificial Intelligence > Galactic > Virtuality > Affiliate Scholar > Rick Searle

Print Email permalink (7) Comments (11495) Hits •  subscribe Share on facebook Stumble This submit to reddit submit to digg


Box of a Trillion Souls - Stephen Wolfram thinks we’ll end up in the Matrix


Rick Searle
By Rick Searle
Utopia or Dystopia

Posted: Mar 18, 2016

“The cybernetic structure of a person has been refined by a very large, very long, and very deep encounter with physical reality.”—Jaron Lanier

Stephen Wolfram may, or may not, have a justifiable reputation for intellectual egotism, but I like him anyway.

I am pretty sure this is because, whenever I listen to the man speak I most often  walk away no so much with answers as a whole new way to frame questions I had never seen before, but sometimes I’m just left mesmerized, or perhaps bewildered, by an image he’s managed to draw.

A while back during a talk/demo of at the SXSW festival he managed to do this when he brought up the idea of “a box of a trillion souls”. He didn’t elaborate much, but left it there, after which I chewed on the metaphor for a few days and then returned to real life, which can be mesmerizing and bewildering enough.

A couple days ago I finally came across an explanation of the idea in a speech by Wolfram over at John Brockman’s Edge.org  There, Wolfram also opined on the near future of computation and the place of  humanity in the universe. I’ll cover those thoughts first before I get to his box full of souls.

One of the things I like about Wolfram is that, uncommonly for a technologist, he tends to approach explanations historically. In his speech he lays out a sort of history of information that begins with information being conveyed genetically with the emergence of life, moves to the interplay between individual and environment with the development of more complex life, and flowers in spoken language with the appearance of humans.

Spoken language eventually gave rise to the written word, though it took almost all of human history for writing to become nearly as common as speaking. For most of that time reading and writing were monopolized by elites. A good deal of mathematics, as well has moved from being utilized by an intellectual minority to being part of the furniture of the everyday world, though more advanced maths continues to be understandable by specialists alone.

The next stage in Wolfram’s history of information, the one we are living in, is the age of code. What distinguishes code from language is that it is “immediately executable” by which I understand him to mean that code is not just some set of instructions but, when run, the thing those instruction describe itself.

Much like reading, writing and basic mathematics before the invention of printing and universal education, code is today largely understood by specialists only. Yet rather than endure for millennia, as was the case with the monopoly of writing by the clerisy, Wolfram sees the age of non-universal code to be ending almost as soon as it began.

Wolfram believes that specialized computer languages will soon give way to “natural language programming”.  A fully developed form of natural language programming would be readable by both computers and human beings- numbers of people far beyond those who know how to code, so that code would be written in typical human languages like English or Chinese. He is not just making idle predictions, but has created a free program that allows you to play around with his own version of a NLP.

Wolfram makes some predictions as to what a world where natural language programming became ubiquitous- where just as many people could code as could now write- might look like. The gap between law and code would largely disappear. The vast majority of people, including school children, would have at the ability to program computers to do interesting things, including perform original research. As computers become embedded in objects the environment itself will be open to the programming of everyone.

All this would seem very good for us humans and would be even better given that Wolfram sees it as the prelude to the end of scarcity, including the scarcity of time that we now call death. But then comes the AI. Artificial intelligence will be both the necessary tool to explore the possibility space of the computational universe and the primary intelligence via which we interact with the entirety of the realm of human thought.  Yet at some threshold AI might leave us with nothing to do as it will have become the best and most efficient way to meet our goals.

What makes Wolfram nervous isn’t human extinction at the hands of super-intelligence so much as what becomes of us after scarcity and death have been eliminated and AI can achieve any goal- artistic ones included- better than us. This is Wolfram’s  vision of the not too far off future, which given the competition with even current reality, isn’t near sufficiently weird enough. It’s only when he starts speculating on where this whole thing is ultimately headed that anything so strange as Boltzmann brains make their appearance, yet something like them does and no one should be surprised given his ideas about the nature of computation.

One of Wolfram’s most intriguing, and controversial, ideas is something he calls computational equivalence. With this idea he claims not only that computation is ubiquitous across nature, but that the line between intelligence and merely complicated behavior that grows out of ubiquitous natural computation is exceedingly difficult to draw.

For Wolfram the colloquialism that “the weather has a mind of its own” isn’t just a way of complaining that the rain has ruined your picnic, but, in an almost panpsychic or pantheistic way, captures a deeper truth that natural phenomenon are the enactment of a sort of algorithm, which, he would claim, is why we can successfully model their behavior with other algorithms we call computer “simulations.” The word simulations needs quotes because, if I understand him, Wolfram is claiming that there would be no difference between a computer simulation of something at a certain level of description and the real thing.

It’s this view of computation that leads Wolfram to his far future and his box of a trillion souls. For if there is no difference between a perfect simulation and reality, if there is nothing that will prevent us from creating perfect simulations, at some point in the future however far off, then it makes perfect sense to think that some digitized version of you, which as far as you are concerned will be you, could end up in a “box”, along with billions or trillions of similar digitized persons, with perhaps millions or more copies of  you.   

I’ve tried to figure out where exactly this conclusion for an idea I otherwise find attractive, that is computational equivalence, goes wrong other just in terms of my intuition or common sense. I think the problem might come down to the fact that while many complex phenomenon in nature may have computer like features, they are not universal Turing machines i.e. general purpose computers, but machines whose information processing is very limited and specific to that established by its makeup.

Natural systems, including animals like ourselves, are more like the Tic-Tac-Toe machine built by the young Danny Hillis and described in his excellent primer on computers, that is still insightful decades after its publication- The Pattern on the Stone. Of course, animals such as ourselves can show vastly more types of behavior and exhibit a form of freedom of a totally different order than a game tree built out of circuit boards and lightbulbs, but, much like such a specialized machine, the way in which we think isn’t a form of generalized computation, but shows a definitive shape based on our evolutionary, cultural and personal history. In a way, Wolfram’s overgeneralization of computational equivalence negates what I find to be his as or more important idea of the central importance of particular pasts in defining who we are as a species, people and individuals.

Oddly enough, Wolfram falls into the exact same trap that the science-fiction writer Stanislaw Lem fell into after he had hit upon an equally intriguing, though in some ways quite opposite understanding of computation and information.

Lem believed that the whole system of computation and mathematics human beings use to describe the world was a kind of historical artifact for which there much be much better alternatives buried in the way systems that had evolved over time processed information. A key scientific task he thought would be to uncover this natural computation and find ways to use it in the way we now use math and computation.

Where this leads him is to precisely the same conclusion as Wolfram, the possibility of building a actual world in the form of simulation. He imagines the future designers of just such simulated worlds:

“Imagine that our Designer now wants to turn his world into a habitat for intelligent beings. What would present the greatest difficulty here? Preventing them from dying right away? No, this condition is taken for granted. His main difficulty lies in ensuring that the creatures for whom the Universe will serve as a habitat do not find out about its “artificiality”. One is right to be concerned that the very suspicion that there may be something else beyond “everything” would immediately encourage them to seek exit from this “everything” considering themselves prisoners of the latter, they would storm their surroundings, looking for a way out- out of pure curiosity- if nothing else.

…We must not therefore cover up or barricade the exit. We must make its existence impossible to guess.” ( 291 -292)

 

Yet it seems to me that moving from the idea that things in the world: a storm, the structure of a sea-shell, the way particular types of problems are solved are algorithmic to the conclusion that the entirety of the world could be hung together in one universal  algorithm is a massive overgeneralization. Perhaps there is some sense that the universe might be said to be weakly analogous, not to one program, to a computer language (the laws of physics) upon which an infinite ensemble of other programs can be run but is structured so as to make some programs more likely to be run while deeming others impossible. Nevertheless, which programs get run is subject to some degree of contingency- all that happens in the universe is not determined from initial conditions. Our choices actually count.

Still, such a view continues to treat the question of corporal structure as irrelevant, whereas structure itself may be primary.

The idea of the world as code, or DNA as a sort of code is incredibly attractive because it implies a kind of plasticity which equals power. What gets lost however, is something of the artifact like nature of everything that is, the physical stuff that surrounds us, life, our cultural environment. All that is exists as the product of a unique history where every moment counts, and this history, as it were, is the anchor that determines what is real. 

Asserting the world is or could be fully represented as a simulation either implies that such a simulation possesses the kinds of compression and abstraction, along with the ahistorical plasticity that comes with mathematics and code or it doesn’t, and if it doesn’t, it’s difficult to say how anything like a person, let alone, trillions of persons, or a universe could actually, rather than merely symbolically, be contained in a box even a beautiful one.

For the truly real can perhaps most often be identified by its refusal to be abstracted away or compressed and by its stubborn resistance to our desire to give it whatever shape we please.


Rick Searle, an Affiliate Scholar of the IEET, is a writer and educator living the very non-technological Amish country of central Pennsylvania along with his two young daughters. He is an adjunct professor of political science and history for Delaware Valley College and works for the PA Distance Learning Project.
Print Email permalink (7) Comments (11496) Hits •  subscribe Share on facebook Stumble This submit to reddit submit to digg


COMMENTS


Astonishing stuff, RS, and here is a bit more to hear from Wolfram’s lecture 10 days ago at the SETI Institute. If you are impatient as I am, skip 1 hour and 7 minutes into Wolfram’s presentation. There we get to the crux of Wolfram’s ideas on the conputational universe and intelligent life.

https://m.youtube.com/#/watch?v=Re9eB_j6m-0





I am reminded of Baudrillard’s Simulacres & Simulation in which he describes the simulated topographical “map” becoming the hyperreality in which people “reside”

He wrote this philosophical treatise in 1981 and I’m wondering if Wolfram is familiar with it, given the similarities in his thinking and rationale for AI. 

https://en.wikipedia.org/wiki/Simulacra_and_Simulation

I am also reminded of Theodore Sturgeon’s short story (1941) called ‘The Microcosmic god’ and his telling of synthetic lifeforms (“Neoterics” in the novel).  There is a strong sense that these two concepts have infiltrated Wolfram’s thinking and influenced his ideas.

https://en.wikipedia.org/wiki/Microcosmic_God

Other than that, thank you for the precis of Wolfram’s discourse on Edge.  I have it bookmarked but life has rather prevented me from really diving into the article.  Your review has made the TL:DR much more digestible.

Regards

Michelle





Thanks much for sharing that spud100. Here’s a more direct link for anyone who wants to check it out:

https://youtu.be/Re9eB_j6m-0





Much thanks, R.S.

Yah, Michelle Microcosmic God was excellent, especially when the neo life evolves from mix, and crawls to the top of the test tube, and breeeps!





Thanks for the references, Michelle. Simulacra and simulation especially seems to address a lot of the questions I’ve been thinking about of late and is now at the top of my reading list.





[…then it makes perfect sense to think that some digitized version of you, which as far as you are concerned will be you, could end up in a “box”, along with billions or trillions of similar digitized persons, with perhaps millions or more copies of you…]

Readers of Mr. Searle’s essay will be both stimulated and entertained by Greg Egan’s science fiction novel, Diaspora. Egan’s 30th century earth is populated by multiply-cloned post-human digital entities living in vast supercomputer cities, or “polises”.

Consideration of central questions facing Transhumanists currently are embedded in an astonishing plot, narrated with scientific and conceptual rigor. The book occupies a well-deserved place in the Transhumanist canon; it was on the list of suggested readings for a recent Transhumanist conference.

The Wikipedia entry https://en.wikipedia.org/wiki/Diaspora_(novel) provides insight and elaboration.





Thanks for the suggestion, Nick.

Readers might also want to check out Greg Egan’s Permutation City which explores similar issues as well. Egan later asked some compelling question about the ethics of simulation itself. Here’s a quote from an interview:

“Q6: What do you regret most about Permutation City?

A6: Something quite separate from the issues with the Dust Theory mentioned above, although these are all valid points. What I regret most is my uncritical treatment of the idea of allowing intelligent life to evolve in the Autoverse. Sure, this is a common science-fictional idea, but when I thought about it properly (some years after the book was published), I realised that anyone who actually did this would have to be utterly morally bankrupt. To get from micro-organisms to intelligent life this way would involve an immense amount of suffering, with billions of sentient creatures living, struggling and dying along the way. Yes, this happened to our own ancestors, but that doesn’t give us the right to inflict the same kind of suffering on anyone else.

This is potentially an important issue in the real world. It might not be long before people are seriously trying to “evolve” artificial intelligence in their computers. Now, it’s one thing to use genetic algorithms to come up with various specialised programs that perform simple tasks, but to “breed”, assess, and kill millions of sentient programs would be an abomination. If the first AI was created that way, it would have every right to despise its creators. “

http://gregegan.customer.netspace.net.au/PERMUTATION/FAQ/FAQ.html

And here’s me spouting off on the same topic:

http://utopiaordystopia.com/2013/03/17/the-ethics-of-a-simulated-universe/

Kind regards,

Rick





YOUR COMMENT (IEET's comment policy)

Login or Register to post a comment.

Next entry: VR Will Create Multiple Existences - “meatspace” will not be considered the only true reality

Previous entry: Body-based Trades and the Ethics of Divided Labour

HOME | ABOUT | FELLOWS | STAFF | EVENTS | SUPPORT  | CONTACT US
SECURING THE FUTURE | LONGER HEALTHIER LIFE | RIGHTS OF THE PERSON | ENVISIONING THE FUTURE
CYBORG BUDDHA PROJECT | AFRICAN FUTURES PROJECT | JOURNAL OF EVOLUTION AND TECHNOLOGY

RSSIEET Blog | email list | newsletter |
The IEET is a 501(c)3 non-profit, tax-exempt organization registered in the State of Connecticut in the United States.

East Coast Contact: Executive Director, Dr. James J. Hughes,
56 Daleville School Rd., Willington CT 06279 USA 
Email: director @ ieet.org     phone: 860-428-1837

West Coast Contact: Managing Director, Hank Pellissier
425 Moraga Avenue, Piedmont, CA 94611
Email: hank @ ieet.org