IEET > GlobalDemocracySecurity > Vision > Fellows > Susan Schneider > Artificial Intelligence > Cyber > SciTech
The Problem of AI Consciousness
Susan Schneider   Mar 22, 2016  

Some things in life cannot be offset by a mere net gain in intelligence.

The last few years have seen the widespread recognition that sophisticated AI is under development. Bill Gates, Stephen Hawking, and others warn of the rise of “superintelligent” machines: AIs that outthink the smartest humans in every domain, including common sense reasoning and social skills. Superintelligence could destroy us, they caution. In contrast, Ray Kurzweil, a Google director of engineering, depicts a technological utopia bringing about the end of disease, poverty and resource scarcity.

Whether sophisticated AI turns out to be friend or foe, we must come to grips with the possibility that as we move further into the 21st century, the greatest intelligence on the planet may be silicon-based.

It is time to ask: could these vastly smarter beings have conscious experiences — could it feel a certain way to be them?

When we experience the warm hues of a sunrise, or hear the scream of an espresso machine, there is a felt quality to our mental lives. We are conscious.

A superintelligent AI could solve problems that even the brightest humans are unable to solve, but being made of a different substrate, would it have conscious experience? Could it feel the burning of curiosity, or the pangs of grief? Let us call this “the problem of AI consciousness.”

If silicon cannot be the basis for consciousness, then superintelligent machines — machines that may outmode us or even supplant us — may exhibit superior intelligence, but they will lack inner experience. Further, just as the breathtaking android in Ex Machina convinced Caleb that she was in love with him, so too, a clever AI may behave as if it is conscious.

In an extreme, horrifying case, humans upload their brains, or slowly replace the parts of their brains underlying consciousness with silicon chips, and in the end, only non-human animals remain to experience the world. This would be an unfathomable loss. Even the slightest chance that this could happen should give us reason to think carefully about AI consciousness.

The philosopher David Chalmers has posed “the hard problem of consciousness,” asking: why does all this information processing need to feel a certain way to us, from the inside? The problem of AI consciousness is not just Chalmers’ hard problem applied to the case of AI, though. For the hard problem of consciousness assumes that we are conscious. After all, each of us can tell from introspection that we are now conscious. It asks: why we are we conscious? Why does all our information processing feel a certain way from the inside?

In contrast, the problem of AI consciousness asks whether AI, being silicon-based, is even capable of consciousness. It does not presuppose that AI is conscious — that is the question. These are different problems, but they are both problems that science alone cannot answer.

I used to view the problem of AI consciousness as having an easy solution. Cognitive science holds that the brain is an information-processing system and that all mental functions are computations. Given this, it would seem that AIs can be conscious, for AIs have the same kind of minds as we do: computational ones. Just as a text message and a voice message can convey the same information, so too, both brains and sophisticated AIs can be conscious.

I now suspect the issue is more complex, however. It is an open question whether consciousness simply goes hand-in-hand with sophisticated computation for two reasons.

First, a superintelligent AI may bypass consciousness altogether. In humans, consciousness is correlated with novel learning tasks that require concentration, and when a thought is under the spotlight of our attention, it is processed in a slow, sequential manner. Only a very small percentage of our mental processing is conscious at any given time. A superintelligence would surpass expert-level knowledge in every domain, with rapid-fire computations ranging over vast databases that could encompass the entire internet. It may not need the very mental faculties that are associated with conscious experience in humans. Consciousness could be outmoded.

Second, consciousness may be limited to carbon substrates only. Carbon molecules form stronger, more stable chemical bonds than silicon, which allows carbon to form an extraordinary number of compounds, and unlike silicon, carbon has the capacity to more easily form double bonds. This difference has important implications in the field of astrobiology, because it is for this reason that carbon, and not silicon, is said to be well-suited for the development of life throughout the universe.
If the chemical differences between carbon and silicon impact life itself, we should not rule out the possibility that these chemical differences also impact whether silicon gives rise to consciousness, even if they do not hinder silicon’s ability to process information in a superior manner.

These two considerations suggest that we should regard the problem of AI consciousness as an open question. Of course, from an ethical standpoint, it is best to assume that a sophisticated AI may be conscious. For any mistake could wrongly influence the debate over whether they might be worthy of special ethical consideration as sentient beings. As the films Ex Machina and I, Robot illustrate, any failure to be charitable to AI may come back to haunt us, as they may treat us as we treated them.

Indeed, future AIs, should they ever wax philosophical, may pose a “problem of carbon-based consciousness” about us, asking if biological, carbon-based beings have the right substrate for experience. After all, how could AI ever be certain that we are conscious?

Susan Schneider is a Fellow of the IEET and Associate Professor in the Department of Philosophy, University of Connecticut. Her books include The Language of Thought: a New Direction, The Blackwell Companion to Consciousness (with Max Velmans), and Science Fiction and Philosophy.


I’m curious about this - consciousness doesn’t have a solid definition, and we don’t really understand the conditions that create it. One of the most popular working theories on consciousness is that it is a byproduct of sufficiently dense information with sufficient interlinking. If that theory is correct then AI will have to develop consciousness as a necessity of the way it works.
Of course that theory is not adequately tested yet.

On the contrary,  super intelligent AI would be far more conscious as they’ll understand the basis for consciousness and incorporate an optimal Version in their code. So they’ll grapple with questions like, “are humans truly conscious? “

Before “consciousness” is deemed important to AI, it seems to me the first task at hand would be to define the word, which as logic11 has pointed out, has yet to be done. With a definition in hand, we might then move on to the questions of how and whether it’s important, and whether or not it will be absent in an AI.

The paragraph spent discussing the differences between carbon chemistry and silicon chemistry strikes me as stunningly silly and irrelevant. The chemical activity of silicon has very little to do with the electronics that would be the basis of an AI.

Firstly, I wouldn’t listen to philosophers - they tend to create complication where none exists.  As I recently remarked on Scott Aaronson’s blog, virtually all the most profound ideas in human history are ridiciously simple, and can be described in just a few lines of math and a few sentences of plain English.  Don’t listen to philosophers, listen to scientists who have actually investigated the issues rather than sitting around in a room making stuff up.

There’s already a perfectly good theory of consciousness, which looks like it might be the answer:  The Information Integrated Theory (IIT) of Tononi and Koch.  Consciousness is just highly integrated information.

I came up with a good analogy on Scott’s blog:  think of the phase transitions that occur between GAS >>> WATER >>> ICE

Someone that didn’t know anything about physics and chemistry would find it hard to understand how ice could possibly arise from water (they look so different!), but in fact they are the same - it’s just that a phase transition has occcured due to differences in temperature. 

Now think of information, and imagine that there are different ‘phases’ or states that information could be in according to how ‘coherent’ or ‘integrated’ that information is.  Consider the following analogies:

MATTER         =  WATER

Imagine that reality actually entirely consists of pure information floating around everywhere.  Mathematics can be considered to be information that is weakly or poorly integrated (it has a low coherence value). 

Now imagine turning a knob that ‘turns up’ the coherence of the information.  Suddenly the information undergoes a ‘phase transition’, and what was before just pure mathematics will appear to us as solid matter. 

This is exactly analogus to turning a knob to turn up the temperature, and having gas turn into water.  (In the case of information, coherence plays the role of temperature in this analogy).

To go from matter to consciousness, just turn the knob to increase coherence again, and the information will again undergo a phase transition - changing it’s appearance from matter to consciousness.  That’s why you can say that consciousness is literally ‘crystallized information’.  It’s exactly analogous to the transition from water to ice.

To get somewhat more technical, there are 3 states of existence ( Math, Matter and Consciousness), and ‘Coherence’ is the correct ‘measure’ of ‘strength of existence’.  Math has a weak strength of existence, Matter a medium strength of existence and Consciousness is the strongest type of existence.

There are implications here for things like anthropic reasoning, because the theory implies that more complex (coherent) types of consciousness have higher measure - you are much more likely to exist as a human than an ant!

So yes, AI can be conscious, and it doesn’t depend on material properties, but only on how the information is organized.

Whether or not an AI actually will be conscious depends on how it’s designed.  Consciousness and Intelligence are not the same thing, and as you point out in the article, much of our own thought processes are in fact unconscious.  Consciousness is just the tip of the iceberg.

If you’re talking about a pure Bayesian AI, that works according to decision theory and probability theoey , then no, that type of AI would not be conscious, since there simply isn’t enough information integration.

To get a fully conscious AI, you’d need an AI based on categorization (concepts) and high-level models of itself (narratives).  This type of AI should have enough information integration to be conscious.

Great article Susan!

Re “The problem of AI consciousness asks whether AI, being silicon-based, is even capable of consciousness… consciousness may be limited to carbon substrates only…”

Consciousness could turn out to be limited to carbon substrates only, or, more generally, the silicon substrates that we are presently able to engineer might just not have what it takes to run consciousness. Yes indeed. But “AI being silicon-based” is an assumption - we could learn how to build artificial carbon substrates that, while being able to run consciousness, perform much better than biological brains in other respects. For example, it has been suggested that quantum features of brain tissue could play a critical role in consciousness. But that doesn’t mean that AIs can’t be conscious. It only means that, to build conscious AIs, we will need to develop substrates that exhibit the same quantum features.

Highly unlikely that consciousness has anything to do with the specific material properties of the brain Giulio.  Whether it’s silicion or carbon based makes no difference.

(a) The brain doesn’t do anything mechanical, it’s clearly an information processing system

(b)  Church-Turing demonstrated that a general purporse computer like the brain (a ‘universal turing machine’ or UTM) can emulate any specfic computer (programs run on different machines are identical in the abstract, so physical details not important).

(c)  Robin Hanson argument : brain is a signal processor; signal processing is an abstract process that is insulated from underlying physical details.

No evidence for QM effects either, brain looks entirely classical.

I think the IIT (Integrated Information Theory) is the best scientific theory of consciousness so far.  According to IIT, consciousness is information processing, and degree of consciousness depends on how integrated the information is.

The IIT is a wonderfully simple and elegant theory, even if it’s not quite right yet I think it’s very much along the right lines.  ( It may still need some additional elements to work).

From the cog-sci perspective, the purpose of consciousness is very likely high-level planning that involves a model of the self and continuous generation of a stream of narratives.  This can fit in with IIT.

It might not be popular, but I actually LIKE ORCH OR. I know, I know. But at this point I’d say it’s no less than IIT, which Scott Aaronson believes is utterly wrong.

Any scientific theory of conscious mind should explain (1) why consciousness exists at all (the “Hard Problem”)  (2) how consciousness could be locally or globally bound by a pack of membrane-bound, supposedly classical neurons (the phenomenal binding / combination problem) (3) how consciousness exerts the causal power to allow us to discuss its existence (the problem of causal impotence vs causal over-determination) (4) how and why consciousness has its countless textures and the interdependencies of their different values (the “palette problem”). Yet above all, any adequate scientific theory of consciousness should offer novel, precise and empirically falsifiable predictions, not mere retrodictions. A good scientific conjecture should be in Popper’s sense “risky”. The predictions should be robust and replicable. Further, the outcome of a well-designed experimental test should - by prior agreement - satisfy both proponents and critics.

I won’t outline my own views here; rather, I’m curious how well IIT satisfies the criteria above?


I think your list of requirements for a good theory of consciousness is rather too stringent, especially point (1). 

Someone could make the same argument against physics, by claiming ‘there’s a “hard problem” of physics, where did matter and energy come from in the first place?  But in fact physics still works just fine without an answer.  Similary, Koch and Tononi would say it’s perfectly valid just to take consciousness as a given, and work from there.

In terms of asking where consciousness comes from in the first place, there may be an answer, but that would require adding some additional elements to IIT, namely, aspects of metaphysics.

Although I don’t agree with Penrose’s views on consciousness, I did think his discussion of metaphysics in ‘The Emperor’s New Mind’ was very erudite, and it’s good starting point for thinking about these issues.

Penrose argued for a platonic view of mathematics, namely that mathematics has an eternal existence independent of the human mind .  He then pointed out that there appear to be ‘3 worlds’ (Mathematics, Physics and Mind), and metaphysics is really about the relationship between these 3 worlds.  I think that’s correct.

So to answer the ‘hard problem’ of consciousness, we are really asking for a theory that precisely explains the relationship between Penrose’s ‘3 worlds’.

I sketched a possible answer that fits in with IIT in my earlier post.  I suggested that there are 3 types of ‘existence’, each one corresponding to one of the 3 ‘worlds’ of Penrose.  I then suggested that the properties that all 3 have in common are *Information* and *Coherence*. 

Information can exist with varying degrees of coherence, and it’s changes in coherence that cause ‘phase changes’ in information states, enabling information to appear as mathematics, matter and consciousness.  Consciousness is just information with particularly high coherence.

In terms of cognitive science, I think I understand consciousness very well.  I would say that consciousness is an information processing system engaged in *self-modelling*, and this model takes the form of *narratives*.  So consciousness basically is a self-model of high-level action-plans in the form of narratives.

IIT approaches conciousness from a different angle, namely information theory.  Given that consciousness is about information processing, and given that what most clearly distinguishes different information processing systems is the degree of information integration, then IIT just seems very plausible.

Max Tegmark recently did some work on ITT in which he defined practical complexity measures that would allow empricial tests of the theory.  Koch and Tononi have also recently talked about empirical tests in the form of a ‘magnetic pulse’ that could be sent through the brain to get a rough measure of information integration.




Mjgeddes, many thanks. I suspect some of our background assumptions are very different; but here goes.

The “Hard Problem” of consciousness is a euphemism, IMO. By analogy, imagine if Biblical literalists spoke of the Hard Problem of fossils. If the existence of a phenomenon is inconsistent with the ontology of our preferred theory of the world, then we’d probably do best to update our theory - or alternatively, discard it altogether. By these lights, materialism in traditional sense is false; but I’d be loath to give up non-materialist physicalism – not yet any rate. (cf.

Yes, the origin of matter and energy is a challenging problem. I wouldn’t call it a Hard Problem in the objectionable sense above because their existence isn’t inconsistent with our best scientific theory of the universe. Indeed, physics itself hints at the answer. Insofar as negative gravitational energy exactly cancels positive mass-energy, the total amount of energy in the universe is exactly zero
(cf. Of course, here we get into murky “philosophical” questions about why anything exists at all. (cf.

Integrated Information Theory? I’ve no problem with crazy assumptions – some of mine are intuitively crazier – but IIT’s apparent lack of any novel, precise and falsifiable empirical predictions is scientifically troubling. If Max Tegmark really has devised a truly predictive IIT test, this is great news; I just haven’t seen it yet.

Consciousness as “self-modelling”? Many investigators find the idea appealing. However, perhaps recall that our most intense forms of consciousness, ranging from orgasmic bliss to unbearable agony to uncontrollable panic, are characterised precisely by a breakdown or absence of meta-cognition. Conversely, our most sophisticated forms of meta-cognition and higher-order intentionality are phenomenally thin. So I think a better case can be made that the evolutionary roots of conscious mind run extraordinarily deep.

Well David, I skimmed through your paper ‘Non-materialist physicalism’.  Yes I have a very different ontology.

Penrose’s book ‘The Emperor’s New Mind’ was the book that convinced me that physicalism is false - the arguments Penrose presented for Platonism ( a realm of pure mathematics that exists beyond space and time) were very strong.

I do equate consciousness with an *intrinsic* property, but not of *physics*.  (Remember, I think physicalism is false).  I’d say that consciousness is the intrinsic property of *reality* .  Physics is the structural-functional property of reality.  Mathematics is the representational property; 3 different properties, 1 reality.

Mjgeddes, any quantum mind theory that doesn’t fall into the realm of not-even-wrong woo should make novel, precise and empirically falsifiable predictions. The Hameroff-Penrose Orch-OR hypothesis satisfies the latter criterion insofar as Orch-OR predicts a collapse-like deviation from the unitary Schrödinger dynamics that should in principle be experimentally detectable. Quantum mind theories (like the one I explore) that don’t propose modifying or supplementing the unitary Schrödinger dynamics are in principle experimentally falsifiable too - though credible lifetimes of individual neuronal superpositions in the CNS are so ludicrously short that most theorists (e.g. Max Tegmark) regard such timescales as the reductio ad absurdum of quantum mind rather than a testable hypothesis.

Testing platonic realism is harder. The very best mathematicians seem to be platonists; I incline to strict ultrafinitism.


Well I certainly don’t support any quantum mind theories (although I agree with some of Penrose’s metaphysics, I don’t agree with his particular theory of consciousness).  I’d be amazed if quantum physics had anything to do with consciousness. 

In the case of the brain, you have an extremely complex structure, strongly suggesting that it’s the high-level organizational properties of the brain that are important for consciousness, not specific physics effects.  Then, you have to consider the fact that there’s absolutely no evidence for QM effects in the brain, it all seems entirely classical.

I read your paper; I never cease to be amazed at the theories that really smart people such as yourself can come up with.  Your ideas seem hugely complex - simple consideration of Occam’s razor makes your theory seem wildly implausible to me, but you should take this a compliment!  Penrose’s theory is even less plausible than yours, since he proposes modifications to standard QM, whereas at least you stick with ordinary QM.


Mjgeddes, wild differences in intuitions of (im)plausibility of theories of consciousness serve to illustrate why experimental testability is so critical. The problem with our naive assumption that membrane-bound neurons function as discrete, decohered classical objects is that such quasi-classicality makes explaining why we aren’t so-called micro-experiental zombies impossible: at most, we should be aggregates of Jamesian “mind-dust”. The combination/binding problem is ultimately why a smart, scientifically literate guy like David Chalmers feels driven to his naturalistic dualism. Chalmers scrupulously examines the case for non-materialist physicalism; but Chalmers believes that the conjecture can’t cope with the apparent structural mismatch between phenomenal mind and the microstructure of the brain.

Complex? Surely invoking nothing beyond the unitary Schrödinger dynamics to explain our minds is parsimonious: we’ve no experimental or theoretical reason to expect the superposition principle of QM breaks down in the CNS or anywhere else. If you find the purported solution incredible, so do I – thermally-induced decoherence in the CNS is insanely rapid - which is why I’m urging something on the lines of the interferometry experiment outlined.

Ask an audience “Who said ‘The robin flies with quantum coherence’?”
(1) Physical Review Letters
(2) Deepak Chopra
Most intelligent people will reply (2). The answer is actually (1)

None of this remotely goes to show human minds aren’t classical, though it does highlight the advantages of experiment over robust commonsense.


Ha,ha, yes,  “The robin flies with quantum coherence”, a ridicious Chopra-style headline.  I checked out the link, and it’s only talking about a receptor in the robin’s eye for detecting magnetic fields, so nothing to do with consciousness.  I breathed a sigh of relief. 

Look, I understand what your thinking is, you’ve really thought very hard about the consquences of physicalism, and you’ve tried hard to save physicalism.  Your thinking is really taking reductionism and physicalism seriously, and just carrying it right through to its logical consequences ( you also reject mathematical platonism and lean towards strict ultrafinitism).

But I feel the end result of your thinking really points to big problems with physicialism and reductionism, and my view is that it’s much simpler just to abandon these two postulates together.

From my point-of-view, it’s much simpler just to accept, as Penrose suggests, that there exist ‘3 Worlds’ - Mathematical, Physical and Mental, and the questions as to ‘why’ they exist in the first place (‘the hard problems’) are ultimately unanswerable.  We just have to accept them as fundamental (intrinsic) elements of reality.

Nor can we hope to ‘reduce’ the 3-worlds to 1.  We cannot ‘reduce’ mathematical or mental properties to physical ones.  We *can* obtain scientific understanding of them, but I think we just need to accept them on their own terms first.

Scott Aaronson, one of the smartest guys on the net, agrees with me on this I think.  From the discussions on his blog, he does seem to accept that at some level, mathematics, physics and consciousness are irreducible…we just have to start by accepting them as fundamental, intrinsic elements of reality and reasoning from there.  In this, both Aaronson and myself agree with David Chalmers.

The fact is that the two current leading scientific theories of consciousness are the Self-Modelling Theory (consciousness as a self-simulation), and the Information Integration theory (consciousness as integrated information). 

And really, both of these theories have only been successful because they started by taking consciousness as a given and reasoning about it on its own terms, not by trying to explain it away or ‘reduce’ it to something else.


YOUR COMMENT Login or Register to post a comment.

Next entry: Danaher Awarded by Irish Research Council

Previous entry: “Humanist Missionaries” aim to secularize Africa via Children’s Education