Continuing the IEET's 2014 editor's choice award to three articles for their outstanding writing and conceptual analysis contributing to transhumanism and technoprogressivism we are proud to announce #1.
Continuing the IEET's 2014 editor's choice award to three articles for their outstanding writing and conceptual analysis contributing to transhumanism and technoprogressivism we are proud to announce #1.The #1 article by hits was Ramez Naam's “2014 Was a Good Year: Better Than You Remember”
The following piece was first published here on Jan 7, 2014 and is the #1 pick for the IEET editor's choice award.
In the article "Rule by Algorithm? Big Data and the Threat of Algocracy" John Danaher lays out concepts of the utmost importance to the future of humanity dealing with Artificial Intelligence (AI) and Big Data. Danaher put's the problem of AI and Big Data articulately by saying "algorithms take over from the messy, human process of democratic decision-making. Citizens become beholden to them, unsure of how they work, but afraid to disregard their guidance. This creates a sort of prison of “invisible barbed wire” which constrains our intellectual and moral development, as well as our lives more generally" and concludes his article with concepts we can all ponder about for years (or a few) to come: "Enhancement technologies might help to equalise the balance of power between ordinary humans and epistemically elite humans. Whether they could do the same for the balance of power between humans and algorithms is another matter. The gap is much wider. AIs have significant advantages over humans when it comes to data-mining and processing. The only kinds of enhancement technology that could bridge the gap are the truly radical (and hypothetical) forms. The ones that would give rise to a genuinely posthuman form of existence. Even if this form of existence is possible, concepts like freedom and self-determination may no longer have any meaning in such a world. Now there’s a thought…"
What kind of society are we creating? With the advent of the internet-of-things, advanced data-mining and predictive analytics, and improvements in artificial intelligence and automation, we are the verge of creating a global “neural network”: a constantly-updated, massively interconnected, control system for the world. Imagine what it will be like when every “thing” in your home, place of work, school, city, state and country is connected to a smart device?
And when all the data from that device is analysed and organised by search algorithms? And when this in turns feeds into some automated control system?
What kind of world do you see? Should we be optimistic or pessimistic? I've addressed this question in several posts over the past year. I thought it might be useful to collect the links to all those posts in one place. So that's what I'm doing here.
As you'll see, most of those posts have been concerned with the risks associated with such technologies. For instance, the threat they may pose to transparency, democratic legitimacy and traditional forms of employment. But just to be clear, I am not a technophobe -- quite the contrary in fact. I'm interested in the arguments people make about technology. I like to analyse them, break them down into their key components, and see how they stand up to close, critical scrutiny. Sometimes I end up agreeing that there are serious risks; sometimes I don't.
Anyway, I hope you enjoy reading these entries. This is a topic that continues to fascinate me and I will write about it more in the future.
(Note: I had no idea what to call this series of posts. So I just went with whatever came into my head. The title might be somewhat misleading insofar as "Big Data" isn't explicitly mentioned in all of these posts, though it does feature in many of them)
1. Rule by Algorithm? Big Data and the Threat of Algocracy
This was the post that kicked everything off. Drawing upon some work done by Evgeny Morozov, I argued that increasing reliance on algorithm-based decision-making processes may pose a threat to democratic legitimacy. I'm currently working on a longer paper that develops this argument and assesses a variety of possible solutions.
2. Big Data, Predictive Algorithms and the Virtues of Transparency (Part One, Part Two)
These two posts looked at the arguments from Tal Zarsky's paper "Transparent Predictions". Zarsky assesses arguments in favour of increased transparency in relation to data-mining and predictive analytics.
3. What's the case for sousveillance? (Part One, Part Two)
This was my attempt to carefully assess Steve Mann's case for sousveillance technologies (i.e. technologies that allow us to monitor social authorities). I suggest that some of Mann's arguments are naive, and that it is unlikely that sousveillance technologies will resolve problems of technocracy and social inequality.
4. Big Data and the Vices of Transparency
This followed up on my earlier series of posts about Tal Zarsky's "Transparent Predictions". In this one I looked at what Zarsky had to say about the vices of increased transparency.
5. Equality, Fairness and the Threat of Algocracy
I was going through a bit of Tal Zarsky-phase back in April, so this was another post assessing some of his arguments. Actually, this one looked at his most interesting argument (in my opinion anyway). In this one, Zarsky claimed that automated decision-making processes should be welcomed because they could reduce implicit bias.
6. Will Sex Workers be Replaced by Robots? (A Precis)
This was an overview of the arguments contained in my academic article "Sex Work, Technological Unemployment and the Basic Income Guarantee". That article looked at whether advances in robotics and artificial intelligence threaten to displace human sex workers. Although I conceded that this is possible, I argued that sex work may be one of the few areas that is resilient to technological displacement.
7. Is Modern Technology Creating a Borg-Like Society?
This post looked at a recent paper by Lipschutz and Hester entitled "We are the Borg! Human Assimilation into the Cellular Society". The paper argued that recent technological developments pushed us in the direction of a Borg-like society. I tried to clarify those arguments and then asked the important follow-up: is this something we should worry about? I identified three concerns one ought to have about the drive toward Borg-likeness.
8. Are we heading for technological unemployment? An Argument
This was my attempt to present the clearest and most powerful argument for technological unemployment. The argument drew upon the work of Andrew McAfee and Erik Brynjolfsson in The Second Machine Age. Although I admit that the argument has flaws -- as do all arguments about future trends -- I think it is sufficient to warrant serious critical reflection.
9. Sousveillance and Surveillance: What kind of future do we want?
This was a short post on surveillance technologies. It looked specifically at Steve Mann's attempt to map out four possible future societies: the univeillant society (one that rejects surveillance and embraces sousveillance); the equiveillant society (one that embraces surveillance and sousveillance); the counter-veillance society (one that rejects all types of veillance); and the McVeillance society (one that embraces surveillance but rejects sousveillance).
10. Procedural Due Process and Predictive Analytics
Big data is increasingly being used to "score" human behaviour in order to predict future risks. Legal scholars Frank Pasquale and Danielle Keats Citron critique this trend in their article "The Scored Society". I analyse their arguments and offer some mild criticisms of the policy proposals.
Rule by Algorithm? Big Data and the Threat of Algocracy
Unfortunately, Morozov’s article is not written in a style that renders its argumentative structure immediately transparent. I shouldn’t complain: not everyone writes in the style of an analytic philosopher; not everyone aspires to reduce human language to a series of number propositions and conclusions. Nevertheless, I have set myself the task re-presenting Morozov’s argument in a more formal garb, and subjecting it to some critical scrutiny. That's what this blog post is about.
The discussion is broken-down into three sections. First, I’ll talk in general terms about the problem Morozov sees with data-mining technologies. Second, I present what I take to be Morozov’s central argument, which I call the argument from the threat of algocracy. This, I suggest, is similar to an argument found in the work of the political philosopher David Estlund. Finally, I look at the suggested solutions to the problem, noting how Morozov’s solution differs from one that I have defended in the past.
1. The Problem of Invisible Barbed Wire
The way in which our data is being monitored and processed has been well-documented. A recent article by Alice Marwick in the NY Review of Books gives a good overview of the phenomenon. Interested readers should check it out. What follows here is just a summary.
In brief, modern technology has made it possible for pretty much all of our movements, particularly those we make “online”, to be monitored, tracked, processed, and leveraged. We can do some of this leveraging ourselves — as do the proponents of self-experiments and members of the quantified self movement — by tracking our behaviour along various metrics and using this to improve our diets, increase our productivity and so forth. To many, this is a laudatory and valuable endeavour.
But, of course, governments and corporations can also take advantage of these data-tracking and processing technologies and use it to pursue their ends. Governments can do it to monitor terrorist activity, track welfare fraud, and prevent tax evasion. Companies can do it to track consumer preferences, create targetted advertising and manipulate consumer purchasing decisions.
You might think that this is all to the good. Who doesn’t want to stop terrorism, stamp out tax evasion and have a more pleasant shopping experience? But Morozov points out that it has a more sinister side. He worries about the role of data-mining in creating a system of algorithmic regulation, one in which our decisions are “nudged” in particular directions by powerful data-processing algorithms. This is worrisome because the rational basis of these algorithms will not be transparent. As he puts it himself:
The discussion is broken-down into three sections. First, I’ll talk in general terms about the problem Morozov sees with data-mining technologies. Second, I present what I take to be Morozov’s central argument, which I call the argument from the threat of algocracy. This, I suggest, is similar to an argument found in the work of the political philosopher David Estlund. Finally, I look at the suggested solutions to the problem, noting how Morozov’s solution differs from one that I have defended in the past.
You might think that this is all to the good. Who doesn’t want to stop terrorism, stamp out tax evasion and have a more pleasant shopping experience? But Morozov points out that it has a more sinister side. He worries about the role of data-mining in creating a system of algorithmic regulation, one in which our decisions are “Thanks to smartphones or Google Glass, we can now be pinged whenever we are about to do something stupid, unhealthy or unsound. We wouldn’t necessarily need to know why the action would be wrong: the system’s algorithms do the moral calculus on their own. Citizens take on the role of information machines that feed the techno-bureaucratic complex with our data. And why wouldn’t we, if we are promised slimmer waistlines, cleaner air, or longer (and safer) lives in return?
(Morozov, 2013)
In other words, the algorithms take over from the messy, human process of democratic decision-making. Citizens become beholden to them, unsure of how they work, but afraid to disregard their guidance. This creates a sort of prison of “invisible barbed wire” which constrains our intellectual and moral development, as well as our lives more generally:
[The problem] here is the construction of what I call “invisible barbed wire” around our intellectual and social lives. Big data, with its many interconnected databases that feed on information and algorithms of dubious provenance impose severe constraints on how we mature politically and socially…
The invisible barbed wire of big data limits our lives to a space that might look quiet and enticing enough, but is not of our own choosing and that we cannot rebuild or expand. The worst part is that we do not see it as such. Because we believe that we are free to go anywhere, the barbed wire remains invisible.
(Morozov, 2013)
The upshot: big data is undermining democracy by depriving us of our ability to think for ourselves, determine our own path in life, and critically engage with governmental decision-making.
2. The Argument from the Threat of Algo-cracy
Morozov’s argument is clothed in some ominous language and clever metaphors (yes: I like the point about “invisible barbed wire”) but for all that it will be familiar to students of political philosophy. People have long worried about the prospect of epistemic elites taking over governmental decision-making, just as people have also long argued in favour of such takeover (Plato and John Stuart Mill being two who spring to mind).
Morozov’s argument is the argument from the threat of algocracy (i.e. rule by algorithm). At the heart of this argument is the debate about what legitimates governmental decision-making in the first place. What gives governments the right to set policies and impose limits on our behaviour? What makes a system of government just and worthwhile? Generally speaking, there are three schools of thought on the matter:
2. The Argument from the Threat of Algo-cracyInstrumentalists: hold that legitimation derives from outcomes. Policies and regulations are designed to accomplish certain goals (in moral terms: they are designed to help us realise fundamental human goods like joy, sociality, friendship, knowledge, freedom etc.). They are legitimate if (and only if) they accomplish those goals.
Proceduralists: hold that legitimation comes from the properties of the decision-making procedures themselves. They defend their view by arguing that we have no idea what the correct mix of regulations is in advance. What matters is that the procedure through which these regulations are adopted is itself just; that it gives people the opportunity to voice their concerns; that it is comprehensible to them; that it gives due weight to their concerns; and so forth.
Pluralists: hold that some combination of just procedures and good outcomes is needed.
Democratic governance is often defended on pluralist grounds, although some theorists emphasise the instrumental goods over the procedural ones, and vice versa. The general idea being that democratic governance has just procedures and leads to good outcomes, at least more so than alternative systems of governance.
The problem with justifying governance along purely instrumentalist lines is that it can give rise to the threat of epistocracy. This threat has been most clearly articulated by David Estlund. As he sees it, if it is really the case that procedure-independent goods legitimate systems of government, it is likely that that some people (the epistemic elites) have better knowledge or foresight of the policies that will lead to those outcomes than others. Indeed, some of these others (the epistemically incompetent) might actively thwart or undermine the pursuit of the procedure-independent goods. Consequently, and following the instrumentalist logic, we should hand over decision-making control to these epistemic elites.
This looks to be an unwelcome conclusion. It seems to reduce the people affected by governmental decisions to little more than moral patients: receptacles of welfare and other positive outcomes, who do not actively shape and determine the course of their own lives. Hence people tend to fall back on pluralist and/or proceduralist approaches to legitimacy, which value the active contribution to and participation in decision-making. (I should add, here, that people can also resolve the problem by appealing to freedom or self-determination as one of the procedure-independent goods toward which governance should be directed).
Morozov’s compaint is essentially the same as that of Estlund. The only difference being that where Estlund is concerned about human epistemic elites (like experts and other technocrats), Morozov is concerned about algorithms, i.e. non-human artificial intelligences that mine and manipulate our data. Morozov couches his argument in terms of the threat to democracy, but what he is specifically talking about are both procedural goods associated with democratic governance (comprehensibility, participation, deliberation) and the other goods that should be protected by such procedures (freedom, self-determination, autonomy etc.). His concern is that over-reliance on data-mining algorithms will undermine these goods, even if at the same time they help us to achieve certain others.
To phrase his argument in slightly more formal terms:
The problem with justifying governance along purely instrumentalist lines is that it can give rise to the threat of epistocracy. This threat has been most clearly articulated byThe Argument from the Threat of Algocracy
- (1) Legitimate democratic governance requires decision-making procedures that protect freedom and autonomy, and that allow for individual participation in and comprehension of those decision-making procedures.
- (2) Reliance on data-mining algorithms undermines freedom and autonomy and prevents active participation in and comprehension of decision-making procedures.
- (3) Therefore, reliance on data-mining algorithms is a threat to democratic governance.
This argument relies on a normative premise (1) and a factual/predictive premise (2). I think the normative premise is reasonably sound. Indeed, I would add in the explicit claim — implied in Morozov’s article — that this form of governance is desirable, all else being equal. The only question then is whether all else is indeed equal. Proponents of algocracy — like proponents of epistocracy — could argue that the procedure-independent gains from algorithmic policy-making will outweigh the procedural and autonomy-related costs. Morozov needs it to be the case that the goods singled out by his argument are more important than any of those putative gains (assuming he argues from consequentialism) or that their impingement is blocked by deontological constraints. I think he would have a strong argument to make on consequentialist grounds; I’m less inclined to the deontological view.
I suspect that the factual/predictive premise is going to prove more controversial. Is it true that the widespread use of data-mining algorithms will have the kind of negative impact envisioned by Morozov? It certainly seems to be true that the majority people don’t comprehend the basis on which algorithms make their decisions. Whether this must be the case is less clear to me. I’m not well-versed in how modern algorithms are coded. Still, I suspect there is a good case to be made for this. That is to say: I suspect algorithms throw up many surprises, even for the engineers who create them, and will continue to do so, particularly as they become more complex.
A slightly more tricky issue has to do with the potentially autonomy-enhancing effects of data-mining technologies. The message from fans of self-experimentation and self-quantification, who are often obsessed with mining data about themselves, seems to be that technologies of this sort can greatly enhance the control one has over one’s life. They can lead to greater self-awareness and goal fulfillment, allowing us to pick and choose the behavioural strategies that lead to the best outcomes for us. That would seem to suggest that premise (2) is false in certain instances.
But that’s not to say that it is generally false. Morozov is probably right in thinking that top-down use of such technologies by governments and corporations is a problem. Nevertheless, the experiences of these individual users does suggest that there is a way in which the technology could be harnessed in a positive manner. Morozov is aware of this argument, and resists it by arguing that the forces of capitalism and bureaucracy are such that the top-down uses will tend to dominate the bottom-up uses. This seems to be what is happening right now and this is what is important.
3. Solutions?
So what is to be done about the threat of algocracy? Morozov complains that current solutions to data-mining miss the point. They conceive of the problem in terms of the right to privacy, and hence craft solutions that give people greater control over their data, either through robust legal protections of that data, or market-based systems for owning and trading in that data (the suggestion in the article is that market-based solutions are distinct from legal ones, though this is clearly not the case: the kinds of market-based solution that are being advocated rely on the law of property and contract).
Instead, Morozov — in a section titled “Sabotage the System. Provoke more Questions” — argues for a four-pronged solution (there is some overlap between the prongs):
Instead, Morozov — in a section titled “Sabotage the System. Provoke more Questions” — argues for a four-pronged solution (there is some overlap between the prongs):A. Politicise the problem: we must, as he puts it “articulat[e] the existence — and profound political consequences — of the invisible barbed wire”. We must expose the “occasionally antidemocratic character” of big data.
B. Learn how to sabotage the system: we must arm ourselves by resisting the trend toward self-monitoring and refusing to make money off our own data. “Privacy can then reemerge as a political instrument for keeping the spirit of democracy alive”.
C. Create “provocative digital services”: we must facilitate technologies that allow us to see the negative effects of data-mining. For example “instead of yet another app that could tell us how much money we can save by monitoring our exercise routine, we need an app that could tell us how many are likely to lose health insurance if the insurance industry has as much data as the NSA.”
D. Abandon preconceptions: we must cast-off any preconceptions we may have about how our digital services work and interconnect.
In short, what we need is some good old-fashioned, consciousness-raising and political activism.
I think there is merit to this. I think we could be more critical about the uses of technology. That’s why I find Morozov’s brand of anti-techno-utopianism somewhat refreshing. But I definitely worry that this is all just a bit too idealistic. If the forces of capitalism and bureaucracy are as powerful as Morozov seems to suggest elsewhere in the article, is some bottom-up activism going to be enough to overturn it?
In my article “On the Need for Epistemic Enhancement: Democratic Legitimacy and the Enhancement Project”, I suggested that broader use of enhancement technologies could help stave off the threat of epistocracy. I’m not convinced I’m correct about this, incidentally, but it does prompt the thought: would broader use of enhancement technologies also help stave off the threat of algocracy?
Probably not. Enhancement technologies might help to equalise the balance of power between ordinary humans and epistemically elite humans. Whether they could do the same for the balance of power between humans and algorithms is another matter. The gap is much wider. AIs have significant advantages over humans when it comes to data-mining and processing. The only kinds of enhancement technology that could bridge the gap are the truly radical (and hypothetical) forms. The ones that would give rise to a genuinely posthuman form of existence. Even if this form of existence is possible, concepts like freedom and self-determination may no longer have any meaning in such a world. Now there’s a thought…