Short story: Soul in the Loop
Marcelo Rinesi
2017-11-27 00:00:00
URL




Every shower she takes makes you more certain she will have killed herself before her daughter's tenth birthday, and these days she's taking one every time she logs off. You aren't allowed to tell her, but the NDA you made her sign has so many post-employment clauses she wouldn't be likely to find a job elsewhere anyway.

A daughter, two parents with Alzheimer, and the obsolete skillset of a radiologist and former e-sports semi-pro: She's as good a match for this job as any human could be, and only humans are allowed to. That's the point.

The politics of deploying killer robots require humans watching what they do to prevent them from doing the a posteriori unacceptable, but the business side of the equation — and somewhere in the company's software stack there's a piece of mathematics modeling just that — compels humans to barely if ever stop the robots from taking the shot. Armies don't pay for robots that don't shoot. The Oxford Protocol supervisors are there to ensure they could, theoretically, be stopped, and to suffer the legal consequences if it becomes convenient for somebody to.

So she logs in ten hours a day to watch the death of people she could have saved — people who might or might not be innocents, people whose names she'll never know — at the cost of risking homelessness for herself, her daughter, and two helpless people who once raised her and she still loves. She never stops a robot. She just takes a shower immediately after every session, the company's contract-mandated monitoring of home network logging it as another data point in her profile.

The company's behavioral prediction models indicate that compulsive showering correlates with late-stage burnout, which means you should start choosing a replacement for her from the vast and growing pool of the economically deprecated. Some of them would last longer than others, and some would actually enjoy their jobs. You always pick the ones who don't, the ones who eventually need a shower every time they log off, and sometime after that require a replacement of their own.

You understand the business case for this company policy, yet find ironic that you would be barred from doing the job you choose people for. But it's not like you don't enjoy your own.

.finis.