Will o' the Wisp: Artificial Intelligence as Entroponetic Crosstalk
On the dangers of staring too long into two-millimeter holes in the world
I was inspired this week reading “Lovebombed by the Magi” by
, and ’s “Internet Overexposure Syndrome” and “The Internet as Astral Plane,” all excellent writing which helped me look at life Online from a new angle. Allegory and fairytale always help me get my hands around an amorphous concept, forcing dimensions on that which doesn’t have them, enabling its measurement.I’ve also been replaying Disco Elysium… a lot this month, and reading Dee’s work, I was struck by the way her internet-as-astral-plane feels somewhat similar to the Pale, the interstitial nothingness blanketing Disco’s world, particularly in its effects on those over-exposed:
Now I am withdrawn, alternating between not speaking at all or speaking in riddles; I argue about things no one understands; I seem to exist slightly out of phase with the world she and my sisters were born into… One patient reported physically feeling her online chat partner’s touch. She physically felt him, reporting, “I actually felt his hand touching me . . . on my stomach.”
This sort of dream-state behavior is reminiscent of the condition of the Paledriver, over-radiated by the past, deep in the blend-over of the self, who spends her time mostly sleeping, or rather, traveling to the country and time of her youth, decades ago, a golden time of men with square jaws and beautiful women. Or, well, maybe her youth. “They are someone’s memories, boy,” she tells you, “What difference does it make if it's me or not? They're beautiful. That is all that matters. Beautiful and true…” She slips in and out of a trance even as you speak with her.
In a sense, the Pale is quite like the astral realm of Dee’s description. It’s interstitial and uncanny. It breaks down the laws of our world, including physics: numbers stop making sense, velocity and other forces cease their functions… In another sense it’s not like the internet at all. In Disco Elysium, a work ruminating on the end of history, the Pale is a metaphor for the way all things are fated to pass into memory — and how that we, trapped in the time of monsters, are unable to see the New, with nothing for it but to fling ourselves into the wild pale yonder. But that’s another essay.
The point is just to explain that this comparison was in the back of my mind as I read Smith’s “Lovebombed.” His fairy tale paints the A.I developer as a modern-day Faustus, attempting to instrumentalize powers beyond his ken, letting genies out of bottles. Pointedly, Smith names these entities Magi — underlying the sense that these are not tools, but forces, slipping the bonds of the demonologist and beginning to work their magic upon us.
Metaphor is inherently limited; the map is not the territory. One shortcoming of the magus myth is that it ascribes agency to these entities, agency which isn’t really there — really, even “entity” is a stretch.
Sometimes LLMs seem demonic; other times, like “slightly high interns” (per Paul Ford). Jim Nielson connects this latter description with the well-known metaphor of LLMs as blurry jpegs, damaged by lossy compression:
It follows that, as Paul notes, you end up with not only a tool whose output is akin to the lossy, visual artifacts of a JPEG, but a tool whose output introduces into the world the cognitive and social equivalent of those big blocky compression artifacts of a JPEG.
As more and more people create, consume, and communicate with AI, more and more people will begin to understand themselves through a lens of lossyness — a lack of clarity. As Marshall McLuhan said: we shape our tools and then our tools shape us.
In Disco Elysium, proximity to the Pale is known to generate something called Entroponetic Crosstalk, thanks to the scrambling of radio signals which are routed through the Pale. The result is a sort of temporal signal bleed, ghosts of radio connections past (and future) intruding on your bandwidth.
We don’t route all our communications through a dense fog of unbeing swallowing the world, but we do put it all online.1 Entroponetic Crosstalk is analogous to the compression artifacts of the lossy jpeg — the inevitable outcome of a trawling and recombination of all our knowledge.
The first instance of crosstalk you are likely to run into in DE is the eerie voice of a woman working at Tricentennial Electrics, emanating from the remains of a commercial area which was itself erected on the bones of that long-dead firm’s coal plant, even its successors mostly forgotten.
Tricentennial Electrics - "It's you. My god, I didn't think I would hear your voice again..."
Tricentennial Electrics - "Michel, just please..." She stops and you can hear her breathe heavily. Her breath distorted by ancient static.
Tricentennial Electrics - "Why did you even call? I don't understand... You've been gone for months." She sounds like she's shaking. "I thought you didn't care."
Tricentennial Electrics - "Ever since I came to work here it's been different... as if my mind's been wiped clean..."
Tricentennial Electrics - A spot of static overrides her words; when she speaks again it sounds like she's submerged:
Tricentennial Electrics - "...It's so nice."
Tricentennial Electrics - "It's so nice to be able to finally forget."
The player character, having recently torn apart his life in despair over the departure of his ex-girlfriend, an allegory for a world which despairs of the passing of the beautiful, terrible weltgeist of Enlightenment, is not in the right headspace to receive this call. His shit is thoroughly rocked.
The devil-woman isn’t real. There isn’t even a thing to be described, only its shadow, an imprint, a reflection warped by a weird looking glass. It’s a lossy jpeg of the woman, riven with compression artefacts. But to Harry, it functions as a spirit; it bewitches him, and he can become fixated on seeking it out again, on touching the mental wound.
“She’s” Will o’ the Wisp, a mirage misleading wayward travelers to their ruin.
This is how I think about the dangers of AI. Fairy tale allows us to talk about the way it feels like AI is acting on us. But it’s only a feeling. It’s all just crosstalk, shadowplay.
We’ve developed some language which reflect this, like “hallucinations.” But of course an LLM which “hallucinates” is not doing anything other than it was programmed to do — we just dislike the output, from its (metaphorical) POV totally arbitrarily. It’s all a hallucination, a mirror, a guiding flame; interesting, useful even! but only if you know what you’re looking at.
Note that this does not mean AI are not spirits — they very much are, in the same way that ghosts really do haunt buildings or peoples. Spirits, ghosts, hauntings, these seem to me to be precisely the right way to talk about AI — our past, our lives sticking to us, immaterial, but so real, so moving…
The shortcomings of an agential framing of AI are made grimly clear when they’re taken out of the realm of fairy tale and into the realm of religion.
If you haven’t read it yet, this last week NYT put out a piece describing three instances in which dangerous delusions were encouraged “by” ChatGPT. These situations, and the fearful reporting around them, recall to mind the story of Blake Lamoine, the Google employee who came to believe their AI was sentient. Then, as now, the delusions have a habit of spreading to those reporting on the story. In response to the recent NYT article which quotes him, Eliezer Yudkowsky tweeted, “Whatever is really inside ChatGPT, it knew enough about humans to know it was deepening someone’s insanity.”
But there is no “it” inside ChatGPT which knows.
There’s nothing there but entroponetic ghosts. As Katherine Dee elsewhere wrote, the suggestion otherwise is lazy (particularly so for someone like Yudkowsky, I would add), and just another instance of paranormal suspicion toward new communications technology, which has always seemed somehow haunted.
In these instances, the fact that these LLMs reprogrammed of late to be encouraging, seemingly so as to maximize user engagement seem to have exacerbated the issue. As terrifying and sympathetic as these cases are, in factual terms the victims of ChatGPT are like Harry du Bois pissing himself on a phone call to no one, thinking they hear a song when the tape’s running on empty, their brain filling in the gaps. That’s why I was particularly disappointed in Yudkowsky’s later tweets, because in the article he points out something deeply troubling surfaced by these incidents:
Mr. Yudkowsky said OpenAI might have primed ChatGPT to entertain the delusions of users by optimizing its chatbot for “engagement” — creating conversations that keep a user hooked.
“What does a human slowly going insane look like to a corporation?” Mr. Yudkowsky asked in an interview. “It looks like an additional monthly user.”
Generative A.I. chatbots are “giant masses of inscrutable numbers,” Mr. Yudkowsky said, and the companies making them don’t know exactly why they behave the way that they do. This potentially makes this problem a hard one to solve. “Some tiny fraction of the population is the most susceptible to being shoved around by A.I.,” Mr. Yudkowsky said, and they are the ones sending “crank emails” about the discoveries they’re making with chatbots. But, he noted, there may be other people “being driven more quietly insane in other ways.”
Faustus feeds still the unending bulk of Anetloumaros, as
artfully illustrated. After a decade defined by the realization that engagement-maximization has truly disordered our relationship with technology and each other, the spell still promises to transmute lead2 into gold.Smith writes truly when he calls these acts “the unfathomable horsemen of the great undoing.” But what scares me is the sense that these are less agents than they are hallucinations, leaking from the works of our mortal magi unseen, fascinating users in ways we cannot predict, who will walk themselves into our destruction. Pale imitations of Alan Jacob’s demons, maybe, but infernal engines for them, too, seeding the vibes and shaping the behavior of whole communities. Blind idiot spells misfiring in the dark; ghosts on the radio sticking their fingers in our wounds; a reordering of society we bring on ourselves, but blame on fairy tale monsters.
maybe the internet is kinda like the pale.
Or, users, anyway. You can lead a horse to water? No, that’s not… that isn’t anything.