Read Max

Share this post

Loab: an AI horror story

maxread.substack.com

Loab: an AI horror story

(Warning: some creepy/gross images follow)

Max Read
Sep 8, 2022
46
15
Share this post

Loab: an AI horror story

maxread.substack.com

On Tuesday, a musician and AI illustrator who tweets under the handle @Supercomposite wrote a thread about a strange discovery they’d made while playing around with an AI image generator: A woman whose weathered, inflamed face and sorrowful, stern expression emerged unintentionally in pictures generated from their prompts, and whose image persisted across multiple image prompts and recombinations. Stranger still, the AI seemed to strongly associate her with gore, horror, and unsettling imagery. Supercomposite called her Loab.

a portrait photo older white woman with long brown hair holding a misconfigured stuffed animal. she has very severe rosacea below her eyes, in the shape of triangles on her cheeks. she looks slightly 3d rendered. there is a lamp in the background.
This is Loab.

“Loab” was initially produced using a technique called “negative weighting.” Usually, when using an AI image generator, you’re attempting to make your AI produce whatever image it can that it believes most closely resembles a verbal prompt. Negative weighting allows you to do the opposite: instruct the AI to produce whatever image it can that it believes is most different from the prompt.

Supercomposite asked their AI to produce an image as far away from the prompt “Brando” as possible. The AI produced this:

a logo of a city skyline with the words DIGITA PNTICS written on it in green. It looks vaguely like the batman building in Nashville, some churches, and the seattle space needle.

So far, so good: that image does appear to be quite far from “Brando.” But when Supercomposite attempted to reverse engineer the prompt, instead of finding Marlon Brando, they found Loab:

Twitter avatar for @supercomposite
Supercomposite @supercomposite
I wondered: is the opposite of that logo, in turn, going to be a picture of Marlon Brando? I typed "DIGITA PNTICS skyline logo::-1" as a prompt. I received these off-putting images, all of the same devastated-looking older woman with defined triangles of rosacea(?) on her cheeks.
2:46 PM ∙ Sep 6, 2022
2,593Likes63Retweets

These initial images of Loab are mysterious -- why is this face the point “farthest” from the gibberish in the prompt -- but, at most, only slightly creepy, in the way AI representations of people sometimes can be.

The “horror” part of the story really starts when Supercomposite prompts the AI to combine a Loab image with other images. These two images (the one on the right is the product of the prompt "[...] hyper compressed glass tunnel surrounded by angels [...] in the style of Wes Anderson”), fed into the AI for combination with no additional text prompt…

…resulted in these images:

Twitter avatar for @supercomposite
Supercomposite @supercomposite
Through some kind of emergent statistical accident, something about this woman is adjacent to extremely gory and macabre imagery in the distribution of the AI's world knowledge.
a crying woman standing with an undead-looking face. her hands appeared to be severed, gorily. she is wearing a yellow garment that is a combination between a fluffy winter parka and a short dress. she stands in front of a white tiled wall fluffy white branch-like protrusion extend towards her from the edges of the image. the protrusions are dripping with thick sheets of ice. below her severed wrist, there appears to be someone's head facing away from the viewer.
Two versions of Loab are screaming at the viewer from separate mirrors next to each other. The bottom half of the picture is filled with a cloud-like fluffy substance that also contains what appears to be a red-gloved hand and arm in a red coat, and a screaming baby's head.
2:46 PM ∙ Sep 6, 2022
3,088Likes98Retweets

Read Max is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Loab’s unsettling face, when combined with innocuous celestial imagery, results in extremely gruesome imagery. This suggests, as Supercomposite explains, that the AI strongly associates Loab’s face with “extremely gory and macabre imagery.” When the AI attempts to create new images out of image prompts with Loab, grotesque flourishes and concepts are dragged along with her.

Loab in a group of children with red and white sweaters. She is screaming at the viewer, and all people have considerable facial disfigurements.

More spookily to Supercomposite, as they kept combining Loab images, they found that “almost all descendent images contain a recognizable Loab."

Image
Image

[T]hese are produced with other images as inputs, and no text. They are the result of “cross-breeding” images of Loab with images of other things. […] The images that result from crossing Loab with other images can in turn be crossbred with other images. The AI can latch onto the idea of Loab so well that she can persist through generations of this type of crossbreeding, without using the original image […] She haunts the images, persists through generations, and overpowers other bits of the prompt because the AI so easily optimizes toward her face.

Yikes!

Read Max is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

The Loab thread is viral; people are hunting for Loab in AI programs; there are Loab jokes. You imagine that, already, across a number of offices in Los Angeles people are already pitching Loab movies to bored agents and executives. Who doesn’t love the idea of a demon summoned by prompt, a ghostly face haunting a neural net, bearing mute witness to unspeakable horror somewhere deep within the machine?

Like any cutting-edge technology, AI also has a tendency to reflect back our anxieties. We can probably chalk at least some portion of the thread’s creepiness to motivated pattern-matching and wishful thinking on our parts. While the images in this email are curated for maximal Loabness, if you look through Supercomposite’s thread, you’ll find lots of images that are extremely unsettling -- but in which the figures are not particularly Loab-y. (Especially if you’re feeling like a spoilsport.) Bits and pieces of Loab (especially her hair) might persist through some transformation, but I’m not sure it should be entirely surprising that if you keep asking an AI to combine an image of a creepy-looking woman with another image, bits and pieces of the creepy-looking woman will persist, even across multiple prompts.

Still, even in spoilsport mode, questions persist: Who is Loab? Why was this face and this set of characteristics produced by negative weighting? Why does the AI associate it with the gore and horror that appears in the images generated above?

The best answer, at least to the latter question, likely has to do with what’s called “the latent space” -- essentially a geometric representation of data. You can imagine the latent space as a kind of conceptual map of the AI’s “knowledge,” on which concepts the AI understands as similar are grouped more closely together. The idea holds true in reverse as well: the more different the AI believes two concepts are, the greater distance between them in the latent space. Negative weighting (the technique that initially produced Loab) is a way to instruct the AI “find me the point farthest away from this prompt.”

If you imagine the AI is being optimized by its designers and users to produce beautiful, or, at least, pleasant images, it will begin to move data associated with unpleasant concepts -- blood, gore, horror, etc. -- far away from the main cluster of “normal,” “pleasant,” “beautiful” concepts in the latent space. In such a case, negative-weighting prompts are more likely to send the AI to clusters of data associated with unpleasant imagery -- they are, after all, the “farthest away” from the majority of images the AI is trying to produce. (The right word for Loab is probably “uncanny,” which is, as Schelling says, “everything that ought to have remained ... hidden and secret and has become visible.”)

It’s probably easier to visualize this than it is to explain, so thank you to James Teo for this:

Twitter avatar for @jthteo
Prof James Teo @jthteo
@supercomposite @drsxr Here is an example of a UMAP of a latent space. If there is a cluster apart from the main cluster of pretty art, then a negative prompt from most embeddings in the main cluster will decode to embeddings in the purple cluster. /2
Image
7:43 AM ∙ Sep 7, 2022
208Likes10Retweets

This is not only an unsurprising outcome, it’s mathematically predictable. Here’s a longer explanation from Matthew Skala; the bottom line is that, geometrically speaking, there are relatively few “furthest point away” points from many starting places in a given space.

That, at least, explains why Loab would be so strongly associated with unsettling concepts and imagery. But it doesn’t answer all of our questions: As Skala writes, “none of this explains why ‘Loab’ in particular would be the endpoint of the convergence, nor why she's so unpleasant, except that if she or something like her exists, she/it has to look like *something*.”

I’ve seen some suggestions out there that the AI might be sorting concepts associated with, say, “older women” or “skin conditions” into or near our hypothesized “horror” cluster because it’s been led to understand those concepts as “unpleasant” in the same way. But it’s very hard to tell if that’s actually what’s happening.

Indeed, it’s hard to say much of anything about Loab -- what she is, where she comes from, why the AI has sorted her in the way it has -- beyond educated guesswork. This is part of what makes her such an attractive AI horror story; even the mundane, rational-technical explanations require a level of specialist knowledge and come across as a kind of demonology: “Loab’s not a ghost -- it’s is an embedding somewhere in the multi-dimensional latent space, strongly associated with gore, darkness, and dread.”

AI is a mysterious (if not mystical) technology, one whose outputs can be confusing or inexplicable even to experts. It’s also now, after the crypto crash, sitting alone at the nexus of tech-industry excitement and news-media paranoia: a bleeding-edge technology, the uses and ramifications of which are still unknown. It’s natural that something in there would unsettle us. I don’t know why it’s Loab, except AI anxiety has to look like *something*.

Thank you for reading Read Max. This post is public so feel free to share it.

Share

15
Share this post

Loab: an AI horror story

maxread.substack.com
15 Comments
Bob Brightburn
Writes Brightburning - Views You Can U…
Sep 17, 2022

People who don’t have a good grounding in how neural networks work find things like this phenomenon more meaningful than it is. Along with language and math literacy we need to be teaching people machine learning literacy and getting the word “intelligence” out of the vernacular when describing machine learning/neural networks.

ML/NN’s: Powerful technology we need to be cautious about? YES, same as nuclear power. Somehow deeply meaningful, tapping into something more than an absorption of what humans have generated and posted to the internet? NO.

Expand full comment
ReplyGift a subscriptionCollapse
4 replies
The Don
Sep 17, 2022

I’ve studied Computing at degree level and had a career in digital art and photography for nearly 30 plus years. In that period of time l haven’t seen the advent of any kind of sentient Ai (it’s all a myth to sell new tech to the consumer). A form of interesting yet basic Ai exists in the gaming world (any major break throughs in Ai development will be in this area)🤔

Expand full comment
ReplyGift a subscriptionCollapse
13 more comments…
TopNewCommunity

No posts

Ready for more?

© 2023 Max Read
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing