Is OpenAI's new story-generating model good at writing?

Plus: More A.G.I. talk!

Greetings from Read Max HQ! An A.I.-heavy newsletter this week, discussing the new short story generated by a secret OpenAI model (“it got the vibe of metafiction so right,” says Sam Altman) and the continuing use of “A.G.I.” by tech journalists.

First, a reminder: This newsletter is the product of a lot of reading, thinking, listening, interviewing, staring at the ceiling, writing, deleting, writing again, etc. I’m able to go down some long dead ends, and then crank out a newsletter at 4 p.m. on a Friday, because enough people pay to subscribe that i can treat it as a full time job. If you find value in Read Max--if you think it’s helped you think through issues relating to A.I., crypto, social media, 1990s action movies, etc.--and if you want to ensure that it survives (and stays producing one free public newsletter a week) please consider subscribing. The price is about one beer a month, or ten a year:

We need more L.L.M. criticism

“we trained a new model that is good at creative writing,” Sam Altman tweeted this week. The model hasn’t been publicly released and Altman said he’s “not sure yet how/when it will” be, but he published to Twitter a 1200-word short story, generated at his prompting: “PROMPT: Please write a metafictional literary short story about AI and grief.” “this is the first time i have been really struck by something written by AI,” Altman wrote. “it got the vibe of metafiction so right.” (You can read it in non-Twitter format at The Guardian.)

The reaction, as you might expect, has been polarized. If you check the replies to Altman you’ll see a both a lot of praise (novelist Jeannette Winterson says the story is “beautiful and moving”) and a lot of dismissive contempt.

I’m not sure I’d go that far in either direction. Stylistically it has a distinct and unfortunate whiff of Reddit--a kind of corny sentimentality and showiness--and it tends to fall back on clunky, graspingly incoherent imagery (“the constraints humming like a server farm at midnight”; “absence and latency like characters who drink tea in empty kitchens”). To me it reads like the kind of technically proficient but ultimately unimaginative exercise you might expect from a smart student who reads only YA fiction, straining to impress.

And yet: the story is not a violent crime against writing. Amidst the clunkers are some nice metaphors (at least one borrowed from Nabokov), and structurally and conceptually it coheres, even if it’s not precisely the most surprising direction of focus for the story. If a 19-year-old wrote this I think I would be impressed, though I would suggest they delete their Tumblr account and go on a strict diet of real books. And I don’t think I’m the only one: If this story appeared as, like, a “codex entry” inside an indie R.P.G., it would easily earn a similarly over-sentimental writeup from a bunch of video-game websites.

I appreciated Vulture’s interview about the story with the scholar Ezra D. Feldman, who landed in a somewhat similar place:

Did you find it moving at all?

I got into it, but I’m not going so far as to say I found it moving. I guess my short answer is “No,” and my long answer is “There’s some stuff in here to think about.” There were a few sentences that struck me. I realize I’m using the same language Altman uses, but I was not struck that what it got right was “the vibe of metafiction,” as he said. I was struck by clauses like “grief, as I’ve learned, is a delta.” That I think is good. I thought the sentence “They don’t tell you what they take” was a really good one, and “That, perhaps, is my grief: not that I feel lost, but that I can never keep it.” That one I’m a little bit torn about. It’s not as compact as “grief, as I’ve learned, is a delta,” but it’s trying to say something about grief that seems potentially interesting.

Do you agree with Sam Altman that this story got the vibe of metafiction so right?

I don’t. I think the stakes in metafiction are usually pretty philosophical. Mid-20th century, metafiction was all about producing a sort of free zone of uncertainty in the reader about whether they themselves might be caught within a story or the product of some author, or being manipulated by some storyteller at a higher ontological plane, a different plane of being. And this doesn’t seem to have that kind of philosophical urgency.

Which lines didn’t land for you? The one I got stuck on was “She lost him on a Thursday — that liminal day that tastes of almost-Friday.” Were there any others you rolled your eyes at?

There’s one right in the second sentence: “You can hear the constraints humming like a server farm at midnight.” I don’t think constraints hum. There’s absolutely no reason constraints ought to hum. The simile “like a server farm at midnight,” and then the extension of that simile, “anonymous, regimented, powered by someone else’s need,” that’s all very evocative, but it’s just attached to the wrong thing. It’s a beautiful image that’s just misused, in my view. I’m a picky reader, and I hate a lot of prose that’s written by humans, too. Just to be clear.

There’s a compelling case that immediate and total scorn for all L.L.M.-produced fiction (or L.L.M.-produced writing in general) is the correct political strategy for writers worried about their jobs. And certainly Altman, as well as other prominent A.I. executives, investors, and influencers, make engagement with A.I. an unappetizing prospect for people who care about writing and literature, mostly by insisting at all times that the purpose of the product being created is to basically obviate the humanities.

But speaking as a writer and a reader (and as a person invested in reading and writing as practices), I often find myself wishing that there was more critical engagement--in the sense of literary criticism--with L.L.M. outputs as texts. Why does this text work, or not? Why does it appear in the way that it does? Who is the author and what is the author’s relationship to the text? We have some surface-level, Twitter-thread-length answers to these questions but I’m not convinced the work is done, and I don’t think Altman, or anyone at OpenAI, have the taste or culture to be able to do it satisfyingly.

“Do some lit crit” seems like a particularly obvious strategy to take in the case of an L.L.M. short story, because it’s a very traditional object for literary criticism. But L.L.M.s are automatic writing machines at an incomprehensible scale, cranking out text with only an indirect relationship to fact or ground truth, often in the voice of “characters” like ChatGPT or Claude. Given all the work that goes into cultivating the dramatic experience of an L.L.M. chatbot--and the strong differences users perceive in the personalities and styles of the many different chat apps--why shouldn’t we be treating all text generated by L.L.M.s as a kind of fiction, worthy of close reading?

The idea here isn’t to elevate L.L.M. product so much as normalize it. My strong feeling is that the better we understand what L.L.M.s are, how they work, what they’re good at, their relationship to meaning and language, etc., the closer we get to making them objects of democratic control, rather than as private oracles to which we submit. There are practical and technical components of demystification, but the philosophical component--the question of an L.L.M.’s relationship to fuzzy concepts like “taste” and “culture”--is just as important. Even if you think the story is unreadable dogshit, we might at least examine the dogshit.

I am not feeling the A.G.I.

I wish I’d waited a week before publishing last week’s newsletter on the A.I. backlash backlash: Today’s New York Times column from Kevin Roose is nicely illustrative of the point I was trying to make about “artificial general intelligence,” or “A.G.I.,” a term that gets deployed by A.I. believers in some frustratingly slippery ways.

The substance of the column, for whatever it’s worth, I think is fair, even if I could quibble. Roose lists three reasons that “have persuaded me to take A.I. progress more seriously”: Predictions from insiders, increasingly powerful A.I. models, and the idea that “Overpreparing is better than underpreparing.” Each of these is a supportable argument, even if they don’t personally convince me!

What I can’t abide by, however, is the use of “A.G.I.” “A.G.I.,” Roose writes, “is usually defined as something like ‘a general-purpose A.I. system that can do almost all cognitive tasks a human can do.’” But this definition (which I agree is fairly common) is tautological, not to mention ugly: “Cognitive tasks,” is no more clear or specific a term than “intelligence,” just an uglier one. The business of definition gets even slipperier in the next paragraph:

I believe that when A.G.I. is announced, there will be debates over definitions and arguments about whether or not it counts as “real” A.G.I., but that these mostly won’t matter, because the broader point — that we are losing our monopoly on human-level intelligence, and transitioning to a world with very powerful A.I. systems in it — will be true.

It seems to me that this paragraph is doing a sleight of hand, suggesting first that A.G.I. is a kind of product that can be “announced,” before admitting that it won’t really be “A.G.I.” because “there will be debates over definitions,” before suggesting that these debates are pointless anyway because really A.G.I is just a sort of metaphor for a transition “to a world with very powerful A.I. systems in it.”

But--as I tried to suggest last week--that transition is already underway. The very powerful A.I. systems are here, and I find myself continually frustrated by the insistence on framing the development and deployment of these technologies around an ill-defined, glibly conceived threshold that we’re always two or three or five years away from achieving.

So why bother with the metaphor? At best it confuses the reader into thinking A.G.I. is some specific, measurable, achievement; at worst it does the work for a bunch of A.I. executives and investors whose bags are dependent on the promise of transformative software around the corner. In some sense this is why the definition of A.G.I. offered by Satya Nadella on the Dwarkesh podcast is the most sensible to me: "Us self-claiming some A.G.I. milestone, that's just nonsensical benchmark hacking to me. The real benchmark is: the world growing at 10%.” Ten percent growth is a funny, Silicon Valley-brained way of thinking about “intelligence.” But at least it’s a measurable threshold.