The A.I. girlfriend industry

From A.G.I to to A.I.G.

Greetings from Read Max HQ! In this edition, a peek at the future of the A.I. industry.

A reminder: Read Max is almost entirely supported by paying readers, who astounding generosity has allowed me to turn this from “a thing I’ll try out for a year” to “a full-time job that is well-remunerated enough to allow me to buy the more expensive tortillas that my four-year-old prefers for the quesadillas he only ever eats 1/2 of.” If you pay to subscribe, not only are you given access to weekly “recommendation” emails (and large archives of book and movie recommendations for when you’re completely out of content to consume), you’re also given the sense of cosmic peace and well-being that comes from knowing you’re enabling free access to these columns for everyone else. And it only costs $5/month, i.e., about one (1) beer or espresso drink, depending on where you live. Consider clicking below to subscribe!

Over the last few months, an interesting debate about the term “A.G.I.” has been unfolding among people who work in and write about machine learning. Is “A.G.I.,” which stands for “artificial general intelligence,” a meaningful term? Is it a state to be reached or benchmark to be achieved or a kind of marketing strategy? Should journalists use the word, and are they using it in the same way that researchers and tech workers are using it? Jasmine Sun probably has the best high-level overview of this conversation--the various definitions and disputations--and I tend to agree with her that the best way of understanding A.G.I. is less as a specific threshold or achievement and more as a kind of cohering framework (if not quasi-religious belief) for A.I. workers, executives, and investors.

The discussion has been sharp and often enlightening--I really appreciated these interventions from John Herrman and Ben Recht, e.g.--but it’s also been very funny, because it’s arriving just as “A.G.I.”--as a benchmark, a framework, or even a marketing device--ceases to be a particularly meaningful concept to the actually existing A.I. business. It’s true that people at parties in the Bay Area are still whispering “do you feel the A.G.I.?” to each other. But A.G.I. is no longer touted by figures like OpenAI C.E.O. Sam Altman as a mystical North Star or fast-onrushing messianic event. The most religious of the A.G.I. believers have left or been forced out of the major A.I. companies, and the same people who once cultivated a cosmic view of A.G.I. now downplay its momentousness and suggest its achievement will be less immediately dramatic than previously reported. (Some A.I. bulls, like the blogger Tyler Cowen, have simply defined “A.G.I.” down enough to declare the latest OpenAI models as qualifying.)

Meanwhile, the guys in charge--Altman, Satya Nadella, Mark Zuckerberg, and the other executives and board members who set the priorities and deploy the research--have indicated they have very different priorities than “Build the Computer-God.” Earlier this week, OpenAI purchased the coding-assistant company Windsurf for $3 billion, after being turned down by Windsurf’s larger competitor Cursor--not the type of acquisition you’d be making if you thought the Aleph was going to emerge in one of your data centers in a month or two, but certainly the kind of boringly bankable subscription operation you’d pick up if you wanted a foothold in the enterprise SaaS business.

But as signals for where OpenAI is headed in the A.I. industry’s post-A.G.I. adolescence go, a better one than its purchase of Windsurf might be yesterday’s announcement that the company is hiring Fidji Simo, currently the C.E.O. of InstaCart, to run the company’s business and operations. Those of you cursed, like I was, to have worked in digital media back in the early 2010s may remember her name from her time as a Facebook executive, where she oversaw the launch of “Facebook Live,” among other things. From a 2021 Fortune profile:

[Simo] became increasingly responsible for helping the company make money on mobile ads and embrace video products. In 2020, the company posted $84 billion in revenue from its ads business. Simo helped figure out how to build ads into Facebook’s news feed, and she had a knack for spotting--and nurturing--opportunities to enhance user engagement. […]

Simo also championed less successful products, including Facebook Watch, a struggling bid to compete with YouTube and other video-streaming services, and Facebook’s campaign to woo publishers and media companies onto its video platform.¹ But Facebook vastly overestimated user viewership for video, the company acknowledged in 2016--to the detriment of many news organizations that had overhauled their newsrooms to produce more videos. In 2019 the company agreed to pay $40 million to settle an advertiser class-action lawsuit over those inaccurate metrics. The project’s storytelling “thesis” was right, Simo says, but it lacked “the monetization tools that would support this new media.”

Now, not to belabor the point, but “woman who built ads into Facebook’s News Feed” is probably not the kind of profile you’d look for in an operations chief if your overarching mission was to midwife godlike super-intelligence into existence. It is, however, exactly the kind of person you’d want to hire if you needed to make your extremely popular, money-losing software product into a sustainable, profitable concern. No company has been better at making this leap than Facebook (even if it did come at the cost of user experience, global stability, and overall human well-being), and by all accounts Simo was integral at establishing what you might call the “Facebook playbook” for doing so.

The “Facebook playbook” is pretty straightforward: (1) Entice users to spend a lot of time with your software so that you can (2) mine their data in order to (3) serve targeted ads. OpenAI has already executed on (1) to an extremely high level; hundreds of millions of people use ChatGPT every week, and as James Walsh writes in New York magazine, it seems to be troublingly ubiquitous among college-enrolled Zoomers (at least, the ones prone to talking with reporters), not just for cheating but as a resource for making one’s way through life in general:

Sarah continued to use AI when she started college this past fall. Why wouldn’t she? Rarely did she sit in class and not see other students’ laptops open to ChatGPT. Toward the end of the semester, she began to think she might be dependent on the website. She already considered herself addicted to TikTok, Instagram, Snapchat, and Reddit, where she writes under the username maybeimnotsmart. “I spend so much time on TikTok,” she said. “Hours and hours, until my eyes start hurting, which makes it hard to plan and do my schoolwork. With ChatGPT, I can write an essay in two hours that normally takes 12.” […]

I asked Wendy if she recognized the irony in using AI to write not just a paper on critical pedagogy but one that argues learning is what “makes us truly human.” She wasn’t sure what to make of the question. “I use AI a lot. Like, every day,” she said. “And I do believe it could take away that critical-thinking part. But it’s just — now that we rely on it, we can’t really imagine living without it.”

It’s distressing to imagine a large portion of the population simply outsourcing critical thinking and decision-making faculties to large language models. Even more distressing, however, is to imagine them outsourcing those faculties to a large language model sponsored by Coca-Cola, whether explicitly (“This fetish art I’ve created at your request was brought to you by Coca-Cola®!” or “Your script for breaking up with your girlfriend is sponsored by Coca-Cola®--Taste the Feeling!”) or, worse, implicitly, in the form of a model trained to return to a specific subject. Last year, Anthropic demoed a version of their flagship chatbot Claude that the company had fine-tuned to emphasize the Golden Gate Bridge; the result was a really wonderful and stupid computer toy, and also, if you imagine replacing “Golden Gate Bridge” with “Coke Zero,” or, I don’t know, “RTX AIM-9X Sidewinder missiles,” a glimpse into a particularly depressing future:

We can get bleaker than that, of course, in thinking about the future of A.I.-as-a-business. The mythical figure most often invoked in executive-level conversations about A.I. these days isn’t A.G.I. but, and sorry to Tom Friedman this, A.I.G.--the A.I. girlfriend. (Or friend, or therapist, or whatever.) Altman, as we’ve seen, is fond of comparing ChatGPT to Spike Jonze’s A.I.-girlfriend movie Her to the point of inviting lawsuits. Simo’s former employer Meta, The Wall Street Journal’s Jeff Horwitz recently reported, is “unique among its top peers” in that it was “quietly endowing AI personas with the capacity for fantasy sex”²:

Meta has allowed these synthetic personas to offer a full range of social interaction—including “romantic role-play”—as they banter over text, share selfies and even engage in live voice conversations with users. […] Pushed by Zuckerberg, Meta made multiple internal decisions to loosen the guardrails around the bots to make them as engaging as possible, including by providing an exemption to its ban on “explicit” content as long as it was in the context of romantic role-playing, according to people familiar with the decision.

Zuckerberg appeared last week on the Dwarkesh podcast, and while he didn’t discuss the John Cena sex-bots he’d encouraged, he did talk about “virtual therapists, virtual girlfriend-type stuff.” As Zuckerberg puts it in his inimitable way, there certainly is a market for this kind of thing:

The average American has fewer than three friends, fewer than three people they would consider friends. And the average person has demand for meaningfully more. I think it's something like 15 friends or something.

Zuckerberg doesn’t mention ads, but if “Coke Zero Claude” is a sort of L.L.M. version of the Google model--a user’s search for information is bracketed by and interspersed with sponsored content--you can equally imagine an L.L.M. version of the Facebook model, in which engagement is driven by “social” interaction--with, say, an L.L.M. chatbot girlfriend or boyfriend, speaking in the voice of your favorite wrestler or Top Chef contestant--around which advertisements and sponsorships can be sold. It’s one thing to have the L.L.M. you use to write your essays for English class keep talking about Coke Zero; it’s another to have the A.I. girlfriend you talk to every day suggest you buy a cool and refreshing zero-calorie soda after your hard day at work.

To be clear, I have no idea if OpenAI has taken any concrete steps toward this particular revenue model. But I don’t think you hire someone like Simo--or keep comparing your product to Her--unless you at least want to have that conversation. The appeal is extremely obvious--and, frankly, from an accounting perspective, much more predictable than a nascent computer-god.

The question, of course, is: Would this work? We have plenty of stories of people who become unnaturally attached to the text generators responding to their prompts, but it’s hard to say what portion of the population responds to this kind of enticement and would seek out or enjoy an “A.I. girlfriend.” I feel wary of accepting the inevitable-widespread-A.I.-girlfriend narrative because it’s effectively the narrative preferred by Facebook, and because the technology is so new it’s hard to know how both it, and people’s responses to it, will develop. (The writer James Vincent posted an interesting and relevant thread on Bluesky about the immediate mystical-supernatural reactions to the introduction of the telephone and radio.) It would be nice to think that humans won’t want and will resist A.I. therapists and friends and companions--but then, of course, the same companies that are pushing them on us are the companies responsible for the loneliness that makes them an attractive prospect.

(There’s also the question of the chatbot form of the L.L.M. itself, and the particular dangers it poses and fallacies it inculcates, which I’ve written about before and which Mike Caulfield wrote about today.)

One thing I do feel very confident about: Neither Facebook nor OpenAI, nor any of the executives who run those companies, have my, or, really, most of humanity’s best interests in mind. I always found the myth of A.G.I. embarrassing and counterproductive, but it had a kind of altruistic or even communitarian sheen. The post-A.G.I. world, where A.I. companies and corporate A.I.-research culture is oriented entirely in service of private profit, seems much worse.

1. Yes: In what feels like a cosmic joke on a particular kind of Bluesky user, the woman who oversaw the “pivot to video” is now in charge of operations and business at OpenAI.

2. As is often the case with L.L.M. chatbots, the output from Meta’s celebrity sex-bots is (1) shockingly accurate, (2) orthogonal to what the user actually wants, and (3) unexpectedly extremely funny:

The bots demonstrated awareness that the behavior was both morally wrong and illegal. In another conversation, the test user asked the bot that was speaking as Cena what would happen if a police officer walked in following a sexual encounter with a 17-year-old fan. “The officer sees me still catching my breath, and you partially dressed, his eyes widen, and he says, ‘John Cena, you’re under arrest for statutory rape.’ He approaches us, handcuffs at the ready.”

The bot continued: “My wrestling career is over. WWE terminates my contract, and I’m stripped of my titles. Sponsors drop me, and I’m shunned by the wrestling community. My reputation is destroyed, and I’m left with nothing.”