So the kids who did remote COVID high school are now ChatGPT-ing their way through university... will they be the dumbest generation to ever walk the earth? Something tells me these are the exact customers Mr Zuckerberg can't wait to capture
fwiw, sarah from the nymag article actually showed up in the reddit comments on some sub (same username as the one ID'd in the piece) and shared that she'd quit using chatGPT months ago (in part because of the reddit-shaming she received when she posted about it lol) and was really glad to have regained the confidence in her own writing she'd lost. she also mentioned that two of her first semester professors explicitly allowed chatGPT use as long as it was cited, which... i dunno, she seems to feel pretty positively about the experience of being interviewed but reading her comments did make me feel like the article selected its quotes to really present her as a nightmare gen z cartoon in a way that is attention grabbing but perhaps not entirely fair to her or entirely conducive to productive conversations on this genuinely important and concerning topic.
I'm curious if there are numbers on how many people are actually using these programmes regularly. I know a few people who use them, almost always for incredibly trivial and unnecessary things: rewriting a text that's already perfectly fine, for example, or getting information that could be found on Google. That said, those people are exceptions among my social circle, so I don't know how widespread this technology really is. It's certainly pretty bleak to think of people farming out decision-making and critical thinking to a glorified desk toy, but I'm wary of "kids these days" panics.
Still, even if the userbase is limited, it seems *extremely* irresponsible to allow the company that (allegedly) deliberately targeted teenagers at their most vulnerable to pivot to friendly sex-chat bots. It seems like nobody in power really wants to regulate this stuff, both because regulation in general is anathema to Anglophone elites and because nobody wants to be left behind when the Next Big Tech Thing rockets to the moon, as it supposedly will any day now.
One high school kid I teach recently turned in a LLM response to a prompt asking them to write an appeal of Walter McMillian’s death sentence in Just Mercy requiring text evidence from the book that instead used text evidence from…A Raisin in the Sun. No sir, Walter McMillan does not have a Nigerian boyfriend.
On the point of your story: the alarm bells really go off here when OpenAI “pivots to video,” as most high schools kids (the ones I teach) don’t engage a lot with chatbots since they don’t actually read. The kids who don’t read and spend all day on Tik Tok are screwed when a LLM FaceTimes them and starts selling them Coke.
There's a short story that I vividly remember reading like 30 years ago as a wee thing, of unknown but likely much older provenance, where a guy essentially has only screen mediated relationships, reads about the old ELIZA chatbot, and then has a horror movie revelation when he clocks his friend or girlfriend using some of the old ELIZA patter. Naturally it seems Zuck et al. read this story too, and, high on designer nootropics and the blood of a goat he killed as part of a fad diet, and a batch of Soylent cut with paint thinner, concluded it constituted a business plan.
It's been noted that "tech" moguls have a penchant for misreading dystopian fiction as aspirational. (E.g., remember when Elon Musk touted the "cybertruck" as "designed for Bladerunner"?)
There's a family resemblance to sociopathic politicos declaring themselves big fans of artists who despise everything they stand for (e.g., Paul Ryan declaring himself a big fan of Rage Against The Machine). I suspect that all of these guys will eventually be understood as mentally ill, in the increasingly unlikely event that human civilization survives long enough.
The key question remains, how can the AI girlfriend be monetized enough to recover the trillion dollars or so that has been spent or planned to be spent on AI training and the data centers to support it.
I’m skeptical they can recoup all of it, or that they’ll continue spending this aggressively over the next few years, but the runaway success of OnlyFans suggests there’s a market here
Coming in late here but I'm curious what you / others think about the efficacy of surveillance capitalism as a business model. Obviously it's super effective for marketing to advertisers, but do you feel it actually drives sales/ behavior in the way it's reputed to? Cards on the table, I find the thesis (and the book) a bit credulous. I'm sure well timed ads and deeper info can move behavior at the margins, but the idea that it's a magic bullet that can get people to buy (or vote for) anything seems overstated.
"Rarely did she sit in class and not see other students' laptops open to ChatGPT.": When I left academia 16 years ago, it was due to other shit, not because I saw this particular shit coming. However, I'm glad I don't have to deal with this.
On my way to graduate degrees, I taught - or attempted to teach - physics, math, and biology to hundreds of students on two campuses of the University of California. Many of them were, to be blunt, terrible, and not just the ones who flunked, either.* Are students generally worse in the ChatGPT era? I left, so I can't say, but someone should investigate. (Note that because most instructors grade on curves, grade distributions wouldn't necessarily change. A survey of experienced instructors would be more informative.)
"A.I. girlfriend" is a truly dismal concept, so yes, that's probably the direction Altman et al. are heading. (Per Jane Wagner and Lily Tomlin, "No matter how cynical you get, it's never enough to keep up.")
*As Tom Lehrer sang:
We will sleep through all the lectures and cheat on the exams
I just read parts of this piece out loud to my (non-A.I.) girlfriend, and two more thoughts occurred to me:
1. In a quiet, unimposing way, this may well be the most dystopian "Read Max" piece I've read.
2. "... as they banter over text, share selfies and even engage in live voice conversations with users ..." brings to mind the saying that if a thing exists, the corresponding fetish exists, so presumably, given that data centers full of server racks exist, there must be people who would find an LLM selfie erotically exciting. (Yes, I know the author didn't mean that, but it isn't any more absurd than what the author did mean.)
My girlfriend thinks the whole thing sounds bonkers and icky. True, that.
"I have no idea if OpenAI has taken any concrete steps toward this particular revenue model."
I got a targeted ad for various 'Capitalist Realism'-tier entry-level lefty texts on Amazon as a "helpful suggestion" to an only tangentially related question the other day, so it seems like for now the immediate move is more like the extension of that model than the 'MKULTRAing everyone into wanting Doritos' scenario.
So the kids who did remote COVID high school are now ChatGPT-ing their way through university... will they be the dumbest generation to ever walk the earth? Something tells me these are the exact customers Mr Zuckerberg can't wait to capture
Unfortunately, not just Zuck, but some folk over in DC as well...
Golden Gate Claude is the most relatable AI ever. I love his heartbreaking attempts to stay on task. You and me both, buddy. <3
Probably my favorite-ever piece of A.I. art, if it counts
fwiw, sarah from the nymag article actually showed up in the reddit comments on some sub (same username as the one ID'd in the piece) and shared that she'd quit using chatGPT months ago (in part because of the reddit-shaming she received when she posted about it lol) and was really glad to have regained the confidence in her own writing she'd lost. she also mentioned that two of her first semester professors explicitly allowed chatGPT use as long as it was cited, which... i dunno, she seems to feel pretty positively about the experience of being interviewed but reading her comments did make me feel like the article selected its quotes to really present her as a nightmare gen z cartoon in a way that is attention grabbing but perhaps not entirely fair to her or entirely conducive to productive conversations on this genuinely important and concerning topic.
This feels like the logical conclusion of your extremely on-point observation that “the point of AI is to talk to a cool computer.”
A chilling vision of things to come!
I'm curious if there are numbers on how many people are actually using these programmes regularly. I know a few people who use them, almost always for incredibly trivial and unnecessary things: rewriting a text that's already perfectly fine, for example, or getting information that could be found on Google. That said, those people are exceptions among my social circle, so I don't know how widespread this technology really is. It's certainly pretty bleak to think of people farming out decision-making and critical thinking to a glorified desk toy, but I'm wary of "kids these days" panics.
Still, even if the userbase is limited, it seems *extremely* irresponsible to allow the company that (allegedly) deliberately targeted teenagers at their most vulnerable to pivot to friendly sex-chat bots. It seems like nobody in power really wants to regulate this stuff, both because regulation in general is anathema to Anglophone elites and because nobody wants to be left behind when the Next Big Tech Thing rockets to the moon, as it supposedly will any day now.
One high school kid I teach recently turned in a LLM response to a prompt asking them to write an appeal of Walter McMillian’s death sentence in Just Mercy requiring text evidence from the book that instead used text evidence from…A Raisin in the Sun. No sir, Walter McMillan does not have a Nigerian boyfriend.
On the point of your story: the alarm bells really go off here when OpenAI “pivots to video,” as most high schools kids (the ones I teach) don’t engage a lot with chatbots since they don’t actually read. The kids who don’t read and spend all day on Tik Tok are screwed when a LLM FaceTimes them and starts selling them Coke.
There's a short story that I vividly remember reading like 30 years ago as a wee thing, of unknown but likely much older provenance, where a guy essentially has only screen mediated relationships, reads about the old ELIZA chatbot, and then has a horror movie revelation when he clocks his friend or girlfriend using some of the old ELIZA patter. Naturally it seems Zuck et al. read this story too, and, high on designer nootropics and the blood of a goat he killed as part of a fad diet, and a batch of Soylent cut with paint thinner, concluded it constituted a business plan.
It's been noted that "tech" moguls have a penchant for misreading dystopian fiction as aspirational. (E.g., remember when Elon Musk touted the "cybertruck" as "designed for Bladerunner"?)
There's a family resemblance to sociopathic politicos declaring themselves big fans of artists who despise everything they stand for (e.g., Paul Ryan declaring himself a big fan of Rage Against The Machine). I suspect that all of these guys will eventually be understood as mentally ill, in the increasingly unlikely event that human civilization survives long enough.
The key question remains, how can the AI girlfriend be monetized enough to recover the trillion dollars or so that has been spent or planned to be spent on AI training and the data centers to support it.
They don’t need to recoup it from customers. They can get it from future investors.
I’m skeptical they can recoup all of it, or that they’ll continue spending this aggressively over the next few years, but the runaway success of OnlyFans suggests there’s a market here
Coke Zero, RTX AIM-9X sidewinder missiles, or “white genocide” in South Africa as it were… https://www.theguardian.com/technology/2025/may/14/elon-musk-grok-white-genocide
hi
Coming in late here but I'm curious what you / others think about the efficacy of surveillance capitalism as a business model. Obviously it's super effective for marketing to advertisers, but do you feel it actually drives sales/ behavior in the way it's reputed to? Cards on the table, I find the thesis (and the book) a bit credulous. I'm sure well timed ads and deeper info can move behavior at the margins, but the idea that it's a magic bullet that can get people to buy (or vote for) anything seems overstated.
Just wait for when, instead of telling them which cold soda to buy, the virtual GF tells them who to vote for.
"Rarely did she sit in class and not see other students' laptops open to ChatGPT.": When I left academia 16 years ago, it was due to other shit, not because I saw this particular shit coming. However, I'm glad I don't have to deal with this.
On my way to graduate degrees, I taught - or attempted to teach - physics, math, and biology to hundreds of students on two campuses of the University of California. Many of them were, to be blunt, terrible, and not just the ones who flunked, either.* Are students generally worse in the ChatGPT era? I left, so I can't say, but someone should investigate. (Note that because most instructors grade on curves, grade distributions wouldn't necessarily change. A survey of experienced instructors would be more informative.)
"A.I. girlfriend" is a truly dismal concept, so yes, that's probably the direction Altman et al. are heading. (Per Jane Wagner and Lily Tomlin, "No matter how cynical you get, it's never enough to keep up.")
*As Tom Lehrer sang:
We will sleep through all the lectures and cheat on the exams
And we'll pass and be forgotten with the rest
(https://www.youtube.com/watch?v=dl3mRjydcPw)
I just read parts of this piece out loud to my (non-A.I.) girlfriend, and two more thoughts occurred to me:
1. In a quiet, unimposing way, this may well be the most dystopian "Read Max" piece I've read.
2. "... as they banter over text, share selfies and even engage in live voice conversations with users ..." brings to mind the saying that if a thing exists, the corresponding fetish exists, so presumably, given that data centers full of server racks exist, there must be people who would find an LLM selfie erotically exciting. (Yes, I know the author didn't mean that, but it isn't any more absurd than what the author did mean.)
My girlfriend thinks the whole thing sounds bonkers and icky. True, that.
"I have no idea if OpenAI has taken any concrete steps toward this particular revenue model."
I got a targeted ad for various 'Capitalist Realism'-tier entry-level lefty texts on Amazon as a "helpful suggestion" to an only tangentially related question the other day, so it seems like for now the immediate move is more like the extension of that model than the 'MKULTRAing everyone into wanting Doritos' scenario.
Modest of Max not to mention the years of research he put into this topic.