"What it suggests is that Google’s problem here is not so much a misunderstanding of what LLMs are good at and what they’re for, but--more troublingly--a misunderstanding of what Google is good at and what it’s for."
Yes, yes, and yes. I've been working with web analytics and SEO for a long time, and while most people click on the top answer on the page without thinking about it because we have trained them Google is always right, many others prefer the list of links. Thank you for articulating the "I prefer a list of links" point of view because most people in search, publishing, and marketing think that if you're not at position 1, all is lost forever. But the data says plenty of people click on the archival links, and often.
It also seems that Pichai/Raghavan's vision of Google is starkly different from Page/Brin's vision, in that they are executives looking to make more money, versus idealistic grad students trying to change the world with the product they built. Not that Page and Brin aren't profit-motivated dopes, but with the company's most recent responses insisting that audiences are wrong in pointing out AI-overview errors, I don't think Pichai is fully on board with Don't Be Evil.
That's what struck me from Zitron's piece a couple of weeks ago: why was Raghavan panicked about getting more clicks in 2019? Google consistently has an 80-90% global market share. Does any other company have an 80% global market share of anything? (that is an honest question) But they are trying to get more money-making clicks because their research product doesn't make enough money somehow.
I don't know Google is making significant edits to their existing wildly popular and profitable research product except to seem cool and relevant for all the SV investors and colleagues who went gaga over ChatGPT. And there are likely business reasons that I don't understand. Because the tech industry is obsessed with going up and to the right forever and monetizing every incremental opportunity instead of building stable products for smart audiences.
I come to this from a slightly different angle but have landed in a similar place: I worked as a research librarian at a university for almost twenty years. We used Google a lot, and we used library research tools a lot, and I had a *lot* of dialogues with students (and occasionally professors) trying to find stuff. What really jumped out at me in Max's piece was the multiplicity of possibilities when someone has an ambiguous question--and students at the beginning phase of their research have lots of ambiguous questions! An LLM summary tries to close the question rather than offering possibilities for further exploration. It's intensely frustrating.
I no longer work in the library world but I still do a lot of searching, and what I've noticed lately is how often the result I want *isn't* in the top results set--I'm usually on page 2 or 3 before I even see something close to what I'm after, and everything before that is obvious spam, shopping sites, and ads. The AI summary just becomes another obstacle, and aside from quick factual queries (the time difference between Seattle and London, say), I'm increasingly not using Google at all.
That's what Midwest business folklore calls "creeping decrementalism"! Creeping decrementalism is when you deliberately change product quality and people naturally migrate away. For example, the company decides that there shouldn't be as much sugar in the cereal because sugar is addictive and bad for the general population, and the company is getting bad PR. So the company gradually removes sugar from the cereal recipe over the course of a few years, then five years later sales down 5% year over year because people have gradually stopped buying from the product, despite the slight improvement in public perception. (Can one Google "creeping decrementalism" and find a good result? Nope! Have I ever taken an econ class where I might have learned about such micro effects? No! But I swear to god it comes up in meetings and justifies budgetary decisions.)
If Google keeps adding garbage at the top of the SERP, people will stop using it, eventually, sure. But it takes an extremely time for a large organization to record that change because the volume of people who use Google for different reasons is soooo freakin massive. Maybe. I don't know what Google is trying to do. I think they are just messy in big corporate ways and don't have a north star besides proving to the DOJ they're not a monopoly (which they are).
But with the original post and this comment, I'm screenshotting your descriptions of how you use Google and using the anecdata as proof that yes, people do occasionally visit to the second or third page of search results to find what they want. There is absolutely a perception that people never visit anything but the top result in Google, despite all sorts of data to the contrary.
I've never heard that term before and yet I instantly recognized what you describe!
To add to your anecdata, a discussion I used to have in library research classes when we talked about web searching (which basically meant Google most of the time) was deliberately analyzing the first few pages of search results to see what kinds of entities/sources they were getting pulled from. Since I mostly worked with science and healthcare students we got a lot of government websites, NGOs, institutions such as the Mayo Clinic, with an increasing proportion of commercially published scientific literature over time as those platforms figured out that exposing their metadata and abstracts led to more engagement (and then I'd have a conversation with students about how to get that stuff through the library rather than paying directly for it).
Anyway, the point of that discussion was always to use the results to further target their research, and if they were getting a lot of stuff that seemed spurious or questionable, what to do about that. Google and to some extent other web search engines seem to encourage just going with that top result, and my intention was to encourage students to slow down, be thoughtful about what they wanted, and not take it for granted that what the search engine was showing them was necessarily the most suitable thing.
One thing worth calling out regarding Google's AI answers is how it represents a massive shift in their business strategy.
As a platform, Google connected people who wanted stuff from the internet (answers, whatever) with people who had that stuff, and they skimmed a bit off the top via sponsored results.
Implicit in AI answers is a desire to keep people on Google itself. They no longer want their users to click through to that link to Reddit, or whatever. They have a bunch of users, and they want to keep them there. In that sense they're now behaving much more like a social media company: Facebook, Twitter, etc, which of course are notoriously hostile to external links. But if the underlying ethos becomes keeping users on google.com, then the value of sponsored results would seem to diminish in value. Why would I the advertiser pay for a link that you are actively trying to keep people from following?
Yeah, and this is another aspect of the "platform problem," too—publishers are a really important group within the multi-sided marketplace that is Google Search, and by adding AI Overviews Google is (at best) violating the illusion of fairness and (at worst) cutting them out of the platform network entirely.
Max - thanks for another great newsletter. I'm intrigued by the last segment in particular about Whatsapp. It's possible that I'm just an elder millenial or maybe I just got my social groups tracked into gchat/hangouts/google chat too early, but I don't really understand the ways in which whatsapp is different from any old messaging app.
Isn't it just texting? Aren't you describing just a big group chat? I understand discord a little better, (as a sort of gamer-american) but does whatsapp have discord-like features with servers and channels? Even the few larger discord servers I've checked out just seem impossible to keep up with, though, so I don't fully understand how people have time to use them effectively once they hit a certain size. Even the example you cite about the 400-member whatsapp group sounds impossible to keep tabs on!
Is there anything you've seen written about how these messaging apps have become something more? Are there specific features they've added/changed to make them more able to serve as hubs of misinformation or organizing over time? Or is the thesis here just that any messaging app can become Like This? Again - I'm probably just old now, but I'm really curious about this question and want to understand it better.
The item is mostly just a joke--to the extent there's a point it's that the past decade of elite media anxiety the supposedly democracy-undermining features of social media has tended to focus on the public-facing platforms where ordinary people are gathering, consuming information, and organizing themselves (for better and often for worse), and not on the private spaces where actual elites are radicalizing and coordinating, often to much more specific and much worse aims than the vague problems of "misinformation" supposedly corrupting the electorate. WhatsApp itself is great; it's how I know anything is going on at my son's school!
Ha well I appreciate the reply and sorry I missed the joke!
I do think that there's something interesting about the way people sometimes talk about things like telegram, whatsapp, discord (recognizing these are slightly different things). They're just messaging apps but they do seem to have some other power beyond what just texting does.
Just chiming in since already on comments page, but Whatsapp is immensely popular in pretty much every country except the U.S. It's like the metric system of chat apps. What makes it different is just scale, and that Americans don't notice it as much.
What if people are spending more time with AI search because they are getting preposterous answers and either have to keep asking to get what they want or because they are getting a bunch of funny things to post. I don't understand how spending MORE time searching would be seen as better... other than for google because they can show more ads.
Kagi (a little boutique paid search engine that I am growing increasingly attached to) is still more bullish on LLMs than I would like, but it solves the summary/lying issue in a simple UX way- there's a 'summarize' button next to the link, for that link alone. The source is obvious, it's easy to follow into the rabbit hole if something feels off- it still feels like a search-adjacent function rather than some internet-eating oracular exercise.
I’m surprised there is no “here’s how to leave Google since it sucks now” discourse. Have you seen anything prominent? I’m not terminally online but I’ve heard at least four separate “AI Overview” summations, but nothing on how to change from Google to DuckDuckGo.
Yes I get the convenience lock-in of Android, Chrome, GMail, Google Maps, etc. If anything, Google Search is the least painful of these to change right?
Very insightful thoughts on AI search in the context of "neutral" platforms. I think this is all going to backfire spetacularly, just don't know when or how.
"What it suggests is that Google’s problem here is not so much a misunderstanding of what LLMs are good at and what they’re for, but--more troublingly--a misunderstanding of what Google is good at and what it’s for."
Yes, yes, and yes. I've been working with web analytics and SEO for a long time, and while most people click on the top answer on the page without thinking about it because we have trained them Google is always right, many others prefer the list of links. Thank you for articulating the "I prefer a list of links" point of view because most people in search, publishing, and marketing think that if you're not at position 1, all is lost forever. But the data says plenty of people click on the archival links, and often.
It also seems that Pichai/Raghavan's vision of Google is starkly different from Page/Brin's vision, in that they are executives looking to make more money, versus idealistic grad students trying to change the world with the product they built. Not that Page and Brin aren't profit-motivated dopes, but with the company's most recent responses insisting that audiences are wrong in pointing out AI-overview errors, I don't think Pichai is fully on board with Don't Be Evil.
That's what struck me from Zitron's piece a couple of weeks ago: why was Raghavan panicked about getting more clicks in 2019? Google consistently has an 80-90% global market share. Does any other company have an 80% global market share of anything? (that is an honest question) But they are trying to get more money-making clicks because their research product doesn't make enough money somehow.
I don't know Google is making significant edits to their existing wildly popular and profitable research product except to seem cool and relevant for all the SV investors and colleagues who went gaga over ChatGPT. And there are likely business reasons that I don't understand. Because the tech industry is obsessed with going up and to the right forever and monetizing every incremental opportunity instead of building stable products for smart audiences.
I come to this from a slightly different angle but have landed in a similar place: I worked as a research librarian at a university for almost twenty years. We used Google a lot, and we used library research tools a lot, and I had a *lot* of dialogues with students (and occasionally professors) trying to find stuff. What really jumped out at me in Max's piece was the multiplicity of possibilities when someone has an ambiguous question--and students at the beginning phase of their research have lots of ambiguous questions! An LLM summary tries to close the question rather than offering possibilities for further exploration. It's intensely frustrating.
I no longer work in the library world but I still do a lot of searching, and what I've noticed lately is how often the result I want *isn't* in the top results set--I'm usually on page 2 or 3 before I even see something close to what I'm after, and everything before that is obvious spam, shopping sites, and ads. The AI summary just becomes another obstacle, and aside from quick factual queries (the time difference between Seattle and London, say), I'm increasingly not using Google at all.
That's what Midwest business folklore calls "creeping decrementalism"! Creeping decrementalism is when you deliberately change product quality and people naturally migrate away. For example, the company decides that there shouldn't be as much sugar in the cereal because sugar is addictive and bad for the general population, and the company is getting bad PR. So the company gradually removes sugar from the cereal recipe over the course of a few years, then five years later sales down 5% year over year because people have gradually stopped buying from the product, despite the slight improvement in public perception. (Can one Google "creeping decrementalism" and find a good result? Nope! Have I ever taken an econ class where I might have learned about such micro effects? No! But I swear to god it comes up in meetings and justifies budgetary decisions.)
If Google keeps adding garbage at the top of the SERP, people will stop using it, eventually, sure. But it takes an extremely time for a large organization to record that change because the volume of people who use Google for different reasons is soooo freakin massive. Maybe. I don't know what Google is trying to do. I think they are just messy in big corporate ways and don't have a north star besides proving to the DOJ they're not a monopoly (which they are).
But with the original post and this comment, I'm screenshotting your descriptions of how you use Google and using the anecdata as proof that yes, people do occasionally visit to the second or third page of search results to find what they want. There is absolutely a perception that people never visit anything but the top result in Google, despite all sorts of data to the contrary.
I've never heard that term before and yet I instantly recognized what you describe!
To add to your anecdata, a discussion I used to have in library research classes when we talked about web searching (which basically meant Google most of the time) was deliberately analyzing the first few pages of search results to see what kinds of entities/sources they were getting pulled from. Since I mostly worked with science and healthcare students we got a lot of government websites, NGOs, institutions such as the Mayo Clinic, with an increasing proportion of commercially published scientific literature over time as those platforms figured out that exposing their metadata and abstracts led to more engagement (and then I'd have a conversation with students about how to get that stuff through the library rather than paying directly for it).
Anyway, the point of that discussion was always to use the results to further target their research, and if they were getting a lot of stuff that seemed spurious or questionable, what to do about that. Google and to some extent other web search engines seem to encourage just going with that top result, and my intention was to encourage students to slow down, be thoughtful about what they wanted, and not take it for granted that what the search engine was showing them was necessarily the most suitable thing.
I was hoping you'd bring some of your experience/expertise to bear here!
I mean, the subject line was catnip on a day of procrastination.
One thing worth calling out regarding Google's AI answers is how it represents a massive shift in their business strategy.
As a platform, Google connected people who wanted stuff from the internet (answers, whatever) with people who had that stuff, and they skimmed a bit off the top via sponsored results.
Implicit in AI answers is a desire to keep people on Google itself. They no longer want their users to click through to that link to Reddit, or whatever. They have a bunch of users, and they want to keep them there. In that sense they're now behaving much more like a social media company: Facebook, Twitter, etc, which of course are notoriously hostile to external links. But if the underlying ethos becomes keeping users on google.com, then the value of sponsored results would seem to diminish in value. Why would I the advertiser pay for a link that you are actively trying to keep people from following?
Yeah, and this is another aspect of the "platform problem," too—publishers are a really important group within the multi-sided marketplace that is Google Search, and by adding AI Overviews Google is (at best) violating the illusion of fairness and (at worst) cutting them out of the platform network entirely.
Max - thanks for another great newsletter. I'm intrigued by the last segment in particular about Whatsapp. It's possible that I'm just an elder millenial or maybe I just got my social groups tracked into gchat/hangouts/google chat too early, but I don't really understand the ways in which whatsapp is different from any old messaging app.
Isn't it just texting? Aren't you describing just a big group chat? I understand discord a little better, (as a sort of gamer-american) but does whatsapp have discord-like features with servers and channels? Even the few larger discord servers I've checked out just seem impossible to keep up with, though, so I don't fully understand how people have time to use them effectively once they hit a certain size. Even the example you cite about the 400-member whatsapp group sounds impossible to keep tabs on!
Is there anything you've seen written about how these messaging apps have become something more? Are there specific features they've added/changed to make them more able to serve as hubs of misinformation or organizing over time? Or is the thesis here just that any messaging app can become Like This? Again - I'm probably just old now, but I'm really curious about this question and want to understand it better.
The item is mostly just a joke--to the extent there's a point it's that the past decade of elite media anxiety the supposedly democracy-undermining features of social media has tended to focus on the public-facing platforms where ordinary people are gathering, consuming information, and organizing themselves (for better and often for worse), and not on the private spaces where actual elites are radicalizing and coordinating, often to much more specific and much worse aims than the vague problems of "misinformation" supposedly corrupting the electorate. WhatsApp itself is great; it's how I know anything is going on at my son's school!
Ha well I appreciate the reply and sorry I missed the joke!
I do think that there's something interesting about the way people sometimes talk about things like telegram, whatsapp, discord (recognizing these are slightly different things). They're just messaging apps but they do seem to have some other power beyond what just texting does.
It wasn't a very good joke and wasn't very well told so I appreciate the opportunity to explain it!
Just chiming in since already on comments page, but Whatsapp is immensely popular in pretty much every country except the U.S. It's like the metric system of chat apps. What makes it different is just scale, and that Americans don't notice it as much.
I was gonna say. I believe two things influenced the spread of WhatsApp abroad, or at least in Chile when I encountered it the first time:
- Android is much more popular outside the US, and it doesn't have a decent text messaging app.
- "Texting for free" on your WiFi was a huge draw back when WhatsApp landed.
What if people are spending more time with AI search because they are getting preposterous answers and either have to keep asking to get what they want or because they are getting a bunch of funny things to post. I don't understand how spending MORE time searching would be seen as better... other than for google because they can show more ads.
even setting aside the questionable opsec, managing a 400-person WhatsApp group sounds like my idea of hell
Kagi (a little boutique paid search engine that I am growing increasingly attached to) is still more bullish on LLMs than I would like, but it solves the summary/lying issue in a simple UX way- there's a 'summarize' button next to the link, for that link alone. The source is obvious, it's easy to follow into the rabbit hole if something feels off- it still feels like a search-adjacent function rather than some internet-eating oracular exercise.
I’m surprised there is no “here’s how to leave Google since it sucks now” discourse. Have you seen anything prominent? I’m not terminally online but I’ve heard at least four separate “AI Overview” summations, but nothing on how to change from Google to DuckDuckGo.
Yes I get the convenience lock-in of Android, Chrome, GMail, Google Maps, etc. If anything, Google Search is the least painful of these to change right?
Very insightful thoughts on AI search in the context of "neutral" platforms. I think this is all going to backfire spetacularly, just don't know when or how.