Search was fragmented. Google centralised it. AI is breaking it apart again.

Web search has had exactly one settled era, and it's ending. For most of the 1990s, there was no single dominant way to find something online.

Then, for roughly twenty-five years, there was effectively one ruleset that mattered: Google's. That consolidation is breaking apart again now — the interface layer where answers get assembled, fragmenting across ChatGPT, Claude, Perplexity and Gemini.

In the mid-1990s, picking a search engine meant picking a genuinely different philosophy.

Yahoo started in 1994 as a human-curated directory — real people sorting websites into categories, the digital equivalent of a phone book.

AltaVista, launched by Digital Equipment Corporation in December 1995, took a different approach entirely: full-text indexing at a scale that was extraordinary for the time, ranking heavily on keyword relevance.

Excite, Lycos, WebCrawler, HotBot and Ask Jeeves each carved out their own variation between those two poles.

Neither dominant model had a reliable way to measure whether a page actually deserved to rank.

Human curation didn't scale with the web's growth, and keyword relevance was trivially gameable: repeated terms crammed into meta tags, white text on white backgrounds, invisible text readable by crawlers but not visitors, and pages built primarily around the vocabulary an engine was known to reward.

By the late 1990s, using many of these engines meant wading through results that had been deliberately manipulated to appear relevant.

AltaVista in particular became notorious for exactly this problem. Its enormous index was technologically impressive, but indexing more of the web did not solve the problem of deciding which pages deserved to come first.

Google, launched by Larry Page and Sergey Brin in September 1998, introduced a third model.

Rather than trusting what a page claimed about itself or relying on a person to catalogue it, PageRank measured what the rest of the web said about that page, treating a link as a vote of confidence weighted by the authority of whoever cast it.

It was a genuinely difficult signal to fake at the time.

And Google's results were dramatically better because of it.

Instead of asking only whether a page contained the right words, Google could ask something closer to: does the rest of the web appear to think this page matters?

On 26 June 2000, Yahoo named Google its default web-search provider, layering Google's ranked results underneath Yahoo's own directory and navigational system.

That decision captured the shift happening across the web. Human curation could still organise parts of the internet, but algorithmic search had become necessary to deal with its scale.

Google had solved a problem its competitors had not.

But better ranking technology explains how Google entered the market.

It doesn't fully explain how Google came to dominate it.

Search quality attracted users away from AltaVista and the large portals.

Google's business model then turned that quality advantage into something increasingly self-reinforcing.

Advertising made every query commercially valuable when Google launched AdWords in October 2000. Search no longer needed to be merely a useful gateway into the web; it could be an extraordinarily profitable product in its own right.

Distribution mattered too.

The Yahoo agreement put Google results in front of people who had not necessarily chosen Google themselves. Later default-search arrangements with browser companies, device manufacturers and other distribution partners made Google the path of least resistance into the web for an increasing proportion of internet users.

More usage generated more query and interaction data, creating further opportunities to improve search and advertising products.

More revenue funded better infrastructure, faster crawling, more sophisticated ranking systems and further distribution.

Each part reinforced the others.

And importantly, this consolidation happened gradually rather than overnight.

Google accounted for roughly 36.5% of US searches by mid-2005, ahead of Yahoo at around 30.5% and MSN at approximately 15.5%, but still some distance from the overwhelmingly dominant position it would later occupy.

By 2008, Google's share had climbed past 60%.

The fragmentation of the 1990s was steadily consolidating around one increasingly dominant search engine and, consequently, one increasingly dominant set of rules governing online visibility.

That distinction — quality helping Google win the market initially, followed by distribution, data and capital compounding that advantage — becomes important much later in this story.

In August 2024, a US federal court would find that Google had unlawfully maintained its search monopoly through exclusive distribution agreements.

But long before regulators reached that conclusion, the commercial reality for businesses had already changed.

Once Google became the default route into the web for most users, Google's rules stopped being merely one technology company's preferences.

They became, functionally, the terms of doing business online.

By the early 2000s, a retailer's organic ranking had become a real revenue line, not a marketing experiment.

That dependency is what made Google's November 2003 Florida update land so hard.

SEO at the time bore little resemblance to what most practitioners would consider acceptable today. Heavy keyword optimisation, reciprocal linking, doorway pages, hidden text and aggressive link acquisition were common across parts of the industry because Google's systems still rewarded enough of those behaviours to make them commercially worthwhile.

Florida became widely associated with Google's first serious attempt to reduce the effectiveness of heavily over-optimised commercial pages.

It shipped roughly two weeks before the peak of the holiday shopping season.

For retailers, that timing turned an algorithm update into a direct commercial event.

Businesses that had spent months building rankings around the techniques Google had previously rewarded or tolerated suddenly found those rankings disappearing at the most valuable point of the year.

Contemporary reporting and site-owner accounts documented apparent false positives, with legitimate retailers reporting substantial organic traffic losses alongside the manipulative sites the update was intended to catch.

There was no modern Search Console diagnosis explaining why an individual site had fallen.

No detailed list of changed signals.

No simple way for a business to distinguish between:

• a genuine quality problem,

• an algorithmic classification error,

• a competitor becoming more relevant,

• or an entirely new ranking system behaving differently.

There was simply a different search result and, for some businesses, a much smaller revenue number attached to it.

Florida's long-term importance wasn't the particular techniques it weakened. Most were eventually going to become unsustainable anyway.

Its real legacy was demonstrating something more fundamental:

an acquisition channel a business had come to depend on could change suddenly, without warning, according to rules controlled entirely by somebody else.

Every major enforcement episode that follows is a variation on that same exposure.

Panda began rolling out in February 2011 and affected roughly 12% of English-language queries in its first release — unusually broad reach for a single ranking change.

Rather than addressing one obvious technical manipulation, Panda attempted to identify patterns associated with low-quality sites.

Google never published the classifier's complete signal set.

But its public guidance increasingly emphasised questions such as whether readers would trust a site, whether articles contained original analysis, whether content appeared mass-produced, whether pages provided substantial value compared with competing results, and whether a user would feel comfortable trusting the site with sensitive information.

The consequences could extend beyond an individual bad page.

If enough of a domain exhibited low-value characteristics, visibility could decline more broadly, potentially affecting stronger pages alongside weaker ones.

That changed the unit of SEO risk.

A business was no longer necessarily being judged only on whether this particular URL answered this particular keyword well enough.

Google was increasingly asking what kind of publisher the site appeared to be as a whole.

Content farms were hit hard.

But that broader classification also created the potential for collateral damage.

Community websites, user-generated resources, forums and niche publishers could share superficial characteristics with the businesses Panda was designed to suppress: short pages, repetitive templates, large quantities of user-created material or high advertising density.

Google subsequently invited feedback from webmasters who believed their sites had been affected incorrectly.

Recovery was also difficult because early Panda versions did not continuously reassess sites in real time.

A publisher could remove or rewrite large quantities of material and still need to wait for another Panda refresh before knowing whether the work had made any difference.

That gap between corrective action and measurable recovery could last months.

Panda was eventually integrated more deeply into Google's core systems.

But its conceptual legacy remained:

Google had moved from judging individual ranking tactics toward judging entire patterns of publishing behaviour.

Panda was principally a content-quality story.

At almost exactly the same time, another much older weakness in Google's system was becoming impossible to ignore.

Links.

In February 2011, the New York Times investigated why J.C. Penney was ranking extremely well for a striking range of valuable generic commercial searches.

Thousands of links were discovered across unrelated and often low-value websites, pointing towards J.C. Penney pages using unusually precise commercial anchor text.

It looked exactly like what it was designed to accomplish: manipulate the PageRank-based authority signal Google had originally built its success around.

Google manually intervened.

J.C. Penney's rankings for affected queries fell sharply.

The retailer dismissed its SEO agency and said publicly that it had not authorised or been aware of the link placements.

Whatever the exact chain of responsibility, the important commercial lesson was straightforward:

Google's consequence attached to the domain benefiting from the manipulation, regardless of how much visibility the brand itself had into how those links had been created.

The incident also exposed something bigger.

Paid and manufactured links were already prohibited by Google's guidelines.

The problem wasn't the absence of a rule.

It was enforcement capacity.

Google could manually investigate individual high-profile websites.

It could not manually review every backlink profile on the internet.

That created an enforcement gap.

And an entire industry had grown inside it.

Penguin launched in April 2012.

It is sometimes remembered as the update that made paid links or manipulative link building against Google's rules.

It didn't.

Those rules already existed.

Google had publicly discussed paid-link enforcement for years.

What Penguin changed was the scale and consistency with which link manipulation could be identified algorithmically.

Link brokers, low-quality directories, reciprocal schemes, exact-match anchor campaigns and other methods had become part of mainstream SEO partly because they continued to work despite being officially discouraged or prohibited.

A genuine commercial ecosystem had developed around exploiting the distance between Google's written policy and its ability to enforce that policy.

Agencies sold the tactics.

Clients bought SEO campaigns without necessarily understanding precisely how authority was being manufactured.

Link networks sold placements.

Training courses taught exact-match anchors and scalable acquisition techniques.

Penguin substantially narrowed that gap.

And because the system evaluated link profiles businesses had accumulated over years, the consequences landed on historical decisions as well as new ones.

Recovery could be painfully slow.

Google introduced its Disavow Links tool later in 2012, giving site owners a mechanism to identify backlinks they wanted Google to disregard.

But Penguin initially ran through periodic updates rather than continuously.

Removing or disavowing manipulative links did not necessarily produce an immediate reassessment.

Businesses could spend months cleaning up a profile and still have no certainty about whether the work had succeeded until another Penguin refresh occurred.

Eventually, Penguin 4.0 in September 2016 changed the model significantly.

Penguin became real-time and more granular, with Google increasingly devaluing spammy links rather than using them to demote an entire site.

Recovery could therefore happen as Google's crawlers reassessed the affected links rather than waiting years for a named refresh.

But the wider lesson had already been established.

Google had created a valuable signal.

An industry had learned how to maximise it.

A grey and black market had developed around the gap between policy and enforcement.

And when Google's ability to enforce the rule finally caught up, businesses carrying years of accumulated optimisation decisions absorbed the cost.

Full piece: Florida, Panda and Penguin — When Google's Enforcement Hit Real Businesses

The Knowledge Graph, launched in 2012, pushed this transformation further.

Google increasingly moved beyond treating searches as strings of words and towards understanding entities — people, organisations, products, places and concepts — together with the relationships between them.

Instead of merely returning documents about an entity, Google could assemble structured information about it directly within Search.

Knowledge panels made this visible to users.

Featured snippets extended the principle to ordinary informational queries by extracting an answer from a source and placing it above the conventional organic results.

People Also Ask expanded the search journey into a network of related questions without requiring the user to leave Google between them.

Calculators, conversions, sports results, weather information and other direct-answer features resolved still more queries without a conventional website visit.

The publisher bargain was already changing long before generative AI.

Publishers gave Google permission to crawl and index their information because being indexed created discoverability and traffic.

Google increasingly used that information to make its own results page more useful, sometimes reducing the need for the click that had historically made the exchange commercially valuable to the publisher.

AI Overviews did not invent that tension.

They dramatically expanded it.

Entity understanding also changed what it meant for a brand to be visible.

A business could have perfectly optimised pages yet still be poorly understood as an organisation, product provider or category participant.

Visibility increasingly depended not only on ranking documents, but on how Google's wider systems represented the entity those documents described.

At the same time, Google's ranking systems became better at understanding language itself.

The practical consequences for SEO accumulated gradually.

Exact-match keyword targeting became less necessary.

Creating one page for every slightly different phrase became less useful.

Topical relationships became more important.

A comprehensive page could rank across many semantically related searches.

Search optimisation increasingly became an exercise in satisfying an underlying information need rather than reproducing a query string.

Google also became increasingly concerned with whether a result was merely relevant or genuinely trustworthy.

The August 2018 core update, nicknamed "Medic" by the SEO industry, became particularly associated with volatility across health, finance and other topics where inaccurate information could materially affect people's lives.

Google never confirmed a specific "Medic algorithm".

But industry attention increasingly focused on Expertise, Authoritativeness and Trustworthiness — E-A-T — concepts described in Google's Search Quality Rater Guidelines.

Those guidelines did not function as a direct checklist of ranking factors.

They described the qualities Google wanted its systems to recognise.

The Product Reviews Update, beginning in 2021, continued this direction by attempting to reward reviews demonstrating deeper research and first-hand product knowledge over thin affiliate pages that merely summarised manufacturers' specifications.

The Helpful Content Update in August 2022 made the people-first distinction more explicit still.

Google said it wanted to reduce the visibility of content created primarily to gain search traffic rather than genuinely help an intended audience.

Four months later, Google added another "E" — Experience — to its quality framework, distinguishing first-hand experience from more formal expertise.

That progression foreshadowed the language Google would use again in its 2026 guidance for generative Search: originality, first-hand knowledge, unique perspective and content that adds something beyond the commodity information already available everywhere else.

Classic SEO already contained versions of this problem.

Featured snippets could answer a simple question above the results.

Knowledge panels could satisfy an entity query without requiring a website visit.

Weather boxes, calculators and other direct features frequently resolved an intent entirely within Google.

Generative answers make those outcomes central rather than peripheral.

A brand can now receive:

A mention without a citation

A citation without a click

A recommendation without a measurable session

Visibility based entirely on a third-party source ** An inaccurate representation with no obvious mechanism for correction**

Influence over a buyer without ever appearing in conventional analytics

Research from Pew found users clicked a traditional Google result during around 8% of visits in which an AI summary appeared, compared with roughly 15% when one did not.

That difference matters.

But click data only describes one part of the behaviour.

It cannot tell you whether somebody:

• remembered a cited brand,

• trusted an answer,

• shortlisted a vendor,

• discussed the recommendation internally,

• searched the company directly later,

• or converted through another channel days or weeks afterwards.

That creates a measurement problem as important as the traffic problem.

For most of Google's dominant era, businesses could reasonably treat Google visibility as a proxy for search visibility.

That is becoming less defensible.

ChatGPT, Claude, Perplexity and Google's own generative products do not necessarily retrieve the same sources, perform the same query expansion, make the same citation decisions or represent brands in the same way.

Some combine proprietary crawling, model knowledge and third-party search infrastructure in ways that are only partly disclosed.

Behaviour can also vary within the same product depending on whether live web retrieval is invoked, which model is in use, and what kind of question has been asked.

A high organic ranking in Google therefore does not guarantee the same company will be visible in ChatGPT.

Being heavily cited in Perplexity does not guarantee Claude will choose the same sources.

A strong company website does not guarantee the assistant will derive its understanding of that brand from the company's own pages.

This is a fundamentally different optimisation environment.

Full piece: AI Overviews and the Publisher Bargain

All of this is happening while Google's control over search is under unprecedented legal scrutiny.

In August 2024, a US federal court found that Google had unlawfully maintained a monopoly in general search through exclusive distribution agreements.

The important distinction is consistent with the earlier history.

The ruling was not that Google had originally become successful because its search technology was bad or because users did not want it.

Google's early product was genuinely better than many competitors.

The legal question concerned the mechanisms through which that advantage was subsequently maintained.

In September 2025, the court imposed behavioural remedies, including restrictions on exclusive default-search arrangements and requirements to share certain search data with qualifying competitors, while rejecting the more severe structural breakup sought by the US Department of Justice.

Google is appealing the underlying monopoly ruling.

This does not mean algorithm changes themselves are illegal, or that AI Overviews are inherently anticompetitive.

It does provide important context.

A company that began as one competitor in a fragmented search market eventually accumulated enough control over the route through which people access online information that courts are now actively examining the mechanisms sustaining that control.

And just as that happens, the interface through which people discover information is beginning to fragment again.