Photo via The Straits Times

AI giants promise publishers new way to block some scraping

Read the original at The Straits Times ↗
Cite

The Straits Times. (2026, October 31). AI giants promise publishers new way to block some scraping. The Straits Times. https://www.straitstimes.com/world/ai-giants-promise-publishers-new-way-to-block-some-scraping

Dated from the outlet's own feed. Check the original for its byline.

The Straits Times · collected 2026-10-31 · by The Straits Times

Quick Summary

On October 30, major AI companies including OpenAI, Google, Microsoft, and Anthropic announced plans to help publishers block content scraping more easily by providing a standardized command recognized by all participating firms. This new protocol aims to streamline the process of preventing AI models from training on copyrighted material without permission; however, full implementation is not expected until 2027 for some companies like Microsoft. The move comes amid legal challenges and growing concerns over unauthorized use of published content by AI systems.
Written by AI on 2026-11-01, from this article's own text only (the story summary below draws on all the coverage).

AI analysis

Story summary

Google, OpenAI, Microsoft, and Anthropic pledged on October 30, 2026, to provide publishers with a single standardized method to prevent their content from being scraped by AI models. Currently, publishers must use multiple blocking commands for each bot that collects data, but the new protocol will allow them to enter one standard instruction recognized and followed by any adopting AI company. Microsoft plans to implement this starting February 1, 2027, while Google aims to do so sometime in 2027; OpenAI and Anthropic have already begun or plan to roll it out soon. This announcement comes amid growing criticism and legal battles from publishers who are facing a surge in bots scraping their articles for AI training purposes. For instance, the New York Times sued OpenAI and Microsoft in December 2023 over allegations of using millions of its articles without permission.

Written for “AI Giants Aid Publishers Scraping” on 2026-11-21, grounded in this article and the 0 other(s) covering the same event.

Compared with similar articles

This article reads unscored and hedges 14% of its claims. Each row says how that neighbour differs.
New York Post
Leans strongly right Hedges 4 of 21 claims Outlet record 66/100
“While both articles discuss AI companies' interaction with news publishers regarding web scraping, Article A focuses on conservative groups warning against it in court filings, whereas Article B reports on a promise by AI giants to provide a new method for publishers to block scraping.”

Story

AI Giants Aid Publishers Scraping
Technology · 1 article(s) covering the same event.
How this article was measured signals, leaning charts, publisher record, byline, topics and claims

Signals How these are calculated →

Claims extracted
14
sentences that state something as happening or true
Uncertain
14%
2 of 14 hedged ("reportedly", "may", "according to")
Leaning
not political
takes no side on a contested political question
Publisher signals
64/100
The Straits Times's corrections and hedging, across everything collected from it. Not a measure of accuracy.
Outlets on this story
1
Technology
Same framing elsewhere
1
articles that frame the event the way this one does
Analyzed 2026-11-01 · how these are computed

How this is being covered How these are calculated →

Article leaning vs. publisher reliability
Source leaning vs. consistency

Publisher

The Straits Times · 5674 article(s) · 7 correction(s) detected
Running correction rate · 7 correction(s)

Who wrote this

The Straits Times
3966 article(s) here · 1 carrying a prediction
The environmental consequences of this war will last for generations,” Lewis Pugh wrote in a post on his website.
He predicts that pairing more use of these Wolbachia mosquitoes with existing steps to fight mosquito-borne disease could “knock dengue down to much lower levels” in many places.
The Estonian police said on Nov 20 the closure would “make it harder for Russian intelligence services to recruit and influence people, and help prevent sanctions evasion”.
2026-11-21 · assertive framing · Queues in Russian town as Estonia border closure looms
COP31 climate summit risks failure as talks go deep into overtime ANTALYA, Turkey, Nov 21 - The COP31 climate summit will either produce a good deal or no deal at all, the European Union's top climate official said on Saturday as talks stretched deep into overtime.
He was considered one of the leaders of the Beltran Leyva cartel, which had reportedly allied with the Los Mayors faction of the powerful Sinaloa Cartel.
Trump told European leaders they were spending a lot on Ukraine, Zelenskiy says KYIV, Nov 21 - US President Donald Trump told European leaders this week they were spending a lot on the war in Ukraine and could use that money at home if a peace deal were reached with Russia, Ukrainian President Volodymyr Zelenskiy said on Saturday.
Denmark would present its proposals at a meeting of EU ambassadors in Brussels on Monday.
NATO will defend every inch of Polish territory, Rutte says WARSAW, Nov 21 - NATO will defend every inch of Polish territory, NATO Secretary General Mark Rutte said in Warsaw, a day after Poland said Russia violated its airspace with three objects that were probably two missiles and a drone.
The cuts proposed by Antonio Costa, who chairs summits of EU leaders, would reduce the overall seven-year budget by €40 billion to €1.582 trillion compared to a proposal presented by the Irish EU presidency in October, and by €180.6 billion compared to the original Commission proposal.
Catherine, Britain’s Princess of Wales, will reflect on her experience with cancer and recovery in a documentary to be broadcast internationally in November, Kensington Palace said on Nov 21.
Also by The Straits Times
Nothing else under this byline is closely related to this article, so these are simply their most recent.
All 3966 articles by The Straits Times →

Topics

Anthropic ChatGPT Google Microsoft OpenAI

Subjects

Microsoft ORG · 5× OpenAI ORG · 3× Anthropic ORG · 2× Google ORG · 2× California GPE · 1× ChatGPT ORG · 1× Cloudflare ORG · 1× SAN FRANCISCO GPE · 1× Times ORG · 1× the New York Times ORG · 1×

Narrative

Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol.
framing: assertive · carried by 1 article(s) · first seen 2026-11-01
Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol.
2026-11-01 · The Straits Times
AI giants promise publishers new way to block some scraping · assertive framing

Claims (14 extracted, 2 hedged)

AI giants promise publishers new way to block some scraping AI generated SAN FRANCISCO, California – asserted
giants → promise → scraping
OpenAI, Google, Microsoft and Anthropic pledged on Oct 30 to give media outlets and other websites a single way to prevent their content from being scraped to train artificial intelligence models, amid growing criticisms and legal battles from news publishers. asserted
OpenAI → pledge → publishers
Publishers currently have to implement multiple blocking commands for every bot that crawls the web to collect data, but the new proposed solution will allow them to enter a single, standardised instruction that is recognised and honoured by any AI company that adopts the protocol. asserted
that → have → protocol
The solution will not fully take effect for months, though. asserted
solution → take → months
Microsoft will implement it beginning Feb 1, 2027, and Google sometime in 2027. asserted
Microsoft → implement → 2027
ChatGPT-maker OpenAI said it will roll it out “in the coming months”, while Claude-maker Anthropic said it has “already started” rolling it out. asserted
it → say → it
The announcement comes as publishers are facing a surge in bots that scrape their articles to feed AI models. asserted
that → come → models
Some, like the New York Times, have taken the matter to court. asserted
Some → take → court
In December 2023, the Times sued OpenAI and Microsoft for allegedly using millions of its articles without authorisation or payment to train ChatGPT. uncertain
Times → sue → ChatGPT
The two tech companies are invoking an exception to copyright law known as fair use. asserted
companies → invoke → use
Meanwhile, internet infrastructure provider Cloudflare has blocked AI bots by default on new customer sites since July 2025, and a standard called Really Simple Licensing already allows publishers to charge bots that visit their pages. asserted
that → block → pages
The new setting will apply only to future training of models but does not prevent bots from crawling otherwise, and content scraping will be allowed by default until the setting is changed by a publisher, Microsoft said in a blog post. Opting out of AI training could reduce the chances that a publisher’s content will surface in AI apps and experiences, Microsoft and Google said in separate blog posts. uncertain
Microsoft → apply → posts
“We believe creating a common signal that conveys such preferences to multiple providers at once is an important step. asserted
that → believe → providers
And we look forward to working with other AI companies on the standards and their implementation across the industry,” Meta spokesman Andy Stone wrote on X, conveying his support for the effort. asserted
Stone → look → effort
Give feedback · no sign-in needed

No sign-in needed for now. Your note is sent without a name, so it cannot be answered.

History Support