Meta Pauses Terms That Would Let It Crawl Business Websites for AI. What Now?

On 5 November 2026, Meta indefinitely paused an update to its commercial terms that would have allowed it to crawl and use external websites linked to Meta business accounts for artificial intelligence. In a notice on the Meta Business Help Center, Meta confirmed the changes “are now paused and will not take effect on November 20, 2026” while consultations continue. The existing Commercial Terms from 6 March 2024 stay in force. But site owners should not relax: the policy has been paused, not withdrawn.

What the proposed terms would have allowed

First reported by Nieman Lab on 15 October, the update moved from a 30 October rollout to 20 November before Meta halted it. As detailed by Ars Technica, the text required businesses to authorise automated crawlers, bots, and scrapers to index external domains associated with their services and let Meta use content from those pages. Crucially, Meta could create test credentials, which experts warned could bypass paywalls. While earlier crawling provisions in Ad Creative Generative AI Terms applied only to ads, this applied to all business accounts without an opt-out.

Meta defended the change as a consolidation of scattered terms, asserting it had no plan to ignore robots.txt and that crawling practices remained unchanged. Meta later removed the word “train” from the draft, though critics argued that “develop” was just as broad.

Why Meta Backed Down

Publisher alarm was immediate because virtually every news organization operates a Meta business account. Lawsuits by media groups led courts in Norway and Brazil to issue preliminary injunctions against the rollout. In the United States, Senators Adam Schiff and Marsha Blackburn urged Mark Zuckerberg to withdraw the policy. Meta removed the preview on 5 November, with a spokesperson telling Nieman Lab that the pause lets Meta “take our time to get this right” during consultations.

Meanwhile, the UK Competition and Markets Authority launched a Competition Act 1998 investigation into potential abuse of dominance, saying the terms could let Meta collect content from business and publisher websites for products including its AI features “without those businesses getting a choice”. The pause has not ended that investigation.

Why Arab businesses should care

This matters to Arab businesses, online stores, creators, and news sites, which depend heavily on Facebook and Instagram pages to reach customers. In our view, good Arabic content is still scarce on the web compared with English, and that scarcity is exactly what makes it valuable to anyone training models, Meta included, which is deep in the model race with releases such as its open Muse Spark Open. Yet none of the reported legal actions came from an Arab country. Regional publishers are relying entirely on legal decisions made elsewhere. Two questions remain open: whether Meta-ExternalAgent has already crawled these sites for training (no source says), and when, or whether, Meta will bring the terms back and in what wording.

What to Do Now

Site owners should review their configurations against Meta’s official web crawler documentation:

• Open yourdomain/robots.txt and check whether it addresses Meta-ExternalAgent.
• Decide per crawler: blocking Meta-ExternalAgent concerns training, while blocking Meta-WebIndexer means Meta AI is less likely to cite and link you.
• Keep FacebookExternalHit unblocked: it builds the title and thumbnail preview when your links are shared.
• Note that FacebookExternalHit and Meta-ExternalFetcher may bypass robots.txt in some cases, such as security checks or user-requested tasks.
• Remember that robots.txt changes take up to 24 hours to take effect.
• Keep a copy of the current terms page and watch the Business Help Center notice for the next update.

# robots.txt
User-agent: Meta-ExternalAgent
Disallow: /

A reprieve, not a solution

Meta’s pause gives website owners breathing room, but mistaking a tactical delay for a settled victory would be dangerous. Platforms routinely test how much autonomy they can extract from creators before facing regulatory pushback. Because none of the reported challenges came from an Arab court or regulator, regional publishers cannot afford complacency. Arab site owners should set their crawler rules today rather than trust that Meta’s next revision will respect their content tomorrow.