Resources · Standards

What each AI standard does. Thirteen standards, eight jobs.

Thirteen open standards tell AI systems something about your content: what it is, who may fetch it, what may be done with it, what it costs, and what happened after. In a vendor pitch they can sound interchangeable, but they do eight different jobs, and they stack. Two of them move money: x402 and MPP.

Updated

Declaring is not the same as being paid

Ask your vendors. Put this to your CDN, bot-management vendor or marketplace, verbatim: "Which of these standards do you read, which do you enforce at the door, and who is paying under any of them through you today?"

Thirteen standards, eight jobs

A machine that buys content has to find it, identify itself, learn the terms, pay and report what it did. Each standard below does one or two of those jobs. The companies that sell access are on Content marketplaces, which also explains how a Broker orders offers for agents.

StandardIts job, and what it answersStatus, and who stands behind itWho uses it
robots.txt Declare terms: allow or block, per crawler IETF standard, RFC 9309 (2022). Its rules "are not a form of access authorization" [1] On sites using TollBit, 30% of AI bot scrapes in Q4 2025 ignored it (TollBit's own data) [2]
AIPREF Declare terms: a preference about AI training and AI use IETF working-group draft -08 (September 2026), without group consensus yet; the robots.txt training token is an individual draft [4, 5] Anthropic has recognized the training token since October 30, 2026; Microsoft from February 1, 2027; Google intends to in early 2027; OpenAI "will honor" it [6, 7, 8, 9]
TDMRep Declare terms: reserve text-and-data-mining rights W3C Community Group final report (2024), "not a W3C Standard" [11] More than 50 adopters on the group's own list, among them Hachette Livre, Elsevier, Springer Nature, Wiley and Radio France [12]
RSL Declare terms: state license terms and prices; it also specifies a license server (OLP) RSL 1.0, a Recommendation of RSL's Technical Steering Committee, December 2025 [14, 15] More than 3,000 supporters, says ILC; one AI company licensed through ILC, Yahoo Scout [16]
IAB Tech Lab CoMP Declare terms: describe content, its price and where its license is, for a marketplace API IAB Tech Lab, final V1, April 2026 [21] FORA carries CoMP: its fields travel whole on each FORA offer [22]
llms.txt Help discovery: a curated index for AI readers A proposal by Jeremy Howard, v2 August 2026; no standards body [31] "thousands of sites publish an llms.txt file", by its author's count [31]
Web Bot Auth Identify the caller: a signature on each agent request IETF working-group draft, October 2026 [32] ChatGPT Work's cloud browser signs with it [33]. Cloudflare verifies, and FORA agents sign, under two different earlier drafts [23, 34]
MCP Connect: how an agent calls a tool or a seller The Agentic AI Foundation, under the Linux Foundation; revision of July 28, 2026 [35, 36] Cashmere, AskNews and Newstex sell content through MCP servers; FORA's Identity Service is one [22, 37, 38, 39]
C2PA Prove origin: who made this, and has it changed C2PA 2.5 (October 2026), a Linux Foundation project; ISO 22144 at committee draft [41, 42, 43] Steering-committee members include Adobe, BBC, Google, Meta, Microsoft and OpenAI [44]
FORA Declare terms, transact and report: a signed offer per term, the buyer's signed acceptance, a signed link, and the buyer's usage report The open protocol this site describes. Apache-2.0, release v1.2.1, with FORA Server, an open-source implementation [22, 24] Any company can run its own marketplace on it. See About FORA
x402 Settle, which moves money: payment over HTTP 402 x402 Foundation, under the Linux Foundation; 40 members by July 2026, Visa, Mastercard and American Express among them [48] In AWS WAF on CloudFront since June 2026; in Cloudflare's Monetization Gateway, a closed beta since September 30, 2026 [49, 50]
MPP Settle, which moves money: payment over HTTP 402, per charge, session or subscription Co-authored by Tempo and Stripe; its core scheme is an individual Internet-Draft [52, 53] Cloudflare's Agents SDK names it as one of its two payment protocols; AWS called MPP support "coming soon" in June 2026 [49, 54]
SPUR Content Telemetry Report: what an AI system did with content after the fetch SPUR Coalition, 1.0, September 2026 [55] AskNews says it weights its publisher pool by its events [56]
robots.txtDeclare terms: allow or block, per crawler
Status, and who stands behind it
IETF standard, RFC 9309 (2022). Its rules "are not a form of access authorization" [1]
Who uses it
On sites using TollBit, 30% of AI bot scrapes in Q4 2025 ignored it (TollBit's own data) [2]
AIPREFDeclare terms: a preference about AI training and AI use
Status, and who stands behind it
IETF working-group draft -08 (September 2026), without group consensus yet; the robots.txt training token is an individual draft [4, 5]
Who uses it
Anthropic has recognized the training token since October 30, 2026; Microsoft from February 1, 2027; Google intends to in early 2027; OpenAI "will honor" it [6, 7, 8, 9]
TDMRepDeclare terms: reserve text-and-data-mining rights
Status, and who stands behind it
W3C Community Group final report (2024), "not a W3C Standard" [11]
Who uses it
More than 50 adopters on the group's own list, among them Hachette Livre, Elsevier, Springer Nature, Wiley and Radio France [12]
RSLDeclare terms: state license terms and prices; it also specifies a license server (OLP)
Status, and who stands behind it
RSL 1.0, a Recommendation of RSL's Technical Steering Committee, December 2025 [14, 15]
Who uses it
More than 3,000 supporters, says ILC; one AI company licensed through ILC, Yahoo Scout [16]
IAB Tech Lab CoMPDeclare terms: describe content, its price and where its license is, for a marketplace API
Status, and who stands behind it
IAB Tech Lab, final V1, April 2026 [21]
Who uses it
FORA carries CoMP: its fields travel whole on each FORA offer [22]
llms.txtHelp discovery: a curated index for AI readers
Status, and who stands behind it
A proposal by Jeremy Howard, v2 August 2026; no standards body [31]
Who uses it
"thousands of sites publish an llms.txt file", by its author's count [31]
Web Bot AuthIdentify the caller: a signature on each agent request
Status, and who stands behind it
IETF working-group draft, October 2026 [32]
Who uses it
ChatGPT Work's cloud browser signs with it [33]. Cloudflare verifies, and FORA agents sign, under two different earlier drafts [23, 34]
MCPConnect: how an agent calls a tool or a seller
Status, and who stands behind it
The Agentic AI Foundation, under the Linux Foundation; revision of July 28, 2026 [35, 36]
Who uses it
Cashmere, AskNews and Newstex sell content through MCP servers; FORA's Identity Service is one [22, 37, 38, 39]
C2PAProve origin: who made this, and has it changed
Status, and who stands behind it
C2PA 2.5 (October 2026), a Linux Foundation project; ISO 22144 at committee draft [41, 42, 43]
Who uses it
Steering-committee members include Adobe, BBC, Google, Meta, Microsoft and OpenAI [44]
FORADeclare terms, transact and report: a signed offer per term, the buyer's signed acceptance, a signed link, and the buyer's usage report
Status, and who stands behind it
The open protocol this site describes. Apache-2.0, release v1.2.1, with FORA Server, an open-source implementation [22, 24]
Who uses it
Any company can run its own marketplace on it. See About FORA
x402Settle, which moves money: payment over HTTP 402
Status, and who stands behind it
x402 Foundation, under the Linux Foundation; 40 members by July 2026, Visa, Mastercard and American Express among them [48]
Who uses it
In AWS WAF on CloudFront since June 2026; in Cloudflare's Monetization Gateway, a closed beta since September 30, 2026 [49, 50]
MPPSettle, which moves money: payment over HTTP 402, per charge, session or subscription
Status, and who stands behind it
Co-authored by Tempo and Stripe; its core scheme is an individual Internet-Draft [52, 53]
Who uses it
Cloudflare's Agents SDK names it as one of its two payment protocols; AWS called MPP support "coming soon" in June 2026 [49, 54]
SPUR Content TelemetryReport: what an AI system did with content after the fetch
Status, and who stands behind it
SPUR Coalition, 1.0, September 2026 [55]
Who uses it
AskNews says it weights its publisher pool by its events [56]
Eight layers, each a different job: declare terms (robots.txt, AIPREF, TDMRep, RSL, which specifies a license server, CoMP, FORA), help discovery (llms.txt), identify the caller (Web Bot Auth), connect (MCP), prove origin (C2PA), transact (FORA), settle, which moves money (x402, MPP), and report (SPUR Content Telemetry, FORA's usage reports).
Thirteen standards, eight jobs.
Text version

Eight layers, each a different job: declare terms (robots.txt, AIPREF, TDMRep, RSL, which specifies a license server, CoMP, FORA), help discovery (llms.txt), identify the caller (Web Bot Auth), connect (MCP), prove origin (C2PA), transact (FORA), settle, which moves money (x402, MPP), and report (SPUR Content Telemetry, FORA's usage reports).

How to read it. Each row does a different job, so the rows don't rank against each other: a road sign isn't worse than a till for not taking payment. The last column mixes kinds of adoption, and each cell says which.

Transacting agrees one sale; settling moves the money. RSL declares terms, and its specification describes a license server. FORA spans the terms, the transaction and the report, and leaves the money to your operator.

Where each job stands

30%
of AI bot scrapes ignored robots.txt on sites using TollBit, in Q4 2025, by TollBit's own data [2]
3,000+
RSL supporters, by ILC's count [16]
4
AI companies committed to honor AIPREF's training refusal [6, 7, 8, 9]
1
AI company paying under an RSL-based license: Yahoo Scout [16]

What to do this quarter

  1. Write down what training, grounding and display are each worth to you. Grounding means an AI answer uses your page as its source. None of these standards can tell you that.
  2. Generate your rules. The AI rules generator writes robots.txt groups for AI crawlers, AIPREF's training refusal, the line that attaches the search-only contract, and TDMRep's header and tag.
  3. Adopt the search-only contract, with your legal team. Set your fee in your site's terms and point robots.txt at the contract: the steps.
  4. Publish RSL if you have license terms to state. Behind FORA's edge function, keep it at /rsl.txt, which the edge passes through to every client [27].
  5. See what an agent would be offered. Paste one of your pages on For Publishers. No account needed.

Charging through your site's terms: the search-only contract

The Movement for an Open Web publishes the Search Only Terms Contract, a legal text a publisher points to from robots.txt and incorporates into its site terms. Indexing for search and a person's own reading are free. A fee is owed for "scraping, bulk extraction, dataset creation, or any AI-related use", for each article, image, video or audio file, at the amount the publisher sets.

A contract covers what the edge function can't see: your open pages, and crawlers that hide what they are. How to adopt the contract →

Proof the buyer agreed to your terms

FORA asks more of you than an RSL file, which a publisher can write in a minute [17]: an Exchange and keys, because it also transacts and takes the report. For a publisher choosing what to do this quarter, that difference may matter more than every comparison below the divider.

What FORA gives back is proof. You set several terms for a page: free for search, a price per AI answer, a yearly deal for one licensee. The Exchange signs an offer for each term the buyer may see. The buyer signs the one it picked, with its own registered key. The Exchange issues a signed link against that acceptance, and both sides keep the record [23, 28].

A buyer that says it didn't know the terms has its own signature on them. You can check it in the Exchange operator's publisher console, which verifies both signatures in your browser [27]. None of the other license formats, RSL's license server included, carries the buyer's own signature on the terms it accepted.

The rest of the page compares the specifications clause by clause. Everything above reads without it.

What a license file can say: uses, price, place and duties

Three of the thirteen are license formats: RSL 1.0, CoMP and FORA's licensing core. All three say what you may do, who may buy, where, at what price, and what you owe afterward. TDMRep and AIPREF say only the first.

On uses they speak almost the same language. RSL defines six use words, from all to search [14]. CoMP takes the same six names, four definitions word for word [20]. FORA starts from RSL's use words and adds copyright uses such as reproduce and sync, 23 in all. It also accepts AIPREF's train-ai as a spelling of ai-train [28]. RSL and FORA both scope a term by place and kind of user, and both can prohibit as well as permit [14, 28].

On what you owe afterward they part. RSL's reporting rule: a client that "does not recognize or cannot comply with the profile MUST treat that activity as not licensed" [14]. CoMP has a citation flag and a reporting address [20]. FORA has obligations, credit and notices among them [28]. An unknown RSL element "MUST render the document non-conformant", so one element a client doesn't know affects the whole file [14]. A FORA agent that can't evaluate a term declines only that term [28].

Where the three license formats differ

RSL's <standard> element points at a legal text elsewhere, such as a Creative Commons license [14]. FORA incorporates by reference too, with the document's digest beside its address, so the document can't be swapped after an offer is signed [28].

See it clause by clause. The Licensing Core explorer shows 12 licenses, each as a FORA term, an RSL file and a CoMP package.

Price and credit.

Pick "AI answers, pay per crawl": RSL loses the credit, CoMP keeps it.

Caps and reports.

Pick "Annual AI access deal": the cap is in neither file.

A shared license.

Pick "Collective blanket license": neither RSL nor CoMP can pin the document's version.

RSL's license server, on paper

RSL specifies a license server [14]. Supertab sells one, and Daily Kos's RSL file names it [57, 58]. No AI company is named as a buyer.

In RSL's words, the server "Implements the Open License Protocol (OLP) to issue license tokens and enforces payment requirements on behalf of Publishers". A license token is the pass it issues; a site or CDN can ask which license and asset it covers. The file can name x402 as the way to pay [14].

The difference that matters in a dispute is who signs. RSL's client doesn't sign the license its token covers, and <standard> takes an address with no digest (Appendix A) [14]. TollBit's content tokens name a license type and a license ID, and TollBit signs them, not the buyer [59]. CoMP records how a crawler plans to identify itself, which is not its assent [20].

What ILC does with it. ILC, the collective that co-authored RSL, keeps the transaction out of the content path [16, 18]. Its RSL file names no license server and points, through <standard>, at ILC's collective license [19]. An AI company takes one blanket license, for a fee ILC doesn't publish, and reports its own use [18]. Yahoo Scout, ILC's first licensee, pays under that contract [16]. How ILC compares with other collectives →

Two sequences side by side. RSL license server: your RSL file names a license server; the client asks for one exact license; the server issues a License Token; the file can name x402 as the way to pay; the client doesn't sign the license its token covers. FORA Exchange: you set several terms for a page; the Exchange signs an offer for each; the buyer signs the one it picked (proof the buyer agreed to your terms); a signed link is issued against that acceptance; both sides keep the record. The operator bills the buyer, and FORA names no payment rail.
RSL's license server as the spec describes it, beside FORA's Exchange.
Text version

Two sequences side by side. RSL license server: your RSL file names a license server; the client asks for one exact license; the server issues a License Token; the file can name x402 as the way to pay; the client doesn't sign the license its token covers. FORA Exchange: you set several terms for a page; the Exchange signs an offer for each; the buyer signs the one it picked (proof the buyer agreed to your terms); a signed link is issued against that acceptance; both sides keep the record. The operator bills the buyer, and FORA names no payment rail.

Provenance is not licensing: where C2PA fits

C2PA defines signed provenance metadata bound to a file: who made it, with what tool, and what has been done to it since. Content Credentials is the consumer name [40]. Adobe, Arm, BBC, Intel, Microsoft and Truepic founded it in 2021 [45].

C2PA's core specification removed its training-and-mining assertion in version 2.0; the Creator Assertions Working Group's specification now carries that signal [41, 46]. A publisher who adopts C2PA has licensed nothing. It has evidence: a signed manifest with a content hash shows what was delivered and whether it changed. FORA carries C2PA validation results as a signed attestation about a resource [22]. Text has been in the specification since 2.3, but if text "is only partially copied", its manifest "may be truncated or absent" [41].

How the standards plug into each other

Two pairs that can contradict each other. RSL states which uses are permitted; CoMP leaves uses to the license it links, so the two can disagree unseen [14, 20]. If you publish both, check that they say the same thing. robots.txt governs fetching by crawler name and TDMRep reserves a use. The TDMRep group calls them complementary, but "this complementarity is not specified in the TDMRep document and remains subject to interpretation" [13].

Four standards feed into FORA: CoMP's fields travel on each offer, C2PA's results arrive as an attestation, an MCP assistant reaches FORA through the Identity Service, and FORA's request signatures are a profile of Web Bot Auth. An Exchange operator can collect over x402 or MPP; FORA Server ships no adapter for either.
Where FORA plugs into the other standards.
Text version

Four standards feed into FORA: CoMP's fields travel on each offer, C2PA's results arrive as an attestation, an MCP assistant reaches FORA through the Identity Service, and FORA's request signatures are a profile of Web Bot Auth. An Exchange operator can collect over x402 or MPP; FORA Server ships no adapter for either.

What you can learn about use after delivery

SPUR's Content Telemetry 1.0 specifies "a telemetry schema for tracking AI agent content usage from retrieval through user engagement". Your server or CDN can report retrieval "with no cooperation from the AI agent required"; only the AI company can report grounding, citation and presentation [55]. RSL's reporting element can point at a profile like this one [14].

In FORA, the buyer reports its usage to the Exchange that sold the item, and each report names the transaction it covers [22]. For disputes and payouts, see About FORA.

Questions

Do I have to choose between RSL and FORA?

No. FORA doesn't replace RSL or CoMP. RSL publishes your terms in a file anyone can read. FORA sells under the terms you set in the publisher console, one signed offer at a time.

Is TDMRep enough to reserve AI training in the EU?

TDMRep is a machine-readable form of the reservation the EU's copyright directive recognizes. In the TDMRep report's summary of Article 4, mining is allowed unless "expressly reserved by their rightsholders in an appropriate manner, such as machine-readable means" [11]. Whether yours meets the test is a question for your lawyer.

What isn't on the map?

IPTC's photo metadata has a data-mining property inside the image file [60]. ODRL is the policy language TDMRep uses [11, 61]. The European Commission's consultation lists Liccium's TDM·AI among opt-out protocols. The Commission hasn't published its list of generally agreed ones [62].

Sources

All sources

robots.txt and AIPREF

  1. RFC 9309: Robots Exclusion Protocol IETF
  2. State of the Bots, Q3–Q4 2025 TollBit
  3. AI Preferences (aipref) working group IETF
  4. A Vocabulary For Expressing AI Usage Preferences, draft -08 IETF AIPREF working group
  5. Machine-Readable Training Preference Signal IETF, individual Internet-Draft
  6. robots.txt controls Anthropic Anthropic help center, article 8896518 (no link)
  7. Your content, your choice: clearer preferences for training and grounding Microsoft, Bing blog
  8. Google's common crawlers Google for Developers
  9. Helping websites express AI training preferences OpenAI
  10. State of the Bots, Q1 2025 TollBit

TDMRep

  1. TDM Reservation Protocol (TDMRep), Final Community Group Report W3C TDMRep Community Group
  2. TDMRep adopters W3C TDMRep Community Group, on GitHub
  3. Text and Data Mining Reservation Protocol Community Group W3C

RSL and ILC

  1. Really Simple Licensing (RSL) 1.0 specification RSL Technical Steering Committee
  2. RSL AI Licensing 1.0 now an official industry standard RSL, press release
  3. The Internet Licensing Collective launches with Yahoo Scout as first commercial partner Internet Licensing Collective, press release
  4. RSL quickstart RSL
  5. The ILC Platform Internet Licensing Collective
  6. ILC's RSL file Internet Licensing Collective

CoMP

  1. CoMP 1.0: Content Metadata Marketplace Supply Specification IAB Tech Lab, on GitHub
  2. CoMP: Content Monetization Protocols initiative IAB Tech Lab

FORA

  1. What Is FORA? FORA documentation
  2. Authentication FORA documentation
  3. FORA protocol repository FORA-Protocol on GitHub
  4. Broker FORA documentation
  5. Search-Only Contract FORA documentation
  6. FORA Server FORA-Protocol on GitHub
  7. Licensing Terms FORA documentation
  8. The licensing core explorer FORA
  9. Publisher Onboarding FORA documentation

llms.txt

  1. The /llms.txt file Jeremy Howard, llmstxt.org

Web Bot Auth

  1. HTTP Message Signatures for automated traffic (draft-ietf-webbotauth-httpsig-protocol) IETF webbotauth working group
  2. ChatGPT Work's cloud browser allowlisting OpenAI Help Center
  3. Web Bot Auth Cloudflare docs

MCP

  1. Model Context Protocol specification, revision 2026-07-28 Model Context Protocol
  2. MCP joins the Agentic AI Foundation Model Context Protocol blog
  3. MCP server documentation Cashmere
  4. MCP server AskNews docs
  5. Newstex MCP Server Newstex

C2PA

  1. C2PA: Coalition for Content Provenance and Authenticity C2PA
  2. C2PA Technical Specification 2.5 C2PA
  3. About C2PA C2PA
  4. ISO/CD 22144, Authenticity of information: Content credentials ISO
  5. Membership C2PA
  6. C2PA founding press release C2PA
  7. Training and Data Mining Assertion 1.1 Creator Assertions Working Group

Payments: x402 and MPP

  1. x402 x402 Foundation
  2. Linux Foundation announces operational launch of x402 Foundation x402 Foundation
  3. AWS WAF announces AI traffic monetization AWS, What's New
  4. Monetization Gateway beta: charge AI agents for consumption with HTTP 402 Cloudflare blog
  5. Machine Payments Protocol MPP
  6. Frequently asked questions MPP
  7. The "Payment" HTTP Authentication Scheme IETF, individual Internet-Draft
  8. Agentic Payments, Agents SDK Cloudflare docs

SPUR Content Telemetry

  1. SPUR Content Telemetry 1.0 SPUR Coalition, on GitHub
  2. From Retrieval to Citation: How AskNews Makes Publisher Usage Traceable AskNews blog

Others mentioned

  1. Supertab Connect Supertab
  2. Daily Kos's RSL file Daily Kos
  3. Tokens TollBit docs
  4. IPTC Photo Metadata Standard IPTC
  5. ODRL Information Model 2.2 W3C
  6. Stakeholder consultation on AI and copyright compliance European Commission

Contracts: the search-only contract

  1. Search Only Terms Contract Movement for an Open Web

Sources are checked on the date of publication.

Declare what you sell. Then make it purchasable.

Generate your robots.txt and headers, attach the search-only contract, and see what an agent would be offered for one of your pages through a FORA marketplace, which any company can run.

Generate your rules See your offers