AFR v1.1

The AFR standard

AFR (AI-Friendliness Rating) scores every website 0–100 on how open it is to AI crawlers and agents. It measures only public, reproducible facts: every sub-check is one HTTP GET or DNS lookup anyone can repeat, and every awarded point links to its raw evidence. Grades are coordinates on an openness scale — not a moral verdict.

The three-layer model

The total is 100 = 35 + 40 + 25. Legibility carries the most weight because machine-readable content is what AI-friendliness actually means: a site that is technically open but unreadable still fails on AI usefulness.

Layer Question Weight
L1 Reachability Can AI reach it? 35
L2 Legibility Can it be read? 40
L3 Actionability Can it act? 25

Every sub-check

Grouped exactly as the standard defines them. Points are fixed; nothing here depends on opinion, popularity, or payment.

L1 Reachability 35 points

Reachability — 35 points. Can an AI crawler get to your content at all? 25 of these points sit in the robots.txt AI policy, scored per crawler family; missing robots.txt means the protocol default (allow everything), so it scores full marks by definition.

A1 robots.txt AI crawler policy 25 pts

Scored per AI crawler family: search/citation 9, hybrid 6, training 10. Each user-agent is judged by the most specific matching rule group (RFC 9309); a partial Disallow such as /private/ counts as allowed, a site-wide Disallow scores 0 for that family.

ID Family User-agent Points What it measures
A1.OAI-SearchBot Search / citation OAI-SearchBot 4 Awarded when robots.txt allows OAI-SearchBot (Search / citation family); the most specific matching rule group decides.
A1.PerplexityBot Search / citation PerplexityBot 3 Awarded when robots.txt allows PerplexityBot (Search / citation family); the most specific matching rule group decides.
A1.ChatGPT-User Search / citation ChatGPT-User 2 Awarded when robots.txt allows ChatGPT-User (Search / citation family); the most specific matching rule group decides.
A1.Google-Extended Hybrid Google-Extended 3 Awarded when robots.txt allows Google-Extended (Hybrid family); the most specific matching rule group decides.
A1.Applebot-Assistant Hybrid Applebot-Assistant 2 Awarded when robots.txt allows Applebot-Assistant (Hybrid family); the most specific matching rule group decides.
A1.Bingbot Hybrid Bingbot 1 Awarded when robots.txt allows Bingbot (Hybrid family); the most specific matching rule group decides.
A1.GPTBot Training GPTBot 3 Awarded when robots.txt allows GPTBot (Training family); the most specific matching rule group decides.
A1.ClaudeBot Training ClaudeBot 2 Awarded when robots.txt allows ClaudeBot (Training family); the most specific matching rule group decides.
A1.CCBot Training CCBot 1 Awarded when robots.txt allows CCBot (Training family); the most specific matching rule group decides.
A1.Bytespider Training Bytespider 1 Awarded when robots.txt allows Bytespider (Training family); the most specific matching rule group decides.
A1.meta-externalagent Training meta-externalagent 1 Awarded when robots.txt allows meta-externalagent (Training family); the most specific matching rule group decides.
A1.Amazonbot Training Amazonbot 1 Awarded when robots.txt allows Amazonbot (Training family); the most specific matching rule group decides.
A1.Applebot-Extended Training Applebot-Extended 1 Awarded when robots.txt allows Applebot-Extended (Training family); the most specific matching rule group decides.
Group total 25

A2 No technical wall 5 pts

A crawler that does not run JavaScript must still be able to read the homepage, and the homepage must not be a forced login wall.

ID Family User-agent Points What it measures
A2.no-js-body — — 3 Readable body for a no-JS user agent (≥80 words of text, or <noscript> content).
A2.no-login-wall — — 2 No site-wide forced login wall on the homepage.
Group total 5

A3 Basic availability 5 pts

HTTPS with a valid certificate, a homepage answering in under 2.5 seconds (median of three), and no suspicious redirect chain.

ID Family User-agent Points What it measures
A3.https-valid — — 2 HTTPS with a valid certificate.
A3.response-time — — 2 Homepage response < 2.5s (median of 3 requests).
A3.redirect-chain — — 1 No suspicious redirect chain (≤2 hops, same site family).
Group total 5

L2 Legibility 40 points

Legibility — 40 points. Once a crawler is here, can it understand the page? llms.txt, structured data, semantic HTML, a sitemap, and basic content signals. Note: llms.txt is a machine-readability bonus, not a guarantee that an AI will cite you.

B1 llms.txt family 14 pts

A machine-readable map of your content. Honesty note: llms.txt is a readability bonus and is not a ranking signal for AI citation.

ID Family User-agent Points What it measures
B1.llms-txt — — 8 /llms.txt exists and returns 200.
B1.format — — 4 Valid format: H1 + intro + parseable Markdown link list.
B1.llms-full-txt — — 2 /llms-full.txt exists.
Group total 14

B2 Structured data 8 pts

JSON-LD or microdata, plus the entity types that let an agent tell what the page is about.

ID Family User-agent Points What it measures
B2.json-ld — — 2 Page contains JSON-LD or microdata.
B2.core-entity — — 4 Core entity type present: Organization / WebSite / Article / Product.
B2.rich-type — — 2 Rich type present: FAQPage / HowTo / BreadcrumbList.
Group total 8

B3 Semantic HTML 8 pts

Heading structure, image alt coverage, Open Graph tags, and the html lang declaration.

ID Family User-agent Points What it measures
B3.unique-h1 — — 1 Exactly one <h1> per page.
B3.heading-order — — 2 Heading levels do not skip (h1 → h2 → h3).
B3.img-alt — — 2 <img> alt coverage ≥ 60%.
B3.og-tags — — 2 og:title / og:description present.
B3.html-lang — — 1 <html lang> declared.
Group total 8

B4 Sitemap 4 pts

A reachable /sitemap.xml that carries lastmod and covers the homepage links we sample.

ID Family User-agent Points What it measures
B4.sitemap-reachable — — 2 /sitemap.xml reachable (robots-declared or direct hit).
B4.lastmod — — 1 Sitemap entries carry <lastmod>.
B4.link-coverage — — 1 ≥80% of 10 sampled homepage links appear in the sitemap.
Group total 4

B5 Content signals 6 pts

Enough extractable text, machine-readable dates, and a reachable About/contact path.

ID Family User-agent Points What it measures
B5.word-count — — 2 Homepage extractable text ≥ 500 words.
B5.machine-dates — — 2 Machine-readable dates (<time datetime> or ISO date).
B5.about-contact — — 2 About / contact path reachable from the homepage.
Group total 6

L3 Actionability 25 points

Actionability — 25 points. Can the AI do the user's job for them: call an API, read agent instructions, connect over MCP, or use your in-site AI features?

C1 Public API 8 pts

An API an agent can actually call, human-readable documentation, and a machine-readable OpenAPI spec.

ID Family User-agent Points What it measures
C1.public-api — — 5 Public API detected (llms.txt / common paths / dev subdomain).
C1.api-docs — — 2 Human-readable API documentation.
C1.openapi-spec — — 1 Machine-readable OpenAPI spec (/.well-known/openapi.yaml or schema reference).
Group total 8

C2 Agent instructions 4 pts

/agents.md and agent usage policy inside llms.txt (the Optional section).

ID Family User-agent Points What it measures
C2.agents-md — — 2 /agents.md exists.
C2.llms-agent-policy — — 2 Agent usage policy / instructions inside llms.txt.
Group total 4

C3 MCP interface 6 pts

A publicly available MCP server declared by the site or its documentation, with a reachable URL.

ID Family User-agent Points What it measures
C3.mcp-server — — 6 Public MCP server declared and its URL is reachable.
Group total 6

C4 In-site AI features 7 pts

Assistant, AI search or generative features on the site — the only check that allows human sampling on top of automation.

ID Family User-agent Points What it measures
C4.ai-features — — 7 In-site AI assistant / AI search / generative features.
Group total 7

Grade thresholds

Current thresholds: A ≥75 / B ≥ 60 / C ≥ 42 / D ≥ 25, calibrated on 2026-11-10 against the first Top-1000 scan distribution. After that, thresholds only move at quarter boundaries following a distribution review, and historical scores keep the engine version they were computed with.

Grade Range Label Meaning
A 75–100 AI-Open Fully open
B 60–74 AI-Friendly Friendly
C 42–59 AI-Neutral Neutral
D 25–41 AI-Restricted Restricted
F 0–24 AI-Closed Closed

Strategy tags

A second, independent axis derived from robots.txt alone. The tag sits next to the grade and never deducts points — it explains the policy behind the score.

  • AI-Open Allows every AI crawler: search, hybrid, and training.
  • Middle-Path Allows search-type crawlers, blocks training crawlers.
  • Training-Open Allows training crawlers while restricting search-type crawlers.
  • AI-Closed Blocks all AI crawlers site-wide.
  • Unconfigured No AI-specific rules in robots.txt; protocol defaults apply.

Engine versions

Every scan records the engine version that produced it, so any score on this site can be reproduced bit-for-bit later.

Written standard AFR v1.1
Scoring engine afr-1.1.0
Scanner scanner-0.1.0
Crawler user-agent afrScanner/0.1 (+https://afr.im/bot; AI-friendliness research crawler)

Every snapshot stores engine_version; historical scores are never re-graded with a newer engine — they stay attached to the version that produced them.

Where the written standard and the engine differ, the engine version recorded on each scan governs recomputation.

Recompute any score yourself

The scoring engine is a pure function: no IO, no clock, no network. Clone the repository, run the test suite, and feed a recorded snapshot back into the engine.

1. Clone and run the engine test suite (Node ≥ 22, zero dependencies):

git clone https://github.com/AFR-IM/AFR-web
cd AFR-web
npm test   # = node --test packages/*/test/*.test.ts

2. Feed a recorded snapshot ({ signals, afr } per site) back into the engine:

import { scoreAfr } from './packages/afr/src/index.ts';
import snapshot from './scans/github.com.json' with { type: 'json' };

const result = scoreAfr(snapshot.signals);
console.log(result.score, result.grade, result.engineVersion);

Neutrality pledge

Paying never changes a score.

Read it verbatim — this is the promise the product is built on.

  • Paid placements never change ranking or grades. The natural list is always a pure AFR-score descending sort; money buys exposure in a separate Sponsored block, never a better position in the natural order and never a better-looking grade.
  • Every sponsored slot shows the site's real grade ring and score — an F-grade site can buy exposure, but it cannot buy an A-grade appearance.
  • Ads live in visually separated blocks with a permanent "Sponsored" label, never mixed into natural results, and hovering an ad never changes a grade color.
  • Verification, badges, evidence chains and score-change notifications stay free forever. Nothing in the free tier is throttled to force an upgrade.

Scores cannot be bought: payments never affect ratings.

Data sources

Seed set
Tranco daily Top 1M (tranco-list.eu, free for academic use with attribution).
Scan evidence
Public snapshots committed to github.com/AFR-IM/AFR-data — git versioning is the audit trail.
Corrections
Pull requests to the data repository (corrections/) trigger a re-scan; verified owners may publish a statement.

Machine-readable standard

The whole standard is served as JSON at /api/v1/method (layers, per-sub-check points, grade thresholds) so tools can consume it without scraping this page.

Open /api/v1/method

AFR v1.1 is published under CC BY 4.0 — reuse it with attribution.