AFR v1.1
The AFR standard
AFR (AI-Friendliness Rating) scores every website 0–100 on how open it is to AI crawlers and agents. It measures only public, reproducible facts: every sub-check is one HTTP GET or DNS lookup anyone can repeat, and every awarded point links to its raw evidence. Grades are coordinates on an openness scale — not a moral verdict.
The three-layer model
The total is 100 = 35 + 40 + 25. Legibility carries the most weight because machine-readable content is what AI-friendliness actually means: a site that is technically open but unreadable still fails on AI usefulness.
| Layer | Question | Weight |
|---|---|---|
| L1 Reachability | Can AI reach it? | 35 |
| L2 Legibility | Can it be read? | 40 |
| L3 Actionability | Can it act? | 25 |
Every sub-check
Grouped exactly as the standard defines them. Points are fixed; nothing here depends on opinion, popularity, or payment.
L1 Reachability 35 points
Reachability — 35 points. Can an AI crawler get to your content at all? 25 of these points sit in the robots.txt AI policy, scored per crawler family; missing robots.txt means the protocol default (allow everything), so it scores full marks by definition.
A1 robots.txt AI crawler policy 25 pts
Scored per AI crawler family: search/citation 9, hybrid 6, training 10. Each user-agent is judged by the most specific matching rule group (RFC 9309); a partial Disallow such as /private/ counts as allowed, a site-wide Disallow scores 0 for that family.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| A1.OAI-SearchBot | Search / citation | OAI-SearchBot | 4 | Awarded when robots.txt allows OAI-SearchBot (Search / citation family); the most specific matching rule group decides. |
| A1.PerplexityBot | Search / citation | PerplexityBot | 3 | Awarded when robots.txt allows PerplexityBot (Search / citation family); the most specific matching rule group decides. |
| A1.ChatGPT-User | Search / citation | ChatGPT-User | 2 | Awarded when robots.txt allows ChatGPT-User (Search / citation family); the most specific matching rule group decides. |
| A1.Google-Extended | Hybrid | Google-Extended | 3 | Awarded when robots.txt allows Google-Extended (Hybrid family); the most specific matching rule group decides. |
| A1.Applebot-Assistant | Hybrid | Applebot-Assistant | 2 | Awarded when robots.txt allows Applebot-Assistant (Hybrid family); the most specific matching rule group decides. |
| A1.Bingbot | Hybrid | Bingbot | 1 | Awarded when robots.txt allows Bingbot (Hybrid family); the most specific matching rule group decides. |
| A1.GPTBot | Training | GPTBot | 3 | Awarded when robots.txt allows GPTBot (Training family); the most specific matching rule group decides. |
| A1.ClaudeBot | Training | ClaudeBot | 2 | Awarded when robots.txt allows ClaudeBot (Training family); the most specific matching rule group decides. |
| A1.CCBot | Training | CCBot | 1 | Awarded when robots.txt allows CCBot (Training family); the most specific matching rule group decides. |
| A1.Bytespider | Training | Bytespider | 1 | Awarded when robots.txt allows Bytespider (Training family); the most specific matching rule group decides. |
| A1.meta-externalagent | Training | meta-externalagent | 1 | Awarded when robots.txt allows meta-externalagent (Training family); the most specific matching rule group decides. |
| A1.Amazonbot | Training | Amazonbot | 1 | Awarded when robots.txt allows Amazonbot (Training family); the most specific matching rule group decides. |
| A1.Applebot-Extended | Training | Applebot-Extended | 1 | Awarded when robots.txt allows Applebot-Extended (Training family); the most specific matching rule group decides. |
| Group total | 25 | |||
A2 No technical wall 5 pts
A crawler that does not run JavaScript must still be able to read the homepage, and the homepage must not be a forced login wall.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| A2.no-js-body | — | — | 3 | Readable body for a no-JS user agent (≥80 words of text, or <noscript> content). |
| A2.no-login-wall | — | — | 2 | No site-wide forced login wall on the homepage. |
| Group total | 5 | |||
A3 Basic availability 5 pts
HTTPS with a valid certificate, a homepage answering in under 2.5 seconds (median of three), and no suspicious redirect chain.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| A3.https-valid | — | — | 2 | HTTPS with a valid certificate. |
| A3.response-time | — | — | 2 | Homepage response < 2.5s (median of 3 requests). |
| A3.redirect-chain | — | — | 1 | No suspicious redirect chain (≤2 hops, same site family). |
| Group total | 5 | |||
L2 Legibility 40 points
Legibility — 40 points. Once a crawler is here, can it understand the page? llms.txt, structured data, semantic HTML, a sitemap, and basic content signals. Note: llms.txt is a machine-readability bonus, not a guarantee that an AI will cite you.
B1 llms.txt family 14 pts
A machine-readable map of your content. Honesty note: llms.txt is a readability bonus and is not a ranking signal for AI citation.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| B1.llms-txt | — | — | 8 | /llms.txt exists and returns 200. |
| B1.format | — | — | 4 | Valid format: H1 + intro + parseable Markdown link list. |
| B1.llms-full-txt | — | — | 2 | /llms-full.txt exists. |
| Group total | 14 | |||
B2 Structured data 8 pts
JSON-LD or microdata, plus the entity types that let an agent tell what the page is about.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| B2.json-ld | — | — | 2 | Page contains JSON-LD or microdata. |
| B2.core-entity | — | — | 4 | Core entity type present: Organization / WebSite / Article / Product. |
| B2.rich-type | — | — | 2 | Rich type present: FAQPage / HowTo / BreadcrumbList. |
| Group total | 8 | |||
B3 Semantic HTML 8 pts
Heading structure, image alt coverage, Open Graph tags, and the html lang declaration.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| B3.unique-h1 | — | — | 1 | Exactly one <h1> per page. |
| B3.heading-order | — | — | 2 | Heading levels do not skip (h1 → h2 → h3). |
| B3.img-alt | — | — | 2 | <img> alt coverage ≥ 60%. |
| B3.og-tags | — | — | 2 | og:title / og:description present. |
| B3.html-lang | — | — | 1 | <html lang> declared. |
| Group total | 8 | |||
B4 Sitemap 4 pts
A reachable /sitemap.xml that carries lastmod and covers the homepage links we sample.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| B4.sitemap-reachable | — | — | 2 | /sitemap.xml reachable (robots-declared or direct hit). |
| B4.lastmod | — | — | 1 | Sitemap entries carry <lastmod>. |
| B4.link-coverage | — | — | 1 | ≥80% of 10 sampled homepage links appear in the sitemap. |
| Group total | 4 | |||
B5 Content signals 6 pts
Enough extractable text, machine-readable dates, and a reachable About/contact path.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| B5.word-count | — | — | 2 | Homepage extractable text ≥ 500 words. |
| B5.machine-dates | — | — | 2 | Machine-readable dates (<time datetime> or ISO date). |
| B5.about-contact | — | — | 2 | About / contact path reachable from the homepage. |
| Group total | 6 | |||
L3 Actionability 25 points
Actionability — 25 points. Can the AI do the user's job for them: call an API, read agent instructions, connect over MCP, or use your in-site AI features?
C1 Public API 8 pts
An API an agent can actually call, human-readable documentation, and a machine-readable OpenAPI spec.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| C1.public-api | — | — | 5 | Public API detected (llms.txt / common paths / dev subdomain). |
| C1.api-docs | — | — | 2 | Human-readable API documentation. |
| C1.openapi-spec | — | — | 1 | Machine-readable OpenAPI spec (/.well-known/openapi.yaml or schema reference). |
| Group total | 8 | |||
C2 Agent instructions 4 pts
/agents.md and agent usage policy inside llms.txt (the Optional section).
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| C2.agents-md | — | — | 2 | /agents.md exists. |
| C2.llms-agent-policy | — | — | 2 | Agent usage policy / instructions inside llms.txt. |
| Group total | 4 | |||
C3 MCP interface 6 pts
A publicly available MCP server declared by the site or its documentation, with a reachable URL.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| C3.mcp-server | — | — | 6 | Public MCP server declared and its URL is reachable. |
| Group total | 6 | |||
C4 In-site AI features 7 pts
Assistant, AI search or generative features on the site — the only check that allows human sampling on top of automation.
| ID | Family | User-agent | Points | What it measures |
|---|---|---|---|---|
| C4.ai-features | — | — | 7 | In-site AI assistant / AI search / generative features. |
| Group total | 7 | |||
Grade thresholds
Current thresholds: A ≥75 / B ≥ 60 / C ≥ 42 / D ≥ 25, calibrated on 2026-11-10 against the first Top-1000 scan distribution. After that, thresholds only move at quarter boundaries following a distribution review, and historical scores keep the engine version they were computed with.
| Grade | Range | Label | Meaning |
|---|---|---|---|
| A | 75–100 | AI-Open | Fully open |
| B | 60–74 | AI-Friendly | Friendly |
| C | 42–59 | AI-Neutral | Neutral |
| D | 25–41 | AI-Restricted | Restricted |
| F | 0–24 | AI-Closed | Closed |
Strategy tags
A second, independent axis derived from robots.txt alone. The tag sits next to the grade and never deducts points — it explains the policy behind the score.
- AI-Open Allows every AI crawler: search, hybrid, and training.
- Middle-Path Allows search-type crawlers, blocks training crawlers.
- Training-Open Allows training crawlers while restricting search-type crawlers.
- AI-Closed Blocks all AI crawlers site-wide.
- Unconfigured No AI-specific rules in robots.txt; protocol defaults apply.
Engine versions
Every scan records the engine version that produced it, so any score on this site can be reproduced bit-for-bit later.
| Written standard | AFR v1.1 |
|---|---|
| Scoring engine | afr-1.1.0 |
| Scanner | scanner-0.1.0 |
| Crawler user-agent | afrScanner/0.1 (+https://afr.im/bot; AI-friendliness research crawler) |
Every snapshot stores engine_version; historical scores are never re-graded with a newer engine — they stay attached to the version that produced them.
Where the written standard and the engine differ, the engine version recorded on each scan governs recomputation.
Recompute any score yourself
The scoring engine is a pure function: no IO, no clock, no network. Clone the repository, run the test suite, and feed a recorded snapshot back into the engine.
1. Clone and run the engine test suite (Node ≥ 22, zero dependencies):
git clone https://github.com/AFR-IM/AFR-web
cd AFR-web
npm test # = node --test packages/*/test/*.test.ts 2. Feed a recorded snapshot ({ signals, afr } per site) back into the engine:
import { scoreAfr } from './packages/afr/src/index.ts';
import snapshot from './scans/github.com.json' with { type: 'json' };
const result = scoreAfr(snapshot.signals);
console.log(result.score, result.grade, result.engineVersion); Neutrality pledge
Paying never changes a score.
Read it verbatim — this is the promise the product is built on.
- Paid placements never change ranking or grades. The natural list is always a pure AFR-score descending sort; money buys exposure in a separate Sponsored block, never a better position in the natural order and never a better-looking grade.
- Every sponsored slot shows the site's real grade ring and score — an F-grade site can buy exposure, but it cannot buy an A-grade appearance.
- Ads live in visually separated blocks with a permanent "Sponsored" label, never mixed into natural results, and hovering an ad never changes a grade color.
- Verification, badges, evidence chains and score-change notifications stay free forever. Nothing in the free tier is throttled to force an upgrade.
Scores cannot be bought: payments never affect ratings.
Data sources
- Seed set
- Tranco daily Top 1M (tranco-list.eu, free for academic use with attribution).
- Scan evidence
- Public snapshots committed to github.com/AFR-IM/AFR-data — git versioning is the audit trail.
- Corrections
- Pull requests to the data repository (corrections/) trigger a re-scan; verified owners may publish a statement.
Machine-readable standard
The whole standard is served as JSON at /api/v1/method (layers, per-sub-check points, grade thresholds) so tools can consume it without scraping this page.
AFR v1.1 is published under CC BY 4.0 — reuse it with attribution.