Agent Disco Agent discovery, graded.
Agent Disco check

Common Crawl index presence

← Back to all checks

passive · Category LLM training data · Informational (not graded) · Key llm_training.common_crawl

Informational (not graded): queries the Common Crawl CDX endpoint for the most recent monthly snapshot and counts the target's pages. Common Crawl's corpus underpins most open-source LLM training runs, so presence is a proxy for "an LLM may already know this site." Reported for context — corpus presence is a popularity signal, not a live agent-visibility lever, so it does not affect the score.