Docs/Getting Started

Core Concepts

Understand scores, site keys, API keys, and integration types in CrawlReady.

AI Readiness Score

A single 0–100 score that measures how ready your site is for AI crawlers and agents. It combines three sub-scores: Crawlability (50% weight), Agent Readiness (25%), and Agent Interaction (25%). Floor rule: the overall score cannot exceed 60 if any sub-score falls below 20, regardless of how well the other two perform.

Site Key

A unique token in the format cr_live_* that identifies your site for analytics ingest. Generated automatically when you register a domain. You pass the site key to the middleware — it's included in every ingest event sent to /api/v1/ingest. The site key is not a secret, but it is unique per domain and rate-limited. It is different from an API key.

API Key

A Bearer token used for authenticated API access — running scans, managing sites, listing results. Created from the dashboard or via POST /api/v1/api-keys. The full secret is shown exactly once at creation time; store it immediately. API keys are scoped to your user or organization and subject to tier-based rate limits. Different from a site key: API keys authenticate you, site keys identify your site.

Integration Types

Middleware runs server-side on every inbound request before your application handles it. It reads the User-Agent header, detects AI crawlers, and fires an ingest event asynchronously without blocking the response. Because it runs at the edge, it captures crawlers that never execute JavaScript — the bots most likely to be indexing your content for AI answers. Coverage is approximately 100% of bot traffic.

Script tag runs client-side after the page loads in a browser. It only fires when JavaScript executes successfully, which means any crawler that does not run a JS engine (the majority of AI crawlers) is invisible to it. Use the script tag only when you cannot deploy server-side middleware.

Need Help?

Can't find what you're looking for? Reach out and we'll help.