Express

Middleware Setup

Install the package:

npm install @crawlready/middleware
# or
pnpm add @crawlready/middleware

Add the crawlready middleware to your Express app before your route handlers:

import express from 'express';
import { crawlready } from '@crawlready/middleware/express';

const app = express();

app.use(crawlready({
  siteKey: process.env.CRAWLREADY_SITE_KEY!,
}));

// Your routes follow here
app.get('/', (req, res) => {
  res.send('Hello world');
});

app.listen(3000);

Add your site key to your environment:

CRAWLREADY_SITE_KEY=cr_live_xxxxx

The middleware detects AI crawlers from the User-Agent header and fires an analytics beacon in the background. It calls next() immediately and never delays the response.

Configuration options

| Option | Type | Default | Description | |---|---|---|---| | siteKey | string | required | Your site key from the CrawlReady dashboard (format: cr_live_*) | | botFilter | 'ai-only' \| 'all' | 'ai-only' | Track only AI crawlers, or all 86+ known bots | | endpoint | string | CrawlReady ingest URL | Override the ingest endpoint (useful for testing) | | onBotDetected | function | — | Called before each ingest event; return false to suppress it |

Verify Your Integration

After deploying, use the Verify Integration button in your CrawlReady dashboard. It sends a synthetic GPTBot request to your site and confirms that a visit is recorded within 15 seconds.

If the check times out:

• Confirm the middleware is registered before your route handlers.
• Confirm CRAWLREADY_SITE_KEY is set in the environment where the process runs.
• If your app is behind a reverse proxy, ensure the User-Agent header is forwarded through.

Going Further

• API Reference — trigger scans and retrieve scores programmatically
• Analytics Dashboard — understand your crawler traffic data
• Analytics Overview — what the metrics mean