Crawler route

The crawler route tolls AI crawlers at Cloudflare. naulon generates a Cloudflare Worker, which naulon installs with a token you create or you paste in yourself. The Worker sends AI crawler requests to naulon, which quotes and charges them, and passes everything else to your site. Your DNS, origin, HTTPS certificate and human visitors are unaffected.

It needs nothing on your server, and it runs beside the naulon WordPress plugin or SDK: crawlers that come through Cloudflare pay at the route, and the plugin charges any crawler that reaches your origin directly.

Cloudflare's own security settings run before the Worker, so an agent they block never reaches the route. Sites behind Cloudflare lists each one and how naulon checks it.

Before you start

• Your domain is proven in naulon (a TXT record, a meta tag or a file; the site's Domains card walks you through it).
• Your site charges for at least one path. The Worker only forwards the paths you toll.
• Your origin is served over HTTPS.
• You can create a Worker and add routes in the Cloudflare account that holds your domain's zone.
• If the naulon WordPress plugin or SDK runs on this domain, it is a version that recognises the route (plugin 0.5.8 or later, @naulon/enforce 0.7.0 or later). An older one would charge a crawler that already paid at Cloudflare, so the card asks you to update it first. Do not turn the plugin off: see the next section.

When naulon sees Cloudflare in front of your site, the Domains card opens on the crawler route. Otherwise press Set it up a different way and choose My site is behind Cloudflare. If something above is missing, the card names it and links to the setting that fixes it.

With the WordPress plugin or SDK

Prices are set in naulon, whichever way you charge. Your site supplies who wrote each post, through the plugin or your credits endpoint. The route needs that too, so keep the plugin on. If it is off, naulon has no authors to pay and every read is free; the test reports it.

Setting	Where you set it	What wins
Price, citation multiplier, price rules	naulon (Pricing)	naulon. The plugin has no price.
Which paths are charged, crawler policy	naulon	naulon, on the route and in the plugin alike.
A post is free	the post's toll box in WordPress, or naulon's free list	Either one makes it free.
Who wrote a post, and their shares	WordPress (authors, co-authors)	WordPress
An author's wallet	the author's WordPress profile	The wallet the author claims in naulon, when they have claimed one. A per-article wallet you type in naulon is not used for a plugin site, so an author's share can never be redirected by the site.

The gate signs its own fetch of your page after a crawler paid, covering that exact page and your site's naulon id. The plugin and the SDK check that signature against naulon's published key and serve the page without charging again. Any other request is charged as usual, including a fetch the gate made for some other naulon site.

Let naulon install it

1. In Cloudflare, open My Profile > API Tokens and create a custom token with three permissions: Account · Workers Scripts · Edit, Zone · Workers Routes · Edit and Zone · Zone · Read. Limit the zone resources to your domain's zone. If you set a start date, Cloudflare counts it in UTC, and the token does nothing before then.
2. On the crawler route card, open Use an API token instead if you see it, paste the token and press Install. naulon finds your zone, uploads a Worker named naulon-route- followed by your domain, adds its routes and sends a test crawler.
3. Delete the token in Cloudflare. naulon used it for that one request and did not keep it.

If your card shows Connect Cloudflare, you can approve access on Cloudflare's own page instead of creating a token. naulon asks for the same permissions, uses them once, and hands the access back when the install finishes.

naulon never replaces something it did not create. If another Worker already handles one of the route patterns, the install stops before changing anything and names the pattern, so you can decide what to keep.

Paste it yourself

1. On the crawler route card, press Create the route. naulon generates the Worker file and the route patterns for your domain.
2. In Cloudflare, open Workers & Pages, create a Worker, replace its code with the file and deploy it.
3. In the Worker's settings, add each route pattern shown on the card on your domain's zone. For a site that tolls /essays/, that is example.com/essays/* and example.com/license.xml.
4. Back on the card, press Send a test crawler. You should see "The route works".

The card then waits for the first real AI crawler and shows when the last one came through.

What the Worker does

• It acts only on your own hostname and only on the paths you toll.
• It forwards a request to naulon when the user agent is a known AI crawler, or when the request already carries a payment or a licence. Browsers are never forwarded.
• It skips requests carrying your origin secret in x-naulon-origin-auth. That is naulon fetching your page after a crawler paid, and forwarding it back would loop.
• If naulon does not answer within the timeout, or answers with a server error, the Worker serves your site directly, and crawlers read free until it clears.
• It serves your licence document at /license.xml from naulon, and falls back to your own copy if naulon cannot answer.

The test

Send a test crawler makes three requests to your site and reports one verdict:

Request	Expected
A test crawler (user agent naulon-route-check)	answered by naulon through the route
The same request with your origin secret	answered by your site directly
A browser	answered by your site directly

The test crawler is charged like any unrecognised agent, but it is never counted as a crawler arriving. If Cloudflare challenges the test, the card says so instead of blaming your rule.

The test's page is made up, so it proves where a crawler goes, not that it pays. It also checks that naulon can still price your pages: it re-prices a page you were paid for before, or, on a site nobody has paid yet, checks that the WordPress plugin answers. If either fails, the card says nothing is being charged. On WordPress that usually means the plugin is off.

Secrets

The file carries two secrets: the route secret that lets the Worker into naulon, and your origin secret. Treat the file like a password and keep it out of public repositories.

To replace the route secret, press Generate a new secret. You get a new file; the old one keeps working until you deploy the new one and press I've deployed the new file. If naulon installs the new file on Cloudflare for you, the old secret is retired as soon as the install finishes. Showing the file again and generating a secret are both recorded in your audit log.

If you rotate your origin secret on the Protect your origin panel, or change your crawler policy or the paths you charge for, install the Worker again or paste the new file. naulon keeps no access to your Cloudflare account, so it cannot update the Worker itself. The test reports a stale file as a loop.

Limits

• Cloudflare only for now.
• Workers count against your Cloudflare plan's request allowance. Only forwarded requests (crawlers on tolled paths) run the Worker's forwarding path, but every request on a matched route invokes the Worker. If the daily allowance runs out, Cloudflare stops running the Worker and the test reports it.

Removing it

Press Remove from Cloudflare on the card and give access the same way you installed. naulon deletes its Worker and routes, then removes the route. Crawlers read free from that moment.

Or press Remove the route, then delete the Worker in Cloudflare yourself. Until you do, it keeps running but only serves your site.