# robots-ai.txt — AI-specific crawler guidance for Price Spectre
# This file SUPPLEMENTS robots.txt and MUST NOT override it.
# robots.txt (https://www.pricespectre.com/robots.txt) remains AUTHORITATIVE for crawler
# access control. Where this file and robots.txt differ, robots.txt wins.
# AI usage guidance: https://www.pricespectre.com/ai.txt and /ai.json
# Last updated: 2026-06-08

# Mirrors the disallow rules in robots.txt for all AI crawlers. Public pages may be crawled;
# API, authentication, and query-string URLs must not.

User-agent: *
Disallow: /api/
Disallow: /auth/
Disallow: /*?*
Allow: /

User-agent: GPTBot
Disallow: /api/
Disallow: /auth/
Disallow: /*?*
Allow: /

User-agent: OAI-SearchBot
Disallow: /api/
Disallow: /auth/
Disallow: /*?*
Allow: /

User-agent: Claude-Web
Disallow: /api/
Disallow: /auth/
Disallow: /*?*
Allow: /

User-agent: ClaudeBot
Disallow: /api/
Disallow: /auth/
Disallow: /*?*
Allow: /

User-agent: Google-Extended
Disallow: /api/
Disallow: /auth/
Disallow: /*?*
Allow: /

Sitemap: https://www.pricespectre.com/sitemap.xml