# CauseComp — AI answer engines and search crawlers are welcome.
User-agent: GPTBot
User-agent: OAI-SearchBot
User-agent: ClaudeBot
User-agent: Claude-Web
User-agent: PerplexityBot
User-agent: Google-Extended
User-agent: Googlebot
User-agent: Bingbot
User-agent: *
Allow: /
Disallow: /admin
Disallow: /account
Disallow: /login
Disallow: /logout
Disallow: /reset
Disallow: /set-password
Disallow: /register
Disallow: /check-registration
Disallow: /billing/
Disallow: /stripe/
Disallow: /executive/predict
Disallow: /executive/export
Disallow: /executive/export-multi
Disallow: /executive/comparables
Disallow: /executive/report-pdf
Disallow: /executive/metros
Disallow: /executive/email-benchmark
Disallow: /workforce/predict
Disallow: /workforce/export
Disallow: /workforce/onet
Disallow: /workforce/private-sectors
Disallow: /workforce/postings-status
Disallow: /workforce/metros
Disallow: /roles/download

Sitemap: https://www.causecomp.org/sitemap.xml