PerplexityBot: Perplexity's Answer-Retrieval Crawler
Perplexity · Answer-retrieval crawler · Last verified October 2026
PerplexityBot crawls pages so Perplexity's answer engine can retrieve and cite them — Perplexity's main discovery path to your content. Perplexity cites sources inline more aggressively than most engines, so invisibility here means invisibility across a fast-growing answer surface. It honors robots.txt according to Perplexity's docs.
What is PerplexityBot?
Crawls pages so Perplexity's answer engine can retrieve and cite them. This is Perplexity's main discovery path to your content.
The user-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot
Does PerplexityBot respect robots.txt?
Yes, per Perplexity's documentation (claimed since 2024). Caveat: third parties have reported undeclared fetching that bypassed blocks. If you see PerplexityBot traffic despite a Disallow, enforce at the firewall rather than trusting the file.
How to block PerplexityBot in robots.txt
User-agent: PerplexityBot Disallow: /
How to explicitly allow PerplexityBot
Useful when your robots.txt has broad Disallow rules and you want to carve out an exception:
User-agent: PerplexityBot Allow: /
One bot per User-agent block; specific tokens beat User-agent: *.
What blocking PerplexityBot does (and doesn't do)
Blocking PerplexityBot removes you from Perplexity's index — no crawling, no citations. Given how prominently Perplexity surfaces its sources, this is a high-visibility-cost block. Only block it if you've deliberately decided Perplexity isn't an audience you want.
How to check if PerplexityBot is crawling your site
Search your access logs for the token:
grep -i "PerplexityBot" /var/log/nginx/access.log | wc -l
On Cloudflare, filter Firewall Events or Analytics by User Agent. If the count is zero over a month, the bot isn't visiting you — either your robots.txt blocks it, or it simply hasn't found you worth crawling.
Beyond robots.txt
robots.txt is a request, not a barrier. For server- and CDN-level blocking (nginx, Apache, Cloudflare WAF), see blocking beyond robots.txt on the overview page.
PerplexityBot vs. related bots
Changelog
Last verified October 2026.
What does PerplexityBot see on your site?
Operator docs: Perplexity · All AI crawlers