Showing 1-2
of 2 posts
Technical
Robots.txt for AI Crawlers: The Complete 2026 Guide
AI crawlers like GPTBot, ClaudeBot, and PerplexityBot are visiting your site. Your robots.txt determines whether they can index your content for AI search results. Here's how to configure it correctly in 2026.
AI Infrastructure
ADP 2.1+: Proving AI Crawlers Actually Use Your Infrastructure
We built proof infrastructure that tracks AI crawler hits on ADP endpoints and monitors which press releases get cited. Here's the technical implementation.