Back to Resources
Allowing BrandHalo crawlers through your firewall
·5 min read
BrandHalo uses automated crawlers to run compliance scans on your website channels and AI readiness checks at brandhalo.io/aiready. If your firewall or WAF blocks these crawlers, scans may fail or return incomplete results, including HTTP 403 responses and security challenge pages.
This guide explains how to add BrandHalo to your firewall exemptions so scans can reach your homepage and site files reliably.
Add these User-Agent strings to your firewall allowlist:
BrandHalo-AIChecker - AI readiness scans (brandhalo.io/aiready)BrandHalo-Crawler - Channel crawls for brand monitoringMost firewalls and WAFs support rules that allow traffic based on User-Agent. Create an allow rule that matches requests where the User-Agent header contains BrandHalo. The action should be "Allow" or "Skip" (skip remaining rules) so BrandHalo crawlers can access your site.
If you use Cloudflare, add a Custom rule with action "Skip" so BrandHalo crawlers can reach your site. Use quick add to open a prefilled rule in your dashboard, then choose which WAF components to skip and deploy.
Rule name
Allow BrandHalo crawlers
Rule expression
(http.user_agent contains "BrandHalo-AIChecker") or (http.user_agent contains "BrandHalo-Crawler")
After Cloudflare opens, sign in, pick your domain, review the prefilled rule, select Skip targets (for example all remaining custom rules and Super Bot Fight Mode rules), then deploy.
For AWS WAF, Sucuri, Imperva, or similar services, create a rule that allows requests where the User-Agent header contains BrandHalo. Consult your provider's documentation for adding User-Agent allow rules.
BrandHalo runs on serverless technology, which by design does not have fixed egress IP addresses. User-Agent allowlisting is the recommended approach. If you require IP-based allowlisting (for example, on an Enterprise plan with dedicated egress IPs), please contact us at hello@brandhalo.io.
Once your exemptions are in place, run another scan at brandhalo.io/aiready to confirm we can reach your homepage. For ongoing monitoring, compliance scans, and llms.txt generation, create a BrandHalo account and add your site as a channel.
Part of BrandHalo's resource library on brand governance, connectors, Brand Agent, and keeping your brand consistent across AI tools and channels.
View all articlesBrandHalo gives you the infrastructure to centralise, monitor, and protect your brand across every channel and AI touchpoint.
You've read the guide. Now put it into practice.