Guides
Configuration you can copy, with the reasoning behind it and an honest note on what each measure does not cover.
Block AI training crawlers without losing search traffic
GPTBot and ClaudeBot crawl for training. OAI-SearchBot and PerplexityBot decide whether assistants can cite you. Blocking all of them costs referral traffic — here is how to separate the two, with the exact directives.
ReferenceThe complete robots.txt list for AI crawlers
A named-crawler reference: which user agents crawl for training, which power AI search citations, which ignore robots.txt entirely, and the exact directives that work for each.
Browser boundaryThe security headers checklist: CSP, HSTS and the five that actually matter
Most guides list twenty directives you will never ship. These are the ones that block real attacks, what each breaks when misconfigured, and how to roll them out without downtime.
Abuse boundaryRate limiting patterns that survive real traffic
Fixed windows let attackers double their burst at the boundary. Token buckets need shared state. How to pick a limiter that holds up on serverless, at the edge, and behind a CDN.
ArchitectureWAF vs application middleware: where should your security boundary live?
Edge WAFs catch traffic before it reaches you but cannot see application context. Middleware sees everything but runs on your compute. The honest tradeoff, and when running both is worth it.
AI boundaryPrompt injection defense for web applications that call LLMs
If your app passes user input, page content or tool output into a model, you have an injection surface WAF rules do not cover. What actually helps, and what only looks like it helps.