mirror of
https://github.com/TecharoHQ/anubis.git
synced 2025-10-04 05:41:59 +08:00
de7dbfe6d6* Split up AI filtering files Create aggressive/moderate/permissive policies to allow administrators to choose their AI/LLM stance. Aggressive policy matches existing default in Anubis. Removes `Google-Extended` flag from `ai-robots-txt.yaml` as it doesn't exist in requests. Rename `ai-robots-txt.yaml` to `ai-catchall.yaml` as the file is no longer a copy of the source repo/file. * chore: spelling * chore: fix embeds * chore: fix data includes * chore: fix file name typo * chore: Ignore READMEs in configs * chore(lib/policy/config): go tool goimports -w Signed-off-by: Xe Iaso <me@xeiaso.net> --------- Signed-off-by: Xe Iaso <me@xeiaso.net> Co-authored-by: Xe Iaso <me@xeiaso.net>
7 lines
No EOL
405 B
YAML
7 lines
No EOL
405 B
YAML
# Blocks all AI/LLM bots used for training or unknown/undocumented purposes.
|
|
# Permits user agents with explicitly documented non-training use, and published IP allowlists.
|
|
- import: (data)/bots/ai-catchall.yaml
|
|
- import: (data)/crawlers/ai-training.yaml
|
|
- import: (data)/crawlers/openai-searchbot.yaml
|
|
- import: (data)/clients/openai-chatgpt-user.yaml
|
|
- import: (data)/clients/mistral-mistralai-user.yaml |