Commit graph

2 commits

Author SHA1 Message Date
JP Hastings-Spital
05b5f19b26 Remove a bunch of AI crawlers
Pulled from Cory's post: https://coryd.dev/posts/2024/go-ahead-and-block-ai-web-crawlers/
2024-03-27 21:09:17 +00:00
JP Hastings-Spital
b7e9b3cf17 Use robots.txt template 2023-08-19 19:52:30 +01:00