3 commits

Author	SHA1	Message	Date
JP Hastings-Spital	098d40be14	Exclude Apple’s AI training in robots.txt ... See: https://support.apple.com/en-us/119829#datausage	2024-06-25 08:40:27 +01:00
JP Hastings-Spital	05b5f19b26	Remove a bunch of AI crawlers ... Pulled from Cory's post: https://coryd.dev/posts/2024/go-ahead-and-block-ai-web-crawlers/	2024-03-27 21:09:17 +00:00
JP Hastings-Spital	b7e9b3cf17	Use robots.txt template	2023-08-19 19:52:30 +01:00