mapple
CDCloudflare Developers
•Created by mapple on 5/10/2023 in #pages-help
robots.txt
Hi - I'm moving a previous wordpress site to a static site using pages and a cloudflare custom domain. I've added a robots.txt file to the files uploaded - when I visit mysite.com/robots.txt or mysite.com/anystring it's showing the index page - how do I stop that behaviour - I'd rather the user/crawler was served a 404 for mysite.com/anystring than the home page?
10 replies