are you using WebDirect and/or Custom Web Publishing?
Both. Have been running website in fms since they forced us to a version or two ago. Using web direct and its predecessors as long as they have been available.
Did not have this issue until I am guessing fms 15. Didn't find it until people started saying they could not find us on the web.
Commenting out out the lines does not appear to stop the header issue but I do not have it in my site code.
We WANT google etc to traverse our sites. This issue is preventing that.
OK, I don't know what "headers" can be changed (if any) with WebDirect. Or even if conf changes would make a difference in WebDirect.
This may be a question for TS, if you want t post to Report a Product Issue.