Thank you for the post.
A little more information may help us to narrow this down.
What version of Mac OS X is the server running? Is Instant Web Publishing only in use or both Custom Web Publishing and Instant Web Publishing?
The server is running Mac OS X Server 10.6.8 on a 2 X 2GHz Daul Core Xeon with 32 GB of RAM.
I have a solution that uses IWP and another (less-used) soltuion that uses PHP.
No other services are running on the server (only FMP Server Advanced).
I also authenticate using an external Open Directory.
@ Joel Anderson
What you are experiencing is being discussed in this thread:
It's what happens when IWP "quits abnormally", an issue that is much worse in FM12 than ever before.
When your users get the message you quoted in their browsers, go to your Filemaker Server Console. You will see that at the top left of the console, the IWP stop/start green/red buttons have switched to green-active. Click it, and that solves the problem. Restarting Filemaker Server completely, or even the machine itself, is better because it clears the memory being used by the web publishing engine.
Switch on the email alerts from FM Server, and you will be getting a "701" email every time it happens, so you can go and fix it before your IWP users get too frustrated...