A new problem has arisen which would seem to prevent the installation of Filemaker Server 15 on OS X machines running OS X Server.
The Filemaker Server 15 installer checks if standard Web ports 80 and 443 are open before it will proceed with installation. It does so even if the option for a multi-machine installation is selected, and the target machine specified as the Filemaker-only server. No option is given within the Installer to install Web services on other ports, or to proceed anyway. The installer simply will not proceed if it detects ports 80 and 443 in use.
One might think that the remedy would be to simply disable web services in the Server application. Unfortunately, so long as OS X Server is installed, an Apache service is kept running—solely, as far as I can tell, in order to display this message:
This is a remarkably annoying oversight Apple might wish to address, either in Filemaker Server, OS X Server, or (gasp!) by making the OS X Server environment aware of, and even capable of managing, Filemaker Server.
Until they see fit to do something sensible, however, the Filemaker Server installer can be tricked into proceeding with the installation through the brutal but harmless expedient of repeatedly killing OS X Server's Apache httpd services during the FMS installation.
The following Bash one-liner works (enter this into Terminal before running the Filemaker Server installer, give your password for sudo the first time):
while sleep 5; do sudo killall -9 httpd; done
You can safely ignore any "No matching processes were found" messages. If you have the watch command installed (via Homebrew, etc.), you could also use watch to rerun the command every 2 seconds, like so:
sudo watch killall -9 httpd
There's probably a more elegant way to do this, but either works as a quick and dirty way past the installer port checks.
Just remember to kill your Bash loop or watch command once Filemaker's installed (Ctrl-C in Terminal). And make sure you do NOT tell FMS to enable Web services during the Deployment stage, on any system running OS X Server even with Web services disabled, or you may be in for a world of discomfort.
Hope this helps somebody.