Is "macOS Server" officially compatible with "FileMaker 17 Server"?
Have you heard about Apple abandoning MacOS Server as you may know it now? Instead, now you're supposed to get third-party utilities to do what much of what MacOS Server did for you. After much work with MacOS Server, I was so disgusted with Apple's disregard for existing infrastructure, I abandoned OS Server entirely.
If you look at other posts about FMA on macOS server, you will see that there is a conflict between the macOS server installed Web server and the one FMS installs. People recommend to install FMS on macOS without the server app installed. And as fmpdude mentioned, the server app is shrinking.
I would ratchet up the advice from "not a good idea" to "don't even think about it".
I've had two client outages caused by having the MacoS "Server" application installed on a FMS server, both times by well meaning third party IT guys.
Just to add my two cents, don't do it. It's always been recommended from FMI (as far as I know) that the only thing running on a FMS machine should be FMS. OS X Server will cause headaches on the web side. Other services can cause hiccups as well - Time Machine, File Sharing, etc.. As cheap as Mac minis are, I'd strongly recommend picking up a fast, 16 GB RAM, with SSD and use that as your FMS and use the current machine for OS X Server. we've had great success doing so for groups upwards of 100 mixed connection users. Another tidbit from experience, if you have installed Server (I'd say anything) on to a computer you plan to use for FMS, wipe the drive and start fresh. You don't want some little gremlin from a previous installation or configuration mucking up your efforts. There's been more than one thread here about such trials. In addition to installing FMS, go through all system preferences and turn off energy saver, allow for restart after power loss (but don't automatically open the databases), screen savers are ok if you must, but turn time machine off along with anything else that is going to impact performance.
P.S. As to the upcoming changes to MacOS Server, most of the core features are no longer going to be bundled by Apple later this year (presumably around the next OS release). They do have a list Prepare for changes to macOS Server - Apple Support of alternative open source services that you can add to replace the lost functionality, albeit without the cohesive interface and simple implementation.
Retrieving data ...