Thank you for the post.
At this time, the only FileMaker Server versions tested to be compatible with Mac OS X 10.9 Mavericks are FileMaker Server 12.0v6 (Advanced) and FileMaker Server 13.
"Pre FileMaker 11 version of FileMaker have not been tested or certified with Mac OS X Mavericks. These products were developed and released before Mac OS X v10.9 Mavericks became available.
While these earlier versions may install and run on Mac OS X v10.9 Mavericks, you may encounter installation and stability issues for which there is no resolution."
For more information please review the following: