Having just done install and configuration for FMS13 on both a single-machine and two-machine setup on Mac OS X 10.8, here are a few things I have learned by trial and error, along with a couple of helpful pages posted by others:
1) We are using a GoDaddy wildcard certificate (*.companyname.com.au). It was originally created from a Mac OS X Server 10.6 Certificate Request from a private key created on that machine. The certificate was successfully installed on several Mac OS X Server 10.7 machines as well by copying and installing the private key, wildard cert and intermediate cert.
2) Since I couldn't use that private key in FMS13's Certificate Request, I used FMS13 certificate command to generate a new serverKey.pem serverand Request.pem as per the docs for the domain *.companyname.com.au. I then used that CSR to re-key our original GoDaddy cert. That generated a different cert with the same expiry date etc., but based on a different private key. That certificate installed without complaint on both FMS installations (on the Worker machine in the two-machine setup). Then I restarted the worker (two machine)/server (single machine).
3) After installing the re-keyed cert, the "chain of trust" in the browser wasn't right. I then installed the GoDaddy intermediate in the System keychain and restarted. Now the certificate chain is fine - it shows the new intermediate certificate (I know it's the new one, it has a later expiry than the original) as well as the correct *.companyname.com.au cert.
1) Once the certificate is installed and the intermediate cert is in the System keychain, WebDirect is fine. However, if you switch on "Require secure connections" in the Database Server->Security section of the Admin Console and restart the database server, FileMaker Pro clients cannot connect at all. In addition, files do not appear on the WebDirect start page. So it is now switched off. So we have secure connections to WebDirect and Admin Console, but not in FMPro/Go.
1) To install the cert on both machines, generate the request on the Worker, then copy serverKey.pem and serverRequest.pem from /Library/Filemaker Server/CStore/ to the same location on the Master. Copy the certificate files supplied in response to the certificate request to the master as well, and run the "fmsadmin certificate import" command, pointing to "your" certificate. make sure that if there is an intermediate certificate as well, import that into the System keychain using /Applications/Utilities/Keychain Access.app
2) When I first installed the certificate on the Master, I could not connect using FMPro/Go, and no files were visible on the WebDirect landing page. I had "Require secure connections" switched on. Not knowing what I know now, I uninstalled and reinstalled FMS from both the Master and the Worker, then reinstalled on both, but only installed the certificate on the Worker. As a result we have secure connections for WebDirect and FMPro/Go but not for Admin Console.
If there's something I have missed, please feel free to share.