Creating and Installing an SSL Certifcate for FMS 17 Using KeyTool

Document created by fmpdude on Jun 14, 2018Last modified by fmpdude on Jun 16, 2018
Version 9Show Document
  • View in full screen mode

Installing an SSL Certificate Into FileMaker Server 17 Using KeyTool/KeyStore

 

Approximately 1,980 words

Approximate reading time: 8 minutes

 

Introduction

After all the postings on the FileMaker forum where people have issue after issue installing SSL, FMS 17, surprisingly, removes CSR creation from the Administration Console. Some might argue, that removal has made SSL in FMS 17 even more disjointed and laborious. Others might argue the other way.

 

So, why not just manage the entire SSL process yourself (including CSR generation)?

 

One good reason you might not want to manage the entire SSL process yourself could be that you only use FMP/FMS. In this case, then, learning a standard technique--but yet another way of doing something--may not be worth it for you. Or, perhaps you want to minimize working with command-line utilities. On the positive side, FMS’ now-required command-line CSR creation does to some work for you as discussed below. Yet, FMS’ SSL implementation behind the scenes is possibly doing in code what we are doing below using KeyTool.

 

Installing an SSL Certificate into FileMaker Server 17 is relatively simple though there are several steps, any one of which will make the import fail. When the import fails, FileMaker Server 17 simply says: “Import failed”. Great. Thanks.

 

Moreover, the FileMaker documentation isn’t always completely “helpful” though the good folks at FMI are there to help you — if you want to use the FMS 17 approach for generating a CSR (that is, using the FMS command-line utility).

 

Unlike other approaches, this quick article will not use FileMaker Server 17 to generate the CSR. Instead, using a common tool installed on all computers (all operating systems), called “keytool” we’ll generate what we need ourselves and make sure we put the required files in the right locations.

 

!!!CAUTIONS!!!

PLEASE: DO NOT ASSUME THIS PROCESS WILL WORK FOR YOU. THERE ARE MANY MANUAL STEPS AND MANY MOVING PARTS. ALSO THERE ARE TWO WORKFLOWS AS SHOWN BELOW IN THE DIAGRAM: (1) THE FMS SSL INSTALLATION AND (2) PURCHASING THE CERTIFICATE ITSELF.

I TESTED THIS PROCESS SUCCESSFULLY WITH THE INEXPENSIVE NAMECHEAP.COM ($8.88/YR) CERTIFICATE WITH FMS 16 AND FMS17 BUT I HAVE NO FMS EXPERIENCE WITH OTHER CERTIFICATES.

HAVE FULL BACKUPS OF EVERYTHING BEFORE YOU START!

 

Certificate Used

This article will demonstrate the NameCheap (namecheap.com) $8.88/year low-end certificate. If you have just a testing server, this certificate may be all you need. Higher-end certificates can take more work as they are more serious about verifying who you really are. Thus, there could be more steps than we discuss below for a higher-end (more trusted) certificate, but you’ll get the general idea anyway.

I’ve successfully installed this low-end certificate with both FMS 16 and FMS 17.


https://www.namecheap.com/security/ssl-certificates.aspx

 

Diagram of steps described below

 

 

What is KeyTool?

 

From the Oracle website:keytool is a key and certificate management utility. It allows users to administer their own public/private key pairs and associated certificates for use in self-authentication (where the user authenticates himself/herself to other users/services) or data integrity and authentication services, using digital signatures.

 

It also allows users to cache the public keys (as certificates) of their communicating peers.

 

A certificate is a digitally signed statement from one entity (person, company, etc.), saying that the public key (and some other information) of some other entity has a particular value. (See Certificateshttps://docs.oracle.com/javase/7/docs/technotes/tools/solaris/keytool.html#Certificates.) When data is digitally signed, the signature can be verified to check the data integrity and authenticity. Integrity means that the data has not been modified or tampered with, and authenticity means the data indeed comes from whoever claims to have created and signed it.”

https://docs.oracle.com/javase/7/docs/technotes/tools/solaris/keytool.html

 

The link above has lots of information and other usage examples, but we’ll actually go through most of the process here using an example.

 

What is a KeyStore File?

A KeyStore file is a repository for your certificates and related files including private keys. That’s really all it is. It’s just a file. Many environments will accept a KeyStore file to implement SSL.  In these environments, you go through part of the process (described below) and then, at the end, import the certificate back into the KeyStore. This article won’t discuss using a KeyStore for your SSL certificate’s final result since FMS 17 doesn’t work that way.

 

Creating the KeyStore

Using the KeyTool command at your command line, we’ll need to issue a few commands. That’s it, and we’re done. Along the way, as described below, you need to get (buy) a certificate to complete the process. When you renew the certificate (“renew” is a misnomer), you start over from the beginning.

 

Let’s assume you have already created a sub-domain at your registrar for your FMS installation. Also, we assume that you’ve installed FMS 17 and it’s working OK on that sub-domain. If you need help with the sub-domain configuration, post on the forum for additional help with that part.

 

Our pretend company and sub-domain will be: fms.myfms17.com. We will need to reference this sub-domain in steps below.

 

Step 1. Create the KeyStore (assuming you are in your Desktop Folder)

 

$ keytool -genkey -alias fms.myfms17.com -keyalg RSA -keystore keyStore.jks -keysize 2048

 

You will need to enter a password. Don’t forget that password and don’t share it.

 

After you run the command above, you should see a “keyStore.jks” on your desktop. It doesn’t matter what you name the file as long as you reference it correctly in future steps.

 

Create the CSR

The CSR, or signing request, kicks off everything. Here you enter details about your company, the common name, address, and other details. A CSR may fail at the CA if you don’t enter the domain name as the “common name”. If you’re using a different certificate than the one here, you may need to contact your certificate vendor for more information.

 

A CSR creates an encoded request you use to purchase the certificate from a Certificate Authority (CA) like NameCheap.com Creating the CSR also created a private key inside the KeyStore file.  We’ll need to extract that private key in a bit.

 

Note: if you use FMS’ 17’s command-line tool, FMS does the same thing. That is, FMS creates the CSR (ServerRequest.csr) and it creates the ServerKey.pem file (the private key) and puts these files in the CSTORE folder for you. By default, and probably always, this CSTORE directory is: “C:\Program Files\FileMaker\FileMaker Server\CStore”.

 

But, let’s create the CSR using the cool KeyTool command.

 

Step 2. Create the CSR from our KeyStore File (assuming Desktop folder).

 

$ keytool -certreq -alias fms.myfms17.com -keystore keyStore.jks -file fms.myfms17.com.csr

 

You may be prompted in any of these steps to re-enter the password you created in Step 1.

When this command is done, you should have a fms.myfms17.com.csr file on your desktop. You will need to use this file to purchase your certificate.

 

Getting your SSL Certificate

Purchasing an SSL certificate is more or less the same regardless of where you go. With NameCheap.com, you copy and paste your CSR’s file contents from step 2 into the requested window at its website. Typically, and the steps may very from which level certificate you purchased, you then need to verify via email (or other) that you are who you said you were in the CSR.

The $8.88 certificate is from Comodo so you will get an automated email from them asking you to visit a website (using a link in the email) and enter a code in the email. Up to this point, before you entered the code from the email, the certificate you purchased is only “in progress”. However, once you enter that code, the CA (NameCheap.com in our example) will show the certificate issued.

Once the CA shows the certificate is issued you can download it as a zip file from the CA vendor. (Some other CA vendors may vary on the exact steps.)

 

Once the CA shows the certificate is issued you can download it as a zip file from the CA vendor. (Some other CA vendors may vary on the exact steps.)

What Do We Need From the Certificate Zip File?

Here are the files you will typically see in the zip file.  For FMS 17, we only care about the first two shown below as we will use both of these files when we import the certificate into FMS 17.

 

1. fms_myfms17_com.ca-bundle  (combined certificate in PEM format)

2. fms_myfms17_com.ca-crt  (single certificate in PEM format)

3. fms_myfms17_com.ca-p7b (is a certificate in PKCS#7 format --> designed for IIS and Tomcat servers). We won’t need 3 here.

 

Extract these files and move them to the Desktop of where you have FMS 17 installed. We will reference the first two above when we do the actual SSL import.

 

Getting that Private Key From our KeyStore

FileMaker 17’s serverKey.pem file is the private key generated when we generated the CSR, but for us, it’s currently locked away in the KeyStore file. To get that file, we will use two steps:

 

Step 3. Create an intermediate PKCS12 file:

$ keytool -importkeystore -srckeystore keystore.jks -destkeystore intermediate.p12 -deststoretype PKCS12

 

Step 4: Extract the private key

$ openssl pkcs12 -in intermediate.p12 -out serverKey.pem -nodes

 

Now you should see the “serverKey.pem” file on your Desktop.

 

We now have all the files we necessary to import the FMS 17 certificate.

 

Finally, Install the Certificate Already!

(Strong Suggestion: backup files in this directory before you overwrite them in this step!)

 

Copy these two files to (by default) C:\Program Files\FileMaker\FileMaker Server\CStore directory:

 

◆ The CSR itself, called serverRequest.csr (this is the CSR you generated in step 2 above, rename if necessary.)

◆ The private key we just extracted: serverKey.pem

 

At this point, we can use the FMS 17 import certificate dialog to import the certificate.

In the FMS 17 import dialog, you need to reference three files to import the certificate:

 

◆ The first button is the path to your actual “.crt” file you extracted from the CA ZIP file (“fms_myfms17_com.ca-crt “ for our example)

◆ The second button is the path you use to point to serverKey.pem in your CStore directory (this is the file you just copied in to that directory      above.)

◆  The third and final button you point to your “bundle” file you extracted from the zip file.

         ⁃    (fms_myfms17_com.ca-bundle)

 

Click import!

Hopefully, it worked for you. Back in FMS 16, we were required to re-order the bundle file certificates in the reverse order from which they were received from namecheap.com. However, in FMS 17, that re-ordering no longer is needed (and reordering the bundle file made the import fail in my testing.)

 

If all went well, you should see something like the below after your successful certificate import into FMS 17.

(The “Issued To” result on the right above is fictitious and only for emphasis.)

 

Getting Help

Since SSL installation can be tricky in the best of circumstances, but usually goes well. If all doesn’t go well, you have lots of resources:

• NameCheap.com (if you use them, of course) has excellent 24/7 chat support and a dedicated SSL team that will help you by email. Don’t discount asking the CA vendor for help. They may be extremely helpful — especially since you’re using their certificates.

• The FileMaker Forum. I’m sure you know about this one. Lots of dedicated folks to help you there.

• Of course, online searches can help with that one little bit that’s not working, too.

 

Conclusion

Installing an SSL certificate into FMS 17 takes several steps some of which are outside of FMS itself. One of those outside steps is purchasing the SSL certificate itself and the CA workflow.

 

Since KeyTool is a standard approach and what FMS maybe itself using in the background (there is a Keystore file in the CStore directory already by default), I decided to just take FMS out of the picture for CSR creation.

There are other approaches out there as well. Some of these approaches are more GUI-based than this approach. The benefit of this approach is that it works everywhere, so why maintain multiple approaches?

 

Hopefully, this quick article was useful for you.

5 people found this helpful

Attachments

    Outcomes