I am sure you are not going to find an article about this in the Knowledge Base as this will happen rarely but when it does it's very very frustrating.
The FMS 15 installer will not install because it tells you that Port 5003 is in use and it's not because FileMaker Pro client is running. You can try to un-install FMS 14 reboot, and NADA. NOTHING. What the heck is blocking Port 5003 you may ask ? Well let's thank our friends at CISCO.
Turns out the Cisco AnytimeConnect VPN software is listening on Port 5003. How so ? Well, you need to ask them. More troubling is that if you monitor this app with LittleSnitch or other app it uploads and downloads megabytes of data. Strange and scary.
How can you find out who is listening on Port 5003 -- sounds very James Bondish no.... but easy to do. Open the Terminal App located in your APPLICATIONS>UTILITIES folder.
Type in the terminal:
sudo lsof -i :5003
* It will ask you to authenticate with your computer login's password. Do it.
You will see something like this back:
acwebseca 342 root 27u IPv4 0x2ce1cfd76a1f00bb 0t0 TCP localhost:fmpro-internal (LISTEN)
Aha! There you have it, this process called acwebseca is listening and blocking Port 5003. If you Google it you will see that is part of CISCO's VPN software product for Mac.
So how do you kill it ?
Not so fast... it turns out trying to kill it with Activity Viewer app on Mac OS X will only cause it to restart. Frustration is just starting....
Maybe if it's not the CiscoVPN software, killing (whatever process it may be) might work but sure start there see how far you get.
Time to call FileMaker Support! Problem is you will be on hold and then when you talk to them they will tell you "never seen that".
So you just wasted 1-2 hours.
Here what you have to do. Find the CiscoVPN software under your APPLICATIONS FOLDER>CISCO. There you should see the uninstaller.
Run the uninstaller and BINGO! It should allow you to install.
If it's another application blocking you will find it using Terminal and need to find the uninstaller so it removes all the System Extensions, Daemons, and other good stuff that it installs in the LIBRARY folder.
Another very useful set of scripts if you are battling with the Mac OS X Server App installed previously fighting with FileMaker Server's own web server trying to connect is to unload/turn off Apache. This happens a lot with Mac Mini's that had Mac OS X Server and FileMaker Server running together which of course is not recommended anymore. These commands will ensure that Apache is turned off and FileMaker launch Daemons are killed and restarted so you can complete your installation of FileMaker Server. Save them because there are not easy to find!
Resetting Filemaker to be primary web server in event of an issue:
sudo launchctl unload -w /System/Library/LaunchDaemons/org.apache.httpd.plist
sudo launchctl unload /Library/LaunchDaemons/com.filemaker.*
sudo launchctl load /Library/LaunchDaemons/com.filemaker.*