You did not describe the steps you took to connect to the hosted database.
Here are the instruction for hosting and opening a hosted database.
Host - Server
You will open database on host computer (server).
Turn file sharing in this database within FM, when sharing is turned on the host ip address will be displayed.
Select the users that have access to the database. In testing I would just select all users
Then Click OK
IP - Client - Note FMP allows up to 5 concurrent connections
1st connect from your local network, so Turn off 3G/4G so you know you are on your local network
Select the Host icon , then a list of your shared database will be displayed.
Select the desired database and login with your user information.
From outside network / 3G / 4G
Setup port forwarding using port 5003 to the ip address listed on the host computer when you turned on sharing.
Port 5003 will have to be open on your firewall
Go to a website such as www.ipchicken.com or www.whatismyipaddress.com to get your public ip address
Select the network path icon next to the gear icons.
Enter your public ip address publicipaddress or publicipaddress/filename