Unfortunately, FileMaker (like many other Apple apps, TextEdit, etc.) has decided that you are not smart enough to create a new window, so will open one for you whenever you activate the application.
I can see that this might make sense for a web browser (where I rely on it), some sense for an email program (mostly, but not always), but less and less sense for other applications, and almost no sense at all for a database.
About the only way I know of to not see it is to use something like AppleScript. I keep a little AppleScript droplet right next to the FileMaker icon on the Dock. By "droplet" I mean an AppleScript saved as an application, no startup screen, etc., which runs by double-clicking it. The code is:
tellapplication "FileMaker Pro Advanced"
launch -- open if not open, does not open dialog
activate -- bring to front
As you can see, the "launch" command opens the app without such things as dialogs and new windows.
If you have several versions of FileMaker Pro Advanced it will open the last one installed; which is usually what you want.