I have Mac Pro 2013 (darth vader garbage can edition) running El Capitan.
If I option-double-click on my Mac Book pro in finder, the selected database file is opened and prompts for the userid and password as expected. If I option-double-click on my Mac Pro 2013, the selected database file is opened and no prompt occurs for the userid and password. This problem occurs with Filemaker Pro and Filemaker Pro Advanced.
I have tried this with a wireless logitech K750 keyboard, the wireless Apple keyboard and the wired apple keyboard with same result. I even changed the double click speed in preferences to see what effect that would have. I have tried doing with Filemaker closed/open and still no prompt for the userid and password.