Thank you for your post.
I have not encountered this issue, and I know that doesn't help you. In addition, I do not see any report of this behavior reported.
Are there any Layout script triggers enabled? For instance, OnLayoutEnter or OnLayoutKeystroke?
Do you have any plug-ins installed and enabled? If so, which plug-ins?
Is this occurring with one file or all files? If one file, is it occurring on one layout or all layouts?
Any other information you can provide may be helpful in determining the cause.
Sometimes, "weird mouse behavior" can be traced to a specific brand of hardware or to interference from another application also installed and (perhaps) running on the machine. So you may need to search for any possible common denominators...
A Mac OS X specific issue does exist, but obviously, i am not able to replicate it on demand and i don't know if you are really running into it or not. Mine is windows management related.
That is, even FileMaker Pro is the front / activated application, its first window is like inactive (you can see it looking its clearer title bar).
When this issue occurs, a double click is needed where a simple clic is generally enough.
Then, the way to restore a normal state is to activate the Finder and then, reactivate FileMaker Pro, using the Dock for instance.
So, the next time the issue occurs, you can try this process above, that not need a restart nor a quitting, and maybe post back here.
I know that not helps a lot but it is the most i can suggest at this time...
Good luck !
Oh sorry, i am not sure the Dock to activate / inactivate is enough in this case. If not, use instead [cmd]+[Tab]