This seems very similar to an issue I've experienced and reported (http://forums.filemaker.com/posts/4abb9fa829). It's actually more similar to the initial problem that made me dig into this (when I also had to force-quit), than to somewhat milder issue I'm able to "easily" reproduce.
Have you tried reproducing this in a new file?
I've been working extensively on one file so can't say I've noticed it elsewhere. Have yet to work out how to reporoduce the issue but will keep an eye on what might be triggering it and try it in another file.