You can certainly move it to your Applications folder. However, each time you run, it will make a copy on your desktop. As far as I know, there's no way to stop this behavior as it is inherent to OS X's Java implementation, not to the FileMaker admin app itself.
OK, while this doesn't STOP it from happening once it's already started, but I know how to avoid in the first place. When you first launch the FMS10 Admin Console Page and you click "Start Admin Console", when it downloads the app it asks you where you want to save it to. If you pick the Applications folder, when you launch the app, it doesn't appear on the Desktop. I tried renaming the app to something a little more friendly, and when I launched it, it replicated itself, using the original name, in the Applications folder. So, it looks like, upon launch, it checks to see if it exists in the original download location, and if not, then it creates itself. Doesn't make sense to me (and I've never seen this before, even with other java apps), but oh well. While I can't call it something a little nicer, at least I can keep it in the Applications folder. I can confirm this working in OS X 10.5.6
Cool tip! Note, further, that you can rename the application at the time it asks you where to save it. You can start from scratch by junking your original app, then visiting http://www.yourdomain.com:16000/ to re-initiate the app download.