No, you do not have to pay royalties to FileMaker for apps you have created and sold. You just simply need to purchase a copy of FileMaker Pro Advanced to create the runtimes.
Some good information about runtimes in the help file: http://www.filemaker.com/11help/html/fmpa_tools.24.12.html
Be sure to pay attention to the information FMI provides about your obligations when distributing runtimes, including notices which have to appear in them, and your tech support to users.
Because you are the software provider when distributing a runtime, you take on some obligations, just like FMI does when selling you a copy of FileMaker. Just make sure you know what they are, and are ready.