It will give you that message when you use an upgrade licence, and the installer can not find an older version of FM on the machine. In the next step you will be able to enter the licence code of the older FM install to verify that you are installing an upgrade.
Hope that helps,
Ruben van den Boogaard
This message should appears when you are trying to install an Upgrade version of FileMaker Pro (maybe the serial used it's an upgrade version and the installer try to find in your PC an older version that qualify for the upgrade; nowdays in this particular case you can only upgrade to FMP 12 if you have installed a FMP or FMPA 10 or 11).
Hope this helps (and remember to validate my response here, thanks).