My guess (and it's a pure speculation based on what you are saying) is that FileMaker hadn't started work on that 'feature' until after 12.0v1 was released - and so hadn't added the logic to FM12 to 'not open' a file based on that flag until a later vrev of 12.
Will be interesting to see which vrev of FM12 this check was added. Not sure there is much that FM can do about it now however...
I suggest you add a check to the opener script in your solution and if FM12 - show an error dialog and 'Exit Application' - just as a double check.
Just did a quick test with FMPA13 and Mavericks - created new db from starter solution and tried to open with FMPA12.
“Research Notes.fmp12” was created by version 13.0 of FileMaker and cannot be opened by this version (v12.0).
Please use version 13.0 to view this file.
I made a brand new empty db with FMPA13 - noted that the minimum open was set to FMP 12.
"Book tours are almost designed to beat out of an author any affection he has for his book." ~~ Michael Lewis