I am currently using FMP11 on a Mac Running OS10.6 Snow Leopard.
Likely your file is damaged in a way that doesn't trigger an error in 8.5 but does in 11. Try reovering the file and see what is reported by the recover process.
I already tried that but the resulting recover does not even come close to the state that the 8.5 version. There are thousands of records in the DB that we don't have the time to look through every record for consistency. Is there any thing else I can do?
"I already tried that but the resulting recover does not even come close to the state that the 8.5 version."
I'm afraid I don't understand what you mean by that sentence.
Recover with 8.5 (You never know, it might work)
Best option is to find an older back up that does not trigger this behavior when opened with filemaker 11. If you can find such a copy, save a clone and import your data from your latest copy of the file, while both files are open in 8.5. Then open the file with filemaker 11 and see what you get.