You can use FileMaker 11 to open version 5 files. FileMaker 11 will generate new copies of your existing files converted to FileMaker 11 format. The conversion doesn't convert every detail of your original files perfectly so you then have to test and make adjustments to scripts and a few other details like accounts/Password settings but it is possible to then use these converted files with FileMaker 11.
Here's a knowledge base article with more on the subject: http://help.filemaker.com/app/answers/detail/a_id/750/kw/converting
Note: you can download the free trial copy of FileMaker 11 and use it to convert copies of your files to see how smoothly the conversion goes with your specific files and not have to spend $$ to do so.