Thank you for your post.
Obviously, something has changed in your environment, and from your information, it is difficult to determine where that problem(s) exist. Since you are able to close files but not open, the connection from the Admin to the Server appears fine. However, I would quit the Admin and restart it to see if something was lost in the connection. I doubt this would be the problem, but it is the least evasive.
Assuming memory has become corrupt, find a time to close all the files on the server, stop the server, and reboot the machine. This will reset the memory environment. Do the same on the Admin machine. See if you are then able to open the files.
If the above fails, then move your database files to the Desktop (or if in a folder, move the folder to the Desktop), uninstall and reinstall FileMaker Server 8. Move your database files back to the appropriate folder(s) and see if you can open them.
Please let me know if you continue to have difficulty.
Thank you for your reply, TSGal!
Finally figured out the solution when I was going through FM server admin and looking at configurations. It turns out that I had exceeded the number of files allowed to be hosted by FM server. So I changed that number and now, I can open my files.