Thank you for your post.
Unfortunately, I'm unable to duplicate the problem. This is what I have done.
I have files hosted on both a Mac and Windows servers, running FileMaker Server 10 on both platforms.
I launched FileMaker Pro 9.0v3 on Mac OS 10.5.7, accessed database files on both servers that contain tables with Container fields. I waited a few minutes, put my cursor in a Container field and selected "Insert -> Picture". After the drive began spinning, I was prompted for the file name and I could insert the file without any problem.
Are you using FileMaker Server 9? 10? Previous version?
Have you tried it with FileMaker Pro 10?
Any additional information that I can try here to duplicate the issue would be helpful.