10 Replies Latest reply on Apr 26, 2015 1:02 PM by wimdecorte

    FileMaker Server quitting! ... global temporary file damaged.

    robwoof

      FM Server Advanced 11.0v3, Apple XServe running Mac OS X Server 10.6.8

       

      Four times in the last three months FileMaker Server has stopped itself due to a problem with a temp file. The Server log entries look like this:

       

      2011-11-29 10:05:10.953 +1100Error618ADS FM Host 01Database or temporary file "filemac:/Server HD/private/var/folders/4O/4OlgV5ExGaG6EqE0XkE2XE+++TA/-Tmp-/FMTEMPFM4ed29ffd0001.tmp" is damaged and has been closed. (824)
      2011-11-29 10:05:10.953 +1100Error54ADS FM Host 01FileMaker Server quitting! An unrecoverable error occurred: global temporary file damaged.

       

      These entries are then followed by events showing files being closed and users being kicked off, and then FileMaker Server stops. Since the database files are closed properly, no consistency checks occur. FileMaker Server starts without complaint via the Admin Console.

       

      The location of the problem file is inaccessible (owner fmserver, no other access), so I can't delete the offending file. Each time this error has occurred, the file name given starts with FMTEMPFM and ends with 0001.tmp, but it's not the same file name each time.

       

      The server has three internal drives. The boot drive has FMServer installed on it (obviously). The other two drives contain the database files and backups, respectively. All disks have lots of free space. I will be running a disk check on the machine late tonight when users have disconnected.

       

      Any ideas?

       

      Cheers,

      Rob

        • 1. Re: FileMaker Server quitting! ... global temporary file damaged.
          sporobolus

          on 2011-11-28 18:30 robwoof wrote

          FM Server Advanced 11.0v3, Apple XServe running Mac OS X Server 10.6.8

           

          Four times in the last three months FileMaker Server has stopped itself due to a problem with a temp file. The Server log entries look like this:

           

          | 2011-11-29 10:05:10.953 1100 | Error | 618 | ADS FM Host 01 | Database or temporary file "filemac:/Server HD/private/var/folders/4O/4OlgV5ExGaG6EqE0XkE2XE++TA/-Tmp-/FMTEMPFM4ed29ffd0001.tmp" is damaged and has been closed. (824) |

          | 2011-11-29 10:05:10.953 +1100 | Error | 54 | ADS FM Host 01 | FileMaker Server quitting! An unrecoverable error occurred: global temporary file damaged. |

           

          i would want to see the console & system log entries from around the same time;

          i wonder if there is another problem with the system and that this is just a

          symptom

          • 2. Re: FileMaker Server quitting! ... global temporary file damaged.
            Mike_Mitchell

            I realize this is a bit of an old thread, but I'm having the same issue on a Windows 2008 R2 box. From looking at it, it seems FMS creates a series of these temporary files each time the service is restarted. In my instance, some of them grow larger than others - in some cases, several GB - until the server finally gives up as CPU and memory grow to ridiculous proportions.

             

            I was able to isolate at least some of the problem back to some corrupted databases. Once I pulled them out of the Databases directory, the problem settled down some, but it's still occurring. Just not as frequent.

             

            Anybody else see this before? It's becoming a real headache.

            • 3. Re: FileMaker Server quitting! ... global temporary file damaged.
              wimdecorte

              This could be a case where other software (anti-virus often, sometimes backups) interferes with FMS.  If there is AV active on the box, disable it for a while and see if the problem goes away.

               

              Also scour the system and application event logs for any signs that other things are running into issues.  Could obviously be hardware related if the disk has bad sectors etc.

              • 4. Re: FileMaker Server quitting! ... global temporary file damaged.
                Mike_Mitchell

                Thanks, Wim. Unfortunately, I'm not able to disable AV software. The computer security people would have a litter of kittens.   :\

                 

                However, from looking at the Temp directory, it seems these files correspond to databases. As I repair corrupted files, the number of these temp files with abnormally large file size continues to shrink. I don't know what FMS uses them for, but there appears to be a correlation. After repairing a couple of these files and restarting, the server lasted about 24 hours without an incident. When I got another call this morning, there was only one temp file with unusual file size. I found only one file acting strangely, so perhaps we finally ran it to ground.

                 

                I would LOVE to get the AV software disabled on these boxes. My backups come up corrupted on a fair frequency, and we occasionally run into issues like this. All I get from COMPSEC is, "Well, that's a problem with their (FileMaker's) product." Grr.

                • 5. Re: FileMaker Server quitting! ... global temporary file damaged.
                  BruceHerbach

                  Mike,

                   

                  I'm sure you know this,  but FileMaker requires that the database folder not be touched by backup or Anti-virus software.  Having been an IT guy at one point in my career,  I was on the other side of some of these conversations. I hate to admit it, but what you got was my usual first response to a request of this type.  I would then wait for a reasoned response with some backup information, like the FileMaker FMS installation guide.  If something like that was provided I was willing to exclude a specific folders from the backups or AV scans.

                   

                  I'm a bit concerned that you say your back up files show corruption and that you are having these issues for a while.  I'm afraid that you will get this last file fixed today and have the problem again in the near future.  It may get to a point were you can't fix the files... If you can go back to COMPSEC and let them know they are the most likely cause of the issue.   Maybe they will be more helpful.

                   

                  Best of Luck

                  Bruce

                  • 6. Re: FileMaker Server quitting! ... global temporary file damaged.
                    Mike_Mitchell

                    Thanks, Bruce. Yes, I'm quite aware this is a no-no. I've gone over it several times - with documentation in hand - with them, and so have the server admins. We get, "DOE requires all files be scanned per policy. If this causes a problem with FileMaker, then that's a flaw in their product."

                     

                    I can try again, but I have my doubts.

                     

                    Thanks again.

                    • 7. Re: FileMaker Server quitting! ... global temporary file damaged.
                      wimdecorte

                      Mike_Mitchell wrote:

                       

                      We get, "DOE requires all files be scanned per policy.

                       

                      This is the crux of the problem.  As long as they think of FM files as "files" like Word or Excel then they will not change their stance.  But FM files when put on FMS transcend mere "files".  They are now hosted databases.  They should treat them like SQL server or Oracle.  That may be a shift in thinking that they can get behind.

                      • 8. Re: FileMaker Server quitting! ... global temporary file damaged.
                        Mike_Mitchell

                        Yeah. Since I’m still sitting here two days later, rebuilding files - er, databases - I might have some traction.   :-\

                         

                        Thanks, guys.

                         

                        One thing: I’m seeing a LOT of hung client sessions. Same IP address, multiple listings as “client” from fmsadmin (I have to use the command line on this remote). The CPU usage is still hovering at around 25%, but the logs don’t show anything. It looks like the server is trying to establish communication and can’t, and gets hung up. Consistent with damaged / corrupted databases?

                        • 9. Re: FileMaker Server quitting! ... global temporary file damaged.
                          BruceHerbach

                          Maybe you can have them scan the backups only by showing the frequency at which the main files become backups?

                           

                          Best of luck.

                           

                          Out of curiosity,  does anyone know if other server based database systems require that no other system touch the database while it is being served? 

                          • 10. Re: FileMaker Server quitting! ... global temporary file damaged.
                            wimdecorte

                            The other ones are not file-based like FM is.  And there is specialized AV software for it.  That's where the model breaks; FMS requires exclusive lock on the files because of its sharing mechanism.  And traditional AV software for file-based things like Word and Excel does not work well in that model.