13 Replies Latest reply on Jun 28, 2011 4:32 PM by philmodjunk

    What is the cause of file corruption?

    sjfmacct

      Title

      What is the cause of file corruption?

      Your post

      Each week, we boot FileMaker server one time.
      Before booting. we
      1. make a copy of all FileMaker files
      2. disconnect all users in admin console
      3. close all files in admin console
      4. stop FileMaker server in cmd-line,
         fmsadmin stop server
      5. stop FileMaker service in in Microsoft Service


      Although we carefully, cautiously and properly go through every step, still
      every a few weeks, the primary file (3+GB, the largest in DB) got corrupted.
      We have to replace it with a backup file.

      Usually it stops at step 4. After typing password, it hangs forever.
      If you keep going through step 5, it will also show "stopping" forever.

      Thanks for any help.

        • 1. Re: What is the cause of file corruption?
          sjfmacct

          no sign of corruption on client side and on the server before booting.

          • 2. Re: What is the cause of file corruption?
            philmodjunk

            Why do you do these steps?

            What version of server are you using?

            • 3. Re: What is the cause of file corruption?
              sjfmacct

              FMS v10

              Step 2-4 were added to reduce the chance of corruption. They may help in some ways.

              • 4. Re: What is the cause of file corruption?
                philmodjunk

                How do you do step 1?

                How do you know the files are being corrupted?

                When you check your server admin controlled schedules, do you see any back up schedules with the status of "running" instead of "OK"?

                • 5. Re: What is the cause of file corruption?
                  sjfmacct

                  we use Filemaker backup feature in admin console for step 1

                  the file needs long verification each time when this happens. Sometimes, they can't be fixed.

                  we wait all backup finish (OK) before we can start step 2.

                  • 6. Re: What is the cause of file corruption?
                    philmodjunk

                    1) good to know you aren't attempting to copy an open file either manually or via a third party back up utility.

                    3) there's a known bug where server 10 will "hang" a scheduled back up. Your answer rules out this possibility--though a hung back up schedule would explain the difficuties you appear to be having in shutting down the server in step 4--which is why i asked about this.

                    I don't see how following these steps will have any effect on file corruption.

                    Rather, it sounds like the files are already corrupted (There are many ways a file can be damaged) and that this corruption is creating problems when you back up and/or close the files in question.

                    Have you tried taking a copy of these files down off the server and running a recover on them?

                    • 7. Re: What is the cause of file corruption?
                      sjfmacct

                      Yes, we did recovery two weeks ago. We also did it before.

                       

                      It is possible the file is already corrupted before shutting down the server.

                      Is there any way we can shutdown server smoothly even with corrupted files ?

                       

                      How can we tell the files have been corrupted when they are running.

                      Basically we can’t tell. Each time before booting, everything is fine.

                      This is why we try to figure it out if booting process itself is the cause of corruption.

                      • 8. Re: What is the cause of file corruption?
                        philmodjunk

                        Yes, we did recovery two weeks ago. We also did it before.

                        And what results were reported by the recover process?

                        What is the reason for shutting down the server in the first place? We leave our servers up an running 24/7 except for restarts after installing system updates.

                        There is no guaranteed method for telling that a file is corrupted. That's one of the reasons why I asked how you knew they were corrupted. Crashing files are one possibility and the most frequent symptom.

                        Scripts that ran correctly before but now fail even though no change was made to the system MAY be a symptom.

                        Simple finds that fail to find data you know is present may be due to a corroupted field index. Likewise sorts that fail to sort correctly.

                        Problems with a specific layout can be due to a corrupted layout.

                        Other possibilities exist.

                        Fact is a file can be corrupted and appear to be fully normal for quite some time until a specific action happens to encounter the corruption in just the wrong way and a problem occurs.

                        Things to keep in mind about Recover:

                        1. Recover does not detect all problems
                        2. Recover doesn't always fix all problems correctly
                        3. Best Practice is to never put a recovered copy back into regular use or development. Instead, replace the damaged file with an undamaged back up copy if this is at all possible. You may have to save a clone of the back up copy and import all data from your recovered copy to get a working copy with the most up to date information possible.

                        And there's a third party produced utility: FMDIff that claims to be able to detect problems that are not detected/correct by a file Recover.

                        • 9. Re: What is the cause of file corruption?
                          sjfmacct

                          recovery always successful, no error reported.

                          Is it basically true that step 2 to 5 doesn't really help anything because it neither increases or decreases the chance of corruption?

                           

                          • 10. Re: What is the cause of file corruption?
                            philmodjunk

                            More to the point, why close the files in the first place? What does that accomplish for you?

                            Once you've closed the file, stopping or starting the server and the service should have no effect on it one way or the other.

                            • 11. Re: What is the cause of file corruption?
                              sjfmacct

                              hard part is here, it is what we don't understand.

                              After closing the files ( corrupted or not),  suppose stopping FileMaker service and the server has no effect with these files any more. But time by time,

                              the service is still hard to stop. Whenever hard-stopping happens, the file or files need long time verification or repair before being able to use.

                              If hard-stopping doesn't happen, no verification is needed.

                               

                               

                               

                              • 13. Re: What is the cause of file corruption?
                                philmodjunk

                                Please note the first sentence: "If you need to restart your FileMaker Server..."

                                What I am asking is why you need to restart it in the first place.

                                The only time we ever need to restart the server is if a scheduled back up "hangs"--a known issue with FileMaker Server 10. When this occurs we can tell because the schedule's status shows "running" and no new backups take place. The admin console goes "deaf"--no longer updates to correctly report client activity and won't let you close files or stop the server, but individual clients can access and use the databse normally. When this happens, we carefully make sure that all clients have quit FileMaker and then Reboot the computer. We are unable to close the files nor use Console nor command line to stop the server due to this bug and and have instead to reboot the server and then wait for a consistency check to complete on all files before we can again access the database from a client computer.

                                Other than that, we let the server run 24/7.