1 2 3 Previous Next 31 Replies Latest reply on Aug 14, 2017 2:56 PM by TSGal

    File Corruption - Access Privileges Damaged or Tampered With

    honeycomb

      OS X 10.11.6 El Capitan

      FMS 16.0.1

       

      Four times in the past month one of our database files has become corrupted. Each time it happened I was alerted by a user who said they got an error that the "Script cannot be found or has been deleted." When I open the file (hosted on FMS), although it opens, I find all the scripts are missing from the file. If I try to open a backup of this file (clone or full backup) I get the lovely error message, "The access privileges in this file have been damaged or possibly tampered with."

       

      I can not run FM Recovery on the damaged file because of the corruption. However, FM Recovery on a backup of the file just prior to corruption returns no damages. I was able to use FM Diff to analyze the file, but it too, returned no damages.

       

      This is the same file that has experienced a font substitution change ( Font substitution problem). The thread for the discussion for some reason has been removed. The font substitution occurred several times while the file was hosted using FMS v15. Since moving to FMS v16 the substitution has not reoccurred. The author of the thread also had another thread going wondering if FMS was corrupting their file...

      FileMaker Server might be corrupting our file??

       

      Here's some background on some of the issues....

       

      We haven't really experienced any other issues with the file other than the font substitution problem until recently (the past couple months). The file is approx. 11 GB and has anywhere from 50-90 users connected most of the work day. For years we were able to make small database schema changes on the live file (like fixing a calculation, or adding a new table occurrence), then one day it locked up during an update and we had to force quit FMP. Users logged in at the time of the lock up were able to continue working but if they logged out and tried re-opening the file, it would never open. We had to reboot the server for users to reconnect. We did restore to a backup prior to the force quit just to be safe.

       

      We have also made changes to privilege sets on the live file, though not very often. We have never had an issue doing this but a couple months ago the file did lock up while updating changes to privilege sets. We had to force quite FMP. When we went back in to Security, the changes we made were actually saved. We ran Recovery on the file and found no corruption, so we stuck with the file.

       

      Fast forward a couple weeks after the Privilege set issue and the file experienced the "Script could not be found..." error for the first time. Again, the file can be accessed on FMS (but no scripts are in it), but a local version of the file can't be opened due to the "Access privileges have been damaged" error. Each time this has happened, we have reverted to a backup.

       

      We don't know what's causing the issue. I haven't been able to find a common denominator each time the corruption happens. Though it does seem that it occurs about the time a backup runs. The most recent corruption happened last night. The 8 pm backup finished at 8:02 pm. I logged in about 8:04 pm and saw the "Script could not be found" error. The 8pm backup was fine. The file was corrupt between 8pm and 8:04 pm.

       

      Looking at the logs, I am seeing this each time this issue occurs....

      FMS log.png

       

       

      I'd like to avoid having to rebuild the file if at all possible.

        • 1. Re: File Corruption - Access Privileges Damaged or Tampered With
          scottworld

          Hi honeycomb, the thread that you mentioned entitled Weird font substitution problem was actually MY THREAD! And there were 161 messages in it. I didn't delete it or remove it, so it seems like someone at FMI must have removed it. I can actually still see the thread privately on my end, but nobody else can see it publicly. So someone must have done something to it behind-the-scenes at FMI, unless it is a bug in the Jive Forum Software. I wonder if FMI doesn't want people knowing about this font bug?

          • 2. Re: File Corruption - Access Privileges Damaged or Tampered With
            honeycomb

            My apologies scottworld. I have notified TSGal of the missing thread . . . they are looking into it.

            • 3. Re: File Corruption - Access Privileges Damaged or Tampered With
              TSGal

              honeycomb:

               

              Thank you for your posts.

               

              Recover will do its best to find damage/corruption and clean it up.  It is not a cure-all.  With that said, if the 8PM backup is fine but the 8:02 PM backup is damaged, then please send in a clone of the 8PM backup so we can determine the cause of the corruption.  I have sent you a private message with instructions where to send the clone.

               

              TSGal

              FileMaker, Inc.

              • 4. Re: File Corruption - Access Privileges Damaged or Tampered With
                FileKraft

                TSGal, the reference to the issue Font substitution problem links to this:

                 

                 

                Unauthorized

                 

                 

                 


                Access to this place or content is restricted. If you think this is a mistake, please contact your administrator or the person who directed you here.

                 

                • 5. Re: File Corruption - Access Privileges Damaged or Tampered With
                  shaunbicheno

                  I had the same problem with FMS15 back in March, just after going live with a new system.

                   

                  I am fairly certain that it came from my messing a little with the access privileges while people were logged on and I had to go back to backup as well.

                   

                  I am growing concerned that FMS15 & 16 have become far less tolerant of schema changes and indeed of database design practices, than previous versions. With an almost total absence of guidance in this matter however all I can do is speculate.

                  1 of 1 people found this helpful
                  • 6. Re: File Corruption - Access Privileges Damaged or Tampered With
                    honeycomb

                    Have you had subsequent issues with your db file? Although we've gone back to "clean" backups prior to corruption, it's happened 4 times in a month. We are backing up every half-hour now, 24 hrs a day.

                    • 7. Re: File Corruption - Access Privileges Damaged or Tampered With
                      scottworld

                      This isn't a direct reply to what you guys are talking about above, but we are still having ongoing weekly problems with our file... where FileMaker keeps changing the fonts on all of our layouts. My original thread (which had 161 replies) is no longer visible on this forum -- perhaps FMI doesn't want people knowing about this bug? But the font problem is still happening to us on a weekly basis, and my latest thread about this topic is (at least, temporarily) visible here: FileMaker keeps changing fonts in our file

                      • 8. Re: File Corruption - Access Privileges Damaged or Tampered With
                        shaunbicheno

                        I've not had a re-occurrence of the "tampered" corruption, but the file seems acutely sensitive to querying relational records. I too have looked at recover logs and gone through the arduous process of DDR and Top Call Stats analysis. The wait times spike inexplicably and the whole workgroup will have to suffer interminable dialog boxes saying "Find in progress, processing query..."

                         

                        There are a number of posts on this dialog already and I have had discussions on this with some fairly high-profile developers, who have also experienced them and/or a dramatic drop in performance after upgrading to FMS15 or 16. For this reason I can not be sure that the two issues are related.

                        • 9. Re: File Corruption - Access Privileges Damaged or Tampered With
                          shaunbicheno

                          I have replied to you in the thread you linked to in your post scottworld. I trust you're OK keeping this thread about the access privileges "tampered" corruption.

                          • 10. Re: File Corruption - Access Privileges Damaged or Tampered With
                            honeycomb

                            It seems similar to this thread for sure. All Scripts DELETED

                            1 of 1 people found this helpful
                            • 11. Re: File Corruption - Access Privileges Damaged or Tampered With
                              honeycomb

                              Coincidentally (or not), my file also has experienced an acute sensitivity to querying relational records. Like you, I don't know if this issue is related to the "access privileges damaged" or "all scripts deleted" issues.

                               

                              We implemented a new module (basically, just a transactional table to track status changes), which has a table that is queried on a lot and for weeks it ran smoothly. Then one Monday morning the wait times were astronomical, making the file nearly unusable. The only difference from the preceding Friday and the doomsday Monday was that we ran a batch script to bring data from the "old" setup into our new setup, which created an additional hundred thousand records or so. Deleting the newly added records didn't do much to fix the problem.

                               

                              This table is very flat, just a handful of fields, none of which are unstored calculations. The relationship in question was nothing out of the ordinary. The table is called "StatusHx". The relationship is a self join to show previous StatusHx records of the same parent record. Here's the relationship.

                              Screen Shot 2017-08-07 at 11.25.21 AM.png

                               

                              All of the fields are indexed. The join on "KpStatusHx > KpStatusHx" is the one that caused the slowdown. "KpStatusHx" is just the primary key (a numeric serial number). The sort on the relationship is inconsequential really . . . it's just sorting descending on an indexed timestamp field (removing the sort criteria does nothing to fix the problem).

                               

                              The number of records that would result from the relationship would be anywhere from 0 to a max of 10 or 12. I didn't even have to perform a query via the relationship to get the slowdown . . .  just loading the portal of the relationship could take 30 seconds.

                               

                              I've implemented joins like this for years and have never seen a slowdown like this. Even on a local copy of this file, loading the portal records takes a noticeable split second. Removing the join on KpStatusHx makes the portal load instantaneously.

                               

                              Thankfully, this relationship was only needed as a part of a script. I was able to work around it by doing a find for the correct records as opposed to accessing them via the portal relationship.

                              • 12. Re: File Corruption - Access Privileges Damaged or Tampered With
                                FileKraft

                                how are your cache settings?

                                (most issues we experience are related to lazy evaluation - index not yet written back to disk etc ..

                                refresh issues,

                                all consistent with questioned reliability for PSoS etc ..)

                                 

                                also index on records with record open state 1 or 2 might not be written back quickly enough so Pause of 1 sec or partial second might alter your experience which i consider  kludge.

                                • 13. Re: File Corruption - Access Privileges Damaged or Tampered With
                                  shaunbicheno

                                  My woes may be narrowing down to a link table to link sales orders together. It is very simple and does not have any aggregate calc but whenever somebody is using a "linked order" everything seems to slow down to a halt.

                                   

                                  I am of the opinion that FMP could be doing more to support these multiple reports of performance loss. I am not alone in thinking that the last 2 versions are - in theory - a triumph but that it seems that the ground has changed from under us, that things which worked well before no longer work so well and there is no signs that these issues are being seriously looked into.

                                   

                                  In the meantime, it is us that are taking the heat from FMP's paying customers and I for one am getting tired of apologising, shrugging my shoulders and telling them that I'd doing the best I can. I need to hear something meaningful from FM on this: If the corruption were tracked down to some kind of problem with the file then I would create a new file and import the tables, hook them up again and import the scripts. It would be a pain but ultimately worth it if I had some proper intel that it might just sort these go-slows out.

                                  • 14. Re: File Corruption - Access Privileges Damaged or Tampered With
                                    honeycomb

                                    I don't think it's cache issue, but I'll look into it. Keeping the relationship but changing the join an "=" instead of ">" makes the delay go away completely. Even with one record being returned in the portal, there is a delay when the ">" operator is used. If it's an "=" it loads immediately.

                                    2 of 2 people found this helpful
                                    1 2 3 Previous Next