1 2 3 4 Previous Next 51 Replies Latest reply on Apr 20, 2015 8:11 PM by CarstenLevin

    Scary Movie!!! - watch a FileMaker file loose its data

    CarstenLevin

      Let me be very clear: This is the first time I have seen this happen. And do not show it to children or people with faint psyche.

      scary1.png

      A short prelude:

      • A friend of mine build his own FileMaker solution. It is hosted at a very reliable FileMaker hosting company (a competitor to my own company).
      • He called me last Friday: "My FileMaker database is loosing data, and I am sure that it is not me who are doing something wrong".
      • I got acces to the database running at the server and could see that he was probably right.
      • I got the hosting company to send me a backup of the database from just before the problem occurred.

       

      The solution is originally build with FileMaker 9 or 10 by a business owner who have used FileMaker at his own company for many years. He is building his own solutions and although he is not "a developer" or certified or anything he know what he is doing.

       

      He claimed that when he added a new record and entered data it would at the same time delete another record. I had a look and at once saw that the new record I created got the correct serial number (106), but then I tried putting a field with the Record ID on the layout ... and even though I just created the record it had got Record ID "1". This should of course be impossible.

      record_id.png

      And ... should you not believe it: Have a look at this Scary Movie.

       

        • 1. Re: Scary Movie!!! - watch a FileMaker file loose its data
          CarstenLevin

          Additional information:

          • When my friends got the problem the file was running on a very stable Filemaker 12 server and used by FileMaker Pro 12 clients. Two users max.
          • I got the backup file from som hours before the problem occurred.
          • I Repeated the creation of new records after uploading the files to a FileMaker 13 server. The same problem occurred.
          • I tried it on my own mac, locally with FileMaker 13v5 and the same happened.

           

          Did the following, each time with a new copy of the file I got from the hosting service:

          • Recovered the file: No problems found.
          • Checked the file with FMdiff, no problems, never crashed ...
          • Deleted the indexation within the solution and created them again.

           

          And I also did something else that I found interesting:

          I made a new record and wrote the short name "PA" into the field for the short name. PA was the short name of Peter Andreas before his record was lost. Got an alarm: "PA" is not unique ... so even though Peter's record is taken over by Ursula and with ursula as short name ... the short name "PA" does still exist.

           

          IMPORTANT: Having used FileMaker since 2.1 and being responsible for a lot of FileMaker Servers and solutions I must say that I have not seen problems like his since FileMaker 6. Many years ago. And this is probably the first time I have been able to reproduce a problem like this.

           

          I am listening: Any advice or explanation.

           

          BTW: My friend is now rebuilding his database from scratch in FileMaker Pro 13. I thought him about Anchor Bouy, proper naming of TO/TOG's, using themes and many other new techniques. His new solutions will be much better than anything he previously had:-)

          • 2. Re: Scary Movie!!! - watch a FileMaker file loose its data
            iceman7

            I agree: Very frigtening!

             

            I've witnessed my client claim the same thing in recent months....that several of their staff report losing data. I cannot replicate their issues nor confirm them. Indeed I was resolutely incredulous: Couldn't happen, I thought!

            • 3. Re: Scary Movie!!! - watch a FileMaker file loose its data
              CarstenLevin

              Hi Iceman7,

               

              Having worked with FileMaker for 20 years I have had some reports of people claiming that they where loosing data in their FileMaker solutions. Except for the case that I am showing here I have always been able to find the error made by the developers within the solution and pinpoint that it was not FileMaker loosing data but just an error.

              Deleting a record without establishing context.

              Cascading delete.

              etc.

              This case I am referring to here is the first I have ever seen where FileMaker lost/overwrote data.

              So in all I find FileMaker very very stable and reliable.

              • 4. Re: Scary Movie!!! - watch a FileMaker file loose its data
                CarstenLevin

                Just got an email .... friendly .... but claiming that it seems that I am giving the impression that FileMaker is not reliable when I post a message like this one. To the sender ... please post it here, I will not be offended at all.

                 

                And it is very important for me to state, as I have already done, that FileMaker is indeed an extremely stable and reliable platform. We rarely if ever see any problem with integrity and never before did I see any data loss. And never when the files was hosted on a server (again the case mentioned here is an exception).

                 

                And it is a very old solution developed by a non-developer, although pretty experienced with FileMaker. And the solution was originally developed on a local desktop/laptop and may have crashed more than once before being correctly deployed on a server.

                 

                And my conclusion was that the file was damaged and that it should be rebuild.

                 

                BTW: I also showed my friend how to separate data from GUI and logic, building a two file solution instead of the old one file solution.

                • 5. Re: Scary Movie!!! - watch a FileMaker file loose its data
                  taylorsharpe

                  I used to run into weird things like this when I was more often working on non-hosted machines.  The problem being that force quitting and other such things tend to introduce corruption.  I've seen something happen like this after using a hacking security program to reset the Admin password.  But fortunately we were just using that export the data and didn't need to keep the file that was apparently corrupted by the security hack.  FileMaker's recovery feature often recovers files and makes them useful, but not always, which is why they don't recommend using a recovered file going forward, only to recover data from it. 

                   

                  One thing that you didn't mention that I've seen fix a problem like this before was an apparent corruption in the layout.  Try creating a new layout with the same table occurrence and try adding new records and see if it still happens.  If so, then the original layout is the corruption issue and not underlying schema such as indexing. 

                   

                  But I do have to say after getting most clients on FMS, these types of problems really have almost gone away for my clients.  I just don't even tell clients there is a peer-to-peer option. 

                  • 6. Re: Scary Movie!!! - watch a FileMaker file loose its data
                    iceman7

                    I've never personally witnessed FileMaker lose data either...and I too have been using FileMaker (off and on) for 20 years. Not that I can remember anyway.

                     

                    Several staff members using one of my solutions swear they have lost work. I think there might have been some very limited loss due to some repeated server disconnects (51) we were witnessing. I believe we have since resolved the cause of those disconnects.

                    • 7. Re: Scary Movie!!! - watch a FileMaker file loose its data
                      Markus Schneider

                      STrange.

                       

                      Assuming that there are no custom menues, relationships not been seen, etc: File seems to be broken.

                       

                      THis could happen, when a table has serial numbers (field definition), a developer opens manage database and goes to that table - and a user creates a record by script. No error-msg, no new record - but one record will be overwritten (but then, there appears no empty record...)

                       

                      SInce You have FMDiff: Try to contact Winfried Huslik...

                      • 8. Re: Scary Movie!!! - watch a FileMaker file loose its data
                        CarstenLevin

                        Hi Markus,

                        Usually I would contact Winfried and let him have a look at the file, he is very very knowing and competent. But since I am sure that the file is damaged and since I would probably not ever recommend my friend to continue with this file I will not do so this time.

                        I also believe that the result of rebuilding the solution based on modern techniques, the developer conventions my company is using, with Anchor Bouy, separation etc. etc. is a good idea. The relational diagram in the existing solution is a very impressive spider web:-)

                        • 9. Re: Scary Movie!!! - watch a FileMaker file loose its data
                          Markus Schneider

                          I do also think that rebuilding the db is what has to be done. Using 'modern' methods is the best.

                          BUt: We got a lot of older FM7/8 databases with incredible spider-web -> they all run! No problem other than losing yourself in that structure..

                           

                          WHen a file becomes broken, it will crash, will show empty fields, will no longer accept the credentials (etc.) - but I havn't seen this before (more than 20 years with FileMaker)..

                          • 10. Re: Scary Movie!!! - watch a FileMaker file loose its data
                            Benjamin Fehr

                            verry scary indeed!

                            mschneider

                            - I see 3 found records out of 62

                            - one engages the "New Record" functions

                            - there should be 4 out of 63 Records now!

                            Though whatever functions taking place, it is not create "New Record"! Explanations could be found in any corruption in Custom Menu though as if cmd/N alias New Record was assigned to any script.

                             

                            I'ld like to catch the attention of Fred(CH) who once showed me a trick with custom menu in order to have a simple "Find" OR cmd/F triggering a designated script (means doing anything but what it actually was supposed to).


                            Some questions:

                            - is this taking place on FMServer only and if though, is there a chance to do some tests with FMPro only?

                            - what do you see when you have Script-Debugger engaged and then perform New Record via the Menu OR via cmd / N?

                            - has anybody have similar experiences with FMPro OR even FM Runtimes?

                            - was this occurrence reported to FMI as a issue report?

                            • 11. Re: Scary Movie!!! - watch a FileMaker file loose its data
                              BruceHerbach

                              A few odd observations:

                               

                              First it created a new record,  but got ID #1.  Is it possible that the something/someone reset the record ID counter so that the new record got the ID of an existing record?  You should still have an additional record but duplicate IDs.

                               

                              Second,  is there a custom menu replacing the New record command with a script?  If so is there an error in the script?

                               

                              Third when you tried to enter peters short name,  it reported that this had to be unique and that value was taken.  Did you try doing a search on the field to see if it could find the offending record?

                               

                              Last, did you try reseting the indexes on the records in the table.  While you are not seeing corruption,  it sounds like the index is incorrect.

                               

                              HTH

                              • 12. Re: Scary Movie!!! - watch a FileMaker file loose its data
                                Benjamin Fehr

                                Anyway, this Movie is not gonna make it to the Oscars

                                • 13. Re: Scary Movie!!! - watch a FileMaker file loose its data
                                  CarstenLevin

                                  Hi Bruce,

                                   

                                  First it created a new record,  but got ID #1.  Is it possible that the something/someone reset the record ID counter so that the new record got the ID of an existing record?  You should still have an additional record but duplicate IDs.

                                  No, my friend does not know how to reset the record id counter. How is this done, did not know that it could be done. It is get get(recordID) that was used. I believe that the recordID is not among those attributes to a record that we can change.

                                  I know it can be done by creating a clone and then importing data once again (restarting recordID from 0) but since I do not trust the file I do not find this to be relevant.

                                  And your suggestion: Custom menu ... would still not create a new record with recordID = 1.

                                  If I search for Peters short name I will find Ursula. So behind the surface the ghosts of the original data is lurking.

                                  And I did of course delete the index'es and reindex. Same result. Strange, strange, strange.

                                  You can see it happening here:

                                  1 2 3 4 Previous Next