We have been running FMP 11 on Windows for months without experiencing this kind of problem.
Have you tested chained deletion via portals or other such potential side effect operations, was there any change in the OS etc.
There is usually an explaination for this kind of behaviour, reverting back to the older version might be a good option until you can figure out what is happening.
Better to regain the users confidence.
With regards to file corruption and rebuilding a file (Which may not be the cause of your issue here)...
You may be able to rebuild the file's indexes and resolve the problem. If you save a clone of your file and import the data into this clone, it should rebuild all indexes during import and this might resolve the issue.
When you ran a recover on the files, were any problems reported by the recover process?
"as the records have embedded images in them and its not feasible or possible to completely rebuild the databases with the image files." How are these images embedded? If they are store by reference container fields, you can import these references into container field in the new file just like any other data.
"When you ran a recover on the files, were any problems reported by the recover process?"
No, it didnt report any issues.
"How are these images embedded? If they are store by reference container fields, you can import these references into container field in the new file just like any other data."
I am wary of attempting this, they are images in container fields on the data base records, but many of the original images no longer exist except within the records. Given the history with these databases its seems unlikely I would be able to rebuild with the images intact.
"There is usually an explaination for this kind of behaviour, reverting back to the older version might be a good option until you can figure out what is happening."
I am sure there is an explanation!! I just wish I knew what it was, I am reluctant to revert back to an older version because I fear it may add more issues rather than reduce them, and I would still have no confidence that we would not suffer further data loss in the future.
I am more inclined to migrate all the data out of FMP and get something like an SQL database built from scratch at this stage.
Its really not possible to continue on with the constant threat of data loss, so unless I can identify a cause and rectify it then I will have to look at more stable and secure alternatives.
Unfortunately the lack of meaningful support from FileMaker limits the options, essentially they have told me that this happens 'sometimes' that data base corruption is usually the cause, and have offered no more that the standard PDF on data recovery. Given the absence of a 'clean' backup that we know is un-corrupted and the difficulties posed by the images in the container field that document was really no help.
If you believe its a corruption issue and need to get the images, then you could create a script to extract all the images, and capture the image path for each record.
Once you have extracted the images you can clean out the container fields, clone as per PhilModJunks suggestion and re import the image again using another script, this time you can consider using a container reference rather than embedding the images, if you think it would help.
Thanks Conor, I guess I could try doing all of that with another copy of the databases on one of my personal computers so that it was completely isolated and if successful then migrate it all back to the server.
Do you have any ideas where to start with a script to extract the images and image paths? I have not looked at how FMP handles images within the database.
I really appreciate you taking the time to assist me with this.
I would try importing the images first before trying more labor intensive approaches. Since you are importing images into a new copy, if it doesn't work as it should, you have lost nothing as your current file is unchanged. If it works, you've saved yourself hours of work. I really doubt you'll have trouble importing the images if they are present in the original file to begin with.
If recovering a file doesn't result in a message telling you problems were found it is most likely that there were no problems with the file structure itself. It's possible that the recover simply didn't detect the problem--recover isn't perfect--but it's more likely that you may have a problem with an index and importing all your data into a clone of your file will rebuild all your indexes and this may very well correct the problems you are having.
It's also a good idea to play detective and try to identify ways that records or data may be deleted in unexpected ways:
- Incorrectly implemented scripts with Go To Related Records can modify or delete large numbers of records in one swoop.
- Delete options in relationships can delete related records when a record in a related table is deleted
- Layout based script triggers can fire unexpectedly in the middle of another script's execution, such as when another script uses Go To Layout to switch to a layout where an OnLayoutLoad trigger is defined.
There's no magic way to identify these causes, but if you analyze your design and ask your users questions, you might spot a possible cause for your disappearing records.
For more on the hazards of Go To Related Records: The Complete Go To Related Record
Thanks Phil, I am perservering with the trouble shooting! I will investigate all your suggestions.
Should I be running FMP Server on the server? Does it offer any real world advantages over the standard install for the type of use I have described?
Server provides a number of advantages:
- Better protection against possible file corruption if a client crashes
- Automated Backups that can be performed without closing the file
- Better accesse control
- Support for large numbers of users
- Faster response times
It's also much more expensive and requires a server with suitable hardware specs (You can find these in the knowledge base). So you get more but you also pay more.
Does anyone know if this was ever resolved?
I have a very large DB holding a few hundres records and have run into a similar problem. we recently discovered that once a record has been created and a few days later the billing department gets arround to creating the bill for that record its no longer there.
This is causing many problems since we can not bill and this ends up in a big loss for the company. Fortunately we are keeping daily backups from where we cwn recove the record but having to do this on a weekly basis is just not practical.
As galumay did, you'll need to do some careful analysis of your database design and function to check all possible causes. There are database features that if improperly implemented, can delete or hide records in unexpected ways. After you've ruled out those causes, you can consider whether a corrupted index or other file damage might be at fault... Don't forget to consider and investigate user error also.