Impossible to day without access to your deployment and going through all the logs
- if youare on OSX: permissions issue on the backup folders
- running out of disk space on the system drive, drive where FMS is installed on, drive where the backups are written to
- outside backup mechanism that is trying to backup the live files or the backup folders and preventing FMS from completing the backup
- OS-level indexing on the live files or the backup folders
- Anti-virus on the live files or the backup folders
So if I was making a change on the live database while the backup was running that could of caused the error?
could happen I guess; when you commit a change (like save a layout, get out of "define database", save a script) then FileMaker server has to lock parts of the file to replace the old schema elements with the new ones.
It is more likely I think that some of the other factors are at play, were there active users in the database as you were making changes?
Yes there were active users while I made the change. And I made a modification of a calculation couple minutes before the backup error occurred. For some reason the change took a long time to commit. More slow than usual.
That would explain it. When you change a calculation FM has to both update the schema and get a lock on that and get a lock on ALL the records as it has to update the result. If any other user is trying to maintain a lock on a record you can certainly have lock contention going on.
It's one of the reasons why the best practice is not to do development on live files...
Were you aware of that best practice? If yes: what was your evaluation of it? Any particular reason why you thought it would not happen to you? Not trying to be flippant here, but as a community we seem to have some trouble in having the best practices accepted or even plain acknowledged. Is it because they are not know, is it because developers think they don't apply to their situation?
First, while this is NOT your situation, should someone search for the 8003 error and find this thread, there is a known bug in FMS 11 that caused this occasionally. Even though this was not a commonly found bug, FMI engineers released an update for FMS 11 AFTER FMS 12 was released. Impressed me and rescued a client who was running into this occasionally ... until the client was ready to upgrade to FMS 12.
Second, best practices are great guidelines to follow to insure the best possible outcomes. But they are guidelines and not mandates. There are many respected developers who develop on live files. The key is to know the risks involved and do a risk/reward analysis of your specific situation. Developing live on a small database with a handful of occasional users is decidedly different than doing so in a busy production environment with lots of active users. You SHOULD be very hesitant to go into Manage Database on a live file under any circumstances, because the risks are great. But so are the rewards. Evaluate your own situation and decide what’s best for you (but if you do develop live, do so with a ROBUST backup scheme in place).
Chad Adams of Skeleton Key did an excellent video on one of the key dangers of working on a live database: http://fmrift.wordpress.com/2010/06/16/possible-dangers-when-working-on-a-live-database/#more-78