My suspicion is that the container fields have to remain due to the new hard-linking of backups to unchanged data, in this case the unchanged externally-stored container content data from older backups.
It would be nice to get input from FileMaker staff on this.
As that may very well be the reason behind the error, I would say that this then becomes a bug, since it is an error being generated by a designed process. But still there is the issue of container copies being kept but the databases referencing them being removed...
Continuing this thought/suspicion, which if true, causes a problem in the backups folder, since the idea of keeping a Maximum number of days worth of copies is meant to purge out-dated files, keeping the storage allocation of backups from increasing daily.
This is actually what the new Hard-linking in the backup system does to reduce file retention. The old FM files are removed when no longer needed, and only the Hard-linked container data which is still referenced in the newer backups is kept from the older backups. This allows FM Server to avoid the time and disk-space to repeatedly write container data that hasn't changed.
The net result is less disk space used and less time taken to perform the backup. Hard-links are a new file-management system of FMServer12, and one cannot expect them to behave as a standard OS file directory system.
(Remember that manually changing anything in the backup directories can corrupt your backups. Backups should be copied when needed, not moved. This allows the hard-link system to function as intended without user interference.)
But I have container data in EACH daily backup folder. 14 daily copies of container data. And when FMS goes to delete the old 15+ day old folders, it succeeds in removing the databases but returns this error when attempting to remove all the folder contents, leaving all these container files. And for me, I have 1.5G of container files in each of the daily 14 folders. So, .... I hope this detail paints the picture more fully!
I have 1.5G of container files in each of the daily 14 folders.
I don't have an answer for your overall question but I wanted to chime in on this one aspect: unless all of container data was changed between backups, you don't have 1.5GB in each of the backup folders. That's one of the major changes in 12: any file that is not changed between backups is hard-linked and does not take up additional disk space. So for instance if none of the container data changed since the oldest backup then those 14 backups together will use 1.5GB overall, not 14 times 1.5GB....
Is this affecting what I am seeing at the Desktop level?
The Backup folder size is showing 9.8G.
Yet each of the 14 daily folders are 4-5G each.
The math doesn't add up, which is, I assume, what your are pointing out.
That brings up a new potential problem: I am daily copying the complete Backup folder over to another XServe RAID drive, not replacing current contents. This was theoretically giving me a continual backup strategy beyond the 14 day Maximum setting on the FMS machine.
Am I getting the expected results? If what you say is what I think it is, then I'm not. I won't be able to go open a 27-day old database and have it be a full copy as I am assuming! Yikes! Glad this is my test server and not my live server - yet....
You are being deceived by the new Hard_Linking of files. You cannot read info about hard links at the OS level the same way you would read a Get Info or Properties tab because the hard-link really is reading the linked data it's linked to, rather than the link itself. It's a little like shortcuts or alias files, but you can't see them as such.
The only Unexpected result is that you are seeing an Error, and you shouldn't get an error in FMS12 for normal behavior.
I asked Dave Simerly to get FM staff to chime in on this due to the unexpected error.
And, once again, you should never be touching the backup files except to COPY to them to another location; not to open, view, or move them. You don't want to mess up the Hard Links by moving or changing anything inside the backup area of the server. This is an entirely different type of file llinking technology than FM has used before.
Appleworld may be dealing with a hard apple,
Hard links are maintained only on the OS drive where they are created. The Hard Link points the copying process to the original file. A file copied to a new drive or computer from a Hard Link is copying the original file, not the Hard Link. If you review your external backups you will see they are complete and contain copies of the original files.
Did you ever get this resolved? I'm now having this issue with one of my backup scripts.
Is this simply a reported error with the oldest backup being deleted but leaving the containers intact, or is it messing with the files themselves.
This appears to be a bug in the reporting only. It should be leaving the older containers which are hardlinked, while deleting the rest of the oldest backup files. It simply shouldn't be reporting an error when this happens.
I suggest reporting this to FM Inc as a bug in the hopes they will fix the erroneous error report in the next server rev.
Stephen Huston wrote:
It should be leaving the older containers which are hardlinked, while deleting the rest of the oldest backup files.
That's not accurate. In Windows Explorer or the OSX Finder, the oldest of the backed up files (FM and RC) and the folder they are in should go away completely, whether some of the files are hard linked or not. If they are hard linked to other backup sets then their inode (the space on the hard disk) will not be reclaimed, but the files as we see them in Windows Explorer or Finder will be deleted.
The bug is that the older backup folder and RC data files are still visible in Windows Explorer / Finder.
I was referring to the "source" files to which the later backups are hardlinked. Don't those container files have to be preserved for the new hardlinks to work?
No, there is no concept of source file or original file in hard-linking. At least not on the level of the files as we see them in Windows Explorer / Finder. Hard linking is a mechanism that happens on a much deeper level of the OS than what is exposed to us.
The only "original" is the inode. What we see are just representations of the various hard link references to that inode, hard links are nothing but a string of text that is the path and filename. If there are multiple hard links to an inode then you basically see a list of paths and filenames that all point to the same inode. In essence, in Windows and OSX alike, every file that we see in Windows Explorer and Finder is a hard link reference to an inode.
What FMS does when it deletes an older backup is it removes the hard link reference to that path and filename (what we see in WE / F). If it happens to be the only hard link reference in the list for the underlying inode then the inode is "deleted".
If the actual disk information for the container data is not really those folders, then the failure to delete them should be reported as a bug to FMI. My suspicion is that there is a problem with trying to delete the "first" hardlink if the data still exists. My assumption before reading your post was that the data itself was there and should not be deleted to preserve the validity of later hardlinks. If your explanation is correct instead (not doubted you, just news to me) then the failure to delete the oldest hardlinks for Appleworld and some others is a bug in FM operations.
My suspicion would still be that there is a basic conflict when trying delete the hardlinks just because they are hardlinks, and the source data has to be protected. FMI will need to address this if the actual container data storage area of the disk is off limits to the visible OS.