External Containers in FMS16 backup folders, unexpected behavior

Discussion created by sleepy62 on Mar 12, 2018
Latest reply on Mar 13, 2018 by sleepy62

Hi, We are hosting a large solution on a FM16 Server on Win Server 2016.


I have been researching the use of containers for the doc management area of our solution. We have been using TroiFile and a web server but for the smaller requirements we have been using external containers. So I am quite used to how they work and have never had a serious problem with broken links etc. In fact I have found the data structure to be far more robust than some have indication in these forums.


That said our doc repository can grow to several TB with thousand of documents so I was concerned about storage space and backup performance. After reading every bit of documentation I could find and following extensive experimentation I can conclude:


Backup performance seems not to be an issue as it only backs up changed or new files. Non changed items are referenced by hard links to previous backups. At least that is my understanding.


Storage space does appear to be an issue. Although the system is supposed to be creating hard links to previously stored backups, it still seems to be allocating space to them. All of the database backup copies appear to be exactly the same size even when no changes have been made. The parent backup folder also grows correspondingly in size. This is unexpected but perhaps my understanding of how hard links work is flawed.


In addition, the external container files inside the backup folder are hidden so is difficult to see what is going on. I know they are there because I can see the parent folder properties and I also did a “DIR -force” in the power shell and the files do show up.


The fact that these files are hidden is unexpected as I don’t recall that being the case with FM12 and 13.


When I copy the folders to an ftp server the mode changes and they do become visible. However if I replace the live system with one of the backups the container files are still invisible in the live databases folder. Although the database is working perfectly with all containers showing up populated. This seems wrong.


Is this normal? I can find no discussion on this phenomena.


I realize that it is ill advised to go digging in the RC_Data folders but a clear understanding of how the backup system works is critical. In particular I am concerned that the following a crash recovery the live database will end up with multiple hard links to backup folders, if that makes any sense.


Im OK with things going on inside the filemaker DB file that I don’t need to know about, but when files are being shuffled around in my OS I would like to know what exactly is being done.  It all seems a bit opaque.


I would appreciate and comment or guidance anyone might have.


PS. A bit of extra info, I've just noticed that the files in the Progressive backups RC folder are visible in explorer. However those in Scheduled backup are not. However I tried turning off the "Hide System OS Files" option in Windows and all the files are now visible (albeit shaded out indicating they are supposed to be hidden). I cannot understand why in on instance they would be declared hidden system files and in another they are not.