AnsweredAssumed Answered

Successful Synchronization without Server

Question asked by mikewson on Feb 3, 2015
Latest reply on Feb 4, 2015 by mikewson


Successful Synchronization without Server


Over the last several years we have successfully developed a robust synchronization product that does NOT require FileMaker Server. The Mobile client runs on either FileMaker Pro or FileMaker Go and syncs with a Master copy wherever it is. It supports any number of laptops or iPads. Syncing to a Master copy of a solution without the file being hosted on Server, is critical for us since the vast majority of our clients do not have Server. For those companies with Server, the solution still works.
Several well known products are available used for syncing with a hosted file on Server. This note, however, provides a brief framework for those needing to build a sync system where the solution is only hosted over peer-to-peer, without the special functions that Server provides.
Five Key Concepts:
1- Two Files.  On each Mobile unit (Laptop or iPad) are two files: Mobile and Sync Connector.
The Mobile file is simply a pared down version of Master with fewer tables. The Mobile file has no direct external file references to Master. All connections take place through Sync Connector. This is because FileMaker gives us no control over opening and closing connections. So, by putting the external file references in a separate file that we can open and close, we can control our connections to Master. The Sync Connector file servers as a link between the Mobile solution and the Master solution. Sync Connector contains no tables! It is only for linking. Sync Connector file has one external file ref to the Master db and a second local file ref that is only used for preloading data at set up. Sync Connector file has exactly one TO to each syncing table in Master and one TO to each syncing table in Mobile. Each TO has exactly one layout representing it. Every layout is total blank with no fields! All scripts that perform the syncing are in Sync Connector.
2- Pre-Load Data.  Preload the Mobile files from bulk tables with standard import scripts while Mobile is temporarily in same directory. Imports are about a hundred times faster if done locally on same machine. Remove syncing is only for records newly created or modified after the PreLoad.
3- Packing. During Sync, any records to be sent (up or down) are packed into a $$DataBlock from one layout, then they are unpacked on the corresponding layout. For example, Mobile Customer records are packed from the Mobile Customer layout, and then unpacked on the Master Customer layout. Note that the two layouts are in Sync Connector.
“Packing” is a quasi XML syntax where field contents is wrapped in opening and closing tags based on the Field name. Carriage returns delineate new records.
By packing the record data in to $DataBlocks, the transmission errors are almost totally eliminated and syncing is dramatically faster.
4- IDs must be managed. For example, each iPad has its own iPad number that is used in conjunction with a serial ID to identify records and downstream related records on that iPad. But when those records are sent up to Master, they will be given new standard Primary Key IDs. Those new Primary Keys must be pushed back down to Mobile to replace the iPad Specific IDs. Records on Master and Mobile that are downstream of the synced record must also have their IDs updated.
5- Modification Timestamp, when compared with Last Sync TimeStamp, is used to find records that have been modified. Instead of the normal auto-enter function, which set updates after any change, we have a custom calculation used as an auto-enter that ignores changes made by the syncing scripts. That is, it only updates the modification timestamp field when a user directly changes a field. Changes made by running scripts do not update the time stamp.