3 Replies Latest reply on Nov 21, 2016 12:10 PM by keywords

    Data changes while exporting - what happens?


      Hi all,


      I am working on a solution that includes some potentially very lengthy exports ( as in hours). This solution requires me to keep track of the last time the data was moved, i. e. I will store a timestamp so I can pick up the changed data next time. Now I am wondering if I need to safe that timestamp before or after the export. I assume before but would love to hear from you. Especially interesting: what happens if the data is changed during the export? Does the export consider that?






        • 1. Re: Data changes while exporting - what happens?

          It would be helpful to explain in more detail as to what you are doing and why you are doing it. Having the "big picture" in mind helps us to help you and might lead us to make entirely different suggestions.


          I've never tested it, but I suspect that changes made to date while it is in the midst of a lengthy export might or might not show in the exported data simply based on whether the change was committed to the server before or after those records were fetched from the server in order to provide data to the export.


          You might find it safer to do this type of task offline from a recent back up copy of the file pulled off the server for that purpose. You can then set your timestamp to be that of the date/time of your backup copy.

          • 2. Re: Data changes while exporting - what happens?



            the big picture is somewhat like this:

            The solution is used by different clients, some large some small. It is a bit of a legacy database, there is a lot of givens at this point I just have to work with, one of them being a large amount  of unstored calcs. While the plan is to get rid of them step by step, it is not doable right now. The part I am doing the export for, is a universal reporting engine in which clients can compose a variety of different reports depending on their needs. I guess you can all imagine what happens when you do complex finds and sorts on calc fields - it gets slooooooow. So right now the solution is to export the relevant data (which evaluates all the calc) and have an indexed reporting data-warehouse. This process will happen over night, so we are to talking about a very busy system at that time. And while it is unlikely that data changes in the middle of the night when I am running those exports, I need to make sure to capture everything.


            I hope this helps to put this in context a bit!

            • 3. Re: Data changes while exporting - what happens?

              Create a Reports table (or separate file) so each export is a new record in that table. Each report will then remain as its own archive record.