2 Replies Latest reply on Jul 31, 2013 1:16 PM by PeterDurant

    Slowly importing records

    PeterDurant

      Title

      Slowly importing records

      Post

           My contact management database currently has 1.2 million records with 240 fields of various types.  The database size is about 3Gig and is hosted on a cloud server. 

           I am loading in about 150,000 new contacts at a time until I get to the point of about 3 million records.  In order to load the records I have to upload the database from the server to my computer, load the records and then download back to the server since if I try to load to the server it always times out.

           That's not so much a problem but here IS the problem: With each new 150,000 records the loading process gets slower - much slower.  It now takes 24 hours to load these records.  The last batch took about 18 hours.  By the time I get to the last set of 150,000 records it will take a week to load!  Does anyone have any suggestions that might speed this up?

        • 1. Re: Slowly importing records
          philmodjunk

               Go through your tables and check the storage options on all your fields. Turn off indexing and make calculations unstored in every place that it is possible to do so. The delays during import are most likely due to the need to evaluate each stored calculation (small part) and to update/build field and value indexes on your fields. (Note if you find you need to turn storage or indexing back on for a field after import, you can do so, but there will be  a delay while FileMaker makes the needed updates after you make that change and your table will be locked against users/scripts making changes while it does so.

          • 2. Re: Slowly importing records
            PeterDurant

                 Thanks Phil, as always that solved the problem. smiley

                 Peter