AnsweredAssumed Answered

When is a file too big and what do you do about it?

Question asked by peterdurant on Mar 17, 2017
Latest reply on Mar 20, 2017 by peterdurant

I am creating a voter database with one layout and about 25 fields:


Voter ID

Election 1 Name (i.e. 2016 Presidential Election)

Election 1 Date  (i.e. 11/08/2016)

Election 1 Abr  (i.e. PRES-16)

Election 2 Name...


There are about 6 million unique voters with anywhere from maybe 300,000 to 3.5 million people voting in each election (depending on the election).  I have to import each election separately (if the voter ID shows up in the import file then they have voted). I Update Existing Records based on Voter ID and add remaining records as new.


I can only get about 2 elections imported before the import gets VERY slow (can take up to 6 hours whereas the first one took 5 minutes).  by the 3rd election the whole thing crashes FM and I can get no further.


I have tried importing from Excel (as long as there were less than 1.4 million voters), I have tried making individual FM databases for each election and then importing importing into a master FM database and I have tried directly from CSV file.  Right now I have one database for each election but dealing with 10 years of election history and multiple elections per year is getting a little unwieldy.


So my question is: Is this just too big a solution for FM to handle and if so (or not) how would you handle this?