1 2 3 Previous Next 120 Replies Latest reply on Jul 20, 2017 9:18 AM by fmpdude

    FM Crashes During Large Import (Over the Limit)??

    osensnolf

      I am using FM14 x64 and trying to import a large 2GB CSV into Filemaker but it crashes.  I also have data in a FMP12 database that I am trying to import directly into a newer FMP12 database.

       

      I am  having crashing issues importing large files into FileMaker. So, last night, I decided to load the latest FileMaker x64 demo on four computers and try to import the data. When I got up this morning, FM had closed on each computer without finishing the import.

       

      On three of the computers, Filemaker closed without giving an error message. Those three use SSD's. On my 4th computer which uses SATA, I did receive an error. Not sure if hard drive type has anything to do with it.

       

      This happens if I try to import from a CSV file or if I try to import from an existing FMP12 file.

       

      Below is the crash report. Does this provide any insight?

       

      Thanks for your help.

       


      Problem signature:
      Problem Event Name: APPCRASH
      Application Name: FileMaker Pro.exe
      Application Version: 14.0.3.313
      Application Timestamp: 5601d22b
      Fault Module Name: FileMaker Pro.exe
      Fault Module Version: 14.0.3.313
      Fault Module Timestamp: 5601d22b
      Exception Code: c0000005
      Exception Offset: 000000000020a1cd
      OS Version: 6.1.7601.2.1.0.768.3
      Locale ID: 1033
      Additional Information 1: eb32
      Additional Information 2: eb325102729a5b61009b1683466d6130
      Additional Information 3: bb82
      Additional Information 4: bb82403c7da318b5ce6ca6f6950ad6a4

        • 1. Re: FM Crashes During Large Import (Over the Limit)??

          Wow.

           

          Although this sounds like an FM-internal issue (bug), and hopefully FM will respond to this thread and ask for a copy of your file so they can test, but there are a few things I would check. Also, see the link to a possible solution below.

           

          1. Does the import fail at the same place? I'd check the table and see the last record imported. Then, check the CSV to make sure nothing's hokey there. Shouldn't make an import crash, but still.

           

          2. 2 GB isn't a lot in the grand scheme of things, but I'd also check to make sure the import didn't eat any remaining disk space. That would definitely stop the import.

           

          3. Check memory. You probably have 16 or 32 gig RAM machines, but it would be interesting to see if machines configured differently, got further in the import.

           

          I did see this other posting for the c0000005 exception code you might want to check out:

           

          Faulting module name: DBEngine.dll

           

          Good luck!

           

          - m

          • 2. Re: FM Crashes During Large Import (Over the Limit)??
            osensnolf

            Hi morkus.

             

            I can send the file without data but with data, the Filemaker database is 19GB.

             

            The error (closing of Filemaker) happens at different records.  It does usually happen after 1GB or 2 million rows has been imported.  The computer that tends to get the furthest with the import has Windows Home Server installed on it (and I do realize this is not supported).  All other computers have Windows 7 with SP1 - none have the same updates applied so I do not think it is a bad Microsoft bug.

             

            My Ultrabook has an i3 with 4GB RAM.  All others have at least 6GB or 8GB RAM.

             

            All have plenty of diskspace for read and write.

             

            I'm wondering if I will see a difference if I uninstall Advanced and stick with Pro.  I do not edit the database layout - I only add records and edit them.

             

            I bet this has happened at least 20 different time on several different FM versions.  The computer that has the SATA drive is the only one that has ever generated an error.  The computers with SSD just close out.  All SSD's are different.  Everything about each machine is different.  v14 and my database are the only thing that are the same as even different data has been tested.

            • 3. Re: FM Crashes During Large Import (Over the Limit)??
              Markus Schneider

              can't You split the csv? Maybe in 2 pieces, if the import fails, split into another 2...

              if one of the splited files can be imported and the other not, it's could be a problem with the csv..

               

              Can You have the TaskManager running while importing? RAM?

              • 4. Re: FM Crashes During Large Import (Over the Limit)??
                osensnolf

                Hi Markus.

                 

                Yes, if I split the file, I usually have success if it is less than 200MB or less than 1Million records.  I can do that but it is very time consuming as I have to split into at least 15 files and be at the computer to start each file.  On top of that, I have to make a copy of the DB after each successful import in case the next one crashes (so that I do not have to start from the beginning -- lesson learned).

                 

                That said, this option does work but it would be best if I could find the problem that is causing larger imports to make FM crash.

                 

                Going to uninstall v14 and install v13 on a machine and see if that works.

                 

                Thanks

                • 5. Re: FM Crashes During Large Import (Over the Limit)??
                  siplus

                  I had good results by doing a Pause/resume script 0 seconds followed by a Flush Cache to Disk every 100'000 records imported. YMMV.

                  2 of 2 people found this helpful
                  • 6. Re: FM Crashes During Large Import (Over the Limit)??
                    osensnolf

                    I can see how that will help.

                     

                    Can you PLEASE send that to me so that I can have my developer integrate it to see if that will help?  If so, I'll pay you for your time.

                     

                    osensnolf hotmail com

                    • 7. Re: FM Crashes During Large Import (Over the Limit)??
                      siplus

                      Take a look at TroiFile plugin, especially TrFile_GetContents.

                      • 8. Re: FM Crashes During Large Import (Over the Limit)??

                        Did you check out the link I included that describes the c0000005 exception?

                         

                        Also, if you use Windows computers, maybe try a Mac FileMaker version too.

                         

                        Although all the tips presented here are fine and good, you shouldn't have to resort to special tricks or third-party utilities.

                         

                        If this were me, I would contact FileMaker as it's most likely a bug in the product based on what I am hearing.

                         

                        Good luck.

                         

                        - m

                         

                        P.S. In any case, if you send me the field structure and your table name and the approximate number of records, I'll generate a 2+ GB file using my cool free data generator and see how it goes here.  I use the Mac version of 14 Pro Adv.

                        • 9. Re: FM Crashes During Large Import (Over the Limit)??
                          TSPigeon

                          osensnolf:

                           

                          Thank you for your post.

                           

                          I sent you a private message with a request for your file. Please include the table name I'll need to import data to.

                           

                          TSPigeon

                          FileMaker, Inc.

                          • 10. Re: FM Crashes During Large Import (Over the Limit)??
                            osensnolf

                            Take a look at TroiFile plugin, especially TrFile_GetContents.

                            -- Do you have the direct link or code for the script so that I can be sure I am using exactly what you are using?

                             

                             

                             

                            Did you check out the link I included that describes the c0000005 exception?

                            -- Yes, I did review that prior to you including it and reviewed it again.  I do not have a Mac to test on.

                             

                             

                            If this were me, I would contact FileMaker.

                            -- I did that earlier today but it says it can take up to 2 days to reply.

                             

                             

                            I'll PM you regarding your request.

                            • 11. Re: FM Crashes During Large Import (Over the Limit)??
                              osensnolf

                              Will reply to your PM.  Thanks.

                              • 12. Re: FM Crashes During Large Import (Over the Limit)??
                                osensnolf

                                UPDATE:  I started a new import on the four same computers yesterday.  Within 3 hours, FileMaker had closed or locked up on three of the computers.  On the fourth computer, I uninstalled v14 and installed v13.  So far it has been importing successfully for 17 hours.

                                 

                                I'm not sure if v13 is slower because I could not find the x64 version or what, but it is super slow.  At this rate, it will be another 3.5 days before it finishes importing my 16M rows (total of 5.5GB CSV file).

                                 

                                I do not have any other updates from anyone else.

                                 

                                Thanks for the continued help.

                                • 13. Re: FM Crashes During Large Import (Over the Limit)??

                                  Since FileMaker tech support (TSPIgeon above) reached out to you, I did not download the file and try to do an import separately.

                                   

                                  FileMaker Tech Support, in my experience, is excellent.

                                   

                                  I wish FM would come out with more frequent updates, and even add a few new (sorely missing) features in point releases, but overall, I have only good things to say about them and have had good experiences.

                                   

                                  It's not cool that 14 doesn't work as well as 13 and I'm sure FM isn't happy about that, either.

                                   

                                  Hope you and FM get to the bottom of this issue soon.

                                   

                                  - m

                                  • 14. Re: FM Crashes During Large Import (Over the Limit)??
                                    osensnolf

                                    UPDATE: The PC with v13 shows that the import was complete but it only imported 4M records.  The good news is that it did not crash so I can find where it left off and resume.  It's the first time I've seen it say import is complete even though it isn't.  Usually if I see their message, it really did complete.

                                     

                                    I have v14 setup on one of the previous failed PC's and split the CSV into files of 800k rows each.  So far I have imported 3 files without any issues.  In the past, this approach was fine.  The problem is if the data is over 1M rows or over a certain size (which I have yet to pinpoint).

                                     

                                    Without a doubt, v13 is operating much differently (stable) than v14.

                                     

                                    v13 does seem to import much slower.  Maybe because it is 32-bit?  FMv14 x32 and x64 both cause import issues.

                                     

                                    Thanks

                                    1 2 3 Previous Next