1 2 3 Previous Next 37 Replies Latest reply on Sep 18, 2012 7:18 AM by farokh

    FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.

    farokh

      Summary

      FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.

      Product

      FileMaker Pro

      Version

      12.0v2 and Advanced 12.0v2

      Operating system version

      OS X 10.7.4

      Description of the issue

      When attempting to run a script that takes a long time, FM Pro is crashing. I have both FM Pro and Advanced 12.0v2. The script in question potentially could run for many hours. It's exporting about 427,000 PDF images that are stored in a FM Pro database that I am trying to convert over to using references instead of the actual images.

      Here's the Problem Report:

      Process:         FileMaker Pro [54293]
      Path:            /Applications/FileMaker Pro 12 Advanced/FileMaker Pro Advanced.app/Contents/MacOS/FileMaker Pro
      Identifier:      com.filemaker.client.advanced12
      Version:         12.0.2 (12.0.2)
      Code Type:       X86 (Native)
      Parent Process:  launchd [167]

      Date/Time:       2012-08-21 20:03:42.658 -0400
      OS Version:      Mac OS X 10.7.4 (11E53)
      Report Version:  9

      Interval Since Last Report:          421276 sec
      Crashes Since Last Report:           2
      Per-App Interval Since Last Report:  7520 sec
      Per-App Crashes Since Last Report:   2
      Anonymous UUID:                      70098317-FB17-4BC7-9CEA-5417CBC9E81B

      Crashed Thread:  0  Dispatch queue: com.apple.main-thread

      Exception Type:  EXC_BAD_ACCESS (SIGBUS)
      Exception Codes: KERN_PROTECTION_FAILURE at 0x0000000000000000

      VM Regions Near 0:
      --> __PAGEZERO             0000000000000000-0000000000001000 [    4K] ---/--- SM=NUL  /Applications/FileMaker Pro 12 Advanced/FileMaker Pro Advanced.app/Contents/MacOS/FileMaker Pro
          __TEXT                 0000000000001000-0000000000b06000 [ 11.0M] r-x/rwx SM=COW  /Applications/FileMaker Pro 12 Advanced/FileMaker Pro Advanced.app/Contents/MacOS/FileMaker Pro

      Application Specific Information:
      objc[54293]: garbage collection is OFF

      Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
      0   DLI_PDFL91                         0x02ecf3e2 emit_byte + 14
      1   DLI_PDFL91                         0x02ecf42d emit_marker + 27
      2   DLI_PDFL91                         0x02ecfb1a write_file_header + 26
      3   DLI_PDFL91                         0x02ec9027 jpeg_write_coefficients + 409
      4   DLI_PDFL91                         0x02e92072 CacheJPEG + 1344
      5   DLI_PDFL91                         0x02e939aa dtl_StoreGraphics + 4323
      6   DLI_PDFL91                         0x02e94868 dtl_verify_graphic_ex + 683
      7   DLI_PDFL91                         0x02e948da dtl_verify_graphic + 40
      8   DLI_PDFL91                         0x02e6d8c7 dlpdfimagecreatefromfile + 137
      9   com.filemaker.client.advanced12     0x0066fab9 APDFLImageVS::CreatePDFImageFromFile(FMPDF::Doc, _t_ASFixedRect*, bool&, char const*) + 41
      10  com.filemaker.client.advanced12     0x006700cf APDFLImageVS::PlaceEncodedBytesInContentCommon(FMPDF::Content, _t_ASFixedPoint, int, int, int, _t_ASFixedRect*) + 831
      11  com.filemaker.client.advanced12     0x00670a7b APDFLImageVS::PlaceInContentCommon(FMPDF::Content, _t_ASFixedPoint, int, int, int, _t_ASFixedRect*) + 251
      12  com.filemaker.client.advanced12     0x00678602 APDFLGraphicsEnv::DrawImage(Draco::CoordRect const&, APDFLImageVS*) + 754
      13  com.filemaker.client.advanced12     0x006789c7 APDFLGraphics::DrawImage(Draco::CoordRect const&, IImage*) + 487
      14  com.filemaker.client.advanced12     0x006975ab ProWebViewerView::DrawView(IGraphics*, Draco::CoordRect const&) + 1947
      15  libViewSystem.dylib                0x0514266d BaseView::RenderToIGraphicsUsingTransform(IGraphics*, Draco::Coord3x3Matrix const&, Draco::CoordRect const&, Draco::Coord3x3Matrix const*) const + 175
      16  libViewSystem.dylib                0x05141ad7 BaseView::RenderToAncestor(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*) + 73
      17  libViewSystem.dylib                0x05146020 BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 488
      18  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      19  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      20  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      21  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      22  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      23  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      24  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      25  libViewSystem.dylib                0x051460fc BaseView::UpdateRectListToAncestor_secondary(IView*, IGraphics*, Draco::CoordRect const&, Draco::Coord3x3Matrix const*, unsigned int, bool, bool) + 708
      26  libViewSystem.dylib                0x051464c1 BaseView::UpdateRectListToAncestor(IView*, IGraphics*, Draco::CoordRectQuickList const&, Draco::Coord3x3Matrix const*, unsigned int, bool) + 769
      27  libViewSystem.dylib                0x05141b1d BaseView::UpdateRectListToAncestor(IView*, IGraphics*, Draco::CoordRectQuickList const&, Draco::Coord3x3Matrix const*) + 61
      28  libViewSystem.dylib                0x0514464a BaseView::PrintViewRelativeToAncestor(IGraphics*, IView*, Draco::CoordRect const&) + 64
      29  libViewSystem.dylib                0x05143576 BaseView::PrintView(IGraphics*) + 216
      30  com.filemaker.client.advanced12     0x00817cf4 DSDocPrintable::PrintPageInRect(IViewPrintOperation*, IGraphics*, unsigned long, Draco::CoordRect const&, Draco::Coord3x3Matrix const&) + 3046
      31  com.filemaker.client.advanced12     0x0067e6c7 XViewPrintOperationPDF::RunOperation() + 1175
      32  com.filemaker.client.advanced12     0x008167ec DSDocPrintJob::ExecutePrintJob() + 92
      33  com.filemaker.client.advanced12     0x0068ffcf FMPDF::Feature::ExecuteFeature(FMDocWindow&, PrintSettings const&, unsigned short) + 95
      34  com.filemaker.client.advanced12     0x006902a7 DoPDFFeatureOperation(FMDocWindow&, Draco::PDFSettings const&, PrintSettings const&, unsigned short, Draco::UniversalPath const&) + 87
      35  com.filemaker.client.advanced12     0x004b5275 DoSaveAsPDF(FMDocWindow&) + 1509
      36  com.filemaker.client.advanced12     0x001a23bc _ZL14DoImportExportP11FMDocWindowj + 572
      37  com.filemaker.client.advanced12     0x001a7234 AppHandleCommand(FMDocWindow*, unsigned int, bool) + 4468
      38  com.filemaker.client.advanced12     0x001aa0ec AppExecuteCommand(FMDocWindow*, unsigned int, bool, bool) + 220
      39  com.filemaker.client.advanced12     0x00328dfa ScriptRuntime::DispatchStep(bool&) + 362
      40  com.filemaker.fmengine.framework     0x0419467a Draco::ScriptRuntimeBase::Execute() + 522
      41  com.filemaker.fmengine.framework     0x04195133 Draco::ScriptRuntimeBase::DoNextStep() + 371
      42  com.filemaker.fmengine.framework     0x0419525d Draco::ScriptRuntimeBase::DoRunLoop() + 109
      43  com.filemaker.client.advanced12     0x0032215f ScriptRuntime::OnIdle() + 127
      44  com.filemaker.fmengine.framework     0x0416f010 Draco::FMSession::OnIdle(bool) + 272
      45  com.filemaker.client.advanced12     0x0017b802 FMAppSession::OnIdle(bool) + 66
      46  com.filemaker.client.advanced12     0x0016abf5 CFMProApp::Idle(bool) + 69
      47  com.filemaker.client.advanced12     0x0016ad1d CFMProApp::DispatchNullEvent() + 109
      48  com.filemaker.client.advanced12     0x004b37a9 -[MFAppDelegate HandleIdleNotification] + 25
      49  com.apple.Foundation               0x920c0df1 __-[NSNotificationCenter addObserver:selector:name:object:]_block_invoke_1 + 49
      50  com.apple.CoreFoundation           0x99909903 ___CFXNotificationPost_block_invoke_1 + 275
      51  com.apple.CoreFoundation           0x998d4688 _CFXNotificationPost + 2776
      52  com.apple.Foundation               0x920c0db7 -[NSNotificationCenter postNotification:] + 115
      53  com.apple.Foundation               0x92128803 postQueueNotifications + 948
      54  com.apple.CoreFoundation           0x998ed0ce __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 30
      55  com.apple.CoreFoundation           0x998ed00d __CFRunLoopDoObservers + 413
      56  com.apple.CoreFoundation           0x998bfa5d __CFRunLoopRun + 1261
      57  com.apple.CoreFoundation           0x998bf1dc CFRunLoopRunSpecific + 332
      58  com.apple.CoreFoundation           0x998bf088 CFRunLoopRunInMode + 120
      59  com.apple.HIToolbox                0x9a59e723 RunCurrentEventLoopInMode + 318
      60  com.apple.HIToolbox                0x9a5a5a8b ReceiveNextEventCommon + 381
      61  com.apple.HIToolbox                0x9a5a58fa BlockUntilNextEventMatchingListInMode + 88
      62  com.apple.AppKit                   0x982530d8 _DPSNextEvent + 678
      63  com.apple.AppKit                   0x98252942 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 113
      64  com.apple.AppKit                   0x9824ecb1 -[NSApplication run] + 911
      65  com.filemaker.client.advanced12     0x0006e7c0 CMacApp_Main + 576
      66  com.filemaker.client.advanced12     0x001700e3 main + 67
      67  com.filemaker.client.advanced12     0x00050ce1 start + 53

      Steps to reproduce the problem

      I run a script, eventually FM Pro crashes. RAM usage keeps increasing until the crash. I have run into this in both FM Pro and FM Pro Advanced.

      Expected result

      Successful completion of the script.

      Actual result

      FM Pro crashes.

      Exact text of any error message(s) that appear

      No error message.

      Configuration information

      This is on an early 2008 Mac Pro with 4 GB RAM, 1TB hard drive and a single 2.8 Ghz quad core CPU.

      Workaround

      None.

        • 1. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
          philmodjunk

          Any time that you get crashes, it's good practice to run a recover on your file. Not only can file damage be the cause of your crash, a crash can damage your file.

          Things to keep in mind about Recover:

          While Recover almost always detects and fully corrects any problems with your file...

          1. The recovered copy may behave differently even if recover reports "no problems found".
          2. Recover does not detect all problems
          3. Recover doesn't always fix all problems correctly
          4. Best Practice is to never put a recovered copy back into regular use or development. Instead, replace the damaged file with an undamaged back up copy if this is at all possible. You may have to save a clone of the back up copy and import all data from your recovered copy to get a working copy with the most up to date information possible.
          • 2. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
            farokh

            I'm not sure what this has to do with the crash. Even if the database has a problem, which I doubt since there are other scripts that run over the entire database without an issue, Filemaker Pro should never crash.

            However, I was under the assumption that if the database was damaged to the point where a recovery was necessary that FMP would inform me of that fact while it scanned the file when I opened it again.

            This database is over 113 gigabytes and so every time it crashes, I lose about 20-30 minutes while Filemaker checks the file.

            • 3. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
              philmodjunk

              To repeat. A damaged file can cause a crash and a crash can damage your file.

              The consistency check you get when you open your file after the crash is a "quick check" for obvious problems. It is no where near as comprehensive as a recover.

              I hate to be the one to tell you this, but damage in a file can lie hidden in your file for years until just the right combination of factors trips over the damage and you get unexpected results, a "hang" or a "crash". This isn't common, but it does happen.

              I strongly recommend that you recover your file and test the recovered file to see if it still crashes when you run this script. If nothing else, you can report back to the forum that recover did not repair the problem. This will then suggest looks in other directions for the cause.

              You might also, if possible, test your file on a different computer. If it only crashes on a specific computer, that points towards issues with that computer.

              • 4. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                farokh

                I had already started the recovery, just to see what would happen. It's been running since 5:00pm ET, it's now 12:09am and it's still going, no end in sight. Not even a progress bar showing or barber pole. I have to use Activity Monitor to even see that something is happening.

                It's on step 6 of 17, recovering a table. The last two entries in the Recovery.log are:

                2012-08-22 18:09:08.758 -0400     images.fmp12     0     Recovering: library catalog 'claimant images'
                2012-08-22 19:59:23.643 -0400     images.fmp12     0       Rebuilding library catalog 'claimant images' order list: quick reference

                I guess I'll check it again in the morning and see where it's at.

                • 5. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                  farokh

                  It's now 7:38am, and it's at the same point. How long should I expect recovery to take on a 113 GB file?

                  • 6. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                    farokh

                    OK, it's now 12:16pm ET, still no change, no evidence of actual progress.

                    Also, I forgot to mention that the crashes happen at different points in the process. It would seem to me that if there was a problem with the database, it would crash at the same point each time, no? I will fail at different points each time, and there's no specific pattern. IE if it failed at point x the first time, it failed at x+y the next time, and then x-z the next time, which I would think would rule out a database corruption issue.

                    • 7. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                      farokh

                      So, now it's about 6:00pm ET, and still no change.

                      And no response at all from Filemaker.

                      I even tried calling tech support and got put on hold forever.

                      Not very promising.

                      • 8. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                        farokh

                        OK, it's now 9:00am ET the 2nd day.

                        Nothing has changed, and I really don't have any idea what FM Pro is doing. It's using 100% of one core and yet there's almost no disk IO going on. I'm attaching a screen shot.

                        Can ANYONE explain what is going on? I'm on the verge to recommending that my client move to something else instead of spending a bunch of  money on upgrading to 12.

                        • 9. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                          philmodjunk

                          How large is the file?

                          There's clearly been a change as your last post shows that you are on "step 6", but even recovering a 4 gigabyte file doesn't normally take this long.

                          Is the file you are recovering on a local hard drive? Are you recovering to a local hard drive.

                          I'm just looking for all the possible factors here.

                          • 10. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.

                            Farokh Irani:

                            Thank you for your post.

                            As Phil mentioned, recovery will take longer depending on the size of the file. The bigger the file, then the longer the recovery process takes. If the file is taking this much time to recover, consider making a clone with no records and running recovery on the clone.

                            If the exporting of the 427,000 PDFs is crashing before completion, then consider exporting in smaller found sets, 100 PDFs or so a time.

                            TSFalcon
                            FileMaker, Inc.

                             

                            • 11. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                              farokh

                              1) How will recovering a clone (which has no records) be of any help? I need to be able to get the PDFs OUT of the database.

                              2) I'm exporting the images one record at a time. Unfortunately, I can't include the script here since FM Pro is busy recovering the database.

                              3) again, no one seems able to explain why FM Pro is using 100% CPU (one full core of a quad core CPU) yet the disk IO is basically flat. This database has a total of 8 fields, only one of which is the image. The rest are text, numbers or timestamp. It's not a complicated database and has only a single relationship to another database based on one field. It boggles my mind, regardless of the size of the actual database file, that the recovery process isn't beating up the disk. It should not be taking FM Pro all that CPU and very little disk IO. I would imagine that the recovery process would mean scanning through the file and making sure to get the data out of each record.

                              • 12. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                                philmodjunk

                                1) How will recovering a clone (which has no records) be of any help?

                                If the internal structure of the file is damaged, recovering a clone of the file and then importing the data from your damaged file into your recovered clone may produce a copy of your file that works for you.

                                3) A database often represents a "black box". We can see what goes in and what comes out, but we can't see what is going wrong inside. If there was a known bug in TSFalcon's official tech support database, TSFalcon could tell you that and what options are available that might allow you to work around the issue. But it appears there is no such known issue. Either you have discovered a new one or your file is damaged. These tests are intended to either fix the problem or rule out file damage as a cause of the problem.

                                • 13. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                                  elmpine

                                  Just a side thought: Are you exporting all the pdfs to the same folder ?

                                  Not sure where mac OS is up to with this but there used to be a limit on the number of files in a folder ( at least there was in windows) before the OS would throw a wobbly. It's possible that Filemaker is not getting the right response code or a very delayed code from the OS and is therefore crashing.

                                  elmpine

                                  • 14. Re: FM Pro and FM Pro Advanced 12.0v2 memory leak when using Save Records as PDF script step.
                                    farokh

                                    No, I'm only putting a few hundred in each folder. Besides the most it got to before crashing was about 1900 records.

                                    1 2 3 Previous Next