10 Replies Latest reply on Jan 11, 2017 2:12 PM by TSGal

    Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.

    eworbit

      While wondering why our FM solution routinely goes down every few

      days/everyday, I copied it over to my test FM15 (Mac OS Server).    I

      can semi-reliable cause my solution to crash the server by using the

      record selector on a layout which is linked to a 600 record shadow

      table.   I randomly select a few various records and then I get the

      dreaded SDOD - "Sort dialog of death". In my attempts to debug this, I

      dragged everything off the layout and removed any triggers.   I seem to

      get it to crash when I browser some records, pause for about 2 or 3

      minutes.  Then when I try to go to the next record, then SDOD pops up.

       

      When I eventually get the redundant SDODs I watch the TopCallStats.log

      (see attached) to see the repeating list of queries.   I have to force

      quit FM Pro client but the ODBC Query lines still continue, however it

      replaces my name with an id number.   This goes on and on and on even

      without anyone connected to the server.

       

      At this point I can use Admin Console to close the file. Amazingly, the

      ODBC Queries are still happening.  There is no hosted file anymore.  I

      have to restart FM Server to get them to stop.

       

      On my FM 14 production system (also Mac OS Server), I normally can't

      cleanly close the file because I suspect too many hung processes are

      going on from all the different users.    If just one person gets the

      SDOD, then everyone (all windows FM Pro v14.0.5) will be hung up and

      it's there's no remedy other than to reboot the Mac server.

       

      See attached.   Any insight?

       

      -Eric Wood

       

       

        • 1. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
          TSGal

          eworbit:

           

          Thank you for your post.

           

          What is the shadow table being accessed?

           

          If you stop the database server, does the query lines still continue?

           

          What is the actual query being sent?

           

          Please provide the exact "SDOD" message?

           

          TSGal

          FileMaker, Inc.

          • 2. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
            eworbit

            What is the shadow table being accessed?

            The Shadow table being access is a order header and line items tables (views).  These are the typical queries:

            SELECT TransId,BatchId,SoldToCustId,BillToCustId,TransType,LocId,CustLevel,ShipToId,ShipToName,ShipToAttn,ShipToAddr1,ShipToAddr2,ShipToDivision,ShipToCity,ShipToRegion,ShipToCountry,ShipToPostalCode,ShipVia,ShipToDAS,TermsCode,DistCode,InvcNum,PrintInvcStatus,PrintPackStatus,PrintPickStatus,InvcDate,TransDate,PODate,CustPONum,ShipNum,ReqShipDate,ActShipDate,Rep1Id,Rep1Pct,Rep2Id,Rep2Pct,TaxableYN,TaxOnFreight,TaxClassFreight,TaxClassMisc,SalesTax,Freight,Misc,CurrencyID,ExchRate,NetDueDate,TaxAmtAdj,TaxAdj,TaxLocAdj,TaxClassAdj,ShipToPhone,Notes,Rep1CommRate,Rep2CommRate,FreightQuoted,FreightAvailable,FreightTerms,FreightAcctNum,CcAuthAmount,ShipFromAttn,ShipFromName,ShipFromDivision,ShipFromAddr1,ShipFromAddr2,ShipFromCity,ShipFromRegion,ShipFromCountry,ShipFromPostalCode,ShipFromPhone,ShipFromFax,FreightCost,ShipToRes,CustName,OrderTotal,TotCost,DiscAmt,FMQuoteDate,FMQuoteId,WebOrderId,GoogleTransId,PaypalTransId,OtherTransId,PickPrintDate,PickPrintName,PickPrintUser,InvcPrintDate,InvcPrintUser,RouteToDate,RouteToUser,RouteToUserBy,AckDeliveryBy,AckDeliveryDate,AckDeliveryDestination,AckDeliveryName,AckDeliveryType,GrossProfit,ProfitMargin,TotalWeight,DropShipExists,VendorId,LinkedPoNumber,CcAuthCode,CcAuthDate,CcAuthUserName,CcAuthNum,AvailToShip,CouponCode,ShipToVerifyDate,NumPkgs,TrackNumList,OrderWeight FROM IPI.dbo.trav_SoTransHeaderInfo_view WHERE (TransId=N'00898673' AND BatchId=N'######')

            SELECT TransID,EntryNum FROM IPI.dbo.trav_SoTransDetailInfo_view WHERE (TransID IN (N'00898673'))

            SELECT TransID,EntryNum,LocId,ItemId,Descr,AddnlDescr,TaxClass,TotQtyOrder,AcctCode,GLAcctSales,GLAcctCOGS,GLAcctInv,UnitsSell,PriceID,UnitPrice,UnitCost,PromoID,ReqShipDate,Rep1Id,Rep1Pct,Rep1CommRate,Rep2Id,Rep2Pct,Rep2CommRate,PriceAdjType,PriceAdjAmt,CouponCode,DropShipYn,PoTransId,VendorId,QtyShip,GP,ExtendedCost,ExtendedPrice,TravEntryNum,NonDevPrice FROM IPI.dbo.trav_SoTransDetailInfo_view WHERE (TransID=N'00898673' AND EntryNum=1)

             

            When using the record selector to bounce around to different "orders", these are the typical queries.  Because there is a portal for the line items, FM does not fetch the magically 25 records in a batch.  So that's not the problem.

             

            If you stop the database server, does the query lines still continue?

            Yes! I have attached some screen shots to illustrated all the steps that trigger the crash, the order screen, go to a random order record, the logs, etc.

             

            What is the actual query being sent?

            It should be like the queries above.  But as I watch SQL Server Profiler, FM Server actually does *not* send any queries to the SQL Server.  This is fortunate since FM Server isn't trying to also bring down my Windows Server.  Thank goodness! Although it does start logging ODBC Queries in the TopCallStats.log file.

             

            Please provide the exact "SDOD" message?

            I *think* that the user/client that happens to trigger this "deadlock" situation, they end up getting the beach-ball or circle of death.  Everyone else as they move around will get stuck Sort Dialogs or Find Dialogs, circle of deaths, etc.  because evidently the ODBC queuing mechanism has barfed inside FM Server.  So the messages that clients get vary.  I didn't want to mislead us that this was significant now.

             

            I'll ask some rhetorical questions:

            Can I trigger this by doing individual finds instead of dragging the record selector?  Dunno. Its just more fun to use the mouse!

            Does it seem like FM Server barfs whenever you sit dormant on this layout for a few minutes and then want to browse to a new record?  YES!  I can't pinpoint it but it seems that FM Server's ODBC queuing mechanism timesout or purges to where a client tries to wake it up and it can't.  All I know is I can crash it without going anywhere else.

            Can any user (not just admins) trigger the crash.  Most definitely!  Doesn't maker FM Pro or Pro Advanced.

            Did this same problem happen in FM Server 13?  I think I remember so.  It happens in 14 and 15.

             

            Well thank you for listening.

            • 3. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
              TSGal

              eworbit:

               

              Thank you for the additional information.

               

              None of your answers look out of the ordinary, so I have forwarded your posts and files to our Development and Testing departments for review.  When I receive any feedback, I will let you know.

               

              TSGal

              FileMaker, Inc.

              • 4. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                TSGal

                eworbit:

                 

                Testing has asked the following questions:

                 

                1. What SQL Server version are you using?  Is it installed on Mac OS X or Windows?  What version?

                 

                2. What ODBC driver and its version are you using?

                 

                3. What is the SQL Server schema being used?

                 

                4. How are you mapping SQL Server database into FileMaker?

                 

                5. Please provide a clone of the file.  I have sent you a private message with instructions where to send the file.

                 

                6. Is there a crash log?  If so, also provide it.  You can find the crash log from the Console.app and in the System Reports folder, find any crash log named "fmxxxx.crash".

                 

                7. Please also provide the Event.log (from Library/FileMaker Server/Logs/)

                 

                8. When you reproduce on the FileMaker Server 15 test server, how many users are connected?

                 

                TSGal

                FileMaker, Inc.

                • 5. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                  TSGal

                  eworbit:

                   

                  I received your clone and other information.  Thank you.

                   

                  Testing has looked over your crash report and noticed the crash report had several instances of Actual Technologies.

                   

                  What is the Actual ODBC SQL Server driver version?  This information can be found from:

                  ODBC Manager -> System DSN tab > Double click the SQL Server DSN to open Actual SQL Server DSN Configuration dialog, and click About dialog.

                   

                  Do you use any plug-in in the FileMaker file to perform any SQL queries?

                   

                  TSGal

                  FileMaker, Inc.

                  • 6. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                    eworbit

                    No plugins used at all.

                     

                    For my FM14 server I use Actual Technologies is 3.3.6.  For my

                    development FM15 server I use Actual Technologies 4.0.3.    I had

                    upgraded the dev box from 3.3.6 to 4.0.3 a few weeks ago but the FMS

                    still hung.   So 4.0.3 wasn't the answer.

                     

                    I see there is now an Actual Technologies 4.0.4 version.   What I will

                    do is upgrade the development box to 4.0.4, and see if the solution

                    still hangs and re-supply the log files from the dev box instead of our

                    production box.

                     

                    -Eric Wood

                    • 7. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                      TSGal

                      eworbit:

                       

                      Thank you for the information.

                       

                      When opening the file, I am prompted for Account Name and Password, and neither Admin/<no password> nor admin/admin work.  Please send another message with the login credentials.

                       

                      TSGal

                      FileMaker, Inc.

                      • 8. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                        eworbit

                        sorry about that.

                        admin

                        FMadmin15

                        you can reset the password after you open it.

                        thanks,

                        -Eric

                        • 9. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                          TSGal

                          eworbit:

                           

                          Thank you.  The information has been sent to Development and Testing.

                           

                          TSGal

                          FileMaker, Inc.

                          • 10. Re: Infinite TopCallStats.log ODBC queries. Eventually Server crash. SDOD.
                            TSGal

                            eworbit:

                             

                            Development and Testing are unable to reproduce the issue, and they have asked that you turn on the driver logs in your development environment.  The results from these logs may help pinpoint the issue.  I have sent you a private message with instructions where to send these files.

                             

                            TSGal

                            FileMaker, Inc.