If memory is an issue, some Windows performance monitoring will show that quickly. I doubt that it is but it's a good idea to rule it out.
In the log viewer: any repeated errors? Also look in the Windows application log, not just the FMS log that you see in the admin console. From the one entry that you discribe it doesn't look like the script process (SASE) on is getting killed, just that it loses the connection to the database engine. Could it be some unhandled errors in the script? The FMS log will tell you what errors happen in the script.
What do the FMS performance counter show you? Any long delays in there for either disk i/o or network?
Looking at Windows Perfomance Monitor (I am not familiar with it so don't know what is 'normal')
A steady 7,050Mb available
80 to 200 cache faults a sec
On the physical disk have at least 97% idle time
Processors (Total) max out at 1.2% processor time with 0 in % interupt time and Parking status
No errors in the log viewer except a 401 for another script that completed 15 minutes prior the failure.
Sorry, can't find a Windows application log - should it be somewhere other than in C:\Windows\logs ?
No FMS errors apart from as described - the Warning in the details part of the error is 30
No delays to speak of for disk i/o or network.
Have noticed though, that running the script on FMP on the same box that the client disappears from the Connected Client list occaasionally. Could it be cache clearing or something like that? Client cache settings are the default.
The normal varies per deployment and you derive it from the baseline performance monitoring you normally do "when things change": like adding a module to the solution, adding a bunch of users, changing hardware, major new OS or FMS versions...
Without the baseline it is very difficult to say what is normal and what isn't, except for obvious outliers. The numbers you quote seem normal enough.
To view the Windows application log, you need to open the Event Viewer in Windows, you can find it under "administrative tools"
I think I have identified the problem although yet to be confirmed. Nothing untoward appears in either the server statistics or performance monitor.
The database engine does certainly not become unresponsive, users are working away fine. memory and processor usage minimal.
The one 'hint' seems to be that when I ran the same script in FileMaker Pro on the server itself I got a 'not responding' message after a while. This was while I was watching Performance monitor. The database engine was still running at the time, and plenty but minimal disk and network activity - no hardware memory errors.
Part of the routine sends api requests to Amazon, and I think that, although my maths tells me I am not, I must be flooding them with too many requests and they stop responding for a while, which stops the script in it's track waiting for some xml to be returned and processed. Amazon have very clear throttling limits on their APIs, and I believe I am well within these - but of course their documentation could be wrong!
I will know more when I break the routine down into smaller chunks, it should take about 4 hours but I am going to break it down to about 30 minute bites.
Are you using a plugin for that or a straight import?
If a plugin, there will always be timeout issues, just like there would be with a client, as FM waits for the plugin function to return some result. If you're using a plugin which one are you using?
Usig a plugin built with the latest released version of ScriptMaster.
Getting more and more convinced the problem is the delay in getting the result of the cal.
Have no suspiscions about the plugin, have used one very similar for about 9 months with a faster (latency and upload/download) internet connection and a different api with datasets 4 times the size.
Ok, an update and any ideas sought, it is the plugin, or a clash of scripts using the plugin.
This are the errors I am getting in the error log, I get a lot of these with different threads
Aug 28, 2012 11:32:46 AM com.prosc.fmkit.Plugin setLastError
SEVERE: PluginFunction com.prosc.beanshell.CompiledFunction@f39b3a got an error with converted args [UPDATE BookDetails SET LowestUsedPrice=null, LowestUsedPriceCurrency='£', AmazonTotalUsed=4 ,AmazonSalesRank= WHERE zkAmazonASIN='0216889219', |]
at com.prosc.beanshell.FMPro$executeSql.call(Unknown Source)
Aug 28, 2012 11:32:46 AM com.prosc.beanshell.BeanShellPlugin setLastError
SEVERE: ScriptMaster bound variables: Parameters:
Aug 28, 2012 11:42:08 AM com.prosc.fmkit.PluginBridge pushContext
WARNING: pushContext called with function (com.prosc.beanshell.CompiledFunction@1542a75) when stack size is 1; this either means it was called re-entrantly, an idle handler was called without waiting for a return from a previous call, or pop was not called from last invocation. Current thread: Thread-27875
There are 2 server side scripts that overlap calling the same plugin - ScriptMaster built.
Any ideas gratefully received, about to try building another plugin so that each script calls a different one to see if that helps
Can you try the same functionality with the regular ScriptMaster plugin, just to rule out anything in the plugin creation process?
Good idea Wim, will try that, but may have an issue because when I tried to register the command that gets the xml as a script step last month it didn't work and rather than find out why I just created the plugin.
Will try again tonight and see if I can get that working
I'm still on FMS11, I'm not sure if the backup efficiency is greatly increased in FMS12. But are you running a backup at the same time overnight? The backups could be causing general performance issues, or scripting issues even. You could try disabling backups at the same time.