WebViewers, Javascript and FileMaker: Enhanced integration possibilities via WebWorkers

Discussion created by steve_ssh on Oct 22, 2014
Latest reply on Sep 19, 2015 by user19752




I'd like to share the results of a personal project that I have been working on.


Before Getting Started:


To help folks get a sense of whether this post might be of interest, I'll say the following up front:


• This is experimental work; though very promising, it is not a confirmed technique for our toolboxes just yet.


• It is not cross-platform. It works well on OSX, and it looks as though it will work adequately well on iOS. It does not work on Windows.


• This post is not best-suited for those looking for an "Intro to WebViewers and FileMaker" type of post.


Those just getting started with this topic, and who are new to these technologies might find more joy and less confusion by looking at any of the great posts that are out there which are geared towards getting started with this topic.





Back in the days of FmGo11 and FmGo12, I spent much time working on WebViewer projects to help fill in some of the limitations that I felt existed in terms of portals and list views on FMGo. Much was possible with WebViewers and some basic Javascript ability, but there was one feature that I was not able to adequately achieve:


This feature was the ability to feed more data to an existing WebViewer without having to reload the entire WebViewer contents. I wanted to be able to push data updates from FM, or have the WebViewer pull in data updates from FM without the use of plugins or an external server. Most of the techniques that I have seen involve the use of reloading the WebViewer will a full set of data (most of the time via the data URL scheme) whenever data needs to transfer from FM proper to the WebViewer.



The Challenge and Questions:


The challenge that I presented myself with came in the form of three questions:


1) How might it be possible to push data updates to the WebViewer without having to lose the WebViewer's currently running code?


2) If this were possible, what might be some of the use cases? What are some of the cool things that this would allow for which would not be possible otherwise?


3) If this were possible, what would the API look like if we wanted to make this be as seamless and easy-to-use for developers who are already moderately well versed in both FM and Javascript?


How could such functionality be wrapped up so that all the details are taken care of, and a developer could just use it without a lot of preliminary setup or provisions?



First Thoughts:


My first thoughts with respect to solving question #1 (how might this be possible?), were things such as the following:



• Export the WebViewer HTML to a local file, and use Javascript to check for changes to the location URL where FM might be able to add small bits of data.


• Have a second WebViewer where FMP could load some code which stores data in one of the HTML5 persistent storage caches, and have the main WebViewer periodically check for updates to the same cache.


• Have FM export text content out to the temp directory which the WebViewer could periodically read in as script files and look for changes.



First Obstacle:


Beyond possible security restrictions about what the WebViewer does and does not have access to, the first real obstacle was that I did not want to have to have the WebViewer code constantly polling something for changes, because this would inhibit the use of the WebViewer functioning as a UI for the user: While constantly checking for updates elsewhere, the WebViewer can not respond to user actions, and, on the other hand, if we don't check for updates frequently enough, we lose the snappy response that we would hope to have in response to being triggered by FM.



WebWorkers Run In a Separate Thread:


WebWorkers are a means by which a web page can spawn off a truly separate thread of code that can work on something in the background, while the main page's code is able to carry on as normal, i.e. without any hint of having to share the resources with the background process. Realizing this, my investigation turned into a study of what WebWorker features are available within a FileMaker WebViewer, and what could be done with these features to help solve the questions posed above. The answers that I found were that, on OSX, the FileMaker WebViewer is capable of using pretty much all of the WebWorker features that I could ask for (on Windows, unfortunately, these features do not seem to be available).



The Resulting Concept:


Here is a sketch of what I thought might work:


- Send HTML and Javascript to a WebViewer using the data URL scheme.

- Javascript in this HTML payload spawns and controls a WebWorker thread.

- The WebWorker thread is given the task of monitoring a file in the FileMaker temp directory.

- When FileMaker wants to push a command/message/response to the WebViewer, it writes the data to the temp file.

- The WebWorker detects the update to the temp file and reads in the data.

- The WebWorker sends a notification to the main WebViewer thread, informing it of the new message, and providing any message payload.

- (Up until this time, the main WebViewer thread has been able to respond to any user interactions in a totally unhindered manner).

- When the main WebViewer thread receives the notification from the WebWorker, it processes the request, which could be something like updating the UI or performing a calculation, etc.


All of this takes place without the need to reload the WebViewer. This is particularly significant in cases where the user may have interacted with the UI to such a point where it could be difficult to reload the WebViewer without it noticeably interrupting the user.




As far as sending messages back into FM from the WebViewer, things are already really good:


We have the ability to send data back into FM via the FMP URL, and on OSX and iOS this is working great.




When we combine these two abilities:


When we combine these two abilities, i.e. messaging FM-->JS and messaging JS-->FM, it is a very powerful combination -- one which truly lets us have unbroken "dialogs" back and forth between FileMaker proper and the WebViewer.



Perhaps an analogy would be a telephone conversation:


If someone calls me up and tells me something, neither of us expect that for me to be able to reply and say something back to them that we should have to both hang up the phone and then I call them back and we start our conversation all over again (until the next time that they say something, at which point we hang up the phones and I call them back...). Such is how I felt about reloading the WebViewer just to push more data to it.


With the WebWorker at our side, it seems to be able to send individual messages back and forth between the two sides that it feels *almost* feels as though there is an open two-way channel of communication between FileMaker and the WebViewer, i.e. a "real" telephone conversation, where hanging up each time someone says something is not required.




Moving along to Question #2: What might this sort of two-way communication be good for?


The first things that come to mind for me are:


1) Any sort of application that has a UI that involves significant rendering.


The ability to push updates without destroying and rebuilding the canvas/screen contents could potentially be significant.



This might include:


- games

- mapping applications

- drawing applications:


Regarding drawing applications:


This could possibly open up doors to realtime collaborative drawing within FM, where various team-members could all mark up the same image with their annotations (at the same time).



2) Another possible application that comes to mind would be the ability to control HTML5 audio from FMGo.


Without going into too many of the details, suffice it to say that there are some Javascript restrictions with respect to playing sound on a mobile device. With the ability to control a WebViewer's currently running code, it seems to be possible to open a WebViewer in a window in FmGo, send that window to the background, but still be able to control its WebViewer from the frontmost FM window context. The implication here is that that background WebViewer could act as a dedicated on-demand audio player, controllable from within FM proper. Per the current iOS restrictions, it appears as though once the user has given their consent for that WebViewer to play just one snippet of audio, the WebViewer then has free ability to use scripting to play any other audio as long as the WebViewer page is not reloaded. This is not at the top of my list of things to implement, but it certainly is a tantalizing challenge. I have tried the concepts out a little bit, and I see promise. So far, the main obstacle that I could foresee would be performance issues: being a device of limited horsepower, I really don't know if, in practice, this would be a feasible project.




Moving along to Question #3: What might the API look like?


This was a really fun challenge, and, though I've just wrapped up what I believe is a really cool API for this, I believe that it could get much better than what I've done on my first pass at this.


My philosophy for the API was to keep each environment, stylistically speaking, on its own turf as much as possible.


The goal was to have the necessary coding from the FileMaker end to seem so naturally "FileMakerish", that it would be a piece of cake for any intermediate FMP developer. Likewise, I wanted the Javascript side of things to be just ordinary Javascript coding, without having to ever "think like a FileMaker developer" as they coded. This has been a fun challenge, and as I implied above, I don't think I have got it just right just yet, but I do have what I consider to be a respectable start. (My hope is that others may take some of these ideas and run with them and toss back their concepts and improvements, etc..)



For now, I have named the system "jsv". Below I'll give little sketches of aspects of it.




Breakdown of Components.


The result that I have settled on for now is a family of 5 Custom functions.


These functions work in conjunction with a single, straight-forward FileMaker script that, essentially, takes care of the business of exporting data to the temp directory.


(The script requires the use of one available global text field.)




The CF's break down as follows:


1) One CF is used to initialize a WebViewer. You supply it with your basic HTML, and it outputs the exact data to send to the WebViewer.


2) Another CF is used any time that you want to send a command from FM to the WebViewer. This CF expects a Command argument, and it also accepts any data payload that you are sending along with the command.


3 & 4) Two CF's are dedicated to the task of organizing payload data which is fed into CF #2 above. One of these CF's is for Key-Value pair data, the other is specifically for handling data returned from an ExecuteSQL call, i.e. data that comes in a "rows-of-fields" format.


5) The last CF is also a data wrapper. Its specific task is to package up data that FM returns back to the Javascript environment.




Callbacks and Error Handling:


Beyond just getting the nuts and bolts of two-way communication happening, I wanted to try to really make things simple for a developer to use this. As such, there were two features that I thought would be worth implementing, which go beyond just low-level messaging concerns:


1) Callbacks:


I wanted to make it easy for either system to receive a callback in response to a message that it sends to "the other side". As such, under the hood, there are message ids assigned to each message passed back and forth, and each system is able to return a result which gets properly routed back to the correct callback handler without the developer having to concern themselves with how to match a particular callback message with its intended recipient.



2) Error Handlers:


FileMaker is able to assign an error handler script to its messages that it sends to the Javascript environment. In the event that an error is thrown when attempting to invoke the target Javascript function, the designated FileMaker script will be invoked, along with any available error information.




Script Aliases:


This feature is somewhat experimental, and I go back and forth between feeling that it is an important addition to the code and, at other times feeling that it is just code bloat due to my enthusiasm about all this stuff. The idea is that I did not want developers to have to hard-code names of FileMaker scripts into their code anywhere, and yet the FMP URL technique requires knowledge of the exact name of the target script. The result is that I added a scheme whereby the Javascript code can refer to "script aliases", i.e. strings that remain constant and always point back to the correct script, despite script renaming. This is one of the features where I imagine someone will be able to show a better path that the one I went down. I'd welcome that. For now, I'm still on the fence as to whether this is useful or bloat.



Exciting Topics:


One technology that is on my list of stuff to learn soon is AngularJS. Given this interest, one of the ideas in the back of my head was that I wanted the API of jsv to be very easy to couple with Angular. So far this looks very promising. I attribute this to being able to really just work in a Javascript-intuitive way on the WebViewer end of things, along with the capabilities that ExecuteSQL provides on the FM end of things.


Another technology that I think will be easy to integrate with will be Google Maps, and I look forward to building some kind of demo where the UI automatically refreshes data-driven markers on a displayed map without having to redraw/refresh the map, and without requiring a plugin or a server beyond FMS.


I took a stab at integration at both of these (AngularJS and Google Maps), in the most basic introductory way just to do a proof of concept. I've included these proof of concept examples as part of a demo file which I intend to attach to this post.





Demo File:


As I just mentioned, I have a demo file to share. It is a bit rough around the edges, but it contains a fairly comprehensive walk-through of the concepts that I've described above. Everyone is welcome to check it out, and I hope that many do, but my guess is that you'd need a little bit of Javascript familiarity to be able to really have a good time with it. The demo file will only work on OSX (it might work on iOS -- I tried it a minute ago on a medium-old iPad and I only saw one bug). It will not work on Windows. If you have not made the move to FM13 yet, you would be best off to access the file from a host so that the JS-->FM interaction will work properly. I recommend trying the demo file out using FMP/A and not FMGo -- at least at first.


Additional Note:


I just walked myself through the demo file for one last check of it, and I realize that it's all about code. There's little to no glitz or sizzle to it -- it really only has meaning to someone who is willing to dig into code and see how something works. There's plenty to check out and appreciate there, but, without a natural enjoyment for reading/examining code, it will probably not be a very enticing demo. Sorry about that -- since I really enjoy reading code, I believe I lost perspective about what would be captivating for a demo...






If I were to have one FMP-related regret for 2014, it is that I will be missing the Pause-On-Error which is coming up next week. There are lots of folks that I know that I would enjoy meeting there and learning from. Not having been at POE before, I don't know if it's the sort of place where one can pull others aside and share some code -- it sounds like that may be the case. If so, and if I were there, this is what I would bring to share. Though I am admittedly very excited about the API part of this project, in some sense the most important part is just getting the idea out there about what might be possible with WebWorkers as part of our WebViewer bag of tricks.



Thanks for reading this. I know that my posts characteristically are never-concise, and I appreciate you taking the time to get this far with this one.


Kind regards -- I'll attach the demo file to this post.







Update 29/30 October 2014:


There is now a third code archive attached to this post:


This most recent archive aims to illustrate just the very basics of setting up a WebWorker to listen for messages from FMP.


I have stripped away as much extra complexity as possible, so that developers could look at just the code they needed to see in order to see how it works and/or grab some code snippets/ideas for their own use.


It only has one example, which is a very simple one, and all of the underlying code is exposed and readily accessible.


I hope it is helpful to anyone who wants to learn this idea and play with it some.



Archive NameDescriptionRuns OnFM Version Required
Jsv_demo_20141030_02Extensive demo of jsv, which is a small library of CFs that wrap this WebWorker functionality into an API that implements and hides the related grunt work.OSX 10.6.8 or laterFMP13.0v3 or FMP12-hosted
GoogleMapDemo_20141030_02A demo file which illustrates the use of the jsv library to dynamically control Map Markers in a GoogleMap displayed in a WebViewer, without reloading the WebViewer.OSX 10.7 or laterFMP13.0v3
SimpleFmWebWorker_20141030_04A simple demo file which shows just the basics of WebViewer/WebWorker use described in this post. Most appropriate for grabbing code snippets.OSX 10.7 or laterFMP12 or later



Notes Regarding Google Map Demo File:


1) Use of an iPad along side of FMP/A is optional.


2) Every once in a while I have noticed that a command does not go through. I suspect this has to do with my methodology for triggering scripts across the network, and that the general webworker technique is still sound.