13 Replies Latest reply on Nov 7, 2012 1:03 AM by psijmons

    Data Separation - Local UI and dynamic data source

    Steve Wright

      My existing solution uses a launcher file which allows the user to enter their IP address into a field then (via url) opens up the hosted ui and hosted data file, but its really pointless transferring all of the UI data across the network, especially if moving to 12 since the file sizes are generally larger due to the underlying UI code.

       

      I am contemplating a complete re-write from scratch, so have been experimenting with having the UI locally and connecting to a hosted data file, this brings up one problem..

       

      How could I allow the end user's to specify the IP of the data source in a similar way to above, considering that my solution has all admin access removed and this is the way I intend to keep it for various reasons. Basically, Manage External Data Sources is not accessible when deployed.

       

       

      So far, I have come up with a few ideas, but only one of them seems to fit:

      0. Variables are not allowed, so that ideas out of the window.

      1. I tried opening the local UI, then opening the data file via url and hoped it 'recognises it as the source', why I thought that would work I have no idea, but tried it anyway.

      2. I tried adding a range of ip's, but this is impractical, slow and not really where I want to be heading.

      3. I have considered having users change their 'computer name' again, impractical really.

      4. I have considered having users set one of x static ip addresses, but I could never account for the variations realistically, so again impractical.

      5. I tried fmnet:/*/data which is not recommended by FileMaker according to various documentation, however in my limited test cases does seem to work.

       

      So I guess my other question is, considering test 5 works ( fmnet:/*/data ) what are the implications of using it, why is it not recommended, does anybody use it?

       

      I know * is a wildcard, therefore FM will be searching the network in some form or other but its considerably faster (in my tests) than for instance having

      fmnet:/192.168.1.1/data followed by

      fmnet:/192.168.1.2/data etc etc..

       

       

      Maybe I am approaching it wrong, or maybe there really is no solution other than manually editing the data source path.

       

      I would love to know how others handle this because a local UI does appeal to me, its just the above gets in the way of a simplified deployment.

      FMI have advised against using the only method I have found to work so far and allow easy deployment with no intervention (ie editing the solution files)

       

       

       

       

       

      EDIT:

      I forgot to add that the wildcard usage is advised against here:

       

      FileMaker does not recommend using an asterisk (*) as a wildcard character in network file paths as it slows FileMaker network traffic. When possible, replace an asterisk with the appropriate IP address or use variables in file paths. If you have converted a database from a previous version of FileMaker Pro, review the converted data sources and replace any asterisks with known IP addresses or network file paths.

      src:http://www.filemaker.com/11help/html/create_db.8.32.html


       

      However, It would be great if it was clarified what is meant by "slows network traffic" in terms of slow whilst discovering the database, or also slow whilst interacting with the database files. Of course being unreliable is also a concern.

       

      And probably the most important fact, I am only concerned with Local Network's

       

       

       

       

      Many thanks,

       

      Steve

        • 1. Re: Data Separation - Local UI and dynamic data source
          psijmons

          hello Steve

          I thought about this problem a lot, as the advantage of pushing the UI files to the user is very appealing.

          But I never came up with a solution.

           

          My clients all get a launcher file that has their IP nr hard coded. This will open the correct set of UI and data files (on their own server).

          When the UI files are on client machines, you need the IP nrs hardcoded in the UI files, no way around it as far as I can see.

          Having a whole list of IP nrs in the External Data Source is not a viable option, unless you can set them all on the same local IP nr.

          Which I can't across all my licensees, and several need external IP nrs.

           

          I will be curious to see if someone has a bright solution for this.

          Peter

          • 2. Re: Data Separation - Local UI and dynamic data source
            DavidJondreau

            I don't understand why you're trying to do this.

             

            It sounds like you're trying to use one interface file on multiple client machines to connect to different host machines on an internal network...which is weird, why not have one host on a dedicated machine on the network?

            • 3. Re: Data Separation - Local UI and dynamic data source
              Steve Wright

              Hi David,

               

              I think you may have mis understood, or I may not have been clear

               

              There will only ever be 1 host, hosting 1 data file (per customer, calling them a client may confuse matters)

              The plan was to have a copy of a single 'UI' file on multiple client machines connecting to the only hosted data file.

               

              Why, because the UI without data can be in excess of 100mb going by the 12 conversion of our current solution, thats 100mb of data that may or may not be transferred over the network for the UI alone, it makes a lot more sense to hold this client side and only transfer the data.  At least, thats how I see it.

               

               

              The solution is a vertical market solution not in-house or custom, therefore we like to keep the deployement as simple as possible.

               

              Having to edit a customers IP address manually for each sale of our solution is out of the question, considering it may also have to be later edited if something changes with their network and that our files have privelages removed. 

               

              Short of finding a reliable solution, the alternative is what we currently do and that is host the UI & data file, using a simple launcher to open the remote UI (which in turn opens the remote data)  This allows us to deploy with no editing of the files before hand.

               

              Using a wildcard is looking like a possibility if only I can figure out the actual problems it could cause, because in my test's so far I cant really find any.  The good thing is, if for any reason the wildcard cannot connect then it could easily fall back to launching the hosted UI instead (by way of the current launcher method).

               

              Hope that makes more sense.

              • 4. Re: Data Separation - Local UI and dynamic data source
                Steve Wright

                I may have an answer on windows, its something I currently use during web development on a local machine to manage faux domains.

                 

                Basically, instead of hard coding the ip into filemaker, I could prompt the user for the server IP during install and automatically append it to the windows hosts file.

                • C:\Windows\System32\Drivers\etc\hosts
                • Privelages may be an issue, however the installer I use does apparently support this.

                 

                By appending something like:

                • 192.168.1.100  mySolutionDataSourceName

                 

                 

                This would allow me to add the data source filepaths as:

                • fmnet:/mySolutionDataSourceName/data
                • fmnet:/*/data
                • or vice versa

                 

                 

                Since the solution itself is locked out, this also provides a way to make changes on a client machine if required, such as a server IP change, without having to provide access to manage data sources, or provide a replacement file with the hardcoded IP.  Im not sure of the equivalent on Mac OSX but im sure it exists.

                 

                 

                I would still like to know why a wildcard is not recommended, because since trying it between PC, Mac, iPhone, iPad and Peer to Peer sharing from FM Pro, it works.. I have not noticed any cause for concern at least on my network, but in all fairness I have not tested with a full on solution, with large real world data sets (yet).

                • 5. Re: Data Separation - Local UI and dynamic data source
                  DavidJondreau

                  OK, that makes sense.

                   

                  Why not make it a scripted process? You can either check to see if the data file is available or check against some stored value if the filepath has been set yet and if it's not, run a script (with Run script with Full Access Privileges checked) that uses the Open Manage Data Sources[] script step.

                   

                  Let the users do it themselves.

                  • 6. Re: Data Separation - Local UI and dynamic data source
                    Steve Wright

                    Hi David,

                     

                    You got me there...I never actually thought of opening the "Manage Data Sources" window via a script with admin privelages, instead I thought I would over complicate matters but to be fair, I have only just started thinking of having a local UI so dont find myself needing to touch the data sources often.

                     

                    This is certainly the solution for me, even though the user could potentially delete the data source, breaking relationships it would only be in the UI file, so easily replaced and of course this makes it easier to modify should the need arise in the future.

                     

                    Many thanks.

                     

                     



                    • 7. Re: Data Separation - Local UI and dynamic data source
                      caromsoft

                      You might want to test running the UI locally in a real world setting before you go too far down this road. My solution has a UI that is about 54 MB in size right now, and like you I was worried about the speed. I also employ some layout tricks which should slow the solution down even more over a WAN. When I tried running the UI locally I didn't see a huge difference in speed, which surprised me. Whatever difference in speed there was was offset in my mind with the logisitcal problems of getting an updated UI to all the client machines, instead of just to the server(s).

                       

                      Of course every solution is different, and it might make a big difference with yours. Good luck!

                      1 of 1 people found this helpful
                      • 8. Re: Data Separation - Local UI and dynamic data source
                        psijmons

                        Hello Steve,

                         

                        when you put a wildcard in the source filepath, the launch file may search for a very long time and eventually time out before a proper connection is made.
                        But the heart of the puzzle is how to transfer the hard-coded IP in the customer launch file, then set that in an anonymous UI file on the customer machine, which in turn will connect to the correct remote data file.

                        It owuld be nice if you could use a param in the fmnet path but unfortunately, the moment you touch a script that needs to connect to the data file it will fail because the param is not yet processed.

                        Or am I missing something?

                        • 9. Re: Data Separation - Local UI and dynamic data source
                          psijmons

                          the logisitcal problems of getting an updated UI to all the client machines, instead of just to the server(s)

                          Carom,

                          That is the easy part because you can have version control on the UI files. When the user opens a UI file that is old, a newer version is pushed from a container field in the data file to the desktop.

                          • 10. Re: Data Separation - Local UI and dynamic data source
                            Steve Wright

                            Argh... I just lost my original reply, that will teach me to write directly to web form without copying before posting

                             

                            @caromsoft

                            I have considered that the benefits will not be what I had expected, luckily enough this is interchangeable.  It would be extremely easy to revert it to a hosted UI and hosted data file, so I have nothing to lose aside from the time spent on the initial issue...at least up until I can perform more thorough testing with real world data. 

                             

                            Also, I already have a process in place for updating client side files, although my existing solution only uses a launcher client side, we do use various plugins so need to ensure the client machines have the correct versions of these etc.. I forsee the process being very similar.

                             

                            @psijmons

                            If the wildcard's symptoms are solely a slow down in initial discovery, then it still maybe a viable option (at least for some clients) I like the simplicity of it and simplicity is required since most of our clients are not (how do you say this nicely)  in tune with computers.

                             

                            Essentially, the hard coded IP will be exposed via the scripted way of invoking the the 'manage data sources' allowing the client to make changes with no real consequence other than maybe deleting the data source and having to obtain a new client file.  I can place money on the fact that this WILL happen.


                            Since they would be setting the IP in the UI, it would not need to be set in any other file.  The UI is essentially both the UI and the launcher, it could even fall back to launching a hosted UI via our existing method of opening the URL based on a global field..  Why? no idea but it could

                             

                            The biggest hurdle I was trying to avoid was to have to 'build' (in the same way you do a runtime) a launcher file for each customer to cater for their IP addresses, since I could see now way to define it after distribution.  I just didnt look close enough.  With that out of the way I can at least move forward to the next hurdle which may or may not exist.

                             

                             

                            Ive just been called out, so ill have to skip the proof reading and hope above makes sense.

                             

                            Thanks everybody so far for your input, its appreciated

                            • 11. Re: Data Separation - Local UI and dynamic data source
                              psijmons

                              Giving users access to the Open Manage DataSources dialog is not an acceptable routine for me. Users should be presented with a username / pw dialog and that's it.

                               

                              My Launcher file is an extremely small FMP file: one layout, one script, that will directly open on a specific key combination to OpenManageDatabase. So very quickly, I can prepare a launcher for a new customer and hard code their IP. The filename of the launcher carries a customer acronym that is checked when a data file is opened. If there is a mismatch, FileMaker will quit. Just as an additional precaution.

                               

                              Note that the * wildcart in the filepath you mention in the FMI instructions above is only for a filepath you use for import / export xlsx, txt, pdf, etc. files, a wildcart will not be allowed in a fmnet path.

                              • 12. Re: Data Separation - Local UI and dynamic data source
                                Steve Wright

                                Note that the * wildcart in the filepath you mention in the FMI instructions above is only for a filepath you use for import / export xlsx, txt, pdf, etc. files, a wildcart will not be allowed in a fmnet path.

                                 

                                 

                                You can define a data source using fmnet:/*/datafilename and it works. I didn't think you could either until I tried it

                                [edit] Guess I should add this

                                But it's not recommended as it slows FileMaker network traffic  according to
                                http://www.filemaker.com/help/html/odbc_ess.20.5.html

                                [/edit]


                                As a test, I have around me 2 Windows PC's  an iMac, iPad, iPhone and Macbook.

                                 

                                I setup a small data file with about 5 tables and some random data

                                I setup another file to act as the with a portal showing data from the first file.  The data source is set as : fmnet:/*/data nothing more, nothing less.

                                I then proceeded to host the data file between all possible desktop's via FM Pro peer to peer.

                                Likewise, I tried connecting via all possible client systems


                                It worked on all accounts and at the same speed as my launcher. 
                                In all cases the data file was only available via one acting host, it was not in the local files, so there was no possibility of being mistaken and actually loading a local data file. Likewise, the interface was not hosted, so I wasnt just opening a remote UI with valid data source.

                                 

                                 

                                 

                                Re your launcher:  Im not sure I understand why you need to hard code anything if it is just a launcher?


                                As with my existing deployment, I do not have to hard code any IP address, nor do I have to provide access to data sources instead when the user installs the client, they simply enter their IP address into the launcher.   It stores this in a global and attempts to auto connect upon next loading, if it fails they are asked to check / enter their IP again or can choose Open Remote if they really need to.


                                It does this simply by scripting an OpenURL[ fmp7://" & table::field & "/data" ]

                                I believe its just fmp://" & table::field &"/data"   in 12.

                                 

                                When the remote data & ui file is opened a script closes the launcher file.

                                The client version is stored in a text file per client machine ( this is exported each time the launcher is loaded for accuracy)

                                The remote ui file re-imports this into a single global field for checking it is a suitable version, if not the remote UI informs the user and performs the appropriate action.

                                 

                                Deploying the clients this way saved me ever having to hardcode anything, one launcher suites all clients which is what I was hoping for with the client side UI.




                                • 13. Re: Data Separation - Local UI and dynamic data source
                                  psijmons

                                  hello Steve,

                                  Good suggestion!, I completely overlooked the option to use the OpenURL command.

                                  I will be testing such a setup shortly.

                                   

                                  BTW, my version control is done on the very first script in the UI file with the version as the script name. It contains only 1 script step: Exit Script [ GetScriptName ] which is picked up by the parent script and compared to the latest version.

                                   

                                  hardcoded IP: the reason for that is that for some customers I need to add not just the local IP but also an external IP.