10 Replies Latest reply on Feb 12, 2014 10:54 AM by opencirrus

    Advice on stage releasing a FM DB


      Hello, applogies if this seems to be a basic question, but I am working to develop a FM13 Pro database for corporate access from 5 concurrent users (locally), which will extend to 10 in the coming months. Each user, has a valid licence of FM Pro, and we plan to buy extra licences as they're required. In the future, if we go beyond 10 users, then I will consider FM Server.


      I need to find the best way to stage-release the DB for user testing and user data-input, whilist I continue to develop the solution further. At this stage, I see the only way to acheive this is to release a copy for use (live copy on a local host) whilst remotely working on the next phase elsewhere. I need to ensure no dataloss during this cycle, and the only way I can see of acheiving this, is to export records (table by table), then inport records into the new development as each phase is completed, and tested.


      Can anyone advise if there is a better way to acheive a phased release?


        • 1. Re: Advice on stage releasing a FM DB

          I assume you are hosting this on a server.  If not, that is the first recommendation to use FileMaker Server. 


          You can always download a copy and then rehost it on the server so both files are there and develop from the new file and when things are as you want, move them over to the production database. 


          All DB companies and educators and even FM will tell you not to develop on a production database.  But I do it all the time for small systems like yours (e.g., <25 users).  I usually work on layouts that are copies of the ones I want to.  I try to minimize the amount of time in Manage>Databases or do schema changes off hours or at lunch time.  When other users are using it a lot, I focus work on scripts which can be worked on without any problem while others are using the same database.  I also have progressive backups going as well as regularly scheduled backups so I can go backwards if needed.  And you have to thoroughly think through what you are doing so as not to hinder the current production processes and break the current work flow and user interface.  If you're good at it, you can solve problems much more quickly without having to take down the database and plan migrations. 


          I know it drives my consultant friends who work on Oracle and MS SQL server crazy, but I make live schema changes on FM databases quite frequently. The larger and more complex the system, the less this is adviseable.  You can totally hose a database... or make changes that you did not predict the impacts.  And while not very likely, there are a few weird situations where making schema changes on live databases can corrupt it (e.g., have Manage>Databases open and loose your network connection, etc.). 


          For such a small number of users, I would recommend making live changes so that you don't have to worry about exporting and reimporting (migration).  If you're concerned, do it at off peak times or in the evening.  But you're probably fine to do it live during the day in most cases.  Doing it live will avoid the issue of not loosing data because someone is making changes inbetween your export and reimporting.  

          • 2. Re: Advice on stage releasing a FM DB

            My thought is that if you are allowing users to enter data THAT NEEDS TO BE PRESERVED you are not truly doing a testing itteration you are releasing the DB to "production".

            You wrote "whilst remotely working on the next phase elsewhere" and I take it to mean you have fileA.fmp12 that is ready to be exposed to users, copy it as fileB.fmp12, give fileA.fmp12 to users for production, and continue development on fileB.fmp12.

            If that is the actual case and the data input by users of fileA must be preserved in fileB then you have to have some sort of a data sync functionality to move the data from fileA to fileB when fileB is ready for deployment. Their are many hurdles to overcome with such a process.. for example what if you add a field in fileB that does not exist in fileA or... what if you delete a field in fileB that exists in fileA??? Your sync process must be able to handle all the possible issues.

            It can be done its just got to be a thoughtful well tested process. 


            See taylorsharpe's last paragraph but don't do it without FM server.

            • 3. Re: Advice on stage releasing a FM DB

              Thanks for your input, I may need to organise my time to work on the production DB live. I can always test functionaliy on a mirror copy to minimise work on the production version, so thanks for that. Can I ask, though, why do I need FM server? Perhaps I'm not understanding something here, but for such a small number of concurrent users, is this really necessay? To start I only need 5 users, which is within the share max for FM Pro running as a host? Once I have go beyond 5, then I can upgrade to FM Server. I also take you note on progressice backups to minimise any damage.

              • 4. Re: Advice on stage releasing a FM DB

                Peer to Peer file sharing is inefficient at best.  I recommend it only for people on a really tight budget working with data that is not important.  The Server gives you much more funcitonality in terms of administrative functions like backups and schedule scripting, etc., none of which you can easily do with Peer to Peer sharing.  You may think it is easy to do backups in a local setup, but copying live FileMaker databases will corrupt them, so you have to shut them down to do backups and most people don't do this properly on a peer-to-peer system and they corrupt their data and I hear them get mad that FileMaker doesn't work well, etc., when it really does if you do it right.  If you don't need to share or your data is relatively unimportant, then you don't need a server.  But I highly recommend against making some production database that needs sharing of valuable data without using a server.  I hate to say what I think should be obvious, but that is the whole purpose of a server.  Client applications were never meant for sharing and the sharing they do (peer to peer) is just a workaround for what really needs done properly with a server.  If I were running FM, I would completely get rid of peer-to-peer sharing because I think it causes more problems than it solves.  But FM has a long history of supporting it, so it is still a feature. 


                Obviously you can make peer-to-peer work, but take it from a long time developer, you really need to take another look at getting a FileMaker Server.  Yes, I know it costs more, but you will save a lot of headaches by doing it the right way first.  Best of luck!

                • 5. Re: Advice on stage releasing a FM DB

                  I usually refrain from giving advise on how to deal with an in-house developed database as my methods are frowned on by the developer community and I would rather not give out frowned on advise. BUT, since taylorsharpe (whom I feel is an professional developer) has uttered his thoughts first, I will gladly voice my opinions on working with an in-house database. I have developed in-house and for others. They are very different in they way you approach it.


                  I have developed 3 major in-house database systems over the past 25+ years. This latest rewrite is a 6 file, 200+ table, 1000's of fields and scripts database that is involved in all aspects of our business (except accounting, I use Moneyworks). It's a fairly large system by FileMaker standards.


                  For any multi-user FileMaker database, I would host it via FileMaker Server. It's not even an option.


                  I develop solely on the live database, with users still connected. It doesn't effect the users at all (other than their screen seemingly to magically update). This includes all schema, layout and scripting changes. Admittedly, there are only 6 users currently, but I have worked this way on a 50+ user database in the past without any impact to the users. It really only impacts the developer as he/she may not be able to save their changes till the users commits their record(s).

                  It is much, much quicker to develop live and, frankly, I think it is safely developing as a server client. (I even think a lost connection while in Manage>Database is safe, although I'm not brave enough to test it)

                  I don't use the separation model and why I decided on 6 files instead of 1, I don't know. 1 would have been more efficient. I was probably thing about importing all of those table into a clone if it ever came up.


                  Here is the biggest time saver of them all.

                  You don't need to know exactly what you want the database to do beforehand, just wing it. (that should make a few developers cringe) You should have a reasonably good idea of how the major components are going to interact, but for the details, just wing it. If it doesn't work out, change it. I don't believe it is possible to anticipate every need. FileMaker won't care how many times you make changes to the database. You only have a few users and you are right there to explain your changes and get feedback.


                  With that, I am a big fan of consistancy. Keep the interface similar throughout (duh). Choose conventions and stick with them. If they don't work out, change them, but change them throughout all of the database. I like human readable field names and scripts. I don't like integration data into the table and field names. What the heck is "gt_psn_fn_id"?  You don't need all of that extra stuff in a field name. I do name most fields so that they will sort together in logical groups, though. (e.g. Name_First, Name_Last)


                  Those are some of my opinions, and just like my conventions, "I'm sticking with it".


                  (post kind of turned into a rant, didn't it)



                  • 6. Re: Advice on stage releasing a FM DB

                    Thanks Kyle, to be honest, I'm winging here it - at some point very soon, I have to release a production copy before it's completely tested, and I know there will be some major changes as problems I didn't foresee come to light as the DB is used in anger. I still may chose to work with a test copy to play and test functions and relationships, then any live updates are minimal. However, we will only have 5 concurrent users (in reality, only 1-2 connected at the same time i'm sure). Point taken in using server, I will upgrade one of the 5 FM Pro licences i have to server, this will not be an issue. Due the nature of this project, all licences are volume licences anyway, and costs are not really an issue. One more question, do you host as a web server (for browsers) or does each of your clients have FM installed?

                    • 7. Re: Advice on stage releasing a FM DB

                      You wil breathe easier knowing server is handling the connections/transactions for the users, backups, and everything else it does.

                      • 8. Re: Advice on stage releasing a FM DB

                        Each of our users have a regular copy of FileMaker installed. I use FileMaker Adanced.

                        I may use some iPads to collect production information in the future, but currently, we access the database directly using regular FMP clients.


                        FileMaker (via server) is a tough program. It is not nearly as fragile as some portray. When things happen that cannot be explained, it is very easy to blame corruption, but it's usually just pilot error. I recently fell victim to that myself. (Script trigger interactions can become convoluted.)


                        BTW, I test and test and test, but as soon as a new user interacts with the database, all kinds of glaring issues pop up. Always amazes me how I can miss such obvious issues.



                        • 9. Re: Advice on stage releasing a FM DB

                          opencirrus wrote:


                          as the DB is used in anger.


                          Regardless of the platform, if you allow the situation to get to this point you've lost.  So take care to not allow the situation to get there in the first place.  This is not a technical problem that can be solved with better design / better scripts.  This is all about managing expectations.



                          opencirrus wrote:


                          One more question, do you host as a web server (for browsers) or does each of your clients have FM installed?


                          That's scary.  This is not an after-thought quesiton.  You need to develop with the client's infrastructure in mind.  Developing for the web is NOTHING like devloping for FM clients.

                          • 10. Re: Advice on stage releasing a FM DB

                            Thanks for your answer - I'm sure things won't actually get "angry".


                            I'm developing for FM clients in the first instance - "developing for the web is Nothing like developing for FM clients" - really? That's not what FM have to say, surely you can develop for FM client, then consider what we can achieve or not through a browser?