1 2 Previous Next 17 Replies Latest reply on Jan 1, 2015 3:23 PM by wimdecorte

    Rebuild or Convert?


      When i started at my current employer 12 years ago they had a solution for names and adresses in place, built in Filemaker 4.

      Since then the solution got built on by me (A newbie at Filemaker at the time) and my colleague (A newbie with some knowledge), it evolved as did Filemaker and when we look at the system now it looks more like a hodgpodge of 96(!) files stuck together than anything else.

      Nowadays it contains billing information, names, adressen, Fax correspondence, order bills, forms and even a form of communication with our window calculationsoftware for the sake of ERP, CRM and Warehousing etc. etc. etc.

      The system works but is cumbersome, untangible and gets more and more impossible to fathom when making adjustments.

      With everything we know now we would have definitely built the system in a different way but that's hindsight.


      So, now the million dollar question....we want to change the system drastically but we have to retain all history because it most of it isnt even history yet (customer communication, billing etc).....what would you do?


      Rebuild the whole thing or just keep on plowing through the history of an outdated Filemaker system (And converted to yet a new version soon) ?


      I'd love to see your thoughts.....

        • 1. Re: Rebuild or Convert?

          Hello Rob,


          My first FM application was built a few years ago, in FM9, and I've had a similar decision to make (although it sounds like your system is much more complex, and certainly a lot older).  I decided to go for a re-write, for these main reasons:


          1. Much of the design was flawed, and re-designing an existing (converted) system would have been harder (and less satisfying) than building to a new design.  Re-building is a good chance to put right your past mistakes (and the mistakes of others).
          2. FM12 has so many new features which, again, greatly impact on the design process.  The ExecuteSQL script step, for example, is going to greatly simplify my relationship graph, and modal dialog windows are also simplifying many of the validation routines.  These are just a couple of examples - there are many.
          3. Many UI features in the old system (such as portal row adding and deleting, etc.) relied on old techniques, which are now much smoother because a) I'm more proficient with FM, and b) FM is better.  I felt it would be a mistake to re-engineer so many shared features - again, more satisfying to do it "more right" in a newly created system.


          Obviously, rebuilding is a lot of work.  But I'm not regretting it so far.  As much as anything, I'm really enjoying the sense of doing it "right" this time around, and feel confident that the new incarnation will be a much better thing than the system it's replacing, so it'll be worth it in the end.


          Good luck, whichever way you decide to go!



          • 2. Re: Rebuild or Convert?

            we want to change the system drastically but we have to retain all history because it most of it isnt even history yet (customer communication, billing etc).....what would you do?


            Don't re-invent the wheel. However complex or ungainly as your system is, it represents 12 years of work.


            Creating a new application with the same functionality from scratch will take you weeks or months. All the code will be new, untried, potentially buggy and the application will be substantially different. It will have to be extremely successful to overcome user resistance to change.


            I suggest that you move the entire thing to the most current version just as it is. Test it to ensure that nothing breaks in the process. At that point you have a working application. This may take hours or days.


            If you have the time and budget you can then identify changes that you would like to make. At this point you are improving a working application. Any changes you make will be incremental and may be appreciated for the benefits that they bring.



            • 3. Re: Rebuild or Convert?

              Ditto. This is usually what I do. You can use the converted solution while it is being revised.


              Make a PLAN before changing so that you know the optimum way to revise. You can create new interface file(s) and point to the data in the converted file(s).


              -- sent from my iPhone4 --

              Beverly Voth


              • 4. Re: Rebuild or Convert?



                I'd strongly advise you look through Technet archives regarding converted solutions. We're pleased that FileMaker took such a bold step and direction change in moving to the FMP12 file format, but in our experience, since v12 was released, converted files do give a lot of problems. We have no evidence, but our suspicions are that the problems people have been experiencing depend on the settings of the .fp7 file at the point of conversion, therefore you get a lot of 'we have this problem' with 'we've never seen that on our version' type threads.


                I do agree that evolved business processes shouldn't be discarded and to use this as an opportunity to retain those things that work well and improve/enhance those that are could be better, or add those that don't currently exist.


                We've been through file conversions roughly every 7 years (.fp3, .fp5, .fp7 to .fp12) and each has presented their own advantages and challenges. You can of course convert your existing solution to v12 and see how it goes and, as Beverly suggests use it if it tests out OK. You haven't mentioned whether the existing system is a .fp7 solution, but should it be earlier you'll have all sorts of window problems and, even going from .fp7 to .fmp12 we've had to go through every check box on every layout as they've failed to convert correctly.


                From the sound of things 96 separate files must be a scripting nightmare, so longer term we'd recommend setting up (at least) 2 new files created in FileMaker 12 and move to the separation model with the business intelligence held in a user facing file, with one or more data files feeding into this. You should be able to import your existing tables into the data files, but assuming you're calling scripts to and from other files, the table occurrence graph, layouts (definitely) and scripts will need to be set up from scratch. However, as you have a working system, albeit a big one, you should be able to use this as a template for the new system (just don't copy any objects from the old to the new).


                We have mature .fp7 systems we've converted and despite approaching 30 years FileMaker experience each, found ourselves unable to set layout colours or format fields with pop-up menus. Thankfully 12.0.v4 has resolved some of the more disasterous problems we experienced with the earlier versions.


                Taking the above approach, you should be able to import your data using matching field names when ready to migrate, you'll be able to use variables throughout, apply themes to layouts and have a fraction of the script and relationships complexity you must have with the number of files you're dealing with. You may also want to consider whether there is likely to be a mobile access requirement in the future and consider replacing a lot of the cross table calculations and 'thin out' the complexity to improve performance over the Internet.


                I hope this is of help. We've 1000 to 3000 hour solutions we have the same problem with and are still considering whether to follow our own advice given above, but these were mostly designed around the separation model, so we may decide to put up with some file conversion pain on those files that don't consist of multiple, multiple files.





                • 5. Re: Rebuild or Convert?

                  It sounds like you have a very complex, mission critical, system. One thing I wouldn't do is let a file change drive your re-write schedule. We've found zero 11 to 12 issues in the last 9 months, and previously migrated all of our clients from v6 to the v7 structure. New features may provide the tipping point, but manage the re-write on your own schedule and business cycle.


                  If you do decide to rebuild, I suggest carefully analyzing your solution and breaking it down into functional modules, probably oriented around a task. Contact management, lead generation, sales orders, inventory management, and the like. Then you can go about rewriting parts of the system, rather than the whole thing at once.


                  You might look first at the most discrete, non-interrelated process. This will allow you to establish the look and programming methodologies that you'll extend throughout the system. Deploying one section will give you field testing and user feedback regarding your choices.


                  I'd then target the module that will give you the best return on investment. This would be the module that is the most troublesome to users, and likely the module where most active development is taking place. It may also be productive to target the "core" module (often Contacts) that all of the other systems plug into - sometime restructuring the core module is required to effectively re-write the peripheral systems.


                  Re-writing by module will be a little extra work, as you have to "plug" the new modules into the old system, but will lead to a faster deployment and more manageable bugs.


                  As you move forward, think hard about what and how much history to transfer. The whole point of the rewrite is change things; you don't want your thinking "twisted" to accommodate the old structure. After you are done, you can develop conversion routines to move the old data to the new system. But some things may not be worth converting - there is generally a lot of seldom used info in a big system. The old system can be "left around" to look up this occasional data. You could also export this information to flat table files that users can query, without an extensive interface.


                  Good luck!

                  Bob Gossom

                  • 6. Re: Rebuild or Convert?

                    Hello, Rob.


                    I've done this in a number of different ways. I will say that, in your case, 96 files is a lot. Andy's comment about a "scripting nightmare" in such a scenario is borne out by my experience, as scripts have to jump around between files to make things work. I can tell you that, in one project, I had one script that was over 150 pages long (printed). Once the solution was consolidated down, that script shrank to 2 pages. So moving all your code into a single interface file makes a lot of sense (just for maintainability). But it doesn't necessarily have to happen all at once.


                    There are different paths you can take with the upgrade. The attached document lays some of them out. It's old, but most of it is still valid.


                    You've already gotten some very good advice. So I'll just say that the decision of which path to pursue will depend on:


                    1) How much time you have available to build - and TEST - the new build

                    2) How much downtime you can afford on the live system

                    3) How much vestigial garbage you have in the current system (most build-as-you go systems accumulate a fair amount of this)

                    4) How much of the existing look-and-feel you really want to retain


                    Best of luck.



                    • 7. Re: Rebuild or Convert?



                      A wee test to try before making any decision and I'd welcome feedback from anyone who have said they've had no problems at all with their conversions. The background to this is that we detest the solid black shadows on fields formatted as pop-up menus and try to soften these or even remove them if the field is in a report and we're displaying the 2nd value only from a value list.


                      In FMP 11 create a brand new database with a single field, add to a layout, format the field as a pop up menu and add a simple value list to this. Now set the 'Line' value to 'None' and then change the colour and observe both results.


                      Now in FMP 12 do the same with a new database created in v12 using the 'Classic' layout and repeat and observe the results. In our experience the Classic view formats the pop up field as a Drop Shadow, which is greyed out in the formatting pallete but you can't get rid of the damn thing - ironcially Drop Shadow isn't available in v12! Try all options, including remove styles.


                      You could also try converting the original test .fp7 file and repeat. Have a pop up menu formatted and one at default and observe the results.


                      In our experience you cannot remove the black shadow line in v12, so in a report, for example, that was displaying someone's name but was actually a Human Resources ID field you suddenly get horrible black outlines.


                      When you do your conversion your layouts will effectively be using the Classic theme, whereas all the nice new themes appear to work very nicely, unless you are trying to squeeze a lot of data onto a layout (poor design I know, but sometimes that is what the customer wants!). We find ourselves creating stand alone v12 databases using themes, copy and paste fields then use the format painter to transfer the styles to some of our legacy fields.


                      Again to repeat, conversion does seem to depend on the state of your database at the point of conversion and, I guess, how fussy you are about the design. We're producing commercial products, so the appearance is very important to us. We have had some databases that converted and all the check boxes had to be reformatted, whereas other databases were fine!


                      This is only a small example, but with 96 files to deal with, small things have a habit of growing. We do love most of the new features that v12 provided us with and look forward to seeing what FileMaker will add with the next version. Our v11 cloud server is now very undersubscribed, whereas our v12 servers are continually expanding and we will continue to migrate our LAN based customers' solutions to the .fmp12 format.


                      Hope this is of interest.



                      • 8. Re: Rebuild or Convert?

                        Hey, Andy.


                        I'd say your experience is typical. I've had few or no problems with converted solutions. However, one thing I did notice was the "embossed" and "engraved" settings that used to exist converted poorly. I ended up replacing such elements with version 12-native layouts.


                        Another glitch I noticed was that line heaviness sometimes gets out of whack. For example, if the original solution used lots of hairline widths for lines, I ended up converting them to 1 pt to get them to render consistently.


                        There are a few other cases, such as sliding objects with field borders that don't seem to render correctly. You can search the forum and find threads that discuss some of these issues.


                        That said, from what I've read, most of the interface difficulties with conversion come from folks who took great pains (like yourself) to design custom layout "look and feel" to overcome some visually unattractive element in the default layout objects. There were a LOT of very unhappy folks at DevCon 2012, whose custom interface designs were totally blown by the v12 conversion. Those who used more or less what FileMaker provided had far fewer issues.





                        • 9. Re: Rebuild or Convert?

                          Thanks Mike


                          I think we can summarise our particular experience as 'look back with pain and forward with optimism!'



                          • 10. Re: Rebuild or Convert?


                            • 11. Re: Rebuild or Convert?

                              Rob, we did 2 major conversions on our main product fp5 > fp7 > fp12 

                              We did a careful analysis on the pro's and con's of a rebuild and both times we decided to rebuild from the ground up for a number of reasons:

                              • weed out the code we would design in a much better (faster and DRY) way in the new versions (parameter handling, globals, ExecuteSQL just to name a few that appeared over time)

                              • implement data separation structure

                              • complete overhaul of UI and navigation structure (both times, UI expectations change over time)


                              Actual data migration was never an issue as long as you keep your field names intact. If you want to update your naming convention, you will need to do a manual mapping for imports.

                              But in the fp7 > fp12 conversion we did spend a lot of time making 2 very versatile and portable template layouts, one for form view and one for list view, once we got those in place, development went very fast.



                              1. make a general workflow plan from scratch, really start with a blanc sheet here
                              2. form a small group of stakeholders and prepare mockups with e.g. Balsamiq software to get feedback on your ideas
                              3. be careful with an ever growing list of requirements from stakeholders, decide what is essential and what is luxury (and moves to priority low)
                              4. build from the ground up (and plan some serious time for this), connections with outside systems will probably be a lot easier in the latest FMP.
                              5. Form a small group of testers (usually those with lots of experience in the actual data entries) and debug as much as possible on a parallel system, while your old system is still being used.
                              6. Use their feedback and get them involved, people are usually aversive to change but once they see the new opportunities they will grow into it
                              7. Build a solid scenario that will migrate the flat data fp4 > fp5 > fp7 > fp12 (you will need FMPA in all of these versions, or involve a 3rd party who does this routinely). test, test, test
                              8. Train more staff on the parallel test system, more debugging possibly
                              9. set a weekend for migration
                              10. run all conversions using actual data during the weekend
                              11. GO


                              There are a lot of decisions you need to make before you start building your new solution (like the way you design your ERD, data separation Y/N?, etc. All I can say is take your time in the beginning to think things over, it will pay back later in the project.




                              • 12. Re: Rebuild or Convert?

                                Well, it looks like there's enough to consider....

                                I'm leaning toward redeveloping the entire solution but i'll take some serious time looking into it with regards to all of your tips and advices.


                                Whatever the outcome, i had some real insights from your reply's while trying to determine what the next step should be.


                                Thanks in advance and i'll post what we decide to do!

                                • 13. Re: Rebuild or Convert?

                                  Good choice.


                                  The client is typically happy with an initial conversion, but the extra fields, code, and table occurrences drive me nuts. It's harder to continue developing a converted solution when the client asks for a modification -- I wonder if I should fish or cut bait. Do I forge ahead with getting new features built, or do I clean up what I find along the way? Do I constantly create new DDR and poke around to see if a global field is in use?


                                  In my opinion, conversions are better at the end of the day, but sometimes you and the client need to know about how long that day will be.


                                  Somewhat in summary of the good advice above: I've always been happy with redevelopment, and rarely been personally happy with a conversion. Only when a rebuild was going to take too much time (to the client -- cost too much) do I simply convert.

                                  • 14. Re: Rebuild or Convert?



                                    You have received a lot of comments some. I haven't read them all so if this is redundant, sorry.


                                    The first red flag is 96 files. The second red flag is your comment "The system works but is cumbersome, intangible and gets more and more impossible to fathom when making adjustments."


                                    These 2 factors alone would cause me rewrite the system from scratch in FM 12. What would stop me? The Budget but you have an advantage being an in house developer.


                                    Besides it would be a lot more fun creating a new solution!


                                    John Morina

                                    Pueblo Systems, Inc.

                                    CCQ-FM, Inc.

                                    1 2 Previous Next