8 Replies Latest reply on Jul 12, 2012 2:00 PM by ADNPlus

    Some Raw Layout Mode Speed Data

    Charlie Bailey

      While working in layout mode in FMPA 12.0v1, I have noticed a general trend where copy/paste operations get progressively slower as the total object count on the layout increases. I have done tests in several different environments including client/server (LAN), local files, and with and without an SSD drive, and the trends that I present here are apparent in all scenarios. The numbers below come from a test using a local file on a MacBook Air, 1.8 GHz Intel Core i7, 4GB RAM, FMPA 12.0v1, OSX 10.7.3. The test involves creating a number of simple graphic objects (natively drawn squares in this case) and duplicating them in succession using copy/paste. Starting with a single square, duplicating until you have ten squares, grouping these, and then duplicating until you have 100, grouping, duplicating, etc... until you have 100,000 object on the screen. The time for the paste operation isn't measurable (at least not with my iPhone timer app) until I get to about 1,000 objects. From there, the paste operations get progressively longer as more objects are added to the layout.

       

      OperationTime (sec)Total Objects on Layout (at end of paste)
      Paste 1,000 (grouped) objects3.21,000
      Paste 1,000 (grouped) objects3.62,000
      Paste 1,000 (grouped) objects3.83,000
      Paste 1,000 (grouped) objects4.14,000
      Paste 1,000 (grouped) objects4.25,000
      Paste 1,000 (grouped) objects4.66,000
      Paste 1,000 (grouped) objects4.97,000
      Paste 1,000 (grouped) objects5.28,000
      Paste 1,000 (grouped) objects5.69,000
      Paste 1,000 (grouped) objects5.810,000

       

      OperationTime (sec)
      Total Objects on Layout (at end of paste)
      Paste 10,000 (grouped) objects29.810,000
      Paste 10,000 (grouped) objects33.320,000
      Paste 10,000 (grouped) objects36.830,000
      Paste 10,000 (grouped) objects42.440,000
      Paste 10,000 (grouped) objects46.250,000
      Paste 10,000 (grouped) objects51.460,000
      Paste 10,000 (grouped) objects55.470,000
      Paste 10,000 (grouped) objects62.480,000
      Paste 10,000 (grouped) objects70.890,000
      Paste 10,000 (grouped) objects81.6100,000

       

      OperationTime (sec)Total Objects on Layout (at end of paste)
      Paste 1 object3.210,001
      Paste 1 object3.010,002
      Paste 1 object3.010,003
      Paste 1 object3.010,004
      Paste 1 object3.010,005
      Paste 1 object3.210,006

       

      OperationTime (sec)Total Objects on Layout (at end of paste)
      Paste 1 object66.2*100,001
      Paste 1 object70.1*100,001 (repeat of previous test)
      Paste 1 object50.7100,002
      Paste 1 object51.6100,002 (repeat of previous test)
      Paste 1 object51.1100,003
      Paste 1 object51.1100,003 (repeat of previous test)
      Paste 1 object50.8100,004
      Paste 1 object50.8100,005

      *Not sure what's going on with these numbers, they're clearly very different from the subsequent tests.

       

      OperationTime (sec)Total Objects on Layout (at end of paste)
      Paste 1 object01
      Paste 100 objects0.5100
      Paste 1,000 objects3.31,000
      Paste 3,000 objects8.93,000
      Paste 6,000 objects17.96,000
      Paste 10,000 objects29.810,000

       

      Destination Layout Object Count
      Browse Mode Nav (sec)
      Layout Mode Nav (sec)Browse Mode to Layout Mode Switch (sec)
      1000
      100000
      1,0000.91.00.3
      3,0001.31.40.7
      10,0003.33.33.3
      30,0008.49.410.5
      100,00036.247.550.3

       

      Arguably, some of my test scenarios are a little ridiculous as I can't imagine needing to put 100,000 objects, or even 10,000 objects on a layout (though I admit to having created layouts with 1,000+ objects at one point or another). I just needed to get the times into a range that was measurable with my iPhone stopwatch.

       

      Almost all of these tests were also performed using FMPA v11.0v4 (FMPA11 appears to have a limit of 26,817 layout objects which no doubt has been reported before - so I was unable to test layouts with more than 26,817 objects in FMPA 11.0v4). I don't present those numbers here because in all cases, the paste operation in FMPA 11.0v4 was so fast as to be unmeasurable with my timer. So clearly, there is something very different at work with FMPA 12.0v1.

       

      A general sense of layout mode sluggishness was apparent on layouts with 3,000 or more objects. Operations such as object drags, object draws, and multiple object selection all were noticably slower than on layouts with 1,000 or fewer objects. Layouts with 10,000 objects were usable but very slow to respond to the above operations. Layout mode operations on the 100,000 object layout were painfully slow.

       

      Of note - FMPA 12.0v1 never crashed during any of these tests.

       

      I'm not sure how much this data will be useful in real world scenarios, but at least we have some quantitative data to work with. My test files are attached.

       

       

      Charlie Bailey

      inRESONANCE

        • 1. Re: Some Raw Layout Mode Speed Data
          ADNPlus

          Interesting. Users though, will feel the pain more with mere scrolling. I give you two converted files and the csv data source dragged onto FMP11 and FMP12. Clearly v11 is faster when scrolling and searching. I tested only straight on client, OS X (10.6.8).

           

           

          [UPDATE 7/12/12]

           

          Installed the FM12 update and the scrolling speed issue seems to be resolved when testing with the same files. Looking good...for now.

          • 2. Re: Some Raw Layout Mode Speed Data
            HOnza

            Charlie, thanks for sharing your test results.

            My quick guess is that this has a lot to do with the unlimited undo.

             

            Many software developers discivered that it's very hard to find a good balance between undoability and performance.

            Today's hardware makes it much more possible to maintain acceptable speed when having unlimited undo.

            The general practice is to always use the latest hardware for development - and layout mode work is development.

            But only time will show whether specifically layout mode speed issues will be rare, or if they force FMI to respons with some solution, such as ability to disable the unlimited undo.

             

            HOnza

            • 3. Re: Some Raw Layout Mode Speed Data
              Charlie Bailey

              HOnza,

               

              You may be right about the unlimited undo theory, I too expect that this is somehow related to that feature. The data presented here is from experiments done without saving the layout between trials. I looked into whether or not saving the layout between paste operations would affect the times, and found that it did not.

               

              Charlie

              • 4. Re: Some Raw Layout Mode Speed Data
                Charlie Bailey

                Cesar,

                 

                I did some quick quantitative tests with your files and found that yes - scrolling does seem to be slower with FMPA 12.0v1 than in FMPA 11.0v4. The test was performed by scrolling to the top of the page, starting the timer while clicking and holding at the bottom of the scroll bar. Technically, this isn't really scrolling, it's paging, but it's the only way that I can think of to do a quantitative test since I don't have scrolling arrows in OSX Lion and using the intertial two finger trackpad scrolling would undoubtedly give varied results between trials. The effect of paging on scroll speed can be seen when comparing trials with different number of rows visible.

                 

                TestTime (sec)Note
                Scroll through 5000 records (FMPA 12.0v1)23.960 rows, 12 columns visible
                Scroll through 5000 records (FMPA 12.0v1)31.910 rows, 12 columns visible
                Scroll through 5000 records (FMPA 12.0v1)11.060 rows, 3 columns visible
                Scroll through 5000 records (FMPA 12.0v1)25.810 rows, 3 columns visible
                Scroll through 5000 records (FMPA 11.0v4)10.860 rows, 12 columns visible
                Scroll through 5000 records (FMPA 11.0v4)21.010 rows, 12 columns visible
                Scroll through 5000 records (FMPA 11.0v4)4.260 rows, 3 columns visible
                Scroll through 5000 records (FMPA 11.0v4)22.210 rows, 3 columns visible

                 

                Now since this is the performance tuning forum, and not the FileMaker 12 is slower than FileMaker 11 forum, I did some tests to see if I could speed up the scrolling (paging) in FMPA 12.0v1, but came up basically empty. All I could see was that showing more rows increases the paging speed, and even that effect only seems to be significant when the number of rows displayed is under 15.

                 

                 

                TestTime (sec)Note
                Scroll through 5000 records (FMPA 12.0v1)33.8All fields indexed, 10 rows, 12 columns visible
                Scroll through 5000 records (FMPA 12.0v1)36.0All fields unindexed, 10 rows, 12 columns visible
                Scroll through 5000 records (FMPA 12.0v1)31.8Unindexed, 60 rows, 12 columns, list view
                Scroll through 5000 records (FMPA 12.0v1)27.6Unindexed, 60 rows, 12 columns, table view
                Scroll through 5000 records (FMPA 12.0v1)28.5Unindexed, 50 rows, 12 columns, table view
                Scroll through 5000 records (FMPA 12.0v1)28.2Unindexed, 40 rows, 12 columns, table view
                Scroll through 5000 records (FMPA 12.0v1)27.4Unindexed, 30 rows, 12 columns, table view
                Scroll through 5000 records (FMPA 12.0v1)27.7Unindexed, 20 rows, 12 columns, table view
                Scroll through 5000 records (FMPA 12.0v1)26.2Unindexed, 15 rows, 12 columns, table view
                Scroll through 5000 records (FMPA 12.0v1)36.5Unindexed, 10 rows, 12 columns, table view

                 

                - Charlie

                • 5. Re: Some Raw Layout Mode Speed Data
                  HOnza

                  OK, good to know that the unlimited undo does not seem to be the bottleneck here.

                  It may be also related to the increased complexity of the layout.

                   

                  Regarding the scrolling speed and you inability to come up with a way to optimize it:

                   

                  When I teach peple how to optimize bottlenecks we often have to go through thinking ouside of the box...

                   

                  Basically there are two ways to optimize a bottleneck:

                  1. Make it run faster
                  2. Use it less often

                   

                  Sometimes things simply must be slower (not saying it's this case and not saying it must be so much slower).

                  Sometimes it's difficult to make the mind shift to not mind, but it's actually possible in most cases.

                  But possible does not always mean reasonable, of course.

                   

                  I am just always trying to look at issues from different angles, and include even the least usual angles ;-)

                   

                  HOnza

                  • 6. Re: Some Raw Layout Mode Speed Data
                    ADNPlus

                    Thanks Charlie, I am glad that the latest update seems to address this and I think making our voices heard as well as analysis such as yours contributed to getting this to the attention of the FM Programming group. I was worried. It's one thing to notice these issues as a developer and a whole other thing when end users notice them and as a developer nothing can be done about them. Let's see what the near future brings and hope it nothing less than a pleasant FileMaker solution building experience.

                    • 7. Re: Some Raw Layout Mode Speed Data
                      Charlie Bailey

                      I just repeated the scrolling tests on the same hardware, once with FMPA 12.0v1 and then again after upgrading to FMPA 12.0v2, using the Crispy11.zip files referenced above in this thread. It's interesting that I got somewhat different numbers than my previous test in FMPA 12.0v1, but the relative numbers (comparing v1 to v2) are still encouraging. The same disclaimer that this really isn't scrolling applies here (it's really paging).

                       

                      TestTime (sec)Note
                      Scroll through 5000 records (FMPA 12.0v1)18.660 rows, 12 columns visible
                      Scroll through 5000 records (FMPA 12.0v1)42.810 rows, 12 columns visible
                      Scroll through 5000 records (FMPA 12.0v1)9.160 rows, 3 columns visible
                      Scroll through 5000 records (FMPA 12.0v1)24.410 rows, 3 columns visible
                      Scroll through 5000 records (FMPA 12.0v2)12.360 rows, 12 columns visible
                      Scroll through 5000 records (FMPA 12.0v2)24.610 rows, 12 columns visible
                      Scroll through 5000 records (FMPA 12.0v2)5.760 rows, 3 columns visible
                      Scroll through 5000 records (FMPA 12.0v2)24.710 rows, 3 columns visible

                       

                      There was no significant difference between v1 and v2 with the 10 rows, 3 columns test, but in all other cases, v2 outperformed v1.

                      • 8. Re: Some Raw Layout Mode Speed Data
                        ADNPlus

                        Thanks for the update. In my test of the 10 rows, 3 columns visible I see almost no difference in performance. FM11 is speedier but the difference is almost negligible from a user stand point. This with both copies sitting on the desktop. I also tested a different database hosted of my FM12 server in Colorado (I am in San Diego), and performace is great. Using OSX 10.6 here. Barring any unforseen hitches, I am moving forward with more solutions conversions to v12. I am not all that crazy about the new design tools but it'll be easier to get used to them now. Keep on truckin'