12 Replies Latest reply on Jun 14, 2017 11:36 AM by Stephen Huston

    Container thumbnail rendering issues

    justinc

      I have been trying to improve the performance of a list view layout and have run into some oddities or issues with a container field.  The layout in question is a list view that has an image shown on it in each row.  It's not huge - 58x87 (WxH).

       

      * The Image container field is local to the layout's base table.   (I de-normalized these thumbnails - along with nearly everything else shown on the layout.  The Body part objects that are shown at first rendering are all local to the base table.)

      * I recently reconfigured the DB (File -> Manage -> Containers) to enable Thumbnails and to Permanently store them.

      * I am using FMPA 15.03, and the file is hosted on FMS 15.03 also.

      * The container field on the layout is configured to 'reduce or enlarge' the image.

       

      Considering that I'm trying to improve the performance of the layout (i.e. scrolling to be smooth and quick - no hesitations or lag)

      1)  Is there a better way to configure the DB to store these images, for better performance?

       

      2)  The caching in 15:

             When I close the file and reopen it (not restarting FM itself, just closing the file), and navigate to this layout, the images will again show the 'Loading...' symbol the first time through.  I can scroll down the page and easily out-pace the ability of FM to load the images and it just shows the gray circle.  But once I have slowly stepped through the entire list so that all the images have loaded up, it scrolls quite nicely - there is no lag and no stray loading of images.

           Isn't 15 supposed to retain the cache between file closings?  I would expect the same 'no lag and no loading of images' behavior in 15 after closing the file - IF the cache were being maintained and IF the thumbnails were being permanently stored.

        • 1. Re: Container thumbnail rendering issues
          bigtom

          How big are the original images?

          Create a calculated container field with the value: GetThumbnail(image_field;58;87). Use this field on your list layout. this should speed things up.

           

          If it works out well you can change the field to un-calculated and loop through setting the images via script which will be even faster.

           

          It is best if the large images are in a separate related table as the do not need to transfer when to the layout unless needed in the large size. Put the small thumbnails field in the layouts table or another related table.

          • 2. Re: Container thumbnail rendering issues
            justinc

            We reduce the originals to a thumbnail size already, upon the initial import:  145 x 200.

             

            I'll try the calculated field trick and see how it goes.

            • 3. Re: Container thumbnail rendering issues
              bigtom

              So you no longer have the large images in the table?

               

              I have found that 1.5x the container size works well enough for retina displays so it doesn't look bad.

               

              Do you have SSL and progressive download enabled? Not sure how much that helps with images.

               

              I have done a list view with 150x150 images and thousands of records. I can scroll fast through the list and get the loading icon for a second or less before the images appear. How long are your loading?

               

              LAN or WAN connection?

              • 4. Re: Container thumbnail rendering issues
                justinc

                This is a WAN file.  SSL is not enabled on this server.

                 

                Correct: the related 'Images' table stores only a cropped down version (145x200) of the original image (whatever size was imported).  This related 'Image' container is then further scaled down to match whatever size is on the layout (in this particular case, 58x87) using the 'reduce/enlarge to fit' option on the layout-object definition.

                 

                Taking your suggestion I created some AE-calc container fields that matched the exact size that I was using on the layout (58x87) and prepopulated those.  I wasn't seeing a whole lot better performance (i.e. scroll time to reach the last record).

                 

                My testing found set is 150 records.  Using FMP 14 and closing/opening the file between each run (to remove possible caching effects), my baseline performance is ~20 seconds to reach the last record, when scrolling from top to bottom by holding down the mouse button while pointing at the bottom of the scroll bar.  This is a very jerky and laggy process, and it doesn't even yield a fully-loaded cache; that is, I can scroll back upwards through the found set and still find many records that haven't loaded the image.  When paused on any given record that has the 'Loading...' placeholder image in it, yes, the image does show up quite quickly there after (<1s), but it's the scrolling through all the records that is getting me.

                 

                It really doesn't seem like the thumbnails are getting cached.  When I run this layout using 15 (after closing/opening the file), it still takes it 13-15 seconds to reach the bottom.  But after I carefully scroll through the found set to ensure that all records have loaded the image, it only takes ~3s to scroll from top to bottom.  I get similar performance when using 14; but as soon as I close/open the file again in either version, I get all the 'Loading...' placeholders and it takes ~15 seconds again to scroll through things.

                 

                And using the 'permanent' thumbnail storage, I would assume that it wouldn't have to constantly re-size the images for this layout.  (That's why I removed padding and borders at one point - to ensure that the layout object sized would fully match the 'getthumbnail()' size I was using in the AE calc - but perhaps there is still some of that 'resizing' effect going on.)

                 

                And I realize that it still has to transfer the images - that's fine.  I don't really care if the images are present right off the bat anyway.  They can load up in that <1s window after the user stops scrolling.  What IS the problem here is that the UI is very laggy and hitching up when scrolling down the list.

                • 5. Re: Container thumbnail rendering issues
                  bigtom

                  Ok. 20s is a little slow for only 150 records.

                   

                  This really comes down to testing a few things to find the issue. Caching was improved in 15.

                   

                  I would do a test layout of a list view with only the images and nothing else. This will let you know if the speed is really with the images. I would also test this with a very narrow table. Maybe one with only the image thumb in it. Are the images you are using from the layout table or a related table?

                   

                  Test the upload speed of FMS and the download speed at the client. This is often overlooked. How far is the server from the client? Avg Ping time?

                   

                  Another test step would be to see how much a local UI file will help. Usually big gains with this, but adds complexity. Having another Server closer to the client with the UI makes a difference if you don't want to go local.

                  • 6. Re: Container thumbnail rendering issues
                    justinc

                    I was expecting more effect from the 15 caching improvements.  Maybe that's why I've been concentrating on these images so much - it doesn't seem that the images are being cached.

                     

                    If I close and re-open the file, go to the same layout with the same found set, it is still taking 10-12 seconds (in my improved version) to scroll through things.  And all the while the new 'loading...' symbol is displayed on all the images/containers as it loads them up.  But once you spend a careful amount of time scrolling through the entire list (i.e. letting it fully load everything), you can breeze through the list in 3 seconds with not lagginess, hitches, delays, or anything.  Which implies to me that the live-cache is doing something differently than the 'saved' cache, or whatever it is that 15 purports to improve on.

                     

                    And that was partly my hope with enabling thumbnails with permanent storage, too - that it wouldn't have to create/re-scale the images each time through.  The 'Loading...' could be simply due to transferring the image data, and not always caused by thumbnail creation.  But I did get improved performance when I made an AE calc container field that was the exact size of the field on the layout.  I still get the 'Loading...' symbol with this setup, though, which I would now guess is caused by actual data transfer/rendering as opposed to thumbnail creation.  But with storing the thumbnail I would expect that I could have 1 container field, and FileMaker would kind of take care of creating thumbnails for the different layouts that are each a slightly different size, and store them...somewhere.  (Since they are permanent, I would guess that they get tacked onto the record's storage somewhere.)  So instead of me explicitly having to create new AE calc fields for these different sizes, I thought FM would handle it.  But I seemed to get better performance after making my own fields.

                    • 7. Re: Container thumbnail rendering issues
                      bigtom

                      Yeah, you need to make all the thumbnail sizes yourself one way or another. If you have a 600x600 image and you display that in a 50x50 object on the layout it is just a big image in a small box at the full image file size.

                      • 8. Re: Container thumbnail rendering issues
                        justinc

                        I made a version of this layout, and did a couple of runs with various fields.  I even did some runs using the related field (i.e. the master 'Images' table that holds a 145x200 copy of the imported image)

                         

                        photos and name fields only5s
                        Name field only1.5s
                        Photo field only5s
                        RELATED table, Photo field only; crop-to-frame set6s
                        RELATED table, Photo field only; 'Reduce to fit'set5s

                         

                        The images added about 4 seconds to the process.  Interestingly, the related table didn't appear to add much overhead to the result.  (This timing is not very scientific - just my finger on a stopwatch.  And I only did one timed run of each.)

                         

                        If that's the case in general (related table not adding additional time), then I might just create the various sized fields in the related table, instead of spreading them around in the tables that the layouts are based in.  I do kind of like the idea of not showing any related data, though, if I can help it.  And the AE calc on these layout-base-table versions is simple and easy to maintain.

                        • 9. Re: Container thumbnail rendering issues
                          bigtom

                          Sounds like a slow network connection on one end or the other.

                          • 10. Re: Container thumbnail rendering issues
                            Stephen Huston

                            When you close the file on the client machine, that forces it to recache from the server on reopening that file. Caching within the file settings won't store it locally when the file is closed.

                            • 11. Re: Container thumbnail rendering issues
                              justinc

                              Stephen, yes, that appears to match what I was just discovering.  I did some more research and reading about the newer container system (I found a whitepaper on it: "In-Depth - Container Fields", from Susan Prosser).  This paper describes the 'Permanent' storage option for thumbnails as being stored permanently at the server.

                               

                              I was thinking/hoping that the thumbnails might be cached locally, so that they didn't have to get transferred again and again.  But it appears that container data will always get transferred from the server, regardless - at least until it becomes part of the live-cache.  Once a list of records has been thoroughly browsed so that all data for all records has been cached, the scrolling and loading performance is very good.  It's just that first run through it.

                               

                              Maybe the larger issue is that the layout-rendering process or thread (and thus I also mean the data-transfer process to execute that rendering) takes precedence over the scrolling of the layout.  So if I scroll-down by three pages in rapid succession, the system pauses/hitches/lags on pages 2 and 3, attempting to load the data and render those records as I scroll by them.  But...how does the system know?  It doesn't really - I think that it should perhaps a better 'abort' process where a set of requests for data can be aborted mid-stream more easily.  And that the rendering engine, while it now does load portal and container data asynchronously, could do a better job of interrupting that process if the UI moves on to another set of records.

                              • 12. Re: Container thumbnail rendering issues
                                Stephen Huston

                                The lag problem you are seeing is expected -- maybe not preferred, but expected -- behavior when scrolling a list view with container field images on the list layout. If you can use a list view layout without the image field on the layout for scrolling, the image field data won't try to cache during scrolling. That's really the only work-around I know for this issue so far.