Thank you for your post.
Keep in mind that Table view allows you to view multiple records at one time. If you scroll to an area where the Container field is present and/or the contents have not yet been displayed, then the contents of that Container field load. As a possible workaround, you may want to consider a second Layout that displays the fields without the Container fields. Then, in Table View, you can scroll through the records without any delay. You can even set up a button on a field so that when clicked, it switches back to Form View on the original layout (displaying the Container fields).