I would certainly agree. You put edit fields on a list view allowing for over 2k records? Think of how much HTML and longpoling that's processing!
Simple alternative: turn your rows into merge fields to only display the list view with text data. Then take the user to a form view or even a popover to edit a row. Much higher performance.
You could also setup a pagination system to handle large lists. It's a technique that heavy portal users have been doing for years that I've been fooling around with.
But yes, the less records/fields/data you have loaded, the faster WD will perform.
Nothing is editable on this layout thankfully. For selection purposes only. The end users don't seem to like using the QuickFind to make the found set manageable. They asked for a list and against my recommendations, they got a list. WD wasn't in the original plan and the system has grown way past its original scope. The portal pagination is what I've recommended to the client. Its just a matter of whether they're willing to pay for it.
but are you using merge text or fields to show the data? The former is the faster one. I would strip out any custom styling you have in the body object, use whatever the base of the theme is with merge text only. That will at least optimize it as far as it can go until you get pagination in place.