Thank you for your post, and I apologize for the late reply.
Limiting the number of records is good, but how much data is being displayed for each record? If each record has 500 fields with lots of text, then it will be definitely slow things down.
It is recommended FileMaker Server run on a dedicated server. If not, this could affect performance.
Other performance considerations is the network and how it is set up. Any information on the network setup may be helpful.