Those stats look fine but that screenshot was not taken with the system under load. When the users complain about the coffee cups, look at the Client stats part of that window.
You describe only 2 of the 4 typical bottlenecs (CPU and memory). Disk I/O is more important than that and there is also the network throughput. What other activity is going on, on the server? Any virus scanning, shadow copies?
I would suggest you set up some performance monitoring to include the 4 typical bottlenecks and all FMS counters to see what happens on the machine during the slowdowns. You may not see anything if the problem is more downstream from the server with a bad cable or failing switch.