PS, going into find mode and typing the same search into the correct field produces instant results
You tell QuickFind which fields to search in Layout mode. By default, QuickFind searches all the fields on your layout, including the unstored calculation fields, which is what's taking forever.
Any field with a yellow magnifying glass in the corner is going to take a while, green is good. You can select the fields you don't want QuickFind to search and turn off the option in the Inspector-->Data tab.
I started off by ignoring all of the fields for the field service calculator. Success. Now I have that tool in the pocket should I run into this agian.