I think part of the issue was that the contacts relationship was missing the value for "Include only related values starting from" I filled in what I think is the correct value. Then again this may have been a function of the download.
So after fixing this it still seemed to be a problem. I had opened the interface file with out explicitly opening the data file. I then opened the data file and found that as long as I clicked out side of the field and then click in the Contact field I get the correct list.
So I added an Onvalidate script trigger that committs the record. This seems to make it come up with the correct list all of the time.
Thanks Bruce, that does seem to fix things in this test file.
I will try that out on the bigger solutions and see if I can better results.