Ok I'm not sure how, or what made a difference, but it seems to be working now! It may have been the second import I have in that script and I added a second Show All Records between the Import Records steps. I'm not sure if that made a difference or not, if I find out for sure I'll post here just in case anyone has a similar problem.
and I added a second Show All Records between the Import Records steps
That's the key difference. Import Matching matches values against all the records in the target table's current found set, not against all records in the table.