Duplicates in which table?
I would guess that you have this structure in your CSV file from what you report:
Customer A Order Number 1 Item 1 data
Customer A Order Number 1 ITem 2 data
Customer A Order Number 1 Item 3 data
Customer B Order Number 2 Item 1 data
Customer B Order Number 2 Item 2 data
and so forth....
If so, when you import into your table of orders from this CSV data, you will get the Order and customer data imported into your table "for my customers information and order number", multiple times, one for each item ordered.
If that's what you have, you can set up a "unique values, validate always" validation rule on the Order Number field in this table (not the other) and import your records from the CSV. This will then filter out the duplicates during the import.
On the other hand:
The CSV file that I am importing from always contains the entire set of orders that have not shipped. I continue to over write the .csv file to capture any modifications to the order.
Suggests another issue. If you are importing the same data a second time, because it has not yet shipped, this will generate duplicated data in either table. You may be able to avoid this if you:
a) show all records in the target table
b) use the Update matching records in found set option to update existing records with the option to add new records if they don't match selected so that new rows of data produces new records.
But this assumes that a match field or combination of match fields can be specified to get this update type of import to work...
I have two individual files exportedCustomers exportedOrders that I am importing from. I do have match lines setup, and I forgot to mention that I do not ALWAYS get duplicate entries. Its just occassionally.
Your comment about showing all records is intriguing to me. Do I have to add a step to show all records in my import script?
Do I have to add a step to show all records in my import script?
yes. Otherwise, there's a chance of getting duplicated data like you describe should the matching record you want to update not happen to be in the current found set of records in the specified target table occurrence.