You'll need to do one of the following:
1) Remove the records with duplicated values from your table
2) Modify the relationship to your portal in some way so that the dulicates are not related records
3) modify the portal filter to exclude the duplicates
With 3), you use a one row portal with the same filter and other properties but only one row and place your summary field from the portal's table inside that portal row to display the appropriate total, count or other aggregate value. But this is only a "displayed" value, you can't use it in a calculation.