clicking the layout background commits records opened for editing and this commit is needed before the summary fields can update.
Your delete portal row button can include a commit records step to make this automatic. OnObjectSave Script triggers on the fields that you might edit to change a summary total can also commit records--but this gets a bit trickier as the commit also removes the focus from the portal and if you are editing a portal row at the bottom of the portal, the portal scroll thumb snaps back to the top when the records in the portal are committed and this can scroll your portal row out of view. Sometimes you have to script the process to save the active portal row number in a variable so that your script can commit records, then use go to portal row with the variable to put the focus back into the correct portal row.
Or you can just keep clicking the layout back ground...