Please note that you have posted to the part of the forum used to report possible software bugs. If you post questions like this in the FM Pro Forum (See tab at top of this screen), it will be seen and responded to by more people and more quickly.
What you describe is indeed possible, but you do not describe the design of your database so I am unable to go into much detail on how to do it.
Create a layout just for this report.
Remove the body layout part and use part setup to add a sub summary part. Choose a field for the "when sorted by" setting that will group your records by customer.
Put the customer name field and a summary field that computes the "total of" your invoice amount field in this sub summary part.
Perform a find for just unpaid invoices and sort them by the same field you specified as the "when sorted by" field for your sub summary part and you'll get a list of customers with upaid invoices and the amounts that they owe.
This is the basic idea, you may need to adjust this if it is possible for a customer to paritally pay off an invoice.