You would create a third field to represent the address you want displayed:
Then use address_with_default in your report.
When you create address_with_default be sure to select "calculation returns TEXT" - I think it defaults to NUMBER. For this calculation you will use the If statement to control your logic. You can use the IsEmpty function to determine if address is present or not.
Brilliant thanks Travis, I'll give that a try now.