What version / platform do you use? I got correct result with WindowNames ( "filename" ) on Win7 / FM15.
If you failed to "close", you might missed "current file only" check box in Close Windows script step. If you failed to "get names", it can be bug.
This worked for me if the file is "standalone". I think if the file is referenced from other file, you need to close rerereing file first, so there may be no way to full automate. I close application.
Exit Loop If [ Let ( $windows = WindowNames ( "filename" ) ; $windows = "" ) ]
Close Window [ Name: GetValue ( $windows ; 1 ) ]