I suggest posting a copy of your script(s) involved. From your description, I'm unsure if you have two windows open or three. You can copy and paste a script from a Filemaker advanced Database Design Report or by printing the script as a PDF and copying the text from the PDF file.
The scripts are quite simple. In playing around with it this morning, I also discovered something that I had not noticed before.
The new record layout, as I mentioned, is set to not automatically commit entries. I wanted it that way so the user could fill in the required and optional fields and change them, if needed. Committing the record is by a button, and that all seems to work fine. The record cannot be committed until all required fields are entered.
What I want is that the user can at any time go to the help screen for guidance on how to complete the fields and then return back to the layout keeping all the entries as they were but not yet committed.
On the new record screen there is a Help button with a simple script:
New Window 
Go to Layout ["Help [New Activity Record]" (Admin)]
(All Help screens are hosted in a general Admin table, the new record layout is off the Activity table.)
Clicking the Help button works fine -- the Help screen pops up (full screen). On the Help screen is a Return button, and that is scripted as:
Close Window [Current Window]
Now, what I noticed is that when the Help screen is open, there are only two windows available, the original new record layout and the Help layout. However, when I close the Help window via the Return button, a NEW window pops up that is identical to the original new record window. If the user then tries to enter/modify data in the new window, a flash screen pops up that entries cannot be modified as the "record" is already open in another window. I have looked, and the original new record layout has not created a new record. Closing the second (duplicate) window lets the user continue as I had wanted.
I somehow think that the second window is caused by the fact that the record in the original window has not yet been committed, but cannot figure out how or why.
Without being able to directly examine and test your file, I can't be sure, but suspect something else in your database is performing a script with a new window script step in it. If you have a copy of filemaker advanced, I'd enable the debugger and test this process--watching for a script to kick in that creates the third window.
I'm speculating here that you might have a layout based script trigger that is firing off a script that contains a New Window step or a Go To Related Record (with new window option specified) step that is creating your extra window.