There are several scenarios you can play with.
You could put a "cancel" button on your layout B that both deletes the new record and also returns them to Layout A.
You could use a scripted find or Go To Related records to pull up a found set on Layout B that consists of only the records linked to the current record on Layout A. Special handling would be needed if you use GTRR and this is the first such record.
You could also open a custom dialog with input fields to collect all the data for the new record. If the user clicks Ok, the script continues, creates the new record and returns the user to the original layout. If they click cancel, the dialog is closed and no new record is created.
You can also set up global fields on a layout---possibly shown in a new window and a save button creates the new record, populates the fields from the global fields and the ID of the current Layout A record and returns the user to the original layout. A Cancel button just clears the globals and returns the user to the original layout or window.
And now with FileMaker 13, I can see how a PopOver button might be used for such a purpose....
Thanks for the answers Phil. I really like the idea of opening a custom dialog BEFORE the record is created. At one point, I made the button on layout A just go to layout B and make the user click another button to create the new record. I abandoned that for some reason (can't recall, but I'm sure it was some issue with reverting and getting back to layout A). Since it's only one field that needs to be selected for this new record, I like the idea of doing all the prompting first, then creating (or aborting). I really want to have a drop down for the user to choose the field though, so I'm not sure that'll work.
The global fields on a layout thing seems doable too. Will require some more research on my part.
PopOver button? I AM on 13 now. I'll google that. :-)
THANKS for the ideas.