This should not be annoying at all. Changing views *is* making a change to the layout since it determines how the layout will be displayed (default display) when back at Browse mode.
When you are developing, you may want to view it in table mode while working but for the design, have it always display in form mode as the default. If FM assumes you want to change it and doesn't ask, it will come up for Users as table mode. If FM assumes you DON'T want it to change, you can't make design changes. No, you can't stop FM from asking and it's a good thing. :^)
I disagree that this is a good thing. If you are trying to get FM to work like an application, you shouldn't be prompted if all you are doing is changing the view. Ideally, I want the table view to be the default, but as read only. If a user wants to see the details or enter a new item, I want them to use the form. This is how most applications work. Any way to get FM to do this?
The work around is to set up different layouts for each view and lock them in layout setup.. so that they cannot be changed. Then add buttons to select the view by changing layouts. (You can make the header visible for table view in layout setup so that you have a place for such buttons.)
(Table view isn't the greatest layout view to make available to users. It has options that can really confuse the unwary or uninformed when data seems to disappear from view on them due to an option they selected. I almost always set up a list view layout where I can have better control over what options the users have when working with the layout.)