I'm also using Web Clips created with the Apple Configurator, but when I launch mine it says "Back to Safari" not "Back to <name of web clip>". As far as I know there's no way to turn this off - even hiding the top toolbar doesn't hide it.
FYI here's the steps I take to create the Web Clip which I documented for my own future reference on my blog:
Glad to hear this behavior is confirmed, thank you for your feedback. In Configurator web clip payload settings, if you tick the "start full screen" checkbox it will do as I say. Unticking it will initially launch from Safari with your noted effect.
I am pretty sure this is an iOS 9 behavior and nothing to do with FM. If there isn't a workaround I will probably remove the web clip, because if you accidentally tap "back" you'll reload your solution, I believe forcibly without a dialog (though I need to confirm this).
The "Back to <name of application>" was a new feature of iOS 9 - it happens in all other applications, such as when clicking a link in an email etc. I use it all the time and it's one of my favourite features of iOS 9 - I couldn't see a preference to enable/disable this in the Settings app however.