Would a Runtime Solution work for you?
Solution 1: Split your solution into two files. Put all your data table in File 1. Put all layouts, scripts, value lists etc. in File 2 and use external data source references to link to the tables in File 1. Now, most updates can be deployed simply by swapping out File 2 while the data in File 1 remains unchanged.
Solution 2: Write scripts that use Import records to copy the data from the original file into the new version. Such a script can ask the user to find and open the original file--establishing a known file path to the file and then import from every table in the original file into the corresponding file in the new copy. Set Next Serial value can be used to update serial number settings. The contents of global fields will import into the new copy just like any other data field.
Thanks for the ideas. Hadn't thought of splitting the data from the layouts, etc.