not automatically, as you're editing... I'd imagine this could be dangerous. What if you are removing the last record in order to add another one?
The best I could imagine is to have a calc field on the parent that checks if there are related records, then runa script that searches for these records, delete (or better still, archive) the records that don't have children.
You could schedule the script run run on server, if you really wanted.
Its not an dangerous as you might think. Its very common in other programming to enable garbage collection.
In the case you are stating as an example you would definitely need to add a record before removing the last one. The whole idea is the parent record is irrelevant when it doesn't have any child records. Its really more of a backwards way of looking at things because the "Parent" name makes you assume that is the master record or the record you would be working from. All this "Parent" table would be doing is grouping records together with some additional data. When there is no items in the group it doesn't need to exist.
Anyway its more a memory management issue so a script would probably be fine although automatic would be much cleaner.