Your calculation field may be all that you need here.
You have two issues: the length of the line--which can be controlled by a calculation or conditional format and the amount of space taken up by each sub summary part.
Simply using a calculation or conditional format to control the length of the line will still leave you with extra space between your listed records when date and product ID change so you really need a single sub summary part here that will appear if the date or the product ID changes.
If your horzontal line is a text object you can use conditional formatting to change the font size to 120+ to make that text object invisible so you can layer two such objects on top of each other and use conditional formatting to control which is visible.
If your horizontal line is a graphic, you can place graphics in two global container fields and use a calculation field with "container" specified as the result type to control which is visible in each sub summary layout part instance.
Thanks for the ideas. I hadn't considered using container fields nor am I familiar with them. Let me do a little exploring and I'll report back. You are right that this would work best with a single subsummary part. I could unconditionally draw the short line at control breaks and then just turn on or off an additional segment if there was a break on date.