I think I understand what you're trying to do. I created a quick demo that it would help.
The key is to:
1. Find your data
2. Sort it by the hour
and then set up the chart to be based on the found set, using the hour field, and then set the display to be "Summarized groups of records".
Study my chart in the set up of the object. Study the "Chart" and "Data Source" panels.
Charts are tricky things to understand. But once you do, the concept sticks and you can set up a chart with relative ease.
Here's a great resource that will help you out.
I wasn't sorting the records after I entered the layout; i did it before. Thanks for all your help.