Yep and that can be annoying. Preview assumes that you will print the report using the Records Being Browsed option. You can click the book control to see other pages in the previewed version of the report. If you have one record per page, you can also use a script that captures the current record number (get ( RecordNumber ) and then uses Go To Record/Request/Page to go to that page after entering preview mode.
If you only want to preview the current record, you can also isolate the current record in a found set of just that record and then Preview only has the one record to preview.