Product and version: FileMaker Pro Advanced 16 (16.0v5) and 17 (17.0v1). Not tested on other versions.
OS and version: macOS 10.12.6. Not tested on Windows or other macOS versions.
Hardware: MacBook Pro Retina 13
Description: When viewing the Styles tab in the Inspector and the inspector window is not tall enough to show all the available styles, it won't scroll using the mouse or trackpad when the cursor is over the body of the inspector.
With the other 3 tabs in the Inspector, you can scroll normally when the cursor is anywhere over the inspector window.
How to Replicate: Go into layout mode and go to a layout that is based on a Style that has lots of text options – such as Enlightened. Switch the Inspector to the Styles tab and make the height of it small'ish (a couple of inches). If you have the Inspector docked with FMPA 17, just make the whole window smaller in height. Select a text object in the layout. See screenshot. With the cursor hovering in the main part of the window (over the various text styles), you can't scroll down to see the additional styles. You CAN scroll if you hover the cursor over the small scroll bar area on the right side of the inspector window. If you switch to any of the other 3 inspector tab's, they will scroll with just the cursor anywhere over the main part of the window.
Workaround: Place cursor right over the right edge of the inspector window and you can scroll as long as the cursor remains in this narrow area.
Notes: I tried changing the settings in the OS (System Preferences > General > Show Scroll Bars) to each of the 3 options (Automatically, When Scrolling, Always). The behavior in the Styles tab of the Inspector is the same.