You could use either the mac pro or a windows 2012 server... Either OS will work fine.. We develop on Mac's and have about half windows workstations.. We went with a windows 2012 server due to the fact we could not rack mount a mac.. (that is the only reason)
But I will say the windows server has been really stable and working great.. Either a 4way or 6 way would do. If you might want to implement some of the new web direct features then go 6way.
I would not run any server on a desktop product like windows 8. It is meant for the desktop not server.
Stop the RAID madness. It will be slower than the built-in Mac Pro SSD.
You don't need RAID. You need backup (which is not RAID). The only kind of RAID you may need is RAID 1, to be able to reboot the server quickly. But it will slow you down cause you won't be able to use built-in ssd.
With a mac, and carbon copy cloner you do a clone each night on another bootable drive + backup of database elsewhere. If something goes wrong on built-in disk you reboot on the clone, and put back the backuped data. and bam you'r back in business in 10 min max.
Also, Filemaker Server don't use more than 4 cores at most, if you don't use web direct (web direct can use more cores). You need to have the highest Ghz possible because the useful task in filemaker use at most 100% of CPU. So privilege clock frequency. The best would be high Ghz hasswell processors (not available in mac pro, the xeon are inferior). So you have more choice in PC world (unless you use an mackintosh which is what I do, and its rock solid, even over clocked )
Finally, I prefer the Mac environment it's easier to maintain and offers great automation thanks to unix command line and applescript