Thank you for the post.
FileMaker Server 11 and FileMaker Server 12 could be installed to the same machine; however, page 11 of the FileMaker Server 11 Getting Started Guide and page 9 of the FileMaker Server 12 Getting Started Guide state, "You cannot run two different versions of FileMaker Server on the same machine at the same time."
Both programs use the same ports for the database server.
"For best performance, run FileMaker Server on a dedicated machine reserved for use as a database server." See the FileMaker Server Getting Started Guides linked above for additional information about setup and configuration.
I put both on the same machine by having one running in a virtual machine with its own virtual IP. In my case it's VMWare Fusion and Windows Server 2008 on top of OS X. Works like a charm.