While FileMaker Support recommended against installing anything else on the same server as FileMaker Server, in some users circumstances it cannot be avoided.
What follows is the culmination of investigation and resolutions to achieve a solid performing server sharing by both FileMaker Server 11 and Lasso Professional 8.6.
At the crux of the issue was the treatment of the 32-bit vs 64-bit connector. Lasso 8.6 is built with both 32 and 64-bit connectors and when installed alone ran without issues.
When FileMaker Server 11 was installed alone, it too ran without issues using the native 64-bit connector for IIS 7 in 32-bit compatibilty mode.
Put the two together, and the fun starts.
That said, the end solution using a customized installation process for Lasso Professional 8.6 and reconfiguring IIS is now clearly documented in a tech article on the LassoSoft site for any who need to resolve this in future and successfully share their server with the two technologies.