Your conditional relationship must be filtered to only include the record you want to judge upon.
Note that this might mean that your portal comes from one relationship and your conditional format may come through another.
You can test run this by working with your portal relationship...it should only show the record of interest. Once you have that relationship worked out, reset your portal back to where it was, then define your conditional relationship.
If you can post the pertinent section of your relationship graph, the answers you get may be more specific.
Thanks Ninja. I've tidied up my relationships graph. It was a huge mess. Here it is: