EvaluationError() evaluates an expression, i.e. a formula. The expression you are evaluating is the RESULT of evaluating R1. If you were on a record of Rick Whitelaw, the actual test performed would be:
EvaluationError( "WhitR" ) ≠ 504
and it would return true, because the error is 0.
In any case, I don't think a calculation can pre-test for field validation triggering. What you can do is test independently for the uniqueness of the proposed value: define a calculation field that returns the short name, and a self-join relationship matching this with the previously assigned short names - then see if there is a related record. Or use a script.
I had assumed the EvaluationError function would evaluate Left ( Last Name ;4 ) & Left ( First Name;1 ) as an expression and return 504 in the field's native context. Now I know it's not possible. Scripting it will be.
I had assumed the EvaluationError function would evaluate Left ( Last Name ;4 ) & Left ( First Name;1 ) as an expression
For this to happen, you would need to enter the expression as text, e.g.:
EvaluationError( "Left ( Last Name ; 4 ) & Left ( First Name ; 1 )" )
Of course, that wouldn't get you closer to your goal, because the expression itself contains no errors.