You can add your own custom validation error message to use in place of the standard message and you can disallow letting the user override the warning which limits the validation error dialog to returning to the field to correct the error or reverting the record which gets you pretty close.
If you want total control of this yourself, add a self join to Manage database Relationships that matches on your calculation field:
JoinTable::CalcField = JoinTable2::CalcField.
Then you can use the OnRecordCommit script trigger or possibly an OnObjectExit trigger to check JoinTable2 for related records. If there's more than one you have a duplicate.