Yes, in that your calc-based record level access is likely to behave as an unstored calc to resolve for each record. How much impact is hard to estimate except via trial.
Scripted restrictions take more work but may make a smaller hit.
Do you need to hide records fully, or just disable editing? Even hiding specific fields via Hide When in the Inspector may be less of a hit, once implemented, but, if many fields/layouts, longer to implement.
You may be able to mitigate the delays by keeping found sets small and not performing finds that rely on record level access to omit records.
In other words, specify the account name as part of the find criteria and don't rely on RLA to drop out records the user is not permitted to view. (I haven't tested this, so it's possible that you'll still get a delay depending on how FileMaker implements the RLA feature under the hood. I'm assuming that RLA is a constrain on the found set like constrain found set or an Omit type request. If I'm right, then keeping the found set produced by your find small should avoid such delays.)