Make a script which count the user in the table and compare with the number of user that you want to limit
Or give him the only other password to your file and set the privilege set to allow modifying for both scripts and layouts. Not sure if this will allow creating a new layout. That's a lot more access than the typical user should have unless they are a developer--in which case you usually give them full access to the file.
The right answer is to have a startup script like this.
If you are having more than one solution on your server you can have a central file with a record for each solution with a field stating the max number of users, and then check up against this. But it is fine just to build it into each individual file.
In this case I have chosen not to count users with full access, thus not throwing the developer out:-)
A good reference: https://fmhelp.filemaker.com/docs/14/en/fmp14_functions_ref.pdf