That would depend on how you design your database.
Have you used Manage | Security to define accounts, passwords to limit their level of access to the database?
Do any of the layouts they are permitted to access also provide access to your referenced table such that they can change data in that table?
Yes I set permissions. The referenced table is accessed by certain depts, but my concern is not here. My concern is: can using a value list with the option to add values turned on, ruin the referenced table or add records to the referenced table?
What kind of value list are we discussing? Custom Values or values drawn from a table?
If custom values, then enabling the option to edit the value list gives the user complete access to the value list, they can edit or delete any value in the list, but then they have no access via this option to any table. The only ability to affect a table is determined by any aspect of your design that permits edit level access to that table and the only damage that they can do is to the data stored in that table.
If values from a table, then the option to edit values is not available--and again you are looking at what aspects of your design permit the user to modify data in a table.