All values (means all the values from that field from all records) are displayed.
So, You can't remove a single value without removing that record...
But: You can have a separate table, just for the value list or a separate field that has a value under some conditions (i.e. if an item is active/valid)
With "use values from field" Value lists, you can delete the record that is the source of that value, clear the value from the field in that record or, if this is a conditional value list, you can change the match field value so record's values are no longer part of the listed values.