One way would be to put the field or fields that you update via replace field contents in a separate related record so that editing other fields still update the modification date but not the one that you update in this fashion.
There are also a number of methods offered for setting up an audit trail on a table such that each change to each field is individually tracked.
And please note that someone could "access" a record without actually modifying it. You might track all cases where a user simply makes the record the current record on their layout using a script performed by the OnRecordLoad trigger. An OnRecordCommit Trigger might be used to log when a record is modified via a script and then you can set the script up to do nothing when a global variable is set to a particular value. Then you can use a script to set that variable, replace field contents and then return the variable to its original value as a way to modify the records via a script without logging this as an "edit" of the individual records.
Thanks. Could you point me in the right direction to get me started on "methods offered for setting up an audit trail on a table such that each change to each field is individually tracked"?
Unless you are up for a fairly involved bit of scripting and setting up script triggers on every field, you may want to web search using a phrase such as "FileMaker Audit Trail" to see what is found. I believe there are several third party produced solutions out there that you can investigate.