1 2 Previous Next 19 Replies Latest reply on Apr 23, 2013 11:16 AM by Paul Jansen

    Auditing the Audit Log (or Quis custodiet ipsos custodes)

    DrewTenenholz

      All --

       

      For security reasons, I have an audit trail that records every time a record is created, viewed, edited, or deleted.

       

      For hyper-security reasons, I need to be able to prove that the audit trail has not been tampered with.

       

      Access privileges are already set on the audit trail file (which is a separate file), so that is not the issue. I want to find a way to (easily) show whether a specific audit trail record has been altered, or worse yet, removed. What are your ideas?

       

      -- Drew Tenenholz

       

      I've tried recording a count of the number of audit trail records for each 'parent' record in yet another location within the system. The problem is that this is slowing down normal operations of the system - it is expected that there will be hundreds to thousands of audit trail records per 'parent'. This is because even viewing the 'parent' is a recordable event.

       

      It is necessary to NOT use Server-Side scripting, this is a solution that is sold to customers who will not have the technical ability to create and use SSS. So, the challenge is to stay within FMPro.

        • 1. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
          mbraendle

          What comes quickly to my mind:

          - checking against a separate checksum

          - rebuilding the database according to the audit log and comparing this file to the original file (using relationships). This, however, doesn't cover the views.

          • 2. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
            steve_ssh

            Hello Drew,

             

            Is the use of a plugin allowed in a potential solution?

             

            I realize that you said "stay within FMPro", but, given the context, I'm uncertain as to whether that means "no server-side scripting", or whether that means "nothing external at all".

             

            Kind regards,

             

            -steve

            • 3. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
              DrewTenenholz

              Martin --

               

              Your answer is helpful, but I'm still missing a piece.  Using a checksum as part of the audit trail record itself solves the issue of whether or not the record has been tampered with.

               

              I'm also looking for a solution to indicate whether there is a gap in the audit trail records, indicating that an audit trail record has been deleted.  Anything that counts the number of records created (either all audit trail records or audit trail records for a specific parent record) seems to put a lot of drag on the system as this value needs to be calculated with each new audit trail record, and I make a large number of them.

               

              Unless someone has a better idea, we've settled on a method whereby every new audit trail record stores the checksum of it's predecessor in addition to it's own checksum.  This way, any deletions in the audit trail will leave an identifiable gap in the checksum sequences that would be very difficult to recreate, even if you knew how the system worked and somehow had the ability to interleave a new (false) audit trail record.

               

              --  Drew Tenenholz

               

              P.S.  the -- is used in place of the more proper comma.  I hope it doesn't mean I'm rating your reaponse poorly or something.  I use mail for the list, and wouldn't know if this has an effet on the web interface for messages.

              • 4. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                DrewTenenholz

                Steve --

                 

                Plugins are welcome here.  We are already using: 360Works ScriptMaster, a host of Troi plugins, and a few others.

                 

                -- Drew

                • 5. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                  mbraendle

                  Does the double minus mean that my answer was not helpful? ;-)

                  • 6. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                    DrewTenenholz

                    Martin --

                     

                    Your answer is helpful, but I'm still missing a piece.  Using a checksum as part of the audit trail record itself solves the issue of whether or not the record has been tampered with.

                     

                    I'm also looking for a solution to indicate whether there is a gap in the audit trail records, indicating that an audit trail record has been deleted.  Anything that counts the number of records created (either all audit trail records or audit trail records for a specific parent record) seems to put a lot of drag on the system as this value needs to be calculated with each new audit trail record, and I make a large number of them.

                     

                    Unless someone has a better idea, we've settled on a method whereby every new audit trail record stores the checksum of it's predecessor in addition to it's own checksum.  This way, any deletions in the audit trail will leave an identifiable gap in the checksum sequences that would be very difficult to recreate, even if you knew how the system worked and somehow had the ability to interleave a new (false) audit trail record.

                     

                    --  Drew Tenenholz

                     

                    P.S.  the -- is used in place of the more proper comma.  I hope it doesn't mean I'm rating your reaponse poorly or something.  I use mail for the list, and wouldn't know if this has an effet on the web interface for messages.

                    • 7. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                      steve_ssh

                      Hi Drew,

                       

                      I've been thinking about this problem throughout the day, and was pondering possible solutions similar to what you are doing, i.e. each new audit record contains a reference to its predecessor (hashed/checksummed, etc.).

                       

                      I thought this a good avenue to pursue, but I did not come up with any satisfying means to cover the hack scenario where the attacker deletes a number of records off the end, thus removing a chunk of records, but leaving no gaps.  The undeleted records would presumably still validate as ok.  As I see it, newly added (valid) records would only detect a gap if there were some additional factor to consider which allowed detection of a sequence break at the time of creating the new record.

                       

                      Were you able to cover this scenario as well, or is there a flaw to my thinking about this?

                       

                      Just curious & best regards,

                       

                      -steve

                      • 8. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                        mbraendle

                        Drew,

                         

                        I was rather thinking of using an outside software that calculates the checksum of the whole file. However, that means touching the .fp7 or .fmp12 file (which means it should best be closed in this moment, e.g. by acting on a backup copy).

                        • 9. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                          RalphLearmont

                          DrewTenenholz wrote:

                           

                          All --

                           

                          For security reasons, I have an audit trail that records every time a record is created, viewed, edited, or deleted.

                           

                          For hyper-security reasons, I need to be able to prove that the audit trail has not been tampered with. 

                           

                          Access privileges are already set on the audit trail file (which is a separate file), so that is not the issue.  I want to find a way to (easily) show whether a specific audit trail record has been altered, or worse yet, removed. 

                           

                           

                          On the assumption that your Audit trail file is also a filemaker file, my thought is to audit the Audit file.  That means you have the auditing software being applied twice.

                           

                          In the second case, the "second" audit file will reveal any tampering with the first.  If there has been no tampering, the second file will show "New" records as being the only "event".  That is, by demonstrating that no modifications, or deletions etc. have been made to the first Audit file, that ought to provide enough proof.   (Unless of course there are doubts over the authenticity of your second Audit file)!   At some point, there has to be some measure of trust..

                          • 10. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                            DrewTenenholz

                            Martin --

                             

                            At 2:54 AM -0700 4/20/13, MartinBraendle wrote:

                            I was rather thinking of using an outside software that calculates the checksum of the whole file. However, that means touching the .fp7 or .fmp12 file (which means it should best be closed in this moment, e.g. by acting on a backup copy).

                            This is also an interesting idea.  As you indicate, a little bit tricky to accomplish from within the solution.

                             

                            The requirement is not specific about implementation, but it does require a user of the system (possibly only an admin-level access user) to be able to perform the check without having to invoke knowledge of computers outside of the solution....

                             

                            -- Drew Tenenholz

                            • 12. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                              DrewTenenholz

                              Ralph --

                               

                              Thanks for the suggestion.  As a developer, I'm OK with doing it that way.  However, this is both a practical exercise and an abstract one.  Since I'm being asked for a way to prove that the audit trail has not been tampered with, creating an identically programmed, yet parallel audit trail is insufficient since anyone who can defeat the first audit trail can do the same on the second.  Basically, I can set up audit trails ad infinitum yet this will not show, in an abstract way, that we can detect tampering with the data.

                               

                              -- Drew Tenenholz

                               

                               

                              >On the assumption that your Audit trail file is also a filemaker file, my thought is to audit the Audit file.  That means you have the auditing software being applied twice.

                              >In the second case, the "second" audit file will reveal any tampering with the first.  If there has been no tampering, the second file will show "New" records as being the only "event".  That is, by demonstrating that no modifications, or deletions etc. have been made to the first Audit file, that ought to provide enough proof.   (Unless of course there are doubts over the authenticity of your second Audit file)!   At some point, there has to be some measure of trust..

                              • 13. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                                Malcolm

                                Checksums as signatures on the audit trail?  If checksums were posted to another database the discovery of tampering could be as simple as "stored checksum <> live checksum".

                                 

                                Malcolm

                                • 14. Re: Auditing the Audit Log (or Quis custodiet ipsos custodes)
                                  DrewTenenholz

                                  Steve --

                                   

                                  At 6:52 PM -0700 4/19/13, steve_ssh wrote:

                                  I thought this a good avenue to pursue, but I did not come up with any satisfying means to cover the hack scenario where the attacker deletes a number of records off the end, thus removing a chunk of records, but leaving no gaps.  The undeleted records would presumably still validate as ok.

                                  Yes, that same scenario occurred to me the next morning, before reading your response.  And, I don't think I've protected against that situation.

                                   

                                  In practice, this is an electronic medical records solution where we are trying to protect against the situation where a practitioner attempts to alter a patient record (presumably after something like an adverse event) to alter information that was previously there (such as a note saying the patient was warned about a potential problem when they we not actually informed).  Since the mere act of reading the older patient note creates a link in the chain, it becomes harder (from a practical point of view), to delete all traces of all future activity on a record when what you are trying to do is add something to the past.

                                   

                                  If the user were trying to remove all trace of having ever accessed (and thus altered) the record, it might be possible to delete the audit trail and deny that you ever spoke to the patient, but then there are independent means to contradict that like co-pay receipts, calendar entries, and elements outside my system.

                                   

                                  Also, the audit trail is on the patient record, with a new audit record created when ANY user accesses the record (e.g. when the patient leaves the exam room and makes a follow-up appointment at the front desk two minutes later).  This also makes it harder in practical terms to alter the patient record without the audit trail indicating something is awry.

                                   

                                  Maybe the most frustrating thing about this question is that there is no clear guidance in the requirements (these are U.S. federal requirements) for what  exactly is 'good enough'.  They say you need to prove the audit trail has not been tampered with and leave it to the developer to figure out both what that means and how to implement it.  I was hoping someone with some experience (maybe from banking, police work, etc.) could help explain what they had done to satisfy a similar requirement.

                                   

                                  -- Drew Tenenholz

                                  1 2 Previous Next