2 Replies Latest reply on Jul 19, 2012 11:03 PM by cortical

    export count error - semi-silent

    cortical

      FM11Adv on Mas OSX (10.7.4 and 10.6.8)

       

      The file set is a data separation set and the issue is with the data file. A copy of the master set was loaded to a server ( FMS 11Adv on OSX) for testing the data in a new version, some data edits made , and a schedule used to save a backup. File data_source originates from the backup.

       

      The data_source indicates 2089 records in the Projects table, but an import or export results in 2100 records. 11 invisible records are appearing in the output that are not indicated ( except for one instance) in the source.

      No recover log or error is reported at any stage, back on the dev box importing the updated data into a clone of the dev master.

       

       

      Importing data from a working file into a master

       

       

      test 1

      The source file is named source_data.fp7, and contains 2089 records as reported in the status bar and Manage database table details (image a). NOTE that the current record is 2100 of 2089 total.

       

      The destination file is a clone of the master data file (image a)

       

      An import results in 2100 records in the destination. ( image b)

       

      test 2

      A clone of the cloned master file is made and named target.fp7

      The import is repeated and although the source status slider indicates 2089 total, the import result is 2100 ( image c)

       

      test 3

      a simple message get found count on the source_data

      image d

       

      test 4

      an export from source_date results in 2100 record in the export file

       

       

       

      So 11 records in the source file which the source file exports, but itself does not recognize

       

       

      test 5 - save a copy of the source_data using copy_of current file

      in the file 2089 records; result 2100 records

       

      test 6

      save a compacted copy of source_data, result 2089 records

      the file size is noticeably smaller 51.6 MB cf 67.4 MB for the save compacted

       

      test 7

      export from the compacted copy; result 2100 records

       

      test 8

      repeat save compacted copy of source_data; result 2089 records

       

      test 9

      export from test 8 ; result 2100 records

       

      test 10

      quit FM, archive the source data, transfer the archive to a different computer, unzip

      source = 2089

      export = 2100

       

      At this point there have been no recover logs or error messages at any process.

       

      test 11

      consistency check on the source_data file; 0 bad blocks

       

      So the error would clearly appear to be in the source file but persists despite the variations.

       

      The records are identifiable in the export as nulls ((image nulls), and can be deleted in the intermediate export, and the remainder presumptively valid records imported into the target.

      This all make one rather uneasy. A somewhat occult corruption.

        • 1. Re: export count error - semi-silent

          This appears to be an index problem, although usually the offending records will have ? in them. Just last night I converted a file to v12 and read through the import log and found several fields in one table that had a reindex error, and if I'm correct in my assumption this is one method of finding the problem fields.

           

          If the number of indexed fields (include calculations) is small, try reindexing all of them. If large, maybe try converting to v12 and look at the Conversion Log and see if you find anything like the following (without a zero at the end of the line)

          Rebuilt value index for field 'Create TimeStamp'; 3257 item(s) now exist, difference from old index is -1

          • 2. Re: export count error - semi-silent
            cortical

            It does appear to be an indexing issue.

             

            FM 12 conversion resulted in 2089 records, which export as 2089

             

            The conversion log differneces from old are all 0

             

            Turned oof all indexing i the problem table in teh source fp7 file, closed, quit, re-opened and exported 2089

            Imported from eth source file into the destination table ( a copy) and 2089.

             

            So issue corrected.  But I still feel very uneasy about it.

             

            Thanks