1 Reply Latest reply on Nov 20, 2012 10:43 AM by dreed

    Indexing Help on a Large File

    dreed

      Hello,

       

      I have a file with about 1.2 million records and about 800 GB of external container data. I've been going through the process of using the Scribe plugin to read text contents of the supported file types (pdf, office docs, etc) into a text field in this table for searching. There will be about 250,000 records in the table that will contain data in this text field and currently my script has processed ~80,0000 of them. I stopped the processing of the script to check on some errors the plugin reported (older, unsupported office doc types), and while I was doing this I decided to turn on indexing for this field, since it will be necessary for searching the text contents.

       

      How long do you think it might take to build this index, considering that some of these fields may contain pages of text? many of the fields are empty or contain only a few lines of text, but since I turned on the index and commited the change from Manage Database, Filemaker (12v3) has been unresponsive, just giving the indeterminate progress bar. It's been >12 hours at this point.

        • 1. Re: Indexing Help on a Large File
          dreed

          36 hours.... I get a progress bar with records remaining occasionally, before going back to unresponsive.  At this rate it might be done by next Monday... wish it was faster, but I guess as long as it gets done I'll be happy.

           

          Perhaps I should have used ScribeFileAsKeywords() instead of ScribeFileAsText() and then just done a value index instead of full word index.