AnsweredAssumed Answered

Fastest Method to Export CSV

Question asked by SethRedlus on Oct 9, 2017
Latest reply on Oct 10, 2017 by bigtom

Howdy All-


Hoping for some opinions and insight here. I have custom scripted an export of my data to CSV to import it into excel and wanted to get thoughts on the fastest method - and to see what folks think a realistic export time should be.


My file is a little strange - I have a vertical table of answers to a survey. The average survey is 100 questions, so there are 100 records added to the "Answers" table for reach response. That means, for about 300 people who take the survey, my "Answers" table has 300x100=30000 records. I have built an export layout where each survey has a portal that shows all 100 answers for that survey and sorts them according to question number. I then loop through all the surveys to get access to all the data. Part of the benefit of this is that I can easily customize the order that the fields are exported in.


Because I chose a CSV export (so I wouldn't have to buy an excel import/export plugin), I have constructed the export as a script that uses the BaseElements WritetoFile function. It loops through each response and adds it to a variable. At the end of each record, it appends the variable to the file.


I exported about 100 records and it took quite a while - I'd say a good 3-5 minutes coming from a remote filmmaker server (all via gigabit connection).


Any tips and tricks to make it faster? Happy to include scripts or samples if anyone thinks there is something I can clear up. It's worth mentioning that if someone thinks an excel plugin would be faster, then I'd gladly go that route - just wouldn't want to do it and end up in the same place.


Thanks all!