Voters 5
-
-
J
-
E
-
-
M
5
Collect data for exports multi-threaded
- Released
|
J |
mojorisin |
Collect the data for exports multi-threaded before writing it to disk.
Allow exports to follow the same thread setting as scans, or give them their own.
Estimated: V9.1
Activity Newest / Oldest
Team TreeSize
Status changed to: Released
Team TreeSize
Status changed to: In progress
Team TreeSize
Status changed to: Planned
mojorisin
Ok, sure, you can't write multithreaded to the disk. But from what I can tell, it seems that the export process basically has two steps. Correct me if I'm wrong:
1. Gather and process all exported information (generate the file)
2. Write the gathered information to disk (write file to disk)
I say this because it took me over an hour to export approx. 1.6 million items to an HTML file. The resulting file was around 1.3GB in size. Obviously, it does not normally take over an hour to write a file of that size, if all you're doing is writing the file (e.g. a file copy). Additionally, I monitored my CPU usage (per-thread) during this hour-long export process and noted that it was getting taxed heavily on a few threads (maybe 4?). So there must be something else going on with the export process that heavily uses the CPU, and that is what I would like to multithread, i.e. put it on 8 or 12 threads and increase the priority.
Team TreeSize
OK, thanks for your feedback. I re-phrased the request. Did you have activated any non default columns like MD5-hash or permissions?
mojorisin
No, nothing like that which would greatly increase the export time. It takes that long even with default columns. Sorry for late response.
Team TreeSize
Status changed to: Regretfully declined
Team TreeSize
Well, you cannot write multithreaded in a single file, as this is a kind of streams that an application is writing. Please contact our support if the export is too slow.