UploadMultiple performance is affected by contents of the destination folder

Question

In my previous tests UploadMultiple worked really well but now that I am using it in a production environment I got reports of very slow transfers (much slower than uploading each file individually without concurrency).

I am making some tests and the behavior is really easy to reproduce and seems to be related to the amount of files located in the destination folder. I am uploading a 7 MB folder containing only 3 files using UploadMultiple with concurrency enabled and these are the average upload times:

FTP Server 1

- Empty destination folder: ~ 00:00:04

- Destination folder with 45 MB (516 files): ~ 00:01:05

- Destination folder with 700 MB (1780 files): ~ 00:01:40

FTP Server 2

- Empty destination folder: ~ 00:00:08

- Destination folder with 45 MB (516 files): ~ 00:01:23

- Destination folder with 700 MB (1780 files): ~ 00:03:05

The performance should not be affected by the contents of the destination folder as I am not using Synchronize.

As you can see the performance is just unacceptable for such a small upload. Can you please advice?

Comments

I enabled logs and I can confirm that UploadMultiple actually calls the Synchronize method to upload the files. Also I am monitoring the files that are uploaded in the FTP server and most of the time elapsed is NOT the actual file uploads. For example, with the bigger destination folder the operation takes 3 minutes, but the files start being uploaded until the last 5 seconds. This means that most of the time is probably spent getting the listings of all the contents of the destination folder to perform the synchronization, which is completely unnecessary for my scenario.

I really need a way to upload a folder with concurrency but without the Sync mode, just like FileZilla does if you ask it to upload one complete folder, the transfers just start immediately and takes the same regardless of the contents of the destination folder (in my test ~8 seconds)

How can I do this?

Answer 1

This is a good point.  We've got an experimental fix ready for you to test.  Can you please open a support ticket here so that we can send it to you?  Please also send us your purchase order number.