edtFTPj/PRO - slow performance?

Question

Hi,

I've recently downloaded a trial version of edtFTPj/PRO to try out the SFTP capabilities. I've been comparing it with JSch and JSCAPE's Secure FTP Factory.

I've set up a local SFTP server on my computer (the server from CoreFTP.com) and haven't configured the server in any way beyond the bare minimum required. I've then run a series of tests to time up- and downloading to/from files and to/from memory.

I was surprised to find that edjFTPj/PRO seemed to be 4-5 times slower than the other two for uploads, and 3-4 times slower for downloads. My figures were as follows:

Upload 10Mb of data from memory: 1.33 Mbyte/s (Jsch = 6.18 Mbyte/s, Jscape = 4.50 Mbyte/s)
Download 10Mb of data to memory: 1.39 Mbyte/s (Jsch = 4.89 Mbyte/s, Jscape = 4.65 Mbyte/s)
Upload 100Mb of data from a file: 0.12 Mbyte/s (Jsch = 0.60 Mbyte/s, Jscape = 0.42 Mbyte/s)
Download 100Mb of data to a file: 0.14 Mbyte/s (Jsch = 0.57 Mbyte/s, Jscape = 0.49 Mbyte/s)

I also find that disconnecting takes an age - a suspicious looking 60 seconds, in fact. The Core FTP Server monitor indicates that the final operation (regardless of whether that is a send, receive or delete) appears to be hanging. Calling the quitImmediately() method fixes this - but makes me wonder whether there's an SFTP protocol failure (either in the server or the client). The other two libraries disconnect without exhibiting this problem.

I want to give edjFTPj/PRO a fair crack of the whip, so wondered if there might perhaps be some configuration I might consider to improve the performance.

Note from EnterpriseDT developers:
A major SFTP optimization was introduced in edtFTPj/PRO 3.1.0.
It now more than matches the speed of other Java SFTP clients.

Many thanks for any suggestions,
Alistair.

Answer 1

We haven't benchmarked for a little while so we'll run a few tests and post the results here.

Re the disconnect, you'll find that the server is failing to send the acknowledgement of a closed SFTP channel, which the client is waiting for. There's a timeout of 60 s by default. The other clients aren't waiting for it.

You can disable waiting for the ack by using setConfigFlags(SSHFTPClient.ConfigFlags.DISABLE_WAIT_FOR_CHANNEL_CLOSE) and you shouldn't get that 60 s.

Perhaps we should enable that flag by default, although we're a little reluctant as really the ack should be waited for.

Answer 2

Realistic benchmarking requires using a network - if tests are run on localhost (as above), they won't return realistic results. Across a network, cpu bound operations become almost irrelevant, whereas they can dominate in local tests.

One different we'ved noticed in testing is that j/pro by default uses 3des-cbc, while jsch uses aes-128 - a much faster algorithm (but one not supported as widely).

If the same algorithms are used, the differences are less apparent.

We've done some initial tests with j/pro and jsch using 3des-cbc on a 100 Mb/s LAN:

Upload file: j/pro 0.28 MB/s jsch 0.58 MB/s
Download file: j/pro 0.76 MB/s jsch 0.88 MB/s

We'll continue to look at ways we can improve our upload speed, but download speeds are comparable.

Answer 3

Many thanks for taking the time to look at this, Bruce. My tests were all local, so I guess they factored out any real network activity.

Answer 4

Just to follow up on this thread, we've made some significant SFTP performance enhancements to be in the forthcoming 3.1.0 release. Email support for a pre-release jar.