How can I speed up SFTP?

Question

I have built a rollout type of application that uploads a large number of files and folders to an Akamai NetStorage server. I have all file I/O for this rollout application encapsulated within a class I wrote in order to hide protocol-specific features from the application. The application simply specifies to use either FTP or SFTP and the file I/O class takes care of the rest. Internally, my file I/O class makes use of your edtFTPnet/PRO classes ProFTPClient, SSHFTPClient, and polymorphically IFileTransferClient to do its magic.

In the typical case when someone needs to use the rollout tool, about 270 MB worth of data (split across 1300+ file and 500+ folders) gets uploaded to the Akamai NetStorage server. Using FTP, uploading all of these files/folders takes about 7 minutes. (NOTE: we get a huge speed benefit out of running 10 simultaneous upload threads, each with its own, independent, FTP Client.) But using SFTP, this same exact process takes about 24 minutes. In general, we're seeing that it takes about 3-5 times longer to do the same thing under SFTP as it does under FTP. Why is it so slow? Any suggestions for speeding this up?

One thing I already do under both FTP and SFTP is to use connection pooling. I

Answer 1

The main reason why FTP is so much faster than SFTP is that FTP is unencrypted. The raw file is simply written to a socket and read by the server.

In SFTP, the file is broken into packets, and every packet is encrypted on the client side, transmitted, and decrypted on the server side. There is a lot of overhead in doing so.