Exceptions with AsyncFileTransferClient and SFTP shared keys

Question

I'm trying to use SFTP pre-shared key login with AsyncFileTransferClient, and I'm running into several unexpected exceptions, that occur seemingly at random. The way I'm using the client is a main thread that lists a directory, and for each file that matches a REGEX download it via an asyncRename followed by an asyncDownloadByteArray. I've added an errorListener, and I think most of these are coming from that. The first error I'm getting, although this seems to have been fixed in v3.7.0 is:

com.enterprisedt.net.ftp.ssh.SSHFTPException: The SSH client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
at com.enterprisedt.net.ftp.ssh.SCPClient.checkConnection(Unknown Source)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.chdir(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.FTPConnection.setDirectory(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.Task.A(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.Task.configureFreeConnRemoteDirs(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.RenameTask.run(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor$_A.run(Unknown Source)


As I said, that seems to be fixed in v3.7.0, so I'm not so worried about it, but I'm getting a couple others using v3.7.0. One of them is:

java.io.IOException: Null response to FXP_REAL_PATH
at com.enterprisedt.net.j2ssh.sftp.SftpSubsystemClient.getAbsolutePath(Unknown Source)
at com.enterprisedt.net.j2ssh.SftpClient.ls(Unknown Source)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.D(Unknown Source)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.dirDetails(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.DirectoryListTask.run(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor$_A.run(Unknown Source)

and another is:

com.enterprisedt.net.ftp.FTPException ReplyCode=-1 - The SFTP Subsystem could not be initialized
com.enterprisedt.net.ftp.FTPException: The SFTP Subsystem could not be initialized
at com.enterprisedt.net.ftp.async.AsyncResult.endAsyncInternal(Unknown Source)
at com.enterprisedt.net.ftp.async.DirectoryListResult.endAsync(Unknown Source)
at com.enterprisedt.net.ftp.SecureFileTransferClient.directoryList(Unknown Source)
at com.telescope.Insight.Transport.Impl.SftpTransport.run(SftpTransport.java:156)

Similar, but different stack trace:

com.enterprisedt.net.j2ssh.SshException: The SFTP Subsystem could not be initialized
at com.enterprisedt.net.j2ssh.SshClient.openSftpChannel(Unknown Source)
at com.enterprisedt.net.j2ssh.SftpClient.<init>(Unknown Source)
at com.enterprisedt.net.j2ssh.SshClient.openSftpClient(Unknown Source)
at com.enterprisedt.net.j2ssh.SshClient.openSftpClient(Unknown Source)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.connect(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.ConnectTask.connect(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.ConnectTask.run(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor.createConnection(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor$_A.run(Unknown Source)

I also appear to be getting some read timeouts:
Read timed out
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.net.SocketInputStream.read(SocketInputStream.java:182)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.A(Unknown Source)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.run(Unknown Source)
at java.lang.Thread.run(Thread.java:619)
com.enterprisedt.net.ftp.FTPException ReplyCode=-1 - Read timed out
com.enterprisedt.net.ftp.FTPException: Read timed out
at com.enterprisedt.net.ftp.async.AsyncResult.endAsyncInternal(Unknown Source)
at com.enterprisedt.net.ftp.async.DirectoryListResult.endAsync(Unknown Source)
at com.enterprisedt.net.ftp.SecureFileTransferClient.directoryList(Unknown Source)
at com.telescope.Insight.Transport.Impl.SftpTransport.run(SftpTransport.java:156)

and for some reason I'm getting some that are interrupted:
java.io.IOException: The thread was interrupted
at com.enterprisedt.net.j2ssh.sftp.SftpSubsystemClient.A(Unknown Source)
at com.enterprisedt.net.j2ssh.sftp.SftpSubsystemClient.openDirectory(Unknown Source)
at com.enterprisedt.net.j2ssh.SftpClient.ls(Unknown Source)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.D(Unknown Source)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.dirDetails(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.DirectoryListTask.run(Unknown Source)
at com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor$_A.run(Unknown Source)

Answer 1

Hi, if you can send us a zipped log file at the ALL level it should be a lot clearer what is happening. support at enterprisedt dot com.

Answer 2

I have just emailed support. Can you please have someone look at this ASAP? We are getting close to launch and this looks like it may be interfering with the application running and pulling the data in a timely fashion.

Answer 3

Ok, if anyone's interested, here's a PSA to NOT try and re-use the AsyncResult.getClient() and use that for any of the synchronous methods. At least, not without calling endAsync() first. :oops:

Answer 4

For AsyncFileTransferClient the rule is to NOT call a synchronous method from inside an asynchronous method's callback.

The synchronous method call in the callback will be waiting to get a free connection, while your asynchronous call that resulted in the callback has not yet freed the connection (and can't until the synch call has completed). Deadlock if you only have one connection. Even with more than one connection, as soon as you have multiple asynchronous method calls you get the same result.

So only use asynchronous methods inside asynchronous method callbacks.