FTPThread's appear to be hung at getConnection()

Question

I've been using the SecureFileTransferClient to do some regular, plain old FTP work, and it's worked fine up until now. My user threads are waiting on a synchronous directoryList() call (each has it's own SecureFileTransferClient instance). The edtFTP thread that appears to be servicing that request is sitting there, apparently doing nothing. I do have a few log messages from exceptions that came up the stack to my user threads, but after a short time, they stop appearing as I'm guessing that's when the FTPThread's hung. Really not sure what is going on, as every other time when there was a connection issue it's all worked fine, especially with throwing an exception back up the stack for me to catch.

I'm on v3.8.1.

User thread:

java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
com.enterprisedt.net.ftp.async.AsyncResult.waitTillComplete(Unknown Source)
com.enterprisedt.net.ftp.async.AsyncResult.endAsyncInternal(Unknown Source)
com.enterprisedt.net.ftp.async.DirectoryListResult.endAsync(Unknown Source)
com.enterprisedt.net.ftp.SecureFileTransferClient.directoryList(Unknown Source)
com.telescope.Insight.Transport.Impl.FileTransferConnectionThread.run(FileTransferConnectionThread.java:232)

FTPThread:

Name: FTPThread_edt_2
State: WAITING on com.enterprisedt.net.ftp.async.internal.FTPConnectionPool@c4a80a1
Total blocked: 277  Total waited: 490,092

Stack trace: 
 java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
com.enterprisedt.net.ftp.async.internal.FTPConnectionPool.getConnection(Unknown Source)
com.enterprisedt.net.ftp.async.internal.FTPTaskProcessor$_A.run(Unknown Source)

Log messages:

2011-08-19 09:05:00,778 ERROR Impl.FileTransferConnectionThread - com.enterprisedt.net.ftp.ControlChannelIOException - Operation timed out
2011-08-19 09:06:16,057 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:06:26,918 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:06:40,304 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:06:54,442 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:07:31,066 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:07:41,927 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:07:55,312 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:08:09,451 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:08:46,300 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:08:56,934 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:09:10,319 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out
2011-08-19 09:09:24,459 ERROR Impl.FileTransferConnectionThread - ReplyCode=-1 com.enterprisedt.net.ftp.FTPException - Operation timed out

Answer 1

We've taken a look at this. The problem seems to be that all other connections are in use - a free one can't be obtained.

In this case an exception should be thrown after the timeout period, and I've changed the code to accommodate this. You may also be able to increase the size of the pool to avoid the problem.

Please email us for a new jar file to test out.

Answer 2

Thanks. Odd thing is, there were 3 other FTPThread's per SecureFileTransferClient instance (4 threads per instance total), and from what I could see they were not in use. Could one of the other threads have gotten in a state where it thought the connection was in use, but not had one? This appears to be the first time we've encountered this problem, although I almost would have expected it to appear sooner given our use, which was exactly the same as weeks past.

Answer 3

It depends on how many connections were being used rather than the number of threads.

Answer 4

Ah. So there's a limit to the total number of connections (JVM wide)? Or is it per instance? I'm pretty sure it's running more or less at defaults from a thread/number of connections standpoint.

Answer 5

No, there's a configurable number of FTP connections with SecureFileTransferClient.

Answer 6

Ah, so it's per-instance of SecureFileTransferClient, ok, that's what I thought. I don't believe I changed that from the default, and it was only 1 of my threads using a single SecureFileTransferClient instance, so since I was using all synchronous operations, I doubt I could have exceeded the maximum number of threads for the instance.