infrequent thread hangs

Question

Have been using the FTPClient for some pretty simple downloads but have it in a long running thread loop. About every 4-5 days the getter thread hangs. Have been wandering through the code to see if there was anything I should be doing explicitly. Should I be setting an explicit timeout on the FTPActiveDataSocket.setTimeout( someTime) in order to force a timeout on the sock via the FTPActiveDataSocket.setTimeout( someTime)'s call to:

sock.setSoTimeout(millis);

Answer 1

You should really quit the session once your immediate downloads are complete, and re-contruct the FTPClient each time you need to download a bunch of files.

Have been using the FTPClient for some pretty simple downloads but have it in a long running thread loop. About every 4-5 days the getter thread hangs. Have been wandering through the code to see if there was anything I should be doing explicitly. Should I be setting an explicit timeout on the FTPActiveDataSocket.setTimeout( someTime) in order to force a timeout on the sock via the FTPActiveDataSocket.setTimeout( someTime)'s call to:

sock.setSoTimeout(millis);

Answer 2

I'm transferring 3 text files on an hourly basis, and I see this exception every couple of days. I've already isolated the code in a thread that exits once the transfer is complete, and starts over with a new FTPClient when its time for more files. This kept the program from failing, but still leaves me wondering. I was experiencing something similar with my own FTP implementation, which is what prompted me to find edtFTPj in the first place, so the problem may be with the JRE. I'm using 1.4.1_03, on a couple versions of Windows (2000 and 2003 Server). BTW, the server I'm d/l'ing from is ProFTPD 1.2.8.

java.lang.NullPointerException
at com.enterprisedt.net.ftp.FTPClient.initGet(FTPClient.java:1088)
at com.enterprisedt.net.ftp.FTPClient.getASCII(FTPClient.java:1110)
at com.enterprisedt.net.ftp.FTPClient.get(FTPClient.java:1021)
at ats.metapp.AMRCInput.getData(AMRCInput.java:187)
at ats.metapp.AMRCInput.access$500(AMRCInput.java:27)
at ats.metapp.AMRCInput$2.construct(AMRCInput.java:124)
at ats.common.SwingWorker$2.run(SwingWorker.java:118)
at java.lang.Thread.run(Unknown Source)

It's being thrown by the following code:

    // make the ftp connection to the server
    try {
      ftp = new FTPClient(ftpHost, DEFAULT_PORT, FTP_TIMEOUT);
      Logger.setLevel(Level.ERROR);
      ftp.setTimeout(FTP_TIMEOUT);
      ftp.setType(FTP_MODE);
      ftp.login(ftpUser, ftpPass);
      ftp.chdir(ftpDir);
 
      // get the files from AMRC
      for (int i = 0; i < 3; i++) {
        try {
          ftp.get(filename[i], filename[i]);
        }
        catch (IOException ex) {
          lastException[i] = ex;
        }
        catch (FTPException ex) {
          lastException[i] = ex;
        }
      }
    }
    catch (IOException ex) {
      firstException = ex;
    }
    catch (FTPException ex) {
      firstException = ex;
    }
    finally {
      try {
        ftp.quit();
      catch (Exception ignore) {}
    }

Answer 3

Can you clarify, do you get the exception when you use a new FTPClient for each batch of files?

I'm transferring 3 text files on an hourly basis, and I see this exception every couple of days. I've already isolated the code in a thread that exits once the transfer is complete, and starts over with a new FTPClient when its time for more files. This kept the program from failing, but still leaves me wondering. I was experiencing something similar with my own FTP implementation, which is what prompted me to find edtFTPj in the first place, so the problem may be with the JRE. I'm using 1.4.1_03, on a couple versions of Windows (2000 and 2003 Server). BTW, the server I'm d/l'ing from is ProFTPD 1.2.8.

Answer 4

This won't solve your problem, but it might help you find out what is going on. It appears that there is a bug in FTPClient that masks the true problem.

Looking inside the source for FTPClient, in initGet, there is a call to createDataSocket(). If this has any problems and throws either an IOException or an FTPException, the exception is caught and a flag is set to close the connection. But an exception coming from createDataSocket will result in the variable 'data' being left initialized to null if it is the first file in the batch, since the get() is the first FTPClient operation you call that will try to make the data connection. After the exception is caught, the finally clause will try to close the connection (since the flag is set.) This will result in a NullPointerException which will be thrown instead of the original exception.

I would suggest changing the finally clause in FTPClient.initGet to be:

finally {
if (close && (data != null)) {
try {
data.close();
} catch (IOException ignore) {}
}
}

It won't solve all of your problems, but it will allow the original FTPException or IOException propogate out of the function. This should give you a better indication of what your actual problem is.

Answer 5

Can you clarify, do you get the exception when you use a new FTPClient for each batch of files?

Yes, I'm still seeing the exception. The only improvement I've made so far is to isolate the exception to a thread which exits without taking the whole app down. (Yes, I could try catching the exception, but that seems kinda self-defeating to me.) The module in question is part of a meteorological data collection and distribution system used in Antarctica.

Answer 6

Oops, I didn't realize that I wasn't logged in for the last post. Thanks to wstoppler for the info :D

Answer 7

Well spotted, this could easily be the cause of the NullPointerException (which is at the data.close() line). Thanks for pointing this out.

It doesn't tell us why the createDataSocket() has failed, but you should get the exception you are interested in now.

We'll put this fix into 1.4.2, but you (vehornr) might want to try modifying this line, recompiling and trying this fix out.

Oh, and there's a similar problem with initPut() which we'll fix as well.

This won't solve your problem, but it might help you find out what is going on. It appears that there is a bug in FTPClient that masks the true problem.

Looking inside the source for FTPClient, in initGet, there is a call to createDataSocket(). If this has any problems and throws either an IOException or an FTPException, the exception is caught and a flag is set to close the connection. But an exception coming from createDataSocket will result in the variable 'data' being left initialized to null if it is the first file in the batch, since the get() is the first FTPClient operation you call that will try to make the data connection. After the exception is caught, the finally clause will try to close the connection (since the flag is set.) This will result in a NullPointerException which will be thrown instead of the original exception.

I would suggest changing the finally clause in FTPClient.initGet to be:

finally {
if (close && (data != null)) {
try {
data.close();
} catch (IOException ignore) {}
}
}

It won't solve all of your problems, but it will allow the original FTPException or IOException propogate out of the function. This should give you a better indication of what your actual problem is.

Answer 8

Hi Support,

I was just going to post about this although I am not sure if I am experiencing the exact same problem, here is what I am noticing:

I perform a single put with a single FTPClient in a thread (there maybe several concurrent threads putting different files to the same remote host). I also built a JSP dashboard to monitor the progress of each When the file being transferred is particularly large ( 600+ MB ) I noticed that the thread stays alive and the put appears to have stopped at 99% just a few bytes short of completion according to the progress monitor. However, when I manually log into the remote host I see that the transfer was indeed successful, 100%. So if PUT never completes then that explains why my thread won't die. I am unable to replicate this situation when transferring smaller files. I am doing all transfers in PASV, TYPE I. My hypothesis is that certain FTP servers may be immediately closing the socket (or IO stream) after transfer completes, causing the data to throw the NullPointerException.

Not sure if this is related as well, but should I always do a ftp.quit() after a single put or just let the thread finish and die? If the connection has timed out after a transfer, I have seen the server return "500 connection timed out". I'm afraid if the server is closing after a lengthy transfer that errant Exceptions may be throw when everything was actually successful.

Thanks!

Answer 9

You should always quit() if the session is finished, i.e. if you don't expect to be performing any operations in the next couple of minutes.

Are you sure put() doesn't complete? Look at the logs to check - it might be that the last few bytes aren't recorded in the monitor.

Hi Support,

I was just going to post about this although I am not sure if I am experiencing the exact same problem, here is what I am noticing:

I perform a single put with a single FTPClient in a thread (there maybe several concurrent threads putting different files to the same remote host). I also built a JSP dashboard to monitor the progress of each When the file being transferred is particularly large ( 600+ MB ) I noticed that the thread stays alive and the put appears to have stopped at 99% just a few bytes short of completion according to the progress monitor. However, when I manually log into the remote host I see that the transfer was indeed successful, 100%. So if PUT never completes then that explains why my thread won't die. I am unable to replicate this situation when transferring smaller files. I am doing all transfers in PASV, TYPE I. My hypothesis is that certain FTP servers may be immediately closing the socket (or IO stream) after transfer completes, causing the data to throw the NullPointerException.

Not sure if this is related as well, but should I always do a ftp.quit() after a single put or just let the thread finish and die? If the connection has timed out after a transfer, I have seen the server return "500 connection timed out". I'm afraid if the server is closing after a lengthy transfer that errant Exceptions may be throw when everything was actually successful.

Thanks!

Answer 10

Regarding put() not finishing...

Looking at the code, any monitor interval > 512 where (filesize mod monitor_interval != 0) will never report the last bytes transferred. However this was not the problem with put().

The problem is that with a PASV connection, servers can frequently appear to hang and never send back a close/complete reply to the client when the transfer completes [a known FTP caveat to large scale users/admins]. My solution was to set a timeout and catch SocketTimeoutException. At this point, I check the total size transferred (from the monitor) and compare it against the actual filesize in bytes (I only do binary so no ASCII CR/LF issues). I evaluate this within a dynamic tolerance to fuzzy logic "guess" whether the transfer completed sucessfully.

Just wanted to follow up :D