Our Products:   CompleteFTP  edtFTPnet/Free  edtFTPnet/PRO  edtFTPj/Free  edtFTPj/PRO
0 votes
4.4k views
in Java FTP by (160 points)
Hi !

We are in a process of doing the final testing before implementing our new FTP API using EDTFTPJ-PRO-1.3.3 version to connect via SSL to an FTP server on AIX using Proftd version 1.2.10rc3.

So far things were looking goot, but since we started our load / stress test we realized that our connections always remain at the QUIT state and never gets released.

Any one is aware of that kind of issue ?

Thanks for your help

7 Answers

0 votes
by (51.6k points)
Yes, this happens because ProFTPD doesn't acknowledge the SSL closure signal as it should, so edtFTPj/PRO appears to hang while it's waiting. The following flag switches off SSL closure for both control and data channels:
ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_SSL_CLOSURE);

If you want to be more cautious you can switch it off on the control channel only by:
ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_CONTROL_SSL_CLOSURE);

The latter should be sufficient, but if you find that data channels failing to close then you'll need to use the former.

- Hans (EnterpriseDT)
0 votes
by (180 points)
Hi,

In the EDT API docs there is this text:


ProFTPD FTP servers (at least until version 1.2.9) don't close SSL connections on the data-channel in the standard way, so clients connecting to servers of this type must use the DISABLE_DATA_WAIT_ON_CLOSE flag. The control channel closure conforms to the standard.

Are you sure that:

ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_SSL_CLOSURE);

is correct and not

ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_DATA_WAIT_ON_CLOSE );

or

ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_DATA_SSL_CLOSURE);

Thanks,
0 votes
by (51.6k points)
In FTP SSL closure is supposed to work as follows:
  1. The client sends an I-want-to-close message to the server.
  2. The client waits for the server's reply.
  3. Once the server receives the message it replies with its own I-want-to-close message and shuts down the connection.
  4. Once the client receives the server's reply it closes the connection.
Keep in mind that FTP uses multiple connections for each session - one for sending commands (the control channel) and one for each file transfer (the data channels).

The SSL closure-related flags in SSLFTPClient may be explained as follows:
  • DISABLE_CONTROL_WAIT_ON_CLOSE stops the client from waiting for the server's reply when closing the control channel. In other words, it sends the I-want-to-close message but then just closes the connection straight afterwards.
  • DISABLE_CONTROL_SSL_CLOSURE stops the client from sending the I-want-to-close message on the control channel - it simply closes the control channel without warning.
  • DISABLE_DATA_WAIT_ON_CLOSE stops the client from waiting for the server's reply when closing a data channel.
  • DISABLE_DATA_SSL_CLOSURE stops the client from sending the I-want-to-close message on a data channel
  • DISABLE_WAIT_ON_CLOSE stops the client from waiting for the server's reply when closing all channels - control and data.
  • DISABLE_SSL_CLOSURE turns off SSL closure altogether for all channels - control and data.

It's actually quite safe to disable SSL closure on the control channel since the server and client have already agreed to close by means of the standard QUIT command.

The data channels are a different matter since the means of determining the end of a transfer is in fact the closure of the data channel. In other words, when the client uploads a file, the server only knows that the client has finished when it detects that the client has closed that data channel. This makes it vulnerable to 'truncation attacks', which are attacks where the channel is closed before a transfer is complete. In standard FTP, the server has no way of knowing that such an attack has occurred. SSL closure prevents this happening in FTPS since both parties must agree that they're going to close the channel before they do so. So if the channel is closed without an SSL closure being completed then the receiving end should assume that there's been a truncation attack and therefore discard the data that it's just received.

Unfortunately compliance is poor in this area. Our experience is that a lot of servers don't perform an SSL closure exchange (they simply close the connection abruptly) and are therefore vulnerable to truncation attacks. It's not a major problem though since truncation attacks are quite difficult to carry out (requiring perfect timing and a high level of access) and are often not a great risk in terms of loss.

Anyway, it is for this reason that we've had to introduce these flags.

What does this mean for you?
  • DISABLE_DATA_SSL_CLOSURE should be off if possible in order to reduce truncation attacks. Having said that, I've never heard of a server that will reject an upload, as it should, if there was no SSL closure, which really makes it rather pointless. If you're concerned about truncation attacks when downloading then you should keep it off, otherwise you may turn it on in order to increase compatibility.
  • DISABLE_CONTROL_SSL_CLOSURE is safe to turn on since the client and server effectively go through their own closure exchange using the standard FTP QUIT command and reply. And again, I've never heard of a server providing the option to abandoning an entire session if the control channel was not closed using an SSL closure.
  • The XXX_WAIT_ON_CLOSE flags should prevent hanging, but since they stop the client from confirming that the SSL closure exchange was completed they do weaken security slightly.

Is that clear? I'm afraid I've rambled on a bit here, but it is a rather complex topic.

- Hans (EnterpriseDT)
0 votes
by (180 points)
Well thank you for the in depth explanation.

So to summarize, are you saying that in relation to the ProFTPd problem described in the documentation for the SSLFTPClient.ConfigFlags Class that says:

ProFTPD FTP servers (at least until version 1.2.9) don't close SSL connections on the data-channel in the standard way, so clients connecting to servers of this type must use the DISABLE_DATA_WAIT_ON_CLOSE flag. The control channel closure conforms to the standard.

using either DISABLE_SSL_CLOSURE or DISABLE_DATA_WAIT_ON_CLOSE as described above will result in the same effect? Is that correct?

Also, I've been working on the same issue as the author of this topic, and know that the code using the EDT API which caused all those connections on the ProFTPd server to hang in a QUIT state was already using the following config flag:

ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_SSL_CLOSURE);

If DISABLE_SSL_CLOSURE accomplishes the same effect as DISABLE_DATA_WAIT_ON_CLOSE and the sessions in QUIT state on the server still hang, are there any other suggested actions to take that might help to release them?

Or does setting DISABLE_DATA_WAIT_ON_CLOSE in addition to DISABLE_SSL_CLOSURE or maybe even instead of, behave differently?

Thanks,
0 votes
by (162k points)
DISABLE_SSL_CLOSURE turns off SSL closure altogether for all channels. So if there is still a hanging problem, it isn't with SSL closure. Please email a log set to a debug level of ALL to support at enterprisedt dot com and we'll take a look.
0 votes
by (160 points)
Thank you very much for these answers

But,

When I opend my posting here is what we had in the code :

ftp.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_SSL_CLOSURE);

Then we tried:

ftp.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_SSL_CLOSURE | SSLFTPClient.ConfigFlags.DISABLE_DATA_WAIT_ON_CLOSE);

And we are still getting the Issue with connections remaining on QUIT and generating high CPU and server to crash

We will now try the following:
ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_CONTROL_SSL_CLOSURE);

But I would like to know if it should be combined with:
ftpClient.setConfigFlags(SSLFTPClient.ConfigFlags.DISABLE_SSL_CLOSURE); or on it's own...


Also I am looking at how the ftp.quit was developped by the consultant and I'm not sure it is quite right...
as there is not much to it... based on what I read on the forum for ftp.quit.

catch (Exception ex) {
if (debugThis) {throw new ServiceException(ex.toString()+"\n"+ message.toString());}
else {throw new ServiceException(ex.toString());}
} finally {
if (ftp != null) {
try {
ftp.quit();

} catch (IOException ex) {
} catch (FTPException ex) {
}



Thank you so much for all of your help...
0 votes
by (162k points)
If you are still getting a problem when using DISABLE_SSL_CLOSURE send us a log file and a code snippet of what is being done.

Categories

...