NullPointerException on chdir in SSHFtp

Question

Hi,

Have a problem with the SSHFTPClient that I cannot explain. I have a process that continually monitors a directory on a remote server. It normally works without a hinge, but since I switched from using normal ftp to sshftp, I sometimes get an unexpected NullPointerException when performing a directory change. The directory is checked a couple of times a minute (with the directory change coded to happen before every check). Here is the stacktrace:

java.lang.NullPointerException
at com.enterprisedt.net.j2ssh.sftp.SftpSubsystemClient.getAttributes(SftpSubsystemClient.java:486)
at com.enterprisedt.net.j2ssh.SftpClient.cd(SftpClient.java:216)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.chdir(Unknown Source)
at se.sek.integration.services.FTPHelper.SSHFtpDownloader.getFile(SSHFtpDownloader.java:64)
at se.sek.integration.services.FTPHelper.Downloader.run(Downloader.java:185)
at java.lang.Thread.run(Unknown Source)

The following code is used to perform the directory change:

client.chdir(m_ftpDownloadDir);

Why is this happening and is there anything I can do to fix the behaviour??

Thanks,
Henrik

Answer 1

SFTP operates quite differently to FTP - a completely different protocol. However it shouldn't throw a NullPointerException.

What is the value of ftpDownloadDir? Does the check ever succeed on a directory that it is failing for?

Hi,

Have a problem with the SSHFTPClient that I cannot explain. I have a process that continually monitors a directory on a remote server. It normally works without a hinge, but since I switched from using normal ftp to sshftp, I sometimes get an unexpected NullPointerException when performing a directory change. The directory is checked a couple of times a minute (with the directory change coded to happen before every check). Here is the stacktrace:

java.lang.NullPointerException
at com.enterprisedt.net.j2ssh.sftp.SftpSubsystemClient.getAttributes(SftpSubsystemClient.java:486)
at com.enterprisedt.net.j2ssh.SftpClient.cd(SftpClient.java:216)
at com.enterprisedt.net.ftp.ssh.SSHFTPClient.chdir(Unknown Source)
at se.sek.integration.services.FTPHelper.SSHFtpDownloader.getFile(SSHFtpDownloader.java:64)
at se.sek.integration.services.FTPHelper.Downloader.run(Downloader.java:185)
at java.lang.Thread.run(Unknown Source)

The following code is used to perform the directory change:

client.chdir(m_ftpDownloadDir);

Why is this happening and is there anything I can do to fix the behaviour??

Thanks,
Henrik

Answer 2

SFTP operates quite differently to FTP - a completely different protocol. However it shouldn't throw a NullPointerException.

What is the value of ftpDownloadDir? Does the check ever succeed on a directory that it is failing for?

Yes, I am aware that the protocol is different (underlying ssh protocol). The value of ftpDownloadDir varies (we are polling a couple of different directories on a Solaris machine - one example is "/usr/sonic/payflow").

The behaviour is that the process works (including changing directory, listing files, downloading files, delting files, etc) for quite a long time (a day or so) and then suddenly throws this exception and then doesn't recover.

The SSHFTPClient instance is NOT null at this point! (first thing I checked)

Regards,
Henrik

Answer 3

Is it possible that the SFTP session has timed out?

From the code it looks like the only way for this to occur is if the connection state is closed.

Could you take a look at the log with it set to DEBUG?

The behaviour is that the process works (including changing directory, listing files, downloading files, delting files, etc) for quite a long time (a day or so) and then suddenly throws this exception and then doesn't recover.

The SSHFTPClient instance is NOT null at this point! (first thing I checked)

Regards,
Henrik

Answer 4

How do I go about getting this log that you mention and how do I set it to debug? (I'm connecting from a windows machine to a Solaris running openssh version 4.1).

I also have the same problem as another post on this forum of getting an IOException "The connection did not complete" when connecting a large number of client threads at "client"-side server startup.

Can the problems be related?? Any recommended settings for the openssh server?

/Henrik

Answer 5

There's info in the Dev Guide on logging, or take a look at the examples included. They all log to the console.

BTW, we have just implemented setting of file permissions, send us an email if you want to try it out.

How do I go about getting this log that you mention and how do I set it to debug? (I'm connecting from a windows machine to a Solaris running openssh version 4.1).

I also have the same problem as another post on this forum of getting an IOException "The connection did not complete" when connecting a large number of client threads at "client"-side server startup.

Can the problems be related?? Any recommended settings for the openssh server?

/Henrik

Answer 6

Ok, got the loggin working. setting the level to debug produced a rediculous amount of text, so we'll try with the INFO level log first:

INFO [puretls] 8 nov 2005 16:26:42.281 : PureTLS debug level=0
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:26:42.312 : Licence expiry date: 31 dec 2099
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:26:42.312 : Production licence
INFO [com.enterprisedt.net.ftp.ssh.SSHFTPClient] 8 nov 2005 16:26:42.328 : SSHFTPClient settings validated.
INFO [cryptix] 8 nov 2005 16:26:42.359 : GLOBAL_TRACE=false
INFO [cryptix] 8 nov 2005 16:26:42.359 : GLOBAL_DEBUG=false
INFO [cryptix] 8 nov 2005 16:26:42.359 : GLOBAL_DEBUG_SLOW=false
[05/11/08 16:26:53] ID=SuretiesFromGwbToCrs_FTP (info) Connection to endpoint 'E_SEKFtpTest' (re)established

[2005-11-08 16:27:01] ExchangeRatesFromMxgToCrs_FTP: class com.enterprisedt.net.ftp.ssh.SSHFTPException: The SFTP client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
[2005-11-08 16:27:01] ExchangeRatesFromMxgToCrs_FTP: IOException: The connection did not complete, errorcount: 1 -> retry
[05/11/08 16:27:03] ID=TreasuryFromGwbToCrs_FTP (info) Connection to endpoint 'E_SEKFtpTest' (re)established

[05/11/08 16:27:11] ID=RiskoFromGwbToCrs_FTP (info) Connection to endpoint 'E_SEKFtpTest' (re)established

INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:27:14.984 : Licence expiry date: 31 dec 2099
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:27:14.984 : Production licence
INFO [com.enterprisedt.net.ftp.ssh.SSHFTPClient] 8 nov 2005 16:27:14.984 : SSHFTPClient settings validated.
ERROR [com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon] 8 nov 2005 16:27:15.31 : The Transport Protocol thread failed : null
java.lang.ClassCastException
at com.enterprisedt.net.j2ssh.transport.kex.DhGroup1Sha1.performClientExchange(DhGroup1Sha1.java:158)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolClient.performKeyExchange(TransportProtocolClient.java:392)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.beginKeyExchange(TransportProtocolCommon.java:681)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.onMsgKexInit(TransportProtocolCommon.java:1229)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.startBinaryPacketProtocol(TransportProtocolCommon.java:986)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.run(TransportProtocolCommon.java:373)
at java.lang.Thread.run(Unknown Source)
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:27:26.671 : Licence expiry date: 31 dec 2099
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:27:26.671 : Production licence
INFO [com.enterprisedt.net.ftp.ssh.SSHFTPClient] 8 nov 2005 16:27:26.671 : SSHFTPClient settings validated.
[2005-11-08 16:27:45] ExchangeRatesFromMxgToCrs_FTP: class com.enterprisedt.net.ftp.ssh.SSHFTPException: The SFTP client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
[2005-11-08 16:27:45] ExchangeRatesFromMxgToCrs_FTP: IOException: The connection did not complete, errorcount: 2 -> retry
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:27:57.296 : Licence expiry date: 31 dec 2099
INFO [com.enterprisedt.util.license.LicenseProperties] 8 nov 2005 16:27:57.296 : Production licence
INFO [com.enterprisedt.net.ftp.ssh.SSHFTPClient] 8 nov 2005 16:27:57.296 : SSHFTPClient settings validated.
ERROR [com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon] 8 nov 2005 16:27:57.328 : The Transport Protocol thread failed : null
java.lang.ClassCastException
at com.enterprisedt.net.j2ssh.transport.kex.DhGroup1Sha1.performClientExchange(DhGroup1Sha1.java:158)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolClient.performKeyExchange(TransportProtocolClient.java:392)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.beginKeyExchange(TransportProtocolCommon.java:681)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.onMsgKexInit(TransportProtocolCommon.java:1229)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.startBinaryPacketProtocol(TransportProtocolCommon.java:986)
at com.enterprisedt.net.j2ssh.transport.TransportProtocolCommon.run(TransportProtocolCommon.java:373)
at java.lang.Thread.run(Unknown Source)

If you find it hard to read, I don't blame you... The above is a snippet of the log produced. The server is trying to initialize 6 connections to the ssh server during startup. 5 of the 6 connections succeded, while the 6:th failed, throwing a ClassCastException as shown in the snippet above.

My code then tries to reconnect again, but ClassCastException is thrown once again...

This is how I make the initial connection:

client = new SSHFTPClient();
client.getValidator().setHostValidationEnabled(false);
client.setRemoteHost(m_ftpHost);
client.setRemotePort(m_ftpPort);
client.setTimeout(m_timeout);
client.setAuthentication(m_ftpUser, m_ftpPassword);
// connect to the server
client.connect();

To reconnect I do the following and then the above:

try {
   client.quit();
   client = null;
} catch (Exception qe) {
   if (m_log_actions) SEKLogger.log(m_esbComponentName+": "+qe.getClass()+": "+qe.getMessage());
   client = null;
}

Any suggestions as to how I move on?? I have been using this technique for remote file polling for 1

Answer 7

Hi, sorry about changing the direction of this post, but the problem in my previous post must superseed my original (which will still be an issue when I get the connection problem solved...).

Besides, I suspect the problems may be related...??

Regards,
Henrik

Answer 8

For both cases can you please email support at enterprisedt dot com with the log files with DEBUG set (yes, there will be a bit there).

Hi, sorry about changing the direction of this post, but the problem in my previous post must superseed my original (which will still be an issue when I get the connection problem solved...).

Besides, I suspect the problems may be related...??

Regards,
Henrik