Java FTP Client Download Problem

Question

I have been using the edtFTPj for about 2 years now with no problems. I am using it to automatically download the latest files from an FTP server. In my code, I retrieve the directory listing before trying to download the files. For some reason this is not working on some folders, but it is on others. Some folders return no files. When I log into the ftp from the command prompt or IE, I see a number of files within the folder.

My question is whether or not anyone has experienced this problem with edtFTPj? Also, is there a size limit on the directory list return that I'm not aware of? Or is there anything else that could prohibit the listing of the directory contents using dir() method?

Thanks,
Dan

Answer 1

Some people have reported this problem - it has been very hard to diagnose. There is no size limit.

If you can log in using ftp.exe and capture the file listing from "dir" and post it (from one of the offending directories) together with the log file we will try to figure it out.

Also, if you email support at enterprisedt dot com we will send you a version with more logging that might help diagnose this.

I have been using the edtFTPj for about 2 years now with no problems. I am using it to automatically download the latest files from an FTP server. In my code, I retrieve the directory listing before trying to download the files. For some reason this is not working on some folders, but it is on others. Some folders return no files. When I log into the ftp from the command prompt or IE, I see a number of files within the folder.

My question is whether or not anyone has experienced this problem with edtFTPj? Also, is there a size limit on the directory list return that I'm not aware of? Or is there anything else that could prohibit the listing of the directory contents using dir() method?

Thanks,
Dan

Answer 2

Thanks for the reply. I included some logging using the setMessageLogging() method. My application iterates through a predetermined set of folders which reside on the server. Below is the results from the log:

---> PASV
227 Entering Passive Mode (205,172,241,205,109,126)
---> NLST AD
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,127)
---> NLST AU
550 AU/*: No such file or directory.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,128)
---> NLST BL
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,129)
---> NLST CC
550 CC/*: No such file or directory.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,130)
---> NLST FI
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,131)
---> NLST MA
550 MA/*: No such file or directory.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,132)
---> NLST OI
550 Arguments too long
---> PASV
227 Entering Passive Mode (205,172,241,205,109,133)
---> NLST SD
550 Arguments too long
---> PASV
227 Entering Passive Mode (205,172,241,205,109,134)
---> NLST SP
550 SP/*: No such file or directory.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,135)
---> NLST SR
550 Arguments too long
---> PASV
227 Entering Passive Mode (205,172,241,205,109,136)
---> NLST TA
550 TA/*: No such file or directory.
---> PASV
227 Entering Passive Mode (205,172,241,205,109,137)
---> NLST TR
150 Opening BINARY mode data connection for file list.
226 Transfer complete.

Answer 3

The problem here is that the server doesn't support arguments to NLST. Instead, you must cd into each directory and then call dir() (no args).

Unfortunately edtFTPj is swallowing a 550 error - we'll change this to throw an exception instead.

Thanks for the reply. I included some logging using the setMessageLogging() method. My application iterates through a predetermined set of folders which reside on the server. Below is the results from the log:

Answer 4

I made the suggested switch from dir(directory Name) to using the chdir(directory Name) in conjunction with the dir() method and I received similar results. Below is the output I received from the trace. It appears to get into each directory, but the directory list is where it fails. This only occurs on some folders, and not all. I abbreviated the output because the last folder worked successfully and it was a repeat of each file in the folder being downloaded.

---> CWD AD
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,69)
---> NLST
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> CWD ..
250 CWD command successful.
---> CWD AU
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,70)
---> NLST
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD BL
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,71)
---> NLST
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> CWD ..
250 CWD command successful.
---> CWD CC
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,72)
---> NLST
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD MA
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,73)
---> NLST
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD OI
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,74)
---> NLST
550 Arguments too long
---> CWD ..
250 CWD command successful.
---> CWD SD
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,75)
---> NLST
550 Arguments too long
---> CWD ..
250 CWD command successful.
---> CWD SP
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,76)
---> NLST
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD SR
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,77)
---> NLST
550 Arguments too long
---> CWD ..
250 CWD command successful.
---> CWD TA
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,78)
---> NLST
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD TR
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,79)
---> NLST
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> PASV
227 Entering Passive Mode (205,172,241,205,71,80)
---> RETR TR_SCH2DT_000050247_2004110100_20050407154736.CSV.gz
150 Opening BINARY mode data connection for TR_SCH2DT_000050247_2004110100_20050
407154736.CSV.gz (495 bytes).
226 Transfer complete.
---> PASV

Answer 5

Ok, one more thing to try is dir("."), i.e. supplying the current directory, ".", as the name.

Answer 6

No Dice .... Trace output below

---> CWD AD
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,120)
---> NLST .
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> CWD ..
250 CWD command successful.
---> CWD AU
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,121)
---> NLST .
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD BL
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,122)
---> NLST .
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> CWD ..
250 CWD command successful.
---> CWD CC
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,123)
---> NLST .
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD FI
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,124)
---> NLST .
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> CWD ..
250 CWD command successful.
---> CWD MA
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,125)
---> NLST .
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD OI
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,126)
---> NLST .
550 Arguments too long
---> CWD ..
250 CWD command successful.
---> CWD SD
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,127)
---> NLST .
550 Arguments too long
---> CWD ..
250 CWD command successful.
---> CWD SP
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,128)
---> NLST .
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD SR
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,129)
---> NLST .
550 Arguments too long
---> CWD ..
250 CWD command successful.
---> CWD TA
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,130)
---> NLST .
550 *: No such file or directory.
---> CWD ..
250 CWD command successful.
---> CWD TR
250 CWD command successful.
---> PASV
227 Entering Passive Mode (205,172,241,205,251,131)
---> NLST .
150 Opening BINARY mode data connection for file list.
226 Transfer complete.
---> CWD ..
250 CWD command successful.

Answer 7

ok, let's find out what it does accept.

log in using command line ftp.

then type 'debug'

then type 'dir' and copy the output into a text editor

and also 'ls' and copy the output

then post the results

Answer 8

As requested, the results are posted below. I executed the commands on both the root and one of the subfolders (SR). The results of the dir on the root folder were too long to post (there were thousands of files in that subdirectory), so I cut out most of that.

ftp> debug
Debugging On .
ftp> dir
---> PORT 172,28,20,46,7,35
200 PORT command successful.
---> LIST
150 Opening ASCII mode data connection for directory listing.
total 1607
drwxrwx--- 2 1110 1110 1560 Jul 13 17:55 AD
drwxrwx--- 2 1110 1110 48 Sep 26 2002 AU
drwxrwx--- 2 1110 1110 3144 Jul 28 08:42 BL
drwxrwx--- 2 1110 1110 48 Sep 26 2002 CC
drwxrwx--- 2 1110 1110 8112 Aug 3 08:06 FI
drwxrwx--- 2 1110 1110 48 Sep 26 2002 MA
drwxrwx--- 2 1110 1110 183936 Aug 3 08:06 OI
drwxrwx--- 2 1110 1110 415152 Aug 2 13:57 SD
drwxrwx--- 2 1110 1110 48 Nov 23 2004 SP
drwxrwx--- 2 1110 1110 204048 Aug 2 13:57 SR
drwxrwx--- 2 1110 1110 48 Sep 26 2002 TA
drwxrwx--- 2 1110 1110 2352 Jul 13 09:49 TR
226 Transfer complete.
ftp: 720 bytes received in 0.01Seconds 72.00Kbytes/sec.
ftp> ls
---> PORT 172,28,20,46,7,38
200 PORT command successful.
---> NLST
550 No files found.
ftp> cd SR
---> CWD SR
250 CWD command successful.
ftp> dir
---> PORT 172,28,20,46,7,39
200 PORT command successful.
---> LIST
150 Opening ASCII mode data connection for directory listing.
total 25480
-rw-rw---- 2 1110 1110 715 Apr 8 2005 SR_ARRDISTR_000050247_200
5030100_20050407180314.CSV.gz
-rw-rw---- 2 1110 1110 703 May 11 2005 SR_ARRDISTR_000050247_200
5040100_20050506194347.CSV.gz
-rw-rw---- 2 1110 1110 716 Jun 14 10:37 SR_ARRDISTR_000050247_200
5050100_20050613193155.CSV.gz
-rw-rw---- 2 1110 1110 705 Jul 12 14:03 SR_ARRDISTR_000050247_200
5060100_20050711155305.CSV.gz

......

-rw-rw---- 2 1110 1110 286 Jun 15 11:32 SR_SUPPLYICAP_000050247_2
005070100_20050615111435.CSV.gz
-rw-rw---- 2 1110 1110 258 Jul 15 18:12 SR_SUPPLYICAP_000050247_2
005080100_20050715210924.CSV.gz
226 Transfer complete.
ftp: 315105 bytes received in 6.11Seconds 51.58Kbytes/sec.
ftp> ls
---> PORT 172,28,20,46,7,40
200 PORT command successful.
---> NLST
550 Arguments too long
ftp> debug
Debugging Off .
ftp> quit
221-You have transferred 0 bytes in 0 files.
221-Total traffic for this session was 742 bytes in 0 transfers.
221-Thank you for using the FTP service on misftp0.iso-ne.com.
221 Goodbye.

Answer 9

The problem is apparent from your listings.

The dir() command uses NLST, and your server does not appear to work with NLST - it is reporting "550 Arguments too long". Perhaps this is because there are too many files in the directory.

One option is to try the dirDetails() command, e.g. change into each directory and use dirDetails(".") or dirDetails(""). This uses LIST, which looks as though it is supported from your listing.

As requested, the results are posted below. I executed the commands on both the root and one of the subfolders (SR). The results of the dir on the root folder were too long to post (there were thousands of files in that subdirectory), so I cut out most of that.

Answer 10

Using the new method you mentioned appears to have worked. Thank you very much for all your help.