directory parsing problem on microsoft ftp service

Question

I've got edtftpj 1.4.0. I have a couple of customers who are using it to log into a Microsoft FTP server and its getting an exception when trying to parse the directory entries. It appears that the following exception happens with java 1.4.1 but is fixed with 1.4.2.

===============
Java(TM) Plug-in: Version 1.4.1_02
Using JRE version 1.4.1_02 Java HotSpot(TM) Client VM

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:37.984 : 220 srv-test-02 Microsoft FTP Service (Version 5.0).

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:37.984 : ---> USER video1

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.15 : 331 Password required for video1.

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.15 : ---> PASS ********

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.31 : 230 User video1 logged in.
DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.31 : ---> TYPE I

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.46 : 200 Type set to I.
DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.78 : ---> SYST

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.93 : 215 Windows_NT version 5.0

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.93 : ---> PASV

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.109 : 227 Entering Passive Mode (10,1,1,14,7,163).

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.109 : ---> LIST iqlib/

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.125 : 125 Data connection already open; Transfer starting.

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.125 : 226 Transfer complete.
DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.187 : ---> PASV

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.203 : 227 Entering Passive Mode (10,1,1,14,7,165).

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.203 : ---> LIST iqlib/IQeye3/

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.203 : 125 Data connection already open; Transfer starting.

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.218 : 226 Transfer complete.
DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.265 : ---> PASV

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.265 : 227 Entering Passive Mode (10,1,1,14,7,167).

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.265 : ---> LIST iqlib/IQeye3/040630/

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.281 : 125 Data connection already open; Transfer starting.

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.281 : 226 Transfer complete.

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.343 : ---> PASV

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.343 : 227 Entering Passive Mode (10,1,1,14,7,169).

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.343 : ---> LIST iqlib/IQeye3/040630/15/

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.359 : 125 Data connection already open; Transfer starting.

DEBUG [FTPControlSocket] 30 Jun 2004 15:17:38.359 : 226 Transfer complete.

dir exception

java.text.ParseException: Failed to retrieve name: 06-30-04 03:09PM <DIR> 06

at WindowsFileParser.parse(WindowsFileParser.java:123)

at FTPFileFactory.parse(FTPFileFactory.java:120)

at FTPClient.dirDetails(FTPClient.java:1554)

=====================

This string:
06-30-04 03:09PM <DIR> 06

seems perfectly legal to me. Have you seen anything like this? Can this be fixed in edtftpj 1.4.1 that you just released?

Answer 1

I verified part of my own question. This exception occurs with java 1.4.1_x and works with current 1.4.2_04. However I don't know why this is. It would be nice not to have to require 1.4.2 just for this reason. Maybe WindowsFileParser can be "fixed" to work with 1.4.1.

Answer 2

Since we were struggling with parsing issues ourselves (although for us it was the issue of selecting the correct parser for the FTP server), we took a look at your issue as well. I don't know why it works for java 1.4.2, but here is what we found. (The code we are looking at is revision 1.3 of WindowsFileParser.)

The problem appears to be that the parser splits the line based on whitespace. In your case, the filename (directory name) is 06. This splitting is good for breaking out the date, etc, but not good for determining the filename. If the filename has whitespace in it, the name will be split up. To avoid this, the original string is searched, looking for the split string for the filename. For example, if the text was

06-30-04 03:09PM <DIR> Filename

It would search for "Filename". If the text was

06-30-04 03:09PM <DIR> My Filename

it would search for "My".

The problem is that the directory name in your case is "06". When searching the original string, it matches on the month of date part. The index returned from String.indexOf will be 0 (start of string). The code interprets this case as a failure and throws and exception.

You were a bit lucky in that the 06 matches the beginning of the string and that is detected as an error condition in the parsing. If the 06 matched somewhere in the middle of the string, the error would have been discovered later after creating a FTPFile object with a bad name.

It seems as though this problem (or a related problem) would be present anytime the filename (or the first part of a filename if the filename had whitespace in it) matches something else in the directory listing line.

Answer 3

Thanks for pointing this out, although I really don't understand why the Java version should make a difference.

We've actually fixed a similar bug in the UnixFileParser, but didn't apply it to WindowsFileParser. (silly us).

It uses the same splitting technique, but searches for a number of fields, not just the filename. This minimises (pretty much eliminates) the chances of duplication.

You can take a look at UnixFileParser in 1.4.1 to see what I mean.

We'll try the same approach for the WindowsFileParser.

Since we were struggling with parsing issues ourselves (although for us it was the issue of selecting the correct parser for the FTP server), we took a look at your issue as well. I don't know why it works for java 1.4.2, but here is what we found. (The code we are looking at is revision 1.3 of WindowsFileParser.)

The problem appears to be that the parser splits the line based on whitespace. In your case, the filename (directory name) is 06. This splitting is good for breaking out the date, etc, but not good for determining the filename. If the filename has whitespace in it, the name will be split up. To avoid this, the original string is searched, looking for the split string for the filename. For example, if the text was

Answer 4

We have a preliminary build of 1.4.2 that you can test out if you wish - send an email to support at enterprisedt dot com and ask for 1.4.2 beta, and we'll send you the jar file. Please report back.

Thanks for pointing this out, although I really don't understand why the Java version should make a difference.

We've actually fixed a similar bug in the UnixFileParser, but didn't apply it to WindowsFileParser. (silly us).

It uses the same splitting technique, but searches for a number of fields, not just the filename. This minimises (pretty much eliminates) the chances of duplication.

You can take a look at UnixFileParser in 1.4.1 to see what I mean.

We'll try the same approach for the WindowsFileParser.

Since we were struggling with parsing issues ourselves (although for us it was the issue of selecting the correct parser for the FTP server), we took a look at your issue as well. I don't know why it works for java 1.4.2, but here is what we found. (The code we are looking at is revision 1.3 of WindowsFileParser.)

The problem appears to be that the parser splits the line based on whitespace. In your case, the filename (directory name) is 06. This splitting is good for breaking out the date, etc, but not good for determining the filename. If the filename has whitespace in it, the name will be split up. To avoid this, the original string is searched, looking for the split string for the filename. For example, if the text was