Intermittent FTP Failures

Question

Hi,
I have an application which starts by downloading a number of files and getting fileinfos from 3 different folders on a remote FTP server (UNIX). I am currently using edtFTPnet-Free V1.3.0 (previously up until about 2-3 weeks ago I was using edtFTPnet-Free V1.2.3).

Over the last 2 months or so users have been experiencing a large number of FTP failures. With some users consistantly experiencing significantly more failures than others (some high volume users have experienced relatively few failures (myself included) while other lower volume users have experienced many failures). In fact one user may experience problems while at the same time others do not. The FTP server Hosts report no issues with the server over the same period.

I changed the code so that files are now only downloaded if the server file is newer than that on the local drive and one folder of three now returns filtered (at source) fileinfos. (There are a maximum of 9 files to download with the largest approximately 200KB. Total download time if all files are to be downloaded is typically 10-15 seconds. With getting the fileinfos from the 3 folders taking the longest time.).

This new startup scheme has improved the situation somewhat but failures still occur.

I have also produced a smaller test code (with more diagnostics) which some of the users have. This is almost identical to the main application except that it downloads the files to a different local folder.

Today one user reported 5 successive failures with the main code - extract from log file follows:

DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.015 : ---> SIZE ./htdocs/fymfiles_2/FYMYMNames.ini
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.218 : 213 4663
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.218 : ---> MDTM ./htdocs/fymfiles_2/FYMYMNames.ini
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.421 : 213 20080805054633
DEBUG [FTPConnection] 5 Aug 2008 06:21:27.421 : Combining absolute path 'C:\Keim\tmmp\FYM' with relative path 'C:\Keim\tmmp\FYM\server1\FYMYMNames.ini'
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.437 : ---> PASV
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.718 : 227 Entering Passive Mode (213,171,193,5,228,184)
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.718 : Server supplied address=213.171.193.5
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.718 : Server supplied port=58552
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.718 : Substituting server supplied IP (213.171.193.5) with remote host IP (213.171.193.5)
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.718 : NewPassiveDataSocket(213.171.193.5,58552)
DEBUG [FTPControlSocket] 5 Aug 2008 06:21:27.921 : ---> RETR ./htdocs/fymfiles_2/FYMYMNames.ini
ERROR [FTPControlSocket] 5 Aug 2008 06:23:28.453 : Exception in SendCommand : System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.StreamReader.ReadBuffer()
at System.IO.StreamReader.ReadLine()
at EnterpriseDT.Net.Ftp.FTPControlSocket.ReadReply()
at EnterpriseDT.Net.Ftp.FTPControlSocket.SendCommand(String command)
CAUSED BY:
System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
WARN [FTPClient] 5 Aug 2008 06:23:28.593 : SYST command failed - setting Unix as default parser : EnterpriseDT.Net.Ftp.FTPException: The FTP client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
EnterpriseDT.Net.Ftp.FTPException: The FTP client has not yet connected to the server. The requested action cannot be performed until after a connection has been established.
at EnterpriseDT.Net.Ftp.FTPClient.CheckConnection(Boolean shouldBeConnected)
at EnterpriseDT.Net.Ftp.FTPClient.GetSystem()
at EnterpriseDT.Net.Ftp.FTPClient.DirDetails(String dirname)
DEBUG [FTPFileFactory] 5 Aug 2008 06:23:28.593 : Selected UNIX parser

Between each failure he also ran the test code without errors. Equivalent Extract from log file attached:

DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.453 : ---> SIZE ./htdocs/fymfiles_2/FYMYMNames.ini
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.656 : 213 4663
DEBUG [FTPConnection] 5 Aug 2008 06:25:32.656 : Combining absolute path 'C:\Keim\tmmp\FYM' with relative path 'C:\Keim\tmmp\FYM\server2\FYMYMNames.ini'
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.656 : ---> PASV
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.859 : 227 Entering Passive Mode (213,171,193,5,219,2)
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.859 : Server supplied address=213.171.193.5
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.859 : Server supplied port=56066
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.859 : Substituting server supplied IP (213.171.193.5) with remote host IP (213.171.193.5)
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:32.859 : NewPassiveDataSocket(213.171.193.5,56066)
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:33.062 : ---> RETR ./htdocs/fymfiles_2/FYMYMNames.ini
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:33.281 : 150-Accepted data connection
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:33.281 : 150 4.6 kbytes to download
DEBUG [FTPClient] 5 Aug 2008 06:25:33.281 : Transferred 4663 bytes from remote host
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:33.687 : 226-File successfully transferred
DEBUG [FTPControlSocket] 5 Aug 2008 06:25:33.687 : 226 0.000 seconds (measured here), 158.22 Mbytes per second

Any Help or suggestions on how to improve this situation would be gratefully received.

thanks,

Jim

Answer 1

I'm seeing the same issue as of late. I'm using verion 4.1.1 which isn't the latest version, but having the same types of Exceptions with much larger files. I'm transfering files that can be upto 1 gig in size. I've had to rework code to catch the SocketException and reprocess if something failes. The problem with this was I had a finally block of code which was trying to close the FTP Connection which was outside of my try/catch block so periodically my thread would exit execution and I would need to restart the service. I have sense added in another layer to catch this Exception and reprocess as usual.

Answer 2

This is always a bit of a problem with the FTP protocol - because it is always opening sockets for transfers and listings, it isn't always that reliable.

The most recent version of edtFTPnet/PRO (5.1.1) now has a retry facility that automatically retries and resumes transfers in the event of this kind of failure.

Answer 3

I believe I have paid for a copy up through the 5.0 version but not sure about the 5.1 version. How far back does retry functionality go and are there any major changes in the exposed interfaces between the 4.x and the 5.x libraries?

Answer 4

Retry was only introduced in 5.1.0. Check your purchase email for how long you can get upgrades - a minimum of 12 months, or forever with Lifetime Support/Updates. Or email us with your details.

There are plenty of changes, but recompilation should be all that is required.