Our Products:   CompleteFTP  edtFTPnet/Free  edtFTPnet/PRO  edtFTPj/Free  edtFTPj/PRO
Ask a Question

Continual error on uploading many small xml files

0 votes
2.6k views
in .NET FTP by (120 points)
Hi

We have set up a new IIS server 6.0 with FTP service. We can ftp ok with filezilla but it is very slow. But when we are using the FTP code within our application we get lots of errors and this results in very slow transfers and eventually not all files are transferred. We are transferring a series of xml files in directories, nothing huge in file size. Here is a snippet of the debug code and this pattern is repeated over and over..

Also I just discovered that some files that are uploaded are truncated and the whole file has not been uploaded only a part of it. Which with xml files is catastrophic! In fact any file that has an error thrown as below will be truncated this occurs every 5-10 seconds according to the log.

Many thanks for any help. Cheers Tim 

DEBUG [EnterpriseDT.Net.Ftp.FTPClient] 25 Aug 2010 15:50:30.269 : Connecting to 10.62.127.120:21
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.269 : 220 Microsoft FTP Service
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.269 : ---> USER sitefinity_writer
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.269 : ---> USER sitefinity_writer
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.269 : 331 Password required for sitefinity_writer.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.269 : 331 Password required for sitefinity_writer.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.269 : ---> PASS ********
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.269 : ---> PASS ********
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.269 : 230-Warning! Use by any any unauthorized individual (or) for an unauthorized purpose is a violation of Federal and/or State law. Violators will be prosecuted.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.269 : 230-Warning! Use by any any unauthorized individual (or) for an unauthorized purpose is a violation of Federal and/or State law. Violators will be prosecuted.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : 230 User sitefinity_writer logged in.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : 230 User sitefinity_writer logged in.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : ---> TYPE A
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : ---> TYPE A
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : 200 Type set to A.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : 200 Type set to A.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : ---> CWD /xml/careergrp/
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : ---> CWD /xml/careergrp/
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : 250 CWD command successful.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : 250 CWD command successful.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : ---> PASV
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : ---> PASV
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : 227 Entering Passive Mode (10,62,127,120,10,42).
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : 227 Entering Passive Mode (10,62,127,120,10,42).
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : ---> STOR CarGrpAQUA.xml
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : ---> STOR CarGrpAQUA.xml
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.472 : 125 Data connection already open; Transfer starting.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.472 : 125 Data connection already open; Transfer starting.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.675 : 226 Transfer complete.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.675 : 226 Transfer complete.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.675 : ---> PASV
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.675 : ---> PASV
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.675 : 227 Entering Passive Mode (10,62,127,120,10,43).
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.675 : 227 Entering Passive Mode (10,62,127,120,10,43).
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.675 : ---> STOR CarGrpAROMAMASS.xml
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.675 : ---> STOR CarGrpAROMAMASS.xml
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.675 : 125 Data connection already open; Transfer starting.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.675 : 125 Data connection already open; Transfer starting.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.878 : 226 Transfer complete.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.878 : 226 Transfer complete.
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.878 : ---> PASV
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.878 : ---> PASV
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.878 : 227 Entering Passive Mode (10,62,127,120,10,44).
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.878 : 227 Entering Passive Mode (10,62,127,120,10,44).
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.878 : ---> STOR CarGrpAUTOMTVE.xml
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.878 : ---> STOR CarGrpAUTOMTVE.xml
DEBUG [EnterpriseDT.Net.Ftp.FTPControlSocket] 25 Aug 2010 15:50:30.878 : 125 Data connection already open; Transfer starting.
INFO [iTafeSA.PageBuilder.FtpClient] 25 Aug 2010 15:50:30.878 : 125 Data connection already open; Transfer starting.
WARN [EnterpriseDT.Net.Ftp.FTPClient] 25 Aug 2010 15:50:31.409 : Caught exception closing data socket : Unable to write data to the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at System.IO.StreamWriter.Dispose(Boolean disposing)
   at System.IO.StreamWriter.Close()
   at EnterpriseDT.Net.Ftp.FTPClient.PutASCII(Stream srcStream, String remoteFile, Boolean append)

1 Answer

0 votes
by (162k points)
FTP uses a new TCP socket for each data transfer. If you have lots of rapid transfers in a single process, the number of sockets allocated to that process get used up. Because of the way TCP/IP works, it can take 2-4 minutes to free a closed socket - so after a while you start getting these errors.

The only solution is to either 1) slow the transfers down or 2) use another protocol like SFTP which doesn't consume a socket per transfer or 3) zip the files up and transfer as one big file, unzipping at the other end

Categories

EnterpriseDT 2020
...