SFTP - Renamed File - then tried to Rename again - code=2

Question

Hi.

I am trying the following

ftp.connect
ftp.download(filename)
ftp.rename(ftp.path,ftp.path & ".processing")
...process downloaded file
ftp.rename(ftp.path & ".processing",otherftppath)
ftp.close

I am getting an error on the last rename there:

ALL [SFTPMessage] 23 Sep 2009 17:57:30.247 : Reply : Name=SSH_FXP_STATUS,Type=101,RequestID=17
errorCode=2
errorMessage=No such file or directory, file: /pub/company/org/DEV/apath/test of variant.XLSX.processing
language=en

now - if I go look on the SFTP server - the file does exists.
I know I have been generic here - but the data is sensitive to my customer - so i cant share too much .

Sometimes I get this error - sometimes I dont - but its too consistent to ignore.

Thoughts?

Answer 1

I don't really follow what you're doing here. It looks like you are:

1. downloading the file to your local disk
2. renaming the file on the remote machine by appending '.processing'
3. processing the file on your local disk
4. renaming the file on the remote machine by removing '.processing'

This doesn't make any sense to me. Why would you rename the remote file while you're processing the local file? Unless (possibly) you are going to upload it again after you've processed it. Did you leave something out?

Answer 2

I don't really follow what you're doing here. It looks like you are:

1. downloading the file to your local disk
2. renaming the file on the remote machine by appending '.processing'
3. processing the file on your local disk
4. renaming the file on the remote machine by removing '.processing'

This doesn't make any sense to me. Why would you rename the remote file while you're processing the local file? Unless (possibly) you are going to upload it again after you've processed it. Did you leave something out?

does it really matter why i am doing it?

ok, here is why - the Customer owns the remote system - they want to keep all their files on their SFTP site
I have a process that fires on a timer - to go and look for Files on the SFTP server with a certain extension
My process is threaded - meaning there is a slight chance I could be working on one file from the SFTP site - (i.e processing) when another thread kicks off to look for files - so to prevent getting the same file again - it iis renamed on the SFTP site to Processing.

When completed - the customer wants the orifinal file moved to a Completed or FAiled folder - depending on how the processing went.

make sense?

Answer 3

The fact that you have multiple threads competing for files may be the reason for the problem. It seems to me that the gap between the download and the rename leaves open the possibility that two threads could be trying to rename the same file at the same time. In other words, what's to stop two threads from starting to download the same file at the same time. This in itself wouldn't cause a problem, but when they've finished they both try to rename the file. Since that can only be done once, the second one will fail.

If this is the problem then it could be solved by either (1) renaming the file before downloading it or (2) keeping a list of files that are currently being processed in memory and using that to prevent clashes instead of your ".processing" scheme.

BTW, sorry for the petulant tone of my last message.

Answer 4

The fact that you have multiple threads competing for files may be the reason for the problem. It seems to me that the gap between the download and the rename leaves open the possibility that two threads could be trying to rename the same file at the same time. In other words, what's to stop two threads from starting to download the same file at the same time. This in itself wouldn't cause a problem, but when they've finished they both try to rename the file. Since that can only be done once, the second one will fail.

If this is the problem then it could be solved by either (1) renaming the file before downloading it or (2) keeping a list of files that are currently being processed in memory and using that to prevent clashes instead of your ".processing" scheme.

BTW, sorry for the petulant tone of my last message.

currently - i am controlling the threads - so - it is single threaded.

I didnt have this issue when I ran NON-SFTP - i was able to do the rename and continue to do other FTP commands with no issue.

I have noticed - that i can do all sorts of the FTP commands -(GetFileInfos(), DownloadFile(), Rename() ) but as soon as I do the Rename() - any command i try to run after that fails.
I have tried to even start a new FTP object and connection - to no avail.

I am trying tto get into production with this code - and currently - this is the only thing preventing me from it
I will try to whip up a sample VB.NET unit test - anywhere I can post it?

Answer 5

Can you please e-mail a DEBUG level log file (zipped up) to support@enter...

Thanks

Answer 6

The fact that you have multiple threads competing for files may be the reason for the problem. It seems to me that the gap between the download and the rename leaves open the possibility that two threads could be trying to rename the same file at the same time. In other words, what's to stop two threads from starting to download the same file at the same time. This in itself wouldn't cause a problem, but when they've finished they both try to rename the file. Since that can only be done once, the second one will fail.

If this is the problem then it could be solved by either (1) renaming the file before downloading it or (2) keeping a list of files that are currently being processed in memory and using that to prevent clashes instead of your ".processing" scheme.

BTW, sorry for the petulant tone of my last message.

currently - i am controlling the threads - so - it is single threaded.

I didnt have this issue when I ran NON-SFTP - i was able to do the rename and continue to do other FTP commands with no issue.

I have noticed - that i can do all sorts of the FTP commands -(GetFileInfos(), DownloadFile(), Rename() ) but as soon as I do the Rename() - any command i try to run after that fails.
I have tried to even start a new FTP object and connection - to no avail.

I am trying tto get into production with this code - and currently - this is the only thing preventing me from it
I will try to whip up a sample VB.NET unit test - anywhere I can post it?

I got a work-around - but curious - the ftp.ServerDirectory - what is that supposed to be? it seems like it is always the same as the WorkingDirectory.

Answer 7

WorkingDirectory is an obsolete property that's only there for backward compatibility. We renamed the property to ServerDirectory when we introduced the LocalDirectory property, since the name 'WorkingDirectory' was not specific enough.

Answer 8

WorkingDirectory is an obsolete property that's only there for backward compatibility. We renamed the property to ServerDirectory when we introduced the LocalDirectory property, since the name 'WorkingDirectory' was not specific enough.

Thanks

What I found I had to do - was to Change my working directory to the 'root' of the logged in FTP user. even though I was using Fully qualified path names in the Rename() call.