Putting large file we sometimes get "Permission Denied" at about the one GB location

Question

Using EDTFTPjPro version 5.0.1 - when putting a large multi-GB file, we sometimes get "Permssion Denied" exception at just below the one GB level of bytes arrived at the server.  It's always about the same, but never the exact same number of bytes arrived at server.  I attempted to re-create using a slightly modified version of one of the sample programs, but it would not kick (this was done from my desktop) - the actual process is done server to server.  Due to the volume of traffic, not feasible to turn on logging in Production.  Any ideas as to what could be going on?  As I said - this is a sometime thing. We can always start the put, but it gets that exception thrown.

Answer 1

Without a log it is difficult to say. Is this FTPS or SFTP? You could enable logging at the INFO or ERROR level - that will give minimal logging but will provide some clues. Also, the server log may provide details on the issue if you have access to it.

Comments

We do have the exception - it is a "Permission denied" put exception.  Maybe this has to do with implicit permissions as to file size with the user on the target system?  I just tried again with the demo Java program using the same user id that has the error, and it went fine; file was 1.7  GB.

---

It is SFTP over port 22.  I just checked again and a significant number of the transfers failed at the same exact point of having transferred 989,414,493 bytes.

---

If it fails at this point it is possible there is an issue with rekeying. After about 1 GB, in SFTP new keys are exchanged. What does the log say?

---

I came across this entry: https://github.com/paramiko/paramiko/issues/49  - is that parameter settable in the SSHFTPClient ?

---

No, it isn't applicable. There's not a hard limit. If you can post a server or client error message and stack trace that might help.

---

Still a bit fuzzy, but it is starting to look like it may be a problem with compression.  I have still not been able to duplicate the error outside of Prod, but in setting up the test in our DEV environment, another problem kicked in which was solved by setting compression to false.  The interface still managed to transfer about the same number of bytes, just under a thousand MB.