Query Regarding Complete FTP Server Capabilities

Question

Hello
We are in the process of evaluating the Complete FTP Server to see if it meets our needs.
History:
The company I am currently working for purchased a 1 year license for CompleteFTP Server Professional several years ago. Due to complications listed below the product was never actually used and the organisation let the license expire. I have the enviable task of tracking down the technical issues that caused the product to remain unused several years ago and see if they can be rectified with newer versions of the Server or Client.


We have the following situation:
1. AIX Server will Push a file via SFTP to a Windows Complete FTP Server using its own FTP Client. The File will be in ASCII(Unicode) with Line Endings set to (LF). The expected encoding will be ASCII(Windows) with Line Endings set to (CR LF)

2. A custom java application using EnterpriseDT edtFTPj/PRO will take the file, log details about it and push it to a z/OS mainframe using FTPS. It should be received by z/OS in the EBCDIC format with newline characters converted appropriately

When we tested this situation several years ago (with CompleteFTP Server 4.1) we discovered that the line endings in step 1 were not being converted from (LF) to (CR LF). I believe this was the fault of the FTP client on the AIX server, possibly also the fault of the SFTP protocol as it does not handle Data Types as cleanly.

When we tested step 1 with Line endings manually set to CR LF the entire process worked.
When we left it as above the file appeared on a single line on the z/OS server.
We were not able to put a process in place on the AIX side to manually change the line endings before the file push due to possible conflicts with other legacy applications calling the same SFTP scripts.

Given the above restrictions. Does the newer versions of CompleteFTP Server have the capabilities to automatically change the line endings for incoming files to match ASCII(Windows), either through custom scripts/extensions, overriding whatever Data Type is pushed from the client or by some other method.

If we were to upgrade to CompleteFTP Server Enterprise edition, would the Multi-Protocol Gateway have the capability to convert line endings as the situation above requires?

The answer provided will help us determine whether a license renewal/license upgrade for EnterpriseDT CompleteFTP Server is warranted or if it will continue to sit unused in the organisation.

Note: I do not have permission to make a decision regarding the purchase of an EnterpriseDT license, I can only pass any findings along.

Answer 1

Step 1 should work - but it may be that the SFTP AIX client does not support ASCII transfers. SFTP does not have ASCII and binary mode transfers like FTP does - it is up to the client to transform the EOL chars based on what OS it thinks the server is using.

If the client can't be fixed, there's a couple of options:

1) You are already using edtFTPj/PRO - why not use it for the SFTP client on AIX? It will allow you to transfer in ASCII mode.

OR

2) It should be relatively straightforward to use a process trigger on CompleteFTP to run a script (like unix2dos) to change the line endings to what they should be.

Answer 2

You are already using edtFTPj/PRO - why not use it for the SFTP client on AIX? It will allow you to transfer in ASCII mode.

Unfortunately, Legacy Applications and the cost of development on AIX will likely prevent this. However I will raise it when the decision making process occurs

It should be relatively straightforward to use a process trigger on CompleteFTP to run a script (like unix2dos) to change the line endings to what they should be.

I like this answer.
I will investigate whether anyone attempted to solve the issue with Process Triggers back in the day.

In the meantime. Would the MPG work with the scenario mentioned earlier? Or would it still require the Process Triggers to be set up.

Answer 3

Unfortunately, Legacy Applications and the cost of development on AIX will likely prevent this. However I will raise it when the decision making process occurs

Just so you know, edtFTPj/PRO includes the ability to run FTP scripts from the command-line - see here

In the meantime. Would the MPG work with the scenario mentioned earlier? Or would it still require the Process Triggers to be set up.

Yes you can use the MPG, but you have to keep in mind that data passes straight through CompleteFTP without ever being written to disk. Therefore, if you need to convert line-endings, you'd need to develop a custom file-system extension that does that on-the-fly. The process triggers would still work, so you can keep doing logging and whatever else you need in there. Or you could do that in the extension.

If, however, you use edtFTPj/PRO on the AIX box, and therefore (hopefully) don't need to convert line-endings, then the MPG should work as required without the need for a custom extension.

- Hans (EnterpriseDT)

Answer 4

Thank you for the very quick responses

Answer 5

Thank for for the assistance so far, it is looking very likely that CompleteFTP Server will be the solution chosen going forward.

I am commencing an analysis of the existing CompleteFTP Server product and noticed that the config file for version 4.1/5 of the Server consisted of a .sdf file that could be opened by any good SQL Server CE file viewer.

I also noticed that the .sdf file had no password access requirement on it.

I would like to know if this has changed in the latest version of the completeFTP product, if so, is it an internal password, or a password that can be changed by an administrator User.
Does the structure of the SQL CE database change significantly between versions?


I have a requirement to monitor Server Certificates and notifying appropriate people prior to expiry.
Can Triggers or some other system be used within Complete FTP Server to notify users/administrators when Certificates are due to expire?

If there is no in-built functionality within Complete FTP than I am contemplating writing a program to extract the certificate information directly out of the .sdf file and read it.

Answer 6

There's no password on config.sdf - it is essentially just a configuration file. If someone has access to it, they already own your server.

The structure doesn't change too much - mostly an extra field or some additional data. You can find out the exact changes between versions by inspecting the SQL in

C:\Program Files (x86)\Complete FTP\Server\Sql\Config

There's currently no trigger system to notify admins of when certificates are due to expire. We've made a note of this suggestion and will have a think about how this might be implemented. There should be no problem writing a script to check the certificates - the certificate fields are obvious and won't change. You can write something to extract to a file, and then use OpenSSL to parse them.

Answer 7

There's currently no trigger system to notify admins of when certificates are due to expire. We've made a note of this suggestion and will have a think about how this might be implemented. There should be no problem writing a script to check the certificates - the certificate fields are obvious and won't change. You can write something to extract to a file, and then use OpenSSL to parse them.

That was my plan. I thought I should verify the password/structure and the like first before I start designing/developing a system based on undocumented (well not publicly documented) configuration file structures.

Answer 8

Well.
I have gotten the request submitted to see about re-licensing the server so we can update it.

In the meantime.
http://www.enterprisedt.com/products/co ... acros.html
has no mention of %WindowsPath%
Yet
http://www.enterprisedt.com/products/co ... pting.html

mentions %WindowsPath%, as does the version history.

Is my understanding correct that %FilePath% will return the FTP Filesystem path whereas %WindowsPath% will return where the file is physically located on the hard drive?

Answer 9

That understanding is correct.

We'll amend the list to include %WindowsPath%

Answer 10

Regarding the Enterprise edition. Especially with regards to Clustering.
I note it stated that clustered sites could be used for failover or load balancing.
Is there anything in place for automatic load-balancing or failover? Or does it come down to the connecting FTP client application pointing to the correct IP Address.
I am curious to see if we should be requesting a license upgrade rather than a license renewal in order to address some high availability concerns.

Also:
http://www.enterprisedt.com/products/co ... tings.html
specified a %FileLength% variable with the following description: "Length of the file in bytes, kB, MB or GB"

Can we use the formatting options to force the FileLength to always return bytes? So far when playing with the trial version of the software I have not been able to accomplish this.