Large File OutOfMemoryError

Question

Is there any trick to transferring (get) large files greater than 1GB? I keep getting the error. This is version 1.5.2.

Answer 1

Can't say we've tried files > 1 GB. Post the stack trace.

Is there any trick to transferring (get) large files greater than 1GB? I keep getting the error. This is version 1.5.2.

Answer 2

All I get is

Exception in thread main java.lang.OutOfMemoryError.

I am executing with -Xms1024m -Xmx1024m. 2048m gives the same result.

Answer 3

Post a snippet of your code that exhibits the problem

All I get is

Exception in thread main java.lang.OutOfMemoryError.

I am executing with -Xms1024m -Xmx1024m. 2048m gives the same result.

Answer 4

OK, here it is. It is essentially the Demo.java with a few changes.

import com.enterprisedt.net.ftp.FTPClient;
import com.enterprisedt.net.ftp.FTPMessageCollector;
import com.enterprisedt.net.ftp.FTPTransferType;
import com.enterprisedt.net.ftp.FTPConnectMode;
import com.enterprisedt.util.debug.Level;
import com.enterprisedt.util.debug.Logger;

public class DownloadCronus {

/**
* Log stream
*/
private static Logger log = Logger.getLogger(DownloadCronus.class);

/**
* Standard main()
*
* @param args standard args
*/
public static void main(String[] args) {

// we want remote host, user name and password
if (args.length < 3) {
usage();
System.exit(1);
}

// assign args to make it clear
String host = args[0];
String user = args[1];
String password = args[2];

Logger.setLevel(Level.ALL);

FTPClient ftp = null;

try {
// set up client
ftp = new FTPClient();
ftp.setRemoteHost(host);
FTPMessageCollector listener = new FTPMessageCollector();
ftp.setMessageListener(listener);
//ftp.setAutoPassiveIPSubstitution(true);

// connect
log.info("Connecting");
ftp.connect();

// login
log.info("Logging in");
ftp.login(user, password);

// set up passive ASCII transfers
log.debug("Setting up BINARY transfers");
ftp.setType(FTPTransferType.BINARY);
ftp.setConnectMode(FTPConnectMode.ACTIVE);
ftp.setTransferBufferSize(1000);

ftp.chdir("..");
ftp.chdir("IQR");

// copy file from server
log.info("Getting file");
ftp.get("IKOB.OCT04.MATCHED");

// Shut down client
log.info("Quitting client");
ftp.quit();

String messages = listener.getLog();
log.debug("Listener log:");
log.debug(messages);

log.info("Test complete");
} catch (Exception e) {
log.error("DownloadCronus failed", e);
}
}

/**
* Basic usage statement
*/
public static void usage() {

System.out.println("Usage: DownloadCronus remotehost user password");
}

}

Answer 5

The problem line is quoted. This method returns a byte[], and you don't have enough memory for a 1 GB byte array.

Presumably you want to download to a file, therefore you need

ftp.get("IKOB.OCT04.MATCHED", "IKOB.OCT04.MATCHED");

to download to a file of the same name. Of course make sure you have enough disk space.

            // copy file from server
            log.info("Getting file");
            ftp.get("IKOB.OCT04.MATCHED");

Answer 6

Thanks. That works.