Odd…I’ve never seen this behavior..

I ran the following program on a win xp sp2 with 2 GB ram. I gave the VM 700 MB and asked it to allocate 670 MB and it was able to consistenly allocate that memory in 1 chunk. In your case, you are allocating multiple chunks of memory which has its memory managment overheads but I would think that it would not be as bad as you state.

I tested the program in 1.4, 1.5 and 1.6 of sun’s jdk and all of them pretty much behave similarly.

here’s the program.

import java.lang.Runtime;
import java.text.DecimalFormat;

public class Test {
public static void main(String [] args) {
//byte [] array = new byte[1024 * 1024 * 700];

if (args.length == 0) {
usage();
}

final int count = Integer.valueOf(args[0]);
System.out.println(“alloc.value = ” + args[0]);

byte [] array = new byte[count];
System.out.println(“jvm version ” + System.getProperties().getProperty(“java.vm.version”));
System.out.println(“total memory = ” +
(new DecimalFormat(“###,###,###.##”)).format(Runtime.getRuntime().totalMemory())
+ “\nfree memory = ” +
(new DecimalFormat(“###,###,###.##”)).format(Runtime.getRuntime().freeMemory()));
}

private static void fail(String msg) {
System.err.println(msg);
System.exit(-1);
}

private static void usage() {
fail(“ERROR : USAGE Test “);
}
}

Our solution was to create a ChunkedByteBuffer that allocates data in 64k chunks and throws an interface in front that decorates it as a standard list. Using the chunked methodology we are able allocate 1.2gigs of the 1.4gig available on the heap.

For now the code is buried in our commercial project, but I’ll chat with our project lead if there is any interest in it.

A second issue is that each JVM will store some amount of metadata per object for GC, locks etc. This will steal away some heap space. The amount may vary with JVMs, so again you may want to try a different one.

The latest 1.4 version of JRockit can be downloaded here and it is free (as in beer):
http://commerce.bea.com/products/weblogicjrockit/jrockit_prod_fam.jsp

– Henrik

Nice to know about your technique. Can you post some code ? It will be nice to add it to your toolkit.

But, I was successfully able to allocate a single array of 670,000,000(670 MB) bytes when the JVM was given a -Xmx value of 700 MB. And I was using a Sun jdk 1.4.x, 1.5.x and 1.6x.

I can understand that if the heap gets badly defragmented over time, then I might NOT be able to allocate the above chunk.

Any idea why some of us are seeing limitations of 300-400 MB ?

Thanks,

BR,
~A

I further noticed that I could allocate upto 1425 MB for the JVM on my windows XP box. The java program itself could allocate 1350MB from the JVM.

I could not increase it beyond that on Windows, though. The ONLY way I know that could increase it beyond that would be to add the /3GB switch to the operating system boot.ini file which should let me allocate about 2.4 GB for the java process.

BR,
~A