Time to Take out the Garbage

To understand how to boost the performance of an application server, it is necessary to understand what is meant by Java garbage collection.

Programmers who write programs in C or C++ are allowed to create and delete objects anywhere in memory.  Java does not allow this, instead the Java application runs inside the JVM (Java virtual machine.)  The Java program is not allowed to create objects outside the virtual machine.  Because C and C++ programs can do that, they can crash with memory pointer errors when they try to write to invalid addresses.

With Java, garbage collection is the process by which the Java program frees up memory by making available locations in memory that were previously used to create variables but now have gone out of scope.

What does this mean?  Consider an example.  Suppose you write a subroutine like this:

Function double(y)

x=2

return y*x

End

The variable x, which is equal to 2, goes out of scope after the function has run.  That means there is no longer any reference to it, also called a “pointer.”  The value x is still stored in memory, but the program cannot access it, because the pointer is gone.  Hence this is space that could be erased and used for something else.

The application server knows which addresses inside the JVM are stale.  So when available memory gets low, the application server executes what is called garbage collection.  This causes a performance hit, because all threads stop running (meaning your application stops running) while the application server wipes clean these areas so they can be used again.

Oracle says that if the JVM spends 1% of its time doing garbage collection on a 32 processor system, that results in a 20% loss in throughput. 10% garbage collection results in a 75% performance hit on a 32 processor system.  They define “throughput” as the percentage of time spent not spent in garbage collection.  (I am not sure how they calculate that number.)

The simplest garbage collector (called the “serial garbage collector”) loops through all the data fields in memory and then deletes those for which there is no reference (pointer). That is somewhat slow.  A faster algorithm, the parallel collector, delete objects using the assumption that newer objects are more likely to go out of scope before old ones.  That makes sense if you think, for example, that that object which is the connection to the database servers lasts as long as the program is running.  So you could call it “old.” Then as each data record is read from the database it could be called “young”, because its lifespan is short, as it is replaced with the data record which is read next.

This parallel approach groups objects into what are called “generations,” another reference to time.  When a generation fills up, what is called a minor garbage collection occurs.  Objects in the young generation that cannot be deleted, because there are still in use (meaning there still exists a reference), are moved to what is called the “tenured” generation.  When the tenured generation runs out of room for more data, the JVM does a full garbage collection.  The full garbage collection takes significantly more time than the minor garbage collection, freezing up the application longer.

To tune the JVM to boost performance, you can select which garbage collector to use (meaning which garage collection algorithm) plus the size of the generations relative to the size of the memory allocated to the JVM.

Oracle describes this with an example. You can cause the application server to output to the log garbage collection statistics by adding -verbose:gc option to the command line.  Then the log will look something like this:

[GC 325407K->83000K(776768K), 0.2300771 secs] [GC 325816K->83372K(776768K), 0.2454258 secs] [Full GC 267628K->83769K(776768K), 1.8479984 secs]

The first two garbage collections shown here were minor garbage collections, since they do not say “Full.”  Reading left-to-right, the numbers shown are the memory up by objects before and after the garbage collection.  The total amount of memory allocated to the JVM is shown on the right. In this example, the first line says that data objects consumed roughly 317 MB of memory, before garbage collection ran in that generation.  After that, memory used to store data objects in that generation fell to roughly 81 MB. The JVM in this case is allocated 758 MB to create objects.  This minor garbage collection took 0.23 seconds to execute.  You can see that the major garbage collection took significantly longer:  1.85 seconds.  How much that affected the end-user depends on the type of application, network latency, data in cache, and so forth.

When you set options in the application server for the JVM, you specify which garbage collection algorithm to use with the -XX:+UseSerialGC option.  (That syntax shown here indicates serial, which Oracle says is suited to single processor servers.  You would need to investigate the reasoning behind that before selecting that option.)

The goal of JVM performance tuning should be to minimize pauses due to garbage collection, which is equivalent to saying maximum throughput.  In order to do that, it would be necessary to study this topic in detail.  That task could be assigned to the architect and developers, as they are sitting around waiting for garbage collection to finish.