More horsepower doesn't always mean better performance
Sergiy is CTO of SA Consulting and can be contacted at [email protected].
The era of 64-bit computing started with the release of the Alpha processor by Digital Equipment Corp. in 1992. Later, several other major computer hardware companies, namely Hewlett-Packard, Fujitsu, Sun Microsystems, and IBM, moved into the 64-bit market with their offerings. In 1995, the Fujitsu-owned HAL Computer launched the industry's first workstations based on a 64-bit SPARC CPU, SPARC64. Shortly after HAL's announcement, Sun launched the long expected Ultra 1 and 2 workstations, which used Sun's 64-bit UltraSPARC processor. In 1997, IBM released RS64, the first 64-bit PowerPC RISC chip. In 1998, IBM supplemented RS64 with a 64-bit SMP chip, POWER3. It took another five years for the 64-bit computing to come to mass market.
In this article, I examine two modern 64-bit platforms widely available in the sub-$5000 range—the AMD64 and PowerPC64. In the process, I evaluate the performance of 32- and 64-bit Java Virtual Machines from two major JVM vendors, Sun Microsystems and IBM, using the SPECjvm98 and SPECjbb2000 benchmarks from Standard Performance Evaluation Corp. (http://www.spec.org/).
The AMD64 is a 64-bit platform from Advanced Micro Devices (AMD) that extends the industry-standard x86 instruction set architecture. It was designed to deliver full compatibility with existing x86 applications and operating systems without paying performance penalties when working in 32-bit mode. In April 2003, AMD announced the availability of Opteron, the first processor supporting AMD64 architecture.
In 2003, IBM introduced PowerPC 970, a single-core processor that was derived from IBM's POWER4 dual-core CPU and brought processing power of the 64-bit PowerPC architecture to desktops and low-end servers. Additionally, the PowerPC 970, like the POWER4, was also able to process 32-bit instructions natively without any performance penalty. Soon after this, in August 2003, Apple Computer started shipping Power Mac G5 computers featuring PowerPC 970.
Java Backgrounder
Around the time when the first 64-bit processor came to life, the history of Java technology started. Java is a robust, general-purpose, object-oriented, architecture-neutral, portable, secure, multithreaded programming language, with implicit memory management. Java's object-oriented features are mostly the same as C++, with the addition of interfaces and extensions for more dynamic method resolution. Unlike C++, Java does not support operator overloading, multiple inheritance, or automatic type coercion. Robustness is mostly achieved by extensive dynamic (runtime) checking and a built-in exception-handling mechanism. The Java compiler generates bytecode instructions that are independent of any specific architecture, and thus provides architecture neutrality. Additional portability is achieved by specifying the sizes of the primitive data types and the behavior of arithmetic operators on these types. For example, int always means a signed two's complement 32-bit integer, and float always means a 32-bit IEEE 754 floating-point number. Java also has a set of synchronization primitives that are based on the widely used monitor and condition variable paradigm. Automatic garbage collection (GC) simplifies the task of Java programming and dramatically decreases the number of bugs, but makes the system somewhat more complicated.
In 1991, one year before Digital Equipment Corp. introduced the first 64-bit processor, Sun Microsystems initiated "the Green Project." The goal of the project was to anticipate and plan for the "next wave" of computing. The initial conclusion of the project was that the world would soon see the fusion of mobile digital devices and computers. In the summer of 1992, the Green Team presented *7 ("StarSeven"), a working demo of an interactive handheld entertainment device controller with an animated touch-screen user interface. The device was able to control a number of different platforms by using Oak, an entirely new programming language. The main feature of Oak, developed by James Gosling, was that it was a completely processor-independent language. During the next several years, the language was retargeted for the Internet and later became known as "Java." The name "Oak" was dismissed because of the copyright issues.
In May 1995, Sun formally announced Java, a programming language that lets developers write a program once and run it on multiple operating systems and hardware platforms ("write once, run anywhere"). In 1996, Sun released the Java Development Kit (JDK 1.0), and shortly thereafter, 10 major operating-system vendors announced their plan to distribute Java technology with their products—including Microsoft, which licensed Java from Sun for five years at a cost of approximately $3.75 million per year. In October 1996, Sun announced the first Just-in-Time compiler for the Java platform. JDK 1.1, shipped in February 1997, was downloaded more than 220,000 times within the next three weeks after its release. By the beginning of the next year, this number reached 2 million.
At the end of 1998, the Java 2 platform was released. Roughly half a year later, in the middle of 1999, Sun announced the three editions of the Java platform: J2ME (Java 2 Micro Edition), for mobile, wireless, and other resource-constrained environments; J2SE (Java 2 Standard Edition), for desktop environments; and J2EE (Java 2 Enterprise Edition), for Java-based application servers. J2EE laid out a framework for a number of Java development technologies that have already gained widespread use, such as Enterprise JavaBeans (EJB) and JavaServer Pages (JSP). The next upgrade to Java technology, J2SE 1.3, appeared in May 2000 and several weeks later it gained industry support from Apple with Mac OS X.
J2SE 1.4 was released in February 2002 and was a major new release of the Java platform. It contained 62 percent more classes and interfaces than J2SE 13. Among other features, it provided extensive XML support, support for secure sockets (using the SSL and TLS protocols), new I/O API, logging, regular expressions, and assertions.
In September 2004, the most recent release of Java, J2SE 5.0 (internal version number 1.5.0) codename "Tiger," became publicly available. Tiger contains the first significant updates to Java since its 1.0 release in 1996, including support for generics, autoboxing and autoboxing of primitive types, enhanced for loops, enumerated types, formatted I/O, and varargs.
The Java Virtual Machine (JVM) is a specification for software responsible for running Java programs compiled into a special instruction set—Java bytecode. The JVM is an abstract computing machine and is responsible for Java hardware and OS independence, the small size of Java compiled code, and has the ability to prevent malicious programs from executing. The Java Virtual Machine does not assume any particular implementation technology, hardware, or operating system. There are several JVM components, whose 32-bit and 64-bit version performance may differ, adding to the general performance difference between 32-bit and 64-bit binaries. Among other things, for example, they include the Just-in-Time (JIT) compiler and garbage collection (GC).
The JIT compiler has been a part of JVM since JDK 1.0.2, when Java was viewed only as a client-side technology. The JIT compiler implements dynamic translation of Java bytecode to hardware machine code before execution. The idea behind JIT is that Java bytecode is smaller and easier to compile than the original Java source code. The result is that the time spent compiling Java bytecode on any platform to machine code is much less than the time to compile machine code straight from the Java source. In addition, JITed code can run as fast as statically compiled code. In 32-bit and 64-bit JVMs, the corresponding JITs take somewhat different time to compile Java bytecode to the final actual machine code and can apply different optimizations, affecting the total performance difference between the two versions both in client- and server-side applications.
Garbage collection is an automatic memory-management system, which reclaims memory no longer needed by objects. From the point of view of software engineering, this provides one of the biggest advantages of Java—programmers can forget about low-level memory-management details. Garbage collection also removes the two big sources of bugs: incomplete deallocation (memory leaks) and premature deallocation (corrupted pointers). Garbage collection accounts for a significant portion of the running time of the Java application, since it has to be performed regularly to free the Java heap of inaccessible objects. Since the size of data in the Java heap in 32-bit and 64-bit differs in one way or another, the garbage collection performance difference also contributes to the general performance difference between 32-bit and 64-bit JVMs.
64-Bit Backgrounder
64-bit computing comprises several key elements, most importantly 64-bit addressing. In practical terms, 64-bit addressing is achieved with 64-bit integer registers (or general-purpose registers in RISC terms). 64-bit registers let 64-bit pointers fit into a single register. The advantage of 64-bit pointers is that they make it possible to address huge (as for present times) amounts of memory. While a 32-bit processor is capable of utilizing only 232 bytes or about 4GB of memory, a 64-bit processor theoretically can address 264 bytes or about 18 billion GB of memory. The practical limit of addressable memory in modern 64-bit systems is usually lower, depending on specific hardware architecture and operating systems. For example, in Linux-based operating systems, the addressable memory is limited to 242 bytes or 4096GB due to the current design of internal Linux kernel data structures. Obviously, this is still good enough to break space limitations of the current 32-bit systems.
The second important aspect of 64-bit computing is 64-bit integer arithmetic. Again, this is a simple consequence of having 64-bit wide integer registers capable of storing much larger integer quantities. The direct result of this may be a significant performance improvement for certain types of applications dealing with intensive integer computations of large data.
The third, but equally important characteristic of 64-bit computing is the use of 64-bit operating systems and applications. Such software must support all 64-bit features of the hardware, including 64-bit addressing and arithmetic. Usually it comes with some extra benefits, such as the ability to operate on more files and larger files (although this also may be a feature of certain 32-bit software).
The benefits of 64-bit computing show up in a number of applications. Database servers use a large address space for scalability, maintaining larger buffer pools, caches, and sort heaps in memory to reduce the volume of I/O they perform. They can also allocate more per-user memory, support many more users, and work with much larger files. Simulation and other computationally intensive programs benefit from keeping much larger arrays of data entirely in memory. Finally, a large group of Java programs—J2EE application servers—have been enjoying the benefits of 64-bit computing for some time now, utilizing modern 64-bit Java implementations.
The major drawback of 64-bit computing comes from the fact that 64-bit binaries are typically larger than their 32-bit counterparts. As a consequence, with a larger final machine code size and a given size of cache and translation lookaside buffer (TLB), the chances of both cache and TLB misses increase. This, in turn, may decrease the performance and negate the 64-bit benefits.
Performance Evaluation
The test systems I used to examine the performance of the 32- and 64-bit JVMs from Sun and IBM are two 64-bit dual-CPU workstations—an AMD64-based Opteron system and the PowerPC64-based Apple Power Mac G5. Both workstations run Linux-based 64-bit operating systems, Fedora Core 2, and a beta version of Terra Soft Solutions Y-HPC accordingly, featuring Linux kernel 2.6.x.
I used the SPECjvm98 and SPECjbb2000 benchmarks from SPEC to gauge the performance of the JVMs. SPECjvm98 measures the client-side performance of Java Virtual Machines using these seven applications:
- _201_compress, a popular compression program.
- _202_jess, a Java version of NASA's CLIPS rule-based expert system.
- _209_db, data management benchmarking software.
- _213_javac, the JDK Java compiler.
- _222_mpegaudio, an MPEG-3 audio decoder.
- _227_mtrt, a dual-threaded program that ray traces an image file.
- _228_jack, a real parser-generator.
SPECjbb2000 (the Java Business Benchmark) is a benchmark for evaluating the performance of server-side Java, which emulates a three-tier system, a common type of server-side Java applications.
With the SPECjvm98 benchmark, I measured the total execution time of every benchmark application in seconds, where the lower value is better. The heap size for all SPECjvm98 benchmarks is varied by the JVM between a minimum heap size of 16MB and maximum heap size of 32MB. In the SPECjbb2000 benchmark, I measured the number of operations per second for three different heap sizes. Here, the higher value corresponds to the higher performance. Each benchmark application was run three times. For final results, the best runs are reported.
Figures 1 and 2 show the performance of 32-bit and 64-bit Linux versions of the Sun Java 2 Standard Edition Development Kit 5.0 (J2SE 1.5.0) in SPECjvm98 and SPECjbb2000 benchmarks on the AMD64 platform. In three SPECjvm98 benchmark applications out of seven—_201_compress, _222_mpegaudio, and _228_jack—the 64-bit version of the JVM shows a better performance than its 32-bit counterpart. In SPECjbb2000, the performance of the 64-bit version is higher only for large enough heap sizes. In the case of 256MB heap size, the fact that the total amount of live data is larger in the 64-bit version of the JVM causes more frequent garbage collections, which decreases the application throughput.
Figures 3 and 4 show the performance of 32-bit and 64-bit versions of the IBM Developer Kit for Linux, Java 2 Technology Edition, Version 1.4.2 GA in SPECjvm98 and SPECjbb2000 benchmarks on the AMD64 platform. For this JVM, a different set of three out of seven benchmark applications—_209_db, _213_javac, and again _228_jack—shows better performance in the 64-bit environment. In the SPECjbb2000 benchmark, the 64-bit version of IBM's JVM does not show better performance than its 32-bit counterpart in any of the three tested heap sizes.
Figures 5 and 6 illustrate the performance of 32-bit and 64-bit versions of the IBM Developer Kit for Linux, Java 2 Technology Edition, Version 1.4.2 GA in SPECjvm98 and SPECjbb2000 benchmarks on the PowerPC64 platform. Here, in all SPECjvm98 benchmark applications and in SPECjbb2000, performance of the 64-bit JVM is worse than the performance of its 32-bit counterpart.
Conclusion
Based on the benchmark results for the PowerPC64 platform running a Linux-based operating system (the only 64-bit OS available for Apple Power Mac G5 today) and IBM JVM, you can conclude that on this platform, any application that does not require 64-bit features should be used on a 32-bit JVM because, in all cases, performance of the 64-bit JVM here is lower than the performance of its 32-bit counterpart.
On the AMD64 platform running a Linux-based OS and both Sun and IBM JVMs, you see that (in the general case) it is hard to predict performance difference for Java apps run on the 32-bit JVMs and their 64-bit counterparts. The performance benefits here are both application and JVM dependent. In a case when maximum performance is required, it is necessary to benchmark a specific application in its specific execution environment to be able to evaluate potential benefits of switching to 64-bit computing.
There are several important things worth noting here. First, although both SPECjvm98 and SPECjbb2000 are industry-standard benchmarks, they are limited in scope. Thus, the obtained results may hold for a large set of Java applications, but not for the whole range. Second, with more and more widespread adoption of 64-bit computing, we can expect continuous improvement of the 64-bit tools, including the 64-bit JVMs, which may further improve their performance. Third, we tested only several specific combinations of hardware platforms, operating systems, and JVMs. Thus, the results of the JVM benchmarks on the 64-bit edition of Windows XP/2003 or the upcoming 64-bit Mac OS X may provide quite different insights.
DDJ
