Memory Performance Testing

Executive Summary
Kingston Technology Co. contracted Mindcraft, an independent testing lab, to perform a quantitative performance study that answered the following questions:

1. How much does Web server performance improve as you add more memory?
   The essential graphic to answer these questions is a chart showing Web server operations per second as a function of the amount of memory. In another chart, we show the effect the amount of memory has on response time.
2. How much more can an application server do with additional memory?
   We answer this question in a chart showing the improvement of application server performance as a function of the amount of memory in the server.
3. How much can a LDAP (Lightweight Directory Access Protocol) directory server do with more memory?
   We chart the results by showing performance as a function of memory size.

Top Of Page

2. Scope of Work

The goal of this benchmark was to show the performance benefits of more memory in Windows NT-based servers. We answer the following questions for the applications we tested:

• How much more work can an application do with each memory configuration?
• When is it better to add memory to a system than to add processors?
• How many users can the system support with this much memory?

Server Computer

For testing purposes, we selected a Compaq ProLiant 5000. This system uses 200MHz Pentium Pro CPUs, and can be configured with one to four processors. We tested the system with 64MB of memory, and we expanded up to 1GB of memory.

• Web Server Testing
  Mindcraft used SPECweb96 to test Web server performance. The file working size was 622MB. We used Microsoft's IIS (Internet Information Server) 4.0 Web server software.
• Application Server Testing
  Mindcraft used the Ziff-Davis ServerBench benchmark to test application server performance as a function of memory size.
  Mindcraft set up the machines to use Windows NT 4.0 as the client operating system. They used 24 client systems simulating a total of 72 clients.
• Directory Server Testing
  Mindcraft used DirectoryMark to test the performance of an LDAP (Lightweight Directory Access Protocol) directory server test.
  Mindcraft used a directory size of 50,000, 200,000 entries, and 400,000 entries

Top Of Page

3. What We Found

Web Server

• Double your memory, cut your response time by more than half.
  A web server houses HTML files that transfer to a client in response to HTTP requests. It can also use applications to generate HTML code. The bottom line is, people don't want to wait around for a slow web page. While performing this benchmark test we found that by increasing the system's memory from 128MB to 256MB, with the system running 1CPU, a user can cut the server response time by 63%. Also, by doubling memory from 256MB to 512MB, a user will improve the system response time by 59%.
  If running a server with 2 CPUs, and system memory is upgraded from 512MB to 1GB, the system will improve response time by 36%.
  By looking at these results we can say:
  Double your memory = cut your response time in half
  Specweb96 Response Time Chart
  
• Increase your server performance by over 500%
  We also measured how many operations per second can be performed on the server as the amount of RAM is increased.
  This benchmark shows that the user can increase web server performance by 540% by upgrading a system from 128MB to 512MB, and running the server with 1 CPU.
  When upgrading from 512 MB to 1GB, and running a server with 2CPUs, a user can increase performance by 51%.
  Specweb96 Operations per second
  
  It is important to notice in both charts that when running a system with 512MB, whether it is running one CPU or two CPUs, performance does not change significantly. This indicates that memory plays the key role in performance increase. These charts show that in this case increasing the amount of RAM on a server will be recommended, rather than increasing the number of CPUs.

Application Server

• Support three to ten times as many clients by increasing your memory
  We tested the application server performance by adding clients in sets of four. We started testing with 1 client, and increased it to a maximum of 72 clients accessing the server at on time.
  Tests demonstrate that increasing your memory in an application server significantly improves the application server performance.
  By increasing base memory from 128MB to 256MB the server was able to support five times as many people before the transactions per second dropped. By increasing memory from 256MB to 512MB the server was able to support ten times as many people.
  So by doubling the server memory a user can dramatically increase the number of clients supported on an application server.
  ServerBench Score
  
  This chart illustrates that the number of clients supported on a server is directly related to the amount of RAM installed in the system.
• Increase the system memory on an application server = increase the number of clients supported
  We also found out that whether running one CPU or two CPUs with 512MB, a performance drop will take place at the same point (see chart below). Once again, this shows that memory plays a very important role on application servers.
  When running two CPUs the highest performance increase happens when 1GB of RAM is added to the system. At this point a server can support the highest amount of clients without showing significant performance drop.
  

Directory Server

We tested three size databases:

• Double your memory, increase your directory server performance by an average of 1000%
  On our first scenario, a 110MB database, we found out that by increasing the base memory from 128MB to 256MB a server will experience a performance increase of 947%. This is a very dramatic performance increase for clients accessing requests from the server.
  50,000 entry database scenario (110MB database)
  
  On our second scenario, a database of 375MB, we found out that by doubling memory from 256MB to 512MB, access request time will improve by 3000%.
  200,000 entry database scenario (375MB database)
  
  Finally, on our third scenario, a database of 690MB, by doubling memory from 512MB to 1GB, system response time will experience a performance increase of 248%.
  400,000 entry database scenario (690MB database)
  

Top Of Page

4. Mindcraft Certification

Mindcraft was commissioned by Kingston Technology Company to produce an independent and unbiased assessment of the benefits of memory on Windows NT Servers.
The tests were conducted on the second quarter of 1998.
These tests should be reproducible by others who use the same test lab configurations as well as the computer software configurations used in this benchmark test.

Top Of Page

5. About Mindcraft

Mindcraft is a service-oriented, independent test lab. The company was founded in 1985 to provide high quality services and products to vendors and end users who want to test software, systems, and network products.
Mincraft is the largest Accredited POSIX Testing Laboratory in the world. As part of their accreditation by The National Voluntary Laboratory Accreditation Program (NVLAP, part of the United States National Institute of Standards and Technology), they have developed a rigorous quality system that meets international standards.
For more information about Mindcraft go to: http://www.mindcraft.com/

Top Of Page

6. About Kingston

Kingston Technology is the world's largest independent manufacturer of memory products for servers, workstations, desktops, portables, and electronic devices. Over the last ten years, Kingston has diversified its product lines to include processor upgrades, flash memory, networking hardware, and storage products. With strictly regulated ISO-registered-facilities in the United States (ISO 9001), Ireland (ISO 9002), and Taiwan (ISO 9002), Kingston markets its products through an extensive worldwide network of distributors, major reseller chains and independent dealers.
In August of 1996, Kingston became part of SOFTBANK Corp. SOFTBANK Holdings Inc. is the holding company for all of SOFTBANK Corporation's U.S.-based activities. Its major operating companies include Ziff-Davis, Kingston Technology Company, SOFTBANK Content Services, and UTStarcom. SOFTBANK is the largest shareholder of Yahoo! and E*Trade, as well as a minority investor in The Rights Exchange, GeoCities, CyberCash, First Virtual Holdings, and E-LOAN. In addition, through affiliated venture funds in the U.S. and Japan, the SOFTBANK Group has made more than 70 investments in Internet companies. Access the SOFTBANK website at http://www.softbank.com/. Visit the Kingston home page on the Internet at http://www.kingston.com/.