SPARC T3 reiterates Siebel CRM's Supremacy on T-series Hardware

It's been mentioned and proved several times that Sun/Oracle's T-series hardware is the best fit to deploy and run Siebel CRM. Feel free to browse through the list of Siebel benchmarks that Sun published in the past on T-series:

        2004-2010 : A Look Back at Sun Published Oracle Benchmarks

Oracle Corporation announced the availability of SPARC T3 servers in Oracle OpenWorld 2010, and sure enough there is a Siebel CRM benchmark on SPARC T3-1 server to support the server launch event. Check the following web page for high level details of the benchmark.

        SPARC T3-1 Server Posts a High Score on New Siebel CRM 8.1.1 Benchmark

I intend to provide the missing pieces of information in this blog post.

First of all, it is not a "Platform Sizing and Performance Program" (PSPP) benchmark. Siebel 8.1.1 was used to run the benchmark, and there is no Siebel PSPP benchmark kit available as of today for v8.1.1. Hence the test results from this benchmark exercise are not directly comparable to the Siebel 8.0 PSPP benchmark results.

Workload

The benchmark workload consists of a mix of Siebel Financial Services Call Center and Siebel Web Services / EAI transactions. The FINS Call Center transactions create a bunch of Opportunities, Quotes and Orders, where as the Web Services / EAI transactions submit new Service Requests (SR), search for and update existing SRs. The transaction mix is 40% FINS Call Center transactions and 60% Web Services / EAI transactions.

Software Versions

• Siebel CRM 8.1.1
• Oracle RDBMS 11g R2 (11.2.0.1), 64-bit
• iPlanet Web Server 7.0 Update 8, 32-bit
• Solaris 10 09/10 in the application-tier and
• Solaris 10 10/09 in the web- and database-tiers

Hardware Configuration

• Application Server : 1 x SPARC T3-1 Server (2 RU system)
  • One socket 16-Core 1.65 GHz SPARC T3 processor, 128 hardware threads, 6 MB L2 Cache, 64 GB RAM
• Web Server + Database Server : 1 x Sun SPARC Enterprise T5240 Server (2 RU system)
  • Two socket 16-Core 1.165 GHz UltraSPARC T2 Plus processors, 128 hardware threads, 4 MB L2 Cache, 64 GB RAM

Virtualization Technology

iPlanet Web Server and the Oracle 11g Database Server were configured on a single Sun SPARC Enterprise T5240 Server. Those software layers were isolated from each other with the help of Oracle Solaris Containers virtualization technology. Resource allocations are shown below.

Tier	#vCPU	Memory (GB)
Database	96	48
Web	32	16

Test Results

#vUsers	Avg Trx Resp Time (sec)		Business Trx Throughput/HR		Avg CPU Utilization (%)			Avg Memory Footprint (GB)
	FINS	EAI	FINS	EAI	App	DB	Web	App	DB + Web
13,000	0.43	0.2	48,409	116,449	58	42	37	52	35

Why stop at 13K users?

Notice that the average CPU utilization on the application server node (SPARC T3-1) is only ~58%. The application server node has room to accommodate more online vusers - however, there is not enough free memory left on the server to scale beyond 13,000 concurrent users. That is the main reason to stop at 13,000 user count in this benchmark.

Siebel Best Practices

Check the following presentation:

        Siebel on Oracle Solaris : Best Practices, Tuning Tips

Acknowledgments

Credit to all our peers at Oracle Corporation who helped us with the hardware, workload, verification and validation etc., in a timely manner. Also Jenny deserves special credit for spending enormous amount of time running the benchmark with patience.

Original blog post URL:
http://blogs.sun.com/mandalika/entry/sparc_t3_reiterates_siebel_s

Read More

Is it really Solaris Versus Windows & Linux?

(Even though the title explicitly states "Solaris Versus .. ", this blog entry is equally applicable to all the operating systems in the world with few changes.)

Lately I have seen quite a few e-mails and heard few customer representatives talking about the performance of their application(s) on Solaris, Windows and Linux. Typically they go like the following with a bunch of supporting data (all numbers) and no hardware configuration specified whatsoever.

• "Transaction X is nearly twice as slow on Solaris compared to the same transaction running on Windows or Linux"
• "Transaction X runs much faster on my Windows laptop than on a Solaris box"

Lack of awareness and taking the hardware completely out of the discussions and context are the biggest problems with complaints like these. Those claims make sense only when the underlying hardware is the same in all test cases. For example, comparing a single user, single threaded transaction running on Windows, Linux and Solaris on x86 hardware is appropriate (as long as the type and speed of the processor are identical), but not against Solaris running on SPARC hardware. This is mainly because the processor architecture is completely different for x86 and SPARC platforms.

Besides, these days Oracle offers two types of SPARC hardware - 1. T-series and 2. M-series, which serve different purposes though they are compatible with each other. It is hard to compare and analyze the performance discrimination between different SPARC offerings (T- and M-series) too with no proper understanding of the characteristics of the CPUs in use. Choosing the right hardware for the right job is the key.

It is improper to compare the business transactions running on x86 with SPARC systems or even between different types of SPARC systems, and to incorrectly attribute the hardware strength or weakness to the operating system that runs on top of the bare metal. If there is so much of discrepancy among different operating environments, it is recommended to spend some time understanding the nuances in testing hardware before spending enormous amounts of time trying to tune the application and the operating system.

The bottomline: in addition to the software (application + OS), hardware plays an important role in the performance and scalability of an application - so, unless the testing hardware is the same for all test cases on different operating systems, don't you just focus on the operating system alone and make hasty decisions to switch to other operating platforms. Carefully choose appropriate hardware for the task in hand.

Read More

OOW 2010 : Accelerate and Bullet-Proof Your Siebel CRM Deployment with Oracle's Sun Servers

The best practices slides from today's OpenWorld presentation can be downloaded from the following location.

        Siebel on Oracle Solaris : Best Practices, Tuning Tips

The entire presentation with proper disclaimers and Oracle Solaris Cluster specific slides will be posted on Oracle's web site soon. Stay tuned.

Read More

Identifying Ideal Oracle Database Objects for Flash Storage and Accelerators

(Originally posted on blogs.sun.com at:
http://blogs.sun.com/mandalika/entry/identifying_ideal_oracle_database_objects)

The Sun Storage F5100 Flash Array and Sun Flash Accelerator F20 PCIe Card help accelerate I/O bound applications such as databases. The following are some of the guidelines to identify Oracle database objects that can benefit by using the flash storage. Even though the title explicitly states "Oracle", some of these guidelines are applicable to other databases and non-database products. Exercise discretion, evaluate and experiment before implementing these recommendations as they are.

• Heavily used database tables and indexes are ideal for flash storage

  • - The database workloads with no I/O bottlenecks may not show significant performance gains

  • - The database workloads with severe I/O bottlenecks can fully realize the benefits of flash devices

    • Top 5 Timed Foreground Events section in any AWR report that was collected on the target database system is useful in finding whether disk I/O is a bottleneck

      • Large number of Waits and the large amount of time in DB spent waiting for some blocked resource under User I/O Wait Class is an indication of I/O contention on the system

• Identify the I/O intensive tables and indexes in a database with the help of Oracle Enterprise Manager Database Control, a web-based tool for managing Oracle database(s)

  • - The "Performance" page in OEM Database Control helps you quickly identify and analyze performance problems
  • - Historical and the real-time database activity can be viewed from the "performance" page.
    • The same page also provides information about the top resource consuming database objects

• An alternate way to identify the I/O intensive objects in a database is to analyze the AWR reports that are generated over a period of time especially when the database is busy

  • - Scan through the SQL ordered by .. tables in each AWR report
  • - Look for the top INSERT & UPDATE statements with more elapsed and DB times

    • The database tables that are updated frequently & repeatedly, along with the indexes created on such tables are good candidates for the flash devices

  • - SQL ordered by Reads is useful in identifying the database tables with large number of physical reads

    • The database table(s) from which large amounts of data is read/fetched from physical disk(s) are also good candidates for the flash devices

      • To identify I/O intensive indexes, look through the explain plans of the top SQLs that are sorted by Physical Reads

• Examine the File IO Stats section in any AWR report that was collected on the target database system

  • - Consider moving the database files with heavy reads, writes and relatively high average buffer wait time to flash volumes

• Examine Segments by Physical Reads, Segments by Physical Writes and Segments by Buffer Busy Waits sections in AWR report

  • - The database tables and indexes with large number of physical reads, physical writes and buffer busy waits may benefit from the flash acceleration

• Sun flash storage may not be ideal for storing Oracle redo logs

  • - Sun Flash Modules (FMOD) in F5100 array and F20 Flash Accelerator Card are optimized for 4K sector size

    A redo log write that is not aligned with the beginning of the 4K physical sector results in a significant performance degradation

  • - In general, Oracle redo log files default to a block size that is equal to the physical sector size of the disk, which is typically 512 bytes

    • Majority of the recent Oracle Database platforms detect the 4K sector size on Sun flash devices
    • Oracle database automatically creates redo log files with a 4K block size on file systems created on Sun flash devices

      • However with a block size of 4K for the redo logs, there will be significant increase in redo wastage that may offset expected performance gains

F5100 Flash Storage and F20 PCIe Flash Accelerator Card as Oracle Database Smart Flash Cache

In addition to the I/O intensive database objects, customers running Oracle 11g Release 2 or later versions have the flexibility of using flash devices to turn on the "Database Smart Flash Cache" feature to reduce physical disk I/O. The Database Smart Flash Cache is a transparent extension of the database buffer cache using flash storage technology. The flash storage acts as a Level 2 cache to the (Level 1) SGA. Database Smart Flash Cache can significantly improve the performance of Oracle databases by reducing the amount of disk I/O at a much lower cost than adding an equivalent amount of RAM.

F20 Flash Accelerator offers an additional benefit - since it is a PCIe card, the I/O operations bypass disk controller overhead.

The database flash cache can be enabled by setting appropriate values to the following Oracle database parameters.

 db_flash_cache_file
 db_flash_cache_size

Check Oracle Database Administrator's Guide 11g Release 2 (11.2) : Configuring Database Smart Flash Cache documentation for the step-by-step instructions to configure Database Smart Flash Cache on flash devices.

Read More

PeopleSoft NA Payroll 500K EE Benchmark on Solaris : The Saga Continues ..

(Original post is at:
http://blogs.sun.com/mandalika/entry/peoplesoft_na_payroll_500k_ee)

Few clarifications before we start.

Difference between 240K and 500K EE PeopleSoft NA Payroll benchmarks

Not too long ago Sun published PeopleSoft NA Payroll 240K EE benchmark results with 16 job streams and 8 job streams. First of all, I want to make sure everyone understands the fact that PeopleSoft NA Payroll 240K and 500K EE benchmarks are two completely different benchmarks. The 240K database model represents a large sized organization where as 500K database model represents an extra-large sized organization. Vendors [who benchmark] have the flexibility of configuring 8, 16, 24 or 32 parallel job streams (or threads) in those two benchmarks to parallellize the work being done.

Now that the clarifications are out of the way, here is the direct URL for the 500K Payroll benchmark results that Oracle|Sun published last week. (document will be updated shortly to fix the branding issues)

    PeopleSoft Enterprise Payroll 9.0 using Oracle for Solaris on a Sun SPARC Enterprise M5000 (500K EE 32 job streams)

What's changed at Sun & Oracle?

The 500K payroll benchmark work was started few months ago when Sun is an independent entity. By the time the benchmark work is complete, Sun was part of Oracle Corporation. However it has no impact whatsoever on the way we have been operating and interacting with the PeopleSoft benchmark team for the past few years. We (Sun) still have to package all the benchmark results and submit for validation just like any other vendor. It is still the same laborious process that we have to go through from both ends of Oracle (that is, PeopleSoft & Sun). I just mentioned this to highlight Oracle's non-compromising nature on anything at any level in publishing quality benchmarks.

SUMMARY OF 500K NA PAYROLL BENCHMARK RESULTS

The following bar chart summarizes all the published benchmark results by different vendors. Each 3D bar on X-axis represent one vendor, and the Y-axis shows the throughput (#payments/hour) achieved by corresponding vendor. Actual throughput and the vendor name is also shown in each of the 3D bar for clarity. Common sense dictates that higher the throughput, the better it is.

The numbers in the following table were extracted from the very first page of the benchmark results white papers where Oracle|PeopleSoft highlights the significance of the results and the actual numbers that are of interest to the customers. The results in the following table are sorted by the hourly throughput (payments/hour) in the descending order. The goal of this benchmark is to achieve as much hourly throughput as possible. Click on the link that is underneath the hourly throughput values to open corresponding benchmark result.

Oracle PeopleSoft North American Payroll 9.0 - Number of employees: 500,480 & Number of payments: 750,720

Vendor	OS	Hardware Config	#Job Streams	Elapsed Time (min)	Hourly Throughput Payments per Hour
Sun	Solaris 10 10/09	1x Sun SPARC Enterprise M5000 with 8 x 2.53 GHz SPARC64 VII Quad-Core CPUs & 64G RAM 1 x Sun Storage F5100 Flash Array with 40 Flash Modules for data, indexes. Capacity: 960 GB 1 x Sun Storage 2510 Array for redo logs. Capacity: 272 GB. Total storage capacity: 1.2 TB	32	50.11	898,886
IBM	z/OS 1.10	1 x IBM System z10 Enterprise Class Model 2097-709 with 8 x 4.4 GHz IBM System z10 Gen1 CPUs & 32G RAM 1 x IBM TotalStorage DS8300. Total capacity: 9 TB	8*	58.96	763,962
HP	HP-UX B.11.31	1 x HP Integrity rx7640 with 8 x 1.6 GHz Intel Itanium2 9150 Dual-Core CPUs & 64G RAM 1 x HP StorageWorks EVA 8100. Total capacity: 8 TB	32	96.17	468,370

This is all public information. Feel free to compare the hardware configurations and the data presented in all three rows and draw your own conclusions. Since all vendors used the same benchmark toolkit, comparisons should be pretty straight forward.

Sun Storage F5100 Flash Array, the differentiator

Of all these benchmark results, clearly the F5100 storage array is the key differentiator. The payroll workload is I/O intensive, and requires low latency storage for better throughput (it is implicit that less latency means less I/O waits).

There is a lot of noise from some of the outside blog readers (I do not know who those readers are or who they work for) when Sun published the very first NA Payroll 240K EE benchmark with eight job streams using an F5100 array that has 40 flash modules (FMOD). Few people thought it is necessary to have those many flash modules to get that kind of performance that Sun demonstrated in the benchmark. Now that we have the 500K benchmark result as well, I want to highlight another fact that it is the same F5100 that was used in all the three NA Payroll benchmarks that Sun published in the last 10 months. Even though other vendors increased the number of disk drives when moved from 240K to 500K EE benchmark environment, Sun hasn't increased the flash modules in F5100 -- the number of FMODs remained at 40 even in 500K EE benchmark. This fact implicitly suggests at least two things -- 1. F5100 array is resilient, scales and performs consistently even with increased load. 2. May be 40 flash modules are not needed in 240K EE Payroll benchmark. Hopefully this will silence those naysayers and critics now.

While we are on the same topic, the storage capacity in the other array that was used to store the redo logs was in fact reduced from 5.3 TB in a J4200 array that was used in 240K EE/16 job stream benchmark to 272 GB in a 2510 array that was used in 500K EE/32 job stream benchmark. Of course, in both cases, the redo logs consumed only 20 GB on disk - but since the arrays were connected to the database server, we have to report the total capacity of the array(s) whether it is being used or not.

Notes on CPU utilization and IBM's #job streams

Even though I highlighted the I/O factor in the above paragraph, it is hard to ignore the fact that the NA Payroll workload is CPU intensive too. Even when multiple job streams are configured, each stream runs as a single-thread process -- hence it is vital to have a server with powerful processors for better [overall] performance.

Observant readers might have noticed couple of interesting things.

1. The maximum average CPU usage that Sun reported in 500K EE benchmark in any scenario by any process is only 43.99% (less than half of the total processing capacity)

   The reason is simple. The SUT, M5000, has eight quad-core processors and each core is capable of running two hardware threads in parallel. Hence there are 64 virtual CPUs on the system, and since we ran only 32 job streams, only half of the total available CPU power was in use.

   Customers in a similar situation have the flexibility to consolidate another workload onto the same system to take advantage of the available/remaining CPU cycles.

2. IBM's 500K EE benchmark result is only with 8 job streams

   I do not know the exact reason - but if I have to speculate, it is as good as anyone's guess. Based on the benchmark results white paper, it appears that the z10 system (mainframe) has eight single core processors, and perhaps that is why they ran the whole benchmark with only eight job streams.

Also See:

Benchmark Results White Papers

• PeopleSoft Enterprise Payroll 9.0 using Oracle for Solaris on a Sun SPARC Enterprise M4000 -- 240K EE 16 stream benchmark
• PeopleSoft Enterprise Payroll 9.0 using Oracle for Solaris on a Sun SPARC Enterprise M4000 -- 240K EE 8 stream benchmark
  

  Best Practices White Paper

• Best Practices for Oracle PeopleSoft Enterprise Payroll for North America using the Sun Storage F5100 Flash Array or Sun Flash Accelerator F20 PCIe Card
  

  Blogs

• Expensive non-performance by Joerg Moellenkamp
• PeopleSoft NA Payroll 240K EE Benchmark with 16 Job Streams : Another Home Run for Sun
• PeopleSoft North American Payroll on Sun Solaris with F5100 Flash Array : A blog Reprise
• App benchmarks, incorrect conclusions and the Sun Storage F5100
• Oracle PeopleSoft Payroll (NA) Sun SPARC Enterprise M4000 and Sun Storage F5100 World Record Performance

Read More

Book Review: Oracle Database 11g – Underground Advice for Database Administrators

(06/15/2010: This blog post will be edited multiple times to add reviews for the remaining chapters in the book.)

Author: April C. Sims
Publisher: Packt
Target Audience: Oracle Database Administrators

Chapter #1 "When to step away from the keyboard" starts off with an interesting example, cautions the DBAs to be self-restraint but encourages to do the right thing at the end of the day. I liked the idea of listing out a whole bunch of graphical and command line Oracle tools [with brief descriptions] that an Oracle DBA may need in performing some of the day-to-day activities. Also couple of pages were dedicated to list out various tasks performed by Oracle DBAs on a daily, weekly, monthly, quarterly & yearly basis. It was interesting. And finally the chapter concludes with a bunch of useful tips for the administrators to avoid making unwanted errors.

The only thing that probably didn't fit in this chapter is the very brief discussion on staying away from dinosaurs. In my opinion, it is completely off-topic.

Chapter #2 "Maintaining Oracle Standards" is available for download. Get it from this location and read it yourself. You be the judge.

Chapter #3 "Tracking the Bits and Bytes". The first half of the chapter talks about Oracle Data Block and the methods to view the data at the block level, the finest level of granularity that contains the actual data. The author tried and succeeded with a decent follow up that briefly explains how transaction integrity is maintained in Oracle database. The hands on exercise makes the reader sweat a little, but may help understand the material that was presented earlier, better. The key is to focus and try to understand what is happening when running all those scripts and commands. I would like a much simpler example though. Admittedly this is not something that Oracle administrators do everyday, but it does not hurt to gain some insight into Oracle internal workings and to be prepared to leverage this knowledge when disaster strikes.

The second half of the chapter was dedicated for Log Miner, a PL/SQL package utility that can be used to extract the database transactions that have been executed over a period of time. April did a nice job briefly explaining why protecting the [physical] redo, undo and the archive log files is very important -- to keep the data & database transactions from falling into the wrong hands. An example using "Flashback Transaction Blackout" method was shown to demonstrate how to use log miner utility to retrieve the changes that were done to the database few minutes ago.

I am not impressed with the example in page 92 in section Identifying data in undo segments by flashing back to timestamp. There are a bunch of SQL statements in the example with no output from a test environment. I strongly believe that showing the actual output keeps the material interesting and easy to follow.

Also I did not like the idea of pointing to blogs and random web sites, as they may disappear any time without a warning.

To be continued ..
________________
Technorati Tags:
Oracle | Database | RDBMS | DBA | Book | Review | Packt

Read More

Oracle RDBMS : Flushing a Single SQL Statement out of the Object Library Cache

It is well known that the entire shared pool can be flushed with a simple ALTER SYSTEM statement.

SQL> ALTER SYSTEM FLUSH SHARED_POOL;

System altered.

What if the execution plan of a single SQL statement has to be invalidated or flushed out of the shared pool so the subsequent query execution forces a hard parse on that SQL statement. Oracle 11g introduced a new procedure called PURGE in the DBMS_SHARED_POOL package to flush a specific object such as a cursor, package, sequence, trigger, .. out of the object library cache.

The syntax for the PURGE procedure is shown below.

procedure PURGE (
        name VARCHAR2, 
        flag CHAR DEFAULT 'P', 
        heaps NUMBER DEFAULT 1)

Explanation for each of the arguments is documented in detail in $ORACLE_HOME/rdbms/admin/dbmspool.sql file.

If a single SQL statement has to be flushed out of the object library cache, the first step is to find the address of the handle and the hash value of the cursor that has to go away. Name of the object [to be purged] is the concatenation of the ADDRESS and HASH_VALUE columns from the V$SQLAREA view. Here is an example:

SQL> select ADDRESS, HASH_VALUE from V$SQLAREA where SQL_ID like '7yc%';

ADDRESS   HASH_VALUE
---------------- ----------
000000085FD77CF0  808321886

SQL> exec DBMS_SHARED_POOL.PURGE ('000000085FD77CF0, 808321886', 'C');

PL/SQL procedure successfully completed.

SQL> select ADDRESS, HASH_VALUE from V$SQLAREA where SQL_ID like '7yc%';

no rows selected

Note to Oracle 10g R2 Customers

The enhanced DBMS_SHARED_POOL package with the PURGE procedure is included in the 10.2.0.4 patchset release.

10.2.0.2 and 10.2.0.3 customers can download and install RDBMS patch 5614566 to get access to these enhancements in DBMS_SHARED_POOL package.

Also see:

• Oracle Support Document ID 457309.1 "How To Flush an Object out the Library Cache [SGA]"
• Oracle Support Document ID 751876.1 "DBMS_SHARED_POOL.PURGE Is Not Working On 10.2.0.4"
• DBMS_SHARED_POOL.PURGE() procedure documentation

(Original post is at:
http://blogs.sun.com/mandalika/entry/oracle_rdbms_flushing_a_single)

Read More