Configuring and Tuning Databases on the Solaris Platform

Description

• Copyright 2002
• Edition: 1st

• Book
• ISBN-10: 0-13-083417-3
• ISBN-13: 978-0-13-083417-1

Using this book, database professionals can dramatically improve the performance and cost-effectiveness of virtually any database application running on the Solaris® platform. Solaris database optimization specialist Allan Packer presents start-to-finish guidance on optimizing all four leading Solaris databases: Oracle, DB2, Sybase, and Informix. Configuring and Tuning Databases on the Solaris Platform brings together best-practice guidelines for every aspect of database tuning on Sun servers. Drawing on years of expertise as a Sun consultant, Packer presents detailed recommendations for data layout, CPU and memory configuration, and system sizing. He introduces a systematic, drill-down monitoring approach designed to help database professionals identify and resolve system bottlenecks that inhibit database performance. Packer demystifies database tuning by providing a quick primer that explains database architecture, key database concepts, and how to use key subsystems such as query optimizers and buffer cache. He describes and critiques each leading database benchmark, offering guidance for using benchmarks -- and not misusing them.



Extras

Related Articles

Drill-Down Monitoring of Database Servers

Sun StorEdge Instant Image 3.0 and Oracle8i Database Best Practices



Sample Content

Online Sample Chapter

Tuning Databases on Solaris: Role of the Buffer Cache

Table of Contents

I. DATABASES ON SUN SERVERS.

1. Sun's Relationship with Database Vendors .

The Nature of the Relationship. Engineering. Marketing. Joint Sales Activities. The Major Database Suppliers. Sun and DB2 for Solaris. Sun and Oracle. Sun and Sybase. Sun and Informix. Other Databases on Sun.

2. The Role of Database Engineering at Sun.

Sun's Database Engineering Group. Related Efforts Within Sun. The Ambassador Program.

3. Solaris Optimizations for Databases.

Scheduling Optimizations. Preemption Control. Dispatch Table Modifications. I/O Optimizations. The pread and pwrite System Calls. Kernel Asynchronous I/O. Unix File System Enhancements. Other I/O Optimizations. Other Optimizations. Compiler Optimizations. Intimate Shared Memory. Dynamic Reconfiguration. Dynamic Intimate Shared Memory. Reconfiguration Coordination Manager.

4. Hardware Architecture and Databases.

Hardware Architectures. Symmetric Multiprocessor (SMP) Systems. Nonuniform Memory Architecture (NUMA) Systems. Massively Parallel Processor (MPP) Systems. Chip-Level Multiprocessing (CLMP) Systems. Databases on SMP and NUMA Systems. Shared-Disk Databases on SMPs. Shared-Nothing Databases on SMPs. Should You Split Up? To Partition or Not to Partition?

II. Database Architecture.

5. Introduction to Database Architecture.

Architecture of Relational Databases. Data Store. Database Engine. Query Optimizer. Database Shared Memory. Logger. Pagecleaners. Database Recovery Process. Locking Subsystem. System Monitor. Command Interface. APIs. Database Monitoring. Process Models. 2n Architectures. Multithreaded Architectures. Parallel Processing. Distributed Databases. Replicated Databases. Transaction Monitors. Transactions and Ad Hoc Queries. User Multiplexing. Transaction Routing.

6. Database Workloads.

Online Transaction Processing (OLTP). Decision Support Systems (DSS). Data Warehouses and Data Marts. Operational Data Stores. Data Mining. Batch Workloads. Online Analytical Processing (OLAP).

7. The Role of the Buffer Cache.

Overview of the Buffer Cache. Monitoring the Buffer Cache. An Acceptable Cache Hit Rate. The Cache Hit Rate Confusion. Cache Hit Rate Guidelines. A Worked Example. Sizing the Buffer Cache. Influence of Buffer Cache Size on Throughput. Influence of Buffer Cache Size on Data Cache Hit Rate. Influence of Page Size on Buffer Cache Effectiveness.

8. The Role of the Database Optimizer.

Query Optimizers. Query Compilation. Query Optimization. Factors Affecting Query Optimization. Optimization Methods. Table Access. Table Join Order. Join Methods. Executing the Query Plan. Reducing the Workload. Data Partitioning. Denormalization. Concatenated Indexes. Exotic Optimizations. Expert Intervention.

9. Oracle Architecture.

Process Model. Memory Management. System Global Area (SGA). Program Global Area (PGA). Physical Data Storage. Tablespaces. Redo Log Files. Control Files. Logical Data Storage. Segments. Extents. Blocks. Partitions. System Tablespace. Parallel Processing. The Parallel Query Option (PQO). Other Parallel Capabilities.

10. Sybase Architecture.

Process Model. Memory Management. Data Cache. Procedure Cache. Physical Data Storage. Logical Data Storage. Tables and Indexes. Table Partitioning. System Databases. Parallel Processing. Adaptive Server IQ with Multiplex. Column Storage. Indexing. I/O and Caches. Parallelism. IQ Multiplex. Interfaces.

11 Informix XPS Architecture.

Process Model. Coservers. Virtual Processors. Memory Management. Buffer Pool. DS Memory. Dynamic Memory Allocation. Page Cleaning. Physical Data Storage. Logical Data Storage. Pages. Extents. Dbspaces. Dbslices. Tables and Indexes. Logical and Physical Logs. Fragmentation. Tblspaces. System Database. Parallel Processing. Query Parallelism. Colocated Joins.

12. DB2 for Solar is Architecture.

Process Model. Memory Management. Bufferpools. Bufferpool Management. Physical Data Storage. Tablespaces. Log Files. Logical Data Storage. System Tablespaces. Parallel Processing. DB2 Universal Database Enterprise Edition (EE). DB2 Universal Database Enterprise-Extended Edition (EEE). Other DB2-Related Products.

III. SIZING AND CONFIGURING SUN DBMS SERVERS.

13. Sizing Systems for Databases.

Basis of a Sizing Estimate. Minimum Requirements. Limitations of Estimates. The Right Questions. What type of workload is it? What is the status of the application? How many users? Do processing requirements vary with work shifts? How are users connected to the database server? What are the response time expectations? What I/O capacity and throughput are required? How much memory is required? What is the expected system growth rate? Using Published TPC Results for Sizing. Using TPC-C to Size Real-World OLTP Servers. Using TPC-D or TPC-R to Size Real-World DSS Servers. Using TPC-H to Size Real-World DSS Servers. Using Remote Terminal Emulators. Summary of Rules of Thumb. A General-Purpose OLTP Sizing Tool. Background. Establishing Metrics. The Search for Simplifying Assumptions. Gathering the Raw Data. Asking the Right Questions. Building a Model. Validating the Model. The Resulting Tool. Conclusions. Bibliography.

14. Configuring Systems.

Solaris Configuration. Memory Interleaving. Benefits of Memory Interleaving. Evaluation of Interleaving.

15. Configuring CPU.

Managing Workloads. Domains. Processor Sets. Resource Management. CPU Performance. Process Binding. Solaris Scheduling Classes.

16. Configuring The Network.

Performance Considerations. Availability Considerations.

17. Data Layout.

Storage Subsystems. Individually Addressable Disks. RAID-Based Arrays. Storage Area Networks (SANs). Network Attached Storage (NAS). Volume Managers. Veritas Volume Manager (Veritas). Solaris Volume Manager (SVM). RAID Manager (RM6). Component Manager. GUI Administration. Relative Performance. Data Layout Technologies. Introduction to RAID. Concatenation. RAID 0: Striping. RAID 1: Mirroring. RAID 0+1: Striping Plus Mirroring. RAID 1+0: Mirroring and Striping. RAID 3: Striping with Dedicated Parity Disk. RAID 5: Striping with Distributed Parity. Summary of RAID Technology. Database Files on UFS. Database Files on Raw Devices. Write Caches. Data Layout Strategies. Efficient Data Layout. Raw Devices vs. UFS. The Right and Wrong Way to Stripe. Placing Data and Indexes. Laying Out Data on a Single Disk. Write Caches for Improved Performance. High Availability. Disk Failure Planning. Performance Implications of Mirroring. Elimination of Single Points of Failure. Other Data Layout Issues. Database File Naming Using Symlinks. Migration Between Raw Devices and File Systems. RAID 5 Performance with Databases. The Implications of RAID for Database Optimizers. Volume Manager Pros and Cons. Storage Cocktails. Data Layout Recommendations. A Worked Example.

IV. PERFORMANCE MONITORING AND TUNING.

18. Troubleshooting Methods.

Problem-Solving Strategy Development. The Nature of the Problem. Possible Contributors to the Problem. Drilling Down to Find the Root Cause.

19. Major Contributors to Poor Performance.

Performance Problem Identification. Poorly Designed Applications. Poor Database Design and Implementation. Problems with the User Environment. Poor Data Layout. The Next Step.

20. System Performance Monitoring Tools.

Basic Solaris Tools. Monitoring Intervals. Other Monitoring Tools.

21. Drill-Down Monitoring.

STEP 1. Monitoring Memory. STEP 2. Monitoring Disks. STEP 3. Monitoring Networks. STEP 4. Monitoring CPUs. STEP 5. Monitoring and Tuning a Database. EXTRA STEP: Checking for ISM.

22. Monitoring and Tuning Oracle.

Managing Oracle Behavior. Running Administrative Commands. Viewing Current Oracle Tunable Parameters. Changing Tunable Parameters for Oracle. Making Dynamic Parameter Changes Persistent. Viewing and Changing Hidden Parameters. Monitoring Error Messages. Using Oracle Enterprise Manager. Monitoring Oracle System Tables. Generating Explain Plans. Calculating the Buffer Cache Hit Rate. Monitoring Oracle with utlbstat/utlestat. The Library Cache. User Connections. Systemwide Wait Events. Latch Wait Events. Buffer Busy Wait Statistics. Rollback Segments. Modified init.ora Parameters. Dictionary Cache Statistics. Tablespace and Database File I/O Activity. Date, Time, and Version Details. Monitoring the Shared Pool. Tuning Oracle. Tuning init.ora. Setting Tunable Parameters for OLTP Workloads. Setting Tunable Parameters for DSS Workloads. Applying Other Tuning Tips. Using Oracle with File Systems. Optimizing Oracle Load Performance. Planning for Indexes. Using an SGA Larger Than 2 Gbytes. Reconfiguring Oracle9i Dynamically. Oracle9i Dynamic System Global Area. How Oracle Chooses Between ISM and DISM. The Benefits of Using Dynamic SGA. Recovering Oracle. The Influence of Checkpoints on Recovery Time. The Influence of Checkpoints on Performance. The v$instance_recovery view. Other Parameters Influencing Recovery.

23. Monitoring and Tuning Sybase.

Sybase ASE Monitoring. Sybase Central Monitor Output. Sybase sp_sysmon Stored Procedure Output. Configuring Sybase ASE. Calculating the Buffer Cache Hit Rate. Monitoring Error Messages. Generating Query Plans. Tuning Sybase ASE. Tuning Memory. Tuning I/O. Tuning the CPU. Tuning Parallel Features. Tuning Other Aspects

24. Monitoring and Tuning Informix XPS.

Informix XPS Monitoring. Examining Informix Utilities. Changing Informix XPS Tunable Parameters. Monitoring Error Messages. Generating Query Plans. Monitoring Buffer Pool Behavior. Monitoring Scan Type. Monitoring Queries. Tuning Informix XPS. Tuning Parallel Features. Tuning Memory. Tuning I/O. Tuning CPU. Tuning Log Buffers. Optimizing Load Performance. Tuning Inter-Coserver Communication.

25. Monitoring and Tuning DB2 for Solaris.

Monitoring DB2. Changing DB2 Tunable Parameters. Monitoring Error Messages. Generating an Explain Plan. Monitoring the Cost of SQL Statements. Monitoring DB2 Processes. Managing Bufferpools. Calculating the Cache Hit Rate. Tuning DB2 for Solaris for DSS Workloads. Choosing a Partitioning Method. Choosing a Page Size. Tuning I/O. Tuning Bufferpools. Tuning Parameters That Influence the Optimizer. Tuning Other Important Parameters. Tuning DB2 for Solaris for OLTP Workloads. Choosing a Page Size. Tuning I/O. Tuning Bufferpools. General Tuning Tips. Balancing Tablespace Activity. Controlling Lock Activity. Configuring Agent Processes. Limiting Open Files.

26. Metrics: How to Measure and What to Report.

Common Performance Metrics. System Metrics. Application Metrics. Application Instrumentation. System and Application Metrics Combined. Choosing Statistics. Scalability Demystified. Lies, Damn Lies, and Statistics. When the Same Thing Isn't. Comparison of Methods. Interval Discrepancies. Data Collection and Presentation Discrepancies. Conclusion and Recommendations.

V. OTHER TOPICS.

27. Benchmarking.

Industry-Standard Database Benchmarks. Usefulness of Database Benchmarks. Introduction to TPC-C. Benefits of TPC-C. Limitations of TPC-C. Introduction to TPC-D, TPC-R, and TPC-H. TPC-D. TPC-R. TPC-H. Introduction to TPC-W. TPC Results in a Competitive Environment. Running Your Own Benchmark. Reasons for Running a Benchmark. Factors That Make a Benchmark Meaningful. Parameters to be Measured. Benchmark Requirements. Running the Benchmark. Competitive Benchmarks. What Often Goes Wrong with Benchmarks? Conclusion

28. Java Interfaces, Middleware, and Databases.

Java in the Database. Java Interfaces to Databases. JDBC. SQLJ. J2EE Middleware. J2EE Middleware Suppliers. The J2EE Framework. JavaServer Pages. The Java Servlet API. XML. Enterprise JavaBeans. Importance of J2EE to End Users. J2EE Availability. Other Related Technologies. In Conclusion.

Appendix A: References.

Preface

PREFACE

Many books have been written on database tuning—I personally own enough volumes on Oracle tuning to fill a small shelf. Very few, though, approach con-figuring, tuning, monitoring, and troubleshooting from the perspective of the system as a whole, treating the database as one of a number of key components. I have set out to approach database configuration and tuning from this broader context.

To my knowledge, this book is the first attempt at tackling database tuning for multiple databases. Since many aspects of database architecture and tuning are common across the major products, I have separated out generic topics such as database concepts, hardware architectures, the buffer cache and the optimizer. I hope that the opportunity to see the big picture and to compare and contrast the different implementations outweighs the inconvenience of having to follow cross-references at times.

This book is also the first published guide to database configuring and tuning for Solaris users (although most of the principles and much of the detail should be applicable to other UNIX systems). I trust that it will enhance your experience of running databases on the Solaris Operating Environment.

My aim has been to identify the highlights. An overwhelming array of statistics are reported by Solaris and the various databases; rather than attempting to define every statistic and tunable parameter, I focus on those likely to have the most impact on common database workloads.

Finally, no one person could claim a full mastery of all the topics covered in this book, and I certainly make no such assertion on my own behalf. I have tried to distill the knowledge I have acquired during my 12 years with Sun—5 years in the field organization as a Systems Engineer specializing in databases, followed by 7 years in Database Engineering (now Performance and Availability Engineering)—and add to it the research and insights of my colleagues.

I was fortunate to join an engineering group that actively pursued engineering relationships with all four database vendors. This involvement has offered me direct participation in performance projects with the engineering groups at Oracle in Redwood Shores, IBM at the DB2 Development Lab in Toronto, and Sybase at Emeryville. My contact with Informix has been peripheral, and in writing the Informix chapters I have relied heavily on the wisdom and experience of my colleagues.

Intended Audience

This book should appeal to the following groups of readers:

• Database administrators looking for practical tuning advice and for a broader understanding of the system as a whole.
• System administrators and other technical staff wanting to expand their understanding of the architecture and management of databases.
• Developers hoping to better understand the context in which their applications are used. The suggestions on benchmarking applications during the entire development cycle should be of particular interest.
• Specialists in one database wanting to learn more about other databases. I know many people who have expressed a desire to better understand the other major databases, but found the task too daunting. I hope this book will simplify that process by providing enough material to answer the important questions without being overwhelming.
• System users who simply want to understand the jargon associated with databases and the servers on which they run.

I have tried to go into enough depth to satisfy those looking for detailed configuration and tuning suggestions, while still making the content accessible to people who are not database or Solaris gurus.

Organization of This Book

This book is organized in five parts. Parts One, Two, and Five are best viewed as reference material. Parts Three and Four lend themselves to hands-on tuning.

Part One: Databases on Sun Servers — Provides background information on Sun's relationship with database vendors and Sun's Database Engineering organization, and discusses the optimizations made to the Solaris Operating Environment to improve database performance and availability. The section concludes with a review of the major hardware architectures (including those not supported by Sun) and their implications for database deployment.

Part Two: Database Architecture — Addresses database concepts, database workloads, and explores in detail the database optimizer and the role and sizing of the database buffer cache. Separate chapters discuss the architecture of Oracle, Sybase, Informix XPS, and DB2 for Solaris; for these chapters I provide a consistent format to make it easier for those wanting to contrast an unfamiliar database with one that is more familiar.

Part Three: Sizing and Configuring Sun DBMS Servers — Focuses on system sizing and configuring the CPUs, memory, and networks of a database server. The section concludes with a detailed discussion of data layout technologies, strategies, and recommendations.

Part Four: Performance Monitoring and Tuning — Deals with the issues and tools associated with performance monitoring and tuning, and offers a drill-down method for identifying and resolving system bottlenecks. Detailed configuration and tuning advice is provided for each of Oracle, Sybase, Informix XPS, and DB2 for Solaris, and the section concludes with a discussion on how to monitor applications and what metrics to collect.

Part Five: Other Topics — The final section of the book covers benchmarks and the role of Java technology in databases. The first chapter examines the pros and cons of industry-standard benchmarks and investigates the process and potential pitfalls involved in developing your own benchmark. The final chapter explores direct support for Java technology in databases and the Java 2 Enterprise Edition (J2EE) technology that is central to the burgeoning middleware market.

An appendix lists sources for supplementary information.

Book Website

The scripts and utilities referred to in the book are available on the book website, and I will also use this site to post updates to the material in the book. The website can be found at:

http://www.solarisdatabases.com

A Note from the Author

I welcome your feedback, suggestions, and general comments. If you like the book, please let me know! If you find flaws, errors, or omissions, I would also be glad to hear of them. In reading Configuring and Tuning Databases for the Solaris Platform, I hope you receive even a small proportion of the benefit I have gained from writing it.

—Allan Packer
Allan.Packer@Sun.COM



Updates

Submit Errata

