Informix Product Family

Informix
Version 11.70

IBM Informix Performance Guide



SC27-3544-06
Informix Product Family
Informix
Version 11.70

IBM Informix Performance Guide



SC27-3544-06
 Note
Before using this information and the product it supports, read the information in “Notices” on page C-1.

This edition replaces SC27-3544-05.

This document contains proprietary information of IBM. It is provided under a license agreement and is protected
by copyright law. The information contained in this publication does not include any product warranties, and any
statements provided in this manual should not be interpreted as such.
When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any
way it believes appropriate without incurring any obligation to you.
© Copyright IBM Corporation 1996, 2014.
US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract
with IBM Corp.
Contents
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
About this publication . . . . . . . . . . . . . . . . . . .
xiii . . . . . . . . . . . .
Topics beyond the scope of this publication . . . . . . . . . . .
xiii . . . . . . . . . . . .
Types of users . . . . . . . . . . . . . . . . . . . . .
xiv . . . . . . . . . . . .
Software dependencies . . . . . . . . . . . . . . . . . .
xiv . . . . . . . . . . . .
Assumptions about your locale . . . . . . . . . . . . . . .
xiv . . . . . . . . . . . .
Demonstration databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
What's new in performance for Informix, version 11.70 . . . . . . . . . . . . . . . . . . . . . xv
Example code conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
Additional documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxii
Compliance with industry standards . . . . . . . . . . . . . . . . . . . . . . . . . . xxii
How to provide documentation feedback . . . . . . . . . . . . . . . . . . . . . . . . . xxii

Chapter 1. Performance basics . . . . . . . . . . . . . . . . . . . . . . . . . 1-1

Developing a basic approach to performance measurement and tuning . . . . . . . . . . . . . . . 1-1
Quick start for acceptable performance on a small database . . . . . . . . . . . . . . . . . . . 1-2
Performance goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
Measurements of performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
Response time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Cost per transaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
Resource utilization and performance . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
Resource utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8
CPU utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9
Memory utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-10
Disk utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11
Factors that affect resource utilization . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Maintenance of good performance . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14

Chapter 2. Performance monitoring and the tools you use . . . . . . . . . . . . . 2-1

Evaluate the current configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
Create a performance history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2
The importance of a performance history . . . . . . . . . . . . . . . . . . . . . . . . 2-2
Tools that create a performance history . . . . . . . . . . . . . . . . . . . . . . . . . 2-3
Monitor performance with the OpenAdmin Tool (OAT) for Informix . . . . . . . . . . . . . . . . 2-7
Monitor database server resources . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7
Monitor resources that impact CPU utilization . . . . . . . . . . . . . . . . . . . . . . 2-7
Monitor memory utilization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8
Monitor disk I/O utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10
Monitor transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12
Using the onlog utility to monitor transactions . . . . . . . . . . . . . . . . . . . . . . 2-12
Using the onstat utility to monitor transactions . . . . . . . . . . . . . . . . . . . . . . 2-13
Monitor sessions and queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13
Monitoring memory usage for each session . . . . . . . . . . . . . . . . . . . . . . . 2-13
Using the SET EXPLAIN statement . . . . . . . . . . . . . . . . . . . . . . . . . . 2-14

Chapter 3. Effect of configuration on CPU utilization. . . . . . . . . . . . . . . . 3-1

UNIX configuration parameters that affect CPU utilization . . . . . . . . . . . . . . . . . . . 3-1
UNIX semaphore parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1
UNIX file-descriptor parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
UNIX memory configuration parameters . . . . . . . . . . . . . . . . . . . . . . . . 3-3
Windows configuration parameters that affect CPU utilization . . . . . . . . . . . . . . . . . . 3-3
Configuration parameters and environment variables that affect CPU utilization . . . . . . . . . . . . 3-3
Specifying virtual processor class information. . . . . . . . . . . . . . . . . . . . . . . 3-4
Setting the MULTIPROCESSOR configuration parameter when using multiple CPU VPs . . . . . . . . 3-10

© Copyright IBM Corp. 1996, 2014 iii

 Setting the SINGLE_CPU_VP configuration parameter when using one CPU VP. . . . . . . . . . . 3-10
Optimizing access methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10
Limiting PDQ resources in queries . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11
Limiting the performance impact of CPU-intensive queries . . . . . . . . . . . . . . . . . . 3-12
Limiting the number of PDQ scan threads that can run concurrently . . . . . . . . . . . . . . 3-12
Configuring poll threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13
Enabling fast polling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17
Network buffer pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17
Network buffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18
Support for private network buffers . . . . . . . . . . . . . . . . . . . . . . . . . 3-18
Network buffer size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19
Virtual processors and CPU utilization. . . . . . . . . . . . . . . . . . . . . . . . . . 3-20
Adding virtual processors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20
Automatic addition of CPU virtual processors . . . . . . . . . . . . . . . . . . . . . . 3-20
Monitoring virtual processors. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-21
CPU virtual processor private memory caches . . . . . . . . . . . . . . . . . . . . . . 3-24
Connections and CPU utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25
Multiplexed connections and CPU utilization . . . . . . . . . . . . . . . . . . . . . . 3-25
MaxConnect for multiple connections UNIX . . . . . . . . . . . . . . . . . . . . . . . 3-26

Chapter 4. Effect of configuration on memory utilization . . . . . . . . . . . . . . 4-1

Shared memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1
Resident portion of shared memory . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Virtual portion of shared memory . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
Message portion of shared memory . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
Estimating the size of the resident portion of shared memory . . . . . . . . . . . . . . . . . 4-3
Estimating the size of the virtual portion of shared memory . . . . . . . . . . . . . . . . . . 4-4
Estimating the size of the message portion of shared memory . . . . . . . . . . . . . . . . . 4-6
Configuring UNIX shared memory . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6
Freeing shared memory with onmode -F . . . . . . . . . . . . . . . . . . . . . . . . 4-7
Configuration parameters that affect memory utilization . . . . . . . . . . . . . . . . . . . . 4-8
Setting the size of the buffer pool, logical-log buffer, and physical-log buffer . . . . . . . . . . . . 4-9
The LOCKS configuration parameter and memory utilization . . . . . . . . . . . . . . . . . 4-15
The RESIDENT configuration parameter and memory utilization . . . . . . . . . . . . . . . . 4-17
The SHMADD and EXTSHMADD configuration parameters and memory utilization . . . . . . . . . 4-17
The SHMTOTAL configuration parameter and memory utilization . . . . . . . . . . . . . . . 4-18
The SHMVIRTSIZE configuration parameter and memory utilization . . . . . . . . . . . . . . 4-19
The SHMVIRT_ALLOCSEG configuration parameter and memory utilization. . . . . . . . . . . . 4-19
The STACKSIZE configuration parameter and memory utilization . . . . . . . . . . . . . . . 4-20
Configure and monitor memory caches . . . . . . . . . . . . . . . . . . . . . . . . . 4-21
Data-dictionary cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-22
Data-distribution cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-23
Monitor and tune the SQL statement cache . . . . . . . . . . . . . . . . . . . . . . . 4-25
Session memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-38
Data-replication buffers and memory utilization . . . . . . . . . . . . . . . . . . . . . . 4-39
Memory latches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-39
Monitoring latches with command-line utilities . . . . . . . . . . . . . . . . . . . . . . 4-39
Monitoring latches with SMI tables . . . . . . . . . . . . . . . . . . . . . . . . . . 4-40
Encrypted values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-40

Chapter 5. Effect of configuration on I/O activity . . . . . . . . . . . . . . . . . 5-1

Chunk and dbspace configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1
Associate disk partitions with chunks . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Associate dbspaces with chunks . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Placing system catalog tables with database tables . . . . . . . . . . . . . . . . . . . . . 5-2
I/O for cooked files for dbspace chunks . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Direct I/O (UNIX) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Direct I/O (Windows) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Concurrent I/O (AIX only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Enabling the direct I/O or concurrent I/O option (UNIX). . . . . . . . . . . . . . . . . . . 5-4

iv IBM Informix Performance Guide

 Confirming the use of the direct or concurrent I/O option (UNIX) . . . . . . . . . . . . . . . . 5-4
Placement of critical data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Consider separate disks for critical data components . . . . . . . . . . . . . . . . . . . . 5-5
Consider mirroring for critical data components . . . . . . . . . . . . . . . . . . . . . . 5-5
Configuration parameters that affect critical data . . . . . . . . . . . . . . . . . . . . . . 5-8
Configure dbspaces for temporary tables and sort files. . . . . . . . . . . . . . . . . . . . . 5-8
Creating temporary dbspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10
Specify temporary tables in the DBSPACETEMP configuration parameter . . . . . . . . . . . . . 5-11
Override the DBSPACETEMP configuration parameter for a session . . . . . . . . . . . . . . . 5-11
Estimating temporary space for dbspaces and hash joins. . . . . . . . . . . . . . . . . . . 5-12
PSORT_NPROCS environment variable . . . . . . . . . . . . . . . . . . . . . . . . 5-13
Configure sbspaces for temporary smart large objects. . . . . . . . . . . . . . . . . . . . . 5-14
Creating temporary sbspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14
Specify which sbspaces to use for temporary storage . . . . . . . . . . . . . . . . . . . . 5-15
Placement of simple large objects . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-15
Advantage of blobspaces over dbspaces . . . . . . . . . . . . . . . . . . . . . . . . 5-16
Blobpage size considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16
Factors that affect I/O for smart large objects . . . . . . . . . . . . . . . . . . . . . . . 5-20
Disk layout for sbspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20
Configuration parameters that affect sbspace I/O . . . . . . . . . . . . . . . . . . . . . 5-20
onspaces options that affect sbspace I/O . . . . . . . . . . . . . . . . . . . . . . . . 5-21
How the Optical Subsystem affects performance . . . . . . . . . . . . . . . . . . . . . . 5-24
Environment variables and configuration parameters for the Optical Subsystem . . . . . . . . . . . . 5-25
STAGEBLOB, an Optical Subsystem configuration parameter . . . . . . . . . . . . . . . . . 5-25
OPCACHEMAX, an Optical Subsystem configuration parameter . . . . . . . . . . . . . . . . 5-26
INFORMIXOPCACHE, an Optical Subsystem environment variable . . . . . . . . . . . . . . . 5-26
Table I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-26
Sequential scans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-26
Light scans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-27
Unavailable data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
Configuration parameters that affect table I/O . . . . . . . . . . . . . . . . . . . . . . . 5-28
How DATASKIP affects table I/O . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
Background I/O activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
Configuration parameters that affect checkpoints . . . . . . . . . . . . . . . . . . . . . 5-30
Configuration parameters that affect logging. . . . . . . . . . . . . . . . . . . . . . . 5-34
Configuration parameters that affect page cleaning . . . . . . . . . . . . . . . . . . . . 5-39
Configuration parameters that affect backup and restore. . . . . . . . . . . . . . . . . . . 5-41
Configuration parameters that affect rollback and recovery . . . . . . . . . . . . . . . . . . 5-42
Configuration parameters that affect data replication and auditing . . . . . . . . . . . . . . . 5-43
LRU tuning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-45

Chapter 6. Table performance considerations . . . . . . . . . . . . . . . . . . . 6-1

Placing tables on disk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-1
Isolating high-use tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2
Placing high-use tables on middle partitions of disks . . . . . . . . . . . . . . . . . . . . 6-3
Using multiple disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3
Backup and restore considerations when placing tables on disks . . . . . . . . . . . . . . . . 6-4
Factors affecting the performance of nonfragmented tables and table fragments . . . . . . . . . . . 6-5
Estimating table size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5
Estimating data pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5
Estimating pages that simple large objects occupy . . . . . . . . . . . . . . . . . . . . . 6-8
Managing the size of first and next extents for the tblspace tblspace . . . . . . . . . . . . . . . . 6-10
Managing sbspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-10
Estimating pages that smart large objects occupy . . . . . . . . . . . . . . . . . . . . . 6-10
Improving metadata I/O for smart large objects . . . . . . . . . . . . . . . . . . . . . 6-12
Monitoring sbspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13
Changing storage characteristics of smart large objects . . . . . . . . . . . . . . . . . . . 6-17
Managing extents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19
Choosing table extent sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-20
Monitoring active tblspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23
Monitoring the upper limit on extents and extent interleaving . . . . . . . . . . . . . . . . . 6-23

Contents v
 Reclaiming unused space within an extent . . . . . . . . . . . . . . . . . . . . . . . 6-26
Managing extent deallocation with the TRUNCATE keyword . . . . . . . . . . . . . . . . . 6-27
Defragment partitions to merge extents . . . . . . . . . . . . . . . . . . . . . . . . 6-28
Storing multiple table fragments in a single dbspace . . . . . . . . . . . . . . . . . . . . . 6-29
Displaying a list of table and index partitions . . . . . . . . . . . . . . . . . . . . . . . 6-29
Changing tables to improve performance . . . . . . . . . . . . . . . . . . . . . . . . . 6-29
Loading and unloading tables . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-29
Dropping indexes for table-update efficiency . . . . . . . . . . . . . . . . . . . . . . 6-32
Creating and enabling referential constraints efficiently . . . . . . . . . . . . . . . . . . . 6-32
Attaching or detaching fragments . . . . . . . . . . . . . . . . . . . . . . . . . . 6-34
Altering a table definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-35
Denormalize the data model to improve performance . . . . . . . . . . . . . . . . . . . . 6-42
Shortening rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-42
Expelling long strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-43
Splitting wide tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-44
Redundant data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-45
Reduce disk space in tables with variable length rows . . . . . . . . . . . . . . . . . . . . 6-46
Reduce disk space by compressing tables and fragments. . . . . . . . . . . . . . . . . . . . 6-46

Chapter 7. Indexes and index performance considerations . . . . . . . . . . . . . 7-1

Types of indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1
B-tree indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1
Forest of trees indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2
R-tree indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4
Indexes that DataBlade modules provide . . . . . . . . . . . . . . . . . . . . . . . . 7-4
Estimating index pages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4
Index extent sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4
Estimating conventional index pages. . . . . . . . . . . . . . . . . . . . . . . . . . 7-5
Managing indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-8
Space costs of indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-8
Time costs of indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-8
Unclaimed index space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9
Indexes on columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-10
Nonunique indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-12
Improve query performance with a forest of trees index . . . . . . . . . . . . . . . . . . . . 7-13
Detecting root node contention . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-13
Creating a forest of trees index . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-14
Disabling and enabling a forest of trees index . . . . . . . . . . . . . . . . . . . . . . 7-15
Performing a range scan on a forest of trees index . . . . . . . . . . . . . . . . . . . . . 7-15
Determining if you are using a forest of trees index . . . . . . . . . . . . . . . . . . . . 7-16
Finding the number of hashed columns and subtrees in a forest of trees index . . . . . . . . . . . 7-16
Creating and dropping an index in an online environment . . . . . . . . . . . . . . . . . . . 7-16
When you cannot create or drop indexes online . . . . . . . . . . . . . . . . . . . . . 7-17
Creating attached indexes in an online environment . . . . . . . . . . . . . . . . . . . . 7-18
Limiting memory allocation while creating indexes online . . . . . . . . . . . . . . . . . . 7-18
Improving performance for index builds . . . . . . . . . . . . . . . . . . . . . . . . . 7-18
Estimating memory needed for sorting . . . . . . . . . . . . . . . . . . . . . . . . 7-19
Estimating temporary space for index builds . . . . . . . . . . . . . . . . . . . . . . 7-20
Storing multiple index fragments in a single dbspace . . . . . . . . . . . . . . . . . . . . . 7-20
Improving performance for index checks . . . . . . . . . . . . . . . . . . . . . . . . . 7-20
Indexes on user-defined data types . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-21
Defining indexes for user-defined data types . . . . . . . . . . . . . . . . . . . . . . 7-22
Using an index that a DataBlade module provides . . . . . . . . . . . . . . . . . . . . . 7-27
Choosing operator classes for indexes . . . . . . . . . . . . . . . . . . . . . . . . . 7-27

Chapter 8. Locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1

Locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1
Locking granularity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1
Row and key locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-1
Page locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-2

vi IBM Informix Performance Guide

 Table locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3
Database locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-3
Configuring the lock mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-4
Setting the lock mode to wait . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5
Locks with the SELECT statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5
Isolation level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-5
Locking nonlogging tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8
Update cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-8
Locks placed with INSERT, UPDATE, and DELETE statements . . . . . . . . . . . . . . . . . 8-10
The internal lock table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-10
Monitoring locks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-11
Configuring and managing lock usage . . . . . . . . . . . . . . . . . . . . . . . . . 8-12
Monitoring lock waits and lock errors . . . . . . . . . . . . . . . . . . . . . . . . . 8-13
Monitoring the number of free locks . . . . . . . . . . . . . . . . . . . . . . . . . 8-14
Monitoring deadlocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-15
Monitoring isolation levels that sessions use . . . . . . . . . . . . . . . . . . . . . . . 8-15
Locks for smart large objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-16
Byte-range locking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-16
Lock promotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-19
Dirty Read isolation level and smart large objects . . . . . . . . . . . . . . . . . . . . . 8-20

Chapter 9. Fragmentation guidelines . . . . . . . . . . . . . . . . . . . . . . 9-1

Planning a fragmentation strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-1
Fragmentation goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-2
Examining your data and queries . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-4
Considering physical fragmentation factors . . . . . . . . . . . . . . . . . . . . . . . 9-5
Distribution schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-6
Choosing a distribution scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-7
Designing an expression-based distribution scheme . . . . . . . . . . . . . . . . . . . . . 9-8
Suggestions for improving fragmentation . . . . . . . . . . . . . . . . . . . . . . . . 9-9
Strategy for fragmenting indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-10
Attached indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-10
Detached indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-12
Restrictions on indexes for fragmented tables . . . . . . . . . . . . . . . . . . . . . . 9-13
Strategy for fragmenting temporary tables . . . . . . . . . . . . . . . . . . . . . . . . 9-13
Distribution schemes that eliminate fragments . . . . . . . . . . . . . . . . . . . . . . . 9-14
Fragmentation expressions for fragment elimination . . . . . . . . . . . . . . . . . . . . 9-14
Query expressions for fragment elimination . . . . . . . . . . . . . . . . . . . . . . . 9-15
Effectiveness of fragment elimination . . . . . . . . . . . . . . . . . . . . . . . . . 9-16
Improve the performance of operations that attach and detach fragments . . . . . . . . . . . . . . 9-19
Improving ALTER FRAGMENT ATTACH performance . . . . . . . . . . . . . . . . . . . 9-20
Improving ALTER FRAGMENT DETACH performance . . . . . . . . . . . . . . . . . . . 9-26
Forcing out transactions when altering table fragments . . . . . . . . . . . . . . . . . . . 9-27
Monitoring fragment use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9-28
Monitoring fragmentation with the onstat -g ppf command. . . . . . . . . . . . . . . . . . 9-28
Monitoring fragmentation with SET EXPLAIN output . . . . . . . . . . . . . . . . . . . 9-29

Chapter 10. Queries and the query optimizer . . . . . . . . . . . . . . . . . . 10-1

The query plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1
The access plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1
The join plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-2
Example of query-plan execution . . . . . . . . . . . . . . . . . . . . . . . . . . 10-4
Query plans that include an index self-join path . . . . . . . . . . . . . . . . . . . . . 10-8
Query plan evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-9
Report that shows the query plan chosen by the optimizer . . . . . . . . . . . . . . . . . . 10-9
Sample query plan reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-12
XML query plans in IBM Data Studio . . . . . . . . . . . . . . . . . . . . . . . . 10-19
Factors that affect the query plan . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-19
Statistics held for the table and index . . . . . . . . . . . . . . . . . . . . . . . . . 10-20
Filters in the query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-20

Contents vii
 Indexes for evaluating a filter . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-21
Effect of PDQ on the query plan . . . . . . . . . . . . . . . . . . . . . . . . . . 10-22
Effect of OPTCOMPIND on the query plan . . . . . . . . . . . . . . . . . . . . . . . 10-22
Effect of available memory on the query plan . . . . . . . . . . . . . . . . . . . . . . 10-23
Time costs of a query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-23
Memory-activity costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-24
Sort-time costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-24
Row-reading costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-25
Sequential access costs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-26
Nonsequential access costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-26
Index lookup costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-26
In-place ALTER TABLE costs . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-27
View costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-27
Small-table costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-28
Data-mismatch costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-28
Encrypted-value costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-29
GLS functionality costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-29
Network-access costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-29
Optimization when SQL is within an SPL routine. . . . . . . . . . . . . . . . . . . . . . 10-31
SQL optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-31
Execution of an SPL routine . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-33
SPL routine executable format stored in UDR cache . . . . . . . . . . . . . . . . . . . . 10-33
Trigger execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-34
Performance implications for triggers . . . . . . . . . . . . . . . . . . . . . . . . . 10-35

Chapter 11. Optimizer directives . . . . . . . . . . . . . . . . . . . . . . . . 11-1

What optimizer directives are . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-1
Optimizer directives that are embedded in queries. . . . . . . . . . . . . . . . . . . . . 11-1
External optimizer directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-1
Reasons to use optimizer directives . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-2
Preparation for using directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-3
Guidelines for using directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-3
Types of optimizer directives that are supported in SQL statements . . . . . . . . . . . . . . . . 11-3
Access-method directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-4
Join-order directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-5
Join-method directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-6
Optimization-goal directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-7
Star-join directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-8
EXPLAIN directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-8
Example of directives that can alter a query plan . . . . . . . . . . . . . . . . . . . . . 11-10
Configuration parameters and environment variables for optimizer directives . . . . . . . . . . . . 11-12
Optimizer directives and SPL routines . . . . . . . . . . . . . . . . . . . . . . . . . 11-13
Forcing reoptimization to avoid an index and previously prepared statement problem . . . . . . . . . 11-13
External optimizer directives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-15
Creating and saving external directives . . . . . . . . . . . . . . . . . . . . . . . . 11-15
Enabling external directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-15
Deleting external directives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-16

Chapter 12. Parallel database query (PDQ) . . . . . . . . . . . . . . . . . . . 12-1

What PDQ is . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-1
Structure of a PDQ query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-1
Database server operations that use PDQ . . . . . . . . . . . . . . . . . . . . . . . . . 12-2
Parallel update and delete operations . . . . . . . . . . . . . . . . . . . . . . . . . 12-2
Parallel insert operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-2
Parallel index builds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-3
Parallel user-defined routines . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-4
Hold cursors that use PDQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-4
SQL operations that do not use PDQ . . . . . . . . . . . . . . . . . . . . . . . . . 12-4
Update statistics operations affected by PDQ . . . . . . . . . . . . . . . . . . . . . . 12-5
SPL routines and triggers and PDQ . . . . . . . . . . . . . . . . . . . . . . . . . . 12-5

viii IBM Informix Performance Guide

 Correlated and uncorrelated subqueries . . . . . . . . . . . . . . . . . . . . . . . .12-5
OUTER index joins and PDQ . . . . . . . . . . . . . . . . . . . . . . . . . . . .12-5
Remote tables used with PDQ . . . . . . . . . . . . . . . . . . . . . . . . . . .12-6
The Memory Grant Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12-6
The allocation of resources for parallel database queries . . . . . . . . . . . . . . . . . . . .12-7
Limiting the priority of decision-support queries . . . . . . . . . . . . . . . . . . . . .12-8
Adjusting the amount of memory for DSS and PDQ queries . . . . . . . . . . . . . . . . . 12-11
Limiting the number of concurrent scans . . . . . . . . . . . . . . . . . . . . . . . 12-11
Limiting the maximum number of PDQ queries . . . . . . . . . . . . . . . . . . . . . 12-12
Managing PDQ queries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-12
Analyzing query plans with SET EXPLAIN output . . . . . . . . . . . . . . . . . . . . 12-12
Influencing the choice of a query plan . . . . . . . . . . . . . . . . . . . . . . . . 12-13
Setting the PDQ priority dynamically. . . . . . . . . . . . . . . . . . . . . . . . . 12-13
Enabling the database server to allocate PDQ memory . . . . . . . . . . . . . . . . . . . 12-13
User control of PDQ resources . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-15
DBA control of resources for PDQ and DSS queries . . . . . . . . . . . . . . . . . . . . 12-15
Monitoring resources used for PDQ and DSS queries . . . . . . . . . . . . . . . . . . . . 12-16
Monitoring PDQ resources by using the onstat Utility . . . . . . . . . . . . . . . . . . . 12-16
Identifying parallel scans in SET EXPLAIN output . . . . . . . . . . . . . . . . . . . . 12-18

Chapter 13. Improving individual query performance . . . . . . . . . . . . . . . 13-1

Test queries using a dedicated test system . . . . . . . . . . . . . . . . . . . . . . . . 13-1
Display the query plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-1
Improve filter selectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-2
Filters with user-defined routines . . . . . . . . . . . . . . . . . . . . . . . . . . 13-2
Avoid some filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-3
Use join filters and post-join filters . . . . . . . . . . . . . . . . . . . . . . . . . . 13-3
Automatic statistics updating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-6
How AUS works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-7
AUS expiration policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-8
Viewing AUS statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-10
Rescheduling AUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-10
Disabling AUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-11
Update statistics when they are not generated automatically . . . . . . . . . . . . . . . . . . 13-11
Update the statistics for the number of rows . . . . . . . . . . . . . . . . . . . . . . 13-12
Drop data distributions if necessary when upgrading . . . . . . . . . . . . . . . . . . . 13-13
Creating data distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-13
Updating statistics for join columns . . . . . . . . . . . . . . . . . . . . . . . . . 13-16
Updating statistics for columns with user-defined data types . . . . . . . . . . . . . . . . . 13-16
Update statistics in parallel on very large databases . . . . . . . . . . . . . . . . . . . . 13-17
Adjust the amount of memory and disk space for UPDATE STATISTICS . . . . . . . . . . . . . 13-17
Data sampling during update statistics operations . . . . . . . . . . . . . . . . . . . . 13-18
Display data distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-18
Improve performance by adding or removing indexes . . . . . . . . . . . . . . . . . . . . 13-20
Replace autoindexes with permanent indexes . . . . . . . . . . . . . . . . . . . . . . 13-20
Use composite indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-20
Indexes for data warehouse applications. . . . . . . . . . . . . . . . . . . . . . . . 13-21
Configure B-tree scanner information to improve transaction processing . . . . . . . . . . . . . 13-22
Determine the amount of free space in an index page . . . . . . . . . . . . . . . . . . . 13-29
Optimizer estimates of distributed queries . . . . . . . . . . . . . . . . . . . . . . . . 13-29
Buffer data transfers for a distributed query . . . . . . . . . . . . . . . . . . . . . . 13-30
The query plan of a distributed query . . . . . . . . . . . . . . . . . . . . . . . . 13-30
Improve sequential scans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-31
Enable view folding to improve query performance . . . . . . . . . . . . . . . . . . . . . 13-31
Reduce the join and sort operations . . . . . . . . . . . . . . . . . . . . . . . . . . 13-32
Avoid or simplify sort operations . . . . . . . . . . . . . . . . . . . . . . . . . . 13-32
Use parallel sorts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-32
Use temporary tables to reduce sorting scope . . . . . . . . . . . . . . . . . . . . . . 13-33
Configuring memory for queries with hash joins, aggregates, and other memory-intensive elements. . . . 13-34
Optimize user-response time for queries . . . . . . . . . . . . . . . . . . . . . . . . . 13-35
Optimization level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-35

Contents ix
 Optimization goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-35
Optimize queries for user-defined data types . . . . . . . . . . . . . . . . . . . . . . . 13-38
Parallel UDRs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-38
Selectivity and cost functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-39
User-defined statistics for UDTs . . . . . . . . . . . . . . . . . . . . . . . . . . 13-40
Negator functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-40
Optimize queries with the SQL statement cache . . . . . . . . . . . . . . . . . . . . . . 13-40
When to use the SQL statement cache . . . . . . . . . . . . . . . . . . . . . . . . 13-41
Using the SQL statement cache . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-41
Monitoring memory usage for each session . . . . . . . . . . . . . . . . . . . . . . . 13-43
Monitoring usage of the SQL statement cache . . . . . . . . . . . . . . . . . . . . . . 13-46
Monitor sessions and threads . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-47
Monitor sessions and threads with onstat commands . . . . . . . . . . . . . . . . . . . 13-48
Monitor sessions and threads with ON-Monitor (UNIX) . . . . . . . . . . . . . . . . . . 13-53
Monitor sessions and threads with SMI tables . . . . . . . . . . . . . . . . . . . . . . 13-54
Monitor transactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13-55
Display information about transactions . . . . . . . . . . . . . . . . . . . . . . . . 13-55
Display information about transaction locks . . . . . . . . . . . . . . . . . . . . . . 13-56
Display statistics on user sessions . . . . . . . . . . . . . . . . . . . . . . . . . . 13-57
Display statistics on sessions executing SQL statements. . . . . . . . . . . . . . . . . . . 13-58

Chapter 14. The onperf utility on UNIX . . . . . . . . . . . . . . . . . . . . . 14-1

Overview of the onperf utility . . . . . . . . . . . . . . . . . . . . . . . . . . . .14-1
Basic onperf utility functions . . . . . . . . . . . . . . . . . . . . . . . . . . . .14-1
onperf utility tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14-3
Requirements for running the onperf utility . . . . . . . . . . . . . . . . . . . . . . . .14-3
Starting the onperf utility and exiting from it . . . . . . . . . . . . . . . . . . . . . . .14-4
The onperf user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14-5
Graph tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14-5
Query-tree tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-11
Status tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-11
Activity tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-12
Why you might want to use onperf . . . . . . . . . . . . . . . . . . . . . . . . . . 14-12
Routine monitoring with onperf . . . . . . . . . . . . . . . . . . . . . . . . . . 14-13
Diagnosing sudden performance loss . . . . . . . . . . . . . . . . . . . . . . . . . 14-13
Diagnosing performance degradation . . . . . . . . . . . . . . . . . . . . . . . . . 14-13
onperf utility metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-13
Database server metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-13
Disk-chunk metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-15
Disk-spindle metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-15
Physical-processor metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-15
Virtual-processor metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-16
Session metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-16
Tblspace metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-17
Fragment metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14-18

Appendix A. Case studies and examples . . . . . . . . . . . . . . . . . . . . A-1

Case study of a situation in which disks are overloaded . . . . . . . . . . . . . . . . . . . . A-1

Appendix B. Accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1

Accessibility features for IBM Informix products. . . . . . . . . . . . . . . . . . . . . . . B-1
Accessibility features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
Keyboard navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
Related accessibility information . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
IBM and accessibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1
Dotted decimal syntax diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1

Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1
Privacy policy considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-3
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-3

x IBM Informix Performance Guide

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-1

Contents xi
xii IBM Informix Performance Guide
Introduction
This introduction provides an overview of the information in this publication and
describes the conventions it uses.

About this publication

This publication provides information about how to configure and operate IBM®
Informix® to improve overall system throughput and to improve the performance
of SQL queries. This publication also includes information about performance
tuning issues and methods, fragmentation guidelines, and a complete description
of the onperf utility

Information in this publication can help you perform the following tasks:
v Monitor system resources that are critical to performance
v Identify database activities that affect these critical resources
v Identify and monitor queries that are critical to performance
v Use the database server utilities (especially onperf, ISA and onstat) for
performance monitoring and tuning
v Eliminate performance bottlenecks by:
– Balancing the load on system resources
– Adjusting the configuration parameters or environment variables of your
database server
– Adjusting the arrangement of your data
– Allocating resources for decision-support queries
– Creating indexes to speed up retrieval of your data

Performance measurement and tuning encompass a broad area of research and

practice and can involve information beyond the scope of this publication.

Performance issues related to dimensional databases and data warehouse queries

are described in the IBM Informix Data Warehouse Guide.
Related concepts:
Performance tuning dimensional databases (Data Warehouse Guide)

Topics beyond the scope of this publication

Attempts to balance the workload often produce a succession of moderate
performance improvements. Sometimes the improvements are dramatic. However,
in some situations a load-balancing approach is not enough. The following types of
situations might require measures beyond the scope of this publication:
v Application programs that require modification to make better use of database
server or operating-system resources
v Applications that interact in ways that impair performance
v A host computer that might be subject to conflicting uses
v A host computer with capacity that is inadequate for the evolving workload
v Network performance problems that affect client/server or other applications

© Copyright IBM Corp. 1996, 2014 xiii

 No amount of database tuning can correct these situations. Nevertheless, they are
easier to identify and resolve when the database server is configured properly.

Important: Although broad performance considerations also include reliability and

data availability as well as improved response time and efficient use of system
resources, this publication discusses only response time and system resource use.
For discussions of improved database server reliability and data availability, see
information about switchover, mirroring, and high availability in your IBM
Informix Administrator's Guide. For information about backup and restore, see the
IBM Informix Backup and Restore Guide.

Types of users
This publication is written for the following users:
v Database administrators
v Database server administrators
v Database-application programmers
v Performance engineers

This publication assumes that you have the following background:

v A working knowledge of your computer, your operating system, and the utilities
that your operating system provides
v Some experience working with relational databases or exposure to database
concepts
v Some experience with computer programming
v Some experience with database server administration, operating-system
administration, or network administration

Software dependencies
This publication assumes that you are using IBM Informix, Version 11.70.

Assumptions about your locale

IBM Informix products can support many languages, cultures, and code sets. All
the information related to character set, collation and representation of numeric
data, currency, date, and time that is used by a language within a given territory
and encoding is brought together in a single environment, called a Global
Language Support (GLS) locale.

The IBM Informix OLE DB Provider follows the ISO string formats for date, time,
and money, as defined by the Microsoft OLE DB standards. You can override that
default by setting an Informix environment variable or registry entry, such as
DBDATE.

If you use Simple Network Management Protocol (SNMP) in your Informix

environment, note that the protocols (SNMPv1 and SNMPv2) recognize only
English code sets. For more information, see the topic about GLS and SNMP in the
IBM Informix SNMP Subagent Guide.

The examples in this publication are written with the assumption that you are
using one of these locales: en_us.8859-1 (ISO 8859-1) on UNIX platforms or
en_us.1252 (Microsoft 1252) in Windows environments. These locales support U.S.
English format conventions for displaying and entering date, time, number, and
currency values. They also support the ISO 8859-1 code set (on UNIX and Linux)

xiv IBM Informix Performance Guide

 or the Microsoft 1252 code set (on Windows), which includes the ASCII code set
plus many 8-bit characters such as é, è, and ñ.

You can specify another locale if you plan to use characters from other locales in
your data or your SQL identifiers, or if you want to conform to other collation
rules for character data.

For instructions about how to specify locales, additional syntax, and other
considerations related to GLS locales, see the IBM Informix GLS User's Guide.

Demonstration databases
The DB-Access utility, which is provided with your IBM Informix database server
products, includes one or more of the following demonstration databases:
v The stores_demo database illustrates a relational schema with information about
a fictitious wholesale sporting-goods distributor. Many examples in IBM
Informix publications are based on the stores_demo database.
v The superstores_demo database illustrates an object-relational schema. The
superstores_demo database contains examples of extended data types, type and
table inheritance, and user-defined routines.

For information about how to create and populate the demonstration databases,
see the IBM Informix DB-Access User's Guide. For descriptions of the databases and
their contents, see the IBM Informix Guide to SQL: Reference.

The scripts that you use to install the demonstration databases are in the
$INFORMIXDIR/bin directory on UNIX platforms and in the %INFORMIXDIR%\bin
directory in Windows environments.

What's new in performance for Informix, version 11.70

This publication includes information about new features and changes in existing
functionality.

The following changes and enhancements are relevant to this publication. For a
complete list of what's new in this release, see the release notes or the information
center at http://pic.dhe.ibm.com/infocenter/idshelp/v117/topic/com.ibm.po.doc/
new_features.htm.

Introduction xv
Table 1. What's new in the IBM Informix Performance Guide for version 11.70.xC8
Overview Reference
Dynamic private memory caches for CPU virtual “CPU virtual processor private memory caches” on page
processors 3-24

Private memory caches for CPU virtual processors now

change size automatically as needed. You create private
memory caches by setting the VP_MEMORY_CACHE_KB
configuration parameter to the initial total size of the
caches. The size of a private memory cache increases and
decreases automatically, depending on the needs of the
associated CPU virtual processor. Previously, the size of
private memory caches was limited to the value of the
VP_MEMORY_CACHE_KB configuration parameter. You
can preserve the previous behavior by including a comma
and the word STATIC after the size value of the
VP_MEMORY_CACHE_KB configuration parameter.

The onstat -g vpcache command now shows the target

size for each bin in the cache before draining starts and
the last time that each bin was drained.
In-place alter operations on serial data types “Conditions for in-place alter operations” on page 6-36

The ALTER TABLE statement converts the following

column data types with in-place alter operations:
v SERIAL to SERIAL8
v SERIAL to BIGSERIAL
v SERIAL8 to BIGSERIAL
v BIGSERIAL to SERIAL8

Previously such data types were converted with slow

alter operations. In-place alter operations require less
space than slow alter operations and make the table
available to other sessions faster.

xvi IBM Informix Performance Guide

Table 1. What's new in the IBM Informix Performance Guide for version 11.70.xC8 (continued)
Overview Reference
Temporarily prevent constraint validation “Creating and enabling referential constraints efficiently”
on page 6-32
You can significantly increase the speed of loading or
migrating large tables by temporarily preventing the
database server from validating foreign-key referential
constraints. You can disable the validation of constraints
when you create constraints or change the mode of
constraints to ENABLED or FILTERING.
v You include the NOVALIDATE keyword in an ALTER
TABLE ADD CONSTRAINT statement or in a SET
CONSTRAINTS ENABLED or SET CONSTRAINTS
FILTERING statement.
v If you plan to run multiple ALTER TABLE ADD
CONSTRAINT or SET CONSTRAINTS statements, run
the SET ENVIRONMENT NOVALIDATE ON statement
to disable the validation of foreign-key constraints
during the current session.

When you migrate data, include the -nv option in the

dbimport command.

The NOVALIDATE keyword prevents the database server

from checking every row for referential integrity during
ALTER TABLE ADD CONSTRAINT and SET
CONSTRAINTS operations on foreign-key constraints.
When those statements finish running, the database
server automatically resumes referential-integrity
enforcement of those constraints in subsequent DML
operations.

Use this feature only on tables whose enabled foreign-key

constraints are free of violations, or when the referential
constraints can be validated after the tables are loaded or
migrated to the target database.
Faster creation of foreign-key constraints “Creating and enabling referential constraints efficiently”
on page 6-32
When you run the ALTER TABLE ADD CONSTRAINT
statement, some foreign-key constraints can be created
faster if the table has a unique index or a primary-key
constraint that is already defined on the columns in the
foreign-key constraint.

Foreign-key constraints are not created faster, however, if

the constraint key or index key includes columns of
user-defined or opaque data types, including BOOLEAN
and LVARCHAR, or if other restrictions are true for the
foreign-key constraint or for the referenced table.

Introduction xvii
Table 2. What's new in the IBM Informix Performance Guide for version 11.70xC4
Overview Reference
Data sampling for update statistics operations “Data sampling during update statistics operations” on
page 13-18
If you have a large index with more than 100 000 leaf
pages, you can generate index statistics based on
sampling when you run UPDATE STATISTICS statements
in LOW mode. Gathering index statistics from sampled
data can increase the speed of the update statistics
operations. To enable sampling, set the
USTLOW_SAMPLE configuration parameter or the
USTLOW_SAMPLE option of the SET ENVIRONMENT
statement.

Table 3. What's new in the IBM Informix Performance Guide for version 11.70xC3
Overview Reference
Automatic read-ahead operations “Configuration parameters that affect table I/O” on page
5-28
You can enable the database server to use read-ahead
operations automatically to improve performance. Most AUTO_READAHEAD configuration parameter
queries can benefit from processing the query while (Administrator's Reference)
asynchronously retrieving the data required by the query.
The database server can automatically use asynchronous
operations for data or it can avoid them if the data for
the query is already cached. Use the
AUTO_READAHEAD configuration parameter to
configure automatic read-ahead operations for all queries,
and use the SET ENVIRONMENT AUTO_READAHEAD
statement to configure automatic read-ahead operations
for a particular session.

The RA_THRESHOLD configuration parameter is

deprecated with this release.

The RA_THRESHOLD configuration parameter is

deprecated with this release.
Reserving memory for critical activities “The LOW_MEMORY_RESERVE configuration parameter
and memory utilization” on page 4-15
You can enable the server to reserve a specific amount of
memory for use when critical activities (such as rollback LOW_MEMORY_RESERVE configuration parameter
activities) are needed and the server has limited free (Administrator's Reference)
memory. If you enable the new
LOW_MEMORY_RESERVE configuration parameter by
setting it to a specified value in kilobytes, the critical
activities can complete even when you get out-of-memory
errors. You can also dynamically adjust the value of the
LOW_MEMORY_RESERVE configuration parameter with
the onmode -wm or -wf command.
Configuring the server response to low memory Configure the server response when memory is critically
low (Administrator's Guide)
You can configure the actions that the server takes to
continue processing when memory is critically low. You LOW_MEMORY_MGR configuration parameter
can specify the criteria for terminating sessions based on (Administrator's Reference)
idle time, memory usage, and other factors so that the
targeted application can continue and avoid
out-of-memory problems. Configuring the low memory
response is useful for embedded applications that have
memory limitations.

xviii IBM Informix Performance Guide

Table 4. What's new in the IBM Informix Performance Guide for version 11.70xC2
Overview Reference
Improve network connection performance and scalability “Improve connection performance and scalability” on
page 3-16
You can improve the performance and scalability of
network connections on UNIX operating systems by NUMFDSERVERS configuration parameter
using the NUMFDSERVERS configuration parameter. Use (Administrator's Reference)
the configuration parameter to adjust the maximum
number of poll threads for the server to use when
migrating TCP/IP connection across virtual processors
(VPs). Adjusting the number of poll threads, for example,
by increasing the number of poll threads beyond the
default number, is useful if the database server has a high
rate of new connect and disconnect requests or if you
find a high amount of contention between network
shared file (NSF) locks. For example, if you have multiple
CPU VPs and poll threads and this results in NSF
locking, you can increase the value of NUMFDSERVERS
(and you can increase the number of poll threads
specified in the NETTYPE configuration parameter) to
reduce NSF lock contention.

Table 5. What's new in the IBM Informix Performance Guide for version 11.70xC1
Overview Reference
Less root node contention with forest of trees indexes “Forest of trees indexes” on page 7-2

If you have many concurrent users who routinely “Improve query performance with a forest of trees index”
experience delays due to root node contention, you might on page 7-13
improve query performance if you convert your B-tree
index to a forest of trees index. A forest of trees index is CREATE INDEX statement (SQL Syntax)
similar to a B-tree index, but has multiple root nodes and
potentially fewer levels. You create forest of trees indexes HASH ON clause (SQL Syntax)
with the new HASH ON clause of the CREATE INDEX
statement of SQL.
Automatic light scans on tables “Light scans” on page 5-27

Informix now automatically performs light scans when

appropriate. In the previous release, you had to set
configuration parameters to enable Informix to perform
these scans.
Defragmenting partitions “Defragment partitions to merge extents” on page 6-28

A frequently updated table can become fragmented over

time and this fragmentation degrades performance when
the table is accessed by the server. You can improve
performance by defragmenting partitions to merge
noncontiguous extents. Defragmenting a table brings data
rows closer together and avoids partition header page
overflow problems. Use the SQL administration API
task() or admin() function with the defragment argument
and specify the table name or partition number that you
want to defragment.

Introduction xix
Table 5. What's new in the IBM Informix Performance Guide for version 11.70xC1 (continued)
Overview Reference
Automatically add CPU virtual processors “Automatic addition of CPU virtual processors” on page
3-20
When the database server starts, it checks that the
number of CPU virtual processors is at least half the
number of CPU processors on the database server
computer. This ratio of CPU processors to CPU virtual
processors is a recommended minimum to ensure that the
database server performs optimally in most situations. If
necessary, the database server automatically increases the
number of CPU virtual processors to half the number of
CPU processors.
Automatically optimize data storage “Defragment partitions to merge extents” on page 6-28

You can automatically compress, shrink, repack, and

defragment tables and fragments by enabling the
auto_crsd Scheduler task. You can disable or update the
thresholds for each of these operations.
Large pages support on Linux “Resident portion of shared memory” on page 4-2

Large pages for non-message shared memory segments “Virtual portion of shared memory” on page 4-2
that reside in physical memory are now enabled by
default on Linux platforms. Previously, large pages were
supported only on AIX® and Solaris systems. The use of
large pages can provide performance benefits in large
memory configurations. To enable or disable support for
large pages, use the IFX_LARGE_PAGES environment
variable.
Improving performance by reducing buffer reads “Improve performance by adding or removing indexes”
on page 13-20
If you enable the new BATCHEDREAD_INDEX
configuration parameter, the optimizer automatically BATCHEDREAD_INDEX configuration parameter
chooses to fetch a set of keys from an index buffer, (Administrator's Reference)
reducing the number of buffer times a buffer is read.
Alerts for tables with in-place alter operations “In-place alter” on page 6-35

You can see which tables have outstanding in-place alter

operations, which can cause slight performance
degradation. Each table with an outstanding in-place alter
operation has an informative alert row in the ph_alert
table in the sysadmin database.
Automatic allocation of secondary partition header pages “Managing extents” on page 6-19
for extending the extent map

If you have a table that needs more extents and the

database server runs out of space on the partition header
page, the database server now automatically allocates
extended secondary partition header pages to
accommodate new extent entries. The database server can
now allocate an unlimited number of extents for any
partition, unless the size of a table dictates a limit to the
number of extents.

xx IBM Informix Performance Guide

Table 5. What's new in the IBM Informix Performance Guide for version 11.70xC1 (continued)
Overview Reference
Partitioning table and index storage by an INTERVAL “Distribution schemes” on page 9-6
strategy

You can define a storage distribution strategy for tables or

indexes that partitions data into a set of fragments that
are each based on an interval value of the fragment key,
which must be a column expression that references a
single column of a numeric, DATE, or DATETIME data
type. When rows are inserted that do not fit in the range
fragments, the database server automatically creates new
interval fragments without DBA intervention.

This kind of fragmentation strategy is useful when all

possible fragment key values in a growing table are not
known, and the DBA does not want to allocate fragments
for data that is not yet loaded.
Partitioning table and index storage by a LIST strategy “Distribution schemes” on page 9-6

You can define a storage distribution strategy for tables or

indexes that partitions data into a set of fragments that
are each based on a list of discrete values of the fragment
key. Each value in the list must be unique among the lists
for fragments of the same object. Query performance can
improve through fragment elimination when the
fragment key for a table has a finite set of values, and
queries on the table specify equality predicates on the
fragment key.
New editions and product names For more information about the Informix product family,
go to http://www.ibm.com/software/data/informix/.
IBM Informix Dynamic Server editions were withdrawn
and new Informix editions are available. Some products
were also renamed. The publications in the Informix
library pertain to the following products:
v IBM Informix database server, formerly known as IBM
Informix Dynamic Server (IDS)
v IBM OpenAdmin Tool (OAT) for Informix, formerly
known as OpenAdmin Tool for Informix Dynamic
Server (IDS)
v IBM Informix SQL Warehousing Tool, formerly known
as Informix Warehouse Feature

Example code conventions

Examples of SQL code occur throughout this publication. Except as noted, the code
is not specific to any single IBM Informix application development tool.

If only SQL statements are listed in the example, they are not delimited by
semicolons. For instance, you might see the code in the following example:
CONNECT TO stores_demo
...

DELETE FROM customer

WHERE customer_num = 121

Introduction xxi
 ...

COMMIT WORK
DISCONNECT CURRENT

To use this SQL code for a specific product, you must apply the syntax rules for
that product. For example, if you are using an SQL API, you must use EXEC SQL
at the start of each statement and a semicolon (or other appropriate delimiter) at
the end of the statement. If you are using DB–Access, you must delimit multiple
statements with semicolons.

Tip: Ellipsis points in a code example indicate that more code would be added in
a full application, but it is not necessary to show it to describe the concept that is
being discussed.

For detailed directions on using SQL statements for a particular application

development tool or SQL API, see the documentation for your product.

Additional documentation
Documentation about this release of IBM Informix products is available in various
formats.

You can access Informix technical information such as information centers,

technotes, white papers, and IBM Redbooks® publications online at
http://www.ibm.com/software/data/sw-library/.

Compliance with industry standards

IBM Informix products are compliant with various standards.

IBM Informix SQL-based products are fully compliant with SQL-92 Entry Level
(published as ANSI X3.135-1992), which is identical to ISO 9075:1992. In addition,
many features of IBM Informix database servers comply with the SQL-92
Intermediate and Full Level and X/Open SQL Common Applications Environment
(CAE) standards.

How to provide documentation feedback

You are encouraged to send your comments about IBM Informix user
documentation.

Use one of the following methods:

v Send email to [email protected].
v In the Informix information center, which is available online at
http://www.ibm.com/software/data/sw-library/, open the topic that you want
to comment on. Click the feedback link at the bottom of the page, complete the
form, and submit your feedback.
v Add comments to topics directly in the information center and read comments
that were added by other users. Share information about the product
documentation, participate in discussions with other users, rate topics, and
more!

Feedback from all methods is monitored by the team that maintains the user
documentation. The feedback methods are reserved for reporting errors and

xxii IBM Informix Performance Guide

omissions in the documentation. For immediate help with a technical problem,
contact IBM Technical Support at http://www.ibm.com/planetwide/.

We appreciate your suggestions.

Introduction xxiii
xxiv IBM Informix Performance Guide
Chapter 1. Performance basics
Performance measurement and tuning issues and methods are relevant to daily
database server administration and query execution.

Performance measurement and tuning encompass a broad area of research and

practice and can involve information beyond the scope of this publication. For
more information about the types of information contained in the publication and
the topics that are beyond the scope of this publication, see “About this
publication” on page xiii.

These topics:
v Describe a basic approach for performance measurement and tuning
v Provide guidelines for a quick start to obtain acceptable initial performance on a
small database
v Describe roles in maintaining good performance

Developing a basic approach to performance measurement and tuning

To maintain optimum performance for your database applications, develop a plan
for measuring system performance, making adjustments to maintain good
performance and taking corrective measures when performance degrades. Regular,
specific measurements can help you to anticipate and correct performance
problems.

By recognizing problems early, you can prevent them from affecting users
significantly. Early indications of a performance problem are often vague; users
might report that the system seems sluggish. Users might complain that they
cannot get all their work done, that transactions take too long to complete, that
queries take too long to process, or that the application slows down at certain
times during the day.

To determine the nature of the problem, you must measure the actual use of
system resources and evaluate the results.

Users typically report performance problems in the following situations:

v Response times for transactions or queries take longer than expected.
v Transaction throughput is insufficient to complete the required workload.
v Transaction throughput decreases.

An iterative approach to optimizing database server performance is recommended.

If repeating the steps found in the following list does not produce the desired
improvement, insufficient hardware resources or inefficient code in one or more
client applications might be causing the problem.

To optimize performance:
1. Establish performance objectives.
2. Take regular measurements of resource utilization and database activity.
3. Identify symptoms of performance problems: disproportionate utilization of
CPU, memory, or disks.

© Copyright IBM Corp. 1996, 2014 1-1

 4. Tune the operating-system configuration.
5. Tune the database server configuration.
6. Optimize the chunk and dbspace configuration, including placement of logs,
sort space, and space for temporary tables and sort files.
7. Optimize the table placement, extent sizing, and fragmentation.
8. Improve the indexes.
9. Optimize background I/O activities, including logging, checkpoints, and page
cleaning.
10. Schedule backup and batch operations for off-peak hours.
11. Optimize the implementation of the database application.
12. Repeat steps 2 through 11.

Quick start for acceptable performance on a small database

If you have a small database with each table residing on only one disk and using
only one CPU virtual processor, you can take specific measurements to help you
anticipate and correct performance problems.

To achieve acceptable initial performance on a small database:

1. Generate statistics of your tables and indexes to provide information to the
query optimizer to enable it to choose query plans with the lowest estimated
cost.
These statistics are a minimum starting point to obtain good performance for
individual queries. For guidelines, see “Update statistics when they are not
generated automatically” on page 13-11. To see the query plan that the
optimizer chooses for each query, see “Display the query plan” on page 13-1.
2. If you want a query to run in parallel with other queries, you must turn on the
Parallel Database Query (PDQ) feature.
Without table fragmentation across multiple disks, parallel scans do not occur.
With only one CPU virtual processor, parallel joins or parallel sorts do not
occur. However, PDQ priority can obtain more memory to perform the sort. For
more information, see Chapter 12, “Parallel database query (PDQ),” on page
12-1.
3. If you want to mix online transaction processing (OLTP) and decision-support
system (DSS) query applications, you can control the amount of resources a
long-running query can obtain so that your OLTP transactions are not affected.
For information about how to control PDQ resources, see “The allocation of
resources for parallel database queries” on page 12-7.
4. Monitor sessions and drill down into various details to improve the
performance of individual queries.
For information about the various tools and session details to monitor, see
“Monitoring memory usage for each session” on page 13-43 and “Monitor
sessions and threads” on page 13-47.

Performance goals
When you plan for measuring and tuning performance, you should consider
performance goals and determine which goals are the most important.

Many considerations go into establishing performance goals for the database server
and the applications that it supports. Be clear and consistent about articulating
performance goals and priorities, so that you can provide realistic and consistent

1-2 IBM Informix Performance Guide

 expectations about the performance objectives for your application. Consider the
following questions when you establish performance goals:
v Is your top priority to maximize transaction throughput, minimize response time
for specific queries, or achieve the best overall mix?
v What sort of mix between simple transactions, extended decision-support
queries, and other types of requests does the database server typically handle?
v At what point are you willing to trade transaction-processing speed for
availability or the risk of loss for a particular transaction?
v Is this database server instance used in a client/server configuration? If so, what
are the networking characteristics that affect its performance?
v What is the maximum number of users that you expect?
v Is your configuration limited by memory, disk space, or CPU resources?

The answers to these questions can help you set realistic performance goals for
your resources and your mix of applications.

Measurements of performance
You can use throughput, response time, cost per transaction, and resource
utilization measures to evaluate performance.

Throughput, response time, and cost per transaction are described in the topics
that follow.

Resource utilization can have one of two meanings, depending on the context. The
term can refer to the amount of a resource that a particular operation requires or
uses, or it can refer to the current load on a particular system component. The term
is used in the former sense to compare approaches for accomplishing a given task.
For instance, if a given sort operation requires 10 megabytes of disk space, its
resource utilization is greater than another sort operation that requires only 5
megabytes of disk space. The term is used in the latter sense to refer, for instance,
to the number of CPU cycles that are devoted to a particular query during a
specific time interval.

For a discussion about the performance impact of different load levels on various
system components, see “Resource utilization and performance” on page 1-7.

Throughput
Throughput measures the overall performance of the system. For transaction
processing systems, throughput is typically measured in transactions per second
(TPS) or transactions per minute (TPM).

Throughput depends on the following factors:

v The specifications of the host computer
v The processing overhead in the software
v The layout of data on disk
v The degree of parallelism that both hardware and software support
v The types of transactions being processed

Ways to measure throughput

The best way to measure throughput for an application is to include code in the
application that logs the time stamps of transactions as they commit.

Chapter 1. Performance basics 1-3

 If your application does not provide support for measuring throughput directly,
you can obtain an estimate by tracking the number of COMMIT WORK statements
that the database server logs during a given time interval. You can use the onlog
utility to obtain a listing of logical-log records that are written to log files. You can
use information from this command to track insert, delete, and update operations
as well as committed transactions. However, you cannot obtain information stored
in the logical-log buffer until that information is written to a log file.

If you need more immediate feedback, you can use onstat -p to gather an estimate.
You can use the SET LOG statement to set the logging mode to unbuffered for the
databases that contain tables of interest. You can also use the trusted auditing
facility in the database server to record successful COMMIT WORK events or other
events of interest in an audit log file. Using the auditing facility can increase the
overhead involved in processing any audited event, which can reduce overall
throughput.
Related information:
Auditing data security (Security Guide)

Standard throughput benchmarks

The Transaction Processing Performance Council (TPC) provides standard
benchmarks that allow reasonable throughput comparisons across hardware
configurations and database servers. IBM is an active member in good standing of
the TPC.

The TPC provides the following standardized benchmarks for measuring

throughput:
v TPC-A
This benchmark is used for simple online transaction-processing (OLTP)
comparisons. It characterizes the performance of a simple transaction-processing
system, emphasizing update-intensive services. TPC-A simulates a workload that
consists of multiple user sessions connected over a network with significant disk
I/O activity.
v TPC-B
This benchmark is used for stress-testing peak database throughput. It uses the
same transaction load as TPC-A but removes any networking and interactive
operations to provide a best-case throughput measurement.
v TPC-C
This benchmark is used for complex OLTP applications. It is derived from
TPC-A and uses a mix of updates, read-only transactions, batch operations,
transaction rollback requests, resource contentions, and other types of operations
on a complex database to provide a better representation of typical workloads.
v TPC-D
This benchmark measures query-processing power in terms of completion times
for very large queries. TPC-D is a decision-support benchmark built around a set
of typical business questions phrased as SQL queries against large databases (in
the gigabyte or terabyte range).

Because every database application has its own particular workload, you cannot
use TPC benchmarks to predict the throughput for your application. The actual
throughput that you achieve depends largely on your application.

1-4 IBM Informix Performance Guide

 Response time
Response time measures the performance of an individual transaction or query.
Response time is typically treated as the elapsed time from the moment that a user
enters a command or activates a function until the time that the application
indicates that the command or function has completed.

The response time for a typical Informix application includes the following
sequence of actions. Each action requires a certain amount of time. The response
time does not include the time that it takes for the user to think of and enter a
query or request:
1. The application forwards a query to the database server.
2. The database server performs query optimization and retrieves any
user-defined routines (UDRs). UDRs include both SPL routines and external
routines.
3. The database server retrieves, adds, or updates the appropriate records and
performs disk I/O operations directly related to the query.
4. The database server performs any background I/O operations, such as logging
and page cleaning, that occur during the period in which the query or
transaction is still pending.
5. The database server returns a result to the application.
6. The application displays the information or issues a confirmation and then
issues a new prompt to the user.

Figure 1-1 contains a diagram that shows how the actions just described in steps 1
through 6 contribute to the overall response time.

DB-Access Database server

custno custname
SELECT*in 1234 XYZLTD
Database Background 1235 XSPORTS

User enters Application Database Database Database Database Client application

request (not forwards server server server server receives, processes,
included in request to optimizes retrieves or performs modifies data and displays results
response database query and adds selected background I/O values and from database server.
time). server. retrieves records. (sometimes sends results
user-defined affects to client.
routines. response time).

Overall response time

Figure 1-1. Components of the response time for a single transaction

Response time and throughput

Response time and throughput are related. The response time for an average
transaction tends to decrease as you increase overall throughput.

However, you can decrease the response time for a specific query, at the expense of
overall throughput, by allocating a disproportionate amount of resources to that
query. Conversely, you can maintain overall throughput by restricting the resources
that the database allocates to a large query.

Chapter 1. Performance basics 1-5

 The trade-off between throughput and response time becomes evident when you
try to balance the ongoing need for high transaction throughput with an
immediate need to perform a large decision-support query. The more resources
that you apply to the query, the fewer you have available to process transactions,
and the larger the impact your query can have on transaction throughput.
Conversely, the fewer resources you allow the query, the longer the query takes.

Response-time measurement
To measure the response time for a query or application, you can use the timing
commands and performance monitoring and timing functions that your operating
system provides.

Operating-system timing commands:

Your operating system typically has a utility that you can use to time a command.
You can often use this timing utility to measure the response times to SQL
statements that a DB-Access command file issues.
UNIX Only
If you have a command file that performs a standard set of SQL
statements, you can use the time command on many systems to obtain an
accurate timing for those commands.
The following example shows the output of the UNIX time command:
time commands.dba
...
4.3 real 1.5 user 1.3 sys
The time output lists the amount of elapsed time (real), the user CPU time,
and the system CPU time. If you use the C shell, the first three columns of
output from the C shell time command show the user, system, and elapsed
times, respectively. In general, an application often performs poorly when
the proportion of system CPU time exceeds one-third of the total elapsed
time.
The time command gathers timing information about your application. You
can use this command to invoke an instance of your application, perform a
database operation, and then exit to obtain timing figures, as the following
example illustrates:
time sqlapp
(enter SQL command through sqlapp, then exit)
10.1 real 6.4 user 3.7 sys
You can use a script to run the same test repeatedly, which allows you to
obtain comparable results under different conditions. You can also obtain
estimates of your average response time by dividing the elapsed time for
the script by the number of database operations that the script performs.

Operating-system tools for monitoring performance:

Operating systems usually have a performance monitor that you can use to
measure response time for a query or process.
Windows Only
You can often use the Performance Logs and Alerts that the Windows
operating system supplies to measure the following times:
v User time
v Processor time

1-6 IBM Informix Performance Guide

 v Elapsed time

Timing functions within your application:

Most programming languages have a library function for the time of day. If you
have access to the source code, you can insert pairs of calls to this function to
measure the elapsed time between specific actions.
ESQL/C Only
For example, if the application is written in IBM Informix ESQL/C, you
can use the dtcurrent() function to obtain the current time. To measure
response time, you can call dtcurrent() to report the time at the start of a
transaction and again to report the time when the transaction commits.

Elapsed time, in a multiprogramming system or network environment where

resources are shared among multiple processes, does not always correspond to
execution time. Most operating systems and C libraries contain functions that
return the CPU time of a program.

Cost per transaction

The cost per transaction is a financial measure that is typically used to compare
overall operating costs among applications, database servers, or hardware
platforms. You can measure the cost per transaction.

To measure the cost per transaction:

1. Calculate all the costs associated with operating an application. These costs can
include the installed price of the hardware and software; operating costs,
including salaries; and other expenses. These costs can include the installed
price of the hardware and software; operating costs, including salaries; and
other expenses.
2. Project the total number of transactions and queries for the effective life of an
application.
3. Divide the total cost over the total number of transactions.

Although this measure is useful for planning and evaluation, it is seldom relevant
to the daily issues of achieving optimum performance.

Resource utilization and performance

A typical transaction-processing application undergoes different demands
throughout its various operating cycles. Peak loads during the day, week, month,
and year, as well as the loads imposed by decision-support (DSS) queries or
backup operations, can significantly impact any system that is running near
capacity. You can use direct historical data derived from your particular system to
pinpoint this impact.

You must take regular measurements of the workload and performance of your
system to predict peak loads and compare performance measurements at different
points in your usage cycle. Regular measurements help you to develop an overall
performance profile for your database server applications. This profile is critical in
determining how to improve performance reliably.

Chapter 1. Performance basics 1-7

 For the measurement tools that the database server provides, see “Database server
tools” on page 2-3. For the tools that your operating system provides for
measuring performance impacts on system and hardware resources, see
“Operating-system tools” on page 2-3.

Utilization is the percentage of time that a component is actually occupied, as

compared with the total time that the component is available for use. For instance,
if a CPU processes transactions for a total of 40 seconds during a single minute, its
utilization during that interval is 67 percent.

Measure and record utilization of the following system resources regularly:

v CPU
v Memory
v Disk

A resource is said to be critical to performance when it becomes overused or when

its utilization is disproportionate to that of other components. For instance, you
might consider a disk to be critical or overused when it has a utilization of 70
percent and all other disks on the system have 30 percent. Although 70 percent
does not indicate that the disk is severely overused, you can improve performance
by rearranging data to balance I/O requests across the entire set of disks.

How you measure resource utilization depends on the tools that your operating
system provides for reporting system activity and resource utilization. After you
identify a resource that seems overused, you can use the performance-monitoring
utilities that the database server provides to gather data and make inferences about
the database activities that might account for the load on that component. You can
adjust your database server configuration or your operating system to reduce those
database activities or spread them among other components. In some cases, you
might need to provide additional hardware resources to resolve a performance
bottleneck.

Resource utilization
Whenever a system resource, such as a CPU or a particular disk, is occupied by a
transaction or query, the resource is unavailable for processing other requests.
Pending requests must wait for the resources to become available before they can
complete.

When a component is too busy to keep up with all its requests, the overused
component becomes a bottleneck in the flow of activity. The higher the percentage
of time that the resource is occupied, the longer each operation must wait for its
turn.

You can use the following formula to estimate the service time for a request based
on the overall utilization of the component that services the request. The expected
service time includes the time that is spent both waiting for and using the resource
in question. Think of service time as that portion of the response time accounted
for by a single component within your computer, as the following formula shows:
S= P/(1-U)
S is the expected service time.
P is the processing time that the operation requires after it obtains the
resource.
U is the utilization for the resource (expressed as a decimal).

1-8 IBM Informix Performance Guide

 As Figure 1-2 shows, the service time for a single component increases dramatically
as the utilization increases beyond 70 percent. For instance, if a transaction requires
1 second of processing by a given component, you can expect it to take 2 seconds
on a component at 50 percent utilization and 5 seconds on a component at 80
percent utilization. When utilization for the resource reaches 90 percent, you can
expect the transaction to take 10 seconds to make its way through that component.

Elapsed 12
time (as a 10
multiple of 8
processing 6
time) in 4
minutes
2
0
0 10 20 30 40 50 60 70 80 90 100
Resource utilization (%)

Figure 1-2. Service Time for a Single Component as a Function of Resource Utilization

If the average response time for a typical transaction soars from 2 or 3 seconds to
10 seconds or more, users are certain to notice and complain.

Important: Monitor any system resource that shows a utilization of over 70

percent or any resource that exhibits symptoms of overuse as described in the
following sections.

When you consider resource utilization, also consider whether increasing the page
size of a standard or temporary dbspace is beneficial in your environment. If you
want a longer key length than is available for the default page size of a standard
or temporary dbspace, you can increase the page size.

CPU utilization
Estimates of CPU utilization and response time can help you determine if you
need to eliminate or reschedule some activities.

You can use the resource-utilization formula in the previous topic (“Resource
utilization” on page 1-8) to estimate the response time for a heavily loaded CPU.
However, high utilization for the CPU does not always indicate a performance
problem. The CPU performs all calculations that are needed to process
transactions. The more transaction-related calculations that it performs within a
given period, the higher the throughput will be for that period. As long as
transaction throughput is high and seems to remain proportional to CPU
utilization, a high CPU utilization indicates that the computer is being used to the
fullest advantage.

On the other hand, when CPU utilization is high but transaction throughput does
not keep pace, the CPU is either processing transactions inefficiently or it is
engaged in activity not directly related to transaction processing. CPU cycles are
being diverted to internal housekeeping tasks such as memory management.

You can easily eliminate the following activities:

v Large queries that might be better scheduled at an off-peak time

Chapter 1. Performance basics 1-9

 v Unrelated application programs that might be better performed on another
computer

If the response time for transactions increases to such an extent that delays become
unacceptable, the processor might be swamped; the transaction load might be too
high for the computer to manage. Slow response time can also indicate that the
CPU is processing transactions inefficiently or that CPU cycles are being diverted.

When CPU utilization is high, a detailed analysis of the activities that the database
server performs can reveal any sources of inefficiency that might be present due to
improper configuration. For information about analyzing database server activity,
see “Database server tools” on page 2-3.

Memory utilization
Memory is not managed as a single component, such as a CPU or disk, but as a
collection of small components called pages.

The size of a typical page in memory can range from 1 to 8 kilobytes, depending
on your operating system. A computer with 64 megabytes of memory and a page
size of 2 kilobytes contains approximately 32,000 pages.

When the operating system needs to allocate memory for use by a process, it
scavenges any unused pages within memory that it can find. If no free pages exist,
the memory-management system has to choose pages that other processes are still
using and that seem least likely to be needed in the short run. CPU cycles are
required to select those pages. The process of locating such pages is called a page
scan. CPU utilization increases when a page scan is required.

Memory-management systems typically use a least recently used algorithm to select

pages that can be copied out to disk and then freed for use by other processes.
When the CPU has identified pages that it can appropriate, it pages out the old
page images by copying the old data from those pages to a dedicated disk. The
disk or disk partition that stores the page images is called the swap disk, swap space,
or swap area. This paging activity requires CPU cycles as well as I/O operations.

Eventually, page images that have been copied to the swap disk must be brought
back in for use by the processes that require them. If there are still too few free
pages, more must be paged out to make room. As memory comes under increasing
demand and paging activity increases, this activity can reach a point at which the
CPU is almost fully occupied with paging activity. A system in this condition is
said to be thrashing. When a computer is thrashing, all useful work comes to a halt.

To prevent thrashing, some operating systems use a coarser memory-management

algorithm after paging activity crosses a certain threshold. This algorithm is called
swapping. When the memory-management system resorts to swapping, it
appropriates all pages that constitute an entire process image at once, rather than a
page at a time.

Swapping frees up more memory with each operation. However, as swapping

continues, every process that is swapped out must be read in again, dramatically
increasing disk I/O to the swap device and the time required to switch between
processes. Performance is then limited to the speed at which data can be
transferred from the swap disk back into memory. Swapping is a symptom of a
system that is severely overloaded, and throughput is impaired.

1-10 IBM Informix Performance Guide

 Many systems provide information about paging activity that includes the number
of page scans performed, the number of pages sent out of memory (paged out), and
the number of pages brought in from memory (paged in):
v Paging out is the critical factor because the operating system pages out only
when it cannot find pages that are free already.
v A high rate of page scans provides an early indicator that memory utilization is
becoming a bottleneck.
v Pages for terminated processes are freed in place and simply reused, so
paging-in activity does not provide an accurate reflection of the load on memory.
A high rate of paging in can result from a high rate of process turnover with no
significant performance impact.

Although the principle for estimating the service time for memory is the same as
that described in “Resource utilization and performance” on page 1-7, you use a
different formula to estimate the performance impact of memory utilization than
you do for other system components.

You can use the following formula to calculate the expected paging delay for a
given CPU utilization level and paging rate:
PD= (C/(1-U)) * R * T
PD is the paging delay.
C is the CPU service time for a transaction.
U is the CPU utilization (expressed as a decimal).
R is the paging-out rate.
T is the service time for the swap device.

As paging increases, CPU utilization also increases, and these increases are
compounded. If a paging rate of 10 per second accounts for 5 percent of CPU
utilization, increasing the paging rate to 20 per second might increase CPU
utilization by an additional 5 percent. Further increases in paging lead to even
sharper increases in CPU utilization, until the expected service time for CPU
requests becomes unacceptable.

Disk utilization
Because transfer rates vary among disks, most operating systems do not report
disk utilization directly. Instead, they report the number of data transfers per
second (in operating-system memory-page-size units.)

Because each disk acts as a single resource, you can use the following basic
formula to estimate the service time, which is described in detail in “Resource
utilization” on page 1-8:
S= P/(1-U)

To compare the load on disks with similar access times, simply compare the
average number of transfers per second.

If you know the access time for a given disk, you can use the number of transfers
per second that the operating system reports to calculate utilization for the disk. To
do so, multiply the average number of transfers per second by the access time for
the disk as listed by the disk manufacturer. Depending on how your data is laid

Chapter 1. Performance basics 1-11

 out on the disk, your access times can vary from the rating of the manufacturer. To
account for this variability, you should add 20 percent to the access-time
specification of the manufacturer.

The following example shows how to calculate the utilization for a disk with a
30-millisecond access time and an average of 10 transfer requests per second:
U = (A * 1.2) * X
= (.03 * 1.2) * 10
= .36
U is the resource utilization (this time of a disk).
A is the access time (in seconds) that the manufacturer lists.
X is the number of transfers per second that your operating system reports.

You can use the utilization to estimate the processing time at the disk for a
transaction that requires a given number of disk transfers. To calculate the
processing time at the disk, multiply the number of disk transfers by the average
access time. Include an extra 20 percent to account for access-time variability:
P = D (A * 1.2)
P is the processing time at the disk.
D is the number of disk transfers.
A is the access time (in seconds) that the manufacturer lists.

For example, you can calculate the processing time for a transaction that requires
20 disk transfers from a 30-millisecond disk as follows:
P = 20 (.03 * 1.2)
= 20 * .036
= .72

Use the processing time and utilization values that you calculated to estimate the
expected service time for I/O at the particular disk, as the following example
shows:
S = P/(1-U)
= .72 / (1 - .36)
= .72 / .64
= 1.13

Factors that affect resource utilization

The performance of your database server application depends many factors,
including hardware and software configuration, your network configuration, and
the design of your database.

You must consider these factors when you attempt to identify performance
problems or make adjustments to your system:
v Hardware resources
As discussed earlier in this chapter, hardware resources include the CPU,
physical memory, and disk I/O subsystems.
v Operating-system configuration
The database server depends on the operating system to provide low-level
access to devices, process scheduling, interprocess communication, and other
vital services.

1-12 IBM Informix Performance Guide

 The configuration of your operating system has a direct impact on how well the
database server performs. The operating-system kernel takes up a significant
amount of physical memory that the database server or other applications
cannot use. However, you must reserve adequate kernel resources for the
database server to use.
v Network configuration and traffic
Applications that depend on a network for communication with the database
server, and systems that rely on data replication to maintain high availability, are
subject to the performance constraints of that network. Data transfers over a
network are typically slower than data transfers from a disk. Network delays
can have a significant impact on the performance of the database server and
other application programs that run on the host computer.
v Database server configuration
Characteristics of your database server instance, such as the number of CPU
virtual processors (VPs), the size of your resident and virtual shared-memory
portions, and the number of users, play an important role in determining the
capacity and performance of your applications.
v Dbspace, blobspace, and chunk configuration
The following factors can affect the time that it takes the database server to
perform disk I/O and process transactions:
– The placement of the root dbspace, physical logs, logical logs, and
temporary-table dbspaces
– The presence or absence of mirroring
– The use of devices that are buffered or unbuffered by the operation system
v Database and table placement
The placement of tables and fragments within dbspaces, the isolation of high-use
fragments in separate dbspaces, and the spreading of fragments across multiple
dbspaces can affect the speed at which the database server can locate data pages
and transfer them to memory.
v Tblspace organization and extent sizing
Fragmentation strategy and the size and placement of extents can affect the
ability of the database server to scan a table rapidly for data. Avoid interleaved
extents and allocate extents that are sufficient to accommodate growth of a table
to prevent performance problems.
v Query efficiency
Proper query construction and cursor use can decrease the load that any one
application or user imposes. Remind users and application developers that
others require access to the database and that each person's activities affect the
resources that are available to others.
v Scheduling background I/O activities
Logging, checkpoints, page cleaning, and other operations, such as making
backups or running large decision-support queries, can impose constant
overhead and large temporary loads on the system. Schedule backup and batch
operations for off-peak times whenever possible.
v Remote client/server operations and distributed join operations
These operations have an important impact on performance, especially on a host
system that coordinates distributed joins.
v Application-code efficiency
Application programs introduce their own load on the operating system, the
network, and the database server. These programs can introduce performance
problems if they make poor use of system resources, generate undue network

Chapter 1. Performance basics 1-13

 traffic, or create unnecessary contention in the database server. Application
developers must make proper use of cursors and locking levels to ensure good
database server performance.

Maintenance of good performance

Performance is affected in some way by all system users: the database server
administrator, the database administrator, the application designers, and the client
application users.

The database server administrator usually coordinates the activities of all users to
ensure that system performance meets overall expectations. For example, the
operating-system administrator might need to reconfigure the operating system to
increase the amount of shared memory. Bringing down the operating system to
install the new configuration requires bringing the database server down. The
database server administrator must schedule this downtime and notify all affected
users when the system will be unavailable.

The database server administrator should:

v Be aware of all performance-related activities that occur.
v Educate users about the importance of performance, how performance-related
activities affect them, and how they can assist in achieving and maintaining
optimal performance.

The database administrator should pay attention to:

v How tables and queries affect the overall performance of the database server
v The placement of tables and fragments
v How the distribution of data across disks affects performance

Application developers should:

v Carefully design applications to use the concurrency and sorting facilities that
the database server provides, rather than attempt to implement similar facilities
in the application.
v Keep the scope and duration of locks to the minimum to avoid contention for
database resources.
v Include routines within applications that, when temporarily enabled at runtime,
allow the database server administrator to monitor response times and
transaction throughput.

Database users should:

v Pay attention to performance and report problems to the database server
administrator promptly.
v Be courteous when they schedule large, decision-support queries and request as
few resources as possible to get the work done.

1-14 IBM Informix Performance Guide

Chapter 2. Performance monitoring and the tools you use
You can use performance monitoring tools to create a performance history, to
monitor database resources at scheduled times, or to monitor ongoing transaction
or query performance.

This chapter also contains cross-references to topics that about how to interpret the
results of performance monitoring

The kinds of data that you need to collect depend on the kinds of applications that
you run on your system. The causes of performance problems on OLTP (online
transaction processing) systems are different from the causes of problems on
systems that are used primarily for DSS query applications. Systems with mixed
use provide a greater performance-tuning challenge and require a sophisticated
analysis of performance-problem causes.

Evaluate the current configuration

Before you begin to adjust the configuration of your database server, evaluate the
performance of your current configuration. You can view the contents of your
configuration file with onstat commands or IBM OpenAdmin Tool (OAT) for
Informix.

To alter certain database server characteristics, you must bring down the database
server, which can affect your production system. Some configuration adjustments
can unintentionally decrease performance or cause other negative side effects.

If your database applications satisfy user expectations, avoid frequent adjustments,

even if those adjustments might theoretically improve performance. If your users
are reasonably satisfied, take a measured approach to reconfiguring the database
server. When possible, use a test instance of the database server to evaluate
configuration changes before you reconfigure your production system.

When performance problems relate to backup operations, you might also examine
the number or transfer rates for tape drives. You might need to alter the layout or
fragmentation of your tables to reduce the impact of backup operations. For
information about disk layout and table fragmentation, see Chapter 6, “Table
performance considerations,” on page 6-1 and Chapter 7, “Indexes and index
performance considerations,” on page 7-1.

For client/server configurations, consider network performance and availability.

Evaluating network performance is beyond the scope of this publication. For
information about monitoring network activity and improving network availability,
see your network administrator or see the documentation for your networking
package.

Determine whether you want to set the configuration parameters that help
maintain server performance by automatically adjusting properties of the database
server while it is running, for example:
v AUTO_AIOVPS: Adds AIO virtual processors when I/O workload increases.
v AUTO_CKPTS: Increases the frequency of checkpoints to avoid transaction
blocking.

© Copyright IBM Corp. 1996, 2014 2-1

 v AUTO_LRU_TUNING: Manages cached data flushing as the server load
changes.
v AUTO_READAHEAD: Changes the automatic read-ahead mode or disables
automatic read-ahead operations for a query.
v AUTO_REPREPARE: Reoptimizes SPL routines and reprepares prepared objects
after a schema change.
v AUTO_STAT_MODE: Enables or disables the mode for selectively updating only
stale or missing data distributions in UPDATE STATISTICS operations.
v DYNAMIC_LOGS: Allocates additional log files when necessary.
v LOCKS: Allocates additional locks when necessary.
v RTO_SERVER_RESTART: Provides the best performance possible while meeting
the recovery time objective after a problem.
Related reference:
onstat -c command: Print ONCONFIG file contents (Administrator's Reference)

onstat -g cfg command: Print the current values of configuration parameters

(Administrator's Reference)

Create a performance history

As soon as you set up your database server and begin to run applications on it,
you should begin scheduled monitoring of resource use. As you accumulate data,
you can analyze performance information.

To accumulate data for performance analysis, use the command-line utilities

described in “Database server tools” on page 2-3 and “Operating-system tools” on
page 2-3 in operating scripts or batch files.

The importance of a performance history

If you have a history of the performance of your system, you can begin to track
the cause of problems as soon as users report slow response or inadequate
throughput.

If a history is not available, you must start tracking performance after a problem
arises, and you might not be able to tell when and how the problem began. Trying
to identify problems after the fact significantly delays resolution of a performance
problem.

To build a performance history and profile of your system, take regular snapshots
of resource-utilization information.

For example, if you chart the CPU utilization, paging-out rate, and the I/O transfer
rates for the various disks on your system, you can begin to identify peak-use
levels, peak-use intervals, and heavily loaded resources.

If you monitor fragment use, you can determine whether your fragmentation
scheme is correctly configured. Monitor other resource use as appropriate for your
database server configuration and the applications that run on it.

2-2 IBM Informix Performance Guide

 Choose tools from those described in the following sections, and create jobs that
build up a history of disk, memory, I/O, and other database server resource use.
To help you decide which tools to use to create a performance history, this chapter
briefly describes the output of each tool.

Tools that create a performance history

When you monitor database server performance, you use tools from the host
operating system and command-line utilities that you can run at regular intervals
from scripts or batch files.

You also use performance monitoring tools with a graphical interface to monitor
critical aspects of performance as queries and transactions are performed.

Operating-system tools
The database server relies on the operating system of the host computer to provide
access to system resources such as the CPU, memory, and various unbuffered disk
I/O interfaces and files. Each operating system has its own set of utilities for
reporting how system resources are used.

Different implementations of some operating systems have monitoring utilities

with the same name but different options and informational displays.

UNIX Only

The following table lists some UNIX utilities that monitor system resources.

UNIX Utility Description

vmstat utility Displays virtual-memory statistics
iostat utility Displays I/O utilization statistics
sar utility Displays a variety of resource statistics
ps utility Displays active process information

For details on how to monitor your operating-system resources, consult the

reference manual or your system administration guide.

To capture the status of system resources at regular intervals, use scheduling tools
that are available with your host operating system (for example, cron) as part of
your performance monitoring system.

Windows Only

You can often use the Performance Logs and Alerts that the Windows operating
system supplies to monitor resources such as processor, memory, cache, threads,
and processes. The Performance Logs and Alerts also provide charts, alerts,
reports, and the ability to save information to log files for later analysis.

For more information about how to use the Performance Logs and Alerts, consult
your operating-system manuals.

Database server tools

The database server provides tools and utilities that capture snapshot information
about your configuration and performance.

Chapter 2. Performance monitoring and the tools you use 2-3

 You can use these utilities regularly to build a historical profile of database activity,
which you can compare with current operating-system resource-utilization data.
These comparisons can help you discover which database server activities have the
greatest impact on system-resource utilization. You can use this information to
identify and manage your high-impact activities or adjust your database server or
operating-system configuration.

The database server tools and utilities that you can use for performance
monitoring include:
v IBM OpenAdmin Tool (OAT) for Informix
v The onstat utility
v The onlog utility
v The oncheck utility
v The ON-Monitor utility (on UNIX only)
v The onperf utility (on UNIX only)
v DB-Access and the system-monitoring interface (SMI), which you can use to
monitor performance from within your application
v SQL administration API commands

You can use onstat, onlog, or oncheck commands invoked by the cron scheduling
facility to capture performance-related information at regular intervals and build a
historical performance profile of your database server application. The following
sections describe these utilities.

You can use SQL SELECT statements to query the system-monitoring interface
(SMI) from within your application.

The SMI tables are a collection of tables and pseudo-tables in the sysmaster
database that contain dynamically updated information about the operation of the
database server. The database server constructs these tables in memory but does
not record them on disk. The onstat utility options obtain information from these
SMI tables.

You can use cron and SQL scripts with DB-Access or onstat utility options to
query SMI tables at regular intervals.

Tip: The SMI tables are different from the system catalog tables. System catalog
tables contain permanently stored and updated information about each database
and its tables (sometimes referred to as metadata or a data dictionary).

You can use ON-Monitor to check the current database server configuration.

You can use onperf to display database server activity with the Motif window
manager.
Related concepts:
The onlog utility (Administrator's Reference)
The oncheck Utility (Administrator's Reference)
DB-Access User's Guide (DB-Access Guide)
System catalog tables (SQL Reference)
Chapter 14, “The onperf utility on UNIX,” on page 14-1
Related reference:

2-4 IBM Informix Performance Guide

 The onstat utility (Administrator's Reference)
The System-Monitoring Interface Tables (Administrator's Reference)
SQL administration API portal: Arguments by functional category
(Administrator's Reference)

Performance information that IBM Informix Server Administrator provides:

IBM Informix Server Administrator (ISA) is a browser-based tool that provides

Web-based system administration for the entire range of IBM Informix database
servers.

ISA is the first in a new generation of browser-based, cross-platform administrative

tools. It provides access to every Informix database server command-line function
and presents the output in an easy-to-read format.

The database server CD-ROM distributed with your product includes ISA. For
information on how to install ISA, see the following file on the CD-ROM.
Table 2-1. Operating system file
Operating System File
UNIX /SVR_ADM/README
Windows \SVR_ADM\readme.txt

With ISA, you can use a browser to perform these common database server
administrative tasks:
v Change configuration parameters temporarily or permanently
v Change the database server mode between online and offline and its
intermediate states
v Modify connectivity information in the sqlhosts file
v Check dbspaces, sbspaces, logs, and other objects
v Manage logical and physical logs
v Examine memory use and adding and freeing memory segments
v Read the message log
v Back up and restore dbspaces and sbspaces
v Run various onstat commands to monitor performance
v Enter simple SQL statements and examine database schemas
v Add and remove chunks, dbspaces, and sbspaces
v Examine and manage user sessions
v Examine and manage virtual processors (VPs)
v Use the High-Performance Loader (HPL), dbimport, and dbexport
v Manage Enterprise Replication
v Manage an IBM Informix MaxConnect server
v Use the following utilities: dbaccess, dbschema, onbar, oncheck, ondblog,
oninit, onlog, onmode, onparams, onspaces, and onstat

You also can enter any Informix utility, UNIX shell command, or Windows
command (for example, oncheck -cd; ls -l).

Chapter 2. Performance monitoring and the tools you use 2-5

 Performance information that the onstat utility displays:

The onstat utility displays a wide variety of performance-related and status

information contained within the SMI tables. You can use the onstat utility to
check the current status of the database server and monitor the activities of the
database server.

For a complete list of all onstat options, use the onstat - - command. For a
complete display of all the information that onstat gathers, use the onstat -a
command.

Tip: Profile information displayed by onstat commands, such as onstat -p,

accumulates from the time the database server was started. To clear performance
profile statistics so that you can create a new profile, run the onstat -z. If you use
onstat -z to reset statistics for a performance history or appraisal, ensure that other
users do not also enter the command at different intervals.

The following table lists some of the onstat commands that display general
performance-related information.
Table 2-2. onstat commands that display performance information
onstat command Description
onstat -p Displays a performance profile that includes
the number of reads and writes, the number
of times that a resource was requested but
was not available, and other miscellaneous
information
onstat -b Displays information about buffers currently
in use
onstat -l Displays information about the physical and
logical logs
onstat -x Displays information about transactions,
including the thread identifier of the user
who owns the transaction
onstat -u Displays a user activity profile that provides
information about user threads including the
thread owner's session ID and login name
onstat -R Displays information about buffer pools,
including information about buffer pool page
size.
onstat -F Displays page-cleaning statistics that include
the number of writes of each type that
flushes pages to disk
onstat -g Requires an additional argument that
specifies the information to be displayed

For example, onstat -g mem displays

memory statistics.

For more information about options that provide performance-related information,

see “Monitoring fragmentation with the onstat -g ppf command” on page 9-28 and
“Monitor database server resources” on page 2-7.

2-6 IBM Informix Performance Guide

 Related reference:
onstat -g monitoring options (Administrator's Reference)

Monitor performance with the OpenAdmin Tool (OAT) for Informix

The IBM OpenAdmin Tool (OAT) for Informix provides multiple ways to gather,
view, and analyze performance data.

With OAT, you can:

v Collect performance statistics.
v Find and eliminate database server performance bottlenecks.
v Identify and monitor queries that are critical to performance.
v Improve checkpoint performance and manage LRU queues.
v Manage compression of data in tables and table-fragment rows
v Monitor critical system resources (CPU, memory, disk, virtual processors).
v Monitor and track locking.
v Optimize the disk layout.
v Tune the buffer cache.
v Use query drill-down.
v View the SQL statement cache.
v Use automatic statistics update.
v View historical performance graphs.
v Explore user sessions.

Monitor database server resources

Monitor specific database server resources to identify performance bottlenecks and
potential trouble spots and to improve resource use and response time.

One of the most useful commands for monitoring system resources is onstat -g
and its many options.

Monitor resources that impact CPU utilization

Threads, network communications, and virtual processors impact CPU utilization.
You can use onstat -g arguments to monitor threads, network communications,
and virtual processors.

Use the following onstat -g command options to monitor threads.

onstat -g Option Description

act Displays active threads.
ath Displays all threads.

The sqlexec threads represent portions of

client sessions; the rstcb value corresponds to
the user field of the onstat -u command.
cpu Displays the last time the thread ran, how
much CPU time the thread used, the number
of times the thread ran, and other statistics
about all the threads running in the server.

Chapter 2. Performance monitoring and the tools you use 2-7

 onstat -g Option Description
rea Displays ready threads.
sle Displays all sleeping threads.
sts Displays maximum and current stack use per
thread.
tpf tid Displays a thread profile for tid.

If tid is 0, this argument displays profiles for

all threads.
wai Displays waiting threads, including all
threads waiting on mutex or condition, or
yielding.

Use the following onstat -g command options to monitor the network.

onstat -g Command Option Description

ntd Displays network statistics by service.
ntt Displays network user times.
ntu Displays network user statistics.
qst Displays queue statistics.

Use the following onstat -g command options to monitor virtual processors.

onstat -g Command Option Description

glo Displays global multithreading information,
including CPU-use information about virtual
processors, the total number of sessions, and
other multithreading global counters.
sch Displays the number of semaphore
operations, spins, and busy waits for each
VP.
spi Displays spin locks that are acquired by
virtual processors after they have spun more
than 10,000 times.

To reduce contention, reduce the number of

virtual processors, reduce the load on the
computer, or, on some platforms, use the
no-age or processor affinity options of
virtual processors. If sh_lock mutexes have
highly contended spin locks, create private
memory caches for CPU virtual processors
by setting the VP_MEMORY_CACHE_VP
configuration parameter.
wst Displays wait statistics.

Monitor memory utilization

You can use some specific onstat -g command options to monitor memory
utilization.

2-8 IBM Informix Performance Guide

Use the following onstat -g options to monitor memory utilization. For overall
memory information, omit table name, pool name, or session id from the commands
that permit those optional parameters.
Table 2-3. onstat -g Options for monitoring memory utilization
Argument Description
ffr pool name | session id Displays free fragments for a pool of shared
memory or by session
dic table name Displays one line of information for each
table cached in the shared-memory
dictionary

If you provide a specific table name as a

parameter, this argument displays internal
SQL information about that table.
dsc Displays one line of information for each
column of distribution statistics cached in the
data distribution cache.
mem pool name | session id Displays memory statistics for the pools that
are associated with a session

If you omit pool_name | session id, this

argument displays pool information for all
sessions.
mgm Displays Memory Grant Manager resource
information, including:
v The values of the PDQ configuration
parameters
v Memory and scan information
v Load information, such as the number of
queries that are waiting for memory, the
number of queries that are waiting for
scans, the number of queries that are
waiting for queries with higher PDQ
priority to run, and the number of queries
that are waiting for a query slot
v Active queries and the number of queries
at each gate
v Statistics on free resources
v Statistics on queries
v The resource/lock cycle prevention count,
which shows the number of times the
system immediately activated a query to
avoid a potential deadlock
nsc client id Displays shared-memory status by client ID

If you omit client id, this argument displays

all client status areas.
nsd Displays network shared-memory data for
poll threads
nss session id Displays network shared-memory status by
session id

If you omit session id, this argument

displays all session status areas.

Chapter 2. Performance monitoring and the tools you use 2-9

 Table 2-3. onstat -g Options for monitoring memory utilization (continued)
Argument Description
osi Displays information about your operating
system resources and parameters, including
shared memory and semaphore parameters,
the amount of memory currently configured
on the computer, and the amount of memory
that is unused

Use this option when the server is not online.

prc Displays one line of information for each
user-defined routine (SPL routine or external
routine written in C or Java™ programming
language) cached in the UDR cache
seg Displays shared-memory-segment statistics

This argument shows the number and size of

all attached segments.
ses session id Displays memory usage for session id

If you omit session id, this argument

displays memory usage for all sessions.
ssc Displays one line of information for each
query cached in the SQL statement cache
stm session id Displays memory usage of each SQL
statement for session id

If you omit session id, this argument

displays memory usage for all sessions.
ufr pool name | session id Displays allocated pool fragments by user or
session

Related reference:
onstat -g monitoring options (Administrator's Reference)

Monitor disk I/O utilization

You can use some specific onstat -g arguments and the oncheck utility to
determine if your disk I/O operations are efficient for your applications.

Using onstat -g to monitor I/O utilization

You can use some specific onstat -g command arguments to monitor disk IO.

Use the following onstat -g command arguments to monitor disk I/O utilization.

onstat -g Argument Description

iof Displays asynchronous I/O statistics by
chunk or file

This argument is similar to the onstat -d,

except that information about nonchunk files
also appears. This argument displays
information about temporary dbspaces and
sort files.
iog Displays asynchronous I/O global
information

2-10 IBM Informix Performance Guide

onstat -g Argument Description
ioq Displays asynchronous I/O queuing statistics
iov Displays asynchronous I/O statistics by
virtual processor

For a detailed case study that uses various onstat outputs, see Appendix A, “Case
studies and examples,” on page A-1.

Using the oncheck utility to monitor I/O utilization

Disk I/O operations are usually the longest component of the response time for a
query. You can use the oncheck Utility to monitor disk I/O operations.

Contiguously allocated disk space improves sequential disk I/O operations,

because the database server can read in larger blocks of data and use the
read-ahead feature to reduce the number of I/O operations.

The oncheck utility displays information about storage structures on a disk,

including chunks, dbspaces, blobspaces, extents, data rows, system catalog tables,
and other options. You can also use oncheck to determine the number of extents
that exist within a table and whether or not a table occupies contiguous space.

The oncheck utility provides the following options and information that apply to
contiguous space and extents.

Option Information
-pB Blobspace simple large object (TEXT or BYTE data)

For information about how to use this option to determine the efficiency of
blobpage size, see “Determine blobpage fullness with oncheck -pB output”
on page 5-18.
-pe Chunks and extents

For information about how to use this option to monitor extents, see
“Checking for extent interleaving” on page 6-24 and “Eliminating interleaved
extents” on page 6-24.
-pk Index key values.

For information about how to improve the performance of this option, see
“Improving performance for index checks” on page 7-20.
-pK Index keys and row IDs

For information about how to improve the performance of this option, see
“Improving performance for index checks” on page 7-20.
-pl Index-leaf key values

For information about how to improve the performance of this option, see
“Improving performance for index checks” on page 7-20.
-pL Index-leaf key values and row IDs

For information about how to improve the performance of this option, see
“Improving performance for index checks” on page 7-20.
-pp Pages by table or fragment

For information about how to use this option to monitor space, see
“Considering the upper limit on extents” on page 6-24.

Chapter 2. Performance monitoring and the tools you use 2-11

 Option Information
-pP Pages by chunk

For information about how to use this option to monitor extents, see
“Considering the upper limit on extents” on page 6-24.
-pr Root reserved pages

For information about how to use this option, see “Estimating tables with
fixed-length rows” on page 6-5.
-ps Space used by smart large objects and metadata in sbspace.
-pS Space used by smart large objects and metadata in sbspace and storage
characteristics

For information about how to use this option to monitor space, see
“Monitoring sbspaces” on page 6-13.
-pt Space used by table or fragment

For information about how to use this option to monitor space, see
“Estimating table size” on page 6-5.
-pT Space used by table, including indexes

For information about how to use this option to monitor space, see
“Performance of in-place alters for DDL operations” on page 6-40.

For more information about using oncheck to monitor space, see “Estimating table
size” on page 6-5. For more information about concurrency during oncheck
execution, see “Improving performance for index checks” on page 7-20.
Related concepts:
The oncheck Utility (Administrator's Reference)

Monitor transactions
You can use the onlog and onstat utilities to monitor transactions.

Using the onlog utility to monitor transactions

The onlog utility displays all or selected portions of the logical log. This utility can
help you identify a problematic transaction or gauge transaction activity that
corresponds to a period of high utilization, as indicated by your periodic snapshots
of database activity and system-resource consumption.

This onlog utility can take input from selected log files, the entire logical log, or a
backup tape of previous log files.

Use onlog with caution when you read logical-log files still on disk, because
attempting to read unreleased log files stops other database activity. For greatest
safety, back up the logical-log files first and then read the contents of the backup
files. With proper care, you can use the onlog -n option to restrict onlog only to
logical-log files that have been released.

To check on the status of logical-log files, use onstat -l.

Related concepts:
The onlog utility (Administrator's Reference)

2-12 IBM Informix Performance Guide

 Using the onstat utility to monitor transactions
If the throughput of transactions is not very high, you can use some onstat utility
commands to identify a transaction that might be a bottleneck.

Use the following onstat utility commands to monitor transactions.

onstat command Description

onstat -x Displays transaction information such as
number of locks held and isolation level.
onstat -u Displays information about each user thread
onstat -k Displays locks held by each session
onstat -g sql Displays last SQL statement this session
executed

Related reference:
The onstat utility (Administrator's Reference)

Monitor sessions and queries

Monitoring sessions and threads is important for sessions that perform queries as
well as sessions that perform inserts, updates, and deletes. Some of the information
that you can monitor for sessions and threads allows you to determine if an
application is using a disproportionate amount of the resources.

To monitor database server activity, you can view the number of active sessions
and the amount of resources that they are using.

Monitoring memory usage for each session

You can use some specific onstat -g command arguments to get memory
information for each session.

Use the following command arguments to get memory information for each
session.

onstat -g command argument Description

ses Displays one-line summaries of all active
sessions
ses session id Displays session information by session id
sql session id Displays SQL information by session

If you omit session id, this argument displays

summaries of all sessions.
stm session id Displays amount of memory used by each
prepared SQL statement in a session

If you omit session id, this argument displays

information for all prepared statements.

For examples and discussions of session-monitoring command-line utilities, see

“Monitoring memory usage for each session” on page 13-43 and “Monitor sessions
and threads” on page 13-47.

Chapter 2. Performance monitoring and the tools you use 2-13

 Using the SET EXPLAIN statement
You can use the SET EXPLAIN statement or the EXPLAIN directive to display the
query plan that the optimizer creates for an individual query.

For more information, see “Display the query plan” on page 13-1.

2-14 IBM Informix Performance Guide

Chapter 3. Effect of configuration on CPU utilization
The combination of operating-system and Informix configuration parameters can
affect CPU utilization. You can change the settings of the Informix configuration
parameters that directly affect CPU utilization, and you can adjust the settings for
different types of workloads.

Multiple database server instances that run on the same host computer perform
poorly when compared with a single database server instance that manages
multiple databases. Multiple database server instances cannot balance their loads
as effectively as a single database server. Avoid multiple residency for production
environments in which performance is critical.

UNIX configuration parameters that affect CPU utilization

Your database server distribution includes a machine notes file that contains
recommended values for UNIX configuration parameters. Because the UNIX
parameters affect CPU utilization, you should compare the values in the machine
notes file with your current operating-system configuration.

The following UNIX parameters affect CPU utilization:

v Semaphore parameters
v Parameters that set the maximum number of open file descriptors
v Memory configuration parameters

UNIX semaphore parameters

Semaphores are kernel resources with a typical size of 1 byte each. Semaphores for
the database server are in addition to any that you allocate for other software
packages. You can set some UNIX semaphore parameters.

Each instance of the database server requires the following semaphore sets:
v One set for each group of up to 100 virtual processors (VPs) that are started with
the database server
v One set for each additional VP that you might add dynamically while the
database server is running
v One set for each group of 100 or fewer user sessions connected through the
shared-memory communication interface

Tip: For best performance, allocate enough semaphores for double the number of
ipcshm connections that you expect. Use the NETTYPE configuration parameter to
configure database server poll threads for this doubled number of connections.

Because utilities such as onmode use shared-memory connections, you must

configure a minimum of two semaphore sets for each instance of the database
server: one for the initial set of VPs and one for the shared-memory connections
that database server utilities use. The SEMMNI operating-system configuration
parameter typically specifies the number of semaphore sets to allocate. For
information about how to set semaphore-related parameters, see the configuration
instructions for your operating system.

© Copyright IBM Corp. 1996, 2014 3-1

 The SEMMSL operating-system configuration parameter typically specifies the
maximum number of semaphores per set. Set this parameter to at least 100.

Some operating systems require that you configure a maximum total number of
semaphores across all sets, which the SEMMNS operating-system configuration
parameter typically specifies. Use the following formula to calculate the total
number of semaphores that each instance of the database server requires:
SEMMNS = init_vps + added_vps + (2 * shmem_users) + concurrent_utils
init_vps
is the number of virtual processors (VPs) that are started with the database
server. This number includes CPU, PIO, LIO, AIO, SHM, TLI, SOC, and
ADM VPs. The minimum value is 15.
added_vps
is the number of VPs that you intend to add dynamically.
shmem_users
is the number of shared-memory connections that you allow for this
instance of the database server.
concurrent_utils
is the number of concurrent database server utilities that can connect to
this instance. It is suggested that you allow for a minimum of six utility
connections: two for ON-Archive or ON-Bar and four for other utilities
such as onstat, and oncheck.

If you use software packages that require semaphores, the SEMMNI configuration
parameter must include the total number of semaphore sets that the database
server and your other software packages require. You must set the SEMMSL
configuration parameter to the largest number of semaphores per set that any of
your software packages require. For systems that require the SEMMNS
configuration parameter, multiply SEMMNI by the value of SEMMSL to calculate
an acceptable value.
Related concepts:
“Configuring poll threads” on page 3-13

UNIX file-descriptor parameters

Some operating systems require you to specify a limit on the number of file
descriptors that a process can have open at any one time. To specify this limit, use
an operating-system configuration parameter, typically NOFILE, NOFILES, NFILE,
or NFILES.

The number of open file descriptors that each instance of the database server needs
depends on the number of chunks in your database, the number of VPs that you
run, and the number of network connections that your database server instance
must support.

Use the following formula to calculate the number of file descriptors that your
instance of the database server requires:
NFILES = (chunks * NUMAIOVPS) + NUMBER_of_CPU_VPS + net_connections
chunks is the number of chunks to be configured.
net_connections
is the number of network connections that you specify in either of the
following places:

3-2 IBM Informix Performance Guide

 v sqlhosts file
v NETTYPE configuration entries
Network connections include all but those specified as the ipcshm
connection type.

Each open file descriptor is about the same length as an integer within the kernel.
Allocating extra file descriptors is an inexpensive way to allow for growth in the
number of chunks or connections on your system.

UNIX memory configuration parameters

The configuration of memory in the operating system can affect other resources,
including CPU and I/O.

Insufficient physical memory for the overall system load can lead to thrashing, as
“Memory utilization” on page 1-10 describes. Insufficient memory for the database
server can result in excessive buffer-management activity. For more information
about configuring memory, see “Configuring UNIX shared memory” on page 4-6.

Windows configuration parameters that affect CPU utilization

The Informix distribution includes a machine notes file that contains recommended
values for Informix configuration parameters on Windows. Compare the values in
this file with your current ONCONFIG configuration file settings.

Informix runs in the background. For best performance, give the same priority to
foreground and background applications.

On Windows, to change the priorities of foreground and background applications,

go to Start > Settings > Control Panel, open the System icon, and click the
Advanced Tab. Select the Performance Options button and select either the
Applications or Background Services radio button.

The configuration of memory in the operating system can impact other resources,
including CPU and I/O. Insufficient physical memory for the overall system load
can lead to thrashing, as “Memory utilization” on page 1-10 describes. Insufficient
memory for Informix can result in excessive buffer-management activity. When you
set the Virtual Memory values in the System icon on the Control Panel, ensure
that you have enough paging space for the total amount of physical memory.

Configuration parameters and environment variables that affect CPU

utilization
Some configuration parameters and environment variables affect CPU utilization.
You might need to adjust the settings of these parameters and variables when you
consider methods of improving performance.

The following configuration parameters in the database server configuration file

have a significant impact on CPU utilization:
v DS_MAX_QUERIES
v DS_MAX_SCANS
v FASTPOLL
v MAX_PDQPRIORITY
v MULTIPROCESSOR

Chapter 3. Effect of configuration on CPU utilization 3-3

 v NETTYPE
v OPTCOMPIND
v SINGLE_CPU_VP
v VPCLASS
v VP_MEMORY_CACHE_KB

The following environment variables affect CPU utilization:

v OPTCOMPIND
v PDQPRIORITY
v PSORT_NPROCS

The OPTCOMPIND environment variable, when set in the environment of a client

application, indicates the preferred way to perform join operations. This variable
overrides the value that the OPTCOMPIND configuration parameter sets. For
details on how to select a preferred join method, see “Optimizing access methods”
on page 3-10.

The PDQPRIORITY environment variable, when set in the environment of a client

application, places a limit on the percentage of CPU VP utilization, shared memory,
and other resources that can be allocated to any query that the client starts.

A client can also use the SET PDQPRIORITY statement in SQL to set a value for
PDQ priority. The actual percentage allocated to any query is subject to the factor
that the MAX_PDQPRIORITY configuration parameter sets. For more information
about how to limit resources that can be allocated to a query, see “Limiting PDQ
resources in queries” on page 3-11.

PSORT_NPROCS, when set in the environment of a client application, indicates

the number of parallel sort threads that the application can use. The database
server imposes an upper limit of 10 sort threads per query for any application. For
more information about parallel sorts and PSORT_NPROCS, see “Configure
dbspaces for temporary tables and sort files” on page 5-8.
Related concepts:
Environment variables (SQL Reference)
Related reference:
Database configuration parameters (Administrator's Reference)

Specifying virtual processor class information

Use the VPCLASS configuration parameter to specify a class of virtual processors,
the number of virtual processors that the database server should start for a specific
class, and the maximum number allowed.

To execute user-defined routines (UDRs), you can define a new class of virtual
processors to isolate UDR execution from other transactions that execute on the
CPU virtual processors. Typically you write user-defined routines to support
user-defined data types.

If you do not want a user-defined routine to affect the normal processing of user
queries in the CPU class, you can use the CREATE FUNCTION statement to assign
the routine to a user-defined class of virtual processors. The class name that you
specify in the VPCLASS configuration parameter must match the name specified in
the CLASS modifier of the CREATE FUNCTION statement.

3-4 IBM Informix Performance Guide

For guidelines, on using the cpu and num options of the VPCLASS configuration
parameter, see “Setting the number of CPU VPs.”
Related reference:
VPCLASS configuration parameter (Administrator's Reference)
CREATE FUNCTION statement (SQL Syntax)

Setting the number of CPU VPs

You can configure the number of CPU virtual processors (VPs) that the database
server uses. Do not allocate more CPU VPs than there are CPU processors
available to service them.

When the database server starts, the number of CPU VPs is automatically
increased to half the number of CPU processors on the database server computer,
unless the SINGLE_CPU_VP configuration parameter is enabled. However, you
might want to change the number of CPU VPs based on your performance needs.

Use the following guidelines to set the number of CPU VPs:

Uniprocessor computers
For uniprocessor computers, specify one CPU VP:
VPCLASS cpu,num=1
Dual-processor computers
For dual-processor systems, you might improve performance by running
with two CPU VPs. To test if performance improves, set the num field of
the VPCLASS configuration parameter to 1 in the onconfig file and then
add a CPU VP dynamically at run time by running the onmode -p
command.
Multiprocessor computers that are primarily database servers
For multiprocessor systems with four or more CPUs that are primarily
used as database servers, set the num option of the VPCLASS
configuration parameter in the onconfig file to one less than the total
number of processors. For example, if you have four CPUs, use the
following specification:
VPCLASS cpu,num=3

When you use this setting, one processor is available to run the database
server utilities or the client application.
Multiprocessor computers that are not primarily database servers
For multiprocessor systems that you do not use primarily to support
database servers, you can start with somewhat fewer CPU VPs to allow for
other activities on the system and then gradually add more if necessary.
Multi-core or hardware multithreading computers with logical CPUs
For multiprocessor systems that use multi-core processors or hardware
multithreading to support more logical CPUs than physical processors, you
can assign the number of CPU VPs according to the number of logical
CPU VPs available for that purpose. The amount of processing that an
additional logical CPU can provide might be only a fraction of what a
dedicated physical processor can support.
On systems, where multi-core processors are installed, the optimal
configuration in most cases is the same as for systems with a number of

Chapter 3. Effect of configuration on CPU utilization 3-5

 individual processors equal to the total number of cores. Setting the
number of CPU VPs to N-1, where N is number of cores is close to optimal
for CPU-intensive workloads.
On computers where the CPU uses multiple threads per core, operating
systems show more logical processors than actual processing cores. To take
advantage of more CPU threads, the database server must be configured
with the number of CPU VPs in the range between N and M, where N is
number of cores and M is total number of logical CPUs reported by
system. The number of CPU VPs where optimal performance is achieved
depends on the workload.
When increasing the number of CPU VPs to use more threads per core, the
expected gain in performance is only a fraction of what dedicated physical
processor or core can provide.
If you are migrating Informix from multi-CPU/multicore systems to
systems with multiple threads per core, take special care in regard to
processor affinity. When binding Informix CPU VPs to the logical
processors of the operating system, you must be aware of the architecture
for the CPU. If you are not sure, do not use the CPU affinity so that the
operating system schedules CPU VPs to logical processors with available
resources. Using affinity without understanding the relationship between
the logical CPUs and processing cores can result in severe performance
degradation.
For example, to bind each of 8 configured CPU VPs to a separate core on
an 8-core system with two threads per core (16 logical CPUs), use the
following setting:
VPCLASS cpu,num=8,aff=(0-14/2)
Related concepts:
“Automatic addition of CPU virtual processors” on page 3-20
Related reference:
VPCLASS configuration parameter (Administrator's Reference)

Disabling process priority aging for CPU VPs

Use the noage option of the VPCLASS configuration parameter to disable process
priority aging for database server CPU VPs on operating systems that support this
feature. Priority aging occurs when the operating system lowers the priority of
long-running processes as they accumulate processing time. You might want to
disable priority aging because it can cause the performance of the database server
processes to decline over time.

Your database server distribution includes a machine notes file that contains
information about whether your version of the database server supports this
feature. For information about where to find this machine notes file, see the
Introduction to this guide.

Specify the noage option of VPCLASS if your operating system supports this
feature.
Related reference:
VPCLASS configuration parameter (Administrator's Reference)

3-6 IBM Informix Performance Guide

Specifying processor affinity
Use the aff option of the VPCLASS parameter to specify the processors to which
you want to bind CPU VPs or AIO VPs. When you assign a CPU VP to a specific
CPU, the VP runs only on that CPU. However, other processes can also run on that
CPU.

The database server supports automatic binding of CPU VPs to processors on

multiprocessor host computers that support processor affinity. Your database server
distribution includes a machine notes file that contains information about whether
your version of the database server supports this feature.

You can use processor affinity for the purposes that the following sections describe.
Related reference:
VPCLASS configuration parameter (Administrator's Reference)

Distributing computation impact:

You can use processor affinity to distribute the computation impact of CPU virtual
processors (VPs) and other processes. On computers that are dedicated to the
database server, assigning CPU VPs to all but one of the CPUs achieves maximum
CPU utilization.

On computers that support both database server and client applications, you can
bind applications to certain CPUs through the operating system. By doing so, you
effectively reserve the remaining CPUs for use by database server CPU VPs, which
you bind to the remaining CPUs with the VPCLASS configuration parameter. Set
the aff option of the VPCLASS configuration parameter to the numbers of the
CPUs on which to bind CPU VPs. For example, the following VPCLASS setting
assigns CPU VPs to processors 4 to 7:
VPCLASS cpu,num=4,aff=(4-7)

When specifying a range of processors, you can also specify an incremental value
with the range that indicates which CPUs in the range should be assigned to the
virtual processors. For example, you can specify that the virtual processors are
assigned to every other CPU in the range 0-6, starting with CPU 0.
VPCLASS CPU,num=4,aff=(0-6/2)

The virtual processors are assigned to CPUs 0, 2, 4, 6.

If you specify VPCLASS CPU,num=4,aff=(1-10/3), the virtual processors are assigned

to every third CPU in the range 1-10, starting with CPU 1. The virtual processors
are assigned to CPUs 1, 4, 7, 10.

When you specify more than one value or range, the values and ranges do not
have to be incremental or in any particular order. For example you can specify
aff=(8,12,7-9,0-6/2).

The database server assigns CPU virtual processors to CPUs in a circular pattern,
starting with the first processor number that you specify in the aff option. If you
specify a larger number of CPU virtual processors than physical CPUs, the
database server continues to assign CPU virtual processors starting with the first
CPU. For example, suppose you specify the following VPCLASS settings:
VPCLASS cpu,num=8,aff=(4-7)

The database server makes the following assignments:

Chapter 3. Effect of configuration on CPU utilization 3-7
 v CPU virtual processor number 0 to CPU 4
v CPU virtual processor number 1 to CPU 5
v CPU virtual processor number 2 to CPU 6
v CPU virtual processor number 3 to CPU 7
v CPU virtual processor number 4 to CPU 4
v CPU virtual processor number 5 to CPU 5
v CPU virtual processor number 6 to CPU 6
v CPU virtual processor number 7 to CPU 7
Related reference:
VPCLASS configuration parameter (Administrator's Reference)

Isolating AIO VPs from CPU VPs:

On a system that runs database server and client (or other) applications, you can
bind asynchronous I/O (AIO) VPs to the same CPUs to which you bind other
application processes through the operating system. In this way, you isolate client
applications and database I/O operations from the CPU VPs.

This isolation can be especially helpful when client processes are used for data
entry or other operations that require waiting for user input. Because AIO VP
activity usually comes in quick bursts followed by idle periods waiting for the
disk, you can often interweave client and I/O operations without their unduly
impacting each other.

Binding a CPU VP to a processor does not prevent other processes from running
on that processor. Application (or other) processes that you do not bind to a CPU
are free to run on any available processor. On a computer that is dedicated to the
database server, you can leave AIO VPs free to run on any processor, which
reduces delays on database operations that are waiting for I/O. Increasing the
priority of AIO VPs can further improve performance by ensuring that data is
processed quickly once it arrives from disk.

Avoiding a certain CPU:

The database server assigns CPU VPs to CPUs serially, starting with the CPU
number you specify in this parameter. You might want to avoid assigning CPU
VPs to a certain CPU that has a specialized hardware or operating-system function
(such as interrupt handling).

Setting the number of AIO VPs

Use the aio and num options of the VPCLASS configuration parameter to indicate
the number of AIO virtual processors that the database server starts initially.

If your operating system does not support kernel asynchronous I/O (KAIO), the
database server uses AIO virtual processors (VPs) to manage all database I/O
requests.

If the VPCLASS configuration parameter does not specify the number of AIO VPs
to start in the onconfig file, then the setting of the AUTO_AIOVPS configuration
parameter controls the number of AIO VPs:
v If AUTO_AIOVPS is set to 1 (on), the number of AIO VPs initially started is
equal to the number of AIO chunks, up to a maximum of 128.

3-8 IBM Informix Performance Guide

v If AUTO_AIOVPS is set to 0 (off), the number of AIO VPs started is equal to the
greater of 6 or twice the number of AIO chunks, up to a maximum of 128.

The recommended number of AIO virtual processors depends on how many disks
your configuration supports. If KAIO is not implemented on your platform, you
should allocate one AIO virtual processor for each disk that contains database
tables. You can add an additional AIO virtual processor for each chunk that the
database server accesses frequently.

You can use the AUTO_AIOVPS configuration parameter to enable the database
server to automatically increase the number of AIO virtual processors and
page-cleaner threads when the server detects that AIO virtual processors are not
keeping up with the I/O workload.

The machine notes file for your version of the database server indicates whether
the operating system supports KAIO. If KAIO is supported, the machine notes
describe how to enable KAIO on your specific operating system.

If your operating system supports KAIO, the CPU VPs make asynchronous I/O
requests to the operating system instead of AIO virtual processors. In this case,
configure only one AIO virtual processor, plus two additional AIO virtual
processor for every file chunk that does not use KAIO.

If you use cooked files and if you enable direct I/O using the DIRECT_IO
configuration parameter, you can reduce the number of AIO virtual processors. If
the database server implements KAIO and if direct I/O is enabled, the database
server will attempt to use KAIO, so you probably do not need more than one AIO
virtual processor. Temporary dbspaces do not use direct I/O. If you have
temporary dbspaces, you will probably need more than one AIO virtual processors.

Even when direct I/O is enabled with the DIRECT_IO configuration parameter, if
the file system does not support either direct I/O or KAIO, you still must allocate
two additional AIO virtual processors for every active dbspace chunk that is not
using KAIO.

The goal in allocating AIO virtual processors is to allocate enough of them so that
the lengths of the I/O request queues are kept short (that is, the queues have as
few I/O requests in them as possible). When the I/O request queues remain
consistently short, I/O requests are processed as fast as they occur. Use the onstat
-g ioq command to monitor the length of the I/O queues for the AIO virtual
processors.

Allocate enough AIO VPs to accommodate the peak number of I/O requests.
Generally, allocating a few extra AIO VPs is not detrimental. To start additional
AIO VPs while the database server is in online mode, use the onmode -p
command. You cannot drop AIO VPs in online mode.
Related reference:
AUTO_AIOVPS configuration parameter (Administrator's Reference)
VPCLASS configuration parameter (Administrator's Reference)

Chapter 3. Effect of configuration on CPU utilization 3-9

 Setting the MULTIPROCESSOR configuration parameter when
using multiple CPU VPs
If you are running multiple CPU VPs, set the MULTIPROCESSOR configuration
parameter to 1. When you set MULTIPROCESSOR to 1, the database server
performs locking in a manner that is appropriate for a multiprocessor. Otherwise,
set this parameter to 0.

The number of CPU VPs is used as a factor in determining the number of scan
threads for a query. Queries perform best when the number of scan threads is a
multiple (or factor) of the number of CPU VPs. Adding or removing a CPU VP can
improve performance for a large query because it produces an equal distribution of
scan threads among CPU VPs. For instance, if you have 6 CPU VPs and scan 10
table fragments, you might see a faster response time if you reduce the number of
CPU VPs to 5, which divides evenly into 10. You can use onstat -g ath to monitor
the number of scan threads per CPU VP or use onstat -g ses to focus on a
particular session.
Related reference:
MULTIPROCESSOR configuration parameter (Administrator's Reference)

Setting the SINGLE_CPU_VP configuration parameter when

using one CPU VP
If you are running only one CPU VP, set the SINGLE_CPU_VP configuration
parameter to 1. Otherwise, set this parameter to 0.

Important: If you set the SINGLE_CPU_VP parameter to 1, the value of the num
option of the VPCLASS configuration parameter must also be 1.

Note: The database server treats user-defined virtual-processor classes (that is, VPs
defined with VPCLASS) as if they were CPU VPs. Thus, if you set
SINGLE_CPU_VP to nonzero, you cannot create any user-defined classes.

When you set the SINGLE_CPU_VP parameter to 1, you cannot add CPU VPs
while the database server is in online mode.
Related reference:
SINGLE_CPU_VP configuration parameter (Administrator's Reference)
VPCLASS configuration parameter (Administrator's Reference)

Optimizing access methods

The OPTCOMPIND configuration parameter helps the query optimizer choose an
appropriate access method for your application. When the optimizer examines join
plans, OPTCOMPIND indicates the preferred method for performing the join
operation for an ordered pair of tables.

If OPTCOMPIND is equal to 0, the optimizer gives preference to an existing index

(nested-loop join) even when a table scan might be faster. If OPTCOMPIND is set
to 1 and the isolation level for a given query is set to Repeatable Read, the
optimizer uses nested-loop joins.

When OPTCOMPIND is equal to 2, the optimizer selects a join method based on

cost alone even though table scans can temporarily lock an entire table. For more
information about OPTCOMPIND and the different join methods, see “Effect of
OPTCOMPIND on the query plan” on page 10-22.
3-10 IBM Informix Performance Guide
 To set the value for OPTCOMPIND for specific applications or user sessions, set
the OPTCOMPIND environment variable for those sessions. Values for this
environment variable have the same range and semantics as for the configuration
parameter.
Related reference:
OPTCOMPIND configuration parameter (Administrator's Reference)

Setting the value of OPTCOMPIND within a session

You can set or change the value of OPTCOMPIND within a session for different
kinds of queries. To do this, use the SET ENVIRONMENT OPTCOMPIND
command, not the OPTCOMPIND configuration parameter.

For a DSS query, you should set the value of OPTCOMPIND to 2 or 1, and you
should be sure that the isolation level is not set to Repeatable Read. For an OLTP
query, you could set the value to 0 or 1 with the isolation level not set to
Repeatable Read.

The value that you enter using the SET ENVIRONMENT OPTCOMPIND
command takes precedence over the default setting specified in the ONCONFIG
file. The default OPTCOMPIND setting is restored when the current session
terminates. No other user sessions are affected by SET ENVIRONMENT
OPTCOMPIND statements that you execute.
Related concepts:
OPTCOMPIND Environment Option (SQL Syntax)

Limiting PDQ resources in queries

The MAX_PDQPRIORITY configuration parameter limits the percentage of parallel
database query (PDQ) resources that a query can use. Use MAX_PDQPRIORITY to
limit the impact of large CPU-intensive queries on transaction throughput.

To limit the impact of large CPU-intensive queries on transaction throughput

Set the value of the MAX_PDQPRIORITY configuration parameter to an integer

that represents a percentage of the following PDQ resources that a query can
request:
v Memory
v CPU VPs
v Disk I/O
v Scan threads

When a query requests a percentage of PDQ resources, the database server

allocates the MAX_PDQPRIORITY percentage of the amount requested, as the
following formula shows:
Resources allocated = PDQPRIORITY/100 * MAX_PDQPRIORITY/100

For example, if a client uses the SET PDQPRIORITY 80 statement to request 80

percent of PDQ resources, but MAX_PDQPRIORITY is set to 50, the database
server allocates only 40 percent of the resources (50 percent of the request) to the
client.

For decision support and online transaction processing (OLTP), setting

MAX_PDQPRIORITY allows the database server administrator to control the
impact that individual decision-support queries have on concurrent OLTP

Chapter 3. Effect of configuration on CPU utilization 3-11

 performance. Reduce the value of MAX_PDQPRIORITY when you want to allocate
more resources to OLTP processing. Increase the value of MAX_PDQPRIORITY
when you want to allocate more resources to decision-support processing.

For more information about how to control the use of PDQ resources, see “The
allocation of resources for parallel database queries” on page 12-7.
Related reference:
MAX_PDQPRIORITY configuration parameter (Administrator's Reference)

Limiting the performance impact of CPU-intensive queries

The DS_MAX_QUERIES configuration parameter specifies a maximum number of
decision-support queries that can run at any one time. Queries with a low PDQ
priority use proportionally fewer resources, so a larger number of those queries
can run simultaneously. You can use the DS_MAX_QUERIES configuration
parameter to limit the performance impact of CPU-intensive queries.

The DS_MAX_QUERIES configuration parameter controls only queries with a PDQ

priority that is nonzero.

The database server uses the value of DS_MAX_QUERIES with

DS_TOTAL_MEMORY to calculate quantum units of memory to allocate to a
query. For more information about how the database server allocates memory to
queries, see “The DS_TOTAL_MEMORY configuration parameter and memory
utilization” on page 4-12.
Related concepts:
“The DS_TOTAL_MEMORY configuration parameter and memory utilization” on
page 4-12
Related reference:
DS_MAX_QUERIES configuration parameter (Administrator's Reference)

Limiting the number of PDQ scan threads that can run

concurrently
The DS_MAX_SCANS configuration parameter limits the number of PDQ scan
threads that can run concurrently. This configuration parameter prevents the
database server from being flooded with scan threads from multiple
decision-support queries.

To calculate the number of scan threads allocated to a query, use the following
formula:
scan_threads = min (nfrags, (DS_MAX_SCANS * pdqpriority / 100
* MAX_PDQPRIORITY / 100) )
nfrags is the number of fragments in the table with the largest number of
fragments.
pdqpriority
is the PDQ priority value set by either the PDQPRIORITY environment
variable or the SQL statement SET PDQPRIORITY.

Reducing the number of scan threads can reduce the time that a large query waits
in the ready queue, particularly when many large queries are submitted
concurrently. However, if the number of scan threads is less than nfrags, the query
takes longer once it is underway.

3-12 IBM Informix Performance Guide

 For example, if a query needs to scan 20 fragments in a table, but the scan_threads
formula lets the query begin when only 10 scan threads are available, each scan
thread scans two fragments serially. Query execution takes approximately twice as
long as if 20 scan threads were used.
Related reference:
DS_MAX_SCANS configuration parameter (Administrator's Reference)

Configuring poll threads

The NETTYPE configuration parameter configures poll threads for each connection
type that your instance of the database server supports. If your database server
instance supports connections over more than one interface or protocol, you must
specify a separate NETTYPE configuration parameter for each connection type.

You typically include a separate NETTYPE parameter for each connection type that
is associated with a dbservername. You list dbservernames in the
DBSERVERNAME and DBSERVERALIASES configuration parameters. You
associate connection types with dbservernames in the sqlhosts information. For
details about connection types and the sqlhosts information, see connectivity
information in your.IBM Informix Administrator's Guide.
Related reference:
“UNIX semaphore parameters” on page 3-1
NETTYPE configuration parameter (Administrator's Reference)

Specifying the connection protocol

The first NETTYPE entry, which specifies the protocol for a given connection type,
applies to all dbservernames associated with that type. Subsequent NETTYPE
entries for that connection type are ignored.

NETTYPE entries are required for connection types that are used for outgoing
communication only even if those connection types are not listed in the sqlhosts
information.
UNIX Only
The following protocols apply to UNIX platforms:
v IPCSHM
v TLITCP
v IPCSTR
v SOCTCP
v TLIIMC
v SOCIMC
v SQLMUX
v SOCSSL
Windows Only
The following protocols apply to Windows platforms:
v SOCTCP
v IPCNMP
v SQLMUX
v SOCSSL
Related reference:

Chapter 3. Effect of configuration on CPU utilization 3-13

 NETTYPE configuration parameter (Administrator's Reference)

Specifying virtual-processor classes for poll threads

Each poll thread configured or added dynamically by a NETTYPE entry runs in a
separate VP. A poll thread can run in one of two VP classes: NET and CPU. For
best performance, use a NETTYPE entry to assign only one poll thread to the CPU
VP class. Then assign all additional poll threads to NET VPs.
Related reference:
NETTYPE configuration parameter (Administrator's Reference)

Specifying the number of connections and poll threads

The optimum number of connections per poll thread is approximately 300 for
uniprocessor computers and up to 350 for multiprocessor computers, although this
can vary depending on the platform and database server workload.

A poll thread can support 1024 or perhaps more connections. If the FASTPOLL
configuration parameter is enabled, you might be able to configure fewer poll
threads, but should test the performance to determine the optimal configuration
for your environment.

Each NETTYPE entry configures the number of poll threads for a specific
connection type, the number of connections per poll thread, and the
virtual-processor class in which those poll threads run, using the following
comma-separated fields. There can be no white space within or between these
fields.
NETTYPE connection_type,poll_threads,conn_per_thread,vp_class
connection_type
identifies the protocol-interface combination to which the poll threads are
assigned. You typically set this field to match the connection_type field of a
dbservername entry that is in the sqlhosts information.
poll_threads
is the number of poll threads assigned to the connection type.
conn_per_thread
is the number of connections per poll thread. Use the following formula to
calculate this number:
conn_per_thread = connections / poll_threads
connections
is the maximum number of connections that you expect the
indicated connection type to support. For shared-memory
connections (ipcshm), double the number of connections for best
performance.
This field is used only for shared memory connections on
Windows. Other connection methods on Windows ignore this
value.
For shared memory connection, the value of conn_per_thread is the
maximum number of connections per thread. For network
connections, the value of conn_per_thread can be exceeded.
vp_class
is the class of virtual processor that can run the poll threads. Specify CPU
if you have a single poll thread that runs on a CPU VP. For best
performance, specify NET if you require more than one poll thread. If you

3-14 IBM Informix Performance Guide

 are running Windows, specify NET in all cases. The default value for this
field depends on the following conditions:
v If the connection type is associated with the dbservername that is listed
in the DBSERVERNAME parameter, and no previous NETTYPE
parameter specifies CPU explicitly, the default VP class is CPU. If the
CPU class is already taken, the default is NET.
v If the connection type is associated with a dbservername that the
DBSERVERALIASES parameter specifies, the default VP class is NET.

If the value of conn_per_thread exceeds 350 and the number of poll threads for the
current connection type is less than the number of CPU VPs, you can improve
performance by specifying the NET CPU class, adding poll threads (do not exceed
the number of CPU VPs), and recalculating the value of conn_per_thread. The
default value for conn_per_thread is 50.

Important: Each ipcshm connection requires a semaphore. Some operating systems

require that you configure a maximum number of semaphores that can be
requested by all software packages that run on the computer. For best
performance, double the number of actual ipcshm connections when you allocate
semaphores for shared-memory communications. See “UNIX semaphore
parameters” on page 3-1.

If your computer is a uniprocessor and your database server instance is configured

for only one connection type, you can omit the NETTYPE parameter. The database
server uses the information provided in the sqlhosts information to establish
client/server connections.

If your computer is a uniprocessor and your database server instance is configured

for more than one connection type, include a separate NETTYPE entry for each
connection type. If the number of connections of any one type significantly exceeds
300, assign two or more poll threads, up to a maximum of the number of CPU
VPs, and specify the NET VP class, as the following example shows:
NETTYPE ipcshm,1,200,CPU
NETTYPE tlitcp,2,200,NET # supports 400 connections

For ipcshm, the number of poll threads correspond to the number of memory
segments. For example, if NETTYPE is set to 3,100 and you want one poll thread,
set the poll thread to 1,300.

If your computer is a multiprocessor, your database server instance is configured

for only one connection type, and the number of connections does not exceed 350,
you can use NETTYPE to specify a single poll thread on either the CPU or the
NET VP class. If the number of connections exceeds 350, set the VP class to NET,
increase the number of poll threads, and recalculate conn_per_thread.

Important: You should carefully distinguish between poll threads for network
connections and poll threads for shared memory connections, which should run
one per CPU virtual processor. TCP connections should only be in network virtual
processors, and you should only have the minimum needed to maintain
responsiveness. Shared memory connections should only be in CPU virtual
processors and should run in every CPU virtual processor.
Related concepts:
“Improve connection performance and scalability” on page 3-16
Related reference:

Chapter 3. Effect of configuration on CPU utilization 3-15

 NETTYPE configuration parameter (Administrator's Reference)
VPCLASS configuration parameter (Administrator's Reference)

Improve connection performance and scalability

You can improve connection performance and scalability by specifying information
in the NUMFDSERVERS and NS_CACHE configuration parameters and by using
multiple listen threads.

Informix SQL sessions can migrate across CPU VPs. You can improve the
performance and scalability of network connections on UNIX by using the
NUMFDSERVERS configuration parameter to specify a number for the poll threads
to use when distributing a TCP/IP connection across VPs. Specifying
NUMFDSERVERS information is useful if the database server has a high rate of
new connect and disconnect requests or if you find a high amount of contention
between network shared file (NSF) locks.

You should also review and, if necessary, change the information in the NETTYPE
configuration parameter, which defines the number of poll threads for a specific
connection type, the number of connections per poll thread, and the
virtual-processor class in which those poll threads run. You specify NETTYPE
configuration parameter information as follows:
NETTYPE connection_type,poll_threads,conn_per_thread,vp_class

On UNIX, if vp_class is NET, poll_threads can be a value that is greater than or

equal to 1. If vp_class is CPU, the number of poll_threads can be 1 through the
number of CPU VPs. On Windows, poll_threads can be value that is greater than
or equal to 1.

For example, suppose you specify 8 poll threads in the NETTYPE configuration
parameter, as follows:
NETTYPE soctcp,8,300,NET

You can also specify 8 in the NUMFDSERVERS configuration parameter to enable

the server to use all 8 poll thread to handle network connections migrating
between VPs.

You can use the NS_CACHE configuration parameter to define the maximum
retention time for an individual entry in the host name/IP address cache, the
service cache, the user cache, and the group cache. The server can get information
from the cache faster than it does when querying the operating system.

You can improve service for connection requests by using multiple listen threads.
When you specify DBSERVERNAME and DBSERVERALIASES configuration
parameter information for onimcsoc or onsoctcp protocols, you can specify the
number of multiple listen threads for the database server aliases in your sqlhosts
information. The default value of number is 1.

The DBSERVERNAME and DBSERVERALIASES configuration parameters define

database server names (dbservernames) that have corresponding entries in the
sqlhosts information. Each dbservername parameter in the sqlhosts information has
a nettype entry that specifies an interface/protocol combination. The database
server runs one or more poll threads for each unique nettype entry.

You can use the onstat -g ath command to display information about all threads.
Related concepts:
3-16 IBM Informix Performance Guide
 Name service maximum retention time set in the NS_CACHE configuration
parameter (Administrator's Guide)
“Specifying the number of connections and poll threads” on page 3-14
“Monitor threads with onstat -g ath output” on page 13-49
Related reference:
NETTYPE configuration parameter (Administrator's Reference)
NUMFDSERVERS configuration parameter (Administrator's Reference)
NS_CACHE configuration parameter (Administrator's Reference)
DBSERVERNAME configuration parameter (Administrator's Reference)
DBSERVERALIASES configuration parameter (Administrator's Reference)
Multiple listen threads (Administrator's Guide)

Enabling fast polling

You can use the FASTPOLL configuration parameter to enable or disable fast
polling of your network, if your operating-system platform supports fast polling.
Fast polling is beneficial if you have a large number of connections.

For example, if you have more than 300 concurrent connections with the database
server, you can enable the FASTPOLL configuration parameter for better
performance.
Related reference:
FASTPOLL configuration parameter (Administrator's Reference)

Network buffer pools

The sizes of buffers for TCP/IP connections affect memory and CPU utilization.
Sizing these buffers to accommodate a typical request can improve CPU utilization
by eliminating the need to break up requests into multiple messages.

However, you must use this capability with care; the database server dynamically
allocates buffers of the indicated sizes for active connections. Unless you carefully
size buffers, they can use large amounts of memory. For details on how to size
network buffers, see “Network buffer size” on page 3-19.

The database server dynamically allocates network buffers from the global memory
pool for request messages from clients. After the database server processes client
requests, it returns buffers to a common network buffer pool that is shared among
sessions that use SOCTCP, IPCSTR, or TLITCP network connections.

This common network buffer pool provides the following advantages:

v Prevents frequent allocations and deallocations from the global memory pool
v Uses fewer CPU resources to allocate and deallocate network buffers to and
from the common network buffer pool for each network transfer
v Reduces contention for allocation and deallocation of shared memory

The free network buffer pool can grow during peak activity periods. To prevent
large amounts of unused memory from remaining in these network buffer pools
when network activity is no longer high, the database server returns free buffers
when the number of free buffers reaches specific thresholds.

Chapter 3. Effect of configuration on CPU utilization 3-17

 The database server provides the following features to further reduce the allocation
and deallocation of and contention for the free network buffers:
v A private free network buffer pool for each session to prevent frequent
allocations and deallocations of network buffers from the common network
buffer pool or from the global memory pool in shared memory
v Capability to specify a larger than 4-kilobyte buffer size to receive network
packets or messages from clients

As the system administrator, you can control the free buffer thresholds and the size
of each buffer with the following methods:
v NETTYPE configuration parameter
v IFX_NETBUF_PVTPOOL_SIZE environment variable
v IFX_NETBUF_SIZE environment variable and b (client buffer size) option in the
sqlhosts information

Network buffers
The database server implements a threshold of free network buffers to prevent
frequent allocations and deallocations of shared memory for the network buffer
pool. This threshold enables the database server to correlate the number of free
network buffers with the number of connections that you specify in the NETTYPE
configuration parameter.

The database server dynamically allocates network buffers for request messages
from clients. After the database server processes client requests, it returns buffers
to the network free-buffer pool.

If the number of free buffers is greater than the threshold, the database server
returns the memory allocated to buffers over the threshold to the global pool.

The database server uses the following formula to calculate the threshold for the
free buffers in the network buffer pool:
free network buffers threshold =
100 + (0.7 * number_connections)

The value for number_connections is the total number of connections that you
specified in the third field of the NETTYPE entry for the different type of network
connections (SOCTCP, IPCSTR, or TLITCP). This formula does not use the
NETTYPE entry for shared memory (IPCSHM).

If you do not specify a value in the third field of the NETTYPE parameter, the
database server uses the default value of 50 connections for each NETTYPE entry
corresponding to the SOCTCP, TLITCP, and IPCSTR protocols.

Support for private network buffers

The database server provides support for private network buffers for each session
that uses SOCTCP, IPCSTR, or TLITCP network connections.

For situations in which many connections and sessions are constantly active, these
private network buffers have the following advantages:
v Less contention for the common network buffer pool
v Fewer CPU resources to allocate and deallocate network buffers to and from the
common network buffer pool for each network transfer

3-18 IBM Informix Performance Guide

 The IFX_NETBUF_PVTPOOL_SIZE environment variable specifies the size of the
private network buffer pool for each session. The default size is one buffer.

Use the onstat utility commands in the following table to monitor the network
buffer usage.

Command Output Field Description

onstat -g ntu q-pvt The current number and highest number of
buffers that are free in the private pool for this
session
onstat -g ntm q-exceeds The number of times that the free buffer
threshold was exceeded

The onstat -g ntu command displays the following format for the q-pvt output
field:
current number / highest number

If the number of free buffers (value in q-pvt field) is consistently 0, you can
perform one of the following actions:
v Increase the number of buffers with the environment variable
IFX_NETBUF_PVTPOOL_SIZE.
v Increase the size of each buffer with the environment variable
IFX_NETBUF_SIZE.

The q-exceeds field indicates the number of times that the threshold for the shared
network free-buffer pool was exceeded. When this threshold is exceeded, the
database server returns the unused network buffers (over this threshold) to the
global memory pool in shared memory. Optimally, this value should be 0 or a low
number so that the server is not allocating or deallocating network buffers from
the global memory pool.
Related reference:
IFX_NETBUF_PVTPOOL_SIZE environment variable (UNIX) (SQL Reference)
IFX_NETBUF_SIZE environment variable (SQL Reference)

Network buffer size

The IFX_NETBUF_SIZE environment variable specifies the size of each network
buffer in the common network buffer pool and the private network buffer pool.

The default buffer size is 4 kilobytes.

The IFX_NETBUF_SIZE environment variable allows the database server to

receive messages longer than 4 kilobytes in one system call. The larger buffer size
reduces the amount of overhead required to receive each packet.

Increase the value of IFX_NETBUF_SIZE if you know that clients send greater
than 4-kilobyte packets. Clients send large packets during any of the following
situations:
v Loading a table
v Inserting rows greater than 4 kilobytes
v Sending simple large objects

Chapter 3. Effect of configuration on CPU utilization 3-19

 The b option for sqlhosts allows the client to send and receive greater than 4
kilobytes. The value for the sqlhosts option should typically match the value for
IFX_NETBUF_SIZE.

You can use the following onstat command to see the network buffer size:
onstat -g afr global | grep net

The size field in the output shows the network buffer size in bytes.
Related reference:
Connectivity configuration (Administrator's Guide)
IFX_NETBUF_SIZE environment variable (SQL Reference)

Virtual processors and CPU utilization

While the database server is online, you can start and stop virtual processors (VPs)
that belong to certain classes.

You can use onmode -p or ON-Monitor to start additional VPs for the following
classes while the database server is online: CPU, AIO, PIO, LIO, SHM, TLI, and
SOC. You can drop VPs of the CPU class only while the database server is online.

You should carefully distinguish between poll threads for network connections and
poll threads for shared memory connections, which should run one per CPU
virtual processor. TCP connections should only be in network virtual processors,
and you should only have the minimum needed to maintain responsiveness.
Shared memory connections should only be in CPU virtual processors and should
run in every CPU virtual processor

Adding virtual processors

Whenever you add a network VP (SOC or TLI), you also add a poll thread. Every
poll thread runs in a separate VP, which can be either a CPU VP or a network VP
of the appropriate network type.

Adding more VPs can increase the load on CPU resources, so if the NETTYPE
value indicates that an available CPU VP can handle the poll thread, the database
server assigns the poll thread to that CPU VP. If all the CPU VPs have poll threads
assigned to them, the database server adds a second network VP to handle the poll
thread.

Automatic addition of CPU virtual processors

When the database server starts, the number of CPU virtual processors (VPs) is
automatically increased to half the number of CPU processors on the computer.
This ratio of CPU processors to CPU VPs is a recommended minimum to ensure
that the database server performs optimally in most situations.

During start up, the database server calculates a target number of CPU VPs that
represents an even number equal to or greater than half the number of CPU
processors and compares the target number with the currently allocated number of
CPU VPs. The database server adds the necessary number of CPU VPs to equal the
target number.

If fewer than eight CPU VPs are configured, the server can dynamically add CPU
VPs to a total (configured plus added) of eight.

3-20 IBM Informix Performance Guide

 The SINGLE_CPU_VP configuration parameter must be set to 0 for CPU VPs to be
automatically added. When CPU VPs are automatically added, the value of the
VPCLASS configuration parameter is not updated in the onconfig file; therefore,
the value of the VPCLASS configuration parameter for CPU VPs might not be the
same as the actual number of configured CPU VPs.

Use the auto_tune_cpu_vps task in the Scheduler to control the automatic addition
of CPU VPs. To prevent the automatic addition of CPU VPs, disable the
auto_tune_cpu_vps task in the ph_task table in the sysadmin database:
UPDATE ph_task
SET tk_enable = ’F’
WHERE tk_name = ’auto_tune_cpu_vps’;

The following table shows possible configurations and how many CPU VPs would
be added automatically in each situation.
Table 3-1. Example of how CPU VPs are automatically added
Automatically added
CPU processors Target CPU VPs Allocated CPU VPs CPU VPs
8 4 3 1
3 2 2 0
24 8 6 2

Related concepts:
“Setting the number of CPU VPs” on page 3-5

Monitoring virtual processors

Monitor the virtual processors to determine if the number of virtual processors
configured for the database server is optimal for the current level of activity.

To monitor virtual processors:

v Use command-line utilities, such as onstat-g ioq to view information. See “Using
some onstat-g commands to monitor virtual processors”
v Use the AUTO_AIOVPS configuration parameter to enable the database server
to automatically increase the number of AIO virtual processors and page-cleaner
threads when the server detects that AIO virtual processors are not keeping up
with the I/O workload.
v Query SMI tables. See “Using SMI tables to monitor virtual processors” on page
3-23.

Using some onstat-g commands to monitor virtual processors

You can use the onstat-g glo, onstat-g rea, and onstat-g ioq commands to monitor
virtual processors.

Monitor virtual processors with the onstat-g glo command:

Use the onstat-g glo command to display information about each virtual processor
that is running and to display cumulative statistics for each virtual-processor class.

The onstat -g glo command provides the following types of information:

v How many session threads that are running
v How often threads switch, yield, or need to spin many times to obtain a latch or
resource

Chapter 3. Effect of configuration on CPU utilization 3-21

 v The virtual processor classes that are running and how much time each class
spent running
v The number of virtual processors that are running for each virtual processor
class
v The virtual processors that are running and how much time each virtual
processor spent running
v The efficiency of each virtual processor

Use the onstat -g rea command to determine whether you need to increase the
number of virtual processors.
Related concepts:
“Monitor virtual processors with the onstat-g rea command”
Related reference:
onstat -g glo command: Print global multithreading information
(Administrator's Reference)

Monitor virtual processors with the onstat-g rea command:

Use the onstat-g rea command to monitor the number of threads in the ready
queue.

onstat-g rea displays this information:

v The status field in the output shows the value ready when the thread is in the
ready queue.
v The vp-class output field shows the virtual processor class on which the thread
executes.

If the number of threads in the ready queue is growing for a class of virtual
processors (for example, the CPU class), you might have to add more of those
virtual processors to your configuration.

Ready threads:
tid tcb rstcb prty status vp-class name

6 536a38 406464 4 ready 3cpu main_loop()

28 60cfe8 40a124 4 ready 1cpu onmode_mon
33 672a20 409dc4 2 ready 3cpu sqlexec

Figure 3-1. onstat-g rea output

Related concepts:
“Monitor virtual processors with the onstat-g glo command” on page 3-21
Related reference:
onstat -g rea command: Print ready threads (Administrator's Reference)

Monitor virtual processors with the onstat-g ioq command:

Use the onstat-g ioq command to determine whether you need to allocate
additional AIO virtual processors.

The onstat-g ioq command displays the length of the I/O queues under the
column len, as the figure below shows. You can also see the maximum queue

3-22 IBM Informix Performance Guide

length (since the database server started) in the maxlen column. If the length of the
I/O queue is growing, I/O requests are accumulating faster than the AIO virtual
processors can process them. If the length of the I/O queue continues to show that
I/O requests are accumulating, consider adding AIO virtual processors.

onstat -g ioq

AIO I/O queues:

q name/id len maxlen totalops dskread dskwrite dskcopy
adt 0 0 0 0 0 0 0
msc 0 0 1 12 0 0 0
aio 0 0 4 89 68 0 0
pio 0 0 1 1 0 1 0
lio 0 0 1 17 0 17 0
kio 0 0 0 0 0 0 0
gfd 3 0 3 254 242 12 0
gfd 4 0 17 614 261 353 0

onstat -d
Dbspaces
address number flags fchunk nchunks flags owner name
a1de1d8 1 1 1 1 N informix rootdbs
a1df550 2 1 2 1 N informix space1
2 active, 32,678 maximum
Chunks
address chk/dbs offset size free bpages flags pathname
a1de320 1 1 0 75000 66447 PO- /ix/root_chunk
a1df698 2 2 0 500 447 PO- /ix//chunk1
2 active, 32,678 maximum

Figure 3-2. onstat-g ioq and onstat -d output

Each chunk serviced by the AIO virtual processors has one line in the onstat-g ioq
output, identified by the value gfd in the q name column. You can correlate the
line in onstat -g ioq with the actual chunk because the chunks are in the same
order as in the onstat -d output. For example, in the onstat-g ioq output, there are
two gfd queues. The first gfd queue holds requests for root_chunk because it
corresponds to the first chunk shown in the onstat -d output. Likewise, the second
gfd queue holds requests for chunk1 because it corresponds to the second chunk
in the onstat -d output.

If the database server has a mixture of raw devices and cooked files, the gfd
queues correspond only to the cooked files in onstat -d output.
Related reference:
onstat -g ioq command: Print I/O queue information (Administrator's
Reference)

Using SMI tables to monitor virtual processors

You can get information from system-monitoring interface (SMI) tables to use to
monitor virtual processors.

You must connect to the sysmaster database to query the SMI tables. Query the
sysvpprof SMI table to obtain information about the virtual processors that are
currently running. This table contains the following columns.

Column Description
vpid ID number of the virtual processor

Chapter 3. Effect of configuration on CPU utilization 3-23

 Column Description
class Class of the virtual processor
usercpu Seconds of user CPU consumed
syscpu Seconds of system CPU consumed

CPU virtual processor private memory caches

Each CPU virtual processor (VP) can optionally have a private memory cache to
speed access time to memory blocks.

All memory allocations that are requested by threads in the database server are
fulfilled by memory pools. When a memory pool has insufficient memory blocks to
satisfy a memory allocation request, blocks are allocated from the global memory
pool. Because all threads use the same global memory pool, contention can occur.
Private memory caches allow each CPU virtual processor to retain its own set of
memory blocks that can be used to bypass the global memory pool. The initial
allocation for private memory caches is from the global memory pool. When the
blocks are freed, they are freed to the private memory cache on a specific CPU
virtual process. When a memory allocation is requested, the thread first checks
whether the allocation can be satisfied by blocks in the private memory cache.
Otherwise, the thread requests memory from the global memory pool.

To determine whether private memory caches might improve performance for your
database server, run the onstat -g spi command and look for the sh_lock mutex. If
the values from the onstat -g spi command output show contention for the
sh_lock mutex, try creating private memory caches.

You set the VP_MEMORY_CACHE_KB configuration parameter to enable private

memory caches by specifying the initial size of all private memory caches. The size
of each private memory cache increases or decreases automatically based on the
workload of the associated CPU VP. By default, the total size of private memory
caches can exceed the value of the VP_MEMORY_CACHE_KB configuration
parameter, but cannot exceed the value of the SHMTOTAL configuration
parameter. Alternatively, you can limit the total size of private memory caches to
the size value of the VP_MEMORY_CACHE_KB configuration parameter.

You can view statistics about CPU VP private memory caches by running the
onstat -g vpcache command. You can view statistics about memory pools by
running the onstat -g mem command.

Attention: If you have multiple CPU VPs, private memory caches can increase
the amount of memory that the database server uses.
Related reference:
VP_MEMORY_CACHE_KB configuration parameter (Administrator's
Reference)
onstat -g vpcache command: Print CPU VP private memory cache statistics
(Administrator's Reference)
onstat -g mem command: Print pool memory statistics (Administrator's
Reference)
onstat -g spi command: Print spin locks with long spins (Administrator's
Reference)

3-24 IBM Informix Performance Guide

Connections and CPU utilization
Some applications have a large number of client/server connections. Opening and
closing connections can consume a large amount of system CPU time.

The following topics describe ways that you might be able to reduce the system
CPU time required to open and close connections.

Multiplexed connections and CPU utilization

Many traditional nonthreaded SQL client applications use multiple database
connections to perform work for a single user. Each database connection
establishes a separate network connection to the database server. The multiplexed
connection facility provides the ability for one network connection in the database
server to handle multiple database connections from a client application.

Multiplexed connections enable the database server to create multiple database

connections without consuming the additional computer resources that are
required for additional network connections.

When a nonthreaded client uses a multiplexed connection, the database server still
creates the same number of user sessions and user threads as with a
nonmultiplexed connection. However, the number of network connections
decreases when you use multiplexed connections. Instead, the database server uses
a multiplex listener thread to allow the multiple database connections to share the
same network connection.

To improve response time for nonthreaded clients, you can use multiplexed
connections to execute SQL queries. The amount of performance improvement
depends on the following factors:
v The decrease in total number of network connections and the resulting decrease
in system CPU time
The usual cause for a large amount of system CPU time is the processing of
system calls for the network connection. Therefore, the maximum decrease in
system CPU time is proportional to the decrease in the total number of network
connections.
v The ratio of this decrease in system CPU time to the user CPU time
If the queries are simple and use little user CPU time, you might experience a
sizable reduction in response time when you use a multiplexed connection. But
if the queries are complex and use a large amount of user CPU time, you might
not experience a performance improvement.
To get an idea of the amounts of system CPU time and user CPU times per
virtual processor, use the onstat -g glo option.

To use multiplexed connections for a nonthreaded client application, you must take
the following steps before you bring up the database server:
1. Define an alias using the DBSERVERALIASES configuration parameter. For
example, specify:
DBSERVERALIASES ids_mux
2. Add an SQLHOSTS entry for the alias using sqlmux as the nettype entry, which
is the second column in the SQLHOSTS file. For example, specify:
ids_mux onsqlmux ......
The other fields in this entry, the hostname and servicename, must be present,
but they are ignored.

Chapter 3. Effect of configuration on CPU utilization 3-25

 3. Enable multiplexing for the selected connection types by specifying m=1 in the
sqlhosts file or registry that the client uses for the database server connection.
4. On Windows platforms, you must also set the IFX_SESSION_MUX
environment variable.

Warning: On Windows, a multithreaded application must not use the multiplexed

connection feature. If a multithreaded application enables the multiplexing option
in the sqlhosts registry entry and also defines the IFX_SESSION_MUX
environment variable, it can produce disastrous results, including crashing and
data corruption.
Related concepts:
Multiplexed connections (ESQL/C Guide)
Related tasks:
Supporting multiplexed connections (Administrator's Guide)

MaxConnect for multiple connections UNIX

IBM Informix MaxConnect is a networking product for Informix database server
environments on UNIX. You can use Informix MaxConnect to manage large
numbers (from several hundred to tens of thousands) of client/server connections.
Informix MaxConnect is best for OLTP data transfers, but is not recommended for
large multimedia data transfers.

Informix MaxConnect provides the following performance advantages for medium

to large OLTP configurations:
v Reduces CPU requirements on the database server by reducing the number of
physical connections.
Informix MaxConnect multiplexes connections so that the ratio of client
connections to database connections can be 100:1 or higher.
v Improves end-user response time by increasing system scalability to many
thousands of connections
v Reduces operating-system overhead by aggregating multiple small packets into
one transfer operation

To obtain maximum performance benefit, install Informix MaxConnect on either a

dedicated computer to which Informix clients connect or on the client application
server. Either of these configurations offloads the CPU requirements of handling a
large number of connections from the database server computer.

To monitor Informix MaxConnect, use the onstat -g imc command on the database
server computer and use the imcadmin command on the computer where Informix
MaxConnect is located.

For more information about installing, configuring, monitoring, and tuning

Informix MaxConnect, see the IBM Informix MaxConnect User's Guide.

Important: Informix MaxConnect and the IBM Informix MaxConnect User's Guide
ship separately from IBM Informix.

3-26 IBM Informix Performance Guide

Chapter 4. Effect of configuration on memory utilization
The combination of operating-system and Informix configuration parameters can
affect memory utilization.

You can change the settings of the Informix configuration parameters that directly
affect memory utilization, and you can adjust the settings for different types of
workloads.

Consider the amount of physical memory that is available on your host when you
allocate shared memory for the database server by setting operating-system
configuration parameters. In general, if you increase space for database server
shared memory, you can enhance the performance of your database server. You
must balance the amount of shared memory that is dedicated to the database
server against the memory requirements for VPs and other processes.
Related concepts:
“The Memory Grant Manager” on page 12-6

Shared memory
You must configure adequate shared-memory resources for the database server in
your operating system. Insufficient shared memory can adversely affect
performance.

The database server threads and processes require shared memory to share data by
sharing access to segments of memory.

The shared memory that Informix uses can be divided into the following parts,
each of which has one or more shared memory segments:
v Resident portion
v Virtual portion
v Message portion

The resident and message portions are static; you must allocate sufficient memory
for them before you bring the database server into online mode. (Typically, you
must reboot the operating system to reconfigure shared memory.) The virtual
portion of shared memory for the database server grows dynamically, but you
must still include an adequate initial amount for this portion in your allocation of
operating-system shared memory.

The amount of space that is required is the total that all portions of database server
shared memory need. You specify the total amount of shared memory with the
SHMTOTAL configuration parameter.

The LOCKS configuration parameter specifies the initial size of the lock table. If
the number of locks that sessions allocate exceeds the value of LOCKS, the
database server dynamically increases the size of the lock table. If you expect the
lock table to grow dynamically, set SHMTOTAL to 0. When SHMTOTAL is 0, there
is no limit on total memory (including shared memory) allocation.
Related reference:
LOCKS configuration parameter (Administrator's Reference)

© Copyright IBM Corp. 1996, 2014 4-1

 SHMTOTAL configuration parameter (Administrator's Reference)

Resident portion of shared memory

The resident portion of shared memory includes areas of shared memory that
record the state of the database server, including buffers, locks, log files, and the
locations of dbspaces, chunks, and tblspaces.

The settings that you use for the LOCKS, LOGBUFF, and PHYSBUFF configuration
parameters help determine the size of the resident portion.

The BUFFERPOOL configuration parameter determines the number of buffers

allocated to the resident segment when the database server is started. Subsequent
buffer pools that are added while the database server is running are moved into
virtual memory until the database server is restarted.

In addition to these configuration parameters, which affect the size of the resident
portion, the RESIDENT configuration parameter can affect memory use. When a
computer supports forced residency and the RESIDENT configuration parameter is
set to a value that locks the resident or resident and virtual portions, the resident
portion is never paged out.

The machine notes file for your database server indicates whether your operating
system supports forced residency.

On AIX, Solaris, and Linux systems that support large pages, the
IFX_LARGE_PAGES environment variable can enable the use of large pages for
non-message shared memory segments that are locked in physical memory. If large
pages are configured by operating system commands and the RESIDENT
configuration parameter specifies that some or all of the resident and virtual
portions of shared memory are locked in physical memory, Informix uses large
pages for the corresponding shared memory segments, provided sufficient large
pages are available. The use of large pages can offer significant performance
benefits in large memory configurations.
Related reference:
“Configuration parameters that affect memory utilization” on page 4-8
IFX_LARGE_PAGES environment variable (SQL Reference)

Virtual portion of shared memory

Informix uses the virtual portion of shared memory to allocate memory to each
database server subsystem, as needed.

The virtual portion of shared memory for the database server includes the
following components:
v Large buffers, which are used for large read and write I/O operations
v Sort-space pools
v Active thread-control blocks, stacks, and heaps
v User-session data
v Caches for SQL statements, data-dictionary information, and user-defined
routines
v A global pool for network-interface message buffers and other information

4-2 IBM Informix Performance Guide

 The SHMVIRTSIZE configuration parameter in the onconfig file provides the
initial size of the virtual portion. As the need for additional space in the virtual
portion arises, the database server adds shared memory in increments that the
SHMADD configuration parameter specifies. The EXTSHMADD configuration
parameter configures the size of the virtual-extension shared memory segments
that are added for user-defined routines and DataBlade routines. The limit on the
total shared memory allocated to the database server is specified by the
SHMTOTAL parameter.

The size of the virtual portion depends primarily on the types of applications and
queries that you are running. Depending on your application, an initial estimate
for the virtual portion might be as low as 100 KB per user or as high as 500 KB per
user, plus an additional 4 megabytes if you intend to use data distributions.

When a computer supports forced residency and the RESIDENT configuration

parameter is set to a value that locks virtual segments, the virtual segments that
are locked are never paged out.

On AIX, Solaris, and Linux systems that support large pages, the
IFX_LARGE_PAGES environment variable can enable the use of large pages for
non-message shared memory segments that are locked in physical memory. If large
pages are configured by operating system commands and the RESIDENT
configuration parameter specifies that some or all of the resident and virtual
portions of shared memory are locked in physical memory, Informix uses large
pages for the corresponding shared memory segments, provided sufficient large
pages are available. The use of large pages can offer significant performance
benefits in large memory configurations.
Related tasks:
“Creating data distributions” on page 13-13
Related reference:
“Configuration parameters that affect memory utilization” on page 4-8
IFX_LARGE_PAGES environment variable (SQL Reference)
EXTSHMADD configuration parameter (Administrator's Reference)
SHMADD configuration parameter (Administrator's Reference)
SHMTOTAL configuration parameter (Administrator's Reference)
SHMVIRTSIZE configuration parameter (Administrator's Reference)

Message portion of shared memory

The message portion of shared memory contains the message buffers that the
shared-memory communication interface uses. The amount of space required for
these buffers depends on the number of user connections that you allow using a
given networking interface.

If a particular interface is not used, you do not need to include space for it when
you allocate shared memory in the operating system.

Estimating the size of the resident portion of shared memory

You can use formulas to estimate the size of the resident portion (in KB) of shared
memory when you allocate operating-system shared memory.

Chapter 4. Effect of configuration on memory utilization 4-3

 The result of your calculations is an estimate that normally, slightly exceeds the
actual memory that is used for the resident portion of shared memory.

The following estimate was calculated to determine the resident portion of shared
memory on a 64-bit server. The sizes that are shown are subject to change, and the
calculation is approximate.

To estimate the size of the resident portion of shared memory

1. Estimate the size of the data buffer, using the following formula:
buffer_value = (BUFFERS * pagesize) + (BUFFERS * 254) + 250000
pagesize
is the shared-memory page size, as onstat -b shows it on the last line in
the buffer size field.
If you have multiple buffer pools, add the buffer sizes for each buffer pool
together.
2. Calculate the values in the following formulas:
locks_value = LOCKS * 44 44->128
logbuff_value = LOGBUFF * 1024 * 3
physbuff_value = PHYSBUFF * 1024 * 2
3. Calculate the estimated size of the resident portion in KB, using the following
formula:
rsegsize = 1.02 * (buffer_value + locks_value + logbuff_value
+ physbuff_value + 51,200 51,200->1,200,000) / 1024

Estimating the size of the virtual portion of shared memory

You can use a formula to estimate the initial size of the virtual portion of shared
memory. You specify the initial size in the SHMVIRTSIZE configuration parameter.

The formula for estimating an initial size of the virtual portion of shared memory
is as follows:
shmvirtsize = fixed overhead + shared structures +
(mncs * private structures) +
other buffers

To estimate an SHMVIRTSIZE value with the preceding formula:

1. Estimate the value for the fixed overhead portion of the formula as follows:
fixed overhead = global pool +
thread pool after booting
a. Run the onstat -g mem command to obtain the pool sizes allocated to
sessions.
b. Subtract the value in the freesize field from the value in the totalsize to
obtain the number of bytes allocated per session.
c. Estimate a value for the thread pool after booting variable. This variable is
partially dependent on the number of virtual processors.
2. Estimate the value of shared structures with the following formula:
shared structures = AIO vectors + sort memory +
dbspace backup buffers +
data-dictionary cache size +
size of user-defined routine cache +
histogram pool +
STMT_CACHE_SIZE (SQL statement cache) +
other pools (See onstat display.)
3. Estimate the next part of the formula, as follows:

4-4 IBM Informix Performance Guide

 a. Estimate the value of mncs (which is the maximum number of concurrent
sessions) with the following formula:
mncs = number of poll threads *
number connections per poll thread
The value for number of poll threads is the value that you specify in the
second field of the NETTYPE configuration parameter.
The value for number of connections per poll thread is the value that you
specify in the third field of the NETTYPE configuration parameter.
You can also obtain an estimate of the maximum number of concurrent
sessions when you run the onstat -u command during peak processing. The
last line of the onstat -u output contains the maximum number of
concurrent user threads.
b. Estimate the value of private structures, as follows:
private structures = stack + heap +
session control-block structures
stack Generally 32 KB but dependent on recursion in user-defined
routines. You can obtain the stack size for each thread with the
onstat -g sts option.
heap About 15 KB. You can obtain the heap size for an SQL statement
when you use the onstat -g stm option.
session control-block structures
The amount of memory used per session. The onstat -g ses option
displays the amount of memory, in bytes, in the total memory
column listed for each session id.
c. Multiply the results of steps 3a and 3b to obtain the following part of the
formula:
mncs * private structures
4. Estimate the value of other buffers to account for private buffers allocated for
features such as lightweight I/O operations for smart large objects (about 180
KB per user).
5. Add the results of steps 1 through 4 to obtain an estimate for the
SHMVIRTSIZE configuration parameter.

Tip: When the database server is running with a stable workload, you can use
onstat -g seg to obtain a precise value for the actual size of the virtual portion of
shared memory. You can then use the value for shared memory that this command
reports to reconfigure SHMVIRTSIZE.

To specify the size of segments that are added later to the virtual shared memory,
set the SHMADD configuration parameter. Use the EXTSHMADD configuration
parameter to specify the size of virtual-extension segments that are added for
user-defined routines and DataBlade routines.

The following table contains a list of additional topics for estimating the size of
shared structures in memory.
Table 4-1. Information for shared-memory structures
Shared-Memory Structure More Information
Sort memory “Estimating memory needed for sorting” on
page 7-19
Data-dictionary cache “Data-dictionary configuration” on page 4-22

Chapter 4. Effect of configuration on memory utilization 4-5

 Table 4-1. Information for shared-memory structures (continued)
Shared-Memory Structure More Information
Data-distribution cache (histogram pool) “Data-distribution configuration” on page 4-23
User-defined routine (UDR) cache “SPL routine executable format stored in UDR
cache” on page 10-33
SQL statement cache “Enabling the SQL statement cache” on page
13-42 “Monitor and tune the SQL statement
cache” on page 4-25
Other pools To see how much memory is allocated to the
different pools, use the onstat -g mem command.

Related concepts:
“Session memory” on page 4-38
Related reference:
SHMVIRTSIZE configuration parameter (Administrator's Reference)
NETTYPE configuration parameter (Administrator's Reference)
onstat -g mem command: Print pool memory statistics (Administrator's
Reference)

Estimating the size of the message portion of shared memory

You can estimate the size of the message portion of shared memory in kilobytes.

Estimate the size of the message portion of shared memory, using the following
formula:
msegsize = (10,531 * ipcshm_conn + 50,000)/1024
ipcshm_conn
is the number of connections that can be made using the shared-memory
interface, as determined by the NETTYPE parameter for the ipcshm
protocol.
Related reference:
NETTYPE configuration parameter (Administrator's Reference)

Configuring UNIX shared memory

On UNIX, you can configure shared-memory segments for the database server.

On UNIX, perform the following steps to configure the shared-memory segments

that your database server configuration needs. For information about how to set
parameters related to shared memory, see the configuration instructions for your
operating system.

To configure shared-memory segments for the database server:

1. If your operating system does not have a size limit for shared-memory
segments, take the following actions:
a. Set the operating-system configuration parameter for maximum segment
size, typically SHMMAX or SHMSIZE, to the total size that your database
server configuration requires. This size includes the amount of memory that
is required to start your database server instance and the amount of shared
memory that you allocate for dynamic growth of the virtual portion.

4-6 IBM Informix Performance Guide

 b. Set the operating-system configuration parameter for the maximum number
of segments, typically SHMMNI, to at least 1 per instance of the database
server.
2. If your operating system has a segment-size limit, take the following actions:
a. Set the operating-system configuration parameter for the maximum segment
size, typically SHMMAX or SHMSIZE, to the largest value that your system
allows.
b. Use the following formula to calculate the number of segments for your
instance of the database server. If there is a remainder, round up to the
nearest integer.
SHMMNI = total_shmem_size / SHMMAX
total_shmem_size
is the total amount of shared memory that you allocate for the
database server use.
3. Set the operating-system configuration parameter for the maximum number of
segments, typically SHMMNI, to a value that yields the total amount of shared
memory for the database server when multiplied by SHMMAX or SHMSIZE. If
your computer is dedicated to a single instance of the database server, that total
can be up to 90 percent of the size of virtual memory (physical memory plus
swap space).
4. If your operating system uses the SHMSEG configuration parameter to indicate
the maximum number of shared-memory segments that a process can attach,
set this parameter to a value that is equal to or greater than the largest number
of segments that you allocate for any instance of the database server.

For additional tips on configuring shared memory in the operating system, see the
machine notes file for UNIX or the release notes file for Windows.
Related concepts:
“The SHMADD and EXTSHMADD configuration parameters and memory
utilization” on page 4-17

Freeing shared memory with onmode -F

You can run the onmode -F command to free shared-memory segments that are
unavailable or no longer needed for a process.

The database server does not automatically free the shared-memory segments that
it adds during its operations. After memory has been allocated to the database
server virtual portion, the memory remains unavailable for use by other processes
running on the host computer. When the database server runs a large
decision-support query, it might acquire a large amount of shared memory. After
the query completes, the database server no longer requires that shared memory.
However, the shared memory that the database server allocated to service the
query remains assigned to the virtual portion even though it is no longer needed.

The onmode -F command locates and returns unused 8-kilobyte blocks of shared
memory that the database server still holds. Although this command runs only
briefly (one or two seconds), onmode -F dramatically inhibits user activity while it
runs. Systems with multiple CPUs and CPU VPs typically experience less
degradation while this utility runs.

You should run onmode -F during slack periods with an operating-system

scheduling facility (such as cron on UNIX). In addition, consider running this

Chapter 4. Effect of configuration on memory utilization 4-7

 utility after you perform any task that substantially increases the size of database
server shared memory, such as large decision-support queries, index builds, sorts,
or backup operations.
Related reference:
onmode -F: Free unused memory segments (Administrator's Reference)

Configuration parameters that affect memory utilization

A large number of configuration parameters in the ONCONFIG file affect memory
utilization and performance.

The following configuration parameters significantly affect memory utilization:

v BUFFERPOOL
v DS_NONPDQ_QUERY_MEM
v DS_TOTAL_MEMORY
v EXTSHMADD
v LOCKS
v LOGBUFF
v LOW_MEMORY_MGR
v LOW_MEMORY_RESERVE
v PHYSBUFF
v RESIDENT
v SHMADD
v SHMBASE
v SHMTOTAL
v SHMVIRTSIZE
v SHMVIRT_ALLOCSEG
v STACKSIZE
v Memory cache parameters (see “Configure and monitor memory caches” on
page 4-21)
v Network buffer size (see “Network buffer pools” on page 3-17)

The SHMBASE parameter indicates the starting address for database server shared
memory. When set according to the instructions in the machine notes file or release
notes file, this parameter has no appreciable effect on performance. For the path
name of each file, see the Introduction to this guide.

The DS_NONPDQ_QUERY_MEM parameter increases the amount of memory that

is available for non-PDQ queries. You can only use this parameter if PDQ priority
is set to zero. For more information, see “Configuring memory for queries with
hash joins, aggregates, and other memory-intensive elements” on page 13-34.

The following sections describe the performance effects and considerations

associated with some of the configuration parameters that are listed at the
beginning of this section.
Related concepts:
“Resident portion of shared memory” on page 4-2
“Virtual portion of shared memory” on page 4-2
Related reference:

4-8 IBM Informix Performance Guide

 LOW_MEMORY_MGR configuration parameter (Administrator's Reference)
LOW_MEMORY_RESERVE configuration parameter (Administrator's
Reference)

Setting the size of the buffer pool, logical-log buffer, and

physical-log buffer
The values that you specify for the BUFFERPOOL, DS_TOTAL_MEMORY,
LOGBUFF, and PHYSBUFF configuration parameters depend on the type of
applications that you are using (OLTP or DSS) and the page size.

Table 4-2 lists suggested settings for these parameters or guidelines for setting the
parameters.

For information about estimating the size of the resident portion of shared
memory, see “Estimating the size of the resident portion of shared memory” on
page 4-3. This calculation includes figuring the size of the buffer pool, logical-log
buffer, physical-log buffer, and lock table.
Table 4-2. Guidelines for OLTP and DSS applications
Configuration Parameter OLTP Applications DSS Applications
BUFFERPOOL The percentage of physical Set to a small buffer value and
memory that you need for increase the
buffer space depends on the DS_TOTAL_MEMORY value
amount of memory that is for light scans, queries, and
available on your system and sorts.
the amount of memory that is
used for other applications. For operations such as index
builds that read data through
the buffer pool, configure a
larger number of buffers.
DS_TOTAL_MEMORY Set to a value from 20 to 50 Set to a value from 50 to 90
percent of the value of percent of SHMTOTAL.
SHMTOTAL, in kilobytes.
LOGBUFF The default value for the Because database or table
logical log buffer size is 64 logging is usually turned off
KB. for DSS applications, you can
set LOGBUFF to 32 KB.
If you decide to use a smaller
value, the database server
generates a message a
message that indicates that
optimal performance might
not be obtained. Using a
logical log buffer smaller than
64 KB, impacts performance,
not transaction integrity.

If the database or application

is defined to use buffered
logging, increasing the
LOGBUFF size beyond 64 KB
improves performance.

Chapter 4. Effect of configuration on memory utilization 4-9

 Table 4-2. Guidelines for OLTP and DSS applications (continued)
Configuration Parameter OLTP Applications DSS Applications
PHYSBUFF The default value for the Because most DSS applications
physical log buffer size is 128 do not physically log, you can
KB. set PHYSBUFF to 32 KB.

If the
RTO_SERVER_RESTART
configuration parameter is
enabled, use the 512 kilobyte
default value for PHYSBUFF.

If you decide to use a value

that is smaller than the
default value, the database
server generates a message
that indicates that optimal
performance might not be
obtained. Using a physical log
buffer that is smaller than the
default size impacts
performance, not transaction
integrity.

Related reference:
BUFFERPOOL configuration parameter (Administrator's Reference)
DS_TOTAL_MEMORY configuration parameter (Administrator's Reference)
LOGBUFF configuration parameter (Administrator's Reference)
PHYSBUFF configuration parameter (Administrator's Reference)
RTO_SERVER_RESTART configuration parameter (Administrator's Reference)

The BUFFERPOOL configuration parameter and memory

utilization
The BUFFERPOOL configuration parameter specifies the properties of buffer pools.
The information that you define in the BUFFERPOOL configuration parameter
fields affects memory use.

You can have multiple buffer pools if you have dbspaces that use different page
sizes. The onconfig configuration file contains a BUFFERPOOL line for each page
size. For example, on a computer with a 2 KB page size, the onconfig file can
contain up to nine lines, including the default specification. When you create a
dbspace with a different page size, a buffer pool for that page size is created
automatically, if it does not exist. A BUFFERPOOL entry for the page size is added
to the onconfig file. The values of the BUFFERPOOL configuration parameter
fields are the same as the default specification.

The BUFFERPOOL configuration parameter controls the number of data buffers

available to the database server. These buffers are in the resident portion of shared
memory (buffer pool) and are used to cache database data pages in memory.

Increasing the number of buffers increases the likelihood that a needed data page
might already be in memory as the result of a previous request. However,
allocating too many buffers can affect the memory-management system and lead to

4-10 IBM Informix Performance Guide

excess operating system paging activity. To take advantage of the large memory
available on 64-bit addressing machines, you can increase the size of the buffer
pool.

The size of the buffer pool has a significant effect on database I/O and transaction
throughput.

The size of the buffer pool is equal to the number of buffers multiplied by the page
size. The percentage of physical memory that you need for buffer space depends
on the amount of memory that you have available on your system and the amount
that is used for other applications. For systems with a large amount of available
physical memory (4 GB or more), buffer space might be as much as 90 percent of
physical memory. For systems with smaller amounts of available physical memory,
buffer space might range from 20 - 25 percent of physical memory.

For example, suppose that your system has a page size of 2 KB and 100 MB of
physical memory. You can set the value in the buffers field to 10,000 - 12,500,
which allocates 20 - 25 MB of memory.

Calculate all other shared-memory parameters after you specify the size of the
buffer pool.

Note: If you use non-default page sizes, you might need to increase the size of
your physical log. If you frequently update non-default pages, you might need a
150 - 200 percent increase of the physical log size. Some experimentation might be
needed to tune the physical log. You can adjust the size of the physical log as
necessary according to how frequently the filling of the physical log triggers
checkpoints.

You can use onstat -g buf to monitor buffer pool statistics, including the
read-cache rate of the buffer pool. This rate represents the percentage of database
pages that are already present in a shared-memory buffer when a query requests a
page. (If a page is not already present, the database server must copy it into
memory from disk.) If the database server finds the page in the buffer pool, it
spends less time on disk I/O. Therefore, you want a high read-cache rate for good
performance. For OLTP applications where many users read small sets of data, the
goal is to achieve a read cache rate of 95 percent or better.

If the read-cache rate is low, you can repeatedly increase buffers and restart the
database server. As you increase the BUFFERPOOL value of buffers, you reach a
point at which increasing the value no longer produces significant gains in the
read-cache rate, or you reach the upper limit of your operating-system
shared-memory allocation.

Use the memory-management monitor utility in your operating system (such as

vmstat or sar on UNIX) to note the level of page scans and paging-out activity. If
these levels rise suddenly or rise to unacceptable levels during peak database
activity, reduce the size of the buffer pool.

Smart large objects and buffers

Depending upon your situation, you can take one of the following actions to
achieve better performance for applications that use smart large objects:
v If your applications frequently access smart large objects that are 2 KB or 4 KB
in size, use the buffer pool to keep them in memory longer. Use the following
formula to increase the value of the buffers field:

Chapter 4. Effect of configuration on memory utilization 4-11

 Additional_buffers = numcur_open_lo *
(lo_userdata / pagesize)
In this formula:
– numcur_open_lo is the number of concurrently opened smart large objects that
you can obtain from the onstat -g smb fdd command.
– lo_userdata is the number of bytes of smart-large-object data that you want to
buffer.
– pagesize is the default page size in bytes for the computer.
As a rule, try to have enough buffers to hold two smart-large-object pages for
each concurrently open smart large object. The additional page is available for
read-ahead purposes.
v Use lightweight I/O buffers in the virtual portion of shared memory.
Use lightweight I/O buffers only when you read or write smart large objects in
operations greater than 8000 bytes and seldom access them. That is, if the read
or write function calls read large amounts of data in a single-function
invocation, use lightweight I/O buffers.
When you use lightweight I/O buffers, you can prevent the flood of smart large
objects into the buffer pool and leave more buffers available for other data pages
that multiple users frequently access.
Related concepts:
“Lightweight I/O for smart large objects” on page 5-23
“BUFFERPOOL and its effect on page cleaning” on page 5-40
Related reference:
BUFFERPOOL configuration parameter (Administrator's Reference)
Monitor buffers (Administrator's Guide)

The DS_TOTAL_MEMORY configuration parameter and memory

utilization
The DS_TOTAL_MEMORY configuration parameter places a ceiling on the amount
of shared memory that a query can obtain. You can use this parameter to limit the
performance impact of large, memory-intensive queries. The higher you set this
parameter, the more memory a large query can use, and the less memory is
available for processing other queries and transactions.

For OLTP applications, set DS_TOTAL_MEMORY to 20 - 50 percent of the value of

SHMTOTAL, in KB. For applications that involve large decision-support (DSS)
queries, increase the value of DS_TOTAL_MEMORY to 50 - 80 percent of
SHMTOTAL. If you use your database server instance exclusively for DSS queries,
set this parameter to 90 percent of SHMTOTAL.

A quantum unit is the minimum increment of memory that is allocated to a query.

The Memory Grant Manager (MGM) allocates memory to queries in quantum
units. The database server uses the value of DS_MAX_QUERIES with the value of
DS_TOTAL_MEMORY to calculate a quantum of memory, according to the
following formula:
quantum = DS_TOTAL_MEMORY / DS_MAX_QUERIES

The database server can adjust the size of the quantum dynamically when it grants
memory. To allow for more simultaneous queries with smaller quanta each,
increase the value of the DS_MAX_QUERIES configuration parameter.
Related concepts:

4-12 IBM Informix Performance Guide

“The Memory Grant Manager” on page 12-6
“Limiting the performance impact of CPU-intensive queries” on page 3-12
Related reference:
DS_TOTAL_MEMORY configuration parameter (Administrator's Reference)

Algorithm for determining DS_TOTAL_MEMORY:

The database server derives a value for DS_TOTAL_MEMORY if you do not set
the DS_TOTAL_MEMORY configuration parameter or if you set this configuration
parameter to an inappropriate value.

Whenever the database server changes the value that you assigned to
DS_TOTAL_MEMORY, it sends the following message to your console:
DS_TOTAL_MEMORY recalculated and changed from old_value Kb
to new_value Kb

The variable old_value represents the value that you assigned to

DS_TOTAL_MEMORY in your configuration file. The variable new_value represents
the value that the database server derived.

When you receive the preceding message, you can use the algorithm to investigate
what values the database server considers inappropriate. You can then take
corrective action based on your investigation.

The following sections document the algorithm that the database server uses to
derive the new value for DS_TOTAL_MEMORY.

Deriving a minimum for decision-support memory:

In the first part of the algorithm that the database server uses to derive the new
value for the DS_TOTAL_MEMORY configuration parameter, the database server
establishes a minimum amount for decision-support memory.

When you assign a value to the DS_MAX_QUERIES configuration parameter, the

database server sets the minimum amount of decision-support memory according
to the following formula:
min_ds_total_memory = DS_MAX_QUERIES * 128 kilobytes

When you do not assign a value to the DS_MAX_QUERIES configuration

parameter, the database server uses the following formula instead, which is based
on the value of information in the VPCLASS configuration parameter:
min_ds_total_memory = NUMBER_CPUVPS * 2 * 128 kilobytes

Deriving a working value for decision-support memory:

In the second part of the algorithm that the database server uses to derive the new
value for the DS_TOTAL_MEMORY configuration parameter, the database server
establishes a working value for the amount of decision-support memory.

The database server verifies this amount in the third and final part of the
algorithm.

Chapter 4. Effect of configuration on memory utilization 4-13

 When the DS_TOTAL_MEMORY configuration parameter is set:

When the DS_TOTAL_MEMORY configuration parameter is set, the database

server checks whether the SHMTOTAL configuration parameter is set and then
determines which formula to use to calculate the amount of decision-support
memory.

When SHMTOTAL is set, the database server uses the following formula to
calculate the amount of decision-support memory:
IF DS_TOTAL_MEMORY <= SHMTOTAL - nondecision_support_memory THEN
decision_support_memory = DS_TOTAL_MEMORY
ELSE
decision_support_memory = SHMTOTAL -
nondecision_support_memory

This algorithm effectively prevents you from setting DS_TOTAL_MEMORY to

values that the database server cannot possibly allocate to decision-support
memory.

When SHMTOTAL is not set, the database server sets decision-support memory
equal to the value that you specified in DS_TOTAL_MEMORY.
Related reference:
DS_TOTAL_MEMORY configuration parameter (Administrator's Reference)

When the DS_TOTAL_MEMORY configuration parameter is not set:

When the DS_TOTAL_MEMORY configuration parameter is not set, the database

server uses other sources to calculate a value for the amount of decision-support
memory.

When SHMTOTAL is set, the database server uses the following formula to
calculate the amount of decision-support memory:
decision_support_memory = SHMTOTAL -
nondecision_support_memory

When the database server finds that you did not set SHMTOTAL, it sets
decision-support memory as in the following example:
decision_support_memory = min_ds_total_memory

For a description of the variable min_ds_total_memory, see “Deriving a minimum

for decision-support memory” on page 4-13.

Checking the derived value for decision-support memory:

In the final part of the algorithm that the database server uses to derive the new
value for the DS_TOTAL_MEMORY configuration parameter, the database server
verifies that the amount of shared memory is greater than min_ds_total_memory and
less than the maximum possible memory space for your computer.

When the database server finds that the derived value for decision-support
memory is less than the value of the min_ds_total_memory variable, it sets
decision-support memory equal to the value of min_ds_total_memory.

When the database server finds that the derived value for decision-support
memory is greater than the maximum possible memory space for your computer, it
sets decision-support memory equal to the maximum possible memory space.

4-14 IBM Informix Performance Guide

 The LOGBUFF configuration parameter and memory utilization
The LOGBUFF configuration parameter determines the amount of shared memory
that is reserved for each of the three buffers that hold the logical-log records until
they are flushed to the logical-log file on disk. The size of a buffer determines how
often it fills and therefore how often it must be flushed to the logical-log file on
disk.

If you log smart large objects, increase the size of the logical-log buffers to prevent
frequent flushing to the logical-log file on disk.
Related reference:
LOGBUFF configuration parameter (Administrator's Reference)

The LOW_MEMORY_RESERVE configuration parameter and

memory utilization
The LOW_MEMORY_RESERVE configuration parameter reserves a specific amount
of memory, in kilobytes, for the database server to use when critical activities are
needed and the server has limited free memory.

If you enable the new LOW_MEMORY_RESERVE configuration parameter by

setting it to a specified value in kilobytes, critical activities, such as rollback
activities, can complete even when you receive out-of-memory errors.
Related reference:
LOW_MEMORY_RESERVE configuration parameter (Administrator's
Reference)
onstat -g seg command: Print shared memory segment statistics
(Administrator's Reference)

The PHYSBUFF configuration parameter and memory utilization

The PHYSBUFF configuration parameter determines the amount of shared memory
that is reserved for each of the two buffers that serve as temporary storage space
for data pages that are about to be modified. The size of a buffer determines how
often it fills and therefore how often it must be flushed to the physical log on disk.

Choose a value for PHYSBUFF that is an even increment of the system page size.
Related reference:
PHYSBUFF configuration parameter (Administrator's Reference)

The LOCKS configuration parameter and memory utilization

The LOCKS configuration parameter specifies the initial size of the lock table. The
lock table holds an entry for each lock that a session uses. Each lock uses 120 bytes
within a lock table. You must provide for this amount of memory when you
configure shared memory.

If the number of locks needed by sessions exceeds the value set in the LOCKS
configuration parameter, the database server attempts to increase the lock table by
doubling its size. Each time that the lock table overflows (when the number of
locks needed is greater than the current size of the lock table), the database server
increases the size of the lock table, up to 99 times. Each time that the database
server increases the size of the lock table, the server attempts to double its size.
However, the server will limit each actual increase to no more than the maximum
number of added locks shown in Table 4-3 on page 4-16. After the 99th time that

Chapter 4. Effect of configuration on memory utilization 4-15

 the database server increases the lock table, the server no longer increases the size
of the lock table, and an application needing a lock receives an error.

The following table shows the maximum number of locks allowed on 32-bit and
64-bit platforms
Table 4-3. Maximum number of locks on 32-bit and 64-bit platforms
Maximum
Maximum Number of
Maximum Number of Locks Added Maximum
Number of Dynamic Lock Per Lock Table Number of
Platform Initial Locks Table Extensions Extension Locks Allowed
32-bit 8,000,000 99 100,000 8,000,000 + (99 x
100,000)
64-bit 500,000,000 99 1,000,000 500,000,000 + (99
x 1,000,000)

The default value for the LOCKS configuration parameter is 20,000.

To estimate a different value for the LOCKS configuration parameter, estimate the
maximum number of locks that a query needs and multiply this estimate by the
number of concurrent users. You can use the guidelines in the following table to
estimate the number of locks that a query needs.

Locks per TEXT or BYTE

Statement Isolation Level Table Row Key Data CLOB or BLOB Data
SELECT Dirty Read 0 0 0 0 0
SELECT Committed Read 1 0 0 0 0
SELECT Cursor Stability 1 1 0 0 1 lock for the CLOB or
BLOB value or (if
byte-range locking is
used) 1 lock for each
range
SELECT Indexed 1 Number of Number of 0 1 lock for the CLOB or
Repeatable Read rows that rows that BLOB value or (if
satisfy satisfy byte-range locking is
conditions conditions used) 1 lock for each
range
SELECT Sequential 1 0 0 0 1 lock for the CLOB or
Repeatable Read BLOB value or (if
byte-range locking is
used) 1 lock for each
range
INSERT Not applicable 1 1 Number of Number of 1 lock for the CLOB or
indexes pages in TEXT BLOB value
or BYTE data
DELETE Not applicable 1 1 Number of Number of 1 lock for the CLOB or
indexes pages in TEXT BLOB value
or BYTE data
UPDATE Not applicable 1 1 2 per Number of 1 lock for the CLOB or
changed key pages in old BLOB value or (if
value plus new byte-range locking is
TEXT or BYTE used) 1 lock for each
data range

4-16 IBM Informix Performance Guide

 Important: During the execution of the SQL statement DROP DATABASE, the
database server acquires and holds a lock on each table in the database until the
entire DROP operation completes. Make sure that the value for LOCKS is large
enough to accommodate the largest number of tables in a database.
Related concepts:
“Configuring and managing lock usage” on page 8-12
Related reference:
LOCKS configuration parameter (Administrator's Reference)

The RESIDENT configuration parameter and memory

utilization
The RESIDENT configuration parameter specifies whether shared-memory
residency is enforced for the resident portion of database server shared memory.
This configuration parameter works only on computers that support forced
residency.

The resident portion in the database server contains the buffer pools that are used
for database read and write activity. Performance improves when these buffers
remain in physical memory.

You should set the RESIDENT parameter to 1. If forced residency is not an option
on your computer, the database server issues an error message and ignores this
configuration parameter.

On machines that support 64-bit addressing, you can have a very large buffer pool
and the virtual portion of database server shared memory can also be very large.
The virtual portion contains various memory caches that improve performance of
multiple queries that access the same tables (see “Configure and monitor memory
caches” on page 4-21). To make the virtual portion resident in physical memory in
addition to the resident portion, set the RESIDENT parameter to -1.

If your buffer pool is very large, but your physical memory is not very large, you
can set RESIDENT to a value greater than 1 to indicate the number of memory
segments to stay in physical memory. This specification makes only a subset of the
buffer pool resident.

You can turn residency on or off for the resident portion of shared memory in the
following ways:
v Use the onmode utility to reverse temporarily the state of shared-memory
residency while the database server is online.
v Change the RESIDENT parameter to turn shared-memory residency on or off the
next time that you start database server shared memory.
Related reference:
RESIDENT configuration parameter (Administrator's Reference)

The SHMADD and EXTSHMADD configuration parameters and

memory utilization
The SHMADD configuration parameter specifies the size of each increment of
shared memory that the database server dynamically adds to the virtual portion.
The EXTSHMADD configuration parameter specifies the size of a virtual-extension

Chapter 4. Effect of configuration on memory utilization 4-17

 segment that is added when user-defined routines or DataBlade routines run in
user-defined virtual processors. Trade-offs are involved in determining the size of
an increment.

Adding shared memory uses CPU cycles. The larger each increment, the fewer
increments are required, but less memory is available for other processes. Adding
large increments is generally preferred; but when memory is heavily loaded (the
scan rate or paging-out rate is high), smaller increments allow better sharing of
memory resources among competing programs.

The range of values for SHMADD is 1024 through 4294967296 KB for a 64-bit
operating system and 1024 through 524288 KB for a 32-bit operating system. The
following table contains recommendations for setting SHMADD according to the
size of physical memory.

Memory Size SHMADD Value

256 MB or less 8192 KB (the default)
257 - 512 MB 16,384 KB
Larger than 512 MB 32,768 KB

The range of values for EXTSHMADD is the same as the range of values of
SHMADD.

Note: A shared memory segment can be as large as 4 terabytes, depending on

platform limits and the value of the SHMMAX kernel parameter. Use the onstat -g
seg command to display the number of shared-memory segments that the database
server is currently using.
Related tasks:
“Configuring UNIX shared memory” on page 4-6
Related reference:
SHMADD configuration parameter (Administrator's Reference)
EXTSHMADD configuration parameter (Administrator's Reference)

The SHMTOTAL configuration parameter and memory

utilization
The SHMTOTAL configuration parameter places an absolute upper limit on the
amount of shared memory that an instance of the database server can use.

If the SHMTOTAL configuration parameter is set to 0 or left unassigned, the

database server continues to attach additional shared memory as needed until no
virtual memory is available on the system.

You can usually set the SHMTOTAL configuration parameter to 0, except in the
following cases:
v You must limit the amount of virtual memory that the database server uses for
other applications or other reasons.
v Your operating system runs out of swap space and performs abnormally. In this
case, you can set SHMTOTAL to a value that is a few megabytes less than the
total swap space that is available on your computer.
v You are using automatic low memory management.
Related reference:
4-18 IBM Informix Performance Guide
 SHMTOTAL configuration parameter (Administrator's Reference)

The SHMVIRTSIZE configuration parameter and memory

utilization
The SHMVIRTSIZE parameter specifies the size of the virtual portion of shared
memory to allocate when you start the database server. The virtual portion of
shared memory holds session- and request-specific data as well as other
information.

Although the database server adds increments of shared memory to the virtual
portion as needed to process large queries or peak loads, allocation of shared
memory increases time for transaction processing. Therefore, you should set
SHMVIRTSIZE to provide a virtual portion large enough to cover your normal
daily operating requirements. The size of SHMVIRTSIZE can be as large the
SHMMAX configuration parameter allows.

The maximum value of SHMVIRTSIZE, which must be a positive integer, is:

v 4 terabytes on a 64-bit database server
v 2 gigabytes on a 32-bit database server

For an initial setting, it is suggested that you use the larger of the following values:
v 8000
v connections * 350

The connections variable is the number of connections for all network types that are
specified in the sqlhosts information by one or more NETTYPE configuration
parameters. (The database server uses connections * 200 by default.)

Once system utilization reaches a stable workload, you can reconfigure a new
value for SHMVIRTSIZE. As noted in “Freeing shared memory with onmode -F”
on page 4-7, you can instruct the database server to release shared-memory
segments that are no longer in use after a peak workload or large query.
Related reference:
SHMVIRTSIZE configuration parameter (Administrator's Reference)

The SHMVIRT_ALLOCSEG configuration parameter and

memory utilization
The SHMVIRT_ALLOCSEG configuration parameter specifies a threshold at which
the database server should allocate memory. This configuration parameter also
defines an alarm event security-code that is activated if the server cannot allocate
the new memory segment, thus ensuring that the database server never runs out of
memory.

When you set the SHMVIRT_ALLOCSEG configuration parameter, you must:

v Specify the percentage of memory used or the whole number of kilobytes
remaining on the server. You cannot use negative values and values between 0
and .39.
v Specify the alarm event-security code, which is a value ranging from 1 (not
noteworthy) to 5 (fatal). If you do not specify an event-security code, the server
sets the value to 3, which is the default value.

Example 1:

Chapter 4. Effect of configuration on memory utilization 4-19

 SHMVIRT_ALLOCSEG 3000, 4

This specifies that if the database serve has 3000 kilobytes remaining in virtual
memory and additional kilobytes of memory cannot be allocated, the server raises
an alarm level of 4.

Example 2:
SHMVIRT_ALLOCSEG .8, 4

This specifies that if the database server has twenty percent remaining in virtual
memory and additional kilobytes of memory cannot be allocated, the server raises
an alarm level of 4.
Related reference:
Event Alarm Parameters (Administrator's Reference)
SHMVIRT_ALLOCSEG configuration parameter (Administrator's Reference)

The STACKSIZE configuration parameter and memory

utilization
The STACKSIZE configuration parameter indicates the initial stack size for each
thread. The database server assigns the amount of space that this parameter
indicates to each active thread. This space comes from the virtual portion of
database server shared memory. You can reduce the amount of shared memory
that the database server adds dynamically.

To reduce the amount of shared memory that the database server adds
dynamically, estimate the amount of the stack space required for the average
number of threads that your system runs and include that amount in the value
that you set for the SHMVIRTSIZE configuration parameter.

To estimate the amount of stack space that you require, use the following formula:
stacktotal = STACKSIZE * avg_no_of_threads
avg_no_of_threads
is the average number of threads. You can monitor the number of active
threads at regular intervals to determine this amount. Use onstat -g sts to
check the stack use of threads. A general estimate is between 60 and 70
percent of the total number of connections (specified in the NETTYPE
parameters in your ONCONFIG file), depending on your workload.

The database server also executes user-defined routines (UDRs) with user threads
that use this stack. Programmers who write user-defined routines should take the
following measures to avoid stack overflow:
v Do not use large automatic arrays.
v Avoid excessively deep calling sequences.
v For DB-Access only: Use mi_call to manage recursive calls.

If you cannot avoid stack overflow with these measures, use the STACK modifier
of the CREATE FUNCTION statement to increase the stack for a particular routine.
Related reference:
STACKSIZE configuration parameter (Administrator's Reference)

4-20 IBM Informix Performance Guide

Configure and monitor memory caches
The database server uses caches to store information in memory instead of
performing a disk read or another operation to obtain the information. These
memory caches improve performance for multiple queries that access the same
tables. You can set some configuration parameters to increase the effectiveness of
each cache. You can view information about memory caches by running onstat
commands.

The following table lists the main memory caches that have the greatest effect on
performance and how to configure and monitor those caches.
Table 4-4. Main memory caches
onstat
Cache Name Cache Description Configuration Parameters command
Data Dictionary Stores information about the table DD_HASHSIZE: The maximum onstat -g dic
definition (such as column names and number of buckets in the cache.
data types).
DD_HASHMAX: The number of
tables in each bucket
Data Distribution Stores distribution statistics for a column. DS_POOLSIZE: The maximum onstat -g dsc
number of entries in the cache.

DS_HASHSIZE: The number of

buckets in the cache.
SQL Statement Stores parsed and optimized SQL STMT_CACHE: Enable the SQL onstat -g ssc
statements. statement cache.

STMT_CACHE_HITS: The number

of times anSQL statement is run
before it is cached.

STMT_CACHE_NOLIMIT: Prohibit
entries into the SQL statement cache
when allocated memory exceeds the
value of the STMT_CACHE_SIZE
configuration parameter.

STMT_CACHE_NUMPOOL: The
number of memory pools for the
SQL statement cache.

STMT_CACHE_SIZE: The size of

the SQL statement cache, in KB.
UDR Stores frequently used user-defined PC_POOLSIZE: The maximum onstat -g prc
routines and SPL routines. number of user-defined routines
and SPL routines in the cache.

PC_HASHSIZE: The number of

buckets in the UDR cache.

Related concepts:
“SPL routine executable format stored in UDR cache” on page 10-33
Related reference:
onstat -g cac command: Print information about caches (Administrator's
Reference)

Chapter 4. Effect of configuration on memory utilization 4-21

 onstat -g dsc command: Print distribution cache information (Administrator's
Reference)
onstat -g prc command: Print sessions using UDR or SPL routines
(Administrator's Reference)
onstat -g ssc command: Print SQL statement occurrences (Administrator's
Reference)
Database configuration parameters (Administrator's Reference)

Data-dictionary cache
The first time that the database server accesses a table, it retrieves the information
that it needs about the table (such as the column names and data types) from the
system catalog tables on disk. After the database server has accessed the table, it
places that information in the data-dictionary cache in shared memory.

Figure 4-1 shows how the database server uses this cache for multiple users. User 1
accesses the column information for tabid 120 for the first time. The database
server puts the column information in the data-dictionary cache. When user 2, user
3 and user 4 access the same table, the database server does not have to read from
disk to access the data-dictionary information for the table. Instead, it reads the
dictionary information from the data-dictionary cache in memory.

User 2 User 3 User 4 User 1

Shared-memory data-dictionary cache

syscolumns
syscolumns
tabid colname
tabid colname
120 lname
120 lname
120 fname
120 fname

Figure 4-1. Data-dictionary cache

The database server still places pages for system catalog tables in the buffer pool,
as it does all other data and index pages. However, the data-dictionary cache offers
an additional performance advantage, because the data-dictionary information is
organized in a more efficient format and organized to allow fast retrieval.

Data-dictionary configuration
The database server uses a hashing algorithm to store and locate information
within the data-dictionary cache. The DD_HASHSIZE and DD_HASHMAX
configuration parameters control the size of the data-dictionary cache.

To modify the number of buckets in the data-dictionary cache, use DD_HASHSIZE

(must be a prime number). To modify the number of tables that can be stored in
one bucket, use DD_HASHMAX.

For medium to large systems, you can start with the following values for these
configuration parameters:
v DD_HASHSIZE: 503
v DD_HASHMAX: 4

4-22 IBM Informix Performance Guide

 With these values, you can potentially store information about 2012 tables in the
data-dictionary cache, and each hash bucket can have a maximum of 4 tables.

If the bucket reaches the maximum size, the database server uses a least recently
used mechanism to clear entries from the data dictionary.
Related reference:
DD_HASHSIZE configuration parameter (Administrator's Reference)
DD_HASHMAX configuration parameter (Administrator's Reference)

Data-distribution cache
The query optimizer uses distribution statistics generated by the UPDATE
STATISTICS statement in the MEDIUM or HIGH mode to determine the query
plan with the lowest cost. The first time that the optimizer accesses the distribution
statistics for a column, the database server retrieves the statistics from the
sysdistrib system catalog table on disk and places that information in the
data-distribution cache in memory.

Figure 4-2 shows how the database server accesses the data-distribution cache for
multiple users. When the optimizer accesses the column distribution statistics for
User 1 for the first time, the database server puts the distribution statistics in the
data-distribution cache. When the optimizer determines the query plan for user 2,
user 3 and user 4 who access the same column, the database server does not have
to read from disk to access the data-distribution information for the table. Instead,
it reads the distribution statistics from the data-distribution cache in shared
memory.

User 2 User 3 User 4 User 1

Shared-memory data-dictionary cache

sysdistrib
sysdistrib
tabid colno … endcat
120 1 tabid colno …
120 2 endcat

Figure 4-2. Data-distribution cache

The database server initially places pages for the sysdistrib system catalog table in
the buffer pool as it does all other data and index pages. However, the
data-distribution cache offers additional performance advantages. It:
v Is organized in a more efficient format
v Is organized to allow fast retrieval
v Bypasses the overhead of the buffer pool management
v Frees more pages in the buffer pool for actual data pages rather than system
catalog pages
v Reduces I/O operations to the system catalog table

Data-distribution configuration
The database server uses a hashing algorithm to store and locate information
within the data-distribution cache. The DS_POOLSIZE configuration parameter

Chapter 4. Effect of configuration on memory utilization 4-23

 controls the size of the data-distribution cache and controls the total number of
column distributions that can be stored in the data-distribution cache.

To modify the number of buckets in the data-distribution cache, use the

DS_HASHSIZE configuration parameter.

The following formula determines the number of column distributions that can be
stored in one bucket.
Distributions_per_bucket = DS_POOLSIZE / DS_HASHSIZE

To modify the number of distributions per bucket, change either the

DS_POOLSIZE or DS_HASHSIZE configuration parameter.

For example, with the default values of 127 for DS_POOLSIZE and 31 for
DS_HASHSIZE, you can potentially store distributions for about 127 columns in
the data-distribution cache. The cache has 31 hash buckets, and each hash bucket
can have an average of 4 entries.

The values that you set for DS_HASHSIZE and DS_POOLSIZE, depend on the
following factors:
v The number of columns for which you run the UPDATE STATISTICS statement
in HIGH or MEDIUM mode and you expect to be used most often in frequently
run queries.
If you do not specify columns when you run UPDATE STATISTICS for a table,
the database server generates distributions for all columns in the table.
You can use the values of DD_HASHSIZE and DD_HASHMAX as guidelines for
DS_HASHSIZE and DS_POOLSIZE. The DD_HASHSIZE and DD_HASHMAX
specify the size for the data-dictionary cache, which stores information and
statistics about tables that queries access.
For medium to large systems, you can start with the following values:
– DD_HASHSIZE 503
– DD_HASHMAX 4
– DS_HASHSIZE 503
– DS_POOLSIZE 2000
Monitor these caches by running the onstat -g dsc command to see the actual
usage, and you can adjust these parameters accordingly.
v The amount of memory available
The amount of memory that is required to store distributions for a column
depends on the level at which you run UPDATE STATISTICS. Distributions for a
single column might require between 1 KB and 2 MB, depending on whether
you specify medium or high mode or enter a finer resolution percentage when
you run UPDATE STATISTICS.

If the size of the data-distribution cache is too small, the following performance
problems can occur:
v The database server uses the DS_POOLSIZE value to determine when to remove
entries from the data-distribution cache. However, if the optimizer needs the
dropped distributions for another query, the database server must reaccess them
from the sysdistrib system catalog table on disk. The additional I/O and buffer
pool operations to access sysdistrib on disk adds to the total response time of
the query.
The database server tries to maintain the number of entries in data-distribution
cache at the DS_POOLSIZE value. If the total number of entries reaches within
4-24 IBM Informix Performance Guide
 an internal threshold of DS_POOLSIZE, the database server uses a least recently
used mechanism to remove entries from the data-distribution cache. The number
of entries in a hash bucket can go past this DS_POOLSIZE value, but the
database server eventually reduces the number of entries when memory
requirements drop.
v If DS_HASHSIZE is small and DS_POOLSIZE is large, overflow lists can be long
and require more search time in the cache.
Overflow occurs when a hash bucket already contains an entry. When multiple
distributions hash to the same bucket, the database server maintains an overflow
list to store and retrieve the distributions after the first one.
If DS_HASHSIZE and DS_POOLSIZE are approximately the same size, the
overflow lists might be smaller or even nonexistent, which might waste memory.
However, the amount of unused memory is insignificant overall.

You might want to change the values of the DS_HASHSIZE and DS_POOLSIZE
configuration parameters if you see the following situations:
v If the data-distribution cache is full most of the time and commonly used
columns are not listed in the distribution name field, try increasing the values
of the DS_HASHSIZE and DS_POOLSIZE configuration parameters.
v If the total number of entries is much lower than the value of the DS_POOLSIZE
configuration parameter, you can reduce the values of the DS_HASHSIZE and
DS_POOLSIZE configuration parameters.
Related reference:
DD_HASHSIZE configuration parameter (Administrator's Reference)
DD_HASHMAX configuration parameter (Administrator's Reference)
DS_POOLSIZE configuration parameter (Administrator's Reference)
onstat -g dsc command: Print distribution cache information (Administrator's
Reference)

Monitor and tune the SQL statement cache

The SQL statement cache stores optimized SQL statements so that multiple users
who run the same SQL statement can achieve some performance improvements.

These performance improvements are:

v Reduced response times because they bypass the optimization step, as Figure 4-3
on page 4-26 shows
v Reduced memory usage because the database server shares query data
structures among users

For more information about the effect of the SQL statement cache on the
performance of individual queries, see “Optimize queries with the SQL statement
cache” on page 13-40.

Figure 4-3 on page 4-26 shows how the database server accesses the SQL statement
cache for multiple users.
v When the database server runs an SQL statement for User 1 for the first time,
the database server checks whether the same exact SQL statement is in the SQL
statement cache. If it is not in the cache, the database server parses the
statement, determines the optimal query plan, and runs the statement.
v When User 2 runs the same SQL statement, the database server finds the
statement in the SQL statement cache and does not optimize the statement.

Chapter 4. Effect of configuration on memory utilization 4-25

 v Similarly, if User 3 and User 4 run the same SQL statement, the database server
does not optimize the statement. Instead, it uses the query plan in the SQL
statement cache in memory.

User 1 User 2 User 3 User 4

1. Check Cache 1. Check Cache

2. Parse 2. Parse

3. Execute 3. Optimize

4. Execute

SQL Statement Cache SCC Memory Pool

Query Structures Query Structures

Figure 4-3. Database server actions when using the SQL statement cache

Prepared statements and the statement cache

Prepared statements are inherently cached for a single session. This means that if a
prepared statement is executed many times or if a single cursor is opened many
times, the session uses the same prepared query plan.

If a session prepares a statement and then executes it many times, the SQL
statement cache does not affect performance, because the statement is optimized
just once during the PREPARE statement.

However, if other sessions also prepare that same statement, or if the first session
prepares the statement several times, the statement cache usually provides a direct
performance benefit, because the database server only calculates the query plan
once. Of course, the original session might gain a small benefit from the statement
cache, even if it only prepares the statement once, because other sessions use less
memory, and the database server does less work for the other sessions

4-26 IBM Informix Performance Guide

SQL statement cache configuration
The value of the STMT_CACHE configuration parameter enables or disables the
SQL statement cache.

For more information about how the value of the STMT_CACHE configuration
parameter enables the SQL statement cache, see “Enabling the SQL statement
cache” on page 13-42 describes.

Figure 4-4 shows how the database server uses the values of the pertinent
configuration parameters for the SQL statement cache. Further explanation follows
the figure.

User 1 User 4

A
A Allocate memory from Pools
1. Check Cache 1. Check Cache specified by parameter:
- Match - No Match - STMT_CACHE_NUMPOOL

B Before Add Entry, check parameters:

2. Parse 2. Parse
- STMT_CACHE_HITS
- STMT_CACHE_SIZE
B - STMT_CACHE_NOLIMIT
3. Execute 3. Optimize

SQL Statement Cache Pool

4. Execute
Query Structures

SQL Statement Cache

Full Entries on SSC LRU Queues:

Query Structures Query Structures

Query Structures Query Structures

Figure 4-4. Configuration parameters that affect the SQL statement cache

When the database server uses the SQL statement cache for a user, it means the
database server takes the following actions:
v Checks the SQL statement cache first for a match of the SQL statement that the
user is executing
v If the SQL statement matches an entry, executes the statement using the query
memory structures in the SQL statement cache (User 2 in Figure 4-4)
v If the SQL statement does not match an entry, the database server checks if it
qualifies for the cache.
For information about what qualifies an SQL statement for the cache, see SQL
statement cache qualifying criteria.

Chapter 4. Effect of configuration on memory utilization 4-27

 v If the SQL statement qualifies, inserts an entry into the cache for subsequent
executions of the statement.

The following parameters affect whether or not the database server inserts the SQL
statement into the cache (User 1 in Figure 4-4 on page 4-27):
v STMT_CACHE_HITS specifies the number of times the statement executes with
an entry in the cache (referred to as hit count). The database server inserts one of
the following entries, depending on the hit count:
– If the value of STMT_CACHE_HITS is 0, inserts a fully cached entry, which
contains the text of the SQL statement plus the query memory structures
– If the value of STMT_CACHE_HITS is not 0 and the statement does not exist
in the cache, inserts a key-only entry that contains the text of the SQL
statement. Subsequent executions of the SQL statement increment the hit
count.
– If the value of STMT_CACHE_HITS is equal to the number of hits for a
key-only entry, adds the query memory structures to make a fully cached
entry.
v STMT_CACHE_SIZE specifies the size of the SQL statement cache, and
STMT_CACHE_NOLIMIT specifies whether or not to limit the memory of the
cache to the value of STMT_CACHE_SIZE. If you do not specify the
STMT_CACHE_SIZE parameter, it defaults to 524288 (512 * 1024) bytes.
The default value for STMT_CACHE_NOLIMIT is 1, which means the database
server will insert entries into the SQL statement cache even though the total
amount of memory might exceed the value of STMT_CACHE_SIZE.
When STMT_CACHE_NOLIMIT is set to 0, the database server inserts the SQL
statement into the cache if the current size of the cache will not exceed the
memory limit.

The following sections on STMT_CACHE_HITS, STMT_CACHE_SIZE,

STMT_CACHE_NOLIMIT, STMT_CACHE_NUMPOOL and provide more details
on how the following configuration parameters affect the SQL statement cache and
reasons why you might want to change their default values.

Number of SQL statement executions

When the SQL statement cache is enabled, the database server inserts a qualified
SQL statement and its memory structures immediately in the SQL statement cache
by default.

If your workload has a disproportionate number of ad hoc queries, use the

STMT_CACHE_HITS configuration parameter to specify the number of times an
SQL statement is executed before the database server places a fully cached entry in
the statement cache.

When the STMT_CACHE_HITS configuration parameter is greater than 0 and the

number of times the SQL statement has been executed is less than
STMT_CACHE_HITS, the database server inserts key-only entries in the cache.
This specification minimizes unshared memory structures from occupying the
statement cache, which leaves more memory for SQL statements that applications
use often.

Monitor the number of hits on the SQL statement cache to determine if your
workload is using this cache effectively. The following sections describe ways to
monitor the SQL statement cache hits.
Related concepts:

4-28 IBM Informix Performance Guide

 “Too many single-use queries in the SQL statement cache” on page 4-32
Related reference:
STMT_CACHE_HITS configuration parameter (Administrator's Reference)

Monitoring the number of hits on the SQL statement cache:

To monitor the number of hits in the SQL statement cache, run the onstat -g ssc
command.

The onstat -g ssc command displays fully cached entries in the SQL statement
cache. Figure 4-5 shows sample output for onstat -g ssc.

onstat -g ssc

Statement Cache Summary:

#lrus currsize maxsize Poolsize
#hits nolimit
4 49456 524288 57344
0  1

Statement Cache Entries:

lru hash ref_cnt
hits flag heap_ptr database user

-----------------
------------------------------------------------------------
0 153 0
0 -F a7e4690 vjp_stores virginia
SELECT * FROM customer, orders
WHERE customer.customer_num = orders.customer_num

AND order_date > "01/01/07"

1 259 0
0  -F aa58c20 vjp_stores virginia

SELECT * FROM customer, orders
WHERE customer.customer_num = orders.customer_num

AND order_date > "01/01/2007"

2 232 0
1  DF aa3d020 vjp_stores virginia

SELECT C.customer_num, O.order_num
FROM customer C, orders O, items I
WHERE C.customer_num = O.customer_num

AND O.order_num = I.order_num

3 232 1
1  -F aa8b020 vjp_stores virginia

SELECT C.customer_num, O.order_num
FROM customer C, orders O, items I
WHERE C.customer_num = O.customer_num
AND O.order_num = I.order_num

Total number of entries: 4.

Figure 4-5. onstat -g ssc output

To monitor the number of times that the database server reads the SQL statement
within the cache, look at the following output columns:
v In the Statement Cache Summary portion of the onstat -g ssc output, the #hits
column is the value of the SQL_STMT_HITS configuration parameter.
In Figure 4-5, the #hits column in the Statement Cache Summary portion of the
output has a value of 0, which is the default value of the STMT_CACHE_HITS
configuration parameter.

Chapter 4. Effect of configuration on memory utilization 4-29

 Important: The database server uses entries in the SQL statement cache only if
the statements are exactly the same. The first two entries in Figure 4-5 on page
4-29 are not the same because each contains a different literal value in the
order_date filter.
v In the Statement Cache Entries portion of the onstat -g ssc output, the hits
column shows the number of times that the database server ran each individual
SQL statement from the cache. In other words, the column shows the number of
times that the database server uses the memory structures in the cache instead
of optimizing the statements to generate them again.
The first time that it inserts the statement in the cache, the hits value is 0.
– The first two SQL statements in Figure 4-5 on page 4-29 have a hits column
value of 0, which indicates that each statement is inserted into the cache but
not yet run from the cache.
– The last two SQL statements in Figure 4-5 on page 4-29 have a hits column
value of 1, which indicates that these statements ran once from the cache.
The hits value for individual entries indicates how much sharing of memory
structures is done. Higher values in the hits column indicate that the SQL
statement cache is useful in improving performance and memory usage.

For a complete description of the output fields that onstat -g ssc displays, see
“SQL statement cache information in onstat -g ssc output” on page 4-36.

Determining the number of nonshared entries in the SQL statement cache:

To determine how many nonshared entries exist in the SQL statement cache, run
onstat -g ssc all.

The onstat -g ssc all option displays the key-only entries in addition to the fully
cached entries in the SQL statement cache.

To determine how many nonshared entries exist in the cache:

1. Compare the onstat -g ssc all output with the onstat -g ssc output.
2. If the difference between these two outputs shows that many nonshared entries
exist in the SQL statement cache, increase the value of the STMT_CACHE_HITS
configuration parameter to allow more shared statements to reside in the cache
and reduce the management overhead of the SQL statement cache.

You can use one of the following methods to change the STMT_CACHE_HITS
parameter value:
v Update the ONCONFIG file to specify the STMT_CACHE_HITS configuration
parameter. You must restart the database server for the new value to take effect.
You can use a text editor to edit the ONCONFIG file. Then bring down the
database server with the onmode -ky command and restart with the oninit
command.
v Increase the STMT_CACHE_HITS configuration parameter dynamically while
the database server is running:
You can use any of the following methods to reset the STMT_CACHE_HITS
value at run time:
– Issue the onmode -W command. The following example specifies that three
(3) instances are required before a new query is added to the statement cache:
onmode -W STMT_CACHE_HITS 2

4-30 IBM Informix Performance Guide

 – Call the ADMIN or TASK function of the SQL administration API. The
following example is equivalent to the onmode command in the previous
example:
EXECUTE FUNCTION TASK("ONMODE", "W", "STMT_CACHE_HITS", "2");
If you increase STMT_CACHE_HITS dynamically without updating the
configuration file, and the database server is subsequently restarted, the
STMT_CACHE_HITS setting reverts the value in the ONCONFIG file. Therefore,
if you want the setting to persist after subsequent restarts, modify the
ONCONFIG file.

Monitoring and tuning the size of the SQL statement cache

If the size of the SQL statement cache is too small, performance problems can
occur. You can monitor the effectiveness of the size of the SQL statement cache.

The following performance problems can occur:

v Frequently executed SQL statements are not in the cache
The statements used most often should remain in the SQL statement cache. If the
SQL statement cache is not large enough, the database server might not have
enough room to keep these statements when other statements come into the
cache. For subsequent executions, the database server must reparse, reoptimize,
and reinsert the SQL statement into the cache. Try increasing
STMT_CACHE_SIZE.
v The database server spends a lot of time cleaning the SQL statement cache
The database server tries to prevent the SQL statement cache from allocating
large amounts of memory by using a threshold (70 percent of the
STMT_CACHE_SIZE parameter) to determine when to remove entries from the
SQL statement cache. If the new entry causes the size of the SQL statement
cache to exceed the threshold, the database server removes least recently used
entries (that are not currently in use) before inserting the new entry.
However, if a subsequent query needs the removed memory structures, the
database server must reparse and reoptimize the SQL statement. The additional
processing time to regenerate these memory structures adds to the total response
time of the query.

You can set the size of the SQL statement cache in memory with the
STMT_CACHE_SIZE configuration parameter. The value of the parameter is the
size in kilobytes. If STMT_CACHE_SIZE is not set, the default value is 512
kilobytes.

The onstat -g ssc output shows the value of STMT_CACHE_SIZE in the maxsize
column. In Figure 4-5 on page 4-29, this maxsize column has a value of 524288,
which is the default value (512 * 1024 = 524288).

Use the onstat -g ssc and onstat -g ssc all options to monitor the effectiveness of
size of the SQL statement cache. If you do not see cache entries for the SQL
statements that applications use most, the SQL statement cache might be too small
or too many unshared SQL statement occupy the cache. The following sections
describe how to determine these situations.
Related reference:
STMT_CACHE_SIZE configuration parameter (Administrator's Reference)

Chapter 4. Effect of configuration on memory utilization 4-31

 Changing the size of the SQL statement cache:

You can analyze onstat -g ssc all output to determine if the SQL statement cache is
too small. If the size of the cache is too small, you can change it.

To determine if the size of the SQL statement cache is too small:

1. Run onstat -g ssc all to determine if the cache is too small.
2. Look at the values in the following output columns in the Statement Cache
Entries portion of the onstat -g ssc all output:
v The flags column shows the current status of an SQL statement in the cache.
A value of F in the second position indicates that the statement is currently
fully cached.
A value of - in the second position indicates that only the statement text
(key-only entry) is in the cache. Entries with this - value in the second
position appear in the onstat -g ssc all output, but not in the onstat -g ssc
output.
v The hits column shows the number of times the SQL statement has been
executed, excluding the first time it is inserted into the cache.
If you do not see fully cached entries for statements that applications use most
and the value in the hits column is large for the entries that do occupy the
cache, then the SQL statement cache is too small.

To change the size of the SQL statement cache:

1. Update the value of the STMT_CACHE_SIZE configuration parameter.
2. Restart the database server for the new value to take effect.
Related reference:
STMT_CACHE_SIZE configuration parameter (Administrator's Reference)

Too many single-use queries in the SQL statement cache:

When the database server places many queries that are only used once in the
cache, they might replace statements that other applications use often. You can
view onstat -g ssc all output to determine if too many unshared SQL statements
occupy the cache. If so, you can prevent unshared SQL statements from being fully
cached.

Look at the values in the following output columns in the Statement Cache
Entries portion of the onstat -g ssc all output. If you see a lot of entries that have
both of the following values, too many unshared SQL statements occupy the cache:
v flags column value of F in the second position
A value of F in the second position indicates that the statement is currently fully
cached.
v hits column value of 0 or 1
The hits column shows the number of times the SQL statement has been
executed, excluding the first time it is inserted into the cache.

Increase the value of the STMT_CACHE_HITS configuration parameter to prevent

unshared SQL statements from being fully cached.
Related concepts:
“Number of SQL statement executions” on page 4-28
Related reference:

4-32 IBM Informix Performance Guide

 STMT_CACHE_HITS configuration parameter (Administrator's Reference)

Memory limit and size

Although the database server tries to clean the SQL statement cache, sometimes
entries cannot be removed because they are currently in use. In this case, the size
of the SQL statement cache can exceed the value of the STMT_CACHE_SIZE
configuration parameter.

The default value of the STMT_CACHE_NOLIMIT configuration parameter is 1,

which means the database server inserts the statement even though the current
size of the cache might be greater than the value of the STMT_CACHE_SIZE
parameter.

If the value of the STMT_CACHE_NOLIMIT configuration parameter is 0, the

database server does not insert either a fully-qualified or key-only entry into the
SQL statement cache if the size will exceed the value of STMT_CACHE_SIZE.

Use the onstat -g ssc option to monitor the current size of the SQL statement
cache. Look at the values in the following output columns of the onstat -g ssc
output:
v The currsize column shows the number of bytes currently allocated in the SQL
statement cache.
In Figure 4-5 on page 4-29, the currsize column has a value of 11264.
v The maxsize column shows the value of STMT_CACHE_SIZE.
In Figure 4-5 on page 4-29, the maxsize column has a value of 524288, which is
the default value (512 * 1024 = 524288).

When the SQL statement cache is full and users are currently executing all
statements within it, any new SQL statements that a user executes can cause the
SQL statement cache to grow beyond the size that STMT_CACHE_SIZE specifies.
When the database server is no longer using an SQL statement within the SQL
statement cache, it frees memory in the SQL statement cache until the size reaches
a threshold of STMT_CACHE_SIZE. However, if thousands of concurrent users are
executing several ad hoc queries, the SQL statement cache can grow very large
before any statements are removed. In such cases, take one of the following
actions:
v Set the STMT_CACHE_NOLIMIT configuration parameter to 0 to prevent
insertions when the cache size exceeds the value of the STMT_CACHE_SIZE
parameter.
v Set the STMT_CACHE_HITS parameter to a value greater than 0 to prevent
caching unshared SQL statements.

You can use one of the following methods to change the STMT_CACHE_NOLIMIT
configuration parameter value:
v Update the ONCONFIG file to specify the STMT_CACHE_NOLIMIT
configuration parameter. You must restart the database server for the new value
to take effect.
v Use the onmode -W command to override the STMT_CACHE_NOLIMIT
configuration parameter dynamically while the database server is running.
onmode -W STMT_CACHE_NOLIMIT 0
If you restart the database server, the value reverts the value in the ONCONFIG
file. Therefore, if you want the setting to remain for subsequent restarts, modify
the ONCONFIG file.

Chapter 4. Effect of configuration on memory utilization 4-33

 Related reference:
STMT_CACHE_HITS configuration parameter (Administrator's Reference)

Multiple SQL statement cache pools

Under some circumstances when the SQL statement cache is enabled, the database
server allocates memory from one pool for query structures.

These circumstances are:

v When the database server does not find a matching entry in the cache
v When the database server finds a matching key-only entry in the cache and the
hit count reaches the value of the STMT_CACHE_HITS configuration parameter

This one pool can become a bottleneck as the number of users increases. The
STMT_CACHE_NUMPOOL configuration parameter allows you to configure
multiple sscpools.

You can monitor the pools in the SQL statement cache to determine the following
situations:
v The number of SQL statement cache pools is sufficient for your workload.
v The size or limit of the SQL statement cache is not causing excessive memory
management.
Related reference:
STMT_CACHE_NUMPOOL configuration parameter (Administrator's
Reference)

Number of SQL statement cache pools:

When the SQL statement cache (SSC) is enabled, the database server allocates
memory from the SSC pool for unlinked SQL statements. The default value for the
STMT_CACHE_NUMPOOL configuration parameter is 1. As the number of users
increases, this one SSC pool might become a bottleneck.

The number of longspins on the SSC pool indicates whether or not the SSC pool is
a bottleneck.

Use the onstat -g spi option to monitor the number of longspins on an SSC pool.
The onstat -g spi command displays a list of the resources in the system for which
a wait was required before a latch on the resource could be obtained. During the
wait, the thread spins (or loops), trying to acquire the resource. The onstat -g spi
output displays the number of times a wait (Num Waits column) was required for
the resource and the number of total loops (Num Loops column). The onstat -g spi
output displays only resources that have at least one wait.

Figure 4-6 shows an excerpt of sample output for onstat -g spi. Figure 4-6 indicates
that no waits occurred for any SSC pool (the Name column does not list any SSC
pools).

Spin locks with waits:

Num Waits Num Loops Avg Loop/Wait Name
34477 387761 11.25 mtcb sleeping_lock
312 10205 32.71 mtcb vproc_list_lock

Figure 4-6. onstat -g spi output

4-34 IBM Informix Performance Guide

If you see an excessive number of longspins (Num Loops column) on an SSC pool,
increase the number of SSC pools in the STMT_CACHE_NUMPOOL configuration
parameter to improve performance.
Related reference:
STMT_CACHE_NUMPOOL configuration parameter (Administrator's
Reference)

Size of SQL statement cache pools and the current cache:

Use the onstat -g ssc pool option to monitor the usage of each SQL statement
cache (SSC) pool.

The onstat -g ssc pool command displays the size of each pool. The onstat -g ssc
option displays the cumulative size of the SQL statement cache in the currsize
column. This current size is the size of memory allocated from the SSC pools by
the statements that are inserted into the cache. Because not all statements that
allocate memory from the SSC pools are inserted into the cache, the current cache
size could be smaller than the total size of the SSC pools. Normally, the total size
of all SSC pools does not exceed the STMT_CACHE_SIZE value.

Figure 4-7 shows sample output for onstat -g ssc pool.

onstat -g ssc pool

Pool Summary:
name class addr totalsize freesize #allocfrag #freefrag
sscpool0 V a7e4020 57344 2352 52 7

Blkpool Summary:
name class addr size #blks

Figure 4-7. onstat -g ssc pool output

The Pool Summary section of the onstat -g ssc pool output lists the following
information for each pool in the cache.

Column Description
name The name of the SQL statement cache (SSC)
pool
class The shared-memory segment type in which
the pool has been created. For SSC pools, this
value is always “V” for the virtual portion of
shared-memory.
addr The shared-memory address of the SSC pool
structure
totalsize The total size, in bytes, of this SSC pool
freesize the number of free bytes in this SSC pool
#allocfrag The number of contiguous areas of memory
in this SSC pool that are allocated
#freefrag The number of contiguous areas of memory
that are not used in this SSC pool

Chapter 4. Effect of configuration on memory utilization 4-35

 The Blkpool Summary section of the onstat -g ssc pool output lists the following
information for all pools in the cache.

Column Description
name The name of the SSC pool
class The shared-memory segment type in which
the pool has been created. For SSC pools, this
value is always “V” for the virtual portion of
shared-memory.
addr The shared-memory address of the SSC pool
structure
totalsize The total size, in bytes, of this SSC pool
#blks The number of 8-kilobyte blocks that make
up all the SSC pools

SQL statement cache information in onstat -g ssc output

The onstat -g ssc command displays summary information for the SQL statement
cache.

The onstat -g ssc command displays the following information for the SQL
statement cache.
Table 4-5. SQL statement cache information in onstat -g ssc output
Column Description
#lrus The number of LRU queues. Multiple LRU
queues facilitate concurrent lookup and
insertion of cache entries.
currsize The number of bytes currently allocated to
entries in the SQL statement cache
maxsize The number of bytes specified in the
STMT_CACHE_SIZE configuration
parameter
poolsize The cumulative number of bytes for all pools
in the SQL statement cache. Use the onstat -g
ssc pool option to monitor individual pool
usage.
#hits Setting of the STMT_CACHE_HITS
configuration parameter, which specifies the
number of times that a query is executed
before it is inserted into the cache
nolimit Setting of STMT_CACHE_NOLIMIT
configuration parameter

The onstat -g ssc command lists the following information for each fully cached
entry in the cache. The onstat -g ssc all option lists the following information for
both the fully cached entries and key-only entries.

Column Description
lru The LRU identifier
hash The hash-bucket identifier

4-36 IBM Informix Performance Guide

Column Description
ref_cnt The number of sessions currently using this
statement
hits The number of times that users read the
query from the cache, excluding the first time
the statement entered the cache
flags Shows flag codes.

The flag codes for position 1 are:

D Indicates that the statement has
been dropped
A statement in the cache can be
dropped (not used any more) when
one of its dependencies has
changed. For example, when you
run UPDATE STATISTICS for the
table, the optimizer statistics might
change, making the query plan for
the SQL statement in the cache
obsolete. In this case, the database
server marks the statement as
dropped the next time that it tries to
use it.
- Indicates that the statement has not
been dropped

The flag codes for position 2 are:

F Indicates that the cache entry is
fully cached and contains the
memory structures for the query
- Indicates that the statement is not
fully cached
A statement is not fully cached
when the number of times the
statement has been executed is less
than the value of the
STMT_CACHE_HITS configuration
parameter. Entries with this - value
in the second position appear in the
onstat -g ssc all but not in the
onstat -g ssc output.
heap_ptr Pointer to the associated heap for the
statement
database Database against which the SQL statement is
executed
user User executing the SQL statement
statement Statement text as it would be used to test for
a match

Chapter 4. Effect of configuration on memory utilization 4-37

Session memory
The database server uses the virtual portion of shared memory mainly for user
sessions. The majority of the memory that each user session allocates is for SQL
statements. The amount of used memory can vary from one statement to another.
You can determine which session and which statements have high memory
utilization.

Use the following utility options to determine which session and prepared SQL
statements have high memory utilization:
v onstat -g mem
v onstat -g stm

The onstat -g mem option displays memory usage of all sessions. You can find the
session that is using the most memory by looking at the totalsize and freesize
output columns. The following figure shows sample output for onstat -g mem.
This sample output shows the memory utilization of three user sessions with the
values 14, 16, 17 in the names output column.

onstat -g mem

Pool Summary:
name class addr totalsize freesize #allocfrag #freefrag
...
14 V a974020 45056 11960 99 10
16 V a9ea020 90112 10608 159 5
17 V a973020 45056 11304 97 13
...
Blkpool Summary:
name class addr size #blks
mt V a235688 798720 19
global V a232800 0 0

Figure 4-8. onstat -g mem output

To display the memory allocated by each prepared statement, use the onstat -g stm
option. The following figure shows sample output for onstat -g stm.

onstat -g stm

session 25 --------------------------------------------------
sdblock heapsz statement ('*’ = Open cursor)
d36b018 9216 select sum(i) from t where i between -1 and ?
d378018 6240 *select tabname from systables where tabid=7
d36b114 8400 <SPL statement>

Figure 4-9. onstat -g stm output

The heapsz column in the output in Figure 4-9 shows the amount of memory used
by the statement. An asterisk (*) precedes the statement text if a cursor is open on
the statement. The output does not show the individual SQL statements in an SPL
routine.

To display the memory for only one session, specify the session ID in the onstat -g
stm option. For an example, see “Monitor session memory with onstat -g mem
and onstat -g stm output” on page 13-52.

4-38 IBM Informix Performance Guide

 Related tasks:
“Estimating the size of the virtual portion of shared memory” on page 4-4

Data-replication buffers and memory utilization

Data replication requires two instances of the database server, a primary one and a
secondary one, running on two computers. If you implement data replication for
your database server, the database server holds logical-log records in the
data-replication buffer before it sends them to the secondary database server.

The data-replication buffer is always the same size as the logical-log buffer.

Memory latches
The database server uses latches to control access to shared memory structures
such as the buffer pool or the memory pools for the SQL statement cache. You can
obtain statistics on latch use and information about specific latches. These statistics
provide a measure of the system activity.

The statistics include the number of times that threads waited to obtain a latch. A
large number of latch waits typically results from a high volume of processing
activity in which the database server is logging most of the transactions.

Information about specific latches includes a listing of all the latches that are held
by a thread and any threads that are waiting for latches. This information allows
you to locate any specific resource contentions that exist.

You, as the database administrator, cannot configure or tune the number of latches.
However, you can increase the number of memory structures on which the
database server places latches to reduce the number of latch waits. For example,
you can tune the number of SQL statement cache memory pools or the number of
SQL statement cache LRU queues. For more information, see “Multiple SQL
statement cache pools” on page 4-34.

Warning: Never stop a database server process that is holding a latch. If you do,
the database server immediately initiates an abort.

Monitoring latches with command-line utilities

You can obtain information about latches by running onstat -p or onstat -s.

Monitoring latches with onstat -p

Run onstat -p to obtain the values in the lchwaits field. This field stores the
number of times that a thread was required to wait for a shared-memory latch.

Figure 4-10 shows an excerpt of sample onstat -p output that shows the lchwaits
field.

...
ixda-RA idx-RA da-RA logrec-RA RA-pgsused
lchwaits
5 0 204 0 148
12

Figure 4-10. Partial onstat -p output showing the lchwaits field

Related reference:

Chapter 4. Effect of configuration on memory utilization 4-39

 onstat -p command: Print profile counts (Administrator's Reference)

Monitoring latches with onstat -s

Run onstat -s to obtain general latch information. The output includes the
userthread column, which lists the address of any user thread that is waiting for a
latch.

You can compare this address with the user addresses in the onstat -u output to
obtain the user-process identification number.

Figure 4-11 shows sample onstat -s output.

...
Latches with lock or userthread set
name address lock wait userthread
LRU1 402e90 0 0 6b29d8
bf[34] 4467c0 0 0 6b29d8
...

Figure 4-11. onstat -s output

Monitoring latches with SMI tables

You can query the sysprofile SMI table to obtain the number of times a thread
waited for a latch.

The latchwts column of the sysprofile table contains the number of times that a
thread waited for a latch.

Encrypted values
An encrypted value uses more storage space than the corresponding plain text
value because all of the information needed to decrypt the value except the
encryption key is stored with the value.

Omitting the hint used with the password can reduce encryption overhead by up
to 50 bytes. If you are using encrypted values, you must make sure that you have
sufficient space available for the values.

Note: Embedding zero bytes in the encrypted result is not recommended.

Related concepts:
Column-level encryption (Security Guide)
Related reference:
Calculating storage requirements for encrypted data (SQL Syntax)

4-40 IBM Informix Performance Guide

Chapter 5. Effect of configuration on I/O activity
The configuration of your database server affects I/O activity.

The following factors affect I/O activity:

v The assignment of chunks and dbspaces can create I/O hot spots, or disk
partitions with a disproportionate amount of I/O activity.
v Your allocation of critical data, sort areas, and areas for temporary files and
index builds can place intermittent loads on various disks.
v How you configure read-ahead can increase the effectiveness of individual I/O
operations.
v How you configure the background I/O tasks, such as logging and page
cleaning, can affect I/O throughput.

Chunk and dbspace configuration

The number of disks that you use and the configuration of your chunks, dbspaces,
and blobspaces affect the performance of your database server. You can improve
performance by planning disk use and the configuration of chunks, dbspaces, and
blobspaces.

All the data that resides in a database is stored on disk. The speed at which the
database server can copy the appropriate data pages to and from disk determines
how well your application performs.

All the data that resides in a database is stored on disk. The Optical Subsystem
also uses a magnetic disk to access TEXT or BYTE data that is retrieved from
optical media. The speed at which the database server can copy the appropriate
data pages to and from disk determines how well your application performs.

Disks are typically the slowest component in the I/O path for a transaction or
query that runs entirely on one host computer. Network communication can also
introduce delays in client/server applications, but these delays are typically
outside the control of the database server administrator. For information about
actions that the database server administrator can take to improve network
communications, see “Network buffer pools” on page 3-17 and “Connections and
CPU utilization” on page 3-25.

Disks can become overused or saturated when users request pages too often.
Saturation can occur in the following situations:
v You use a disk for multiple purposes, such as for both logging and active
database tables.
v Disparate data resides on the same disk.
v Table extents become interleaved.

The various functions that your application requires, as well as the

consistency-control functions that the database server performs, determine the
optimal disk, chunk, and dbspace layout for your application. The more disks that
you make available to the database server, the easier it is to balance I/O across
them. For more information about these factors, see Chapter 6, “Table performance
considerations,” on page 6-1.

© Copyright IBM Corp. 1996, 2014 5-1

 This section outlines important issues for the initial configuration of your chunks,
dbspaces, and blobspaces. Consider the following issues when you decide how to
lay out chunks and dbspaces on disks:
v Placement and mirroring of critical data
v Load balancing
v Reduction of contention
v Ease of backup and restore

Together with round-robin fragmentation, you can balance chunks over disks and
controllers, saving time and handling errors. Placing multiple chunks on a single
disk can improve throughput.

Associate disk partitions with chunks

You should assign chunks to entire disk partitions. When a chunk coincides with a
disk partition (or device), it is easy to track disk-space use, and you avoid errors
caused by miscalculated offsets.

The maximum size for a chunk is 4 terabytes.

Associate dbspaces with chunks

You should associate a single chunk with a dbspace, especially when that dbspace
is to be used for a table fragment.

For more information about table placement and layout, see Chapter 6, “Table
performance considerations,” on page 6-1.

Placing system catalog tables with database tables

When a disk that contains the system catalog for a particular database fails, the
entire database remains inaccessible until the system catalog is restored. Because of
this potential inaccessibility, do not cluster the system catalog tables for all
databases in a single dbspace. Instead place the system catalog tables with the
database tables that they describe.

To create a system catalog table in the table dbspace:

1. Create a database in the dbspace in which the table is to reside.
2. Use the SQL statements DATABASE or CONNECT to make that database the
current database.
3. Enter the CREATE TABLE statement to create the table.

I/O for cooked files for dbspace chunks

On UNIX, you can control the use of direct I/O for cooked files used for dbspace
chunks.

On UNIX, you can allocate disk space in two ways:

v Use files that are buffered through the operating system. These files are often
called cooked files.
v Use unbuffered disk access, also called raw disk space.

When dbspaces reside on raw disk devices (also called character-special devices), the
database server uses unbuffered disk access. A raw disk directly transfers data
between the database server memory and disk without also copying data.

5-2 IBM Informix Performance Guide

 While you should generally use raw disk devices on UNIX systems to achieve
better performance, you might prefer to use cooked files, which are easier to
allocate and manage than raw devices. If you use cooked files, you might be able
to get better performance by enabling the Informix direct I/O option.

In addition, Informix supports a separate concurrent I/O option on AIX operating

systems. If you enable concurrent I/O on AIX, you get both unbuffered I/O and
concurrent I/O. With concurrent I/O, writing to two parts of a file can occur
concurrently. (On some other operating systems and file systems, enabling direct
I/O also enables concurrent I/O as part of the same file system direct I/O feature.)

To determine the best performance, perform benchmark testing for the dbspace
and table layout on your system.

Direct I/O (UNIX)

On UNIX, you can use direct I/O to improve the performance of cooked files.
Direct I/O can be beneficial because it avoids file system buffering. Because direct
I/O uses unbuffered I/O, it is more efficient for reads and writes that go to disk
(as opposed to those reads and writes that merely access the file system buffers).

Direct I/O generally requires data to be aligned on disk sector boundaries.

Direct I/O also allows the use of kernel asynchronous I/O (KAIO), which can
further improve performance. By using direct I/O and KAIO where available, the
performance of cooked files used for dbspace chunks can approach the
performance of raw devices.

If your file system supports direct I/O for the page size used for the dbspace
chunk, the database server operates as follows:
v Does not use direct I/O by default.
v Uses direct I/O if the DIRECT_IO configuration parameter is set to 1.
v Uses KAIO (if the file system supports it) with direct I/O by default.
v Does not use KAIO with direct I/O if the environment variable KAIOOFF is set.

If Informix uses direct I/O for a chunk, and another program tries to open the
chunk file without using direct I/O, the open will normally succeed, but there can
be a performance penalty. The penalty can occur because the file system attempts
to ensure that each open sees the same file data, either by switching to buffered
I/O and not using direct I/O for the duration of the conflicting open, or by
flushing the file system cache before each direct I/O operation and invalidating the
file system cache after each direct write.

Informix does not use direct I/O for temporary dbspaces.

Related reference:
DIRECT_IO configuration parameter (UNIX) (Administrator's Reference)

Direct I/O (Windows)

Direct I/O is used for dbspace chunks on Windows platforms regardless of the
value of the DIRECT_IO configuration parameter.

Concurrent I/O (AIX only)

On AIX operating systems, you can use concurrent I/O in addition to direct I/O
for chunks that use cooked files. Concurrent I/O can improve performance,

Chapter 5. Effect of configuration on I/O activity 5-3

 because it allows multiple reads and writes to a file to occur concurrently, without
the usual serialization of noncompeting read and write operations.

Concurrent I/O can be especially beneficial when you have data in a single chunk
file striped across multiple disks.

Concurrent I/O, which you enable by setting the DIRECT_IO configuration

parameter to 2, includes the benefit of avoiding file system buffering and is subject
to the same limitations and use of KAIO as occurs if you use direct I/O without
concurrent I/O. Thus, when concurrent I/O is enabled, you get both unbuffered
I/O and concurrent I/O.

If Informix uses concurrent I/O for a chunk, and another program (such as an
external backup program) tries to open the same chunk file without using
concurrent I/O, the open operation will fail.

Informix does not use direct or concurrent I/O for cooked files used for temporary
dbspace chunks.
Related reference:
DIRECT_IO configuration parameter (UNIX) (Administrator's Reference)

Enabling the direct I/O or concurrent I/O option (UNIX)

Use the DIRECT_IO configuration parameter to enable the direct I/O option on
UNIX or the concurrent I/O option on AIX.

Prerequisites:
v You must log on as user root or informix.
v Direct I/O or concurrent I/O must be available and the file system must
support direct I/O for the page size used for the dbspace chunk.

To enable direct I/O, set the DIRECT_IO configuration parameter to 1.

To enable concurrent I/O with direct I/O on AIX operating systems, set the
DIRECT_IO configuration parameter to 2.

If you do not want to enable direct I/O or concurrent I/O, set the DIRECT_IO
configuration parameter to 0.
Related reference:
DIRECT_IO configuration parameter (UNIX) (Administrator's Reference)

Confirming the use of the direct or concurrent I/O option

(UNIX)
You can confirm and monitor the use of direct I/O or concurrent I/O (on AIX) for
cooked file chunks.

You can confirm the use of direct I/O or concurrent I/O by:
v Displaying onstat -d information.
The onstat -d command displays information that includes a flag that identifies
whether direct I/O, concurrent I/O (on AIX), or neither is used for cooked file
chunks.
v Verifying that the DIRECT_IO configuration parameter is set to 1 (for direct I/O)
or 2 (for concurrent I/O).

5-4 IBM Informix Performance Guide

 Related reference:
DIRECT_IO configuration parameter (UNIX) (Administrator's Reference)
onstat -d command: Print chunk information (Administrator's Reference)

Placement of critical data

The disk or disks that contain the system reserved pages, the physical log, and the
dbspaces that contain the logical-log files are critical to the operation of the
database server. The database server cannot operate if any of these elements
becomes unavailable. By default, the database server places all three critical
elements in the root dbspace.

To arrive at an appropriate placement strategy for critical data, you must make a
trade-off between the availability of data and maximum logging performance.

The database server also places temporary table and sort files in the root dbspace
by default. You should use the DBSPACETEMP configuration parameter and the
DBSPACETEMP environment variable to assign these tables and files to other
dbspaces. For details, see “Configure dbspaces for temporary tables and sort files”
on page 5-8.

Consider separate disks for critical data components

If you place the root dbspace, logical log, and physical log in separate dbspaces on
separate disks, you can obtain some distinct performance advantages. The disks
that you use for each critical data component should be on separate controllers.

This approach has the following advantages:

v Isolates logging activity from database I/O and allows physical-log I/O requests
to be serviced in parallel with logical-log I/O requests
v Reduces the time that you need to recover from a failure
However, unless the disks are mirrored, there is an increased risk that a disk
that contains critical data might be affected in the event of a failure, which will
bring the database server to a halt and require the complete restoration of all
data from a level-0 backup.
v Allows for a relatively small root dbspace that contains only reserved pages, the
database partition, and the sysmaster database
In many cases, 10,000 kilobytes is sufficient.

The database server uses different methods to configure various portions of critical
data. To assign an appropriate dbspace for the root dbspace and physical log, set
the appropriate database server configuration parameters. To assign the logical-log
files to an appropriate dbspace, use the onparams utility.

For more information about the configuration parameters that affect each portion
of critical data, see “Configuration parameters that affect critical data” on page 5-8.

Consider mirroring for critical data components

Consider mirroring for the dbspaces that contain critical data. Mirroring these
dbspaces ensures that the database server can continue to operate even when a
single disk fails.

Chapter 5. Effect of configuration on I/O activity 5-5

 However, depending on the mix of I/O requests for a given dbspace, a trade-off
exists between the fault tolerance of mirroring and I/O performance. You obtain a
marked performance advantage when you mirror dbspaces that have a
read-intensive usage pattern and a slight performance disadvantage when you
mirror write-intensive dbspaces.

Most modern storage devices have excellent mirroring capabilities, and you can
use those devices instead of the mirroring capabilities of the database server.

When mirroring is in effect, two disks are available to handle read requests, and
the database server can process a higher volume of those requests. However, each
write request requires two physical write operations and does not complete until
both physical operations are performed. The write operations are performed in
parallel, but the request does not complete until the slower of the two disks
performs the update. Thus, you experience a slight performance penalty when you
mirror write-intensive dbspaces.

Consider mirroring the root dbspace

You can achieve a certain degree of fault tolerance with a minimum performance
penalty if you mirror the root dbspace and restrict its contents to read-only or
seldom-accessed tables.

When you place update-intensive tables in other, nonmirrored dbspaces, you can
use the database server backup-and-restore facilities to perform warm restores of
those tables in the event of a disk failure. When the root dbspace is mirrored, the
database server remains online to service other transactions while the failed disk is
being repaired.

When you mirror the root dbspace, always place the first chunk on a different
device than that of the mirror. The MIRRORPATH configuration parameter should
have a different value than ROOTPATH.
Related reference:
MIRRORPATH configuration parameter (Administrator's Reference)
ROOTPATH configuration parameter (Administrator's Reference)

Consider mirroring smart-large-object chunks

You can achieve higher availability and faster access if you mirror chunks that
contain metadata pages.

An sbspace is a logical storage unit composed of one or more chunks that store
smart large objects, which consist of CLOB (character large object) or BLOB (binary
large object) data.

The first chunk of an sbspace contains a special set of pages, called metadata, which
is used to locate smart large objects in the sbspace. Additional chunks that are
added to the sbspace can also have metadata pages if you specify them on the
onspaces command when you create the chunk.

Consider mirroring chunks that contain metadata pages for the following reasons:
v Higher availability
Without access to the metadata pages, users cannot access any smart large
objects in the sbspace. If the first chunk of the sbspace contains all of the
metadata pages and the disk that contains that chunk becomes unavailable, you
cannot access a smart large object in the sbspace, even if it resides on a chunk on

5-6 IBM Informix Performance Guide

 another disk. For high availability, mirror at least the first chunk of the sbspace
and any other chunk that contains metadata pages.
v Faster access
By mirroring the chunk that contains the metadata pages, you can spread read
activity across the disks that contain the primary chunk and mirror chunk.
Related concepts:
Sbspaces (Administrator's Guide)

Mirroring and its effect on the logical log

The logical log is write intensive. If the dbspace that contains the logical-log files is
mirrored, you encounter a slight double-write performance penalty. However, you
can adjust the rate at which logging generates I/O requests to a certain extent by
choosing an appropriate log buffer size and logging mode.

For details on the slight double-write performance penalty, see “Consider

mirroring for critical data components” on page 5-5.

With unbuffered and ANSI-compliant logging, the database server requests a flush
of the log buffer to disk for every committed transaction (two when the dbspace is
mirrored). Buffered logging generates far fewer I/O requests than unbuffered or
ANSI-compliant logging.

With buffered logging, the log buffer is written to disk only when it fills and all
the transactions that it contains are completed. You can reduce the frequency of
logical-log I/O even more if you increase the size of your logical-log buffers.
However, buffered logging leaves transactions in any partially filled buffers
vulnerable to loss in the event of a system failure.

Although database consistency is guaranteed under buffered logging, specific

transactions are not guaranteed against a failure. The larger the logical-log buffers,
the more transactions you might need to reenter when service is restored after a
failure.

Unlike the physical log, you cannot specify an alternative dbspace for logical-log
files in your initial database server configuration. Instead, use the onparams utility
first to add logical-log files to an alternative dbspace and then drop logical-log files
from the root dbspace.
Related reference:
The onparams Utility (Administrator's Reference)

Mirroring and its effect on the physical log

The physical log is write intensive, with activity occurring at checkpoints and
when buffered data pages are flushed. I/O to the physical log also occurs when a
page-cleaner thread is activated. If the dbspace that contains the physical log is
mirrored, you encounter a slight double-write performance penalty.

For details on the slight double-write performance penalty, see “Consider

mirroring for critical data components” on page 5-5.

To keep I/O to the physical log at a minimum, you can adjust the checkpoint
interval and the LRU minimum and maximum thresholds. (See “CKPTINTVL and
its effect on checkpoints” on page 5-31 and “BUFFERPOOL and its effect on page
cleaning” on page 5-40.)

Chapter 5. Effect of configuration on I/O activity 5-7

Configuration parameters that affect critical data
The configuration parameters that configure the root dbspace and the logical and
physical logs affect critical data.

You can use the following configuration parameters to configure the root dbspace:
v ROOTNAME
v ROOTOFFSET
v ROOTPATH
v ROOTSIZE
v MIRROR
v MIRRORPATH
v MIRROROFFSET

These parameters determine the location and size of the initial chunk of the root
dbspace and configure mirroring, if any, for that chunk. (If the initial chunk is
mirrored, all other chunks in the root dbspace must also be mirrored). Otherwise,
these parameters have no major impact on performance.

The following configuration parameters affect the logical logs:

v LOGSIZE
v LOGBUFF

LOGSIZE determines the size of each logical-log files. LOGBUFF determines the
size of the three logical-log buffers that are in shared memory. For more
information about LOGBUFF, see “The LOGBUFF configuration parameter and
memory utilization” on page 4-15.

The following configuration parameters determine the location and size of the
physical log:
v PHYSDBS
v PHYSFILE

Configure dbspaces for temporary tables and sort files

Applications that use temporary tables or large sort operations require a large
amount of temporary space. To improve performance of these applications, use the
DBSPACETEMP configuration parameter or the DBSPACETEMP environment
variable to designate one or more dbspaces for temporary tables and sort files.

Depending on how the temporary space is created, the database server uses the
following default locations for temporary table and sort files when you do not set
DBSPACETEMP:
v The dbspace of the current database, when you create an explicit temporary
table with the TEMP TABLE clause of the CREATE TABLE statement and do not
specify a dbspace for the table either in the IN dbspace clause or in the
FRAGMENT BY clause
This action can severely affect I/O to that dbspace. If the root dbspace is
mirrored, you encounter a slight double-write performance penalty for I/O to
the temporary tables and sort files.
v The root dbspace when you create an explicit temporary table with the INTO
TEMP option of the SELECT statement

5-8 IBM Informix Performance Guide

 This action can severely affect I/O to the root dbspace. If the root dbspace is
mirrored, you encounter a slight double-write performance penalty for I/O to
the temporary tables and sort files.
v The operating-system directory or file that you specify in one of the following
variables:
– In UNIX, the operating-system directory or directories that the
PSORT_DBTEMP environment variable specifies, if it is set
If PSORT_DBTEMP is not set, the database server writes sort files to the
operating-system file space in the /tmp directory.
– In Windows, the directory specified in TEMP or TMP in the User
Environment Variables window on Control Panel > System.
The database server uses the operating-system directory or files to direct any
overflow that results from the following database operations:
– SELECT statement with GROUP BY clause
– SELECT statement with ORDER BY clause
– Hash-join operation
– Nested-loop join operation
– Index builds

Warning: If you do not specify a value for the DBSPACETEMP configuration

parameter or the DBSPACETEMP environment variable, the database server uses
this operating-system file for implicit temporary tables. If this file system has
insufficient space to hold a sort file, the query that performs the sort returns an
error. Meanwhile, the operating system might be severely impacted until you
remove the sort file.

You can improve performance with the use of temporary dbspaces that you create
exclusively to store temporary tables and sort files. Use the DBSPACETEMP
configuration parameter and the DBSPACETEMP environment variable to assign
these tables and files to temporary dbspaces.

When you specify dbspaces in either the DBSPACETEMP configuration parameter

or the DBSPACETEMP environment variable, you gain the following performance
advantages:
v Reduced I/O impact on the root dbspace, production dbspaces, or
operating-system files
v Use of parallel sorts into the temporary files (to process query clauses such as
ORDER BY or GROUP BY, or to sort index keys when you execute CREATE
INDEX) when you specify more than one dbspace for temporary tables and
PDQ priority is set to greater than 0.
v Improved speed with which the database server creates temporary tables when
you assign two or more temporary dbspaces on separate disks
v Automatic fragmentation of the temporary tables across dbspaces when
SELECT....INTO TEMP statements are run

The following table shows statements that create temporary tables and information
about where the temporary tables are created.

Chapter 5. Effect of configuration on I/O activity 5-9

Statement That
Creates Temporary FRAGMENT BY Where Temp Table
Table Database Logged WITH NO LOG clause clause Created
CREATE TEMP Yes No No Root dbspace
TABLE
CREATE TEMP Yes Yes No One of dbspaces that
TABLE are specified in
DBSPACETEMP
CREATE TEMP Yes No Yes Cannot create temp
TABLE table. Error 229/196
SELECT ..INTO TEMP Yes Yes No Fragmented by
round-robin only in
the non-logged
dbspaces that are
specified in
DBSPACETEMP

Important: Use the DBSPACETEMP configuration parameter or the

DBSPACETEMP environment variable for better performance of sort operations
and to prevent the database server from unexpectedly filling file systems. The
dbspaces that you list must be composed of chunks that are allocated as
unbuffered devices.
Related concepts:
“Specify temporary tables in the DBSPACETEMP configuration parameter” on
page 5-11
Related reference:
DBSPACETEMP configuration parameter (Administrator's Reference)
CREATE TEMP TABLE statement (SQL Syntax)
INTO TEMP clause (SQL Syntax)

Creating temporary dbspaces

You can create a dbspace for the exclusive use of temporary tables and sort files.
The database server does not perform logical or physical logging of temporary
dbspaces, and temporary dbspaces are never backed up as part of a full-system
backup.

To create a dbspace for the exclusive use of temporary tables and sort files, use
onspaces -t. For best performance, use the following guidelines:
v If you create more than one temporary dbspace, create each dbspace on a
separate disk to balance the I/O impact.
v Place no more than one temporary dbspace on a single disk.

You cannot mirror a temporary dbspace that you create with onspaces -t.

Important: In the case of a database with logging, you must include the WITH NO
LOG clause in the SELECT... INTO TEMP statement to place the explicit temporary
tables in the dbspaces listed in the DBSPACETEMP configuration parameter and
the DBSPACETEMP environment variable. Otherwise, the database server stores
the explicit temporary tables in the root dbspace.
Related reference:

5-10 IBM Informix Performance Guide

 DBSPACETEMP configuration parameter (Administrator's Reference)
create tempdbspace argument: Create a temporary dbspace (SQL
administration API) (Administrator's Reference)
onspaces -c -d: Create a dbspace (Administrator's Reference)

Specify temporary tables in the DBSPACETEMP configuration

parameter
The DBSPACETEMP configuration parameter specifies a list of dbspaces in which
the database server places temporary tables and sort files by default. Some or all of
the dbspaces that you list in this configuration parameter can be temporary
dbspaces, which are reserved exclusively to store temporary tables and sort files.

If the database server inserts data into a temporary table through a SELECT INTO
TEMP operation that creates the TEMP table, that temporary table uses
round-robin distributed storage. Its fragments are created in the temporary
dbspaces that are listed in the DBSPACETEMP configuration parameter or in the
DBSPACETEMP environment variable. For example, the following query uses
round-robin distributed storage:
SELECT col1 FROM tab1
INTO TEMP temptab1 WITH NO LOG;

The DBSPACETEMP configuration parameter lets the database administrator

restrict which dbspaces the database server uses for temporary storage.

Important: The DBSPACETEMP configuration parameter is not set in the

onconfig.std file. For best performance with temporary tables and sort files, use
DBSPACETEMP to specify two or more dbspaces on separate disks.
Tips:
v If you work on a small system with a limited number of disks and
cannot place temporary dbspaces on different disk drives, you might
consider using 1 (or possibly 2) temporary dbspaces. This can reduce the
logging that is associated with the temporary dbspaces.
v If you have many disk drives, you can parallelize many operations (such
as sorts, joins, and temporary tables) without having multiple temporary
dbspaces. The number of temporary dbspaces that you have relates to
how much you want to spread the I/O out. A good starting place is 4
temporary dbspaces. If you create too many small temporary dbspaces,
you will not have enough space for nonparallel creation of large objects.
Related concepts:
“Configure dbspaces for temporary tables and sort files” on page 5-8
“Distribution schemes” on page 9-6
Related reference:
DBSPACETEMP configuration parameter (Administrator's Reference)
CREATE TEMP TABLE statement (SQL Syntax)

Override the DBSPACETEMP configuration parameter for a

session
To override the DBSPACETEMP configuration parameter, you can use the
DBSPACETEMP environment variable for both temporary tables and sort files.

Chapter 5. Effect of configuration on I/O activity 5-11

 This environment variable specifies a comma- or colon-separated list of dbspaces
in which to place temporary tables for the current session.

Important: Use the DBSPACETEMP configuration parameter or the

DBSPACETEMP environment variable for better performance of sort operations
and to prevent the database server from unexpectedly filling file systems.

You should use DBSPACETEMP rather than the PSORT_DBTEMP environment

variable to specify sort files for the following reasons:
v DBSPACETEMP typically yields better performance.
When dbspaces reside on character-special devices (also known as raw disk
devices), the database server uses unbuffered disk access. I/O is faster to
unbuffered devices than to regular (buffered) operating-system files because the
database server manages the I/O operation directly.
v PSORT_DBTEMP specifies one or more operating-system directories in which to
place sort files.
These operating-system files can unexpectedly fill on your computer because the
database server does not manage them.

Estimating temporary space for dbspaces and hash joins

You can estimate and increase the amount of temporary space for dbspaces and for
hash joins. If you do this, you can prevent the possible overflow of memory to
temporary space on disk.

You can use the following guidelines to estimate the amount of temporary space to
allocate:
v For OLTP applications, allocate temporary dbspaces that equal at least 10 percent
of the table.
v For DSS applications, allocate temporary dbspaces that equal at least 50 percent
of the table.

A hash join, which works by building a table (the hash table) from the rows in one
of the tables in a join, and then probing it with rows from the other table, can use
a significant amount of memory and can potentially overflow to temporary space
on disk. The hash table size is governed by the size of the table used to build the
hash table (which is often the smaller of the two tables in the join), after applying
any filters, which can reduce the number of rows and possibly reduce the number
of columns.

Hash-join partitions are organized into pages. Each page has a header. The header
and tuples are larger in databases on 64-bit platforms than in builds on 32-bit
platforms. The size of each page is the base page size (2K or 4K depending on
system) unless a single row needs more space. If you need more space, you can
add bytes to the length of your rows.

You can use the following formula to estimate the amount of memory that is
required for the hash table in a hash join:
hash_table_size = (32 bytes + row_size_smalltab) * num_rows_smalltab

where row_size_smalltab and num_rows_smalltab refer to the row size and the
number of rows, respectively, in the smaller of the two tables participating in the
hash join.

5-12 IBM Informix Performance Guide

 For example, suppose you have a page head that is 80 bytes in length and a row
header that is 48 bytes in length. Because each row must be aligned to 8 bytes, you
might need to add up to 7 bytes to the row length, as shown in these formulas:
per_row_size = 48 bytes + rowsize + mod(rowsize, 8)
page_size = base_page_size (2K or 4K)
rows_per_page = round_down_to_integer((page_size - 80 bytes) / per_row_size)

If the value of rows_per_page is less than one, increase the page_size value to the
smallest multiple of the base_page_size, as shown in this formula:
size = (numrows_smalltab / rows_per_page) * page_size

You can use the DS_NONPDQ_QUERY_MEM configuration parameter to

configure sort memory for all queries except PDQ queries. Its setting has no effect,
however, if the PDQ priority setting is greater than zero.

For more information, see “Hash join” on page 10-3 and “Configuring memory for
queries with hash joins, aggregates, and other memory-intensive elements” on
page 13-34.
Related reference:
DS_NONPDQ_QUERY_MEM configuration parameter (Administrator's
Reference)

PSORT_NPROCS environment variable

The PSORT_NPROCS environment variable specifies the maximum number of
threads that the database server can use to sort a query. When a query involves a
large sort operation, multiple sort threads can execute in parallel to improve the
performance of the query.

When the value of PDQ priority is 0 and PSORT_NPROCS is greater than 1, the
database server uses parallel sorts. The management of PDQ does not limit these
sorts. In other words, although the sort is executed in parallel, the database server
does not regard sorting as a PDQ activity. When PDQ priority is 0, the database
server does not control sorting by any of the PDQ configuration parameters.

When PDQ priority is greater than 0 and PSORT_NPROCS is greater than 1, the
query benefits both from parallel sorts and from PDQ features such as parallel
scans and additional memory. Users can use the PDQPRIORITY environment
variable to request a specific proportion of PDQ resources for a query. You can use
the MAX_PDQPRIORITY parameter to limit the number of such user requests. For
more information about MAX_PDQPRIORITY, see “Limiting PDQ resources in
queries” on page 3-11.

The database server allocates a relatively small amount of memory for sorting, and
that memory is divided among the PSORT_NPROCS sort threads. Sort processes
use temporary space on disk when not enough memory is allocated. For more
information about memory allocated for sorting, see “Estimating memory needed
for sorting” on page 7-19.

Important: For better performance for a sort operation, set PSORT_NPROCS

initially to 2 if your computer has multiple CPUs. If the subsequent CPU activity is
lower than I/O activity, you can increase the value of PSORT_NPROCS.

For more information about sorts during index builds, see “Improving
performance for index builds” on page 7-18.

Chapter 5. Effect of configuration on I/O activity 5-13

Configure sbspaces for temporary smart large objects
Applications can use temporary smart large objects for text, image, or other
user-defined data types that are only required during the life of the user session.
These applications do not require logging of the temporary smart large objects.
Logging adds I/O activity to the logical log and increases memory utilization.

You can store temporary smart large objects in a permanent sbspace or a

temporary sbspace.
v Permanent sbspaces
If you store the temporary smart large objects in a regular sbspace and keep the
default no logging attribute, changes to the objects are not logged, but the
metadata is always logged.
v Temporary sbspaces
Applications that update temporary smart large objects stored in temporary
sbspaces are significantly faster because the database server does not log the
metadata or the user data in a temporary sbspace.

To improve performance of applications that update temporary smart large objects,

specify the LOTEMP flag in the mi_lo_specset_flags or ifx_lo_specset_flags API
function and specify a temporary sbspace for the temporary smart large objects.
The database server uses the following order of precedence for locations to place
temporary smart large objects:
v The sbspace you specify in the mi_lo_specset_sbspace or ifx_lo_specset_sbspace
API function when you create the smart large object
Specify a temporary sbspace in the API function so that changes to the objects
and the metadata are not logged. The sbspace you specify in the API function
overrides any default sbspaces that the SBSPACETEMP or SBSPACENAME
configuration parameters might specify.
v The sbspace you specify in the IN Sbspace clause when you create an explicit
temporary table with the TEMP TABLE clause of the CREATE TABLE statement
Specify a temporary sbspace in the IN Sbspace clause so that changes to the
objects and the metadata are not logged.
v The permanent sbspace you specify in the SBSPACENAME configuration
parameter, if you do not specify an sbspace in the SBSPACETEMP configuration
parameter

If no temporary sbspace is specified in any of the above methods, then the

database server issues the following error message when you try to create a
temporary smart large object:
-12053 Smart Large Objects: No sbspace number specified.

Creating temporary sbspaces

To create an sbspace for the exclusive use of temporary smart large objects, use
onspaces -c -S with the -t option.

For best performance, use the following guidelines:

v If you create more than one temporary sbspace, create each sbspace on a
separate disk to balance the I/O impact.
v Place no more than one temporary sbspace on a single disk.

5-14 IBM Informix Performance Guide

 The database server does not perform logical or physical logging of temporary
sbspaces, and temporary sbspaces are never backed up as part of a full-system
backup. You cannot mirror a temporary sbspace that you create with onspaces -t.

Important: In the case of a database with logging, you must include the WITH NO
LOG clause in the SELECT... INTO TEMP statement to place the temporary smart
large objects in the sbspaces listed in the SBSPACETEMP configuration parameter.
Otherwise, the database server stores the temporary smart large objects in the
sbspace listed in the SBSPACENAME configuration parameter.
Related tasks:
Creating a temporary sbspace (Administrator's Guide)
Related reference:
onspaces -c -S: Create an sbspace (Administrator's Reference)

Specify which sbspaces to use for temporary storage

The SBSPACETEMP configuration parameter specifies a list of sbspaces in which
the database server places temporary smart large objects by default. Some or all of
the sbspaces that you list in this configuration parameter can be temporary
sbspaces, which are reserved exclusively to store temporary smart large objects.

Important: The SBSPACETEMP configuration parameter is not set in the

onconfig.std file. For best performance with temporary smart large objects, use
SBSPACETEMP to specify two or more sbspaces on separate disks.
Related reference:
SBSPACETEMP configuration parameter (Administrator's Reference)

Placement of simple large objects

You can store simple large objects in either the same dbspace in which the table
resides or in a blobspace.

A blobspace is a logical storage unit composed of one or more chunks that store
only simple large objects (TEXT or BYTE data). For information about sbspaces,
which store smart large objects (such as BLOB, CLOB, or multirepresentational
data), see “Factors that affect I/O for smart large objects” on page 5-20.

If you use a blobspace, you can store simple large objects on a separate disk from
the table with which the data is associated. You can store simple large objects
associated with different tables in the same blobspace.

You can create a blobspace with the onspaces utility or with an SQL administration
API command that uses the create blobspace argument with the admin() or task()
function.

You assign simple large objects to a blobspace when you create the tables with
which simple large objects are associated, using the CREATE TABLE statement.

Simple large objects are not logged and do not pass through the buffer pool.
However, frequency of checkpoints can affect applications that access TEXT or
BYTE data. For more information, see “LOGSIZE and LOGFILES and their effect
on checkpoints” on page 5-32.
Related reference:

Chapter 5. Effect of configuration on I/O activity 5-15

 CREATE TABLE statement (SQL Syntax)
create blobspace argument: Create a blobspace (SQL administration API)
(Administrator's Reference)
onspaces -c -b: Create a blobspace (Administrator's Reference)

Advantage of blobspaces over dbspaces

If you store simple large objects in a blobspace on a separate disk from the table
with which it is associated, instead of storing the objects in a dbspace, you can
obtain some performance advantages.

The performance advantages of storing simple large objects in a blobspace are:

v You have parallel access to the table and simple large objects.
v Unlike simple large objects stored in a dbspace, blobspace data is written
directly to disk. Simple large objects do not pass through resident shared
memory, which leaves memory pages free for other uses.
v Simple large objects are not logged, which reduces logging I/O activity for
logged databases.

For more information, see “Storing simple large objects in the tblspace or a
separate blobspace” on page 6-8.

Blobpage size considerations

Blobspaces are divided into units called blobpages. The database server retrieves
simple large objects from a blobspace in blobpage-sized units. You specify the size
of a blobpage in multiples of a disk page when you create the blobspace.

The optimal blobpage size for your configuration depends on the following factors:
v The size distribution of the simple large objects
v The trade-off between retrieval speed for your largest simple large object and the
amount of disk space that is wasted by storing simple large objects in large
blobpages

To retrieve simple large objects as quickly as possible, use the size of your largest
simple large object rounded up to the nearest disk-page-sized increment. This
scheme guarantees that the database server can retrieve even the largest simple
large object in a single I/O request. Although this scheme guarantees the fastest
retrieval, it has the potential to waste disk space. Because simple large objects are
stored in their own blobpage (or set of blobpages), the database server reserves the
same amount of disk space for every blobpage even if the simple large object takes
up a fraction of that page. Using a smaller blobpage allows you to make better use
of your disk, especially when large differences exist in the sizes of your simple
large objects.

To achieve the greatest theoretical utilization of space on your disk, you can make
your blobpage the same size as a standard disk page. Then many, if not most,
simple large objects would require several blobpages. Because the database server
acquires a lock and issues a separate I/O request for each blobpage, this scheme
performs poorly.

In practice, a balanced scheme for sizing uses the most frequently occurring
simple-large-object size as the size of a blobpage. For example, suppose that you
have 160 simple-large-object values in a table with the following size distribution:

5-16 IBM Informix Performance Guide

v Of these values, 120 are 12 kilobytes each.
v The other 40 values are 16 kilobytes each.

You can choose one of the following blobpage sizes:

v The 12-kilobyte blobpage size provides greater storage efficiency than a
16-kilobyte blobpage size, as the following two calculations show:
– 12 kilobytes
This configuration allows the majority of simple-large-object values to require
a single blobpage and the other 40 values to require two blobpages. In this
configuration, 8 kilobytes is wasted in the second blobpage for each of the
larger values. The total wasted space is as follows:
wasted-space = 8 kilobtyes * 40
= 329 kilobytes
– 16 kilobytes
In this configuration, 4 kilobytes is wasted in the extents of 120 simple large
objects. The total wasted space is as follows:
wasted-space = 4 kilobtyes * 120
= 640 kilobytes
v If your applications access the 16-kilobyte simple-large-object values more
frequently, the database server must perform a separate I/O operation for each
blobpage. In this case, the 16-kilobyte blobpage size provides better retrieval
speed than a 12-kilobyte blobpage size.

The maximum number of pages that a blobspace can contain is 2147483647.

Therefore, the size of the blobspace is limited to the blobpage size x 2147483647.
This includes blobpages in all chunks that make up the blobspace.

Tip: If a table has more than one simple-large-object column and the data values are not
close in size, store the data in different blobspaces, each with an appropriately sized
blobpage.

Optimize blobspace blobpage size

When you are evaluating blobspace storage strategy, you can measure efficiency by
two criteria: blobpage fullness and the blobpages required per simple large object.

Blobpage fullness refers to the amount of data within each blobpage. TEXT and
BYTE data stored in a blobspace cannot share blobpages. Therefore, if a single
simple large object requires only 20 percent of a blobpage, the remaining 80
percent of the page is unavailable for use.

However, avoid making the blobpages too small. When several blobpages are
needed to store each simple large object, you increase the overhead cost of storage.
For example, more locks are required for updates, because a lock must be acquired
for each blobpage.

Obtain blobspace storage statistics

To help you determine the optimal blobpage size for each blobspace, use the
oncheck -pB command.

The command lists the following statistics for each table (or database):
v The number of blobpages used by the table (or database) in each blobspace
v The average fullness of the blobpages used by each simple large object stored as
part of the table (or database)

Chapter 5. Effect of configuration on I/O activity 5-17

 Determine blobpage fullness with oncheck -pB output
The oncheck -pB command displays statistics that describe the average fullness of
blobpages. These statistics provide a measure of storage efficiency for individual
simple large objects in a database or table.

If you find that the statistics for a significant number of simple large objects show
a low percentage of fullness, the database server might benefit from changing the
size of the blobpage in the blobspace.

Both the oncheck -pB and onstat -d update commands display the same
information about the number of free blobpages. The onstat -d update command
displays the same information as onstat -d and an accurate number of free
blobpages for each blobspace chunk.

Execute oncheck -pB with either a database name or a table name as a parameter.
The following example retrieves storage information for all simple large objects
stored in the table sriram.catalog in the stores_demo database:
oncheck -pB stores_demo:sriram.catalog

oncheck -pB Output

Figure 5-1 shows the output of this command.

BLOBSpace Report for stores_demo:sriram.catalog

Total pages used by table 7

BLOBSpace usage:
Space Page Percent Full
Name Number Pages 0-25% 26-50% 51-75 76-100%
-------------------------------------------------------------
blobPIC 0x300080 1 x
blobPIC 0x300082 2 x
------
Page Size is 6144 3

bspc1 0x2000b2 2 x
bspc1 0x2000b6 2 x
------
Page Size is 2048 4

Figure 5-1. Output of oncheck -pB

Space Name is the name of the blobspace that contains one or more simple large
objects stored as part of the table (or database).

Page Number is the starting address in the blobspace of a specific simple large
object.

Pages is the number of the database server pages required to store this simple
large object.

Percent Full is a measure of the average blobpage fullness, by blobspace, for each
blobspace in this table or database.

Page Size is the size in bytes of the blobpage for this blobspace. Blobpage size is
always a multiple of the database server page size.

5-18 IBM Informix Performance Guide

The example output indicates that four simple large objects are stored as part of
the table sriram.catalog. Two objects are stored in the blobspace blobPIC in
6144-byte blobpages. Two more objects are stored in the blobspace bspc1 in
2048-byte blobpages.

The summary information that appears at the top of the display, Total pages used
by table is a simple total of the blobpages needed to store simple large objects. The
total says nothing about the size of the blobpages used, the number of simple large
objects stored, or the total number of bytes stored.

The efficiency information displayed under the Percent Full heading is imprecise,
but it can alert an administrator to trends in the storage of TEXT and BYTE data.

Interpreting blobpage average fullness:

You can analyze the output of the oncheck -pB command to calculate average
fullness.

The first simple large object listed in “Determine blobpage fullness with oncheck
-pB output” on page 5-18 is stored in the blobspace blobPIC and requires one
6144-byte blobpage. The blobpage is 51 to 75 percent full, meaning that the size is
between 0.51 * 6144 = 3133 bytes and 0.75 * 6144 = 4608. The maximum size of this
simple large object must be less than or equal to 75 percent of 6144 bytes, or 4608
bytes.

The second object listed under blobspace blobPIC requires two 6144-byte
blobpages for storage, or a total of 12,288 bytes. The average fullness of all
allocated blobpages is 51 to 75 percent. Therefore, the minimum size of the object
must be greater than 50 percent of 12,288 bytes, or 6144 bytes. The maximum size
of the simple large object must be less than or equal to 75 percent of 12,288 bytes,
or 9216 bytes. The average fullness does not mean that each page is 51 to 75
percent full. A calculation would yield 51 to 75 percent average fullness for two
blobpages where the first blobpage is 100 percent full and the second blobpage is 2
to 50 percent full.

Now consider the two simple large objects in blobspace bspc1. These two objects
appear to be nearly the same size. Both objects require two 2048-byte blobpages,
and the average fullness for each is 76 to 100 percent. The minimum size for these
simple large objects must be greater than 75 percent of the allocated blobpages, or
3072 bytes. The maximum size for each object is slightly less than 4096 bytes
(allowing for overhead).

Analyzing efficiency criteria with oncheck -pB output:

You can analyze the output of the oncheck -pB command to determine if there is a
more efficient storage strategy.

Looking at the efficiency information for that is shown for blobspace bspc1 in
Figure 5-1 on page 5-18, a database server administrator might decide that a better
storage strategy for TEXT and BYTE data would be to double the blobpage size
from 2048 bytes to 4096 bytes. (Blobpage size is always a multiple of the database
server page size.) If the database server administrator made this change, the
measure of page fullness would remain the same, but the number of locks needed
during an update of a simple large object would be reduced by half.

Chapter 5. Effect of configuration on I/O activity 5-19

 The efficiency information for blobspace blobPIC reveals no obvious suggestion
for improvement. The two simple large objects in blobPIC differ considerably in
size, and there is no optimal storage strategy. In general, simple large objects of
similar size can be stored more efficiently than simple large objects of different
sizes.

Factors that affect I/O for smart large objects

An sbspace is a logical storage unit, composed of one or more chunks, in which
you can store smart large objects (such as BLOB, CLOB, or multi representational
data). Disk layout for sbspaces, the settings of certain configuration parameters,
and some onspaces utility options affect I/O for smart large objects.

The DataBlade® API and the Informix ESQL/C application programming interface
also provide functions that affect I/O operations for smart large objects.

Important: For most applications, you should use the values that the database
server calculates for the disk-storage information.
Related concepts:
Sbspaces (Administrator's Guide)
Related reference:
What is Informix ESQL/C? (ESQL/C Guide)
DataBlade API overview (DataBlade API Guide)

Disk layout for sbspaces

You create sbspaces on separate disks from the table with which the data is
associated. You can store smart large objects associated with different tables within
the same sbspace. When you store smart large objects in an sbspace on a separate
disk from the table with which it is associated, the database server provides some
performance advantages.

These performance advantages are:

v You have parallel access to the table and smart large objects.
v When you choose not to log the data in an sbspace, you reduce logging I/O
activity for logged databases.

To create an sbspace, use the onspaces utility. You assign smart large objects to an
sbspace when you use the CREATE TABLE statement to create the tables with
which the smart large objects are associated.
Related reference:
onspaces -c -S: Create an sbspace (Administrator's Reference)
CREATE TABLE statement (SQL Syntax)

Configuration parameters that affect sbspace I/O

The SBSPACENAME, BUFFERPOOL, and LOGBUFF configuration parameters
affect the I/O performance of sbspaces.

The SBSPACENAME configuration parameter indicates the default sbspace name if

you do not specify the sbspace name when you define a column of data type
CLOB or BLOB. To reduce disk contention and provide better load balancing, place
the default sbspace on a separate disk from the table data.

5-20 IBM Informix Performance Guide

 The BUFFERPOOL configuration parameter specifies the default values for buffers
and LRU queues in a buffer pool for both the default page size buffer pool and for
any non-default pages size buffer pools. The size of your memory buffer pool
affects I/O operations for smart large objects because the buffer pool is the default
area of shared memory for these objects. If your applications frequently access
smart large objects, it is advantageous to have these objects in the buffer pool.
Smart large objects only use the default page size buffer pool. For information
about estimating the amount to increase your buffer pool for smart large objects,
see “The BUFFERPOOL configuration parameter and memory utilization” on page
4-10.

By default, the database server reads smart large objects into the buffers in the
resident portion of shared memory. For more information on using lightweight I/O
buffers, see “Lightweight I/O for smart large objects” on page 5-23.

The LOGBUFF configuration parameter affects logging I/O activity because it

specifies the size of the logical-log buffers that are in shared memory. The size of
these buffers determines how quickly they fill and therefore how often they need
to be flushed to disk.

If you log smart-large-object user data, increase the size of your logical-log buffer
to prevent frequent flushing to these log files on disk.
Related reference:
SBSPACENAME configuration parameter (Administrator's Reference)
BUFFERPOOL configuration parameter (Administrator's Reference)
LOGBUFF configuration parameter (Administrator's Reference)

onspaces options that affect sbspace I/O

When you create an sbspace with the onspaces utility, you specify information that
affects I/O performance. This information includes the size of extents, the
buffering mode (and whether you want the server to use lightweight I/O), and
logging.

Sbspace extents
As you add smart large objects to a table, the database server allocates disk space
to the sbspace in units called extents. Each extent is a block of physically
contiguous pages from the sbspace.

Even when the sbspace includes more than one chunk, each extent is allocated
entirely within a single chunk so that it remains contiguous. Contiguity is
important to I/O performance.

When the pages of data are contiguous, disk-arm motion is minimized when the
database server reads the rows sequentially. The mechanism of extents is a
compromise between the following competing requirements:
v The size of some smart large objects is not known in advance.
v The number of smart large objects in different tables can grow at different times
and different rates.
v All the pages of a single smart large object should ideally be adjacent for best
performance when you retrieve the entire object.

Because you might not be able to predict the number and size of smart large
objects, you cannot specify the extent length of smart large objects. Therefore, the

Chapter 5. Effect of configuration on I/O activity 5-21

 database server adds extents only as they are needed, but all the pages in any one
extent are contiguous for better performance. In addition, when the database server
creates a new extent that is adjacent to the previous extent, it treats both extents as
a single extent.

The number of pages in an sbspace extent is determined by one of the following

methods:
v The database server calculates the extent size for a smart large object from a set
of heuristics, such as the number of bytes in a write operation. For example, if
an operation asks to write 30 kilobytes, the database server tries to allocate an
extent the size of 30 kilobytes.
v The final size of the smart large object as indicated by one of the following
functions when you open the sbspace in an application program:
– For DB-Access: the DataBlade API mi_lo_specset_estbytes function. For more
information about the DataBlade API functions to open a smart large object
and set the estimated number of bytes, see the IBM Informix DataBlade API
Programmer's Guide.
– For ESQL/C: the Informix ESQL/C ifx_lo_specset_estbytes function. For more
information about the Informix ESQL/C functions to open a smart large
object and set the estimated number of bytes, see the IBM Informix ESQL/C
Programmer's Manual.

These functions are the best way to set the extent size because they reduce the
number of extents in a smart large object. The database server tries to allocate the
entire smart large object as one extent (if an extent of that size is available in the
chunk).
v The EXTENT_SIZE flag in the -Df option of the onspaces command when you
create or alter the sbspace
Most administrators do not use the onspaces EXTENT_SIZE flag because the
database server calculates the extent size from heuristics. However, you might
consider using the onspaces EXTENT_SIZE flag in the following situations:
– Many one-page extents are scattered throughout the sbspace.
– Almost all smart large objects are the same length.
v The EXTENT SIZE keyword of the CREATE TABLE statement when you define
the CLOB or BLOB column
Most administrators do not use the EXTENT SIZE keyword when they create or
alter a table because the database server calculates the extent size from
heuristics. However, you might consider using this EXTENT SIZE keyword if
almost all smart large objects are the same length.

Important: For most applications, you should use the values that the database
server calculates for the extent size. Do not use the DataBlade API
mi_lo_specset_extsz function or the Informix ESQL/C ifx_lo_specset_extsz
function to set the extent size of the smart large object.

If you know the size of the smart large object, it is recommended that you specify
the size in the DataBlade API mi_lo_specset_estbytes() function or Informix
ESQL/C ifx_lo_specset_estbytes() function instead of in the onspaces utility or the
CREATE TABLE or the ALTER TABLE statement. These functions are the best way
to set the extent size because the database server allocates the entire smart large
object as one extent (if it has contiguous storage in the chunk).

Extent sizes over one megabyte do not provide much I/O benefit because the
database server performs read and write operations in multiples of 60 kilobytes at

5-22 IBM Informix Performance Guide

the most. However, the database server registers each extent for a smart large
object in the metadata area; therefore, especially large smart large objects might
have many extent entries. Performance of the database server might degrade when
it accesses these extent entries. In this case, you can reduce the number of extent
entries in the metadata area if you specify the eventual size of the smart large
object in the mi_lo_specset_estbytes() function or ifx_lo_specset_estbytes()
function.

For more information, see “Improving metadata I/O for smart large objects” on
page 6-12.

Lightweight I/O for smart large objects

Instead of using the buffer pool, the administrator and programmer have the
option to use lightweight I/O. Lightweight I/O operations use private buffers in the
session pool of the virtual portion of shared memory.

By default, smart large objects pass through the buffer pool in the resident portion
of shared memory. Although smart large objects have lower priority than other
data, the buffer pool can become full when an application accesses many smart
large objects. A single application can fill the buffer pool with smart large objects
and leave little room for data that other applications might need. In addition, when
the database server performs scans of many pages into the buffer pool, the
overhead and contention associated with checking individual pages in and out
might become a bottleneck.

Important: Use private buffers only when you read or write smart large objects in
read or write operations greater than 8080 bytes and you seldom access them. That
is, if you have infrequent read or write function calls that read large amounts of
data in a single function invocation, lightweight I/O can improve I/O
performance.
Related concepts:
“The BUFFERPOOL configuration parameter and memory utilization” on page
4-10

Advantages of lightweight I/O for smart large objects:

Lightweight I/O provides some performance advantages, because the database

server is not using the buffer pool.

Lightweight I/O provides the following advantages:

v Transfers larger blocks of data in one I/O operation
These I/O blocks can be as large as 60 kilobytes. But the bytes must be adjacent
for the database server to transfer them in a single I/O operation.
v Bypasses the overhead of the buffer pool when many pages are read
v Prevents frequently accessed pages from being forced out of the buffer pool
when many sequential pages are read for smart large objects

When you use lightweight I/O buffers for smart large objects, the database server
might read several pages with one I/O operation. A single I/O operation reads in
several smart-large-object pages, up to the size of an extent. For information about
when to specify extent size, see “Sbspace extents” on page 5-21.

Chapter 5. Effect of configuration on I/O activity 5-23

 Specifying lightweight I/O for smart large objects:

To specify the use of lightweight I/O when creating the sbspace, use the
BUFFERING tag of the -Df option in the onspaces -c -S command.

The default value for BUFFERING is ON, which means to use the buffer pool. The
buffering mode that you specify (or the default, if you do not specify) in the
onspaces command is the default buffering mode for all smart large objects stored
within the sbspace.

Important: In general, if read and write operations to the smart large objects are
less than 8080 bytes, do not specify a buffering mode when you create the sbspace.
If you are reading or writing short blocks of data, such as 2 kilobytes or 4
kilobytes, leave the default of “buffering=ON” to obtain better performance.

Programmers can override the default buffering mode when they create, open, or
alter a smart-large-object instance with DataBlade API and the Informix ESQL/C
functions. The DataBlade API and the Informix ESQL/C application programming
interface provide the LO_NOBUFFER flag to allow lightweight I/O for smart large
objects.

Important: Use the LO_NOBUFFER flag only when you read or write smart large
objects in operations greater than 8080 bytes and you seldom access them. That is,
if you have infrequent read or write function calls that read large amounts of data
in a single function invocation, lightweight I/O can improve I/O performance.
Related reference:
onspaces -c -S: Create an sbspace (Administrator's Reference)
What is Informix ESQL/C? (ESQL/C Guide)
DataBlade API overview (DataBlade API Guide)

Logging
If you decide to log all write operations on data stored in sbspaces, logical-log I/O
activity and memory utilization increases.

For more information, see “Configuration parameters that affect sbspace I/O” on
page 5-20.

How the Optical Subsystem affects performance

The Optical Subsystem extends the storage capabilities of the database server for
simple large objects (TEXT or BYTE data) to write-once-read-many (WORM)
optical subsystems. The database server uses a cache in memory to buffer initial
TEXT or BYTE data pages requested from the Optical Subsystem.

The memory cache is a common storage area. The database server adds simple
large objects requested by any application to the memory cache if the cache has
space. To free space in the memory cache, the application must release the TEXT or
BYTE data that it is using.

A significant performance advantage occurs when you retrieve TEXT or BYTE data
directly into memory instead of buffering that data on disk. Therefore, proper
cache sizing is important when you use the Optical Subsystem. You specify the
total amount of space available in the memory cache with the OPCACHEMAX
configuration parameter. Applications indicate that they require access to a portion

5-24 IBM Informix Performance Guide

 of the memory cache when they set the INFORMIXOPCACHE environment variable. For
details, see “INFORMIXOPCACHE, an Optical Subsystem environment variable”
on page 5-26.

Simple large objects that cannot fit entirely into the space that remains in the cache
are stored in the blobspace that the STAGEBLOB configuration parameter names.
This staging area acts as a secondary cache on disk for blobpages that are retrieved
from the Optical Subsystem. Simple large objects that are retrieved from the
Optical Subsystem are held in the staging area until the transactions that requested
them are complete.

The database server administrator creates the staging-area blobspace with the
onspaces utility or with ON-Monitor (UNIX only.

You can use onstat -O to monitor utilization of the memory cache and
STAGEBLOB blobspace. If contention develops for the memory cache, increase the
value listed in the configuration file for OPCACHEMAX. (The new value takes
effect the next time that the database server starts shared memory.) For a complete
description of the Optical Subsystem, see the IBM Informix Optical Subsystem Guide.

Environment variables and configuration parameters for the Optical

Subsystem
The STAGEBLOB and OPCACHEMAX configuration parameters and the
INFORMIXOPCACHE environment variable affect the performance of the Optical
Subsystem.

STAGEBLOB, an Optical Subsystem configuration parameter

The STAGEBLOB configuration parameter identifies the blobspace that is to be
used as a staging area for TEXT or BYTE data that is retrieved from the Optical
Subsystem, and it activates the Optical Subsystem.

If the configuration file does not list the STAGEBLOB parameter, the Optical
Subsystem does not recognize the optical-storage subsystem.

The structure of the staging-area blobspace is the same as all other database server
blobspaces. When the database server administrator creates the staging area, it
consists of only one chunk, but you can add more chunks as desired. You cannot
mirror the staging-area blobspace. The optimal size for the staging-area blobspace
depends on the following factors:
v The frequency of simple-large-object storage
v The frequency of simple-large-object retrieval
v The average size of the simple large object to be stored

To calculate the size of the staging-area blobspace, you must estimate the number
of simple large objects that you expect to reside there simultaneously and multiply
that number by the average simple-large-object size.
Related reference:
STAGEBLOB configuration parameter (Administrator's Reference)

Chapter 5. Effect of configuration on I/O activity 5-25

 OPCACHEMAX, an Optical Subsystem configuration
parameter
The OPCACHEMAX configuration parameter specifies the total amount of space
that is available for simple-large-object retrieval in the memory cache that the
Optical Subsystem uses.

Until the memory cache fills, it stores simple large objects that are requested by
any application. Simple large objects that cannot fit in the cache are stored on disk
in the blobspace that the STAGEBLOB configuration parameter indicates. You can
increase the size of the cache to reduce contention among simple-large-object
requests and to improve performance for requests that involve the Optical
Subsystem.
Related reference:
OPCACHEMAX configuration parameter (UNIX) (Administrator's Reference)

INFORMIXOPCACHE, an Optical Subsystem environment

variable
The INFORMIXOPCACHE environment variable sets the size of the memory cache that
a given application uses for simple-large-object retrieval.

If the value of this variable exceeds the maximum that the OPCACHEMAX
configuration parameter specifies, OPCACHEMAX is used instead. If
INFORMIXOPCACHE is not set in the environment, the cache size is set to
OPCACHEMAX by default.
Related reference:
INFORMIXOPCACHE environment variable (SQL Reference)

Table I/O
One of the most frequent functions that the database server performs is to bring
data and index pages from disk into memory. Pages can be read individually for
brief transactions and sequentially for some queries. You can configure the number
of pages that the database server brings into memory, and you can configure the
timing of I/O requests for sequential scans.

You can also indicate how the database server is to respond when a query requests
data from a dbspace that is temporarily unavailable.

The following sections describe these methods of reading pages.

For information about I/O for smart large objects, see “Factors that affect I/O for
smart large objects” on page 5-20.

Sequential scans
When the database server performs a sequential scan of data or index pages, most
of the I/O wait time is caused by seeking the appropriate starting page. To
dramatically improve performance for sequential scans, you can bring in a number
of contiguous pages with each I/O operation.

The action of bringing additional pages along with the first page in a sequential
scan is called read ahead.

5-26 IBM Informix Performance Guide

 The timing of I/O operations that are needed for a sequential scan is also
important. If the scan thread must wait for the next set of pages to be brought in
after working its way through each batch, a delay occurs. Timing second and
subsequent read requests to bring in pages before they are needed provides the
greatest efficiency for sequential scans. The number of pages to bring in and the
frequency of read-ahead I/O requests depends on the availability of space in the
memory buffers. Read-ahead operations can increase page cleaning to unacceptable
levels if too many pages are brought in with each batch or if batches are brought
in too often.
Related reference:
Read-ahead operations (Administrator's Guide)

Light scans
Some sequential scans of tables can use light scans to read the data. A light scan
bypasses the buffer pool by utilizing session memory to read directly from disk.

Light scans can provide performance advantages over use of the buffer pool for
sequential scans and skip scans of large tables. These advantages include:
v Bypassing the overhead of the buffer pool when many data pages are read
v Preventing frequently accessed pages from being forced out of the buffer pool
when many sequential pages are read for a single query.

Light scans occur under these conditions:

v The optimizer chooses a sequential scan or a skip-scan of the table.
v The amount of data in the table exceeds one MiB.
v The query meets one of the following locking conditions:
– The isolation level is Dirty Read (or the database has no transaction logging).
– The table has at least a shared lock on the entire table and the isolation level
is not Cursor Stability.

Note: A sequential scan in Repeatable Read isolation automatically acquires a

share lock on the table.

Tables that cannot be accessed by light scans

Light scans are only performed on user tables whose data rows are stored in
tblspaces. Light scans are not used to access indexes, or to access data stored in
blobspaces, smart blob spaces, or partition blobs. Similarly, light scans are not used
to access data in the system catalog tables, nor in the tables and pseudotables of
system databases like sysadmin, sysmaster, sysuser, and sysutils.

Configuration settings that affect light scans

If the BATCHEDREAD_TABLE configuration parameter or the

IFX_BATCHEDREAD_TABLE session environment option to the SET
ENVIRONMENT statement is set to 0, light scans are not used to access tables that
have variable length rows, or tables where the row length is greater than the
pagesize of the dbspace in which the table is contained. A variable length row
includes tables that have a variable length column, such as VARCHAR,
LVARCHAR or NVARCHAR, as well as tables that are compressed.

You can use the IFX_BATCHEDREAD_TABLE session environment option of the

SET ENVIRONMENT statement, or the onmode -wm command, to override the

Chapter 5. Effect of configuration on I/O activity 5-27

 setting of the BATCHEDREAD_TABLE configuration parameter for the current
session. You can use the onmode -wf command to change the value of
BATCHEDREAD_TABLE in the ONCONFIG file.

Example of onstat output during a light scan

If you have a long-running scan, you can view output from the onstat -g scn
command to check the progress of the scan, to determine how long the scan will
take before it completes, and to see whether the scan is a light scan or a bufferpool
scan.

The following example shows some of the output from onstat -g scn for a light
scan. The word Light in the Scan Type field identifies the scan as a light scan.
SesID Thread Partnum Rowid Rows Scan’d Scan Type Lock Mode Notes
17 48 300002 207 15 Light Forward row lookup
Related reference:
BATCHEDREAD_TABLE configuration parameter (Administrator's Reference)
onstat -g scn command: Print scan information (Administrator's Reference)

Unavailable data
Another aspect of table I/O pertains to situations in which a query requests access
to a table or fragment in a dbspace that is temporarily unavailable. When the
database server determines that a dbspace is unavailable as the result of a disk
failure, queries directed to that dbspace fail by default. The database server allows
you to specify dbspaces that, when unavailable, can be skipped by queries,

For information about specifying dbspaces that, when unavailable, can be skipped
by queries, see “How DATASKIP affects table I/O” on page 5-29.

Warning: If a dbspace containing data that a query requests is listed in the

DATASKIP configuration parameter and is currently unavailable because of a disk
failure, the data that the database server returns to the query can be inconsistent
with the actual contents of the database.

Configuration parameters that affect table I/O

The AUTO_READAHEAD configuration parameter changes the automatic
read-ahead mode or disables automatic read-ahead for a query. In addition, the
DATASKIP configuration parameter enables or disables data skipping.

Automatic read-ahead processing helps improve query performance by issuing

asynchronous page requests when Informix detects that the query is encountering
I/O. Asynchronous page requests can improve query performance by overlapping
query processing with the processing necessary to retrieve data from disk and put
it in the buffer pool. You can also use the AUTO_READAHEAD environment
option of the SET ENVIRONMENT statement of SQL to enable or disable the value
of the AUTO_READAHEAD configuration parameter for a session.
Related reference:
AUTO_READAHEAD configuration parameter (Administrator's Reference)

5-28 IBM Informix Performance Guide

 How DATASKIP affects table I/O
The DATASKIP configuration parameter allows you to specify which dbspaces, if
any, queries can skip when those dbspaces are unavailable as the result of a disk
failure. You can list specific dbspaces and turn data skipping on or off for all
dbspaces.

When data skipping is enabled, the database server sets the sixth character in the
SQLWARN array to W..

Warning: The database server cannot determine whether the results of a query are
consistent when a dbspace is skipped. If the dbspace contains a table fragment, the
user who executes the query must ensure that the rows within that fragment are
not needed for an accurate query result. Turning DATASKIP on allows queries
with incomplete data to return results that can be inconsistent with the actual state
of the database. Without proper care, that data can yield incorrect or misleading
query results.
Related concepts:
SQLWARN array (SQL Tutorial)
Related reference:
DATASKIP Configuration Parameter (Administrator's Reference)

Background I/O activities

Background I/O activities do not service SQL requests directly. Many of these
activities are essential to maintain database consistency and other aspects of
database server operation. However, they create overhead in the CPU and take up
I/O bandwidth.

These overhead activities take time away from queries and transactions. If you do
not configure background I/O activities properly, too much overhead for these
activities can limit the transaction throughput of your application.

The following list shows some background I/O activities:

v Checkpoints
v Logging
v Page cleaning
v Backup and restore
v Rollback and recovery
v Data replication
v Auditing

Checkpoints occur regardless of whether much database activity occurs; however,

they can occur with greater frequency as activity increases. Other background
activities, such as logging and page cleaning, occur more frequently as database
use increases. Activities such as backups, restores, or fast recoveries occur only as
scheduled or under exceptional circumstances.

For the most part, tuning your background I/O activities involves striking a
balance between appropriate checkpoint intervals, logging modes and log sizes,
and page-cleaning thresholds. The thresholds and intervals that trigger background
I/O activity often interact; adjustments to one threshold might shift the
performance bottleneck to another.

Chapter 5. Effect of configuration on I/O activity 5-29

 The following sections describe the performance effects and considerations that are
associated with the configuration parameters that affect these background I/O
activities.

Configuration parameters that affect checkpoints

The RTO_SERVER_RESTART, CKPTINTVL, LOGSIZE, LOGFILES, PHYSFILE, and
ONDBSPACEDOWN configuration parameters affect checkpoints.

RTO_SERVER_RESTART and its effect on checkpoints

The RTO_SERVER_RESTART configuration parameter specifies the amount of time,
in seconds, that Informix has to recover from an unplanned outage.

The performance advantage of enabling this configuration parameter is:

v Enabling fast recovery to meet the RTO_SERVER_RESTART policy by seeding
the buffer pool with the data pages required by log replay.

The performance disadvantages of enabling this configuration parameter are:

v Increased physical log activity which might slightly impact transaction
performance
v Increased checkpoint frequency, because the physical log space is depleted more
quickly (You can increase the size of the physical log to avoid the increase in
checkpoint frequency.)

When RTO_SERVER_RESTART is enabled, the database server:

v Attempts to make sure nonblocking checkpoints do not run out of critical
resources during checkpoint processing by triggering more frequent checkpoints
if transactions might run out of physical or logical log resources, which would
cause transaction blocking.
v Ignores the CKPTINTVL configuration parameter.
v Automatically controls checkpoint frequency to meet the RTO policy and to
prevent the server from running out of log resources.
v Automatically adjusts the number of AIO virtual processors and cleaner threads
and automatically tunes LRU flushing.

The database server prints warning messages in the message log if the server
cannot meet the RTO_SERVER_RESTART policy.
Related reference:
RTO_SERVER_RESTART configuration parameter (Administrator's Reference)

Automatic checkpoints, LRU tuning, and AIO virtual processor tuning:

The database server automatically adjusts checkpoint frequency to avoid

transaction blocking. The server monitors physical and logical log consumption
along with information about past checkpoint performance. Then, if necessary, the
server triggers checkpoints more frequently to avoid transaction blocking.

You can turn off automatic checkpoint tuning by setting onmode -wf
AUTO_CKPTS to 0, or setting the AUTO_CKPTS configuration parameter to 0.

Because the database server does not block transactions during checkpoint
processing, LRU flushing is relaxed. If the server is not able to complete checkpoint
processing before the physical log is full (which causes transaction blocking), and if
you cannot increase the size of the physical log, you can configure the server for

5-30 IBM Informix Performance Guide

more aggressive LRU flushing. The increase in LRU flushing impacts transaction
performance, but reduces transaction blocking. If you do not configure the server
for more aggressive flushing, the server automatically adjusts LRU flushing to be
more aggressive only when the server is unable to find a low priority buffer for
page replacement.

When the AUTO_AIOVPS configuration parameter is enabled, the database server

automatically increases the number of AIO virtual processors and page-cleaner
threads when the server detects that AIO virtual processors are not keeping up
with the I/O workload.

Automatic LRU tuning affects all buffer pools and adjusts lru_min_dirty and
lru_max_dirty values in the BUFFERPOOL configuration parameter.
Related concepts:
“LRU tuning” on page 5-45
Related reference:
AUTO_CKPTS configuration parameter (Administrator's Reference)
AUTO_AIOVPS configuration parameter (Administrator's Reference)
BUFFERPOOL configuration parameter (Administrator's Reference)

CKPTINTVL and its effect on checkpoints

If the RTO_SERVER_RESTART configuration parameter is not on, the CKPTINTVL
configuration parameter specifies the frequency, in seconds, at which the database
server checks to determine whether a checkpoint is needed.

When the RTO_SERVER_RESTART configuration parameter is on, the database

server ignores the CKPTINTVL configuration parameter. Instead, the server
automatically triggers checkpoints in order to maintain the
RTO_SERVER_RESTART policy.

The database server can skip a checkpoint if all data is physically consistent when
the checkpoint interval expires.

Checkpoints also occur in either of these circumstances:

v Whenever the physical log becomes 75 percent full
v If a high number of dirty partitions exist, even if the physical log is not 75
percent full.
This occurs because when the database server checks if the physical log is 75
percent full, the server also checks if the following condition is true:
(Physical Log Pages Used + Number of Dirty Partitions) >=
(Physical Log Size * 9) /10)
A partition, which represents one page going into the physical log during
checkpoint processing and has a page that maintains information (such as the
number of rows and number of data pages) about the partition, becomes dirty
when the partition is updated.

If you set CKPTINTVL to a long interval, you can use physical-log capacity to
trigger checkpoints based on actual database activity instead of an arbitrary time
unit. However, a long checkpoint interval can increase the time needed for
recovery in the event of a failure. Depending on your throughput and
data-availability requirements, you can choose an initial checkpoint interval of 5,
10, or 15 minutes, with the understanding that checkpoints might occur more
often, depending on physical-logging activity.
Chapter 5. Effect of configuration on I/O activity 5-31
 The database server writes a message to the message log to note the time that it
completes a checkpoint. To read these messages, use onstat -m.
Related reference:
CKPTINTVL configuration parameter (Administrator's Reference)

LOGSIZE and LOGFILES and their effect on checkpoints

The LOGSIZE and LOGFILES configuration parameters indirectly affect
checkpoints because they specify the size and number of logical-log files. A
checkpoint can occur when the database server detects that the next logical-log file
to become current contains the most-recent checkpoint record.

If you need to free the logical-log file that contains the last checkpoint, the
database server must write a new checkpoint record to the current logical-log file.
If the frequency with which logical-log files are backed up and freed increases, the
frequency at which checkpoints occur increases. Although checkpoints block user
processing, they no longer last as long. Because other factors (such as the
physical-log size) also determine the checkpoint frequency, this effect might not be
significant.

When the dynamic log allocation feature is enabled, the size of the logical log does
not affect the thresholds for long transactions as much as it did in previous
versions of the database server. For details, see “LTXHWM and LTXEHWM and
their effect on logging” on page 5-38.

The LOGSIZE, LOGFILES, and LOGBUFF configuration parameters also affect

logging I/O activity and logical backups. For more information, see “Configuration
parameters that affect logging” on page 5-34.
Related concepts:
Estimate the number of logical-log files (Administrator's Guide)
Related reference:
LOGFILES configuration parameter (Administrator's Reference)
LOGSIZE configuration parameter (Administrator's Reference)

Checkpoints and the physical log

The PHYSFILE configuration parameter specifies the size of the initial physical log.
A checkpoint occurs when either the physical log becomes 75 percent full or a high
number of dirty partitions exist.

The rate at which transactions generate physical log activity can affect checkpoint
performance. To avoid transaction blocking during checkpoint processing, consider
the size of the physical log and how quickly it fills.

For example, operations that do not perform updates do not generate

before-images. If the size of the database is growing, but applications rarely update
the data, little physical logging occurs. In this situation, you might not need a large
physical log.

Similarly, you can define a smaller physical log if your application updates the
same pages. The database server writes the before-image of only the first update
that is made to a page for the following operations:
v Inserts, updates, and deletes for rows that contain user-defined data types
(UDTs), smart large objects, and simple large objects
v ALTER statements
5-32 IBM Informix Performance Guide
v Operations that create or modify indexes (B-tree, R-tree, or user-defined indexes)

Because the physical log is recycled after each checkpoint, the physical log must be
large enough to hold before-images from changes between checkpoints. If the
database server frequently triggers checkpoints because it runs out of physical log
space, consider increasing the size of the physical log.

If you increase the checkpoint interval or if you anticipate increased update

activity, you might want to increase the size of the physical log.

The physical log is an important part of maintaining RTO_SERVER_RESTART

policy. To ensure that you have an abundance of space, set the size of the physical
log to at least 110 percent of the size of all buffer pools.

You can use the onparams utility to change the physical log location and size. You
can change the physical log while transactions are active and without restarting the
database server.
Related concepts:
Strategy for estimating the size of the physical log (Administrator's Guide)
Related reference:
PHYSFILE configuration parameter (Administrator's Reference)
Change the physical-log location and size (Administrator's Guide)

ONDBSPACEDOWN and its effect on checkpoints

The ONDBSPACEDOWN configuration parameter specifies the response that the
database server makes when an I/O error indicates that a dbspace is down. By
default, the database server identifies any dbspace that contains no critical data as
down and continues processing. Critical data includes the root dbspace, the logical
log, or the physical log.

To restore access to that database, you must back up all logical logs and then
perform a warm restore on the down dbspace.

The database server halts operation whenever a disabling I/O error occurs on a
nonmirrored dbspace that contains critical data, regardless of the setting for
ONDBSPACEDOWN. In such an event, you must perform a cold restore of the
database server to resume normal database operations.

The value of ONDBSPACEDOWN has no affect on temporary dbspaces. For

temporary dbspaces, the database server continues processing regardless of the
ONDBSPACEDOWN setting. If a temporary dbspace requires fixing, you can drop
and recreate it.

When ONDBSPACEDOWN is set to 2, the database server continues processing to

the next checkpoint and then suspends processing of all update requests. The
database server repeatedly retries the I/O request that produced the error until the
dbspace is repaired and the request completes or the database server administrator
intervenes. The administrator can use onmode -O to mark the dbspace down and
continue processing while the dbspace remains unavailable or use onmode -k to
halt the database server.

Chapter 5. Effect of configuration on I/O activity 5-33

 Important: This 2 setting for ONDBSPACEDOWN can affect the performance for
update requests severely because they are suspended due to a down dbspace.
When you use this setting for ONDBSPACEDOWN, be sure to monitor the status
of the dbspaces.

When you set ONDBSPACEDOWN to 1, the database server treats all dbspaces as
though they were critical. Any nonmirrored dbspace that becomes disabled halts
normal processing and requires a cold restore. The performance impact of halting
and performing a cold restore when any dbspace goes down can be severe.

Important: If you decide to set ONDBSPACEDOWN to 1, consider mirroring all

your dbspaces.
Related reference:
ONDBSPACEDOWN configuration parameter (Administrator's Reference)

Configuration parameters that affect logging

The LOGBUFF, PHYSBUFF, LOGFILES, LOGSIZE, DYNAMIC_LOGS, LTXHWM,
LTXEHWM and TEMPTAB_NOLOG configuration parameters affect logging.

Logging, checkpoints, and page cleaning are necessary to maintain database

consistency. A direct trade-off exists between the frequency of checkpoints or the
size of the logical logs and the time that it takes to recover the database in the
event of a failure. Therefore, a major consideration when you attempt to reduce the
overhead for these activities is the delay that you can accept during recovery.

LOGBUFF and PHYSBUFF and their effect on logging

The LOGBUFF and PHYSBUFF configuration parameters affect logging I/O
activity because they specify the respective sizes of the logical-log and physical-log
buffers that are in shared memory. The size of these buffers determines how
quickly the buffers fill and therefore how often they need to be flushed to disk.
Related reference:
LOGBUFF configuration parameter (Administrator's Reference)
PHYSBUFF configuration parameter (Administrator's Reference)

LOGFILES and its effect on logging

The LOGFILES configuration parameter, which specifies the number of logical-log
files, affects logging.

When you initialize or restart the database server, it creates the number of
logical-log files that you specify in the LOGFILES configuration parameter.

You might add logical-log files for the following reasons:

v To increase the disk space allocated to the logical log
v To change the size of your logical-log files
v To enable an open transaction to roll back
v As part of moving logical-log files to a different dbspace
Related concepts:
Estimate the number of logical-log files (Administrator's Guide)
Related reference:
LOGFILES configuration parameter (Administrator's Reference)

5-34 IBM Informix Performance Guide

Calculating the space allocated to logical log files:

If all of your logical log files are the same size, you can calculate the total space
allocated to the logical log files.

To calculate the space allocated to these files, use the following formula:
total logical log space = LOGFILES * LOGSIZE

If you add logical-log files that are not the size specified by the LOGSIZE
configuration parameter, you cannot use the LOGFILES * LOGSIZE expression to
calculate the size of the logical log. Instead, you need to add the sizes for each
individual log file on disk.

Use the onstat -l utility to monitor logical-log files.

LOGSIZE and its effect on logging

The LOGSIZE configuration parameter specifies the size of each logical log file. It
is difficult to predict how much logical-log space your database server system
requires until the system is fully in use.

The size of the logical log space (LOGFILES * LOGSIZE) is determined by these
policies:
Recovery time objective (RTO)
This is the length of time you can afford to be without your systems. If
your only objective is failure recovery, the total log space only needs to be
large enough to contain all the transactions for two checkpoint cycles.
When the RTO_SERVER_RESTART configuration parameter is enabled and
the server has a combined buffer pool size of less that four gigabytes, you
can configure the total log space to 110% of the combined buffer pool sizes.
Too much log space does not impact performance; however, too little log
space can cause more frequent checkpoints and transaction blocking.
Recovery point objective (RPO)
This describes the age of the data you want to restore in the event of a
disaster. If the objective is to make sure transactional work is protected, the
optimum LOGSIZE should be a multiple of how much work gets done per
RPO unit. Because the database server supports partial log backup, an
optimal log size is not critical and a non-optimal log size simply means
more frequent log file changes. RPO is measured in units of time. If the
business rule is that the system cannot lose more than ten minutes of
transactional data if a complete site disaster occurs, then a log backup
should occur every ten minutes.
You can use the Scheduler, which manages and executes scheduled
administrative tasks, to set up automatic log backup.
Long Transactions
If you have long transactions that require a large amount of log space, you
should allocate that space for the logs. Inadequate log space impacts
transaction performance.

Choose a log size based on how much logging activity occurs and the amount of
risk in case of catastrophic failure. If you cannot afford to lose more than an hour's
worth of data, create many small log files that each hold an hour's worth of
transactions. Turn on continuous-log backup. Small logical-log files fill sooner,
which means more frequent logical-log backups.

Chapter 5. Effect of configuration on I/O activity 5-35

 If your system is stable with high logging activity, choose larger logs to improve
performance. Continuous-log backups occur less frequently with large log files.
Also consider the maximum transaction rates and speed of the backup devices. Do
not let the whole logical log fill. Turn on continuous-log backup and leave enough
room in the logical logs to handle the longest transactions.

The backup process can hinder transaction processing that involves data located on
the same disk as the logical-log files. If enough logical-log disk space is available,
however, you can wait for periods of low user activity before you back up the
logical-log files.
Related concepts:
The Scheduler (Administrator's Guide)
Related reference:
LOGSIZE configuration parameter (Administrator's Reference)

Estimating logical-log size when logging dbspaces:

To estimate the size of logical logs, use a formula or onstat -u information.

Use the following formula to obtain an initial estimate for LOGSIZE in kilobytes:
LOGSIZE = (connections * maxrows * rowsize) / 1024) / LOGFILES

In this formula:
v connections is the maximum number of connections for all network types
specified in the sqlhosts information by one or more NETTYPE parameters. If
you configured more than one connection by setting multiple NETTYPE
configuration parameters in your configuration file, sum the users fields for each
NETTYPE parameter, and substitute this total for connections in the preceding
formula.
v maxrows is the largest number of rows to be updated in a single transaction.
v rowsize is the average size of a row in bytes. You can calculate rowsize by adding
up the length (from the syscolumns system catalog table) of the columns in a
row.
v 1024 is a necessary divisor because you specify LOGSIZE in kilobytes.

To obtain a better estimate during peak activity periods, execute the onstat -u
command. The last line of the onstat -u output contains the maximum number of
concurrent connections.

You need to adjust the size of the logical log when your transactions include
simple large objects or smart large objects, as the following sections describe.

You also can increase the amount of space devoted to the logical log by adding
another logical-log file.
Related tasks:
Adding logical-log files manually (Administrator's Guide)

Estimating the logical-log size when logging simple large objects:

To obtain better overall performance for applications that perform frequent updates
of TEXT or BYTE data in blobspaces, reduce the size of the logical log.

5-36 IBM Informix Performance Guide

Blobpages cannot be reused until the logical log to which they are allocated is
backed up. When TEXT or BYTE data activity is high, the performance impact of
more frequent checkpoints is balanced by the higher availability of free blobpages.

When you use volatile blobpages in blobspaces, smaller logs can improve access to
simple large objects that must be reused. Simple large objects cannot be reused
until the log in which they are allocated is flushed to disk. In this case, you can
justify the cost in performance because those smaller log files are backed up more
frequently.

Estimating the logical-log size when logging smart large objects:

If you plan to log smart-large-object user data, you must ensure that the log size is
considerably larger than the amount of data being written. Smart-large-object
metadata is always logged even if the smart large objects are not logged.

Use the following guidelines when you log smart large objects:
v If you are appending data to a smart large object, the increased logging activity
is roughly equal to the amount of data written to the smart large object.
v If you are updating a smart large object (overwriting data), the increased logging
activity is roughly twice the amount of data written to the smart large object.
The database server logs both the before-image and after-image of a smart large
object for update transactions. When updating the smart large objects, the
database server logs only the updated parts of the before and after image.
v Metadata updates affect logging less. Even though metadata is always logged,
the number of bytes logged is usually much smaller than the smart large objects.

DYNAMIC_LOGS and its effect on logging

The dynamic log file allocation feature prevents hanging problems that are caused
by rollbacks of a long transaction because the database server does not run out of
log space. The DYNAMIC_LOGS configuration parameter specifies whether the
dynamic log file allocation feature is off, on, or causes the server to pause to allow
the manual addition of a logical log file.

Dynamic log allocation allows you to do the following actions:

v Add a logical log file while the system is active, even during fast recover.
v Insert a logical log file immediately after the current log file, instead of
appending it to the end.
v Immediately access the logical log file even if the root dbspace is not backed up.

The default value for the DYNAMIC_LOGS configuration parameter is 2, which

means that the database server automatically allocates a new logical log file after
the current log file when it detects that the next log file contains an open
transaction. The database server automatically checks if the log after the current
log still contains an open transaction at the following times:
v Immediately after it switches to a new log file while writing log records (not
while reading and applying log records)
v At the beginning of the transaction cleanup phase which occurs as the last phase
of logical recovery
Logical recovery happens at the end of fast recovery and at the end of a cold
restore or roll forward.
v During transaction cleanup (rollback of open transactions), a switch to a new log
file log might occur

Chapter 5. Effect of configuration on I/O activity 5-37

 The database server also checks after this switch because it is writing log records
for the rollback.

When you use the default value of 2 for DYNAMIC_LOGS, the database server
determines the location and size of the new logical log for you:
v The database server uses the following criteria to determine on which disk to
allocate the new log file:
– Favor mirrored dbspaces
– Avoid root dbspace until no other critical dbspace is available
– Least favored space is unmirrored and noncritical dbspaces
v The database server uses the average size of the largest log file and the smallest
log file for the size of the new logical log file. If not enough contiguous disk
space is available for this average size, the database server searches for space for
the next smallest average size. The database server allocates a minimum of 200
kilobytes for the new log file.

If you want to control the location and size of the additional log file, set
DYNAMIC_LOGS to 1. When the database server switches log files, it still checks
if the next active log contains an open transaction. If it does find an open
transaction in the next log to be active, it does the following actions:
v Issues alarm event 27 (log required)
v Writes a warning message to the online log
v Pauses to wait for the administrator to manually add a log with the onparams -a
-i command-line option

You can write a script that will execute when alarm event 27 occurs to execute
onparams -a -i with the location you want to use for the new log. Your script can
also execute the onstat -d command to check for adequate space and execute the
onparams -a -i command with the location that has enough space. You must use
the -i option to add the new log right after the current log file.

If you set DYNAMIC_LOGS to 0, the database server still checks whether the next
active log contains an open transaction when it switches log files. If it does find an
open transaction in the next log to be active, it issues the following warning:
WARNING: The oldest logical log file (%d) contains records
from an open transaction (0x%p), but the Dynamic Log
Files feature is turned off.
Related concepts:
Fast recovery (Administrator's Guide)
Related reference:
DYNAMIC_LOGS configuration parameter (Administrator's Reference)

LTXHWM and LTXEHWM and their effect on logging

With the dynamic log file feature, the long transaction high watermarks are no
longer as critical as in database server versions prior to Version 9.3, because the
server does not run out of log space unless you use up the physical disk space
available on the system. The LTXHWM and LTXEHWM configuration parameters
define long transaction watermarks.

The LTXHWM parameter still indicates how full the logical log is when the
database server starts to check for a possible long transaction and to roll it back.
LTXEHWM still indicates the point at which the database server suspends new

5-38 IBM Informix Performance Guide

 transaction activity to locate and roll back a long transaction. These events should
be rare, but if they occur, they can indicate a serious problem within an
application.

Under normal operations, use the default values for LTXHWM and LTXEHWM.
However, you might want to change these default values for one of the following
reasons:
v To allow other transactions to continue update activity (which requires access to
the log) during the rollback of a long transaction
In this case, you increase the value of LTXEHWM to raise the point at which the
long transaction rollback has exclusive access to the log.
v To perform scheduled transactions of unknown length, such as large loads that
are logged
In this case, you increase the value of LTXHWM so that the transaction has a
chance to complete before reaching the high watermark.
Related reference:
LTXEHWM configuration parameter (Administrator's Reference)
LTXHWM configuration parameter (Administrator's Reference)

TEMPTAB_NOLOG and its effect on logging

The TEMPTAB_NOLOG configuration parameter allows you to disable logging on
temporary tables. You can do this to improve performance and to prevent Informix
from transferring temporary tables when using High-Availability Data Replication
(HDR).

To disable logging on temporary tables, set the TEMPTAB_NOLOG configuration

parameter to 1.
Related reference:
TEMPTAB_NOLOG configuration parameter (Administrator's Reference)

Configuration parameters that affect page cleaning

Several configuration parameters, including the CLEANERS and
RTO_SERVER_RESTART configuration parameters, affect page cleaning. If pages
are not cleaned often enough, an sqlexec thread that performs a query might be
unable to find the available pages that it needs.

If the sqlexec thread cannot find the available pages that it needs, the thread
initiates a foreground write and waits for pages to be freed. Foreground writes
impair performance, so you should avoid them. To reduce the frequency of
foreground writes, increase the number of page cleaners or decrease the threshold
for triggering a page cleaning.

Use onstat -F to monitor the frequency of foreground writes.

The following configuration parameters affect page cleaning:

v BUFFERPOOL, which contains lrus, lru_max_dirty, and lru_min_dirty values
Information that was specified with the BUFFERS, LRUS, LRU_MAX_DIRTY,
and LRU_MIN_DIRTY configuration parameters before Version 10.0 is now
specified using the BUFFERPOOL configuration parameter.
v CLEANERS
v RTO_SERVER_RESTART

Chapter 5. Effect of configuration on I/O activity 5-39

 CLEANERS and its effect on page cleaning
The CLEANERS configuration parameter indicates the number of page-cleaner
threads to run. For installations that support fewer than 20 disks, one page-cleaner
thread is recommended for each disk that contains database server data. For
installations that support between 20 and 100 disks, one page-cleaner thread is
recommended for every two disks. For larger installations, one page-cleaner thread
is recommended for every four disks.

If you increase the number of LRU queues, you must increase the number of
page-cleaner threads proportionally.
Related reference:
CLEANERS configuration parameter (Administrator's Reference)

BUFFERPOOL and its effect on page cleaning

The BUFFERPOOL configuration parameter specifies the number of least recently
used (LRU) queues to set up within the shared-memory buffer pool. The buffer
pool is distributed among LRU queues. Configuring more LRU queues allows
more page cleaners to operate and reduces the size of each LRU queue.

For a single-processor system, set the lrus field of the BUFFERPOOL configuration
parameter to a minimum of 4. For multiprocessor systems, set the lrus field to a
minimum of 4 or to the number of CPU VPs, whichever is greater.

The lrus, lru_max_dirty, and lru_min_dirty values control how often pages are
flushed to disk between checkpoints. Automatic LRU tuning, as set by the
AUTO_LRU configuration parameter, affects all buffer pools and adjusts the
lru_min_dirty and lru_max_dirty values in the BUFFERPOOL configuration
parameter.

If you increase the lru_max_dirty and lru_min_dirty values to improve transaction

throughput, do not change the gap between the lru_max_dirty and lru_min_dirty.

When the buffer pool is very large and transaction blocking is occurring during
checkpoint processing, look in the message log to determine which resource is
triggering transaction blocking. If the physical or logical log is critically low and
triggers transaction blocking, increase the size of the resource that is causing the
transaction blocking. If you cannot increase the size of the resource, consider
making LRU flushing more aggressive by decreasing the lru_min_dirty and
lru_max_dirty settings so that the server has fewer pages to flush to disk during
checkpoint processing.

To monitor the percentage of dirty pages in LRU queues, use the onstat -R
command. When the number of dirty pages consistently exceeds the lru_max_dirty
limit, you have too few LRU queues or too few page cleaners. First, use the
BUFFERPOOL configuration parameter to increase the number of LRU queues. If
the percentage of dirty pages still exceeds the lru_max_dirty limit, update the
CLEANERS configuration parameter to increase the number of page cleaners.
Related concepts:
“The BUFFERPOOL configuration parameter and memory utilization” on page
4-10
Number of LRU queues to configure (Administrator's Guide)
Related reference:
BUFFERPOOL configuration parameter (Administrator's Reference)

5-40 IBM Informix Performance Guide

 RTO_SERVER_RESTART and its effect on page cleaning
The RTO_SERVER_RESTART configuration parameter allows you to use recovery
time objective (RTO) standards to set the amount of time, in seconds, that Informix
has to recover from a problem after you restart Informix and bring it into online or
quiescent mode.

When this configuration parameter is enabled, the database server automatically

adjusts the number of AIO virtual processors and cleaner threads and
automatically tunes LRU flushing.

Use the AUTO_LRU_TUNING configuration parameter to specify whether

automatic LRU tuning is enabled or disabled when the server starts.
Related reference:
RTO_SERVER_RESTART configuration parameter (Administrator's Reference)
AUTO_LRU_TUNING configuration parameter (Administrator's Reference)

Configuration parameters that affect backup and restore

Four configuration parameters that affect backup and restore on all operating
systems also affect background I/O. Additional configuration parameters affect
backup and restore on UNIX.

The following configuration parameters affect backup and restore on all operating
systems:
v BAR_MAX_BACKUP
v BAR_NB_XPORT_COUNT
v BAR_PROGRESS_FREQ
v BAR_XFER_BUF_SIZE

In addition, the following configuration parameters affect backup and restore on

UNIX:
v LTAPEBLK
v LTAPEDEV
v LTAPESIZE
v TAPEBLK
v TAPEDEV
v TAPESIZE

ON-Bar configuration parameters

BAR_MAX_BACKUP, BAR_NB_XPORT_COUNT, BAR_PROGRESS_FREQ, and
BAR_XFER_BUF_SIZE are some ON-Bar configuration parameters that affect
background I/O.

The BAR_MAX_BACKUP configuration parameter specifies the maximum number

of backup processes per ON-Bar command. This configuration parameter also
defines the degree of parallelism, determining how many processes start to run
concurrently, including processes for backing up and restoring a whole system.
When the number of running processes is reached, further processes start only
when a running process completes its operation.

BAR_NB_XPORT_COUNT specifies the number of shared-memory data buffers for

each backup or restore process.

Chapter 5. Effect of configuration on I/O activity 5-41

 BAR_PROGRESS_FREQ specifies, in minutes, how frequently the backup or restore
progress messages display in the activity log.

BAR_XFER_BUF_SIZE specifies the size, in pages, of the buffers.

Related reference:
BAR_MAX_BACKUP configuration parameter (Backup and Restore Guide)
BAR_NB_XPORT_COUNT configuration parameter (Backup and Restore
Guide)
BAR_PROGRESS_FREQ configuration parameter (Backup and Restore Guide)
BAR_XFER_BUF_SIZE configuration parameter (Backup and Restore Guide)

ontape configuration parameters (UNIX)

On UNIX, LTAPEBLK, LTAPEDEV, LTAPESIZE, TAPEBLK, TAPEDEV, and
TAPESIZE are configuration parameters that affect the ontape utility.

On UNIX, the LTAPEBLK, LTAPEDEV, and TAPESIZE configuration parameters

specify the block size, device, and tape size for logical-log backups made with
ontape. The TAPEBLK configuration parameter specifies the block size for database
backups made with ontape, onload, and onunload.

TAPEDEV specifies the tape device. TAPESIZE specifies the tape size for these
backups.
Related reference:
ON-Bar and ontape configuration parameters and environment variable
(Backup and Restore Guide)

Configuration parameters that affect rollback and recovery

The OFF_RECVRY_THREADS, ON_RECVRY_THREADS,
PLOG_OVERFLOW_PATH, and RTO_SERVER_RESTART configuration parameters
affect recovery. The LOW_MEMORY_RESERVE configuration parameter reserves a
specific amount of memory, in kilobytes, for the database server to use when
critical activities, such as rollback activities, are needed.

OFF_RECVRY_THREADS and ON_RECVRY_THREADS and their

effect on fast recovery
The OFF_RECVRY_THREADS and ON_RECVRY_THREADS configuration
parameters specify the number of recovery threads that operate when the database
server performs a cold restore, warm restore, or fast recovery. The setting of
ON_RECVRY_THREADS controls warm restores, and the setting of
OFF_RECVRY_THREADS controls cold restores and fast recovery.

To improve the performance of fast recovery, increase the number of fast-recovery

threads with the OFF_RECVRY_THREADS configuration parameter. The number
of threads should usually match the number of tables or fragments that are
frequently updated to roll forward the transactions recorded in the logical log.

Another estimate is the number of tables or fragments that experience frequent

updates. On a single-CPU host, the number of threads should be no fewer than 10
and no more than 30 or 40. At a certain point, the overhead that is associated with
each thread outweighs the advantages of parallel threads.

5-42 IBM Informix Performance Guide

 A warm restore takes place concurrently with other database operations. To reduce
the impact of the warm restore on other users, you can allocate fewer threads to it
than you would allocate to a cold restore. However, to replay logical-log
transactions in parallel during a warm restore, specify more threads in the
ON_RECVRY_THREADS configuration parameter.
Related reference:
OFF_RECVRY_THREADS configuration parameter (Administrator's Reference)

ON_RECVRY_THREADS configuration parameter (Administrator's Reference)

PLOG_OVERFLOW_PATH and its effect on fast recovery

The PLOG_OVERFLOW_PATH configuration parameter specifies the location of a
disk file (named plog_extend.servernum) that the database server uses if the
physical log file overflows during fast recovery.

The database server removes the plog_extend.servernum file when the first
checkpoint is performed during a fast recovery.
Related reference:
PLOG_OVERFLOW_PATH configuration parameter (Administrator's Reference)

RTO_SERVER_RESTART and its effect on fast recovery

The RTO_SERVER_RESTART configuration parameter enables you to use recovery
time objective (RTO) standards to set the amount of time, in seconds, that Informix
has to recover from a problem after you restart Informix and bring it into online or
quiescent mode.
Related reference:
RTO_SERVER_RESTART configuration parameter (Administrator's Reference)

The LOW_MEMORY_RESERVE configuration parameter and

memory utilization
The LOW_MEMORY_RESERVE configuration parameter reserves a specific amount
of memory, in kilobytes, for the database server to use when critical activities are
needed and the server has limited free memory.

If you enable the new LOW_MEMORY_RESERVE configuration parameter by

setting it to a specified value in kilobytes, critical activities, such as rollback
activities, can complete even when you receive out-of-memory errors.
Related reference:
LOW_MEMORY_RESERVE configuration parameter (Administrator's
Reference)
onstat -g seg command: Print shared memory segment statistics
(Administrator's Reference)

Configuration parameters that affect data replication and

auditing
Data replication and auditing are optional. If you use these features, you can set
configuration parameters that affect data-replication performance and auditing
performance.

Chapter 5. Effect of configuration on I/O activity 5-43

 To obtain immediate performance improvements, you can disable these features,
provided that the operating requirements for your system allow you to do so.

Configuration parameters that affect data replication

Synchronized data replication can increase the amount of time it take longer to free
the log buffer after a log flush. The DRINTERVAL, DRTIMEOUT, and
HDR_TXN_SCOPE configuration parameters can adjust synchronization and
system performance.

The DRINTERVAL configuration parameter indicates whether the data-replication

buffer is flushed synchronously or asynchronously to the secondary database
server. If this parameter is set to flush asynchronously, it specifies the interval
between flushes. Each flush impacts the CPU and sends data across the network to
the secondary database server.

If the DRINTERVAL configuration parameter is set to 0, the synchronization mode

that is specified by the HDR_TXN_SCOPE configuration parameter is used. The
HDR_TXN_SCOPE configuration parameter specifies whether HDR replication is
fully synchronous, nearly synchronous, or asynchronous.
v In fully synchronous mode, transactions require acknowledgement of completion
on the HDR secondary server before they can complete.
v In asynchronous mode, transactions do not require acknowledgement of being
received or completed on the HDR secondary server before they can complete.
v In nearly synchronous mode, transactions require acknowledgement of being
received on the HDR secondary server before they can complete.

The DRTIMEOUT configuration parameter specifies the interval for which either
database server waits for a transfer acknowledgment from the other. If the primary
database server does not receive the expected acknowledgment, it adds the
transaction information to the file named in the DRLOSTFOUND configuration
parameter. If the secondary database server receives no acknowledgment, it
changes the data-replication mode as the DRAUTO configuration parameter
specifies.
Related concepts:
Replication of primary-server data to secondary servers (Administrator's
Guide)
Fully synchronous mode for HDR replication (Administrator's Guide)
Nearly synchronous mode for HDR replication (Administrator's Guide)
Asynchronous mode for HDR replication (Administrator's Guide)
Related reference:
DRINTERVAL configuration parameter (Administrator's Reference)
DRTIMEOUT configuration parameter (Administrator's Reference)
DRLOSTFOUND configuration parameter (Administrator's Reference)
DRAUTO configuration parameter (Administrator's Reference)
HDR_TXN_SCOPE configuration parameter (Administrator's Reference)
onstat -g dri command: Print high-availability data replication information
(Administrator's Reference)

5-44 IBM Informix Performance Guide

 Configuration parameters that affect auditing
The ADTERR and ADTMODE configuration parameters affect auditing
performance.

The ADTERR configuration parameter specifies whether the database server is to

halt processing for a user session for which an audit record encounters an error.
When ADTERR is set to halt such a session, the response time for that session
appears to degrade until one of the successive attempts to write the audit record
succeeds.

The ADTMODE configuration parameter enables or disables auditing according to

the audit records that you specify with the onaudit utility. Records are written to
files in the directory that the AUDITPATH parameter specifies. The AUDITSIZE
parameter specifies the size of each audit-record file.

The effect of auditing on performance is largely determined by the auditing events

that you choose to record. Depending on which users and events are audited, the
impact of these configuration parameters can vary widely.

Infrequent events, such as requests to connect to a database, have low performance

impact. Frequent events, such as requests to read any row, can generate a large
amount of auditing activity. The more users for whom such frequent events are
audited, the greater the impact on performance.
Related reference:
ADTERR configuration parameter (Security Guide)
ADTMODE configuration parameter (Security Guide)
Related information:
Auditing data security (Security Guide)

LRU tuning
The LRU settings for flushing each buffer pool between checkpoints are not critical
to checkpoint performance. The LRU settings are necessary only for maintaining
enough clean pages for page replacement.

The default settings for LRU flushing are 50 percent for lru_min_dirty and 60
percent for lru_max_dirty.

If your database server has been configured for more aggressive LRU flushing
because of checkpoint performance, you can decrease the LRU flushing at least to
the default values.

The database server automatically tunes LRU flushing when the

AUTO_LRU_TUNING configuration parameter is on and in the following cases:
v A page replacement is forced to perform a foreground write in order to find an
empty page. In this case, LRU flushing is adjusted to be 5 percent more
aggressive for the specific bufferpool where the foreground write took place.
v A page replacement is forced to use a buffer that is marked as high priority,
meaning it is frequently accessed. In this case, LRU flushing is adjusted to be
one (1) percent more aggressive for the specific bufferpool where the page
replacement using high priority buffer took place.

Chapter 5. Effect of configuration on I/O activity 5-45

 v If the RTO_SERVER_RESTART configuration parameter is on and the time it
takes to flush the bufferpool is longer than the recovery time objective, LRU
flushing is adjusted to be 10 percent more aggressive for all bufferpools.

After a checkpoint has occurred, if a page replacement performed a foreground

write during the previous checkpoint interval, the database server increases the
LRU settings by 5 percent and continues to increase the LRU flushing at each
subsequent checkpoint until the foreground write stops or until the lru_max_dirty
for a given buffer pool falls below 10 percent. For example, if a page replacement
performs a foreground write and the LRU settings for a buffer pool are 80 and 90,
the database server adjusts these to 76 and 85.5.

In addition to foreground writes, LRU flushing is tuned more aggressively

whenever a page fault replaces high priority buffers and non-high priority buffers
are on the modified LRU queue. Automatic LRU adjustments only make LRU
flushing more aggressive; they do not decrease LRU flushing. Automatic LRU
adjustments are not permanent and are not recorded in the ONCONFIG file.

LRU flushing is reset to the values contained in the ONCONFIG file on which the
database server starts.

The AUTO_LRU_TUNING configuration parameter specifies whether automatic

LRU tuning is enabled or disabled when the server starts.
Related concepts:
“Automatic checkpoints, LRU tuning, and AIO virtual processor tuning” on page
5-30
Related reference:
AUTO_LRU_TUNING configuration parameter (Administrator's Reference)
RTO_SERVER_RESTART configuration parameter (Administrator's Reference)

5-46 IBM Informix Performance Guide

Chapter 6. Table performance considerations
Some performance issues are associated with unfragmented tables and table
fragments.

Issues include:
v Table placement on disk to increase throughput and reduce contention
v Space estimates for tables, blobpages, sbspaces, and extents
v Changes to tables that add or delete historical data
v Denormalization of the database to reduce overhead

Placing tables on disk

Tables that the database server supports reside on one or more portions of one or
more disks. You control the placement of a table on disk when you create it by
assigning it to a dbspace.

Tables that the database server supports reside on one or more portions of a disk
or disks. You control the placement of a table on disk when you create it by
assigning it to a dbspace. A dbspace consists of one or more chunks. Each chunk
corresponds to all or part of a disk partition. When you assign chunks to dbspaces,
you make the disk space in those chunks available for storing tables or table
fragments.

When you configure chunks and allocate them to dbspaces, you must relate the
size of the dbspaces to the tables or fragments that each dbspace is to contain. To
estimate the size of a table, follow the instructions in “Estimating table size” on
page 6-5.

The database administrator (DBA) who is responsible for creating a table assigns
that table to a dbspace in one of the following ways:
v By using the IN DBSPACE clause of the CREATE TABLE statement
v By using the dbspace of the current database
The most recent DATABASE or CONNECT statement that the DBA issues before
issuing the CREATE TABLE statement sets the current database.

The DBA can fragment a table across multiple dbspaces, as described in “Planning
a fragmentation strategy” on page 9-1, or use the ALTER FRAGMENT statement to
move a table to another dbspace. The ALTER FRAGMENT statement provides the
simplest method for altering the placement of a table. However, the table is
unavailable while the database server processes the alteration. Schedule the
movement of a table or fragment at a time that affects the fewest users.

Other methods exist for moving tables between dbspaces:

v You can unload the data from a table and then move that data to another
dbspace with the SQL statements LOAD and UNLOAD, the onload and
onunload utilities or the High-Performance Loader (HPL).
v You can load data into and unload data from external tables.

Moving tables between databases with LOAD and UNLOAD, onload and
onunload, or HPL involves periods in which data from the table is copied to tape

© Copyright IBM Corp. 1996, 2014 6-1

 and then reloaded onto the system. These periods present windows of
vulnerability during which a table can become inconsistent with the rest of the
database. To prevent the table from becoming inconsistent, you must restrict access
to the version that remains on disk while the data transfers occur.

Depending on the size, fragmentation strategy, and indexes that are associated
with a table, it can be faster to unload a table and reload it than to alter
fragmentation. For other tables, it can be faster to alter fragmentation. You can
experiment to determine which method is faster for a table that you want to move
or re-partition.
Related concepts:
The onunload and onload utilities (Migration Guide)
Related tasks:
Moving data with external tables (Administrator's Guide)
Related reference:
ALTER FRAGMENT statement (SQL Syntax)
LOAD statement (SQL Syntax)
UNLOAD statement (SQL Syntax)
CREATE EXTERNAL TABLE Statement (SQL Syntax)

Isolating high-use tables

You can place a table with high I/O activity on a dedicated disk device. Doing this
reduces contention for the data that is stored in that table.

When disk drives have different performance levels, you can put the tables with
the highest use on the fastest drives. Placing two high-use tables on separate disk
devices reduces competition for disk access when the two tables experience
frequent, simultaneous I/O from multiple applications or when joins are formed
between them.

To isolate a high-use table on its own disk device, assign the device to a chunk,
assign that chunk to a dbspace, and then place the table in the dbspace that you
created. Figure 6-1 shows three high-use tables, each in a separate dbspace, placed
on three disks.

Locate each high-use table

in a separate dbspace, each
on its own partition or disk.

High-use High-use High-use Database

table 1 table 2 table 3

Dbspace 1 Dbspace 2 Dbspace 3

Figure 6-1. Isolating high-use tables

6-2 IBM Informix Performance Guide

Placing high-use tables on middle partitions of disks
To minimize disk-head movement, place the most frequently accessed data on
partitions close to the middle band of the disk (not near the center and not near
the edge). This approach minimizes disk-head movement to reach data in the
high-demand table.

The following figure shows the placement of the most frequently accessed data on
partitions close to the middle band of the disk.

Single chunk in a Create high-use

dbspace table in dbspace.

Locate high-use table on the

middle partitions of the disk.

Disk platter

Figure 6-2. Disk platter with high-use table located on middle Partitions

To place high-use tables on the middle partition of the disk, create a raw device
composed of cylinders that reside midway between the spindle and the outer edge
of the disk. (For instructions on how to create a raw device, see the IBM Informix
Administrator's Guide for your operating system.) Allocate a chunk, associating it
with this raw device, as your IBM Informix Administrator's Reference describes. Then
create a dbspace with this same chunk as the initial and only chunk. When you
create a high-use table, place the table in this dbspace.

Using multiple disks

You can use multiple disks for dbspaces, logical logs, temporary tables, and sort
files.

Using multiple disks for a dbspace

Using multiple disks for a dbspace helps to distribute I/O across dbspaces that
contain several small tables.

A dbspace can include multiple chunks, and each chunk can represent a different
disk. The maximum size for a chunk is 4 terabytes. This arrangement allows you
to distribute data in a dbspace over multiple disks. Figure 6-3 shows a dbspace
distributed over three disks.

Device 0x21 Device 0x27 Device 0x22

Dbspace three_arms

Figure 6-3. A dbspace distributed over three disks

Chapter 6. Table performance considerations 6-3

 Because you cannot use this type of distributed dbspace for parallel database
queries (PDQ), you should use the table-fragmentation techniques described in
“Distribution schemes” on page 9-6 to partition large, high-use tables across
multiple dbspaces.

Using multiple disks for logical logs

You can distribute logical logs in different dbspaces on multiple disks in
round-robin fashion to improve logical backup performance. This scheme allows
the database server to back up logs on one disk, while performing logging
operations on the other disks.

Keep your logical logs and the physical log on separate devices to improve
performance by decreasing I/O contention on a single device. The logical and
physical logs are created in the root dbspace when the database server is
initialized. After initialization, you can move them to other dbspaces.

Spreading temporary tables and sort files across multiple disks

You can spread the I/O associated with temporary tables and sort files across
multiple disks, after defining dbspaces for temporary tables and sort files. This can
improve performance for applications that require a large amount of temporary
space for temporary tables or large sort operations.

To define several dbspaces for temporary tables and sort files, use onspaces -t.
When you place these dbspaces on different disks and list them in the
DBSPACETEMP configuration parameter, you spread the I/O associated with
temporary tables and sort files across multiple disks, as Figure 6-4 illustrates. You
can list dbspaces that contain regular tables in DBSPACETEMP.

Device 0x21 Device 0x27 Device 0x22

Dbspace tmpdbs1 Dbspace tmpdbs2 Dbspace tmpdbs3

DBSPACETEMP= tmpdbs1,tmpdbs2,tmpdbs3

Figure 6-4. Dbspaces for temporary tables and sort files

Users can specify their own lists of dbspaces for temporary tables and sort files
with the DBSPACETEMP environment variable. For details, see “Configure
dbspaces for temporary tables and sort files” on page 5-8.

Backup and restore considerations when placing tables on

disks
When you decide where to place your tables or fragments, remember that if a
device that contains a dbspace fails, all tables or table fragments in that dbspace
are rendered inaccessible, even though tables and fragments in other dbspaces are
accessible. The need to limit data unavailability in the event of a disk failure might
influence which tables you group together in a particular dbspace.

6-4 IBM Informix Performance Guide

 Although you must perform a cold restore if a dbspace that contains critical data
fails, you need only perform a warm restore if a noncritical dbspace fails. The
desire to minimize the impact of cold restores might influence the dbspace that
you use to store critical data.

Factors affecting the performance of nonfragmented tables

and table fragments
Numerous factors affect the performance of an individual table or table fragment.
These include the placement of the table or fragment, the size of the table or
fragment, the indexing strategy that was used, the size and placement of table
extents with respect to one another, and the frequency of access to the table.

Estimating table size

You can calculate the approximate sizes (in disk pages) of tables.

For a description of size calculations for indexes, see “Estimating index pages” on
page 7-4.

The disk pages allocated to a table are collectively referred to as a tblspace. The
tblspace includes data pages. A separate tblspace includes index pages. If simple
large objects (TEXT or BYTE data) are associated with a table that is not stored in
an alternative dbspace, pages that hold simple large objects are also included in the
tblspace.

The tblspace does not correspond to any fixed region within a dbspace. The data
extents and indexes that make up a table can be scattered throughout the dbspace.

The size of a table includes all the pages within the tblspace: data pages and pages
that store simple large objects. Blobpages that are stored in a separate blobspace
are not included in the tblspace and are not counted as part of the table size.

The size of a table includes all the pages within the tblspace: data pages and pages
that store simple large objects. Blobpages that are stored in a separate blobspace or
on an optical subsystem are not included in the tblspace and are not counted as
part of the table size.

The following sections describe how to estimate the page count for each type of
page within the tblspace.

Tip: If an appropriate sample table exists, or if you can build a sample table of
realistic size with simulated data, you do not need to make estimates. You can run
oncheck -pt to obtain exact numbers.

Estimating data pages

How you estimate the data pages of a table depends on whether that table
contains fixed-length or variable-length rows.

Estimating tables with fixed-length rows

You can estimate the size (in pages) of a table with fixed-length rows. A table with
fixed-length rows has no columns of the VARCHAR or NVARCHAR data type.

Perform the following steps to estimate the size (in pages) of a table with
fixed-length rows.

Chapter 6. Table performance considerations 6-5

 To estimate the page size, row size, number of rows, and number of data pages:
1. Use onstat -b to obtain the size of a page.
The buffer size field in the last line of this output displays the page size.
2. Subtract 28 from this amount to account for the header that appears on each
data page. The resulting amount is referred to as pageuse.
3. To calculate the size of a row, add the widths of all the columns in the table
definition. TEXT and BYTE columns each use 56 bytes. If you have already
created your table, you can use the following SQL statement to obtain the size
of a row:
SELECT rowsize FROM systables WHERE tabname =
’table-name’;
4. Estimate the number of rows that the table is expected to contain. This number
is referred to as rows. The procedure for calculating the number of data pages
that a table requires differs depending on whether the row size is less than or
greater than pageuse.
5. If the size of the row is less than or equal to pageuse, use the following formula
to calculate the number of data pages. The trunc() function notation indicates
that you are to round down to the nearest integer.
data_pages = rows / trunc(pageuse/(rowsize + 4))
The maximum number of rows per page is 255, regardless of the size of the
row.

Important: Although the maximum size of a row that the database server
accepts is approximately 32 kilobytes, performance degrades when a row
exceeds the size of a page. For information about breaking up wide tables for
improved performance, see “Denormalize the data model to improve
performance” on page 6-42.
6. If the size of the row is greater than pageuse, the database server divides the
row between pages. The page that contains the initial portion of a row is called
the home page. Pages that contains subsequent portions of a row are called
remainder pages. If a row spans more than two pages, some of the remainder
pages are completely filled with data from that row. When the trailing portion
of a row uses less than a page, it can be combined with the trailing portions of
other rows to fill out the partial remainder page. The number of data pages is
the sum of the home pages, the full remainder pages, and the partial remainder
pages.
a. Calculate the number of home pages.
The number of home pages is the same as the number of rows:
homepages = rows
b. Calculate the number of full remainder pages.
First calculate the size of the row remainder with the following formula:
remsize = rowsize - (pageuse + 8)
If remsize is less than pageuse - 4, you have no full remainder pages.
If remsize is greater than pageuse - 4, use remsize in the following formula to
obtain the number of full remainder pages:
fullrempages = rows * trunc(remsize/(pageuse - 8))
c. Calculate the number of partial remainder pages.
First calculate the size of a partial row remainder left after you have
accounted for the home and full remainder pages for an individual row. In
the following formula, the remainder() function notation indicates that you
are to take the remainder after division:

6-6 IBM Informix Performance Guide

 partremsize = remainder(rowsize/(pageuse - 8)) + 4
The database server uses certain size thresholds with respect to the page
size to determine how many partial remainder pages to use. Use the
following formula to calculate the ratio of the partial remainder to the page:
partratio = partremsize/pageuse
Use the appropriate formula in the following table to calculate the number
of partial remainder pages.

Formula to Calculate the Number of Partial Remainder

partratio Value Pages
Less than .1 partrempages = rows/(trunc((pageuse/10)/remsize) + 1)
Less than .33 partrempages = rows/(trunc((pageuse/3)/remsize) + 1)
.33 or larger partrempages = rows

d. Add up the total number of pages with the following formula:

tablesize = homepages + fullrempages + partrempages

Estimating tables with variable-length rows

You can estimate the size of a table with variable-length rows with columns of the
VARCHAR or NVARCHAR data type.

When a table contains one or more VARCHAR or NVARCHAR columns, its rows
can have varying lengths. These varying lengths introduce uncertainty into the
calculations. You must form an estimate of the typical size of each VARCHAR
column, based on your understanding of the data, and use that value when you
make the estimates.

Important: When the database server allocates space to rows of varying size, it
considers a page to be full when no room exists for an additional row of the
maximum size.

To estimate the size of a table with variable-length rows, you must make the
following estimates and choose a value between them, based on your
understanding of the data:
v The maximum size of the table, which you calculate based on the maximum
width allowed for all VARCHAR or NVARCHAR columns
v The projected size of the table, which you calculate based on a typical width for
each VARCHAR or NVARCHAR column

To estimate the maximum number of data pages:

1. To calculate rowsize, add together the maximum values for all column widths.
2. Use this value for rowsize and perform the calculations described in “Estimating
tables with fixed-length rows” on page 6-5. The resulting value is called
maxsize.

To estimate the projected number of data pages:

1. To calculate rowsize, add together typical values for each of your variable-width
columns. It is suggested that you use the most frequently occurring width
within a column as the typical width for that column. If you do not have access
to the data or do not want to tabulate widths, you might choose to use some
fractional portion of the maximum width, such as 2/3 (.67).
2. Use this value for rowsize and perform the calculations described in “Estimating
tables with fixed-length rows” on page 6-5. The resulting value is called projsize.

Chapter 6. Table performance considerations 6-7

 Selecting an intermediate value for the size of the table
The actual table size should fall somewhere between the projected number of data
pages (projsize) and the maximum number of data pages (maxsize).

Based on your knowledge of the data, choose a value within that range that seems
most reasonable to you. The less familiar you are with the data, the more
conservative (higher) your estimate should be.

Estimating pages that simple large objects occupy

You can estimate the total number of pages for all simple large objects, or you can
estimate the number of pages based on the median size of the simple large objects.

The blobpages can reside in either the dbspace where the table resides or in a
blobspace. For more information about when to use a blobspace, see “Storing
simple large objects in the tblspace or a separate blobspace.”

The following methods for estimating blobpages yield a conservative (high)

estimate because a single TEXT or BYTE column does not necessarily occupy the
entire blobpage within a tblspace. In other words, a blobpage in a tblspace can
contain multiple TEXT or BYTE columns.

To estimate the number of blobpages:

1. Obtain the page size with onstat -b.
2. Calculate the usable portion of the blobpage with the following formula:
bpuse = pagesize - 32
3. For each byte of blobsize n, calculate the number of pages that the byte
occupies (bpages_n) with the following formula:
bpages1 = ceiling(bytesize1/bpuse)
bpages2 = ceiling(bytesize2/bpuse)
...
bpages_n = ceiling(bytesize_n/bpuse)
The ceiling() function indicates that you should round up to the nearest integer
value.
4. Add up the total number of pages for all simple large objects, as follows:
blobpages = bpages1 + bpages2 + ... + bpagesn

Alternatively, you can base your estimate on the median size of simple large
objects (TEXT or BYTE data); that is, the simple-large-object data size that occurs
most frequently. This method is less precise, but it is easier to calculate.

To estimate the number of blobpages based on the median size of simple large
objects:
1. Calculate the number of pages required for simple large objects of median size,
as follows:
mpages = ceiling(mblobsize/bpuse)
2. Multiply this amount by the total number of simple large objects, as follows:
blobpages = blobcount * mpages

Storing simple large objects in the tblspace or a separate

blobspace
When you create a simple-large-object column on magnetic disk, you have the
option of storing the column data in the tblspace or in a separate blobspace. You

6-8 IBM Informix Performance Guide

can often improve performance by storing simple-large-object data in a separate
blobspace, and by storing smart large objects and user-defined data in sbspaces.

You can also store simple large objects on optical media, but this discussion does
not apply to simple large objects stored in this way.

In the following example, a TEXT value is stored in the tblspace, and a BYTE value
is stored in a blobspace named rasters:
CREATE TABLE examptab
(
pic_id SERIAL,
pic_desc TEXT IN TABLE,
pic_raster BYTE IN rasters
)

For information about storing simple-large-object data in a separate blobspace, see

“Estimating pages that simple large objects occupy” on page 6-8.

A TEXT or BYTE value is always stored apart from the rows of the table; only a
56-byte descriptor is stored with the row. However, a simple large object occupies
at least one disk page. The simple large object to which the descriptor points can
reside in the same set of extents on disk as the table rows (in the same tblspace) or
in a separate blobspace.

When simple large objects are stored in the tblspace, the pages of their data are
interspersed among the pages that contain rows, which can greatly increase the
size of the table. When the database server reads only the rows and not the simple
large objects, the disk arm must move farther than when the blobpages are stored
apart. The database server scans only the row pages in the following situations:
v When it performs any SELECT operation that does not retrieve a
simple-large-object column
v When it uses a filter expression to test rows

Another consideration is that disk I/O to and from a dbspace is buffered in shared
memory of the database server. Pages are stored in case they are needed again
soon, and when pages are written, the requesting program can continue before the
actual disk write takes place. However, because blobspace data is expected to be
voluminous, disk I/O to and from blobspaces is not buffered, and the requesting
program is not allowed to proceed until all output has been written to the
blobspace.

For best performance, store a simple-large-object column in a blobspace in either of

the following circumstances:
v When single data items are larger than one or two pages each
v When the number of pages of TEXT or BYTE data is more than half the number
of pages of row data

Estimating tblspace pages for simple large objects

In your estimate of the space required for a table, include blobpages for any simple
large objects that are to be stored in that tblspace. For a table that is both relatively
small and nonvolatile, you can achieve the effect of a dedicated blobspace by
separating row pages and blobpages.

To separate row pages from blobpages within a dbspace:

Chapter 6. Table performance considerations 6-9

 1. Load the entire table with rows in which the simple-large-object columns are
null.
2. Create all indexes. The row pages and the index pages are now contiguous.
3. Update all the rows to install the simple large objects. The blobpages now
appear after the pages of row and index data within the tblspace.

Managing the size of first and next extents for the tblspace tblspace
The tblspace tblspace is a collection of pages that describe the location and
structure of all tblspaces in a dbspace. Each dbspace has one tblspace tblspace.
When you create a dbspace, you can use the TBLTBLFIRST and TBLTBLNEXT
configuration parameters to specify the first and next extent sizes for the tblspace
tblspace in a root dbspace.

You can use the onspaces utility to specify the initial and next extent sizes for the
tblspace tblspace in non-root dbspaces.

Specify the initial and next extent sizes if you want to reduce the number of
tblspace tblspace extents and reduce the frequency of situations when you need to
place the tblspace tblspace extents in non-primary chunks.

The ability to specify a first extent size that is larger than the default provides
flexibility for managing space. When you create an extent, you can reserve space
during creation of the dbspace, thereby decreasing the risk of needing additional
extents created in chunks that are not initial chunks.

You can only specify the first and next extent sizes when you create a dbspace. You
cannot alter the specification of the first and next extents sizes after the creation of
the dbspace. In addition, you cannot specify extent sizes for temporary dbspaces,
sbspaces, blobspaces, or external spaces.

If you do not specify first and next extent sizes for the tblspace tblspace, Informix
uses the existing default extent sizes.
Related tasks:
Specifying the first and next extent sizes for the tblspace tblspace
(Administrator's Guide)
Related reference:
TBLTBLFIRST configuration parameter (Administrator's Reference)
TBLTBLNEXT configuration parameter (Administrator's Reference)

Managing sbspaces
An sbspace is a logical storage unit composed of one or more chunks that store
smart large objects. You can estimate the amount of storage needed for smart large
objects, improve metadata I/O, monitor sbspaces, and change storage
characteristics.

Estimating pages that smart large objects occupy

In your estimate of the space required for a table, you should also consider the
amount of sbspace storage for any smart large objects (such as CLOB, BLOB, or
multi-representative data types) that are part of the table. An sbspace contains
user-data areas and metadata areas.

6-10 IBM Informix Performance Guide

CLOB and BLOB data is stored in sbpages that reside in the user-data area. The
metadata area contains the smart-large-object attributes, such as average size and
whether or not the smart large object is logged. For more information about
sbspaces, see your IBM Informix Administrator's Guide.

Estimating the size of the sbspace and metadata area

The first chunk of an sbspace must have a metadata area. When you add smart
large objects, the database server adds more control information to this metadata
area.

If you add a chunk to the sbspace after the initial allocation, you can take one of
the following actions for metadata space:
v Allocate another metadata area on the new chunk by default.
This action provides the following advantages:
– It is easier because the database server automatically calculates and allocates a
new metadata area on the added chunk based on the average smart large
object size
– Distributes I/O operations on the metadata area across multiple disks
v Use the existing metadata area
If you specify the onspaces -U option, the database server does not allocate
metadata space in the new chunk. Instead it must use a metadata area in one of
the other chunks.

In addition, the database server reserves 40 percent of the user area to be used in
case the metadata area runs out of space. Therefore, if the allocated metadata
becomes full, the database server starts using this reserved space in the user area
for additional control information.

You can let the database server calculate the size of the metadata area for you on
the initial chunk and on each added chunks. However, you might want to specify
the size of the metadata area explicitly, to ensure that the sbspace does not run out
of metadata space and the 40 percent reserve area. You can use one of the
following methods to explicitly specify the amount of metadata space to allocate:
v Specify the AVG_LO_SIZE tag on the onspaces -Df option.
The database server uses this value to calculate the size of the metadata area to
allocate when the -Ms option is not specified. If you do not specify
AVG_LO_SIZE, the database server uses the default value of 8 kilobytes to
calculate the size of the metadata area.
v Specify the metadata area size in the -Ms option of the onspaces utility.
Use the procedure that “Sizing the metadata area manually for a new chunk”
describes to estimate a value to specify in the onspaces -Ms option.

Sizing the metadata area manually for a new chunk

Each chunk can contain metadata, but the sum total must accommodate enough
room for all LO headers (average length 570 bytes each) and the chunk free list
(which lists all the free extents in the chunk).

The following procedure assumes that you know the sbspace size and need to
allocate more metadata space.

To size the metadata area manually for a new chunk:

1. Use the onstat -d option to obtain the size of the current metadata area from
the Metadata size field.

Chapter 6. Table performance considerations 6-11

 2. Estimate the number of smart large objects that you expect to reside in the
sbspace and their average size.
3. Use the following formula to calculate the total size of the metadata area:
Total metadata kilobytes = (LOcount*570)/1024 +
(numchunks*800) + 100
LOcount
is the number of smart large objects that you expect to have in all
sbspace chunks, including the new one.
numchunks
is the total number of chunks in the sbspace.
4. To obtain the additional required area for metadata, subtract the current
metadata size that you obtained in step 1 from the value that you obtained in
step 3.
5. When you add another chunk, specify in the -Ms option of the onspaces -a
command the value that you obtained in step 4.

Example of calculating the metadata area for a new chunk:

This topic contains an example showing how to estimate the metadata size
required for two sbspaces chunks.

Suppose the Metadata size field in the onstat -d option shows that the current
metadata area is 1000 pages. If the system page size is 2048 bytes, the size of this
metadata area is 2000 kilobytes, as the following calculation shows:
current metadata = (metadata_size * pagesize) / 1024
= (1000 * 2048) / 1024
= 2000 kilobytes

Suppose you expect 31,000 smart large objects in the two sbspace chunks. The
following formula calculates the total size of metadata area required for both
chunks, rounding up fractions:
Total metadata = (LOcount*570)/1024 + (numchunks*800) + 100
= (31,000 * 570)/1024 + (2*800) + 100
= 17256 + 1600 + 100
= 18956 kilobytes

To obtain the additional area that is required for metadata:

1. Subtract the current metadata size from the total metadata value.
Additional metadata = Total metadata - current metadata
= 18956 - 2000
= 16956 kilobytes
2. When you add the chunk to the sbspace, use the -Ms option of the onspaces -a
command to specify a metadata area of 16,956 kilobytes.
% onspaces -a sbchk2 -p /dev/raw_dev1 -o 200 -Ms 16956

Improving metadata I/O for smart large objects

The metadata pages in an sbspace contain information about the location of the
smart large objects in the sbspace. Typically, these pages are read intensive. You
can improve metadata I/O by redistributing it.

You can distribute I/O to these pages in one of the following ways:
v Mirror the chunks that contain metadata.

6-12 IBM Informix Performance Guide

 For more information about the implications of mirroring, see “Consider
mirroring for critical data components” on page 5-5.
v Position the metadata pages on the fastest portion of the disk.
Because the metadata pages are the most read-intensive part of an sbspace, place
the metadata pages toward the middle of the disk to minimize disk seek time.
To position metadata pages, use the -Mo option when you create the sbspace or
add a chunk with the onspaces utility.
v Spread metadata pages across disks.
To spread metadata pages across disks, create multiple chunks in an sbspace,
with each chunk residing on a separate disk. When you add a chunk to the
sbspace with the onspaces utility, specify the -Ms option to allocate pages for
the metadata information.
Although the database server attempts to keep the metadata information with its
corresponding data in the same chunk, it cannot guarantee that they will be
together.
v Decrease the number of extents each smart large object occupies.
When a smart large object spans multiple extents, the metadata area contains a
separate descriptor for each extent. To decrease the number of descriptor entries
that must be read for each smart large object, specify the expected final size of
the smart large object when you create the smart large object.
The database server allocates the smart large object as a single extent (if it has
contiguous storage in the chunk) when you specify the final size in either of the
following functions:
– The DataBlade API mi_lo_specset_estbytes function
– The Informix ESQL/C ifx_lo_specset_estbytes function
For more information about the functions to open a smart large object and to set
the estimated number of bytes, see the IBM Informix ESQL/C Programmer's
Manual and IBM Informix DataBlade API Programmer's Guide.
For more information about sizing extents, see “Sbspace extents” on page 5-21.

Important: For highest data availability, mirror all sbspace chunks that contain
metadata.

Monitoring sbspaces
You can monitor the effectiveness of I/O operations on smart large objects. For
better I/O performance, all smart large objects should be allocated in one extent to
be contiguous.

For more information about sizing extents, see “Sbspace extents” on page 5-21.

Contiguity provides the following I/O performance benefits:

v Minimizes the disk-arm motion
v Requires fewer I/O operations to read the smart large object
v When doing large sequential reads, can take advantage of lightweight I/O,
which reads in larger blocks of data (60 kilobytes or more, depending on your
platform) in a single I/O operation

You can use the following command-line utilities to monitor the effectiveness of
I/O operations on smart large objects:
v oncheck -cS, -pe and -pS
v onstat -g smb s option

Chapter 6. Table performance considerations 6-13

 The following sections describe how to use these utility options to monitor
sbspaces.

Monitoring sbspaces with oncheck -cS

The oncheck -cS option checks smart-large-object extents and the sbspace
partitions in the user-data area.

Figure 6-5 shows an example of the output from the -cS option for s9_sbspc.

The values in the Sbs#, Chk#, and Seq# columns correspond to the Space Chunk
Page value in the -pS output. The Bytes and Pages columns display the size of
each smart large object in bytes and pages.

To calculate the average size of smart large objects, you can total the numbers in
the Size (Bytes) column and then divide by the number of smart large objects. In
Figure 6-5, the average number of bytes allocated is 2690, as the following
calculation shows:
Average size in bytes = (15736 + 98 + 97 + 62 + 87 + 56) / 6
= 16136 / 6
= 2689.3

For information about how to specify smart large object sizes to influence extent
sizes, see “Sbspace extents” on page 5-21.

Validating space ’s9_sbspc’ ...

Large Objects
ID Ref Size Allocced Creat Last
Sbs# Chk# Seq# Cnt (Bytes) Pages Extns Flags Modified
---- ---- ----- ---- ---------- -------- ----- ----- ------------------------
2 2 1 1 15736 8 1 N-N-H Thu Jun 21 16:59:12 2007
2 2 2 1 98 1 1 N-K-H Thu Jun 21 16:59:12 2007
2 2 3 1 97 1 1 N-K-H Thu Jun 21 16:59:12 2007
2 2 4 1 62 1 1 N-K-H Thu Jun 21 16:59:12 2007
2 2 5 1 87 1 1 N-K-H Thu Jun 21 16:59:12 2007
2 2 6 1 56 1 1 N-K-H Thu Jun 21 16:59:12 2007

Figure 6-5. oncheck -cS output

The Extns field shows the minimum extent size, in number of pages, allocated to
each smart large object.

Monitoring sbspaces with oncheck -pe

The oncheck -pe option displays information that includes the size in pages of the
chunk, the number of pages used, the number of pages that are free, and a list of
all the tables in the chunk, with the initial page number and the length of the table
in pages. This option also shows if smart large objects occupy contiguous space
within an sbspace.

Execute oncheck -pe to display the following information to determine if the smart
large objects occupy contiguous space within an sbspace:
v Identifies each smart large object with the term SBLOBSpace LO
The three values in brackets following SBLOBSpace LO correspond to the Sbs#,
Chk#, and Seq# columns in the -cS output.
v Offset of each smart large object
v Number of disk pages (not sbpages) used by each smart large object

6-14 IBM Informix Performance Guide

Tip: The oncheck -pe option provides information about sbspace use in terms of
database server pages, not sbpages.

Figure 6-6 shows sample output. In this example, the size field shows that the first
smart large object occupies eight pages. Because the offset field shows that the first
smart large object starts at page 53 and the second smart large object starts at page
61, the first smart large object occupies contiguous pages.

Chunk Pathname Size Used Free

1000 940 60

Description Offset Size

-------------------------------------------------- -------- --------
RESERVED PAGES 0 2
CHUNK FREELIST PAGE 2 1
s9_sbspc:’informix’.TBLSpace 3 50
SBLOBSpace LO [2,2,1] 53 8
SBLOBSpace LO [2,2,2] 61 1
SBLOBSpace LO [2,2,3] 62 1
SBLOBSpace LO [2,2,4] 63 1
SBLOBSpace LO [2,2,5] 64 1
SBLOBSpace LO [2,2,6] 65 1
...

Figure 6-6. oncheck -pe output that shows contiguous space use

Monitoring sbspaces with oncheck -pS

The oncheck -pS option displays information about smart-large-object extents and
metadata areas in sbspace partitions. If you do not specify an sbspace name on the
command line, oncheck checks and displays the metadata for all sbspaces.

Figure 6-7 on page 6-16 shows an example of the -pS output for s9_sbspc.

To display information about smart large objects, execute the following command:
oncheck -pS spacename

The oncheck -pS output displays the following information for each smart large
object in the sbspace:
v Space chunk page
v Size in bytes of each smart large object
v Object ID that DataBlade API and Informix ESQL/C functions use
v Storage characteristics of each smart large object

When you use onspaces -c -S to create an sbspace, you can use the -Df option to
specify various storage characteristics for the smart large objects. You can use
onspaces -ch to change attributes after the sbspace is created. The Create Flags
field in the oncheck -pS output displays these storage characteristics and other
attributes of each smart large object. In Figure 6-7 on page 6-16, the Create Flags
field shows LO_LOG because the LOGGING tag was set to ON in the -Df option.

Chapter 6. Table performance considerations 6-15

 Space Chunk Page = [2,2,2] Object ID = 987122917
LO SW Version 4
LO Object Version 1
Created by Txid 7
Flags 0x31 LO_LOG LO_NOKEEP_LASTACCESS_TIME LO_HIGH_INTEG
Data Type 0
Extent Size -1
IO Size 0
Created Thu Apr 12 17:48:35 2007
Last Time Modified Thu Apr 12 17:48:43 2007
Last Time Accessed Thu Apr 12 17:48:43 2007
Last Time Attributes Modified Thu Apr 12 17:48:43 2007
Ref Count 1
Create Flags 0x31 LO_LOG LO_NOKEEP_LASTACCESS_TIME LO_HIGH_INTEG
Status Flags 0x0 LO_FROM_SERVER
Size (Bytes) 2048
Size Limit -1
Total Estimated Size -1
Deleting TxId -1
LO Map Size 200
LO Map Last Row -1
LO Map Extents 2
LO Map User Pages 2

Figure 6-7. oncheck -pS output

Monitoring sbspaces with onstat -g smb

The onstat -g smb s option displays sbspace attributes.

Use the onstat -g smb s option to display the following characteristics that affect
the I/O performance of each sbspace:
v Logging status
If applications are updating temporary smart large objects, logging is not
required. You can turn off logging to reduce the amount of I/O activity to the
logical log, CPU utilization, and memory resources.
v Average smart-large-object size
Average size and extent size should be similar to reduce the number of I/O
operations required to read in an entire smart large object. The avg s/kb output
field shows the average smart-large-object size in kilobytes. In Figure 6-8 on
page 6-17, the avg s/kb output field shows the value 30 kilobytes.
Specify the final size of the smart large object in either of the following functions
to allocate the object as a single extent:
– The DataBlade API mi_lo_specset_estbytes function
– The Informix ESQL/C ifx_lo_specset_estbytes function
For more information about the functions to open a smart large object and to set
the estimated number of bytes, see the IBM Informix ESQL/C Programmer's
Manual and IBM Informix DataBlade API Programmer's Guide.
v First extent size, next extent size, and minimum extent size
The 1st sz/p, nxt sz/p, and min sz/p output fields show these extent sizes if you
set the extent tags in the -Df option of onspaces. In Figure 6-8 on page 6-17,
these output fields show values of 0 and -1 because these tags are not set in
onspaces.

6-16 IBM Informix Performance Guide

 sbnum 7 address 2afae48
Space : flags nchk owner sbname
-------- 1 informix client
Defaults : LO_LOG LO_KEEP_LASTACCESS_TIME

LO : ud b/pg flags flags avg s/kb max lcks

2048 0 -------- 30 -1
Ext/IO : 1st sz/p nxt sz/p min sz/p mx io sz
4 0 0 -1

HdrCache : max free

512 0

Figure 6-8. onstat -g smb s output

Changing storage characteristics of smart large objects

When you create an sbspace, but do not specify values in the -Df option of the
onspaces -c -S command, you use the defaults for the storage characteristics and
attributes (such as logging and buffering). After you monitor sbspaces, you might
want to change the storage characteristics, logging status, lock mode, or other
attributes for new smart large objects.

The database administrator or programmer can use the following methods to

override these default values for storage characteristics and attributes:
v The database administrator can use one of the following onspaces options:
– Specify values when the sbspace is first created with the onspaces -c -S
command.
– Change values after the sbspace is created with the onspaces -ch command.
Specify these values in the tag options of the -Df option of onspaces. For more
information about the onspaces utility, see the IBM Informix Administrator's
Reference.
v The database administrator can specify values in the PUT clause of the CREATE
TABLE or ALTER TABLE statements.
These values override the values in the onspaces utility and are valid only for
smart large objects that are stored in the associated column of the specific table.
Other smart large objects (from columns in other tables) might also reside in this
same sbspace. These other columns continue to use the storage characteristics
and attributes of the sbspace that onspaces defined (or the default values, if
onspaces did not define them) unless these columns also used a PUT clause to
override them for a particular column.
If you do not specify the storage characteristics for a smart-large-object column
in the PUT clause, they are inherited from the sbspace.
If you do not specify the PUT clause when you create a table with
smart-large-object columns, the database server stores the smart large objects in
the system default sbspace, which is specified by the SBSPACENAME
configuration parameter in the ONCONFIG file. In this case, the storage
characteristics and attributes are inherited from the SBSPACENAME sbspace.
v Programmers can use functions in the DataBlade API and Informix ESQL/C to
alter storage characteristics for a smart-large-object column.

Chapter 6. Table performance considerations 6-17

 For information about the DataBlade API functions for smart large objects, see
the IBM Informix DataBlade API Programmer's Guide. For information about the
Informix ESQL/C functions for smart large objects, see the IBM Informix ESQL/C
Programmer's Manual.

Table 6-1 summarizes the ways to alter the storage characteristics for a smart large
object.
Table 6-1. Altering storage characteristics and other attributes of an sbspace
System-Specified Column-Level Storage
Storage Characteristics Storage Storage
Characteristics Specified by PUT Characteris-tics Characteris-tics
Storage Specified by -Df clause of CREATE Specified by a Specified by an
Character-istic System Default Option in onspaces TABLE or ALTER DataBlade API ESQL/C
or Attribute Value Utility TABLE Function Function
Last-access OFF ACCESSTIME KEEP ACCESS TIME, Yes Yes
time NO KEEP ACCESS
TIME
Lock mode BLOB LOCK_MODE No Yes Yes
Logging status OFF LOGGING LOG, NO LOG Yes Yes
Data integrity HIGH INTEG No HIGH INTEG, Yes No
MODERATE INTEG
Size of extent None EXTENT_SIZE EXTENT SIZE Yes Yes
Size of next None NEXT_SIZE No No No
extent
Minimum 2 kilobytes on MIN_EXT_SIZE No No No
extent size Windows 4
kilobytes on
UNIX
Size of smart 8 kilobytes Average size of all No Estimated size of Estimated size
large object smart large objects a particular smart of a particular
in sbspace: large object smart large
AVG_LO_SIZE Maximum size of object Maximum
a particular smart size of a
large object particular smart
large object
Buffer pool ON BUFFERING No LO_BUFFER and LO_BUFFER
usage LO_ NOBUFFER and LO_
flags NOBUFFER
flags
Name of SBSPACE- Not in -Df option. Name of an existing Yes Yes
sbspace NAME Name specified in sbspace in which a
onspaces -S option. smart large object
resides: PUT ... IN
clause
Fragmenta- None No Round-robin Round-robin or Round-robin or
tion across distribution scheme: expression-based expression-based
multiple PUT ... IN clause distribution distribution
sbspaces scheme scheme
Last-access OFF ACCESSTIME KEEP ACCESS TIME, Yes Yes
time NO KEEP ACCESS
TIME

6-18 IBM Informix Performance Guide

 Altering smart-large-object columns
When you create or modify a table, you have several options for choosing storage
characteristics and other attributes (such as logging status, buffering, and lock
mode) for specific smart-large-object columns.

When you create or modify a table, you can:

v Use the values that were set when the sbspace was created. These values are
specified in one of the following ways:
– With the various tags of the -Df option of the onspaces -c -S command
– With the system default value for any specific tag that was not specified
For guidelines to change the default storage characteristics of the -Df tags, see
“onspaces options that affect sbspace I/O” on page 5-21.
v Use the PUT clause of the CREATE TABLE statement to specify non-default
values for particular characteristics or attributes.
When you do not specify particular characteristics or attributes in the PUT
clause, they default to the values set in the onspaces -c -S command.

Later, you can use the PUT clause of the ALTER TABLE statement to change the
optional storage characteristics of these columns. Table 6-1 on page 6-18 shows
which characteristics and attributes you can change.

You can use the PUT clause of the ALTER TABLE statement to perform the
following actions:
v Specify the smart-large-object characteristics and storage location when you add
a new column to a table.
The smart large objects in the new columns can have different characteristics
than those in the existing columns.
v Change the smart-large-object characteristics of an existing column.
The new characteristics of the column apply only to new smart large objects
created for that column. The characteristics of existing smart large objects remain
the same.

For example, the BLOB data in the catalog table in the superstores_demo database
is stored in s9_sbspc with logging turned off and has an extent size of 100
kilobytes. You can use the PUT clause of the ALTER TABLE statement to turn on
logging and store new smart large objects in a different sbspace.

For information about changing sbspace extents with the CREATE TABLE
statement, see “Extent sizes for smart large objects in sbspaces” on page 6-22.
Related reference:
Sbspace logging (Administrator's Guide)
CREATE TABLE statement (SQL Syntax)

Managing extents
As you add rows to a table, the database server allocates disk space in units called
extents. Each extent is a block of physically contiguous pages from the dbspace.
Even when the dbspace includes more than one chunk, each extent is allocated
entirely within a single chunk, so that it remains contiguous.

Contiguity is important to performance. When the pages of data are contiguous,

and when the database server reads the rows sequentially during read-ahead, light

Chapter 6. Table performance considerations 6-19

 scans, or lightweight I/O operations, disk-arm motion is minimized. For more
information about these operations, see “Sequential scans” on page 5-26, “Light
scans” on page 5-27, and “Configuration parameters that affect sbspace I/O” on
page 5-20.

The mechanism of extents is a compromise between the following competing

requirements:
v Most dbspaces are shared among several tables.
v The size of some tables is not known in advance.
v Tables can grow at different times and different rates.
v All the pages of a table should be adjacent for best performance.

If you have a table that needs more extents and the database server runs out of
space on the partition header page, the database server automatically allocates
extended secondary partition header pages to accommodate new extent entries.
The database server can allocate up to 32767 extents for any partition, unless the
size of a table dictates a limit to the number of extents.

Because table sizes are not known, the database server cannot preallocate table
space. Therefore, the database server adds extents only as they are needed, but all
the pages in any one extent are contiguous for better performance. In addition,
when the database server creates an extent that is next to the previous one, it treats
both as a single extent.

A frequently updated table can become fragmented over time which degrades the
performance every time the table is accessed by the server. Defragmenting a table
brings data rows closer together and avoids partition header page overflow
problems.

Choosing table extent sizes

When you create a table, you can specify extent sizes for the data rows of a table
in a dbspace and for each fragment of a fragmented table, and the smart large
objects in an sbspace. The database server calculates extent sizes for smart large
objects in sbspaces.

Extent sizes for tables in a dbspace

When you create a table, you can specify the size of the first extent, as well as the
size of the extents to be added as the table grows. You can also modify the size of
the first extent in a table in a dbspace, and you can modify the size of new
subsequent extents.

The following sample SQL statement creates a table with a 512-kilobyte initial
extent and 100-kilobyte added extents:
CREATE TABLE big_one (...column specifications...)
IN big_space
EXTENT SIZE 512
NEXT SIZE 100

The default value for the extent size and the next-extent size is eight times the disk
page size on your system. For example, if you have a 2-kilobyte page, the default
length is 16 kilobytes.

You can use the ALTER TABLE statement with the MODIFY EXTENT SIZE clause
to change the size of the first extent of a table in a dbspace. When you change the

6-20 IBM Informix Performance Guide

size of the first extent, Informix records the change in the system catalog and on
the partition page, but only makes the actual change when the table is rebuilt or a
new partition or fragment is created.

You might want to change the size of the first extent of a table in a dbspace in
either of these situations:
v If a table was created with small first extent size and you need to keep adding
a lot of next extents, the table becomes fragmented across multiple extents and
the data is scattered.
v If a table was created with a first extent that is much larger than the amount of
data that is stored, space is wasted.

The following example changes the size of the first extent of a table in a dbspace to
50 kilobytes:
ALTER TABLE customer MODIFY EXTENT SIZE 50;

Changes to the first extent size are recorded into the system catalog table and on
the partition page on the disk. However, changes to the first extent size do not take
effect immediately. Instead, whenever a change that rebuilds the table occurs, the
server uses the new first extent size.

For example, if a table has a first extent size of 8 kilobytes and you use the ALTER
TABLE statement to change this to 16 kilobytes, the server does not drop the
current first extent and recreate it with the new size. Instead, the new first extent
size of 16 kilobytes takes effect only when the server rebuilds the table after actions
such as creating a cluster index on the table or detaching a fragment from the
table.

If a TRUNCATE TABLE statement without the REUSE option is executed before

the ALTER TABLE statement with the MODIFY EXTENT SIZE clause, there is no
change in the current first extent.

Use the MODIFY NEXT SIZE clause to change the size of the next extent to be
added. This change does not affect next extents that already exist.

The following example changes the size of the next extent of a table to 50
kilobytes:
ALTER TABLE big_one MODIFY NEXT SIZE 50;

The next extent sizes of the following kinds of tables do not affect performance
significantly:
v A small table is defined as a table that has only one extent. If such a table is
heavily used, large parts of it remain buffered in memory.
v An infrequently used table is not important to performance no matter what size
it is.
v A table that resides in a dedicated dbspace always receives new extents that are
adjacent to its old extents. The size of these extents is not important because,
being adjacent, they perform as one large extent.

Avoid creating large numbers of extents

When you assign an extent size to these kinds of tables, the only consideration is
to avoid creating large numbers of extents. A large number of extents causes the
database server to spend extra time finding the data. In addition, an upper limit

Chapter 6. Table performance considerations 6-21

 exists on the number of extents allowed. (“Considering the upper limit on extents”
on page 6-24 covers this topic.)

Tips for allocating space for table extents

No upper limit exists on extent sizes except the size of the chunk. The maximum
size for a chunk is 4 terabytes. When you know the final size of a table (or can
confidently predict it within 25 percent), allocate all its space in the initial extent.
When tables grow steadily to unknown size, assign them next-extent sizes that let
them share the dbspace with a small number of extents each.

Allocating space for table extents

To allocate space for table extents:

1. Decide how to allocate space among the tables.
For example, you might divide the dbspace among three tables in the ratio 0.4:
0.2: 0.3 (reserving 10 percent for small tables and overhead).
2. Give each table one-fourth of its share of the dbspace as its initial extent.
3. Assign each table one-eighth of its share as its next-extent size.
4. Monitor the growth of the tables regularly with oncheck.

As the dbspace fills up, you might not have enough contiguous space to create an
extent of the specified size. In this case, the database server allocates the largest
contiguous extent that it can.
Related reference:
TBLTBLFIRST configuration parameter (Administrator's Reference)
TBLTBLNEXT configuration parameter (Administrator's Reference)
MODIFY EXTENT SIZE (SQL Syntax)

Extent sizes for table fragments

When you fragment an existing table, you might want to adjust the next-extent
size because each fragment requires less space than the original, unfragmented
table.

If the unfragmented table was defined with a large next-extent size, the database
server uses that same size for the next-extent on each fragment, which results in
over-allocation of disk space. Each fragment requires only a proportion of the
space for the entire table.

For example, if you fragment the preceding big_one sample table across five disks,
you can alter the next-extent size to one-fifth the original size. The following
example changes the next-extent size to one-fifth of the original size:
ALTER TABLE big_one MODIFY NEXT SIZE 2;
Related reference:
MODIFY NEXT SIZE clause (SQL Syntax)

Extent sizes for smart large objects in sbspaces

When you create a table, you should use the extent size that the database server
calculates for smart large objects in sbspaces. Alternatively, you can use the final
size of the smart large object, as indicated by a particular function when you open
the sbspace in an application program.

6-22 IBM Informix Performance Guide

 You can use the final size of the smart large object when you open one of the
following application programs:
v For DB-Access: Use the DataBlade API mi_lo_specset_estbytes function. For
more information about the DataBlade API functions to open a smart large
object and set the estimated number of bytes, see the IBM Informix DataBlade API
Programmer's Guide.
v For ESQL/C: Use the Informix ESQL/C ifx_lo_specset_estbytes function. For
more information about the Informix ESQL/C functions to open a smart large
object and set the estimated number of bytes, see the IBM Informix ESQL/C
Programmer's Manual.

For more information about sizing extents, see “Sbspace extents” on page 5-21. For
more information, see “Monitoring sbspaces” on page 6-13.

Monitoring active tblspaces

Monitor tblspaces to determine which tables are active. Active tables are those that
a thread has currently opened.

Output from the onstat -t option includes the tblspace number and the following
four fields.
Field Description
npages
Pages allocated to the tblspace
nused Pages used from this allocated pool
nextns Number of extents used
npdata
Number of data pages used

If a specific operation needs more pages than are available (npages minus nused),
a new extent is required. If enough space is available in this chunk, the database
server allocates the extent here; if not, the database server looks for space in other
available chunks. If none of the chunks contains adequate contiguous space, the
database server uses the largest block of contiguous space that it can find in the
dbspace. Figure 6-9 shows an example of the output from this option.

Tblspaces
n address flgs ucnt tblnum physaddr npages nused npdata nrows nextns
0 422528 1 1 100001 10000e 150 124 0 0 3
1 422640 1 1 200001 200004 50 36 0 0 1
54 426038 1 6 100035 1008ac 3650 3631 3158 60000 3
62 4268f8 1 6 100034 1008ab 8 6 4 60 1
63 426a10 3 6 100036 1008ad 368 365 19 612 3
64 426b28 1 6 100033 1008aa 8 3 1 6 1
193 42f840 1 6 10001b 100028 8 5 2 30 1
7 active, 200 total, 64 hash buckets

Figure 6-9. onstat -t output

Monitoring the upper limit on extents and extent interleaving

You can monitor the upper limit on the number of extents. You can also check for
and eliminate extent interleaving.

The maximum number of extents for a partition is 32767.

Chapter 6. Table performance considerations 6-23

 Considering the upper limit on extents
Do not allow a table to acquire a large number of extents because an upper limit
exists on the number of extents allowed. Trying to add an extent after you reach
the limit causes error -136 (No more extents) to follow an INSERT request.

To help ensure that the limit is not exceeded, the database server performs the
following actions:
v The database server checks the number of extents each time that it creates an
extent. If the number of the extent being created is a multiple of 16, the database
server automatically doubles the next-extent size for the table. Therefore, at
every 16th creation, the database server doubles the next-extent size.
v When the database server creates an extent next to the previous extent, it treats
both extents as a single extent.

Checking for extent interleaving

When two or more growing tables share a dbspace, extents from one tblspace can
be placed between extents from another tblspace. When this situation occurs, the
extents are said to be interleaved. Performance suffers when disk seeks for a table
must span more than one extent, particularly for sequential scans.

Interleaving creates gaps between the extents of a table. Figure 6-10 shows gaps
between table extents.

Table 1 extents
Table 2 extents
Table 3 extents

Figure 6-10. Interleaved table extents

Try to optimize the table-extent sizes to allocate contiguous disk space, which
limits head movement. Also consider placing the tables in separate dbspaces.

Check periodically for extent interleaving by monitoring chunks. Execute oncheck

-pe to obtain the physical layout of information in the chunk. The following
information appears:
v Dbspace name and owner
v Number of chunks in the dbspace
v Sequential layout of tables and free space in each chunk
v Number of pages dedicated to each table extent or free space

This output is useful for determining the degree of extent interleaving. If the
database server cannot allocate an extent in a chunk despite an adequate number
of free pages, the chunk might be badly interleaved.

Eliminating interleaved extents

You can eliminate interleaved extents by reorganizing the tables with the
UNLOAD and LOAD statements, creating or altering an index to cluster, or using
the ALTER TABLE statement.

6-24 IBM Informix Performance Guide

Reorganizing dbspaces and tables to eliminate extent interleaving:

You can rebuild a dbspace to eliminate interleaved extents so that the extents for
each table are contiguous.

The order of the reorganized tables within the dbspace is not important, but the
pages of each reorganized table should be contiguous so that no lengthy seeks are
required to read the table sequentially. When the disk arm reads a table
nonsequentially, it ranges only over the space that table occupies.

Table 1 extents
Table 2 extents
Table 3 extents

Figure 6-11. A dbspace reorganized to eliminate interleaved extents

To reorganize tables in a dbspace:

1. For DB-Access users: Copy the tables in the dbspace individually to tape with
the UNLOAD statement in DB-Access.
2. Drop all the tables in the dbspace.
3. Re-create the tables with the LOAD statement or the dbload utility.

The LOAD statement re-creates the tables with the same properties they had
before, including the same extent sizes.

You can also unload a table with the onunload utility and reload the table with the
companion onload utility.
Related concepts:
The onunload and onload utilities (Migration Guide)
Related reference:
LOAD statement (SQL Syntax)
UNLOAD statement (SQL Syntax)

Creating or altering an index to cluster:

Depending on the circumstances, you can eliminate extent interleaving if you

create a clustered index or alter a clustered index. When you use the TO CLUSTER
clause of the CREATE INDEX or ALTER INDEX statement, the database server
sorts and reconstructs the table.

The TO CLUSTER clause reorders rows in the physical table to match the order in
the index. For more information, see “Clustering” on page 7-11.

The TO CLUSTER clause eliminates interleaved extents under the following

conditions:
v The chunk must contain contiguous space that is large enough to rebuild each
table.
v The database server must use this contiguous space to rebuild the table.

Chapter 6. Table performance considerations 6-25

 If blocks of free space exist before this larger contiguous space, the database
server might allocate the smaller blocks first. The database server allocates space
for the ALTER INDEX process from the beginning of the chunk, looking for
blocks of free space that are greater than or equal to the size that is specified for
the next extent. When the database server rebuilds the table with the smaller
blocks of free space that are scattered throughout the chunk, it does not
eliminate extent interleaving.

To display the location and size of the blocks of free space, execute the oncheck
-pe command.

To use the TO CLUSTER clause of the ALTER INDEX statement:

1. For each table in the chunk, drop all fragmented or detached indexes except the
one that you want to cluster.
2. Cluster the remaining index with the TO CLUSTER clause of the ALTER
INDEX statement. This step eliminates interleaving the extents when you
rebuild the table by rearranging the rows.
3. Re-create all the other indexes.

You do not need to drop an index before you cluster it. However, the ALTER
INDEX process is faster than CREATE INDEX because the database server reads
the data rows in cluster order using the index. In addition, the resulting indexes
are more compact.

To prevent the problem from recurring, consider increasing the size of the tblspace
extents.

Using ALTER TABLE to eliminate extent interleaving:

If you use the ALTER TABLE statement to add or drop a column or to change the
data type of a column, the database server copies and reconstructs the table. When
the database server reconstructs the entire table, it rewrites the table to other areas
of the dbspace. However, if other tables are in the dbspace, no guarantee exists
that the new extents will be adjacent to each other.

Important: For certain types of operations that you specify in the ADD, DROP,
and MODIFY clauses, the database server does not copy and reconstruct the table
during the ALTER TABLE operation. In these cases, the database server uses an
in-place alter algorithm to modify each row when it is updated (rather than during
the ALTER TABLE operation). For more information about the conditions for this
in-place alter algorithm, see “In-place alter” on page 6-35.

Reclaiming unused space within an extent

After the database server allocates disk space to a tblspace as part of an extent,
that space remains dedicated to the tblspace. Even if all extent pages become
empty after you delete data, the disk space remains unavailable for use by other
tables unless you reclaim the space.

Important: When you delete rows in a table, the database server reuses that space
to insert new rows into the same table. This section describes the procedures for
reclaiming unused space for use by other tables.

You might want to resize a table that does not require the entire amount of space
that was originally allocated to it. You can reallocate a smaller dbspace and release
the unneeded space for other tables to use.

6-26 IBM Informix Performance Guide

 As the database server administrator, you can reclaim the disk space in empty
extents and make it available to other users by rebuilding the table. To rebuild the
table, use any of the following SQL statements:
v ALTER INDEX
v UNLOAD and LOAD
v ALTER FRAGMENT

Reclaiming space in an empty extent with ALTER INDEX

If the table with the empty extents includes an index, you can run the ALTER
INDEX statement with the TO CLUSTER clause. Clustering an index rebuilds the
table in a different location within the dbspace.

When you run the ALTER INDEX statement with the TO CLUSTER clause, all of
the extents associated with the previous version of the table are released. Also, the
newly built version of the table has no empty extents.
Related concepts:
Clustering (Performance Guide)
Related reference:
ALTER INDEX statement (SQL Syntax)

Reclaiming space in an empty extent by unloading and

re-creating or reloading a table
If the table does not include an index, you can unload the table, re-create the table
(either in the same dbspace or in another one), and reload the data with the
UNLOAD and LOAD statements or the onunload and onload utilities.
Related concepts:
The onunload and onload utilities (Migration Guide)
Related reference:
LOAD statement (SQL Syntax)
UNLOAD statement (SQL Syntax)

Releasing space in an empty extent with ALTER FRAGMENT

You can use the ALTER FRAGMENT statement to rebuild a table. When you run
this statement, it releases space within the extents that were allocated to that table.

For more information about the syntax of the ALTER FRAGMENT statement, see
the IBM Informix Guide to SQL: Syntax.

Managing extent deallocation with the TRUNCATE keyword

TRUNCATE is an SQL keyword that quickly deletes active rows from a table and
the b-tree structures of its indexes, without dropping the table or its schema, access
privileges, triggers, constraints, and other attributes. With this SQL data-definition
language statement, you can depopulate a local table and reuse the table without
re-creating it, or you can release the storage space that formerly held its data rows
and b-tree structures.

Two implementations of TRUNCATE exist:

v The first implementation, called "fast truncate," operates on most tables.
v The second implementation, called "slow truncate," operates on tables that
include opaque or smart large object data types, or inherited indexes that are
defined on ROW types within data type hierarchies.
Chapter 6. Table performance considerations 6-27
 The performance advantages of using the TRUNCATE TABLE statement instead of
the DELETE statement are much better for the fast truncate implementation,
because this implementation does not examine or run all of the rows in a table.
Slow truncation implementation occurs on tables that include opaque or smart
large object data types or inherited indexes that are defined on ROW types within
data types, because the truncate operation examines each row containing these
items.

For more information about using TRUNCATE, see the IBM Informix Guide to SQL:
Syntax.

Defragment partitions to merge extents

You can improve performance by defragmenting partitions to merge
non-contiguous extents.

A frequently updated table can become fragmented over time which degrades the
performance every time the table is accessed by the server. Defragmenting a table
brings data rows closer together and avoids partition header page overflow
problems.

Defragmenting an index brings the entries closer together which improves the
speed at which the table information is accessed.

You cannot stop a defragment request after the request has been submitted.
Additionally, there are specific objects that cannot be defragmented and you cannot
defragment a partition if another operation is running that conflicts with the
defragment request.

Tip: Before you defragment a partition:

v Review the information about important limitations and considerations in
Defragment partitions.
v Run the oncheck -pt and pT command to determine the number of extents for a
specific table or fragment.

To defragment a table, index, or partition, run the EXECUTE FUNCTION

command with the defragment argument. You can specify the table name, index
name, or partition number that you want to defragment.

You can use the onstat -g defragment command to display information about the
active defragment requests.
Related tasks:
Scheduling data optimization (Administrator's Guide)
Related reference:
onstat -g defragment command: Print defragment partition extents
(Administrator's Reference)
oncheck -pt and -pT: Display tblspaces for a Table or Fragment
(Administrator's Reference)
defragment argument: Dynamically defragment partition extents (SQL
administration API) (Administrator's Reference)

6-28 IBM Informix Performance Guide

Storing multiple table fragments in a single dbspace
You can store multiple fragments of the same table or index in a single dbspace,
thus reducing the total number of dbspaces needed for a fragmented table. You
must specify a name for each fragment that you want to store in the same dbspace.
Storing multiple table or index fragments in a single dbspace simplifies the
management of dbspaces.

You can also use this feature to improve query performance over storing each
fragment in a different dbspace when a dbspace is located on a faster device.

For more information, see information about managing partitions in the IBM
Informix Administrator's Guide.

Displaying a list of table and index partitions

Use the onstat -g opn option to display a list of the table and index partitions, by
thread ID, that are currently open in the system.

For an example of onstat -g opn output and an explanation of output fields, see
the IBM Informix Administrator's Reference.

Changing tables to improve performance

You can change tables to improve performance by dropping indexes, attaching or
detaching fragments, and altering table definitions. You can also create databases
for decision-support applications by unloading and loading tables in OLTP
databases.

You might want to change an existing table for various reasons:

v To refresh large decision-support tables with data periodically
v To add or drop historical data from a certain time period
v To add, drop, or modify columns in large decision-support tables when the need
arises for different data analysis

Loading and unloading tables

You can create databases for decision-support applications by periodically loading
tables that have been unloaded from active OLTP databases.

You can use one or more of the following methods to load large tables quickly:
v External tables
v Nonlogging tables
The database server provides support to:
– Create nonlogging or logging tables in a logging database.
– Alter a table from nonlogging to logging and vice versa.
The two table types are STANDARD (logging tables) and RAW (nonlogging
tables). You can use any loading utility such as dbimport or HPL to load raw
tables.
v High-Performance Loader (HPL)
You can use HPL in express mode to load tables quickly.

The following sections describe:

Chapter 6. Table performance considerations 6-29

 v Advantages of logging and nonlogging tables
v Step-by-step procedures to load data using nonlogging tables
Related tasks:
Moving data with external tables (Administrator's Guide)
Related reference:
CREATE EXTERNAL TABLE Statement (SQL Syntax)

Advantages of logging tables

Logging type options specify the logging characteristics that can improve
performance in various bulk operations on the table.

STANDARD, which corresponds to a table in a logged database of previous

versions, is the default logging type that is used when you issue the CREATE
TABLE statement without specifying the table type.

Standard tables have the following features:

v Logging to allow rollback, recovery, and restoration from archives.
v Recovery from backups
v All insert, delete, and update operations
v Constraints to maintain the integrity of your data
v Indexes to quickly retrieve a small number of rows

OLTP applications usually use standard tables. OLTP applications typically have
the following characteristics:
v Real-time insert, update, and delete transactions
Logging and recovery of these transactions is critical to preserve the data.
Locking is critical to allow concurrent