MIMIX Operations 5250
MIMIX Operations 5250
MIMIX Operations—5250
Notices
3
Setting polices for MIMIX Switch Assistant ......................................................... 49
Setting policies when MIMIX Model Switch Framework is not used.................... 49
Policy descriptions..................................................................................................... 50
Chapter 3 Scheduling audits and reports 60
How audits and reports are submitted automatically................................................. 60
Scheduling criteria you can change for audits and reports ................................. 61
Default schedules for automatically submitted audits ............................................... 64
Default schedules for automatically submitted reports.............................................. 67
Changing when automatic audits are allowed to run................................................. 68
Changing scheduling criteria for automatic audits............................................... 68
Changing the selection frequency of priority auditing categories ........................ 69
Disabling or enabling automatically submitted audits.......................................... 70
Changing when automatic reports are allowed to run ............................................... 72
Changing scheduling criteria for automatic reports ............................................. 72
Disabling or enabling automatically submitted reports ........................................ 73
Chapter 4 Where to check status 75
Checking status in application group environments .................................................. 75
Checking status in data group-only environments .................................................... 78
Chapter 5 Checking application group status 80
Displaying application group status........................................................................... 80
Resolving problem statuses for an application group................................................ 82
Resolving a procedure status problem reported for an application group ........... 82
Resolving an *ATTN status for an application group........................................... 83
Resolving other common status values for an application group ........................ 84
Resolving problems reported in the Monitors field .............................................. 85
Resolving status for node entries .............................................................................. 87
Resolving status for data resource group entries...................................................... 88
Verifying the sequence of the recovery domain ........................................................ 90
Changing the sequence of backup nodes ................................................................. 91
Examples of changing the backup sequence ...................................................... 93
Chapter 6 Working with procedures and steps 97
Displaying status of procedures ................................................................................ 98
Responding to a procedure in *MSGW status......................................................... 101
Resolving a *FAILED or *CANCELED procedure status......................................... 102
Displaying status of steps within a procedure run ................................................... 103
Resolving problems with step status ....................................................................... 106
Responding to a step with a *MSGW status ..................................................... 108
Resolving *CANCEL or *FAILED step statuses ................................................ 109
Acknowledging a procedure .................................................................................... 110
Running a procedure............................................................................................... 111
Resuming a procedure ...................................................................................... 112
Overriding the attributes of a step ............................................................... 112
Canceling a procedure ............................................................................................ 114
Displaying scheduling information for automatic reports......................................... 114
Displaying available status history of procedure runs ............................................. 115
4
Chapter 7 Working with data group status 117
Displaying data group status ................................................................................... 118
Identifying and resolving problems reflected in the Data Group column ........... 120
Resolving problems highlighted in the Data Group column......................... 121
Identifying and resolving replication problems reflected in the Source and Target
columns ................................................................................................................... 123
Setting the automatic refresh interval ................................................................ 125
Displaying detailed status for a data group ............................................................. 126
Merged view ...................................................................................................... 126
Object detailed status views .............................................................................. 131
Database detailed status views ......................................................................... 133
Identifying replication processes with backlogs....................................................... 137
Data group status in environments with journal cache or journal state ................... 139
Resolving a problem with journal cache or journal state ................................... 141
Resolving a problem for a journal status of *DIFFJRN...................................... 143
Resolving a deadlock condition for apply processes............................................... 144
Chapter 8 Working with audits 145
Auditing overview .................................................................................................... 146
Components of an audit .................................................................................... 146
Phases of audit processing ............................................................................... 147
Object selection methods for automatic audits.................................................. 147
How priority auditing determines what objects to select.............................. 148
Audit status and results ..................................................................................... 149
Audit compliance ............................................................................................... 150
Guidelines and considerations for auditing ............................................................. 151
Auditing best practices ...................................................................................... 151
Considerations for specific audits...................................................................... 152
Recommendations when checking audit results ............................................... 153
Displaying scheduling information for automatic audits .......................................... 155
Displaying audit runtime status and compliance status........................................... 157
Audits - Summary view and alternate columns ................................................. 157
Audits - Compliance summary view and alternate columns .............................. 160
Resolving auditing problems ................................................................................... 162
Resolving audit runtime status problems .......................................................... 162
Checking the job log of an audit ........................................................................ 166
Resolving audit compliance status problems .................................................... 167
Running an audit immediately ................................................................................. 169
Ending an audit ....................................................................................................... 170
Displaying audit history ........................................................................................... 171
Prioritized audit runs with no selected objects .................................................. 173
Working with audited objects................................................................................... 173
Displaying audited objects from a specific audit run ......................................... 175
Displaying a customized list of audited objects ................................................. 175
Working with audited object history......................................................................... 176
Displaying the audit history for a specific object................................................ 177
Chapter 9 Working with system-level processes 178
Displaying status of system-level processes........................................................... 180
Resolving system manager problems and backlogs ............................................... 181
5
Resolving ASP 1 storage exceeded threshold problems ........................................ 183
Starting system managers or a journal manager .................................................... 184
Ending system managers........................................................................................ 184
Ending a journal manager ....................................................................................... 185
Starting collector services ....................................................................................... 185
Ending collector services......................................................................................... 186
Starting target journal inspection processes ........................................................... 186
Ending target journal inspection processes............................................................. 187
Displaying status of target journal inspection .......................................................... 188
Displaying results of target journal inspection ......................................................... 189
Displaying details associated with target journal inspection notifications.......... 190
Displaying messages for TGTJRNINSP notifications.................................. 190
Identifying the last entry inspected on the target system ........................................ 191
Displaying status of monitors on the local system................................................... 192
Starting a monitor .................................................................................................... 194
Ending a monitor ..................................................................................................... 195
Starting and ending the master monitors ................................................................ 195
Starting the master monitor on the local system ............................................... 196
Ending the master monitor on the local system ................................................ 196
Enabling or disabling a monitor ............................................................................... 197
Enabling a monitor ............................................................................................ 197
Disabling a monitor............................................................................................ 197
Displaying status of a remote journal link................................................................ 199
Starting a remote journal link independently ........................................................... 202
Ending a remote journal link independently ............................................................ 202
Chapter 10 Working with notifications and recoveries 204
What are notifications and recoveries ..................................................................... 204
Displaying notifications............................................................................................ 205
What information is available for notifications ................................................... 206
Detailed information..................................................................................... 207
Options for working with notifications ................................................................ 208
Displaying IFS entries associated with notifications from INSTALL .................. 208
Notifications for newly created objects .................................................................... 210
Displaying recoveries .............................................................................................. 211
What information is available for recoveries in the native user interface........... 212
Detailed information..................................................................................... 212
Options for working with recoveries in the native user interface ....................... 213
Orphaned recoveries in the native user interface.............................................. 213
Determining whether a recovery is orphaned.............................................. 214
Removing an orphaned recovery ................................................................ 214
Chapter 11 Starting and ending replication 215
Before starting replication........................................................................................ 217
Commands for starting replication........................................................................... 217
What is started with the STRMMX command.................................................... 217
STRMMX and ENDMMX messages............................................................ 218
What is started by the default START procedure for an application group ....... 218
Choices when starting or ending an application group...................................... 218
What occurs when a data group is started .............................................................. 221
6
Journal starting point identified on the STRDG request .................................... 222
Journal starting point when the object send process is shared ................... 222
Clear pending and clear error processing ......................................................... 223
Starting MIMIX......................................................................................................... 226
Starting an application group................................................................................... 227
Starting selected data group processes .................................................................. 228
Starting replication when open commit cycles exist ................................................ 230
Checking for open commit cycles...................................................................... 230
Resolving open commit cycles .......................................................................... 230
Before ending replication......................................................................................... 232
Commands for ending replication............................................................................ 232
Command choice by reason for ending replication ........................................... 232
Additional considerations when ending replication............................................ 234
Ending immediately or controlled ...................................................................... 234
Controlling how long to wait for a controlled end to complete ..................... 235
Ending all or selected processes....................................................................... 235
When to end RJ links ........................................................................................ 236
What is ended by the ENDMMX command ....................................................... 236
What is ended by the default END procedure for an application group ............ 237
What occurs when a data group is ended ............................................................... 238
Ending MIMIX.......................................................................................................... 240
Ending with default values................................................................................. 240
Ending by prompting the ENDMMX command.................................................. 240
After you end MIMIX products ........................................................................... 241
Ending an application group.................................................................................... 243
Ending a data group in a controlled manner ........................................................... 244
Preparing for a controlled end of a data group .................................................. 244
Performing the controlled end ........................................................................... 244
Confirming the end request completed without problems ................................. 245
Ending selected data group processes ................................................................... 247
Enabling MIMIX-disabled target constraints............................................................ 248
Enabling constraints for an *INACTIVE data group........................................... 249
Enabling constraints when ending a data group ............................................... 249
What replication processes are started by the STRDG command.......................... 251
What replication processes are ended by the ENDDG command .......................... 255
Chapter 12 Resolving common replication problems 259
Replication problems that MIMIX attempts to recover............................................. 260
Replication errors handled by automatic database recovery ............................ 260
Replication errors handled by automatic object recovery.................................. 261
Working with user journal replication errors ............................................................ 263
Working with files needing attention (replication and access path errors)......... 263
Working with journal transactions for files in error....................................... 266
Placing a file on hold ......................................................................................... 267
Ignoring a held file ............................................................................................. 267
Releasing a held file at a synchronization point ................................................ 268
Releasing a held file .......................................................................................... 268
Releasing a held file and clearing entries.......................................................... 269
Correcting file-level errors ................................................................................. 269
Correcting record-level errors............................................................................ 270
7
Record written in error ................................................................................. 270
Working with tracking entries .................................................................................. 272
Accessing the appropriate tracking entry display .............................................. 272
Holding journal entries associated with a tracking entry ................................... 274
Ignoring journal entries associated with a tracking entry................................... 275
Waiting to synchronize and release held journal entries for a tracking entry .... 275
Releasing held journal entries for a tracking entry ............................................ 276
Releasing and clearing held journal entries for a tracking entry........................ 276
Removing a tracking entry................................................................................. 276
Related tracking entries..................................................................................... 277
Working with objects in error ................................................................................... 278
Using the Work with DG Activity Entries display ............................................... 279
Retrying data group activity entries ................................................................... 281
Retrying a failed data group activity entry ................................................... 281
Determining whether an activity entry is in a delay/retry cycle .......................... 282
Working with message queues ............................................................................... 283
Working with the message log ................................................................................ 284
Removing data group activity history entries........................................................... 285
Chapter 13 Starting, ending, and verifying journaling 286
What objects need to be journaled.......................................................................... 287
Authority requirements for starting journaling.................................................... 288
MIMIX commands for starting journaling................................................................. 289
Forcing objects to use the configured journal.................................................... 290
Journaling for physical files ..................................................................................... 291
Displaying journaling status for physical files .................................................... 291
Starting journaling for physical files ................................................................... 291
Ending journaling for physical files .................................................................... 292
Verifying journaling for physical files ................................................................. 293
Journaling for IFS objects........................................................................................ 294
Displaying journaling status for IFS objects ...................................................... 294
Starting journaling for IFS objects ..................................................................... 294
Ending journaling for IFS objects ...................................................................... 295
Verifying journaling for IFS objects.................................................................... 296
Journaling for data areas and data queues............................................................. 298
Displaying journaling status for data areas and data queues............................ 298
Starting journaling for data areas and data queues .......................................... 298
Ending journaling for data areas and data queues............................................ 299
Verifying journaling for data areas and data queues ......................................... 300
Chapter 14 Switching 302
About switching ....................................................................................................... 303
Planned switch - overview ................................................................................. 303
Unplanned switch - overview............................................................................. 304
Switching application group environments with procedures.............................. 305
Switching data group environments with MIMIX Model Switch Framework ...... 307
Switching an application group................................................................................ 308
Switching a data group-only environment ............................................................... 309
Switching to the backup system ........................................................................ 309
Synchronizing data and starting MIMIX on the original production system ....... 310
8
Switching to the production system ................................................................... 310
Determining when the last switch was performed ................................................... 311
Performing a data group switch............................................................................... 312
Switch Data Group (SWTDG) command................................................................. 314
Chapter 15 Support for virtual switching 316
What is virtual switching? ........................................................................................ 316
MIMIX virtual switch overview ................................................................................. 317
MIMIX environment considerations for virtual switching ......................................... 319
Testing considerations for virtual switching............................................................. 321
Performing a precheck for virtual switching............................................................. 323
Performing a virtual switch ...................................................................................... 323
Chapter 16 Less common operations 327
Starting the TCP/IP server ...................................................................................... 328
Ending the TCP/IP server........................................................................................ 329
Working with objects ............................................................................................... 330
Displaying long object names............................................................................ 330
Considerations for working with long IFS path names ................................ 330
Displaying data group spooled file information.................................................. 330
Viewing status for active file operations .................................................................. 331
Identifying data groups that use an RJ link ............................................................. 332
Identifying journal definitions used with RJ ............................................................. 333
Disabling and enabling data groups ........................................................................ 334
Disabling a data group ...................................................................................... 335
Enabling a data group ....................................................................................... 335
Determining if non-file objects are configured for user journal replication............... 336
Determining how IFS objects are configured .................................................... 336
Determining how data areas or data queues are configured ............................ 337
Using file identifiers (FIDs) for IFS objects .............................................................. 338
Chapter 17 Troubleshooting - where to start 339
Reducing contention between MIMIX and user applications................................... 341
Data groups cannot be ended ................................................................................. 342
Verifying a communications link for system definitions ........................................... 343
Verifying the communications link for a data group................................................. 344
Verifying all communications links..................................................................... 344
Checking file entry configuration manually.............................................................. 345
Data groups cannot be started ................................................................................ 347
Cannot start or end an RJ link................................................................................. 348
Removing unconfirmed entries to free an RJ link.............................................. 348
RJ link active but data not transferring .................................................................... 349
Errors using target journal defined by RJ link.......................................................... 350
Verifying data group file entries............................................................................... 351
Verifying key attributes ............................................................................................ 351
Working with data group timestamps ...................................................................... 352
Automatically creating timestamps .................................................................... 352
Creating additional timestamps ......................................................................... 352
Creating timestamps for remote journaling processing ..................................... 353
Deleting timestamps .......................................................................................... 354
Displaying or printing timestamps ..................................................................... 354
9
Removing journaled changes.................................................................................. 355
Performing journal analysis ..................................................................................... 356
Removing journal analysis entries for a selected file ........................................ 358
Appendix A Interpreting audit results - supporting information 360
Interpreting results for configuration data - #DGFE audit........................................ 361
When the difference is “not found” .......................................................................... 363
Interpreting results of audits for record counts and file data ................................... 364
What differences were detected by #FILDTA.................................................... 364
What differences were detected by #MBRRCDCNT ......................................... 365
Interpreting results of audits that compare attributes .............................................. 367
What attribute differences were detected .......................................................... 367
Where was the difference detected................................................................... 369
What attributes were compared ........................................................................ 370
Appendix B IBM Power™ Systems operations that affect MIMIX 371
MIMIX procedures when performing an initial program load (IPL) .......................... 372
MIMIX procedures when performing an operating system upgrade........................ 374
Prerequisites for performing an OS upgrade on either system ......................... 375
MIMIX-specific steps for an OS upgrade on a backup system.......................... 375
MIMIX-specific steps for an OS upgrade on a production system with switching ...
377
MIMIX-specific steps for an OS upgrade on the production system without switch-
ing............................................................................................................................ 379
MIMIX procedures when upgrading hardware without a disk image change .......... 382
Considerations for performing a hardware system upgrade without a disk image
change..................................................................................................................... 382
MIMIX-specific steps for a hardware upgrade without a disk image change..... 382
Hardware upgrade without a disk image change - preliminary steps .......... 382
Hardware upgrade without a disk image change - subsequent steps ......... 383
MIMIX procedures when performing a hardware upgrade with a disk image change...
385
Considerations for performing a hardware system upgrade with a disk image
change..................................................................................................................... 385
MIMIX-specific steps for a hardware upgrade with a disk image change.......... 386
Hardware upgrade with a disk image change - preliminary steps ............... 386
Hardware upgrade with a disk image change - subsequent steps .............. 387
Handling MIMIX during a system restore ................................................................ 391
Prerequisites for performing a restore of MIMIX ............................................... 391
Index 392
10
Who this book is for
11
problems in replication environments for one or more instances of MIMIX® DR,
MIMIX® Enterprise™ or MIMIX® Professional™. The modern web browser
interfaces available through Vision Solutions Portal (VSP) and the MIMIX portal
application provide a better user experience for MIMIX and are required to use
this book. Some topics may not be applicable to all MIMIX products.
MIMIX Operations with IBM PowerHA for i
This book is for administrators and operators in an IBM i clustering environment
who use MIMIX® for PowerHA® to integrate cluster management with MIMIX
logical replication or supported hardware-based replication techniques. This book
focuses on addressing problems reported in MIMIX status and basic operational
procedures such as starting, ending, and switching.
MIMIX Operations - 5250
This book provides high level concepts and operational procedures for managing
your high availability environment using MIMIX products from a native user
interface. This book focuses on tasks typically performed by an operator, such as
checking status, starting or stopping replication, performing audits, and basic
problem resolution.
Using MIMIX Monitor
This book describes how to use the MIMIX Monitor user and programming
interfaces available with MIMIX products. This book also includes programming
information about MIMIX Model Switch Framework.
Using MIMIX Promoter
This book describes how to use MIMIX commands for copying and reorganizing
active files. MIMIX Promoter functionality is included with MIMIX® Enterprise™.
MIMIX for IBM WebSphere MQ
This book identifies requirements for the MIMIX for MQ feature which supports
replication in IBM WebSphere MQ environments. This book describes how to
configure MIMIX for this environment and how to perform the initial
synchronization and initial startup. Once configured and started, all other
operations are performed as described in the MIMIX Operations - 5250 book.
MIMIX DR limitations
Environments that replicate in one direction between only two systems may not
require the features offered by a high availability product. Instead, a disaster recovery
solution that enables data to be readily available is all that is needed. MIMIX® DR
features many of the capabilities found in Vision Solutions high availability products,
with the following specifications that make it strictly a disaster recovery solution:
• Processor group - MIMIX DR is limited to P05 and P10 systems with 1 to 4 CPUs
(cores). The backup system can be any P-Group model.
• MIMIX Instance - Only one MIMIX Instance is allowed on each participating
system. A MIMIX DR installation comprises two systems that transfer data and
objects between them. Both systems within the instance use the same name for
12
MIMIX DR limitations
13
Sources for additional information
This book refers to other published information. The following information, plus
additional technical information, can be located in the IBM Knowledge Center.
From the Information center you can access these IBM Power™ Systems topics,
books, and redbooks:
• Backup and Recovery
• Journal management
• DB2 Universal Database for IBM Power™ Systems Database Programming
• Integrated File System Introduction
• Independent disk pools
• TCP/IP Setup
• IBM redbook Striving for Optimal Journal Performance on DB2 Universal
Database for iSeries, SG24-6286
• IBM redbook AS/400 Remote Journal Function for High Availability and Data
Replication, SG24-5189
• IBM redbook Power™ Systems iASPs: A Guide to Moving Applications to
Independent ASPs, SG24-6802
The following information may also be helpful if you replicate journaled data areas,
data queues, or IFS objects:
• DB2 UDB for iSeries SQL Programming Concepts
• DB2 Universal Database for iSeries SQL Reference
• IBM redbook AS/400 Remote Journal Function for High Availability and Data
Replication, SG24-5189
14
How to contact us
How to contact us
For contact information, visit our Contact CustomerCare web page.
If you are current on maintenance, support for MIMIX products is also available when
you log in to Support Central.
It is important to include product and version information whenever you report
problems.
15
MIMIX overview
This book provides operational information and procedures for using MIMIX®
Enterprise™, MIMIX® Professional™, or MIMIX® DR through the native user interface
user interface. For simplicity, this book uses the term MIMIX to refer to the
functionality provided unless a more specific name is necessary. Some topics may not
apply to all products.
This MIMIX version provides high availability for your critical data in a production
environment on IBM Power™ Systems through real-time replication of changes and
the ability to quickly switch your production environment to a ready backup system.
These capabilities allow your business operations to continue when you have planned
or unplanned outages in your System i environment. MIMIX also provides advanced
capabilities that can help ensure the integrity of your MIMIX environment.
Replication: MIMIX continuously captures changes to critical database files and
objects on a production system, sends the changes to a backup system, and applies
the changes to the appropriate database file or object on the backup system. The
backup system stores exact duplicates of the critical database files and objects from
the production system.
MIMIX uses two replication paths to address different pieces of your replication
needs. These paths operate with configurable levels of cooperation or can operate
independently.
• The user journal replication path captures changes to critical files and objects
configured for replication through a user journal. When configuring this path,
shipped defaults use the remote journaling function of the operating system to
simplify sending data to the remote system. In previous versions, MIMIX DB2
Replicator provided this function.
• The system journal replication path handles replication of critical system objects
(such as user profiles, program objects, or spooled files), integrated file system
(IFS) objects, and document library object (DLOs) using the system journal. In
previous versions MIMIX Object Replicator provided this function.
Configuration choices determine the degree of cooperative processing used between
the system journal and user journal replication paths when replicating database files,
IFS objects, data areas, and data queues.
Switching: One common use of MIMIX is to support a hot backup system to which
operations can be switched in the event of a planned or unplanned outage. If a
production system becomes unavailable, its backup is already prepared for users. In
the event of an outage, you can quickly switch users to the backup system where they
can continue using their applications. MIMIX captures changes on the backup system
for later synchronization with the original production system. When the original
production system is brought back online, MIMIX assists you with analysis and
synchronization of the database files and other objects.
Automatic verification and correction: MIMIX assists in maintaining availability and
switch-readiness in your replication environment by automatically detecting problems
16
during replication, verifying replicated objects as well as configuration integrity with
audits, and automatically attempting to correct any problems detected.
MIMIX is shipped with these capabilities enabled. Incorporated best practices for
maintaining availability and switch-readiness are key to ensuring that your MIMIX
environment is in tip-top shape for protecting your data. User interfaces allow you to
fine-tune to the needs of your environment.
Analysis: MIMIX also provides advanced analysis capabilities through the MIMIX
portal application for Vision Solutions Portal (VSP). When using the VSP user
interface from a web browser, you can view a list of the objects replicated by MIMIX
and a list of objects with detected problems or changes for which automatic recovery
actions have been requested or are in progress. You can use reports to view which
objects on a system are and are not identified to MIMIX configuration. You can also
check historical arrival and backlog rates for replication to help you identify trends in
your operations that may affect MIMIX performance.
Uses: MIMIX is typically used among systems in a network to support a hot backup
system. Simple environments have one production system and one backup system.
More complex environments have multiple production systems or backup systems.
MIMIX can also be used on a single system.
You can view the replicated data on the backup system at any time without affecting
productivity. This allows you to generate reports, submit (read-only) batch jobs, or
perform backups to tape from the backup system. In addition to real-time backup
capability, replicated databases and objects can be used for distributed processing,
allowing you to off-load applications to a backup system.
The topics in this chapter include:
• “MIMIX concepts” on page 18 summarizes key concepts that you need to know
about MIMIX.
• “Best practices for maintaining your MIMIX environment” on page 24 summarizes
recommendations from Vision Solutions.
• “Authority to products and commands” on page 24 identifies authority levels to
MIMIX functions when additional security features provided by Vision Solutions
are used.
• “Accessing the MIMIX Main Menu” on page 26 describes the MIMIX Basic Main
menu and the MIMIX Intermediate Main Menu.
17
MIMIX concepts
The following subtopics organize the basic concepts associated with MIMIX® into
related groups. More detailed information is available in the MIMIX Administrator
Reference book.
Product concepts
MIMIX installation - The network of IBM Power™ Systems systems that transfer data
and objects among each other using functions of a common MIMIX product. A MIMIX
installation is defined by the way in which you configure the MIMIX product for each of
the participating systems. A system can participate in multiple independent MIMIX
installations.
Replication - The activity that MIMIX performs to continuously capture changes to
critical database files and objects on a production system as they occur, send the
changes to a backup system, and apply the changes to the appropriate database file
or object on the backup system.
Switch - The process by which a production environment is moved from one system
to another system and the production environment is made available there. A switch
may be performed as part of a planned event such as for system maintenance, or an
unplanned event such as a power or equipment failure. MIMIX provides customizable
functions for switching.
Audits - Audits are predetermined programs that are used to check for differences in
replicated objects and other conditions between systems. Audits run and can correct
detected problems automatically. Policies control when audits run and many other
aspects of how audits are performed. Additional auditing concepts and
recommendations are described in the auditing chapter of this book.
Automatic recovery - MIMIX provides a set of functions that are able to detect and
automatically correct problems in a MIMIX installation. Processes that can detect
problems and initiate automatic recovery actions include database replication, object
replication, target journal inspection, audits, and virtual switch procedures. Through
policies, you have the ability to disable automatic recovery at the installation or data
group level.
Application group - A MIMIX construct used to group and control resources from a
single point in a way that maintains relationships between them. The use of
application groups is best practice for MIMIX Professional and MIMIX Enterprise and
required for MIMIX for PowerHA.
Data group - A MIMIX construct that is used to control replication activities. A data
group is a logical grouping of database files, data areas, objects, IFS objects, DLOs,
or a combination thereof that defines a unit of work by which MIMIX replication activity
is controlled. A data group may represent an application, a set of one or more
libraries, or all of the critical data on a given system. Application environments may
define a data group as a specific set of files and objects.
Prioritized status - MIMIX assigns a priority to status values to ensure that problems
with the highest priorities, those for detected problems or situations that require
18
MIMIX concepts
immediate attention or intervention, are reflected on the highest level of the user
interface. Additional detail and lower priority items can be viewed by drilling down to
the next level within the interfaces. Those interfaces are the Work with Systems
display and depending on your configuration, either the Work with Application Groups
display or the Work with Data Groups display.
Policies - A policy is a mechanism used to enable, disable, or provide input to a
function such as replication, auditing, or MIMIX Model Switch Framework. For most
policies, the initially shipped values apply to an installation. However, policies can be
changed and most can also be overridden for individual data groups. Policies that
control when audits are automatically performed can be set only for each specific
combination of audit rule and data group.
Notifications - A notification is the resulting automatic report associated with an
event that has already occurred. The severity of a notification is reflected in the
overall status of the installation. Notifications can be generated by a process,
program, command, or monitor. Because the originator of notifications varies, it is
important to note that notifications can represent both real-time events as well as
events that occurred in the past but, due to scheduling, are being reported in the
present.
Recoveries - The term recovery is used in two ways. The most common use refers to
the recovery action taken by a MIMIX process to correct a detected difference when
automatic recovery polices are enabled. The second use refers to a temporary report
that provides details about a recovery action in progress that is created when the
recovery action starts and is removed when it completes.
19
Information from the journal entries is either replicated to the target system or used to
identify objects to be replicated to the target system.
A target system is the system on which MIMIX replication activity between two
systems completes.
Management system and network system - These terms define the role of a
system relative to how the products interact within a MIMIX installation. These roles
remain associated with the system within the MIMIX installation to which they are
defined. One system in the MIMIX installation is designated as the management
system and the remaining one or more systems are designated as network systems.
A management system is the system in a MIMIX installation that is designated as the
control point for all installations of the product within the MIMIX installation. The
management system is the location from which work to be performed by the product
is defined and maintained. Often the system defined as the management system also
serves as the backup system during normal operations.
A network system is any system in a MIMIX installation that is not designated as the
management system (control point) of that MIMIX installation. Work definitions are
automatically distributed from the management system to a network system. Often a
system defined as a network system also serves as the production system during
normal operations.
Journaling concepts
MIMIX uses journaling to perform replication and to support newer analysis
functionality.
Journaling and object auditing - Journaling and object auditing are techniques that
allow object activity to be logged to a journal. Journaling logs activity for selected
objects of specific object types to a user journal. Object auditing logs activity for all
objects to the security audit journal (QAUDJRN, the system journal), including those
defined to a user journal. MIMIX relies on these techniques and the entries placed in
the journal receivers for replicating logged activity.
Journal - An IBM i system object that identifies the objects being journaled and the
journal receivers associated with the journal. The system journal is a specialized
journal on the system which MIMIX uses.
Journal receiver - An IBM i system object that is associated with a journal and
contains the log of all activity for objects defined to the journal.
Journal entry - A record added to a journal receiver that identifies an event that
occurred on a journaled object. MIMIX uses file and record level journal entries to
recreate the object on a designated system.
Remote journaling - A function of IBM i that allows you to establish journals and
journal receivers on one system and associate them with specific journals and journal
receivers on another system. Once the association is established, the operating
system can use the pair of journals to replicate journal entries in one direction, from
the local journal to the remote journal on the other system. MIMIX uses remote
journaling to transfer replicated data from the source system to the target system and
to distributed information among systems.
20
MIMIX concepts
Configuration concepts
MIMIX configuration provides considerable flexibility to enable supporting a wide
variety of customer environments. Configuration is implemented through sets of
related commands. The following terms describe configuration concepts.
Definitions - MIMIX uses several types of named definitions to identify related
configuration choices.
• System definitions identify systems that participate in a MIMIX installation. Each
system definition identifies one system.
• Transfer definitions identify the communications path and protocol to be used
between systems.
• Journal definitions identify journaling environments that MIMIX uses for replication
Each journal definition identifies a system and characteristics of the journaling
environment on that system.
• Data group definitions identify the characteristics of how replication occurs
between two systems. Each data group definition determines the direction in
which replication occurs between the systems, whether that direction can be
switched, and the default processing characteristics for replication processes.
• Application group definitions identify whether the replication environment does or
does not use IBM i clustering. When clustering is used, the application group also
defines information about an application or proprietary programs necessary for
controlling operations in the clustering environment.
Data group entries - A data group entry is a configuration construct that identifies a
source of information to be replicated by or excluded from replication by a data group.
Each entry identifies at least one object and its location on the source system.
Classes of data group entries are based on object type. MIMIX uses data group
entries to determine whether a journal entry should be replicated. Data groups that
replicate from both the system journal and a user journal can have any combination of
data group entries.
Remote journal link (RJ link) - An RJ link is a MIMIX configuration element that
identifies an IBM i remote journaling environment used by user journal replication
processes. An RJ link identifies the journal definitions that define the source and
target journals, primary and secondary transfer definitions for the communications
path used by MIMIX, and whether the IBM i remote journal function sends journal
entries asynchronously or synchronously.
Cooperative processing - Cooperative processing refers to MIMIX techniques that
efficiently replicate certain object types by using a coordinated effort between the
system journal and user journal replication paths. Configuration choices in data group
definitions and data group entries determine the degree of cooperative processing
used between the system journal and user journal replication paths when replicating
database files, IFS objects, data areas, and data queues.
Tracking entries - Tracking entries identify objects that can be replicated using
advanced journaling techniques and assist with tracking the status of their replication.
A unique tracking entry is associated with each IFS object, data area, and data queue
that is eligible for replication using advanced journaling. IFS tracking entries identify
21
eligible, existing IFS objects while object tracking entries identify eligible, existing data
areas and data queues.
Process concepts
The following terms identify MIMIX processes. Some, like the system manager, are
required to allow MIMIX to function. Others, like procedures, are used only when
invoked by users.
Replication path - A replication path is a series of processes used for replication that
represent the critical path on which data to be replicated moves from its origin to its
destination. MIMIX uses two replication paths to accommodate differences in how
replication occurs for user journal and system journal entries. These paths operate
with configurable levels of cooperation or can operate independently.
• The user journal replication path captures changes to critical files and objects
configured for replication through a user journal. When configuring this path,
shipped defaults use the remote journaling function of the operating system to
simplify sending data to the remote system. The changes are applied to the target
system.
• The system journal replication path handles replication of critical system objects
(such as user profiles, program objects, or spooled files), integrated file system
(IFS) objects, and document library object (DLOs) using the system journal.
Information about the changes are sent to the target system where it is applied.
System manager - System manager processes automatically distribute configuration
and status changes among systems in a MIMIX installation. Although often thought of
as one entity per system, there are actually multiple system manager processes
associated with each system. Each system manager process consists of a remote
journal link (RJ Link) that transmits entries in one direction between a pair of systems
and a system manager processing job that applies the entries to MIMIX files on the
remote system (or target side) of the RJ link. Between each pair of communicating
systems there are always two system manager processes that transmit in opposite in
directions. In environments with more than two systems, MIMIX restricts which
systems can communicate based on their designation as a management (*MGT) or
network (*NET) system.
Journal manager - The journal manager is a job on each system that MIMIX uses to
maintain the journaling environment on that system. By default, MIMIX performs both
change management and delete management for journal receivers used by the
replication process.
Collector services - A group of jobs that are necessary for MIMIX to track historical
data and to support using the MIMIX portal application within the Vision Solutions
Portal. One or more collector service jobs collect and combine MIMIX status from all
systems.
Cluster services - When configured for use in an IBM i clustering environment,
MIMIX uses the cluster services function provided by IBM i to integrate the system
management functions needed for clustering. Cluster services must be active in order
for a cluster node to be recognized by the other nodes in the cluster. MIMIX integrates
22
MIMIX concepts
starting and stopping cluster services into status and commands for controlling
processes that run at the system level.
Target journal inspection - A set of MIMIX processes that read journals on a system
being used as the target system for replication. Each target journal inspection process
identifies people or processes other than MIMIX that created, changed, or deleted
replicated objects on the target system. The list of affected objects and any automatic
recovery actions initiated by MIMIX can be displayed in the Objects Changed on
Target window within the MIMIX portal application in Vision Solutions Portal. The
window can be accessed from multiple portlets and is filtered accordingly.
Procedures and steps - Procedures and steps are a highly customizable means of
performing operations. A set of default procedures is shipped with MIMIX for
frequently used operations. These default procedures include the ability to start or
end replication for an application group, perform pre-check activity for switching,
perform a planned or unplanned switch of an application group, perform virtual switch
testing, prepare MIMIX for running a backup, and run data protection reports for
nodes. Each operation is performed by a procedure that consists of a sequence of
steps and multiple jobs. Each step calls a predetermined step program to perform a
specific sub-task of the larger operation. Steps also identify runtime attributes that
determine how the procedure will start processing the step and how it will respond if
the step ends in error.
Log space - A MIMIX object that provides an efficient storage and manipulation
mechanism for replicated data that is temporarily stored on the target system during
the receive and apply processes.
1. MIMIX DR installations running version 8.1.02.00 or higher support virtual failover (virtual
switching)
23
Virtual switch - A way to test production applications on a node that could assume
the role of the production system in the event of a switch without actually performing a
planned switch. A virtual switch allows you to designate a backup node in an
application group for testing, sets up the MIMIX environment to allow user-performed
testing while the production environment and MIMIX remain active, and recovers the
MIMIX environment when testing is complete.
MIMIX Model Switch Framework - A set of programs and commands that provide a
consistent framework to be used when performing planned or unplanned switches in
environments that do not use application groups. Typically, a model switch framework
is customized to your environment through its exit programs.
MIMIX Switch Assistant - A guided user interface that guides you through switching
using your default MIMIX Model Switch Framework. MIMIX Switch Assistant is
accessed from the MIMIX Basic Main Menu and does not support application groups.
24
Authority to products and commands
25
Accessing the MIMIX Main Menu
The MIMIX command accesses the main menu for a MIMIX installation. The MIMIX
Main Menu has two assistance levels, basic and intermediate. The command defaults
to the basic assistance level, shown in Figure 1, with its options designed to simplify
day-to-day interaction with MIMIX. Figure 2 shows the intermediate assistance level.
The options on the menu vary with the assistance level. In either assistance level, the
available options also depend on the MIMIX products installed in the installation
library and their licensing. The products installed and the licensing also affect
subsequent menus and displays.
Accessing the menu - If you know the name of the MIMIX installation you want, you
can use the name to library-qualify the command, as follows:
Type the command library-name/MIMIX and press Enter. The default name of
the installation library is MIMIX.
If you do not know the name of the library, do the following:
1. Type the command LAKEVIEW/WRKPRD and press Enter.
2. Type a 9 (Display product menu) next to the product in the library you want on the
Vision Solutions Installed Products display and press Enter.
Changing the assistance level - The F21 key (Assistance level) on the main menu
toggles between basic and intermediate levels of the menu. You can also specify the
Assistance Level (ASTLVL) parameter on the MIMIX command.
26
Accessing the MIMIX Main Menu
Note: On the MIMIX Basic Main Menu, options 5 (Start or complete switch using
Switch Asst.) and 10 (Availability Status) are not recommended for
installations that use application groups.
27
MIMIX policies
28
Environment considerations for policies
29
which replication is permitted. If any pair of systems supports simultaneous bi-
directional replication, determine the winning system in each pair and determine
the direction to be audited. Set the audit level policy to permit auditing for the data
group that replicates in the chosen direction. Disable auditing for the data group
which replicates in the other direction. You may also want to consider changing
the values of the Run rule on system policy for the installation or the audited data
groups to balance processing loads associated with auditing.
• In environments that permit multiple management systems in the same
installation, in addition to evaluating the direction of replication permitted within
each pair of systems, you must also consider whether the systems defined by
each data group are both management systems. If any pair of systems supports
simultaneous bi-directional replication, choose the winning system and change
the Audit level policies for each data group so that only one direction is audited.
You may need to change the Run rule on system policy to prevent certain data
groups from being audited from specific management systems.
30
Setting policies - general
save and restore objects within the replication environment. Some applications may
fail when they cannot acquire a lock on an object. Refer to our Support Central for
FAQs that list specific applications whose data groups should be excluded from
auditing. For those excluded data groups, you can still run compares to determine if
objects are not synchronized between source and target systems. Care must be
taken to recover from these unsynchronized conditions. The applications may need to
be ended prior to manually synchronizing the objects.
To exclude a data group from audits, use the instructions in “Preventing audits from
running” on page 46.
Disabling audits and recovery when using the MIMIX CDP feature
The functions provided by the MIMIX CDP™ feature1 create an environment in which
source system changes have been transferred to the target system but have not been
applied. Any data group which uses this feature must disable automatic comparisons
and automatic recovery actions for the data group.
Do the following from the management system:
1. From the command line type SETMMXPCY and press F4 (Prompt).
2. For the Data group definition, specify the full three-part name of the data group
that uses the MIMIX CDP feature.
3. Press Enter to see all the policies and their current values.
4. For Automatic object recovery, specify *DISABLED.
5. For Automatic database recovery, specify *DISABLED.
6. For Automatic audit recovery, specify *DISABLED.
7. For Audit level, select *DISABLED.
8. To accept the changes, press Enter.
31
Do the following from the management system:
1. From the command line type SETMMXPCY and press F4 (Prompt).
2. Verify that the value specified for Data group definition is *INST.
3. Press Enter to see all the policies and their current values.
4. Specify a value for the policy you want. Use F1 (Help) to view descriptions of
possible values.
5. To accept the changes, press Enter.
32
Policies which affect an installation
Table 1. Policies that can be set only at the installation level and shipped default values.
Directory depth2 3
33
about each step within the procedure. The policy specifies how many days to keep
history information and the minimum number of runs to keep. You can specify a
different number of runs to keep for switch procedure runs than what is kept for other
types of procedures.
Each procedure run is evaluated individually against the policy and its history
information is retained until the specified minimum days and minimum runs are both
met. When a procedure run exceeds these criteria, system cleanup jobs will remove
the historical information for that procedure run from all systems. The values specified
at the time the cleanup jobs run are used for evaluation.
To change the procedure history retention policy for the installation, do the following:
1. From the command line type SETMMXPCY and press F4 (Prompt).
2. Verify that the value *INST is specified for the Data group definition prompt:
3. Press Enter to see all the policies and their current values.
4. Locate the Procedure history retention policy. The current values are displayed.
Specify values for the elements you want to change.
5. To accept the changes, press Enter.
1. The ASP threshold for recoveries (RCYASPTHLD) policy is available on instances running
MIMIX service pack 8.1.23.00 or higher.
34
Policies which affect replication
For more information about the Automatic object recovery and Automatic database
recovery polices, see “Replication problems that MIMIX attempts to recover” on
page 260 and “When to disable automatic recovery for replication and auditing” on
page 30.
35
For information about how the Source capture delay policy is used by recoveries and
considerations for changing the policy, see “Changing the source capture delay for
recoveries” on page 36.
These topics in the MIMIX Administrator Reference book provide information about
the following policies:
• For information about the DB apply cache policy, see topic “Configuring database
apply caching”.
• For information about the Access path maintenance policy, see topic “Optimizing
access path maintenance”.
• For information about delay retry processing in object replication and the Number
of third delay retry attempts and Third delay retry interval policies, see Parameters
for automatic retry processing in topic “Tips for data group parameters”.
36
Changing the source capture delay for recoveries
To change the source capture delay used during replication, do the following:
Note: This procedure changes a policy value at the installation level. The installation
level value can be overridden by a data group level policy value. Therefore, if
a data group has a value other than *INST for this policy, that value remains in
effect.
Changing the source capture delay used during virtual switch testing
During the application testing phase of a virtual switch, apply processes are ended
and replicated transactions wait to be applied. Recovery actions initiated by
replication processes and target journal inspection during virtual switch testing use a
configured delay time with a default value of 5 minutes. Optionally, you can change
the delay time.
The source capture delay value needs to be an acceptable balance between
increased risk and resource usage for the amount of time that you expect to remain in
the application testing phase of a virtual switch. During virtual switch testing, your
environment is already exposed because transactions accumulate on the target node
and are not applied until after the testing phase of the switch completes. The volume
of transactions, recovery actions, and how frequently recovery actions capture source
information can affect apply process backlogs and resource usage. All transactions
37
that have accumulated for apply processes during testing, including those for new
recoveries, are applied during the recovery phase of a virtual switch. The virtual
switch recovery phase must complete before a planned or unplanned switch of the
application group can be performed.
During virtual switch testing:
• No delay allows the information needed for recoveries to be immediately captured
and sent to the target. This represents the least risk in the event of the loss of the
source node. However, frequent access of source objects may interfere with
applications and increase communication usage, and the increased transactions
may quickly increase apply process backlogs.
• A delay time may reduce the number of captures needed for an object or file,
reduce communications usage, and place fewer recovery-related transactions into
the queues for apply processing. However, the objects and records identified by
new recoveries created during virtual switch testing are not synchronized, and
their source information will not be sent to the target node until the delay time is
reached.
• The value Delay Until Recovery (*VRTSWTRCY) delays all source captures for
target journal inspection recovery actions created during the application testing
phase of a virtual switch until the virtual switch recovery phase starts. If your
environment has workload or storage constraints, this may be more efficient. The
source information needed to synchronize affected target node files and objects is
not captured and sent to the target node at or near the time the errors occurred,
which is an exposure in the event of the loss of the source node. Also, the virtual
switch recovery phase may be slower because it includes all delayed capture
operations as well as applying transactions that accumulated while testing.
To change the source capture delay used during the application testing phase of a
virtual switch, do the following:
Note: This procedure changes a policy value at the installation level. The installation
level value can be overridden by a data group level policy value. Therefore, if
a data group has a value other than *INST for this policy, that value remains in
effect.
38
Changing the source capture delay for recoveries
39
Policies which affect auditing behavior
The policies identified in Table 3 affect the behavior of all audits in an installation
regardless of whether the audit are automatically submitted or manually invoked.
These policies can be set for the installation as well as overridden for a data group.
When set for a specific data group, these policies affect all audits for the data group.
The shipped initial values for both levels are indicated.
When the Set MIMIX Policies (SETMMXPCY) command specifies a data group
definition value of *INST, the policies being changed are effective for all data groups in
the installation, unless a data group-level override exists. When the data group
definition specifies a name, policies which specify the value *INST inherit their value
from the installation-level policy value and polices which specify other values are in
effect for only the specified data group.
Table 3. Policies that affect behavior of audits and their shipped default values
Audit Severity
• Not run audit status *WARNING *INST
• Ignored objects audit status *WARNING *INST
40
Policies which affect auditing behavior
Table 3. Policies that affect behavior of audits and their shipped default values
Several policies affect when audits automatically run and how those audits select
objects. However, those policies are no longer preferred for setting audit scheduling
and object selection criteria. Instead, use the Set MIMIX Schedules (SETMMXSCD)
command as described in “Scheduling audits and reports” on page 60.
41
Changing auditing policies
This topic describes how to change specific policies that affect auditing behavior and
when automatic audits will run. MIMIX service providers are specifically trained to
provide a robust audit solution that meets your needs.
42
Changing auditing policies
43
Note: Most audits check for threshold conditions in all database and object
replication processes, including the RJ link. #FILDTA audits only check for
threshold warning conditions in the RJ link and database replication
processes. #DLOATR audits only check for threshold warning conditions in
object replication processes.
Restricting audit activity in an installation based on data group state: Do the
following from the management system:
Note: This procedure changes a policy value at the installation level. The installation
level value can be overridden by a data group level policy value. Therefore, if
a data group has a value other than *INST for this policy, that value remains in
effect.
44
Changing auditing policies
*IGNOBJ statuses affects the order in which audits are listed on the Work with Audits
display (WRKAUD command) and in the Audits portlet in Vision Solutions Portal
(VSP), and the icon associated with these statuses in the Audits portlet.
All audit statuses have an associated severity. Only the severity associated with these
statuses can be changed with the Audit severity (AUDSEV) policy:
• *NOTRUN - indicates that an audit was submitted but did not run due to the status
of replication processes or policy settings.
• *IGNOBJ - indicates than an audit could not check some objects, typically due to
replication activity, locks, or authorizations. All other objects checked were
considered equal or had detected differences that were automatically recovered.
The policy setting for the severity of the *IGNOBJ status also affects whether any
ignored objects are counted as differences. The policy’s effect on how ignored objects
are counted occurs regardless of whether the audit status is *IGNOBJ or a more
severe audit status.
Note: This procedure changes a policy value at the installation level. The installation
level value can be overridden by a data group level policy value. Therefore, if
a data group has a value other than *INST for this policy, that value remains in
effect.
1. This policy is available for instances running MIMIX service pack 8.0.07.00 or higher. For
instances running earlier software levels, all audits are checked for compliance.
45
are evaluated for compliance. This prevents the potential for compliance errors for
audits that have no configured objects to audit and for the File Member Record Count
(#MBRRCDCNT) audit when the audit level policy value is *LEVEL30. For a detailed
description of this value and other possible values, press F1 (Help) for the Audits
included for compliance (AUDINCMPL) policy in the Set MIMIX Policies
(SETMMXPCY) command.
Note: This procedure changes a policy value at the installation level. The installation
level value can be overridden by a data group level policy value. Therefore, if
a data group has a value other than *INST for this policy, that value remains in
effect.
46
Changing auditing policies
47
Policies for switching with model switch framework
In environments that do not use application groups, MIMIX Switch Assistant (which
implements MIMIX Model Switch Framework) is usually used for switching1. MIMIX
Model Switch Framework cannot be used to switch application groups.
Table 4 identifies the policies associated with switching using MIMIX Model Switch
Framework and the shipped default values of those policies.
For these policies, MIMIX Switch Assistant uses only the policy values specified for
the installation. If MIMIX cannot determine whether a MIMIX Model Switch
Framework is defined, the switch framework policy is *DISABLED.
If the SETMMXPCY command specifies a data group name, the switch framework is
required to be *INST. The switch thresholds are *DISABLED by default but can be
changed.
The policies in Table 4 have no effect on application group switching.
48
Policies for switching with model switch framework
3. Press Enter to see all the policies and their current values.
4. At the Default model switch framework prompt, specify the name of the switch
framework to use for switching this installation.
5. To accept the changes, press Enter.
49
Policy descriptions
For a complete description of all policy values, see online help for the Set MIMIX
Policies (SETMMXPCY) command. The Data group definition and Audit rule
parameters on the command are not technically policies, but they must have values
specified in order to set a value for any policy.
Data group definition - Select the scope of the policies to be set. When the value
*INST is specified, the policies being set by the command apply to all systems and
data groups in the installation, with the exception of any policy for which a data group-
level override exists. When a three-part qualified name of a data group is specified,
the policies being set by the command apply to only that data group and override the
installation-level policy values.
Audit rule - The audit rule must specify a value of *NONE when setting a value for a
policy.
Note: The Audit rule, Audit schedule, and Priority audit policies are no longer
preferred for changing scheduling criteria for automatically submitted audits.
For more information about the recommended way to change audit
scheduling, see “Scheduling audits and reports” on page 60.
Automatic object recovery — Determines whether to enable functions that
automatically attempt to correct common object replication errors that may occur
during replication from the system journal and changes reported by target journal
inspection. When this policy is enabled, a recovery action is automatically started
when an error is detected. Recovery actions that require synchronizing information
from the source node to the target node use the time specified in Source capture
delay (SRCCAPDLY) policy.
Automatic database recovery — Determines whether to enable functions that
automatically attempt to correct common file errors that may occur during replication
from the user journal and changes reported by target journal inspection. When this
policy is enabled and an error is detected, MIMIX will generate a recovery and attempt
to replicate the transaction again. Recovery actions that require synchronizing
information from the source node to the target node use the time specified in Source
capture delay (SRCCAPDLY) policy
Automatic audit recovery — Determines whether to enable automatic recovery
actions for audits. When enabled, audits that support automatic recovery will check
their comparison results for known problems and will attempt to correct any detected
errors by automatically starting recovery actions.
Note: An audit that is currently running is not affected by a change to this policy until
the next run of the audit.
Object recovery notify on success — Determines whether object replication
processes that successfully process an object while using the configured third
delay/retry cycle will send an informational (*INFO) notification. This policy is only
valid when the Automatic object recovery policy is enabled.
Audit notify on success — Determines whether activity initiated by audits,
including recovery actions, will automatically send an informational (*INFO)
notification upon successful completion. If an audit is run when the Automatic audit
50
Policy descriptions
recovery policy is disabled, successful notifications are sent only for the compare
phase of the audit.
Notification severity — Determines the severity level of the notifications sent when
a rule ends in error. This policy determines the severity of the notification that is sent,
not the severity of the error itself. The policy is in effect whether the rule is invoked
manually or automatically.
This policy is useful for setting up an order precedence for notifications at the data
group level. For example, if you set this policy for data group CRITICAL to be
*ERROR when the value for the installation-level policy is *WARNING, any error
notifications sent from data group CRITICAL will have a higher severity than those
from other data groups.
Object only on target — Determines how recovery actions should handle correcting
objects configured for replication that exist only on the target node. The only-on-target
error may have been detected by replication processes, target journal inspection, or
audits (#OBJATR, #IFSATR, #DLOATR, #FILATR, and #FILATRMBR). This policy is
used by the resulting recovery actions when the Automatic recovery policies
(Database, Object, and Audit) are enabled and a virtual switch is not in progress.
See “Policies in environments with more than two nodes or bi-directional replication”
on page 29 for additional information.
Journaling attribute difference action — Determines the recovery action to take for
scenarios in which audits have detected differences between the actual and
configured values of journaling attributes for objects journaled to a user journal. This
type of difference can occur for the Journal Images attribute and the Journal Omit
Open/Close attribute. Differences found on either the source or target object are
affected by this policy.
MIMIX configured higher
Determines the recovery for correcting a difference in which the MIMIX
configuration specifies an attribute value that results in a higher number of journal
transactions than the object's journaling attribute.
MIMIX configured lower
Determines the recovery action for correcting a difference in which the MIMIX
configuration specifies an attribute value that results in a lower number of journal
transactions than the object's journaling attribute.
DB apply threshold action — Determines what action to pass to the Compare File
Data (CMPFILDTA) command or the Compare Record Count (CMPRCDCNT)
command when it is invoked with *DFT specified for its DB apply threshold
(DBAPYTHLD) parameter. The command’s parameter determines what to do if the
database apply session backlog exceeds the threshold warning value configured for
the database apply process. This policy applies whenever these commands are used
and the backlog exceeds the threshold.
The shipped default for this policy causes the requested command to end and may
cause the loss of repairs in progress or inaccurate counts for members. You can also
set this policy to allow the request to continue despite the exceeded threshold.
DB apply cache — Determines whether to use database (DB) apply cache to
improve performance for database apply processes. When this policy is enabled,
51
MIMIX uses buffering technology within database apply processes in data groups that
specify *YES for journal on target (JRNTGT). This policy is not used by data groups
which specify JRNTGT(*NO) or by data groups whose target journals use journal
caching or journal standby functionality provided by the IBM feature for High
Availability Journal Performance (IBM i option 42). As of MIMIX 8.0, the
DBAPYCACHE policy is shipped so that it is enabled at the installation level.
Upgrading an existing MIMIX 7.1 or older installation to MIMIX 8.0 or later does not
change DBAPYCACHE. If the installation you are upgrading did not use the
DBAPYCACHE policy, follow the instructions in the MIMIX Administrator Reference
book to enable it.
Note: When DB apply cache is used, before and after journal images are sent to the
local journal on the target system.This will increase the amount of storage
needed for journal receivers on the target system if before images were not
previously being sent to the journal.
Access path maintenance — Determines whether MIMIX can optimize access path
maintenance during database apply processing as well as the maximum number of
jobs allowed per data group when performing delayed maintenance. Enabling
optimized access path maintenance improves performance for the database apply
process. To make any change to this policy effective, end and restart the database
apply processes for the affected data groups. For more information about optimizing
access path maintenance, see the MIMIX Administrator Reference book.
Optimize for DB apply
Specify whether to enable optimized access path maintenance. When enabled,
the database apply processes are allowed to temporarily change the value of the
access path maintenance attribute for eligible replicated files on the target system.
Eligible files include physical files, logical files, and join logical files with keyed
access paths that are not unique and that specify *IMMED for their access path
maintenance.
Maximum number of jobs
Specify the maximum number of access path maintenance jobs allowed for a data
group when optimized access path maintenance is enabled. The actual number of
jobs varies as needed between a minimum of one job and the specified value. The
default value is 99.
Maximum rule runtime — Determines the maximum number of minutes an audit can
run when the Automatic audit recovery policy is enabled. The compare phase of the
audit is always allowed to complete regardless of this policy’s value. The elapsed time
of the audit is checked when the recovery phase starts and periodically during the
recovery phase. When the time elapsed since the rule started exceeds the value
specified, any recovery actions in progress will end. This policy has no effect on the
#MBRRCDCNT audit because it has no recovery phase. The shipped default for this
policy of 1440 minutes (24 hours) prevents running multiple instances of the same
audit within the same day. Valid values are 60 minutes through 10080 minutes (1
week).
Audit warning threshold — Determines how many days can elapse after an audit
was last performed before an indicator is set. When the number of days that have
elapsed exceeds the threshold, the indicator is set to inform you that auditing needs
52
Policy descriptions
your attention. The shipped default value of 7 days is at the limit of best practices for
auditing.
Note: It is recommended that you set this value to match the frequency with which
you perform audits. It is possible for an audit to be prevented from running for
several days due to environmental conditions or the Action for running audit
policy. You may not notice that the audit did not run when expected until the
Audit warning threshold is exceeded, potentially several days later. If you run
all audits daily, specify 1 for the Audit warning threshold policy. If you do not
run audits daily, set the value to what makes sense in your MIMIX
environment. For example, if you run the #FILDTA audit once a week and run
all other audits daily, the default value of 7 would cause all audits except
#FILDTA to have exposure indicated. The value 1 would be appropriate for the
daily audits but the #FILDTA audit would be identified as approaching out of
compliance much of the time.
Audit action threshold — Determines how many days can elapse after an audit was
last performed before an indicator is set. When the number of days that have elapsed
exceeds the threshold, the indicator is set to inform you that action is required
because the audit is out of compliance. The shipped default of 14 days is the
suggested value for this threshold, which is 7 days beyond the limit of best practices
for auditing.
Note: It is recommended that you set this value to match the frequency with which
you perform audits. It is possible for an audit to be prevented from running for
several days due to environmental conditions or the Action for running audit
policy. You may not notice that the audit did not run when expected until the
Audit action threshold is exceeded, potentially several days later. If you run all
audits daily, specify 1 for the Audit action threshold policy. If you do not run
audits daily, set the value to what makes sense in your MIMIX environment.
For example, if you run the #FILDTA audit once a week and run all other
audits daily, the default value of 14 would cause all audits except #FILDTA to
have exposure indicated. The value 2 would be appropriate for the daily audits
but the #FILDTA audit would be identified as approaching out of compliance
much of the time.
Audits included for compliance — Specifies which audits will be evaluated for
compliance. For each indicated audit, MIMIX evaluates whether the last run of the
audit exceeds the days specified for the Audit warning and Audit action thresholds
(AUDWARN and AUDACT).
To specify multiple audits by name, a plus sign (+) in the + for more prompt, and press
Enter.
Audit level — Determines the level of comparison that an audit will perform when a
MIMIX rule which supports multiple levels is invoked against a data group. The policy
is in effect regardless of how the rule is invoked. The amount of checking performed
increases with the level number. This policy makes it easy to change the level of audit
performed without changing the audit scheduling or rules. No auditing is performed if
this policy is set to *DISABLED.
The audit level you choose for audits depends on your environment, and especially
on the data compared by the #FILDTA, #DLOATR, and #IFSATR audits. When
53
choosing a value, consider how much data there is to compare, how frequently it
changes, how long the audit runs, how often you run the audit, and how often you
need to be certain that data is synchronized between source and target systems.
Note: Best practice is to use level 30 to perform the most extensive audit. If you use
a lower level, consider its effect on how often you need to guarantee data
integrity between source and target systems.
Regardless of the level you use for daily operations, Vision Solutions strongly
recommends that you perform audits at audit level 30 before the following events to
ensure that 100 percent of the data is valid on the target system:
• Before performing a planned switch to the backup system.
• Before switching back to the production system.
For additional information, see “Guidelines and considerations for auditing” on
page 151 and “Changing auditing policies” on page 42.
Run rule on system — Determines the system on which to run audits. This policy is
used when audits are invoked with *YES specified for the value of the Use run rule on
system policy (USERULESYS) parameter on the Run Rule (RUNRULE) or Run Rule
Group (RUNRULEGRP) command. When *YES is specified in these commands, this
policy determines the system on which to run audits. While this policy is intended for
audits, any rule that meets the same criteria will use this policy.
The policy’s shipped default value, *MGT, runs audits from the management system.
In multi-management environments where both systems defined to a data group are
management systems, the value *MGT will run audits only on the target system.
You can also set the policy to run audits from the network system, the source or target
system, or from a list of system definitions. When both systems of a data group are in
the specified list, the target system is used.
When choosing the value for the Run rule on system policy, also consider your
switching needs.
Action for running audits — Determines the type of audit actions permitted when
certain conditions exist in the data group. If a condition exists at the time of an audit
request, audit activity is restricted to the specified action. If multiple conditions exist
and the values specified are different, only the most restrictive of the specified actions
is allowed. If none of the conditions are present, the audit requests are performed
according to other policy values in effect.
Inactive data group
Specify the type of auditing actions allowed when any replication process required
by the data group is inactive. For example, a data group of TYPE(*ALL) is
considered inactive if any of its database or object replication processes is in a
state other than active. This element has no effect on the #FILDTA and
#MBRRCDCNT audits because these audits can run only when the data group is
active.
Replication process in threshold
Specify the type of auditing actions allowed when a threshold warning condition
exists for any process used in replicating the class of objects checked by an audit.
If a checked process has reached its configured warning value, auditing is
54
Policy descriptions
restricted to the specified actions. Most audits check for threshold conditions in all
database and object replication processes, including the RJ link. #FILDTA audits
only check for threshold warning conditions in the RJ link and database replication
processes. #DLOATR audits only check for threshold warning conditions in object
replication processes.
Audit severity — Specifies the severity to associate with audits that have a status of
*NOTRUN or *IGNOBJ. The selected severity determines whether audits with these
statuses roll up into overall status of MIMIX. The severity also affects the order in
which audits are displayed in interfaces, the color displayed in interfaces that use
color, and the icon displayed with the status in Vision Solutions Portal interfaces.
Not run audit status
Specifies the severity to associate with audits that have a status of *NOTRUN.
Audits with this status are the result of the Action for running audits (RUNAUDIT)
policy, or the data group was inactive when a #FILDTA or #MBRRCDCNT audit
was requested.
Ignored objects audit status
Specifies the severity to associate with audits that have a status of *IGNOBJ.
Audits with this status indicate that some objects were ignored because they were
considered active or could not be compared, typically due to locks or
authorizations. The value for this policy also determines whether any ignored
objects in an audit are counted as differences regardless of the audit status.
Audit history retention — Determines criteria for retaining historical information
about audit results and the objects that were audited. History information for an audit
includes timestamps indicating when the audit was performed, the list of objects that
were audited, and result statistics. Each audit, a unique combination of audit rule and
data group, is evaluated separately and its history information is retained until the
specified minimum days and minimum runs are both met. When an audit exceeds
these criteria, system manager cleanup jobs will remove the historical information for
that audit from all systems and will remove the audited object details from the system
on which the audit request originated. The values specified at the time the cleanup
jobs run are used for evaluation.
Minimum days
Specify the minimum number of days to retain audit history for each completed
audit. Valid values range from 0 through 365 days.The shipped default is 7 days.
Minimum runs per audit
Specify the minimum number of completed audits for which history is to retained.
Valid values range from 1 through 365 runs. The shipped default is 1 completed
audit.
Object details
Specify whether to retain the list of audited objects and their audit status for each
completed audit of library-based objects. The specified value in effect at the time
an audit runs determines whether object details for that run are retained. The
specified value has no effect on cleanup of details for previously completed audit
runs. Cleanup of retained details occurs at the time of audit history cleanup. The
shipped default is *YES.
55
DLO and IFS details
Specify whether to retain the list of audited objects and their audit status for each
completed audit of DLO and IFS objects. The specified value in effect at the time
an audit runs determines whether object details for that run are retained. The
specified value has no effect on cleanup of details for previously completed audit
runs. Cleanup of retained details occurs at the time of audit history cleanup. The
shipped default is *YES.
Note: When large quantities of objects are eligible for replication, specifying *YES to
retain either Object details or DLO and IFS details may use a significant
amount of disk storage. Consider the combined effect of the quantity of
replicated objects for all data groups, the number of days to retain history, the
number of audits to retain, and the frequency in which audits are performed.
Synchronize threshold size — Determines the threshold, in megabytes (MB), to
use for preventing the synchronization of large objects during recovery actions
initiated by database replication, object replication, or audits. When any of the
Automatic system journal recovery, Automatic user journal recovery, or Automatic
audit recovery policies are enabled, all initiated recovery actions use this policy value
for the corresponding synchronize command's Maximum sending size (MB)
parameter. This policy is useful for preventing performance issues when
synchronizing large objects.
Number of third delay retry attempts — Determines the number of times to retry a
process during the third delay/retry interval. This policy is used when the Automatic
system journal recovery policy is enabled. Object replication processes use this policy
value when attempting recovery of an in-use condition that persists after the data
group’s configured values for the first and second delay/retry intervals are exhausted.
The shipped default is 100 attempts.
This policy and its related policy, Third delay retry interval, can be disabled so that
object replication does not attempt the third delay/retry interval but still allow
recoveries for other errors.
Third delay retry interval — Determines the delay time (in minutes) before retrying a
process in the third delay/retry interval. This policy is used when the Automatic
system journal recovery policy is enabled. Object replication processes use this policy
value when attempting recovery of an in-use condition that persists after the data
group’s configured values for the first and second delay/retry intervals are exhausted.
The shipped default is 15 minutes.
Source capture delay — Determines the delay (in minutes) between detecting the
need for a recovery action that requires a synchronization request and capturing the
affected file records, attributes, authorities, or the entire object from the source
system. Recovery actions initiated by replication processes and target journal
inspection can delay these capture operations. There are two delay settings, one for
recoveries during normal replication, and one for recoveries during the testing phase
of a virtual switch. During the recovery phase of a virtual switch, MIMIX ignores both
delay settings in this policy and immediately captures any needed source information
for recoveries that have been identified. Recovery actions are only performed when
the Automatic database recovery and Automatic object recovery policies are enabled.
Changes to either value become effective automatically as the replication manager
checks for recoveries to process.
56
Policy descriptions
1. The library ratio monitor and the policy it uses require a license key for MIMIX for PowerHA.
57
CMPRCDCNT commit threshold — Determines the threshold at which a request to
compare record counts (CMPRCDCNT command or #MBRRCDCNT audit) will not
perform the comparison due to commit cycle activity on the source system. The value
specified is the maximum number of uncommitted record operations that can exist for
files waiting to be applied at the time the compare request is invoked. Each database
apply session is evaluated against the threshold independently. As a result, it is
possible that record counts will be compared for files in one apply session but will not
be compared for files in another apply session. For additional information see the
MIMIX Administrator Reference book.
Directory depth — Determines the number of directory levels to be checked by the
directory data protection report. The policy value at the time the report runs
determines the number of sub-directories included in the resulting report. The default
is 3.
ASP threshold for recoveries — Specifies the threshold as a percent of storage
used by the system auxiliary storage pool (ASP 1) at which certain types of recoveries
are delayed. When the percentage of storage used by the system auxiliary storage
pool (ASP1) on the source system exceeds the specified value, MIMIX delays
processing of new recoveries for member data of *FILE objects. Other recoveries are
not affected and continue to be processed. The delayed recoveries are automatically
submitted after the storage used by ASP1 on the source system is less than the
threshold value. The default is 95.
Procedure history retention — Specifies criteria for retaining historical information
about procedure runs that completed, completed with errors, or that have an
acknowledged status. History information for a procedure includes timestamps
indicating when the procedure was run and detailed information about each step
within the procedure. System cleanup jobs use the policy values at the time they run
to evaluate whether procedure history information can be removed. Each procedure
run is evaluated separately. If the history information includes more runs than the
minimum runs setting, then any runs older than the minimum days are removed by
system cleanup jobs.
Minimum days
Specifies the minimum number of days to retain procedure run history. The default
value is 7.
Minimum runs per procedure
Specifies the minimum number of completed procedure runs for which history is to
retained. This value applies to procedures of all other types except *SWTPLAN
and *SWTUNPLAN. The default value is 1.
Min. runs per switch procedure
Specifies the minimum number of completed switch procedure runs for which
history is to retained. This value applies to procedures of type *SWTPLAN and
*SWTUNPLAN that are used to switch an application group. The default value is
12.
The Audit schedule and Priority audit policies are no longer preferred for changing
audit scheduling criteria. For more information about using the preferred method, see
“Scheduling audits and reports” on page 60.
58
Policy descriptions
59
Scheduling audits and reports
This chapter describes how MIMIX automatically submits jobs to run audits and data
protection reports and how you can change scheduling criteria. The shipped default
schedules are also identified.
• “How audits and reports are submitted automatically” on page 60 describes how
the MIMIX Scheduler job is started and how it submits automatic requests, and
identifies what scheduling criteria you can change for audits and reports.
• “Default schedules for automatically submitted audits” on page 64 identifies the
shipped default schedules for both prioritized object auditing and scheduled object
auditing.
• “Default schedules for automatically submitted reports” on page 67 identifies the
shipped default schedule for data protection reports.
• “Changing when automatic audits are allowed to run” on page 68 describes how
to change scheduling criteria for both types of automatic auditing, how to change
the selection frequency of priority auditing categories, and how to disable or
enable automatic auditing.
• “Changing when automatic reports are allowed to run” on page 72 describes how
to change scheduling criteria for data protection reports and how to disable or
enable automatic reports.
Each audit rule1 for a data group has a separate schedule for each type of auditing,
scheduled object auditing or prioritized object auditing. The job scheduler submits
requests for each auditing type.
60
How audits and reports are submitted automatically
The scheduler job on a system remains active whenever the master monitor is active.
The master monitor is started by an autostart entry in the MIMIXSBS subsystem as
well as by the Start MIMIX (STRMMX) command. Starting other processes may not
automatically start the master monitor. You may need to manually start it on each
system with the STRMSTMON command to ensure that automatic scheduling can
occur. For example, starting data groups (STRDG command or processes which
invoke it) does not ensure that the master monitors are active.
61
Scheduling audits and reports
Schedule — Specifies the scheduling information for the specified data protection
report procedure or audit. MIMIX uses this information to automatically submit
requests to run the report or audit.
• When the Schedule type is *AUDIT, the values you specify for Schedule elements
determine when audits that compare all objects selected by data group
configuration entries for the specified rule are performed.
• When the Schedule type is *PROC, the values you specify for Schedule elements
determine when the procedure will run on the node to check whether existing
libraries, directories, or folders on that node are protected by MIMIX replication.
Scheduled dates are entered and displayed in job date format. When the job date
format is Julian, the equivalent month and day are used to determine when to
schedule requests.
State
Specify whether the schedule is enabled or disabled for the specified procedure or
audit. The value *ENABLED is required to allow automatic submission.
Frequency
Specifies how often MIMIX submits a request to run the scheduled activity. The
values specified for other elements further qualify the specified frequency.
Scheduled date
Specifies a date, in job date format, on which MIMIX submits the request. If the
job is to be submitted more than once, this date is used to calculate subsequent
submission dates.
Note: Scheduled date and Scheduled day are mutually exclusive
Scheduled day
Specifies the days of the week on which MIMIX submits the request. If today is the
day of the week that is specified and the scheduled time has already passed, the
request is submitted on the next occurrence of a specified day.
For example, if *FRI is specified for Scheduled day and 12:00:00 is specified for
Scheduled time and you are setting this policy at 11:00 a.m. on a Friday, the
request is submitted the same day. If you are setting the policy at 4:00 p.m. on a
Friday or at 11:00 a.m. on a Monday, the request is submitted the following Friday.
Scheduled time
Specifies the time on the local system at which MIMIX submits the request on the
scheduled date or day. Although the time can be specified to the second, the
activity involved in submitting a job and the load on the system may affect the
exact time at which the job is submitted.
Time can be specified with or without a time separator.
Without a time separator, specify a string of 4 or 6 digits (hhmm or hhmmss)
where hh = hours, mm = minutes, and ss = seconds. Valid values for hh range
from 00 to 23. Valid values for mm and ss range from 00 to 59.
With a time separator, specify a string of 5 or 8 digits where the time separator
specified for your job is used to separate the hours, minutes, and seconds. If this
command is entered from the command line, the string must be enclosed in
apostrophes. If a time separator other than the separator specified for your job is
62
How audits and reports are submitted automatically
63
Scheduling audits and reports
An unchanged object is one that has not been modified since the last time it was
audited.
Audited with no diff.
Specifies the frequency at which objects with no differences are considered for
auditing. An object with no differences is one that has not been modified since the
last time it was audited and has been successfully audited on at least three
consecutive audit runs.
Table 5. Initial values for automatically submitting audits shown in context of the parame-
ters on the Set MIMIX Schedules (SETMMXSCD) command.
Schedule (SCHEDULE)
State *ENABLED1
Frequency *WEEKLY1
Scheduled date
Scheduled day *SUN
Scheduled time Exact time varies by rule, see Table 6.
Relative day of month
Every day, a prioritized audit starts one or more times each hour during the time range
specified in its scheduling criteria. Other criteria specified with an audit’s priority time
range determine the auditing frequency assigned to the auditing priority categories.
64
Default schedules for automatically submitted audits
A scheduled audit runs once at its specified time on the days or dates for its
frequency as specified in its schedule. The shipped value for start time of each audit
rule is staggered, beginning at 2 a.m.
Table 6 shows the initially set times for priority audits versus scheduled audits.
Table 6. Initial times for automatically submitted audits for a new data group.
n/a1 2:00 a.m. #DGFE Checks configuration for files using sdn_DGFE
cooperative processing.
Uses the Check Data Group File Entries
(CHKDGFE) command.
65
Scheduling audits and reports
Table 6. Initial times for automatically submitted audits for a new data group.
All other 2:05 a.m. #OBJATR Compares all attributes for all object sdn_OBJATR
audits: types supported for replication.
3 a.m. Uses the Compare Object Attributes
to (CMPOBJA) command
8 a.m. 2:10 a.m. #FILATR Compares all file attributes. sdn_FILATR
Uses the Compare File Attributes
(CMPFILA) command.
66
Default schedules for automatically submitted reports
Table 7. Initial values for automatically submitting reports, shown in context of the parame-
ters on the Set MIMIX Schedules (SETMMXSCD) command.
Schedule (SCHEDULE)
State *ENABLED
Frequency *WEEKLY
Scheduled date
Scheduled day *SUN
Scheduled time 3:00 a.m.
Relative day of month
67
Changing when automatic audits are allowed to run
This topic describes how to change scheduling criteria for when automatic audits will
run. To effectively audit your replication environment you may need to fine-tune when
MIMIX is allowed to automatically submit prioritized object audits and scheduled
object audits. MIMIX service providers are trained to provide a robust audit solution
that meets your needs.
For both types of auditing, consider:
• How much time or system resource can you dedicate to audit processing each
day, week, or month?
• How often should all data within the database be audited? Business requirements
as well as time and system resources need to be considered.
• Does automatic scheduling conflict with regularly scheduled backups?
• Are there jobs running at the same time as audits that could lock files needing to
be accessed during recovery?
For scheduled object auditing (which select all objects), also consider:
• Are there are a large number of objects to be compared?
• Are there a large number of objects for which a rule is expected to attempt
recovery?
• Specific audits may have additional needs. See “Considerations for specific
audits” on page 152.
• While you may decide to vary the scheduled times, it is recommended that you
maintain the same relative order indicated in “Default schedules for automatically
submitted audits” on page 64.
68
Changing when automatic audits are allowed to run
b. The current values for the Schedule prompts are displayed. Specify the values
you want for these prompts.
• Frequency
• Scheduled date
• Scheduled day
• Scheduled time
• Relative day of month
5. To change when MIMIX is allowed to submit priority-based runs of the audit every
day, do the following:
a. Press Page Down. If you skipped the previous step you may need to do this
twice.
b. The current values of Priority audit prompts are displayed. Specify the values
for these prompts:
• Start after
• Start until
6. To make the changes effective, press Enter.
69
• Changed objects selected
• Unchanged objects selected
• Audited with no diff.
For descriptions of the priority classes with changeable frequencies, see “How
priority auditing determines what objects to select” on page 148.
4. To make the changes effective, press Enter.
70
Changing when automatic audits are allowed to run
71
Changing when automatic reports are allowed to run
This topic describes how to change scheduling criteria for when MIMIX is allowed to
submit requests to run data protection reports. Each data protection report is a
procedure that runs on a specific system.
When changing the schedule for a report, consider the following:
• Which categories of objects do you need to replicate? If you do not need to
replicate library-based objects, IFS objects in directories, or DLO objects in
folders, you can disable schedules for the respective category of data protection
report.
• On which systems do you need to check whether new libraries, directories, or
folders are protected by MIMIX replication? Data protection reports should be run
on systems in your production environment that are source systems for
replication. You might want to disable schedules for data protection reports on
systems that are not source systems or that can never become source systems
for any application group or data group.
• How frequently are new libraries, directories, or folders created on a system? This
affects how often the reports should be scheduled to run on that system to
evaluate whether data is protected by MIMIX. Business requirements as well as
time and system resources also need to be considered when determining
scheduling frequency.
• What is the setting for the Directory depth policy? This policy determines the
number of directory levels that are checked by the data protection report for
directories. The value of this policy may influence how often or when you want to
run the directory report. Changing the value to include more directories for a
report may cause the report to run longer. Consider the needs of your
environment and system resources available when the report runs.
• Does automatic scheduling conflict with regularly scheduled backups or times
when a system has high processing loads?
• Does heavy creation, deletion, renaming, or locking activity associated with
libraries, directories, or folders occur at the same time that data protection reports
run which might affect the report results?
72
Changing when automatic reports are allowed to run
b. From the Work with Systems display, select option 21 (Procedure status) next
to the system you want and press Enter.
2. The Work with Procedure Status display appears showing all procedures of type
*NODE that can run on the selected system. Type 37 (Change schedule) next to
the procedure you want to change and press Enter.
Note: Only procedures named CRTDPRDIR, CRTDPRFLR, and CRTDPRLIB
are data protection reports that be scheduled for automatic submission by
MIMIX.
3. The Set MIMIX Schedules (SETMMXSCD) command appears, showing the
schedule type as *PROC and the selected procedure and node. Press Enter, then
press Page Down.
4. The current values for the Schedule prompts are displayed. Specify the values
you want for these prompts:
• Frequency
• Scheduled date
• Scheduled day
• Scheduled time
• Relative day of month
5. To make the changes effective, press Enter.
73
schedule type as *PROC and the selected procedure and node. Press Enter, then
press Page Down.
4. The current values for the Schedule prompts are displayed. At the State prompt
specify the appropriate value:
• Specify *DISABLED to disable the report from being submitted automatically
on the specified system.
• Specify *ENABLED to enable the report for automatic submission on the
specified system.
5. To accept the change, press Enter.
74
Checking status in application group environments
The types of statuses to check and where you check them depends on how your
MIMIX environment is configured.
• For configurations with application groups of type *NONCLU (not configured for
an IBM i clustering environment), use “Checking status in application group
environments” on page 75.
Note: This document does not address status of application groups of type *CLU
that are configured for an IBM i clustering environment. If you are using
clustering or have MIMIX for PowerHA configured, use the MIMIX
Operations with IBM PowerHA for i book to check status within a clustering
environment.
• For configurations with only data groups, use “Checking status in data group-only
environments” on page 78.
The MIMIX Availability Status display (WRKMMXSTS command) should not be used
in environments that use application groups and is no longer documented in this
book.
75
Where to check status
Procedures Summary: Work with Application Groups display, option 21 (Procedure status).
Details: Work with Procedures Status display.
See: “Displaying status of procedures” on page 98.
See also: “Resolving problem statuses for an application group” on page 82.
System-level processes Summary: Work with Application Groups display, App Node Status column.
Also, Work with Node Entries display subsets to a summary status by node.
Details: Work with Systems display.
See: “Resolving problem statuses for an application group” on page 82.
See also: For additional problem determination, you may also need “Displaying
status of system-level processes” on page 180.
Replication direction Summary: Work with Application Groups display, Repl.Status column. Also,
(node roles) Work with Data Rsc. Grp. Ent. display, Replication Status column, subsets to a
summary by resource group.
Replication processes
Details: Detail locations vary depending on type of problem. Follow the
Replication errors hierarchy for resolving application group problems.
See: “Resolving problem statuses for an application group” on page 82.
Auditing errors
See also: For additional problem determination, you may also need:
Recoveries in progress • Replication direction: “Resolving status for data resource group entries” on
page 88.
• Replication processes (WRKDG): “Identifying and resolving replication
problems reflected in the Source and Target columns” on page 123.
• Replication errors (WRKDG, highlighted name in Data Group column):
“Identifying and resolving problems reflected in the Data Group column” on
page 120.
• Replication errors (WRKDG, Errors DB and Errors Obj columns): “Working
with files needing attention (replication and access path errors)” on page 263,
“Working with tracking entries” on page 272, and “Working with objects in
error” on page 278.
• Auditing errors: “Resolving auditing problems” on page 162.
• Recoveries in progress: “Displaying recoveries” on page 211.
Notifications Summary: Work with Application Groups display, Notifications field identifies the
highest severity level of any new notifications.
Details: Work with Notifications display.
See: “Displaying notifications” on page 205.
Monitors - status and Summary: Work with Application Groups display, Monitors field identifies the
errors highest severity monitor problem on the local system.
Details: Work with Monitors display.
See: “Resolving problems reported in the Monitors field” on page 85.
See also: “Displaying status of monitors on the local system” on page 192.
76
Checking status in application group environments
Switching status Summary: Work with Application Groups display, AG Proc. Status column
includes switching procedures in the summary status.
Details: Work with Procedures Status display.
See: “Resolving problem statuses for an application group” on page 82.
See also: “Displaying status of procedures” on page 98.
77
Where to check status
System-level processes Summary: Work with Data Groups display, Source Mgr and Target Mgr
columns.
Details: Work with Systems display.
See: “Displaying status of system-level processes” on page 180.
Node procedures Summary: Work with Systems display, option 21 (Proc Sts).
Details: Work with Procedures Status display.
See: “Displaying status of procedures” on page 98.
Replication processes Summary: by system where processes run, Work with Data Groups display,
Source DB, Source OBJ, Target DB, and Target OBJ columns.
See: “Identifying and resolving replication problems reflected in the Source and
Target columns” on page 123.
Replication errors Summary: Work with Data Groups display, two locations:
• Data Group column, when name is highlighted
See: “Identifying and resolving problems reflected in the Data Group column”
on page 120
• Errors DB and Errors Obj columns
See: “Working with files needing attention (replication and access path
errors)” on page 263, “Working with tracking entries” on page 272, and
“Working with objects in error” on page 278.
Auditing errors Summary: installation count on Work with Data Groups display, Audits field.
Details: Work with Audits display.
See: “Resolving auditing problems” on page 162.
Recoveries in progress Summary: installation count on Work with Data Groups display, Recov. field.
Details: Work with Recoveries display, and the Object Correction Activity
window in Vision Solutions Portal.
Note: A recovery in progress indicates that MIMIX detected a problem and is currently
attempting to correct it. It is important that no recoveries are in progress when
certain activity, such as ending MIMIX, is performed. The count on the Work with
Data Groups display includes recoveries from all possible sources. However, the
Work with Recoveries display only lists new or in-progress recoveries from audits
or that are the result of object replication third delay retry processing. Recoveries
that originated from replication processes, target journal inspection, and virtual
switch procedures can only be viewed from Vision Solutions Portal.
See: “Displaying recoveries” on page 211.
Notifications - new Summary: installation count on Work with Data Groups display, Notif. field.
errors or warnings Details: Work with Notifications display.
See: “Displaying notifications” on page 205.
78
Checking status in data group-only environments
Switching status Summary: Switch in progress indicated on Work with Data Groups display,
Source DB, Source OBJ, Target DB, and Target OBJ columns.
See: “Identifying and resolving replication problems reflected in the Source and
Target columns” on page 123.
79
Checking application group status
Monitoring status in environments that use application groups begins at the level of
the application group. Resolving reported problems will require investigation into
additional displays for more details.
Note: This chapter does not include status for application groups that are configured
for an IBM i clustering environment. If you are using clustering or have
MIMIX® for PowerHA® configured, see the MIMIX Operations with IBM
PowerHA for i book for status information within a clustering environment.
80
Displaying application group status
App App App Node Data Rsc Data Node Repl. AG Proc.
Opt Group Status Status Grp Status Status Status Status
__ __________
__ SAMPLEAG *ACTIVE *ACTIVE *COMP
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F6=Create F9=Retrieve F10=View config
F12=Cancel F13=Repeat F18=Subset F23=More options F24=More keys
Note: The App Status, Data Rsc Grp Status, and Data Node Status columns are
always blank for application groups that are not configured for an IBM i
clustering environment. If your environment is configured to support
clustering, use the MIMIX Operations with IBM PowerHA for i book.
Table 10. Work with Application Groups display -ideal status values for replication
App Node Status Status of system-level processes that run the nodes *ACTIVE
within the application group
Data Rsc Grp Status Always blank for non-cluster application groups.
AG Proc Status Status of the most recent run of each procedure *COMP
associated with an application group.
81
Checking application group status
82
Resolving problem statuses for an application group
problem reported in other columns on the Work with Application Groups display.
Status shown in other columns will likely change after active procedures complete
or procedures which require attention have been addressed.
*ACTIVE One or more of the last started runs of the procedures to run are still
active or queued. Wait for the procedure to complete. Do not attempt to
correct other status problems reported on the display until the procedure
completes. Use option 21 (Procedure status) to view the status of the
last started runs of procedures.
*ATTN One or more of the last started runs of the procedures have a status that
requires attention. Resolve the procedure problems before attempting to
correct other status problems reported on the Work with Application
Groups display.
1. Use option 21 (Procedure status) to view the status of the last started
runs of the procedures.
2. The resulting procedures shown on the Work with Procedure Status
display which have status values of *ATTN, *CANCELED, *FAILED,
*MSGW, or *PENDCNL require user action.
3. It may be necessary to check status of the steps within the procedure
to resolve a step problem before the procedure can continue. Use
option 8 (Step status) to display step status for the procedure.
Note: For a virtual switch procedure (type *SWTVRT), a *MSGW status for step
MXVRTRCY is expected and should remain unanswered for the duration
of the testing phase of the virtual switch. Do not respond to the message
for step MXVRTRCY until testing is complete and you are ready for the
recovery phase of the virtual switch procedure. Return to the Work with
Application Groups display to check for other reported problems with the
application group that need to be resolved.
For detailed information, see “Working with procedures and steps” on
page 97.
Note: The status *COMP is a summary status that indicates the most recently
started run of each procedure for the application group has completed,
completed with errors, or has an acknowledged status. For any individual
procedure that completed with errors, user action is recommended to
investigate the cause of the error and assess its implications.
83
Checking application group status
If there are no procedure problems, each of the other columns with an *ATTN status
must be addressed individually, starting from the left-most column.
Table 12. Resolving *ATTN status for columns (except AG Proc Status) on the Work with
Application Groups display
App Node The App Node Status column is a summary of the status of the nodes
Status associated with the application group. The status includes the MIMIX
system manager, journal manager, target journal inspection, and collector
services jobs for the nodes in the application group.
*ATTN indicates that the node status and the MIMIX manager status
values do not match. Investigate the status of the associated nodes and
MIMIX managers using option 12 (Node entries). Then use “Resolving
status for node entries” on page 87.
Table 13. Other problem statuses which may appear in multiple columns on the Work with Application
Groups display
*ATTN Each column has a unique recovery. See “Resolving an *ATTN status for an application
group” on page 83.
84
Resolving problem statuses for an application group
Table 13. Other problem statuses which may appear in multiple columns on the Work with Application
Groups display
*INACTIVE The current status of the resource group or node is inactive. This status is possible in the
Repl. Status column.
• If all columns with a status value are *INACTIVE, the application group may have been
ended intentionally. Use option 9 (Start) to start the application group.
• If this value appears only in the App Node Status column, the application resource
group nodes are all inactive and all MIMIX manager jobs are also inactive. Use option
12 (Node entries) to investigate further. For more information see “Resolving status for
node entries” on page 87.
• If this value appears only in the Repl. Status column, logical replication is not active.
Use option 13 (Data resource groups) to investigate. For more information see
“Resolving status for data resource group entries” on page 88.
*UNKNOWN The current status is unknown. The local node is a network node in a non-cluster
application group and does not participate in the recovery domain. Its status cannot be
determined.
When this status appears in all columns, do one of the following:
• Enter the command WRKSYS. On the Work with Systems display, check the status of
Cluster Services for the local system definition. If necessary, use option 9 (Start) to
start cluster services.
• Sign on to a node that is active and use the WRKAG command to check the
application group status. If the status is still *UNKNOWN, use option 12 (Node entries)
to check the status of Cluster Services on the node.
*VRTSWTSTR These values can appear in the Repl. Status column when a virtual switch procedure is in
*VRTSWTTST progress for an application group of type *NONCLU. These statuses are not necessarily a
*VRTSWTRCY problem. However, complete your virtual switch testing as soon as possible to limit
exposure to your environment. While virtual switch testing is in progress, replicated
transactions accumulate because apply processes and access path maintenance for
affected data groups are ended. These transactions must be processed during virtual
switch recovery, and the virtual switch procedure must complete before any other
operation such as starting, stopping, or switching can be performed for the application
group.
85
Checking application group status
Table 14 shows possible status values for the Monitors field that require user action.
For a complete list of possible values, press F1 (Help).
Table 14. Monitor field status values that may require user action
*ATTN Either one or more monitors on the local system failed or there are both
active and inactive monitors on the local system.
Do the following:
1. Press F14 (Monitors) to display the list of monitors on the local system on the
Work with Monitors display.
2. Check the Status column for status values of FAILED, FAILED/ACT, and
INACTIVE.
3. If the monitor is needed on the local system as indicated in Table 15, use option 9
(Start) to start the monitor.
Table 15. Possible monitors and the nodes on which they should be active
MMNFYNEWE - monitor for new object notification entries Primary node when the
Monitors the source system for the newly created libraries, application group is
folders, or directories that are not already included or configured for logical
excluded for replication by a data group configuration. replication.
86
Resolving status for node entries
Figure 4. Status view of Work with Node Entries display for an application group which does
not participate in a cluster
-------------Current------------- Manager
Opt Node Role Sequence Data Provider Status
__ ________
__ SYSB *PRIMARY *PRIMARY *ACTIVE
__ SYSA *BACKUP 1 *PRIMARY *ACTIVE
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F6=Add F7=Systems F9=Retrieve
F10=View config F11=Sort by node F12=Cancel F18=Subset F24=More keys
For each node listed, check the Manager Status column for status values that require
attention. For a complete list of status values for each field and column, press F1
(Help).
Manager Status - This column indicates the status of all of the MIMIX system
manager, journal manager, target journal inspection, and collector services jobs for
the specified node.
*ATTN At least one of the system manager, journal manager, target journal
inspection, or collector services jobs for the node has failed.
When all the nodes listed do not have the same value, use F7 (Systems) to
access the Work with Systems display.
• Check the status of the system and journal managers, target journal
inspection, and collector services.
• Use option 9 (Start) to start the managers and services that are not active
on the node.
87
Checking application group status
*INACTIVE All system manager, journal manager, target journal inspection, and
collector services jobs for the specified node are inactive. This may be
intentional when MIMIX is ended to perform certain activities.
Use F7 (Systems) to access the Work with Systems display.
Figure 5. The Work with Data Resource Group Entries display for an application group that
does not participate in a cluster.
Resource
Resource Group Node Replication
Opt Group Type Status Status Status
__ __________
__ AGRSGRP *DTA *ACTIVE
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F6=Add F9=Retrieve F12=Cancel
F13=Repeat F18=Subset F19=Load F21=Print list
A Virtual switch status field can appear below the Application group field when a
virtual switch is in progress for the application group (*NONCLU only). The field
identifies the phase of virtual switch activity.
Resource Group Status - This column identifies the status of the data resource
group. In environments that do not include IBM i clustering, this column is always
blank.
88
Resolving status for data resource group entries
Node Status - This column identifies the status of the nodes for the data resource
group. In environments that do not include IBM i clustering, this column is always
blank
Replication Status - The value in this column is a summary status of data replication
activity for the data resource group. The status includes the status of all data group
processes, replication direction, replicated object and file entries, audits, and
recoveries.
Figure 17 identifies status values for replication that require user action.
*ATTN One or more of the following problems exist for data groups within the
data resource group.
• The source system of an active data group is not the primary node of
its application group.
• A data group has a failed state, a process other than apply is inactive
during a virtual switch, an error condition, audit errors, or pending
recoveries.
To prevent damage to data in your environment, it is important that
you begin by determining which system should be the source for
the data groups. Do the following:
1. From this display, use option 8 (Data groups) to check which system
is the current source for the data group.
2. Determine which node has the role of current primary for the
application group. From the Work with Application Groups display,
use option 12 (Node entries), then check the current node role.
If the current primary node is correct and a data group with an incorrect
source system is active, end the data group and contact CustomerCare.
If the data groups in question have the correct source system but the
primary node for the application group is not correct, you need to
change the recovery domain for the application group to make the
correct node become primary. Use “Changing the sequence of backup
nodes” on page 91.
After you have ensured that the data groups have the correct
source system, resolve any error conditions reported on the Work
with Data Groups display.
Note: Not all data groups should necessarily be active. Only the data groups
currently being used for data replication should be active. You will need
to look at the current node roles and data providers for the node entries
to determine which data groups should be active.
*INACTIVE All replication in the data resource group is inactive. This may be normal
if replication was ended to perform certain activities. Use option 8 (Data
groups) to access the Work with Data Groups display.
89
Verifying the sequence of the recovery domain
Ensuring that sequence of the current backup nodes is set properly is critical to a
successful and predictable switch process. The current sequence of backup nodes
should match your recovery guidelines.
Do the following to confirm the sequence of the current backup nodes before
performing a switch and before removing or restoring a backup node from the cluster.
1. From the MIMIX Intermediate Main Menu, type 5 (Work with application groups)
and press Enter.
2. From the Work with Application Groups display, type 12 (Node entries) next to the
application group you want and press Enter.
3. The Work with Node Entries display appears, showing current information for the
nodes. Confirm that the current backup nodes have the sequence order that you
expect.
Note: It is important that you are viewing current information on the status view
of the display. Figure 6 shows an example of how the resulting Work with
Node Entries display appears with current status information. If you see
configured information instead, press F10 (View status).
4. If you need to change the sequence of current backup nodes, use “Changing the
sequence of backup nodes” on page 91.
Figure 6. Example of displaying the current sequence information for backup nodes
-------------Current------------- Manager
Opt Node Role Sequence Data Provider Status
__ ________
__ NODEA *PRIMARY *NONE *ACTIVE
__ NODEB *BACKUP 1 NODEA *ACTIVE
__ NODEC *BACKUP 2 NODEA *ACTIVE
__ NODED *BACKUP 3 NODEA *ACTIVE
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F6=Add F7=Systems F9=Retrieve
F10=View config F11=Sort by node F12=Cancel F18=Subset F24=More keys
90
Changing the sequence of backup nodes
91
g. If the configured backup sequence is what you expect, skip to Step 5 to make
the change effective.
4. To change the sequence of backup nodes, do the following:
a. From the configured view of the Work with Node Entries display, type 2
(Change) next to the backup node whose sequence you want to change.
b. On the Change Node Entry (CHGNODE) display, specify *BACKUP for Role
and press Enter. Then specify either *FIRST or a number for List position and
press Enter.
Note: If you specify a number, it cannot already be used in the configured
sequence list.
c. On the Work with Node Entries display, press F5 (Refresh) to view changes.
d. Repeat Step 4 until the correct sequence is shown on the configuration view.
Note: Gaps in configured sequence numbers are ignored when switching to a
backup. For example, in a configuration with two backup nodes, there is
no operational difference between a backup sequence of 1, 2 and a
backup sequence of 2, 5 as long as the same nodes are specified in the
same relative order.
5. To make the changes to the backup order effective, do the following:
a. Press F12 (Cancel) to return to the Work with Application Groups display.
b. Type 9 (Start) next to the application group you want and press F4 (Prompt).
c. On the Start Application Group (STRAG) display, specify *CONFIG for Current
node roles and press Enter.
d. The Procedure prompt appears. If needed, specify a different value and then
press Enter.
6. Confirm that the node entries have changed. Type 12 (Node entries) next to the
application group and press Enter. If necessary, use F10 to access the status
view. The current backup nodes should be in the new order.
92
Changing the sequence of backup nodes
Figure 7. Example of displaying the configured sequence information for backup nodes:
-----------Configured------------
Opt Node Role Sequence Data Provider
__ ________
__ NODEA *PRIMARY *PRIMARY
__ NODEB *BACKUP 1 *PRIMARY
__ NODEC *BACKUP 4 *PRIMARY
__ NODED *BACKUP 5 *PRIMARY
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F6=Add F7=Systems F9=Retrieve
F10=View status F11=Sort by node F12=Cancel F18=Subset F24=More keys
93
configured backup sequence does not match the relative order of either the current
sequence or the desired sequence.
Status View
-----------Current---------------
Opt Node Role Sequence Data Provider
__ ________
__ NODEA *PRIMARY *PRIMARY
__ NODEB *BACKUP 1 *PRIMARY
__ NODED *BACKUP 2 *PRIMARY
__ NODEC *BACKUP 3 *PRIMARY
Configured View
-----------Configured------------
Opt Node Role Sequence Data Provider
__ ________
__ NODEA *PRIMARY *PRIMARY
__ NODEC *BACKUP 1 *PRIMARY
__ NODEB *BACKUP 2 *PRIMARY
__ NODED *BACKUP 3 *PRIMARY
Each row in Table 19 shows a change to be made to the nodes on the configured
view of the Work with Node Entries display. The rows are in the order that the
changes need to occur to correct this example configuration to the desired order.
Table 19. Order in which to change nodes to achieve the desired configuration for example 1
94
Changing the sequence of backup nodes
Table 19. Order in which to change nodes to achieve the desired configuration for example 1
Example 2 - Correcting the configured primary node and changing the backup
sequence
Table 20 shows a four-node environment where the current backup sequence does
not reflect the desired behavior in the event of a switch. Also, the current and
configured primary node do not match. The configured primary node must be
corrected first, before attempting to correct any backup node sequence problems.
Table 20. Example 2, showing discrepancies in primary node and backup sequences
Status View
-----------Current---------------
Opt Node Role Sequence Data Provider
__ ________
__ NODEA *PRIMARY *PRIMARY
__ NODEB *BACKUP 1 *PRIMARY
__ NODED *BACKUP 2 *PRIMARY
__ NODEC *BACKUP 3 *PRIMARY
Configured View
-----------Configured------------
Opt Node Role Sequence Data Provider
__ ________
__ NODEB *PRIMARY *PRIMARY
__ NODEC *BACKUP 1 *PRIMARY
__ NODEA *BACKUP 2 *PRIMARY
__ NODED *BACKUP 3 *PRIMARY
95
Each row in Table 21 shows a change to be made to the nodes on the configured
view of the Work with Node Entries display. The rows are in the order that the
changes need to occur to correct this example configuration to the desired order.
Table 21. Order in which to change nodes to achieve the desired configuration for example 2.
96
CHAPTER 6 Working with procedures and steps
This chapter describes how to work with procedures and steps. Procedures are used
to perform operations associated with an application group or node. A set of default
procedures is shipped with MIMIX which provides the ability to start, end, perform pre-
check activity for switching, switch application groups, and the ability to run data
protection reports for systems. Each procedure consists of a sequence of steps and
may use multiple jobs which perform specific sub-tasks of the larger operation. Steps
also identify runtime attributes that determine how the procedure will start processing
the step and how it will respond if the step ends in error.
It is important to understand how multiple jobs are used to process steps for a
procedure. A procedure uses multiple asynchronous jobs to run the programs
identified within its steps. Starting a procedure starts one job for the application group
or node. For procedures associated with application groups, an additional job is
started for each of its data resource groups. These jobs operate independently and
persist until the procedure ends. Each persistent job evaluates each step in sequence
for work to be performed within the scope of the step program type. When a job for a
data resource group encounters a step that acts on data groups, it spawns an
additional job for each of its associated data groups. Each spawned data group job
performs the work for that data group and then ends.
This chapter contains the following topics:
• “Displaying status of procedures” on page 98 describes how to display the status
of the most recent runs of procedures for an application group, the conditions
which cause each procedure status value, and the actions required to resolve
problem statuses.
• “Responding to a procedure in *MSGW status” on page 101 describes how to
resolve procedure inquiry messages.
• “Resolving a *FAILED or *CANCELED procedure status” on page 102 describes
how to address procedures with these error statuses.
• “Displaying status of steps within a procedure run” on page 103 describes how to
display status of steps within a procedure as well as the differences between the
collapsed and expanded views of the Work with Step Status display.
• “Resolving problems with step status” on page 106 describes the conditions which
cause each step status value and the actions required to resolve problem
statuses. This includes how to resolve step inquiry messages and failed or
canceled steps.
• “Acknowledging a procedure” on page 110 describes how to manually change a
procedure with a status of *CANCELED, *FAILED, or *COMPERR to an
acknowledged status.
• “Running a procedure” on page 111 describes how to start a user or node
procedure and the parameter that controls the step at which the procedure
begins.
97
Working with procedures and steps
98
Displaying status of procedures
Figure 8 shows an example of the Work with Procedure Status display subsetted to
show only runs of a specific procedure and application group. The list is in reverse
chronological order so that the most recently started procedures are at the top.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F11=Duration F12=Cancel
F13=Repeat F18=Subset F21=Print list
A second view, F11 (Duration), shows columns for the Duration of the procedure and
the Node on which the procedure was started. Procedures of type *NODE that have
not run are blank for these columns and have a status of *NEW.
From the second view, F11 (Schedule summary) shows columns for the Frequency
and Scheduled time for procedures of type *NODE that are data protection reports.
The frequency indicates how often the report is automatically submitted to be run by
MIMIX. The scheduled time identifies when MIMIX will submit the next scheduled run.
Only data protection report procedures can be scheduled.
Timestamps are in the local job time. If you have not already ensured that the
systems in your installation use coordinated universal time (UTC), see “Setting the
system time zone and time” on page 322.
Table 22 identifies the possible status values that can appear on the Work with
Procedure Status display and identifies the action to take to resolve reported
problems.
99
Working with procedures and steps
*ATTN The procedure requires attention. Either there is a step with a status of
*MSGW, or there is an active step and one or more steps with step
status values of *ATTN, *CANCEL, *FAILED, or *IGNERR.
Action Required: Determine the status of each step and the action
required to correct that status. See “Resolving problems with step
status” on page 106.
*PENDCNL A request to cancel the procedure is in progress. When the activity for
the steps in progress at the time of the cancel request ends, the
procedure status changes to *CANCELED.
*QUEUED A request to run the procedure is currently waiting on the job queue.
When the procedure becomes an active job, the procedure status
changes to *ACTIVE.
Resumable *CANCELED Either the procedure was canceled and did not complete, or steps within
the procedure were canceled as a response to inquiry messages from
the steps. The procedure was partially performed.
Action Required: Use “Resolving a *FAILED or *CANCELED
procedure status” on page 102 to determine the state of your
environment and whether to resume the procedure or to acknowledge
its status.
*FAILED The procedure failed. Jobs for one or more steps had errors. Those
steps were configured to end if they failed. The procedure was partially
performed.
Action Required: Use “Resolving a *FAILED or *CANCELED
procedure status” on page 102 to determine the state of your
environment and whether to resume the procedure or to acknowledge
its status.
Acknowledged *ACKCANCEL The procedure was canceled and a user action acknowledged the
cancellation so that the procedure can no longer be resumed.
*ACKFAILED The procedure failed and a user action acknowledged the failure so that
the procedure can no longer be resumed.
*ACKERR The procedure completed with errors and a user action acknowledged
the procedure. It is assumed that the user reviewed the steps with
errors. A status of completed with errors is only possible when the steps
with errors had been configured (within the procedure) to ignore errors
or a user’s response to a step in message wait status was to ignore the
error and continue running the procedure. After the step is
acknowledged, the procedure status changes to *ACKERR.
100
Responding to a procedure in *MSGW status
Completed *COMPERR The procedure completed with errors. One or more steps had errors and
were configured to continue processing after an error.
Action Recommended: Investigate the cause of the error and assess
its implications.
New *NEW The *NODE type procedure is new and has not yet been run for that
node.
Action: Use option 9 (Run) to run the procedure.
101
Working with procedures and steps
102
Displaying status of steps within a procedure run
command. Steps in the collapsed view could have any status except *ACTIVE
or *MSGW. Determine if there are any jobs with status values of *FAILED or
*CANCEL in the expanded view. Other jobs may have processed subsequent
steps before the procedure ended. For detailed information use “Resolving
problems with step status” on page 106.
3. After you have completed your evaluation and have taken any needed corrective
action to resolve why jobs failed or were canceled, determine how to best
complete the procedure. Choices are:
• Resume the procedure. If you resume a failed procedure, processing will begin
with the step that failed. If you resume a canceled procedure, processing will
begin with steps following the canceled step. Optionally, if you were unable to
resolve a problem for a step in error, you can override the attributes of that step
for when the procedure is resumed. See “Resuming a procedure” on page 112.
Note: If a virtual switch procedure fails or is canceled at any step within the
range performed while replication status for the application group
indicates virtual switch activity (starting, testing, or recovering), the only
choice after taking corrective action is to resume the procedure. In this
scenario, you will not be able to run any other procedure to start, stop,
or switch the affected application group until the virtual switch procedure
is resumed and allowed to run to completion.
• Acknowledge the procedure status. Procedures with a status of *CANCELED
or *FAILED can be acknowledged (set to *ACKCANCEL or *ACKFAILED,
respectively) to indicated you have investigated the problem steps and want to
run the procedure again starting at its first step. This option should only be
used after you have evaluated the effect of activity performed by the
procedure. See “Acknowledging a procedure” on page 110.
103
Working with procedures and steps
The steps listed on the Work with Step Status display appear in sequence number
order as defined by steps in the procedure. If the procedure is in progress, the display
shows status for the steps that have run, the start time and status of the step that is in
progress, and blank status and start time for steps that have not yet run.
Collapsed view - There are multiple views of the Work with Step Status display. The
view shown depends on the type of procedure.
Procedures other than *NODE - Figure 9 shows the collapsed view of for
procedures other than type *NODE. Each step of the procedure is shown as a
single row and step status represents the summary of all jobs used by the step. F7
(Expand) provides a view that shows details for additional jobs used to process
each step.
*NODE procedures - Figure 10 shows the display for *NODE procedures, which
only shows summary rows for steps. All steps of a node procedure run on the
node where the procedure started (Started on node field).
Figure 9. Collapsed view of Work with Step Status display for procedures other than
*NODE.
104
Displaying status of steps within a procedure run
Figure 10. Work with Step Status display for *NODE procedures.
More...
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F12=Cancel
F13=Repeat F14=Resume proc. F15=Cancel proc. F18=Subset F24=More keys
Expanded view - Figure 11 shows an example of the expanded view available for
procedures other than type *NODE. In the expanded view, step programs of type
*AGDFN will have one row for each node on which the step runs. Steps which run
step programs at the level of the data resource group or data group are expanded to
have multiple rows so that the status of the step for each data resource group or data
group is visible. For step programs of type *DTARSCGRP, there will be a summary
row for the application group followed by a row for each data resource group within
the application group. For step programs of type *DGDFN, there will be a summary
row for the application group, then for each data resource group, there is a summary
row for the data resource group followed by a row for each of its data groups.
Summary rows are identified by a dash (-) in the columns that are being summarized.
105
Working with procedures and steps
Also, for step programs of type *AGDFN, the Data Rsc. Grp. column and the Data
Group column will always be blank. For step programs of type *DTARSCGRP, the
Data Group column will always be blank.
Figure 11. Expanded view of the Work with Step Status display.
blank The procedure has started but processing has not yet started for the step.
106
Resolving problems with step status
*ATTN The step requires attention. The value *ATTN can only appear in the
collapsed view or on a summary row in the expanded view. If the procedure
status is considered active, at least one job submitted by this step has a
status of *FAILED, *CANCEL or *MSGW. If the procedure status is *FAILED
or *CANCELED, this step has at least one job that has not started or has a
status of *CANCEL or *FAILED.
Action Required: Use F7 to see the expanded view. Determine the specific
data resource group or data group for which the problem status exists. Then
address the status indicated for that job.
*DSBLD The step has been disabled and did not run.
*CANCEL One or more jobs used by the step ended in error. In the expanded view of
or status, the job is identified as *CANCEL or *FAILED. The status is due to the
*FAILED error action specified for the step.
• For *CANCEL status, user action canceled the step. The step ran, ended
in error, and issued an inquiry message. The user’s response to the
message was Cancel.
• For *FAILED status, the step ran, one or more jobs ended in error. The
Action on error attribute specified to quit the job.
The type of step program used by the step determines what happens to other
jobs for the step and whether subsequent steps are prevented from starting,
as follows:
• If the step program is of type *DGDFN, jobs that are processing other data
groups within the same data resource group continue. When they
complete, the data resource group job ends. Subsequent steps that apply
to that data resource group or its data groups will not be started. However,
subsequent steps will still be processed for other data resource groups
and their data groups.
• If the step program is of type *DTARSCGRP, subsequent steps that apply
to that data resource group or its data groups will not be started. Jobs for
other data resource groups may still be running and will process
subsequent steps that apply to their data resource groups and data
groups.
• If the step program is of type *AGDFN, subsequent steps that apply to the
application group will not be started. Jobs for data resource group or data
group steps may still be running and will process subsequent steps that
apply to their data resource groups and data groups.
• If the step program is of type *NODE, subsequent steps will not be started.
When all asynchronous jobs for the procedure finish, the procedure status is
set to *CANCELED or *FAILED, accordingly. If both canceled and failed
steps exist when the procedure ends, the procedure status will be *FAILED.
Action Required: Determine the cause of the problem using “Resolving
*CANCEL or *FAILED step statuses” on page 109.
107
Working with procedures and steps
*IGNERR The step ran and an error occurred, but processing ignored the error and
continued.
Action Recommended: Use option 8 (Work with job) to determine the cause
of the failure. Consider whether any changes are needed for the procedure,
step, or operating environment to prevent this error from occurring again.
*MSGW The step ran and issued a message that is waiting to be answered. One or
more jobs for the step ended in error. The step attribute requires that an
operator respond to the message.
Action Required: Determine which job issued the message, investigate the
problem, and then respond to the inquiry message using “Responding to a
step with a *MSGW status” on page 108.
108
Resolving problems with step status
• A response of I (Ignore) will set the job status to *IGNERR as indicated in the
expanded view of step status, and processing continues as if the job had not
ended in error. Type I and press Enter.
109
Working with procedures and steps
Acknowledging a procedure
Acknowledging a procedure allows you to manually change the status of procedures
that either failed or have errors in order to control where the next attempt to run the
procedure will start. Procedures with a status of *CANCELED, *FAILED, or
*COMPERR can be acknowledged (set to *ACKCANCEL, *ACKFAILED, or
*ACKERR, respectively) to indicated you have investigated the problem steps and
understand the implications.
There is one exception to this for a virtual switch procedure. If a problem occurs in any
step within the range performed while the replication status for the application group
indicates virtual switch activity (starting, testing, or recovering) and the virtual switch
procedure fails or is canceled, you cannot acknowledge the failed or canceled
procedure. The procedure must be resumed.
A procedure of *CANCELED or *FAILED allows you to rerun the procedure from its
first step. Once acknowledged, a procedure with either of these statuses cannot be
resumed from the point where the procedure ended. This is appropriate when you
have determined that your environment will not be harmed if the next attempt to run
starts at the first step.
A *COMPERR procedure that is acknowledged (*ACKERR) can never be resumed
because the procedure completed. By acknowledging a procedure with this status,
you are confirming the problems have been reviewed.
The last run of a procedure with a status of *ACKCANCEL or *ACKFAILED and the
last run of the set of start/end/switch procedures can be returned to their previous
status (*CANCELED or *FAILED, respectively). The next attempt to run the procedure
will resume at the failed or canceled step or at the first step that has not been started.
Note: Acknowledging the last run of a failed or canceled procedure will acknowledge
all previous failed or canceled runs of the procedure.
Important! Before changing status of a procedure, it is important that you evaluate
and understand the effect of the partially performed procedure on your environment.
Changing procedure status does not reverse the actions taken by preceding steps
that completed or the actions performed by other asynchronous jobs which did
complete the same step and then processed subsequent steps. It may not be
appropriate for the next run of the procedure to begin with the first step, for example,
110
Running a procedure
if the failure occurred in a step which synchronizes data or changes states of MIMIX
processes. Likewise, it may not be appropriate to return to the previous status to
resume a procedure run was not recently run.
To change the status of a procedure, do the following:
1. From the Work with Procedure Status display type 13 (Change status) next to the
failed or canceled procedure you want and press Enter.
2. The Change Procedure Status (CHGPROCSTS) display appears. Specify the
value you want for the Status prompt and press Enter.
3. If you specified *ACK in Step 2, the Start time prompt appears with *ALL specified
for Start time (STARTTIME). To acknowledge all previously failed or canceled
runs of the selected procedure, press Enter.
Running a procedure
The procedure type determines what command to use to run the procedure.
Procedure types *USER and *NODE can be run from either the Work with Procedures
or Work with Procedure Status displays using option 9 (Run). *NODE procedures can
only run on the node where the procedure is started. All other procedure types must
be invoked by the application group command associated with the procedure type.
For example a procedure of type *START can only be invoked by the Start Application
Group (STRAG) command.
Multiple uniquely named procedures of type *USER for the same application group
can run at the same time. For procedures of type *END, *START, *SWTPLAN,
*SWTUNPLAN, or *SWTVRT, only one procedure for the application group can run at
a time.
Multiple uniquely named procedures of type *NODE can be run on a specific system
at the same time. Procedures of type *NODE which have the same name can run on
different systems at the same time.
Where should the procedure begin? The value specified for the Begin at step
(STEP) parameter on the request to run the procedure determines the step at which
the procedure will start. The status of the last run of the procedure determines which
values are valid.
The default value, *FIRST, will start the specified procedure at its first step. This value
can be used when the procedure has never been run, when its previous run
completed (*COMPLETED or *COMPERR), or when a user acknowledged the status
of its previous run which failed, was canceled, or completed with errors
(*ACKFAILED, *ACKCANCEL, or *ACKERR respectively).
Other values are for resolving problems with a failed or canceled procedure. When a
procedure fails or is canceled, subsequent attempts to run the same procedure will
fail until user action is taken. You will need to determine the best course of action for
your environment based on the implications of the canceled or failed steps and any
steps which completed.
The value *RESUME will start the last run of the procedure beginning with the step at
which it failed, the step that was canceled in response to an error, or the step
111
Working with procedures and steps
following where the procedure was canceled. The value *RESUME may be
appropriate after you have investigated and resolved the problem which caused the
procedure to end. Optionally, if the problem cannot be resolved and you want to
resume the procedure anyway, you can override the attributes of a step before
resuming the procedure.
The value *OVERRIDE will override the status of all runs of the specified procedure
that did not complete. The *FAILED or *CANCELED status of these procedures are
changed to acknowledged (*ACKFAILED or *ACKCANCEL) and a new run of the
procedure begins at the first step.
.
Resuming a procedure
To resume a procedure with a status of *CANCELED or *FAILED, do the following
from the Work with Step Status display:
1. Investigate and resolve problems for steps with errors. See “Resolving problems
with step status” on page 106.
2. Optional: If the problem cannot be resolved, and you want to resume the
procedure anyway, use option 13 (Override step) to change the configured value
of the step for when the procedure is resumed. See “Overriding the attributes of a
step” on page 112 for considerations and more information.
3. For procedures of type *USER or *NODE, use F14 (Resume proc.). For all other
procedure types, from a command line, enter the appropriate application group
command and specify *RESUME as the value for Begin at step (STEP).
112
Running a procedure
command. The attributes determine whether the step is run or actions if the step
errors for the current run of the procedure when it is resumed.
The OVRSTEP command can be used for a procedure that has a status of active
(*ACTIVE, *ATTN, *MSGW, *PENDCNL, or *QUEUED), *CANCELED or *FAILED
and steps that have a status of *CANCEL or *FAILED. The overridden values apply
only for the current run of the procedure when it is resumed.
Note: Regardless of procedure status, attributes cannot be overridden for a required
MIMIX step or any step with a step status of *COMP or *IGNERR.
A procedure with a status of *CANCELED or *FAILED requires user action to resolve
a problem. If the problem cannot be resolved and you want to resume the procedure
anyway, you can use the OVRSTEP command to disable the step in error or specify
the error action to occur when the step is retried.
Important! Overriding the attributes of a step should only be done after you have
considered how rerunning the step impacts your environment. It is important that
you understand the implications for steps which preceded the cancellation or
failure in the last run of the procedure. Processing for steps that completed is not
reversed.
The changes made when using the OVRSTEP command will only apply to the current
run of the procedure. The attributes that can be changed will vary depending on the
statuses of the specified procedure and step. Consider the following:
• When the specified procedure has a status of *NEW, *ACKCANCEL,
*ACKFAILED, *ACKERR, *COMPLETED, or *COMPERR, no attributes can be
overridden on any step in the procedure.
• When the specified procedure has a status that is considered active (*ACTIVE,
*ATTN, *MSGW, *PENDCNL, or *QUEUED), only the Action on error
(ERRACT) can be overridden.
• When the specified procedure has a status that can be resumed (*CANCELED
or *FAILED), the Action before step (BEFOREACT), Action on error
(ERRACT), or State (STATE) can be overridden only on steps that have not yet
run, that failed, or that were canceled.
Do the following from the Work with Step Status display:
1. Locate the step whose attribute you want to override. If needed, use F7 (Expand)
to view the Expanded view.
2. Type 13 (Override step) next to the step you want and press Enter.
3. On the Override Step (OVRSTEP) display, specify the values you want and press
Enter.
4. For procedures of type *USER or *NODE, use F14 (Resume proc.). For all other
procedure types, from a command line, enter the appropriate application group
command and specify *RESUME as the value for Begin at step (STEP). See
“Resuming a procedure” on page 112.
113
Working with procedures and steps
Canceling a procedure
Use this procedure to cancel a procedure with a status that is considered active. This
includes procedure statuses of: *ACTIVE, *ATTN, *MSGW, *PENDCNL, and
*QUEUED.
Important! Use this command with caution. Processing ends without reversing
any actions performed by completed steps, which may leave your environment in
an undesirable state. For example, ending a switch procedure could result in
partially switched data.
If you cancel a virtual switch procedure when it is at any step within the range
performed while the application group’s replication status indicates a virtual switch
is starting, testing, or recovering, the procedure must be resumed. You will not be
able to run any other procedure to start, stop, or switch the affected application
group until the virtual switch procedure is resumed and allowed to run to
completion.
The status of the procedure will be changed to *PENDCNL. If there are any inquiry
messages waiting for an operator response, they are processed as if the response
was Cancel. When all activity for currently running steps end, the status of the
procedure will be automatically changed to *CANCELED.
To cancel an active procedure, do one of the following:
Note: These steps must be done from the node on which the procedure was started.
• From the Work with Procedure Status display, type 12 (Cancel) next to the
procedure you want and press Enter.
• From the Work with Step Status display, press F15 (Cancel proc.).
A procedure that has been canceled can be resumed later, as long as its status has
not been changed to *ACKCANCEL. When a canceled procedure is resumed,
processing begins immediately after the point where it was ended.
114
Displaying available status history of procedure runs
Only procedures for data protection reports can be scheduled for automatic
submission by MIMIX.
Figure 12. Schedule summary view of the Work with Procedure Status display, showing pro-
cedures for a single node.
App Group
Opt Procedure Type or Node Frequency -Scheduled Time--
__ CRTDPRLIB *NODE SYSTEMA *WEEKLY 02/02/14 03:00:00
__ CRTDPRFLR *NODE SYSTEMA *WEEKLY 02/02/14 03:00:00
__ CRTDPRDIR *NODE SYSTEMA *WEEKLY 02/02/14 03:00:00
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F11=Start time F12=Cancel
F13=Repeat F18=Subset F21=Print list
115
Working with procedures and steps
column.
Note: To view status of all runs of all procedures, you can either press F20
(Procedure status) from the Work with Application Groups display, press F14
(Procedure status) from the Work with Procedures display, or enter the
command: WRKPROCSTS.
116
CHAPTER 7 Working with data group status
This chapter describes common MIMIX operations that help keep your MIMIX
environment running. In order for MIMIX to provide a hot backup of your critical
information, all processes associated with replication must be active at all times.
Supporting service jobs must also be active. MIMIX allows you to display and monitor
the statuses of these processes.
The topics included in this chapter are:
• “Displaying data group status” on page 118 describes how to access the Work
with Data Groups display and identifies where to find information to resolve errors
reported in each column of the display.
• “Displaying detailed status for a data group” on page 126 describes how to access
detailed status for a data group and describes what is shown on the multiple views
of the Data Group Status display.
• “Identifying replication processes with backlogs” on page 137 describes what
fields to check for detailed status of a data group.
• “Data group status in environments with journal cache or journal state” on
page 139 describes the additional information reported within data group status
configured to use MIMIX support for IBM’s High Availability Journal Performance
IBM i option 42, Journal Standby feature and Journal caching. This includes the
topic “Resolving a problem with journal cache or journal state” on page 141.
117
Displaying data group status
From the Work with Data Groups display you can start and end replication, track
replication status, perform a data group switch, as well as work with files, objects, and
tracking entries in error and access displays for data group entries and tracking
entries.
To access the Work with Data Groups display, do one of the following:
• From the MIMIX Basic Main menu, select option 6 (Work with data groups) and
press Enter.
• From the MIMIX Intermediate Main menu, select option 1 (Work with data groups)
and press Enter.
Figure 13. Sample Work with Data Groups display. The display uses letters and colored
highlighting to call your attention to warning and problem conditions. This example shows
items in color which would appear with color highlighting on the display. If you are viewing this
page in printed form, the color may not be shown.
Bottom
F3=Exit F5=Refresh F7=Audits F8=Recoveries F9=Automatic refresh
F10=Legend F13=Repeat F16=DG definitions F23=More options F23=More keys
For each data group listed, you can see the current source system and target system
processes, and the number of errors reported.
The following fields and columns are available.
Audit/Recov./Notif. -These three fields, located in the upper right corner of the Work
with Data Groups display respectively identify the number of audits that require action
or attention, the number of new or in-progress recoveries, and the number of new
notifications that require action or attention. If more than 999 items exist in any field,
the field will display +++. When highlighted in reverse red, a problem exists that
requires action. When highlighted in reverse yellow, your attention may be needed to
prevent a problem from occurring.
#
Displaying data group status
For audits, the Audit severity policy in effect when audits run determines whether
certain audit statuses are reported as errors (red) or warnings (yellow).
For recoveries, it is important that there are no new or in-progress recoveries when
certain activity, such as ending MIMIX, is performed. You can view a list of the objects
that have new or in-progress recoveries from the Object Correction Activity window
for a data group within Vision Solutions Portal.
Data group - When a data group name is highlighted, a problem exists.
Source - The following columns provide summaries of processes that run on the
source system.
Mgr - Represents a summation of the system manager and the journal manager
processes on the source system of the data group. These processes must be
active (A) for replication to occur.
DB - Represents the status of the remote journal link. It is possible to have an
active status in this column even though the data group has not been started.
When the RJ link is active, database changes will continue to be sent to the target
system. MIMIX can read and apply these changes once the data group is started.
For data groups configured for source-send replication, this represents the status
of the database send process.
Obj - Represents a summation of the object processes that run on the source
system. These include the object send, object retrieve and container send
processes.
DA - This column represents the status of the data area polling process when the
data group replicates data areas through the data area poller. This column does
not contain data when data areas are replicated through the user journal with
advanced journaling or through the system journal.
Target - The following columns provide summaries of processes that run on the target
system.
Mgr - Represents a summation of the system manager, journal manager, and
target journal inspection processes on the target system of the data group. Target
journal inspection status includes status of inspection jobs for both target journals
(user and system) for the data group. Manager processes must be active (A) for
replication to occur.
DB - Represents the summation of status for the database reader process, the
database apply process, and access path maintenance jobs1. For data groups
configured for source-send replication, this column represents the summation of
the status of database apply processes and access path maintenance jobs.
Obj - Represents the object apply processes.
Errors - When any errors are indicated in the following columns (DB and Object), they
are highlighted in red.
DB - Represents the sum of the number for database files, IFS objects, *DTAARA
and *DTAQ objects that are on hold due to errors plus the number of logical (LF)
1. Access path maintenance jobs run only if the access path maintenance (APMNT) policy is
enabled.
104
and physical (PF) files that have access path maintenance1 failures for the data
group. To work with a subsetted list of file errors and access path errors, use
option 12 (Files needing attention). For a subsetted list of IFS object errors, use
option 51 (IFS tracking entries not active), For a subsetted list of *DTAARA and
*DTAQ errors, use option 53 (Object tracking entries not active).
Obj - Represents a count of the number of objects for which at least one activity
entry is in a failed state. To work with a subsetted list, use option 13 (objects in
error).
Use the following topics to resolve problems reported on this display:
• “Resolving auditing problems” on page 162.
• “Displaying recoveries” on page 211.
• “Displaying notifications” on page 205.
• “Identifying and resolving problems reflected in the Data Group column” on
page 120
• For Source and Target columns, see topics:
– “Displaying status of system-level processes” on page 180
– “Identifying and resolving replication problems reflected in the Source and
Target columns” on page 123
• For Error columns, see topics:
– “Working with files needing attention (replication and access path errors)” on
page 263,
– “Working with tracking entries” on page 272,
– “Working with objects in error” on page 278.
Table 24. Conditions which highlight the data group name in color.
#
Displaying data group status
Table 24. Conditions which highlight the data group name in color.
104
For additional information about possible actions, see the tables in the following
topics:
• For file entries “Working with user journal replication errors” on page 263
• For IFS and Object tracking entries, see “Working with tracking entries” on
page 272 for additional details.
Journaling problems: If the data group name is highlighted in red or yellow, either
the objects are not being journaled, or they are being journaled to a different journal
than the configured journal.
Do the following to check for and resolve journaling problems:
1. From the Work with Data Groups display, use option 8 (Display status).
2. From the Data Group Status display, press F8 (Database).
3. Check the following:
a. The Jrn State and Cache Src and Tgt fields. These fields are located in the
upper left corner of the Data Group Database Status display. For each system
(Src or Tgt) status of the journal state is shown first, followed by the status of
the journal cache. The example below shows ‘v’ for value in all for status
positions. If any of these fields are highlighted, there is a problem. Use
“Resolving a problem with journal cache or journal state” on page 141.
Jrn State and Cache Src: v v Tgt: v v
b. The Not journaled Src and Tgt fields. These fields are located in the upper right
corner of the Data Group Database Status display. For each system (Src or
Tgt) value, if there is a number other than zero, there is a problem. Either the
objects are not being journaled, or they are being journaled to a different
journal than the configured journal. Continue with these steps to determine the
cause of the problem.
6. Press F11 (View) multiple times to switch the view until the display shows a list of
the objects with journaling problems. Note which objects have problems.
7. Press F12 (Cancel) multiple times to return to the Work with Data Groups display.
8. Access the journaled view to check the journaling status for objects with
problems:
• For files, use option 17 (File entries) for the data group. Then press F10
(journaled view) to see journaling status.
• For IFS tracking entries, use option 50 (IFS tracking entries) for the data
group. Then press F10 (journaled view) to see journaling status.
• For object tracking entries, use option 52 (object tracking entries) for the data
group. Then press F10 (journaled view) to see journaling status.
9. From the journaled view, locate the entries with a journal status (JRN1STS or
JRN2STS) value of *NO or *DIFFJRN.
a. If the journal status value is *NO, continue with these steps.
#
Displaying data group status
Table 25. Possible status values for source and target process summaries
L Inactive RJ link (highlighted red) – The RJ link is currently not active. This status is
only displayed in the database source column when a data group uses MIMIX RJ
support.
Check the status of the database and object replication processes.
• If replication processes are active, use option 44 (RJ links) to start the RJ link.
• If replication processes are all inactive, use option 12 (Start DG) to start the data
group.
• If replication processes have other statuses, correct those problems. Then use
option 44 (RJ links) to start the RJ link.
A Active (highlighted blue) – The process is currently active. For the database source
column, this value indicates that the send/receive processes are active.
R Active RJ link (highlighted blue) – The RJ link is currently active. This status is only
displayed in the database source column when a data group uses MIMIX RJ
support.
104
Table 25. Possible status values for source and target process summaries
J RJ Link in Threshold (highlighted turquoise) – The RJ link has fallen behind its
configured threshold.
Use option 8 (Display status) to view detailed status and determine the extent of
the backlog. For more information about which detailed status fields to check, see
“Identifying replication processes with backlogs” on page 137.
V Virtual switch (highlighted yellow) – The apply processes and access path
maintenance for the data group are ended because the data group is participating
in a virtual switch of an application group. Other processes are active.
The apply processes and access path maintenance will be started automatically
during the recovery phase of the virtual switch procedure. User action is required to
initiate the recovery phase. See “Working with procedures and steps” on page 97.
X Switch mode (highlighted red) – The data group is in the middle of switching
(planned or unplanned) the data source system and status may not be retrievable
or accurate.
Wait for the switch to complete. To check for additional status, do one of the
following:
• In environments that use application groups, check the status of the switch
procedure using the WRKPROCSTS command.
• In environments with only data groups that switch using an implementation of
MIMIX Model Switch Framework, use the command CHKSWTFWK
SWTFWK(name) to check status. You can also go to the MIMIX Basic Main
Menu, select option 5 (Start or complete switch using Switch Asst.).
#
Displaying data group status
Table 25. Possible status values for source and target process summaries
P Partially active (highlighted red) - At least one subprocess is active (or is ended for
a virtual switch), but one or more subprocesses are not active. This status is only
displayed in process columns that represent multiple processes. The data group
name may also be shown in a highlighted field of red.
In the Target DB column, partial status is also possible when all other processes,
including database apply, are active but access path maintenance1 is enabled and
does not have at least one active job.
Use option 8 (Display status) to view detailed status and determine which
replication process is not active.
Then from the Work with Data Groups display, use option 9 (Start DG) to start the
data group.
D Disabled – The process is currently not active and the data group is disabled.
Note: The status value for a disabled data group is the letter D displayed in standard
format. No colored blocks are used.
For more information about disabled data groups, see “Disabling and enabling data
groups” on page 334.
Note: Use F10 (Legend) to view a pop-up window that displays the status values
and colors. To remove the pop-up window, press Enter or F12 (Cancel).
104
Displaying detailed status for a data group
Detailed status is available for one data group at a time.
The Data Group Status display (DSPDGSTS command) uses multiple views to
present status of a single data group. The views identify and provide status for each
of the processes used by the data group. Error conditions for the data group as well
as process statistics and information about the last entry processed by each
replication process are included. Some fields are repeated on more than one view.
Do the following to access detailed status for a data group:
1. Use one of the following to locate the data group you want:
• To select a data group from a list of all data groups in the installation, select
option 6 (Work with data groups) on the MIMIX Basic Main Menu and press
Enter.
• To select a data group from a subsetted list for an application group, from the
Work with Application Groups display use option 13 (Data resource groups) to
select a resource group. On the resulting display use option 8 (Data groups).
2. The Work with Data Groups display appears. Type an 8 (Display status) next to
the data group you want and press Enter.
3. The Data Group Status display shows a merged view of data group activity on the
source and target systems. (See Figure 14.)
Only fields for the type of information replicated by the data group are displayed.
For example, if the data group replicates only objects from the system journal, you
will only see fields for system journal replication. If the data group replicates from
both the system journal and the user journal, you will see fields for both. To see
additional status information for object processes or database processes, do the
following:
• If the data group contains object information, press F7 (Object) to view
additional object status displays. The Data Group Object Status display
appears.
• If the data group contains database information, press F8 (Database) to view
additional database status displays. The Data Group Database Status display
appears. Tracking entry information for advanced journaling is also available.
4. For object information, there are three views. For database information, there are
four views available. Use F11 to change between views.
Note: If the data group contains both database and object information, you can
toggle between object details and database details by using the F7 and F8
keys.
Merged view
The initial view displayed is the merged view. This view summarizes status for the
replication paths configured for the data group. The status of each process is
represented by the color of the box surrounding the process and a status letter. Table
26 shows possible status values.
126
Displaying detailed status for a data group
Figure 14 shows a sample of the merged view of the Data Group Status display. The
data group in this view is configured for user journal replication using remote
journaling and for system journal replication. Also, access path maintenance is
enabled.
Figure 14. Merged view of data group status. The inverse highlighted blocks are not shown
in this example.
Note: Journal sequence numbers shown in the Source Statistics and Target
Statistics areas may be truncated if the journal supports *MAXOPT3 for the
receiver size and the journal sequence number value exceeds the available
display field. When truncation is necessary, the most significant digits (left-
most) are omitted. Truncated journal sequence numbers are prefixed by '>'.
This is shown in Figure 14.
Table 26. Possible values for detailed status. Not all statuses are used by each process.
Yellow When displayed on the Data group field, a problem exists that may
require attention.
127
Table 26. Possible values for detailed status. Not all statuses are used by each process.
Yellow - P One or more of the processes is active but others are inactive. On the
merged view, this status is possible for the Object Send field.
When displayed for the journal manager field on the merged view and
object views 1 and 2, certain types of recoveries are delayed due to the
ASP threshold for recoveries policy while other recoveries are active.
Yellow - V The process is ended for a virtual switch. This status is available only
for apply processes and access path maintenance.
Turquoise - T The process has a backlog which exceeds its configured threshold. On
fields which summarize status for multiple processes, use F7 and F8 to
view the specific threshold. The -T is not shown in statistical fields. If a
threshold condition persists over time, refer to the MIMIX Administrator
Reference book for information about possible resolutions.
Blue - C The RJ Link is in catch-up mode.This status is only possible for the
Database Link process in the merged view and the RJ link field in
some database views.
Green - D The data group is disabled. This also means the data group is currently
inactive.
Top left corner: The top left corner of the Data Group Status display identifies the
data group, the elapsed time, and the status of the transfer definition in use. The
elapsed time is the amount of time that has elapsed since you accessed this display
or used the F10 (Restart statistics) key.
Top right corner: The top right corner of the display contains fields that identify the
number of errors that MIMIX identified but could not correct as well as the files and
objects with detected problems that MIMIX is currently attempting or will attempt to
automatically correct. If the workstation supports colors, the number files and objects
in error will be displayed in red.
• The Database errors/rcy. fields. The first value identifies the number of errors in
user journal replication processes. This includes all file entries, IFS tracking
entries, and object tracking entries in error (*HLDERR status). When access path
maintenance1 is enabled, this also includes the number of logical and physical
files that have access path maintenance failures for the data group. The second
value identifies the number of recoveries that exist for objects replicated through
user journals.
• The Object in error/active fields indicate the number of objects that are failed and
128
Displaying detailed status for a data group
the number of objects with pending activity entries. The first number in these fields
indicates the number of objects defined to the data group that have a status of
*FAILED. The second number indicates the number of objects with active
(pending) activity entries or that have new or in-progress recoveries.
• The State field identifies the state of the remote journal link. The values for the
state field are the same as those which appear on the Work with RJ Links display.
This field is not shown if the data group uses source-send processes for user
journal replication.
More detailed breakouts of error and recovery counts are available on the object
views (F7) and Database views (F8). When you use the MIMIX portal application in
Vision Solutions Portal, you can see the actual objects affected and details about the
recovery actions by using the Correction Activity action from the Data Groups portlet.
Source statistics: The middle of the display shows status and summarized statistics
for the journals being used for replication and the processes that read from them. The
following process fields are possible:
System - Identifies the current source system definition. The status value is an
indication of the success in communicating with that system.
Jrn Mgr - Displays the status of the journal manager process for the source
system.
DA Poll - Displays the status of the data area poller. This field is present only if the
data group replicates data areas using this process.
RJLNK Mon - Displays status of the RJLNK monitor on the source system. This
field is present only for data groups that use remote journaling.
Database (Link or Send) - Identifies the status of the process which transfers user
journal entries from the source system to the target system.
Link - Displayed when the data group is configured for remote journaling. The
status is that of the of the RJ link.
Send -Displayed when the data group id configured for MIMIX source-send
processes. The status is that of the database send process.
Object Send - Displays a summation of status from the object send, object
retrieve, and container send processes. The highest priority status from each
process determines the status displayed. Use F7 (Object view) to see the
individual processes. When the data group uses a shared object send job, either
the value *SHARED or a three-character job prefix is displayed below the Send
process status, The value *SHARED indicates that the data group uses the MIMIX
generated shared object send prefix for this source system. A three-character
prefix indicates this data group uses a shared object send job on this system that
is shared only with other data groups which specify the same prefix.
For the Database and Object processes, additional fields identify current journal
information, the last entry that has been read by the process, and statistics related to
arrival rate, entries not read, and estimating the time to read.
Current - For the Database Send and Object Send processes, this identifies the
last entry in the currently attached journal receiver. This information is used to
show the arrival rate of entries to the journals.
129
Note: If the data group uses remote journaling, current information is displayed
in two rows, Source jrn and RJ tgt jrn. The source journal sequence
number refers to the last sequence number in the local journal on the
source system. The remote journaling target journal sequence number
refers to the last sequence number in the associated remote journal on the
target system.
Transactions per hour - For current journal information, this is based on the
number of entries to arrive on the journal over the elapsed time the statistics have
been gathered. For last read information, this is based on the actual number of
entries that have been read over the elapsed time the statistics have been
gathered.
Last Read - Identifies the journal entry that was last read and processed by the
object send, database send, or database reader.
Transactions per hour - For current journal fields, this is based on the number of
entries to arrive on the journal over the elapsed time the statistics have been
gathered. For last read fields, this is based on the actual number of entries that
have been read over the elapsed time the statistics have been gathered and will
change due to elapsed time and the rate at which entries arrive in the journal.
Entries not read - This a calculation of the number of journal entries between the
last read sequence number and the sequence number of the last entry in the
current receiver for the source journal. An asterisk (*) preceding this field indicates
that the journal receiver sequence numbers have been reset between the last
entry in the current receiver and the last read entry.
Estimated time to read - This is a calculation using the entries not read and the
transactions per hour rate. This calculation is intended to provide an estimate of
the length of time it may take the process (database reader, database send, or
object send) to complete reading the journal entries.
Target statistics: The lower part of the display shows status and summarized
statistics for all target system processing. The following process fields are possible:
System - Identifies the current target system definition. The status value is an
indication of the success in communicating with that system.
Jrn Mgr - Displays the status of the journal manager process for the target system.
DB Rdr - Displays status of the database reader. This field is present only for data
groups that use remote journaling.
AP Maint - Displays status of the access path maintenance1 processes. This field
is only present when optimized access path maintenance has been enabled.
RJLNK Mon - Displays status of the RJLNK monitor on the target system. This
field is present only for data groups that use remote journaling.
Sys Jrn Insp - Displays the status of target journal inspection for the system
journal (QAUDJRN) on the target system of the data group. This field is displayed
130
Displaying detailed status for a data group
when the journal definition for the system journal on the current target system
permits target journal inspection and the data group is enabled and has been
started at least once.
User Jrn Insp - Displays the status of target journal inspection for the user journal
on the target system of the data group. This field is displayed when the journal
definition for the user journal on the current target system permits target journal
inspection and the data group is enabled, performs user journal replication,
permits journaling on target, and has been started at least once.
DB Apply and Obj Apply - Each field displays the combined status for the apply
jobs in use by the process. For each process, additional fields show statistics for
the last received journal sequence number, number of unprocessed entries,
approximate number of transactions per hour being processed, and the
approximate amount of time needed to apply the unprocessed transactions for all
database or object apply sessions.
131
Blue - The number of active jobs is equal to or greater than the minimum number
of processes.
Figure 15 and Figure 18 show the active count highlighted.
132
Displaying detailed status for a data group
133
Problem counts: In the top right corner of database views 1 and 2 (Figure 18 and
Figure 19), these fields display combined counts of replicated entries, new or in-
progress recoveries, and errors for file entries, IFS tracking entries, and object
tracking entries:
• File and Tracking entries - The number of database files, journaled IFS objects,
and journaled data areas and data queues being replicated for the data group.
• Not journaled Src Tgt - The number of files and objects identified by file entries
and tracking entries that are currently not journaled or that are journaled to a
journal that is different than the configured journal on the source and target
systems. If the number of not journaled errors on either system exceeds 99,999,
that system’s field displays +++++.
• Held error and Rcy - The number of files and objects identified by file entries and
tracking entries that are held due to an error (*HLDERR) status, and the number
that have new or in-progress recovery actions for problems detected by MIMIX.
• Access path maint. errors - The number of database files with access path
maintenance errors.
• Held for other reasons - The number of files and objects identified by file entries
and tracking entries that are held or inactive for other reasons.
Database view 4 (Figure 21) separates this information into columns for file entries,
IFS tracking entries, and object tracking entries.
Apply sessions: If a data group has multiple database apply sessions you will see
an entry for each session in the Apply Status column on database views 1, 2, and 3
(Figure 18, Figure 19, and Figure 20). Each session has its own status value. In these
sample figures there is only one apply session (A) which is active (-A).
In database view 1, the Open Commit column displays the value *DLK when there is
an open commit cycle that has caused a deadlock condition for the apply process
which MIMIX could not automatically resolve. User action is necessary to resolve the
commit cycle.
134
Displaying detailed status for a data group
135
Figure 19. Data group database status—view 2.
In this example, the Link status of A and the presence of the Reader status indicates that the
data group uses remote journaling. The display also shows that access path maintenance is
used and active, and that journal standby state is active and journal caching is not active.
136
Identifying replication processes with backlogs
Figure 21. Data group detail status—database view 4. In this example, the combined number
of file and tracking entries and number of recoveries (from database views 1 and 2) are sepa-
rated into columns for file entries, IFS tracking entries, and object tracking entries.
137
appropriate fields for each process.
Table 27. Location of fields which identify backlogs and threshold conditions for replication processes
Process Description
RJ Link The backlog is the quantity of source journal entries that have not been transferred from the
local journal on the source system to the remote journal on the target system. The time
difference between the last entry in each journal can also be an indication of a backlog.
DB Reader The backlog is the quantity of journal entries that are waiting to be read by the process. The
or DB Send time difference between the last entry that was read by the process and the last entry in the
journal on the source system can also be an indication of a backlog. This may be a temporary
condition due to maximized log space capacity. If the log space capacity was reached, the
database reader job will be idle until the database apply job is able to catch up. If the
condition is unable to resolve itself, action may be required.
DB Apply The backlog is the number of entries waiting to be applied to the target system. Each apply
session is listed as a separate entry with its own backlog. If a virtual switch procedure is in
progress, a backlog may occur during the testing and recovery phases of a virtual switch.
Object The backlog is the quantity of journal entries that have not been read from the system journal.
Send The time difference between the last entry that was read by the process and the last entry in
the system journal can also be an indication of a backlog.
Multiple data groups sharing the object send job is one possible cause of a persistent
backlog.
Object The backlog is the number of entries for which MIMIX is waiting to retrieve objects.
Retrieve
Object views 1 and 2 Retrieve Backlog • Retrievers, Act column
• Retrieve Backlog
138
Data group status in environments with journal cache or journal state
Table 27. Location of fields which identify backlogs and threshold conditions for replication processes
Process Description
Container The backlog is the number of packaged objects for entries that are waiting to be sent to the
Send target system.
Object The backlog is the number of entries waiting to be applied to the target system. If a virtual
Apply switch procedure is in progress, a backlog may occur during the testing and recovery phases
of a virtual switch.
Notes:
1. When highlighted, the threshold journal entry quantity criterion is exceeded.
2. When highlighted the threshold time criterion is exceeded.
If you repeatedly have issues with processes in thresholds, the settings which control
them may need adjustment. For more information, see “Fine-tuning backlog warning
thresholds for a data group” in the MIMIX Administrator Reference book.
139
The Jrn State and Cache (Src and Tgt) fields reflect journal standby state and journal
caching actual values for the journals when the IBM high availability performance
enhancements are installed on the systems defined to the data group. These fields
appear on database views 1 and 2 (Figure 18 and Figure 19). The target journal state
and cache values are set on the journal when the database apply session is started.
Journal State - The status values indicate the actual state value for the source and
target journals. Table 28 shows the possible values for each field.
Journal Cache - The status indicate the actual cache value for the source and
target journals. Table 29 shows the possible values for each field.
For each system (Src or Tgt) status of the journal state is shown first, followed by the
status of the journal cache. If a problem exists with journal state or journal cache, the
data group name is also highlighted with the same color. For information about
resolving journal cache or journal state problems, see “Resolving a problem with
journal cache or journal state” on page 141.
Either system White U Unknown. MIMIX was not able to retrieve values, possibly
because the journal environment has not yet been built.
Source Red S Source journal is in standby state but that state is not
expected.
Target Yellow S Target journal state or cache is not as expected and the
database apply session is active
blank blank The IBM feature is installed but the data group is
configured to not journal on the target system.
140
Data group status in environments with journal cache or journal state
Target Yellow Y Target journal cache value not as expected and the
database apply session is active.
blank blank The IBM feature is installed but the data group is
configured to not journal on the target system.
4. Source system journal state (first Src: value) - If the source system state is red
and the value for the journal state is standby (S) or inactive (I), the journal state
must be changed and all data replicated through the user journal must be
synchronized. Do the following:
141
a. Press F12 (Cancel) to return to the Work with Data Groups display. Note which
system is specified as the source system for the data group.
b. Use option 45 (Journal Definitions) to view the journal definitions used for the
data group in error.
c. On the Work with Journal Definitions display, determine the journal name and
library specified for the system that is the source system for the data group.
d. Specify the name and library of the source system journal in the following
command:
CHJRN CHGJRN JRN(library/name) JRNSTATE(*ACTIVE)
e. All data replicated through the user journal must be synchronized. For detailed
information about synchronizing a data group, refer to your Runbook or to the
MIMIX Administrator Reference book.
5. Source system journal cache (second Src: value) - If the source system cache
is yellow, the actual status does not match the configured value in the journal
definition used on the source system. Do the following:
a. Press F12 (Cancel) to return to the Work with Data Groups display. Note which
system is specified as the source system for the data group.
b. Use option 45 (Journal Definitions) to view the journal definitions used for the
data group in error.
c. On the Work with Journal Definitions display, use option 5 (Display next to the
journal definition listed for the source system.
d. Check the value of the Journal caching (JRNCACHE) parameter.
e. Determine which value is appropriate for journal cache, the configured value or
the actual status value. Once you have determined this, either change the
journal definition value or change the journal cache (CHGJRN command) so
that the values match.
6. Target system state (first Tgt: value) or Target system cache (second Tgt:
value) - If the target system state or cache is yellow, the actual value for state or
cache does not match the configured value. Do the following:
a. Press F12 (Cancel) to return to the Work with Data Groups display. Note which
system is specified as the target system for the data group.
b. Use option 45 (Journal Definitions) to view the journal definitions used for the
data group in error.
c. On the Work with Journal Definitions display, use option 5 (Display next to the
journal definition listed for the target system.
d. Check the value of the following parameters, as needed:
• Target journal state (TGTSTATE)
• Journal caching (JRNCACHE)
e. Determine why the actual status of the journal state or journal cache does not
match the configured value of the journal definition used on the target system.
142
Data group status in environments with journal cache or journal state
f. Determine which values are appropriate for journal state and journal cache, the
configured value or the actual status value. Once you have determined this,
either change the journal definition value or change the journal state or cache
(CHGJRN command) so that the values match.
143
journal.
7. Press Enter to start journaling.
8. Press F5 (Refresh) to update and view the current journaling status.
144
CHAPTER 8 Working with audits
Audits are defined by and invoked through rules and influenced by policies. Aspects
of audits include schedules, status, reported results, and their compliance status.
MIMIX is shipped so that auditing can occur automatically. For day-to-day operations,
auditing requires minimal interaction to monitor audit status and results. MIMIX user
interfaces separate audit runtime status, compliance status, and scheduling
information onto different views to simplify working with audits. Compliance errors and
runtime errors require different actions to correct problems.
This chapter provides information and procedures to support day-to-day operations
as well as to change aspects of the auditing environment. The following topics are
included.
• “Auditing overview” on page 146 describes concepts associated with auditing and
describes the differences between automatic priority audits and automatic
scheduled audits.
• “Guidelines and considerations for auditing” on page 151 identifies considerations
for specific audits, auditing best practices, and recommendations for checking the
audit results.
• “Displaying scheduling information for automatic audits” on page 155 describes
how to access views of the Work with Audits display where you can display when
prioritized audits will run and when scheduled audits will run.
• “Displaying audit runtime status and compliance status” on page 157 describes
how to access audit status and shows examples of additional views of the Work
with Audits display for status and compliance information.
• “Resolving auditing problems” on page 162 describes how to check for the
presence of auditing problems, resolve problems with runtime status, check the
job log of an audit, and resolve audit compliance problems.
• “Running an audit immediately” on page 169 describes how to perform an audit
manually.
• “Ending an audit” on page 170 describes how to end an audit that is in progress or
that has been requested and is waiting to run.
• “Displaying audit history” on page 171 describes how to display history for specific
audits of a data group.
• “Working with audited objects” on page 173 describes how to display a list of
objects compared by one or more audits.
• “Working with audited object history” on page 176 describes how to access the
audit history for a specific object.
145
Auditing overview
All businesses run under rules and guidelines that may vary in the degree and in the
methods by which they are enforced. In a MIMIX environment, auditing provides rules
and enforcement of practices that help maintain availability and switch-readiness at
all times.
Not using or limiting audit use does little to confirm the integrity of your data. These
approaches can mean lost time and issues with data integrity when you can least
afford them.
In reality, successful auditing means finding the right balance somewhere between
these approaches:
• Audit your entire replication environment every day. The benefit of this approach
is knowing that your data integrity exposure is limited to data that changed since
the last audit. The trade-off with this approach can be time and resources to
perform audits.
• Audit only replicated data that “needs” auditing. This approach can be faster and
use fewer resources because each audit typically has fewer objects to check. The
trade-offs are determining what needs auditing and knowing when objects were
last audited.
MIMIX makes auditing easy by automatically auditing all objects periodically and
auditing a subset of objects every day. MIMIX also provides the ability to fine-tune
aspects of auditing behavior and their automatic submission and the ability to
manually invoke an audit at any time.
For more information about how automatic audits are scheduled and how to change
schedules, see “Scheduling audits and reports” on page 60.
Components of an audit
Together, three components identify a unique audit. Each component must exist to
allow an audit to run.
Rule - A program by which an audit is defined and invoked. Each rule shipped with
MIMIX pre-defines a compare command to be invoked and the possible actions
that can be initiated, if needed, to correct detected problems. When invoked, each
rule can check only the class of objects associated with its compare command.
Names of rules shipped with MIMIX begin with the pound sign (#) character.
Data group - A data group provides the context of what to check and how results
are reported. Multiple audits (rules) exist for each data group.
Note: Audits are not allowed to run against disabled data groups.
Schedule - Each unique combination of audit rule and data group has its own
schedule, by which it is automatically submitted to run. MIMIX ships default
scheduling information associated with each shipped rule. Scheduling can be
adjusted for individual audits. A manually invoked audit can be thought of as an
immediate override of scheduling information.
146
Auditing overview
Although people use the terms “audit” and “rule” interchangeably, a rule is a
component of an audit. The process of auditing runs a rule program.
147
differences after the compare phase of the previous audit run (this is called
differences object auditing and is only available from VSP).
Prioritized auditing can reduce the impact of auditing on resources and performance.
This benefits customers who cannot complete IFS audits, cannot audit every day, or
do not audit at all because of time or resource issues.
When both types of auditing are used, you can achieve a balance between verifying
data integrity and resources. Either or both types of automatic auditing can be
disabled, although that is not recommended.
Objects not equal Objects for which MIMIX detected Objects in this category
differences since the start of the most have the highest priority
recent audit are in this category. This and are always
includes: selected.
• Objects that had any value other than
*EQ in their most recent audit, including
those with differences that were
automatically resolved.
• Objects for which target system
differences were detected by target
journal inspection or by virtual switch
processing.
• Objects with problems detected by
database replication that are being
tracked for automatic recovery.
• When MIMIX is configured for *DLY
commit mode, objects for which commit
rollback entries still exist in the journal
when those objects were part of a
commit cycle that MIMIX automatically
processed (by a commit operation) to
clear a deadlock condition.
148
Auditing overview
New objects A new object is one that has not been Objects in these
audited since it was created. categories are eligible
for selection according
Changed objects A changed object is one that has been to the category
modified since the last time it was audited. frequency specified in
Unchanged An unchanged object is one that has not the Priority audit policy.
objects been modified since the last time it was
audited.
The #FILDTA audit always selects all members of a file for which auditing is less than 100
percent complete. The occurs in all of the above object selection categories.
Initially, the objects selected by a prioritized audit may be nearly the same as those
selected by a scheduled audit. However, over time the number of objects selected by
a prioritized object stabilizes to a subset of those selected by a scheduled audit.
When both scheduled and priority audits are allowed for the same rule and data
group, MIMIX may not start a prioritized audit if the scheduled audit will start in the
near future.
149
When audit recoveries are enabled, you can control the severity level of the
notifications that are returned when the rule ends in error with the Notification severity
policy.
You can also view job logs associated with notifications and recoveries. Job logs are
accessible from the system on which the audit comparison or recovery job ran.
Audit compliance
Compliance is an indication of whether an audit ran within the time frame of the
compliance thresholds set in auditing policies. The Audits included for compliance
policy1 determines which audits are checked for compliance.
When an audit that is being checked for compliance reaches one of the compliance
thresholds, its audit compliance status is reported as a *ATTN or *ACTREQ on the
Compliance summary view of the Work with Audits display. Audit compliance status is
included in the auditing status that is rolled up to other status values within a MIMIX
instance.
Compliance is checked based on the last completed compare date. Compliance
determines whether the date of the last completed compare completed by an audit is
within the range set by policies. The Audit warning threshold policy and the Audit
action threshold policy define when to indicate that an audit is approaching or
exceeding that range.
For audits configured for scheduled object auditing or both scheduled and prioritized
object auditing, compliance status is based on the last run of a scheduled audit or a
user-invoked audit. For audits configured for only prioritized object auditing,
compliance status is based on the last run, which may have been a prioritized audit or
a user-invoked audit. A user-invoked audit or a scheduled audit checked all objects
that are configured for the data group and within the class of objects checked by the
audit whereas a prioritized audit may have checked only a subset of those objects.
Audit runs that only selected objects that were previously different (*DIF) are never
considered in assessing compliance.
150
Guidelines and considerations for auditing
151
Considerations for specific audits
#DGFE audit - This audit is not eligible for prioritized auditing because it checks
configuration data, not objects. As a result, configuration problems for a data group
can only be detected when a scheduled audit or a manually invoked audit runs.
Run the #DGFE audit during periods of minimal MIMIX activity to ensure that
replication is caught up and that added or deleted objects are reflected correctly in the
journal. If the command is run during peak activity, it may contain errors or indicate
that files are in transition.
In addition to regularly scheduled audits, check your configuration using the #DGFE
audits for your data groups whenever you make configuration changes, such as
adding an application or creating a library. Running the audit prior to audits that
compare attributes ensures that those audits will compare the objects and attributes
you expect to be present in your environment.
#DLOATR audit - This audit supports multiple levels of comparisons. The level used
is controlled by the value of the Audit level policy in effect when the audit runs. The
#DLOATR audit compares attributes as well as data for objects defined to a data
group when audit level 20 or 30 is used. Audit level 10 compares only attributes.
When data is compared the audit may take longer to run and may affect performance.
#FILDTA audit - This audit supports multiple levels of comparisons. The level used is
is controlled by the value of the Audit level policy in effect when the audit runs. The
#FILDTA audit compares all file member data defined for file members defined to a
data group only when audit level 30 is used. Level 10 and level 20 compare 5 percent
and 20 percent of data, respectively. Lower audit levels may take days or weeks to
completely audit file data. New files created during that time may not be audited.
Regardless of the audit level you use for regular auditing, Vision Solutions strongly
recommends running a level 30 audit before switching.
#IFSATR audit - This audit supports multiple levels of comparisons. The level used is
controlled by the value of the Audit level policy in effect when the audit runs. The
#IFSATR audit compares data when audit level 20 or 30 is used. At level 10, only
attributes are compared. Regardless of the audit level you use for regular auditing,
Vision Solutions strongly recommends running a level 30 audit before switching.
For journaled IFS objects (*DIR, *STMF, *SYMLNK) identified by IFS tracking entries,
a prioritized audit run of the #IFSATR audit will select the eligible objects from each
priority category.
For non-journaled IFS directory (*DIR) objects identified for system journal replication,
a prioritized audit run of the #IFSATR audit will select directory objects based on their
eligibility for each priority category and may include stream files (*STMF) and
symbolic links (*SYMLNK) within the directory under these conditions:
• When the *STMF or *SYMLNK object within the selected directory is new or
changed since the last audit run, or had differences in the last audit run.
• When the directory has been added to the configuration or moved into the
configured name space since the last audit run.
• When the directory meets selection criteria for either the unchanged objects or
audited with no differences priority category. This ensures that the objects within
152
Guidelines and considerations for auditing
the directory are periodically checked according to the frequency specified in the
priority audit schedule.
#MBRRCDCNT audit - This audit compares the number of current records
(*CURRDS) and the number of deleted records (*NBRDLTRCDS) for physical files
that are defined to an active data group. Equal record counts suggest but do not
guarantee that files are synchronized.
The #MBRRCDCNT audit does not have a recovery phase. Differences detected by
this audit appear as not recovered in the Audit Summary.
In some environments using commitment control, the #MBRRCDCNT audit may be
long-running. Refer to the MIMIX Administrator Reference book for information about
improve performance of this audit.
153
environment for the cause may determine that a change is needed in the
environment, in the MIMIX configuration, or in both. Trends may also indicate a
MIMIX problem, such as reporting an object as being recovered when it was not.
Report these scenarios to MIMIX CustomerCare. You can do this by creating a
new case using the Case Management page in Support Central.
154
Displaying scheduling information for automatic audits
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Audit summary F11=Schedule settings
F14=Audited objects F17=Sort sched. time F24=More keys
155
Figure 23. Audit Schedule Summary, view - settings for scheduled (full) audits.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Audit summary F11=Priority settings
F14=Audited objects F17=Sort sched. time F24=More keys
Figure 24. Audit Schedule Summary, view - settings for priority audits.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Audit summary F11=Priority settings
F14=Audited objects F17=Sort sched. time F24=More keys
156
Displaying audit runtime status and compliance status
157
status value *IGNOBJ may be considered a warning or informational. These
severities affect the sort order.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Compliance summary F11=Last run
F14=Audited objects F16=Inst. policies F23=More options F24=More key
F11 changes the columns displayed so you can see the policies in effect when the
audit was last run (Figure 26).
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Compliance summary F11=DG definition
F14=Audited objects F16=Inst. policies F23=More options F24=More key
The Last Run columns show the values of policies in effect at the time the audit was
last run through its compare phase.
158
Displaying audit runtime status and compliance status
Recovery identifies the value of the automatic audit recovery policy. When this
policy is enabled, after the comparison completes, MIMIX automatically starts
recovery actions to correct differences detected by the audit. Recovery may also
indicate a value of *DISABLED if a condition checked by the Action for running
audits (RUNAUDIT) policy existed and the policy value for that condition specified
*CMP, preventing audit recoveries from running.
Level identifies the value of the audit level policy. The audit level determines the
level of checking performed during the compare phase of the audit. If an audit was
never run, the value *NONE is displayed in both columns.
159
Audits - Compliance summary view and alternate columns
The Compliance summary view of the Work with Audits display shows the
Compliance column and the full name of the data group checked by the audit (Figure
27). This view also shows the timestamp of when the compare phase ended in the
Compare End column.
The list is initially sorted by compliance status. To sort the list by scheduled time, use
F17.
Audits with a compliance status of *OK includes audits that are in compliance, audits
that have been excluded from compliance checks by the Audits included for
compliance policy, and audits that have never been run because the associated data
group is new or has been recently enabled.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Schedule summary F11=Next scheduled
F14=Audited objects F17=Sort sched. time F24=More keys
160
F11 changes the columns displayed so you can see when the scheduled audit run will
occur (Figure 28). The scheduled date and time in this view do not apply to prioritized
audit runs.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F10=Schedule summary F11=DG definition
F14=Audited objects F17=Sort sched. time F24=More keys
161
Resolving auditing problems
Audits run for individual data groups. To check whether auditing problems exist for
data groups in an installation, do the following:
1. To access the Work with Data Groups display, either select option 6 from the
MIMIX Basic Main menu or select option 1 from the MIMIX Intermediate Main
menu and press Enter.
2. The Audits field located in the upper right of the display identifies the total number
of audits in the installation that require action to correct a problem or to prevent a
situation from becoming a problem.
Work with Data Groups CHICAGO
11:02:05
Type options, press Enter. Audits/Recov./Notif.: 079 / 002 / 003
5=Display definition 8=Display status 9=Start DG
10=End DG 12=Files needing attention 13=Objects in error
14=Active objects 15=Planned switch 16=Unplanned switch ...
---------Source--------- --------Target-------- -Errors-
Opt Data Group System Mgr DB Obj DA System Mgr DB Obj DB Obj
162
Resolving auditing problems
You may also need to view the output file or the job log, which are only available from
the system where the audits ran. In most cases, this is the management system.
Work with Audits
System: AS01
Type options, press Enter.
5=Display 6=Print 7=History 8=Recoveries 9=Run rule 10=End
14=Audited objects 46=Mark recovered ...
Status Action
*FAILED If the failed audit selected objects by priority and its timeframe for starting has not passed,
the audit will automatically attempt to run again.
The rule called by the audit failed or ended abnormally.
• To run the rule for the audit again, select option 9 (Run rule). This will check all objects
regardless of how the failed audit selected objects to audit.
• To check the job log, see “Checking the job log of an audit” on page 166.
163
Table 31. Addressing problems with audit runtime status
Status Action
*ENDED The #FILDTA audit or the #MBRRCDCNT audit ended either because of a policy in effect
or the data group status. The recovery varies according to why the audit ended.
To determine why the audit ended:
1. Select option 5 (Display) for the audit and press Enter.
2. Check the value indicated in the Reason for ending field. Then perform the appropriate
recovery, below.
If the reason for ending is *DGINACT, the data group status became inactive while the
audit was in progress.
1. From the command line, type WRKDG and press Enter.
• If all processes for the data group are active, skip to Step 2.
• If processes for the data group show a red I, L, or P in the Source and Target columns,
use option 9 (Start DG). Note that if the data group was inactive for some time, it may
have a threshold condition after being started. Wait for the threshold condition to clear
before continuing with Step 2.
2. When the data group is active and does not have a threshold condition, return to the
Work with Audits display and use option 9 (Run rule) to run the audit. This will check all
objects regardless of how the ended audit had selected objects to audit.
If the reason for ending is *MAXRUNTIM, the Maximum rule runtime (MAXRULERUN)
policy in effect was exceeded. Do one of the following:
• Wait for the next priority-based audit to run according to its timeframe for starting.
• Change the value specified for the MAXRULERUN policy using “Setting policies -
general” on page 31. Then, run the audit again, either by using option 9 (Run rule) or by
waiting for the next scheduled audit or priority-based audit to run.
If the reason for ending is *THRESHOLD, the DB apply threshold action (DBAPYTACT)
policy in effect caused the audit to end.
1. Determine if the data group still has a threshold condition. From the command line, type
WRKDG and press Enter.
2. If the data group shows a turquoise T in the Target DB column, the threshold exceeded
condition is still present. Wait for the threshold to resolve. If the threshold persists for an
extended time, you may need to contact your MIMIX administrator.
3. When the data group no longer has a threshold condition, return to the Work with Audits
display and use option 9 (Run rule) to run the audit. (This will check all objects.)
164
Resolving auditing problems
Status Action
*DIFFNORCY The comparison performed by the audit detected differences. No recovery actions were
attempted because of a policy in effect when the audit ran. Either the Automatic audit
recovery policy is disabled or the Action for running audits policy prevented recovery
actions while the data group was inactive or had a replication process which exceeded its
threshold.
If policy values were not changed since the audit ran, checking the current settings will
indicate which policy was the cause. Use option 36 to check data group level policies and
F16 to check installation level policies.
• If the Automatic audit recovery policy was disabled, the differences must be manually
resolved.
• If the Action for running audits policy was the cause, either manually resolve the
differences or correct any problems with the data group status. You may need to start
the data group and wait for threshold conditions to clear. Then run the audit again.
To manually resolve differences do the following:
1. Type 7 (History) next to the audit with *DIFFNORCY status and press Enter.
2. The Work with Audit History display appears with the most recent run of the audit at the
top of the list. Type 8 (Display difference details) next to an audit to see its results in the
output file.
3. Check the Difference Indicator column. All differences shown for an audit with
*DIFFNORCY status need to be manually resolved. For more information about the
possible values, see “Interpreting audit results - supporting information” on page 360.
To have MIMIX always attempt to recover differences on subsequent audits, change the
value of the automatic audit recovery policy.
*NOTRCVD The comparison performed by the audit detected differences. Either some attempts to
recover differences failed, or the audit job ended before recoveries could be attempted on
all differences.
Note: For audits using the #MBRRCDCNT rule, automatic recovery is not possible. Other audits,
such as #FILDTA, may correct the detected differences.
Do the following:
1. Type 7 (History) next to the audit with *NOTRCVD status and press Enter.
2. The Work with Audit History display appears with the most recent run of the audit at the
top of the list. Type 8 (Display difference details) next to an audit to see its results in the
output file.
3. Check the Difference Indicator column. Any objects with a value of *RCYFAILED must
be manually resolved. For any objects with values other than *RCYFAILED or
*RECOVERED, run the audit again. For more information about the possible values,
see “Interpreting audit results - supporting information” on page 360.
165
Table 31. Addressing problems with audit runtime status
Status Action
*NOTRUN The audit request was submitted but did not run. Either the audit was prevented from
running by the Action for running audits policy in effect, or the data group was inactive
when a #FILDTA or #MBRRCDCNT audit was requested.
Note: An audit with *NOTRUN status may or may not be considered a problem in your environment.
The value of the Audit severity policy in effect determines whether the audit status *NOTRUN
has been assigned an error, warning, or informational severity. The severity determines
whether the *NOTRUN status rolls up into the overall status of MIMIX and affects the order in
which audits are displayed in interfaces.
A status of *NOTRUN may be expected during periods of peak activity or when data group
processes have been ended intentionally. However, if the audit is frequently not run due to
the Action for running audits policy, action may be needed to resolve the cause of the
problem.
To resolve this status:
1. From the command line, type WRKDG and press Enter.
2. Check the data group status for inactive or partially active processes and for processes
with a threshold condition.
3. When the data group no longer has a threshold condition and all processes are active,
return to the Work with Audits display and use option 9 (Run rule) to run the audit. (This
will check all objects.)
*IGNOBJ The audit ignored one or more objects because they were considered active or could not
be compared because of locks or authorizations that prevented access. All other selected
objects were compared and any detected differences were recovered.
Note: An audit with *IGNOBJ status may or may not be considered a problem in your environment.
The value of the Audit severity policy in effect determines whether the audit status *IGNOBJ
has been assigned a warning or informational severity. The severity determines whether the
*IGNOBJ status rolls up into the overall status of MIMIX, determines whether the ignored
objects are counted as differences, and affects the order in which audits are displayed in
interfaces.
To resolve this status:
1. From the Work with Audits display on the target system, type 7 (History) next to the audit
with *IGNOBJ status and press Enter.
2. The Work with Audit History display appears with the most recent run of the audit at the
top of the list. Type 8 (Display difference details) next to an audit to see its results in the
output file.
3. Check the Difference Indicator column to identify the objects with a status of *UN or *UA.
4. When the locks are released or replication activity for the object completes, do one of
the following:
• Wait for the next prioritized audit to run.
• Run the audit manually using option 9 (Run rule) from the Work with Audits display,
For more information about the values displayed in the audit results, see “Interpreting
audit results - supporting information” on page 360.
166
Resolving auditing problems
You must display the notifications from an audit in order to view the job log. Do the
following:
1. From the Work with Audits display, type 7 (History) next to the audit and press
Enter.
2. The Work with Audit History display appears with the most recent run of the audit
at the top of the list.
3. Use option 12 (Display job) next to the audit you want and press Enter.
4. The Display Job menu opens. Select option 4 (Display spooled files). Then use
option 5 (Display) from the Display Job Spooled Files display.
5. Look for messages from the job log for the audit in question. Usually the most
recent messages are at the bottom of the display.
Message LVE3197 is issued when errors remain after an audit completed.
Message LVE3358 is issued when an audit failed. Check for following
messages in the job log that indicate a communications problem (LVE3D5E,
LVE3D5F, or LVE3D60) or a problem with data group status (LVI3D5E,
LVI3D5F, or LVI3D60).
167
3. To resolve a problem with audit compliance, the audit in question must be run and
complete its compare phase.
• To see when the scheduled run of the audit will occur, press F11.
• To see when both scheduled and prioritized audits will run, press F10 to
access the Audit summary view, then use F11 to toggle between views.
• To run the audit now, select option 9 (Run rule) and press Enter. This action will
select all replicated objects associated with the class of the audit. For more
Information, see “Running an audit immediately” on page 169.
168
Running an audit immediately
169
Ending an audit
Only active or queued audits can be ended. This includes audits with the following
statuses: Currently comparing (*CMPACT), Currently recovering (*RCYACT), or
Currently waiting to run (*QUEUED).
You must end active or queued audits from the system that originated the audit. You
can end active or queued audits from any view of the Work with Audits display. This
procedure uses the Status view.
To end an active or queued audit, do the following:
1. From the MIMIX Intermediate Main Menu, select option 6 (Work with audits) and
press Enter. Then use F10 as needed to access the Audit summary view.
2. Check the value shown in the Audit Status column. Press F1 (Help) for a
description of status values.
3. Type option 10 (End) next to the active or queued audit you want to end and
press Enter.
4. Audits in *CMPACT or *QUEUED status are set back to their previous status
values. Audits in *RCYACT status are set according to the completed comparison
result as well as the results of any completed recovery actions.
170
Displaying audit history
BOTTOM
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F11=Summary results
F12=Cancel F13=Repeat F14=Audited objects F21=Print list
171
The summary results view (Figure 30) shows the total number of objects selected by
the audit and whether the objects selected were the result of a priority audit or a
scheduled audit.
BOTTOM
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F11=Compare results
F12=Cancel F13=Repeat F14=Audited objects F21=Print list
The compare results view (Figure 31) shows the duration of the audit as well as
statistics for the compare phase of the audit.
BOTTOM
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F11=Recovery results
F12=Cancel F13=Repeat F14=Audited objects F21=Print list
172
Working with audited objects
When viewing the Work with Audit History display from the system on which the audit
request originated, you can use options to view the object difference details detected
by the audit (option 8), the job log for the audit (option 9), and a list of objects that
were audited (option 14).
173
Audit start field is located at the top of the display in this case. If the selected audit is
the audit run with the latest start date, (*LAST) will also appear in the Audit start field.
Figure 32. Work with Audited Objects display for a single audit.
Audited Object
Opt Status Type Name
_ *NE *FILE L00SAMPLEA/RJFILE1
_ *EQ *FILE L00SAMPLEA/RJFILE2
_ *EQ *FILE L00SAMPLEA/RJFILE3
_ *RCVD *FILE L00SAMPLEA/RJFILE4
BOTTOM
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F12=Cancel F13=Repeat
F18=Subset F21=Print list F22=Display entire field
When the list includes objects from multiple audits, the display appears as shown in
Figure 33, with the specific audit rule and start time displayed in columns. A > symbol
next to an object name indicates a long object path name exists which can be viewed
with F22.
File member information is not automatically displayed. However, you can use F18 to
change subsetting criteria to include members. When member information is
displayed, the name is in the format: library/file(member). Also, the information
displayed for file members may not be from the most recently performed audit.
Because members can be compared by several audits, the most recent run of each of
those audits is evaluated. The evaluated audit run with most severe status is
displayed, even if it is not the most recently performed audit of the evaluated audit
174
Working with audited objects
runs. For all other objects, the information displayed is from the most recent audit run
that compared the object.
Figure 33. Work with Audited Objects display with all audits displayed.
More...
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F12=Cancel F13=Repeat
F18=Subset F21=Print list F22=Display entire field
You can select option 5 to view the details of the audit in which the object was
compared, such as the audit compare and recovery timestamps, and option 9 to view
auditing history for a specific object.
175
system is included in the audit details, which you can view using option 5
(Display).
2. Press F14 (Audited objects). The Work with Audited Objects (WRKAUDOBJ)
command appears.
3. Specify the Data group definition for which you want to see audited objects.
4. Specify the value you want for Object type and press Enter.
5. Additional fields appear based on the value specified in Step 5. Specify values to
define the criteria for selecting the objects to be displayed.
Note: The value specified for Member (MBR) determines whether member-level
objects are selected for their object history. The members selected are not
automatically displayed in the list. To include any selected members,
press F10 (Additional parameters), then specify *YES for Include member
(INCMBR).
6. Press Enter to display the list of objects from the retained history details.
176
Working with audited object history
displayed, there is only one possible audit rule so the Audit rule field is located at the
upper right of the display.
Figure 34. Work with Audited Obj. History display showing audit history for a file member
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F12=Cancel F13=Repeat
F21=Print list F22=Display entire field
From this display you can use option 5 to view details of the audit in which the object
was compared, such as its audit compare and recovery timestamps, and option 8 to
view object difference details that were detected by the audit.
177
Working with system-level processes
MIMIX uses several processes that run at the system level to support the replication
environment and provide additional functionality.
• System-level processes reported on the Work with Systems display (WRKSYS
command) include the system manager, journal manager, target journal
inspection, collector services, and if needed, cluster services.
• Also, single-system processes that monitor specific functions are reported on the
Work with Monitors display (WRKMON command) on each system in the
installation.
Typically, these processes are automatically started and ended when MIMIX is started
or ended. However, you may need to start or end individual processes when resolving
problems.
The following topics are included in this chapter to help you resolve problems with
system level processes:
• “Displaying status of system-level processes” on page 180 describes how to
check for expected status values on the Work with Systems display and identifies
where to find instructions for resolving unexpected status values.
• “Resolving system manager problems and backlogs” on page 181 describes how
to identify and resolve problems with system manager processes.
• “Starting system managers or a journal manager” on page 184 describes how to
start system manager or journal manager processes. In these instructions, MIMIX
will start both if necessary.
• “Ending system managers” on page 184 describes how to end system manager
processes associated with a specified system and identifies when the system
manager RJ links must also be ended. These instructions include optionally
ending the RJ links.
• “Ending a journal manager” on page 185 describes how to end a journal manager
on a selected system.
• “Starting collector services” on page 185 describes how to start collector services.
• “Ending collector services” on page 186 describes how to end collector services.
• “Starting target journal inspection processes” on page 186 describes how to start
target journal inspection on a selected system.
• “Ending target journal inspection processes” on page 187 describes how to end
target journal inspection on a selected system.
• “Displaying status of target journal inspection” on page 188 describes how to
display the status of a single inspection job on a system and how to resolve
problems with its status.
• “Displaying results of target journal inspection” on page 189 describes where to
find information about the objects identified by target journal inspection.
178
• “Identifying the last entry inspected on the target system” on page 191 describes
how to determine the last entry in the target journal and the last entry processed
by target journal inspection.
• “Displaying status of monitors on the local system” on page 192 describes how to
check status of monitors and which monitors should run on the local system.
• “Starting a monitor” on page 194 describes how to start a monitor on the local
system.
• “Ending a monitor” on page 195 describes how to stop a monitor on the local
system.
• “Starting and ending the master monitors” on page 195 describes which
processes will automatically start or end the master monitors and how to end them
manually when necessary.
• “Enabling or disabling a monitor” on page 197 describes how to enable a monitor
so it can be started or disable a monitor so that it cannot run.
• “Displaying status of a remote journal link” on page 199 describes how to display
the status of remote journal links, including those used by data group replication
processes and those used by system manager processes. This topic also
describes the delivery method and state values possible for RJ links.
• “Starting a remote journal link independently” on page 202 describes how to start
an RJ link for a data group or a system manager process independently from
other MIMIX processes.
• “Ending a remote journal link independently” on page 202 describes how to end
an RJ link for a data group or a system manager process independently from
other MIMIX processes.
179
Displaying status of system-level processes
Status of processes that run at the system level can be viewed from the Work with
Systems display.
1. Do one of the following:
• From MIMIX Intermediate Main Menu, select option 2 (Work with Systems) and
press Enter.
• From the Work with Application Groups display, use option 12 (Node entries).
On the resulting Work with Node Entries display, press F7 (Systems).
The Work with Systems display appears, showing the local system at the top of
the list.
2. Check the status values displayed.
• If one or more processes are *UNKNOWN, use “Verifying all communications
links” on page 344.
• If one or more processes are *INACTIVE, use option 9 (Start) to start them on
a system. Any processes that are not active on the system are preselected1 on
the resulting Start MIMIX Managers display.
• The expected status values for normal operation in a two-system environment
are shown below. If the values are not as expected, additional investigation is
needed.
Expected Status Values:
System Managers - *ACTIVE.
Investigate error statuses of *ACTREQ and *INACTRJ using “Resolving
system manager problems and backlogs” on page 181.
Journal Managers - *ACTIVE2.
Investigate warning statuses of *INACTASP and *RCYASP using “Resolving
ASP 1 storage exceeded threshold problems” on page 183.
Target Journal Inspection
For a system that is currently the target system for replication, the expected
status is *ACTIVE.
For other systems, the expected status is *NOTTGT.
Investigate any other status using “Displaying status of target journal
inspection” on page 188.
Collector Services - *ACTIVE.
Cluster Services
For most environments, the expected status is *NONE.
In environments licensed for MIMIX for PowerHA the expected status for a
system that participates in an IBM i cluster is *ACTIVE.
1. Except cluster services. To start cluster services, MIMIX for PowerHA users must specify
*YES for the Start cluster services prompt.
2. In environments with a MIMIX for PowerHA license and you are only controlling a switchable
independent ASP without MIMIX logical replication capability, the expected status is *NONE.
180
Resolving system manager problems and backlogs
Figure 35. Expected status on the Work with Systems display for a two-system environment.
Bottom
F3=Exit F5=Refresh F9=Automatic refresh F10=Legend F12=Cancel
F13=Repeat F16=System definitions
181
• From the Work with Application Groups display, use option 12 (Node entries).
On the resulting Work with Node Entries display, press F7 (Systems).
The Work with Systems display appears. The status shown in the System
Managers column is a summary of all system manager processes through which
a system communicates, and includes status of their system manager RJ links
and processing jobs.
2. You can attempt to resolve system manager statuses of *ACTREQ, *INACTIVE,
or *INACTRJ from this display by doing the following:
a. Type a 9 (Start) next to the system definition you want and press Enter.
b. When the Start MIMIX Managers display appears press Enter.
c. Press F5 (Refresh).
To check specific system manager processes or check for a backlog, or if the
problem persists after attempting a start request continue with these instructions.
3. Type a 7 (System manager status) next to the system definition you want and
press Enter.
The Work with System Pair Status display appears, showing a list that represents
all of the system manager processes associated with the selected system
definition. Each row identifies a pair of systems between which a system manager
process exists and the direction (source to target) in which it transfers data.
4. Check each represented system manager process for statuses of *INACTIVE or
*INACTRJ and do the following to resolve them:
a. To start the system manager processes for the identified source system of a
pair of systems, type a 9 (Start) next to the pair and press Enter.
b. The Start MIMIX Managers display appears. The value *SYS is preselected for
the Manager prompt. Check the values of other prompts and change them as
needed to ensure that the values are appropriate for the system specified.
c. Press Enter.
The start request will start all system managers associated with the source
system of the pair for which the Start option was selected. In a two-node
environment, this affects both system manager processes between the two
systems. In environments with three or more systems, this affects system
manager processes between the selected system and all other systems with
which it communicates.
d. Press F5 (Refresh).
• If a status of *INACTRJ persists, use “Starting a remote journal link
independently” on page 202.
• If a status of *INACTIVE persists, contact your CustomerCare
representative.
5. For every process with a status of *ACTIVE, check the unprocessed entries count.
Unprocessed entries means that a backlog exists and action may be required.
The count identifies the number of entries from the system manager source that
182
Resolving ASP 1 storage exceeded threshold problems
have not been processed by the target’s processing job, and includes entries that
remote journaling has not sent to the target system.
a. Press F5 (Refresh) to ensure that the data displayed is current.
b. For process that have *ACTIVE status, evaluate the timestamp shown for
unprocessed entries. If the timestamp does not change when data is refreshed
(F5), contact CustomerCare.
183
Starting system managers or a journal manager
To start the system manager processes in which a system participates or to start a
journal manager for a system, do the following:
1. Do one of the following:
• From MIMIX Intermediate Main Menu, select option 2 (Work with Systems) and
press Enter.
• From the Work with Application Groups display, use option 12 (Node entries).
On the resulting Work with Node Entries display, press F7 (Systems).
2. The Work with Systems display appears. Type a 9 (Start) next to the system
definition you want and press Enter.
3. The Start MIMIX Managers display appears. By default, any manager that is not
running will be selected to start. Specify the value for the type of manager you
want to start at the Manager prompt and press Enter.
A start request also starts the RJ links for the system manager processes when
necessary.
184
Ending a journal manager
Note: For the Manager prompt, the value *ALL indicates that the journal
manager will also be ended. To end any system manager processes,
either *ALL or *SYS must be specified. By default, system manager RJ
links are not ended.
4. Do one of the following:
• To end only the system manager processing jobs, press Enter.
• To end system manager RJ links as well as the processing jobs, specify *YES
for the End system manager RJ links prompt. Then press Enter.
185
Ending collector services
To end collector services for a system, do the following
1. Do one of the following:
• From MIMIX Intermediate Main Menu, select option 2 (Work with Systems) and
press Enter.
• From the Work with Application Groups display, use option 12 (Node entries).
On the resulting Work with Node Entries display, press F7 (Systems).
2. The Work with Systems display appears. Type a 10 (End) next to the system
definition you want and press Enter.
3. The End MIMIX Managers display appears. At the Collector services prompt, type
*YES and press Enter.
186
Ending target journal inspection processes
187
Displaying status of target journal inspection
Target journal inspection consists of a set of jobs that read journals on the target
system to check for people or processes other than MIMIX that have modified
replicated objects on the target system. Best practice is to allow target journal
inspection for all systems in your replication environment.
Each target journal inspection process runs on a system only when that system is the
target system for replication. The number of inspection processes depends on how
many journals are used by data groups replicating to that system. On a target system,
there is one inspection job for the system journal and one job for each target user
journal identified in data groups replicating to that system.
Because target journal inspection processes run at the system-level, the best location
to begin checking status is from the Work with Systems display.
1. Do one of the following:
• From MIMIX Intermediate Main Menu, select option 2 (Work with Systems) and
press Enter.
• From the Work with Application Groups display, use option 12 (Node entries).
On the resulting Work with Node Entries display, press F7 (Systems).
2. The Work with Systems display appears. The Journal Inspect. column shows the
summarized status of all journal inspection processes on a system.
• Expected values are either *ACTIVE or *NOTTGT.
• For all other status values, type 11 (Jrn inspection status) next to the system
you want and press Enter.
3. The Work with Journal Inspection Status display appears, listing the subset of
journal definitions for the selected system. The status displayed is for target
journal inspection for the journal associated with a journal definition.
Note: Journal definitions whose journals are not eligible for target journal
inspection are not displayed. This includes journal definitions that identify
the remote journal used in RJ configurations (whose names typically end
with @R) as well as the MXCFGJRN journal definition, which are for
internal use.
Table 32 identifies the status for the inspection job associated with a journal and
how to resolve problems.
Table 32. Status values for a single target journal inspection process.
188
Displaying results of target journal inspection
Table 32. Status values for a single target journal inspection process.
*UNKNOWN The status of the process on the system cannot be determined, possibly
(inverse white) because of an error or communications problem.
Use the procedure in “Verifying all communications links” on page 344.
*ACTIVE Target journal inspection is active for the journal identified in the journal
(inverse blue) definition.
*NEWDG Target journal inspection has not run because all enabled data groups
that use the journal definition as a target journal have never been
started.
The inspection process will start when one or more of the data groups
are started.
*NOTCFG Either the journal definition does not allow target journal inspection or all
enabled data groups that use the journal definition (user journal) prevent
journaling on the target system. Target journal inspection is not
performed for the journal.
For instructions for configuring target journal inspection, see topics
“Determining which data groups use a journal definition” and “Enabling
target journal inspection” in the MIMIX Administrator Reference book.
*NOTTGT The journal definition is not used as a target journal definition by any
enabled data group. Target journal inspection is not performed for the
journal.
This is the expected status when the journal definition is properly
configured for target journal inspection but the system is currently a
source system for all data groups using this journal definition.
189
available allow you to display details of the notification and to view all objects
changed on the target by the user identified in the notification as well as any
automatic recovery actions for those objects.
190
Identifying the last entry inspected on the target system
191
Displaying status of monitors on the local system
Each node or system in the product configuration has MIMIX monitors which run on
that system to check for specific potential problems. The Work with Monitors display
indicates the status and other details about monitors on the local system.
Monitor status must be checked from each system. To check status on a system, do
the following:
Note: Environments that do not have application groups can start with Step 3.
1. Select option 1 (Work with application groups) from the MIMIX Basic Main Menu
and press Enter.
2. The Monitors field located in the upper right corner of the Work with Application
Groups display summarizes the status of the MIMIX monitors on the local system
with one of the following values.
*ACTIVE - All enabled monitors on the local system are active.
*ATTN - Either one or more monitors on the local system failed or there are both
active and inactive monitors on the local system. Investigation is necessary.
*INACTIVE - All enabled monitors on the local system are inactive.
Figure 36. Monitors field on the Work with Application Groups display
192
Displaying status of monitors on the local system
Table 33. Possible monitors and the nodes on which they should be active
MMNFYNEWE - monitor for new object notification entries Primary node when the
Monitors the source system for the newly created libraries, application group is
folders, or directories that are not already included or configured for logical
excluded for replication by a data group configuration. replication.
Note: This monitor is shipped disabled. User action is required to
enable this monitor on the source system within your
MIMIX environment. Once enabled, the monitor will
automatically start with the master monitor.
FAILED/ACT The resulting actions of the monitor failed but the monitor remains
active. If the monitor belongs to a group it will participate in the
decision to trigger the group.
193
Table 34. Possible status values for monitors
FAILED The monitor has failed. The event program for the monitor requested
that the monitor be ended and the status set to failed, an unexpected
error occurred in the monitor, or the maximum number of restarts
have been attempted. If the monitor belongs to a group, it will
participate in the decision to trigger the group.
ENDING The monitor is ending normally. Once the monitor completes its
shutdown activities, the status will be changed to either INACTIVE or
FAILED depending on the return value from the last call to the event
program. User action may be required if the state persists.
INACTIVE The monitor has not been started. The monitor is not active.
User action is necessary to start the monitor.
HELD The monitor has been started, but its activity is suspended until a
release is requested.
DISABLED The monitor has been disabled and is not active. If the monitor
belongs to a group, it will not participate in the starting or triggering of
the group. A disabled monitor is not affected by master monitor
actions. User action is necessary to enable the monitor.
SCHEDULED The monitor is scheduled. This value is only valid for time monitors.
Starting a monitor
Only monitors with status values of Inactive, Failed, or Disabled can be started.
If a group monitor is started, unless you specify otherwise, the request is extended to
all monitors defined to the selected group monitor that have a status that allows them
to be started. The status of a monitor may prevent it from participating in the start
request. Monitors with a disabled status are never started with a group monitor.
Do the following to start a monitor:
1. From the MIMIX Basic Main Menu, select option 12 (Work with monitors) and
press Enter.
2. From the Work with Monitors display, type a 9 (Start) next to the monitor you want
and press F4 (Prompt).
3. The Start Monitor (STRMON) display appears. If the monitor you are starting is a
journal monitor or an extended journal monitor, you will see prompts for Journal
Receiver and Starting large sequence number. Specify values that identify the
starting journal receiver and the first journal entry to be considered for monitoring.
194
Ending a monitor
4. If the selected monitor is a group monitor and you want to start only the group
monitor itself, press F10 (Additional parameters). Then specify *NAME at the
Scope of request prompt.
5. Press Enter.
Ending a monitor
Only monitors with status values of Active, Scheduled, Held, or Failed/Act can be
ended.
If a group monitor is ended, unless you specify otherwise, the request is extended to
all monitors defined to the selected group monitor that have a status that allows them
to be ended. The status of a monitor may prevent it from participating in the end
request.
Do the following to end a monitor:
1. From the MIMIX Basic Main Menu, select option 12 (Work with monitors) and
press Enter.
2. From the Work with Monitors display, type a 10 (End) next to the monitor you
want and press F4 (Prompt).
3. The End Monitor (ENDMON) display appears.
4. If the selected monitor is a group monitor and you want to end only the group
monitor itself, press F10 (Additional parameters). Then specify *NAME at the
Scope of request prompt.
5. Press Enter.
195
Starting the master monitor on the local system
The master monitor will attempt to restart itself a limited number of times when it ends
because of an error or if the job is abnormally ended. If an error condition prevents the
master monitor from restarting itself, the master monitor will end. This causes the
MIMIX scheduler job and all other active monitors to also end. When the master
monitor is started, other monitors that are configured to start with the master monitor
and which are not disabled are started.
Use the following command, specifying the name of the MIMIX installation library:
installation-library/STRMSTMON
The master monitor job name will be the installation library name when it is started.
196
Enabling or disabling a monitor
Enabling a monitor
Enabling a monitor changes its status from DISABLED to INACTIVE.
Once enabled, a monitor can be started and run its normal operations. Once enabled,
a monitor can be started by the master monitor if it is configured to do so. If the
monitor is defined to a group, it will participate in requests to start all monitors in the
group and in decisions for the group monitor.
Do the following to enable a monitor:
1. From the MIMIX Basic Main Menu, select option 12 (Work with monitors) and
press Enter.
2. From the Work with Monitors display, type a 20 (Enable) next to the monitor you
want and press F4 (Prompt).
3. The Change Monitor Status (CHGMONSTS) display appears. The value
*ENABLED should be preselected for the Status prompt.
4. If the selected monitor is a group monitor and you want to enable only the group
monitor itself, press F10 (Additional parameters). Then specify *NAME at the
Scope of request prompt.
5. Press Enter.
Disabling a monitor
A monitor must have a status of Inactive or Failed before it can be disabled.
A monitor with a DISABLED status cannot run its normal operations and is prevented
from starting with the master monitor if configured to do so. If it is defined to a group, it
will not participate in requests to start all monitors in the group and will not participate
in decisions for the group monitor.
DISABLED monitors can be started individually. Once they are started, they will
participate in the triggering decision for any group to which they belong.
Do the following to disable a monitor:
1. From the MIMIX Basic Main Menu, select option 12 (Work with monitors) and
press Enter.
2. From the Work with Monitors display, type a 21 (Disable) next to the monitor you
197
want and press F4 (Prompt).
3. The Change Monitor Status (CHGMONSTS) display appears. The value
*DISABLED should be preselected for the Status prompt.
4. If the selected monitor is a group monitor and you want to disable only the group
monitor itself, press F10 (Additional parameters). Then specify *NAME at the
Scope of request prompt.
5. Press Enter.
198
Displaying status of a remote journal link
To check the status of an RJ link used for by a system manager process, do the
following:
1. From the Work with Systems display (WRKSYS command), type 7 (System
manager status) next to the system you want and press Enter.
2. On the resulting Work with System Pairs display, type 12 (Work with RJ links) and
press Enter.
To check the status of all RJ links in a MIMIX installation, type the command:
installation-library/WRKRJLNK PROCESS(*ALL)
Figure 37. Example showing all RJ links in an environment with two systems and one data
group. The first two RJ links listed are for the data group. The last two are for system manager
processes between the systems.
Bottom
Parameters or command
===> _________________________________________________________________________
F3=Exit F4=Prompt F5=Refresh F6=Add F9=Retrieve F11=View 2
F12=Cancel F13=Repeat F16=Jrn Definitions F18=Subset F21=Print list
199
The Dlvry column indicates configured value for how the IBM i remote journal function
sends the journal entries from the source journal to the target journal. The possible
values for delivery are asynchronous (*ASYNC) and synchronous (*SYNC).
*ASYNC - Journal entries are replicated asynchronously, independent of the
applications that create the journal entries. The applications continue processing
while an independent system task delivers the journal entries. If a failure occurs
on the source system, journal entries on the source system may become trapped
because they have not been delivered to the target system.
*SYNC - Journal entries are replicated synchronously. The applications do not
continue processing until after the journal entries are sent to the target journal. If a
failure occurs on the source system, the target system contains the journal entries
that have been generated by the applications.
The State column represents the composite view of the state of the remote journal
link. Because the RJ link has both source and a target component, the state shown is
that of the component which has the most severe state. Table 35 shows the possible
states of an RJ link, listed in order from most severe to least severe.
Table 35. Possible states for RJ links, shown in order starting with most severe.
State Description
*UNKNOWN The state of the link cannot be checked. Both journals defined to the
remote journal link do not reside on the local system and a transfer
definition does not exist between the local system and the source
system of the remote journal link.
*NOTBUILT The remote journal link is defined to MIMIX but one of the associated
journal environments has not been built.
*SRCNOTBLT The remote journal link is defined to MIMIX but the associated source
journal environment has not been built.
*TGTNOTBLT The remote journal link is defined to MIMIX but the associated target
journal environment has not been built.
*FAILED The remote journal cannot receive journal entries from the source
journal due to an error condition.
*CTLINACT The remote journal link is processing a request for a controlled end.
*INACTPEND An active remote journal link is in the process of becoming inactive. For
asynchronous delivery, this is a transient state that will resolve
automatically. For synchronous delivery, one system is inactive while the
other system is inactive with pending unconfirmed entries.
200
Displaying status of a remote journal link
Table 35. Possible states for RJ links, shown in order starting with most severe.
State Description
201
Starting a remote journal link independently
If they are not already active, the remote journal (RJ) links used by replication and
system manager processes are started by operations that start those processes, such
as the STRMMX, STRDG, or STRMMXMGR commands.
To start a remote journal link separately from other MIMIX processes, do the
following:
1. Do one of the following to access a subset of RJ links for the type of process by
which they are used:
• To access the RJ links for a data group on the Work with Data Groups display,
type 44 (RJ links) next to the data group you want and press Enter.
• To access the RJ links for system manager processes on the Work with
Systems display, type 7 (System manager status) next to the system you want
and press Enter. On the resulting Work with System Pairs display, type 12
(Work with RJ links) next to the pair of systems for the system manager
process you want and press Enter.
2. From the Work with Remote Journal Links display, type a 9 (Start) next to the link
in the list that you want to start and press Enter.
3. The Start Remote Journal Link (STRRJLNK) display appears. Specify the value
you want for the Starting journal receiver prompt.
4. To start remote journaling for the specified link, press Enter.
To end a remote journal link separately from other MIMIX processes, do the following:
1. Do one of the following to access a subset of RJ links for the type of process by
202
Ending a remote journal link independently
203
Working with notifications and recoveries
This topic describes what notifications and recoveries are and how to work with them.
This chapter includes the following topics:
• “What are notifications and recoveries” on page 204 defines terms used for
discussing notifications and recoveries and identifies the sources that create
them.
• “Displaying notifications” on page 205 identifies where notifications are viewed in
the user interfaces and how to work with them.
• “Notifications for newly created objects” on page 210 describes the MIMIX®
AutoNotify™ feature which can be used to monitor for newly created libraries,
folders, or directories.
• “Displaying recoveries” on page 211 identifies where recoveries in progress are
viewed in the user interfaces and how to work with them.
204
Displaying notifications
Displaying notifications
In environments with application groups, the Notifications field located in the upper
right corner of the Work with Application Groups display indicates the highest severity
level of new notifications for the MIMIX installation.
You can see a count of how many new notifications require action or attention for a
MIMIX installation from the Work with Data Groups display.
Do the following:
1. Optional. To access the Work with Data Groups display, either select option 6
from the MIMIX Basic Main menu or select option 1 from the MIMIX Intermediate
Main menu and press Enter.
The Notif. field is located in the upper right of the display. The third number
205
Working with notifications and recoveries
206
Displaying notifications
Detailed information
When you display a notification, you see its description, status, severity, data group,
source, and sender as described above. You also have access to the following
information:
Details - When the source of the notification is a rule, this identifies the command that
was initiated by the rule. When the source of the notification is user-generated, this
indicates the notification detail text specified when the notification entry was added.
When the source of the notification is a monitor, this describes the events that
resulted in the notification.
Output File - If available, this identifies an associated output file. Output file
information associated with a notification is only available from the sender system.
For user-generated notifications, output file information is available only if it was
specified when the notification was added.
1. In environments configured for use with IBM i clustering, several other processes besides
audits can generate recoveries with a source name beginning with the character #.
207
Working with notifications and recoveries
Job - If available, this identifies the job that generated the notification. Job information
associated with a notification is only available from the sender system. For user-
generated notifications, this information is available only if it was specified when the
notification was added.
Option Description
4=Remove Deletes the notification. You are prompted to confirm this choice.
For a notification generated by an audit or a MIMIX rule, the
associated job and output files are also deleted.
This must be performed from the system on which the notification
originated.
8=View results When the information is available, this provides the Name and
Library of the output file (outfile) associated with the notification.
This option is only available from the system on which the
notification originated.1
12=Display job Displays the job log for the job which generated the notification, if
it is available. This option is only available from the system on
which the notification originated
208
Displaying notifications
specified in positions other than the last (trailing) character of the specified name. The
names in these IFS entries are considered advanced generic names.
To check for notifications about advanced generic names in IFS entries and locate the
affected IFS entries:
1. Do one of the following:
• On the Work with Application Groups display, the Notifications field indicates
whether any warning notifications exist. Press F15 (Notifications).
• On the Work with Data Groups display, the third number in the
Audits/Recov./Notif. field displays the number of new notifications. Press F8
(Recoveries), then press F10 (Work with Notifications).
• From a command line, enter: WRKNFY.
2. The Work with Notifications display appears. Notifications about advanced
generic names are identified by the name INSTALL in the Source column.
3. Press F11 (View Timestamp) to see the name of the affected data group.
4. From the command line, specify the data group in the following command:
WRKDGIFSE DGDFN(name system1 system2)
5. On the Work with DG IFS Entries display, check for entries that display either
*INCLD(A) or *EXCLD(A) in the Process Type column.
These entries have embedded asterisks and are considered advanced generic
names, which are important because:
• An advanced generic name in an *INCLD IFS entry will have any asterisk
characters embedded within the name interpreted as literal characters, not
wildcard characters. These entries cannot be changed or copied. They should
be evaluated, and if possible removed to avoid confusion with how asterisks
are interpreted.
• An advanced generic name in an *EXCLD IFS entry will be processed
differently after upgrading to 8.1.12.00 or higher. Asterisks embedded within its
name are processed as wildcard characters which act as a filter to exclude
objects that would otherwise be included by an existing entry with a basic
generic name.
For more information about advanced generic names, see the topic “Identifying IFS
objects for replication” in the MIMIX Administrator Reference book.
209
Notifications for newly created objects
The MIMIX® AutoNotify™ feature can be used to monitor for newly created libraries,
folders, or directories. The AutoNotify feature uses a shipped journal monitor called
MMNFYNEWE to monitor for new objects in an installation that are not already
included or excluded for replication by a data group. The AutoNotify feature monitors
the security audit journal (QAUDJRN), and when new objects are detected, issues a
warning notification.
The MMNFYNEWE monitor is shipped in a disabled state. In order to use this feature,
the MMNFYNEWE monitor must be enabled on the source system within your MIMIX
environment. Once enabled, this monitor will automatically start with the master
monitor.
Notifications will be sent when newly created objects meet the following conditions:
• The installation must have a data group configured whose source system is
the system the monitor is running on.
• The journal entry must be a create object (T-CO) or object management
change (T-OM).
• If the journal entry is a create object (T-CO), then the type must be new (N).
• The journal entry must be for a library, folder, or directory.
• If the journal entry is for a library, it cannot be a MIMIX generated library
since MIMIX generated libraries are not replicated by MIMIX.
• If the journal entry is for a directory, it cannot be the /LAKEVIEWTECH
directory, or any directory under /LAKEVIEWTECH.
• If the journal entry is for a directory, it must be a directory that is supported
for replication by MIMIX.
• The object is not already known (included or excluded) in the installation.
From the native user interface, notifications can be viewed from the Work with
Notifications (WRKNFY) display. The notification message will indicate required
actions. Additionally, when you view notifications from the MMNFYNEWE monitor
from Vision Solutions Portal, you have immediate access to actions to create
configuration entries for it.
210
Displaying recoveries
Displaying recoveries
Active recoveries are an indication of detected problems that are currently being
corrected by automatic recovery actions. Before certain activity, such as ending
MIMIX, it is important that no recoveries are in progress in the installation.
You can see a count of how many recoveries are in progress for a MIMIX installation
from the Work with Data Groups display.
Do the following:
1. To access the Work with Data Groups display, either select option 6 from the
MIMIX Basic Main menu or select option 1 from the MIMIX Intermediate Main
menu and press Enter.
2. The Recov. field is located in the upper right of the display. The middle number
identifies the number of new or in-progress recoveries. This includes recoveries
resulting from all sources.
Work with Data Groups CHICAGO
11:02:05
Type options, press Enter. Audits/Recov./Notif.: 079 / 002 / 003
5=Display definition 8=Display status 9=Start DG
10=End DG 12=Files needing attention 13=Objects in error
14=Active objects 15=Planned switch 16=Unplanned switch ...
---------Source--------- --------Target-------- -Errors-
Opt Data Group System Mgr DB Obj DA System Mgr DB Obj DB Obj
211
What information is available for recoveries in the native user interface
The following information is available for the recoveries listed on the Work with
Recoveries display. The F11 key toggles between views of status, timestamp, and
text of the recoveries. Additional details are available for each recovery through the
Display Recovery Details display.
Each recovery provides a brief description of the recovery process taking place as
well as its current status.
Status - Shows the status of the recovery action.
*ACTIVE - The job associated with the recovery is active.
*ENDING - The job associated with the recovery is ending.
*HELD - The job associated with the recovery is held. A recovery whose source is
a replication process cannot be held.
Data group - Identifies the data group associated with the recovery.
Note: On the Status view of the Work with Recoveries display, the F7 key toggles
between the Source column and the Data Group column. The full three-part
name is available in the Timestamp view (F11).
Date - Indicates the date the recovery process started.
Time - Indicates the time the recovery process started.
Source - Identifies the process, program, or command that generated the recovery.
Names that begin with the character # are generated by automatic recovery actions
for audits.1 Names that begin with the characters ## are generated by automatic
recovery actions for object replication that used the third delay retry processing.
Sender or From System - Identifies the system from which the recovery originated.
Detailed information
When you display a recovery, you see its description, status, data group, source, and
sender as described above. You also have access to the following information.
Details - When the source of the recovery is a rule, this identifies the command run by
the rule in an attempt to recover from the detected error.
Output File - If available, this identifies an associated output file that lists the
detected errors the recovery is attempting to correct. Output file information
associated with a recovery is only available from the sender system.
Job - If available, this identifies the job that is performing the recovery action. Job
information associated with a recovery is only available from the sender system.
1. In environments configured for use with IBM i clustering, several other processes besides
audits can generate recoveries with a source name beginning with the character #.
212
Displaying recoveries
WRKRCY Description
Option
8=View progress Displays a filtered view of the output file associated with the recovery.
MIMIX updates the output file while the recovery is in progress,
identifying the detected errors it is attempting to correct and marking
corrected errors as being recovered. This option is only available from
the system on which the recovery job is running.
10=End job Ends an active recovery job. This action is valid for recoveries with
names that begin with # and is only available from the system on
which the recovery job is running.
12=Display job Displays the job log for the recovery job associated in progress. This
option is only available from the system on which the recovery job is
running.
13=Hold job Places an active recovery job on hold. This action is valid for
recoveries with names that begin with # and is only available from the
system on which the recovery job is running.
14=Release job Releases a held recovery job. This action is valid for recoveries with
names that begin with # and is only available from the system on
which the recovery job is held.
213
When automatic database recovery or automatic object recovery is enabled,
orphaned recoveries are deleted, when possible.
Because recoveries are displayed on both systems, but jobs associated with them are
only accessible from the originating system, you need to verify that the recovery is
orphaned before removing it.
214
CHAPTER 11 Starting and ending replication
215
Starting and ending replication
• “What occurs when a data group is ended” on page 238 describes the behavior of
the ENDDG command.
• “Ending MIMIX” on page 240 provides procedures for using the ENDMMX
command and describes when you may also need to end the MIMIX subsystem.
• “Ending an application group” on page 243 provides a procedure for using the
ENDAG command.
• “Ending a data group in a controlled manner” on page 244 provides procedures
for preparing to end, ending, and confirming that the end completed without
problems.
• “Ending selected data group processes” on page 247 provides a procedure using
the ENDDG command.
• “Enabling MIMIX-disabled target constraints” on page 248 describes the
requirements for enabling foreign key constraints that were previously disabled by
MIMIX and provides procedures for enabling constraints when the data group is
ended or while performing a controlled end.
• “What replication processes are started by the STRDG command” on page 251
describes which replication processes are started with each possible value of the
Start processes (PRC) parameter. Both data groups configured for remote
journaling and data groups configured for MIMIX source-send processing are
addressed.
• “What replication processes are ended by the ENDDG command” on page 255
describes what replication processes are ended with each possible value for the
End Options (PRC) parameter. Both data groups configured for remote journaling
and data groups configured for MIMIX source-send processing are addressed.
216
Before starting replication
217
journal links, and automatic recovery processes on the specified systems. Each
data group starts from the journal receiver in use when the data group ended and
with the sequence number following the last sequence number processed.
Master monitor - Starts the master monitor on each of the specified systems.
Monitors - On each of the specified systems, the master monitor starts the MIMIX
scheduler job and monitors that are not disabled and which are configured to start
with the master monitor.
Application groups - If all systems are specified, all application groups and any
associated data resource groups are started. If IBM i clustering is used, default
processing will start the IBM application CRG.
Note: The STRMMX command does not start promoter group activity. Start promoter
group activity using procedures in the Using MIMIX Promoter book.
Optionally, the command can be used to start MIMIX processes associated with the
local system. When the command request specifies *LOCAL for the System definition
(SYSDFN) parameter, single system processes on the local system are started and
multiple-system processes between the local system and another system are started.
Single system processes include the master monitor, MIMIX scheduler, monitors,
journal manager, target journal inspection, collector services, and cluster services.
Multiple-system processes include system managers, RJ links for system managers
and replication processes, data group replication processes and their supporting
processes. Application groups are not started.
218
Commands for starting replication
behavior for application groups that do not participate in a cluster controlled by the
IBM i operating system (*NONCLU application groups).
What is the scope of the request? The following parameters identify the scope of
the requested operation:
Application group definition (AGDFN) - Specifies the requested application group.
You can either specify a name or the value *ALL.
Resource groups (TYPE) - Specifies the types of resource groups to be
processed for the requested application group.
Data resource group entry (DTARSCGRP) - Specifies the data resource groups to
include in the request. The default is *ALL or you can specify a name. This
parameter is ignored when TYPE is *ALL or *APP.
What is the requested behavior? The following parameters, when available, define
the expected behavior:
Current node roles (ROLE) - Only available on the STRAG command, this
parameter is ignored for non-cluster application groups.
What procedure will be used? The following parameters identify the procedure to
use and its starting point:
Begin at step (STEP) - Specifies where the request will start within the specified
procedure. This parameter is described in detail below.
Procedure (PROC) - Specifies the name of the procedure to run to perform the
requested operation when starting from its first step. The value *DFT will use the
procedure designated as the default for the application group. The value
*LASTRUN uses the same procedure used for the previous run of the command.
You can also specify the name of a procedure that is valid the specified
application group and type of request.
Where should the procedure begin? The value specified for the Begin at step
(STEP) parameter on the request to run the procedure determines the step at which
the procedure will start. The status of the last run of the procedure determines which
values are valid.
The default value, *FIRST, will start the specified procedure at its first step. This value
can be used when the procedure has never been run, when its previous run
completed (*COMPLETED or *COMPERR), or when a user acknowledged the status
of its previous run which failed, was canceled, or completed with errors
(*ACKFAILED, *ACKCANCEL, or *ACKERR respectively).
Other values are for resolving problems with a failed or canceled procedure. When a
procedure fails or is canceled, subsequent attempts to run the same procedure will
fail until user action is taken. You will need to determine the best course of action for
your environment based on the implications of the canceled or failed steps and any
steps which completed.
The value *RESUME will start the last run of the procedure beginning with the step at
which it failed, the step that was canceled in response to an error, or the step
following where the procedure was canceled. The value *RESUME may be
appropriate after you have investigated and resolved the problem which caused the
219
procedure to end. Optionally, if the problem cannot be resolved and you want to
resume the procedure anyway, you can override the attributes of a step before
resuming the procedure.
The value *OVERRIDE will override the status of all runs of the specified procedure
that did not complete. The *FAILED or *CANCELED status of these procedures are
changed to acknowledged (*ACKFAILED or *ACKCANCEL) and a new run of the
procedure begins at the first step.
220
What occurs when a data group is started
221
If a data group is participating in a virtual switch when you initiate a start data group
request, the data group’s apply processes will not be started. The apply processes
remain ended and will only be started by the virtual switch procedure during its
recovery phase.
222
What occurs when a data group is started
*NO *NO Data groups start with regular processing: When CLRPND is *NO,
or • Data group file entry status remains unchanged. the value *CLRPND is
*CLRPND • Hold logs remain unchanged. the same as *NO for
CLRERR.
*NO *YES • Data group file entries in *HLDERR, *HLDRGZ, See File entry states
*HLDRNM, *HLDPRM, and *HLDRLTD status are See Log spaces
cleared.
• Tracking entries in *HLDERR status are cleared.
• Hold log space is deleted.
• Data group object activity entries in any failed
status are changed to CC (Completed by clear
request).
223
Table 38. CLRPND and CLRERR processing (Continued)
*YES *NO Note: CLRPND(*YES) will not start a data group when See File entry apply
there are open commit cycles on files defined to the session assignment
data group.
See Single apply session
• Data group file entries in *HLDRGZ, *HLDRNM, processing
and *HLDPRM status are cleared and reset to
See Log spaces
active.
• Data group tracking entries in *HLDRNM are
cleared and reset to active.
• Data group file entries and tracking entries in
*HLDERR status remain unchanged.
• Data group object activity entries in any active
status are changed to CC (Completed by clear
request).
• If there is a requested status at the time of starting,
it is cleared.
• Journal, hold, tracking entry hold, and apply
history log spaces are deleted.
• The apply session to which data group file entries
are assigned may change.
*YES *YES Note: CLRPND(*YES) will not start a data group when See File entry states
or there are open commit cycles on files defined to the See File entry apply
data group.
*CLRPND session assignment
• Data group file entries in *HLDERR, *HLDRGZ, See Single apply session
*HLDRNM, *HLDPRM, and *HLDRLTD status are processing
cleared and reset to active.
See Log spaces
• Data group file entries in *HLDRTY status remain
When CLRPND is *YES,
unchanged.
the value *CLRPND is
• Data group object activity entries in any failed or the same as *YES for
active status are changed to CC (Completed by CLRERR.
clear request).
• Tracking entries in *HLDERR and *HLDRNM
status are cleared and reset to active.
• Tracking entries are primed if any configuration
changes occurred for data group object entries or
data group IFS entries.
• If there is a requested status at the time of starting,
it is cleared.
• Journal, hold, and apply history log spaces are
deleted.
• The apply session to which data group file entries
are assigned may change.
224
What occurs when a data group is started
File entry states: Files in specific states will not reset to active when you specify *YES on the Clear Error
prompt. If you have set data group file entries to any of these states, the following process exception
applies:
Note: The only states that can be set using the Set Data Group File Entry (SETDGFE) command are *HLD,
*RLSWAIT, *ACTIVE, and *HLDIGN. All other states are the result of internal processing.
• *HLD - Journal entries cached before *YES is specified are discarded. If *ALL or *ALLSRC is specified
on the Start processes prompt, all subsequent entries from the specified starting point will be cached
again.
• *RLSWAIT - Journal entries are discarded as they wait for the synchronization point to arrive in the
journal stream. This occurs regardless of the value specified for Clear Error or Clear Pending.
• *HLDIGN - Journal entries are discarded until the file status is changed to something else.
• *HLDSYNC - Journal entries are ignored since an external process is actively synchronizing the file.
When that event completes normally, the file is set to *RLSWAIT.
File entry apply session assignment: Clear pending processing attempts to load balance the data group
file entries among the defined apply sessions. If the requested apply session in the data group file entry
definition is *ANY, or if it is *DGDFT and the requested apply session for the data group definition is *ANY,
then the apply session to which the data group file entry is assigned may be changed when processing
occurs. For data groups configured to replicate through the user journal and which use basic support for
constraints, the requested apply session may be ignored to ensure that related files are handled by the
same apply session. For data groups configured to allow target constraint management, configured files
with foreign key constraints may be automatically assigned to different apply sessions.
The value specified for System for DB file relations (DBRSYS) determines the system used to determine
database file relationships while assigning files to apply sessions. This parameter is evaluated only when
the start request specifies to clear pending entries for all database apply sessions. The default value, *TGT,
uses the target system to determine the file relationships.
Single apply session processing: In most situations, you will perform clear pending processing on all
apply sessions belonging to a data group by specifying *ALL or *DBALL on the Start processes (PRC)
prompt. MIMIX also supports the ability to perform clear pending processing on a single apply session,
which is useful for recovery purposes in certain error situations. The System for DB file relations (DBRSYS)
parameter is ignored when the start request specifies a specific apply session. To perform clear pending
processing on a single apply session, specify PRC(*DBAPY) and the specific apply session (APYSSN).
Log spaces: Because they have not been applied, journal entries that exist in the journal log space are
considered pending. Journal entries that exist in the hold log space, however, are considered in error. The
Clear pending and Clear error prompts affect which log spaces are deleted (and recreated) when a data
group is started.
225
Starting MIMIX
To start all MIMIX products within an installation library, do the following:
1. If you are starting MIMIX for the first time or starting MIMIX after a system IPL, do
the following:
a. Use the command WRKSBSJOB SBS(MIMIXSBS)to verify that the MIMIX
subsystem is running. If the MIMIXSBS is not already active, start the
subsystem using the STRSBS SBSD(MIMIXQGPL/MIMIXSBS)command.
b. If MIMIX uses TCP/IP for system communication, the TCP/IP servers must be
running. If TCP/IP is not already active, start TCP/IP using the port number
defined in the transfer definitions and the procedures described in “Starting the
TCP/IP server” on page 328.
2. Do one of the following:
• From the MIMIX Basic Main Menu, select option 2 (Start MIMIX) and press
Enter.
• From a command line type STRMMX and press Enter.
3. The Start MIMIX (STRMMX) display appears. Accept the default value for the
System definition prompt and press Enter.
4. If you see a confirmation display, press Enter to start MIMIX.
226
Starting an application group
227
Starting selected data group processes
This procedure can be used to do any of the following:
• Start all or selected processes for a data group, or start a specific database apply
process
• Specify a starting point for journal receivers when starting a data group
• Clear pending and error entries when starting a data group
Data groups that are in an application group: The preferred method of starting
data groups that are part of an application group is to use the Start Application Group
(STRAG) command. The default behavior of the STRDG command helps to enforce
this best practice when necessary by not allowing the command to run when the data
group is participating in a resource group with three or more nodes. (A data resource
group provides the association between one or more data groups and an application
group.) The STRDG request will run when the data group is participating in a resource
group with two nodes. You can also change the value specified for Override if in data
rsc. group (DTACRG) parameter to allow a data group to be started even if it is part of
a resource group.
In application group environments with three or more nodes, it is particularly important
to treat all members of an application group as one entity. For example, a
configuration change that is made effective by starting and ending a single data group
would not be propagated to the other data groups in the same resource group.
However, the same change would be propagated to the other data groups if it is made
effective by ending and starting the parent application group.
When to clear pending entries and entries in error: Table 39 identifies when it is
necessary to clear pending entries for apply processes and clear logs of entries
indicating files in error to establish a new synchronization point when starting a data
group. The reason for starting the data group determines whether you need to clear
only pending entries for transactions waiting to be applied, clear only errors, or both.
Before clearing pending entries, determine if there are any file entries on hold. These
are the transactions that will be lost by clearing pending entries.
When clearing pending entries, most environments can accept the default value for
the System for DB file relations prompt. If necessary, you can specify a value when
directed to by your MIMIX administrator.
228
Starting selected data group processes
Table 39. When to clear pending entries and entries in error when starting a data group
If starting the data group in any of these conditions: Specify these values on the
STRDG command:
After synchronizing database files and objects between two systems Clear pending entries and entries in
Note: This assumes that you have synchronized the objects and database error.
files and have changed the journal receivers using TYPE(*ALL) on • Specify *YES for the Clear
the CHGDGRCV command. pending prompt
• Specify *CLRPND or *YES for
After switching the direction of the data group, when starting
the Clear error prompt
replication on the system that now becomes the source system
For additional information about the STRDG command, refer to the following topics:
• “What occurs when a data group is started” on page 221
• “What replication processes are started by the STRDG command” on page 251
229
5. To start the data group, press Enter.
230
Starting replication when open commit cycles exist
231
Before ending replication
Consider the following:
• If the next time you start the data groups requires that you clear pending entries,
or if you will be performing a switch, you should verify that no activity is still in
progress before you perform these activities. Use the command WRKDGACTE
STATUS (*ACTIVE) to ensure all activity entries completed.
• Data groups that are in a disabled state are not ended. Only data groups that
have been enabled and have been started can be ended.
232
Commands for ending replication
Performing a save from the ENDAG or ENDDG When application groups are used, use
source system the ENDAG command with its default
END procedure. See “What is ended
by the default END procedure for an
application group” on page 237.
Performing a save from the target ENDAG For details about the RUNBACKUP or
system PROC(RUNBACKUP) ENDTGT procedures, see the MIMIX
(preferred) Administrator Reference book.
or
You may be able to end only selected
ENDAG PROC(ENDTGT)
processes on the target system. See
or “Ending selected data group processes”
ENDDG PRC(*ALLTGT) on page 247.
Ending only a selected replication ENDDG See “Ending selected data group
process processes” on page 247.
Changing configuration, such as ENDAG or When application groups are used, use
adding or changing data group ENDDG the ENDAG command.
entries The changes are not available to active
replication processes until the data group
processes are ended and restarted.
233
Additional considerations when ending replication
The following questions will help you determine additional options you may need
when ending replication. All methods of ending replication can accomplish these
activities, but in some, the action is not default or may require additional
programming.
• Do processes need to end in a controlled manner or can they be ended
immediately? Both commands support these options. For more information, see
“Ending immediately or controlled” on page 234
• Do you need to end only a subset of the replication processes? Only ENDDG
supports ending selected processes. For more information see “Ending all or
selected processes” on page 235.
• Do RJ links also need to end? Both replication and system manager processes
use RJ links. In most cases, RJ links for either purpose can remain active. For
scenarios when you should end the RJ links, see “When to end RJ links” on
page 236.
234
Commands for ending replication
permitted to end. This ensures that you have a known point in each journal at which
you can restart replication.
If any processes have a backlog of entries, it may take some time for the entry
created by the request to be processed through the replication path. Any entries that
precede the entry requesting to end are processed first.
A data group that is ended in a controlled manner is prepared for a more effective and
safer start when the start request specifies to clear pending entries. The existence of
commit cycles implies that there is application activity on the source system that
should not be interrupted; replication should be allowed to continue through the end of
the commit cycle. It is preferable to ensure that commit cycles are resolved or
removed before ending a data group. There are conditions in which a data group will
not start if open commit cycles exist. For more information, see “Starting replication
when open commit cycles exist” on page 230.
If the request to perform a controlled end also includes ending the RJ link, the RJ link
is ended after all requested processes end.
Either type of end request may be ignored if the request is submitted just before the
time that MIMIX jobs are restarted daily. For more information about restarting jobs,
see ‘Configuring restart times for MIMIX jobs’ in the MIMIX Administrator Reference
book.
235
determines which processes end with each possible value for the PRC parameter. If
you choose to use this parameter, be sure that you understand what processes will
end. See “What replication processes are ended by the ENDDG command” on
page 255.
236
Commands for ending replication
MIMIX managers and services - Ends the system managers, journal managers,
target journal inspection, and collector services on the specified systems.
Monitors - Ends all individual monitors currently active in the installation library on
the specified systems.
Master monitor - Ends the MIMIX scheduler job and the master monitor on each of
the specified systems.
MIMIX Promoter - Ends promoter group activity on the specified systems.
Audits and Recoveries - All queued audits, all audits in progress, and all
recoveries in progress that are associated with the specified systems are ended.
This includes jobs with locks on the installation library. Queued audits are set to
*NOTRUN and audits in comparison phase are set to *FAILED. Audits in recovery
phase reflect their state of processing at the time of the end request, which may
be *NOTRCVD.
Remote journal links for system managers - If you selected to end system
manager RJ links, all remote journal links associated with system manager
processes for the specified systems are ended.
Note: Cluster services is not ended when MIMIX managers end because cluster
services may be necessary for other applications.
Optionally, the command can be used to end MIMIX processes associated with the
local system. When the command request specifies *LOCAL for the System definition
(SYSDFN) parameter, single system processes on the local system are ended and
multiple-system processes between the local system and another system are ended.
Single system processes include the master monitor, MIMIX scheduler, monitors,
journal manager, target journal inspection, and collector services. Multiple-system
processes include system managers, RJ links for system managers and replication
processes (when requested), data group replication processes and their supporting
processes, promoter group activity, and audits. Application groups and cluster
services are not ended.
237
What occurs when a data group is ended
The End Data Group (ENDDG) command will end replication processes for the
specified data group.
The ENDDG command can be used interactively or programmatically. This command
is invoked by the ENDMMX command and by the ENDAG command running the
default END procedure, using values other than default for some parameters.
When an ENDDG request is processed, MIMIX may take a few minutes while it does
the following for each specified data group:
• The value you specify for the Process (PRC) parameter determines which data
group replication processes will end. The default value ends all data group
replication processes.
• When ending data groups that use a shared object send job, the job is ended by
the last data group to end.
• When ending data groups that perform access path maintenance, the database
apply process signals the access path maintenance job and then ends. The
access path maintenance job uses additional jobs, if needed, to change the
access path maintenance attribute to immediate on all files that MIMIX had
previously changed to delayed. Any files that could not be changed are identified
as having an access path maintenance error before the maintenance jobs end.
• The specified replication processes end in the manner specified for the End
process (ENDOPT) parameter. The command defaults to processing the end
request immediately (*IMMED). When invoked by the ENDMMX command, the
default value specified on ENDMMX is *CNTRLD, which takes precedence. When
invoked by a procedure specified on the ENDAG command, the procedure
determines whether ENDDG is passed parameter values or uses the command
defaults.
• if a controlled end is requested, it uses the specified Wait time and Timeout
options.
• If requested, the RJ link is ended. The RJ link is not automatically ended. In most
cases, the default value *NO for the End remote journaling (ENDRJLNK)
parameter is appropriate. Keeping the RJ link active allows database changes to
continue to be sent to the target system even though the data group is not active.
• The Enable constraints (ENBCST) parameter determines whether the database
apply processes will attempt to enable constraints on target system files that were
previously disabled by MIMIX. The default value, *DFT, will enable constraints if
appropriate configuration conditions exist and the end request specifies all apply
sessions and includes database processes that run on the target system.
• If you have used the MIMIX CDP feature to set a recovery point in a data group
and then end the data group, the recovery point will be cleared. When the data
group is started again, the apply processes will process any available
transactions, including those which may have had corruptions. (Recovery points
are set with the Set DG Recovery Point (SETDGRCYP) command.) If a recovery
window is configured for the data group, its configured duration is not affected by
238
What occurs when a data group is ended
239
Ending MIMIX
For most configurations, It is recommended that you end MIMIX products from the
management system, which is usually the backup system. If your installation is
configured so that the backup system is a network system, you should end MIMIX
from the network system.
Notes:
• If you are ending MIMIX for a software upgrade or to install a service pack, use
the procedures in the software’s ReadMe document.
• The ENDMMX command cannot run when application groups are configured and
there are any active, failed, or canceled procedures.
• Determine whether you also need to end RJ links. See “When to end RJ links” on
page 236.
To end MIMIX, use the following procedures:
1. Use one of the following procedures:
• “Ending with default values” on page 240
• “Ending by prompting the ENDMMX command” on page 240
2. Complete any needed follow-up actions using the information and procedures in
“After you end MIMIX products” on page 241.
240
Ending MIMIX
When to also end the MIMIX subsystem - You will also need to end the MIMIX
subsystem when you need to IPL the system, when upgrading MIMIX software, and
when installing a MIMIX software service pack. The MIMIX subsystem must be ended
from the native user interface. To end the subsystem, do the following:
1. If you are using Vision Solutions Portal (VSP) end the VSP server on the systems
where it runs. This prevents object locking issues. Do one of the following:
• If the VSP server runs on an IBM i platform, use the following command:
VSI001LIB/ENDVSISVR
• If the VSP server runs on a Windows platform, from the Windows Start menu,
select
All Programs > Vision Solutions Portal > Stop Server and click Stop Server.
2. Enter the command WRKSBS. The Work with Subsystems display appears.
3. Type an 8 (Work with subsystem jobs) next to subsystem MIMIXSBS and press
Enter.
4. End any remaining jobs in a controlled manner. Type a 4 (End) next to the job and
press F4 (Prompt). The How to end (OPTION) parameter should have a value of
*YES. Press Enter. If you see a confirmation display, press Enter to continue.
5. Press F12 (Cancel) to return to the Work with Subsystems display.
6. Type a 4 (End subsystem) next to subsystem MIMIXSBS and press Enter.
241
242
Ending an application group
243
Ending a data group in a controlled manner
The following procedures describe how to check for errors before requesting a
controlled end of a data group, how to perform the controlled end request, and how to
confirm that the end completed. Held files must be released and the apply process
must complete operations for journal entries stored in log spaces before you end data
group activity.
Data groups that are in an application group: The preferred method of ending data
groups that are part of an application group is to use the End Application Group
(ENDAG) command.
244
Ending a data group in a controlled manner
processes prompt.
3. If the data group uses remote journaling, verify that the value of the End remote
journaling prompt is what you want.
4. For the Wait Time (WAIT) parameter, specify how long MIMIX should try to end
the selected processes in a controlled manner. Use F1 (Help) to see additional
information about the possible options.
• Specify *SBMRQS to submit a request to end the data groups. The appropriate
actions are issued to end the specified processes, and control is returned to
the caller immediately. When you specify this value, the TIMOUTOPT
parameter is ignored so you can skip Step 5.
• Specify *NOMAX. When you specify this value, MIMIX will wait until all
specified MIMIX processes are ended.
• Specify a numeric value (number-of-seconds). MIMIX waits the specified time
for a controlled end to complete before using the option specified in the
TIMOUTOPT parameter.
5. If you specified a numeric value for the WAIT parameter in Step 4, you can also
use the Timeout Option (TIMOUTOPT) parameter. You can specify what action
you want the ENDDG command to perform if the time specified in the WAIT
parameter is reached:
• The current process should quit and return control to the caller (*QUIT).
• A new request should be issued to end all processes immediately
(*ENDIMMED). When this value is specified, pending activity entries may still
exist after the data group processes are ended.
• An inquiry message should be sent to the operator notifying of a possible error
condition (*NOTIFY). If you specify this value, the command must be run from
the target system.
6. If the data group you are ending is configured to allow target constraint
management, specify whether you want to enable constraints on target files which
were previously disabled by MIMIX. If you specified *SBMRQS for WAIT, the
database apply processes must be already ended by a previous controlled end. If
you specified a number for WAIT, the replication processes must end in the
specified time. The value *DFT will enable constraints for *ACTIVE file entries
after replication processes end.
7. Press Enter to process the command.
245
entries exist when you end the data group and perform a switch, you may lose
these entries when the data group is started following the switch.
Note: To ensure that you are aware of any possible pending or delayed activity
entries, enter the WRKDGACTE STATUS(*ACTIVE) command. Any
activities that are still in progress will be listed. Ensure that all activities are
completed.
3. Ensure that there are no open commit cycles.The next attempt to start the data
group will fail if open commit cycles exist and either the start request specified to
clear pending entries (CLRPND(*YES)) or the commit mode specified in the data
group definition changed. (Certain process, such as performing a hardware
upgrade with a disk image change, converting to MIMIX Dynamic Apply, changing
target constraint management configuration, or enabling a disabled data group,
require a clear pending start.) To verify commit cycles, do the following:
a. Press F8 (Database) to view the Data Group Detail Status display.
b. For each apply session listed, verify that the value shown in the Open Commit
column at the right side of the display is *NO.
c. If open commit cycles exist, restart the data group. You must take action to
resolve the open commit cycles, such as ending or quiescing the application or
closing the commit cycle. Then repeat the controlled end again.
246
Ending selected data group processes
247
Enabling MIMIX-disabled target constraints
MIMIX supports t several ways of enabling constraints on target node files that were
previously disabled by MIMIX.
Default shipped procedures for switching application groups automatically enable
MIMIX-disabled constraints before performing the requested switch. However, you
may need to enable constraints outside the scope of a switch procedure, such as
before manually switching a data group with the Switch Data Group command or
before performing other operations for which you would want constraints enabled on
the target node. Even when target constraint management is not configured, you may
need to enable any remaining constraints that failed to be enabled on the target node.
You can manually request MIMIX to enable constraints in these ways:
• When ending an application group, the shipped procedures END and ENDTGT
include disabled steps MXECSTB and MXECSTR. When enabled, these steps
enable constraints previously disabled by MIMIX. To allow these steps to be
performed by either procedure, you must take action to enable the steps from
within the procedure. Enabling steps must be performed from the native user
interface.
• When ending replication processes for a specific data group using the Stop Data
Group dialog from Vision Solutions Portal (VSP) or the End Data Group (ENDDG)
command from the native user interface. Constraint processing can only occur
when the end request specifies all required values and other configuration
requirements are met.
• For an ended data group. When replication processes have already been ended
by a controlled end request and there are no open commit cycles for the data
group, you can enable constraints using the ENDDG command in the native user
interface or the Enable Constraints action for a data group from VSP.
Only MIMIX-disabled constraints on target node files for which there are existing data
group file entries with a status of *ACTIVE1 can be enabled.
When the following conditions exist and these values are specified on an end data
group request, MIMIX will enable constraints on the target node files after the
specified replication processes end and before the requested command completes:
• Process (PRC) must specify processes that run on the target node (*ALL,
*ALLTGT, *DBALL, *DBTGT, or *DBAPY)
• Configuration requirements for the specified End process (ENDOPT) must be
met.
– When specifying *IMMED for an immediate end, or when specifying *CNTRLD
1. For users of Vision Solutions Portal (VSP), data group file entries are known as file selection
rules. File selection rules are assigned an initial status when they are created, which is not
displayed in VSP. When replication processes start, file selection rules correspond to file
activities and their status become Active unless replication activity for the identified file or
member was previously held or in error. In VSP, only the file activities that are experiencing
database replication problems are displayed on the File Activity tab of the Data Group
Details and Activities portlet.
248
Enabling MIMIX-disabled target constraints
for a controlled end and a wait time (WAIT) value of *SBMRQS, database
apply processes must have already been ended by a previous controlled end
request.
– When specifying *CNTRLD for a controlled end and a number for wait time
(WAIT), the data group must be configured for target constraint management
and its replication processes must end within the specified wait time and have
no open commit cycles.
• Apply sessions (APYSSN) must specify *ALL.
• Enable constraints (ENBCST) must specify or resolve to *YES. The shipped
default value, *DFT, resolves to *YES when all other conditions are met.
If the requirements are not met, the constraints are not enabled and no error message
is issued.
MIMIX-disabled constraints that failed to be enabled may still exist on the target node
after the end data group request completes or after the configuration is changed to no
longer allow target constraint management.
When target constraint management is configured, specifying the value *NO for
ENBCST allows any MIMIX-disabled constraints to remain disabled, improving
performance of this end operation and the next start operation.
249
2. The End Data Group (ENDDG) display appears. At the Process prompt, specify
*ALL, *ALLTGT, *DBALL, *DBTGT, or *DBAPY.
3. Specify *CNTRLD for the End process prompt.
4. Press F9 (All parameters).
5. Specify either a number or the value *NOMAX for the Wait time (seconds) prompt.
6. Confirm that the value *ALL is specified for the Apply session prompt.
7. Specify *YES for the Enable constraints prompt.
8. Press Enter.
250
What replication processes are started by the STRDG command
Table 41. Processes started by data groups configured for MIMIX Remote Journal support. This assumes that all replication processes are inactive when the STRDG
request is made.
RJ Link 1 OBJSND OBJRTV CNRSND STSRCV DBRDR DBAPY2 OBJRCV CNRRCV STSSND OBJAPY
*ALL E Starts1 Starts Starts Starts Starts Starts Starts Starts Starts Starts Starts
*ALLSRC A, E Starts1 Starts Starts Starts Starts Inactive Inactive Starts Starts Starts Inactive
*ALLTGT A, B Inactive1 Inactive Inactive Inactive Inactive Starts Starts Inactive Inactive Inactive Starts
*DBALL A, E Starts1 Inactive3 Inactive3 Inactive3 Inactive3 Starts Starts Inactive3 Inactive3 Inactive3 Inactive3
Notes:
A. Data groups which use cooperative processing should have both database and object processes started to prevent objects and data on the target system from
becoming not fully synchronized.
B. When the RJ link is already active, database replication becomes operational.
C. When the RJ link is already active, database journal entries continue to transfer to the target system over the RJ link
D. When the RJ link is already active, database journal entries continue to transfer to the target system over the RJ link, where they will be processed by the DBRDR.
E. If data group data area entries are configured, the data area polling process also starts when values which start database source processes are selected.
251
What replication processes are started by the STRDG command
Table 41. Processes started by data groups configured for MIMIX Remote Journal support. This assumes that all replication processes are inactive when the STRDG
request is made.
RJ Link 1 OBJSND OBJRTV CNRSND STSRCV DBRDR DBAPY2 OBJRCV CNRRCV STSSND OBJAPY
*OBJALL A, C Inactive1 Starts Starts Starts Starts Inactive4 Inactive 4 Starts Starts Starts Starts
*DBSRC A, C, Starts1 Inactive3 Inactive3 Inactive3 Inactive3 Inactive Inactive Inactive3 Inactive3 Inactive3 Inactive3
E
*DBTGT A, B Inactive1 Inactive3 Inactive3 Inactive3 Inactive3 Starts Starts Inactive3 Inactive3 Inactive3 Inactive3
*OBJSRC A, C Inactive1 Starts Starts Starts Starts Inactive4 Inactive4 Starts Starts Starts Inactive
*OBJTGT A, C Inactive1 Inactive Inactive Inactive Inactive Inactive4 Inactive4 Inactive Inactive Inactive Starts
*DBRDR A, D Inactive1 Inactive3 Inactive3 Inactive3 Inactive3 Starts Inactive Inactive3 Inactive3 Inactive3 Inactive3
*DBAPY A, C Inactive1 Inactive3 Inactive3 Inactive3 Inactive3 Inactive4 Starts4 Inactive3 Inactive3 Inactive3 Inactive3
Notes:
A. Data groups which use cooperative processing should have both database and object processes started to prevent objects and data on the target system from
becoming not fully synchronized.
B. When the RJ link is already active, database replication becomes operational.
C. When the RJ link is already active, database journal entries continue to transfer to the target system over the RJ link
D. When the RJ link is already active, database journal entries continue to transfer to the target system over the RJ link, where they will be processed by the DBRDR.
E. If data group data area entries are configured, the data area polling process also starts when values which start database source processes are selected.
1. This column shows the effect of the specified value on the RJ link when the RJ link is not active. See the Notes for the effect of values when the RJ Link is already active, which is
default behavior.
2. If the access path maintenance (APMNT) policy has been enabled at the installation or data group level, an access path maintenance job is also started. Access path maintenance is
available on installations running 7.1.15.00 or higher.
3. These object replication processes are not available in data groups configured for database-only replication.
4. These database replication processes are not available in data groups configured for object-only replication.
Optionally, data groups can use source-send technology instead of remote journaling for database replication. Data groups created on
earlier levels of MIMIX may still be configured this way.
252
What replication processes are started by the STRDG command
Table 42 identifies the processes that are started by each value for Start processes when source-send technology is used for database
replication. The MIMIX database send (DBSND) process and database receive (DBRCV) process replace the IBM i remote journal
function and the DBRDR process, respectively.
Table 42. Processes started by data groups configured for Source Send replication This assumes that all replication processes are inactive when the STRDG request
is made.
DBSND 1 OBJSND OBJRTV CNRSND STSRCV DBRCV DBAPY2 OBJRCV CNRRCV STSSND OBJAPY
*ALL — Starts 1 Starts Starts Starts Starts Starts Starts Starts Starts Starts Starts
*ALLSRC A Starts 1 Starts Starts Starts Starts Starts Inactive Starts Starts Starts Inactive
*ALLTGT A Inactive Inactive Inactive Inactive Inactive Inactive Starts Inactive Inactive Inactive Starts
*DBALL A Starts 1 Inactive3 Inactive3 Inactive3 Inactive3 Starts Starts Inactive3 Inactive3 Inactive3 Inactive3
*OBJALL A Inactive 4 Starts Starts Starts Starts Inactive4 Inactive4 Starts Starts Starts Starts
*DBSRC A Starts 1 Inactive3 Inactive3 Inactive3 Inactive3 Starts Inactive Inactive3 Inactive3 Inactive3 Inactive3
*DBTGT A Inactive Inactive3 Inactive3 Inactive3 Inactive3 Inactive Starts Inactive3 Inactive3 Inactive3 Inactive3
*OBJSRC A Inactive4 Starts Starts Starts Starts Inactive4 Inactive4 Starts Starts Starts Inactive
*OBJTGT A Inactive4 Inactive Inactive Inactive Inactive Inactive4 Inactive4 Inactive Inactive Inactive Starts
*DBAPY A Inactive4 Inactive3 Inactive3 Inactive3 Inactive3 Inactive4 Starts 4 Inactive3 Inactive3 Inactive3 Inactive3
Notes:
A. Data groups which use cooperative processing should have both database and object processes started to prevent objects and data on the target system from
becoming not fully synchronized.
1. When the database send (DBSND) process starts, the data area polling process also starts.
2. If the access path maintenance (APMNT) policy has been enabled at the installation or data group level, an access path maintenance job is also started. Access path maintenance is
available on installations running 7.1.15.00 or higher.
3. These object replication processes are not available in data groups configured for database-only replication.
4. These database replication processes are not available in data groups configured for object-only replication
253
What replication processes are started by the STRDG command
5. The database reader (*DBRDR) process is not used by data groups configured for source-send replication.
254
What replication processes are ended by the ENDDG command
Table 43. Processes ended by data groups configured for MIMIX Remote Journal support. This assumes that all replication processes are active when the ENDDG
request is made and that the request does not specify to end the RJ link.
RJ Link 1 OBJSND OBJRTV CNRSND STSRCV DBRDR DBAPY2 OBJRCV CNRRCV STSSND OBJAPY
*ALL E Active1 Ends Ends Ends Ends Ends Ends Ends Ends Ends Ends
*ALLSRC A, E Active1 Ends Ends Ends Ends Active Active Ends Ends Ends Active
*ALLTGT — Active1 Active Active Active Active Ends Ends Active Active Active Ends
*DBALL B, E Active1 Active 3 Active 3 Active 3 Active 3 Ends Ends Active 3 Active 3 Active 3 Active 3
Notes:
A. Has no effect on database-only replication. New database journal entries continue to transfer to the target system over the RJ link, where they will be processed.
B. Data groups that use cooperative processing may be affected by the result of this value. Ending database processes while object processes remain active may
result in object activity entries being placed on hold. Similarly, ending object processes while database processes remain active may result in files being placed on
hold due to error.
C. New database journal entries continue to transfer to the target system over the RJ link. Existing entries stored in the log space on the target system before the end
request was processed will be applied.
D. New database journal entries continue to transfer to the target system over the RJ link, where they will be processed by the DBRDR.
E. The data area polling process ends when values which end database source processes are specified.
255
What replication processes are ended by the ENDDG command
Table 43. Processes ended by data groups configured for MIMIX Remote Journal support. This assumes that all replication processes are active when the ENDDG
request is made and that the request does not specify to end the RJ link.
RJ Link 1 OBJSND OBJRTV CNRSND STSRCV DBRDR DBAPY2 OBJRCV CNRRCV STSSND OBJAPY
*OBJALL A, B Active1 Ends Ends Ends Ends Active 4 Active 4 Ends Ends Ends Ends
*DBSRC A, B, Active1 Active 3 Active 3 Active 3 Active 3 Active Active Active 3 Active 3 Active 3 Active 3
E
*DBTGT B Active1 Active 3 Active 3 Active 3 Active 3 Ends Ends Active 3 Active 3 Active 3 Active 3
*OBJSRC A, B Active1 Ends Ends Ends Ends Active 4 Active 4 Ends Ends Ends Active
*OBJTGT A, B Active1 Active Active Active Active Active 4 Active 4 Active Active Active Ends
*DBRDR B, C Active1 Active 3 Active 3 Active 3 Active 3 Ends Active Active 3 Active 3 Active 3 Active 3
*DBAPY B, D Active1 Active 3 Active 3 Active 3 Active 3 Active Ends Active 3 Active 3 Active 3 Active 3
Notes:
A. Has no effect on database-only replication. New database journal entries continue to transfer to the target system over the RJ link, where they will be processed.
B. Data groups that use cooperative processing may be affected by the result of this value. Ending database processes while object processes remain active may
result in object activity entries being placed on hold. Similarly, ending object processes while database processes remain active may result in files being placed on
hold due to error.
C. New database journal entries continue to transfer to the target system over the RJ link. Existing entries stored in the log space on the target system before the end
request was processed will be applied.
D. New database journal entries continue to transfer to the target system over the RJ link, where they will be processed by the DBRDR.
E. The data area polling process ends when values which end database source processes are specified.
1. The RJ link is not ended by the End options (PRC) parameter. New database journal entries continue to transfer to the target system over the RJ link. See the Notes column for addi-
tional details.
2. On installations running 7.1.15.00 or higher, if access path maintenance is enabled, the database apply process signals the access path maintenance job and then ends. The access
path maintenance job uses additional jobs, if needed, to change the access path maintenance attribute to immediate on all files that MIMIX had previously changed to delayed. Any
files that could not be changed are identified as having an access path maintenance error before the maintenance jobs end.
On installations running software earlier than 7.1.15.00, if parallel access path maintenance function is enabled, the associated monitors are also ended when ENDDG command
specifies *DFT for End parallel AP maintenance (PRLAPMNT) and *ALL for the Apply session (APYSSN). When *YES is specified for PRLAPMNT, the function is always ended
regardless of the values specified for PRC or APYSSN.
3. These object replication processes are not available in data groups configured for database-only replication.
256
What replication processes are ended by the ENDDG command
4. These database replication processes are not available in data groups configured for object-only replication.
Optionally, data groups can use source-send technology instead of remote journaling for database replication. Data groups created on
earlier levels of MIMIX may still be configured this way. Table 44 identifies the processes that are ended by each value for End options
when source-send technology is used for database replication. The MIMIX database send (DBSND) process and database receive
(DBRCV) process are replaced by the IBM i remote journal function and the DBRDR process, respectively.
Table 44. Processes ended by data groups configured for Source Send replication This assumes that all replication processes are active when the ENDDG request is
made.
DBSND 1 OBJSND OBJRTV CNRSND STSRCV DBRCV DBAPY2 OBJRCV CNRRCV STSSND OBJAPY
*ALL — Ends 1 Ends Ends Ends Ends Ends Ends Ends Ends Ends Ends
*ALLSRC — Ends 1 Ends Ends Ends Ends Ends Active Ends Ends Ends Active
*ALLTGT — Active Active Active Active Active Active Ends Active Active Active Ends
*DBALL A Ends 1 Active 3 Active 2 Active 2 Active 2 Ends Ends Active 2 Active 2 Active 2 Active 2
*OBJALL A Active 4 Ends Ends Ends Ends Active 3 Active 3 Ends Ends Ends Ends
*DBSRC A Ends 1 Active 2 Active 2 Active 2 Active 2 Ends Active Active 2 Active 2 Active 2 Active 2
*DBTGT A Active Active 2 Active 2 Active 2 Active 2 Active Ends Active 2 Active 2 Active 2 Active 2
*OBJSRC A Active 3 Ends Ends Ends Ends Active 3 Active 3 Ends Ends Ends Active
*OBJTGT A Active 3 Active Active Active Active Active 3 Active 3 Active Active Active Ends
*DBAPY A Active 3 Active 2 Active 2 Active 2 Active 2 Active 3 Ends 3 Active 2 Active 2 Active 2 Active 2
Notes:
A. Data groups that use cooperative processing may be affected by the result of this value. Ending database processes while object processes remain active may
result in object activity entries being placed on hold. Similarly, ending object processes while database processes remain active may result in files being placed on
hold due to error.
1. When the database send (DBSND) process ends, the data area polling process also ends.
257
What replication processes are ended by the ENDDG command
2. On installations running 7.1.15.00 or higher, if access path maintenance is enabled, the database apply process signals the access path maintenance job and then ends. The access
path maintenance job uses additional jobs, if needed, to change the access path maintenance attribute to immediate on all files that MIMIX had previously changed to delayed. Any
files that could not be changed are identified as having an access path maintenance error before the maintenance jobs end.
On installations running software earlier than 7.1.15.00, if parallel access path maintenance function is enabled, the associated monitors are also ended when ENDDG command
specifies *DFT for End parallel AP maintenance (PRLAPMNT) and *ALL for the Apply session (APYSSN). When *YES is specified for PRLAPMNT, the function is always ended
regardless of the values specified for PRC or APYSSN.
3. These object replication processes are not available in data groups configured for database-only replication.
4. These database replication processes are not available in data groups configured for object-only replication
5. The database reader (*DBRDR) process is not used by data groups configured for source-send replication.
258
CHAPTER 12 Resolving common replication
problems
Occasionally, a journaled transaction for a file or object may fail to replicate. User
intervention is required to correct the problem. This chapter provides procedures to
help you resolve problems that can occur during replication processing.
The following topics are included in this chapter:
• “Replication problems that MIMIX attempts to recover” on page 260 identifies the
common replication errors that MIMIX will automatically correct when the
database automatic recovery and object automatic recovery policies are enabled.
• “Working with user journal replication errors” on page 263 includes topics for how
to resolve a file that is held due to an error. It also includes topics about options for
placing a file on hold and releasing held files.
• “Working with tracking entries” on page 272 describes how to use tracking entries
to resolve replication errors for IFS objects, data areas, or data queues that are
replicated cooperatively with the user journal. It also includes topics about options
for placing a tracking entry on hold and releasing held tracking entries.
• “Working with objects in error” on page 278 describes how to resolve objects in
error by working with the data group activities used for system journal replication.
This topic includes information about how to retry failed activity entries and how to
determine whether MIMIX is automatically attempting to retry an activity.
• “Removing data group activity history entries” on page 285 describes how to
manually remove completed entries for system journal replication activity. This
may be necessary if you need to conserve disk space.
• “Working with message queues” on page 283 describes how to use the MIMIX
primary and secondary message queues from the native user interface.
• “Working with the message log” on page 284 describes how to access the MIMIX
message log from either user interface.
•
259
Replication problems that MIMIX attempts to recover
When the Automatic system journal recovery (OBJRCY) and Automatic user journal
recovery (DBRCY) policies are enabled, MIMIX replication processes automatically
attempt to correct detected replication errors. When an error is detected, the
replication process initiates a recovery action to perform the corrective activity.
Replication status remains unchanged for the file entry, tracking entry, or activity entry
associated with the affected file or object while the recovery action is being processed
or is waiting to be processed. If the recovery action is successful, the file or object is
tracked for prioritized auditing. If the recovery action cannot be performed or fails,
then a replication error is reported for the affected file or object.
From the native user interface, counts of new or in-progress recoveries are available
on the Work with Data Groups display and in data group details, but only a limited
subset of replication recoveries can be viewed from the Work with Recoveries display.
When a recovery fails and an error is reported, you can use options available from the
Work with Data Groups display to access details and to manually resolve the
problem.
If you use the MIMIX portal application in Vision Solutions Portal, you can use the
Object Correction Activity window to view new or in-progress recoveries for replication
activity as well as recoveries initiated by other processes, including target journal
inspection, audits, and virtual switch procedures.
The following topics identify the replication errors that MIMIX can detect and attempts
to automatically correct:
• “Replication errors handled by automatic database recovery” on page 260
• “Replication errors handled by automatic object recovery” on page 261
The Automatic system journal recovery (OBJRCY) and Automatic user journal
recovery (DBRCY) policies also enable the ability to automatically correct changes
detected by target journal inspection processes. The types of changes detected and
eligible for automatic correction are described in the MIMIX Administrator Reference
book.
260
Replication problems that MIMIX attempts to recover
Table 45. Database replication errors detected and corrected when automatic database recovery is enabled.
Error Description
File level errors Typically invoked when there is a missing library, file, or member. Also invoked
- and - when an attempt to write a record to a file results in a unique key violation.
Unique-key record level Without database automatic recovery, these conditions result in the file being
error placed in *HLDERR status.
Record level errors Invoked when the database apply process detects a data-level issue while
processing record-level transactions.
Without database automatic recovery, any configured collision resolution
methods may attempt to correct the error. Otherwise, these conditions result in
the file being placed in *HLDERR status.
Errors on IFS objects Invoked during the priming of IFS tracking entries when replicated IFS objects
configured for user are determined to be missing from the target system. Priming of tracking entries
journal replication occurs when a data group is started after a configuration change or when Deploy
Data Grp. Configuration (DPYDGCFG) is invoked.
Errors on data area and Invoked during the priming of object tracking entries when replicated data area
data queue objects and data queue objects are determined to be missing from the target system.
configured for user Priming of tracking entries occurs when a data group is started after a
journal replication configuration change or when the Deploy Data Grp. Configuration (DPYDGCFG)
is invoked.
Errors when DBAPY Invoked when a temporary lock condition or an operating system condition exists
cannot open the file or that prevents the database apply process (DBAPY) from opening the file or
apply transactions to the applying transactions to the file. Without database automatic recovery, users
file typically have to release the file so the database apply process (DBAPY) can
continue without error.
261
Only when all recovery options are exhausted without success is an activity entry
placed in error status. Recovery actions that end in an error do not generate a
separate error notification because the error is already reflected in MIMIX status.
Table 46. Object replication errors detected and corrected when automatic object recovery is enabled.
Error Description
Missing objects An object (library-based, IFS, or DLO) exists on the source system and is within the name
on target space for replication, but MIMIX detects that the object does not exist on the target
system1 system. Without object automatic recovery, this results in a failed activity entry.
Notes:
• Missing spooled files are not addressed.
• Missing objects that are configured for cooperative processing are not synchronized.
However, any problems with authorities (*AUTL or *USRPRF) for the missing objects
are addressed.
Missing parent Any operation against an object whose parent object is missing on the target system.
objects on Without object automatic recovery, this condition results in a failed activity entry due to
target system1 the missing parent object.
Missing Any operation that requires a user profile object (*USRPRF) that does not exist on the
*USRPRF target system. Without object automatic recovery, this results in authority or object owner
objects on issues that cause replication errors.
target system1
Missing *AUTL Any operation that requires a authority list (*AUTL) that does not exist on the target
objects on system.Without object automatic recovery, this results in authority issues that cause
target system1 replication errors.
In-use Applications which hold persistent locks on objects can result in object replication errors if
condition the configured values for delay/retry intervals are exceeded. Default values in the data
group definition provide approximately 15 minutes during which MIMIX attempts to
access the object for replication. If the object cannot be accessed during this time, the
result is activity entries with errors of Failed Retrieve (for locked objects on the source
system) and Failed Apply (for locked objects on the target system) and a reason code of
*INUSE.
Notes:
1. The Number of third delay/retries policy and the Third retry interval policy determine
whether automatic recovery is attempted for this error. These policies are in effect
when the object automatic recovery policy is enabled.
2. Automatic recovery for this error is not attempted when the objects are configured for
cooperative processing.
1. The synchronize command used to automatically recover this problem during replication will correct this error any time
the command is used.
262
Working with user journal replication errors
Working with files needing attention (replication and access path errors)
The DB Errors column on the Work with Data Groups display identifies the number of
errors for user journal replication. Specifically, this column identifies the sum of the
number of database files, IFS, *DTAARA, and *DTAQ objects on hold due to errors
(*HLDERR) plus the number of LF and PF files that have access path maintenance1
failures for a data group. Data group file entries and tracking entries should not be left
in *HLDERR state for any extended time. Access path maintenance errors occur
when MIMIX could not change a file’s access path maintenance attribute back to
immediate.
To access a list of files in error for a data group, do the following:
1. From the MIMIX Basic Main Menu select option 6 (Work with data groups) and
press Enter.
2. The Work with Data Groups display appears. Type 12 (Files needing attention)
next to the data group you want which has errors identified in the DB Errors
column and press Enter.
3. The Work with DG File Entries display appears with a list of file entries for the data
group that have replication errors, access path maintenance2 errors, or both. Do
the following:
a. The initial view shows the current replication status of file entries. Any entry
with a status of *HLD, *HLDERR, *HLDIGN or *HLDRLTD indicates that action
is required. Use Table 47 to identify choices based on the file entry status.
1. Errors for the access path maintenance function are included on installations running MIMIX
7.1.15.00 or higher.
2. Access path maintenance errors can only be reported on data group file entries in installa-
tions running MIMIX 7.1.15.00 or higher.
263
Note: MIMIX retains log spaces for file entries with these statuses so that the
journal entries that are being held can be released and applied to the
target system. File entries should not be left in these states for an
extended period.
b. Use Table 47 to identify choices based on the file entry status and Table 48 to
identify available options from this display.
c. If necessary, take action to prevent the error from happening again. Refer to
the following topics:
• “Correcting file-level errors” on page 269
• “Correcting record-level errors” on page 270
4. Press F10 as needed on the Work with DG File entries display until you see the
access path maintenance view. The AP Maint. Status column identifies any AP
maintenance errors for a file with the value *FAILED and failures for logical files
associated with a file as *FAILEDLF.
Immediate action may not be necessary because MIMIX will attempt to retry
access path maintenance when the data group ends and when it is restarted. To
attempt an immediate retry, use option 40 (Retry AP maintenance).
*ACTIVE Unless an error has occurred, no action is necessary. Entries in the user
journal for the file are replicated and applied. If necessary, any of the
options to hold journal entries can be used.
*HLD User action is required to release the file entry (option 26) so that held
journal entries from the user journal can be applied to the target system.
*HLDERR User action is required. Attempt to resolve the error by synchronizing the
file (option 16).
Note: Transactions and hold logs are discarded for file entries with a status of
*HLDERR and an error code of IG. Such a file must be synchronized.
*HLDIGN User action is required to either synchronize the file (option 16) or to
change the configuration if you no longer want to replicate the file.
Journal entries for the file are discarded. Replication is not occurring and
the file may not be synchronized.
Depending on the circumstances, Release may also be an option.
*HLDRGZ These are transitional states that should resolve to *ACTIVE. If these
*HLDRNM status persist, check the journaling status for the entry. MIMIX retains log
*HLDPRM spaces for the held journal entries for the duration of these temporary
hold requests.
*HLDSYNC
264
Working with user journal replication errors
*HLDRTY The file entry is held because an entry could not be applied due to a
condition which required waiting on some other condition (such as in-
use). After a short delay, the database apply job will automatically
attempt to process this entry again. The preferred action is to allow
MIMIX to periodically retry the file entry. By default, the database apply
job will automatically attempt to process the entry every 5 minutes for up
to 1 hour.
Manually releasing the file entry will cause MIMIX to attempt to process
the entry immediately.
Note: From Vision Solutions Portal, you can view information about locks on the
object on both systems. From the File Activity tab on the Data Group
Details and Activities portlet, select the Compare Details action to view
the Locks tab of the Compare Object Details from Nodes window.
*HLDRLTD User action is required for a file in the same network. View the related
files (option 35). A file that is related due to a dependency, such as a
constraint or a materialized query table, is held. Resolving the problem
for the related held file will resolve this status.
*RLSWAIT The file is waiting to be released by the DB apply process and will be
changed to *ACTIVE. If the status does not change to *ACTIVE, check
the journaling status. If this status persists, you may need to synchronize
(option 16).
*CMPACT These are transitional states that should resolve automatically. The file
*CMPRLS entry represents a member that is being processed cooperatively
*CMPRPR between the CMPFILDTA command and the database apply process.
Table 48. Options for working with file entries from the Work with DG FIle Entries display
9=Start journaling See “Starting journaling for physical files” on page 291.
10=End journaling See “Ending journaling for physical files” on page 292.
11=Verify journaling See “Verifying journaling for physical files” on page 293.
20=Work with file See topic “Working with journal transactions for files in error” on
error entries page 266.
265
Table 48. Options for working with file entries from the Work with DG FIle Entries display
27=Release clear See topic “Releasing a held file and clearing entries” on page 269.
31=Repair member Available for entries with a status of *HLDERR that identify a
data member. See topic ‘Comparing and repairing file data - members
on hold (*HLDERR)’ in the MIMIX Administrator Reference book.
35=Work with related Displays file entries that are related to the selected file by
files constraints or by other dependencies such as materialized query
tables
266
Working with user journal replication errors
b. Select the option (Table 49) you want to use on the journal transaction:
Table 49. Options available from the Work with DG FE on Hold display.
2=Change You can change the contents or characteristics of the journal entry.
Use this option with caution. Any changes can affect the validity of
data in the journal entry.
5=Display You can display details for the specified journal entry associated with
the data group file entry in question.
9=Immediate You can immediately apply a transaction that has caused a file to go
apply on hold. The entry you selected is immediately applied to the file
outside of the apply process. If the apply is successful, the error/hold
entry that was applied is removed from the error/hold log. However, if
the apply fails, a message is issued and the entry remains in the
error/hold log. This process does not release the file; it only applies the
selected entry.
267
Note: Be certain that you want to use the ignore feature. Any ignored transactions
cannot be retrieved. You must replace the object on the target system with a
current version from the source system.
If a file has been on hold for a long time or you expect that it will be, the amount of
storage used by the error/hold log space can be quite large. If you anticipate that you
will need to save and restore the file or replace it for any other reason, it may be best
to just ignore all current transactions.
Do the following:
1. From the MIMIX Basic Main Menu select option 6 (Work with Data Groups) and
press Enter.
2. The Work with Data Groups display appears. Type 17 (File entries) next to the
data group you want and press Enter.
3. The Work with DG File Entries display appears. Type 24 (Ignore file) next to the
entry you want and press Enter.
The status of the file is changed to *HLDIGN. The file entry is ignored. Journal
entries for the file entry, including any hold logs, are discarded.
268
Working with user journal replication errors
The request changes the file entry status to *ACTIVE. Any held journal entries for the
associated file are applied. Normal replication of the file resumes.
While a file is being released, the appropriate apply session suspends its operations
on other files. This allows the released file to catch up to the current level of
processing. If a file or member has been on hold for a long time, this can be lengthy.
Do the following to immediately release a held file or file member:
1. From the MIMIX Basic Main Menu select option 6 (Work with Data Groups) and
press Enter.
2. The Work with Data Groups display appears. Type 17 (File entries) next to the
data group you want and press Enter.
3. The Work with DG File Entries display appears. Type 26 (Release) next to the
entry you want and press Enter
269
Once you diagnose and correct a file-level error, the problem rarely manifests itself
again. Some of the most common file-level errors are:
• Authority: The MIMIXOWN user profile defined in the MIMIX job description does
not have authority to perform a function on the target system. You can prevent this
problem by ensuring that the MIMIXOWN user profile has all object authority
(*ALLOBJ). This guarantees that the user profile has all the necessary authority to
run IBM i commands and has the ability to access the library and files on the
management system. If you are attempting to replicate files that have protected
rows or columns, or system-period temporal tables and their associated history
tables, the MIMIXOWN user profile may not have been granted the necessary
function usage. For more information about requirements for replicating files that
use permissions or masks for row and column access control (RCAC), see the
MIMIX Administrator Reference book. For more information about the MIMIXOWN
user profile and authority, see the Using License Manager book.
• Objects existence or corruption: MIMIX cannot run a function against a file on
the target system because the file or a supporting object (such as logical files)
does not exist or has become damaged. System security is the only way to
prevent an object from being accidentally deleted from the target system. Make
sure that only the correct personnel have the ability to remove objects from the
target system where replicated data is applied. Also, ensure that application
programs do not delete files on the target system when there are no apply
sessions running.
• MIMIX subsystem ended: If the MIMIX subsystem is ended in an immediate
mode while MIMIX processes are still active, files may be placed in a “Held”
status. This is a result of MIMIX being unable to complete a transaction normally.
After MIMIX is restarted, you only need to release the affected files.
270
Working with user journal replication errors
• Journaling was ended: When journaling is ended, transaction images are not
being collected. If users update the files while journaling is not running, no journal
entries are created and MIMIX DB Replicator has no way of replicating the
missing transactions. The best way to prevent this error is to restrict the use of the
Start Journaling Physical File (STRJRNPF) and End Journaling Physical File
(ENDJRNPF) commands.
• User journal replication was restarted at the wrong point: When you change
the starting point of replication for a data group, it is imperative that transactions
are not skipped.
• Apply session restarted after a system failure: This is caused when the target
system experiences a hard failure. MIMIX always updates its user spaces with the
last updated and sent information. When a system fails, some information may not
be forced to disk storage. The data group definition parameter for database apply
processing determines how frequently to force data to disk storage. When the
apply sessions are restarted, MIMIX may attempt to rewrite records to the target
system database.
• Unable to write/update a record: This error is caused when MIMIX cannot
access a record in a file. This is usually caused when there are problems with the
logical files associated with the file or when the record does not exist. The best
way to prevent this error is to make sure that replication is started in the correct
position. This error can also be due to one of the problems listed in topic
“Correcting file-level errors” on page 269.
• Unable to delete a record: This is caused when MIMIX is trying to delete a
record that does not exist or has a corrupted logical file associated with the
physical file. This error can also be due to one of the problems listed in topic
“Correcting file-level errors” on page 269.
271
Working with tracking entries
Tracking entries identify library-based objects (data areas and data queues) and IFS
objects configured for cooperative processing (advanced journaling).
You can access the following displays to work with tracking entries in any status:
• Work with DG IFS Trk. Entries display (WRKDGIFSTE command)
• Work with DG Obj. Trk. Entries display (WRKDGOBJTE command)
These displays provide access for viewing status and working with common problems
that can occur while replicating objects identified by IFS and object tracking entries.
Held tracking entries: Status for the replicated objects is reported on the associated
tracking entries. If a journal transaction is not replicated successfully, the tracking
entry is placed in *HLDERR status. This indicates a problem that must be resolved.
Tracking entries can also be placed in *HLD, *HLDIGN statuses by user action These
statuses are reported as ‘held for other reasons’ and also require user action.
When a tracking entry has a status of *HLD or *HLDERR, MIMIX retains log spaces
so that journal entries that are being held can be released and applied to the target
system. Tracking entries should not be left in these states for an extended period.
Additional information: To determine if a data group has any IFS objects, data
areas, or data queues configured for advanced journaling, see “Determining if non-file
objects are configured for user journal replication” on page 336.
When working with tracking entries, especially for IFS objects, you should be aware of
the information provided in “Displaying long object names” on page 330.
Table 50. Tracking entry options on the Work with Data Groups display
50=IFS trk entries Lists all IFS tracking entries for the selected data group on the
Work with DG IFS Trk. Entries display.
51=IFS trk entries Lists IFS tracking entries for the selected data group with inactive
not active status values (*HLD, *HLDERR, *HLDIGN, *HLDRNM, and
*RLSWAIT) on the Work with DG IFS Trk. Entries display.
272
Working with tracking entries
Table 50. Tracking entry options on the Work with Data Groups display
52=Obj trk entries Lists all object tracking entries for the selected data group on the
Work with DG Obj. Trk. Entries display.
53=Obj trk entries Lists object tracking entries for the selected data group with
not active inactive status values (*HLD, *HLDERR, *HLDIGN, and
*RLSWAIT) on the Work with DG Obj. Trk. Entries display.
4. The tracking entry display you selected appears. Significant capability is available
for addressing common replication problems and journaling problems. Do the
following:
a. Use F10 to toggle between views showing status, journaling status, and the
database apply session in use.
b. Any entry with a status of *HLD, *HLDERR or *HLDIGN indicates that action is
required. The identified object remains in this state until action is taken.
Statuses of *HLD and *HLDERR result in journal entries being held but not
applied. Use Table 51 to identify choices based on the tracking entry status.
c. Use options identified in Table 52 to address journaling problems or replication
problems.
*ACTIVE Unless an error has occurred, no action is necessary. Entries in the user
journal for the IFS object are replicated and applied. If necessary, any of
the options to hold journal entries can be used.
*HLD User action is required to release the entry (option 26) so that held
journal entries from user journal can be applied to the target system.
*HLDERR User action is required. Attempt to resolve the error by synchronizing the
file (option 16).
*HLDIGN User action is required to either synchronize the object (option 16) or to
change the configuration if you no longer want to replicate the object.
Journal entries for the object are discarded. Replication is not occurring
and the object may not be synchronized.
Depending on the circumstances, Release may also be an option.
*HLDRNM This is a transitional state for IFS tracking entries that should resolve to
*ACTIVE. If this status persists, check the journaling status for the entry.
Object tracking entries cannot have this status.
*RLSWAIT If the status does not change to *ACTIVE, you may need to synchronize
(option 16)
1. Evaluate the cause of the problem before taking any action.
273
Table 52. Options for working with tracking entries
5=Display Identifies an object, its replication status, journaling status, and the
database apply session used.
9=Start journaling See “Starting journaling for IFS objects” on page 294 and “Starting
journaling for data areas and data queues” on page 298.
10=End journaling See “Ending journaling for IFS objects” on page 295 and “Ending
journaling for data areas and data queues” on page 299.
11=Verify journaling See “Verifying journaling for IFS objects” on page 296 and
“Verifying journaling for data areas and data queues” on page 300.
25=Release wait See “Waiting to synchronize and release held journal entries for a
tracking entry” on page 275.
27=Release clear See “Releasing and clearing held journal entries for a tracking
entry” on page 276.
35=Work with related Displays IFS tracking entries that have the same file identifier
objects (FID) as the specified tracking entry. See “Related tracking entries”
on page 277.
274
Working with tracking entries
journal entries will be applied. The *HLD status remains until additional action is
taken.
Do the following:
1. Access the IFS or object tracking entry display as described in “Accessing the
appropriate tracking entry display” on page 272.
2. Type 23 (Hold) next to the tracking entry for the object you want and press Enter.
275
press Enter.
276
Working with tracking entries
2. Type 4 (Remove) next to the tracking entry you want to remove and press Enter.
Note: Related tracking entries (entries with the same file identifier) will also be
removed when a tracking entry is removed.
3. You will see a confirmation display. To remove the tracking entry, press Enter.
277
Working with objects in error
Use this topic to work with replication errors for objects replicated through the system
journal.
To access a list of objects in error for a data group, do the following:
1. From the MIMIX Basic Main Menu select option 6 (Work with Data Groups) and
press Enter.
2. The Work with Data Groups display appears. Type 13 (Objects in error) next to
the data group you want which has values shown in the Obj Errors column and
press Enter.
3. The Work with Data Group Activity display appears with a list of the objects in
error for the data group you selected. You can do any of the following:
• Use F10 (Error view) to see the reason why the object is in error.
• Use F11 to change between views for objects, DLOs, IFS objects, and spooled
files.
• Use the options identified in Table 53 to resolve the errors. Type the number of
the option you want next to the object and press Enter
Table 53. Options on the Work with Data Group Activity display for working with objects in
error.
7=Display message Use this option to display any error message that is associated
with the entry.
8=Retry Use this option to retry the data group activity. MIMIX changes the
entry status to pending and attempts the failed operation again.
Note: It is possible to schedule the request for a time when the retry is
more likely to be successful. For more information about retrying
failed entries, see “Retrying data group activity entries” on
page 281.
278
Working with objects in error
Table 53. Options on the Work with Data Group Activity display for working with objects in
error.
12=Work with entries Use this option to access the Work with DG Activity Entries
display. From the display you can display additional information
about replicated journal transactions for the object, including the
journal entry type and access type (if available), as well as see
whether the object is undergoing delay retry processing. You can
also take options to display related entries, view error messages
for a failure, and synchronize the object. For more information,
see “Using the Work with DG Activity Entries display” on
page 279.
14=Remove related Use this option to remove an entry with a status of *FAILED and
any related entries that have a status of *DELAYED. You may
need to take action to synchronize the object associated with the
entry.
Table 54. Options available on the Work with DG Activity Entries display.
5=Display Use this option to display details about the individual entry. The
information available about the object includes whether the object
is undergoing delay retry processing, and journal entry
information, including access type information for T-SF, T-YC, and
T-ZC journal entry types. The subtype is shown for U-MX journal
entries. For more information, see “Determining whether an
activity entry is in a delay/retry cycle” on page 282
7=Display message Use this option to display the error message associated with the
processing failure for the entry.
8=Retry Use this option to retry the data group activity entry. MIMIX
changes the entry status to pending and attempts the failed
operation again as soon as possible.
279
Table 54. Options available on the Work with DG Activity Entries display.
9=Display related Displays entries related to the specified object. For example, use
this option to see entries associated with a move or rename
operation for the object.
12=Display job Displays the job that was processing the object when the error
occurred, if the still job information exists and is on this system.
280
Retrying data group activity entries
Data group activity entries that did not successfully complete replication have a status
of *FAILED. These failed data group activity entries are also called error entries. You
can request to retry processing for these activity entries.
Activity entries with a status of *ACTIVE can also be retried in some circumstances.
For example, you may want to retry an entry that is delayed but which has no
preceding pending activity entry. Or, you may want to retry a pending entry that is
undergoing processing in a delay retry cycle.
The retry request places the activity entry in the queue for processing by the system
journal replication process where the failure or delay occurred. Activity entries with a
status of *FAILED or *DELAYED are set to *PENDING until they are processed.
281
using the settings in effect from the Number of third delay/retries and Third retry
interval (min.) policies. These policies can be set for the installation or for a specific
data group.
282
Working with message queues
283
Working with the message log
The MIMIX message log provides a common location for you to see all messages
related to MIMIX products. A consolidated list of messages for all systems in the
installation library is available on the management system.
Note: The target system only shows messages that occurred on the target system.
LVI messages are informational messages and LVE messages are error or diagnostic
messages. CPF messages are generated by an underlying operating function and
may be passed up to the MIMIX product.
Do the following to access the MIMIX message log:
1. Do one of the following to access the message log display:
• From the MIMIX Basic Main Menu, select option 13 (Work with messages) and
press Enter.
• From the MIMIX Intermediate Main Menu, select option 3 (Work with
messages) and press Enter.
2. The Work with Message Log appears with a list of the current messages. The
initial view shows the message ID and text.
3. Press F11 to see additional views showing the message type, severity, the
product and process from which it originated, whether it is associated with a group
(for MIMIX, a data group), and the system on which it originated.
4. You can subset the messages shown on the display. A variety of subsetting
options are available that allow you to manage the message log more efficiently.
5. To work with a message, type the number of the option you want and press Enter.
The following options are available:
• 4=Remove - Use this option if you want to delete a message. When you select
this option, a confirmation display appears. Verify that you want to delete the
messages shown and press Enter. The message is deleted only from the local
system.
• 5=Display message - Use this option to view the full text of the first level
message and gain access to the second level text.
• 6=Print - Use this option to print the information for the message.
• 8=Display details - Use this option to display details for a message log entry
including its from and to program information, job information, group
information, product, process, originating system, and call stack information.
• 9=Related messages - Use this option to display a list of messages that relate
to the selected message. Related messages include a summary and any detail
messages immediately preceding it. This can be helpful when you have a large
message log list and you want to show the messages for a certain job.
• 12=Display job - If job information exists on the system, you can use this option
to access job information for a message log entry. The Work with Jobs display
appears from which you can select options for displaying specific information
about the job.
284
Removing data group activity history entries
285
Starting, ending, and verifying journaling
This chapter describes procedures for starting and ending journaling. Journaling must
be active on all files, IFS objects, data areas and data queues that you want to
replicate through a user journal. Normally, journaling is started during configuration.
However, there are times when you may need to start or end journaling on items
identified to a data group.
The topics in this chapter include:
• “What objects need to be journaled” on page 287 describes, for supported
configuration scenarios, what types of objects must have journaling started before
replication can occur. It also describes when journaling is started implicitly, as well
as the authority requirements necessary for user profiles that create the objects to
be journaled when they are created.
• “What objects need to be journaled” on page 287 describes, for supported
configuration scenarios, what types of objects must have journaling started before
replication can occur. It also describes when journaling is started implicitly, as well
as the authority requirements necessary for user profiles that create the objects to
be journaled when they are created.
• “MIMIX commands for starting journaling” on page 289 identifies the MIMIX
commands available for starting journaling and describes the checking performed
by the commands. It also includes information for specifying journaling to the
configured journal.
• “Journaling for physical files” on page 291 includes procedures for displaying
journaling status, starting journaling, ending journaling, and verifying journaling for
physical files identified by data group file entries.
• “Journaling for IFS objects” on page 294 includes procedures for displaying
journaling status, starting journaling, ending journaling, and verifying journaling for
IFS objects replicated cooperatively (advanced journaling). IFS tracking entries
are used in these procedures.
• “Journaling for data areas and data queues” on page 298 includes procedures for
displaying journaling status, starting journaling, ending journaling, and verifying
journaling for data area and data queue objects replicated cooperatively
(advanced journaling). IFS tracking entries are used in these procedures.
286
What objects need to be journaled
287
TABLE statement is automatically journaled if the library in which it is created
contains a journal named QSQJRN or if the library is journaled with appropriate
inherit rules.
• New *FILE, *DTAARA, *DTAQ objects - The default value (*DFT) for the Journal at
creation (JRNATCRT) parameter in the data group definition enables MIMIX to
automatically start journaling for physical files, data areas, and data queues when
they are created.
– On systems running IBM i 6.1 or higher releases, MIMIX uses the support
provided by the IBM i command Start Journal Library (STRJRNLIB).
Customers are advised not to re-create the QDFTJRN data area on systems
running IBM i 6.1 or higher.
When configuration requirements are met, MIMIX will start library journaling for
the appropriate libraries as well as enable automatic journaling for the configured
cooperatively processed object types. When journal at creation configuration
requirements are met, all new objects of that type are journaled, not just those
which are eligible for replication.
When the data group is started, MIMIX evaluates all data group object entries for
each object type. (Entries for *FILE objects are only evaluated when the data
group specifies COOPJRN(*USRJRN).) Entries properly configured to allow
cooperative processing of the object type determine whether MIMIX will enforce
library journaling. MIMIX uses the data group entry with the most specific match to
the object type and library that also specifies *ALL for its System 1 object (OBJ1)
and Attribute (OBJATR).
Note: MIMIX prevents library journaling from starting in the following libraries:
QSYS*, QRECOVERY, QRCY*, QUSR*, QSPL*, QRPL*, QRCL*, QRPL*,
QGPL, QTEMP and SYSIB*.
For example, if MIMIX finds only the following data group object entries for library
MYLIB, it would use the first entry when determining whether to enforce library
journaling because it is the most specific entry that also meets the OBJ1(*ALL)
and OBJATR(*ALL) requirements. The second entry is not considered in the
determination because its OBJ1 and OBJATR values do not meet these
requirements.
LIB1(MYLIB) OBJ1(*ALL) OBJTYPE(*FILE) OBJATR(*ALL) COOPDB(*YES)
PRCTYPE(*INCLD)
LIB1(MYLIB) OBJ1(MYAPP) OBJTYPE(*FILE) OBJATR(DSPF) COOPDB(*YES)
PRCTYPE(*INCLD)
288
MIMIX commands for starting journaling
289
Forcing objects to use the configured journal
Journaled objects must use the journal defined in the data group definition in order for
replication to occur. Objects that are journaled to a journal that is different than the
journal defined in the data group definition can result in data integrity issues. MIMIX
identifies these objects with a journaling status of *DIFFJRN. A journal status of
*DIFFJRN should be investigated to determine the reason for using the different
journal. See “Resolving a problem for a journal status of *DIFFJRN” on page 143.
The MIMIX commands STRJRNFE, STRJRNIFSE, and STRJRNOBJE provide the
ability to specify the Force to configured journal (FORCE) prompt which determines
whether to end journaling for the selected objects that are currently journaled to a
different journal than the configured journal (*DIFFJRN), and then start journaling to
the configured journal.
To force journaled objects to use the journal configured in the data group definition,
specify *YES for the FORCE prompt in the MIMIX commands for starting journaling.
FORCE(*NO) is the command default for the STRJRNFE, STRJRNIFSE and
STRJRNOBJE commands when run from the native interface. See “MIMIX
commands for starting journaling” on page 289.
290
Journaling for physical files
291
• To start journaling using the command defaults, press Enter.
• To modify command defaults, press F4 (Prompt) then continue with the next
step.
3. The Start Journaling File Entries (STRJRNFE) display appears. The Data group
definition and the System 1 file identify your selection.
4. Specify the value you want for the Start journaling on system prompt. Press F4 to
see a list of valid values.
If journaling is started on the source system, a journal entry will be generated into
the user journal. As a result, replication processes will start journaling for these
objects on the target system if the data group definition specifies *YES for Journal
on target (JRNTGT).
5. If you want to use batch processing, specify *YES for the Submit to batch prompt.
6. Optional: If the file is journaled to a journal that is different than the configured
journal and you have determined that it is ok to force journaling to the configured
journal, press F10 (Additional parameters) to display the Force to configured
journal (FORCE) prompt.
Change the FORCE prompt to *YES to end journaling to a journal that is different
than the configured journal and then start journaling using the configured journal.
This value will also attempt to start journaling for objects not currently journaled.
Journaling will not be ended for objects already journaled to the configured
journal. For more information, see “Forcing objects to use the configured journal”
on page 290.
7. To start journaling for the physical file associated with the selected data group,
press Enter.
The system returns a message to confirm the operation was successful.
292
Journaling for physical files
3. The End Journaling File Entries (ENDJRNFE) display appears. If you want to end
journaling for all files in the library, specify *ALL for the System 1 file prompt.
4. Specify the value you want for the End journaling on system prompt. Press F4 to
see a list of valid values.
If journaling is ended on the source system, a journal entry will be generated into
the user journal. As a result, replication processes will end journaling for these
objects on the target system if the data group definition specifies *YES for Journal
on target (JRNTGT).
5. If you want to use batch processing, specify *YES for the Submit to batch prompt.
6. To end journaling, press Enter.
293
Journaling for IFS objects
IFS tracking entries are loaded for a data group after they are configured for
replication through the user journal and the data group has been started. However,
loading IFS tracking entries does not automatically start journaling on the IFS objects
they identify. In order for replication to occur, journaling must be started on the source
system for the IFS objects identified by IFS tracking entries using the journal defined
in the data group definition.
This topic includes procedures to display journaling status, and to start, end, or verify
journaling for IFS objects identified for replication through the user journal.
You should be aware of the information in “Considerations for working with long IFS
path names” on page 330.
294
Journaling for IFS objects
1. When the command is invoked from a command line, you can change values specified for
the IFS objects prompts. Also, you can specify as many as 300 object selectors by using the
+ for more values prompt.
2. When the command is invoked from a command line, use F10 to see the FID prompts. Then
you can optionally specify the unique FID for the IFS object on either system. The FID values
can be used alone or in combination with the IFS object path name.
295
To end journaling for IFS objects, do the following:
1. Access the journaled view of the Work with DG IFS Trk. Entries display as
described in “Displaying journaling status for IFS objects” on page 294.
2. From the Work with DG IFS Trk. Entries display, type a 10 (End journaling) next to
the IFS tracking entries you want. Then do one of the following:
• To end journaling using the command defaults, press Enter.
• To modify the command defaults, press F4 (Prompt) and continue with the
next step.
3. The End Journaling IFS Entries (ENDJRNIFSE) display appears. The Data group
definition and IFS objects prompts identify the IFS object associated with the
tracking entry you selected. You cannot change the values shown for the IFS
objects prompts1.
4. Specify the value you want for the End journaling on system prompt. Press F4 to
see a list of valid values.
If journaling is ended on the source system, a journal entry will be generated into
the user journal. As a result, replication processes will end journaling for these
objects on the target system if the data group definition specifies *YES for Journal
on target (JRNTGT).
5. To use batch processing, specify *YES for the Submit to batch prompt and press
Enter. Additional prompts for Job description and Job name appear. Either accept
the default values or specify other values.
6. The System 1 file identifier and System 2 file identifier identify the file identifier
(FID) of the IFS object on each system. You cannot change the values shown2.
7. To end journaling on the IFS objects specified, press Enter.
296
Journaling for IFS objects
group definition and IFS objects prompts identify the IFS object associated with
the tracking entry you selected. You cannot change the values shown for the IFS
objects prompts1.
4. Specify the value you want for the Verify journaling on system prompt. Press F4 to
see a list of valid values.
When *DGDFN is specified, MIMIX considers whether the data group is
configured for journaling on the target system (JRNTGT) and verifies journaling on
the appropriate systems as required.
5. To use batch processing, specify *YES for the Submit to batch prompt and press
Enter. Additional prompts for Job description and Job name appear. Either accept
the default values or specify other values.
6. The System 1 file identifier and System 2 file identifier identify the file identifier
(FID) of the IFS object on each system. You cannot change the values shown2.
7. To verify journaling on the IFS objects specified, press Enter.
“Using file identifiers (FIDs) for IFS objects” on page 338.
297
Journaling for data areas and data queues
Object tracking entries are loaded for a data group after they are configured for
replication through the user journal and the data group has been started. However,
loading object tracking entries does not automatically start journaling on the objects
they identify. In order for replication to occur, journaling must be started on the source
system for objects identified by tracking entries that use the journal defined in the data
group definition.
This topic includes procedures to display journaling status, and to start, end, or verify
journaling for data areas and data queues identified for replication through the user
journal.
298
Journaling for data areas and data queues
299
• To modify the command defaults, press F4 (Prompt) and continue with the next
step.
3. The End Journaling Obj Entries (ENDJRNOBJE) display appears. The Data group
definition and IFS objects prompts identify the object associated with the tracking
entry you selected. Although you can change the values shown for these prompts,
it is not recommended unless the command was invoked from a command line.
4. Specify the value you want for the End journaling on system prompt. Press F4 to
see a list of valid values.
If journaling is ended on the source system, a journal entry will be generated into
the user journal. As a result, replication processes will end journaling for these
objects on the target system if the data group definition specifies *YES for Journal
on target (JRNTGT).
5. To use batch processing, specify *YES for the Submit to batch prompt and press
Enter. Additional prompts for Job description and Job name appear. Either accept
the default values or specify other values.
6. To end journaling on the objects specified, press Enter.
300
Journaling for data areas and data queues
5. To use batch processing, specify *YES for the Submit to batch prompt and press
Enter. Additional prompts for Job description and Job name appear. Either accept
the default values or specify other values.
6. To verify journaling on the objects specified, press Enter.
301
Switching
CHAPTER 14 Switching
Switching is when you temporarily change the roles of the systems. The original
source system (production) becomes the temporary target system and the original
target system (backup) becomes the temporary source system. When the scenario
that required you to switch directions is resolved, you typically switch again to return
the systems to their original roles.
This chapter provides information and procedures to support switching. The following
topics are included:
• “About switching” on page 303 provides information about switching with MIMIX
including best practice and reasons why a switch should be performed. Subtopics
describe:
– What is a planned switch and requirements for a planned switch
– What is an unplanned switch and actions to be completed after the failed
source system is recovered
– The role of procedures for switching environments that use application groups
– The role of MIMIX Model Switch Framework for switching environments that do
not use application groups
• “Switching an application group” on page 308 describes how to run a procedure to
switch an application group.
• “Switching a data group-only environment” on page 309. describes how to switch
from the native user interface.
• “Determining when the last switch was performed” on page 311 describes how to
check the Last switch field which indicates the switch compliance status and
provides the date when the last switch was performed.
• “Performing a data group switch” on page 312 describes how to switch a single
data group using the SWTDG command.
• “Switch Data Group (SWTDG) command” on page 314 provides background
information about the SWTDG command, which is used in all switch interfaces.
In environments licensed for MIMIX DR, only virtual switching1 is supported.
Virtual switching does not reverse the roles of the systems and is not covered in this
chapter. For detailed information about virtual switching, see “Support for virtual
switching” on page 316.
302
About switching
About switching
Replication environments rarely remain static. Therefore, best practice is to perform
regular switches to ensure that you are prepared should you need to perform one
during an emergency.
MIMIX supports two methods for switching the direction in which replication occurs for
a data group. These methods are known as a planned switch and an unplanned
switch.
You may need to perform a switch for any of the following reasons:
• The production system becomes unavailable due to an unplanned outage. A
switch in this scenario is unplanned.
• You need to perform hardware or software maintenance on the production
system. Typically, you can schedule this in advance so the switch is planned.
• You need to test your recovery plan. This activity is also a planned switch. Virtual
switching can also be used to test applications prior to performing a planned
switch test of your recovery plan. Virtual switching is described in “Support for
virtual switching” on page 316.
Historically, the concept of switching consists of three phases: switch to the backup
system, synchronize the systems when the production is ready to use, and switch
back to the production system. This round-trip view of switching assumes your goal is
to return to your original production system as quickly as possible. However, some
customers may have an extended time pass between phase one and the other
phases, or may even view a switch as a one-way trip. MIMIX supports both
conceptual views of switching.
Switching data groups is only a part of performing a switch. MIMIX provides robust
support for customizing switching activity include all the needs of your environment.
Best practice for switching includes performing regular switches. Best practice also
includes performing all audits with the audit level set at level 30 immediately prior to a
planned switch to the backup system and before switching back to the production
system. Best practice is to use Vision Solutions Portal to perform switch operations. If
you use the native user interface, best practice is:
• For application groups, use option 4 (Switch all application groups) from the
MIMIX Basic Main Menu.
• For data group-only environments, use option 5 (start or complete switch using
Switch Asst.) from the MIMIX Basic Main Menu.
303
Switching
• Ensure configuration changes are deployed. You can have MIMIX automatically
deploy configuration changes by ending and starting data group replication, or
you can manually deploy data group configuration within MIMIX. For more
information, see the MIMIX Administrator Reference book.
• After you have ensured configuration changes are deployed, perform a full set of
audits with the audit level policy set to level 30.
• Shut down any applications that use database files or objects defined to the data
group. If any users or other non-MIMIX processes remain active while the switch
is being performed, the data can become not synchronized between systems and
orphaned data may result.
• Ensure that there are no jobs other than MIMIX currently active on the source
system. This may require ending all interactive and batch subsystems other than
MIMIX and ending communications.
• Users should be prevented from accessing either system until after the switch is
complete and the data group is restarted.
• If you use user journal replication processes, you should address any files, IFS
tracking entries, or object tracking entries in error for your critical database files. If
you use system journal replication processes, you should address any object
errors.
You are not required to run journal analysis after a planned switch. MIMIX retains
information about where activity ended so that when you restart the data group, it is
started at the correct point.
When the data group is started, the temporary target system (the production system)
is now being updated with user changes that are being replicated from the temporary
source system (the backup system). Do not allow users onto the production system
until after the production system is caught up with these transactions and you run the
switch process again to revert to the normal roles.
304
About switching
relies on status information on the source system about the last entry that was
applied. This information will be cleared when the data group is restarted.
• Communication between the systems must be active before you restart the data
group. The switch process is complete when you restart the data group. When
the data group is restarted, MIMIX notifies the source system that it is now the
temporary target system.
• New transactions are created on the temporary source system (the backup
system) while the production system (the temporary target system) is unavailable
for replication. After you have completed journal analysis, you can send these
new transactions to the production system to synchronize the databases. Once
the databases are synchronized, you must run the switch process again to revert
to the normal roles before allowing users onto the production system.
When the data group is started after a switch, any pending transactions are cleared.
The journal receiver is already changed by the switch process and the new journal
receiver and first sequence number are used.
1. Procedures are used for switching *CLU and *NONCLU application groups. Application
groups of type *NONCLU do not participate in a cluster controlled by the IBM i operating sys-
tem.
305
Switching
during the switch differ based on whether the current primary node (data source)
is available at the start of the switch procedure. The default value, *PLANNED,
indicates that the primary node is still available and the switch is being performed
for normal business processes (such as to perform maintenance on the current
source system or as part of a standard switch procedure). The value
*UNPLANNED indicates that the switch is an unplanned activity and the data
source system may not be available. The value *VIRTUAL indicates that the
application group environment will be modified to allow application testing to be
performed on a backup node in a process known as a virtual switch.
Node roles (ROLE) - This specifies which set of node roles will be used during the
switch. For a planned or unplanned switch, the specified value determines the
node that becomes the new primary node as a result of the switch. The default
value *CURRENT uses the current order of node roles. If the application group
participates in a cluster, the current roles defined within the CRGs will be used. If
*CONFIG is specified, the configured primary node will become the new primary
node and the new role of other nodes in the recovery domain will be determined
from their current roles. If you specify a name of a node within the recovery
domain for the application group, the node will be made the new primary node and
the new role of other nodes in the recovery domain will be determined from their
current roles. For a virtual switch, the value of this parameter determines the node
to be used for testing applications and does not cause changes to recovery
domain roles. In a virtual switch, the first backup node in the recovery domain is
used unless you specify the name of a backup node.
What procedure will be used? The following parameters identify the procedure to
use and its starting point:
Begin at step (STEP) - Specifies where the request will start within the specified
procedure. This parameter is described in detail below.
Procedure (PROC) - Specifies the name of the procedure to run to perform the
requested operation when starting from its first step. The value *DFT will use the
procedure designated as the default for the application group. The value
*LASTRUN uses the same procedure used for the previous run of the command.
You can also specify the name of a procedure that is valid the specified
application group and type of request.
Where should the procedure begin? The value specified for the Begin at step
(STEP) parameter on the request to run the procedure determines the step at which
the procedure will start. The status of the last run of the procedure determines which
values are valid.
The default value, *FIRST, will start the specified procedure at its first step. This value
can be used when the procedure has never been run, when its previous run
completed (*COMPLETED or *COMPERR), or when a user acknowledged the status
of its previous run which failed, was canceled, or completed with errors
(*ACKFAILED, *ACKCANCEL, or *ACKERR respectively).
Other values are for resolving problems with a failed or canceled procedure. When a
procedure fails or is canceled, subsequent attempts to run the same procedure will
fail until user action is taken. You will need to determine the best course of action for
306
About switching
your environment based on the implications of the canceled or failed steps and any
steps which completed.
The value *RESUME will start the last run of the procedure beginning with the step at
which it failed, the step that was canceled in response to an error, or the step
following where the procedure was canceled. The value *RESUME may be
appropriate after you have investigated and resolved the problem which caused the
procedure to end. Optionally, if the problem cannot be resolved and you want to
resume the procedure anyway, you can override the attributes of a step before
resuming the procedure.
The value *OVERRIDE will override the status of all runs of the specified procedure
that did not complete. The *FAILED or *CANCELED status of these procedures are
changed to acknowledged (*ACKFAILED or *ACKCANCEL) and a new run of the
procedure begins at the first step.
For more information about starting a procedure with the step at which it failed, see
“Resuming a procedure” on page 112.
For more information about customizing procedures, see the MIMIX Administrator
Reference book.
307
Switching an application group
These instructions can be used to perform a planned or unplanned switch of an
application group. An application group can have only one procedure that performs a
start, end, or switch operation running at a time.
Before you begin, review topic “About switching” on page 303 for details about
preparing for a planned switch, actions following an unplanned switch, and
implications of the parameters available on the Switch Application Group (SWTAG)
command.
To switch an application group, do the following:
1. From the Work with Application Groups display, type 15 (Switch) next to the
application group you want and press Enter.
The Switch Application Group (SWTAG) display appears.
2. Verify that the values you want are specified for Resource groups and Data
resource group entry.
3. At the Switch type prompt, specify the type of switch to perform.
• For a planned switch, specify *PLANNED.
• For an unplanned switch, specify *UNPLANNED.
4. Verify that the default value *CURRENT for Node roles prompt is valid for the
switch you need to perform. If necessary, specify a different value.
5. If you are starting the procedure after addressing problems with the previous
switch request, specify the value you want for Begin at step. Be certain that you
understand the effect the value you specify will have on your environment.
6. Press Enter.
7. The Procedure prompt appears. Do one of the following:
• To use the default switch procedure for the specified switch type, press Enter.
• To use a different switch procedure for the application group, specify its name.
Then press Enter.
8. A switch confirmation panel appears. To perform the switch, press F16.
308
Switching a data group-only environment
309
After you complete this phase of the switch you must wait until the original production
system is available again. Then perform the steps in “Synchronizing data and starting
MIMIX on the original production system” on page 310.
310
Determining when the last switch was performed
b. For each data group, select option 8 (Display status) and ensure that the
Unprocessed entry counts for both database and object apply have no values.
3. From the MIMIX Basic Main Menu, select option 5 (Start or complete switch using
Switch Asst.).
4. You will see the Confirm Switch to Production confirmation display. Press F16 to
confirm your choice to switch MIMIX and specify switching options.
5. The Run Switch Framework (RUNSWTFWK) command appears. The default
Switch framework and the value *PROD for the Switch framework process are
preselected and cannot be changed. Do the following:
a. You can change values for other parameters as needed.
b. To start the switch, press Enter.
6. Consult your runbook to determine if any additional steps are needed.
311
Performing a data group switch
Performing a data group switch changes the direction of replication for a data group
through the Switch Data Group (SWTDG) command. Only replication for the selected
data group is switched. You may want to perform a data group switch if you are
having problems with an application that only affects a specific data group or if you
need to manually load balance because of heavily used applications.
Note: You cannot switch a disabled data group. For more information, see
“Disabling and enabling data groups” on page 334.
To perform a data group switch, do the following:
1. If you will be performing a planned switch, do the following:
a. Shut down any applications that have database file or objects defined to the
data group.
b. Ensure that you have addressed any critical database files that are held due to
error or held for other reasons.
c. Ensure there are no pending object activity entries by entering: WRKDGACTE
STATUS(*ACTIVE)
2. From the Work with Data Groups display, type the option for the type of switch you
want next to the data group you want to switch and press Enter.
• Use option 15 for a planned switch
• Use option 16 for an unplanned switch
3. Some of the parameter values that you may want to consider when the Switch
Data Group display appears are:
• If you specified Switch type of *PLANNED and have specified a number for the
Wait time (seconds) parameter, you can specify a value for the Timeout Option
parameter to specify what action you want the SWTDG command to perform if
the time specified in the Wait time (seconds) parameter is exceeded. When
you are performing a planned switch you may want to specify the number of
seconds to wait before all the active data group processes end. If you specify
*NOMAX the switch process will wait until all data group processes are ended.
This could delay the switch process.
• You can use the Conditions that end switch parameter to specify the types of
errors that you want to end the switch operation. For a planned switch, the
default value, *DFT, checks all possible values except *RCY and converts any
new or in-progress recoveries found to replication errors. For an unplanned
switch, *DFT will prevent the switch only when database apply backlogs exist.
• Verify that the value for the Start journaling on new source prompt is what you
want. If necessary, change the value.
4. After the confirmation screen, press F16 to continue.
5. Press Enter. Messages appear indicating the status of the switch request. When
you see a message indicating that the switch is complete, users can begin
processing as usual on the temporary source system.
312
Performing a data group switch
313
Switch Data Group (SWTDG) command
The Switch Data Group (SWTDG) command provides the following parameters to
control how you want your switch operation handled:
• The Wait time (seconds) parameter (WAIT) is used to specify the number of
seconds to wait for all of the active data group processes to end. The function of
the default value *DFT is different for planned switches than it is for unplanned
switches. For a planned switch, the value *DFT is equivalent to the value
*NOMAX. For an unplanned switch, the value *DFT is set to wait 300 seconds (5
minutes) for all of the active data group processes to end.
• If you specify a value for the WAIT parameter you can use the Timeout option
parameter (TIMOUTOPT) to specify what action to take when the wait time you
specified is reached. The function of the default value *DFT is different for planned
switches than it is for unplanned switches. For a planned switch, the value *DFT is
equivalent to the value *QUIT. When the value specified for the WAIT parameter is
reached, the current process quits and returns control to the caller. For an
unplanned switch, the value *DFT is equivalent to the value *NOTIFY. When the
value specified for the WAIT parameter is reached, an inquiry message is sent to
notify the operator of a possible error condition.
• The Conditions that end switch (ENDSWT) parameter is used to specify which
conditions should end the switch process. The function of the default value *DFT
is different for planned switches than it is for unplanned switches.
– For a planned switch, the value *DFT checks all conditions except *RCY. If any
new or in-progress recoveries exist when *DFT is used, they are converted to
held file entries, held tracking entries, or failed object activity entries before
ending the process. The value *ALL provides the most comprehensive
checking for conditions that are not compatible with best practices for
switching. Additionally, the value *ALL ensures that your programs will
automatically include any future ENDSWT parameter values that may be
added to maintain a conservative approach to the switching operation.
– For an unplanned switch, the value *DFT ends the process if there are any
backlogs for the database apply process. However, backlogs on other user
journal processes are not checked and switch processing is not ended even
though conditions may exist which are not compatible with best practices for
switching and may result in the loss of data.
• The Start journaling on new source (STRJRNSRC) parameter is used to specify
whether you want to start journaling for the data group on the new source system.
• The End journaling on new target (ENDJRNTGT) parameter is used to specify
whether you want to end journaling of the data group on the new target system.
• The End remote journaling (ENDRJLNK) parameter is used in a planned switch of
a data group that uses remote journaling. This parameter specifies whether you
want to end remote journaling for the data group. The default behavior is to leave
the RJ link running. You need to consider whether to keep the RJ link active after
a planned switch of a data group. For more information, see “Before ending
replication” on page 232.
314
Switch Data Group (SWTDG) command
315
Support for virtual switching
This chapter describes what is virtual switching, the purpose and activity that occur in
each phase of the virtual switch procedure, application testing considerations, as well
as requirements and considerations for your MIMIX environment.
• “What is virtual switching?” on page 316
• “MIMIX virtual switch overview” on page 317
• “MIMIX environment considerations for virtual switching” on page 319
• “Testing considerations for virtual switching” on page 321
• “Performing a precheck for virtual switching” on page 323
• “Performing a virtual switch” on page 323
316
MIMIX virtual switch overview
to perform a planned switch test. Some applications are not suitable for virtual
switching. Also, a virtual switch cannot test some aspects of your environment, such
as IP impersonation, and does not allow you to validate that the MIMIX configuration
operates as expected from the test node.
1. The PRECHKVRT and SWTVRT procedures are added to existing applications groups when
the instance is upgraded to MIMIX version 8.1 or higher.
317
Support for virtual switching
switch recovery. The remaining steps return the test node to its original purpose
as a backup node in the MIMIX production environment. During recovery, all
foreign key constraints are disabled on files on the test node that are eligible for
replication. Triggers are disabled on all target node files eligible for replication. If
necessary, files on the test node which had their journal image settings changed
during the setup phase are changed back to their original settings. Target node
processes are started beginning with the last processed entry for the receiver and
sequence number, and apply processes begin applying the queued transactions
that accumulated during testing. Constraints on the target node are enabled
unless the data group is configured to allow MIMIX to perform target constraint
management. Target journal inspection is changed to allow sending notifications
again. If there are any remaining objects that could not be recovered, they will be
reflected in the user interface as requiring manual correction through replication
activity entries, notifications, or both. Then replication statuses will no longer
reflect a virtual switch status. Finally, prioritized audits are submitted to verify the
objects changed on the target node during testing, these audits run regardless of
whether prioritized auditing is enabled.
For more information about steps within the SWTVRT procedure, see the MIMIX
Administrator Reference book.
Viewing recovery actions for target objects changed during a virtual switch
When using the MIMIX portal application in Vision Solutions Portal, you can use
actions available in the Application Groups portlet and Data Groups portlet to view the
following information:
318
MIMIX environment considerations for virtual switching
• Object Correction Activity window - lists objects for which recovery actions exist
and shows status of the recovery actions. The window can be filtered to those
only for a virtual switch. There may be multiple problems or changes for an object
but the window lists only one row per object per data group.
• From the Application Groups portlet, you can use the Virtual Switch Activity action
for an application group to view the list objects changed on the test node and their
recovery actions. After the recovery phase of the virtual switch begins, you will be
able to see any objects that recovery actions were unable to recover.
The ability tor view the objects changed during a virtual switch and the recovery
actions are not available in the native user interface.
319
Support for virtual switching
virtual switch activity that occurs during a virtual switch may cause DASD issues.
• For the duration of the virtual switch procedure, the virtual switch controls when
the apply processes and access path maintenance are ended and restarted. You
can start or stop other replication processes as needed on the production node.
• Virtual switching uses target journal inspection processes to identify objects that
have changed on the test node. Starting and stopping MIMIX also starts and stops
target journal inspection. Target journal inspection processes need to remain
active during virtual switch testing to avoid having a period of time where changes
made on the test node are not detected. Data groups affected by the virtual switch
can be started and stopped.
• Virtual switch status is triggered by specific step programs in the virtual switch
procedure and is reflected in the replication status of the application group and its
resource groups and data groups. At the application group level, replication status
will indicate statuses of virtual switch starting, virtual switch testing in progress,
and virtual switch recovery in progress when no other conditions that require
attention exist. While a virtual switch status is displayed, other less severe
conditions can exist and you will need to drill into more detailed interfaces to see
those problems. When using Vision Solutions Portal, an important difference in
the overall status column in the Application Groups portlet is that any virtual switch
status takes precedence over all other possible statuses so that you can easily
identify that a virtual switch is in progress. Detailed columns in that portlet will
reflect any other issues that need your attention.
• A failed or canceled virtual switch procedure can only be acknowledged if the
failure or cancel request occurred before step MXSETVRTS. If the procedure fails
or is canceled after the virtual switch starting status is displayed for replication,
you will not be able to start, stop, or switch the application group until after the
virtual switch procedure is resumed and runs to completion.
• In interfaces for starting a virtual switch, the default value for the test node is the
node identified as the first backup in the current node roles of the recovery
domain. If the application group has three or more nodes in its recovery domain,
you can select to use a different backup node. Replicate nodes cannot be
selected as the test node.
• In cascading environment, do not use the middle node of the cascade as the test
node for a virtual switch.
• Audits are not allowed to run during a virtual switch.
• Target journal inspection will not issue notifications while virtual switch testing is in
progress.
• If you subscribe to MIMIX events through Vision Solutions Portal, be aware that
some events will not be triggered during a virtual switch or when certain virtual
switch states occur. There are unique events that are triggered by virtual switch
status changes.For more information, see the list of MIMIX subscription events
available in online help.
320
Testing considerations for virtual switching
321
Support for virtual switching
configuration selection rules (data group entries) for those items so they are
replicated to the target node.
• Determine if the application needs communications access to the test node. This
may need to be set up at the start of the testing phase.
• If an application changes objects or creates data on other connected servers, the
cloud, or in other applications, you may need to temporarily disable those
capabilities to prevent generating data to those locations. If you cannot disable
this activity, any changes outside of the designated test node may cause
damaged objects and will require manual effort to revert. If disabling is not
possible, the application may not be a good candidate for virtual switch testing.
• If an application hard codes an IP address or system name, the application will not
function properly and will not find the address or system when started on the
virtual switch test node. These scenarios are issues in any switch and need to be
resolved. You may need to work with the application provider to resolve them.
• If your environment uses IP impersonation to change network names outside of
MIMIX as part of your normal planned switch processing, carefully evaluate how
applications interact with these changes to determine if virtual switching testing is
viable for the affected applications.
• Identify what actions you need to perform to start and end the application.
Optionally, you can enable and customize steps STRUSRAPPV and
ENDUSRAPPV with instructions to start and end the application on the test node
as part of the virtual switch procedure.
Testing in MIMIX for MQ environments: If your environment includes the MIMIX for
MQ feature and you perform a virtual switch of an application group that replicates an
IBM WebSphere MQ queue manager, the AMQAJRN journal should not be allowed to
change its journal receiver while in the application testing phase of the virtual switch.
Do not intentionally change this journal receiver during virtual switch testing. If the
receiver for this journal changes during virtual switch testing, a manual recovery
process is required after the virtual switch procedure has completed. You will need to
check for MQ-related topics in the Knowledgebase on Support Central and may need
to contact CustomerCare.
322
Performing a precheck for virtual switching
These instructions use the default Virtual Switch (DFT) procedure2 for a selected
application group from the native user interface. From this interface, you can start the
procedure and monitor its status, but you cannot view a list of objects changed by
testing on the target node or the status of recovery actions for those objects. Those
capabilities are available only from Vision Solutions Portal, which is recommended.
Starting the virtual switch procedure:
To start the virtual switch, do the following:
1. From the Work with Application Groups display, type 15 (Switch) next to the
application group you want and press Enter.
The Switch Application Group (SWTAG) display appears.
2. Specify *VIRTUAL for the Switch type prompt.
3. At the Node roles prompt, specify the node to be used for testing in the virtual
switch. (No changes to the recovery domain roles will occur.) The default value
*CURRENT will use the first backup node in the current node roles. This value is
appropriate for typical 2-node application groups. To use a different backup node
323
in an application groups with multiple backup nodes, specify its name.
4. Press Enter.
5. The Procedure prompt appears. Do one of the following:
• To use the default virtual switch procedure (SWTVRT), press Enter.
• To use a different virtual switch procedure for the application group, specify its
name.
6. Press Enter.
7. You can check status of the virtual switch procedure from the Work with
Procedures display. Refer to instructions in “Working with procedures and steps”
on page 97.
Important!
For problem resolution, a virtual switch procedure differs from other types of
procedures in these ways:
• A virtual switch procedure is intended to stay in *MSGW state at step
MXVRTRCY for the entire time that application tests are being performed on
the designated test node. Do not respond to the message for this step until
testing is complete and you are ready to start the recovery phase of the virtual
switch. For all other steps, handle a *MSGW procedure status like that for any
other procedure.
• If a problem occurs in any step within the range performed while the
application group has a replication status of *VRTSWTSTR, *VRTSWTTST or
*VRTSWTRCY and the virtual switch procedure fails or is canceled, you
cannot acknowledge the failed or canceled procedure. The virtual switch
procedure must be resumed after you have investigated the cause of the
problem for the affected step. Furthermore, you will not be able to run any
other procedure to start, stop, or switch the affected application group until the
virtual switch procedure is resumed and allowed to run to completion. The
range of steps to which this behavior applies begins with step MXSETVRTS
and applies through step MXFAILRCYV.
• If a problem with the source node occurs that requires an unplanned switch
while a virtual switch is in progress, the virtual switch procedure must be
allowed to complete. This is necessary even in environments with more than
one backup node in the application group’s recovery domain. If the virtual
switch is in its testing phase, end the application being tested and respond to
the message waiting at step MXVRTRCY. Allow the virtual switch procedure to
complete before attempting to start the unplanned switch. Any recovery
actions that require capturing data from the source node will fail and will have a
Not Recovered status in the Virtual Switch Activity window in Vision Solutions
Portal.
Testing phase: The MIMIX environment on the designated test node is ready for
application testing when the virtual switch procedure successfully processes step
MXSETVRTT. As a result of this step, the Work with Application Groups display
indicates that virtual switch testing is in progress with the value *VRTSWTTST in the
324
Performing a virtual switch
Repl. Status column if replication processes are active. (Apply processes and access
path maintenance remain ended while in the testing phase.)
Important!
While you are testing, the procedure is designed to wait for a user reply that
indicates testing is complete before it processes step MXVRTRCY. You will see
the status *MSGW on the Work with Procedure Status display for the duration of
the testing phase of the virtual switch. Do not respond to this message until
you have completed testing.
8. If you have not enabled and customized step STRUSRAPPV, you must manually
start the application to be tested on the designated test node.
9. Test the application.
While testing, it is good practice to keep a record of your test node activity that
may require manual cleanup when testing is complete. This includes any setup
that will need to be removed, any spool files created, and any changed objects
that are outside of the scope of replication for the participating data groups.
10. To see the objects eligible for replication that have been changed on the test
node, use the Virtual Switch Activity action for the application group from the
Application Groups portlet in Vision Solutions Portal.
11. For the duration of virtual switch testing, monitor the application group for potential
problems from the Work with Application Groups display. Check for any indication
of problems that require action and take action as needed.
12. When testing is complete, end the application on the test node.
Note: If you enabled step ENDUSRAPPV and customized it with actions to end
the application, you can skip this step. When enabled, the customized step
runs after you respond to the message waiting in Step 14.
13. Perform any necessary manual clean up on the test node, such as deleting any
unnecessary spooled files created during testing.
Recovery phase:
14. To indicate that testing is complete and you are ready for the virtual switch
procedure to start its recovery phase, you must respond to the waiting message.
Do the following:
a. From the Work with Application Groups display, type 21 (Procedure status)
next to the application group you want and press Enter.
b. From the Work with Procedure Status display. type 11 (Display message) next
to the virtual switch procedure (SWTVRT) and press Enter.
c. You will see the message “Procedure name for application group name
requires response. (G C).” To start the recovery phase of the virtual switch,
type G and press Enter.
Note: A response of C (Cancel) will cancel the procedure. If you cancel the
procedure, you must resume the procedure and allow it to run to
completion.
This starts the recovery phase of the virtual switch and changes the replication
325
status for the application group to indicate that virtual switch is recovering
(*VRTSWTRCY on the Work with Application Groups display). Apply processes
will be started and will begin processing the accumulated transactions and
recovery actions.
15. You can check the progress of recovery actions from the Virtual Switch Activity
window within Vision Solutions Portal.
16. When the procedure completes, check the Virtual Switch Activity window for any
objects with failed recoveries. You will need to take action to address these
objects.
Additional information about using the Virtual Switch Activity window to check
recovery status and start manual resolution of failed recoveries is available in online
help for a MIMIX instance from within Vision Solutions Portal.
326
CHAPTER 16 Less common operations
This chapter describes how to perform infrequently used operations that help keep
your MIMIX environment running. The following topics are included:
• “Starting the TCP/IP server” on page 328 contains the procedure for starting the
TCP/IP server.
• “Ending the TCP/IP server” on page 329 contains the procedure for ending the
TCP/IP server.
• “Working with objects” on page 330 contains tips for working with long object and
IFS path names.
• “Viewing status for active file operations” on page 331 describes how to check
status when replicating database files that you are reorganizing or copying with
MIMIX Promoter.
• “Identifying data groups that use an RJ link” on page 332 describes how to
determine which data groups use a remote journal link.
• “Identifying journal definitions used with RJ” on page 333 describes how to
determine whether a journal definition is defined to one or more remote journal
links.
• “Disabling and enabling data groups” on page 334 describes when it can be
beneficial to disable and enable data groups. Procedures for these processes are
included in this topic.
• “Determining if non-file objects are configured for user journal replication” on
page 336 provides procedures for determining whether configured for IFS objects,
data areas, and data queues are configured to be cooperatively processed
through the user journal.
• “Using file identifiers (FIDs) for IFS objects” on page 338 describes file identifiers
(FIDs) which are used by commands to uniquely identify the correct IFS tracking
entries to process.
327
Starting the TCP/IP server
Use this procedure if you need to manually start the TCP/IP server.
Once the TCP communication connections have been defined in a transfer definition,
the TCP server must be started on each of the systems identified by the transfer
definition.
You can also start the TCP/IP server automatically through an autostart job entry.
Either you can change the transfer definition to allow MIMIX to create and manage
the autostart job entry for the TCP/IP server, or you can add your own autostart job
entry. MIMIX only manages entries for the server when they are created by transfer
definitions.
When configuring a new installation, transfer definitions and MIMIX-added autostart
job entries do not exist on other systems until after the first time the MIMIX managers
are started. Therefore, during initial configuration you may need to manually start the
TCP server on the other systems using the STRSVR command.
Note: Use the host name and port number (or port alias) defined in the transfer
definition for the system on which you are running this command.
Do the following on the system on which you want to start the TCP server:
1. From the MIMIX Intermediate Main Menu, select option 13 (Utilities menu) and
press Enter.
2. The Utilities Menu appears. Select option 51 (Start TCP server) and press Enter.
3. The Start Lakeview TCP Server display appears. At the Host name or address
prompt, specify the host name or address for the local system as defined in the
transfer definition.
4. At the Port number or alias prompt, specify the port number or alias as defined in
the transfer definition for the local system.
Note: If you specify an alias, you must have an entry in the service table on this
system that equates the alias to the port number.
5. Press Enter.
6. Verify that the server job is running under the MIMIX subsystem on that system.
You can use the Work with Active Jobs (WRKACTJOB) command to look for a job
under the MIMIXSBS subsystem with a function of PGM-LVSERVER.
328
Ending the TCP/IP server
329
Working with objects
When working with objects, these tips may be helpful.
330
Viewing status for active file operations
331
Identifying data groups that use an RJ link
Use this procedure to determine which data groups use a remote journal link before
you end a remote journal link or remove a remote journaling environment.
1. Enter the command WRKRJLNK and press Enter.
2. Make a note of the name indicated in the Source Jrn Def column for the RJ Link
you want.
3. From the command line, type WRKDGDFN and press Enter.
4. For all data groups listed on the Work with DG Definitions display, check the
Journal Definition column for the name of the source journal definition you
recorded in Step 2.
• If you do not find the name from Step 2, the RJ link is not used by any data
group. The RJ link can be safely ended or can have its remote journaling
environment removed without affecting existing data groups.
• If you find the name from Step 2 associated with any data groups, those data
groups may be adversely affected if you end the RJ link. A request to remove
the remote journaling environment removes configuration elements and
system objects that need to be created again before the data group can be
used. Continue with the next step.
5. Press F10 (View RJ links). Consider the following and contact your MIMIX
administrator before taking action that will end the RJ link or remove the remote
journaling environment.
• When *NO appears in the Use RJ Link column, the data group will not be
affected by a request to end the RJ link or to end the remote journaling
environment.
Note: If you allow applications other than MIMIX to use the RJ link, they will be
affected if you end the RJ link or remove the remote journaling
environment.
• When *YES appears in the Use RJ Link column, the data group may be
affected by a request to end the RJ link. If you use the procedure for ending a
remote journal link independently in topic “Ending a remote journal link
independently” on page 202, ensure that any data groups that use the RJ link
are inactive before ending the RJ link.
332
Identifying journal definitions used with RJ
333
Disabling and enabling data groups
Data groups are in an enabled state when they are created. An enabled data group
may have active or inactive replication processes. MIMIX supports the concept of a
disabled state for data groups in a replication environment. When a data group is
disabled, it cannot be started, ended, switched, or audited. However, its configuration
is retained so that the data group can be enabled and used again when needed.
In environments where multiple data groups exist within a single resource group,
changes to data group configuration entries that identify objects to replicate are
propagated to the data groups within a resource group entry as follows:
• If the configuration entry changes are made from an enabled data group, they are
propagated to all data groups within the resource group entry, including disabled
data groups.
• If configuration entry changes are made from a disabled data group, they are not
propagated to the other data groups in the resource group entry.
Data groups associated with an application group may be enabled and disabled
automatically as part of a switch procedure (SWTAG command). If an application
group includes a data group that you do not want to participate in switching
operations, you can change its data group definition to specify *NO for its Allow to be
switched (ALWSWT) parameter.
Data groups can also be manually enabled or disabled with the Change Data Group
(CHGDG) command.
The ability to disable and enable a data group can be beneficial in a variety of
scenarios.
• Disabling a data group is useful for test environments. If you create a data group
for testing purposes, when testing is complete you can simply end and disable the
data group until it is needed again instead of having to delete the data group to
clean up your environment. This provides the benefit of retaining your configured
data group object, file, IFS, and DLO entries while the data group is not needed.
Additionally, the journal manager does not retain journal receivers that have not
been processed by a disabled data group, which allows you to save storage
space on your system.
• In environments that do not use application groups and have non-switchable data
groups configured to replicate in different directions, the ability to disable data
groups can simplify starting replication. You can avoid having to start each data
group individually by disabling the data groups whose direction is not what you
want. Then you can simply start the remaining data groups and the MIMIX
environment using the Start MIMIX (STRMMX) command.
By default, only enabled data groups are shown on the Work with Data Groups
(WRKDG) display. You can display disabled data groups by specifying *ALL or
*DISABLED on the STATE parameter. Disabled data groups are indicated by a status
of D (in green).
Before a data group can be disabled, its replication processes must be ended
(inactive status) and any constraints on target node objects that were previously
334
Disabling and enabling data groups
disabled by MIMIX must be enabled. In environments that use the MIMIX® CDP™
feature, data group processes cannot be suspended at a recovery point before
ending replication because the request to end the data group will clear any recovery
point.
When a disabled data group is enabled, any pending entries must be cleared when
the data group is started. Specify CLRPND(*YES) on the Start Data Group command.
335
Determining if non-file objects are configured for user
journal replication
MIMIX can take advantage of IBM i journaling functions that provide change-level
details in journal entries in a user journal for object types other than files (*FILE).
When properly configured, MIMIX can cooperatively process IFS stream files, data
areas, and data queues between system journal and user journal replication
processes. This enables changes to data or attributes to be replicated through the
user journal instead of replicating the entire object through the system journal every
time a change occurs.
336
Determining if non-file objects are configured for user journal replication
337
Using file identifiers (FIDs) for IFS objects
Commands used for user journal replication of IFS objects use file identifiers (FIDs) to
uniquely identify the correct IFS tracking entries to process. The System 1 file
identifier and System 2 file identifier prompts ensure that IFS tracking entries are
accurately identified during processing. These prompts can be used alone or in
combination with the System 1 object prompt.
These prompts enable the following combinations:
• Processing by object path: A value is specified for the System 1 object prompt
and no value is specified for the System 1 file identifier or System 2 file identifier
prompts.
When processing by object path, a tracking entry is required for all commands
with the exception of the SYNCIFS command. If no tracking entry exists, the
command cannot continue processing. If a tracking entry exists, a query is
performed using the specified object path name.
• Processing by object path and FIDs: A value is specified for the System 1
object prompt and a value is specified for either or both of the System 1 file
identifier or System 2 file identifier prompts.
When processing by object path and FIDs, a tracking entry is required for all
commands. If no tracking entry exists, the command cannot continue processing.
If a tracking entry exists, a query is performed using the specified FID values. If
the specified object path name does not match the object path name in the
tracking entry, the command cannot continue processing.
• Processing by FIDs: A value is specified for either or both of the System 1 file
identifier or System 2 file identifier prompts and, with the exception of the
SYNCIFS command, no value is specified for the System 1 object prompt. In the
case of SYNCIFS, the default value *ALL is specified for the System 1 object
prompt.
When processing by FIDs, a tracking entry is required for all commands. If no
tracking entry exists, the command cannot continue processing. If a tracking entry
exists, a query is performed using the specified FID values.
338
CHAPTER 17 Troubleshooting - where to start
Occasionally, a situation may occur that requires user intervention. This section
provides information to help you troubleshoot problems that can occur in a MIMIX
environment.
You can also consult our website at www.mimix.com for the latest information and
updates for MIMIX products.
The following topics are included in this chapter:
• “Reducing contention between MIMIX and user applications” on page 341
describes a processing timing issue that may be resolved by specifying an Object
retrieval delay value on the commands for creating or changing data group
entries.
• “Data groups cannot be ended” on page 342 describes possible causes for a data
group that is taking too long to end.
• “Verifying a communications link for system definitions” on page 343 describes
the process to verify that the communications link defined for each system
definition is operational.
• “Verifying the communications link for a data group” on page 344 includes a
process to use before synchronizing data to ensure that the communications link
for the data group is active.
• “Checking file entry configuration manually” on page 345 includes the process for
checking that correct data group file entries exist with respect to the data group
object entries. This process uses the Check DG File Entries (CHKDGFE)
command.
• “Data groups cannot be started” on page 347 describes some common reasons
why a data group may not be starting.
• “Cannot start or end an RJ link” on page 348 describes possible reasons that can
prevent you from starting or ending an RJ link. This topic includes a procedure for
removing unconfirmed entries to free an RJ link.
• “RJ link active but data not transferring” on page 349 describes why an RJ link
may not be transferring data and how to resolve this problem.
• “Errors using target journal defined by RJ link” on page 350 describes why errors
when using a target journal defined by an RJ link can occur and how to resolve
them.
• “Verifying key attributes” on page 351 includes a procedure for verifying key
attributes using the VFYKEYATR (Verify Key Attributes) command.
• “Working with data group timestamps” on page 352 describes timestamps and
includes information for creating, deleting, displaying, and printing them.
• “Removing journaled changes” on page 355 describes the configuration
conditions that must be met using the Remove Journaled Changes
(RMVJRNCHG) journal entry.
339
Troubleshooting - where to start
• “Performing journal analysis” on page 356 describes and includes the procedure
for performing journal analysis of the source system.
340
Reducing contention between MIMIX and user applications
341
Data groups cannot be ended
A controlled end for a data group may take some time if there is a backlog of files to
process or if there are a number of errors that MIMIX is attempting to resolve before
ending.
If you think that a data group is taking too long to end, check the following for possible
causes:
• Check to see how many transactions are backlogged for the apply process. Use
option 8 (Display status) on the Work with Data Groups display to access the
detailed status. A number in the Unprocessed Entry Count column indicates a
backlog. Use F7 and F8 to see additional information.
• Determine which replication process is not ending. Use the command
WRKSBSJOB SBS(MIMIXSBS) to see the jobs in the MIMIXSBS subsystem.
Look for jobs for replication processes that have not changed to a status of END.
For example, abc_OBJRTV, where abc is a 3-character prefix.
• Check the QSYSOPR message log to see if there is message that requires a
reply.
• You can use the WRKDGACTE STATUS(*ACTIVE) command to ensure all data
group activity entries are completed. If a controlled end was issued, all activity
entries must be processed before the object processes are ended.
342
Verifying a communications link for system definitions
343
Verifying the communications link for a data group
Before you synchronize data between systems, ensure that the communications link
for the data group is active. This procedure verifies the primary transfer definition
used by the data group. If your configuration requires multiple data groups, be sure to
check communications for each data group definition.
Do the following:
1. From the MIMIX Basic Main Menu, type an 11 (Configuration menu) and press
Enter.
2. From the MIMIX Configuration Menu, type a 4 (Work with data group definitions)
and press Enter.
3. From the Work with Data Group Definitions display, type an 11 (Verify
communications link) next to the data group you want and press F4.
4. The Verify Communications Link display appears. Ensure that the values shown
for the prompts are what you want.
5. To start the check, press Enter.
6. You should see a message "VFYCMNLNK command completed successfully."
If your data group definition specifies a secondary transfer definition, use the following
procedure to check all communications links.
344
Checking file entry configuration manually
345
• To submit the job for batch processing, accept *YES. Press Enter and continue
with the next step.
9. At the Job description prompts, specify the name and library of the job description
used to submit the batch request. Accept MXAUDIT to submit the request using
the default job description, MXAUDIT.
10. At the Job name prompt, accept *CMD to use the command name to identify the
job or specify a simple name.
11. To start the data group file entry check, press Enter.
346
Data groups cannot be started
347
Cannot start or end an RJ link
In normal operations, unconfirmed entries are automatically handled by the RJ link
monitors.
However, there is a scenario where you may end up with a backlog of unconfirmed
entries that can prevent you from starting or ending an RJ link. This problem can
occur when all of the following are true:
• The data group is not switchable or you do not want to switch it
• A link failure on an RJ link that is configured for synchronous delivery leaves
unconfirmed entries
• The RJ link monitors are not active, either because you are not using them or they
failed as a result of a bad link
To recover from this situation, you should run the Verify Communications Link
(VFYCMNLNK) command to assist you in determining what may by wrong and why
the RJ link will not start.
If you are using an independent ASP, check the transfer definition to ensure the
correct database name has been specified.
You also need to end the remote journal link from the target system. Ending the link
from the target system is a restriction of the IBM remote journal function.
348
RJ link active but data not transferring
349
Errors using target journal defined by RJ link
If you receive errors when using a target journal defined by an RJ link, you may need
to change the journal definition and journaling environment. This situation is caused
when the target journal definition is created as a result of adding an RJ link based on
a source journal definition which specified QSYSOPR as the threshold message
queue.
If you receive errors when using the target journal, do the following:
1. On the Work with Journal Definitions display, locate the target journal definition
that is identified by the errors.
2. Type a 5 (Display) next to the target journal definition and press Enter.
3. Page down to see the value of the Threshold message queue.
• If the value is QSYSOPR, press F12 and continue with the next step.
• For any other value, the cause of the problem needs further isolation beyond
this procedure.
4. Type a 2 (Change) next to the target journal definition and press Enter.
5. Press F9 (All parameters), then page down to locate the Threshold message
queue and Library prompts.
6. Change the Threshold message queue prompt to *JRNDFN and the Library
prompt to *JRNLIB, or to other acceptable values.
7. To accept the change, press Enter.
350
Verifying data group file entries
351
Working with data group timestamps
Timestamps allow you to view the performance of the database send, receive, and
apply processes for a data group to identify potential problem areas, such as a slow
send process, inadequate communications capacity, or excessive overhead on the
target system. Although they can assist you in identifying problem areas, timestamps
are not intended as an accurate means of calculating the performance of MIMIX.
A timestamp is a single record that is passed between all replication processes. The
timestamp originates on the source system as a journal entry, is sent to the target
system, and then processed by the associated apply session. The timestamp record
is updated with the date and time at each of the following areas during the replication
process:
• Created - Date and time the journal entry is created
• Sent - Date and time when the journal entry is sent to the target system
• Received - Date and time when the journal entry is received
• Applied - Date and time when the journal entry is applied
Note: For data groups that use remote journaling, the created and sent timestamps
will be set to the same value. The received timestamp will be set to the time
when the record was read on the target system by the database reader
process.
After all four timestamps have been added, the journal entry is converted and placed
into a file for viewing or printing. You can view timestamps only from the management
system. The system manager must be active to return the timestamps to the
management system.
352
Working with data group timestamps
2. The Work with DG Timestamps display appears. Type a 1 (Create) next to the
blank line at the top of the display and press Enter.
3. The Create Data Group Timestamps display appears. Specify the name of the
data group and the number of timestamps you want to create and press Enter.
Note: You should generate multiple timestamps to receive a more accurate view
of replication process performance.
353
Repeat this procedure for each RJ data group for which you want to generate
timestamps.
Deleting timestamps
You can delete all timestamps or you can select a group of one or more timestamps to
delete.
To delete timestamps for a data group, do the following:
1. From the Work with Data Groups display, type 41 (Timestamps) next to the data
group you want and press Enter.
2. The Work with DG Timestamps display appears. Type a 4 (Delete) next to the
timestamps you want to delete and press Enter.
3. A confirmation screen appears. Press Enter.
354
Removing journaled changes
355
Performing journal analysis
When a source system fails before MIMIX has sent all journal entries to the target
system, unprocessed transactions occur. Unprocessed transactions can also occur if
journal entries are in the communications buffer being sent to the target system when
the sending system fails.
Following an unplanned switch, unprocessed transactions on the original source
system must be addressed in order to prevent data loss before synchronizing data
and starting data groups.
The journal analysis process finds any missing transactions that were not sent to the
target system when the source system went down and an unplanned switch to the
backup was performed. Once unprocessed transactions are located, users must
analyze the journal entries and take appropriate actions to resolve them.
The time at which to perform journal analysis is when the original source system has
been brought back up and before performing the synchronization phase of the switch
(which synchronizes data and starts data groups). Analyze all data groups that were
not disabled at the time of the unplanned switch.
Note: The journal analysis tool is limited to database files replicated from a user
journal. The tool does not identify unprocessed transactions for data areas,
data queues, or IFS objects replicated through a user journal, or database files
configured for replication from the system journal.
From the original source system, do the following:
1. Ensure the following are started:
a. The port communications jobs (PORTxxxxx)
b. The MIMIX system managers using STRMMXMGR SYSDFN(*ALL)
MGR(*SYS) TGTJRNINSP(*NO)
IMPORTANT! Only the system managers should be started at this time. Do not
start journal managers. Also, do not start data groups at this time! Doing so will
delete the data required to perform the journal analysis.
2. From the Work with Data Groups display on the original source system, enter 43
(Journal analysis) next to the data group to be analyzed.
The Journal Analysis of Files display appears.
3. Check the following:
• If a pop-up window with the message “Journal analysis information not
collected” is displayed in the list area, press Enter to collect journal analysis
information, then go to Step 6.
• If there is no pop-up window and message LVI379A is displayed at the bottom
of the display, journal analysis determined that all journal entries have been
applied. There are no unprocessed entries to display. The sequence number of
the last applied journal entry is displayed in the Last applied field. No further
action is needed.
356
Performing journal analysis
357
Figure 38. Sample of one journal entry
File identification
File . . . . : <FILE>
Library . : <LIB>
Member . . . : <MBR>
Journal identification
Journal name . . . . . : <JOURNAL>
Library . . . . . . : <JRNLIB>
Receiver identification
Receiver name . . . . : <RCVR>
Library . . . . . . : <RCVRLIB>
Sequence number . . . : <JOURNAL SEQUENCE>
9. Determine what action you need to take for each unprocessed entry. For example:
• You may need to run the original job again on the current source system to
reproduce the entries.
• If a file has already been updated on the current source system (manually or
otherwise), you may need to merge data from both files. If this is the case, do
not synchronize the files.
• If there are write changes (R-PT entries), these changes should be made on
the current source system before running the synchronization phase of the
switch or starting data groups in order to maintain Relative Record Number
consistency within the file. If this is done after the data group has been started,
the relative record numbers could become unsynchronized between the two
systems.
Note: It is the customer’s responsibility to fix the files.
358
Performing journal analysis
file member are immediately removed from the journal analysis information that is
displayed. It does not delete any other information contained in other MIMIX files.
359
Interpreting audit results - supporting information
Audits use commands that compare and synchronize data. The results of the audits
are placed in output files associated with the commands. The following topics provide
supporting information for interpreting data returned in the output files.
• “When the difference is “not found”” on page 363 provides additional
considerations for interpreting result of not found in priority audits.
• “Interpreting results for configuration data - #DGFE audit” on page 361 describes
the #DGFE audit which verifies the configuration data defined to your
configuration using the Check Data Group File Entries (CHKDGFE) command.
• “Interpreting results of audits for record counts and file data” on page 364
describes the audits and commands that compare file data or record counts.
• “Interpreting results of audits that compare attributes” on page 367 describes the
Compare Attributes commands and their results.
360
Interpreting results for configuration data - #DGFE audit
Table 55. CHKDGFE - possible results and actions for resolving errors
*NODGFE No file entry exists. The object is identified by an object entry which
specifies COOPDB(*YES) but the file entry necessary for cooperative
processing is missing.
Create the DGFE or change the DGOBJE to COOPDB(*NO)
Note: Changing the object entry affects all objects using the object entry. If you
do not want all objects changed to this value, copy the existing DGOBJE
to a new, specific DGOBJE with the appropriate COOPDB value.
*EXTRADGFE An extra file entry exists. The object is identified by a file entry and an
object entry which specifies COOPDB(*NO). The file entry is extra when
cooperative processing is not used.
Delete the DGFE or change the DGOBJE to COOPDB(*YES)
Note: Changing the object entry affects all objects using the object entry. If you
do not want all objects changed to this value, copy the existing DGOBJE
to a new, specific DGOBJE with the appropriate COOPDB value.
*RCYFAILED Automatic audit recovery actions were attempted but failed to correct the
detected error.
Run the audit again.
361
The Option column of the report provides supplemental information about the
comparison. Possible values are:
*NONE - No options were specified on the comparison request.
*NOFILECHK - The comparison request included an option that prevented an
error from being reported when a file specified in a data group file entry does not
exist.
*DGFESYNC - The data group file entry was not synchronized between the
source and target systems. This may have been resolved by automatic recovery
actions for the audit.
One possible reason why actual configuration data in your environment may not
match what is defined to your configuration is that a file was deleted but the
associated data group file entries were left intact. Another reason is that a data group
file entry was specified with a member name, but a member is no longer defined to
that file. If you use #DGFE audit with automatic scheduling and automatic audit
recovery enabled, these configuration problems can be automatically detected and
recovered for you. Table 56 provides examples of when various configuration errors
might occur.
362
When the difference is “not found”
363
Interpreting results of audits for record counts and file
data
The audits and commands that compare file data or record counts are as follows:
• #FILDTA audit or Compare File Data (CMPFILDTA) command
• #MBRRCDCNT audit or Compare Record Count (CMPRCDCNT) command
Each record in the output files for these audits or commands identifies a file member
that has been compared and indicates whether a difference was detected for that
member.
Table 57. Possible values for Compare File Data (CMPFILDTA) output file field Difference
Indicator (DIFIND)
Values Description
*EQ (DATE) Member excluded from comparison because it was not changed or
restored after the timestamp specified for the CHGDATE
parameter.
*FF The file feature is not supported for comparison. Examples of file
features include materialized query tables.
364
Interpreting results of audits for record counts and file data
Table 57. Possible values for Compare File Data (CMPFILDTA) output file field Difference
Indicator (DIFIND)
Values Description
*REP The file member is being processed for repair by another job
running the Compare File Data (CMPFILDTA) command.
*SW The source file is journaled but not to the journal specified in the
journal definition.
See “When the difference is “not found”” on page 363 for additional information.
Table 58. Possible values for Compare Record Count (CMPRCDCNT) output file field Dif-
ference Indicator (DIFIND)
Values Description
365
Table 58. Possible values for Compare Record Count (CMPRCDCNT) output file field Dif-
ference Indicator (DIFIND)
Values Description
*EQ Record counts match. No difference was detected within the record
counts compared. Global difference indicator.
*FF The file feature is not supported for comparison. Examples of file
features include materialized query tables.
*SW The source file is journaled but not to the journal specified in the
journal definition.
See “When the difference is “not found”” on page 363 for additional information.
366
Interpreting results of audits that compare attributes
Table 59. Possible values for output file field Difference Indicator (DIFIND)
*EC The values are based on the MIMIX configuration settings. The 5
actual values may or may not be equal.
*EM Established mapping for file identifier (FID). Attribute indicator only 6
for CMPIFSA *FID attribute.
1. The Compare Attribute commands are: Compare File Attributes (CMPFILA), Compare
Object Attributes (CMPOBJA), Compare IFS Attributes (CMPIFSA), and Compare DLO Attri-
butes (CMPDLOA).
367
Table 59. Possible values for output file field Difference Indicator (DIFIND)
*NA The values are not compared. The actual values may or may not 5
be equal.
*NC The values are not equal based on the MIMIX configuration 3
settings. The actual values may or may not be equal.
*NS Indicates that the attribute is not supported on one of the systems. 5
Will not cause a global not equal condition.
*NM Not mapping consistently for file identifier (FID). Attribute indicator 2
only for CMPIFSA *FID attribute.
For most attributes, when the outfile is viewed from the native user interface, when a
detailed row contains blanks in either of the System 1 Indicator or System 2 Indicator
fields, MIMIX determines the value of the Difference Indicator field according to Table
368
Interpreting results of audits that compare attributes
60. For example, if the System 1 Indicator is *NOTFOUND and the System 2 Indicator
is blank (Object found), the resultant Difference Indicator is *NE.
Table 60. Difference Indicator values that are derived from System Indicator values.
Difference Indicator
System 1 Indicator
Object *NOTCMPD *NOTFOUND *NOTSPT *RTVFAILED *DAMAGED
Found (blank
value)
Object Found *EQ / *NE / *NA *NE *NS *UN *NE
(blank value) *UA / *EC /
*NC
System *NOTCMPD *NA *NA *NE *NS *UN *NE
2 *NOTFOUND *NE / *UA *NE / *UA *EQ *NE / *UA *NE / *UA *NE
Indicator
*NOTSPT *NS *NS *NE *NS *UN *NE
*RTVFAILED *UN *UN *NE *UN *UN *NE
*DAMAGED *NE *NE *NE *NE *NE *NE
Table 61. Possible values for output file fields SYS1IND and SYS2IND
*NOTCMPD Attribute not compared. Due to MIMIX configuration settings, this N/A2
attribute cannot be compared.
369
Table 61. Possible values for output file fields SYS1IND and SYS2IND
*NOTSPT Attribute not supported. Not all attributes are supported on all IBM N/A2
i releases. This is the value that is used to indicate an
unsupported attribute has been specified.
*RTVFAILED Unable to retrieve the attributes of the object. Reason for failure 4
may be a lock condition.
1. The priority indicates the order of precedence MIMIX uses when setting the system indicators fields in the summary
record.
2. This value is not used in determining the priority of summary level records.
For comparisons which include a data group, the Data Source (DTASRC) field
identifies which system is configured as the source for replication.
370
APPENDIX B IBM Power™ Systems operations
that affect MIMIX
The following topics describe how to protect the integrity of your MIMIX environment
when you perform operations such as IPLs and IBM i operating system upgrades.
Only basic procedures for a standard one-to-one MIMIX installation are covered. If
you are operating in a complex environment—if you have cluster, SAP R/3, IBM
WebSphere MQ, or other application considerations, for example—contact your
Certified MIMIX Consultant. Ultimately, you must tailor these procedures to suit the
needs of your particular environment.
These topics describe MIMIX-specific steps only. Refer to the user manuals that
correspond to any additional applications installed in your environment. For
instructions on performing IBM Power™ Systems operations, consult your IBM
manuals or the IBM Information Center at
http://publib.boulder.ibm.com/pubs/html/as400/infocenter.html.
The following topics are included:
• “MIMIX procedures when performing an initial program load (IPL)” on page 372
includes the MIMIX-specific steps for performing an initial program load (IPL) to
help ensure the integrity of your MIMIX environment is not compromised.
• “MIMIX procedures when performing an operating system upgrade” on page 374
describes when and how to perform recommended MIMIX-specific steps while
performing a standard upgrade of IBM i.
• “MIMIX procedures when upgrading hardware without a disk image change” on
page 382 describes MIMIX prerequisites and procedures for performing a
hardware upgrade without a disk image change.
• “MIMIX procedures when performing a hardware upgrade with a disk image
change” on page 385 describes prerequisites for saving and restoring MIMIX
software when upgrading from one system to another.
• “Handling MIMIX during a system restore” on page 391 includes prerequisites for
restoring MIMIX software within a MIMIX system pair, to one system from a save
of the other system when an environment meets the conditions specified.
371
IBM Power™ Systems operations that affect MIMIX
372
MIMIX procedures when performing an initial program load (IPL)
7. Start MIMIX from either the source or target system. The Start MIMIX (STRMMX)
command starts the MIMIX processes for the installation, including the MIMIX
managers and the data groups.
For more information about the STRMMX command, see “Starting MIMIX” on
page 226.
8. If you ended the VSP server, restart it using the following command:
VSI001LIB/STRVSISVR
373
IBM Power™ Systems operations that affect MIMIX
Table 62. IBM i operating system upgrade scenarios and recommended processes for handling MIMIX
during the upgrade
Backup system only 1. Perform the preparation steps described in “Prerequisites for performing
an OS upgrade on either system” on page 375.
2. Follow the procedure in “MIMIX-specific steps for an OS upgrade on a
backup system” on page 375.
Production system only 1. Perform the preparation steps described in “Prerequisites for performing
an OS upgrade on either system” on page 375.
2. Perform one of the following procedures:
• If you need to maintain user access to production applications during the
upgrade, perform a planned switch as described in “MIMIX-specific steps
for an OS upgrade on a production system with switching” on page 377.
Your production operations will be temporarily running on the backup
system.
• If you have more flexibility with scheduling downtime, you can perform the
upgrade without switching as described in “MIMIX-specific steps for an
OS upgrade on the production system without switching” on page 379.
Both backup and production 1. Perform the preparation steps described in “Prerequisites for performing
systems an OS upgrade on either system” on page 375.
2. Upgrade the backup system first following the “MIMIX-specific steps for
an OS upgrade on a backup system” on page 375. By doing this first, you
can ensure that the backup system supports all the capabilities of the
production system and you can work through problems or custom
operations before affecting your production environment.
3. Once you have the verified that the backup system is upgraded and
operating as desired, perform one of the following procedures to upgrade
IBM i on the production system:
• If you need to maintain user access to production applications during the
upgrade, perform a planned switch as described in “MIMIX-specific steps
for an OS upgrade on a production system with switching” on page 377.
Your production operations will be temporarily running on the backup
system.
• If you have more flexibility with scheduling downtime, you can perform the
upgrade without switching as described in “MIMIX-specific steps for an
OS upgrade on the production system without switching” on page 379
374
MIMIX procedures when performing an operating system upgrade
375
IBM Power™ Systems operations that affect MIMIX
376
MIMIX procedures when performing an operating system upgrade
different OS versions, Vision Solutions recommends that the backup node run the
higher OS. The following restrictions and limitations may also apply:
• All objects must be compiled using the Target Release parameter to specify
the IBM i version of the lower level operating system.
• Possible inherent restrictions include those specific to new functionality in
the OS. There may be features in a higher OS release that would not be
available in a lower one, such as new command parameters or APIs.
• Some errors may be encountered during or following a role swap due to the
different OS versions.
12. If you ended the VSP server, restart it using the following command:
VSI001LIB/STRVSISVR
377
IBM Power™ Systems operations that affect MIMIX
378
MIMIX procedures when performing an operating system upgrade
379
IBM Power™ Systems operations that affect MIMIX
4. Wait until the status of each data group becomes inactive (red) by monitoring the
status on the Work with Data Groups (WRKDG) display.
For more information about the WRKDG display, see “Displaying data group
status” on page 118.
5. If you have applications that use commitment control, ensure there are no open
commit cycles. For more information, see “Checking for open commit cycles” on
page 230.
If an open commit cycle exist, restart the data group and repeat Step 3, Step 4,
and Step 5 until there is no open commit cycle for any apply session.
6. Use the following command to end other MIMIX products in the installation library,
end the MIMIX managers, and end the RJ links:
ENDMMX ENDOPT(*CNTRLD) ENDRJLNK(*YES) ENDSMRJ(*YES)
7. End the MIMIX subsystems on the production system and on the backup system.
On each system, type the following on a command line and press Enter:
ENDSBS SBS(MIMIXSBS) OPTION(*IMMED)
8. Complete the operating system upgrade. Allow any upgrade conversions and
access path rebuilds to complete before continuing with the next step.
Note: During the IBM i upgrade, make sure you perform a system save on the
system being upgraded. This step will provide you with a backup of
existing data.
9. Start the MIMIX subsystems on the production system and the backup system as
you would during the synchronization phase of a switch. From each system, type
the following on a command line and press Enter:
STRSBS SBSD(MIMIXQGPL/MIMIXSBS)
10. Ensure the names of the journal receivers match the journal definitions:
a. From the production system, specify the command:
installation-name/WRKJRNDFN JRNDFN(QAUDJRN *LOCAL)
b. Next to the JRNDFN(QAUDJRN *LOCAL) journal definition, specify 14 (Build)
and press F4. Type *JRNDFN for the Source for values parameter and press
Enter.
c. Record the newly attached journal receiver name by placing the cursor on the
posted message and pressing F1 or Help.
11. Using the information you gathered in Step 10, start each data group as follows
(This step also starts the MIMIX managers.):
a. From the WRKDG display, type an 9 (Start DG) next to the data group and
press Enter. The Start Data Group display appears.
b. At the Object journal receiver prompt, specify the receiver name recorded in
Step 10c.
c. At the Object large sequence number prompt, specify *FIRST.
d. At the Clear pending prompt, specify *YES.
380
MIMIX procedures when performing an operating system upgrade
12. Start any applications that you disabled prior to completing the IBM i upgrade
according to your Runbook instructions. These applications are normally started
in the program defined in the QSTRUPPGM system value. Allow users back on
the production system.
Note: MIMIX supports replication for up to two version level differences. If running
different OS versions, Vision Solutions recommends that the backup node run
the higher OS. The following restrictions and limitations may also apply:
• All objects must be compiled using the Target Release parameter to specify
the IBM i version of the lower level operating system.
• Possible inherent restrictions include those specific to new functionality in
the OS. There may be features in a higher OS release that would not be
available in a lower one, such as new command parameters or APIs.
• Some errors may be encountered during or following a role swap due to the
different OS versions.
381
IBM Power™ Systems operations that affect MIMIX
382
MIMIX procedures when upgrading hardware without a disk image change
resolving problems.
2. If upgrading a production system, ensure users are logged off the system and
perform a controlled end of all MIMIX data groups. For more information, see
“Ending a data group in a controlled manner” on page 244.
3. Optional step: If upgrading the source system and performing a planned switch,
follow the steps in your runbook. For more information, see “Switching” on
page 302.
4. Use the following command to end all MIMIX products in the installation library,
end the MIMIX managers, and end the RJ links:
ENDMMX ENDOPT(*CNTRLD) ENDRJLNK(*YES) ENDSMRJ(*YES)
5. Record status information for each data group in case it is needed later. Do the
following:
WRKDG DGDFN(*ALL) OUTPUT(*OUTFILE) OUTFILE(MIMIXQGPL/SWITCH)
6. Optional step: Save the MIMIX software and Vision Solutions Portal by doing a
full system save or by saving the following MIMIX installation libraries and IFS
directories:
• LAKEVIEW
• MIMIXQGPL
• MIMIX-installation-library
• MIMIX-installation-library_0
• MIMIX-installation-library_1
• VSI001LIB
• /LakeviewTech (directory tree)
• /visionsolutions/http/vsisvr
383
IBM Power™ Systems operations that affect MIMIX
8.1.07.00), you must reinstall License Manager using the installation wizard or the
instructions in the “Installing products” topic in the Using License Manager book.
4. Confirm that communications work between the new system and other systems in
the MIMIX environment. For more information, see “Verifying a communications
link for system definitions” on page 343.
5. Optional step: If you need to keep users active, perform a planned switch to the
backup system by following the steps in your runbook. See “Considerations for
performing a hardware system upgrade without a disk image change” on
page 382 to determine if a switch is required.
If you do not have a Runbook, you need to follow your processes for the following:
a. End all user applications, user interfaces, and operations actively running on
the production system.
b. Perform a planned switch to the backup system.
c. Start user applications on the backup system and allow users to access their
applications from the backup system.
6. Start MIMIX from either the source or target system. The Start MIMIX (STRMMX)
command starts the MIMIX processes for the installation, including the MIMIX
managers, data groups, and application groups. For more information about the
STRMMX command, see “Starting MIMIX” on page 226.
7. Run your MIMIX audits to verify the systems are synchronized. See “Running an
audit immediately” on page 169 for more information about running audits.
384
MIMIX procedures when performing a hardware upgrade with a disk image change
385
IBM Power™ Systems operations that affect MIMIX
386
MIMIX procedures when performing a hardware upgrade with a disk image change
387
IBM Power™ Systems operations that affect MIMIX
15. Ensure all automation, including MIMIX exit programs, for MIMIX is available and
configured on the new system.
16. Make any necessary modifications to the QSTARTUP program. This may need to
be modified to start the MIMIX subsystem after an IPL. For more information, see
“Considerations for performing a hardware system upgrade with a disk image
change” on page 385.
17. Start the MIMIX subsystem with the following command:
STRSBS SBSD(MIMIXQGPL/MIMIXSBS)
18. On each system, do the following to clean up journals associated with system
manager RJ links:
a. Use the following command to display a list of the appropriate journal
definitions:
WRKJRNDFN JRNDFN(*ALL *LOCAL) RJLNK(*TGT) PROCESS(*INTERNAL)
b. For each journal definition listed, do the following:
• Type option 17 (Work with jrn attributes) and press Enter.
• Press F15 (Work with receiver directory).
• Type option 4 (Delete) for all receivers in the list. If message CPA7025 is
issued, reply with an “I”.
19. Confirm that communications work between the new system and other systems in
the MIMIX environment. For more information, see “Verifying a communications
link for system definitions” on page 343.
20. Start the system manager with the following command:
STRMMXMGR SYSDFN(*ALL) MGR(*SYS)
21. Optional step: Perform a data group switch by following the steps in your
runbook, then skip to Step 23 for the target system. See “Considerations for
performing a hardware system upgrade with a disk image change” on page 385 to
determine if a switch is required.
Important!
The role of the upgraded system within your production replication environment
determines what step to perform next.
• If the upgraded system is both the source and target of data groups within
your environment, you must perform both Step 22 and Step 23.
• If the upgraded system is only a source system, perform Step 22.
• If the upgraded system is only a target system, perform Step 23.
22. If the source system was upgraded, use this step to ensure that journaling
on the new system is correct and start replication processes:
a. On the source system, type the following on a command line, and press Enter:
WRKJRNDFN JRNDFN(*ALL *LOCAL) PROCESS(*REP)
388
MIMIX procedures when performing a hardware upgrade with a disk image change
b. Press F10 to verify the Receiver Prefix, Library, and all other parameters
(option 5) are correct. Make any necessary changes from the MIMIX
management system before continuing.
c. From the Work with Journal Definitions display, do the following to build the
journaling environment and record the journal receiver names:
• Type option 14 next to a journal definition with a value of *SRC or *NONE in
the RJ Link column and press F4 (Prompt).
• Type *JRNDFN for the Source for values parameter and press Enter.
• Record the newly attached journal receiver name by placing the cursor on
the posted message and pressing F1 or Help. You will need this information
to start data groups.
d. Repeat Step 22c for each journal definition with an RJ Link value of *SRC or
*NONE.
e. For each data group that has the upgraded system as its source system, run
the following commands to ensure that files and objects are journaled to the
correct journal:
• To verify file entries are journaled, run:
VFYJRNFE DGDFN(dgname) FILE1(*ALL) JRNSYS(*SRC)
• To verify IFS entries are journaled, run:
VFYJRNIFSE DGDFN(dgname)JRNSYS(*SRC)
• To verify object entries are journaled, run:
VFYJRNOBJE DGDFN(dgname)JRNSYS(*SRC)
g. For each data group that has the upgraded system as its source system, start
the data group with a clear pending start from the receivers recorded in
Step 22c of this procedure:
STRDG DGDFN(data-group-name) DBJRNRCV(user-journal-receiver)
DBSEQNBR2(*FIRST) OBJJRNRCV(security-journal-receiver)
OBJSEQNBR2(*FIRST) CLRPND(*YES) DTACRG(*YES)
h. User and application activity can be resumed on the system.
23. If the target system was upgraded, use this step to ensure that journaling
on the new system is correct and start replication processes:
i. On the target system, type the following on a command line and press Enter:
WRKJRNDFN JRNDFN(*ALL *LOCAL) PROCESS(*REP)
j. Press F10 to verify the Receiver Prefix, Library, and all other parameters
(option 5) are correct. Make any necessary changes from the MIMIX
management system before continuing.
k. From the Work with Journal Definitions display, do the following to build the
journaling environment:
389
IBM Power™ Systems operations that affect MIMIX
n. For the journal definitions listed on the Work with Journal Definitions display,
do the following:
• Type option 17 (Work with jrn attributes) and press Enter.
• Press F15 (Work with receiver directory).
• Type option 4 (Delete) for all receivers in the list. If message CPA7025 is
issued, reply with an “I”.
o. Repeat Step 23n for each journal definition associated with the target system.
p. For each data group that has the upgraded system as its target system, run the
following commands to ensure that files and objects are journaled to the
correct journal:
• To verify file entries are journaled, run:
VFYJRNFE DGDFN(dgname) FILE1(*ALL) JRNSYS(*TGT)
• To verify IFS entries are journaled, run:
VFYJRNIFSE DGDFN(dgname)JRNSYS(*TGT)
To verify object entries are journaled, run:
VFYJRNOBJE DGDFN(dgname)JRNSYS(*TGT)
r. Use the following command and the printed information collected in Step 6 of
“Hardware upgrade with a disk image change - preliminary steps” on page 386
to start all of the data groups.
Note: For each data group, use the Last read receiver and sequence number
from its printed Object Status view (Step 6c) as the values for
OBJJRNRCV andOBJSEQNBR2, and the values from the printed
Database Status view (Step 6d) as the values for DBJRNRCV and
DBSEQNBR2 in the following command:
STRDG DGDFN(data-group-name) DBJRNRCV(last-read-database-
journal-receiver) DBSEQNBR2(last-read-database-sequence-number)
OBJJRNRCV(last-read-object-journal-receiver) OBJSEQNBR2(last-
read-object-sequence-number) CLRPND(*YES) DTACRG(*YES)
24. From both systems, delete any old receivers with different library or prefix names.
25. Run your MIMIX audits to verify the systems are synchronized. See “Running an
audit immediately” on page 169 for more information about running audits.
390
Handling MIMIX during a system restore
391
Index
Symbols #DLOATR considerations 152
*ATTN #FILDTA considerations 152
application group 83 #IFSATR considerations 152
managers for node 87 #MBRRCDCNT considerations 153
monitors 86 after a configuration change 151
replication 89 authority level to run 24
*CANCEL, step status of 109 automatic starting of 60
*CANCELED, procedure status of 102, 113 before switching 151
*FAILED activity entry 278, 281 best practice 24, 151
*FAILED status bi-directional environment considerations 29
procedure 102, 113 change history retention criteria 43
step 109 change status severity 44
*HLD changing automatic schedule 68
file entry 263 check for compliance 45
tracking entry 272 check for reported problems 162
*HLDERR compare phase 147
file entry 263 comparison levels 53
tracking entry 272 compliance 160
*HLDRLTD file entry 263 compliance threshold 52, 53
*INACTIVE definition of 18
application group 85 differences, resolving 162
node managers 88 displaying compliance status 167
replication 89 displaying history 171
*MSGW status displaying status 157
procedure 101 displaying time of next scheduled run 155
step 108 displaying when prioritized runs are allowed
*UNKNOWN 85 155
enable or disable automatic 70
A ending 170
accessing history 171
MIMIX Main Menu 26 job log 166
activity entries, object last performed 167
confirm delay/retry cycle 282 last successful run 150
failed, resolving 278 no objects selected 173
remove history 285 objects compared 173
retrying 281 policies which affect 40
additional resources 14 policies, runtime behavior 40
application group policies, submitting automatically 61
resolving reported problems 82 prevent from running 46, 70
status of 80 priority selection example 173
application group definition 18, 21 priority, changing selection frequency 69
application node status 84 problems reported in installation 118
applications, reducing contention with 341 recovery phase 147
ASP 1 recovery threshold, storage used ex- results recommendations 153
ceeds 183 retain history of 55
audit rule name and description 65
running immediately 169
default settings for automatic 64 schedule for automatic runs 155
#DGFE considerations 152 status
runtime 162
392
status, compliance 160 switching 303
three or more node considerations 29 best practices
when not to audit 30 auditing 151
audit history bi-directional environment policy consider-
change criteria 43 ations 29
audit level
best practice 54 C
changing before switch 42 cancel
audit results 162 procedure 114
#DGFE rule 361 clear error entries
#FILDTA rule 364 processing 223
#MBRRCDCNT rule 364 when to 228
interpreting, attribute comparisons 367 clear pending entries
interpreting, file data comparisons 364 check for open commits 230
resolving problems 162, 361 open commit cycle prevents 230
troubleshooting 166 processing 223
auditing level, object resolving open commits before 230
set when starting a data group 221 when to 228
used for replication 287 cluster services 22
authority level cold start, replacement for 223
for product access 24 collector services 22
automatic error recovery ending 186
replication, policies for 35 starting 185
system journal replication 261 status 180
user journal replication 260 collector services status 87
automatic recovery command, by mnemonic
audits 50 SETMMXSCD 61
concept 18 command, by name
system journal replication 50 Set MIMIX Schedules 61
user journal replication 50 Work with Audit History 171
AutoNotify feature, MIMIX 210 commands, by mnemonic
auxiliary storage pool, system (ASP1) 34 CHGDG 334
CHGMONSTS 197
B CHGPROCSTS 110
backlog CHKDGFE 345, 361
starting shared object send job 222 CNLPROC 114
system manager 181 CRTDGTSP 352
backlog, identifying a 137 DLTDGTSP 354
backup node sequence DSPDGSTS 126
changing 91 DSPDGTSP 354
examples of changing 93 DSPMMXMSGQ 283
verifying 90 ENDAG 215
backup system 19 ENDDG 215, 232, 238
best practice ENDJRNFE 292
audit level 54, 151 ENDJRNIFSE 295
audit level before switch 42, 54, 151 ENDJRNOBJE 299
audit threshold 52, 53 ENDMMX 215, 232, 240
switch frequency 303, 311 ENDMON 195
switch threshold 57 ENDMSTMON 196
393
ENDRJLNK 202 Display Data Group Timestamps 354
ENDSVR 329 Display MIMIX Message Queue 283
HLDDGLOG 355 End Application Group 215
MIMIX 26 End Data Group 215, 232, 238
RLSDGLOG 355 End Journaling File Entries 292
RUNPROC 111 End Journaling IFS Entries 295
STRAG 215, 217 End Journaling Obj Entries 299
STRDG 215, 217, 221 End Lakeview TCP Server 329
STRJRNFE 291 End Master Monitor 196
STRJRNIFSE 294 End MIMIX 215, 232, 240
STRJRNOBJE 298 End Monitor 195
STRMMX 215, 217, 226 End RJ Link 202
STRMON 194 Hold Data Group Log 355
STRMSTMON 196 MIMIX 26
STRRJLNK 202 Release Data Group Log 355
STRSVR 328 Run Procedure 111
SWTDG 312, 314 Start Application Group 215, 217
VFYCMNLNK 343, 344 Start Data Group 215, 217, 221
VFYJRNFE 293 Start Journaling File Entries 291
VFYJRNIFSE 296 Start Journaling IFS Entries 294
VFYJRNOBJE 300 Start Journaling Obj Entries 298
VFYKEYATR 351 Start Lakeview TCP Server 328
WRKAG 80 Start Master Monitor 196
WRKAUDHST 171 Start MIMIX 215, 217, 226
WRKAUDOBJ 173 Start Monitor 194
WRKAUDOBJH 176 Start RJ Link 202
WRKCPYSTS 331 Switch Data Group 312, 314
WRKDG 118 Verify Communications Link 343, 344
WRKDGACT 278 Verify Journaling File Entry 293
WRKDGACTE 279 Verify Journaling IFS Entries 296
WRKDGFE 263 Verify Journaling Obj Entries 300
WRKDGIFSTE 272 Verify Key Attributes 351
WRKDGOBJTE 272 Work with Application Groups 80
WRKDGTSP 352 Work with Audited Obj. History 176
WRKDTARGE 88 Work with Audited Objects 173
WRKMSGLOG 284 Work with Copy Status 331
WRKNFY 206 Work with Data Group Activity 278
WRKNODE 87 Work with Data Groups 118
WRKPROCSTS 98 Work with Data Rsc. Grp. Ent. 88
WRKRJLNK 199, 332 Work with DG Activity Entries 279
WRKSTEPSTS 103 Work with DG File Entries 263
commands, by name Work with DG IFS Tracking Ent. 272
Cancel Procedure 114 Work with DG Obj Tracking Ent. 272
Change Data Group 334 Work with DG Timestamps 352
Change Monitor Status 197 Work with Message Log 284
Change Procedure Status 110 Work with Node Entries 87
Check Data Group File Entries 345, 361 Work with Notifications 206
Create Data Group Timestamps 352 Work with Procedure Status 98
Delete DG Timestamps 354 Work with RJ Links 199, 332
Display Data Group Status 126 Work with Step Status 103
394
commit cycles D
effect on audit comparison 364, 365 data areas and data queues
commit cycles, open determining configuration of 337
checking for 230 holding user journal entries for 274
checking for after a controlled end 245 resolving problems 273
preventing problems with 235 tracking entries 272
preventing STRDG request 230 verifying journaling 300
commit mode change data group 18
prevents starting with open commits 230 backlogs 137
communications controlled vs. immediate end 234
ending TCP sever 329 definition 21
starting TCP sever 328 determining if RJ link used 332
compare phase 147 disabling 335
compliance enabling 335
audit 160 ending considerations 238
change what audits are checked 45 ending controlled 244
concept 150 ending immediately 247
switch 311 ending selected processes 247
switch, policies for 49 indication of disabled state 334
compliance status recovery point cleared 238
switch 311 starting selected processes 228
concepts state, disabled or enabled 334
auditing 146 status, database view 133
MIMIX 18 status, detailed 126
configuration status, merged view 127
audit after changing 151 status, object view 131
determining data areas and data queues 337 status, summary 118
determining, IFS objects 336 switching 307, 312
results of #DGFE audit after changing 361 timestamps 352
configuration changes deployed 221 when to exclude from auditing 30
constraints data group entry
enabling for *INACTIVE data group 249 description 21
enabling MIMIX-disabled 248 data groups
enabling when ending replication 249 accessing 118
manually enable on target 248 data resource group 88
contacting Vision Solutions 15 replication status summary 89
contention with applications, reducing 341 database apply (DBAPY) status 134
controlled end database apply cache policy 51
confirm end 245 database error recovery, automatic 261
description 234 deadlock condition
procedure 244 resolving 144
wait time 235 definition
cooperative processing 21 application group 21
copying active files 331 data group 21
correcting journal 21
file-level errors 269 remote journal (RJ) link 21
record-level errors 270 system 21
CustomerCare 15 transfer 21
definitions
395
application group 18 included processes 236
delay/retry cycle, confirm object in a 282 using default values 240
delaying captures for replication recoveries 36 using specified values 240
disabled audit 70 when to end RJ links 236
disabled data group 334 ending replication 215
disabled report 73 choices 232
displaying controlled vs. immediate 234
data group spooled file information 330 ending RJ link
data group status details 126 independently 202
long IFS object names 330 when to end 184, 236
RJ link status 199 errors
documents, MIMIX 11 file level 269
record level 270
E system journal replicated objects 278
ending target journal of RJ link 350
audit 170 user journal replicated files 263
collector services 186 user journal replicated objects 272
journal managers 185 example
master monitor 196 priority audit object selection 173
MIMIX managers 184, 185 examples
MIMIXSBS subsystem 241 changing backup node sequence 93
monitor 195
system manager RJ links 184 F
system managers 184 file
target journal inspection 187 file-level errors 269
TCP server 329 hold journal entries 267
ending data group new 287
clears recovery point 238 not journaled 122
considerations when ending 238 record-level errors 270
controlled end 244 replicated 263
controlled end and enable constraints 249 file identifiers (FIDs) 338
controlled end wait time 235 file in error
controlled vs. immediate 234 examine held journal entries 266
how to confirm end 245 resolving 263
immediate end 247 file on hold
processes 235 release and apply held entries 268
processes, effect of 255 release and clear entries 269
processes, specifying selected 247 release at synchronization point 268
when to end RJ link 236 files with protected rows or columns 270
ending journaling
data areas and data queues 299 H
files 292 hardware upgrade
IFS objects 295 MIMIX-specific steps 382
IFS tracking entry 295 no disk image change 382
object tracking entry 299 prerequisites 382
ending MIMIX 240 with a disk image change 385
controlled vs. immediate 234 held error (*HLDERR)
end subsystem, when to also 241 file entry 263
follow up after 241 preferred action for entry 264, 273
396
tracking entry 272 requirements and restrictions 288
history journal cache or state
audited object 176 resolving problems 122, 141
completed audits 171 status 140
displaying audit 171 journal definition 21
history log, removing completed entries 285 defined to RJ Link 333
history of, retaining journal entry
procedures 33 description 20
hold (*HLD) unconfirmed 348
preferred action for held entry 264, 273 Journal manager
put file entry on hold 267 resolving problems 183
put tracking entry on hold 274 journal manager 22
release a held file entry 268 ending 185
release a held tracking entry 276 resolving problems 180
hold ignore (*HLDIGN) starting 184
preferred action for ignored entry 264, 273 status 180
put file entry on hold ignore 267 journal receiver 20
put tracking entry on hold ignore 275 journal status
hold related (*HLDRLTD) 264 resolving problems 143
hot backup 16 journaling 20
data group problem with 120
I ending for data areas and data queues 299
i5/OS upgrade 374 ending for files defined to a data group 292
IFS objects ending for IFS objects 295
determining configuration 336 implicitly started 287
file IDs (FIDs) 338 requirements for starting 287
hold user journal entries for 274 starting for data areas and data queues 298
path names 330 starting for IFS objects 294
resolving problems 273 starting for physical files 291
tracking entries for 272 starting, ending, and verifying 286
verifying journaling 296 verifying for data areas and data queues 300
immediate end verifying for IFS objects 296
description 234 verifying for physical files 293
incomplete tracking entry 234 journaling status
information and additional resources 14 data areas and data queues 298
inspection files 291
target journal 23 IFS objects 294
IPL 372
K
J keyed replication
job log verifying file attributes 351
for audit 166
jobs L
used by procedures 97 last audit performed 150
used by procedures, status of 104 last switch performed 311
journal 20 log space 23
inspection on target system 23 long IFS path names 330
journal at create
requirements 287
397
M N
management system 20 names, displaying long 330
manager network system 20
journal 22 new hardware upgrade 385
system 22 MIMIX-specific steps 386
manager status 87, 180 prerequisites 385
master monitor new objects
ending 195, 196 IFS object journal at create requirements 287
starting 195, 196 journal at create selection criteria 288
menu newly created objects, notification of 210
MIMIX Main 26 node entries 87
message queue, primary and secondary 283 node status
messages application group 84
ENDMMX 218 data resource group 89
STRMMX 218 nodes, policy considerations for multiple 29
MIMIX CDP feature not journaled
exclude from audit 30 resolving problems 122
recovery point cleared 238 notification status 205
MIMIX installation 18 notifications
MIMIX managers definition 19, 204
checking for a backlog 181 displaying 205
ending 185 new problems in installation 118
resolving problems 180, 181, 183 severity level 150, 207
starting 184 status 206
MIMIX Model Switch Framework 24, 307
policy default 57 O
MIMIX rules 146 object
MIMIX subsystem (MIMIXSBS) audited history 176
starting 226 object auditing
when to end 241 concept 20
MIMIX Switch Assistant 24 setting level with STRDG 221
setting default switch framework 48 used for replication 287
setting switch compliance policies 49 object error recovery, automatic 262
MMNFYNEWE monitor 210 object send process
monitor considerations for starting a shared 222
disabling 197 objects
enabling 197 audited object list 173
ending 195 configuration of non-file 336
master 195 displaying long IFS names 330
nodes where needed 193 displaying objects in error 128
starting 194, 195 displaying objects with active entries 129
status 192 in error, resolving 278
monitor for newly created objects 210 new 287
monitors reducing contention 341
nodes where needed 86 tracking entries for data areas and data
status of 85 queues 272
MXSCHED job 60 open commit cycles
audit results 364, 365
prevent problems with 235
398
resolving before starting replication 230 notification severity 51
shown in status 245 object only on target 51
when starting a data group 230 object recovery notify on success 50
operations prioritized audit in effect 155
common, where to start 117 procedure history retention 58
less common 327 run rule on system 54
orphaned recoveries 213 switch action threshold 57
output file fields switch warning threshold 57
Difference Indicator 364, 367 synchronize threshold size 56
System 1 Indicator field 369 third delay retry interval 56
System 2 Indicator field 369 third delay retry interval, number of 56
user journal apply threshold 51
P PPRC replication status 84, 89
path names, IFS 330 problems
planned switch 303 troubleshoot 339
policies 19 problems, journaling
audit, automatically submitting 61 data areas and data queues 298
audit, runtime behavior of 40 files 291
changing values 31 IFS objects 294
for auditing 40 problems, resolving
for replication 35 audit results 361
for switching 48 auditing 162
installation-level only 33 common errors 259
introduction 28 common system level errors 178
multi-node and bi-directional environment data group cannot end 342
considerations 29 data group cannot start 347
policy files in error 263
action for running audits 54 files not journaled 122
ASP threshold for recoveries 58 journal cache or state 122, 141
audit action threshold 53 journal status 143
audit history retention 55 not journaled 122
audit level 53 objects in error 278
audit notify on success 50 open commits when starting data group 230
audit rule 50 RJ link cannot end 348
audit severity 44 RJ link cannot start 348
audit warning threshold 52 system level processes 180
audits to include in compliance 45 tracking entries 272
automatic audit recovery 50 procedure
automatic database recovery 50 acknowledging failed or canceled 110
automatic object recovery 50 begin at step 111, 219, 306
changing source capture delay 36 canceling 114
CMPRCDCNT commit threshold 58 defined 23
data group definition 50 directory data protection report 58
database apply cache 51 displaying status 98
default model switch framework 57 history retention 58
directory depth 58 how to run 111
independent ASP library ratio 57 last run 98
journaling attribute difference action 51 multiple jobs 97
maximum rule runtime 52 multiple jobs, status of 104
next automatic report run 114
399
overriding step attributes 112 changing backup sequence 91
resolve problems 98, 99 verifying sequence 90
resuming canceled or failed 112 recovery phase 147
run type *NODE 112 recovery point
run type *USER 112 cleared by ENDDG 238
run type other than *USER 112 recovery, automatic 18
status 99 related IFS tracking entries 277
status history of a 115 release (*RLS)
step status 103 held file entry 268
procedure history held tracking entry 276
change criteria 33 release clear (*RLSCLR)
procedures 97 file entry 269
change directory depth for data protection re- tracking entry 276
ports 33 release wait (*RLSWAIT)
change history retention criteria 33 file entry 268
history retention 33 tracking entry 275
procedures and steps remote journal
differences in virtual switch 324 i5/OS function 20
processes remote journal (RJ) link 21
system level 178 remote journal environment
production system 19 processes ended by ENDDG 255
publications, IBM 14 processes started by STRDG 251
unconfirmed journal entry 348
Q removing
QSTRUPPGM system value 372, 375 activity history entries 285
duplicate tracking entries 276
R reorganizing, active files 331
recommendations replication
auditing 151 automatic error recovery 35
before planned switch 303 backlogs, identifying 137
checking audit results 153 before starting 217
policies in bi-directional environment 29 commands for ending 232
policies in three or more node environment commands for starting 217
29 direction of 19
starting shared object send 222 ending 215
recoveries policies that affect 35
active in installation 118 recovery source capture delay 37
definition 19, 205 resolve replication errors 259
detected database errors 261 starting 215
displaying count of in progress 211 status summary 84, 89
displaying details 211 supported paths 16
occurring in installation 211 switching 302
orphaned 213 system journal 16, 22
orphaned, identifying 214 user journal 16, 22
orphaned. removing 214 replication path 22
recoveries delayed by ASP threshold 183 replication, problems
recovery troubleshoot 339
source capture delay 36 where to start 259
recovery domain report
automatic starting of 60
400
changing automatic schedule 72 procedure 111
default schedule 67 running
enable or disable automatic 73 audits immediately 169
next scheduled run 114
requirements S
audits 151 schedule
enable constraints manually 248 automatic audit or report 62
journal at create 287 automatic audits, displaying 155
journaling 287 changeable criteria 61
resolving problems changing audit 68
application group 82 changing report 72
application group *ATTN status 83 priority audit 63
application group other problem status val- shipped defaults for auditing 64
ues 84 shipped defaults for reports 67
common replication errors 259 scheduler job, MXSCHED 60
data resource group status 88 servers
node entry status 87 ending TCP 329
system level jobs 180 starting TCP 328
troubleshooting 339 service
resource group, data 88 cluster 22
status 88 status collector 22
restore MIMIX services
prerequisites 391 collector, ending 186
restrictions collector, starting 185
journal at create 288 severity level, notification 207
QDFTJRN data area 288 source system 19
retry objects in error 281 spooled files, displaying MIMIX-created 330
retrying, data group activity entries 281 standby journaling
RJ link 21 IBM i5/OS option 42 139
ending independently 202 overview 139
errors for target journal of 350 starting
identifying data groups that use 332 collector services 185
journal definitions by an 333 master monitor 195, 196
operating independently 202 MIMIX managers 184
removing unconfirmed entries 348 monitor 194
status 199 procedure at step 111, 219, 306
when to end 236 RJ link independently 202
RJ link, ending system manager 184 system and journal managers 184
rule target journal inspection 186
#DGFE 65 TCP server 328
#DLOATR 66 virtual switch 323
#FILATR 66 starting data group
#FILATRMBR 66 at specified journal location 222
#FILDTA 66 deploy configuration 221
#IFSATR 66 prevented by open commit cycles 230
#OBJATR 66 procedure 228
descriptions 65 processes, effect of 251
rules set object auditing level 221
MIMIX 146 when to clear entries 228
run
401
starting journaling ended 270
data areas and data queues 298 ending 241
file entry 291 starting 226
files 291 switch
IFS objects 294 precheck for virtual 323
IFS tracking entry 294 Switch Assistant, MIMIX 24
object tracking entry 298 switch framework
starting MIMIX disable policy when not used 49
included processes 217 specify a default 48
procedure 226 switch testing, virtual 316
starting replication 215 switch, virtual 316
before 217 performing a 323
choices 217 problem resolution differences 324
status switching 302
active file operations 331 application group 308
application group 80 best practice 24, 303, 311
audit compliance 167 change audit level before 42, 54
audits 157 compliance 311
collector services 87 conditions that end 314
data group detail 126 description, planned switch 303
database apply (DBAPY) 134 description, unplanned switch 304
journal cache or state 140 journal analysis after unplanned switch 356
journaling data areas and data queues 298 last switch field 311
journaling files 291 phases of a 303
journaling IFS objects 294 policies for 48
journaling tracking entries 294, 298 reasons for 303
monitors 85, 192 setting switch compliance policies 49
node entries 87 setting switch framework policy 48
notification 206 switch framework vs. SWTDG command 307
notifications 205 SWTDG command details 314
procedures 98 unplanned, actions to complete an 304
recoveries active in installation 211 using option 6 on MIMIX Basic Main Menu
replication, application group level 84 309
replication, data resource group level 89 using STRDG command 312
replication, logical 89 synchronize
replication, PPRC 89 file entry 264
RJ link 199 objects, system journal replicated 280
steps in a procedure 103 tracking entries 274
switch compliance 311 system definition 21
switching 124 system journal replication 16, 22
system and journal managers 87 detailed status 131
system-level processes 180 errors automatically recovered 261
target journal inspection processes 188 journaling requirements 287
Work with Data Groups display 118 system level processes 178
step system manager 22
begin procedure at 111, 219, 306 backlog 181
defined 23 ending 184
resolve problems 106 resolving problems 180, 181
status 103, 106 starting 184
subsystem, MIMIXSBS status 180
402
when to end associated RJ link 184 performing journal analysis 356
system roles unprocessed entries 245
management or network 20 upgrade
production or backup 19 hardware, no disk image change 382
source or target 19 hardware, with a disk image change 385
new hardware 385
T OS/400 374
target constraint management 248 user journal replication 16, 22
target journal inspection 23 detailed status 133
last entry inspected 191 errors automatically recovered 260
results 189 journaling requirements 287
starting 186 non-file objects 336
status 180, 188 tracking entries 272
target system 19 tracking entry 21
threshold
ASP storage used 183 V
audit action 53 verifying
audit warning 52 communications link 343, 344
CMPRCDCNT open commit 58 journaling, IFS tracking entries 296
switch action 57 journaling, object tracking entries 300
switch warning 57 journaling, physical files 293
synchronize size 56 key attributes 351
System ASP percent used 58 viewing status, active file operations 331
system ASP percent used recovery 34 virtual switch
user journal apply 51 application testing considerations 321
timestamps 352 automatic error recovery 318
automatically created 352 check MIMIX prerequisites 323
creating additional 352 compared to test planned switch 316
deleting 354 configuration requirements 319
displaying 354 major phases of 317
printing 354 MIMIX considerations 319
tips operational considerations 319
displaying data group spooled files 330 performing a 323
displaying long IFS object names 330 procedure problem differences 324
removing journaled changes 355 source capture delay during testing 37
working with active file operations 331
tracking entry 21 W
file identifiers (FIDs) 338 wait time, data group controlled end 235
IFS 272 wait time, data group controlled end during
incomplete 234 switch 314
not journaled 122
object 272
related IFS entries 277
removing duplicate 276
transfer definition 21
U
unconfirmed journal entries, removing 348
unplanned switch 304
403