DirectX 9 Audio®

Exposed: Interactive
Audio Development

Edited by Todd M. Fay

With Scott Selfon and Todor J. Fay

Wordware Publishing, Inc.

Library of Congress Cataloging-in-Publication Data

Fay, Todd M.
DirectX 9 audio exposed: interactive audio development / edited by Todd M. Fay.
p. cm.
ISBN 1-55622-288-2 (paperback, companion CD-ROM)
1. Sound—Recording and reproducing—Digital techniques. 2. Computer
games—Programming. 3. DirectX. I. Fay, Todd M.
TK7881.4.D56 2003
794.8'165—dc22 2003014463
CIP

© 2004, Wordware Publishing, Inc.

All Rights Reserved

2320 Los Rios Boulevard

Plano, Texas 75074

No part of this book may be reproduced in any form or by any means

without permission in writing from Wordware Publishing, Inc.

Printed in the United States of America

ISBN 1-55622-288-2

10 9 8 7 6 5 4 3 2 1
0307

DirectX, DirectMusic, and DirectSound are registered trademarks of Microsoft Corporation in the United States and/or
other countries.
All screen shots and game titles used in this book remain the property of their respective companies.
All brand names and product names mentioned in this book are trademarks or service marks of their respective companies.
Any omission or misuse (of any kind) of service marks or trademarks should not be regarded as intent to infringe on the
property of others. The publisher recognizes and respects all marks used by companies, manufacturers, and developers as a
means to distinguish their products.
This book is sold as is, without warranty of any kind, either express or implied, respecting the contents of this book and any
disks or programs that may accompany it, including but not limited to implied warranties for the book’s quality, performance,
merchantability, or fitness for any particular purpose. Neither Wordware Publishing, Inc. nor its dealers or distributors shall
be liable to the purchaser or any other person or entity with respect to any liability, loss, or damage caused or alleged to have
been caused directly or indirectly by this book.

All inquiries for volume purchases of this book should be addressed to Wordware
Publishing, Inc., at the above address. Telephone inquiries may be made by calling:
(972) 423-0090
Contents

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . xix
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . xxi

Unit I: Producing Audio with DirectMusic

Chapter 1 DirectMusic Concepts . . . . . . . . . . . . . . . . . . . . . 3

Interactivity . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Variability . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Adaptability . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Groove Levels . . . . . . . . . . . . . . . . . . . . . . . . . 7
Content-Driven Audio . . . . . . . . . . . . . . . . . . . . . 8
Playback Standardization. . . . . . . . . . . . . . . . . . . . 8
3D Spatialized Audio. . . . . . . . . . . . . . . . . . . . . . 9
DirectMusic Rules and Regulations . . . . . . . . . . . . . . 10
Each DirectMusic Performance Has a Single Tempo
at Any One Time. . . . . . . . . . . . . . . . . . . . . . 10
Primary and Secondary Segments . . . . . . . . . . . . . 11
Crossfades . . . . . . . . . . . . . . . . . . . . . . . . 12
Pause/Resume . . . . . . . . . . . . . . . . . . . . . . . 12
Memory Footprint . . . . . . . . . . . . . . . . . . . . . 13
DirectMusic and MIDI . . . . . . . . . . . . . . . . . . . . . 14

Chapter 2 Linear Playback. . . . . . . . . . . . . . . . . . . . . . . . 17

Waves in Wave Tracks . . . . . . . . . . . . . . . . . . . . 17
Streaming Versus In-Memory Playback . . . . . . . . . . . . 18
Tracks and Parts and Layers, Oh My . . . . . . . . . . . . . . 19
MIDI and DLS Collections . . . . . . . . . . . . . . . . . . . 21
Building DLS Collections . . . . . . . . . . . . . . . . . . 23
Stereo Waves in DLS Collections . . . . . . . . . . . . . 32
Using DLS Collections in Segments . . . . . . . . . . . 33
Authoring Flexible Content for Dynamic Playback . . . . . . . 36
Transitions . . . . . . . . . . . . . . . . . . . . . . . . . . 36

iii
Contents

Chapter 3 Variation . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Variable Playback/Variations . . . . . . . . . . . . . . . . . 41
Auditioning Variations in DirectMusic Producer . . . . . . . 44
Variation Shortcuts . . . . . . . . . . . . . . . . . . . . . . 45
Specifying Variation Behavior Settings . . . . . . . . . . . . . 46
Variation Switch Points . . . . . . . . . . . . . . . . . . . . 50
Influencing Variation Choices . . . . . . . . . . . . . . . . . 51
Tips and Further Options for Variability . . . . . . . . . . . . 53

Chapter 4 Interactive and Adaptive Audio . . . . . . . . . . . . . . . . 55

Variable Intensity Music and Ambience . . . . . . . . . . . . 55
DirectMusic Style Files . . . . . . . . . . . . . . . . . . . 55
Choosing a Pattern from a Style . . . . . . . . . . . . . 60
Building Up Styles . . . . . . . . . . . . . . . . . . . . 60
Using Styles with Segments . . . . . . . . . . . . . . . . . 61
Changing the Playing Patterns . . . . . . . . . . . . . . . 64
Segment-Based Branching and Layering . . . . . . . . . . 66
Branching Playback . . . . . . . . . . . . . . . . . . . 66
Layered Segments . . . . . . . . . . . . . . . . . . . . 68

Chapter 5 DirectMusic Producer . . . . . . . . . . . . . . . . . . . . . 75

Getting Started . . . . . . . . . . . . . . . . . . . . . . . . 75
Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Setting the Chord Reference. . . . . . . . . . . . . . . . . . 78
Creating a Primary Segment . . . . . . . . . . . . . . . . . 79
Making Our Music Cooler . . . . . . . . . . . . . . . . . . 80
Variations . . . . . . . . . . . . . . . . . . . . . . . . . . 82
Styles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
Creating Our First Style . . . . . . . . . . . . . . . . . . 84
Adding Groove Level. . . . . . . . . . . . . . . . . . . . 85

Chapter 6 Working with Chord Tracks and ChordMaps. . . . . . . . . . 87

Chord Tracks . . . . . . . . . . . . . . . . . . . . . . . . . 87
ChordMaps . . . . . . . . . . . . . . . . . . . . . . . . . 91
Chords for Composition . . . . . . . . . . . . . . . . . . . 96
PlayModes . . . . . . . . . . . . . . . . . . . . . . . . . . 98
Note Tweaking . . . . . . . . . . . . . . . . . . . . . . . 101
ChordLevels and Inversions . . . . . . . . . . . . . . . . . 101
Pattern Length and Chord Rhythm . . . . . . . . . . . . . . 103
More on Variation Choices: The Mother of All Windows . . . 104

Chapter 7 DirectX Audio Scripting . . . . . . . . . . . . . . . . . . . 107

Defining the Audio Implementation Problem . . . . . . . . . 107
Creating a Script . . . . . . . . . . . . . . . . . . . . . . 109
Scripting Languages . . . . . . . . . . . . . . . . . . . . . 112

iv
 Contents

Basic Scripting Usage . . . . . . . . . . . . . . . . . . . . 113

Playing a Piece of Audio . . . . . . . . . . . . . . . . . . . 114
Using Transition Segments . . . . . . . . . . . . . . . . . . 116
Stopping a Piece of Audio. . . . . . . . . . . . . . . . . 117
Global DirectMusic Parameters . . . . . . . . . . . . . . . 118
Editing Content while Using Scripts. . . . . . . . . . . . . . 119
Debugging Scripted Content . . . . . . . . . . . . . . . . . 119
Scripting with Variables . . . . . . . . . . . . . . . . . . . 122
Conditional Statements . . . . . . . . . . . . . . . . . . 123
Local Variables . . . . . . . . . . . . . . . . . . . . . . 125
A Two-Way Street: Variables Set by the Scripter . . . . . . 126
Tracking Individual Instances of a Playing Segment . . . 126
Triggering Segment Playback Relative to Playing
Segments . . . . . . . . . . . . . . . . . . . . . . . 127
Script Tracks and Self-Modifying Music . . . . . . . . . . 128
Using Scripting for Resource Management . . . . . . . . . . 129

Unit II: Programming with DirectX Audio

Chapter 8 DirectX Audio Programming 101. . . . . . . . . . . . . . . 135

DirectX Audio = DirectSound + DirectMusic . . . . . . . . . 135
DirectX Audio Building Blocks . . . . . . . . . . . . . . . . 138
Segment . . . . . . . . . . . . . . . . . . . . . . . . . 138
AudioPath . . . . . . . . . . . . . . . . . . . . . . . . 139
Performance . . . . . . . . . . . . . . . . . . . . . . . 139
Loader . . . . . . . . . . . . . . . . . . . . . . . . . . 139
COM. . . . . . . . . . . . . . . . . . . . . . . . . . . 139
HelloWorld! . . . . . . . . . . . . . . . . . . . . . . . . . 141
It’s a 3D World . . . . . . . . . . . . . . . . . . . . . . . 145
Using the CD . . . . . . . . . . . . . . . . . . . . . . . . 147

Chapter 9 The Loader . . . . . . . . . . . . . . . . . . . . . . . . . 149

Why the Loader? . . . . . . . . . . . . . . . . . . . . . . 149
Manage File Relationships . . . . . . . . . . . . . . . . 149
Provide a Convenient File Load API . . . . . . . . . . . . 150
Support Customization . . . . . . . . . . . . . . . . . . 151
Loading a File . . . . . . . . . . . . . . . . . . . . . . . . 151
SetSearchDirectory() . . . . . . . . . . . . . . . . . . . 152
LoadObjectFromFile() . . . . . . . . . . . . . . . . . . . 153
GetObject(). . . . . . . . . . . . . . . . . . . . . . . . 154
SetObject() . . . . . . . . . . . . . . . . . . . . . . . . 155
Managing the File List . . . . . . . . . . . . . . . . . . . . 156
ScanDirectory() . . . . . . . . . . . . . . . . . . . . . . 156
EnumObject(). . . . . . . . . . . . . . . . . . . . . . . 157
EnableCache() . . . . . . . . . . . . . . . . . . . . . . 157

v
Contents

ClearCache() . . . . . . . . . . . . . . . . . . . . . . . 157
ReleaseObject()/ReleaseObjectByUnknown() . . . . . . . . 158
Garbage Collection . . . . . . . . . . . . . . . . . . . . . 158
LoaderView Sample Application . . . . . . . . . . . . . . . 159
LoaderView Source . . . . . . . . . . . . . . . . . . . . . 161
Predefined Object Types . . . . . . . . . . . . . . . . . 162
CAudio Class. . . . . . . . . . . . . . . . . . . . . . . 162
Initialization . . . . . . . . . . . . . . . . . . . . . . . 164
SetObjectType() . . . . . . . . . . . . . . . . . . . . . . 165
GetObjectInfo() . . . . . . . . . . . . . . . . . . . . . . 166
ReleaseItem() . . . . . . . . . . . . . . . . . . . . . . . 167
ReleaseAll() . . . . . . . . . . . . . . . . . . . . . . . . 168
ChangeCacheStatus(). . . . . . . . . . . . . . . . . . . 168
ScanDirectory() . . . . . . . . . . . . . . . . . . . . . . 168
Loading from a Resource . . . . . . . . . . . . . . . . . . 169

Chapter 10 Segments . . . . . . . . . . . . . . . . . . . . . . . . . . 175

The Tao of Segment . . . . . . . . . . . . . . . . . . . . . 175
Anatomy of a Segment . . . . . . . . . . . . . . . . . . 177
How a Segment Loads . . . . . . . . . . . . . . . . . . 179
How a Segment Plays . . . . . . . . . . . . . . . . . . . 179
Performance Channels . . . . . . . . . . . . . . . . . . 180
Inter-Track Communication . . . . . . . . . . . . . . . . 181
Primary, Secondary, and Controlling Segments . . . . . . 182
Primary Segment. . . . . . . . . . . . . . . . . . . . 182
Controlling Segment . . . . . . . . . . . . . . . . . . 183
Secondary Segment . . . . . . . . . . . . . . . . . . 183
Playing Segments . . . . . . . . . . . . . . . . . . . . . . 184
SegmentState. . . . . . . . . . . . . . . . . . . . . . . 184
PlaySegmentEx() . . . . . . . . . . . . . . . . . . . . . 185
StopEx() . . . . . . . . . . . . . . . . . . . . . . . . . 187
Introducing Jones . . . . . . . . . . . . . . . . . . . . . . 188
Starting Jones . . . . . . . . . . . . . . . . . . . . . . 189
Loading a Segment . . . . . . . . . . . . . . . . . . . . 189
Setting Segment Control Parameters. . . . . . . . . . . . 190
Viewing the TimeLine . . . . . . . . . . . . . . . . . . . 191
Viewing Control Parameters. . . . . . . . . . . . . . . . 192
Jones Code . . . . . . . . . . . . . . . . . . . . . . . . . 193
Audio Data Structures. . . . . . . . . . . . . . . . . . . 193
Loading a Segment into Jones . . . . . . . . . . . . . . 196
Segment Options . . . . . . . . . . . . . . . . . . . . . 202
Mode . . . . . . . . . . . . . . . . . . . . . . . . . 202
Resolution . . . . . . . . . . . . . . . . . . . . . . . 204
Delay . . . . . . . . . . . . . . . . . . . . . . . . . 206
Time Signature. . . . . . . . . . . . . . . . . . . . . 208
Transition . . . . . . . . . . . . . . . . . . . . . . . 208

vi
 Contents

Aligned Cut In . . . . . . . . . . . . . . . . . . . . . 209

Invalidation . . . . . . . . . . . . . . . . . . . . . . 211
Use Path . . . . . . . . . . . . . . . . . . . . . . . . 212
Setting Segment Options . . . . . . . . . . . . . . . . . 213
Tracking Control Parameters . . . . . . . . . . . . . . . 217

Chapter 11 AudioPaths . . . . . . . . . . . . . . . . . . . . . . . . . 223

Anatomy of an AudioPath . . . . . . . . . . . . . . . . . . 224
AudioPath Performance Phase. . . . . . . . . . . . . . . 225
AudioPath DirectSound Phase . . . . . . . . . . . . . . . 227
AudioPath Configurations . . . . . . . . . . . . . . . . . . 229
AudioPath File Configuration . . . . . . . . . . . . . . . 229
Predefined Paths . . . . . . . . . . . . . . . . . . . . . 231
Working with AudioPaths . . . . . . . . . . . . . . . . . . 232
Downloading and Unloading with AudioPaths . . . . . . . 232
Playing on an AudioPath . . . . . . . . . . . . . . . . . 233
Embedded AudioPaths . . . . . . . . . . . . . . . . . . 234
Accessing Objects in an AudioPath . . . . . . . . . . . . 235
Setting the AudioPath Volume . . . . . . . . . . . . . . . 237
Shutting Down an AudioPath . . . . . . . . . . . . . . . 237
Adding AudioPaths to Jones . . . . . . . . . . . . . . . . . 238
Using AudioPaths in Jones . . . . . . . . . . . . . . . . 239
Jones AudioPath Structures . . . . . . . . . . . . . . . . 241
Creating an AudioPath . . . . . . . . . . . . . . . . . . 243
Getting AudioPath Stats . . . . . . . . . . . . . . . . . . 245
PChannel Count . . . . . . . . . . . . . . . . . . . . 245
Tool Count. . . . . . . . . . . . . . . . . . . . . . . 246
Buffer Count . . . . . . . . . . . . . . . . . . . . . . 247
DMO Count . . . . . . . . . . . . . . . . . . . . . . 248
Scanning an AudioPath . . . . . . . . . . . . . . . . . . 249
Editing AudioPath Object Parameters . . . . . . . . . . . 253
Accessing with AudioPathItemInfo. . . . . . . . . . . . 254
Accessing Buffer Parameters . . . . . . . . . . . . . . 254
Accessing 3D Parameters. . . . . . . . . . . . . . . . 256
Accessing Tools and DMOs. . . . . . . . . . . . . . . 257
Accessing Volume . . . . . . . . . . . . . . . . . . . 259
Performing with an AudioPath . . . . . . . . . . . . . . . 259

Chapter 12 Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . 261

Anatomy of a Script . . . . . . . . . . . . . . . . . . . . . 262
Routines . . . . . . . . . . . . . . . . . . . . . . . . . 263
Variables . . . . . . . . . . . . . . . . . . . . . . . . . 264
Content . . . . . . . . . . . . . . . . . . . . . . . . . 266
Finding the Routines and Variables . . . . . . . . . . . . 267
Error Handling . . . . . . . . . . . . . . . . . . . . . . 270
Script Tracks . . . . . . . . . . . . . . . . . . . . . . . 270

vii
Contents

Script Language . . . . . . . . . . . . . . . . . . . . . 271

Adding Scripting to Jones . . . . . . . . . . . . . . . . . . 272
Jones Data Structures . . . . . . . . . . . . . . . . . . . 273
Loading the Script . . . . . . . . . . . . . . . . . . . . 273
Enumerating and Calling Routines . . . . . . . . . . . . 276
Enumerating and Accessing Variables . . . . . . . . . . . 277
A Scripting Sample . . . . . . . . . . . . . . . . . . . . . 278
Bad Dream. . . . . . . . . . . . . . . . . . . . . . . . 278
Theme Music . . . . . . . . . . . . . . . . . . . . . . . 279
Setting Activity and Hysteria . . . . . . . . . . . . . . . . 280
Wasp Flies By. . . . . . . . . . . . . . . . . . . . . . . 281
Train . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Thinking . . . . . . . . . . . . . . . . . . . . . . . . . 282
Enough Already! . . . . . . . . . . . . . . . . . . . . . 283

Chapter 13 Sound Effects . . . . . . . . . . . . . . . . . . . . . . . . 285

Sound Effects Programming 101 . . . . . . . . . . . . . . . 286
Allocating a Sound Effects AudioPath . . . . . . . . . . . 286
Playing Sounds on the AudioPath . . . . . . . . . . . . . 288
Activating and Deactivating the AudioPath . . . . . . . . . 289
Controlling the 3D Parameters . . . . . . . . . . . . . . 289
Controlling the Listener . . . . . . . . . . . . . . . . . . 291
Getting Serious . . . . . . . . . . . . . . . . . . . . . . . 293
Minimize Latency . . . . . . . . . . . . . . . . . . . . . 293
Manage Dynamic 3D Resources . . . . . . . . . . . . . 296
Keep Music and Sound Effects Separate . . . . . . . . . . 298
Mingle . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
Separate Sound Effects and Music Environments . . . . . . 304
Volume Control for Sound Effects and Music. . . . . . . . 306
Dynamic 3D Resource Management . . . . . . . . . . . 307
CAudioPath . . . . . . . . . . . . . . . . . . . . . . 307
CAudio . . . . . . . . . . . . . . . . . . . . . . . . 310
CThing . . . . . . . . . . . . . . . . . . . . . . . . 313
CThingManager . . . . . . . . . . . . . . . . . . . . 318
Mingle Implementation. . . . . . . . . . . . . . . . . 324
Low-Latency Sound Effects . . . . . . . . . . . . . . . . 326
Background Ambience . . . . . . . . . . . . . . . . . . 327
Avoiding Truncated Waves . . . . . . . . . . . . . . . . 327
Manipulating the Listener Properties . . . . . . . . . . . . 330
Scripted Control of Sound Effects . . . . . . . . . . . . . 332
Creating Music with the Composition Engine . . . . . . . 334

Chapter 14 Notifications . . . . . . . . . . . . . . . . . . . . . . . . 339

Performance Notifications . . . . . . . . . . . . . . . . . . 340
Notification Categories . . . . . . . . . . . . . . . . . . 340
The Notification Pmsg . . . . . . . . . . . . . . . . . . 341

viii
 Contents

Enabling a Notification . . . . . . . . . . . . . . . . . . 342

Receiving a Notification . . . . . . . . . . . . . . . . . . 343
Polling for Notifications . . . . . . . . . . . . . . . . 343
Waiting on an Event . . . . . . . . . . . . . . . . . . 343
Capturing Notifications with a Tool . . . . . . . . . . . 344
Interpreting the Notification . . . . . . . . . . . . . . . . 346
Lyric Notifications . . . . . . . . . . . . . . . . . . . . . . 346
Receiving a Lyric . . . . . . . . . . . . . . . . . . . . . 347
Interpreting the Lyric . . . . . . . . . . . . . . . . . . . 347
Case Study: Begonia’s Amazing Dance Moves . . . . . . . 348
Script Routines . . . . . . . . . . . . . . . . . . . . . . . 351
Notifications in Jones . . . . . . . . . . . . . . . . . . . . 351
Jones Implementation . . . . . . . . . . . . . . . . . . . . 353
Enabling Notifications . . . . . . . . . . . . . . . . . . 353
Capturing Notifications . . . . . . . . . . . . . . . . . . 357
Retrieving Notifications . . . . . . . . . . . . . . . . . . 361
CAudio Integration . . . . . . . . . . . . . . . . . . . . 361
Case Study: Return to Begonia’s Amazing Dance Moves . . 364

Unit III: DirectX Audio Case Studies

Chapter 15 A DirectMusic Case Study for Russian Squares for

Windows XP Plus Pack . . . . . . . . . . . . . . . . . . . . 369
Introduction and Overview. . . . . . . . . . . . . . . . . . 369
Adaptive Elements . . . . . . . . . . . . . . . . . . . . 370
Music Cells . . . . . . . . . . . . . . . . . . . . . . 370
Variation. . . . . . . . . . . . . . . . . . . . . . . . 370
Music Layers . . . . . . . . . . . . . . . . . . . . . . 371
Cooperative Composing . . . . . . . . . . . . . . . . 371
Creating the Adaptive Audio Design . . . . . . . . . . . . . 371
Russian Squares: How the Game Plays . . . . . . . . . . 371
Aural and Visual Themes . . . . . . . . . . . . . . . . . 372
The Musical Form . . . . . . . . . . . . . . . . . . . . . . 372
The Adaptive Goal and Core Objectives . . . . . . . . . . . 374
Inherent Obstacles . . . . . . . . . . . . . . . . . . . . 375
The DirectMusic Solution. . . . . . . . . . . . . . . . . . . 376
Harmonic Considerations . . . . . . . . . . . . . . . . . . 378
The Production Process . . . . . . . . . . . . . . . . . . . 378
Gathering, Creating, and Assembling DLS Banks . . . . . 379
Setting Up Styles . . . . . . . . . . . . . . . . . . . . . 380
The Band and Mixing . . . . . . . . . . . . . . . . . . . . 380
AudioPaths and Effects . . . . . . . . . . . . . . . . . . 381
Pattern Creation and Groove Level Assignments . . . . . . 382
Continuous Controllers . . . . . . . . . . . . . . . . . . . 382
Creating DirectMusic Embellishments: Intros, Ends, and Breaks 383
Segments . . . . . . . . . . . . . . . . . . . . . . . . . . 384

ix
Contents

Motifs and Secondary Segments. . . . . . . . . . . . . . 384

Testing, Delivery, and Integration . . . . . . . . . . . . . 385

Chapter 16 A DirectMusic Case Study for No One Lives Forever . . . . . 387

An Overview of the Score . . . . . . . . . . . . . . . . . . 387
The Adaptive Concept and Music States . . . . . . . . . . 388
Music Sets . . . . . . . . . . . . . . . . . . . . . . . . 389
Transitions: Getting from A to B and Back Again . . . . . . 390
The Matrix . . . . . . . . . . . . . . . . . . . . . . . . 390
Performance Boundaries . . . . . . . . . . . . . . . . . 391
Stinger Motifs. . . . . . . . . . . . . . . . . . . . . . . 392
Sounds and DLS Banks . . . . . . . . . . . . . . . . . . 392
Integration and Implementation . . . . . . . . . . . . . . 393
Scripting . . . . . . . . . . . . . . . . . . . . . . . . . 393
The Test Player . . . . . . . . . . . . . . . . . . . . . . 394
DirectMusic and the Production Process . . . . . . . . . . . 394
The Prototype and Pre-production . . . . . . . . . . . . . 394
Composing and Sequencing . . . . . . . . . . . . . . 395
DLS Creation . . . . . . . . . . . . . . . . . . . . . 395
The DLS Editor . . . . . . . . . . . . . . . . . . . . . 397
Styles, Patterns, Bands, and Segments:
How They Work Together. . . . . . . . . . . . . . . . 398
Style and Band Setup . . . . . . . . . . . . . . . . . 399
Pattern Creation and Organization . . . . . . . . . . . 400
Instrument-level Variation . . . . . . . . . . . . . . . 401
Composing with the Pattern Editor . . . . . . . . . . . 402
Continuous Controllers . . . . . . . . . . . . . . . . . . 402
Segment Creation . . . . . . . . . . . . . . . . . . . 403
Motif Creation . . . . . . . . . . . . . . . . . . . . . 405
Creating Transition Segments. . . . . . . . . . . . . . 406
The LithTech Player/Tester . . . . . . . . . . . . . . . . . . 408
Preparing DirectMusic Files for the Game . . . . . . . . . 408
Using the Script. . . . . . . . . . . . . . . . . . . . . . 409
Setting Up Music States in the Script . . . . . . . . . . . . 409
The Transition Matrix . . . . . . . . . . . . . . . . . . . 410
Motifs and Secondary Segments. . . . . . . . . . . . . . 410
Using the LithTech Test Player . . . . . . . . . . . . . . 411
Integration and Implementation . . . . . . . . . . . . . . . 412
Integration of DirectMusic Technology . . . . . . . . . . 412
Implementation of the Music Content . . . . . . . . . . . 412
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 413
Addendum: A Guide to the NOLF Media Files . . . . . . . . 414
NOLF Quicktime Videos . . . . . . . . . . . . . . . . . 414
Ambush Music States Audio File . . . . . . . . . . . . . . 415

x
 Contents

Chapter 17 A DirectSound Case Study for Halo . . . . . . . . . . . . . 417

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 417
Audio the Bungie Way . . . . . . . . . . . . . . . . . . 417
History of Halo Audio . . . . . . . . . . . . . . . . . . . 418
Coming to the Xbox . . . . . . . . . . . . . . . . . . . 419
Production . . . . . . . . . . . . . . . . . . . . . . . . . 421
Music . . . . . . . . . . . . . . . . . . . . . . . . . . 421
SFX . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
Surround Sound . . . . . . . . . . . . . . . . . . . . . 426
Attaching Sounds to Animations. . . . . . . . . . . . . . 426
Real-time Tracking of Velocity and Pitch . . . . . . . . . . 427
Cascading Sound Effects . . . . . . . . . . . . . . . . . 428
Dialog . . . . . . . . . . . . . . . . . . . . . . . . . . 428
The Final Mix . . . . . . . . . . . . . . . . . . . . . . . 430
What Worked, What Didn’t, and the Future of Both . . . . . . 430
The Good . . . . . . . . . . . . . . . . . . . . . . . . 430
The Not So Good. . . . . . . . . . . . . . . . . . . . . 431
In Closing. . . . . . . . . . . . . . . . . . . . . . . . . . 432

Chapter 18 A DirectMusic Case Study for Interactive Music on the Web . . 433
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 433
Background . . . . . . . . . . . . . . . . . . . . . . . 433
The Wide Sounds DMX Control . . . . . . . . . . . . . . 436
Functionality . . . . . . . . . . . . . . . . . . . . . . . 436
The Web Site . . . . . . . . . . . . . . . . . . . . . . . . 438
The Adaptive Audio Design . . . . . . . . . . . . . . . . . 439
Style and Aesthetics — What Is Right for the Site? . . . . . 439
Form and Function . . . . . . . . . . . . . . . . . . . . 440
Finding the Hooks . . . . . . . . . . . . . . . . . . . 440
Matching Music to Site Content . . . . . . . . . . . . . 441
Thinking as the User . . . . . . . . . . . . . . . . . . 441
Content Creation . . . . . . . . . . . . . . . . . . . . . . 441
DLS Instrumentation . . . . . . . . . . . . . . . . . . . 442
Sample Format and Compression . . . . . . . . . . . . . 442
Instrument Choices . . . . . . . . . . . . . . . . . . . . 443
The Styles and their Patterns. . . . . . . . . . . . . . . . 444
Styles for Each Content Zone . . . . . . . . . . . . . . . 445
Pattern Structure . . . . . . . . . . . . . . . . . . . . . 445
Groove Levels . . . . . . . . . . . . . . . . . . . . . . 446
The Primary Segments . . . . . . . . . . . . . . . . . . 447
Chord Tracks and ChordMaps . . . . . . . . . . . . . . 448
The Secondary Segments . . . . . . . . . . . . . . . . . 450
AudioPaths and Effects . . . . . . . . . . . . . . . . . . 451
Testing the Functionality . . . . . . . . . . . . . . . . . . 452
Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . 453
Creating the Script . . . . . . . . . . . . . . . . . . . . 453

xi
Contents

Creating an AudioPath Object . . . . . . . . . . . . . . 455

Working with Variables . . . . . . . . . . . . . . . . . . 456
Delivery . . . . . . . . . . . . . . . . . . . . . . . . . 458
Packaging of Run-time Files . . . . . . . . . . . . . . . . 458
Implementation . . . . . . . . . . . . . . . . . . . . . . . 459
Putting It All Together . . . . . . . . . . . . . . . . . . . 460
The “Control” Object . . . . . . . . . . . . . . . . . 460
Attaching Triggers to Anchor Tags . . . . . . . . . . . . . 461
Test, Test, Test! . . . . . . . . . . . . . . . . . . . . . . 462

Chapter 19 A DirectMusic Case Study for Worms Blast . . . . . . . . . . 463

Game Overview. . . . . . . . . . . . . . . . . . . . . . . 463
Music Preparation . . . . . . . . . . . . . . . . . . . . . . 465
Making the Motifs Fit the Background Music . . . . . . . . . 466
Samples and Instruments . . . . . . . . . . . . . . . . . . 466
The Project Structure . . . . . . . . . . . . . . . . . . . . . 467
Using Groove Levels to Set the Mood . . . . . . . . . . . . 468
Motifs for Minor Game Events . . . . . . . . . . . . . . . . 470
Motifs for Major Game Events . . . . . . . . . . . . . . . . 470
Things I Would Have Done Differently . . . . . . . . . . . . 471
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 472

Chapter 20 A DirectMusic Case Study for Asheron’s Call 2:

The Fallen Kings. . . . . . . . . . . . . . . . . . . . . . . 473
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 473
Music for Massively Multiplayer Games . . . . . . . . . . 473
Project Goals . . . . . . . . . . . . . . . . . . . . . . . 475
Repetition . . . . . . . . . . . . . . . . . . . . . . . 475
Adaptive. . . . . . . . . . . . . . . . . . . . . . . . 475
Interactive . . . . . . . . . . . . . . . . . . . . . . . 476
Informative . . . . . . . . . . . . . . . . . . . . . . 476
Code Budget . . . . . . . . . . . . . . . . . . . . . . . 476
Project Overview . . . . . . . . . . . . . . . . . . . . . . 477
Style and Feel . . . . . . . . . . . . . . . . . . . . . . 477
Time Feel. . . . . . . . . . . . . . . . . . . . . . . . . 478
The Stack. . . . . . . . . . . . . . . . . . . . . . . . . 479
Background Score . . . . . . . . . . . . . . . . . . . . 480
Monster Melodies. . . . . . . . . . . . . . . . . . . . . 481
Player Melodies. . . . . . . . . . . . . . . . . . . . . . 481
Dimensions of Musical Space . . . . . . . . . . . . . . . 482
Expanding the Stack . . . . . . . . . . . . . . . . . . . 483
Modes and Chords . . . . . . . . . . . . . . . . . . . . 483
Groove Level . . . . . . . . . . . . . . . . . . . . . . . 485
Technical Considerations . . . . . . . . . . . . . . . . . . 486
Project Organization . . . . . . . . . . . . . . . . . . . 486
Optimizing Patch Changes . . . . . . . . . . . . . . . . 487

xii
 Contents

Optimizing Voices and DLS Banks . . . . . . . . . . . . . 488

Cheap Stereo Separation . . . . . . . . . . . . . . . . . 488
Note Scattering . . . . . . . . . . . . . . . . . . . . . . 489
Evolving Patches . . . . . . . . . . . . . . . . . . . . . 489
Custom Code . . . . . . . . . . . . . . . . . . . . . . 489
DirectMusic Script. . . . . . . . . . . . . . . . . . . . . 490
Postmortem . . . . . . . . . . . . . . . . . . . . . . . . . 493
What Worked. . . . . . . . . . . . . . . . . . . . . . . 494
Player Music as a Purely Social Game . . . . . . . . . 494
Layered Approach . . . . . . . . . . . . . . . . . . . 494
Music as Sound Design . . . . . . . . . . . . . . . . 495
Music Is Informative . . . . . . . . . . . . . . . . . . 495
DirectMusic . . . . . . . . . . . . . . . . . . . . . . 495
What Didn’t Work . . . . . . . . . . . . . . . . . . . . 496
Not What People Expect from an “Interactive”
Music System . . . . . . . . . . . . . . . . . . . . . 496
Learning DirectMusic while Creating the Project . . . . . 496
Lack of Support . . . . . . . . . . . . . . . . . . . . 496
Future Work . . . . . . . . . . . . . . . . . . . . . . . 497
On the CD . . . . . . . . . . . . . . . . . . . . . . . . . 498

Chapter 21 Beyond Games: Bringing DirectMusic into the Living Room . . 501
From “Sound Capture” to “Music Production…” . . . . . . . 502
“…to Nonlinear Music Format” . . . . . . . . . . . . . . . 505
Nonlinear Music Design . . . . . . . . . . . . . . . . . . . 508
Role . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
Functionality . . . . . . . . . . . . . . . . . . . . . . . 511
Granularity . . . . . . . . . . . . . . . . . . . . . . . . 512
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . 514

About the Authors . . . . . . . . . . . . . . . . . . . . . . 517

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523

xiii
This page intentionally left blank.
Foreword

The development of today’s video games is an increasingly complex

and dynamic undertaking. Not only do massive amounts of assets
come crashing together almost magically to make a game, but some-
where in this chaos, all of this “magic” has to be fun for the game
player. Herein lies the proverbial rub. We apply remarkable efforts to
make games exhilarating, challenging, and entertaining, but as both
our technology and skills improve, so does our appetite for yet big-
ger and better games. In this apparently never-ending quest, a grim
reality will eventually rear its ugly head — some of us will be left
behind.
Knowledge is an incredibly powerful thing. The consummate
beauty of the games industry, unlike any other trade that I’ve wit-
nessed, is our willingness to share it with each other. The yearly
Game Developers Conference is a prime example of this grand phe-
nomenon, where “secrets” are available for the asking. But why on
earth would developers who compete directly against each other in
this high-stakes venture consider such a thing?
Legitimacy of our art is why. For reasons unbeknownst to us
mere mortals, even though we are a multi-billion-dollar a year indus-
try, we are still considered the “bastard children” of entertainment.
Perhaps because we appear to be so different from “credible” profes-
sions, it is the subconscious need to prove them wrong that
ultimately drives our exchange of knowledge.
When I was writing The Complete Guide to Game Audio, a regular
comment from established game audio professionals was that in
doing so I would be training the competition. My answer was always,
“Yes, it would.” Yes, it gave away valuable insight into the mysteri-
ous underworld of game audio. Yes, it gave away years of experience
from established craftsmen. Yes, it invited new blood into the circle.
Yes, yes, yes! It did all of those things and more, just as this book,

xv
Foreword

DirectX 9 Audio Exposed: Interactive Audio Development, will. How-

ever, despite the anxiety among us about being overrun by those
looking for new opportunities, nobody complained that this was a bad
thing for the industry. In fact, the continuing trend seems to make
access to the most intimate of details even easier. The free flow of
wisdom is seen by most as the key to improvement and future
advancements of our trade.
This theory is attestable by the formation of several organiza-
tions that not only serve as breeding grounds for new ideas and
efforts of standardization in the industry but as a place where anyone
can ask a question and actually get an answer. Game audio specialists
can tap into a wealth of knowledge at the Game Audio Network Guild
(www.audiogang.org), the Interactive Audio Special Interest Group
(IA-SIG) (www.iasig.org), the Audio Engineering Society (AES)
Game Audio Committee, and even at the yearly Project Bar-B-Q
(www.projectbarbq.com). Or, if you prefer, you can go right to the
source to find your answers.
Console manufacturers also understand this and are getting
involved. Microsoft recently formed the Xbox Registered Content
Creators program, which enables audio makers like us to get
answers directly from them. As we work on game projects for the
Xbox, it’s a great feeling to know someone is available to help us uti-
lize the system’s full audio capabilities (which in turn makes us look
good) and to also be there in case we fall (which also makes us look
good by showing our resourcefulness to our employers and clients).
There is hope that the other platform makers will follow suit.
There are, however, still many wide gaps in the knowledge con-
tinuum that need to be filled in for those of us who make and
implement game audio. Even with our gargantuan efforts at stan-
dardization, there will always be different application techniques that
solve their own specific problems and present audio in unique ways.
There is no getting around this type of individualism or efforts at
creating the ultimate problem solver, and for the overall good of
game creation, there is no need to stifle such attempts either.
Instead, we make the knowledge available so that not only can we
utilize every feature to the fullest extent, but so that someone even
smarter than us can use it as yet another stepping-stone to create

xvi
 Foreword

the next original application. Some of these will eventually become

the heart of game audio management and presentation.
DirectX is a prime example of an application that has been pol-
ished and honed over the years to become an absolute necessity in
today’s game creation. DirectX Audio has developed into the defini-
tive collection of tools and technologies that provide programmers
and audio content creators with the ability to create interactive,
adaptive, and dynamic audio, which was basically unheard of just a
few short years ago. It is rapidly nearing the point of absolute
necessity.
What better way to learn and absorb the key features of a subject
such as DirectX Audio than directly from the creators and users of
this technology? Todd M. Fay (the infamous LAX) has done a
remarkable job bringing together a cohesive team of these experts to
present all the finer details and subtleties that this application has to
offer. By reading what these leading authorities on DirectX Audio
have to say, you will be sharing in the ultimate flow of knowledge,
which will directly translate into increased impact and overall poten-
tial of your game audio. Better yet, your quest will be all the more
triumphant because you won’t be the one getting left behind.
I wish you much luck in your future gaming endeavors, and I
look forward to hearing what you’ve done with your newfound
wisdom.
Work hard. Make us proud.
— Aaron Marks
Game composer/sound designer, On Your Mark Music Productions
Author, The Complete Guide to Game Audio

xvii
This page intentionally left blank.
Acknowledgments

Peter Walsh for inspiring me to produce this book in the first place!
Scott Selfon and Todor and Melissa Fay, for their enormous contribu-
tions to this project. Jason Booth, Scott Morgan, Guy Whitmore,
Bjorn Lynne, Jay Weinland, Marty O’Donnell, Ciaran Walsh, and
Tobin Buttram for lending their amazing insight and authorship to
this book. Jim Hill, Wes Beckwith, Beth Kohler, and the rest of the
team at Wordware for their hard work. Dugan Porter, Scott Snyder,
Shane Kneip, Andrew Barnabas, Mike Verrette, and David Yackley
for advising on this project. The members, sponsors, and directors of
G.A.N.G. for their dedication to excellent interactive audio. Kent and
Kim Quirk for giving me my first title to score. Tommy Tallarico,
Aaron Marks, Laddie Ervin, and Jack Wall for their mentorship and
direction, both personally and professionally. Gary Gottlieb for allow-
ing me to research game audio at UMass. Chuck Schuldiner (R.I.P.),
Claude Debussy, Iron Maiden, BT, and the dozens of brilliant game
composers whose work I’ve enjoyed over the years for their musical
inspiration. Stephen and Kristina Guberski, Kevin Callahan, Rich
Borgatti Jr., the Murphy family, Elizabeth Steffen, Katie Valinch,
Brian Redmond, Greg Capolino, and Ted Regulski for their endless
encouragement. Oh, and Tom Russo for being from Boston.
And to my mother, father, brothers, friends, and family for believ-
ing in all of my crazy ideas (or at least pretending to…)
Cheers!
— Todd M. Fay (aka LAX)

xix
This page intentionally left blank.
Introduction

by Todd M. Fay
DirectMusic is software that allows composers, sound designers, and
audio engineers to create digital musical content with interactive,
variable, and adaptive playback properties. What does that mean?
Moreover, who cares? Chances are that Joe Musician is not terribly
familiar with the terms “interactive,” “variable,” and “adaptive” as
they relate to audio production — and for good reason; musical
pieces featuring these properties have yet to enter the musical main-
stream. Well, we are here to change all of that…

DirectMusic in Interactive Media Development

(aka Computer/Video Games)
While not familiar terms in mainstream music production or motion
picture post, interactive, variable, and adaptive audio playback are
mainstay terms in the world of game audio production. Games, like
other visual media, use audio as a means of communicating informa-
tion and manipulating audience emotion. However, there are some
pitfalls inherent to the medium that audio producers need to deal
with, namely nonlinearity, variable environments, repetition, storage
space, and processing power limitations. These potential roadblocks
require unique creative and technical solutions. DirectMusic exists,
in part, to assist audio producers in dealing with these unique
problems.
DirectMusic is part of DirectX Audio, the aural components of a
suite of software tools (or “library”) collectively referred to as
DirectX. Microsoft created DirectX to give programmers powerful

xxi
Introduction

tools to create games for Windows PCs. And they succeeded;

DirectX made Windows the premier PC platform for games. DirectX
Audio consists of two component parts named DirectMusic and
DirectSound. At first, DirectX had only the DirectSound library. Pro-
grammers used DirectSound for basic playback of a game’s audio
assets. DirectMusic was introduced later on to give composers and
sound designers more control over their contribution to the game.
Today, the two are referred to collectively as DirectX Audio. We
focus mainly on DirectMusic in this text because it is easier to work
with from the perspective of a composer and sound designer and is
in effect more powerful. Just understand that DirectSound came into
existence before DirectMusic, and DirectMusic was built, in part, on
top of DirectSound. This is already more than you need to know, so
let’s avoid entering into anymore details here, as there is a book on
the subject (the one you’re holding).

Greater Control for Audio Producers

In the early days of game development, audio producers were largely
dependent upon audio programmers (software engineers) to get
their sounds and music into their game, a process known as integra-
tion. DirectMusic relieves much of this dependency. Using Direct-
Music Producer, an audio producer can place all of his or her waves,
sequence data, sample sets, and even instructions on how audio
should be played interactively, adaptively, and with variation. The
sum of the audio producer’s work is saved in a file called a Segment
(.sgt files). A Segment is to an adaptive, interactive, and variable
score as a wave and/or MIDI file is to a linear score. Because of the
completeness of the information and data stored in a DirectMusic
Segment, the role of the programmer in integrating is reduced signif-
icantly. Gone are the days of a project’s audio programmer screwing
with volume levels or triggering sounds to the wrong events. The
programmer doesn’t need to know how the audio gets played back;
he only tells the game to play the piece of music, and whatever audio
data and parameters were specified by the composer are used at
playback.

xxii
 Introduction

This is useful for audio producers creating stand-alone music

with DirectMusic. They can save all of their samples, waves,
sequence data, and instructions for adaptation, variation, and
interactivity (if any) in the Segment. The listener then loads the file
into their DirectMusic player program, and voílà — beautiful music!
There will, of course, always be some amount of information that
programmers and composers must communicate to each other. Pro-
grammers must let the composer know how much memory he can
use for sample, wave, and sequence data or whether he can stream
wave data off of the hard disk or CD/DVD. The programmer gener-
ally needs to know how to transition from one piece of music to the
next (measure, beat, marker boundaries, and so on). The program-
mer typically implements things like tempo changes based on game
state. However, a composer (or programmer) can instead take advan-
tage of the DirectX Audio Scripting language for greater control over
these aspects of playback from the programmer. A scripting language
provides the audio producer with the ability to create simple
easy-to-read “programs” (known as routines) that a programmer can
then run based on programmatic events. The language is designed to
be simple to learn but still have a lot of flexibility and power in what
it can do for audio producers. We discuss this more later.

DirectMusic for Music Production

The applications for interactive, variable, and adaptive audio, particu-
larly music, extend beyond computer and video games. The potential
for DirectMusic to alter the way we listen to digital music in our
homes, cars, and anywhere else where we can smuggle a personal
musical player is very exciting. Music written with interactivity,
adaptability, and dynamism allows audiences to take a more active
role in their listening experiences. A recording artist’s next single
could be a DirectMusic module that listeners can intuitively remix
with, introduce new musical parts and effects to, or set to react to
changes in their computing environment. It could even be made to
never play back the same way twice, if listeners so chose!
Admittedly, not everyone is interested in having such an active
role in music listening. Passive listeners can still enjoy enhanced
musical experiences by setting their DirectMusic songs to play back
according to packaged templates. “Factory settings” could provide

xxiii
Introduction

alternate instrumentation, allowing a jazz piece to take on more of a

world, rock, or hip-hop feel for instance, even switching back and
forth between styles and instrument sets. Other templates could
range from intelligent “concert hall” or “arena” reverberation set-
tings using built-in effects processing to completely self-generated
performances based on a loose set of criteria created from scratch
with every play.
There are plenty of tools on the market that allow just about any-
one with a few hundred bucks to begin creating music on his
computer. The differences between your average audio editing or
multitracking program and DirectMusic are the interactive, adaptive,
and variable abilities that you will not find anywhere else (at least
not to this degree). Remember, these kinds of soundtracks, songs,
and soundscapes are for the most part unexplored territory. We
assure you, the terrain in this territory is unlike any you’ve experi-
enced before, either as a composer, a performer, an engineer, or a
listener.
At the time of this book’s publication, Microsoft wasn’t promot-
ing DirectMusic as a mainstream music production tool. DirectMusic
was virtually unknown to the millions of music-producing hobbyists
and professionals looking for new ways to distinguish themselves
among their peers and audiences. Only a few members of a small
community were using DirectMusic for audio production outside of
games. Our solution to bringing interactive, adaptive, and variable
music to the masses? Educate audio producers on how to use
DirectMusic to create new sonic and musical experiences. The more
DirectMusic songs created, the more that people can begin to exper-
iment and appreciate these new musical experiences.
DirectMusic is the chosen platform for this neo-musical produc-
tion crusade for three reasons. First, its depth. There is not, to our
knowledge, any other interactive/adaptive audio solution as compre-
hensive as DirectMusic. Secondly, its reach. Anyone with a Windows
machine and DirectX 6.1 or higher can enjoy music created for
DirectMusic. Oh, and number three: It’s free! We wish we could add
“ease and convenience of use” to our list of reasons to love
DirectMusic, but if that were the case, you would not have much use
for this fine text! DirectMusic Producer (the program Microsoft
gives you to create DirectMusic content) is not user-friendly

xxiv
 Introduction

production software. In all fairness to the designers at Microsoft,

DirectMusic Producer (DMP) isn’t commercial software, and it was
never meant to be. It’s a bit convoluted as an interface, and it cer-
tainly isn’t pretty to look at. Add a slew of alien concepts in audio
production, and you have a significant barrier to entry. Up until now,
there wasn’t even a book on how to get around in DMP, let alone
DirectMusic. The good news is that since DMP is simply an inter-
face for producers to work with DirectMusic, someone somewhere
could build a more user-friendly program to replace DMP, without
sacrificing any of DirectMusic’s power. Of course, it’s pure specula-
tion as to if that may ever happen… In the meantime, DMP gets the
job done. Heck, we often rely on another sequencer/multitrack and
editing software package for the bulk of our production work — only
using DMP when we want to apply some of DirectMusic’s unique
functionality to our productions (dynamism, interactivity, and/or
adaptability).
Without a doubt, the single biggest drawback to producing music
using DirectMusic is the lack of a customizable, standardized, and
widely available player program. See, DirectMusic files are special in
that they cannot just be opened in Windows Media Player, Real
Player, Winamp, or whatever other music player people use day in
and day out. They have to be played back using a program specially
written to play DirectMusic files. Truth be told, WMP, Winamp, Real
Player, and their peers could support DirectMusic files, but they just
don’t. Maybe someday they will, but for now DirectMusic playback
options are limited to launching DirectMusic Producer or writing
your own player. Luckily for all of us, Mr. Todor Fay (not to be con-
fused with Todd M. Fay!) decided to create Jones. Jones is a
DirectMusic player. You can load a DirectMusic file into it and play it
back. Jones is free, and you can distribute it as you see fit! So get it
out to your audience so they can listen to your new musical cre-
ations! There is more on using Jones later…
Excited? Good! Read this book and dig into what DirectMusic
has to offer you! The wisdom in these pages and the software offer
you a grand opportunity for you to distinguish your art from that of
your strictly linear audio-producing peers.

xxv
Introduction

DirectX Audio for Software Engineers

This is short and sweet. DirectX 9 Audio Exposed: Interactive Audio
Development contains a comprehensive tutorial-style look at the
inner workings of the DirectX Audio libraries. Everything you’d like
to know about the DirectMusic API is within these pages. Todor Fay,
the originator of the cornerstone DirectMusic technology, has laid
out much of his wisdom for all of us to learn from. So take a look at
the examples and discussions and get some ideas about how you can
build your own software applications upon the DirectX Audio back-
bone. We look forward to seeing/hearing/using what you come up
with!

xxvi
Unit I

Producing Audio
with DirectMusic

Chapter 1: DirectMusic Concepts

Chapter 2: Linear Playback
Chapter 3: Variation
Chapter 4: Interactive and Adaptive Audio
Chapter 5: DirectMusic Producer
Chapter 6: Working with Chord Track and ChordMaps
Chapter 7: DirectX Audio Scripting

1
This page intentionally left blank.
 Chapter 1

DirectMusic Concepts

D irectMusic and DirectMusic Producer (the tool used to author

music and soundscapes for DirectMusic) introduce some new
concepts in audio production. Learn to appreciate what follows, and
you will see that there is a new musical world waiting to be con-
quered. But first, a quick explanation of terms… .
We use the term “audio producer” throughout this book. An
audio producer is anyone who creates audio to be used as part of a
DirectMusic project. This can be a sound designer, producer, record-
ing engineer, composer; we don’t care. If your focus is creating audio
to be placed in a project, you are an audio producer. If your job is
focused on integrating sounds into a project using code, building
tools/extensions to DirectX Audio, or developing playback software,
or you are a game programmer, then you should perk up when you
see the term “programmer.”

Interactivity
“Interactivity” is both the most overused and misused term in game
audio. Often, when someone speaks of interactive audio, he is refer-
ring either to adaptive audio (audio that changes characteristics, such
as pitch, tempo, instrumentation, etc., in reaction to changes in game
conditions) or to the art and science of game audio in general. A
better definition for interactive audio is any sound event resulting
from action taken by the audience (or a player in the case of a game
or a surfer in the case of a web page, etc.). When a game player
presses the “fire” button, his weapon sounds off. The sound of the
gun firing is interactive. If a player rings a doorbell in a game world,
the ringing of the bell is also an interactive sound. If someone rolls

3
4 n Chapter 1

the mouse over an object on a web site and triggers a sound, that
sound is interactive. Basically, any sound event, whether it be a
one-off, a musical event, or even an ambient pad change, if it comes
into play because of something the audience does (directly or indi-
rectly), it is classified as interactive audio.

Variability
An interesting side effect of the performance of recorded music pro-
duction is the absence of variation in playback. Songs on a CD play
back the same every single time. Music written to take advantage of
DirectMusic’s variability properties can be different every time it
plays. Variation is particularly useful in producing music for games,
since there is often a little bit of music that needs to stretch over
many hours of gameplay. Chapter 21 discusses applications of varia-
tion in music production outside the realm of games.
The imperfection of the living, breathing musician creates the
human element of live musical performances. Humans are incapable
of reproducing a musical performance with 100 percent fidelity.
Therefore, every time you hear a band play your favorite song, no
matter how much they practice, it differs from the last time they
played it live, however subtle the differences. Repetition is per-
ceived as being something unnatural (not necessarily undesirable,
but unnatural nonetheless) and is easily detectable by the human ear.
Variability also plays a role in song form and improvisation. Again,
when a song is committed to a recording, it has the same form and
solos every time someone plays that recording. However, when per-
formed by musicians at a live venue, they may choose to alter the
form or improvise in a way that is different from that used in the
recording.
DirectMusic allows a composer to inject variability into a pre-
pared piece of digital audio, whether a violin concerto or an audio
design modeled to mimic the sounds of a South American rain forest.
Composers and sound designers can introduce variability on differ-
ent levels, from the instrument level (altering characteristics such as
velocity, timbre, pitch, etc.) to the song level (manipulating overall
instrumentation choices, musical style choices, song form, etc.).
Using DirectMusic’s power of variability, composers can create
 DirectMusic Concepts n 5

stand-alone pieces of music that reinvent themselves every time the

listener plays them, creating a very different listening experience
when juxtaposed to a mixed/mastered version of the same music.
I
Composers alter the replay value of their compositions as well by

Unit
allowing their music to reinvent itself upon every listening session.
Avoidance of audio content repetition in games is often impor-
tant. When asked about music for games, someone once said, “At no
time in history have so few notes been heard so many times.” Repe-
tition is arguably the single biggest deterrent to the enjoyment of
audio (both sound effects and music) in games. Unlike traditional lin-
ear media like film, there are typically no set times or durations for
specific game events. A “scene” might take five minutes in one
instance and hours in another. Furthermore, there is no guarantee
that particular events will occur in a specific order, will not repeat, or
will not be skipped entirely. Coupled with the modest storage space
(also known as the “footprint”) budgeted for audio on the media and
in memory, this leaves the audio producer in a bit of a quandary. For
the issue of underscore, a game title with hours of gameplay might
only be budgeted for a few minutes worth of linear music. The audio
director must develop creative ways to keep this music fresh —
alternative versions, version branching, and so on. Audio program-
mers can investigate and implement these solutions using
DirectMusic.
As to events triggering ambience, dialog, and specific sound
effects, these may repeat numerous times, adding the challenge of
avoiding the kind of obvious repetition that can spoil a game’s real-
ism for the player. As we discuss in more detail, DirectMusic
provides numerous methods for helping to avoid repetition. On the
most basic level, audio programmers can specify variations in pitch
and multiple versions of wave, note, and controller data. Even game
content (specific scripted events in the game for instance) can spec-
ify orderings for playback (such as shuffling, no repeats, and so on)
that DirectMusic tracks as the game progresses. Using advanced
features, chord progressions can maintain numerous potential pro-
gression paths, allowing a limited amount of source material to
remain fresh even longer.
6 n Chapter 1

Adaptability
Adaptive audio is audio that changes according to the state of its
playback environment. In many ways, it is like interactive audio in
that it responds to a particular event. The difference is that instead
of responding to feedback from the listener/player, the audio changes
according to changes occurring within the game or playback environ-
ment. Say for instance that a game’s musical score shifts keys with
the rising and setting of the sun within the game world. The player
isn’t causing the sun to set; the game is. Therefore, the score
“adapts” to changes happening within the game’s world. A famous
example of adaptive audio in games occurs in Tetris. The music plays
at a specified tempo, but that tempo will double if the player is in
danger of losing the game.
Avoiding repetition is an excellent first step in a strong audio
implementation for games, as well as reintroducing music listeners
to some potentially intriguing performance characteristics lost when
listening to linear music. Continuing to focus on audio for games for
a moment, audio content triggered out of context to game events is
in many ways less desirable than simple repetition. For instance,
using data compression and/or high-capacity media, a composer
might be able to create hours and hours of underscore, but if this lin-
ear music is played with no regard to the state of the game or the
current events in the game’s plot, it could be misleading or distract-
ing to the user. Adaptive audio (audio that changes according to
changes in certain conditions) provides an important first step for
creating interactivity in underscore.
Do not confuse adaptive audio with variability — if, for instance,
20 variations of a gunshot are created and a single one is randomly
chosen when the player fires a gun, that is variable but not necessar-
ily adaptive. If the gunshot’s reverberation characteristics modulate
as the player navigates various and differing environments, then we
have adaptive sound design. Likewise, a game’s score could have
infinite variability, never playing the same way twice, but it becomes
adaptive when the sultry slow jazz tune fades in over the rock theme
during the player’s introduction to the game’s film noir femme fatale
antagonist. Now contrast variability and adaptation with interactivity;
a character that stops talking when shot by the player is an example
of interactive audio.
 DirectMusic Concepts n 7

Adaptation does not always need to mean “just-in-time” audio

generation. While an underscore that instantly and obviously
responds to game events might be appropriate for a children’s ani-
I
mated title, this kind of music is often inappropriate for a first-person

Unit
shooter, for instance. In such a style of game, where the breakneck
pace of the action constantly modulates the game state, the user can
begin to notice the interactivity of the score. This is dangerous, as
the subtlety of the score is lost on the player, potentially damaging
the designer’s carefully crafted game experience. In a game like
Halo, you do not want the player performing music via an exploit in
the interactive music engine. For this reason, musical changes are
often most satisfying when “immediate” changes are reserved for
important game events and more subtle interactivity is used for
other events and situations.

Groove Levels
One of the more powerful ways that DirectMusic exposes adaptive
audio is with the groove level. Groove level is a musical or emotional
intensity setting that adjusts in real time according to rules you
define as part of your playback environment. You can set up different
moods within a DirectMusic piece and assign those moods to differ-
ent groove levels. For instance, someone could produce a
progressive trance piece with sparse musical activity on groove level
1 and increase the intensity of the music through various groove lev-
els. This can be achieved by adding more notes and instruments, or
by setting higher volumes. You can then assign the different inten-
sity (groove) levels to trigger upon changes in the playback
environment. Say you create a DirectMusic piece for stand-alone
music listening; you could set a rule that switches groove levels
according to the time of day or even the number of windows the lis-
tener has open on the desktop. The possibilities are truly endless.
We discuss groove levels in much more detail in the section on
Style-based Segments in Chapter 4.
8 n Chapter 1

Content-Driven Audio
Adaptive audio becomes really interesting when the audio gets
behind the wheel and drives the playback environment to behave
and/or change in a particular manner. This is commonly referred to
as content-driven audio. DirectMusic’s sequencer provides notifica-
tions that can be used to manipulate the playback environment,
allowing for the interesting possibility of music driving gameplay
rather than the opposite. For instance, a monster around the corner
could wait to jump out at you until an appropriate moment in the
score. Content-driven audio is a particularly unexplored area of
DirectMusic, one that could be put to some interesting uses.

Playback Standardization
DirectMusic can play wave files. The great thing about wave files is
that they can be CD quality, and (speaker quality not withstanding)
they sound the same when played back on any computer. The prob-
lem with wave files is that they are big when compared to MIDI files,
and they often limit adaptability and variation. A partial solution is to
create a custom sample bank that is distributed with the MIDI
sequence data in your DirectMusic file. While in most cases you are
still forced to use minimal file sizes (restricting quality), you don’t
have to worry about different listeners hearing your sequenced
music through different synthesizers/samplers; you’ve given them
the sample bank that you created as part of the DirectMusic file.
Before this standardization took place, you could listen to a MIDI file
of Beethoven’s Fifth through your SoundBlaster, while we listened
to it through the sample set on our Turtle Beach sound card. The
result: We both would have heard very different renditions. This is
an audio producer’s nightmare, since you have no control over the
instruments on which your sequence plays. Luckily, this is no longer
a problem, thanks to DLS-2.
DLS-2 (the Downloadable Sounds format, level 2) is a sound for-
mat used for creating and storing samples, much like the popular
Akai sample format. DirectMusic can use DLS-2 samples. The great
 DirectMusic Concepts n 9

thing about DLS-2 support in DirectMusic is that audio producers

can create their own custom sample bank and include it as part of the
DirectMusic file. This means that no matter who plays the Direct-
I
Music file, they will hear the same sounds that everyone else does

Unit
(as opposed to relying on their sound card or soft synth rendering
the sequence data). DLS also specifies basic synthesis abilities —
specification of which waves comprise an Instrument and how to
modify the source wave data according to MIDI parameters like
pitch bend, modulation, and so on. DirectMusic, or more specifically
the Microsoft Software Synthesizer that DirectMusic uses, supports
DLS-2, which adds numerous additional features, including six stage
envelopes for volume and pitch, two low-frequency oscillators, and
MIDI-controllable filtering per voice. When using DirectMusic,
these sample-based Instruments are stored in DLS Collections,
which can be self-contained files (generally using the .dls file exten-
sion) or embedded directly within pieces of music, similar to
traditional MOD files and the newer .rmid extension to standard
MIDI files, where sampled Instruments can be embedded within the
same file as the note information.

3D Spatialized Audio
Our world is one of three dimensions. Most audio that you’ve heard
played over the radio or television exists in one or two dimensions.
Even surround sound lacks a vertical component (sound coming
from above or below). Audio engineers have developed a technique
that synthesizes the way we hear sound in 3D space. DirectX Audio
has this functionality. This means that a sound can be mixed in space
not only to sound closer, farther away, or to the left or right but also
from above, below, and even behind us, all using just two speakers!
We do not go into detail here on how or why DX Audio is able to do
this. Just know that you have the ability to mix sound in 3D using
DirectX Audio.
10 n Chapter 1

DirectMusic Rules and Regulations

DirectMusic imposes several rules and limitations that you should
keep in mind. We cover the more significant ones here and discuss
more specific restrictions as they arise.

Each DirectMusic Performance Has a Single Tempo at

Any One Time
A DirectMusic Performance is somewhat analogous to a conductor
and an orchestra; the conductor is only able to conduct at a single
speed at any time (unless he is conducting some avant-garde piece of
contemporary music!), and the orchestra as a whole needs to under-
stand where those beats are. All pieces of music playing on a
Performance must play at the same tempo — and a tempo change
imposed by one piece of music will affect all other playing pieces of
music. This becomes most interesting when using DirectMusic for
playing sound effects and ambient sound in addition to music, as
these sounds care nothing for tempo. SFX and ambience may inad-
vertently cut off because of musical tempo changes. There are
several options that you may consider:
n Author all content at the same tempo. This of course defeats
the ability to change speeds based on various events.
n Use clock-time Segments for sound effects. Every Segment
file can specify whether it is meant to use music-time or
clock-time. A music-time Segment bases its timing information on
the tempo of the music. Notes will play for a specific number of
measures, beats, ticks, etc. If the tempo changes, the note will
be played for less or more time — desirable for music but not so
much for other sounds. A clock-time Segment, by contrast, only
pays attention to the system clock and absolute time. Generally
used for Segments that only contain non-note-based wave infor-
mation, a clock-time Segment uses millisecond accuracy for play-
back. A wave told to play for 5.12 seconds will play for exactly
that amount of time, regardless of tempo changes that occur
while it is playing.
 DirectMusic Concepts n 11

n Use multiple DirectMusic Performances. You can always run

more than one DirectMusic Performance simultaneously. The
amount of processing power used (i.e., the CPU overhead) for a
I
second DirectMusic Performance is small, though of course

Unit
there are now additional assets to manage and track. Consider
playing sound effects, ambience, and other audio cues that do not
respond to tempo on one performance (with constant tempo),
while music-oriented sounds will be played on another perfor-
mance (with variable tempo).

Primary and Secondary Segments

The next restriction to keep in mind is the differences between pri-
mary and secondary Segments. A Segment can be either primary or
secondary. Only one primary Segment may play at a time; starting a
new primary Segment will implicitly stop and replace any previously
playing primary Segment. Primary Segments typically dictate tempo,
groove level, chord progression, and other big picture (aka “global”)
performance-level events. By contrast, many secondary Segments
may be playing at the same time. Secondary Segments typically layer
on top of the primary Segment, picking up and using the primary
Segment’s tempo and other settings. However, if a secondary Seg-
ment plays as a controlling secondary Segment, rather than layer on
top of the primary Segment, it will actually replace corresponding
Tracks from the primary Segment. Controlling secondary Segments
are typically reserved for more advanced usage (for instance, chang-
ing the chord progression of the primary Segment, say, when an
antagonist enters the same room as the hero in the game).
Primary Segments are generally the main background music or
ambience. Secondary Segments are often “one-shot” sounds or
“stingers” that play over the primary — perhaps sound effects or
musical motifs. Secondary Segments follow the primary’s chord pro-
gression, so musically motivated secondary Segments can play with
appropriate transposition and voicing.
12 n Chapter 1

Crossfades
Programmers and audio producers should note that crossfades are
not a built-in DirectMusic function for transitioning between Seg-
ments. For now, it is a fact, and so you are going to have to fudge
them. It’s not all bad, as you do have options. These options include
AudioPath volume manipulation, MIDI volume controller use, or
authoring volume crossfades directly into content. Remember that
only one primary Segment can play at a time, which precludes the
possibility of using primary Segments exclusively for crossfades.
Unless multiple DirectMusic Performances are used, at least one of
the Segments we are crossfading between will need to be a second-
ary Segment. For ease of use, we suggest that both be secondary
Segments. The one-tempo limitation we discussed above makes
crossfades between two pieces of music with different tempos diffi-
cult. One of the pieces will be played faster or slower (unless
multiple DirectMusic Performances are used). Of course, this limita-
tion really only applies for sequenced music; prerendered wave files
using different tempos play fine.

Pause/Resume
“Pause” is another feature not implicitly built into DirectMusic.
However, a programmer can track the time when a Segment starts,
as well as its offset when he stops that Segment. Using this informa-
tion (and keeping in mind that the Segment may have been looped),
the programmer can typically restart a Segment from the same loca-
tion to create “resume” behavior. However, this only works for
Segments that use music time. Note that content that includes
numerous tempo changes may be less accurate in restart position
than content with a single tempo.
While this works for waves, MIDI and DLS limitations do not
allow this for sequenced note data. The MIDI standard for note infor-
mation consists of a “note on” and corresponding “note off” event.
Starting playback between the two events will not trigger the note.
Even if it did, remember that DLS Instruments cannot be started
from an offset; they can only be played from the beginning. This
won’t really be an issue for short notes in a dense musical passage.
However, if you have long sustained notes (for instance, a string
pad), keep this in mind if you want to be able to pause/resume your
 DirectMusic Concepts n 13

music. In that particular case, you might want pause/resume func-

tionality to restart the piece of music at a position before the pad
began or even the beginning of the Segment.
I

Unit
Memory Footprint
Memory is typically the most precious resource for games, whether
on the PC or a console. The amount of memory taken up by the
resources of a program or a particular component of a program is
called the footprint. Making use of streaming wave data can help
keep the memory footprint small; only a small amount of any playing
wave is actually in memory at a time. DirectMusic supports wave
streaming within Wave Tracks, with content-specified read-ahead
settings that determine how much of the wave is in memory. How-
ever, the DLS format does not support streaming, so any DLS
Collection instruments play entirely from memory. There are several
optimizations available that we will discuss in Chapter 2 when we
cover Bands (a DirectMusic file that stores DLS Instruments), but
keep in mind that DLS Instruments will occupy a footprint as long as
they are downloaded.
Again, this isn’t a big deal if you are creating music for stand-
alone listening. You’ll have the free range of the system memory to
work with for samples (normally 128MB of memory — more mem-
ory than any pro hardware synthesizer on the 2003 market) on top of
all the streaming that the audience’s PC can handle. But in games,
you are very limited in the amount of memory you can use. Just keep
that in mind.

þ Note for programmers: Just using DirectMusic incurs a memory

footprint, albeit a small one. The size does depend on which
DirectMusic functions you choose to use in your application. For
instance, if you use DirectX Audio Scripting with the Visual Basic
Scripting Edition language (VBScript), that language requires several
libraries of code to be loaded to the tune of almost 1MB of DLLs
(dynamically linked libraries). Granted, other aspects of your program
might depend on the same libraries, but to help keep the memory
footprint more manageable, DirectMusic offers a tiny (~76KB)
optimized DirectX Audio Scripting language (called audiovbscript) as an
alternative to the fully featured VBScript.
14 n Chapter 1

DirectMusic and MIDI

We’ve already alluded several times to DirectMusic’s support for the
MIDI format. This can be a strength or a weakness; the interoper-
ability with this standard sequencing format allows for easy
authoring from more sequencing-focused tools in the composer’s
palette (like Sonar or Cubase SX). But using DirectMusic for sound
effects can work against this support — a sound effect is often meant
to be played once or looped indefinitely. The concept of a finite “note
on” and “duration” is somewhat foreign in this instance. As we’ve
discussed already, the use of clock-time Segments can mitigate this
somewhat for sound effects — they are given a fixed duration in mil-
liseconds that are independent of tempo.
While DirectMusic supports MIDI, it has several significant
aspects that allow it to overcome some of the basic limitations of
MIDI devices. For instance, typical devices are limited to 16 MIDI
channels. The Microsoft Software Synthesizer, the synthesis engine
used by DirectMusic, allows you to reference up to 32 million dis-
crete channels. Using DirectMusic Producer, you will be able to use
1000 of these performance channels, or pchannels, per Segment.
DirectMusic also adds some basic optimizations to MIDI to allow
for easier manipulation of controller data. Traditionally, sweeping a
controller from one value to another consisted of many rapid MIDI
controller events. As a file format (and authoring convenience) opti-
mization, DirectMusic allows the audio producer to instead use
continuous controller curves. The start time, end time, initial value,
end value, and curve type can all be specified. When the content
plays back, the intermediate values are dynamically generated to
remain compatible with traditional MIDI.
A common question raised when a game is layering 20 pieces of
music is whether the composer needs to worry about authoring each
piece of audio onto its own set of pchannels. With traditional MIDI,
you cannot simultaneously play two separate Instruments on the
same channel at the same time — the patch change from one over-
rides the other. Similarly, if two Tracks on the same channel use the
same MIDI controllers, they will override each other. Because
DirectMusic allows the composer to play multiple pieces of content
simultaneously, DirectMusic provides solutions to these basic MIDI
limitations.
 DirectMusic Concepts n 15

From the point of view of MIDI, DirectMusic AudioPaths effec-

tively act as unique synthesizers with their own unique pchannels.
An AudioPath is pretty much what it sounds like — a route that
I
audio data follows through hardware or software before it is output.

Unit
PC AudioPaths can include DirectX Media Objects (DMOs), which
are software audio processing effects. AudioPaths also define the
behavior of content being played onto them, such as whether they
can be positioned in 3D space (kind of like surround sound but differ-
ent — more on this later), whether they should be played as stereo
or mono, and so on. As far as MIDI is concerned, each AudioPath
gets its own unique set of “physical” pchannels (granted, this is all in
software, so we’re not talking about tangible channels). For instance,
if two sequences are authored, both using pchannels 1 through 16,
playing them on separate AudioPaths will keep the two from inter-
acting. If one Segment on AudioPath A has a piano on pchannel 1 and
another Segment on AudioPath B has a French horn on pchannel 1,
they will both play happily. If we tried to play these two Segments
onto a single AudioPath, one of the patch changes would win out, and
one Segment would be playing the wrong instruments.
That said, sometimes playing multiple pieces of content onto the
same AudioPath is desirable. For instance, a Segment could be
authored simply to alter the mix of music played by another Segment
(using volume controller curves). Alternatively, in the example of 3D
audio above, we probably would want to play everything that will
emanate from this single position onto the same AudioPath.
Therefore, we have a solution for multiple simultaneous pieces
of music. But what about MIDI controller curve interaction? Of
course, if the pchannels are kept unique (either at authoring time or
by playing the content onto separate AudioPaths), the MIDI control-
lers won’t conflict. But what about the above example of a Segment
where we just want to alter the mix? If the Segment it’s altering has
its own volume controller curves, the two will conflict, and we might
get the volume jumping rapidly between the two. The classic solu-
tion is to use volume curves in one Segment and expression curves
in the other Segment. This is a common approach in sequenced
music, as both affect perceived volume and apply additively. This way
the audio producer sets Volume (MIDI Continuous Controller #7) in
the primary piece of music, and uses Expression (MIDI Continuous
16 n Chapter 1

Controller #11) to reduce this volume per individual performance

channel in the second, layered piece of music.
DirectMusic provides a more generalized solution for this prob-
lem — not only for volume but also for pitch bend, chorus send,
reverb send, and numerous other MIDI continuous controllers. The
audio producer specifies an additive curve ID for each MIDI continu-
ous controller curve created. Curves that have the same ID override
each other, as with traditional MIDI. Curves that have different IDs
are applied additively (or more accurately for the case of volume, the
process becomes subtractive).
 Chapter 2

Linear Playback

O kay, let’s get cracking. The first thing we want to do is be able

to play sound. We add variability, dynamism, and the rest later.
Let’s focus on playing back a wave file.

þ Note At this point, if you have not already, you must install DirectX
9.0 as well as DirectMusic Producer, which are available on the
companion CD.

Waves in Wave Tracks

To play back a wave file in DirectMusic, you’ll create a Segment out
of it. As already mentioned, the DirectMusic Segment file (.sgt) is
the basic unit of DirectMusic production. Segments are built from
one or more of DirectMusic’s various track types, which can make
sound (via stand-alone waves and/or DLS instrument triggering) or
modify performances (tempo, chord progression, intensity level,
etc.). You can create a Segment out of a wave file in DirectMusic
Producer. Run DirectMusic Producer and go to File>Import File
into Project>Wave File as Segment…. Open a wave file into the
program. Building a Segment from a wave file, we get our first look
at one of the basic track types in DirectMusic — the Wave Track.
Beyond the 32 variation buttons (which we cover in Chapter 3),
Wave Tracks play along the sequencer timeline with other Wave
Tracks and also with MIDI sequences.
If you look at the size of the Segment file, it is much smaller than
the wave file. Here is an important early lesson in content delivery:
Segments will know where the wave and sample files that it needs to
play are but do not store those files as part of itself. There are sev-
eral reasons to do this; if several Segments use the same wave data,
17
18 n Chapter 2

you do not have to worry about having two copies of it in memory. In

addition, if you later want to go back and edit that wave data, you do
not have to worry about copying it into several different places. That
said, there are various reasons that you might instead want to embed
the wave data within the Segment itself — for instance, for the con-
venience of only having to deliver a single file or file load time
considerations.
Files with the extension “.**p” are design-time files. Design-time
files are used in DirectMusic Producer for editing purposes and con-
tain information not necessary for run-time use — for instance, a
Segment used in a game does not need to include information on
what size and position to open editing windows. In addition,
design-time files always reference the content they use, even if you
specify that content should be embedded. For these reasons, when
content is meant to be integrated into a game or a special player, you
should save the Segment as a run-time file (either via the per-file
right-click menu Runtime Save As… option or the global Runtime
Save All Files option from the File menu). When Segments are
run-time saved, you will see the .sgt extension, and wave files will
similarly have the more expected .wav extension. So to summarize,
be sure to save your Segments as design-time (.sgp) files while you
are working on them and as run-time files (.sgt) when they are fin-
ished and ready for distribution. For more details on content delivery,
there is a white paper available on the Microsoft Developer Network
web site (msdn.microsoft.com) called “Delivering the Goods:
Microsoft DirectMusic File Management Tips.”

Streaming Versus In-Memory Playback

A minute-long 44.1 kHz 16-bit stereo wave file is 10MB. 10MB isn’t
a big deal when you run a stand-alone DirectMusic file on a contem-
porary computer, but it is a very large file in the world of game audio.
Do not forget that the rest of a game’s resources need to reside in
memory as well. If the game is written to run on a system with as lit-
tle as 64MB of RAM, you are already in way over your head. Console
games are even more unforgiving. Consider yourself lucky if you get
4MB for your entire sound budget! You can use audio streaming to
alleviate these restrictions. Streaming is a technique that works a lot
 Linear Playback n 19

like a cassette player. In a cassette player, audio data is moved

across the play head a little bit at a time. The play head reads the
data as it comes and plays the appropriate sound. With streaming, a
I
small area in memory called a buffer is created. Wave files are

Unit
moved, bit by bit, through the buffer and read by DirectMusic. The
CD player in your PC uses streaming for playback. If it weren’t for
streaming, you’d have to load the entire music file into your com-
puter’s RAM, which in most cases simply isn’t an option.
DirectMusic uses the following rule for streaming: The wave
streams with 500 msec readahead if it is more than five seconds long. If
it is shorter than five seconds, it resides and plays from memory (RAM).
Readahead determines the size of the buffer. For 500 msec, our 44.1
kHz 16-bit stereo wave file will use 88.2KB of memory (.5 sec x
44100 samples/sec x 2 bytes/sample x 2 channels), a big difference
when compared to 10MB! Memory usage is reduced by a factor of
more than 100! You can override this behavior, choosing to load all
wave data to memory, or specify a different readahead value in the
Compression/Streaming tab of a wave’s property page in
DirectMusic Producer. To get to any Track’s property page, simply
right-click on it in the Track window.

Tracks and Parts and Layers, Oh My

Before we move any further, the concept of Tracks and parts can use
a bit of sorting out. When we created our Segment from a wave file,
it consisted of a single Track, a Wave Track, with a single part (by
default on pchannel 1). Each part specifies its own variation behavior,
what pchannel it is played on, and, in the case of Wave Tracks, vol-
ume for that part as well. By comparison, Tracks specify big picture
(or “global”) behaviors such as clock-time, as well as more exotic
settings like multiple, concurrent Track groups. If you open the prop-
erty page for a Track or part, you’ll notice the separate tabs with
their own settings — the Track properties actually includes both the
Track Group Assignments and Flags tabs in addition to the actual
Track properties tab.
Remember that waves in Wave Track parts never conflict with
DLS Collections on a pchannel; you can play as many waves as you
want simultaneously and still play a DLS Collection instrument on
20 n Chapter 2

the same pchannel without any problem. The reason a Wave Track
part can be assigned to a pchannel is that MIDI controllers can still
be used to drive wave playback (for instance, pan on a mono wave,
pitch bend, volume, etc.). The pchannel that a Wave Track is
assigned to can be altered in the wave part’s property page.
This brings us to the concept of layers, or the various lettered
rows (a, b, c, etc.) that you see displayed in DirectMusic Producer
for a single Wave Track part. As mentioned, you can play as many
waves at one time as you wish. Therefore, layers are purely an aid
for placing waves against the Segment timeline, rather than having
many waves overlap on a tiny area of the screen and not be able to
easily edit them (or tell which wave started and finished when).
Waves can be placed on different layers within a part for easier legi-
bility. All layers are equal, are played simultaneously, and do not take
any additional resources (processor power or memory) when played.

Figure 2-1: Multiple waves on Wave Tracks are split

into “layers” so you can see their beginnings and
endings more easily.

As a last bit of terminology for now, certain parts are actually subdi-
vided even further into strips. In particular, the parts for MIDI-
supporting Tracks (Pattern Tracks and Sequence Tracks) have
separate strips for note information and continuous controller infor-
mation. Pattern Tracks also have an additional variation switch point
strip, which we cover in Chapter 3 when we discuss variations.
 Linear Playback n 21

MIDI and DLS Collections

I
Creating a DirectMusic Segment from a piece of MIDI music is sim-

Unit
ple. In DirectMusic Producer, follow File>Import File into
Project>MIDI File as Segment… . This creates a DirectMusic
Segment from a MIDI file, importing note, MIDI controller, and
other pertinent data along the way. Let’s examine some new Track
types related to MIDI:
n Tempo Track: Specifies the current tempo for the piece of
music. You can override this by playing a primary or controlling
secondary Segment with its own tempo settings.
n Time Signature Track: Sets the time signature for the piece.
Use this to track where measures fall as well as how many subdi-
visions (grids) to give each beat.
n Chord Track: Specifies the key as well as specific chord pro-
gressions for a piece of music. Typically, for an imported MIDI
file, this will just indicate a C major chord. DirectMusic Producer
does not attempt to analyze the imported MIDI file for chordal
information.
n Sequence Track: A sequence is what its name implies.
Sequence Tracks house MIDI sequences. This is where the
majority of MIDI information is imported. Notice that as with
Wave Tracks, Sequence Tracks can (and typically do) consist of
multiple parts. By default, each MIDI channel is brought in as a
separate part on the corresponding pchannel. In addition, each
part can contain its own continuous controller information.
Unlike Pattern Tracks (more on these in Chapter 3), Sequence
Tracks are linear, non-variable sequences; they play the same
every time, and they do not respond to chord changes (though
they do respond to tempo changes).
22 n Chapter 2

Figure 2-2: A Segment with a Sequence Track consisting of multiple

instruments.

n Band Track: Bands are how Performance channels refer to DLS

instrument collections. The Band Track is an area where initial
pan, volume, and patch change settings for Tracks are stored.
This is often an area of confusion, as you can also have volume
and pan controller data within Sequence Track (or Pattern Track)
parts. The Band Track typically just has a single Band with the
initial settings. Subsequent volume changes are typically created
as continuous MIDI controller events in Sequence or Pattern
tracks. Continuous controller events, unlike band settings, allow
you to easily sweep values over time (for example, to fade a
track in or out). If any patch changes occur during performance,
another Band can be placed in the Band Track, though the elimi-
nation of MIDI channel limitations means there is often little rea-
son not to just place each instrument on its own unique and
unchanging channel.
 Linear Playback n 23

Building DLS Collections

Our Segment is now ready for playback. Of course, we’re assuming
I
that the piece of music will be played using a preauthored DLS Col-

Unit
lection, such as the gm.dls collection that comes with Windows
machines. Otherwise, it will be trying to play instruments that don’t
exist on the end user’s machine, and these instruments would be
played silently. If we wanted to use our own instruments, we would
want to build one or more DLS Collections. DirectMusic Producer
provides this authoring ability in DLS Designer. Alternatively, the
DLS-2 format is a widespread standard, and there are several tools
out there for converting from other common wavetable synthesizer
formats to the DLS format.
Creating DLS Collections is often one of the more challenging
tasks when you decide to create real-time rendered (versus
streamed prerendered) audio. Remember that unlike streamed audio,
your DLS Collection will occupy system memory (RAM), which is
typically one of the most precious commodities for an application.
For this reason, you’ll want to create collections that get you the
most bang for your buck in terms of memory versus instrument
range and quality.
Let’s create our first DLS Collection. From the File menu, select
New, and from the dialog that appears, choose DLS Collection and
hit OK. We’re presented with a project tree entry for our DLS Col-
lection (using the design-time .dlp file extension), which has two
subfolders, Instruments and Waves.

Figure 2-3: An empty DLS

Collection in the project tree.

DLS Collections are composed of waves (effectively, ordinary wave

files) and instruments, which consist of instructions for how waves
should map to MIDI notes, along with a fairly sophisticated set of
audio-processing features. Let’s add a few waves for our first instru-
ment, a piano. We can drag our waves right into the Waves folder
from a Windows Explorer window or right-click on the Waves folder
and choose Insert Wave… from the menu that appears.
24 n Chapter 2

Figure 2-4: We’ve inserted four wave

files into our DLS Collection.

We can now adjust the properties for these waves by right-clicking

on them and choosing Properties to bring up their property window.
The most important settings to note are going to be Root Note, loop
points (both in the Wave tab), and compression settings (in the Com-
pression/Streaming tab). We’ll return to compression a bit later. Root
Note specifies the base note that this wave corresponds to, also
known in other wave authoring programs as the “unity pitch” or
“source note.” For our above piano sounds, determining the root
note was made easier by including the root note information right in
the wave file names. DirectMusic Producer will also automatically
use any root note information stored with the wave file, which some
wave authoring tools will provide. Otherwise, we can adjust the root
note manually from the property page, using either the scroll arrows
or by playing the appropriate note on a MIDI keyboard.

Figure 2-5: Setting the proper root note for our

BritePiano_C4 wave.

This particular piano wave doesn’t loop, so we don’t need to worry

about the loop settings. Again, if loop settings had been set on the
source wave file, DirectMusic Producer would automatically use that
information to set loop points.
 Linear Playback n 25

DirectMusic Producer provides some basic editing features on

waves in the Wave Editor, which can be opened by double-clicking
any wave in the Waves folder (or indeed, any separate wave file
I
you’ve inserted into your DirectMusic project).

Unit
Figure 2-6: The Wave Editor. You can specify whether waveform selections
(made by clicking and dragging) should snap to zero crossings via Snap To
Zero, and you can specify a loop point by selecting Set Loop From Selection.

The Wave Editor window supports clipboard functions, so you can

copy to and paste from other wave editing applications if you wish.
You can also try out any loop points you’ve set (or just hear the wave
you’ve created) by playing the wave. As with Segment playback, you
can use the Play button from the Transport Controls toolbar, or use
the Spacebar as a shortcut to audition your wave. Note that you can
only start playing your waves from the beginning, regardless of
where you position the play cursor, as DLS waves cannot be started
from an offset.
Now that we’ve set up our waves, let’s add them to a DLS
instrument so we can play them via MIDI notes. To create an instru-
ment, right-click on the Instruments folder and choose Insert
Instrument.

Figure 2-7: Creating an instrument.

26 n Chapter 2

You’ll notice that each instrument is assigned its own instrument ID,
a unique combination of three values (MSB, LSB, and Patch) that
allows for more than 2 million instruments — a bit more freeing than
the traditional 128 MIDI patch changes. DirectMusic Producer will
make sure that all loaded DLS Collection instruments use unique
instrument IDs, but you should take care if you author your DLS
Collections in different projects to make sure that they don’t conflict.
Otherwise, if two instruments with identical IDs are loaded at the
same time, DirectMusic will have no way of knowing which one you
want to use. The General MIDI DLS Collection (gm.dls) that is
found on all Windows machines uses 0,0,0 through 0,0,128, so our
new instrument probably defaulted to instrument ID (0,1,0).
Let’s open up the Instrument editor by double-clicking on the
instrument in the project tree.

Figure 2-8: The Instrument editor.

 Linear Playback n 27

There are lots of options here that really demonstrate the power of a
DLS-2 synthesizer, but for now let’s just start with our basic instru-
ment. The area above the piano keyboard graphic is where we define
I
which wave or waves should be played for a given note. Each wave

Unit
assigned to a range of notes is called a DLS region. As you can see,
our new instrument defaults to a single region, which plays the wave
BritePiano_C5, over the entire range of MIDI notes. The piano key-
board note with a black dot on it (C5) indicates the root note for the
region. Remember that the root note will play the wave as it was
authored, notes above the root note will pitch the wave up, and notes
below the root note will pitch the wave down.
Looking along the left edge of our region, you can see the labels
DLS1, 1, 2, and so on. These labels identify the various layers of this
instrument. Layers allow you to create overlapping regions, one of
the nice features of the DLS-2 specification. One of the more com-
mon uses is for adding multiple velocity versions of a wave, where as
the note is struck harder, the instrument will switch to a different
wave file. This is particularly effective for percussion instruments.
Notice that each region has a velocity range that can be set in the
Region frame. Multiple layers also mean that a single MIDI note
could trigger a much more complex instrument composed of several
simultaneously played wave files. Remember that regions on a single
layer cannot overlap. The DLS1 layer is the single layer that was
supported in the original DLS-1 specification, and is generally still
used to author any single-layer DLS instruments. For this simple
instrument, we won’t worry about using multiple layers for the
moment.
Our first step is to create regions for the rest of our piano waves.
We want to resize that existing region so there is room on the layer
for the other waves (that, and we probably don’t want to try to play
this wave over the entire range of the MIDI keyboard!). If you move
the mouse over either edge of the region, you’ll see that it turns into
a “resize” icon, and you can then drag in the outer boundaries of the
region.

Figure 2-9: Resizing a region with the mouse. Alternatively, we could

change the Note Range input boxes in the Region frame.
28 n Chapter 2

By the way, this is similar to the functionality for note resizing

(changing start time and duration) in the MIDI-style Tracks (Pattern
Tracks and Sequence Tracks). There are other similarities between
region editing and note editing — the selected region turns red, just
as notes do. The layer we are currently working with turns yellow,
just as the current Track does in the Segment Designer window. And
just as you can insert notes by holding the Ctrl key while clicking,
you can create a new region in the same way.

You can resize the region as long as the mouse is held down and
dragged (or, as before, resize the region by grabbing a side edge of
it). As an alternative to drawing a region with the mouse, you can
right-click and choose Insert Region from the menu that appears.
Every region that we create defaults to use the same wave that
the first region did. So we’ll want to choose a more appropriate wave
from the drop-down aptly labeled Wave. Let’s assign BritePiano_E5
to this new region we’ve created.

We repeat the process for the rest of our waves to build up our
instrument. Notice as you click on each region that you can view its
particular properties for wave, range, root note, etc., in the Region
frame.

Figure 2-10: The completed regions for our piano instrument.

There are several potential issues to discuss here. First, should all
instruments span the entire keyboard? On the plus side, the instru-
ment would make a sound no matter what note was played. On the
minus side, when the source wave is extremely repitched (as
 Linear Playback n 29

BritePiano_C4 is in the downward direction and BritePiano_E5 is in

the upward direction), it can become almost unrecognizable, dramati-
cally changing timbre and quality. Most composers opt for the latter,
I
if only to know whether their music ever goes significantly out of an

Unit
acceptable range for their instruments. If it does, the music can be
adjusted (or the DLS instrument expanded to include additional
regions).
A second question is how far above and/or below the root note a
region should span. That is, does the wave maintain the quality of
the source instrument more as it is pitched up or down? This can
vary significantly from wave to wave (and the aesthetic opinions of
composers also differ quite a bit). For the above example, we did a bit
of both — the bottom three regions encircle notes both above and
below the root note, while the highest piano region (our BritePiano_
E5 wave) extends upward from the root note. How far you can
“stretch” a note can vary quite significantly based on the kind of
instrument and the source waves.
Once you’ve created your instrument, you can try auditioning it
in several ways. An attached MIDI keyboard will allow you to trigger
notes (and indeed, small red dots will pop up on the keyboard graphic
as notes are played). You can also click on the “notes” of the key-
board graphic to trigger the note. The Audition Options menu allows
you to set the velocity of this mouse click and choose whether to
audition all layers or only the selected one. (In the case of our exam-
ple, we only have one layer, so the two options don’t matter.)

Try out the piano instrument, paying particular attention to the tran-
sitions between regions. These are often difficult to match smoothly,
as the pitch shifting of the waves begins to alter their characteristics
and the instruments can begin to develop musical “breaks” where
the tone quality changes significantly. If you are writing a purely
tonal piece of music, it is often useful to author your DLS Collections
such that the tonic and dominant tones (the scale root and the 5th
degree) are the root notes of your regions. That way, the “best” ver-
sion of each wave (not repitched) will be played for what are typically
the most commonly heard tones in the music.
30 n Chapter 2

If you do need to tweak the source waves somewhat, you can

either reauthor them and paste over the previous version in the
wave bank, or you can make manual edits to how the regions play
particular waves. The latter is useful when a wave is just slightly
mispitched when played against another wave or when the levels of
two recorded waves differ slightly. To override the wave’s properties
for this region, open the region’s property window (by right-clicking
and choosing Properties or, as with other property windows, by
clicking on the region when the property window is already open).

Figure 2-11: A DLS region’s property window.

Here we’ve slightly adjusted the fine-tuning for
BritePiano_G4 only as this region plays it back.
If any other regions or instruments used the
wave, their pitch would be unaffected by this
change.

þ Note One common question is whether using overlapping regions

on multiple layers could be used to more smoothly crossfade between
waves. Unfortunately, all regions are triggered at the same velocity, so
there is no easy way to make one region play more quietly than another
for a given MIDI note. With some effort, you could potentially create
several smaller regions over the transition that attenuate the source
waves, gradually bringing their volume up (by overriding attenuation in
the region’s property window) to complete the crossfade.

And that’s enough to create our basic instrument. If we wanted to,

we could adjust any of a number of additional properties on the
instrument’s articulation, which is where much of the power of a
DLS-2 synthesizer lies. The articulation allows you to specify vol-
ume and pitch envelopes, control a pair of low-frequency oscillators
(LFOs), and set up low-pass filtering parameters for the instrument.
You can even create per-region articulations, where a region has its
own behavior that differs from other regions of the instrument. We’ll
 Linear Playback n 31

just set up the instrument-wide articulation for this piano. Since our
original sample already has most of the volume aspects of the enve-
lope built in, we’ll just add a release envelope. This means that when
I
a MIDI note ends, the instrument will fade out over a period of time

Unit
rather than cutting off immediately, much like an actual piano. By
dragging the blue envelope point to the left in the Instrument Articu-
lation frame, we have set the release to be .450 seconds, so this
instrument’s regions will fade out over roughly a half-second when a
MIDI note ends.

Figure 2-12: To add any per-region articulations (presumably different from the
global instrument articulation), right-click on the region and choose Insert
Articulation List.

Our basic piano instrument is now complete. We could open up the

instrument’s property window (as always, by right-clicking on it and
choosing Properties) and give it a better name than Instrument, such
as MyPiano. We then repeat the process for other instruments in our
collection. Again, remember that DLS Collections will be loaded into
memory when a Segment is using them, so you’ll want to budget
their size according to available memory on the system on which
they will be played.
32 n Chapter 2

Stereo Waves in DLS Collections

One unfortunate omission from the DLS-2 specification is that stereo
waves are not supported for regions. DirectMusic Producer works
around this by using DLS-2’s aforementioned support for multiple
layers. When a stereo wave is used in a DLS region, DirectMusic
Producer creates a hidden second region, separates the two source
channels out, and plays the mono pair of channels in sync on the two
regions’ collection.

Figure 2-13: The top single stereo region is

actually stored by DirectMusic Producer as
something closer to the bottom pair of
mono regions.

All of this is transparent to the composer (and to any content that

uses the DLS instrument) — they can use stereo waves the same as
mono waves without having to do anything differently. But it does
have several impacts on content authoring. The primary implication
of DLS-2 not supporting stereo waves is that stereo waves cannot be
compressed in DirectMusic Producer. Otherwise, DirectMusic would
have to somehow figure out how to separate the left and the right
channels to place them in separate mono regions. If you do intend to
use stereo waves and they must be compressed, you must author
them as pairs of mono waves, create regions on two layers, and then
set those regions’ multichannel properties to specify the channels
and phase locking of the waves (so they are guaranteed to always
play in sync).
 Linear Playback n 33

Unit
Figure 2-14: Making our piano’s right channel
wave play in sync with the left channel and on
the right speaker. Both channels would be set
to the same Phase Group ID, and one of them
should have the Master check box checked.

Using DLS Collections in Segments

Now that we’ve assembled our DLS Collection, our Segment needs
to use that collection’s patch changes in order to play its instru-
ments. Remember that patch change information is stored in a Band
in the Band Track. If we created a MIDI-based Segment from
scratch, we would insert a Band Track (right-click in the Segment
Designer window and choose Add Tracks…), and then insert a
Band into the Track (by clicking in the first measure and hitting the
Insert key).
However, let’s assume for our very first piece of music that we
just imported a MIDI file (from the File menu, choose Import File
into Project…, then MIDI File as Segment…). In this case, we
already have a Band created for us containing the patch changes from
the original MIDI file.
We now want to edit this Band to use our new DLS instruments
(rather than the General MIDI ones, or worse, instruments that
don’t exist).

Figure 2-15: Double-click the Band in the Band

Track to open the Band Editor.
34 n Chapter 2

Figure 2-16: The Band Editor window.

Remember that in addition to patch change information, you can set

up the initial volume and pan for each channel. The grid at the right
allows you to do this graphically (vertical corresponding to volume,
horizontal corresponding to left-right pan) by grabbing a channel’s
square and dragging it. Alternatively, you can set up each Perfor-
mance channel by double-clicking on it (or right-clicking and
choosing Properties).
The properties window is where you set the instrument proper-
ties for the channel.

Figure 2-17: The Band Properties dialog box.

An instrument’s priority allows DirectMusic to know which voices

are the most important to keep playing in the event you run out of
voices. Volume and Pan are once again the same functionality seen in
the Band Editor’s right grid. Transpose lets you transpose all notes
 Linear Playback n 35

played onto the channel (“Oct” is for octaves and “Int” is for inter-
vals within an octave). PB Range lets you control how far the wave
will be bent by Pitch Bend MIDI controllers.
I
Range is a somewhat interesting option. It instructs DirectMusic

Unit
that we will only play notes within a certain range, and therefore that
we only need the regions from this instrument that are in that range.
While this can cut down on the number of DLS instrument regions
that are in memory (and thus possibly the size of wave data in mem-
ory), it does mean that notes played outside of this range will fall
silent. Because transposition, chord usage, or other interactive
music features might cause our content to play in a wider-than-
authored range, Range is generally not used (and thus left
unchecked).
Getting back to the task at hand, this channel is currently using
General MIDI patch 89 (remember that the other two aspects of an
instrument ID, the MSB and LSB in the DLS Collection, are typi-
cally zero for General MIDI collections). We want it to instead play
our new instrument, so we click on the button displaying the name
of the current instrument, choose Other DLS… from the drop-
down that appears (the other options are all of the instruments from
gm.dls), and select our DLS Collection and instrument to use.

Figure 2-18: The Choose DLS

Instrument dialog box.

If we now play our Segment, Performance channel one will use our
newly created piano instrument.
36 n Chapter 2

Authoring Flexible Content for Dynamic Playback

Playing back long MIDI sequences isn’t going to be particularly
interactive, so we might want to take this opportunity to consider
alternative solutions to strictly linear scores. For instance, you might
want to “chop up” your score into smaller pieces of music that can
smoothly flow into each other. Authoring music in this kind of seg-
mented (no pun intended) manner can be as simple as creating a
bunch of short looping clips, each of which follows the same progres-
sion, perhaps deviating in terms of instrumentation and ornamenta-
tion. Alternatively, the various sections could be much more distinct,
and you could author transitional pieces of audio content that help
“bridge” different areas of the music. In every case, you need to con-
sider the balance between truly “interactive” audio (that is, music
that can jump to a different emotional state quickly) and musical con-
cepts such as form, melodic line, and so on. That is, how can a piece
of music quickly and constantly switch emotional levels and still
sound musically convincing? Finding a balance here is perhaps one of
the most significant challenges in creating dynamic audio content,
and this challenge is addressed by specific composers in Unit III.

Transitions
Now that we’ve got basic linear pieces of music, let’s discuss moving
from one piece of linear music to another. How do we move from one
piece of music (the “source”) to the next (the “destination”)?
DirectMusic provides an abundance of choices in how such transi-
tions occur. You can specify a Segment’s default behavior in its
property page (right-click on the Segment) on the Boundary tab. On
the positive side, using these settings means you don’t need to tell a
programmer how your musical components should transition
between each other. On the negative side, this locks a Segment into
one specific transition behavior, which is often not desired; it’s quite
common to use different transition types based on both the Segment
that we’re coming from and the Segment that we’re going to. Over-
riding can be a chore, particularly when using DirectX Audio
Scripting. More often than not, audio producers end up having to
communicate their ideas for transitions directly to the project’s
 Linear Playback n 37

programmer for implementation. This is of course if you’re working

on a game. If you are creating stand-alone music for a DirectMusic
player, stick to what you’ve got in terms of transition types!
I
Looking over the boundary options, there are quite a few that are

Unit
fairly straightforward — Immediate, Grid (regarding the tempo grid;
remember that the size of a grid can be specified in the Time Signa-
ture Track), Beat, and Measure. End of Segment and End of
Segment Queue are also fairly easily understood — the former start-
ing this Segment when the previous one ends and the latter starting
this Segment after any other Segments that were queued up to play
when the current Segment ended.
Marker transitions introduce us to another Track type that we
can add to an existing Segment, aptly named the Marker Track. After
adding a Marker Track to a Segment, an audio producer can add
markers (also known as exit switch points) wherever it is deemed
appropriate in the Segment, which will postpone any requested tran-
sition until a specific “legal” point is reached. Just to clarify, the
transition is specified on the Segment that we’re going to, even
though the boundary used will be based on the currently playing
Segment that we’re coming from.
Transitions do not have to occur directly from one piece of music
to another. The content author can create specific (typically short)
pieces of music that make the transition smoother, often “ramping
down” or “ramping up” the emotional intensity, altering the tempo,
or blending elements of the two Segments. A programmer or
scripter can then specify a transition Segment be used in conjunction
with any of the boundaries discussed above.
As a more advanced behavior, when using Style-based playback,
the programmer or scripter can specify particular patterns from the
source and/or destination Segments (for instance, End and Intro,
which plays an ending pattern from the source Segment’s Style and
an intro pattern from the destination Segment’s Style) to similarly
make the transition more convincing. We cover these embellishment
patterns more in depth when we get into Style-based playback in
Chapter 4.
By default, transitions go from the next legal boundary in the
source Segment to the beginning of the destination Segment. There
are, of course, cases where we might want the destination Segment
to start from a position other than the beginning. Most commonly,
38 n Chapter 2

we could intend for the destination Segment to start playing from the
same relative position, picking up at the same measure and beat as
the source Segment that it is replacing.
For this kind of transition, alignment can be used, again in con-
junction with the various boundaries and transition Segment options
above. Align to Segment does exactly what we outlined above. As an
example, if the source Segment is at bar ten beat two, the destina-
tion Segment will start playing at its own bar ten beat two. Align to
Barline looks only at the beat in the source Segment and can be used
to help maintain downbeat relationships. Using the same example as
above, the destination Segment would start in its first bar but at beat
two to align to barline with the source Segment. As another tech-
nique for starting a Segment at a position other than its beginning,
enter switch points (specified in the “lower half” of Marker Tracks)
can be used. In this case, whenever a transition is intended to occur,
the destination Segment will actually force the source Segment to
keep playing until an appropriate enter switch point is reached in the
destination Segment. Alignment generally provides sufficiently satis-
factory transitions with less confusion, so enter switch points are
rarely used. Remember in both cases that the limitations outlined
earlier for pause/resume apply here; starting a Segment from a posi-
tion other than the beginning will not pick up any already sustained
MIDI notes, though it will pick up waves in Wave Tracks, as well as
subsequent MIDI notes.
Transition types can be auditioned within DirectMusic Producer
by using the Transition (A|B) button.

Figure 2-19: The Transition, or A|B button.

By right-clicking on the button, you can set all of the above options
(and then some!).
 Linear Playback n 39

Unit
Figure 2-20: The Transition Options window
grants access to various transition settings.

Then while playing back a Segment, click the Segment you want to
be your destination, and then click the Transition button (or Alt+”,
its keyboard shortcut). The transition will occur as it would in an
application using DirectMusic.

Figure 2-21: Play the Segment.

40 n Chapter 2

Figure 2-22: Highlight the Segment to which you want to transition, and
press the Transition button.
 Chapter 3

Variation

O kay, we understand linear pieces of music and getting from

one piece of music to another. A primary challenge to over-
come is repetition. No matter how smooth our transitions, if we hear
the same pieces of audio repeatedly, they are bound to get stale.
Here is where DirectMusic provides significant strength!

Variable Playback/Variations
Content variability can be achieved on several different levels. On a
“microscopic” level, content can specify ranges for volume and pitch
on individual waves played in Wave Tracks, as well as randomized
volume, start time, and duration on notes in Pattern Tracks.

Figure 3-1: The Wave Properties page allows you to

specify ranges for attenuation and fine-tuning. This
particular example will randomly pitch-shift the wave
up or down by two semitones.

41
42 n Chapter 3

Figure 3-2: The Pattern Note Properties page

allows you to specify variability for velocity, start
time, and duration. This particular note will play
with randomization in start time, duration, and
with a random velocity between 75 and 125.

On the “macroscopic” level, you could of course author multiple Seg-

ments and write a script to choose between them randomly.
Then we have variations, which provide the best of both worlds.
Variations are those 32 intimidating-looking buttons present on Wave
Track parts and Pattern Track parts (in Segment files), as well as in
pattern parts (in Style files, a format which we cover in a bit).

Figure 3-3: The Variation Button window.

Each of those buttons represents one version of the part; for

instance, you could have a drum part where variation 1 includes the
full drum part, variation 2 removes the snare, variation 3 mixes up
the rhythms a bit, and so on.
The primary thing to remember about variations is that at any
given time, a single variation plays per part. This does not mean that
every single part is playing variation 12 at the same time, but it does
mean that if you author 12 variations for your drum Track, only one
is being played at any given time. Each part chooses a new variation
to play whenever its Segment is started or loops.
Before we get into the details of variation behavior and function-
ality, let’s talk a bit about the 32 buttons, which can be among the
trickier aspects of DirectMusic Producer to understand. When you
 Variation n 43

first create an empty part that supports variations, each of the 32

button labels is white. This indicates that these variations are empty.
As soon as you add any note, continuous controller, or wave informa-
I
tion to a variation, that variation’s label becomes black, which

Unit
indicates that it has some information in it.

Figure 3-4: Variation 1 has data in it, while variation 2 is empty.

One potentially confusing aspect of variations is that empty varia-

tions are just as acceptable to be “picked” to be played as variations
with data. For instance, a composer might create three alternate ver-
sions of a drum part on variations 1 through 3. When he then tries to
play the piece of music, he finds that most of the time it’s silent.
Why? Because variations 4 through 32 were enabled and empty
(silent), meaning 29 out of 32 times, an empty variation was chosen.
This brings us to disabling variations. When a variation is dis-
abled, DirectMusic will never “pick” it to be played. By default, all
variations are enabled. This is indicated in DirectMusic Producer by
a solid gray background for the button.

Figure 3-5: Variations 3 and 4 are disabled. Variation

3 has data in it, which typically should be deleted
(since the variation will never be chosen).

To disable one or more variations, make sure the appropriate varia-

tion is pressed by clicking on it; then right-click anywhere over the
32 variation buttons and choose Disable from the right-click menu
that appears. Disabled variations have a checkerboard look to them.
Note that it is possible to have a non-empty variation that is disabled.
In most typical scenarios, this means that you are saving data that
you will never use, so it is generally a good idea to go ahead and
erase any disabled variations when your project is completed.
44 n Chapter 3

Auditioning Variations in DirectMusic Producer

The final aspect of the user interface for variations is the buttons
being selected (“pressed down”) or not selected (“up”).

Figure 3-6: Variation 1 is selected; 2 is not.

The main thing to remember about this aspect of the buttons is that
it is solely for editing and auditioning purposes in DirectMusic Pro-
ducer — whether a button is pressed or not has no bearing on how
the variations will behave in a DirectMusic player or in a game at run
time. Pressing buttons provides for two purposes:
n Editing one or more variations simultaneously. Pressing a
single variation button means that you are now editing only that
variation. Paste from the clipboard, add a note, or drag a wave
into the part, and that data is assigned solely to that variation. If
you select multiple variations, notes and waves added will be
identically added to each of the selected variations.
n Auditioning variations within DirectMusic Producer. Let’s
say you created a four-part piece of music, and each part had ten
enabled variations. You listen to it a few times and notice that
there’s an incorrect note in one variation of the bass part. Rather
than forcing you to keep repeatedly playing the Segment until
that variation gets picked again, DirectMusic Producer has an
audition mode, where the same variation (whichever one is
“pressed”) will be chosen every time the Segment is played.
Only one part can be in audition mode at a time. To select a part
for audition mode, just click anywhere on it. The part that is in
audition mode is highlighted in a slight yellow shade.
 Variation n 45

Unit
Figure 3-7: Part three (piano) is in audition mode.
Within DirectMusic Producer, all other parts will
randomly choose which variation to play from the
enabled variation buttons. Part three will always
play whatever variation button is pressed (until it is
no longer in audition mode).

Remember that audition mode only applies in DirectMusic Producer;

when a Segment is saved and played in a DirectMusic application,
none of its parts will be in audition mode, and all parts will choose
from their enabled variations.
Audition mode provides for some unique behaviors not seen out-
side of DirectMusic Producer. First, as multiple variation buttons can
be selected (“pressed”) at once, you have the opportunity to hear a
single part playing multiple variations at once. As mentioned previ-
ously, only a single variation will be chosen at a time in an actual
DirectMusic application. You can also select disabled variations and
audition them; a DirectMusic application will never play a disabled
variation. Lastly, you can have all of the variations on your auditioned
part deselected, in which case no variation will be chosen, causing
this part to be silent. Again, this behavior is solely an audition fea-
ture of DirectMusic Producer.

Variation Shortcuts
There are a few shortcuts provided for quick selection of which par-
ticular variations you are editing. The vertical bar to the left of the
32 variation buttons selects (and with a second click deselects) all
enabled variations. Double-clicking on a variation will “solo” it (only
46 n Chapter 3

select that variation). Double-clicking again will “unsolo” it, revert-

ing to the previously pressed buttons.
If you are using linear wave files, a shortcut is provided to avoid
needing to work directly with the 32 variation buttons at all. From
the File menu, choose Import>Wave File(s) as Segment Varia-
tion(s)…. From the dialog that appears, choose all of the wave files
that you want to be variations in your Segment. A clock-time Seg-
ment is automatically generated, with a length of the longest wave
file you chose and using one variation per wave file. You’ll notice that
in this case, all of the empty variations (unless of course no varia-
tions are empty) are automatically disabled.

Specifying Variation Behavior Settings

Once you have created a Segment that utilizes variations, the next
logical step is to be able to more finely control how variations are
chosen. For instance, if we had 20 variations of a dialog fragment, we
would really like the person hearing this Segment not to hear the
same variation played twice in a row. Actually, it would often be ideal
to play every possible variation once before we ever get a repeat.
This kind of functionality can be specified by the audio producer from
each part’s properties page.
As usual, right-click a part and choose Properties (or if the
Properties page was already open, click the part to make it the active
property display).

Figure 3-8: Selecting per-part variation behavior

for a Pattern Track part.
 Variation n 47

Unit
Figure 3-9: Selecting per-part variation behavior
for a Wave Track part.

Though presented in different locations on the Pattern Track and

Wave Track property pages, the available options and functionality
are the same:
n In random order: The default behavior — a new variation is
randomly selected from the enabled variations every time the
piece of music plays or loops. Repeats can occur.
n In order from first: The first time the Segment is played, varia-
tion 1 (or the first enabled variation) is played. On each repeat or
loop, the next enabled variation in order is played.
n In order from random: The first time the Segment is played, a
random enabled variation is chosen. On each repeat or loop, the
next enabled variation in order is played.
n With no repeats: Random enabled variations are chosen on
each repeat or loop, but the same variation will not be chosen
twice in a row.
n By shuffling: On each repeat or loop, a random enabled varia-
tion is chosen, but every enabled variation will be played once
before any are repeated.
Recall that a new variation is chosen each time a Segment is played
or it loops. However, there are situations where we might want to
“reset” the variations. For instance, let’s say that each variation is
the next step up in an ascending scale, and we allow the Segment to
loop as the character picks up tokens. If they haven’t picked up any
tokens in the last few minutes, we want to stop the Segment and
start the scale over from the lowest note the next time a new token
48 n Chapter 3

is picked up. This is where the Reset Variation Order on Play option
(available in The Wave Track and Pattern Track property pages) can
be used to control whether DirectMusic “remembers” which varia-
tions were previously played the next time the Segment starts. If,
for instance, a Segment was set at In Order from First and its Pat-
tern Track had Reset Variation Order on Play checked, every time
the Segment was played, it would start from variation 1. If it looped,
it would then play variation 2, 3, and so on.

Figure 3-10: Reset Variation Order on Play

functionality for the Pattern Track.

There are, of course, instances where the audio producer would pre-
fer for two or more parts to always pick their variations identically.
For instance, in a DirectMusic arrangement of a jazz trio, there
might be several “solos” for each of the instruments (parts). We do
not want two instruments to “compete” by playing their solos at the
same time. For these kinds of situations, Lock ID functionality pro-
vides a way to control how different parts choose their variations.
Every part that has the same Lock ID will choose its variation to be
the same. Therefore, for the above scenario, if each part has three
solos to choose from, we could place the drum part’s solos on varia-
tions 1-3, the guitar’s solos on variations 4-6, and the piano’s solos
on variations 7-9, for example. For the variations where the parts do
not have solos, they would either “lay out” or just keep time. When
the Segment was played, each of the parts would choose its varia-
tions in step with the other parts, so only one solo would be played
at a time.
 Variation n 49

Unit
Figure 3-11: This Wave Track part’s Lock ID is set
to 1. Any other Wave Track parts in the Segment
that are set to Lock ID 1 will choose the same
variation as this part.

Note that Lock ID does respond to the other settings that we’ve dis-
cussed that control how variations are chosen; if one locked part has
variation 10 disabled, that variation will never be chosen for that
entire Lock ID group. Similarly, if one part is set to play its variations
in order, all of the parts will follow that part’s lead. Mixing and
matching between various part variation settings (for instance, one
part set to shuffle, another part set to play in order) within a Lock ID
group can lead to somewhat unpredictable variation choices, so it’s
generally recommended that all parts in a Lock ID group use the
same settings.
Another important concept to remember is that Lock ID is
unique per Track, not per Segment. That is, several parts in a Pat-
tern Track that use the same Lock ID will choose the same variation.
However, a Pattern Track part and a Wave Track part using the same
Lock ID will not lock to each other. A limited subset of this kind of
functionality is available in the Wave Track’s property page with the
Lock Variations to Pattern Track option. Checking this box means
that each pchannel of the Wave Track is now locked to the corre-
sponding pchannel of the Pattern Track; a Wave Track part on
pchannel 1 will always choose the same variation as the Pattern
Track part on pchannel 1 if one is present.
50 n Chapter 3

Figure 3-12: Locking a Wave Track’s variation to a

Pattern Track.

Locking a Wave Track’s variations to a Pattern Track is most useful

when you wish to use MIDI continuous controllers to manipulate
waves — panning, MIDI volume control (fade in, fade out, etc.), and
so on. By authoring corresponding variations in the Pattern Track,
you can ensure that a particular wave plays using particular MIDI
controller information. Alternatively, you could keep the Wave Track
unlocked from the Pattern Track, in which case any of your controller
variations might be used over any Wave Track variation.

Variation Switch Points

As we’ve already discussed, a new variation is chosen for each part
whenever that part’s Segment starts playing or loops. But what if we
want variations to be chosen more frequently? In particular, the
audio producer might create small fragments of melodic or rhythmic
lines that he wants to be able to switch between in the midst of a
Segment. Variation switch points provide the audio producer with a
solution to this — namely, the ability to determine if and when a part
can change variations in the middle of playback.
Variation switch points are displayed in DirectMusic Producer in
a separate strip for each Pattern Track part (they are not supported
by Wave Track parts). The two types of switch points are enter
switch points and exit switch points. Enter switch points tell
DirectMusic where it is legal to “jump into” this particular variation.
Exit switch points tell DirectMusic where it is legal to try to find
another variation to change over to. If no variation has an enter
 Variation n 51

switch point corresponding to this exit switch point, the currently

playing variation will continue to play.
I

Unit
Figure 3-13: A variation using variation switch points. This variation can jump to
any other variation at the position of the exit switch point (red blocks), shown as
the dark blocks in the Var Switch Points line. Similarly, this variation can start
playing mid-Segment if any other variation has an exit switch point
corresponding to this variation’s enter switch point (green blocks), the lighter
blocks in the Var Switch Points line.

While opening many possibilities for potential variation mixing and

matching combinations, variation switch points can make composi-
tion much more challenging. Rather than only having to worry about
how different variations meld at Segment loop points, variation
switch points open up the possibility of playback jumping between
variations right in the middle of playback, at whatever boundaries
they wish. Often it is easiest to limit variation switch points to mea-
sure boundaries, although they do support down to grid resolution if
you need it.

Influencing Variation Choices

DirectMusic provides even more acute functionality for controlling
which variations play when, particularly when the audio producer is
using chord progressions. We cover the various aspects of chord pro-
gressions (chords, Chord Tracks, and ChordMaps) in more detail in
Chapter 6, but for now just know that a composer can define a static
chord progression by placing chords in a Chord Track or a dynami-
cally generated progression by using a ChordMap and a Signpost
Track. Regardless of how the chords generate, a composer can elect
52 n Chapter 3

to have variation choice influenced by the function of the currently

requested chord in the current key. This functionality is in the Varia-
tion Choices window, more informally known as “The Mother of All
Windows.” The window opens from the button to the right of the 32
variation buttons:

Figure 3-14: The Variation Choices window.

In this window, each row corresponds to one of the 32 variations.

Each column allows the audio producer to specify exactly what kinds
of chords a variation will or will not be chosen for. For instance, you
might have a melodic line in a variation that sounds great played
over a IV chord (e.g., an F major chord in the key of C), but you have
another variation that provides a stronger option to play on a ii chord
(e.g., a D minor chord in the key of C). By selecting and unselecting
the various buttons in the Variation Choices window, you can add
these kinds of very specific restrictions. An unselected button means
that given a matching set of chord conditions, this variation will not
be chosen. Unlike with the 32 variation buttons, the button pressed/
unpressed state is actually used here to determine in-application
behavior. However, as with the variation buttons themselves, a dis-
abled variation’s row will be grayed out.
The Functions area allows you to specify specific chords that a
variation should or should not be played over. The capitalized version
of the chord (e.g., I) indicates the major version, the lowercase ver-
sion (e.g., i) is minor, and italicized (e.g., i) is for all other chord types
(augmented, diminished, etc.). The Root column allows you to spec-
ify whether a variation should play over notes that are based within
the scale (S), or based on a non-scalar tone that is flatted (b) or
sharped (#). Type allows you to determine whether the variation
should play over a triadic chord (tri), a chord with a sixth or seventh
(6, 7), or more complex chords (Com). Lastly, Dest, or destination,
actually looks ahead to what the next chord in the progression is
 Variation n 53

going to be. If the current chord is leading to the tonic (->I), a domi-
nant chord (->V), or any other chord (->Oth), you can instruct a
variation to never play in one or more of these situations.
I

Unit
Figure 3-15: One possible configuration for a variation.

Look at Figure 3-15. Assuming we’re in the key of C, this variation

will only play if the chord is C major (I), d minor (ii), F major (IV), G
major (V), or a minor (vi). It will also only play over scalar roots, so,
for instance, it would not play over Ab minor. The variation will play
over any chord “type” (triadic, 6th/7th, or complex). Lastly, this vari-
ation will only be chosen if the next chord encountered in the
Segment is a C major (I) or G major (V) chord.
Audio producers and programmers often ask if there is a way to
force a specific variation choice. The Variation Choices window pro-
vides one option for doing this. Without using any of the standard
chord functionality (notes being revoiced and/or transposed to fit the
chord), you could make every variation play over a unique chord and
then choose the particular chord to force that single “legal” variation
to be chosen.

Tips and Further Options for Variability

We’ve now covered several ways to create nonrepetitive content.
There are numerous other ways to add variability to audio content,
including (but not limited to):
n ChordMaps (see Chapter 6)
n Scripting (see Chapter 7)
n Style-based playback (see Chapter 4)
But the most common and powerful way to create nonrepetitive con-
tent is in the areas that we’ve already discussed — particularly using
pitch, volume, and duration variability, along with the 32 variation
buttons available per Wave Track and Pattern Track part.
Avoiding repetition for sound effects and ambience can be
accomplished by using pitch and attenuation randomization in combi-
nation with the use of multiple variations (typically each using a
54 n Chapter 3

unique authored version of a sound’s wave or waves — one for each

variation). If a sound effect is composed of several different aspects,
try varying the timing between them somewhat as well for even
more alternatives.
Let’s take the classic example of a footstep in a game. Tradi-
tionally, you might record five or six footsteps and randomly choose
between them every time your character moves forward on the
screen. But using DirectMusic, you could break the footstep into its
component elements — the sound of a heel, the sound of a toe, and
perhaps a separate scuff for the sole of the foot/shoe/whatever. Even
if you only record three variations of each, by placing each in a sepa-
rate Wave Track part, you now have at least 3 x 3 x 3 = 27 unique
footsteps. If you add appropriate random pitch range to each ele-
ment, you have even more footsteps. Now if you create added
variations where the actual start time of the various elements is
adjusted slightly (the scuff sound comes slightly earlier or later),
you’ve got even more potential “performances” of this footstep, with
no programmer assistance required.
For musical variability, consider occasionally adding or removing
single instrument lines via silent variations. Changes in orchestra-
tion from variation to variation are also a quick way to keep music
changing with each performance. Melodic lines, particularly those
that are closely associated with a chord progression, can be more dif-
ficult to structure convincingly in multiple variations — especially
when you stop to consider that every possible variation of a part will
have to play properly against every possible variation of every other
part. As nonmelodic instruments, drum parts tend to be more
straightforward for adding variations to quickly.
 Chapter 4

Interactive and
Adaptive Audio

S o far, we have covered playing back basic linear audio and adding
aspects of variability to that music. The next step is to make the
audio dynamic — that is, allow it to respond to events that the user
or program creates (i.e., interactive and adaptive audio). For
instance, computer games often want to convey a certain emotional
level based on what the current game state is. If the player’s health
is low and many enemies are around, the composer would often like
the music to be more frantic. This general concept of an “intensity
level” is often difficult to communicate to the programmer and even
more challenging to implement convincingly in an application.

Variable Intensity Music and Ambience

DirectMusic provides several potential solutions for creating
dynamic music. In this chapter, we cover two in particular — the use
of DirectMusic Style files and the use of Segment-based branching.

DirectMusic Style Files

Perhaps the easiest way to implement intensity level is to use
DirectMusic Style (.sty) files, which contain numerous musical frag-
ments, each with an assigned intensity level. Audio producers
compose one or more patterns (musical loops) within a Style. A pat-
tern is composed of a Pattern Track, those same MIDI-supporting
Tracks we’ve already used in Segments. Each pattern is assigned a
groove level range, DirectMusic’s term for musical intensity. When

55
56 n Chapter 4

the programmer alters the groove level based on application events,

a new pattern is chosen automatically based on the new intensity
requested.
When a new Style file is created (File>New…, or the Ctrl+N
shortcut), it appears in the project tree with several folders.

Figure 4-1: A newly created style file. Remember

that design-time file extensions end with p, so
this is a .stp file. The run-time version of the file
will be a .sty file.

The Bands folder contains one or more Bands (information about

per-channel volume, pan, and patch change settings) that can be
dragged into a Segment’s Band Track. Typically, you will just use one
Band per Style, and that Band is automatically inserted when you
drop the Style into a Segment. More information on that in a bit.
Motifs provide a subset of the functionality available to second-
ary Segments (and for history buffs out there, they are actually the
predecessor to secondary Segments). Recall that one or more sec-
ondary Segments can be played at a time and are typically layered on
top of a primary Segment, whereas only a single primary Segment
can be playing at a time. While motifs may appear somewhat conve-
nient (all living in a single file), there are several significant
drawbacks to motifs:
n They cannot be accessed directly using DirectX Audio Scripting.
DirectX Audio scripts can only play and stop Segments, not
motifs (which reside in Styles).
n While they can be played directly by a programmer, the com-
poser needs to manually embed a Band within the motif to
ensure that its expected instrument patch changes are applied
before it is played.
n While motifs support variations, it’s up to the game programmer
to keep using the same instance of the motif in order to avoid
resetting the variation order.
 Interactive and Adaptive Audio n 57

In short, for most cases, secondary Segments are much more useful
and convenient than motifs.
So we’re typically going to have a Style with a single Band, no
I
motifs, and several patterns. Our default Style has a single pattern. If

Unit
we double-click it, the Pattern Editor (looking quite similar to the
Segment Designer window) opens. MIDI controller editing is identi-
cal to the behavior in Pattern Tracks in Segments.

Figure 4-2: The Pattern Editor window.

There is a new Track at the top of the part called the Chords for
Composition Track. This Track, covered in more detail in Chapter 6,
tells DirectMusic what the original function of the music was so that
DirectMusic can make an educated choice when revoicing the music
to work over a new chord or key. For now, we leave it at the C major
default.

þ Note DirectMusic Style files only support Pattern Tracks and the
Chords for Composition Track and therefore DLS Instruments. In
particular, Styles cannot be used in conjunction with Wave Tracks, so
consider Segment-based branching if you are using prerendered
and/or streamed music.

Opening the properties page for a pattern gives us access to groove

range and other pattern-related functions. This is also where you can
adjust the length of this specific pattern.
58 n Chapter 4

Figure 4-3: The properties page for a pattern within a Style.

We’ve discussed groove range earlier. An audio producer can assign

a pattern to a single groove level if they wish or a range of appropri-
ate groove levels. Typically, a range of values is used. This allows for
two things:
n Pattern variability. If two patterns have overlapping ranges, a
requested groove level within the overlap will randomly choose
between the two patterns (all other aspects of the patterns being
equal — see below for more details on how a pattern is chosen).
n Easy addition of patterns later. Let’s say the composer cre-
ates three patterns — a low-intensity pattern (assigned to
groove range 0-25), a medium-intensity pattern (assigned to
groove range 26-75), and a high-intensity pattern (assigned to
groove range 76-100). After trying the application, the composer
might decide another pattern with intensity between “medium”
and “high” is appropriate. He can create a new pattern without
the programmer needing to make any changes to his code. By
contrast, using a single groove level, rather than a range, for
each pattern would probably require the programmer to change
how he determines the groove level.
The Destination Range option can only be used when you know what
groove level your music is heading toward — that is, generally only
when groove levels are authored directly into a Segment via a Groove
Track (see the “Using Styles with Segments” section later in this
chapter). With destination range, DirectMusic will actually “look
ahead” to what groove level is going to be needed next before
 Interactive and Adaptive Audio n 59

choosing which pattern should be played. While this could be some-

what useful when “ramping” from one groove level to another, the
idea of authoring groove levels into content is typically contrary to
I
the desire for dynamic manipulation of groove levels. For this rea-

Unit
son, destination range is rarely used.
The Embellishment area allows you to add even further control
to how a pattern is chosen. Embellishments are generally specific pat-
terns that you can call for without changing groove levels. For
instance, you might be playing a piece of music at groove level 50
and you want to transition to a new piece of music, also at groove
level 50. Rather than temporarily changing the groove level to pick a
convincing ending pattern, you could create a pattern still assigned
to groove level 50 that is an end embellishment. To further strengthen
the flow of the transition, the new Style could have an intro embel-
lishment pattern. Then when the programmer or scripter wants to
start playing the second piece of music, he can say to play ending and
introductory embellishments, and these specially assigned patterns
will be played between the two pieces of music. The naming conven-
tions for embellishments (intro, end, fill, break, and custom) are
actually somewhat arbitrary — from a programmer’s or scripter’s
point of view, there’s no difference between most of them. The
exceptions are that an intro embellishment is always picked from the
Style you are going to, and the end embellishment is always picked
from the Style that you are coming from. Of course, if you’re sticking
with the same Style and just wanted a brief pattern change, even
these have no difference in function. Note that custom embellish-
ment types (with a range from 100-199 to help keep them from
becoming confused with groove ranges) can only be requested pro-
grammatically or within content; DirectX Audio Scripting does not
support them.
The Chord Rhythm option adds yet another layer of control over
how DirectMusic chooses which pattern to play, assuming the Seg-
ment uses chords (in Chord Tracks or via ChordMaps). The audio
producer might have one pattern that is more ideal to a rapidly
changing progression and another that works better under a static
chord. DirectMusic will look for a pattern with a chord change
rhythm that maps closest to the currently playing Segment’s chord
rhythm.
60 n Chapter 4

Choosing a Pattern from a Style

So after all of these settings, how exactly does DirectMusic choose
which pattern to play at a given time? In descending order of
priority:
1. Look for the embellishment requested.
2. Look for the groove level requested.
3. Look for pattern matching destination groove level to next
groove change. Again, as destination groove level is rarely
known ahead of time, this particular rule is rarely used.
4. Longest pattern that fits available space (between now and
end of Segment or next groove change).
5. Look for a pattern with an appropriate chord rhythm.
6. In case of a match in all other settings, randomly choose one
of these matching patterns to play.
The fourth rule is often the most significant to keep in mind. The
argument here is that DirectMusic should pick the musically “stron-
gest” progression, which would mean to prefer to play a single
eight-bar pattern rather than a one-bar pattern eight times. But the
rule also applies when you have multiple longer patterns to choose
between. If you had been anticipating using variable length patterns,
it’s generally not easy to get around this restriction but it does allow
for a simpler composition and musical transition process.

Building Up Styles
The most basic use for Styles is what we like to call additive instru-
mentation. Each pattern is authored identically to the previous
(lower groove level) one but adds another instrument to the mix.
This can work quite effectively and is relatively easy to implement.
The audio producer creates a low groove level pattern (which could
be silence, basic rhythm, or perhaps just subtle ambience). They
then create additional patterns, each at a higher groove level and
each adding a new instrumental line or additional rhythmic
complexity.
Primarily due to the pattern-choosing rules discussed above, it is
often easiest to create every basic pattern with the same length.
Embellishment patterns of differing lengths can help break up the
 Interactive and Adaptive Audio n 61

rhythm somewhat, though all patterns must use the Style’s global
time signature for their playback length. Individual parts in a pattern
can be varying lengths and time signatures to help distract from the
I
feeling of constant time signature.

Unit
By the way, groove level and pattern length can be viewed in a
somewhat easier-to-read interface via the Style Designer window.
The window is opened by double-clicking on the Style itself in the
project tree rather than any of its patterns and displays information
on all of the patterns (their length, groove ranges, embellishments
they are assigned to, and so on).

Figure 4-4: The Style Designer window for a Style with five patterns for varying
groove levels. The highlighted pattern is an ending embellishment (as
indicated by the E in the Embellishment column).

Using Styles with Segments

Remember that the basic unit of playback (as far as the programmer
is concerned) is typically the DirectMusic Segment, not a Style file.
So we’re going to want to create a Segment that actually “plays” our
style. Several Track types come into play when instructing a Seg-
ment on how to use one or more Styles:
n Style Track: This Track is the basis for using Styles with Seg-
ments. Typically, the Style Track will consist of a single Style
placed in the first bar. However, there’s nothing wrong with
changing Styles as a Segment proceeds.
n Groove Track: This is the Track where the audio producer can
set the current groove level for the Segment. This Track also
allows for a Segment to manually request an embellishment
62 n Chapter 4

pattern (for instance, play an intro embellishment at the begin-

ning of the Segment). Inserting a Style into a Style Track auto-
matically makes changes to several other Tracks in a Segment,
inserting new Tracks if necessary.
n Time Signature Track: This Track becomes locked to the time
signature of the current Style, as you can’t override a Style’s
time signature in a Segment. If additional Styles are inserted
later in the Segment, similarly locked time signature events
(which appear “grayed out”) will appear.
n Tempo Track: The default tempo for the Style (specified in the
Style’s property page and the Style Designer window) is inserted
on the first beat of the bar where the Style was entered.
n Band Track: The Style’s default Band is inserted one tick
before the bar where the Style begins playback. This ensures
that all of the instrument changes and volume/pan settings are
applied just before the first notes are heard (assuming no notes
begin before beat one — if any notes do occupy the pickup bar,
you should manually move the Band even earlier). Note that the
Band inserted into the Band Track is a copy of the Style’s Band,
not a “reference” to it. That is, if you now go back and edit the
Style’s Band, those changes are not applied to the Band in the
Segment. Similarly, if you edit the Band in the Segment, those
changes won’t be reflected in the original Style’s Band. We typi-
cally manually drag the Style’s Band into the Segment just to
make sure we are not using an old version of the Band.
So all of these Tracks are automatically filled in for us. But what
about the Groove Track? Because the groove level is something we
want to be dynamically changed by the programmer while the appli-
cation is running, composers don’t typically author groove changes
into the Groove Track beyond an initial setting (typically the lowest
groove level) in the first bar.
 Interactive and Adaptive Audio n 63

Unit
Figure 4-5: A typical Style-based Segment. Groove level 1 is specified in the
first bar. When the Style was inserted, the other tracks were created and/or
modified automatically.

Before we completely dismiss the Groove Track, it is still quite use-

ful for auditioning groove changes within DirectMusic Producer.
After composing a bunch of patterns of varying groove levels, you
might create a long Segment, drop in the Style and a few groove
changes, and listen to how the performance sounds. You could even
use variable groove changes (the +/– option in a groove change’s
property page) or experiment with the embellishment types.
As far as the length of the Segment, you have several options. If
you are using patterns of varying lengths, you’ll generally want to
make the Segment the least common multiple of those lengths. For
instance, if one pattern is four bars and another is three, you’ll want
to make the Segment 12 bars long (and infinitely looping). This just
makes sure that, assuming a single groove level (and often a single
pattern) is played for a long time, the pattern won’t be interrupted or
cut short by several bars due to the Segment ending. Worse, remem-
ber that the pattern chosen is also influenced by how much space is
left before the end of the Segment — so you might get a completely
different pattern chosen right at the end if the pattern length did not
match the Segment length.
For the specialized (and easier to manage) case where all pat-
terns have exactly the same length, creating an infinitely looping
Segment of that same length is generally preferable. This will also
give us some added flexibility in how transitions between the groove
levels occur, particularly with the ability to successfully use align-
ment transitions.
64 n Chapter 4

Changing the Playing Patterns

If you’ve experimented with groove changes to audition your pat-
terns in a Segment, you might notice that the new groove level’s
pattern didn’t always take over immediately. The reason behind this
is that a groove change tells DirectMusic to choose a more appropri-
ate pattern when the currently playing pattern finishes. Certainly, the
application might change groove levels quite a bit, and you wouldn’t
want new patterns abruptly starting and restarting frequently. How-
ever, if you’ve created a lot of 20-bar patterns, the change to the new
groove level could definitely take quite a while and more time than
you would hope.
There are several options for making pattern changes occur
more frequently. The easiest will involve the programmer or scripter.
Every time he adjusts the groove level, he can also restart the cur-
rently playing Segment using any of the transition types we’ve
already discussed. For instance, the audio producer could instruct
the programmer to restart the Segment at the next bar. This means
that a groove change will “take effect” (a new pattern will begin play-
ing) within the next bar. Or, in the above example, the audio
producer could structure his 20-bar patterns such that they’re actu-
ally composed of five four-bar-long sections. The audio producer
could then put a Marker Track in the Segment (note that the mark-
ers are not in the Style file, as Style patterns don’t support Marker
Tracks) at the natural break points for the music, and the program-
mer could restart the Segment on a marker boundary. One difficulty
with this solution is that we are restarting the Segment (and thus
whatever the appropriate pattern is) every time we touch the groove
level. This can be problematic when smaller groove level changes
shouldn’t really affect the pattern we’re playing. For instance, if our
“low level” pattern spans groove levels one through 25, it doesn’t
really matter to us if the groove level changes from one to two. The
same piece of music should just keep playing. But in the simplest
implementation of the above solution, we’re retriggering the Seg-
ment — and thus restarting the pattern — every time the groove
level changes. Depending on how frequently these changes occur,
we could get very intimately acquainted with the first few bars of the
pattern and never hear the 20th onward. One way to combat this is
for the content creator to instruct the programmer as to when the
 Interactive and Adaptive Audio n 65

Segment should really be restarted versus allowing it to continue

with the newly adjusted groove level.
Based on the above issue with potentially restarting the playing
I
pattern from the beginning constantly, another commonly used tech-

Unit
nique is to use alignment to Segment transitions. Remember, in this
kind of transition, we jump to the same relative location in the desti-
nation Segment as we were at in the previous Segment. So if we
were at bar four beat two, we jump to bar four beat two of the new
piece of music. In the particular case of adjusting groove levels, both
the source and destination Segment are the same piece of music.
The retriggering is enough to bump us over to a new pattern more
appropriate for the current groove level. But with the use of align-
ment, we transition over in mid-pattern rather than restarting the
Segment. This technique is a bit trickier to manage than the previ-
ous one. Remember that if we’re starting in the middle of a pattern,
any notes that would have started playing prior to that point would
not be retriggered — so for instance, if our destination had some
nice string pads that had started on the downbeat, we wouldn’t hear
them if our transition occurred on beat two. An added complexity
occurs if our patterns had different lengths, as they now may transi-
tion to musically awkward positions in each other. This is another
reason why it’s often easiest to keep all patterns at the same length.

Figure 4-6: Attempting to transition using Segment alignment can be

difficult for patterns of differing lengths. In the above diagram,
Pattern 1 is four bars long, and Pattern 2 is five bars long. Below them
is a display of the 20 bars of a looping Segment, so we can see how
they line up over time. If we transition from Pattern 1 to Pattern 2 at,
say, the end of bar eight, we’re transitioning at the end of Pattern 1
but into the start of the fourth bar of Pattern 2. If both patterns had
been the same length, we would be making a potentially more
natural transition from the end of Pattern 1 to the beginning of
Pattern 2.
66 n Chapter 4

Segment-Based Branching and Layering

An alternative to DirectMusic Style files for creating variable inten-
sity music and ambience is to use multiple Segments. Remember
that with strictly Style-based playback, you are restricted to DLS
instruments. With Segments, you have the ability to mix and match
between DLS instruments and prerendered streaming wave files.
There are two primary ways that Segments can be used to shift
intensity: branching and layering.

Branching Playback
In the case of branching Segments, we change the Segment we are
playing at a musically appropriate boundary, based on some event
that changes the emotional level we want to convey. In the simplest
implementation, branching just consists of authoring complete
pieces of linear music for each emotional intensity, and then picking
logical places where one piece could jump to the beginning of the
next. Often measures are appropriate boundaries; if not every mea-
sure is, the composer can use markers to dictate which are.
Sometimes the above can result in abrupt and unsatisfying tran-
sitions, particularly if the two pieces of music were not written with
each other in mind. The jolt from playing a slow-moving piece of
music to a pulse-poundingly fast one might be appropriate in some
circumstances, but in others it can distract more than intended. A
potential solution here is to use transition Segments. As we previ-
ously discussed, when you tell a new Segment to start playing, you
can specify a transition Segment that should be played before this
new Segment starts. A common solution to the abruptness issue is
to create quick little “bridges” that ramp up or down the tempo,
smooth out any instrument entrances and exits, and so on. Of
course, this can lead to a bit of extra work on the composer’s part —
you might find yourself writing transition Segments for every possi-
ble configuration (slow to fast, fast to slow, slow piece 1 to slow piece
2, slow piece 2 to slow piece 1, and so on) unless you’re clear as to
what pieces could possibly jump to other pieces. Building up a
roadmap beforehand can aid both in the planning and authoring of
transition Segments.
 Interactive and Adaptive Audio n 67

Unit
Figure 4-7: A sample diagram for five pieces of music. Each arrow indicates
that a piece of music will need to be able to transition to another piece of
music. For this scenario, we assume that you have to somehow “engage”
enemies in combat, so you’ll never jump right to the “fighting” music from
the “no enemies” music. Similarly, if you decide to flee the room, you’ll
disengage from the enemies first, so you don’t have to worry about the
“fighting” music jumping directly to the hallway “ambient” music.

Yet another alternative use of branching is Segment alignment.

Remember that by default, a new Segment will start playing from the
beginning. Using alignment, we can make it start at the same posi-
tion where the last Segment left off. With this technique, we can
achieve results similar to the Style-based playback above, but with
the benefit of streamed waves. Using this technique, you could
author several different versions of the same audio track, with differ-
ent mixes and increased instrumentation in each version. Again,
using markers (in marker tracks) to control what are appropriate
boundaries, the music can seamlessly ramp up or down to another
wave based on the scenario.
68 n Chapter 4

Layered Segments
The other effective Segment-based tool for creating dynamic music
is layering. Here we keep playing the same basic piece of music, but
we layer other pieces of music over the top. This involves the use of
secondary Segments. Remember that only one primary Segment can
be playing at a time, and playing a new primary Segment automati-
cally stops any currently playing ones. But you can layer as many
secondary Segments on top of a primary Segment as you like. Don’t
forget that whether a Segment is played as primary or secondary is
in the hands of the programmer or scripter, so the composer needs
to communicate this information beforehand.
A common use of layered secondary Segments is as motific ele-
ments. Indeed, these secondary Segments replace the somewhat
outdated Style-based motifs, which were much less flexible and
required extra effort from the programmer or scripter in order to use
them. As an example, let’s say our character’s love interest enters
the room, and the love theme plays on top of the basic music track.
The use of Chord Tracks (described in more detail in Chapter 6) can
make this all the more effective. The love theme can be played in the
appropriate key, and even repitch itself based on the chord progres-
sion of the underlying music.
As a brief example, let’s take any old Segment we’ve already
authored and add a Chord Track to it if it doesn’t already have one.
(Right-click on Segment > Add Track(s)…, and choose Chord
Track from the list.) Now let’s right-click on the Chord Track (the
leftmost part of the track that says “Chords”) and bring up its prop-
erties. Odds are the track thinks our piece of music is in the key of C
because we haven’t told it otherwise. Mine happens to be in Eb, so I
update the property page like so:
 Interactive and Adaptive Audio n 69

Unit
Figure 4-8: Chord Track Properties dialog.

We’d probably also want to adjust that C major 7 chord that appears
in measure 1 by default. Let’s make it an Eb major 7 instead. (I’m not
going to bother entering the full chord progression, but we’ll use this
particular chord in a moment.) We right-click on the CM7, choose
Properties, and modify the property page like so:

Figure 4-9: Our Eb major seventh chord. Adjusting layer 1 (the

bottommost chord layer) defaults to automatically editing all
four layers, so you would only need to adjust the chord
(four-octave keyboard on the left) and the scale (one-octave
keyboard on the right) once. Right-click and uncheck
Auto-Sync Level 1 to All to change this behavior.
70 n Chapter 4

Now let’s create our romantic theme. We create a new Segment with
a Pattern Track and a Band Track (making sure that the Band and
pattern pchannels don’t interfere with any from our original piece of
music). As far as the actual music, I’ll just borrow a recognizable
theme from Tchaikovsky’s Romeo and Juliet:

Figure 4-10: Our (well, Tchaikovsky’s) love theme. Note that we’ve authored it in
the key of C (not having a Chord Track implies this) and named the Segment
“LoveTheme.” The Pattern Track part uses performance channel 17 (which can be
adjusted from the property page), and we disabled all variations except our
theme on variation 1. Our band similarly just has an entry for performance
channel 17 telling it what instrument to play.

If we played our theme as a primary Segment, it would play in C

major. But if we play it as a secondary Segment against our piece of
music in the key of Eb, it will transpose to this key. To try it out,
open up the Secondary Segment Toolbar (from the View menu,
choose Secondary Segment Toolbar), and choose our Segment
from one of the drop-downs.

Figure 4-11: The Secondary Segment

toolbar.
 Interactive and Adaptive Audio n 71

You can specify the boundary it will audition on by right-clicking the

Play button. Now play the primary Segment from the main transport
controls (click on the Segment to give it “focus,” which will make its
I
name appear next to the primary segment’s Play button). As it plays,

Unit
play our love theme from the secondary Segment toolbar’s Play but-
ton. The secondary Segment transposes to fit the key of our primary
Segment. Similarly, this theme could be played against any of our lin-
ear compositions in the proper key just by authoring a Chord Track
into those Segments and specifying the proper key and scale in a
chord in the track.
Transposition is one thing, but as a more advanced use, we might
want our theme to actually follow the chord progression of our pri-
mary Segment. Let’s add another chord to our primary Segment and
start off with a dominant-tonic (V-I) progression. We’ll move that Eb
chord to bar two (by dragging or copying and pasting) and insert a Bb
chord in bar 1.

Figure 4-12: Our new chord.

Figure 4-13: Our primary Segment.

72 n Chapter 4

Now when we play our secondary Segment as this primary Segment

plays, the notes will conform to the current chord as well as the
scale. We can adjust the specifics of how our secondary Segment
adjusts itself from its property page, and can even override these
settings on a per-note basis if we wish.

Figure 4-14: The Default Play Mode area determines how our
secondary Segment responds to chord changes. In the default
behavior, chord and scale are used to determine appropriate
notes, but there are also settings to only transpose according
to scale, to remain fixed to the authored notes (useful for drum
tracks), and so on.

Alternatively, secondary Segments can be played as controlling

secondary Segments. In this case, rather than taking on the character-
istics of the primary Segment, they force the primary Segment to
take on their characteristics of tempo, chord progression, and so on
— whichever tracks are present in the secondary Segment are used
to replace to the primary Segment’s. Going back to our previous
example, we can add a Chord Track to our secondary Segment telling
DirectMusic that our theme was indeed authored in the key of C,
over a C major chord. If we then played it as a controlling secondary
Segment over the primary Segment, the primary Segment would
now take on the chord progression of the secondary Segment.
 Interactive and Adaptive Audio n 73

Unit
Figure 4-15: Adding a new Chord Track and C major chord to our
secondary Segment.

Figure 4-16: Setting up a Play button in the

secondary Segment toolbar (by right-clicking on it)
to play a controlling secondary Segment. Don’t
forget that this is an audition setting only; the
composer would want to tell the programmer or
scripter that the Segment should be played as a
controlling secondary Segment.

Remember that Sequence Tracks do not respond to chord changes

— they are static, non-pitch-varying tracks, analogous to traditional
sequenced MIDI tracks. So the fact that our controlling secondary
Segment has switched out the chord progression won’t be heard in
the primary Segment unless any of its notes are played by Pattern
Tracks. But if our music was placed in a Pattern Track, and we had
traced out the chord progression for the music in our Chord Track,
the primary Segment would sound quite different over this new
74 n Chapter 4

chord progression. To repeat a useful scenario for this, background

music authored in a major key could switch to a minor key when the
villain enters the room, just by playing a controlling secondary Seg-
ment composed of minor chords. Indeed, there is nothing that says a
secondary Segment has to be composed of any notes at all — it is
perfectly acceptable to use them for Tempo, Chord, Style, Wave, and
any other tracks.

In this chapter, we have attempted to supply an overview and a basic

understanding of some of the power and flexibility DirectMusic can
bring to an interactive application. It can be used simply to provide a
MIDI-capable synthesizer, with the ability to deliver instrument col-
lections that ensure uniform content playback across multiple sound
cards. It can be used to synchronously play back real-time-rendered
(MIDI) and prerendered (wave) sounds in synch with each other.
DirectMusic provides the ability to seamlessly transition from one
piece of music to another at appropriate (and content-defined) bound-
aries. And it gives a rich set of tools to composers and sound
designers for creating variation and dynamically adjustable content.
Being aware of some of the rules and restrictions for how it can best
be used, I encourage you to experiment with the concepts presented
here. It’s often easiest to gain familiarity with a few tools at a time
rather than being overwhelmed by the options; this chapter has
merely touched the surface of the features DirectMusic supports.
 Chapter 5

DirectMusic Producer

W ow! That was a lot of theory to take in. Now we’re ready to dig
into using the program that does it all: DirectMusic Producer.
DirectMusic Producer is a complex program with a steep learning
curve. While this section is not meant to be artistic, it demonstrates
DirectMusic’s basics, as well as the fundamentals required to use the
program in a production environment. We also revisit some basic
ideas key to using DirectMusic, including variation, styles, and chord
changes. Our goal is to create a simple blues that uses DirectMusic’s
recomposition features to follow chord changes.

Getting Started
Insert the CD included with this book into your CD-ROM drive, as
we refer to MIDI files on this CD for the note data in this chapter.
Open DirectMusic Producer and create a new project by selecting
File>New from the menu and selecting a project in the dialog box.
Name your project, and choose a storage location for its data on your
hard drive. Select File>Import File into Project>MIDI File as
Segment from the menu. Navigate to the Unit I\Chapter 5\Tutorial
Part One directory on the CD and select the file Tutorial_Bass_
Melody.mid. The Segment opens automatically in the main screen
of DirectMusic Producer.

75
76 n Chapter 5

Figure 5-1: A DirectMusic Segment file, created from a MIDI sequence.

This Segment has several tracks of data in it. The first is a Tempo
Track, which controls the speed of playback. The second track is a
Chord Track, which controls how DirectMusic transposes other Seg-
ments over this Segment. The next two tracks are Sequence Tracks
containing note data. The final track is a Band Track, which controls
the instrument patching.
Often, when importing data from sequencing programs, the data
requires some amount of cleanup. In this case, there are several
problems; first, the Segment is two bars long, yet the music in the
Sequence Tracks is only one bar long. To fix this problem, right-click
on the title bar of the window and select Properties.

Figure 5-2: The properties dialog box.

þ Note Almost everything in DirectMusic has a properties dialog; our

Segment itself has one, as well as each track and each piece of data in
each track.

Click on the button next to Length, and set the number of measures
to one.
 DirectMusic Producer n 77

Patterns
I
The music we imported from the MIDI file is incredibly simplistic. It

Unit
is a single bar of a bass line and melody and hardly interesting at
that. However, music often exists as the collective of small patterns
like this one, which play over various chord changes. Our next step
is to make this small piece of music play over a series of chord
changes, using DirectMusic’s transposition functionality to adjust
the notes to match the new chords.
Currently, our music is stored as a Sequence Track in our Seg-
ment. Sequence Tracks do not transpose to follow chords, and play
exactly as the author wrote them. To get our melody and bass line to
transpose over chord changes, we need to use a Pattern Track
instead.
To create a new Pattern Track:
1. Select File>New.
2. Set the number of measures to one.
3. Select Pattern Track from the menu.
4. Select the Segment1.sgp (which was created) from the
menu on the left-hand side.
5. Right-click and select Rename.
6. Rename the Segment to BassSegment.sgp.
We need to copy the data from our old Segment into the new Seg-
ment’s Pattern Track. Select the Tutorial_Bass_Melody.sgt
Segment and maximize track one’s window. This opens the roll edi-
tor, where you can view and edit note data.
78 n Chapter 5

Figure 5-3: The roll editor displays note data,

which you can edit right in DirectMusic Producer.

Select the first note in the window and press Ctrl+A to select all
the notes. Press Ctrl+C to copy this data. Open Pattern Track one
in our new Bass pattern. Click on the first beat in the measure, and
press Ctrl+V to paste the data into the track. You may have to scroll
the track up or down to see the note data.
Create a new Segment named Melody.sgt with a Pattern Track,
and repeat this process with track two. At the end of the process,
you need to point the Pattern Track to channel two by right-clicking
on the Pattern Track to bring up the properties dialog box and set
the pchannel to two.

Setting the Chord Reference

For DirectMusic to transpose a melody to a new chord, it needs to
understand the relationship of the melody to its chord and scale. In
our case, we base the bass line and melody off a C7 chord.
Add a Chord Track to the bass and melody Segments by right-
clicking in the Segment window and selecting Add Track and Chord
Track from the menus. Right-click on the first beat in the measure
of the Chord Track and select Insert to insert a chord.
 DirectMusic Producer n 79

Unit
Figure 5-4: We define the chord and scale reference in the Chord
Properties page. Here we use a C7 chord with a C Mixolydian scale.

On the bottom piano roll, set the chord to C7. Do this by clicking on
the appropriate notes on the piano roll (C2, E2, G2, and Bb2). Now
adjust the scale by changing the B natural to a Bb. Repeat this pro-
cess for the second part as well.

þ Note You can copy the chord from the Chord Track and paste it in
the other Segment’s Chord Track, rather than resetting the notes
manually.

Creating a Primary Segment

Now that both of our parts reference chords and scales, we must cre-
ate a chord progression for them to play over. Create a new Segment
called MasterSegment.sgt, set its length to four bars, and add a
Chord Track. Place a series of chords in this Chord Track for the bass
and melody parts to follow by selecting bar one and pressing Insert
to insert a new chord. Use the following chord progression:
C7 > F7 > G7 > C7
The notes and scales for these chords are shown here:
Chord Chord Notes Scale
C7 C E G Bb C D E F G A Bb
F7 F A C Eb C D Eb F G A Bb
G7 G B E G C D E F G A B

Figure 5-5: Adding your melody and bass line.

80 n Chapter 5

Add two Segment Trigger Tracks to MasterSegment.sgt. A Segment

Trigger Track allows you to include another Segment inside of your
Segment, which is extremely useful for organization. In your Seg-
ment Trigger Track, right-click on the first beat of the first bar and
select Insert from the menu. Select your bass Segment from the
drop-down box in the properties page. Now copy and paste this Seg-
ment into all four measures on the Segment. Repeat this process on
the second track to include the melody as well. Play this Segment.
Notice that your instrument settings vanish, and both tracks sound
like a piano. To fix this, add a Band Track to the Segment, and copy
the Band from your original, imported Segment file into this track.
When you are done, MasterSegment.sgt should look like this:

Figure 5-6: Your completed master Segment.

Play the Segment and notice how the notes now transpose to follow
the chord progression.

Making Our Music Cooler

Play the Segment and notice that both the bass and melody move in
a linear fashion. That is, when the chord changes from C7 to F7, both
lines move up a fourth in parallel. If two musicians were to play this,
it is unlikely that they would move through these changes in such a
strict parallel fashion. DirectMusic allows us a number of ways to
specify how a given part should move through the chord changes.
Open the properties page onto the F7 chord. Notice that there
are four different chord and scale levels. Use these different levels to
 DirectMusic Producer n 81

specify alternate ways in which DirectMusic should interpret the

chord. On the second level of the chord, change the chord mapping
so that the top two notes of the chord are on the bottom of the chord.
I
It should now be composed of C, Eb, F, and A, which is the second

Unit
inversion of the chord. For the G7 chord, set the second chord to the
second inversion as well (in this case, C, E, G, and B). Your chord
should look like this:

Figure 5-7: Setting up alternate ways for DirectMusic to interpret the G7 chord.

Now go back to your melody Segment file, right-click on an area of

the pattern where no notes are found, and bring up the properties
dialog. In the bottom, right-hand corner under Default Play Mode,
set the chord level to Chord Level 2 so that this pattern follows the
chord mappings specified in the second layer of the chord.

Figure 5-8: Adjusting chord levels.

82 n Chapter 5

Play MasterSegment.sgt again. Notice that while the bass is mov-

ing linearly through the chord change, the melody now uses the
inversion to determine the root note of the melody. Save your
project.
In case you are lost, there is a completed version of this section
of the tutorial on the companion CD located in Unit I\Chapter 5\
Tutorial Part One.

Variations
Now that we have a basic piece of music playing over some chord
changes, we want to add some variation to the playback so that each
time our bass line plays, it plays a little bit differently. Open the
BassSegment.sgt that we created earlier, and open the Pattern
Track. Notice the variation bar. By default, DirectMusic Producer
fills in all of these variations with the same data.

Figure 5-9: The variation bar.

Select only the variation to work on. Notice in the above figure that
we selected all variations because every number in the bar is
depressed. You can quickly select or unselect all variations by click-
ing the vertical bar on the leftmost edge of the variation bar. Click
this bar so that every variation number is not depressed, then select
variation 2. Your variation bar should look like this:

Figure 5-10: Only variation 2 is selected.

In the pattern, raise the note on the second beat to an E, and lower
the last note to an E. Your pattern should look like this:
 DirectMusic Producer n 83

Unit
Figure 5-11: Our first bass line variation.

If we play the project now, we would only hear this variation one out
of 32 times, since DirectMusic fills out our 32 Segments for us by
default. To have the variation play more often, disable all variations
except for variations 1 and 2 by deselecting variation 2 and selecting
variations 3 and higher. Right-click on the Variation tab, and select
Disable. Your Variation tab should now look like this:

Figure 5-12: Variations 1 and 2 are enabled, while all others are disabled.

Play the master Segment, and notice how both our original bass line
and our variation play.
Again, we saved a completed version of this section of the tuto-
rial on the companion CD in the folder Unit I\Chapter 5\Tutorial Part
Two.
84 n Chapter 5

Styles
In the previous two sections of this chapter, we stored our musical
data in Pattern Tracks. Pattern Tracks have a lot of power, as they
can follow chord changes in multiple ways and contain up to 32 varia-
tions. However, even more powerful than patterns are Styles. As
defined in Chapter 4, a Style is a collection of patterns, which
DirectMusic chooses based on various parameters. Each pattern
within a Style acts just like Pattern Tracks, containing chord
mappings and variations. Moreover, because you can place multiple
patterns within a Style, they allow you an unlimited number of varia-
tions. More importantly, they respond to the groove level.
While groove level can be used in many ways, it is easiest to
think of it as an intensity filter. Each pattern within a Style has a
groove range in which it is valid. Our melodic line might be simpler
at groove levels one through ten, while a more active one is used
from 11 to 20. When the Style is choosing the pattern to play, it looks
at the current groove level and determines which patterns are valid
within the Style. A typical game implementation involves tying
groove level to the intensity of the current game situation. If there is
a lot of action going on, the groove level raises and the music inten-
sifies, and when the action dies down, the groove level lowers and
the music calms down.

Creating Our First Style

Create a new Style by clicking File>New and selecting Style from
the menu. In the left-hand menu, open the folder in the Style named
Patterns. Double-click on the pattern to open it. Using the same
techniques we used in the first tutorial in this chapter, copy the data
from our melody pattern to this Pattern Track. You will need to set
the pchannel for this part to two, as well. Now select the pattern in
the left-hand menu, and press Ctrl+C to copy it into memory and
Ctrl+V to paste it. Your Style should now have two patterns called
pattern one and pattern two. Open pattern two, and adjust the notes
to make the melody into more of an accompaniment Style line. You
can use the right-click menu on the mouse to add new notes if you
need to. Our accompaniment looks like this:
 DirectMusic Producer n 85

Unit
Figure 5-13: Our accompaniment pattern.

Rename the Style Piano.

Open our MasterSegment file, and use the right-click menu to
delete our current melody Segment Trigger Track. Use the right-
click menu to add a Style Track. Right-click on the first bar in the
Style Track, and select Insert. Select our Piano Style from the
menu.
Press Play on the master Segment and notice that both patterns
are chosen from our Style. Also of note is that we do not have to
place references to our Style in every measure of the master Seg-
ment. This is because a Style Track is assumed to repeat, unlike our
Segments, which have a fixed length.

Adding Groove Level

Go back to our Style, and bring up the properties on pattern two.
Notice that there is a Low and High setting under Groove Range.
This sets which groove levels this pattern is valid. Set these num-
bers to 1 and 10, respectively. This will make this pattern only viable
from groove levels one to ten.
Now select pattern one, and set its groove range from five to
100.
Go back to our master Segment, and use the right-click menu to
add a new track. This time, select Groove Track from the menu.
Select the first bar in the Groove Track and press Insert.
86 n Chapter 5

Play the master Segment, and notice that only pattern two is
used. This is because our groove level is set to 1, which is not a valid
groove level for pattern one to play. Now set the groove level to 6 in
our Groove Track, and play the master Segment. You will notice that
both patterns now play. This is because both patterns are valid, so
DirectMusic will choose between them every measure.
Set the groove level to 11 in our Groove Track. You will notice
that only pattern one plays, because pattern two’s groove range is
too low to play. Save your project.
The wonderful thing about using groove levels to control your
music is that it’s very easy to work with and test on the musical end
and provides a simple mapping for a game to use that doesn’t require
your programmers to understand how your music is composed.
Often, groove level can be tied to something very simple for a game
engine to understand, such as the number of monsters in the area,
and the composer can decide what the music should do in these
cases.
Once again, we saved a completed version of this section of the
tutorial on the companion CD in the folder Unit I\Chapter 5\Tutorial
Part Three.

This chapter offered some insight into the basic mechanics of using
DirectMusic Producer. While DirectMusic Producer can seem daunt-
ing at first, once you become familiar with the interface, you will find
it efficient and well designed. Remember to start small and be pre-
pared to build several projects as a learning experience.
 Chapter 6

Working with Chord

Tracks and ChordMaps

O ne of the most powerful and revolutionary features of

DirectMusic is its ability to handle chord progressions and
scales. We first discussed the power of chords in DirectMusic in
Chapter 4 and took a brief look at how to harness some of that power
in Chapter 5. As you know, audio producers can apply chord changes
to their music by entering chords into a Chord Track. Beyond this,
they can create ChordMaps that define different paths that chord
progressions can follow. Since chords, ChordMaps, and chord pro-
gressions are undoubtedly some of the most complex and daunting
features, we have devoted an entire chapter to the topic.
Be prepared; it is going to be a bumpy but satisfying ride. In the
end, you will be able to do some incredible musical feats with chords
in DirectMusic! This chapter offers a thorough demonstration of
DirectMusic’s chordal and scalar abilities. For another example of
connecting chords, open the file ChordMapTutorial.pro located in the
Unit I\Chapter 6 folder on the companion CD.
Oh, and it is assumed the reader has an intermediate under-
standing of chord and scale theory.

Chord Tracks
This section demonstrates designing music that utilizes nonvariable
Chord Tracks. Two Segments are created using the same pattern,
but each will have their own chords and scales.
First, create a pattern:
1. Create a Style called SimpleStyle.
87
88 n Chapter 6

2. Enter the following notes into the Style’s only pattern:

Figure 6-1: This is a C major Alberti Bass figure (arpeggio) followed by a

sixteenth note scale run.

3. Create an eight-bar Segment called MajorChordTrack.

4. Add a Chord Track (not a ChordMap Track) and a Style Track
to it.
Select SimpleStyle in the tree view and drag it into bar one beat
one of the Style Track.
If you play the Segment, you will hear the pattern that you just
entered play eight times.
Since there are no chords in the Chord Track, the pattern repeat-
edly plays over a C chord. Make things more interesting by adding
some additional harmonies.
1. Select bar one beat one in the Chord Track and hit the Insert
key (or right-click and select Insert).
A new C chord appears.
2. Do the same thing to bar two beat one.
3. Click the up arrow next to 2C five times to make the chord
an F chord.
Next, we want a ii chord. Add a C major chord to bar three and trans-
pose it to D. To make it minor, we need to edit the chord. Start by
 Working with Chord Tracks and ChordMaps n 89

changing the label from M to min. The chord type labels have no
effect on playback; they are there for you to customize. If you prefer
another way of labeling chords, feel free to use that.
I
When we edit the notes on the keyboard display in the Chord

Unit
Properties window, we only want to edit the notes in the row labeled
1 on the left-hand side. Later we will go into what the four chord lev-
els are for. We can edit the chord tones and the underlying scale. To
edit the third chord, click the F# in the chord section. This deletes
that chord tone. Click the F key to give us a D minor chord. It is
okay to leave the chord with only two tones, but in this case, we
want D minor. Here is what it should look like when you are finished:

Figure 6-2: Chord Properties dialog box.

You can generally create a duplicate of anything in DirectMusic Pro-

ducer by Ctrl+dragging it. Ctrl+drag the F chord to bar four. Now
transpose the F chord to G. The Chord Editor page is simply the
properties page, so hit F11 to bring it up again or right-click the
chord and select Properties. You can copy multiple chords simulta-
neously as well. Click the C chord in bar one and Shift+click the G
major chord. Ctrl+drag those four chords to bar five. Standard cut,
copy, and paste works also.
Lastly, we want an A minor chord in bar five instead of C major,
so select that chord and delete it. We could have edited the C major
chord to make it A minor, but it takes fewer steps to copy and trans-
pose another already existing minor chord. Ctrl+drag the D minor
chord into bar five and transpose it down to A minor. The finished
Segment should look like this:
90 n Chapter 6

Figure 6-3: Play the Segment to hear the pattern cycle through all the chords.

Next, we create a minor key version of the same chord progression.

This is going to require more work, since we need to adjust the
scales in addition to the chords.
1. Create a new eight-bar Segment and name it
MinorChordTrack.
2. Add a Style and Chord Track to MinorChordTrack.
3. Drag SimpleStyle into the Style Track.
Before we add any chords, we must adjust the key of the Segment.
Right now, any chord that we create has the underlying scale of C
major because our key is set to C major and we do not want to have
to edit the whole scale on every chord. If you pull out the dividing
line between the tracks and their labels, you can see that the Chord
Track is labeled like this: Chord (Key=C/0#s). If you bring up the
properties window for the Chord Track, you can change the key. Do
this by left-clicking the Chord Track label and hitting F11 if the prop-
erty page is not already open. Select the Key tab. We are going to
use C minor, which has three flats, so the first thing we need to do is
click the Flats radio button. Next, raise the key signature value to
three flats.
Now we can start entering the chords. Insert a chord into bar
one beat one. This chord will be a perfect C minor chord with the
wrong label. Change the label to min. Notice the underlying scale is
C minor instead of C major. Copy the C minor chord to bar two and
transpose it to F minor. For bar three we want a ii7b5 (half-
diminished) chord. Start with a D minor chord and change the A in
the chord to an Ab. You can add a C above the Ab to make it truly a
min7b5 chord, but for our purposes, it will not matter. Label that one
Dmin7b5 or whatever else you like. Bar four will be G major. Use
your keyboard’s Insert key to add the chord this time. Transpose it
up to G major, and change the third to B natural.
 Working with Chord Tracks and ChordMaps n 91

We are still not done with the G chord. The chord is correct, but
the scale is still not. Notice when you changed the Bb to B natural
that the B natural in the scale side turned blue. This is Direct-
I
Music’s way of warning you that there is a chord tone that does not

Unit
have a matching scale degree. You should always include the chord
tones in your scale, or DirectMusic may get confused about how the
scale tones relate to the chord tones. Click the blue B natural to add
it to the scale. It should turn red like the other tones. Lastly, click
the Bb to remove it. The scale is now C harmonic minor. Copy the
first four bars into the last four, like we did with the major Segment.
Transpose the C minor chord in bar five down to Ab, and make it
major instead of minor. This is the final chord progression:
| Cmin | Fmin| Dmin7b5 | GM | Abmaj | Fmin | Dmin7b5 | GM |
In this section, we used DirectMusic chords to take a simple, one-
measure pattern and created two different chord progressions with
it. This can be a quick way to write music if all that is required is to
move the music through different chord progressions. This section
serves as the foundation for the next section on variable chord
progressions.

ChordMaps
ChordMaps are sets of potential chords or chord paths for chord pro-
gressions to follow. Introducing a ChordMap into a Segment allows
DirectMusic to generate chord progressions on the fly, adding a new
element of variability to DirectMusic songs. DirectMusic pieces the
chord progression together within the boundaries provided by the
content creator’s ChordMap parameters. This section includes
examples of variable chord progressions and instructions on how to
use them.
Create a ChordMap called SignPostsOnly by using Ctrl+N and
selecting ChordMap. If you have seen ChordMaps before, you may
have noticed the charts with lots of little connecting lines going
everywhere. The truth is, you do not need to put anything into that
area of the ChordMap to get a variable chord progression.
The most vital thing in a ChordMap is the signpost. Signposts are
groups of chords from which DirectMusic chooses while generating a
92 n Chapter 6

chord progression from the ChordMap. The signpost list is located

on the right side of the ChordMap editor (see Figure 6-4). Under SP
(SignPost), you’ll see the word <new>. Click on that, and a new
chord will be created. You can edit the chord from there. If you
check a box under the number 1 and next to your chord, your sign-
post chord will become a member of signpost group one. When you
start working in your Segment, the signposts are the main way that
you control how the ChordMap is used within the Segment. You will
be able to specify which signpost group you want the engine to
choose from at that given time.
There is an easier method to employ in the creation of chords
here. The chord on the left side of the screen can be dragged into the
signpost list. The chords on the left have nothing to do with the way
the ChordMap sounds. That is just a palette for you to grab chords
from. Just like a painter’s palette, you can change and customize the
palette. All the chords on the palette are editable. To start with, they
are all major chords, but if you look up next to DirectMusic Pro-
ducer’s main View drop-down menu, you can see a ChordMap
drop-down menu. From there, you can change the palette to all
minor, major, major seventh, dominant seventh, minor seventh, or
chords based on the underlying scale of the ChordMap.
To show flats instead of sharps in any of the ChordMap’s various
components (in this case, the palette) click on the ChordMap in the
tree view and go to the Chord Properties window. There is a check
box that lets you specify flats along with the key of the ChordMap. It
is a good idea to set this first, since changing the key will transpose
all chords in your ChordMap.
For this example, add these chords to the signpost list: C, F,
Dmin, G7, Ab7, and Db7. Once you’ve got the chords entered, check
group one for C, group two for Dmin, F, and Ab7, and group three for
G7 and Db7. If you are a theory buff, group one is tonic, group two is
subdominant (or a substitution), and group three is dominant (or a
substitution). Figure 6-4 shows what the signpost list should look
like.
 Working with Chord Tracks and ChordMaps n 93

Unit
Figure 6-4: The signpost list.

Now we can make a Segment to use the ChordMap. Create an eight-

measure Segment and name it SignPostOnly. You will need a
ChordMap Track, a Signpost Track, and a Style Track. To create the
ChordMap and Signpost Tracks, use Ctrl+Ins or right-click and
select Add Tracks. Then select the appropriate track type to create.
Insert SimpleStyle into the Style Track and your new ChordMap into
the ChordMap Track (bar one beat one). Click bar one beat one of
your Signpost Track to insert a signpost. Choose group one. Fill out
the rest of the Segment so the signposts follow this pattern:
| 1 | 2 | 3 | 1 | 2 | | 3 | | (also shown in Figure 6-5)
Now we can generate the chord progression from the ChordMap.
This is done by clicking the dice icon in the Signpost Track. When
you do so, a ChordMap Track appears and is filled in with chords.
Notice that for all bars where groups two and three are entered, the
same chords are used. If you click Compose again, it selects new
chords for groups two and three again and uses only those chords
instead of the previously generated chords. Group one only has one
chord; that’s C major. If you do not want to have to click the dice all
the time, you can call up the signpost track’s properties page (select
the blue track label) and go to the Flags tab. Check both Recompose
on Play and Recompose on Loop. The Recompose on Play flag
automatically selects new chords every time the piece plays. The
Recompose on Loop flag automatically selects new chords every
time the section loops.
Here is what the Segment should look like. Note that the chords
will most likely be different:
94 n Chapter 6

Figure 6-5: Signpost demonstration.

You are probably itching to play with all the neat graphs, so let’s do
that next!
You can define different paths for the chords to take when mov-
ing from one signpost to another. Do this by mapping out chords
between the different signpost chords and drawing lines from chord
to chord. You can experiment with the map that we just created by
dragging all the chords from group two to bar one and all the chords
from group three to bar three. Right-click each of the chords in mea-
sure one and make them beginning signpost chords by selecting
Toggle beginning signpost from the pop-up menu that appears.
Beginning signposts are signified by a green arrow. Similarly,
right-click all the chords in measure three and make them all ending
signpost chords. Ending signposts are signified by a red circle. Put
whatever chords you want into bar two, and start drawing lines from
one chord to another. The lines are drawn from the lowest gray box
on the end of the chord. An empty box is always on the bottom since
a new one is added every time you create a new connection path.

Figure 6-6: Connecting chords.

 Working with Chord Tracks and ChordMaps n 95

If you go back and recompose (i.e., regenerate) the Segment above,

bar six should have one of the connecting chords that you just added.
This happens because once connections between two signposts are
I
drawn, DirectMusic will always use the connection lines if there is

Unit
room in the Segment. The area between bars five and seven is the
only place where there is a move from signpost two to signpost
three, and there is a measure open for the connecting chord. If you
want the chord progression generated by the ChordMap to be able to
choose randomly between going directly from signpost to signpost
and using the connecting chords, draw a line that connects the two
signposts directly. Otherwise, DirectMusic will always use the con-
necting chords if there is room. To delete a connection, click on the
line and press the Delete key.
For another example of connecting chords, open ChordMap-
Tutorial.pro on the companion CD. Check out the bookmark
Connecting Chords. The ChordMap only uses one signpost chord
that is placed into groups one and two. The map (see Figure 6-7) is
meant for a Segment that is five measures in length with signpost
one on bar one and signpost two on bar five. The connecting chords
play on bars two, three, and four. The strategy here is that bar five is
never actually played. The Segment is set up so that it loops after
bar four. Bar five is only there to hold the target signpost so the con-
necting chords can be composed. This is a good strategy for looping
Segments that need to transition back to bar one. You can’t compose
connecting chords back to bar one, so you have to make a duplicate
that never plays.

Figure 6-7: Connecting chords example 2.

96 n Chapter 6

The ChordMap in the example above has to be played in a five-

measure Segment that is in 4/4 time to get the connecting chords to
play properly. If you select the variable editing option in the Chord-
Map Properties dialog, the chords can now occur at different time
intervals than they are laid out in the ChordMap. To change the time
intervals that a chord can play, bring up the properties page for the
connection line. Here you can adjust minimum and maximum
lengths and even the weight — the probability that that connection
will be selected.
One other ChordMap feature is the cadence. This could be useful
in a jazz situation where you just want to put in the keys and have
the ii-V-Is come in automatically. Cadences are good for that, but the
chords do not have to be standard classical cadences. Your cadence
could be bVI, bV, i.
Any signpost can have an associated cadence. The two slots to
the left of the signpost in the signpost list labeled C1 and C2 are for
cadence chords (see Figure 6-8). Drag the chord or chords you want
to use into those slots. Then when you want to cadence to a sign-
post, check the Precede Signpost with Cadence check box in the
properties page for signposts in your Segment’s signpost track.
There is a Cadences bookmark in the accompanying project that
gives an example of this.

Figure 6-8: Cadences.

Chords for Composition

One thing has probably been bothering you: “Do I have to write
everything in C major?” The answer is no, you do not. There is a
feature called chords for composition that defines the context of your
music. This is important since DirectMusic usually remembers
notes as a combination of chord and scale degrees and not as actual
MIDI notes the way a traditional sequencer would.
 Working with Chord Tracks and ChordMaps n 97

If you look at any pattern in a Style, you will see a Chord Track
along the top. This defines the context of the pattern’s notes. All the
notes in the pattern are compared to the notes in the chord above
I
them. If you change the chord to Gb minor, the notes will be inter-

Unit
preted in the context of Gb minor. You could even write in Lydian b7
and have that interpreted correctly by the DirectMusic engine.
Look at the bookmark ChordForComposition. The chord for
composition for the pattern is E7#11, and the music uses the notes
in that scale and chord. In this context, an A# is a natural four
because A# is the fourth scale degree of the underlying scale of the
chord for composition. If this pattern is played over a C major chord,
the A# will be played as an F, since F is the natural four in C major.
Play the Segment above the pattern to hear the unusual E Lydian b7
scale be converted perfectly to C major, C minor, and C Lydian b7.
The last chord is the same as the chord for composition, E7#11.
Naturally, the output is the same as the notes that were originally
entered into the pattern with that chord.
You are not required to stick with one chord for composition. You
can treat the Chords for Composition Track as a regular Chord Track
and enter as many chords as you want. Whatever notes are in the
pattern below the Chord Track are interpreted in the context of their
corresponding chords. Check out the MultipleChordForComposition
bookmark for an example. The pattern has a C major arpeggio fol-
lowed by a G major arpeggio. The chords for composition are C
major and G major, respectively, but the Segment that plays the pat-
tern has only one C major chord. As a result, the G chord is played as
a C major chord also.
Keep in mind that when you run-time save your files, all chords
for composition information are lost, since it is only for the author’s
benefit. DirectMusic remembers notes in terms of functionality, not
MIDI notes (in most cases; we go over the other cases later). We
have had to recover lost files from the run-time version before and
restore our chords for composition information. It can be scary to
open your project and hear all the work you did with lots of strange
harmonies played in a C major context.
The problem above is easy enough to address. If you right-click
the Chords for Composition Track and change the Keep Constant
option from MIDI Notes (M) to Functionality (F), you can enter in
your original chord for composition and everything will be restored
98 n Chapter 6

immediately. Make sure you change the setting back to MIDI Notes
though. MIDI Notes can ruin a project if you accidentally change the
Chords for Composition Track or pattern key, and all your music
gets moved around inappropriately. If you keep the MIDI Notes set-
ting on, the notes in the pattern will never change visibly. If you
change a chord for composition, DirectMusic reevaluates the notes
in the new context.
If you have Keep Functionality selected and you change a chord
for composition, all the notes will move to keep the same functional-
ity with the new chord that it had with the old chord. For instance, if
you have a C major chord for composition with an E in the pattern,
the note is remembered as the third. If you change the chord for
composition to G major with Keep Functionality selected, the E will
transpose to a B since B is third of G major. If you have MIDI Notes
selected, the note will stay E, but it will now be remembered as a
sixth, since E is a sixth in G.
There is a problem using Pattern Tracks, as opposed to patterns
in a Style. The problem comes from the fact that the Segment’s
Chord Track functions as a Chords for Composition Track for the Pat-
tern Track. This can be very confusing, since the notes will be
shown on screen according to whatever chord is in the track at the
time you open the Segment. If you hit Play and the Segment
recomposes, the notes already displayed do not appear to change.
When you begin editing notes, remember to look at your Chord
Track first. Make sure that you are thinking about that chord. If you
recompose, the context will change again. It is probably better to
turn off recomposing while editing a Pattern Track. Even better, put
the Pattern Track in a separate secondary Segment; that way, you
always know that the chords for composition will stay the same.

PlayModes
The chord for composition sets the context of our pattern, but there
are other features that we can use to direct the DirectMusic engine
on how our patterns should respond to chords. Some of these fea-
tures are playmodes, note properties, chord levels, pattern chord
rhythms, and variation choices. We go through them one by one,
learning what each does and how to use them.
 Working with Chord Tracks and ChordMaps n 99

Let’s start with using playmodes. Open up the pattern in the

SimpleStyle Style. Open the properties page for the part. On the
lower right is a section labeled Default Play Mode. The first drop-
I
down box is set to Chord/Scale. When the pattern is played with

Unit
Chord/Scale selected, both the chord position and scale degrees are
considered when deciding what note is played over a certain chord.
The chord position is which chord tone we are talking about. For
instance, in a C major chord, C is chord position one, E is chord posi-
tion two, and G is chord position three. If it were C7, Bb would be
chord position four.
In the sample project, visit the Playmode bookmark for a demon-
stration of the Chord/Scale playmode. Notice that the Segment has a
chord that is not a traditional triad. It is D2, A2, and D2. When
SimpleStyle is played with this chord, the notes are stretched out to
fit the chord because the Chord/Scale playmode specifies that we
take the chord into account as well as the scale. The Sequence
Track’s purpose in this Segment is to show you the notes that play
when the Style’s pattern is played through the playmode Segment.
The notes are a doubling of the Style’s output an octave lower.
Sequence Tracks ignore chords and always play the same thing.
Back to the topic; let’s look at how the engine plays the scale
section of the pattern (starting on beat three) when played through
the playmode Segment. If you do not have the project handy, refer
back to Figure 6-1 for Simple Style’s notes. The first two notes are
what one would expect, but from there it looks confusing. You proba-
bly expect to hear D, E, F, G, A, B, C, D since the pattern has a
straight scale, but because DirectMusic has been told to take chord
position and scale into account, it does things differently. It plays D,
E, A, B, D, E, C, D. The A is played as the third note because A is
the second chord tone in the chord D-A-D. In C major, E was the
second chord tone. So when you are in C major and you write an E,
DirectMusic remembers that as the second chord tone, not as the
third scale degree. Thus, you hear A because with D-A-D, the sec-
ond chord tone is A. In C major, F is one scale degree above the
second chord tone, so over D-A-D, B is one degree above the second
chord tone. This also explains the D and E after that. The D in the
next octave is the third chord tone, like G was in C major, and E is
one above that, just like A was in C major. So why are the last two
notes lower? We have run out of chord tones and are going to the
100 n Chapter 6

next octave. These two notes are actually what we expected; they
just seem strange since they are lower than the two before them.
The second measure plays, ignoring the chord positions because
the default playmode of the Scale Style is set to the scale playmode
instead of the chord/scale playmode. The scale mode obviously only
takes scale position into account. The pattern starts on the root of
the chord and plays the same scale degrees as the original pattern.
Here is a summary of the way pitches are retained when the
chord/scale playmode is specified in a pattern. Pitches have three
components in the chord/scale playmode: chord position + scale off-
set + chromatic offset. So a D# in C major is what in our D-A-D
chord? Well, D# is the second chord position (E) lowered by one
chromatic tone in C major. In D-A-D, A is the second chord tone. G#
is one half-step lower than that, so the answer is G#. In this case,
there was no scale offset.
The difference between the chord playmode and the chord/scale
playmode is in how these playmodes cause the engine to handle sev-
enths, or the fourth chord tone. If chord playmode is specified and
the chord given in the Chord Track does not contain a seventh or
fourth chord tone, that tone will be omitted during playback. This
could be a good thing if you are trying to avoid sevenths when they
are not specified. On the other hand, the chord playmode could have
a negative impact if you have melodic lines that depend on sevenths.
PedalPoint is another playmode that will ignore the chord’s root
and round to the nearest scale note. Unless you really want a true
pedal point in the sense of one note, this may not work harmonically
with what you are doing. PedalPointChord is a really good option for
melodic lines, since it shifts notes to the nearest chord tone. That
doesn’t mean that every note will be a chord tone, though. It means
the generated note will have the same offset from a chord tone as
the original note in the pattern. For instance, a C in a C major pat-
tern (chord pos 1) shifts up to a B in a G major chord (chord pos 2),
and a D in a C major pattern (chord pos 1+1 scale degree) will
become a C in a G major pattern (chord pos 2+1 scale degree). In
English that means if you write a chord tone, it will always play a
chord tone. If you write a non-chord tone, it will play another
non-chord tone. Listen to the flute part in the PedalPointChord Seg-
ment in the ChordMap tutorial for an example.
 Working with Chord Tracks and ChordMaps n 101

All of this information about chord position and scale-offset infor-

mation can be quite confusing at first, but it is worth working
through in order to take full advantage of DirectMusic.
I

Unit
Note Tweaking
Another way to use the pedal point chord playmode is with second-
ary Segments or motifs. If you play a really long note with the Don’t
Cut Off and Regenerate on Chord Change flags specified in the
note’s properties page, some cool harmonies can result. In the tuto-
rial project, visit the Regeneration bookmark. Play the Segment
featuring the Chord Track using the main Play button, and then play
the other Segment with the secondary Segment toolbar. The notes
continue past the end of the Segment, adjusting to every new chord
that comes along! In this case, the pedal point chord playmode set-
ting is not specified in the part but on the notes themselves.
There are other things that you can do to individual notes using
their properties pages. You can tell the engine to harmonize a note to
a chord that has not arrived by tweaking the map to chord value.
This is great for pickups. You can avoid having one note randomly
transpose an octave because it hit a range limit. Do this by setting
groups of notes to transpose together with the Override Inv. Group
feature. You can tweak the scale and chromatic offsets that we dis-
cussed earlier in the Note Properties box. You can tell a note when it
is appropriate for it to cut off. For example, you can tell DirectMusic
to cut off the note if it is not in the new chord’s scale or chord tone
list.

ChordLevels and Inversions

You can specify what kinds of chord tones pattern parts are supposed
to play. This is done with ChordLevels or Inversions. Earlier you
were told to always edit chords in the bottom row labeled 1. The bot-
tom level consists of chords that the various instrument parts within
a pattern respond to, but you can define deeper chord levels. This is
especially useful for more complex harmonies, such as those in jazz.
If you have a G13b9 chord, you probably don’t want the exten-
sions to be played by the lower instruments. The strategy to use is
102 n Chapter 6

to place the basic chord tones in the first level and the upper exten-
sions into the upper chord levels. Open the ChordLevel bookmark to
see an example. The first three chords are simple and have the same
note for each level. When the more complex chords start in bar four,
only the trumpets play the dissonant tones. This is because in the
pattern the trumpet part is set to chord level three via its properties
page under Default Play Mode. All the extension notes are placed in
level three in the chords (see Figure 6-9). That way, the trombones
and bass play the fundamental chord level one tones, and the trum-
pets play the extended level three tones.

Figure 6-9: Chord levels.

You could also define for DirectMusic whether specific inversions

are allowed through the chords. The Inversions bookmark illustrates
this. The first set of chords has different inversions defined for the
second chord level, but the second set does not. The Style’s part is
set to play on level two. You hear the first four bars inverted, but the
last four are not, since all the chord levels are in root position in the
second half of the Segment. One last thing that could be done with
this feature is to implement a polychordal texture as bookmark 21 of
the Demo8 project that comes with what Producer demonstrates.
The Demo8 project is a demo that, in our opinion, is the best way to
learn the basics of DirectMusic. The project has a series of book-
marks that lead you through an excellent tutorial. Find it on the web
at:
http://msdn.microsoft.com/downloads/default.asp?url=/downloads/
topic.asp?url=/msdn-files/028/000/114/topic.xml
 Working with Chord Tracks and ChordMaps n 103

It is also on the DirectX SDK CD. Run this file to install it:
E:\DXF\DXSDK\essentls\DMusProd\DemoContent\DMPDemoCont
ent.exe.
I

Unit
Pattern Length and Chord Rhythm
Now let’s discuss chord rhythm, pattern length, and how Direct-
Music chooses which pattern is appropriate to play a given set of
chords. So far, every Style that we have dealt with had only one pat-
tern, but it is often better to have several patterns to add variety.
One chord change every measure with the same musical motifs
repeatedly can get tiresome. You can compose separate patterns for
each level of chord activity. By default, DirectMusic picks the longest
pattern that fits the chord rhythm in the Segment. This default
behavior only holds true if you have a Groove Track with a groove
level specified. Adding a Groove Track will not hurt your content if
you do not plan to use grooves. Just put any number in the first mea-
sure. For an example, look at the bookmark AdvancedTechniques.
The Style has two patterns. One is one measure long, and the other
is four. What really matters is the chord rhythm. Go to the properties
page of the pattern 4Bar. At the bottom is the chord rhythm section.
You can click the Set button to change the setting. After adjusting
the setting, you can then click the boxes to put an x in every place
where there should be a chord change. The four-measure pattern is
chosen every time that there is no chord change for four bars
because the pattern follows only one chord for four measures. If you
put an x at the beginning of every measure, the 4Bar pattern always
plays because it is the longest pattern with the correct chord rhythm
in all cases.
There are two variations in 4Bar. One variation is a single chord,
but the other variation moves every two beats. Patterns with only
one chord change are a good way to take some liberty with the
chords in your patterns. You can have different chord rhythms in
every variation. Faster, syncopated, and less regular chord rhythms
are sometimes better written out exactly without being subject to
Chord Track’s control. Chord rhythm variability is very easy with
this pattern, since there is only one part and there is no need to do
variation locking to keep the harmonies together.
104 n Chapter 6

You can still build inherent chord progressions into different vari-
ations if you have multiple parts. All you need to do is take advantage
of locking variations. To get this setup to work, you need to make
sure that the ChordMap can play for a longer period of time with only
one chord. In this example’s ChordMap (shown in Figure 6-10), you
will notice a connection that has been drawn from the first signpost
directly to the second target signpost. If there were no connecting
chords, this would not be necessary, but since there are connection
chords, DirectMusic will only take paths that are specified when
moving between these two signposts.

Figure 6-10: Randomly bypassing connecting chords.

More on Variation Choices:

The Mother of All Windows
To get to the Variation Choices screen, press the ? button next to the
variation numbers in your pattern. Figure 6-11 shows only half of the
window. The full window displays a row for all 32 variations in a part.
Each row tells what types of chords that variation can play on.
Pushed buttons are situations that are legal and unpushed buttons
are situations that are forbidden by the content creator. Note that
something will always play, so if a chord is selected and there are no
variations that are allowed to play over that chord, an illegal variation
will be selected at random.
 Working with Chord Tracks and ChordMaps n 105

Unit
Figure 6-11: Variation Choices window.

Figure 6-11 is the string part’s variation choices in the

AdvancedTechniques example. All variations besides 1 through 6 are
disabled, so they appear almost in black. Variation 1 hangs on the
fourth scale degree, so it does not sound very good over major
chords because of the half-step difference with the third. As a result,
all of the major triads are disabled for that variation. The iv chord is
disabled since variation 1 didn’t sound pleasing over that chord to the
author. Variation 2 hangs on the second scale degree, so it does not
sound very good over the diminished chords represented in italics.
There are only two diminished chords in the ChordMap, so those are
the only two disabled. Variations 3 and 4 sounded good to the author
with any chord, so every button is pushed in their rows. Variations 5
and 6 are reserved for the I chord, since they are whole notes bring-
ing resolution.
The other buttons are not used in this example, but they can be
very useful for creating more interesting harmonies. Before these
other buttons are explained, it must be made clear that the scale
degrees represented by the Roman numerals depend on the Chord
Track in the Segment. This, however, does not mean that you are
limited to standard major and minor scales. A key of F with five flats
would result in F Phrygian or F, Gb, Ab, Bb, C, Db, Eb, F. In that
case, ii means Gb minor and VII means Eb major.
The Root section tells whether or not variations can be played on
chords built off scale degrees that have been raised, lowered, or
unchanged. In the F Phrygian example, unchecking S would mean
that the variation could not play on a chord based on F, Gb, Ab, Bb, C,
106 n Chapter 6

Db, or Eb. If # was unchecked, chords based on F#,G, A, B, C#, D,

and E would become illegal.
In the Type section, “tri” means triads, “6,7” means chords with
sixths or sevenths added, and “Com” means complex chords. The
Dest area specifies a chord’s destination.
Before we wrap it up, there is one more trick that you should
know about when dealing with ChordMaps. If you ever need to
transpose a set of chords, there is a trick to doing it without having
to do it chord by chord. Create a dummy ChordMap that you do not
plan to use. Copy your chords into that ChordMap, and then change
the key of the clipboard ChordMap, transposing all of the chords that
you just pasted in. Select those chords, paste them into the Chord-
Map that you are actually using, and you have a set of transposed
chords.
Congratulations! You have just completed an in-depth tutorial on
one of the most intimidating and powerful parts of DirectMusic. We
wish you luck in becoming comfortable and at ease with these revo-
lutionary concepts. There are so many things to explore in the world
of DirectMusic harmony, and there were so few pioneers at the time
of publication. We hope that this chapter has given you the knowl-
edge and understanding necessary to dig in and write some great
interactive music.
 Chapter 7

DirectX Audio Scripting

D irectX Audio Scripting is another tool provided to composers

and programmers working with DirectMusic. Rather than
needing to write low-level code in order to control such things as
Segment playback, the composer can build sets of instructions using
a more basic language. While this language does not cover the full
gamut of behaviors available to the programmer using the C/C++
DirectMusic programming interfaces, it provides the most common
behaviors in an easier-to-understand format. The application then
triggers the composer’s sets of instructions at the appropriate time.
This allows more of the music playback behavior to remain in the
hands of the composer and in a form where the behavior can be
adjusted more easily without having to recompile the application.
The script can be updated in DirectMusic Producer, run-time saved,
and dropped back into the application, and the new behavior will be
used without any additional changes.

Defining the Audio Implementation Problem

While DirectMusic allows a composer or sound designer to create
flexible, nonrepetitive audio content, the audio producer must still
deal with the process of integrating that audio into the interactive
application. Such integration is generally placed in the hands of the
programmer. While the programmer no longer needs to be con-
cerned with what exact content a piece of music (a DirectMusic
Segment) plays, he must still manage quite a bit of information:
n Dynamic volume and groove level changes for Segments
n Managing AudioPaths that Segments are played onto (unless
each Segment contains its own embedded AudioPath)
107
108 n Chapter 7

n Playing Segments at musically appropriate boundaries (generally

decided on by the audio producer)
n Playing Segments as primary or secondary Segments
n DLS Collection download/unload management
This can set up a significant workflow challenge — if the composer
wants a piece of music to transition at a different boundary, he needs
to communicate this to the programmer, wait for the application to be
changed and its code rebuilt, and then he can proceed. This often
leads to the same kind of slow turnaround process that we were try-
ing to avoid, where very few changes will be able to be made simply
because of the time involved with each iteration of the application.
Enter DirectX Audio Scripting. Using a basic, human-readable
language, an audio producer can now author scripts, instructions for
what behaviors should occur when a certain trigger is reached. Each
“trigger” is a script routine, a list of instructions that this trigger
should cause to occur — play a piece of music, stop another piece of
music, adjust the musical intensity, change the volume, and so on.
The programmer, rather than needing to implement these instruc-
tions, simply calls these script routines at appropriate points while
the application runs.
DirectX Audio Scripting provides a subset of the features of the
lower-level C/C++ DirectMusic programming interfaces. That is,
anything you can do with scripting can be done in application code if
desired. Scripting allows the composer to abstract more of the
behavior away from the programmer. For instance, rather than the
programmer needing to know that when a player enters a room, the
ambience for this room should play, the intensity should be adjusted
based on the number of enemies, and the previous ambience should
fade out, he can just send a trigger called EnteredRoom, and the
composer’s script can give the instructions for what that means in
terms of the audio.
The other powerful feature that DirectX Audio Scripting supplies
is the ability to pass information back and forth between the applica-
tion and the music. For instance, the programmer could tell the
script what the current score is, and when the programmer triggers
the routine called AnnounceScore, the sound designer could stitch
together the proper dialog necessary to report the score. Or perhaps
the crowd would react with more energy if the script was made
 DirectX Audio Scripting n 109

aware of the time remaining and how far apart the two teams’ scores
are. All of this is accomplished by using scripting variables, where
data can be passed in both directions; the script can tell the applica-
I
tion when an event has occurred in the music, and the application

Unit
can tell the script what the user is doing and the current state of the
application.
But wait — isn’t all of this scripting a form of programming? Are
we turning the composer into a programmer? In some respects we
are, but hopefully it is with a straightforward language that allows
the composer to do more of what he wants without having to depend
on the programmer. As you’ll see below, many of the most powerful
functions (namely, playing or stopping a piece of music) are very
straightforward. Feel free to cut and paste from both of these exam-
ples, those in the demo project supplied with DirectMusic Producer,
and sample scripts found in the DirectX Software Development Kit
(SDK).

Creating a Script
The first step to building our script is to create a new one in Direct-
Music Producer. As with other file types, you can create a new script
via the File menu by selecting New… and then choosing Script in
the dialog that opens.

Figure 7-1: An empty script as displayed in the project tree.

The first thing we notice is that a script looks an awful lot like a
DirectMusic container file, with separate embed and reference fold-
ers. In fact, script files are a specialized kind of container file. A
script is able to act on any other file that it knows about, as well as
manipulate parameters for the global DirectMusic engine (tempo,
groove level, and so on). With files placed in the Embed Runtime
folder, the content will actually be saved within the Script file when it
is run-time saved, just as they were in a container’s Embed Runtime
110 n Chapter 7

folder. Meanwhile, any files placed in the Reference Runtime folder

are files that the composer intends to save out and deliver for appli-
cation integration separately but that the script is still aware of and
can use. Files can be placed in the two folders by dragging them onto
the folders, or right-clicking on one and choosing Add/Remove
Files ….

Figure 7-2: The Add/Remove Files dialog provides a quick way to

add many files to a script’s container at once.

We discuss more implications of embedding versus referencing later

in the chapter. For now, let’s use the Reference Runtime folder,
remembering that when it comes time to integrate our content into
the application, we need to separately deliver all of the files that the
script uses.
One of the nice things about DirectX Audio Scripting is that
waves and Segments can be mixed together, just as they can by a
programmer. Of course, wave files don’t give you much in the way of
variability nor do you get any tempo or measure information without
building a Segment first. But for some scenarios (for instance, non-
varying dialog), basic wave files are quite useful in scripts, as we can
see in our first set of examples. We’re going to reference a few wave
files, named for the dialog they contain — “Hello,” “My Name Is,”
and “Bob” (if you’ve authored these wave files separately, don’t for-
get to insert them into the project via the File menu’s Insert Files
into Project option). We also have some background music handy in a
pair of Segments called RoomAmbience and ActionMusic.
 DirectX Audio Scripting n 111

Unit
Figure 7-3: The setup for our upcoming examples. Our two
Segments happen to refer to their own DLS Collections, but they
could also (or instead) use General MIDI instruments.

Note first off that the wave My Name Is is referenced by the script
as MyNameIs — without the spaces. The name a script uses to ref-
erence an object is not allowed to contain any spaces, so these were
removed. Other restrictions to be aware of are that a script object
cannot begin with a number, and you can’t have two waves/Segments
with the same name (if you do, DirectMusic Producer will automati-
cally add a number to the end of the name of the second one
inserted).
Now let’s take a look at the Script Designer window. If it isn’t
already open, double-click on the script in the project tree.
112 n Chapter 7

Figure 7-4: An empty script.

There are three frames in the Script Designer. The Source frame is
where we type in the actual routines for this script. When we want
to try out these routines, we double-click them over in the Routines
frame. This simulates the exact function that the programmer will
perform when they “trigger” a script routine. The Variables frame
will show any variables that our script uses along with their current
values. Again, we can click on these to update them just as the pro-
grammer would in the actual application.

Scripting Languages
DirectMusic Producer supports any ActiveX scripting language for
creating scripts, but the two most commonly used languages are
Visual Basic Scripting Edition and AudioVBScript. AudioVBScript is
a DirectMusic-optimized language, specially geared toward music
content playback and occupying a very small memory footprint.
Alternatively, a composer could use the full Visual Basic Scripting
Edition language (“VBScript” for short). While this language occu-
pies more memory and potentially could take more processing
power, it is a fully featured language, providing added flexibility and
additional features, such as string concatenation (the ability to merge
several pieces of text for display or script usage). Generally, start
with AudioVBScript and move up to VBScript if you find that you
 DirectX Audio Scripting n 113

need additional power. All scripts written in AudioVBScript will

work in VBScript, but the opposite is not necessarily true. Of course,
don’t forget that the programmer can also step in for more complex
I
scripting usage to perform some operations in lower-level Direct-

Unit
Music code.
The language that a specific script uses can be selected from that
script’s properties page.

Figure 7-5: The Script Properties window.

A full list of differences between the two languages can be found in

the DirectMusic Producer help documentation under the topic
AudioVBScript Limitations.

Basic Scripting Usage

Let’s create our first scripting routine. Click in the Source frame of
the Script Designer window. A cursor appears, and we can type away.
Routines follow this format:
Sub [name of routine]
[things to do when this routine is triggered]
End Sub

For instance, we’re going to create a routine that introduces a new

character.
Sub IntroduceCharacter
End Sub
114 n Chapter 7

If you click anywhere else in the Script Designer window (or in the
project tree), notice that IntroduceCharacter pops up in the Routines
frame at the top right of the window.

Figure 7-6: IntroduceCharacter in the Routines window.

You could double-click on this routine to trigger it (though of course,

it doesn’t do anything yet). By the way, if our routines ever disappear
from that Routines list, it means that our script has some kind of
typographical error in it. We talk about that more when we get to
debugging in a bit.

Playing a Piece of Audio

Okay, now let’s make our routine do something. The simplest func-
tion is probably to play a wave or a DirectMusic Segment. Let’s play
our Hello wave.
Sub IntroduceCharacter
Hello.play
End Sub

The period (.) is how DirectX Audio Scripting separates the object
that we’re using (in this case, a wave) from the function that we want
to call on it (in this case, playing it). Remember, the script only
knows about objects that were dragged into its embed or reference
folders; if we tried to play a Segment or wave that wasn’t in those
folders, we would get a syntax error. The above routine will play
Hello using all default properties; it will play as a primary Segment,
and it will play at the default authored boundary (most typically,
immediately, but this could be set in the Boundary tab of the Seg-
ment’s properties page). Again, we could try this routine out by
double-clicking on it over in the Routines frame. Now it makes
noise!
 DirectX Audio Scripting n 115

Making scripts as readable as possible by placing comments

helps when you might later need to return to the script to make
edits. Any line of a script that begins with an apostrophe (’) is effec-
I
tively ignored when that script is played back and typically used for,

Unit
among other things, describing what the routine does.
Sub IntroduceCharacter
’We trigger this routine whenever our character meets
’other characters.
Hello.play
End Sub

Let’s make this routine do something a bit more complex; a routine

that only does a single thing doesn’t really save us much over having
the programmer implement it. We’ve got these other two waves that
we wanted to string together into a more complete introduction, so
let’s add them to the script. We wouldn’t want to just sequentially
call Hello.Play, MyNameIs.Play, and Bob.Play — that would fire off
all three waves at the same time. Worse yet, they would each be
played as the primary Segment, which means they would cut each
other off. Instead, we want to use one of the transition boundaries
that DirectMusic provides, the ability to queue Segments to play one
after the other.
Sub IntroduceCharacter
’We trigger this routine whenever our character meets
’other characters.
Hello.play
MyNameIs.play (AtFinish)
Bob.play (AtFinish)
End Sub

Our code now introduces one of the several parameters available

when you tell a Segment to play. Parameters can be optionally
enclosed in parentheses, which often helps for readability. They are
required to be in parentheses when they’re on the right side of an
equation, as we see a bit later on, so it’s often easiest to simply
always use parentheses.
The first parameter that can be used to modify our play call con-
sists of flags that we can use. The AtFinish flag says that this
Segment should wait to play until the previous primary Segment has
finished playing. So our routine is now firing off “Hello” and queuing
up “my name is” and “Bob” to play afterward.
116 n Chapter 7

þ Note Be aware that this kind of dialog “stitching” (piecing together

dialog fragments to reassemble whole sentences) involves many
challenges beyond implementing queued playback in an application.
Writing dialog, coaching, and recording voice talent to get appropriate
and believable inflections present significant challenges that are beyond
the scope of this book. These difficulties are shared by many
applications — sports titles that use stitching for their commentators in
particular.

You can see the list of parameters for playing a Segment by looking
in the DirectMusic Scripting Reference guide, under the topic Seg-
ment.Play. When the Script Designer window is open, a shortcut to
the guide can be found in the Help menu and is also available via the
Ctrl+F2 shortcut. Parameters are separated by commas, though if
you don’t use all of the parameters, their default settings will be
used. Therefore, for these early examples, we will just stick with the
first parameter.
For the flags parameter of Segment.Play, multiple flags can be
combined simply by “adding” them (using the + sign between
them). For instance, we can create a new routine that starts our
room ambience when we enter the room. If any music is playing, we
want the ambience to sneak in at a measure boundary. We play it as a
secondary Segment so that the background score continues to play.
(Recall that only one primary Segment can be playing at a time.)
Sub EnterRoom
RoomAmbience.Play (IsSecondary + AtMeasure)
End Sub

Granted, our dialog example has the same issue; we probably don’t
want our dialog to become the sole primary Segment. Since it
becomes a bit more complex to solve this problem for dialog stitch-
ing, we come back to that in a moment.

Using Transition Segments

The play event also supports three other parameters — an
AudioPath on which to play this Segment, a transition Segment to
play prior to playing this Segment, and a PlayingSegment to play rel-
ative to (use the measure, beat, marker, etc., boundaries). We come
back to the last one in the next section. But for now, let’s take a
 DirectX Audio Scripting n 117

quick look at transition Segments. Remember that each parameter in

a call to a DirectX Audio Scripting function is separated by a comma.
We just want to use the default AudioPath, so we can leave the sec-
I
ond parameter blank or use the special keyword “nothing” to

Unit
indicate that it should be ignored and use the default.
Sub StartActionMusic
ActionMusic.Play (AtMeasure, nothing, RampUpAction)
End Sub

…which is equivalent to:

Sub StartActionMusic
ActionMusic.Play (AtMeasure, , RampUpAction)
End Sub

Both versions of the routine would do the same thing — at the next
measure of the primary Segment, play the RampUpAction Segment
that we’ve authored as a transition Segment; when it finishes, play
ActionMusic.

Stopping a Piece of Audio

Now our music is chugging along, but we need to be able to stop it
eventually (assuming it was authored to loop for a while, or we just
need to get back to authoring for a bit!). The basic stop call is similar
to our basic play call:
Sub ExitRoom
RoomAmbience.Stop
End Sub

Again, by default, the stop occurs on the Segment’s default boundary

(which was probably immediate). You can use the single parameter of
Segment.Stop to specify a more specific boundary or (if you are
using Style-based Segments) to tack an end embellishment pattern
onto the ending of the Segment. Again, see the DirectMusic
Scripting Reference portion of the DirectMusic Producer help file for
a complete list of options. As an example, when we exit the room,
we want to stop our music at the next beat. We could create the fol-
lowing routine:
Sub ExitRoom
RoomAmbience.Stop (AtBeat)
End Sub
118 n Chapter 7

Global DirectMusic Parameters

Scripting also exposes a number of DirectMusic’s global parameters.
A script can modify the master tempo at which all content is playing,
for instance. Or the script can change the master groove level (handy
for dynamic music that uses Style-based playback).
Sub DoubleTime
SetMasterTempo (200)
End Sub

Sub IncreaseIntensityLevel
SetMasterGrooveLevel (GetMasterGrooveLevel + 1)
End Sub

This second example actually uses two global scripting functions,

SetMasterGrooveLevel and GetMasterGrooveLevel. We can find out
the current groove level, increment it by one, and then set the
groove level to the resulting value.
All of these “global” functions are technically operating on a
DirectMusic Performance object, so in the documentation, they can
be found under Performance.[Function]. But since most implementa-
tions only involve a single Performance object, you don’t typically
need to use one in your scripting calls. Remember that multiple per-
formances can be created by a programmer if you need to have
multiple tempos in use at the same time or multiple primary Seg-
ments; each performance is effectively its own instance of the
DirectMusic playback engine.
One note with using the global functions is that you probably
want to create a function that resets all of the global DirectMusic
settings just so that you know you’re starting off from a known state.
Otherwise, after running the above script routines, we’d be playing
at twice the authored tempo and at a higher groove level without
having any way to get back to a regular tempo and groove level!
Sub ResetSettings
SetMasterTempo (100)
SetMasterGrooveLevel (0)
End Sub
 DirectX Audio Scripting n 119

Editing Content while Using Scripts

I
You might sometimes notice that a script “loses track” of a Segment

Unit
you were using. You try to fire off a routine to play that Segment and
you get silence or an older version of the Segment plays. This is
most frequently caused by editing a Segment after setting up your
script. In order to allow for script routine auditioning within Direct-
Music Producer, the script is actually “running” even as you edit it.
But when you edit a piece of music that the script was using,
DirectMusic Producer has a tough choice — the script could update
with each edit to the Segments (which might cause a significant
slowdown to edits if the Segment and/or script were large) or the
script can just accept that it might get out of sync, at which point the
content creator can force a refresh. DirectMusic Producer opts for
the latter. You can force the script to resynchronize with all of its
source content by hitting the Refresh button above the list of rou-
tines. If that is not successful, you can save and reload your script by
right-clicking on it, choosing Save, and then again right-clicking and
choosing Revert to Saved.

Debugging Scripted Content

Let’s take a quick break from scripting to look at how you can debug
(find and correct errors in) scripts. As we discussed before, you
know your script has some kind of syntax error when the routines
disappear from the Routines list in the top-right frame of the Script
Designer window. But where was that error? Rather than having to
hunt through every line of code, DirectMusic Producer can report
errors along with their locations via the Message Window. You can
open it from the Add-Ins menu. This window reports syntax errors
(errors that occur because the script was typed wrong), as well as
run-time errors (errors that occur when the script tries to run a
routine).
Let’s set up one of each and see how the window responds.
SubExitRoom ’Notice – no space between ‘Sub’ and ‘ExitRoom’
RoomAmbience.Stop (AtBeat)
End Sub
120 n Chapter 7

If the Message Window is open and we click outside of the Source

box in the Script Designer (or click the Refresh button above the
Routines frame), the Message Window will display this:

Figure 7-7: The Message Window displays errors in your script.

(The line might be different for you depending on whether you

started a new script or worked from one of the previous examples.)
You can see exactly what line that is by clicking back into your script
Source window and looking at the status bar down at the bottom of
the DirectMusic Producer window. (If the status bar is hidden, dis-
play it by selecting it in the View menu.)

Figure 7-8: The status bar displays the exact line and
column number that your cursor is currently on.

For an example of a run-time error, let’s use that same routine but
try to play a Segment that we haven’t added to either of the script’s
folders (for instance, a typo in the name of the Segment that we
intended to play).
Sub ExitRoom
'Try to play a Segment the script doesn’t know about
SomeOtherSegment.Play (AtBeat)
End Sub

Nothing happens when we click away from the Source window, as

this is a perfectly legal script routine. But when we try to run it (by
double-clicking it in the Routines frame), we get what is shown in
Figure 7-9:
 DirectX Audio Scripting n 121

Unit
Figure 7-9: The Message Window displays an error regarding
an unlocatable Segment.

This lets us know that the script didn’t know where to find the Seg-
ment, so we should add it to the reference or embed the run-time
folder.
The Message Window has one additional use; you can place
statements in your routines that provide information similar to
comment statements but let you know when a routine (or more par-
ticularly, a specific portion of a routine) has been run. These com-
mands are known as trace statements. A programmer can actually
“listen” for these trace statements in a game if you wanted to pro-
vide information (for instance, for karaoke, song title information,
and so on).
Going back to our queued-up dialog, we could add a trace state-
ment that lets us know the script routine completed.
Sub IntroduceCharacter
’We trigger this routine whenever our character meets
’other characters.
Hello.play
MyNameIs.play (AtFinish)
Bob.play (AtFinish)
Trace “Queued up Bob’s introduction.”
End Sub

When we double-click the routine, we hear all three waves play in

order and we also immediately see the following in the Message
Window:

Figure 7-10: The routine works!

122 n Chapter 7

Scripting with Variables

So we now know how to start and stop pieces of audio. For many
scenarios, the next step is to be able to use application information
to dictate what specific audio is triggered. This involves the use of
variables, which are scripting objects that the programmer and the
scripter “share” and can adjust or check. In the most basic examples,
the programmer will be the only one adjusting a variable’s value, and
the script will read that value to decide what to do.
Let’s return to the character introductions. What if we had 20
character names that our player could choose at the beginning of our
application? We wouldn’t want to write 20 routines that the program-
mer would have to choose between when it came time to introduce
ourselves. We could instead use a variable — say, one called
CharacterNumber (remember that variables, like other scripting
objects, cannot contain spaces or start with a digit). The programmer
sets this variable when we choose our name. Then whenever we
introduce ourselves to another character, the programmer can call
our single script routine, which checks the value and uses it to
decide the appropriate wave to play.
Variable use consists of two parts. First, you must declare the
variable so the script knows how to use it (similar to dragging a wave
or Segment into one of the script’s two folders). The format for this
is simply:
Dim [name of variable]

One quick restriction relating to AudioVBScript is that you must

declare any variables that you intend to use above any routines. So
this would be illegal:
Sub EnterRoom
RoomAmbience.Play (IsSecondary + AtMeasure)
End Sub

’illegal – all “dim”s must come before all “sub”s

dim CharacterNumber

Also make sure that you don’t name a variable the same name as one
of your other scripting objects (Segments, waves, and so on). Again,
the Message Window will let you know if this problem occurs.
 DirectX Audio Scripting n 123

Once a variable has been declared, if you click away from the
Sources tab, you’ll see the variable appear in the lower-right frame of
the Script Designer window, as in Figure 7-11.
I

Unit
Figure 7-11: Variables list in the lower-right frame of the Script Designer window.

You can now click on the value field to set the value, just as the pro-
grammer would do in the application. Clicking the Reinitialize button
resets all values to Empty.

Conditional Statements
Now that we’ve created a variable, we use conditional statements to
act based on their value. With conditional statements, the script per-
forms some comparison on the variable to decide how to act (for
instance, what piece of music to play and when to play it). The sim-
plest conditional statements are if … then statements, where if a
condition is true, you will perform one or more actions. The syntax
for this is:
If (expression) then
[Action1]
[Action2]
[...]
End if

Incidentally, indenting the actions that occur within the If/End If

statements are for readability only; there’s no rule that these lines of
code must be spaced a certain way (in fact, tabs and extra spaces are
universally ignored in DirectX Audio Scripting).
As an example of a simple If/End If statement, we could have a
script like this:
dim PlayerHealth

Sub PlayDeathMusic
if (PlayerHealth = 0) then
DeathMusic.Play (AtImmediate)
end if
end sub
124 n Chapter 7

If the programmer tried to trigger the death music when the player
still had health, nothing would happen; only if health was zero would
we hear the Segment called DeathMusic play.
More complex statements will tend to use the if … then/elseif …
then/else format: If a condition is true, do some action; otherwise,
check to see if another condition is true (elseif) and perform their
actions; otherwise, if all of these conditions are false (else), do some
final action. For our character introduction example above, our code
might look like this (let’s simplify to three characters instead of 20):
dim CharacterNumber
'We have three possible options for our character
'1=Bob
'2=Joe
'3=Bill

Sub IntroduceCharacter
'We trigger this routine whenever our character meets
'other characters.
Hello.play
MyNameIs.play AtFinish
if (CharacterNumber=1) then
Bob.play (AtFinish)
elseif (CharacterNumber=2) then
Joe.play (AtFinish)
elseif (CharacterNumber=3) then
Bill.play (AtFinish)
else
Trace "Illegal character number."
end if
End Sub

We can test out the above routine by setting our variable in the
lower-right frame of the Script Designer (just as the programmer
would in application code) and then double-click a routine to trigger
it, again, just as the programmer would.
Notice the final else statement in the above example. Using trace
statements here often comes in handy for debugging; you are alerted
that something caused your variable’s value to go outside of the
range that you expected it to fall into.
 DirectX Audio Scripting n 125

Local Variables
Any time a variable is declared with a dim statement, it is a global
I
variable that the programmer can see and manipulate. There are

Unit
often situations where global variables are not necessary. For
instance, the scripter might do some basic math on several global
variables to decide what piece of music to play. Rather than having to
create another global variable for this value, scripters can use local
variables. Local variables have two primary differences from global
variables — they cannot be seen by the programmer, and they do not
have to be declared. They are implicitly created when they are first
used.
A good example of local variable use is adding randomization to
scripted playback. AudioVBScript has a rand function that can be
used to generate a random number from 1 to the value you specify.
For instance, if we had three pieces of music that would be appropri-
ate for the current scene, we could construct a script routine like the
following:
Sub PlaySceneMusic
x = rand(3)
’x is a local variable – we did not have a
’ “dim x” statement.
’The programmer cannot see it, and its value is
’ not stored after the routine finishes running.
If (x=1) then
LowLevelMusicThematic.Play
ElseIf (x=2) then
LowLevelMusicRhythmic.Play
ElseIf (x=3) then
LowLevelMusic.Play
End if
End Sub

Again, see the DirectMusic Producer documentation page (for Per-

formance.Rand) for more details on this function. Of course, this
provides a small subset of the functionality that you could create
within a single Segment with variations, but it does allow for your
script to randomly choose between several fully composed Segments
without needing to merge them into a giant Segment or needing for
the programmer to get involved.
126 n Chapter 7

A Two-Way Street: Variables Set by the Scripter

Variables can also be set by the scripter and used by the program-
mer. For instance, a sound designer could track how many times a
routine has been called or how many instances of a sound have been
started. Note that tracking when they’ve stopped is often more eas-
ily handled by the programmer.
dim NumberOfTorchesPlayed

Sub StartTorch
Torch.Play (IsSecondary)
NumberOfTorchesPlayed = NumberOfTorchesPlayed + 1
end sub

Tracking Individual Instances of a Playing Segment

Now that we have a basic understanding of variables, let’s return to
the stop function on Segments. One thing to note with the Seg-
ment.Stop call is that we’re stopping the Segment rather than a
specific playing instance of that Segment. For ambience and music,
where there is generally only one instance of any particular piece of
music playing at a time, this isn’t an issue. But let’s say for example
that we have a Segment that plays a torch sound, and the character
in our application is walking through an environment with 50 torches
that we had started playing. Calling Torch.stop would stop all 50
of them. If we have just extinguished one, we want the ability to stop
just that single torch.
To do this, we can keep track of PlayingSegment objects. When-
ever a Segment is played, we can choose to store this particular
playing instance in a variable. We can later stop this particular
PlayingSegment without stopping every other instance of the origi-
nal Segment.
dim MyPlayingTorch

Sub StartMyTorch
’We only carry one torch, so there’s only one
’PlayingSegment we need to keep track of
Set MyPlayingTorch = Torch.Play (IsSecondary)
end sub

Sub StartRoomTorch
Torch.Play IsSecondary
 DirectX Audio Scripting n 127

End sub

Sub StopMyTorch I
’Will only stop our own torch; room torches will
’continue to play.

Unit
MyPlayingTorch.Stop
End Sub

Sub StopAllTorches
Torch.Stop
End Sub

Note that the syntax for using objects in variables is slightly modified
from numeric variables — a Set statement is used when you are
assigning something to an object, rather than a numeric variable. In
this example, we now have a personal torch that we track the
instance of, and if our character extinguishes it, that single instance
can be stopped without affecting other torches in the room. Mean-
while, the room might be filled with other torches that we don’t need
to be able to stop individually, so the script doesn’t keep track of
their individual PlayingSegment objects. The StopAllTorches method
would stop every torch, including our own.

Triggering Segment Playback Relative to Playing

Segments
Now that we can use PlayingSegment objects, we can return to our
original Character Introduction script routine (stitching together
“Hello my name is Bob”) and make it use secondary Segments. By
default, when you play a Segment, it uses the primary Segment as
the reference point to know when to start playing. But we don’t want
that here, or the “my name is” and “Bob” waves aren’t going to play
until the primary Segment finishes (and at that point, they’ll both
play at the same time!). We instead want “my name is” to queue up
relative to the “Hello” Segment (which is going to play as a second-
ary Segment) and then for “Bob” to queue up relative to “my name
is.”
To get the proper stitching behavior, we are going to use the
fourth parameter on the Segment.Play function, oFromPlayingSeg-
ment, which is the PlayingSegment object that this play event should
play relative to. Don’t forget that we still need to fill in the second
and third parameters to the play function, for which we could use the
128 n Chapter 7

keyword “nothing” or just leave blank (but still using commas to

separate them).
Dim PlayingSegHello
Dim PlayingSegMyNameIs

Sub IntroduceCharacter

Set PlayingSegHello = Hello.play (IsSecondary)

Set PlayingSegMyNameIs = MyNameIs.play (AtFinish + IsSecondary, nothing, nothing,
PlayingSegHello)
Bob.play (AtFinish + IsSecondary, nothing, nothing, PlayingSegMyNameIs)

End Sub

Script Tracks and Self-Modifying Music

One of the more interesting aspects of DirectX Audio Scripting is
that script routines do not solely have to be triggered by the pro-
grammer. DirectMusic Segments themselves can fire off script
routines at appropriate times, which allows for the concept of
self-modifying music. A piece of music might play and check the
state of a variable every four bars. If that variable has changed to a
certain value, the music might stop of its own accord or fire off
another piece of music, change tempo, or any of a number of actions.
Segments can trigger script routines by using a Script Track.
When a Segment has a Script Track, you can place specific routine
calls at any point in the Segment, and that routine will be called
when the routine reaches that position. For instance, for our above
example, a Segment might look like this:

Figure 7-12: Every four bars, the script routine CheckNumEnemies is called. If this
Segment was our primary Segment, and the routine CheckNumEnemies fired off
another primary Segment, this Segment would of course stop playing.
 DirectX Audio Scripting n 129

This can lead to all sorts of interesting implementations. Every piece

of music in your application could fire off a script routine call right as
it ends that figures out what piece of music to play next. So you
I
could effectively have a self-running jukebox without the program-

Unit
mer having to code anything. Or the music can poll a variable every
so often and respond accordingly without needing the programmer to
let the script explicitly know that the variable has been updated.
Such a technique is used in the baseball example included with the
DirectX Software Development Kit — an infinitely looped secondary
Segment consisting solely of a script track checks to see if the score
has changed and whether the crowd has responded appropriately
with cheers or boos (without needing the programmer’s involvement
in the least). See the baseball.spt file (which can be opened in
DirectMusic Producer and re-expanded into its source files) for more
details on how this was implemented.

Using Scripting for Resource Management

DirectX Audio Scripting can also be used to allow the composer to
control certain aspects of resource management, particularly when
Segments are loaded, downloaded, and unloaded. Loading is the pro-
cess of reading a Segment in from the disk. Downloading is the
process of determining the waves (streaming or in memory) and/or
DLS instruments the Segment uses and loading their wave data into
memory (or, where available, transferring their wave data onto a
DLS-capable hardware synthesizer, the original concept behind the
term “download”). Unloading is the opposite of downloading, where
DLS instruments and waves no longer in use can be freed.
By default, all of these are managed automatically; when a script
loads, all of the Segments that it references or embeds are automati-
cally loaded and downloaded. Of course, if you have one script for an
entire application, that could be a fairly large resource hit. The
scripter could create one script for every level or one for every
group of sounds that can be inclusively loaded and unloaded. Alterna-
tively, the scripter can opt to control when these processes occur.
130 n Chapter 7

Figure 7-13: You can control whether loading and downloading

automatically occur via the script’s properties page.

With increased control comes increased responsibility. If the script

attempts to play a Segment that it has not yet loaded, an error will
occur. If it attempts to play a Segment that has not been downloaded,
that Segment will probably play silently. Similarly, it will be up to the
scripter to free (unload) content that is no longer needed, or system
resources might be unnecessarily drained. As one extra note of pre-
caution, loading and downloading are synchronous functions, which
means that the script will wait to continue executing content until
the necessary disk transfers are complete.
Sub LoadAndPlayMusic
MySegment.Load
MySegment.DownloadSoundData
’Downloads all DLS instruments and in-memory waves.
MySegment.Play
End Sub

Sub FreeWaveData
MySegment.UnloadSoundData
’Make sure you unload as many times as you download
’for a Segment, as wave data won’t be freed until then.
End Sub

While downloaded instruments are freed via UnloadSoundData,

loaded Segments are not freed until the script is freed. This is gener-
ally not much of an issue, as it is the wave data (DLS instruments
and non-streamed waves) that takes significant memory and needs to
be unloaded as soon as playback is completed.
Once a script has been completed, don’t forget to run-time save
it (right-click and choose Runtime Save As… or Runtime Save
All Files from the File menu); the .spt file that is generated will
 DirectX Audio Scripting n 131

embed any content that was placed in the Embed Runtime folder and
knows how to use any separately delivered content that had been
dragged into the Reference Runtime folder. When scripts are used in
I
a DirectMusic application, encourage your programmer to give you

Unit
the ability to manually fire off every script routine in some manner
while running the application. For instance, if the bad guy music only
gets triggered at the very end of a game, you probably don’t want to
have to go all the way through a level just to hear if the music trig-
gered at an appropriate boundary.
DirectX Audio Scripting provides another tool to the composer
or sound designer for controlling audio behavior without needing to
overly involve the programmer and, more importantly, the ability to
make changes without having to rebuild the entire application. While
the functionality provided is a subset of the functionality made avail-
able programmatically, the functions allow for the vast majority of
basic and intermediate scenarios to be handled solely by the content
author.
This page intentionally left blank.
Unit II

Programming with
DirectX Audio

Chapter 8: DirectX Audio Programming 101

Chapter 9: The Loader
Chapter 10: Segments
Chapter 11: AudioPaths
Chapter 12: Scripting
Chapter 13: Sound Effects
Chapter 14: Notifications

133
This page intentionally left blank.
 Chapter 8

DirectX Audio
Programming 101

W e begin our journey into DirectX Audio programming with a

short overview of the technology and a simple application that
makes a sound.

þ Note Understand that a certain level of programming experience is

required to work with DirectX Audio. You should be familiar with C++
and object-oriented programming concepts. Experience in COM is a
plus but is not required, since we walk through the parts you need to
know. Experience in MFC and Windows programming is valuable as
well but, again, not required. Of course, you need a compiler. All
programming examples were written and tested in Microsoft Visual
C++ 6.0 and Microsoft Visual Studio .NET.

DirectX Audio = DirectSound + DirectMusic

Before we start, let’s look at the DirectX Audio building blocks. To
do that, it helps to have a little history. Microsoft created Direct-
Sound back in 1995. DirectSound allowed programmers to access
and manipulate audio hardware quickly and efficiently. It provided a
low-level mechanism for directly managing the hardware audio chan-
nels, hence the “Direct” in its name.
Microsoft introduced DirectMusic several years later, providing a
low-level, “direct” programming model, where you also talk directly
to a MIDI device, as well as a higher-level content-driven mecha-
nism for loading and playing complete pieces of music. Microsoft
named DirectMusic’s lower-level model the Core Layer and the con-
tent-driven model the Performance Layer, which provides relatively

135
136 n Chapter 8

sophisticated mechanisms for loading and playback of music. The

Performance Layer also manages the Core Layer.

Host Application

DirectMusic
Performance

DirectMusic DirectSound
Core

MIDI Audio Device Wave

Figure 8-1: In DirectX 7.0, DirectMusic and

DirectSound are separate components.

In DirectX 7.0, the host application uses DirectMusic and

DirectSound separately (see Figure 8-1). MIDI and style-based
music are loaded via the DirectMusic Performance Layer and then
fed to the DirectMusic Core, which manages MIDI hardware as well
as its own software synthesizer. Sound effects in the form of wave
files are loaded into memory, prepared directly by the host applica-
tion, and fed to DirectSound, which manages the wave hardware as
well as software emulation. It became clear after the release of
DirectMusic that the content-driven model appealed to many appli-
cation developers and audio designers since the techniques used by
DirectMusic to deliver rich, dynamic music were equally appropriate
for sound effects. Microsoft decided to integrate the two technolo-
gies so that DirectMusic’s Performance Layer would take on both
music and sound effect delivery.
Microsoft added sound effect functionality to the Performance
Layer, including the ability to play waves in Segments and optionally
use clock time instead of music time. The Core Layer assumes wave
mixing via the synthesizer, which is really just a sophisticated mix
engine that happens to be more efficient than the low-level Direct-
Sound mechanisms. This way, the Core Layer premixes and feeds
sound effects into one DirectSound 3D Buffer, dramatically
 DirectX Audio Programming 101 n 137

increasing efficiency. Real-time audio processing significantly

enhances the DirectMusic/DirectSound audio pipeline.
Perhaps the most significant new feature is the introduction of
the AudioPath concept. An AudioPath manages a virtual audio chan-
nel, or “path,” from start to end. This starts with the generation and
processing of MIDI and wave control data, continues through the
synthesizer/mix engine in the Core Layer, on through the audio
effects processing channels, and finally to the final mix, including 3D
spatialization.

Host Application

DirectMusic Performance
II

Unit
AudioPath
AudioPath
AudioPath

DirectMusic DirectSound
Core/Synth

Audio Device Wave

Figure 8-2: In DirectX 8.0, DirectSound and

DirectMusic merge, managed by AudioPaths.

With DirectX 8.0 and later, the Performance Layer controls one or
more AudioPaths, each of which manages the flow of MIDI and wave
data into the DirectMusic software synthesizer, on through multiple
audio channels into DirectSound, where they are processed through
effects, and through their final mix stage into the audio device (see
Figure 8-2).
This integration of DirectSound and DirectMusic, delivering
sounds and music through the Performance Layer via AudioPaths, is
what we call DirectX Audio.
138 n Chapter 8

þ Note When Microsoft delivered DirectX Audio in DirectX 8.0, there

was one unfortunate limitation: The latency through the DirectMusic
Core Layer and DirectSound was roughly 85ms, unacceptably high for
triggered sound effects. Therefore, although a host application could
use the AudioPath mechanism to create rich content-driven sounds and
music and channel them through sophisticated real-time processing, it
had to bypass the whole chain and write directly to DirectSound for low
latency sound effects (in particular, “twitch” triggered sounds like gun
blasts) in order to achieve acceptable latency.
Fortunately, Microsoft improved latency in the release of DirectX
9.0. With this release, games and other applications can finally use the
same technology for all sounds and music.

Sounds like a lot of stuff to learn to use, eh? There is no denying that
all this power comes with plenty of new concepts and programming
interfaces to grok. The good news is you only need to know what
you need to know. Therefore, it is quite easy to make sounds (in fact,
a lot easier than doing the same with the original DirectMusic or
DirectSound APIs) and then, if you want to learn to do more, just dig
into the following chapters!
In that spirit, we shall write a simple application that makes a
sound using DirectX Audio. First, let’s acquaint ourselves with the
basic DirectX Audio building blocks.

DirectX Audio Building Blocks

In order to accomplish the simplest level of DirectX Audio program-
ming, we need to work with three objects — Segment, Performance,
and Loader. It is important to understand AudioPaths and COM pro-
gramming as well.

Segment
A Segment represents any playable audio. It might be a MIDI or wave
file, or it might be a DirectMusic Segment file authored in Direct-
Music Producer. You can load any number of Segments from disk,
and you can play any number of them at the same time. You can even
play the same Segment multiple times overlapping itself.
 DirectX Audio Programming 101 n 139

AudioPath
An AudioPath represents the journey from Segment to synthesizer/
mix engine to audio channel to final mix. A DirectSound Buffer man-
ages the audio channel, since a Buffer really represents one hard-
ware (or software) audio channel. There can be any number of
AudioPaths active at any time and any number of Segments can play
on the same AudioPath at once.

Performance
The Performance manages the scheduling of Segments. Think of it as
the heart of the DirectX Audio system. It allocates the AudioPaths
and connects them, and when a Segment plays, it connects the Seg- II
ment to an AudioPath and schedules its playback.

Unit
Loader
Finally, the Loader manages all file input/output. This is intentionally
separate from the Performance, so applications can override the
loading functionality, should they desire.

COM
COM is short for Component Object Model. Like all DirectX objects,
the Segment, AudioPath, Performance, and Loader are COM objects.
DirectX Audio’s usage of COM is at the most elementary level. Let’s
discuss COM now:
n Individual function interfaces represent each object as a table of
predefined functions that you can call. There is no direct access
to the internal data and functions of the object.
n Interfaces and objects are not the same. An interface provides an
understood way to communicate with an object. Importantly, it is
not limited to one object. For example, there is an interface,
IPersistStream, that provides methods for reading and writing
data to a file. Any COM object that includes an IPersistStream
interface can receive commands to read and write data from any
component that understands IPersistStream.
140 n Chapter 8

n A unique identifier called an IID (interface ID) identifies each

COM interface. An IID is really a GUID (globally unique ID), a
16-byte number guaranteed to be unique. For example,
IID_IDirectMusicPerformance represents the IDirectMusic-
Performance interface.
n Every COM object also has a unique identifier called a CLSID
(class ID), which is also a GUID. Continuing our example,
CLSID_DirectMusicPerformance identifies the DirectMusic Per-
formance object.
n COM has a special function called CoCreateInstance() that you
can use to create most COM objects. CoCreateInstance() finds
the DLL (dynamic-link library) that contains the object’s code,
loads it, and calls a standard entry function to create the object
and then return it to your program. The caller provides the
CLSID of the object and the IID of the desired interface, and
CoCreateInstance() returns the matching object and interface.
n With few exceptions, all COM methods return error codes in a
standardized format called HRESULT. When a method succeeds,
it returns S_OK. There are failure codes for everything from out
of memory to errors specific to the particular object. The SDK
documents all possible return codes for each interface.
n Each object supports the base COM interface, IUnknown, with
its three methods: AddRef(), Release(), and QueryInterface().
These methods provide a standardized way to manage the exis-
tence of the object and access its interfaces.
A program might reference an object with more than one
pointer in more than one place. This easily happens when objects
indirectly reference other objects (for example, a program refer-
ences several Segments that all reference a particular wave file).
It is important for the object to know how many pointers are cur-
rently referencing it. Every time something new points to the
object, it calls AddRef() to let the object know. The object’s
AddRef() implementation increments an internal reference coun-
ter and ensures that the object stays valid until the last external
reference is released via Release(). AddRef() is automatically
invoked when an object is created, so you rarely call it directly.
 DirectX Audio Programming 101 n 141

When a pointer to an object is no longer needed, the refer-

encing owner calls Release() to tell the object that a reference
has gone away. When the reference count drops to zero, indicat-
ing that nothing is using the object anymore, the object must go
away for good. Typically, it cleans up any variables of its own and
then frees its memory.
A COM object can support one or more interfaces. Because
all interfaces are based on IUnknown and must implement the
QueryInterface() method, it is possible to use one interface to
hopscotch to the next by calling QueryInterface() and naming the
desired interface (using the IID). For example, the Segment
object supports both the IDirectMusicSegment interface, which
you use to set Segment parameters, as well as the IPersist-
Stream interface, which the Loader uses to read data into the
II
Segment.

Unit
Therefore, from your perspective, COM programming in DirectX
Audio means you use CoCreateInstance() to create a few objects (the
rest are created automatically by other DirectX Audio objects),
Release() to get rid of them, and on the rare occasion Query-
Interface() to access additional features.
That’s enough explanation about COM. Let’s get on with the
program.

HelloWorld!
No introduction to any API is complete without that ancient but use-
ful ritual known as the “Hello World” application. Our first applica-
tion loads a wave file and plays it. Before we do anything, we need to
hook up the correct include files. DirectX Audio only requires
dmusici.h, as it has all the interfaces, data structures, and GUIDs for
the Performance Layer. In turn, it pulls in the dsound.h and dmusic.h
lower-level header files, so we get everything we need. Here is what
we need to put at the top of the application:
#include <dmusici.h>
142 n Chapter 8

þ Note You need to link with the GUID library that comes with DirectX.
Be sure to include dxguid.lib in your project’s linker settings or you will
get a ton of errors for each IID or CLSID used in your code but not
found by the linker. In VC 6, open the Project Settings… dialog, select
the Link tab, and add dxguid.lib to the Object/Library Modules edit box.
Of course, this has already been set up properly with the projects
included on the companion CD.

Now we can get to the program. Start by declaring the objects

we use. Notice that the objects are all interface pointers. They do
not yet represent real objects.
// The performance manages all audio playback.
IDirectMusicPerformance8* pPerformance = NULL;
// The loader manages all file I/O.
IDirectMusicLoader8* pLoader = NULL;
// Each audio file is represented by a Segment.
IDirectMusicSegment8* pSegment = NULL;

Before we start, initialize the COM system by calling CoInitialize().

This initializes the COM system so that CoCreateInstance() can
function.
// Initialize COM
CoInitialize(NULL);

þ Note If you are working with DirectPlay and using the same thread,
you should instead call CoInitializeEx(NULL,COINIT_MULTITHREADED),
which is what DirectPlay requires. DirectMusic is completely thread-safe,
so it works well with either.

Now, use COM to create the loader object. CoCreateInstance() is

passed the class ID of the loader as well as the interface ID for the
IDirectMusicLoader interface. CoCreateInstance() uses this infor-
mation to query the system registry, find the corresponding
dynamic-link library (DLL), which in this case is dmloader.dll, load
it, and tell the DLL to create one Loader object with the IDirect-
MusicLoader interface.
// Create the loader. This is used to load any DirectX
// Audio object that is read from a file. We use
// it to read a wave file as a Segment.
// The three parameters that really matter are the
// class ID, interface ID, and returned pointer. The
// two other parameters assume more sophisticated COM
// programming, which is not required for DirectX Audio.
 DirectX Audio Programming 101 n 143

CoCreateInstance(
CLSID_DirectMusicLoader, // Class ID of the DirectMusic loader.
NULL, // Ignore COM aggregation.
CLSCTX_INPROC, // Must run within this application.
IID_IDirectMusicLoader8, // Request the IDirectMusicLoader8 interface.
(void**)&pLoader); // Return pointer to interface here.

Likewise, create a Performance.

// Create the Performance. This manages the playback of sound and music.
CoCreateInstance(
CLSID_DirectMusicPerformance, // Class ID of the DirectMusic Performance.
NULL, // Ignore.
CLSCTX_INPROC, // Yada yada fancy COM stuff.
IID_IDirectMusicPerformance8, // Interface we want to get back.
(void**)&pPerformance ); // Returned IDirectMusicPerformance8 pointer.

You must initialize the Performance for playback. A call to II

InitAudio() archives this. At minimum, you must specify which

Unit
AudioPath type you want to use by default and how many channels
you need on that default path. An AudioPath represents a signal path
through which the sounds (Segments) play. AudioPaths can include
real-time effects, such as reverb or compression. You usually create
AudioPaths from configuration files that define their layout, but you
can also create predefined AudioPaths directly.
In our example, we create one stereo AudioPath with no effects,
using the DMUS_APATH_DYNAMIC_STEREO standard AudioPath
identifier.
// Initialize the Performance.
pPerformance->InitAudio(NULL,NULL,NULL,
DMUS_APATH_DYNAMIC_STEREO, // Default AudioPath type.
2, // Only two pchannels needed for this wave.
DMUS_AUDIOF_ALL,NULL);

Now it is time to use the Loader to read the wave file from disk. This
creates a Segment. Note that the Loader’s LoadObjectFromFile()
method has a lot in common with the system CoCreateInstance()
command. Again, you provide a class ID for the type of object you
wish to load, and you provide an interface ID to indicate which inter-
face you expect to use. Not surprisingly, the Loader turns around and
calls CoCreateInstance() to create the object you requested.
Provide the name of the file to the Loader. In our HelloWorld
application, it is, naturally, a wave file titled Hello.wav.
144 n Chapter 8

// Load the demo wave. We are making the assumption that it is in the
// working directory.
// In VC6, this is set in the Project Settings/Debug/General/Working directory: field.
if (SUCCEEDED(pLoader->LoadObjectFromFile(
CLSID_DirectMusicSegment, // Class identifier.
IID_IDirectMusicSegment8, // ID of desired interface.
L"Hello.wav", // Filename.
(LPVOID*) &pSegment // Pointer that receives interface.
)))
{

Once the Loader reads the wave into a Segment, we still need to
prep it for play. The Download() command moves any audio data
(waves and DLS instruments) required by the Segment into the syn-
thesizer, priming it so it can play instantly.
// Install the wave data in the synth
pSegment->Download(pPerformance);

Now we are ready to rock. The Performance has a method,

PlaySegmentEx(), which can play a Segment in all kinds of neat
ways. We just use it to play our Segment in the simplest way for
now. I cover the neat bells and whistles in upcoming chapters.
// Play the wave.
pPerformance->PlaySegmentEx(
pSegment, // Segment to play.
NULL,NULL,0,0,NULL,NULL,NULL);

Wow! Sound!
PlaySegmentEx() returns immediately as the sound starts play-
ing. Normally, this would be no problem, since the application would
continue running. However, HelloWorld does not do anything after
playing the sound, so it needs to wait patiently for eight seconds
(roughly the duration of the sound) before cleaning up and closing
down.
// Wait eight seconds to let it play.
Sleep(8000);

Okay, it should be done by now, so unload the Segment and release

all objects.
// Unload wave data from the synth.
pSegment->Unload(pPerformance);
 DirectX Audio Programming 101 n 145

// Release the Segment from memory.

pSegment->Release();
}

// Done with the performance. Close it down, and then release it.
pPerformance->CloseDown();
pPerformance->Release();

// Same with the loader.

pLoader->Release();

// Bye bye COM.

CoUninitialize();

If you compile this and have trouble getting it to make any sound, the
problem is almost certainly that it cannot find the wave file. Make
sure that your project’s working directory is set to the Media direc- II
tory, where Hello.wav resides. Refer to your compiler’s user manual

Unit
for information on how to set your working directory. Double-click
on the version in the bin directory; it successfully plays because I
placed a copy of Hello.wav there.

It’s a 3D World
Now that we have loaded and played a sound using DirectX Audio,
let’s see how easy it is to add a little code to move that sound in 3D.
To do this, we need to start working with AudioPaths. We create
a 3D AudioPath that mixes all audio into a DirectSound 3D Buffer.
We then use the IDirectSound3D interface on the buffer to move the
sound as it plays.
Go back to the InitAudio() call, and change it to create a 3D
AudioPath as the default path.
// Initialize the performance with a default 3D AudioPath.
pPerformance->InitAudio(NULL,NULL,NULL,
DMUS_APATH_DYNAMIC_3D, // Default AudioPath Type.
2, // Only two pchannels needed for this wave.
DMUS_AUDIOF_ALL,NULL);

This tells the performance to create a 3D AudioPath as its default

path. Later, after the wave has been loaded and is ready to play, we
access that path.
146 n Chapter 8

// Before we play the wave, get the 3D AudioPath so we can set

// the 3D position before and during playback.
IDirectMusicAudioPath *p3DPath = NULL;
pPerformance->GetDefaultAudioPath(&p3DPath);

The AudioPath GetObjectInPath() interface method grants direct

access to any component within the span of the path. It is called
GetObjectInPath(), and you will become quite familiar with it as you
work with DirectMusic. In this case, use GetObjectInPath() to
directly access the 3D Buffer interface.
// Extract the 3D buffer interface from the 3D path.
IDirectSound3DBuffer8 *p3DBuffer = NULL;
p3DPath->GetObjectInPath(0,
DMUS_PATH_BUFFER,0,GUID_NULL,0,
IID_IDirectSound3DBuffer,
(void **) &p3DBuffer);
// Don't need this anymore.
p3DPath->Release();

Now that we have the 3D interface, we can change the spatial posi-
tion of any Segments played on this path. Before we play our sound,
set its initial position in 3D space.
// Initialize the 3D position so the sound does not jump
// the first time we move it.
p3DBuffer->SetPosition(-2,0.2,0.2,DS3D_IMMEDIATE);

Then, go ahead and play the wave.

pPerformance->PlaySegmentEx(
pSegment, // Segment to play.
NULL,NULL,0,0,NULL,NULL,NULL);

Now that the sound is playing, we can progressively move it through

space.
// Loop and slowly move the sound.
float flXPos = (float) -1.99;
for ( ; flXPos < 2.0; flXPos += (float) 0.01 )
{
Sleep(20); // Equivalent to updating at 50fps
p3DBuffer->SetPosition(flXPos, flXPos, (float) 0.2, DS3D_IMMEDIATE);
}

Ooh, isn’t that cool?

 DirectX Audio Programming 101 n 147

We are finished now, so remember to release the buffer while

shutting down.
// Okay, done. Release the buffer interface.
p3DBuffer->Release();

That concludes our whirlwind tour of basic DirectX Audio program-

ming. Believe it or not, we covered most of what you need to know
to get sound and music playing in your application. It can get so
much more fun if you are willing to dig in deeper and really explore,
and that is what we do in the upcoming chapters.

Using the CD II
This is a good time to introduce the Unit II material on the compan-

Unit
ion CD.
Take a look in the Unit II folder on the CD. It includes source
directories for each project discussed in these chapters. Each direc-
tory is labeled with the chapter number and project name. So, for
example, this chapter’s programming projects are 8_Hello and 8_
Hello3D. The Bin directory contains the final executables of each
project, the SourceMedia folder has the DirectMusic Producer
source files to create the content, and the Media directory contains
the final run-time media files for the sample projects.
You can either copy these files by hand or you can run the Setup
program, also in the Unit II folder, to automatically install these files
on your hard drive. It includes an uninstaller, so you can install, play
with the materials, and when done wipe them out to make room for
more on your hard drive.
This page intentionally left blank.
 Chapter 9

The Loader

A lthough the Loader is a decidedly unglamorous feature of

DirectX Audio, it is a good idea to become familiar with it. Not
only does the Loader manage the files you load, it also manages how
they are cached in memory. So, even if you skip past this chapter for
now, be sure to come back and give it a read later.

Why the Loader?

The Loader serves three purposes: managing linkages between
DirectMusic files, providing convenience functions for easy file load-
ing and manipulation, and being replaceable so an application can
write its own customized Loader that can deliver resources from a
packaged file format of its own devising (more on why this is useful
later).

Manage File Relationships

The Loader manages the relatively complex linkages that exist
between DirectMusic files. For example, a Style playback Segment
file might reference both a Style file and a DLS instrument file.
Because multiple Segments might use the same Style and DLS files,
it is extremely inefficient to bundle up all the Style and DLS data into
the Segment. You end up with multiple files bundling redundant
Style and DLS data. Worse, all this data would redundantly load into
memory with the Segment. The solution is to place the shared
Styles and DLS instruments in their own files that the Style play-
back Segment references. At the time the Segment is loaded, the
appropriate Style and DLS files must also be loaded and correctly

149
150 n Chapter 9

linked. The Loader provides a centralized mechanism to manage this

for you.

Segment A Segment B Segment C

Wave
Wave
DLS Collection
Style

Figure 9-1: File referencing in DirectMusic.

In Figure 9-1, three Segment files reference different combinations

of DLS, Style, and wave files. The Loader manages internal pointers
to the DLS, Style, and wave objects, so only one instance of each is
necessary.

þ Note In some circumstances it is reasonable to bundle all of the files

required by a Segment into the Segment itself. This can include waves,
DLS, Styles, and more. You can create Segments with embedded files
with the Segment Designer in DirectMusic Producer. This has the advan-
tage of packaging the Segment as just one file with no dependencies. It
is great for music file players, since there is no need to ship a set of
interconnected files.

Provide a Convenient File Load API

The Loader serves as a convenience function for easy file loading.
All DirectMusic objects that can be loaded from files support the
standard COM file I/O interface IPersistStream. By supporting this,
any DirectMusic object can be loaded in a few steps by:
1. Calling CoCreateInstance() to create an instance of the
object.
2. Calling the object’s QueryInterface() method to retrieve its
IPersistStream interface.
 The Loader n 151

3. Creating a file stream object represented by the standard

IStream interface.
4. Presenting the IStream to the IPersistStream Load()
method, which reads the data from the stream and creates
the instance of a loaded object.
Clearly, this is a lot of work, and yet it is something that every appli-
cation must do. Therefore, it makes sense to provide a standard
helper so the application does not have to do this redundant work.

Support Customization
Applications like games often keep resources bundled up in one large
file. Although DirectMusic’s container format provides an easy way II
to accomplish that, it still does not provide encryption or compres-
sion features that a game might require. Because the Loader uses a

Unit
publicly defined interface, you can create your own Loader and have
it retrieve the data in any way that it pleases.

Loading a File
So, how do you use the Loader to load files? As we saw in the
HelloWorld example, you must first use the COM CoCreate-
Instance() call to create a Loader.
CoCreateInstance(CLSID_DirectMusicLoader, NULL,
CLSCTX_INPROC, IID_IDirectMusicLoader8,
(void**)&pLoader);

This creates the Loader and gets it ready for business.

Remember that you will need to call Release() on the Loader
when you are done using it, which should be after all file I/O is com-
pleted for your application.

þ Note Because the Loader caches files for efficiency, it is very

important that you keep the same Loader around. I have seen situations
where a clever programmer got rid of one pesky variable by creating a
Loader every time a file needed to be loaded and then released it. This
solution worked, but memory usage skyrocketed as each Loader
reloaded the same styles, waves, and DLS Collections instead of
sharing them. In addition, file I/O and download times took longer for
the same reason.
152 n Chapter 9

Once you have an instance of a Loader, you need to tell it where to

look for your file. Call the SetSearchDirectory() method to tell the
Loader where you want it to search for referenced files. You might
also use SetObject() if there are specific referenced files that you
want it to know about. Then, load the file, using either the
LoadObjectFromFile() or GetObject() methods.
Let’s look at each of these methods in detail.

SetSearchDirectory()
SetSearchDirectory() tells the Loader which directory to go to when
searching for a file that is referenced by the file currently being
loaded.
HRESULT SetSearchDirectory(
REFGUID rguidClass,
WCHAR* pwszPath,
BOOL fClear
);

n REFGUID rguidClass: This is the class ID for the type of file

(a REFGUID is a pointer to a GUID or, in this case, CLSID). You
can set the Loader to look in different directories for different
types of files. For example, to set the search directory for DLS
files, this is set to CLSID_DirectMusicCollection. To set the
search directory for all types, use GUID_DirectMusicAllTypes.
n WCHAR *pwszPath: The Loader will look in this directory
path for files. DirectMusic uses Unicode characters for all nam-
ing and filenames, hence the 16-bit WCHAR instead of ASCII
char.
n BOOL fClear: This parameter indicates whether SetSearch-
Directory() should flush the current directory of all references.
The fClear option guards against files with the same name in dif-
ferent directories. SetSearchDirectory() flushes the files in the
old directory from the Loader if this flag is set. However, if an
object is already loaded, the fClear option does not remove it
from the cache.
 The Loader n 153

LoadObjectFromFile()
You can use the Loader to read a file in two ways: GetObject() and
LoadObjectFromFile(). LoadObjectFromFile() is the more conve-
nient approach. GetObject() has all kinds of clever options for loading
from files, streams, or memory. However, LoadObjectFromFile() pro-
vides a very straightforward way to read DirectX Audio objects
directly from files.
HRESULT LoadObjectFromFile(
REFGUID rguidClassID,
REFIID iidInterfaceID,
WCHAR *pwzFilePath,
void ** ppObject
);
II
n REFGUID rguidClassID: This is the class ID for the type of

Unit
object to load. The Loader does not know anything about how to
load a particular type of file. Instead, it calls COM to create an
instance of the object, which then does the actual file reading
and, when finished, returns the object to the caller. For example,
to load a Segment, this parameter is CLSID_DirectMusic-
Segment.

þ Note GUID_DirectMusicAllTypes is not valid for LoadObjectFrom-

File() because it does not define a specific object type.

n REFIID iidInterfaceID: You must also provide an interface

identifier. An interface of the requested type returns in ppObject.
For example, when loading a Segment, you can request the stan-
dard Segment interface IDirectMusicSegment8 with
IID_IDirectMusicSegment8.
n WCHAR *pwzFilePath: The file path provides the file name
and can be a complete path or one relative to the search direc-
tory as set with SetSearchDirectory(). The Loader first assumes
that the file name is a complete path and tries to load from that
absolute address. If that fails, it concatenates the file name onto
the current search directory and tries to look there.
n void ** ppObject: The loaded object returns in ppObject, which
is a pointer to the variable that receives the returned object
interface.
154 n Chapter 9

GetObject()
While LoadObjectFromFile() does the job 90 percent of the time,
there are scenarios where a little more flexibility is needed.
n The Loader can identify every file with alternate information,
including a name, version, date stamp, unique GUID, and, of
course, file name. The application may need to use one or more
of these other variables to search for the file, especially if it is
already cached. LoadObjectFromFile() can only use the file
name.
n The data might be stored in a memory buffer or stream instead
of a file. LoadObjectFromFile() can only load from a file.
GetObject() provides a way to read from alternate sources. It does so
by using the DMUS_OBJECTDESC structure, which allows for a
very thorough description of the object.
typedef struct _DMUS_OBJECTDESC {
DWORD dwSize; // Size of this, should it grow in later versions.
DWORD dwValidData; // Flags indicating which fields have valid data.
GUID guidObject; // Every object can have a unique ID.
// This is saved in the file.
GUID guidClass; // Every object must belong to a predefined class.
FILETIME ftDate; // The date when the object was created.
DMUS_VERSION vVersion; // Major and minor version info.
WCHAR wszName[DMUS_MAX_NAME]; // Friendly name of object.
WCHAR wszCategory[DMUS_MAX_CATEGORY]; // Optional category name.
WCHAR wszFileName[DMUS_MAX_FILENAME]; // The file name, optionally with path.
LONGLONG llMemLength; // If memory is used instead of a file, sets length.
LPBYTE pbMemData; // Pointer to memory data.
IStream *pStream // Or, stream to read data from.
} DMUS_OBJECTDESC, *LPDMUS_OBJECTDESC;

Flag bits in dwValidData indicate which fields are valid. For example,
the flag DMUS_OBJ_NAME means that the string in wszName is a
valid name for the object. Conversely, a cleared bit in dwValidData
indicates that the corresponding data field is empty. The fields that
are most important for tracking resources are wszFileName and
guidObject. These help identify and validate the correct file. The
GUID is guaranteed to be unique but must be authored into the files,
typically via DirectMusic Producer.
In addition to identification fields, the caller can use pbMemData
and pStream to present data in a memory or stream format. We
 The Loader n 155

cover an example of loading a Segment from memory later in this

chapter (see the “Loading from a Resource” section).
HRESULT GetObject(
LPDMUS_OBJECTDESC pDesc,
REFIID riid,
LPVOID FAR * ppObject
);

n LPDMUS_OBJECTDESC pDesc: This is a pointer to a

DMUS_OBJECTDESC structure that describes the object.

þ Note The DMUS_OBJECTDESC structure is quite large because of all

the Unicode strings embedded within it. If you are managing many
structures that have DMUS_OBJECTDESC embedded within them, be
aware that it eats up 848 bytes for each instance. II
n REFIID riid: Like LoadObjectFromFile() or even CoCreate-

Unit
Instance(), you must provide the interface ID for the interface
that you intend to get back. For example, to get back a Segment
interface, pass in IID_DirectMusicSegment8.
n void **ppObject: The loaded object returns in ppObject, which
is a pointer to the variable that receives the returned object
interface.

SetObject()
Sometimes, you need to point the Loader at a file source before
another object references it. This could be because the file path is
broken or, more likely, the reference object loads from memory or an
IStream. Therefore, you need to point the Loader directly at it using
SetObject(). This is almost identical to GetObject(), except it does
not actually load anything. It just tells the Loader where the object
is, and the Loader pulls it in later when needed.
HRESULT SetObject(
LPDMUS_OBJECTDESC pDesc
);

n LPDMUS_OBJECTDESC pDesc: This is a pointer to a

DMUS_OBJECTDESC structure that describes the object. In
particular, the location of the object, be it a file path (wszFile-
Name), memory pointer (pbMemData), or IStream (pStream),
must be filled in and the appropriate flags in dwValidData must
156 n Chapter 9

be set. SetObject() will use that to parse the file header to

retrieve additional information, including the name and GUID.

Managing the File List

The Loader maintains a list of all files that it knows about. This list
serves two purposes:
n To keep track of where various files are located so that when the
caller requests a resource, the Loader can immediately find and
load that resource.
n To hold on to loaded file objects so that multiple requesters can
share them. We refer to this as the cache. When the caller
requests a cached object, the Loader immediately returns the
object with its reference count incremented.
There are several methods of the IDirectMusicLoader interface that
give you control over the file list.

ScanDirectory()
ScanDirectory() searches a directory for a specific type of file, as
defined by its class ID and file extension (the * wildcard is sup-
ported.) For example, you might use it to search for all Style files
within a particular directory. This does not load any of the files.
Instead, it adds all their location information to the file list. To do
this, it parses each file header, looking for all pertinent information
to fill the DMUS_OBJECTDESC record for each file. Therefore,
internal information, including object name and GUID, is retrieved
and stored in the list.
ScanDirectory() is useful primarily in conjunction with Enum-
Object(), which lists the objects one by one. This is useful for
applications that need to display all available files of a particular type.
This is great for media playback and authoring applications but not
needed for games that usually have dedicated soundtracks. As long
as file names are unchanged and SetSearchDirectory() is used appro-
priately to establish where to find them, ScanDirectory() incurs
unnecessary overhead, so avoid using it when possible.
 The Loader n 157

EnumObject()
EnumObject() scans through the file list and retrieves information
about all files of a specific file type, identified by its class ID. For
example, an application might use this to present the user with a list
of all playable media files. It would do so by calling ScanDirectory()
to scan for all Segment files (which can include wave and MIDI files
as well) and then calling EnumObject() to list them.

EnableCache()
EnableCache() and ClearCache() are very important for effective
memory management.
When the Loader opens a file, it keeps an internal reference to II
the loaded object in its file list. The Loader effectively caches the

Unit
file, so a subsequent request simply receives a pointer to the original
object. This is all fine and well, but it has its cost. Some file types,
like Segments, are rarely, if ever, referenced by anything else. There-
fore, it is usually just a memory waste to cache them. EnableCache()
lets the application decide which file types the Loader should cache.
File types that typically are referenced by others, like Styles and
DLS Collections, are great candidates for Loader caching since it
ensures that just one instance of a Style (for example) is created and
then referenced by all objects that use it. Segments are not.

þ Note It is not always the case that there are no objects referencing
Segments. Both Script objects and Segment Trigger Tracks can reference
Segments. In those situations, enabling caching may have merit. Even
then, it is typically rare to have multiple Scripts or Segments referencing
the same Segment.

ClearCache()
ClearCache() provides a quick mechanism for clearing out all objects
of a particular type. For example, you might call ClearCache() for
some or even all file types when moving from one scene to another
in a game.
158 n Chapter 9

þ Note ClearCache() does not remove an object from memory unless

the Loader is the only entity referencing it. For example, if you call
ClearCache() for all DLS Collections, it will remove all references from
the Loader, but any DLS Collections currently in use by loaded
Segments will continue to stay in memory until the last Segment is
released, which in turn does the final release on the DLS Collection.
This means that music and sound effects will continue to play without a
glitch. Ah, the beauty of COM….

ReleaseObject()/ReleaseObjectByUnknown()
ReleaseObject() and ReleaseObjectByUnknown() can be used in lieu
of ClearCache() to release a specific object from the cache. Use
these when you need to remove one but not all objects from the
cache.
These methods are identical in functionality. ReleaseObject()
requires the IDirectMusicObject interface, which all loadable objects
must support. However, it is obnoxious to have to call Query-
Interface() to get that interface in order to then call ReleaseObject().
So, with DirectX 8.0, ReleaseObjectByUnknown() was added to the
API. It lets you use the regular interface for the object, and it inter-
nally queries for the IDirectMusicObject interface.

Garbage Collection
In DX8, DirectMusic introduced three features that could easily
cause reference loops. These are Scripting, Script Tracks, and Seg-
ment Playback Tracks. For example, one script might reference
another script that calls back into the first. On the other hand, a Seg-
ment Trigger Track triggers the same Segment or another that
triggers the original Segment again. These are all perfectly reason-
able scenarios, and they do load and run appropriately. However,
when it comes time to clean up, a self-referencing loop keeps the ref-
erence counts of all objects in the loop incremented by at least one,
and so they never go away, even though nothing actually references
them outside the loop.
The CollectGarbage() method solves this problem. Collect-
Garbage() scans for all objects that have been completely released by
the Loader as well as the application and calls an internal method on
the object, forcing it to release its references to other objects and
 The Loader n 159

thus breaking any cyclic references and removing unneeded objects

from memory.

LoaderView Sample Application

The LoaderView sample demonstrates both how to load files with
the Loader and how to manage the cache. In truth, it goes a little
overboard demonstrating all of the caching features, but by playing
with this, you can get a good understanding of how the Loader man-
ages its file list and how you can consequently control it for your
application.
LoaderView is a windowed application that lets you open a Seg-
ment, play it, view the Loader’s internal lists of all object types that II
were loaded as a result of loading the Segment, and then play with

Unit
the object lists using the standard Loader methods and observe the
behavior. You can run LoaderView by either going to the Unit II\
9_Loader directory and building LoaderView or running it directly
from the Bin directory. LoaderView opens a simple dialog window
with commands for loading a Segment and managing the Loader (see
Figure 9-2).

Figure 9-2: LoaderView.

160 n Chapter 9

Let’s first explore LoaderView, discuss how it uses the various

Loader features, and then look at the code.
Click on the Open… button to open a File dialog.

Search for a Segment, MIDI, or wave file to load. Files that reference
other files are particularly interesting. For example, EvilDonuts.sgt
has references to Style, DLS, and wave files.
When you click on the Open… button, the Loader’s GetObject-
FromFile() method is called to load the Segment. Once the file is
loaded, its name is listed in the box underneath the Open… button.

Click on the Play and Stop buttons to audition the Segment. We talk
a lot more about playing Segments in the next chapter, but at least
we can audition our Segment for now. With the Loader being as dull
a subject as it is, it helps to be able to make noise.
Now, let’s look at what is in the Loader’s cache.

The Type drop-down lets you choose which media

type to enumerate and display. Click on the list to
choose from Segments, Waves, DLS Collections,
Styles, and quite a few others.
 The Loader n 161

The list box below the Type drop-down displays all files of the media
type that you requested that the Loader is currently managing.
Choose a list that displays one or more files. For example, “Evil
Donut” uses two DLS Collections, so you might try that. Note that
they all have “-cached” after their name. This indicates that the
Loader currently has a reference to an instance of the object loaded
in memory.
You can turn off caching. Uncheck the Enable Cache option.
This calls the EnableCache() method with a false flag to disable it.
Disabling a cache also automatically clears it, so notice now that the
display lists these files no longer as cached. Remember: Once the
Loader does not have the file cached, it must reload the file when the
application subsequently requests the file. This incurs file and CPU
overhead and extra memory overhead if the object is still in use
II
since there are two copies of the object in memory once the second

Unit
instance loads.
Go back and press the Play button. Notice that the music still
plays. Although the Loader no longer has its references to the
objects in memory (Style, DLS, etc.), the Segment still does, so it
plays flawlessly. Later, when the Loader and application finally
release the Segment, it in turn releases all objects that it references.
Ah, such is the power and beauty of COM.
Likewise, you can release an individual item. Go to another list,
such as DLS Collections, that has two or more objects. Click on one
to select it, and then click on the Release Item button. Note that
just the one object loses its “cached” designation.
Finally, click on the Scan Files button. This calls ScanDirec-
tory() for the selected file type in the current search directory. The
list updates with all additional files found in the directory. Notice that
ScanDirectory() only retrieves the file information and does not load
and cache the file objects themselves.

LoaderView Source
The CAudio class manages all audio and Loader functionality. The
LoaderView UI (CLoaderDlg) makes calls into CAudio to load and
play Segments. I wrote the code using MFC (Microsoft Foundation
Classes). We do not discuss the UI code here. I do my best to keep
162 n Chapter 9

the UI code and the audio code separated in all programming exam-
ples, so we can focus on just the audio portions. Of course, I include
the UI source in the LoaderView source code project in the Unit II\
9_Loader directory on the companion CD.
CAudio also tracks a table of object descriptors (DMUS_
OBJECTDESC) for the currently selected object type and provides
calls to manipulate the list. This is not enormously useful for regular
applications, but it is useful for exploring the Loader’s caching
behavior, which is what LoaderView is all about. We walk through
the layout of CAudio in just a second.

Predefined Object Types

Although the Loader is completely format agnostic, for convenience
we concentrate on nine familiar DirectX Audio formats. To make
communication between the UI and CAudio as simple as possible, we
predefine these.
// Define the nine categories of DirectX Audio objects we will track
// in the loader's file list.

enum OBJECT_TYPE
{
OT_AUDIOPATH = 0, // AudioPath configuration file.
OT_BAND = 1, // Band file.
OT_CHORDMAP = 2, // ChordMap file.
OT_CONTAINER = 3, // Container - carries a collection of other objects.
OT_DLS = 4, // DLS Instrument Collection file.
OT_SEGMENT = 5, // Segment file.
OT_STYLE = 6, // Style file.
OT_WAVE = 7, // Wave file.
OT_GRAPH = 8 // ToolGraph file.
} ;
#define OBJECT_TYPES 9

CAudio Class
The CAudio class manages the Loader, the Performance, and one
Segment. It has routines for initialization and shutdown. It has rou-
tines to load and play Segments. It has a bunch of routines dedicated
to exploring and managing the Loader’s file lists. Since we cover ini-
tialization and playback in other chapters, here we focus just on the
code for managing the Loader.
Here is the full definition of the CAudio class:
 The Loader n 163

class CAudio
{
public:
// Initialization and shutdown methods.
CAudio();
~CAudio();
HRESULT Init();
void Close();
// Convenience methods for accessing DirectX Audio interfaces.
IDirectMusicPerformance8 * GetPerformance() { return m_pPerformance; };
IDirectMusicLoader8 * GetLoader() { return m_pLoader; };
// Methods for loading, playing, stopping, and removing Segments.
IDirectMusicSegment8 * LoadSegment(WCHAR *pwzFileName);
IDirectMusicSegment8 * GetSegment() { return m_pSegment; };
void PlaySegment();
void StopSegment();
void ClearSegment();
// Methods for managing the loader's file lists and retrieving data from it.
II
DWORD SetObjectType(OBJECT_TYPE otSelected);

Unit
void GetObjectInfo(DWORD dwIndex, char *pszName, BOOL *pfCached);
BOOL GetCacheStatus() { return m_afCacheEnabled[m_otSelected]; };
void ChangeCacheStatus();
void ReleaseAll();
void ReleaseItem(DWORD dwItem);
void ScanDirectory();
private:
// DirectX Audio interfaces for performance, loader, and Segment.
IDirectMusicPerformance8 * m_pPerformance; // The Performance.
IDirectMusicLoader8 * m_pLoader; // The Loader.
IDirectMusicSegment8 * m_pSegment; // Currently loaded Segment.
// Only one object type is active at a time. This reflects the
// current choice in LoaderView's type drop-down list.
OBJECT_TYPE m_otSelected; // Currently selected object type.
// We maintain a table of enumerated objects of the current selected type.
DMUS_OBJECTDESC * m_aObjectTable; // Table of object descriptors.
DWORD m_dwObjectCount; // Size of table.
// We use a static array of class IDs to identify class type in calls to loader.
static const GUID * m_saClassIDs[OBJECT_TYPES];
// We need a flag for each object type to indicate whether caching is enabled.
BOOL m_afCacheEnabled[OBJECT_TYPES];
// Current search directory.
WCHAR m_wzSearchDirectory[DMUS_MAX_FILENAME];
};

The following routines manage the Loader:

n SetObjectType() tells CAudio which object type to view. When
the user selects a type from the drop-down list, LoaderView calls
this. It builds a table of DMUS_OBJECTDESC structures in
m_aObjectTable.
164 n Chapter 9

n GetObjectInfo() retrieves name and caching information from a

specific entry in m_aObjectTable.
n ReleaseItem() tells the Loader to release its cache reference for
just one item.
n ReleaseAll() tells the Loader to release all its cached references.
n ChangeCacheStatus() tells the Loader to change whether or not
it is caching the current object type.
n ScanDirectory() tells the Loader to scan the current working
directory to build a list of available files.
Before we discuss each of these routines, we must initialize the
object type to class ID relationship.

Initialization
CAudio’s Init() method creates the Loader and Performance and
initializes both. It also disables caching for Segments to conserve
memory.
HRESULT CAudio::Init()

{
//Init COM.
CoInitialize(NULL);

// Create loader.
HRESULT hr = CoCreateInstance(
CLSID_DirectMusicLoader, NULL, CLSCTX_INPROC,
IID_IDirectMusicLoader8, (void**)&m_pLoader);
if (SUCCEEDED(hr))
{
// Turn off caching of Segments.
m_pLoader->EnableCache(CLSID_DirectMusicSegment,false);
// Create performance.
hr = CoCreateInstance(
CLSID_DirectMusicPerformance, NULL, CLSCTX_INPROC,
IID_IDirectMusicPerformance8, (void**)&m_pPerformance);
}
if (SUCCEEDED(hr))
{
// Once the performance is created, initialize it.
// We'll use the standard music reverb audio path and give it 128 pchannels.
hr = m_pPerformance->InitAudio(NULL,NULL,NULL,
DMUS_APATH_SHARED_STEREOPLUSREVERB, // Default AudioPath type.
128,DMUS_AUDIOF_ALL,NULL); }
 The Loader n 165

if (FAILED(hr))
{
// No luck, give up.
Close();
}
return hr;
}

We also have a static array of class IDs that translate from predefined
integers to GUIDS, since it is much more convenient to work with
predefined integers because they directly correlate with positions in
the UI’s drop-down list of object types.
const GUID *CAudio::m_saClassIDs[OBJECT_TYPES] = {
&CLSID_DirectMusicAudioPathConfig, // OT_AUDIOPATH
&CLSID_DirectMusicBand, // OT_BAND
&CLSID_DirectMusicChordMap, // OT_CHORDMAP II
&CLSID_DirectMusicContainer, // OT_CONTAINER

Unit
&CLSID_DirectMusicCollection, // OT_DLS
&CLSID_DirectMusicSegment, // OT_SEGMENT
&CLSID_DirectMusicStyle, // OT_STYLE
&CLSID_DirectMusicGraph, // OT_WAVE
&CLSID_DirectSoundWave // OT_GRAPH
};

SetObjectType()
SetObjectType() is called when a new file class is requested for dis-
play. It sets the variable m_otSelected to the new object type and
then builds the m_aObjectTable table of DMUS_OBJECTDESC
structures by using the Loader’s EnumObject() method.
DWORD CAudio::SetObjectType(OBJECT_TYPE otSelected)
{
// We track the objects in an array, so delete the current array.
delete [] m_aObjectTable;
m_aObjectTable = NULL;
m_dwObjectCount = 0;
m_otSelected = otSelected;
HRESULT hr = S_OK;

We need to know how many objects the Loader has of the current
type before we can allocate the memory for the array. However, the
Loader does not provide a routine for directly returning this informa-
tion. Therefore, we use the enumeration method and scan until it
returns S_FALSE, indicating that we passed the range.
166 n Chapter 9

// First, find out how many objects there are

// so we can allocate the right size table.
for (;hr == S_OK;m_dwObjectCount++)
{
DMUS_OBJECTDESC Dummy;
Dummy.dwSize = sizeof(DMUS_OBJECTDESC);
hr = m_pLoader->EnumObject(*m_saClassIDs[m_otSelected],m_dwObjectCount,&Dummy);
}
m_dwObjectCount--;
// Now we know how many, so allocate the table.
if (m_dwObjectCount)
{
m_aObjectTable = new DMUS_OBJECTDESC[m_dwObjectCount];
if (m_aObjectTable)
{

Once we have the array, filling it is simply a matter of calling

EnumObject() a second time — this time iterating through the array.
// Then fill the table with the object descriptors.
DWORD dwIndex = 0;
for (dwIndex = 0;dwIndex < m_dwObjectCount;dwIndex++)
{
// Set the .dwSize field so the loader knows how big it really is and
// therefore which fields can be written to.
m_aObjectTable[dwIndex].dwSize = sizeof(DMUS_OBJECTDESC);
m_pLoader->EnumObject(*m_saClassIDs[m_otSelected],
dwIndex,&m_aObjectTable[dwIndex]);
}
}
}

Finally, return the number of objects counted. The UI can use this
information when displaying the list of objects.
return m_dwObjectCount;
}

GetObjectInfo()
Once the UI has called SetObjectType() to create the table of object
descriptors, it needs to populate its list view of those same objects.
It calls GetObjectInfo() to iterate through the table and retrieve the
name and cache status of each object.
void CAudio::GetObjectInfo(DWORD dwIndex, char *pszName, BOOL *pfCached)
{
if (m_aObjectTable && (dwIndex < m_dwObjectCount))
{
 The Loader n 167

// If the wszName field is valid, use it.

if (m_aObjectTable[dwIndex].dwValidData & DMUS_OBJ_NAME)
{
// Convert from Unicode to ASCII.
wcstombs(pszName,m_aObjectTable[dwIndex].wszName,DMUS_MAX_NAME);
}
// Else if the file name is available, use it.
else if (m_aObjectTable[dwIndex].dwValidData & DMUS_OBJ_FILENAME)
{
// First, get the Unicode name.
WCHAR wzTemp[DMUS_MAX_FILENAME];
wcscpy(wzTemp,m_aObjectTable[dwIndex].wszFileName);
// Then, remove the path.
WCHAR *pwzStart = wcsrchr(wzTemp,'\\');
if (pwzStart) pwzStart++;
else pwzStart = wzTemp;
// Convert to ASCII.
wcstombs(pszName,pwzStart,DMUS_MAX_NAME);
II
}

Unit
// Uh oh. No name.
else
{
strcpy(pszName,"<No Name>");
}
// The DMUS_OBJ_LOADED flag indicates that the resource is currently cached.
*pfCached = (m_aObjectTable[dwIndex].dwValidData & DMUS_OBJ_LOADED) && TRUE;
}
}

ReleaseItem()
ReleaseItem() tells the Loader to release the selected item from its
cache. The item will continue to be listed, but the Loader will no lon-
ger point to it and have it AddRef()’d. Should the item be requested a
second time, the Loader will create a new instance and load it.
void CAudio::ReleaseItem(DWORD dwItem)

{
IDirectMusicObject *pObject;
if (SUCCEEDED(m_pLoader->GetObject(&m_aObjectTable[dwItem],
IID_IDirectMusicObject,
(void **) &pObject)))
{
m_pLoader->ReleaseObject(pObject);
pObject->Release();
}
}
168 n Chapter 9

ReleaseAll()
ReleaseAll() tells the Loader to clear all objects of the currently
selected type from its cache. All objects not currently AddRef()’d by
the application will go away. This is the same as calling Release-
Item() for every item in the list.
void CAudio::ReleaseAll()
{
m_pLoader->ClearCache(*m_saClassIDs[m_otSelected]);
}

ChangeCacheStatus()
You can tell the Loader to enable or disable caching on a class-by-
class basis. ChangeCacheStatus() tells the Loader to flip the caching
for the currently selected type. It does so by tracking the cache state
with an array of Booleans, one for each object type. It flips the state
and calls the Loader’s EnableCache() method to enable or disable the
caching.
void CAudio::ChangeCacheStatus()

{
// Flip the state in the m_afCacheEnabled table.
m_afCacheEnabled[m_otSelected] = !m_afCacheEnabled[m_otSelected];
// Call EnableCache with the new state.
m_pLoader->EnableCache(*m_saClassIDs[m_otSelected],m_afCacheEnabled[m_otSelected]);
}

ScanDirectory()
ScanDirectory() tells the Loader to look in the current working
directory and read all files of the currently selected type. To do so, it
calls the Loader’s ScanDirectory() method and passes it the wildcard
* extension. This tells the Loader to open every single file in the
directory and attempt to parse it with the object defined by the class
ID. If the object cannot parse the file, the Loader does not include it
in the list. For example, the Segment object knows how to parse
Segment, MIDI, and wave files. It fails with all other files, and so the
Loader does not include the failed files in the resulting list. There is
one case where multiple types can parse the same file. Both the
 The Loader n 169

CLSID_DirectSoundWave and CLSID_DirectMusicSegment objects

know how to parse a wave file.
void CAudio::ScanDirectory()
{
m_pLoader->ScanDirectory(*m_saClassIDs[m_otSelected],L"*",NULL);
}

That concludes our tour of the LoaderView application. One last

thing before we go…

Loading from a Resource

You can embed audio objects directly in the executable. To do so,
compile them into the executable as resources and then use the
II
Loader’s ability to read objects from memory pointers.

Unit
As you probably know, all buttons, menus, dialogs, and other
user interface elements are stored in the program’s executable as
resources. You create resources in a resource editor that provides
tools for visually laying out the program’s user interface. The
resource editor is part of the development environment. The final
stage of program compilation binds the resource data into the exe-
cutable. The application makes standard calls to access and use the
resource data. The mechanism for storing and retrieving resource
elements is not limited to menus, icons, and their ilk. You can store
just about any data you want in a resource and have your application
access it when needed. This provides the convenience of binding all
data directly into the executable so there are no extra files to distrib-
ute with the program.
To demonstrate this, let’s use our familiar HelloWorld example
from the previous chapter and embed the wave as a binary resource.
Once it uses resources, our HelloWorld application can no longer be
a console app (a console application is a simple command-line pro-
gram that does not make any calls into the Windows libraries). Calls
to retrieve resources require the Windows APIs. The biggest change
in that regard is that main() is replaced with WinMain(). Then, to
insert the wave, use the resource editor’s option to import a binary
file as a resource (it actually has a specific option for wave files but
not for other DirectMusic media types).
170 n Chapter 9

To access the wave resource, use the system calls Find-

Resource() to locate it, LoadResource() to load it, and Lock-
Resource() to return a pointer to its memory.
Here is the source code:
#include <windows.h> // We need the Windows headers for Windows calls and structures.
#include "resource.h" // Resource IDs.
#include <dmusici.h>

int APIENTRY WinMain(HINSTANCE hInstance,

HINSTANCE hPrevInstance,
LPSTR lpCmdLine,
int nCmdShow)
{
// The performance manages all audio playback.
IDirectMusicPerformance8* pPerformance;
// The loader manages all file I/O.
IDirectMusicLoader8* pLoader = NULL;
// Each audio file is represented by a Segment.
IDirectMusicSegment8* pSegment = NULL;

// Initialize COM
CoInitialize(NULL);

// Create the loader. This is used to load audio objects from files.
CoCreateInstance(CLSID_DirectMusicLoader, NULL,
CLSCTX_INPROC, IID_IDirectMusicLoader8,
(void**)&pLoader);

// Create the performance. This manages the playback of sound and music.
CoCreateInstance(CLSID_DirectMusicPerformance, NULL,
CLSCTX_INPROC, IID_IDirectMusicPerformance8,
(void**)&pPerformance );

// Initialize the performance.

pPerformance->InitAudio(NULL,NULL,NULL,
DMUS_APATH_DYNAMIC_STEREO, // Default AudioPath type.
2, // Only two pchannels needed for this wave.
DMUS_AUDIOF_ALL,NULL);

// Okay, let's read the wave from an embedded resource...

DMUS_OBJECTDESC ObjDesc;

// Find the wave resource in the executable.

HRSRC hFound = FindResource(NULL,”IDR_WAVE”, RT_RCDATA);
// Force it to load into global memory.
HGLOBAL hRes = LoadResource(NULL, hFound);

ObjDesc.dwSize = sizeof(DMUS_OBJECTDESC);
 The Loader n 171

// Must let the loader know what kind of object this is.
ObjDesc.guidClass = CLSID_DirectMusicSegment;
// The only valid fields are the class ID and memory pointer.
ObjDesc.dwValidData = DMUS_OBJ_CLASS | DMUS_OBJ_MEMORY;
// Get the memory pointer from the resource.
ObjDesc.pbMemData = (BYTE *) LockResource(hRes);
// Get the memory length from the resource.
ObjDesc.llMemLength = SizeofResource(NULL, hFound);
// Now, read the Segment from the resource.
if (SUCCEEDED(pLoader->GetObject(
&ObjDesc, IID_IDirectMusicSegment8,
(void**) &pSegment )))
{
// Install the wave data in the synth.
pSegment->Download(pPerformance);

// Play the wave.

pPerformance->PlaySegmentEx(
II
pSegment, // Segment to play.

Unit
NULL,NULL,0,0,NULL,NULL,NULL);

// Wait eight seconds to let it play.

Sleep(8000);

// Unload wave data from the synth.

pSegment->Unload(pPerformance);

// Release the Segment from memory.

pSegment->Release();
}

// Done with the performance. Close it down, and then release it.
pPerformance->CloseDown();
pPerformance->Release();

// Same with the loader.

pLoader->Release();

// Bye bye COM.

CoUninitialize();
return 0;
}

This all works fine and dandy if the resource you are loading is com-
pletely self contained, which is the case of our example wave file.
However, if the Segment references other files, they too need to be
prepared as resources and handed to the Loader using SetObject()
prior to the call to GetObject(). Follow exactly the same steps to pre-
pare the referenced resources to initialize the resource and
172 n Chapter 9

descriptor as before, but call SetObject() instead of GetObject().

SetObject() just tells the Loader where the file object is placed in
memory so that the Loader can pull it in later when it needs to.
Loading from a resource is not the only way to load from mem-
ory. You can read anything you want into a chunk of memory and
then tell the Loader to read it. However, be very careful; after the
GetObject() call has completed, the Loader may continue to need
access to the memory. This is particularly true with waves and DLS
instruments that DirectMusic does not actually read until the appli-
cation instructs it to download a Segment that uses a wave or DLS
instrument from these files. If you call GetObject() with a memory
pointer and then release the memory, you could get a nasty crash
later when you call download on a Segment that references it. To be
safe, wait until all download calls have occurred and you are 100 per-
cent confident that there will be no more. Then, clear the Loader’s
reference with a call to SetObject() with the same DMUS_OBJECT-
DESC descriptor but with NULL in pbMemData.
Here is some sample code that plays a wave file that has been
placed in memory (*pbData). Notice how it calls SetObject() after
downloading the wave data to ensure that the data cannot be pulled
from memory later.
IDirectMusicPerformance8 *g_pPerformance;
IDirectMusicLoader *g_pLoader;

IDirectMusicSegment8 *PlayMemWave(BYTE *pbData,DWORD dwLength)

{
IDirectMusicSegment8 *pSegment = NULL;

DMUS_OBJECTDESC ObjDesc;

ObjDesc.dwSize = sizeof(DMUS_OBJECTDESC);
// Must let the loader know what kind of object this is.
ObjDesc.guidClass = CLSID_DirectMusicSegment;
// The only valid fields are the class ID and memory pointer.
ObjDesc.dwValidData = DMUS_OBJ_CLASS | DMUS_OBJ_MEMORY;
// Assign the memory pointer.
ObjDesc.pbMemData = pbData;
// Get the memory length from the resource.
ObjDesc.llMemLength = dwLength;
// Now, read the Segment from the resource.
if (SUCCEEDED(g_pLoader->GetObject(
&ObjDesc, IID_IDirectMusicSegment8,
 The Loader n 173

(void**) &pSegment )))

{
// At this point, the Segment data has been read, but any wave
// data is still pending reading from the file via the download
// command. If we free the memory before this next call, it
// will cause a crash.
pSegment->Download(g_pPerformance);
// Both DLS and wave downloads cause a copy of the memory to
// be made, so at this point we can free the resource memory.
// However, this does not apply to streamed waves, which always
// read from the source every time they play.
// To be safe, make sure the loader can't find the memory any more.
ObjDesc.pbMemData = NULL;
g_pLoader->SetObject(
&ObjDesc, IID_IDirectMusicSegment8,
(void**) &pSegment )))
// Play the wave.
g_pPerformance->PlaySegmentEx(
II
pSegment, // Segment to play.

Unit
NULL,NULL,0,0,NULL,NULL,NULL);
}
// Return the Segment. Be sure to unload and release it later.
return pSegment;
}

With a better understanding of how the Loader manages files, you

are better equipped to avoid some of the pitfalls and downright enig-
matic behavior that might test your wits. Although we covered a
great deal in this chapter, we still have not touched on the reverse
story — how you can replace the Loader with one of your own. For
some applications (in particular games), it can be useful to do this
because you can then completely control the delivery of game audio
resources and even compress or encrypt them in the process.
Although we do not cover it in this book, there is a great article and
programming example on MSDN. Go to http://msdn.microsoft.com
and search for “Custom Loading in DirectMusic.”
This page intentionally left blank.
 Chapter 10

Segments

I n the previous chapters, we explored the rudiments of playing

Segments because without playing Segments, you don’t have
sound! So, we’ve already skimmed the essentials: loading, down-
loading, playing, and stopping a Segment. Now it’s time to dig in and
really understand what makes a Segment tick because the Segment
is at the core of DirectMusic’s power.
Close your eyes, dear student, take a deep breath, and listen
very closely to what I am about to say: Use this knowledge judi-
ciously, and you will unlock the secrets of the musical universe. Use
it frivolously, and you shall unlock a plague of discordant cacophony!
Okay, maybe that’s overstating it a bit. Let’s just say Segments
are fun.
Then we’ll have some real fun writing a Segment-centric applica-
tion that explores some of the Segment’s more interesting features.

The Tao of Segment

Every sound effect or phrase of music that DirectX Audio plays
starts as a Segment. So far, we’ve only played Segments one at a
time. But that won’t do in a complex application environment. In
interactive media, things happen spontaneously, so you can’t have a
prerecorded (read: canned) audio or music score. The soundtrack
must be assembled in real time as the experience takes place. The
obvious first step toward such a solution is to have many little sound
and music snippets and trigger them as the user goes along. Indeed,
that is why Segments are called “Segments”; the intention is each
represents just a snippet, or Segment, of the entire sound effect or
music score.

175
176 n Chapter 10

Sound Effect Sound Effect

Voice-over

Music Music

Music Effect

Music Effect

Time

Figure 10-1: Multiple overlapping Segments combine to create the complete

score.

But, that’s not enough. In any interactive sound scenario, there are
requirements for reactivity, context, variability, and structure for
both music and sound effects. Reactivity means that the sounds or
music must be capable of responding to immediate actions and
changes in the story. Context means that new sounds or musical
phrases take into consideration the rest of the audio score, so they
reinforce each other, rather than clash. Variability means that sounds
and music don’t sound exactly the same every time, which would
quickly become unnatural. Structure means that sounds and espe-
cially music have different levels of inherent timelines.
Let’s look at sound effects first. By definition, sound effects are
reactive, driven by the unfolding story. When something happens, be
it a car crashing or a monster belching, the game immediately plays
the appropriate sound. But even then, greater sophistication is
highly desirable. The human ear is remarkably quick to identify a
perfectly repeated sound, and after the fourth belch it starts to sound
as if the monster swallowed a tape recorder instead of a deliciously
seasoned and sautéed princess. So, variation in the sound is impor-
tant. Context is important too. If the monster is sad, it would help to
reinforce the emotion in the sound, perhaps by bending the belch
down in pitch.
Music is significantly more challenging. In order to be effective
at reinforcing the user’s experience, a nonlinear score should track
the flow of the media that it supports. Yet, music has multiple layers
 Segments n 177

of structure, from song arrangement at the highest level to phrasing,

harmony, and melodic form. Randomly playing notes in response to
events in the media doesn’t cut it. But, neither does it work to play a
lengthy prerecorded score that seems to absentmindedly wander
outside of an interactive application with a storyline of its own, often
running at cross-purposes to the story. It’s as if the score was com-
posed by Mr. Magoo. Remember the cartoon character that could
barely see and would “blindly” misinterpret situations and forge
ahead on the wrong assumption to comic effect? Do you really want
your score composed by that guy?
Unfortunately, that is what a large percentage of scores for
games and other interactive applications seem like. Not surprisingly,
gamers typically turn the music off after a number of runs. Appar-
ently, the comedy of the ill-fitting score is lost on them!
II
(Pardon me for editorializing a bit here…)

Unit
What does this all mean? The Segment architecture must be rel-
atively sophisticated in order to support not just playing sounds and
music snippets in reaction to events in the application (i.e., game-
play) but also to play them in such a way that they still seem to have
structure, with variation and respect for context. While we are at it,
let’s add another requirement. The Segment mechanism must sup-
port different sound- and music-producing technologies, from MIDI
and DLS to wave playback, from style-based playback to straight
sequencing.
The DirectX Audio Segment architecture was designed to solve
all of these requirements. The Segment, which is manipulated via
the IDirectMusicSegment8 interface, is capable of handling a wide
range of sound-producing technologies, from individual waves to
Style playback. Multiple Segments can be scheduled for playback in
ways that allow their sounds and music to merge, overlap, align, and
even control each other.

Anatomy of a Segment
A Segment is little more than a simple structure that defines a set of
playback parameters, such as duration and loop points for the music
or sounds that it will play. But a Segment can contain any number of
Tracks, each representing a different technology. By technology, I
mean Tracks that play Styles, Tracks that set the tempo, and much
178 n Chapter 10

more. It is choice of Track types and data placed within them that
define a Segment.
If you are familiar with creating Segments in DirectMusic Pro-
ducer, you know that a Segment is constructed by choosing from a
wide palette of Tracks and adding them to the Segment. You know
that some Track types generate sound (Sequence, Style, Wave, etc.),
while others silently provide control information (Chord, Parameter,
Time Signature, etc.), and yet others actively control playback (Seg-
ment Trigger, Script). There are currently about a dozen Track types
included with DirectMusic.
I like to think of a Segment as a little box that manages a list of
Tracks, each represented by a long box indicating its list of actions
over time.

IDirectSegment8

Segment

Wave Track

Time Sig Track

Band Track

Sequence Track

Time

Figure 10-2: Segment with four Tracks.

The Segment in Figure 10-2 has a Wave Track that triggers one or
more waves to play across the duration of the Segment. It also has a
Time Signature track, which provides timing information. It has a
Band Track, which determines which DLS instruments to play and
their initial volume and pan, and it has a Sequence Track, which plays
a sequence of MIDI notes. So, this Segment plays a sequence of
waves as well as MIDI notes, and it provides time signature informa-
tion that the Performance can use to control playback of this
Segment. In a while, we learn that this can also control the playback
of other Segments.
 Segments n 179

You can see how a Segment is ultimately defined by the collec-

tion of Tracks that it contains, since they completely define its
behavior.
Each Track in a Segment is actually a COM object, represented
by the IDirectMusicTrack8 and IPersistStream interfaces. The
IPersistStream interface is used for loading the Track, while the
IDirectMusicTrack8 interface is used for performing it. In a later
chapter, we discuss how to create your own Track type.

How a Segment Loads

Tracks are created at the time a Segment is loaded from file. The
Segment file format stores each Track as a class ID for the Track
type followed by the data. When the Segment is loaded, it reads its II
small set of data and then reads the list of Tracks, each represented

Unit
by a class ID and associated data chunk. For each Track in the file, it
calls CoCreateInstance() to create the Track object. It then calls the
Track’s IPersistStream::Load() method and passes it the Track’s data
chunk in the form of the IStream at the current file position (see
Chapter 9 to better understand file I/O and the Loader). The Track
reads its data and passes control back to the Segment, which contin-
ues to the next Track, assembling a list of created and loaded Tracks
as it goes.

þ Note The Track types do not show up as cached object types in the
Loader. Because they are always embedded within the Segment, there is
no need for the Loader’s linking and caching mechanisms.

How a Segment Plays

Since playback of a Segment is really playback of the Tracks within
the Segment, we look to the Tracks themselves to understand how a
Segment performs. There are two ways that Tracks perform: by
generating playback events (mostly MIDI messages) for the Direct-
Music synth and by delivering control parameters that are read by
other Tracks. Examples of message-generating Tracks include
Sequence, Style, Band, Wave, and Pattern. Examples of control-
generating tracks include Groove Level, Chord/Key, Mute, Tempo,
and Time Signature.
180 n Chapter 10

Performance Channels
At this point, it’s probably a good idea to introduce the concept of
Performance channels, or pchannels. Since everything that the Per-
formance plays goes through a MIDI-style synthesizer, a lot of MIDI
conventions are used. One is the concept of a MIDI channel.
In the MIDI world, each synthesizer can have up to 16 channels.
Each channel represents a musical instrument. Up to 16 different
instruments can be played at the same time by assigning each to one
of the 16 channels. Every performance MIDI message includes the
address of the channel. So, the program change MIDI message tells
the channel which instrument to play. The MIDI Note On and Note
Off messages trigger playing notes using that instrument. More than
one note can play at a time on one channel. Additional MIDI mes-
sages adjust the volume, pan, pitch bend, and many other parameters
for all notes playing on the addressed channel.
There are some powerful things that you can do with this.
Because each message carries its channel address, you can inject
messages into the stream that modify the performance. For example,
you can adjust the volume or reverb of a MIDI performance by send-
ing down the appropriate MIDI control change messages without
touching the notes themselves.
This is all marvelous, but 16 MIDI channels is not enough.
There’s an historical reason that MIDI is limited to 16 channels.
When the MIDI specification was created back around 1982, it was
purely a hardware protocol designed to connect electronic musical
instruments together via a standard cable. To maximize bandwidth,
the number of channels was limited to four bits, or 16. At the time, it
was rare to find a musical device that could play more than one
instrument at a time, let alone 16, so it was reasonable. Twenty
years later, a 16-instrument limit is horribly inadequate.
DirectMusic’s Core Layer addresses this by introducing the con-
cept of channel groups. Each channel group is equivalent to one set
of 16 MIDI channels. In turn, the Performance supports unlimited
channels, called pchannels, and manages the mapping of these from
an AudioPath pchannel (each AudioPath has its own virtual channel
space) to a unique channel group + MIDI channel on the synth.
What this means is every Track in every Segment that plays on the
same AudioPath is sending commands to the synth on the same
 Segments n 181

shared set of pchannels. So, if a Band Track in a Segment sets piano

on pchannel 2, a separate Sequence Track with notes on pchannel 2
will play piano. Likewise, one Segment with a Band Track can con-
trol the instruments of a completely separate Segment on the same
AudioPath (as long as they both play on the same AudioPath, of
course).
In DirectMusic, everything that can make a sound is addressed
via pchannels. This means waves as well as the standard MIDI
notes. In all cases, the sound-generating Tracks create messages
(called pmsgs), which are time stamped and routed via pchannels,
ultimately to the synth where they make sound.
The good news is that all this is really set up in the content by
the audio designer working with DirectMusic Producer. There’s
really nothing the programmer needs to do, but it’s good to under-
II
stand what is going on.

Unit
Inter-Track Communication
While some Track types generate messages on pchannels to send to
the synth, others provide control information. Examples of control
information are time signatures, which are used to identify the next
beat and measure. Groove level is another example, which controls
the intensity level of Style playback Tracks.
As it turns out, control information is always provided by one
track to be used by another. Groove Level, Chord, and Mute Tracks,
for example, are used to determine the playback behavior of a Style
Track. At first glance, it might make sense to just put the groove,
chord, and mute information directly in the Style Track to keep
things simple. But, it turns out that by sharing this information with
multiple customer Tracks, some really powerful things can be
accomplished. For example, multiple Segments can play at the same
time and follow the same chord progression. More on how this
works in a bit…
So, we have a mechanism for cross-track communication. It’s
done via an API for requesting control information, and it’s called
GetParam(). GetParam() methods are provided on the IDirect-
MusicPerformance, IDirectMusicSegment, and IDirectMusicTrack
interfaces. When a Track needs to know a particular parameter, it
calls GetParam() on the Performance, which in turn identifies the
appropriate Segment and calls GetParam() on it. That, in turn, calls
182 n Chapter 10

GetParam() on the appropriate Track. Each command request is

identified by a unique GUID and associated data structure in which
the control data is returned.
The beauty of the GetParam() mechanism is that it works not
only across Tracks within one Segment but also across Tracks in
multiple Segments. This is very important because it allows control
to be shared in sophisticated ways that make the playing of multiple
Segments at once seem more like a carefully orchestrated perfor-
mance. For example, a Segment with a Pattern Track that is
designed to transpose on top of the current chord and key is played
as a musical embellishment. It uses chord and key information gen-
erated by a Chord Track in a separate Style playback Segment. So,
the embellishment seems to lock into the harmonic structure of the
Style Segment, and it all sounds as if it were carefully written to
work together.

Primary, Secondary, and Controlling Segments

So, how does the performance determine which Segment to read the
control information from? As it turns out, there are three levels of
playback priority for a Segment, set at the time the Segment is
played. They are primary, secondary, and controlling.

Primary Segment
A primary Segment is intended primarily for music. Only one pri-
mary Segment may be playing at a time, and it is considered the core
of the music performance. In addition to the music that it plays, the
primary Segment provides the underlying Tempo, Time Signature,
and, if applicable, Chord and Groove Level Tracks. By doing so, it
establishes these parameters for all music in all Segments. So, any
Playback Tracks within that same Segment retrieve their control
information from the neighboring Control Tracks as they play. No
two primary Segments can play at the same time because that would
cause the Control Tracks to conflict. If a primary Segment is playing
and a new one is scheduled to play, the currently playing Segment
stops exactly at the moment that the new Segment starts.
 Segments n 183

Controlling Segment
A controlling Segment has the unique ability to override the primary
Segment’s Control Tracks without interrupting its playback. For
example, a controlling Segment with an alternate chord progression
can be played over a primary Segment and all music shifts to the
replacement key and chord until the controlling Segment is stopped.
So, suddenly the tonal character of the music can make a shift in an
emotionally different direction. Figure 10-3 demonstrates how a con-
trolling Segment with an alternate chord progression overrides the
chords in two subsequent primary Segments.
Likewise, Tempo, Muting, and Groove Level can be overridden.
This is a very powerful way to alter the music in real time, perhaps
in response to stimuli from the host application (i.e., in response to II
gameplay). Like the primary Segment, the controlling Segment is

Unit
primarily for use with music, not sound effects.

Controlling Segment Chord Track

Em F#M

Primary Segment Chord Track Primary Segment Chord Track

CM7 Dm FM7 FM7 Dm GM

CM7 Dm Em F#M Dm GM

Figure 10-3: Underlying chord progression assembled from primary

and controlling Segments.

Secondary Segment
A secondary Segment also has sound-producing Tracks, but it is
devoid of Control Tracks. Importantly, any number of secondary Seg-
ments can be played at the same time. This is important for playing
sound effects as well as music. With sound effects, the usage is sim-
ple. Each secondary Segment contributes additional sounds on top of
the others playing in the same AudioPath. With music, it can be a
184 n Chapter 10

little more sophisticated. Because the secondary Segments retrieve

their control information from the primary and controlling Segments,
they automatically track any of the musical parameters that are
defined by the controlling Tracks. This is very powerful. In particu-
lar, the fact that they can track the chord and time signature means
that musical embellishments can play spontaneously and yet blend in
rhythmically and harmonically.
There is the special case of a self-controlling Segment. It is pos-
sible to create a Segment in DirectMusic Producer that will only
listen to its own Control Tracks, regardless of whether it is played as
a primary, secondary, or controlling Segment. This is useful if you
want to create a Segment that genuinely plays on its own and
ignores the Control Tracks of other Segments.
Whew! Enough talk — let’s do something!

Playing Segments
We already learned everything that we needed to know about loading
and downloading a Segment from the previous two chapters. But
there are a ton of options to explore in how we actually play, stop,
and control a running instance of a Segment. For that, we look at the
SegmentState object and the PlaySegmentEx() and StopEx() calls.

SegmentState
One very important feature of DirectMusic is the ability to play the
same Segment multiple times concurrently. This is useful for sound
effects. For example, you might want to use the same car engine
sound on several cars that are driving at the same time. This is also
very important with musical fragments when the score is built from
multiple Segments.
If the only way to adjust a parameter of the playing Segment or
stop its playback involved applying the operation to the original Seg-
ment object, things would be very bad since that would have to
represent all instances of the playing Segment. We need something
to represent each instance of the playing Segment. DirectMusic pro-
vides such a beast, called the SegmentState. You can request one of
these when you play the Segment and then use it to specifically stop
the Segment later, find out if it is still playing, and even access some
 Segments n 185

of its control parameters. It is represented by the IDirectMusic-

SegmentState interface. We use this to stop a playing Segment in
our programming example.

PlaySegmentEx()
Let’s take a close look at PlaySegmentEx(), the method you call to
actually play a Segment. We skip PlaySegment(), which is an older
version from DirectX 7.0, as it is pretty much the same as
PlaySegmentEx() with fewer options.
HRESULT hr = pPerformance->PlaySegmentEx(
IUnknown * pSource, // Segment to play.
WCHAR * pwzSegmentName, // Unused feature.
IUnknown * pTransition,
DWORD dwFlags,
// Optional transition Segment.
// Control flags.
II
__int64 i64StartTime, // Optional start time.

Unit
IDirectMusicSegmentState** ppSegmentState, // Optional Segment state to track
// playing Segment.
IUnknown * pFrom, // Optional Segment or AudioPath that stops when this starts.
IUnknown * pAudioPath // AudioPath to play this Segment on.
);

Typically, the three most important parameters are pSource,

dwFlags, and pAudioPath. Pass the Segment you want to play to
pSource, specify how you want it played with dwFlags, and deter-
mine which AudioPath it plays on with pAudioPath. (That is optional,
too. You can pass NULL for the default AudioPath.)
Let’s look at all the parameters in detail.
n IUnknown * pSource: This is always an IDirectMusicSegment
or IDirectMusicSegment8. Since IUnknown is a base class of
IDirectMusicSegment, it automatically works without any cast-
ing or calling QueryInterface(). You can just pass the IDirect-
MusicSegment interface, and it will work. So, why isn’t this just
IDirectMusicSegment? The API is designed to support both
IDirectMusicSegment and IDirectMusicSong. However,
IDirectMusicSong is not currently exposed. When Microsoft
eventually enables it, you will be in for a treat.
n WCHAR * pwzSegmentName: This is only an option when
using IDirectMusicSong, which is currently not enabled, so just
ignore for now.
186 n Chapter 10

n IUnknown * pTransition: The IUnknown interface pointer of a

Segment to use to transition to this Segment. The transition
Segment will play first, followed by the new Segment. The
DMUS_SEGF_AUTOTRANSITION flag must be set in dwFlags.
Also, if the transition Segment is using style technology and it
includes the proper ChordMap Track, the chord progression for
the transition is automatically composed to best fit between the
currently playing Segment and the one we are transitioning to.
These intelligent transitions make it much smoother when mov-
ing from one piece of music to another, resulting in a score that
magically stays cohesive even when the story changes dramati-
cally in unexpected ways.
n DWORD dwFlags: This carries the control flags for the new
Segment. These control how the Segment should be scheduled
in terms of timing and behavior. It’s worth taking some time to
explore these. To make things easier, our sample application,
Jones, lets you visually set these and hear what happens. All of
these flags are explained in detail later when we walk through
the source code.
n __int64 i64StartTime: This is the time to begin playing the
Segment. If 0, the Segment will play at the first opportunity, as
defined by dwFlags (there are options to align with the time sig-
nature in different ways, for example). By default, this is
assumed to be MUSIC_TIME units, which track the music time
of the performance. However, if clock-time units are required,
the DMUS_SEGF_REFTIME flag can be set and REFERENCE_
TIME units used. This allows scheduling the start of the Seg-
ment’s playback against a hard clock time, unaffected by the cur-
rent musical tempo. This is particularly useful for sound effects
that need to be scheduled.
n IDirectMusicSegmentState** ppSegmentState: If you
would like to track the playback of the Segment, pass the address
of an IDirectMusicSegmentState interface pointer here. It is
then filled in with an instance of an IDirectMusicSegmentState,
which can then be used to operate on the specific instance of the
playing Segment. This parameter can be NULL if no Segment-
State object is wanted.
 Segments n 187

n IUnknown * pFrom: Sometimes, you might want the new Seg-

ment to override an existing Segment, automatically stopping
that Segment’s playback at the exact time that the new one
starts. Although this is automatically done with primary Seg-
ments, it isn’t with controlling or secondary Segments, since
they can overlap. This also lets you replace a series of Segments
that are all playing on the same AudioPath with the new one.
Therefore, pFrom is the IUnknown interface pointer of the
SegmentState or AudioPath to stop when the new Segment
begins playing. If it is an AudioPath, all SegmentStates playing on
that AudioPath are stopped. This value can be NULL.
n IUnknown * pAudioPath: Every Segment must play on an
AudioPath. You can either provide a specific AudioPath object II
with this parameter or pass NULL, in which case the default
AudioPath is used.

Unit
StopEx()
Often, you need to stop the Segment at some point. Again, there are
two methods — the older Stop() and the newer StopEx(). Since
StopEx() has all the functionality of Stop(), we only look at StopEx().
You have four choices for what you stop:
n An individual instance of the playing Segment (the
SegmentState).
n All instances of the playing Segment.
n All Segments playing on a specific AudioPath.
n All Segments playing on the Performance.
HRESULT hr = m_pPerformance->StopEx(
IUnknown *pObjectToStop, // Segment, SegState, or AudioPath to stop.
__int64 i64StopTime, // Optional stop time.
DWORD dwFlags // Control flags.
);

n IUnknown * pObjectToStop: Pointer to the interface of the

object you’d like to stop. Since all COM objects are based on
IUnknown, the IDirectMusicSegmentState, IDirectMusic-
Segment, or IDirectMusicAudioPath interfaces all work.
• IDirectMusicSegmentState will stop the playing instance of the
Segment.
188 n Chapter 10

• IDirectMusicSegment will stop all instances of the Segment.

• IDirectMusicAudioPath will stop all Segments playing on the
AudioPath.
• NULL will stop all Segments playing.
n __int64 i64StopTime: This defines the time to stop. If 0, the
stop will occur at the first opportunity, as defined by dwFlags
(there are options to align with the time signature in different
ways, for example). By default, this is assumed to be MUSIC_
TIME units, which track the music time of the performance.
However, if clock-time units are required, the DMUS_SEGF_
REFTIME flag can be set and REFERENCE_TIME units used.
This allows scheduling the stop against a hard clock time, unaf-
fected by the current musical tempo.
n DWORD dwFlags: Control flags indicating when the stop
should occur. These are a subset of the play flags, primarily
focused on setting the timing grid. So, only DMUS_SEGF_
BEAT, DMUS_SEGF_DEFAULT, DMUS_SEGF_GRID,
DMUS_SEGF_MEASURE, DMUS_SEGF_REFTIME,
DMUS_SEGF_MARKER, and DMUS_SEGF_SEGMENTEND
are supported (more on these flags in a little bit).

Introducing Jones
This chapter’s programming example is the first installment of a pro-
gram named Jones. Jones starts by loading Segments and exploring
all the options and permutations for playing them (see Figure 10-4).
As we progress through the book, Jones grows to add functionality,
including scripting and AudioPaths, as we dig deeper into DirectX
Audio features.
 Segments n 189

II

Unit
Figure 10-4. Jones demonstrates loading, configuring,
and playing Segments.

Let’s do a quick walk-through of Jones to get acquainted and then

return to look at each operation in detail along with the underlying
code.

Starting Jones
You can run Jones by either going to the Unit II\10_Segments direc-
tory and building Jones or running SegmentJones.exe directly from
the Bin directory.

Loading a Segment
Use the Open… button to read a Segment into Jones. This opens a
File dialog, where you can choose a Segment, MIDI, or wave file to
load. For the purposes of this walk-through, try
StyleSegment.sgt from the Media directory.
The tree view displays the loaded Segment.
To the left of the Segment is a plus sign, indicat-
ing that it has children Segments. Go ahead and
click on it.
190 n Chapter 10

These are also Segments that you can play, and indeed they are
designed to be played at the same time as the main Segment. Where
did they come from? If the Segment references any Styles (which
StyleSegment.sgt does), then the Style’s Bands and motifs are con-
verted into Segments and displayed as additional Segments under
the Segment you loaded from a file. You can play any of these Seg-
ments. This feature in Jones quickly gets you closer to the inherent
interactivity of DirectMusic.
Styles typically include a series of Bands and motifs, all designed
to be played in conjunction with the main Style Segment but with
very different intentions. The Bands change the timbre set. In other
words, each provides a different set of instruments with a different
mix so that by changing Bands you can quickly make strong changes
to the music as it plays. The motifs are melodic fragments that are
designed with the Style in mind and layer effectively on top of it. By
playing the motifs, you add musical elements to the performance. So,
by displaying all of the Bands and motifs that come packaged with
the Style, Jones immediately gives you a quick way to start interac-
tively jamming and appreciating the layered and interactive approach
to delivering music.

Setting Segment Control Parameters

Click on a Segment and all of its control information is displayed on
the right, along with Play and Stop buttons.

Use these controls to set up how you want the currently selected
Segment to play. Click on the Play button to hear the Segment. You
can also double-click on the Segment’s name in the tree view to play
it.
 Segments n 191

Viewing the TimeLine

Once the Segment is playing, Jones displays it as a moving rectangle
in the timeline view underneath.

The portion of the Segment that has played or is in the process of

playing is painted blue, while the part that is still waiting to play is
red. II
The four vertical lines, from left to right, represent the current
time, the latency time, the queue time, and the prepare time. Each

Unit
represents a different stage in the process of playing a Segment. We
start at the current time and work backward through the sound pro-
duction process:
n Current time (black): This is the time at which you hear the
sound. Notice that at the very moment that a Segment rectangle
hits this, you start hearing it play. It would be simpler if we could
use just this for all our timing and ignore latency, queue, and pre-
pare times. But DirectMusic spreads out the different operations
both to increase efficiency (some operations can be done in large
chunks to reduce overhead) and to provide control opportunities
(for example, tools that can adjust the timing of notes in either
direction).
n Latency time (red): This is the time at which the DirectMusic
synth, which is implemented in software, actually does its work.
Depending on the hardware, this is anywhere from a few milli-
seconds to 70ms after the current time. The synth wakes up at
regular intervals, typically every ten or so milliseconds, and pro-
cesses another buffer of data to feed out the AudioPath into the
effects filters. However, because the MIDI events and waves are
all time stamped, the synth places the sounds appropriately in
the buffer at sample accurate resolution, even though it pro-
cesses the audio in relatively large lumps. Latency time is partic-
ularly important to understand because it represents the earliest
192 n Chapter 10

a sound can play. By default, this is used to calculate when to

play a new Segment.
n Queue time (purple): For increased efficiency, the Performance
Layer sends notes down in large chunks rather than individually.
This is particularly important when hardware acceleration is
involved because kernel transitions (i.e., calls down to the core
Windows OS) are expensive. So, the Performance queues up
MIDI and wave events and sends them down to the synth
roughly 50ms ahead of latency time. The starting notes are sent
down immediately, bypassing the queue time wait when a new
Segment plays. Just as latency time indicates the earliest a Seg-
ment can play, queue time indicates the earliest a Segment can
be stopped. Pmsgs in the Performance Layer can be yanked by
stop invalidations, but once they are sent down to the synth, they
are committed for good. The queue time can be adjusted with a
call to IDirectMusicPerformance::SetBumperLength().
n Prepare time (blue): This is the time at which the Segment
Tracks dump pmsgs into the queue. This is usually a full half-
second ahead of current time. This relatively large delay allows
the Tracks to be called infrequently, increasing efficiency. The
delay allows Tool processing to alter the time stamping of pmsgs
pretty dramatically. However, it doesn’t impose a restriction on
the start or stop time of a Segment, so it’s relatively harmless
even though the delay is relatively large. The delay duration can
be adjusted with a call to IDirectMusicPerformance::
SetPrepareTime().

Viewing Control Parameters

Below the timeline, the Tempo, Time Sig, Groove, and Chord param-
eters are displayed.

These four parameters are called control parameters and are

retrieved from the currently playing primary and controlling Seg-
ments. The Tracks in all playing Segments use these parameters to
determine their behavior (for example, transpose a note to follow the
current chord).
 Segments n 193

When you want to stop the Segment, make sure it is selected in

the tree view and then click on the Stop button or just let it play until
it is finished.

Jones Code
The source code for Jones can be split into two categories: UI and
DirectX Audio management. Since this book is about programming
DirectX Audio (not programming Windows with MFC), the source
code for Jones’ UI is not discussed. Extra effort has been made to
separate functionality so that just about everything we care about is
covered by the audio management code (starting with the CAudio
class), and there’s no need to discuss the UI. Of course, the UI code II
is included with the projects on the companion CD and it is docu-

Unit
mented, so you are welcome to use it in any way you’d like.

Audio Data Structures

Jones uses the same classes that we created for the Loader in the
previous chapter — CAudio and CSegment — though with some sig-
nificant changes and additions, which we will discuss.
Jones also uses an additional class: CSegState. CSegState keeps
track of one playing instance of a Segment via an IDirectMusic-
SegmentState8 interface pointer. This can be used to stop the
specific instance of the playing Segment, as well as perform other
operations on that Segment in real time. CSegState also tracks
CSegment for convenience.
Since we can have more than one instance of a Segment playing
at the same time, we manage the CSegStates with CSegStateList. To
facilitate this, CSegState and CSegStateList are based on the list
management classes CMyNode and CMyList, respectively.
class CSegState : public CMyNode
{
public:
// Constructor.
CSegState(CSegment *pSegment, IDirectMusicSegmentState8 *pSegState)
{
m_pSegState = pSegState;
pSegState->AddRef();
m_pSegment = pSegment;
194 n Chapter 10

}
~CSegState()
{
m_pSegState->Release();
}
// We keep a linked list of SegmentStates.
CSegState *GetNext() { return (CSegState *) CMyNode::GetNext(); };
// Access methods.
CSegment *GetSegment() { return m_pSegment; };
IDirectMusicSegmentState8 *GetSegState() { return m_pSegState; };
private:
CSegment * m_pSegment; // The Segment that this is playing.
IDirectMusicSegmentState8 * m_pSegState; // The DirectMusic SegmentState object.
};

/* CSegStateList
Based on CMyList, this manages a linked list of CSegStates.
*/

class CSegStateList : public CMyList

{
public:
// Overrides for CMyList methods.
CSegState * GetHead() { return (CSegState *) CMyList::GetHead(); };
CSegState *RemoveHead() { return (CSegState *) CMyList::RemoveHead(); };
// Clear list and release all references.
void Clear();
};

The CSegment class manages a loaded instance of a Segment. Like

CSegState, it is based on CMyList in order to manage a linked list of
CSegments.
Each CSegment can have a list of children CSegments, managed
by the CSegmentList class.
class CSegmentList : public CMyList
{
public:
// Overrides for CMyList methods.
CSegment *RemoveHead() { return (CSegment *) CMyList::RemoveHead(); };
// Clear list and release all references.
void Clear(IDirectMusicPerformance8 *pPerf);
};

// We create Segments from file as well as extract them from embedded Bands and motifs.
// We'll keep track of how a Segment was created for display purposes.

typedef enum _SEGMENT_SOURCE

{
 Segments n 195

SEGMENT_BAND = 1,
SEGMENT_MOTIF = 2,
SEGMENT_FILE =3
} SEGMENT_SOURCE;

/* CSegment
This manages one Segment.
Segments can also carry a list of children Segments.
*/

class CSegment : public CMyNode

{
public:
CSegment();
~CSegment();
CSegment *GetNext() { return (CSegment *) CMyNode::GetNext(); };
void Init(IDirectMusicSegment8 *pSegment,
IDirectMusicPerformance8 *pPerf,SEGMENT_SOURCE ssType);
II
void Clear(IDirectMusicPerformance8 *pPerf);

Unit
void SetFlags(DWORD dwFlags) { m_dwPlayFlags = dwFlags; };
DWORD GetFlags() { return m_dwPlayFlags; };
void GetName(char *pszBuffer);
SEGMENT_SOURCE GetType() { return m_ssType; };
CSegment * EnumChildren(DWORD dwIndex) {
return (CSegment *) m_ChildSegments.GetAtIndex(dwIndex); };
bool HasChildren() { return !m_ChildSegments.IsEmpty(); };
CSegment *GetTransition() { return m_pTransition; };
void SetTransition(CSegment * pTransition) { m_pTransition = pTransition; };
bool HasEmbeddedAudioPath();
IDirectMusicSegment8 * GetSegment() { return m_pSegment; };
private:
IDirectMusicSegment8 * m_pSegment; // The DirectMusic Segment that
// this manages.
DWORD m_dwPlayFlags; // DMUS_SEGF_ Playback flags.
DMUS_OBJECTDESC m_Desc; // DirectMusic object descriptor.
// Used by Loader.
CSegmentList m_ChildSegments; // Children Segments of this one.
CSegment * m_pTransition; // Transition Segment to
// autotransition to this one.
SEGMENT_SOURCE m_ssType; // Type of Segment (how it was
// created.)
};

CAudio is the top-level class. It initializes and manages the Direct-

Music Loader and Performance objects. It manages the loading and
playback of Segments (CSegment), also keeping track of their
SegmentStates (CSegState) as they play. CAudio provides methods
that track the current time signature, groove level, tempo, and chord
by calling GetParam() and returning the data in string format for
196 n Chapter 10

display. We take a look at these tracking methods a little later on.

For now, let’s take a look at CAudio.
class CAudio
{
public:
CAudio();
~CAudio();
HRESULT Init(); // Open the Performance, Loader, etc.
void Close(); // Shut down.
IDirectMusicPerformance8 * GetPerformance() { return m_pPerformance; };
IDirectMusicLoader8 * GetLoader() { return m_pLoader; };
CSegment *LoadSegment(WCHAR *pwzFileName); // Load a Segment from file.
void PlaySegment(CSegment *pSegment); // Play a Segment.
void StopSegment(CSegment *pSegment); // Stop a Segment.
void StopSegState(CSegState *pSegState); // Stop an individual instance of a
// playing Segment.
CSegment * EnumSegments(DWORD dwIndex) // Enumerate through all loaded
// Segments.
{ return (CSegment *) m_SegmentList.GetAtIndex(dwIndex); };
// Convenience functions that use GetParam to read current parameters and
// convert into strings for display.
bool GetTimeSig(char *pszText); // Current time signature.
bool GetGroove(char *pszText); // Current groove level (no
// embellishments, though.)
bool GetTempo(char *pszText); // Current tempo.
bool GetChord(char *pszText); // Current chord (doesn't bother
// with key.)
private:
IDirectMusicPerformance8 * m_pPerformance; // The DirectMusic Performance object.
IDirectMusicLoader8 * m_pLoader; // The DirectMusic Loader object.
CSegmentList m_SegmentList; // List of loaded Segments.
CSegStateList m_SegStateList; // List of playing Segments.
};

Now that we’ve covered the data structures, let’s walk through the
different parts of the program and examine the code.

Loading a Segment into Jones

For Jones, our CAudio and CSegment classes pick up more sophisti-
cated Segment loading than in the Loader implementation. In
particular, they now check for a Style playback Segment, and if so,
expose any children Band and Style motif Segments embedded
within the Style.
 Segments n 197

þ Note A Style motif, by the way, is a little Segment that is included

within a Style and is designed to play effectively on top of any Segment
that uses the Style. It uses the time signature and chord information to
lock on to the current rhythm and harmony, so it always sounds right,
no matter when it is triggered. It even uses instruments that are
configured as part of the Style design and placed in the Band Track of
the Style Segment.

Loading a Segment is handled by two methods: CAudio::LoadSeg-

ment() and CSegment::Init(). CAudio::LoadSegment() takes a file
path as a parameter, loads the Segment, and returns it wrapped in a
CSegment class. It then calls CSegment::Init(), which prepares for
playback and visualization. Finally, it checks if the Segment has a
Style in it, and if so, it spelunks for any embedded Band and motif
Segments, which it inserts as children of the main Segment. These II
show up as children nodes in the tree view and can be played inde-

Unit
pendently as secondary Segments to layer melodies on top (motifs)
and change the instruments (Bands).
Let’s look at the CAudio::LoadSegment() routine. CAudio::Load-
Segment() first takes the file name to be loaded and extracts the
path, which it hands to the Loader via a call to IDirectMusicLoader::
SetSearchDirectory(). This ensures that referenced files, such as
Styles, and DLS instrument collections will be easily found and con-
nected to the Segment. CAudio::LoadSegment() then loads the
Segment, calls Init() on the Segment, and places it in CAudio’s Seg-
ment list.
CSegment *CAudio::LoadSegment(WCHAR *pwzFileName)
{
wcscpy(m_wzSearchDirectory,pwzFileName);
WCHAR *pwzEnd = wcsrchr(m_wzSearchDirectory,'\\');
if (pwzEnd)
{
// If pwzFileName includes a directory path, use it to set up the search
// directory in the Loader.
// The Loader will look here for linked files, including Styles and DLS
// instruments.
*pwzEnd = 0;

m_pLoader->SetSearchDirectory(GUID_DirectMusicAllTypes,m_wzSearchDirectory,FALSE);
}
CSegment *pSegment = NULL;
IDirectMusicSegment8 *pISegment;
// Now, load the Segment.
if (SUCCEEDED(m_pLoader->LoadObjectFromFile(CLSID_DirectMusicSegment,
198 n Chapter 10

IID_IDirectMusicSegment8,
pwzFileName,
(void **) &pISegment)))
{
// Create a CSegment object to manage playback.
// This also recursively searches for embedded Band and Style motif
// Segments which can be played as secondary Segments.
pSegment = new CSegment(pISegment,m_pPerformance,SEGMENT_FILE);
if (pSegment)
{
m_SegmentList.AddTail(pSegment);
}
pISegment->Release();
}
return pSegment;
}

Next, CSegment::Init() prepares the Segment for playback and dis-

play. CSegment::Init() inserts a DisplayTrack Track in the Segment.
This will be used for visualization. It’s really an advanced topic (cre-
ating your own Track types), so it would be a bit of an unwelcome
tangent right now. If you are interested in how it’s done, take a look
at the source code, which is well documented.
Init() then downloads the Segment to the Performance. This
ensures that all waves and DLS instruments are primed in the synth
for playback.

þ Note Downloading of instruments is done with Bands, which are

pchannel specific. If the pchannel referred to by a Band isn’t in the
default AudioPath, the download will fail.

Next, Init() retrieves the Segment’s name via the IDirectMusic-

Object interface, so there is a friendly name for display. Finally, if this
is a Style playback Segment, it scans the Style for Bands and motifs
and installs them as children Segments in its CSegmentList field and
recursively calls this same Init() routine for each of them. That last
part is relatively complex, and it’s usually something you’d never
need to do. But, it’s a good learning exercise and, besides, we
needed it for Jones.
void CSegment::Init(IDirectMusicSegment8 *pISegment,
IDirectMusicPerformance8 *pPerf,
SEGMENT_SOURCE ssType)

{
 Segments n 199

// Create the DisplayTrack and insert it in the Segment.

CDisplayTrack *pDisplayTrack = new CDisplayTrack(this);
if (pDisplayTrack)
{
IDirectMusicTrack *pITrack = NULL;
pDisplayTrack->QueryInterface(IID_IDirectMusicTrack,(void **)&pITrack);
if (pITrack)
{
pISegment->InsertTrack(pITrack,-1);
}
}

// Assign the Segment and download it to the synth.

m_pSegment = pISegment;
pISegment->AddRef();
pISegment->Download(pPerf);

// Get the default playback flags.

II
pISegment->GetDefaultResolution(&m_dwPlayFlags);

Unit
// If this is a motif, assume secondary Segment and allow it to trigger
// to the time signature even when there is no primary Segment playing.
if (ssType == SEGMENT_MOTIF)
{
m_dwPlayFlags |= DMUS_SEGF_SECONDARY | DMUS_SEGF_TIMESIG_ALWAYS;
}
// If this is a band, just assume secondary.
else if (ssType == SEGMENT_BAND)
{
m_dwPlayFlags |= DMUS_SEGF_SECONDARY;
}
m_dwPlayFlags &= ~DMUS_SEGF_AUTOTRANSITION;

// Get the object descriptor from the Segment. This includes the name.
IDirectMusicObject *pIObject = NULL;
pISegment->QueryInterface(IID_IDirectMusicObject,(void **)&pIObject);
if (pIObject)
{
pIObject->GetDescriptor(&m_Desc);
pIObject->Release();
}
m_ssType = ssType;

// Now, the really fun part. We're going to see if this has any child Segments
// that would come with the Style, if this is a Style playback Segment.
// If so, we can get the motifs and Bands from the Styles and install them as
// Segments.
IDirectMusicStyle8 *pStyle = NULL;
if (SUCCEEDED(m_pSegment->GetParam(
GUID_IDirectMusicStyle,-1,0,0,NULL,(void *)&pStyle)))
{
200 n Chapter 10

// Paydirt! There's a Style in this Segment.

// First, enumerate through its Band lists.
WCHAR wszName[DMUS_MAX_NAME];
DWORD dwEnum;
for (dwEnum = 0;;dwEnum++)
{
// S_FALSE indicates end of the list.
if (pStyle->EnumBand(dwEnum,wszName) == S_OK)
{
// Use the name to retrive the Band.
IDirectMusicBand *pBand;
if (SUCCEEDED(pStyle->GetBand(wszName,&pBand)))
{
// The Band itself is not a Segment, but it has a method for
// creating a Segment.
IDirectMusicSegment8 *pIBandSeg;
if (SUCCEEDED(pBand->CreateSegment((IDirectMusicSegment
**)&pIBandSeg)))
{
// For visualization, give the Band Segments a measure of
// duration.
pIBandSeg->SetLength(768*4);
CSegment *pSegment = new CSegment;
if (pSegment)
{
pSegment->Init(pIBandSeg,pPerf,SEGMENT_BAND);
wcscpy(pSegment->m_Desc.wszName,wszName);
pSegment->m_Desc.dwValidData = DMUS_OBJ_NAME;
m_ChildSegments.AddHead(pSegment);
}
pIBandSeg->Release();
}
pBand->Release(); }
}
else
{
break;
}
}
// Now, enumerate through the Style's motifs.
for (dwEnum = 0;;dwEnum++)
{
if (pStyle->EnumMotif(dwEnum,wszName) == S_OK)
{
IDirectMusicSegment8 *pIMotif;
if (SUCCEEDED(pStyle->GetMotif(wszName,(IDirectMusicSegment
**)&pIMotif)))
{
CSegment *pSegment = new CSegment;
if (pSegment)
 Segments n 201

{
pSegment->Init(pIMotif,pPerf,SEGMENT_MOTIF);
wcscpy(pSegment->m_Desc.wszName,wszName);
pSegment->m_Desc.dwValidData = DMUS_OBJ_NAME;
m_ChildSegments.AddHead(pSegment);
}
pIMotif->Release();
}
}
else
{
break;
}
}
pStyle->Release();
}
} II
þ

Unit
Note If this seems like a lot to do just to get a Segment ready to play,
rest assured that your perception is correct and, no, you wouldn’t
normally need to do all these things. As we’ve already seen in previous
chapters, all you really need to do is load and then download your
Segment. So, don’t feel like memorizing these steps is a prerequisite to
DirectMusic guruhood. However, walking through all this extra fancy
stuff, especially extracting the Bands and motifs, helps to better grasp
the flexibility and inherent opportunities presented by the API.

Once a Segment is loaded, we can display it along with the other

Segments in the tree view. To facilitate this, CAudio provides a
method for enumerating through the Segments.
CSegment * EnumSegments(DWORD dwIndex) // Enumerate through all loaded Segments.
{ return (CSegment *) m_SegmentList.GetAtIndex(dwIndex); };

CSegment::EnumSegments() is based on the CMyList class

GetAtIndex() method, which takes a zero-based integer position in
the list and returns the item at that position or NULL if the end of
the list has been reached.
To display the tree, Jones calls EnumSegments() for the loaded
Segments and then, if there are children Segments to display
indented (motifs and Bands), calls EnumChildren() on each Segment.
EnumChildren() is also based on CMyList::GetAtIndex().
202 n Chapter 10

Segment Options
When you click on a Segment in the tree view, Jones displays every-
thing it knows about the Segment in the box to the right. Almost all
of this is the data from CSegment’s m_dwPlayFlags field, which is
used in the call to PlaySegmentEx(). The display breaks the flags
into mutually exclusive groupings. For example, a Segment can
either play as a primary, secondary, or controlling Segment, so the
two flags that set this are assigned as determined by the choice in
the Mode drop-down list. Notice that StyleSegment.sgt has been set
to play as a primary, whereas the motif and Band Styles underneath
it have all been set to secondary.

Let’s walk through the Segment options and discuss each selection
and how it controls the flags that are passed to PlaySegmentEx(),
starting with the Mode control.

Mode

n Primary: Primary is actually the default behavior. This mode

doesn’t set any flags in order to play a Segment as a primary
Segment.
Select StyleSegment.sgt and play it. Once it is playing,
click again. Notice that a new Segment appears in the timeline
view, and the first Segment eventually goes away, indicating that
the first ended abruptly so that the second could start. Also,
 Segments n 203

notice that where the second Segment starts, the first Segment
stays red, since it no longer plays beyond that point.

n Secondary: This sets the DMUS_SEGF_SECONDARY flag to

enable secondary mode.
With StyleSegment.sgt playing, double-click on one of the
motif or Band Segments to play it. Notice that an additional Seg-
ment starts playing, displayed below the primary Segment. II
Double-click several times, and you will hear several instances
of the same motif. Also, listen to how the motifs track the chord

Unit
changes in the primary Segment so that the motifs always sound
musically correct.

n Controlling: This sets both the DMUS_SEGF_CONTROL and

DMUS_SEGF_SECONDARY flags to indicate that a Segment
should be controlling. The DMUS_SEGF_SECONDARY flag
enables multiple overlapping Segments, which is required to
allow controlling Segments to play at the same time as the pri-
mary and other secondary Segments. DMUS_SEGF_CONTROL
ensures that the Segment’s Control Tracks override the primary
Segment’s Control Tracks.
Go to the Open… button and load ChordChange.sgt. Note
that this Segment loads as a primary Segment. Change its mode
to Controlling. Start StyleSegment.sgt playing, and then play
ChordChange.sgt on top of it. Notice that the music changes its
harmony because the chord progression in ChordChange.sgt
overrides the chord progression in StyleSegment.sgt. Click on
the Stop button to kill ChordChange.sgt, and notice that the
music reverts to the original chord progression.
204 n Chapter 10

Resolution
There is a whole series of options available for when you’d like the
new Segment to begin playing. For musical applications, it’s very
important that the new Segment line up in an appropriate manner.
Typically, you want the Segment to play as soon as possible, yet each
Segment has different criteria as to when it should rhythmically line
up. While one Segment might sound great synchronized on a beat
boundary, another might need to align with a measure in order to
work well. Select this with the Resolution control:

As we walk through each option, experiment with playing both pri-

mary and secondary Segments with each resolution.
n Default: The Segment has a default timing resolution authored
by the content creator directly into it. This menu option sets the
DMUS_SEGF_DEFAULT flag, indicating that whatever was
authored into the Segment should be used. Jones calls the
IDirectMusicSegment::GetDefaultResolution() method to find
out what this is, so you can see it in the display and change it to
something else.
n None: Don’t set any of the resolution flags if you want the Seg-
ment to play as soon as possible. This is useful for sound effects.
Note that if there is no primary or controlling Segment currently
playing, a Segment will play immediately anyway, so you don’t
need to write a special case for starting the Segment instantly
under these circumstances (no primary or controlling Segment).
However, if you do care to synchronize to a “silent” time signa-
ture, set the DMUS_SEGF_TIMESIG_ALWAYS flag.
n Grid: The smallest music resolution is the grid. DirectMusic’s
time signature is made up of three components: BPM (beats per
measure), beat size, and grid. The first two should be familiar to
you. For example, a time signature of 3/4 means there are three
beats to the measure and the beat is a quarter note. This works
 Segments n 205

fine, but it doesn’t allow for a sense of sub-beat resolution, which

can be pretty important. If you want a piece to have a “triplet
feel,” you want to subdivide the beat into threes, rather than
twos or fours. So, the grid resolution, which is authored into the
time signature in DirectMusic Producer, indicates the lowest-
level timing resolution.
When you choose Grid, Jones sets the DMUS_SEGF_GRID
flag and the Segment plays at the next available grid time. The
Segment is still subtly tied into the feel of the music. Be mindful
to keep the Segment simple, or it could conflict with the rhythm
of the other Segments when it is out of phase with their beat
timing.
n Beat: This sets the start time to line up with the next beat. This II
is typically the most useful for layering secondary Segments. It
can also come in handy when a sudden change is required and

Unit
it’s too long to wait for the end of the measure to start a change
in the music. Set this with the DMUS_SEGF_BEAT flag.
n Measure: This sets the start time to line up with the next mea-
sure, or bar. This is typically the most useful for transitioning
between Segments because it keeps the musical meter intact.
Some secondary Segments, which have relatively long and com-
plicated phrasing, might require alignment to the measure to
sound right. Set this with the DMUS_SEGF_MEASURE flag.
n Marker: This sets the start time to line up with a specific
marker placed in a Marker Track in the currently playing primary
Segment. This is useful in situations where transitions on mea-
sure boundaries are not acceptable because they might break up
the musical phrases. Since markers can be placed anywhere,
regardless of the time signature, they can be used in different
ways. For example, markers can be used to set trigger points for
secondary Segments on specific beats. Set this with the DMUS_
SEGF_MARKER flag.
n Segment End: You can set a Segment to start playback when
the current primary Segment finishes. This is useful in two
ways: Segment End can be used to schedule a Segment to play
when the current one finishes or it can be used in conjunction
with the alignment flags (see the section titled “Aligned Cut In”)
206 n Chapter 10

to switch between two Segments that start at the same time. Set
this with the DMUS_SEGF_SEGMENTEND flag.
n End of Queue: We have one final option that looks remarkably
similar to Segment End but is useful in a different way. If you’d
like to queue up a series of Segments to play one after the other,
play them with the DMUS_SEGF_QUEUE flag. This only
applies to primary Segments. This is very useful if you have a
series of Segments that make up the music for a scene and you’d
like to just queue them all up at the beginning and then forget
about it. You can even set up a song arrangement with repeats of
the same Segments in different places. DirectMusic will play the
Segments back in the order in which you queued them with the
PlaySegmentEx() call. However, you can break out of the queue
as soon as you need to. Play a primary Segment that does not
have the DMUS_SEGF_QUEUE flag set, and DirectMusic
immediately flushes all Segments queued in the future and
replaces them with the new Segment.

þ Note It is also possible to specify exactly when a Segment should

play. Usually, this isn’t necessary because DirectMusic’s automated
mechanisms for finding the best time to start a Segment work so well.
However, you can specify an exact time in music- or clock-time units via
the i64StartTime parameter to PlaySegmentEx(). By default, the value
placed in i64StartTime is in MUSIC_TIME units, indicating that it tracks
the musical tempo. However, you can put an absolute REFERENCE_
TIME value in i64StartTime and set the DMUS_SEGF_REFTIME flag.

Delay

You can specify the earliest time to use to calculate the timing reso-
lution for when a Segment should begin playing. Choose a Segment
and, as you read about each delay option, try it out. Watch the
timeline display to see the different delays.
 Segments n 207

n Optimal: By default, DirectMusic looks at the playback mode

and determines which latency to apply. If the Segment is primary
or controlling, and there currently are Segments playing, it sets
the latency to Queue Time. Otherwise, it opts for the fastest
choice, which is Synth Latency.
n Synth Latency: This is the default start time for secondary
Segments, since it is the earliest possible time the Segment can
play. Since secondary Segments just add new notes but don’t
affect any currently playing Segments in any way, it’s best to play
as soon as possible. This sets the DMUS_SEGF_AFTER-
LATENCYTIME flag. Additionally, if a primary Segment starts
when nothing is playing, it plays at Synth Latency.
n Queue Time: This sets the DMUS_SEGF_AFTERQUEUE- II
TIME flag, indicating that the Segment must start playback at

Unit
some point after the Queue Time. Queue Time is important
because it is the last chance to invalidate (turn off) pmsgs just
before they go down to the synth. This is useful if the start of the
new Segment causes invalidation of another Segment, either
because as a primary Segment it overrides the previously play-
ing primary Segment or as a controlling Segment it requires a
regeneration of notes in currently playing Segments. This is the
default timing for primary and controlling Segments.
n Prepare Time: This option sets the DMUS_SEGF_AFTER-
PREPARETIME flag, indicating that the new Segment must not
start playing until after Prepare Time. Although setting Prepare
Time incurs greater delay, it can be useful in several situations:
• If the invalidation caused by starting after Queue Time is
unsatisfactory. Sometimes, invalidations caused by controlling
Segments or new primary Segments chop off and restart notes
in ways that don’t work.
• When a Segment has a streaming wave, it needs more time to
preload the start of the wave if downloading is disabled for that
wave (which dramatically reduces memory overhead) or if it
picks up partway through the Segment.
208 n Chapter 10

Time Signature

This sets the DMUS_SEGF_TIMESIG_ALWAYS flag, indicating that

even if there is no primary or controlling Segment currently playing,
the time signature from the last played primary Segment holds, and
the start time should synchronize with it, depending, of course, on
the resolution flags.
The easiest way to understand the Time Signature (DMUS_
SEGF_TIMESIG_ALWAYS) option is to play with it a bit. If there are
any Segments currently playing, turn them off. Select a motif Seg-
ment from under StyleSegment.sgt. With Time Signature turned off,
click on the Play button several times to get multiple instances of
the Segment playing. Notice that they play immediately but com-
pletely out of sync with each other. Now, turn on the check box and
try again. This time, the Segments start a little later, as they line up
with a silent time signature, but they play in lockstep.

Transition

Sometimes, when there is a dramatic change in the state of the host

application (say, in the story of an interactive game), the music
should be able to respond by changing in a dramatic way. If you are
going to suddenly play something significantly different, it helps if
there is a short transition Segment that can bridge between the pre-
vious and new musical parts. Using DirectMusic Producer, it is
possible to create a transition Segment that can bridge between a
random point in the predecessor Segment and the start of a new
Segment, often picking up elements from both of them on the fly. Set
the DMUS_SEGF_AUTOTRANSITION flag and place the transition
Segment in the pTransition parameter of PlaySegmentEx() to use
the transition Segment. Jones accomplishes this by letting you
choose a template Segment from the list of already loaded Segments,
which it displays in the Transition drop-down list.
 Segments n 209

Aligned Cut In
Sometimes it would be nice to start playback of a Segment immedi-
ately, yet keep it rhythmically aligned with a larger resolution. For
example, you might have a motif that adds a musical layer in
response to some user action that needs to align to the measure but
needs to be heard immediately. Or you’d like to transition to a new
primary Segment immediately, yet keep the meter aligned.
In order to use this feature, you need to do two things:
1. Set the timing resolution for the logical start of the new Seg-
ment. In other words, set the timing resolution as if the new
Segment played from the start. To do this, use one of the
resolution flags that we have already discussed. This ensures
that the new Segment will still line up rhythmically, even II
though it may cut in anywhere in the Segment.

Unit
2. Set the resolution at which the cut-in may occur. This should
be a smaller size than the resolution flag. Why? Resolution
ensures that the two Segments line up rhythmically, allowing
a cut-in to occur at a finer resolution (in other words sooner),
which is the whole purpose of this feature.
The Aligned Cut In menu enables this feature and provides choices
for the timing of the cut.

A typical example is to set Resolution to Measure and Aligned Cut In

to Beat. Instead of waiting for the next measure, the Segment looks
back in time and aligns its start with the last measure boundary and
initiates playback by cutting over on the next beat. So you get the
desired rhythmic effect of measure resolution without waiting for the
next measure. The Aligned Cut In option provides a way to deter-
mine what’s a safe place to make that cut, so it’s not just at an
arbitrary place. Let’s look at each option:
210 n Chapter 10

n Disabled: This option simply doesn’t activate Aligned Cut In, so

none of the alignment flags are set. In order to enable Aligned
Cut In, DMUS_SEGF_ALIGN must be set along with an optional
cut-in resolution flag. The remaining options select each of
these.
n Only on Marker: This is the default behavior because it pro-
vides the most sophisticated control of the cut-in. Cut-in mark-
ers can be placed in the Segment when authored in DirectMusic
Producer. These markers identify ideal points to switch to the
new Segment. Just the DMUS_SEGF_ALIGN flag is set to
enable this behavior. However, there can be circumstances
where there are no markers and a fallback resolution is desired.
These options follow. Understand, though, that if there is a
marker placed in the Segment, it will always override any of
these. Often, though, these options work well, and there is no
need for specific markers.
n Anywhere: This option enables the fallback cut to occur at
music-time resolution (in other words, as soon as possible). It
sets the DMUS_SEGF_VALID_START_TICK flag along with
DMUS_SEGF_ALIGN.
n On Grid: This option enables the fallback cut to occur on a grid
boundary. It sets the DMUS_SEGF_VALID_START_GRID flag
along with DMUS_SEGF_ALIGN.
n On Beat: This option enables the cut to occur on a beat bound-
ary. It sets the DMUS_SEGF_VALID_START_BEAT flag along
with DMUS_SEGF_ALIGN.
n On Measure: Enables the cut to occur on a measure boundary.
It sets the DMUS_SEGF_VALID_START_MEASURE flag along
with DMUS_SEGF_ALIGN.
To try out the different invalidations, once again use StyleSeg-
ment.sgt. This has some markers embedded within it. Play it as a
primary Segment, wait a little, and then click on Play again. Notice
that it picks up on a measure boundary (assuming you didn’t change
the resolution) but starts at the beginning. Now, change Resolution
to Segment End and set Aligned Cut In to Only on Marker. Again
play the Segment and then click on Play a second time. This time,
notice that the new Segment starts playing but it picks up at a
 Segments n 211

predefined point partway into it. You know it starts out partway
because the start of the Segment is green, indicating it was never
played. Also note that the start of the new Segment is all the way off
the left side of the screen because it has aligned with the start of the
first instance of it playing.

Next, experiment by playing motifs on top of a primary Segment. For

the motifs, set Resolution to Measure and Aligned Cut In to On II
Grid. The motifs rhythmically align with the measure but start play-

Unit
ing partway through.

Invalidation
By default, when a primary or controlling Segment plays, it causes
an invalidation of all currently playing Segments. This means that all
notes that were generated at prepare time but not yet committed to
the synth at queue time are flushed, and the Segment is told to
regenerate the notes. Why? Because a change in primary or control-
ling Segment usually means that something changed, and new notes
need to be generated to reflect that. For example, if the groove level
or chord changes, then previously generated notes are probably
wrong, causing cacophony. There are times when you don’t want
invalidation. Invalidation can cause an audible glitch when there are
sustained sounds, be they DLS instruments or straight waves.
Because of this, the introduction of a controlling or primary Segment
can sometimes have an unfortunate effect on other aspects of the
sound environment, sound effects in particular, which couldn’t care
less what the underlying chord change or groove level is.
There are several ways to deal with the various problems caused
by invalidation:
n Set the new Segment to play after prepare time, in which case
there are no invalidations. If the delay is painful, you can adjust it
downward with a call to SetPrepareTime(), but keep an ear out
for timing glitches. Numbers in the 200 to 300ms range, which
212 n Chapter 10

effectively cut it in half, should be safe. If this solution works for

the music, it is usually the best because it is simple and avoids
the overhead of invalidations anyway.
n Create a separate Performance for all sound effects so the two
worlds simply don’t stomp on each other. This is a perfectly rea-
sonable solution and does not incur much additional overhead,
especially since the AudioPaths used are typically different
anyway.
n Use one of two flags that reduce the scope of invalidations.
These are represented in the Invalidation menu.

• All Segments: This is the default behavior. It leaves all invali-

dation intact.
• Only Primary: This sets the
DMUS_SEGF_INVALIDATE_PRI flag. Only the notes in the
currently playing primary Segment are invalidated. All other
Segments are left alone.
• No Segments: This sets the DMUS_SEGF_NOINVALIDATE,
turning off all invalidation. For some circumstances, this is a
perfectly reasonable solution.

Use Path
A Segment can be authored with an embedded AudioPath. However,
that AudioPath is not automatically created and invoked each time
the Segment plays. The reason is simple: Creating multiple Audio-
Paths can be very CPU intensive, especially if they invoke lots of
audio effects. If you are going to play the same Segment over and
over again or if you intend to share that AudioPath with multiple
Segments, then it makes more sense to directly create the
AudioPath and manage its usage explicitly. But there are situations
where you would like an embedded AudioPath to be automatically
used, so there’s a flag to enable that. When a Segment has its
AudioPath embedded within itself, it carries everything necessary to
perform. There is no need to externally match up the Segment with
an appropriate AudioPath that may happen to have filters, etc., set
 Segments n 213

specifically for the Tracks in the Segment. A DirectMusic Media

Player application would benefit from embedded Segments, for
example.

When a Segment has an embedded AudioPath configuration, this

option is enabled. It sets the DMUS_SEGF_USE_AUDIOPATH flag.

Setting Segment Options

The CSegment class stores and retrieves all the information that you
select in the display, with methods for setting and getting data.
All of the Segment flags are stored in one variable,
m_dwPlayFlags, accessed via calls to SetFlags() and GetFlags().
II
Note that this is initially filled with the default flags that were placed

Unit
in the Segment file at author time.
The DirectMusic Segment file format does not have a way to
define transition Segments, but the PlaySegmentEx() call does take
such a parameter. So, CSegment has a field, m_pTransition, which is
a pointer to a CSegment to be played as a transition into the parent
CSegment. m_pTransition is managed via the SetTransition() and
GetTransition() calls, which simply access the variable.
To display the name of the Segment, we have a slightly more
complex routine that retrieves the name from the DMUS_OBJECT-
DESC descriptor that was retrieved at the time the file was loaded
and converts from Unicode to ASCII. In some cases, the Segment
may not have a name because none was authored into it (typically
the case with MIDI and wave files). If so, it uses the file name. If all
else fails, it returns a descriptive error message based on the type of
Segment.
void CSegment::GetName(char *pszBuffer)
{
pszBuffer[0] = 0;
if (m_Desc.dwValidData & DMUS_OBJ_NAME)
{
// Convert from Unicode to ASCII.
wcstombs( pszBuffer, m_Desc.wszName, DMUS_MAX_NAME );
}
else if (m_Desc.dwValidData & DMUS_OBJ_FILENAME)
{
wcstombs( pszBuffer, m_Desc.wszFileName, DMUS_MAX_NAME );
214 n Chapter 10

// Get rid of any file path.

char *pszName = strrchr( pszBuffer,'\\');
if (pszName) strcpy(pszBuffer,++pszName);
}
else
{
static char * s_pszNames[3] = { "Band", "Motif", "File" };
wsprintf( pszBuffer,"%s (no name)",s_pszNames[m_ssType – 1]);
}
}

In order to enable the Use Path check box, we have to find out
whether the Segment actually has an AudioPath configuration
embedded within it. The only way to do this is to actually call the
DirectMusic API to retrieve the configuration and return true if one
exists.
bool CSegment::HasEmbeddedAudioPath()
{
IUnknown *pConfig;
if (SUCCEEDED(m_pSegment->GetAudioPathConfig(&pConfig)))
{
pConfig->Release();
return true;
}
return false;
}

That covers all the routines for managing the Segment’s display data.
Now, let’s actually play the dang thing.
Segment playback is handled by the CAudio::PlaySegment()
method. This is relatively simple because we’re just retrieving the
parameters we need, calling DirectMusic’s PlaySegmentEx() rou-
tine, and then storing the returned IDirectMusicSegmentState.
First, check to see if there is a transition Segment. If so, get the
IDirectMusicSegment interface and set the DMUS_SEGF_
AUTOTRANSITION flag. Then, call PlaySegmentEx() and pass it
the transition Segment as well as an empty SegmentState interface
(IDirectMusicSegmentState). If the call succeeded, create a
CSegState object to track both the SegmentState and Segment that
spawned it. This will be used for tracking and eventually stopping it
later.
void CAudio::PlaySegment(CSegment *pSegment)
{
if (pSegment)
 Segments n 215

{
// Is there a transition Segment?
IDirectMusicSegment8 *pTransition = NULL;
if (pSegment->GetTransition())
{
pTransition = pSegment->GetTransition()->GetSegment();
}
DWORD dwFlags = pSegment->GetFlags();
if (pTransition)
{
dwFlags |= DMUS_SEGF_AUTOTRANSITION;
}
IDirectMusicSegmentState8 *pISegState;
if (SUCCEEDED(m_pPerformance->PlaySegmentEx(
pSegment->GetSegment(), // Returns IDirectMusicSegment
NULL,pTransition, // Use the transition, if it exists.
dwFlags,
0,
// Playback flags.
// No time stamp.
II
(IDirectMusicSegmentState **) &pISegState, // Returned SegState.

Unit
NULL, // No prior Segment to stop.
NULL))) // No AudioPath, just use default.
{
CSegState *pSegState = new CSegState(pSegment,pISegState);
if (pSegState)
{
m_SegStateList.AddHead(pSegState);
}
pISegState->Release();
}
}
}

To stop the Segment, CAudio has two methods. You can stop an indi-
vidual instance of a playing Segment with CAudio::StopSegState(), or
you can stop all playing instances of the Segment with a call to
CAudio::StopSegment(). StopSegState() stops just the individual
instance of a playing Segment as managed by the CSegState object.
It uses the same timing resolution flags that were used to play the
Segment. For example, if a Segment has been set to play on a mea-
sure boundary, it will also stop on a measure boundary. Only some of
the play flags are legal for Stop. These are defined by STOP_FLAGS.
#define STOP_FLAGS (DMUS_SEGF_BEAT | DMUS_SEGF_DEFAULT | DMUS_SEGF_GRID | \
DMUS_SEGF_MEASURE | DMUS_SEGF_REFTIME | DMUS_SEGF_MARKER)

Note that once the Segment is stopped, CSegState is no longer use-

ful, so the caller can and should delete it, which then releases the
216 n Chapter 10

IDirectMusicSegmentState. This is automatically done for you if you

use CAudio::StopSegment().
void CAudio::StopSegState(CSegState *pSegState)
{
if (pSegState)
{
m_pPerformance->StopEx(pSegState->GetSegState(),0,
pSegState->GetSegment()->GetFlags() & STOP_FLAGS);
}
}

CAudio::StopSegment stops all instances of a currently playing Seg-

ment. This could be accomplished by passing the IDirectMusic-
Segment interface directly to StopEx(). However, since we are
tracking all of the SegmentStates via the list of CSegStates, we
should use them so that we can release their memory at the same
time. Scan through the list of all CSegStates and for each that
matches the passed CSegment, call StopSegState(). Then, remove
from the list and delete. The destructor for CSegState releases the
IDirectMusicSegmentState interface, allowing the object to be
reclaimed by DirectMusic.
void CAudio::StopSegment(CSegment *pSegment)
{
if (pSegment)
{
CSegState *pNext;
CSegState *pSegState = m_SegStateList.GetHead();
for (;pSegState;pSegState = pNext)
{
// Get the next pointer now, since pSegState
// may be pulled from the list if it matches.
pNext = pSegState->GetNext();
// Is this from the requested Segment?
if (pSegState->GetSegment() == pSegment)
{
// Stop it.
StopSegState(pSegState);
// Free the SegState.
m_SegStateList.Remove(pSegState);
delete pSegState;
}
}
}
}
 Segments n 217

Tracking Control Parameters

As we discussed earlier, a big piece of DirectMusic’s musical power
comes from its ability to share musical control information across
multiple playing Segments. You’ve heard what happens when you
experiment with playing different combinations of controlling, pri-
mary, and secondary Segments. You can also see it. Across the
bottom of Jones is a display of four control parameters: Tempo, Time
Sig, Groove, and Chord.

At any point in time, each control parameter is generated by the cur-

rent primary Segment or a controlling Segment that is overriding it. II
To help visualize what is going on with the control parameters as
well as demonstrate code to access these, CAudio() includes four

Unit
methods for retrieving each of the parameters via IDirectMusic-
Performance::GetParam() and converting the data into text strings
for display.
CAudio::GetTimeSig() finds out what the current time signature
is and writes it in a string. It calls IDirectMusicPerformance::Get-
Param with the GUID_TimeSignature command and DMUS_
TIMESIGNATURE structure to retrieve the time signature from
whatever Segment is generating it.

þ Note Even when no Segments are playing, there is still a time

signature. Along with tempo, DirectMusic keeps track of the last played
time signature as a special case because it is necessary for scheduling
new Segments.

bool CAudio::GetTimeSig(char *pszText)

{
if (m_pPerformance)
{
// Since we can get parameters across a broad range of time,
// we need to get the current play time in music format for our request.
MUSIC_TIME mtTime;
m_pPerformance->GetTime(NULL,&mtTime);
// Use GUID_TimeSignature command and DMUS_TIMESIGNATURE structure
DMUS_TIMESIGNATURE TimeSig;
if (SUCCEEDED(m_pPerformance->GetParam(GUID_TimeSignature,
-1,0,mtTime,NULL,(void *) &TimeSig)))
{
wsprintf(pszText,"%ld/%ld:%ld",(long)TimeSig.bBeatsPerMeasure,
218 n Chapter 10

(long)TimeSig.bBeat,(long)TimeSig.wGridsPerBeat);
}
else
{
strcpy(pszText,"None");
}
return true;
}
return false;
}

CAudio::GetGroove() follows a similar format. It finds out what the

current groove level is and writes that groove level into a string.
CAudio::GetGroove() calls GetParam() with the GUID_Command-
Param and DMUS_COMMAND_PARAM structure to retrieve the
groove level information from whatever Segment is generating it.
Note that the DMUS_COMMAND_PARAM also includes embel-
lishment information, such as DMUS_COMMANDT_FILL and
DMUS_COMMANDT_BREAK, but GetGroove() ignores these (left
as an exercise for the reader!).
bool CAudio::GetGroove(char *pszText)
{
if (m_pPerformance)
{
// Since we can get parameters across a broad range of time,
// we need to get the current play time in music format for our request.
MUSIC_TIME mtTime;
m_pPerformance->GetTime(NULL,&mtTime);
// Use GUID_CommandParam command and DMUS_COMMAND_PARAM structure
DMUS_COMMAND_PARAM Groove;
if (SUCCEEDED(m_pPerformance->GetParam(GUID_CommandParam,
-1,0,mtTime,NULL,(void *) &Groove)))
{
// Groove level is a number between 1 and 100.
wsprintf(pszText,"%ld",(long)Groove.bGrooveLevel);
}
else
{
strcpy(pszText,"None");
}
return true;
}
return false;
}
 Segments n 219

CAudio::GetTempo() retrieves the current tempo and writes it in a

string. It calls GetParam() with the GUID_TempoParam command
and the DMUS_TEMPO_PARAM structure to retrieve the tempo
from whatever Segment is generating it. Note that even when no
Segments are playing, there still is a tempo. As with time signature,
DirectMusic treats this as a special case because it is necessary for
subsequent Segment scheduling.
bool CAudio::GetTempo(char *pszText)
{
if (m_pPerformance)
{
// Since we can get parameters across a broad range of time,
// we need to get the current play time in music format for our request.
MUSIC_TIME mtTime;
m_pPerformance->GetTime(NULL,&mtTime);
II
// Use GUID_TempoParam command and DMUS_TEMPO_PARAM structure

Unit
DMUS_TEMPO_PARAM Tempo;
if (SUCCEEDED(m_pPerformance->GetParam(GUID_TempoParam,
-1,0,mtTime,NULL,(void *) &Tempo)))
{
// Tempo is a floating point number.
sprintf(pszText,"%3.2f",Tempo.dblTempo);
}
else
{
strcpy(pszText,"None");
}
return true;
}
return false;
}

Finally, CAudio::GetChord() finds the current chord and writes it in a

string. It calls GetParam() with GUID_ChordParam and the DMUS_
CHORD_KEY structure. This actually returns both chord and key
information; if you take a look at DMUS_CHORD_PARAM, it’s quite
complex. There’s a lot of rich harmonic information stored in
DMUS_CHORD_KEY and the array of DMUS_SUBCHORD struc-
tures, allowing multiple parallel chords and much more.
typedef struct _DMUS_SUBCHORD
{
DWORD dwChordPattern; /* Notes in the subchord */
DWORD dwScalePattern; /* Notes in the scale */
DWORD dwInversionPoints; /* Where inversions can occur */
DWORD dwLevels; /* Which levels are supported by this subchord */
220 n Chapter 10

BYTE bChordRoot; /* Root of the subchord */

BYTE bScaleRoot; /* Root of the scale */
} DMUS_SUBCHORD;

typedef struct _DMUS_CHORD_KEY

{
WCHAR wszName[16]; /* Name of the chord */
WORD wMeasure; /* Measure this falls on */
BYTE bBeat; /* Beat this falls on */
BYTE bSubChordCount; /* Number of chords in the list of subchords */
DMUS_SUBCHORD SubChordList[DMUS_MAXSUBCHORD]; /* List of subchords */
DWORD dwScale; /* Scale underlying the entire chord */
BYTE bKey; /* Key underlying the entire chord */
BYTE bFlags; /* Miscellaneous flags */
} DMUS_CHORD_KEY;

Indeed, this is the source of the rather overwhelming Chord Prop-

erties page in DirectMusic Producer.

Figure 10-5: The Chord Properties window

However, for the purposes of this exercise, we will happily only dis-
play the chord name with the root of the first chord, and leave it at
that.
bool CAudio::GetChord(char *pszText)
{
if (m_pPerformance)
{
// Since we can get parameters across a broad range of time,
// we need to get the current play time, in music format, for our request.
MUSIC_TIME mtTime;
m_pPerformance->GetTime(NULL,&mtTime);
// Use GUID_ChordParam command and DMUS_CHORD_KEY structure
DMUS_CHORD_KEY Chord;
if (SUCCEEDED(m_pPerformance->GetParam(GUID_ChordParam,-1,0,mtTime,NULL,
(void *) &Chord)))
{
 Segments n 221

// Display the root of the chord as well as its name. For the root, use a
// lookup table of note names.
static char szRoots[][12] =
{ "C", "C#", "D", "Eb", "E", "F", "F#", "G", "G#", "A", "Bb", "B" };
DWORD dwRoot = Chord.SubChordList[0].bChordRoot + 12;
wsprintf(pszText,"%s%d%s",szRoots[dwRoot%12],dwRoot/12,Chord.wszName);
}
else
{
strcpy(pszText,"None");
}
return true;
}
return false;
}

II
We’ve learned a lot in this chapter. Segment playback is understand-
ably at the heart of DirectX Audio. When used for creating music, it

Unit
can be quite sophisticated. We’ve covered the scheduling and control
of Segment playback, in particular from a music perspective. If you
are using DirectMusic’s deeper music features, I encourage you to
play with Jones and refer back to this chapter as you proceed. A good
understanding of how these work will save you from a lot of head
scratching and trial-by-error programming later on.
This page intentionally left blank.
Chapter 11

AudioPaths

A udioPaths bring a whole new level of flexibility and control to

your music and sound effects. So far, everything we have done
plays through one AudioPath. This has worked well for us in our pro-
gramming examples up to this point; however, it is limited with
respect to adding sound effects and increased musical complexity.
Let’s take a look:
n No way to address 3D: For sound effects work, it is critical
that one or more sound-producing Segments can be routed to a
specific 3D position in space. It is also required that the 3D posi-
tion be directly accessible to the host application so it can be
controlled during gameplay.
n Pchannel collision: Segments authored with the same
pchannels conflict when played at the same time. For example,
instrument choices from one Segment are applied to another, and
a piano part is played by a bassoon. It should be possible to play
two Segments in their own “virtual” pchannel spaces, so they
cannot overlap.
n No independent audio processing: It should be possible to set
up different audio processors to affect different sound effects
paths in different ways, such as applying echo to sounds going
under a tunnel or distortion to an engine. Likewise, different
musical instruments as well as sections can benefit from individ-
ualized audio processing (reverb, chorus, compression, etc.).
n No independent MIDI processing: In the same vein, it should
be possible to set up different MIDI, or pmsg, processors (tools)
to manipulate individual musical parts and sound effects.

223
224 n Chapter 11

In fact, this was the status quo with DirectX 7.0. AudioPaths, intro-
duced in DirectX 8.0, set out to solve these issues.

Anatomy of an AudioPath
So what is an AudioPath? An AudioPath is a DirectMusic object that
manages the route that a musical element or sound effect takes. It
defines everything that is needed to get from the point when a Seg-
ment Track emits a pmsg (Performance message) to when the final
rendered audio finally leaves DirectSound as PCM wave data. This
includes everything from how many pchannels to use to the place-
ment of audio effects and their configuration.
An AudioPath can be as simple or sophisticated as you want it to
be. It can be a 3D path with nothing more than a DirectSound3D
module at the end for positioning, or it can be a complex path with
different pchannels assigned to different audio channels with real-
time audio effects processing.
Let’s dissect the AudioPath from top to bottom and learn about
all the possible steps. Because there’s so much that it can do, you
might find this a bit overwhelming. It’s not important that you mem-
orize all the steps. Just read this and get a feel for the possibilities.
Later, if you want to zero in on a particular feature, you can read
through it again.
The AudioPath can be broken down into two phases, before and
after the synth (see Figure 11-1). Before the synth, the play data is
managed in pmsg form inside DirectMusic’s Performance Layer.
Each pmsg identifies a particular sound to start or stop or perhaps a
control, such as which instrument to play or its volume at any point
in time. The synth converts the pmsg into wave data, at which point
we are in the second phase of the AudioPath where raw wave data is
fed through multiple streaming DirectSound buffers to be processed
via effects, optionally positioned in 3D, and finally mixed to generate
the output.
 AudioPaths n 225

AudioPath

DirectMusic Synth DirectSound

Performance (DirectMusic Core)

Figure 11-1: AudioPath phases.

AudioPath Performance Phase

Figure 11-2 shows the Performance phase in greater detail. This is
the journey a pmsg takes as it travels through the AudioPath from
Track to synth: II

Unit
AudioPath

Performance
Segment
State Graph Graph Map Graph Map Synth

Track Tool Tool Tool

Figure 11-2: AudioPath Performance phase.

Wow! Does that look complicated. Is there danger of the pmsg mak-
ing it through in time for lunch, let alone alive? Have no fear; it’s not
as crazy as it looks. If you are worried about extra latency of CPU
overhead, don’t be. The internal implementation is lightweight and
efficient. Also, most of the objects in the path are optional, so their
steps are simply skipped when they don’t exist.
Let’s walk through the steps:
1. Track emits a pmsg. The pmsg is filled with appropriate data
and time stamped. There is a wide range of available pmsg
types, from MIDI note to wave.
226 n Chapter 11

2. The pmsg enters the Segment’s Tool Graph. This is optional.

The Graph holds a set of one or more Tools, which are pmsg
processors. These can alter the pmsg in real time. Simple
examples of Tools would be echo (make multiple copies at
later timestamps) and transpose (shift the pitch up or down.)
Every Tool is represented by an IDirectMusicTool interface.
Tools are very easy to write and use. In fact, there’s a won-
derful Tool wizard that ships with DX9, so if you have wild
ideas for things you’d like to do in real time to your music or
sounds as they play, I highly recommend giving it a try.
Unfortunately, there isn’t enough room in this book to walk
through the process of creating full-featured Tools. Tools in
the Segment are typically authored directly into the Segment
and only process pmsgs in the Segment itself.
3. The pmsg leaves the Segment and enters the AudioPath’s
Tool Graph. Tools in this Graph process all pmsgs coming
from all Segments playing in the AudioPath. Think of these
as partially global tools, in that they process all pmsgs that
flow through a specific AudioPath, regardless of the Segment.
These Tools are also optional and typically authored via the
AudioPath configuration file.
4. The AudioPath maps the pmsg pchannel from the local
AudioPath defined pchannel range to a unique pchannel in
the Performance. This ensures that Segments played on dif-
ferent AudioPaths cannot collide on the same pchannels.
5. The Performance also has an optional Tool Graph. Tools in
this Graph process pmsgs from all Segments on all Audio-
Paths. These are truly global Tools.
6. Then, the Performance maps the pmsg’s pchannel to a real
MIDI channel and channel group number because the syn-
thesizer operates completely in the MIDI domain. Addi-
tionally, the pmsg is converted into MIDI format data. For
example, a note pmsg is converted to a MIDI note on and a
MIDI note off. Or, a volume curve is converted into a series
of discrete MIDI volume control change commands. The
resulting MIDI events are shipped down to the synth as a
block of MIDI commands.
 AudioPaths n 227

7. Finally, the synth receives the MIDI data and renders it into
one or more channels of wave data. Note that even the type
of synthesizer and which pchannels it connects to is managed
by the AudioPath.
The cool thing is the AudioPath has access and control over almost
all of the steps in this process. This means that via the AudioPath
API, you can access most of these as well.
Let’s continue with the second phase of the AudioPath.

AudioPath DirectSound Phase

The synth, which is managed via DirectMusic’s IDirectMusicPort
interface, generates multiple streams of audio data, which are fed
into one or more DirectSound buffers (see Figure 11-3). Again, the
II
AudioPath retains access to and control over each of the steps on the

Unit
way.

AudioPath

Synth Buffer Final

DMO 3D Mix

Buffer
DMO Send

Mix-in
Buffer DMO

Figure 11-3: AudioPath DirectSound phase.

Let’s walk through these steps:

1. The synth emits one or more streams of audio data. The
AudioPath defines for each pchannel how many audio outputs
should be created and where they should connect. These are
called “buses.” This means that you can literally have every
228 n Chapter 11

single pchannel routed to a different set of effects if you so

desire.
2. Each bus is fed into a DirectSound Buffer. These Buffers are
called “synth-in Buffers” because they receive their input
from the synth.
3. Each Buffer can hold any number of DMOs. The audio data is
fed through each DMO in order. A DMO is an audio effects
processor. Like tools, you can create and insert these into
your AudioPath configuration. DirectMusic ships with a great
set of DMOs, from Reverb to Compression, which you can
use. But be careful; each DMO uses CPU to continuously
process the audio. So the more you add, the more CPU gets
gobbled.
4. If the Buffer is set up as a 3D Buffer, the audio data next
enters the 3D engine where it is placed in 3D space and sent
to the final mix. Otherwise, the data goes directly to the final
mix.
5. A send can be placed in the Buffer to route audio data to
another Buffer. The destination buffer takes its input from
other buffers, so it is called a “Mix-in Buffer.”
6. The Mix-in Buffer also has one or more DMOs, which can
process the data before sending to the final mix. These are
often called global Buffers because they can take input from
any number of regular Buffers. This is a great way to econo-
mize on CPU. When creating the AudioPath configuration in
Producer, place Send Effects in your regular Buffers to send
to a shared Mix-in Buffer, which has some CPU-expensive
processing to do, and just that one instance can do the work
for all.
Pretty amazing, eh? With AudioPaths, you can create sophisticated
routing and processing of your audio all the way from the Segment to
the final mix. There are many cool things you can stick in it and
many ways to configure it. You can create any number of instances of
any AudioPath and each has its own virtual pchannel space, so your
Segments won’t collide.
 AudioPaths n 229

þ Note One word about terminology — although we use the term

“Buffer,” it is anything but that. Historically, a Buffer in DirectSound
represented a genuine memory buffer. You’d create it, copy your audio
data into it, and then play it. As we discussed earlier, this is very
inefficient because it binds the data to the playback device. But, by DX8,
buffers had many new features, including 3D and a kludge to get
around the inefficiency via the DuplicateSoundBuffer command. So, we
continue with the name “Buffer,” but think of it really as “AudioStream”
or something like that.

AudioPath Configurations
Now that we’ve seen all the wonder of what an AudioPath can do, we
need some way to define them. There are two ways to do this: II
n Create from a configuration file loaded from disk

Unit
n Create from a predefined set of useful AudioPath definitions

AudioPath File Configuration

You can create your own definition of an AudioPath and store it in a
file called an AudioPath configuration file. DirectMusic Producer has
a sophisticated AudioPath editor that lets you create, audition, and
save AudioPath configuration files. With it, you can address all the
features that we’ve discussed above, from pchannel assignments to
tools, DMOs, and buffers.
The AudioPath configuration file does not load directly as an
AudioPath. This would be unfortunate because it would mean that
each file could only generate one instance of an AudioPath. Let’s
think of an example: Suppose you’ve designed an AudioPath for play-
ing race car sounds in 3D. You create an AudioPath configuration
with distortion and compression DMOs along with 3D positioning
capability. You want to use the same AudioPath design interchange-
ably for each car that appears on the racetrack, and there could be a
lot of them. You also don’t want the overhead of loading another
AudioPath from disk every time a new car pops into view. Clearly,
that would be ridiculous given that all of the AudioPaths are defined
the same, but, if you simply use the same AudioPath for all cars, it
doesn’t work because:
230 n Chapter 11

n They run in the same pchannel space, so pchannel-based com-

mands like volume and pitch for one car are applied to all.
n Adjustments to the compression, distortion, and 3D position in
the Buffer are all directed to the same object, so, for example, all
cars are heard at the same point in space.
Therefore, you need both an AudioPath configuration that defines the
AudioPath and then a way to create as many AudioPaths from that
configuration as needed. The creation should be very quick and effi-
cient, definitely not involving any file I/O.
In many ways, this is analogous to Segments. You load the Seg-
ment from disk. It represents the definition of a Segment but not the
running instance. When you play the Segment, you get back a Seg-
mentState. The SegmentState is the running instance of the
Segment. Likewise, the AudioPath is the running instance of the
AudioPath configuration.
So, like a Segment, you load the AudioPath configuration from
disk. However, unlike a Segment, the AudioPath configuration
doesn’t have any methods. There’s nothing you can do with it
directly. So, it doesn’t have a unique COM interface of its own.
Instead, it is represented by the base level IUnknown interface.
With that understood, loading an AudioPath configuration is
about the same as loading a Segment. Use the Loader’s LoadObject-
FromFile() method and pass it the file path and class ID for an
AudioPath configuration, CLSID_DirectMusicAudioPath-
Configuration.
IUnknown *pIConfig;
pLoader->LoadObjectFromFile(
CLSID_DirectMusicAudioPathConfig, // AudioPath config class ID.
IID_IUnknown, // No special interface.
pwzFileName, // File path.
(void **) &pIConfig); // Config returned in pIConfig.

Once you have the AudioPath configuration loaded, create as many

AudioPaths from it as you like with the Performance’s CreateAudio-
Path() method.
// Now, use this to create a live AudioPath.
IDirectMusicAudioPath *pIPath = NULL;
m_pPerformance->CreateAudioPath(pIConfig,true,&pIPath);
 AudioPaths n 231

Predefined Paths
Because there are some really useful standard AudioPath designs
that people usually need, DirectMusic comes with a handful of pre-
defined AudioPaths that you can just ask for without first loading a
configuration file. These include:
n DMUS_APATH_DYNAMIC_3D: One bus that feeds into a
mono 3D Buffer. Every instance has its own unique Buffer.
Although you can play stereo waves and DLS instruments into
this path, they will always be rendered in mono, since Direct-
Sound 3D needs to place a mono sound in 3D space (so there is
no way to play stereo sounds in 3D.).
n DMUS_APATH_DYNAMIC_MONO: One bus feeding into a II
mono Buffer without 3D. Again, each instance has a unique
Buffer. Since there is only one bus out, MIDI pan control does

Unit
nothing. But you can use the Buffer Pan command to pan the
entire Buffer in the final mix.
n DMUS_APATH_DYNAMIC_STEREO: Two buses to a stereo
Buffer. The Buffer is not shared, so each instance is unique.
MIDI pan commands work here.
n DMUS_APATH_SHARED_STEREOPLUSREVERB: This is
a standard music AudioPath. It has two Buffers, both of which are
defined as shared, which means that no matter how many Audio-
Paths you create, they all share the same audio processing buff-
ers. The two Buffers are Dry Stereo, which receives the regular
panned left and right signals, and Reverb, which holds a reverb
DMO configured for music and receives the MIDI-controlled
reverb send as a mono input, with stereo out to the mixer. (The
reverb algorithm takes a mono signal and spreads it across the
stereo spectrum, among other things.)
To create a predefined AudioPath, call the Performance’s Create-
StandardAudioPath() method and pass it the identifier for the
predefined type that you’d like. Unlike AudioPath configurations,
which have the number of pchannels built in, the standard types do
not. One music AudioPath might need only 16 pchannels, while
another might require 100. So, CreateStandardAudioPath() also
requires a requested number of pchannels as a parameter.
232 n Chapter 11

IDirectMusicAudioPath *pIPath = NULL;

pPerformance->CreateStandardAudioPath(
DMUS_APATH_DYNAMIC_3D, // Make this a 3D path.
12, // Allocate 12 pchannels.
true, // Activate immediately.
&pIPath); // Returned AudioPath.code)

Working with AudioPaths

Once you’ve created an AudioPath, there’s a lot you can do with it,
from starting and stopping Segments to accessing and adjusting vari-
ous objects in the AudioPath. Let’s look at all of these. We can start
with the one you probably won’t need: downloading instruments and
waves directly to the AudioPath.

Downloading and Unloading with AudioPaths

As we discussed in previous chapters, before a Segment can be
played, all of its DLS instruments and waves must be downloaded to
the synth. As long as there is only one synthesizer, this is a rela-
tively simple proposition; simply download everything directly to the
one synth, regardless of what AudioPath it will play on. Indeed, that
is what we have been doing so far by downloading to the
Performance.
AudioPaths introduce an extra, though optional, level of complex-
ity. Along with the DMOs, buffers, and tools, the AudioPath
configuration also defines what synth (or even multiple synths) to
use. That’s right, even synths are plug-in components in Direct-
Music! This makes things a little more interesting because only the
AudioPath knows which instruments on which pchannels go to which
synths. So, any downloading and unloading of Segments must
include the AudioPath to manage the routing of data to the correct
synths. To do so, pass the IDirectMusicAudioPath interface to the
Segment’s Download() and Unload() commands:
// Download the Segment's instruments and waves
// to the synth(s) managed by the AudioPath.
pSegment->Download(pAudioPath);

// Later, when done, unload the instruments and waves,

// again via the AudioPath.
pSegment->Unload(pAudioPath);
 AudioPaths n 233

Note that if there are multiple copies of the same AudioPath, you
don’t need to download to each one. Downloading to one will get the
instruments and waves in place in the synth for use by all. This is a
common misconception. Also, it’s actually very rare to be using
more than one synth. So, in the 99 percent chance where you are
only using the standard Microsoft DLS2 synth, the easiest solution is
to download to the Performance and not worry about the AudioPaths.
In fact, that is exactly what Jones does.

Playing on an AudioPath
You can play Segments directly on an AudioPath. Or, you can make it
the default AudioPath for the Performance, in which case all Seg-
ments will default to playing on it. To play a Segment directly on the II
AudioPath, pass it as a parameter to PlaySegmentEx():

Unit
pPerformance->PlaySegmentEx(
pSegment,NULL,NULL,0,0,NULL,NULL,pAudioPath);

To stop a Segment playing on an AudioPath, you don’t need to do

anything special. Just pass the Segment or SegmentState to the Per-
formance’s StopEx() method, and it will stop it. However, there’s a
cool trick that you can play if you want to stop everything that is
playing on a particular AudioPath. Try passing the IDirectMusic-
AudioPath interface to StopEx() instead of a Segment or Segment-
State (IDirectMusicSegment or IDirectMusicSegmentState), and
StopEx() will stop all Segments that are currently playing on just the
one AudioPath. This is very useful, especially when playing complex
sounds in 3D and you need to simply shut off all sounds from a par-
ticular source.
pPerformance->StopEx(pAudioPath,0,0);

Likewise, you can take advantage of PlaySegmentEx()’s ability to

stop one or more Segments at the very moment the new Segment
starts. If you pass the IDirectMusicAudioPath pointer in PlaySeg-
mentEx’s pFrom parameter, it will stop all Segments playing on the
AudioPath at the very moment it starts the new Segment. For exam-
ple, you might use this to shut off all engine and tire sounds for a
race car when it hits a wall and an explosion Segment is played. In
the following example, pCarAudioPath is the AudioPath used for all
234 n Chapter 11

sounds coming from the car, and pExplosionSegment is the Segment

with the explosion sound.
pPerformance->PlaySegmentEx(
pExplosionSegment,NULL,NULL,0,0,NULL,pCarAudioPath,pCarAudioPath);

This tells DirectMusic to play the explosion sound on the car’s

AudioPath while shutting down all Segments currently playing on
the same AudioPath.

Embedded AudioPaths
Using DirectMusic Producer, it is possible to embed an AudioPath in
a Segment. This is handy because it lets you attach everything nec-
essary to play a Segment to the Segment itself. For example, if the
Segment were designed to play with a particular reverb on some of
the MIDI channels and a combination of compression and parametric
EQ on some other channels, you could create an AudioPath configu-
ration with the required reverb, compression, and EQ settings and
place it within the Segment. Then, on playback, you’d get exactly the
configuration of effects that was designed with the Segment in mind.
Once an AudioPath has been embedded in a Segment, there are
two ways to use it. You can either request the AudioPath configura-
tion directly by calling the Segment’s GetAudioPathConfig() method
or telling PlaySegmentEx() to automatically use the AudioPath.
Here’s how you use the configuration in the Segment:
// Get the AudioPath configuration from the Segment.
pSegment->GetAudioPathConfig(&pAudioPathConfig);
// Create an AudioPath from the configuration.
pPerformance->CreateAudioPath(pAudioPathConfig,true,&pAudioPath);
// Done with the configuration.
pAudioPathConfig->Release();
// Play the Segment on the AudioPath.
pPerformance->PlaySegmentEx(pSegment,NULL,NULL,0,0,0,0,pAudioPath);
// Release the AudioPath. It will go away when the Segment finishes.
pAudioPath->Release();

Or, just tell PlaySegmentEx() to use the embedded AudioPath:

pPerformance->PlaySegmentEx(pSegment,NULL,NULL,
DMUS_SEGF_USE_AUDIOPATH, // Use the embedded path, if it exists.
0,NULL,NULL);
 AudioPaths n 235

At first glance, it would seem like the latter is always the preferable
solution. Certainly, it is a lot more convenient. However, it is not as
flexible and brings with it a performance cost. Creating and using an
AudioPath, especially if it has its own dynamic buffers and DMO
effects, takes CPU cycles. So, if you intend to use the AudioPath for
the playback of one or more Segments, it quickly becomes smarter
to manage it directly.

Accessing Objects in an AudioPath

With all the wonderful things you can place in an AudioPath, it’s
important that you be able to access them. For example, if you create
an AudioPath with a 3D Buffer, you will need to access the 3D inter-
face on a regular basis to update the 3D position as the object moves II
in space.

Unit
So, there’s a special method, GetObjectInPath(), that you use to
access the objects in an AudioPath both before Segments play on it
as well as once it is actively processing playing Segments. Since
there are so many different types of objects that can reside in an
AudioPath and each has its own interface, GetObjectInPath()
approaches this in a generic way, using a series of parameters to
identify the instance and position of the object and an IID to request
the desired interface.
HRESULT hr = pPath->GetObjectInPath(
DWORD dwPChannel, // Pchannel to search.
DWORD dwStage, // Position in AudioPath.
DWORD dwBuffer, // Which buffer, if in DSound.
REFGUID guidObject, // Class ID of object.
DWORD dwIndex, // Nth item.
REFGUID iidInterface, // Requested interface IID.
void ** ppObject // Returned interface.
);

Gadzooks! That’s a lot of parameters! Have no fear, it will all make

sense, and you typically don’t need most of these. Let’s look at them
in order:
n DWORD dwPChannel: This is the Performance channel to
search. dwPChannel is necessary because a Tool or DMO could
be set to play only on specific channels. If that level of precision
is not needed, DMUS_PCHANNEL_ALL will search all
channels.
236 n Chapter 11

n DWORD dwStage: The AudioPath is broken down into a series

of “stages,” each representing a sequential step in the Audio-
Path’s route. These are represented by a series of hard-coded
identifiers (aka “stages”), starting with DMUS_PATH_AUDIO-
PATH_GRAPH, which represents the Tool Graph embedded in
the AudioPath, all the way to DMUS_PATH_PRIMARY_
BUFFER, which identifies the primary Buffer at the end of
DirectSound. Frequently used stages include DMUS_PATH_
AUDIOPATH_TOOL and DMUS_PATH_BUFFER_DMO.
n DWORD dwBuffer: When accessing a DirectSound Buffer or
DMO embedded within a Buffer, providing the stage and
pchannel is not enough because there could be more than one
Buffer in parallel. Therefore, dwBuffer identifies which Buffer to
search, starting with 0 for the first Buffer and iterating up. Oth-
erwise, this should be 0.
n REFGUID guidObject: For DMOs, Tools, and synth ports, a
class ID is needed to identify which type of object to look for.
Optionally, GUID_All_Objects will search for objects of all
classes.
n DWORD dwIndex: It is entirely possible that more than one of
a particular object type exist at the same stage in the AudioPath.
For example, if there are two DMOs of the same type in a Buffer,
use dwIndex to identify each. Alternately, when guidObject is set
to GUID_All_Objects, dwIndex can be used to iterate through all
objects within one particular stage.
n REFGUID iidInterface: You must provide the IID of the inter-
face that you are requesting. For example, to get the 3D interface
on the Buffer, pass IID_IDirectSound3DBuffer.
n void ** ppObject: This is the address of a variable that receives
a pointer to the requested interface. The interface will be
AddRef()’d by one, so be sure to remember to release it when
done.
Here’s an example of using GetObjectInPath() to update the 3D posi-
tion of a 3D AudioPath:
void Set3DPosition(IDirectMusicAudioPath *pPath,
D3DVALUE x, D3DVALUE y, D3DVALUE z)
{
IDirectSound3DBuffer *p3DBuffer = NULL;
 AudioPaths n 237

// Use GetObjectInPath to access the 3D interface.

if (SUCCEEDED(pPath->GetObjectInPath(
DMUS_PCHANNEL_ALL, // Ignore pchannels.
DMUS_PATH_BUFFER, // Buffer stage.
0, // First buffer.
GUID_All_Objects, // Ignore object type.
0, // Ignore index.
IID_IDirectSound3DBuffer, // 3D buffer interface.
(void **)&p3DBuffer)))
{
// Okay, we got it. Set the new coordinates.
p3DBuffer->SetPosition(x,y,z,DS3D_IMMEDIATE);
// Now release the interface.
p3DBuffer->Release();
}
}
II
Setting the AudioPath Volume

Unit
The AudioPath interface has a very useful method, SetVolume(), that
changes the volume of everything playing on it. SetVolume() takes as
parameters the requested volume setting (which is always a negative
number because it is really attenuation) and the amount of time to
fade or rise to the requested level.
// Drop the volume to -6db and take 200ms to do so.
pAudioPath->SetVolume(-600,200);

þ Note SetVolume() can be very CPU expensive if you aren’t careful. It

works by sending Volume MIDI messages down to the synth for every
pchannel in the AudioPath, and if you specify a duration other than 0
for immediate, it will send a series of these to create a smooth fade. If
you have created an AudioPath with many more pchannels then you are
actually using, you will pay for it with lots of wasted volume messages,
which add up. Create only as many pchannels as you need, and if you
don’t need a fade (i.e., a sudden change in volume is acceptable,
perhaps because nothing is playing), then set the fade duration to 0.

Shutting Down an AudioPath

Running AudioPaths consumes memory and CPU resources, espe-
cially if they have DMOs and dynamic buffers. It’s always a good idea
to get rid of an AudioPath when you are done using it. To do this,
simply call the AudioPath’s Release() method.
238 n Chapter 11

// Bye bye...
pPath->Release();

þ Note If an AudioPath is currently actively playing Segments, it will not

go away when you call Release(). Instead, it will wait until the last of the
Segments playing on it finishes.

In some cases, you might want to keep the AudioPath around for a
short while between uses, but keep it deactivated so CPU is not
wasted. You can do so by deactivating and then reactivating the
AudioPath. Call the AudioPath’s Activate() method, which takes a
Boolean variable to activate or deactivate.
// Don't need the path for a while...
pPath->Activate(false);
// Okay, fire it back up, we need it again...
pPath->Activate(true);

It’s important to note that the operations of creating, releasing, acti-

vating, and deactivating an AudioPath are all glitch free. In other
words, you can add and remove AudioPaths to a playing performance
without fear of an audio hiccup.

Adding AudioPaths to Jones

Okay, let’s have some fun and add AudioPath support to Jones. We
can add some serious features that make it easy to create and
explore AudioPaths. These include:
n Creating AudioPaths, both from configuration files as well as the
predefined standard AudioPaths
n Retrieving AudioPath statistics for display
n Scanning and then displaying for editing many of the objects
(such as Tools, DMOs, and 3D) within the AudioPath
n Playing Segments on AudioPaths
Figure 11-4 shows what Jones with AudioPath support looks like.
Notice how Jones has been rearranged to make room for the new
AudioPaths panel to the right of the Segments panel.
 AudioPaths n 239

II

Unit
Figure 11-4: Jones with AudioPaths.

Let’s walk through this and learn a bit about the code.

Using AudioPaths in Jones

Start by creating a new AudioPath. First, select which type of
AudioPath you’d like to create from the drop-down menu below
AudioPaths.

The choices are:

n Configuration: Creates an AudioPath from an AudioPath config-
uration file. Clicking on Create opens a file dialog, from which
you can choose the configuration file (extension .aud).
n Music: Creates a predefined music AudioPath. This path has two
buffers: Dry Stereo and Reverb. Note that multiple instances all
240 n Chapter 11

share the same two Buffers. All other predefined types have
dynamic buffers, where each instance gets a new buffer.
n 3D: Creates a predefined 3D AudioPath. This path has one mono
buffer with DirectSound 3D capabilities.
n Mono: Creates a predefined Mono AudioPath. This path has one
mono buffer.
n Stereo: Creates a predefined Stereo AudioPath. This path has
one stereo buffer.
Make your choice from the menu, and then click on Create. The
new AudioPath appears in the list just under the Create button. This
list shows all created AudioPaths.
If you’d like, experiment by creating a few paths of different
types. You might try the AudioPath configuration file BigPath.aud,
which comes with many Buffers and DMOs. It is designed with the
Segment AudioPath.sgt in mind.
Click on an AudioPath in the list. This selects it for display as
well as playback. Load a Segment and play it via the AudioPath sim-
ply by selecting both in their respective lists. Experiment by playing
the same Segment in different AudioPaths. Be sure to try
AudioPath.sgt.
The box to the right displays the name of the current path at the
top along with some statistics about it. The statistics itemize the
PChannels, Tools, Buffers, and DMOs that are in the AudioPath.

Below the statistics are lists of components that you can access and
edit within the AudioPath. These include the AudioPath volume as
well as any Tools, DMOs, and Buffers that exist in the path. Double-
click on any item in this list. Most items will open a property page
that you can then edit. For example, click on Volume to edit the vol-
ume in the Volume dialog.
 AudioPaths n 241

If you have a Segment already playing on the AudioPath, you can

adjust any of these parameters in real time and hear the change to
the music immediately. For example, create a Music path. Select it
and start music playing on it. Go to the list and double-click on the
Waves Reverb item. This is the reverb DMO used for music
playback.
II

Unit

While the music plays, you can adjust the reverb parameters and
hear the result immediately. Press the Apply button to activate the
changes.

Jones AudioPath Structures

We add a new class, CAudioPath, to Jones. Each manages a running
instance of an AudioPath. We won’t bother with wrapping the
AudioPath configuration. For that, we’ll load one every time we need
it to create an AudioPath. Since the Loader caches file objects, this
isn’t as expensive as it sounds. CAudioPath keeps track of one run-
ning instance of an AudioPath via an IDirectMusicAudioPath
242 n Chapter 11

interface pointer. This can be used to choose an AudioPath for play-

back and control its parameters in real time.
CAudioPath also provides a series of methods for directly getting
and setting useful parameters, such as Buffer frequency and 3D posi-
tion. Two methods, GetObjectInfo() and GetObject(), were designed
with Jones in mind. They provide a way to iterate through all objects
and then access them individually for display. This is not something
you would do in a regular application, but it’s great for spelunking an
AudioPath.
CAudioPath inherits from the CMyNode base class so that an
unlimited list of CAudioPath objects can be easily managed.
class CAudioPath : public CMyNode
{
public:
// Constructor.
CAudioPath(IDirectMusicAudioPath *pAudioPath,WCHAR *pzwName);
~CAudioPath();
// We keep a linked list of CAudioPaths.
CAudioPath *GetNext() { return (CAudioPath *) CMyNode::GetNext(); };
// Access methods.
IDirectMusicAudioPath *GetAudioPath() { return m_pAudioPath; };
char *GetName() { return m_szName; };
// Methods to report stats on the composition of the AudioPath.
DWORD GetPChannelCount();
DWORD GetToolCount();
DWORD GetBufferCount(bool fMixin);
DWORD GetDMOCount();
// Methods to scan and access objects in the path.
bool GetObjectInfo(DWORD dwIndex, char *pszName, AudioPathItemInfo *pItem);
bool GetObject(AudioPathItemInfo *pItem,REFGUID iidInterface,void **ppObject);
// Methods to get and set useful parameters in the AudioPath.
long GetVolume() { return m_lVolume; };
void SetVolume(long lVolume);
bool GetBufferParams(DWORD dwBuffer, DWORD dwStage, DWORD dwPChannel,
long *plVolume, long *plPan, DWORD *pdwFrequency);
bool SetBufferParams(DWORD dwBuffer, DWORD dwStage, DWORD dwPChannel,
long lVolume, long lPan, DWORD dwFrequency);
bool Get3DParams(DWORD dwBuffer, DWORD dwStage,
DWORD dwPChannel, DS3DBUFFER *p3DParams);
bool Set3DParams(DWORD dwBuffer, DWORD dwStage,
DWORD dwPChannel, DS3DBUFFER *p3DParams);
private:
bool ClassIDToName(REFGUID rguidClassID,char *pszName);
char m_szName[20]; // Name, for display
IDirectMusicAudioPath * m_pAudioPath; // The DirectMusic AudioPath object.
 AudioPaths n 243

long m_lVolume; // Keep track of current volume.

};

CAudioPathList is based on CMyList and manages a linked list of

CAudioPaths.
class CAudioPathList : public CMyList
{
public:
// Overrides for CMyList methods.
CAudioPath *GetHead() { return (CAudioPath *) CMyList::GetHead(); };
CAudioPath *RemoveHead() { return (CAudioPath *) CMyList::RemoveHead(); };
// Clear list and release all references.
void Clear();
};

CAudio introduces a new field to keep track of the list of AudioPaths. II

CAudioPathList m_AudioPathList; // List of created AudioPaths.

Unit
It adds two methods for creating AudioPaths.
CAudioPath *CreateAudioPathFromConfig(WCHAR *pwzFileName);
CAudioPath *CreateStandardAudioPath(DWORD dwType, DWORD dwChannels);

It also adds a CAudioPath parameter to CAudio::PlaySegment(), so

you can now play a Segment on a specific AudioPath.
void PlaySegment(CSegment *pSegment,CAudioPath *pPath); // Play a Segment.

As before, we look only at the code implemented in the audio

classes. We do not pay attention to the MFC Windows code. You
should be able to take these classes and use them equally effectively
with any UI framework, including WTL or direct Windows
programming.

Creating an AudioPath
CAudio::CreateAudioPathFromConfig() loads the AudioPath configu-
ration from the passed file path and uses it to create an AudioPath.
First, it loads the configuration from the file. Then, it retrieves the
name of the configuration by QI’ing for the IDirectMusicObject
interface and using that to get the object descriptor. This normally
wouldn’t be required in an application, but we want to display the
name of the AudioPath in Jones. Next, CreateAudioPathFromConfig()
calls IDirectMusicPerformance8::CreateAudioPath() to create the
AudioPath and, assuming the call succeeds, creates a CAudioPath
244 n Chapter 11

class to wrap the AudioPath. It places the new CAudioPath in its list
of AudioPaths, m_AudioPathList.
Note that CreateAudioPathFromConfig() doesn’t keep the
AudioPath configuration around. Doing so would be redundant, since
the Loader keeps a copy in its own cache. The next time we try to
create an AudioPath from this configuration, the Loader avoids a file
load, so we get the efficiency we need.
CAudioPath *CAudio::CreateAudioPathFromConfig(WCHAR *pwzFileName)
{
CAudioPath *pPath = NULL;
IUnknown *pIConfig;
// First, load the configuration.
if (SUCCEEDED(m_pLoader->LoadObjectFromFile(
CLSID_DirectMusicAudioPathConfig, // AudioPath config class ID.
IID_IUnknown, // No special interface.
pwzFileName, // File path.
(void **) &pIConfig))) // Config returned in pIConfig.
{
// Get the config's name by retrieving the object
// descriptor from the configuration.
DMUS_OBJECTDESC Desc;
Desc.dwSize = sizeof(Desc);
wcscpy(Desc.wszName,L"No Name");
IDirectMusicObject *pIObject = NULL;
pIConfig->QueryInterface(IID_IDirectMusicObject,(void **)&pIObject);
if (pIObject)
{
pIObject->GetDescriptor(&Desc);
pIObject->Release();
}
// Now, use this to create a live AudioPath.
IDirectMusicAudioPath *pIPath = NULL;
m_pPerformance->CreateAudioPath(pIConfig,true,&pIPath);
if (pIPath)
{
// Create a CAudioPath object to manage it.
pPath = new CAudioPath(pIPath,Desc.wszName);
if (pPath)
{
m_AudioPathList.AddTail(pPath);
}
pIPath->Release();
}
pIConfig->Release();
}
return pPath;
}
 AudioPaths n 245

The constructor for CAudioPath simply stores the AudioPath and its
name. The constructor keeps the name in ASCII format because
that’s all we need for the UI.
CAudioPath::CAudioPath(IDirectMusicAudioPath *pAudioPath,WCHAR *pzwName)

{
m_pAudioPath = pAudioPath;
pAudioPath->AddRef();
wcstombs(m_szName,pzwName,sizeof(m_szName));
m_lVolume = 0;
}

Getting AudioPath Stats

Now that we have an AudioPath loaded, let’s see what we can learn II
from it. Unfortunately, there are no straightforward DirectMusic

Unit
methods for reading an AudioPath’s capabilities because the assump-
tion is that every AudioPath is designed specifically for the applica-
tion that uses it, so there would be no reason to interrogate it.
However, for an application like Jones, where we can load any arbi-
trary AudioPath and look at it, we need such a method.
No problem. Ve haff meanss for makink ze AudioPath talk!

PChannel Count
First, we create a method (CAudioPath::GetPChannelCount()) that
figures out how many pchannels belong to the AudioPath in a some-
what hokey way. IDirectMusicAudioPath has a method, Convert-
PChannel(), that is used to convert any pchannel from the AudioPath
virtual pchannel space to the absolute pchannel value in the Perfor-
mance. We call ConvertPChannel() with virtual pchannel values of 0
and counting, up until it fails. This assumes that the pchannels are 0
based. I’ve never run into a situation where they aren’t, so it’s rela-
tively safe, and since this is only used for display, it’s not the end of
the world if we miss some esoteric non-zero-based pchannel
assigments.
DWORD CAudioPath::GetPChannelCount()

{
DWORD dwCount = 0;
if (m_pAudioPath)
{
246 n Chapter 11

HRESULT hr = S_OK;
for (;hr == S_OK;dwCount++)
{
DWORD dwResult;
// Normally, we'd use this to convert from the AudioPath's
// pchannel to its equivalent value in the Performance.
hr = m_pAudioPath->ConvertPChannel(dwCount,&dwResult);
}
dwCount--;
}
return dwCount;
}

Tool Count
To find out how many Tools are embedded in the AudioPath, CAudio-
Path::GetToolCount() calls GetObjectInPath(), iterating through
Tools in the DMUS_PATH_AUDIOPATH_TOOL stage until it fails.
This works because the generic class ID, GUID_All_Objects, is used
instead of a specific Tool class ID.
DWORD CAudioPath::GetToolCount()

{
DWORD dwCount = 0;
for (dwCount = 0; ;dwCount++)
{
IUnknown *pUnknown;
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
DMUS_PCHANNEL_ALL, // Search all pchannels.
DMUS_PATH_AUDIOPATH_TOOL, // Look in the Tool stage.
0, // No buffer!
GUID_All_Objects, // All Tools.
dwCount, // Nth Tool.
IID_IUnknown, // Generic interface.
(void **)&pUnknown)))
{
// We found another Tool, so continue.
pUnknown->Release();
}
else
{
// No more Tools. Quit.
break;
}
}
return dwCount;
}
 AudioPaths n 247

Buffer Count
To figure out how many DirectSound buffers are included in the
AudioPath, we use a similar technique. CAudioPath::GetBuffer-
Count() uses GetObjectInPath() to scan for Buffers. However, it is
complicated by the fact that there are two sets of Buffers: Sink-in,
which take their input from the synth, and Mix-in, which take their
input from other Buffers. To handle this and avoid redundant code,
GetBufferCount() receives one parameter, fMixIn, which determines
whether to look in the DMUS_PATH_MIXIN_BUFFER or DMUS_
PATH_BUFFER stage. It iterates through the Buffers by calling
GetObjectInPath() until it fails.
DWORD CAudioPath::GetBufferCount(bool fMixin)
{
II
DWORD dwPChannel;

Unit
DWORD dwStage;
// If we are searching Mix-in Buffers...
if (fMixin)
{
// Pchannel must be 0.
dwPChannel = 0;
// Set stage to Mix-in Buffers.
dwStage = DMUS_PATH_MIXIN_BUFFER;
}
else
{
// Pchannel must be all channels.
dwPChannel = DMUS_PCHANNEL_ALL;
// Set stage to Sink-in Buffers.
dwStage = DMUS_PATH_BUFFER;
}
DWORD dwCount = 0;
for (dwCount = 0; ;dwCount++)
{
IUnknown *pUnknown;
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
dwPChannel, // Look on all pchannels.
dwStage, // Searching the appropriate stage.
dwCount, // Nth buffer.
GUID_All_Objects, // Ignore class ID (can only get Buffers).
0, // No index.
IID_IUnknown, // Generic interface.
(void **)&pUnknown)))
{
// Hah! We found another Buffer!
pUnknown->Release();
248 n Chapter 11

}
else
{
// No more Buffers. Quit.
break;
}
}
return dwCount;
}

DMO Count
CAudioPath::GetDMOCount() figures out how many DMOs exist in
the AudioPath. It does so by using GetObjectInPath() to scan for
DMOs in each of the Sink-in and Mix-in Buffers.
DWORD CAudioPath::GetDMOCount()
{
DWORD dwCount = 0;
DWORD dwBufferCount = GetBufferCount(false);
// There will be two passes. First the Sink-in Buffers (which
// read from the synth). Then, the Mix-in Buffers, which receive
// from other Buffers.
DWORD dwDMOStage[2] = {
DMUS_PATH_BUFFER_DMO,
DMUS_PATH_MIXIN_BUFFER_DMO };
DWORD dwPChannel[2] = { DMUS_PCHANNEL_ALL, 0 };
for (DWORD dwBufferType = 0; dwBufferType < 2; dwBufferType++)
{
DWORD dwBuffer;
for (dwBuffer = 0; dwBuffer < dwBufferCount;dwBuffer++)
{
// Now, for each Buffer, iterate through the DMOs.
for (DWORD dwDMO = 0; ;dwDMO++)
{
IUnknown *pUnknown;
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
dwPChannel[dwBufferType], // Search all pchannels.
dwDMOStage[dwBufferType], // Buffer DMO stage.
dwBuffer, // Which Buffer to search.
GUID_All_Objects, // Search for all object types.
dwDMO, // Index of DMO.
IID_IUnknown, // Look for base interface.
(void **)&pUnknown)))
{
// DMO was found! Increment.
dwCount++;
pUnknown->Release();
}
 AudioPaths n 249

else
{
// No DMO, move on to the next Buffer.
break;
}
}
}
// Prepare Buffer count for second pass.
dwBufferCount = GetBufferCount(true);
}
return dwCount;
}

Scanning an AudioPath
Jones needs a way to iterate through all of the objects that were
authored into an AudioPath so they can be displayed and double-
II
clicked for editing. These include any DMOs and Tools, as well as

Unit
the Buffer parameters, including 3D. Normally, this is not particu-
larly useful because an application should know what it needs to
access and request them directly. Following that philosophy,
DirectMusic’s AudioPath API was not designed with this sort of
inspection in mind. In other words, there is no direct API that you
can call and query for the nth object in the AudioPath. You can sort of
do it with GetObjectInPath(), but you must iterate through stages,
buffers, and then objects within the buffers.
CAudioPath::GetObjectInfo() does just that, but it gets all the
extra work out of the way for you. It iterates through all objects
within an AudioPath for which we can subsequently throw up a UI of
some sort. It uses a data structure, called AudioPathItemInfo, to
return all of the parameters (stage, buffer, index) that were needed
to access the nth object in the AudioPath, and it stores an identifier
for the type of object (Tool, DMO, Buffer, or 3D) in AudioPathItem-
Info. A subsequent call to GetObject() with the AudioPathItemInfo
structure can quickly access the specified object.
GetObjectInfo() also returns a name for the object. Unfortu-
nately, none of the objects have a way to return a friendly name. So,
GetObjectInfo() improvises as best it can. For DMOs, it looks at the
class ID to see if it is one of the familiar DMOs that ships with
DirectX and, if so, sticks in the name. Otherwise, it just gives
generic names, like “DMO 3,” etc.
250 n Chapter 11

Here’s the code. First, we define the types of objects that we are
looking for. These really reflect UI operations that we know we can
do in Jones with these particular items.
typedef enum _AP_ITEM_TYPE
{
AUDIOPATH_VOLUME = 1, // Set the AudioPath volume.
AUDIOPATH_TOOL = 2, // Open a Tool's property page.
AUDIOPATH_DMO = 3, // Open a DMO's property page.
AUDIOPATH_BUFFER = 4, // Set Buffer parameters.
AUDIOPATH_3D =5 // Set 3D parameters.
} AP_ITEM_TYPE;

The AudioPathItemInfo structure records everything needed to find

the nth item in the AudioPath:
typedef struct _AudioPathItemInfo
{
AP_ITEM_TYPE ItemType; // Which type of object.
DWORD dwStage; // Which stage to look in.
DWORD dwBuffer; // Which Buffer, if applicable.
DWORD dwIndex; // Index into nth object in stage.
DWORD dwPChannel; // Pchannel, if applicable.
} AudioPathItemInfo;

Finally, here’s the big kahuna, GetObjectInfo(). This scans for the nth
item (within the set of things it is looking for) in the AudioPath. To
do so, it starts at the top and keeps decrementing the passed iterator,
dwIndex, until it hits zero, at which point it returns the current item.
bool CAudioPath::GetObjectInfo(
DWORD dwIndex, // Nth item to look for.
char *pszName, // Returned name.
AudioPathItemInfo *pItem) // Returned parameters needed to retrieve item.

{
// Initialize the AudioPathItemInfo.
pItem->dwBuffer = 0;
pItem->dwIndex = 0;
pItem->dwStage = 0;
// If this is the very start, return the volume parameter.
if (dwIndex == 0)
{
strcpy(pszName,"Volume");
pItem->ItemType = AUDIOPATH_VOLUME;
return true;
}
dwIndex--;
// Okay, now see if this is a tool.
 AudioPaths n 251

DWORD dwToolCount = GetToolCount();

if (dwIndex < dwToolCount)
{
// Since Tools don't have accessible names, just
// identify which Tool it is.
sprintf(pszName,"Tool %d",dwIndex+1);
pItem->ItemType = AUDIOPATH_TOOL;
pItem->dwIndex = dwIndex;
pItem->dwStage = DMUS_PATH_AUDIOPATH_TOOL;
return true;
}
dwIndex -= dwToolCount;
// Now, we look at all the items in the Buffers.
// There will be two passes. First the Sink-in Buffers (which
// read from the synth). Then, the Mix-in Buffers, which receive
// from other Buffers.
// Set up the variables that change for the two passes.
// There are two DMO stages in the AudioPath, one for each Buffer type.
II
DWORD dwDMOStage[2] = {

Unit
DMUS_PATH_BUFFER_DMO,
DMUS_PATH_MIXIN_BUFFER_DMO };
// And, there are two Buffer stages.
DWORD dwBufferStage[2] = { DMUS_PATH_BUFFER, DMUS_PATH_MIXIN_BUFFER };
// Pchannels are handled differently for Sink-in vs Mix-in Buffers.
DWORD dwPChannel[2] = { DMUS_PCHANNEL_ALL, 0 };
// Variable to track the two passes.
DWORD dwBufferType;
// Okay, let's do it.
for (dwBufferType = 0; dwBufferType < 2; dwBufferType++)
{
// How many Buffers of this type?
DWORD dwBufferCount = GetBufferCount(dwBufferType == 1);
// Get the pchannel to use for this Buffer type.
pItem->dwPChannel = dwPChannel[dwBufferType];
// For each Buffer (there can easily be more than one...)
pItem->dwBuffer = 0;
for (;pItem->dwBuffer < dwBufferCount;pItem->dwBuffer++)
{
// First, check if we are at the Buffer itself. If so,
// just return success.
if (dwIndex == 0)
{
// Buffers don't have names, so just use a counter.
if (dwBufferType)
{
sprintf(pszName,"MixBuffer %d",pItem->dwBuffer);
}
else
{
sprintf(pszName,"SinkBuffer %d",pItem->dwBuffer);
252 n Chapter 11

}
pItem->dwStage = dwBufferStage[dwBufferType];
pItem->ItemType = AUDIOPATH_BUFFER;
return true;
}
dwIndex--;
IUnknown *pUnknown;
// Okay, it's not the Buffer. Now, iterate through the DMOs.
for (pItem->dwIndex = 0;;pItem->dwIndex++)
{
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
pItem->dwPChannel, // Pchannels.
dwDMOStage[dwBufferType], // Which Buffer DMO stage.
pItem->dwBuffer, // Which Buffer are we looking at?
GUID_All_Objects, // Scan all DMOs.
pItem->dwIndex, // Look for the Nth DMO.
IID_IUnknown, // Just the base interface.
(void **)&pUnknown)))
{
// If index is 0, we've found our DMO.
if (dwIndex == 0)
{
// We'll use IPersist to get the class ID.
IPersist *pPersist;
// Put in a default name.
sprintf(pszName," DMO %d",pItem->dwIndex+1);
// See if it has an IPersist and, if so, get the class ID.
if (SUCCEEDED(pUnknown->QueryInterface(IID_IPersist,
(void **) &pPersist)))
{
CLSID clsid;
pPersist->GetClassID(&clsid);
// With the class ID, we might recognize one
// of our standard DMOs.
ClassIDToName(clsid,pszName);
pPersist->Release();
}
// Fill in the rest.
pItem->dwStage = dwDMOStage[dwBufferType];
pItem->ItemType = AUDIOPATH_DMO;
pUnknown->Release();
return true;
}
pUnknown->Release();
dwIndex--;
}
else
{
// Ran out of DMOs, so go to next Buffer.
break;
 AudioPaths n 253

}
}
pItem->dwIndex = 0;
// Check to see if this Buffer supports 3D.
IDirectSound3DBuffer *p3DBuffer;
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
pItem->dwPChannel, // Pchannel.
dwBufferStage[dwBufferType], // Which Buffer stage.
pItem->dwBuffer, // Which Buffer.
GUID_All_Objects,0, // Don't care about object class.
IID_IDirectSound3DBuffer, // Does it have the 3D interface?
(void **)&p3DBuffer)))
{
// If there's a 3D interface, this must be a 3D Buffer.
p3DBuffer->Release();
if (dwIndex == 0)
{
// Ooh, goody, we got it.
II
strcpy(pszName," 3D");

Unit
pItem->dwStage = dwBufferStage[dwBufferType];
pItem->ItemType = AUDIOPATH_3D;
return true;
}
dwIndex--;
}
}
}
return false;
}

To use GetObjectInfo(), the application calls it in a for loop, incre-

menting the iterator dwIndex and retrieving the name and Audio-
PathItemInfo for each item found until GetObjectInfo() eventually
runs out and returns false. That’s exactly what Jones does by build-
ing a list of all the items, which it displays in the list on the right.
Then, when the user double-clicks on an item in the list, Jones has
what it needs to retrieve the item for editing.

Editing AudioPath Object Parameters

Now that we’ve got this nice list with all the items and their names,
retrieving an object is straightforward because we have everything
we need to call the AudioPath’s GetObjectInPath() method.
254 n Chapter 11

Accessing with AudioPathItemInfo

To directly access any generic object using the AudioPathItemInfo
structure, we have a convenience function, GetObject(), which is
very simple. GetObject() uses the AudioPathItemInfo structure that
was filled in by GetObjectInfo(). It also provides the ID of the inter-
face that it is expecting. Together, these are used to call GetObject-
InPath() to retrieve the desired interface.
bool CAudioPath::GetObject(AudioPathItemInfo *pItem,
REFGUID iidInterface,
void **ppObject)
{
return (SUCCEEDED(m_pAudioPath->GetObjectInPath(
pItem->dwPChannel, // Use the pchannel that GetObjectInfo returned.
pItem->dwStage, // And, the stage that GetObjectInfo returned.
pItem->dwBuffer, // Likewise, use the Buffer.
GUID_All_Objects, // Get any object type.
pItem->dwIndex, // Get the Nth item.
iidInterface, // Caller requested an interface.
ppObject))); // Returned interface.
}

Accessing Buffer Parameters

CAudioPath has two methods (GetBufferParams() and SetBuffer-
Params()) for reading and writing the Buffer volume, pan, and
frequency. When you double-click on a Buffer in the list, Jones opens
a dialog to edit these three parameters and uses these two methods
to retrieve and set the data. Note that these two methods do not use
the AudioPathItemInfo structure. GetBufferParams() and SetBuffer-
Params() can be used much more broadly, since typically the
application already knows specifically what Buffer it needs to access.
Therefore, these routines are great examples of using GetObject-
InPath() to access and manipulate a specific object in an AudioPath.
/* CAudioPath::GetBufferParams
GetBufferParams is a convenience function that retrieves the current
volume, pan, and frequency from the Buffer.
The Buffer is defined by Buffer index, pchannel, and stage, since this
could be a Sink-in or Mix-in Buffer.
*/

bool CAudioPath::GetBufferParams(DWORD dwBuffer,

DWORD dwStage,
DWORD dwPChannel,
 AudioPaths n 255

long *plVolume,
long *plPan,
DWORD *pdwFrequency)

{
bool fSuccess = false;
// We'll be retrieving an IDirectSoundBuffer interface.
IDirectSoundBuffer *pBuffer = NULL;
// Use GetObjectInPath to retrieve the Buffer.
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
dwPChannel, // The pchannel.
dwStage, // Mix-in or Sink-in stage.
dwBuffer, // Which Buffer.
GUID_All_Objects,0, // No need for class ID.
IID_IDirectSoundBuffer,
(void **)&pBuffer)))
{
// We got it. Go ahead and read the three parameters.
II
pBuffer->GetFrequency(pdwFrequency);

Unit
pBuffer->GetPan(plPan);
pBuffer->GetVolume(plVolume);
pBuffer->Release();
fSuccess = true;
}
return fSuccess;
}

/* CAudioPath::SetBufferParams
SetBufferParams is a convenience function that sets the
volume, pan, and frequency for the Buffer.
The Buffer is defined by buffer index, pchannel, and stage, since this
could be a Sink-in or Mix-in Buffer.
*/

bool CAudioPath::SetBufferParams(DWORD dwBuffer,

DWORD dwStage,
DWORD dwPChannel,
long lVolume,
long lPan,
DWORD dwFrequency)

{
bool fSuccess = false;
// We'll be using an IDirectSoundBuffer interface.
IDirectSoundBuffer *pBuffer = NULL;
// Use GetObjectInPath to retrieve the Buffer.
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
dwPChannel, // The pchannel.
dwStage, // Mix-in or Sink-in stage.
dwBuffer, // Which Buffer.
256 n Chapter 11

GUID_All_Objects,0, // No need for class ID.

IID_IDirectSoundBuffer,
(void **)&pBuffer)))
{
// We got it. Go ahead and set the three parameters.
pBuffer->SetFrequency(dwFrequency);
pBuffer->SetPan(lPan);
pBuffer->SetVolume(lVolume);
pBuffer->Release();
fSuccess = true;
}
return fSuccess;
}

Accessing 3D Parameters
CAudioPath has two methods (Get3DParams() and Set3DParams())
for reading and writing the 3D position of a 3D DirectSound Buffer.
Use Set3DParams() to continuously update the position of a 3D
object that is being tracked in space by a 3D AudioPath.
/* CAudioPath::Get3DParams
Get3DParams is a convenience function that retrieves the current
3D parameters from a 3D Buffer.
*/

bool CAudioPath::Get3DParams(DWORD dwBuffer,

DWORD dwStage,
DWORD dwPChannel,
DS3DBUFFER *p3DParams)

{
bool fSuccess = false;
// We'll be using an IDirectSound3DBuffer interface.
IDirectSound3DBuffer *p3DBuffer = NULL;
// Use GetObjectInPath to retrieve the Buffer.
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
dwPChannel,
dwStage,
dwBuffer,
GUID_All_Objects,0,
IID_IDirectSound3DBuffer,
(void **)&p3DBuffer)))
{
// Okay, we got the Buffer. Read it.
fSuccess = true;
p3DBuffer->GetAllParameters(p3DParams);
p3DBuffer->Release();
}
 AudioPaths n 257

return fSuccess;
}

/* CAudioPath::Set3DParams
Set3DParams is a convenience function that writes
3D parameters to a 3D Buffer.
*/

bool CAudioPath::Set3DParams(DWORD dwBuffer,

DWORD dwStage,
DWORD dwPChannel,
DS3DBUFFER *p3DParams)

{
bool fSuccess = false;
// We'll be using an IDirectSound3DBuffer interface.
IDirectSound3DBuffer *p3DBuffer = NULL;
// Use GetObjectInPath to retrieve the Buffer.
II
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(

Unit
dwPChannel,
dwStage,
dwBuffer,
GUID_All_Objects,0,
IID_IDirectSound3DBuffer,
(void **)&p3DBuffer)))
{
// Okay, we got the Buffer. Write to it.
fSuccess = true;
p3DBuffer->SetAllParameters(p3DParams,DS3D_IMMEDIATE);
p3DBuffer->Release();
}
return fSuccess;
}

Accessing Tools and DMOs

CAudioPath doesn’t have any special methods for accessing Tools or
DMOs because they would be nothing more than very thin wrappers
over GetObjectInPath(). Tools and DMOs have four different ways of
providing control, and GetObjectInPath() provides access to all. They
are:
n A specific programming interface: Each object provides its
own COM interface that allows direct program control over its
parameters. For example, the IDirectSoundFXEcho8 interface
provides direct access to read and write the Echo parameters.
258 n Chapter 11

n The IMediaParams interface: This interface provides timeline

control over each parameter in a standardized way. It includes
methods, used in editors, for querying the names and ranges of
each parameter, as well as methods for setting parameters with
continuously changing curves. DirectMusic supports this via the
Parameter Control Track.
n The IPersistStream interface: This interface provides a stan-
dard way to read and write data from a file stream. All DMOs and
Tools must support this if they have any data they must read
from disk. This can also be used to write data directly into an
object, though given the other options, it’s typically not the best
approach.
n A property page: Each DMO and Tool has the option of
providing a property page, which can be used to edit its parame-
ters in an authoring tool. To do so, the DMO or Tool must pro-
vide an ISpecifyPropertyPages interface, which, in turn, leads to
the IPropertyPage interface that manages the property page.
Jones takes the last approach, using GetObjectInPath() to access the
DMO or Tool’s ISpecifyPropertyPages interface to open the property
dialog. The code to do this is remarkably simple:
ISpecifyPropertyPages *pPages;
if (SUCCEEDED(pAudioPath->GetObjectInPath(
DMUS_PCHANNEL_ALL, // Pchannel.
DMUS_PATH_BUFFER_DMO, // Look at the DMO stage.
0,GUID_All_Objects, // Any DMO.
dwIndex, // Nth DMO.
IID_ISpecifyPropertyPages, // Get the property page interface.
(void **)&pPages)))
{
// Success! It has a property page!
CAUUID PageID;
pPages->GetPages(&PageID);
pPages->Release();
// Use the Windows Property page mechanism to view it.
OleCreatePropertyFrame(m_hWnd,0,0,
L"DMO Editor",1,(IUnknown **) &pPages,1,
PageID.pElems,0,0,0);
}
 AudioPaths n 259

Accessing Volume
Volume is a special case because the command to adjust the volume
is built directly into the IDirectMusicAudioPath interface. For com-
pleteness, CAudioPath provides methods for getting and setting the
volume.
void CAudioPath::SetVolume(long lVolume)
{
m_pAudioPath->SetVolume(lVolume,0);
m_lVolume = lVolume;
}

GetVolume() simply returns the last volume that was set.

long GetVolume() { return m_lVolume; };
II
Performing with an AudioPath

Unit
Playing a Segment with an AudioPath is simple; just pass the
AudioPath as a parameter to PlaySegmentEx(). So, we update
CAudio::PlaySegment() to include a CAudioPath as a parameter:
void CAudio::PlaySegment(CSegment *pSegment,CAudioPath *pPath)
{
if (pSegment)
{
IDirectMusicAudioPath *pIPath = NULL;
if (pPath)
{
pIPath = pPath->GetAudioPath();
}
// Is there a transition Segment?
IDirectMusicSegment8 *pTransition = NULL;
if (pSegment->GetTransition())
{
pTransition = pSegment->GetTransition()->GetSegment();
}
DWORD dwFlags = pSegment->GetFlags();
if (pTransition)
{
dwFlags |= DMUS_SEGF_AUTOTRANSITION;
}
IDirectMusicSegmentState8 *pISegState;
if (SUCCEEDED(m_pPerformance->PlaySegmentEx(
pSegment->GetSegment(), // Returns IDirectMusicSegment
NULL,pTransition, // Use the transition, if it exists.
dwFlags, // Playback flags.
260 n Chapter 11

0, // No time stamp.
(IDirectMusicSegmentState **) &pISegState, // Returned SegState.
NULL, // No prior Segment to stop.
pIPath))) // Use AudioPath, if supplied.
{
CSegState *pSegState = new CSegState(pSegment,pISegState);
if (pSegState)
{
m_SegStateList.AddHead(pSegState);
}
pISegState->Release();
}
}
}

Now that we have a decent understanding of what AudioPaths are

and how to use them, we’ve got the primary pieces of the Direct-
Music audio architecture in place. We know how to create an
AudioPath, we know how to play music and sound effects on it, and
we know how to access components within the AudioPath, so we can
alter them in real-time in response to real time events.
Chapter 12

Scripting

A fter DirectX 6.1 shipped with the original DirectMusic, it

quickly became clear from developer feedback that the con-
tent-driven approach was well-liked. Content creators and audio
leads loved the fact that they could create complex sounds and wrap
them up in Segment files.
However, a critical piece was still missing. There were still too
many scenarios where it was necessary to write down instructions
for the programmer, and some of these instructions could be quite
complicated. For example: “When the cuddly monster dies, play a fill
transition Segment and drop the groove level to 23 while playing
byebye.sgt as a secondary Segment aligned to a beat boundary.”
Ouch! This has issues:
n That’s a lot of information for the programmer to get right. Oh,
and how many monsters did you say there were?
n If the content creator wants to change anything (for example,
play a break transition), he or she can’t just change the content
and try again. No, the whole program needs to be recompiled
with the change request.
n The above items slow down the development process and keep
programmers and content creators perpetually irritated with
each other.
Wouldn’t it be nice if the content creator could say instead, “When
the cuddly monster dies, please call the script routine Cuddly-
MonsterBuysFarm, and I’ll take it from there.” Thus, scripting was
born…

261
262 n Chapter 12

Anatomy of a Script
DirectMusic’s scripting support is all built around the Script object.
The CLSID_DirectMusicScript object with its IDirectMusicScript
interface represents the Script object. Like most DirectX Audio
objects, the Loader reads the Script object from a file. Once it has
loaded a script, the application can make direct calls into the script to
run its routines, as well as pass variables back and forth between the
script and the application.
So think of a script as a discrete object that is made up of:
n Routines to call
n Variables that store data as well as provide a means to pass the
data between the application and the script
n Embedded and/or linked content (Segments, waves, and any
other DirectX Audio media) to be referenced by variables and
manipulated by routines
Figure 12-1 shows an example of a script with its internal routines,
variables, and content. The application manages the script and can
make calls to trigger the routines and access the variables. In turn,
the routines and variables access the embedded and linked Seg-
ments, AudioPaths, and other content.

App Script
Segment
Routines Content
ref

Segment
Style

DLS
Variables Collection
AudioPath
ref

Figure 12-1: A script with its internal routines, variables, and content.
 Scripting n 263

Routines
The heart of scripting is, of course, in the code. The code in a Direct-
Music script is all stored in routines. There is no primary routine
that is called first, unlike the required main() in a C program. Instead,
any routine can be called in any order. Each routine exposes some
specific functionality of the Script object. For example, two routines
might be named EnterLevel or MonsterDies. The first would be
called upon entering a level in a game. The second would be called
when a monster dies. The beauty is that the programmer and con-
tent creator only need to agree on the names EnterLevel and
MonsterDies and call them at the appropriate times.
Each routine has a unique name, and that name is used to invoke
the routine via the IDirectMusicScript::CallRoutine() method. II

Unit
HRESULT CallRoutine(
WCHAR *pwszRoutineName,
DMUS_SCRIPT_ERRORINFO *pErrInfo
);

Pass only two parameters — the name of the routine and an optional
structure that is used to return errors. If the routine does indeed
exist (which should always be the case in a debugged project),
CallRoutine() invokes it immediately and does not return until the
routine has completed.
Therefore, the code to call a routine is typically very simple:
// Invoke the script handler when the princess eats the frog by accident
pScript->CallRoutine(L”EatFrog”,NULL); // Yum yum.

Notice that no parameters can be passed to a routine. If the caller

would like to set some parameters for the routine, it must set them
via global variables first. Likewise, if the routine needs to return
something, that something needs to be stored in a variable and then
retrieved by the caller. For example, a routine that creates an Audio-
Path from an AudioPath configuration might store the AudioPath in a
variable, which the caller can then read immediately after calling the
AudioPath creation routine.
So, let’s look at variables.
264 n Chapter 12

Variables
There really are three uses for variables in a script:
n Parameter passing variables: Declare a variable in the script
and use it to pass information back and forth between the appli-
cation and the script. An example might be a variable used to
track the number of hopping objects in a scene, which in turn
influences the musical intensity.
n Internal variables: Declare a variable in the script and use it
purely internal to the script. An example might be an integer
that keeps track of how many times the princess steps on frogs
before the fairy godmother really gets pissed.
n Content referencing variables: Link or embed content (Seg-
ments, AudioPaths, etc.) in the script, and the script automati-
cally represents each item as a variable. For example, if the
Segment FrogCrunch.sgt is embedded in the script, the variable
FrogCrunch is automatically allocated to provide access to the
Segment.
Variables used in any of these three ways are equally visible to the
application calling into the script.
Within the scripting language’s internal implementation, all vari-
ables are handled by the variant data type. A variant is a shape-
shifting type that can masquerade as anything from byte to pointer.
To do so, it stores both the data and tag that indicates which data
type it is. This is why you can create a variable via the “dim” key-
word in a scripting or Basic language and then use it without first
specifying its type. That is all fine and good in scripting, but it is not
a natural way to work in C++. Therefore, IDirectMusicScript gives
you three ways to work with variables and translates from variant
appropriately:
n Number: A 32-bit long, used for tracking numeric values like
intensity, quantity, level, etc.
n Object: An interface pointer to an object. Typically, this is used
to pass objects back and forth with the application. In addition, all
embedded and referenced content is handled as an object
variable.
 Scripting n 265

n Variant: You still have the option to work with variants if you
need to. This is useful for passing character strings and other
types that cannot be translated into interfaces or longs. Fortu-
nately, this is rare.
To accommodate all three types, IDirectMusicScript has Get and Set
methods for each. They are GetVariableNumber(), GetVariable-
Object(), and GetVariableVariant() to retrieve a variable from the
script and SetVariableNumber(), SetVariableObject(), and Set-
VariableVariant() to assign a value to a variable in the script.
Each Get or Set call passes the name of the variable along with
the data itself. For example, the parameters for GetVariable-
Number(), which retrieves a numeric value, are: the Unicode name,
a pointer to a long to fill, and an optional error structure in case II
there’s a failure.

Unit
HRESULT // GetVariableNumber(
WCHAR *pwszVariableName,
LONG *plValue,
DMUS_SCRIPT_ERRORINFO *pErrInfo
);

The code to retrieve a variable is simple:

// Find out how many frog chances the princess has left
long lFrogs;
pScript->GetVariableNumber(L"FrogsLeft",&lFrogs,NULL);

Likewise, setting a variable is straightforward. See the code snippet

below using SetVariableObject() as an example:
HRESULT SetVariableObject(
WCHAR* pwszVariableName,
IUnknown* punkValue,
DMUS_SCRIPT_ERRORINFO* pErrInfo
);

Again, we pass the name of the variable. Since it is treated as an

object, we pass its interface pointer (of which IUnknown is always
the base). We can also pass that optional error structure (more on
that in a bit).
// Pass the script a 3D AudioPath for tracking a hopping frog
IDirectMusicAudioPath *pPath = NULL;
pPerformance->CreateStandardAudioPath(DMUS_APATH_DYNAMIC_3D,3,true,&pPath);
if (pPath)
266 n Chapter 12

{
pScript->SetVariableObject("FrogPath",pPath,NULL);
}
// Remember to release the path when done with it. The script will
// release its pointer to the path on its own.

Retrieving a variable object is a little more involved in that you need

to identify what interface you are expecting. GetVariableObject()
passes an additional parameter: the interface ID of the interface that
it is expecting.
// Retrieve a Segment that is stored in the script
IDirectMusicSegment8 *pSegment;
pScript->GetVariableObject(
"FrogScream", // Name of the variable.
IID_IDirectMusicSegment8, // IID of the interface.
(void **)&pSegment, // Address of the interface.
NULL); // Optional error info.

Content
Obviously, it is important that scripts be able to directly manipulate
the DirectMusic objects that they control. To deal with this, key
objects that you would want to manipulate from within a script all
have scripting extensions. These are the AudioPath, AudioPath Con-
figuration, Performance, Segment, and SegmentState (called a
“playing Segment”). They all exhibit methods that can be called
directly from within the script (internally, this is done via support for
the IDispatch interface). Documentation for these scripting exten-
sions can be found in the DirectMusic Scripting Reference section of
the DirectMusic Producer help file, not the regular DirectX program-
ming SDK.
It is also very important that a script be able to introduce its own
content. It’s not enough to load files externally and present them to
the script with SetVariableObject() because that implies that the
application knows about all of the objects needed to run the script,
which gets us back to the content creator writing down a long list of
instructions for the programmer, etc. Scripting should directly con-
trol which files are needed and when.
The Script object also supports linking and embedding objects at
authoring time. A linked object simply references objects that are
stored in files outside the script file. This is necessary to avoid
redundancy if an object is shared by multiple scripts. If the script is
 Scripting n 267

the sole owner of an object, it can embed it directly, which results in

a cleaner package because there are fewer files. Setting up linked or
embedded content is very simple in DirectMusic Producer. In the
project tree, just drag the objects you want to use in the script into
its Embed Runtime or Reference Runtime folder. This also automat-
ically makes sure the object loads with the script, and if the object
can be directly manipulated, it appears as a variable in the script.

II

Unit
You can have everything you need for a particular section in your
application all wrapped up in one script file, which is wonderfully
clean. Be careful, though. By default, script files automatically down-
load all their DLS and wave instruments to the synthesizer when the
script is initialized. If you have more stuff than you want downloaded
at any one time, you need to manage this directly in your scripting
code and avoid the automatic downloading.
If you ever wondered why you could not simply open a script file
in a text editor, the embedded and linked content is the reason.
Although the code portion of the script is indeed text, the linked and
embedded data objects are all binary in RIFF format, which cannot
be altered in a text editor.

Finding the Routines and Variables

The IDirectMusicScript interface includes two methods, Enum-
Routine() and EnumVariable(), which can be used to find out what is
in the script. These are very useful for writing small applications that
can scan through the script and display all routines and variables and
then directly call them. Jones does that, as you will see in a bit.
HRESULT EnumRoutine(
DWORD dwIndex, // nth routine in the script.
268 n Chapter 12

WCHAR *pwszName // Returned Unicode name of the routine.

);

EnumRoutine() simply iterates through all routines, returning their

names.
HRESULT EnumVariable(
DWORD dwIndex, // nth variable in the script.
WCHAR *pwszName // Return Unicode name of the variable.
);

EnumVariable() does the same for each variable, but it’s a little more
involved because it enumerates all declared variables as well as all
variables that were automatically created for linked and embedded
objects. There is a little trick that you can use to figure out which
type a variable is. Make a call to GetVariableLong(), and if it suc-
ceeds, the variable must be a declared variable; otherwise, it must be
a linked or embedded object. Jones uses this technique.
You can also use GetVariableObject() to search for specific object
types, since it requires a specific interface ID. Here is a routine that
will scan a script looking for all objects that support IDirectMusic-
Object and display their name and type. This should display
everything that is linked or embedded content and can be manipu-
lated as a variable.

þ Note Some content (for example, styles and DLS Collections) cannot
be directly manipulated by scripting, so they do not have variables
assigned to them.

// Search for all content variables within a script

// and display their name and type.

void ScanForObjects(IDirectMusicScript *pScript)

{
DWORD dwIndex;
HRESULT hr = S_OK;
// Enumerate through all variables
for (dwIndex = 0; hr == S_OK; dwIndex++)
{
WCHAR wzName[DMUS_MAX_NAME];
hr = pScript->EnumVariable(dwIndex,wzName);
// hr == S_FALSE when the list is finished.
if (hr == S_OK)
{
IDirectMusicObject *pObject;
 Scripting n 269

// Only objects that can be loaded from file have the

// IDirectMusicObject interface.
HRESULT hrTest = pScript->GetVariableObject(wzName,
IID_IDirectMusicObject,
(void **) &pObject,
NULL);
if (SUCCEEDED(hrTest))
{
// Success. Get the name and type and display them.
DMUS_OBJECTDESC Desc;
Desc.dwSize = sizeof (Desc);
if (SUCCEEDED(pObject->GetDescriptor(&Desc)))
{
char szName[DMUS_MAX_NAME + 50];
if (Desc.dwValidData & DMUS_OBJ_NAME)
{

}
wcstombs(szName,Desc.wszName,DMUS_MAX_NAME); II
else

Unit
{
strcpy(szName,"<unnamed>");
}
// This should be a Segment, AudioPath configuration,
// or another script because only these support IDispatch
// and can be loaded from file.
if (Desc.guidClass == CLSID_DirectMusicSegment)
{
strcat(szName,": Segment");
}
else if (Desc.guidClass == CLSID_DirectMusicAudioPathConfig)
{
strcat(szName,": AudioPath Config");
}
else if (Desc.guidClass == CLSID_DirectMusicScript)
{
strcat(szName,": Script");
}
strcat(szName,"\n");
OutputDebugString(szName);
}
pObject->Release();
}
}
}
}
270 n Chapter 12

Error Handling
Sometimes tracking errors with scripting can be frustrating. Variable
or routine names may be wrong, in which case calls to them fail.
There can be errors in the routine code, but there is no way to step
into the routines when debugging. Therefore, it helps to have a
mechanism for figuring out what went wrong.
Enter the DMUS_SCRIPT_ERRORINFO structure. As we have
seen already, CallRoutine() and all of the methods for manipulating
variables can pass this as an option.
typedef struct _DMUS_SCRIPT_ERRORINFO {
DWORD dwSize;
HRESULT hr;
ULONG ulLineNumber;
LONG ichCharPosition;
WCHAR wszSourceFile[DMUS_MAX_FILENAME];
WCHAR wszSourceComponent[DMUS_MAX_FILENAME];
WCHAR wszDescription[DMUS_MAX_FILENAME];
WCHAR wszSourceLineText[DMUS_MAX_FILENAME];
} DMUS_SCRIPT_ERRORINFO;

As you can see, this structure can provide a wealth of information to

find out where something went wrong, especially if the error was in
the script code. The SDK covers each of these fields quite well, so
let’s skip that. If you are curious, look in DirectMusic>DirectMusic
C/C++ Reference>DirectMusic Structures at Microsoft’s web site.

Script Tracks
You can also trigger the calling of script routines by using a Script
Track in a Segment. This is very powerful because it gives the
opportunity to have time-stamped scripting. You can even use script-
ing to seamlessly control the flow of music playback by calling
routines at decision points in the Segments to decide what to play
next based on current state variables.
Script Tracks are very straightforward to author and use. In
DirectMusic Producer, open a Segment and use the Add Tracks com-
mand to add a Script Track to it. At the point in the timeline where
you want the script routine called, insert a script event. Then choose
from a pull-down menu in the Properties window which routine from
which script to call. Each script event also has options for timing.
The script routine can be called shortly ahead of the time stamp or
 Scripting n 271

exactly at it. The former is useful if you want to make decisions

about what to play once the time stamp is reached. For example, you
would not want to call the routine to queue a new Segment at exactly
the time it should play, since latency would make that impossible.
The Script Track is quite flexible. It supports unlimited script
calls. It allows calls to more than one script, and no two calls have to
use the same timing options.

Script Language II
DirectMusic’s scripting lets you choose between two language

Unit
implementations. It does this by internally supporting a standard
COM interface for managing scripted language, called IActiveScript.
Theoretically, that means that you can use any language supported
by IActiveScript, which includes a wide range of options from
JavaScript to Perl. However, DirectMusic Producer offers content
creators exactly two options, VBScript and AudioVBScript. This is
just fine because they really are the best choices.
VBScript is a standard scripting implementation that Microsoft
provides as part of the Windows operating system. VBScript is a
very full-featured language. However, it is also big, requiring close to
a megabyte to load.
AudioVBScript is a very light and nimble basic scripting language
that was developed specifically for DirectMusic scripting. It is fast
and very small and ships as part of the API. However, it does not
have many of the more sophisticated features found in VBScript.
Typically, scripters should use AudioVBScript. If the scripter runs up
against a brick wall because AudioVBScript does not have a feature
they need (like arrays, for example), then the scripter can simply
switch to VBScript and continue. Since AudioVBScript is a subset of
VBScript, this should be effortless.

þ Note Documentation for the AudioVBScript language and all

scripting commands are located in the DirectMusic Producer help files
under Creating Content>Script Designer>AudioVBScript Language, not
the regular DirectX help.
272 n Chapter 12

Adding Scripting to Jones

As it turns out, adding scripting support to Jones is not hard at all.
We would like to be able to:
n Load a script
n Display all script routines
n Call routines
n Display all script variables
n Set variable values
Jones’ scripting solution adds a series of list boxes on the left side of
the Jones window, as seen in Figure 12-2. The top box lists all cur-
rently loaded scripts. Underneath it, two list boxes display all
routines and variables for the currently selected script. To call a rou-
tine, simply double-click on it or, alternately, select it and press the
Call button. To view a variable, click on it once and view it in the text
box. To change it, edit its value in the text box.

Figure 12-2: Jones with scripting support.

 Scripting n 273

Jones Data Structures

We add a new class, CScript, to our audio library. CScript keeps track
of one instance of a loaded script via its IDirectMusicScript interface.
I considered creating data structures to manage the lists of routines
and variables, but the enumeration routines provided by IDirect-
MusicScript are so straightforward, it seemed silly to do so.
We start with the relatively simple CScript class. It tracks the
name and script interface and provides access methods to both. The
biggest item is the initialization method.
class CScript : public CMyNode
{
public:
CScript(); II
~CScript();
CScript* GetNext() { return (CScript*)CMyNode::GetNext();}

Unit
IDirectMusicScript *GetScript() {return m_pScript; };
void Init(IDirectMusicScript *pScript,
IDirectMusicPerformance8 *pPerformance);
char *GetName() { return m_szName; };
private:
IDirectMusicScript * m_pScript;
char m_szName[DMUS_MAX_NAME]; // Name, for display
};

CScript is managed in a link list by CScriptList. We add one instance

of CScriptList to CAudio to manage all of the scripts, and we add a
method for loading scripts:
CScript *LoadScript(WCHAR *pwzFileName); // Load a script from file.
CScriptList m_ScriptList; // List of loaded scripts.

That is it.

Loading the Script

To load a script, click on the Open… button. This opens the standard
file dialog and prompts you to load a script file with the extension
.spt.
274 n Chapter 12

Select a script file (I would recommend SimpleScript.spt in the

Media directory) and click on Open…. The script name is placed in
the script list, and its routines and variables appear in the two lists
below.
Let’s look at LoadScript(). It uses the Loader to read a script
from the passed file path. Once LoadScript() successfully reads the
script file, it calls the CScript’s Init() method, which prepares the
script for running.
CScript *CAudio::LoadScript(WCHAR *pwzFileName)
{
WCHAR wzSearchDirectory[DMUS_MAX_FILENAME];
wcscpy(wzSearchDirectory,pwzFileName);
WCHAR *pwzEnd = wcsrchr(wzSearchDirectory,'\\');
if (pwzEnd)
{
// If we have a directory path, use it to set
// up the search directory in the Loader.
// The Loader will look here for linked files,
// including Segments, Styles, and DLS instruments.
*pwzEnd = 0;
m_pLoader->SetSearchDirectory(GUID_DirectMusicAllTypes,
wzSearchDirectory,FALSE);
}
CScript *pScript = NULL;
IDirectMusicScript8 *pIScript;
// Now, load the script.
if (SUCCEEDED(m_pLoader->LoadObjectFromFile(
CLSID_DirectMusicScript, // Class ID for script.
IID_IDirectMusicScript8, // Interface ID for script.
pwzFileName, // File path.
(void **) &pIScript))) // Returned IDirectMusicScript.
{
// Create a CScript object to manage it.
pScript = new CScript;
if (pScript)
{
// Initialize and add to list.
pScript->Init(pIScript,m_pPerformance);
m_ScriptList.AddTail(pScript);
}
pIScript->Release();
}
return pScript;
}
 Scripting n 275

CScript::Init() is similarly simple. It gets the script ready to run by

calling the script’s IDirectMusicScript::Init() routine and then
retrieving the script’s name so it can be displayed.
void CScript::Init(IDirectMusicScript *pScript,
IDirectMusicPerformance8 *pPerformance)
{
m_pScript = pScript;
pScript->AddRef();
// Initialize the script for this performance.
// In addition to providing a global performance pointer
// for the scripting, this automatically downloads
// all waves and DLS instruments, if this option was set
// in the script at authoring time (which is usually the case).
pScript->Init(pPerformance,NULL);
// Get the object descriptor from the script. This includes the name.
IDirectMusicObject *pIObject = NULL; II
pScript->QueryInterface(IID_IDirectMusicObject,(void **)&pIObject);

Unit
if (pIObject)
{
DMUS_OBJECTDESC Desc;
Desc.dwSize = sizeof(Desc);
pIObject->GetDescriptor(&Desc);
pIObject->Release();
m_szName[0] = 0;
if (Desc.dwValidData & DMUS_OBJ_NAME)
{
wcstombs( m_szName, Desc.wszName, DMUS_MAX_NAME );
}
else if (Desc.dwValidData & DMUS_OBJ_FILENAME)
{
wcstombs( m_szName, Desc.wszFileName, DMUS_MAX_NAME );
// Get rid of any file path.
char *pszName = strrchr( m_szName,'\\');
if (pszName) strcpy(m_szName,++pszName);
}
else
{
strcpy( m_szName,"Script (no name)");
}
}
}
276 n Chapter 12

Enumerating and Calling Routines

Once the script is loaded, Jones displays the list of available script
routines in the Routines list box. To run a routine, double-click on its
name in the Routines list.

To create that list of routines, Jones calls the IDirectMusicScript::

EnumRoutine() method. I thought of wrapping this somehow in
CScript for completeness, but that seemed silly. So, here’s the code
inside Jones’ UI that enumerates the routines in the process of creat-
ing the list box. Notice that EnumRoutine() signals the end of list
with S_FALSE, not an error condition.
// Enumerate through the script's routines.
DWORD dwIndex;
HRESULT hr = S_OK;
for (dwIndex = 0; hr == S_OK; dwIndex++)
{
// Names are returned in Unicode.
WCHAR wzName[DMUS_MAX_NAME];
hr = pScript->GetScript()->EnumRoutine(dwIndex,wzName);
if (hr == S_OK)
{
// Convert from Unicode to ASCII and add to list box.
char szName[DMUS_MAX_NAME];
wcstombs( szName, wzName, DMUS_MAX_NAME );
m_ctScriptRoutineList.AddString(szName);
}
}

To play the routine, Jones retrieves the routine name from the rou-
tine list and uses it to call CallRoutine().
// Convert routine name to Unicode.
WCHAR wszName[MAX_PATH];
mbstowcs(wszName,szName,MAX_PATH);
 Scripting n 277

// Use the Unicode name to call the routine.

pScript->GetScript()->CallRoutine(wszName,NULL);

Enumerating and Accessing Variables

The list of variables is displayed below the routines.

II
Click on a variable to show its value in the edit box. Type over the

Unit
variable value with a new value and Jones automatically sends the
new value to the script.
Scripting supports three ways to access variables: objects, vari-
ants, and numbers. However, to keep things simple in Jones, we are
only supporting numbers. So, when we enumerate through the vari-
ables using IDirectMusicScript::EnumVariable(), we need to make
sure that the variable is valid as a number and we want to retrieve
the current value, which we can store in our list. We do both with a
call to IDirectMusicScript::GetVariableNumber(). If the call suc-
ceeds, we can display the variable.
DWORD dwIndex;
HRESULT hr = S_OK;
for (dwIndex = 0; hr == S_OK; dwIndex++)
{
WCHAR wzName[DMUS_MAX_NAME];
// Enumerate through all variables
hr = pScript->GetScript()->EnumVariable(dwIndex,wzName);
if (hr == S_OK)
{
long lData = 0;
// But verify that each variable is capable of providing a number.
// If not, ignore it.
hr = pScript->GetScript()->GetVariableNumber(wzName,&lData,NULL);
if (SUCCEEDED(hr))
{
// Success. Convert to ASCII and place in the list box.
char szName[DMUS_MAX_NAME] = "";
wcstombs( szName, wzName, DMUS_MAX_NAME );
278 n Chapter 12

int nIndex = m_ctScriptVariableList.AddString(szName);

m_ctScriptVariableList.SetItemData(nIndex,(DWORD)lData);
}
}
}

Next, we need to be able to edit the variable. When a variable is

selected in the list, Jones pulls the variable data from the list (previ-
ously retrieved with GetVariableNumber()) and places it in the edit
box. Then, in response to user edit, it sets the new value.
// Get the name of the variable.
char szName[MAX_PATH];
m_ctScriptVariableList.GetText(nSelect,szName);
// Convert the name to Unicode.
WCHAR wszName[MAX_PATH];
mbstowcs(wszName,szName,MAX_PATH);
// Use the name to set the variable to the new value.
pScript->GetScript()->SetVariableNumber(wszName,m_lScriptVariable,NULL);

A Scripting Sample
To really understand programming with scripting, it helps to mess
around with the scripting side of things too. To that end, I created a
wacky little script called SimpleScript.spt.

Bad Dream
SimpleScript endeavors to provide a soundtrack to a very strange
scene. Imagine a mechanical world filled with moving metal parts,
and large train-like objects fly by at random intervals with a spray of
steam and grinding metal. You are being hunted by a very annoying
translucent wasp made of cellophane. As the story progresses, there
are more and more mechanical objects to dodge, and as you find
yourself increasingly cornered, your level of hysteria increases.
Occasionally, you have a good idea that causes time to stand still, but
then chaos falls back around your ears and the chase ensues. Even-
tually, you wake up in a cold sweat, and it’s over. To build this
soundscape, we need the following:
n Theme music that plays forever and responds to both changes in
“activity” and “hysteria”
n A routine to start the music
 Scripting n 279

n Variables and routines for updating the activity and hysteria

n A routine to play when the wasp flies by
n A routine to play when a train is encountered
n A routine to play when there is a “really good idea”
n A routine to finish the music when the scene is over
The SimpleScript example incorporates solutions for all of these.
Load it into Jones, so you can call each routine as we discuss it. If
you are feeling adventurous, open it directly in DirectMusic Pro-
ducer and study it in greater depth as we go along.
Before declaring any routines, SimpleScript declares two vari-
ables that will be used to set the Activity and Hysteria levels as the
scene plays. II
' Variables set by the application:

Unit
dim Activity ' Game supplied Activity level
dim Hysteria ' Game supplied Hysteria level

Theme Music
The Segment SkaNSeven.sgt supplies the theme music. It uses a
Style for playback (thanks to Microsoft’s Kelly Craven for creating a
great Style), which has the advantage that Style Segments can
respond to global groove level changes. Because Style Segments are
chord based, other musical effects can track them harmonically.
SkaNSeven.sgt is 42 measures long with measures 10 through 42
looping infinitely, so once it starts playing, it just loops and loops. In
truth, you’d want more variety, but hey, this is called “SimpleScript”
after all.
The first routine, StartLevel, is called by the application when
we first enter the scene. StartLevel establishes the initial Activity
and Hysteria levels by setting these values to zero and then calling
the routine Update (more on that in a second.) Finally, StartLevel
plays SkaNSeven.sgt to get the music looping.
' StartLevel is called when we enter the scene.
sub StartLevel
Activity = 0 ' Init the Activity
OldActivity = 0 ' Init the previous as well
Hysteria = 0 ' Init the Hysteria
Update ' Now use these to set tempo and groove
' Finally, play the music
280 n Chapter 12

SkanSeven.Play AtMeasure
end sub

Go ahead and click on the StartLevel command to hear the music

start.

Setting Activity and Hysteria

Two variables, Activity and Hysteria, are used to control the overall
groove level and tempo of the theme music as it plays. To change
one of these, the application (or Jones) calls IDirectMusicScript::
SetVariableNumber() with the name of the variable (Activity or Hys-
teria) and that variable’s new value.
But changing the variable is not enough. The script can’t actively
do anything with the new information unless one of its routines is
called. So, the script has a routine, Update, that should be called
whenever either variable has changed.
Update first makes sure that Activity and Hysteria are within
reasonable ranges. Then, if it sees that the Activity increased, it
plays a short secondary Segment, called Change.sgt, on a beat
boundary. Then, Update applies formulas to translate the two vari-
ables into appropriate groove level and tempo modifiers.
Why does it play the Segment? When the groove level is
changed, the effect doesn’t take until the next pattern boundary
within the Style, which is usually every measure. Since something
just happened that is supposed to increase the musical activity, it
would seem odd to wait up to a whole measure to hear the music
react. Yet, cutting immediately would ruin the timing. So, we play
the change Segment to add a flurry of activity while we wait for the
groove level to hit at the next measure boundary.
' Every time the Activity or Hysteria variables are updated by
' the application, it needs to call Update so the script can act
' accordingly.
sub Update
' First, make sure everything is in range.
if (Hysteria < 0) then
Hysteria = 0
end if
if (Hysteria > 10) then
Hysteria = 10
end if
if (Activity < 0) then
Activity = 0
 Scripting n 281

end if
if (Activity > 6) then
Activity = 6
end if
' If the activity is increasing, play a motif.
if (OldActivity < Activity) then
Change.play IsSecondary+AtBeat
end if
' Store the activity for next time around.
OldActivity = Activity
' Then, set the levels
SetMasterGrooveLevel Activity*25 - 50
SetMasterTempo Hysteria*2 + 98
end sub

To test this, first change the Activity or Hysteria variable, and then
run the Update routine. In Jones, this means clicking on the variable II
name, editing its value, and then double-clicking on Update.

Unit
Wasp Flies By
When the wasp flies by, the routine Wasp can be called. Wasp plays a
secondary Segment written with a Pattern Track, so it can transpose
on top of the current chord and key. The Segment has many different
variations so that the melody it plays is always a little different. This
wasp has personality! On the other hand, when Activity is high, it
seems intuitive that the wasp should be a little louder and little
angrier. So, the Wasp routine checks the current Activity level before
deciding which of two Segments to play.
' When a Wasp flies by in the game, call the Wasp routine which plays a secondary
' Segment. If the Activity is above 2, play a slightly louder and more intense Wasp.
' In either case, align with the beat.
sub Wasp
if (Activity > 2) then
BigWasp.play IsSecondary+AtBeat
else
LittleWasp.play IsSecondary+ AtBeat
end if
end sub

Train
Now let’s script the behaviors for the train. When a train lumbers by
and the user gets close to the Tracks, the routine NearTracks is
called to play something inspiring. This is a simple sequence with
282 n Chapter 12

some weird train-like sound effects. Since this isn’t musical at all, it
can play immediately and not sound odd.
' When close to train tracks, call NearTracks.
' This isn't rhythmically attached to the music, so
' play it immediately.
sub NearTracks
Train.play IsSecondary+AtImmediate
end sub

Thinking
I don’t know about you, but when I have an original thought, the
world stops and takes notice. Time stands still, multicolored flashing
lights flutter around my head, and fog comes out of my nose. The
Thinking routine attempts to recreate this otherworldly experience.
It does so in an interesting way. It plays a Segment, Pause.sgt, that is
a controlling Segment with a low groove level accompanied by break
and fill embellishments. These alter the theme music by causing it to
track to the low groove level and play the break and fill embellish-
ments, while continuing with its own chord progression. Pause also
plays a few drone-like notes and eerie sounds to telegraph the expe-
rience of having a head bursting full of profound, karmic thoughts.
But, again, with a groove change, we have a situation where the
effect is delayed until the next measure boundary. We need to hear
something sooner! Profound thoughts like mine can’t wait. The
Idea.sgt secondary Segment fills the breach with a brief chorus of
angelic voices.
' When it's time to stop and think, call Thinking.
sub Thinking
' Play a small motif to carry us over since
' the controlling Segment is aligned to a measure.
Idea.play IsSecondary+AtBeat
' Now, the controlling Segment, which plays some
' embellishments and drone sounds while altering
' the groove level.
Pause.play IsControl+AtMeasure
end sub
 Scripting n 283

þ Note There are actually ways to force a groove level or

embellishment to occur sooner. You can cause an immediate
invalidation (see IDirectMusicPerformance::Invalidate), or you can
transition to a new Segment on a beat boundary while making the
groove change.

Enough Already!
When it’s clear that the music itself is causing the hysteria, you can
opt out by calling the EndLevel routine. This takes advantage of
DirectMusic composition technology, which dynamically writes a
transition Segment and plays it to lead into the final Segment,
ScaNSevenEnd.sgt. It’s worth listening to this a few times to see
how it is able to pick chord progressions that wrap up the music II
quite handily, regardless of where the theme was at the moment you
decided to leave.

Unit
' EndLevel is called when we are done.
sub EndLevel
' Play the finish Segment but precede it with a dynamically
' composed transition with a fill and chords to modulate into it.
SkanSevenEnd.Play AtMeasure+PlayFill+PlayModulate
end sub

þ Note If you are previewing in Jones, you will notice that the visual
display temporarily shows nothing when the composed transition
Segment plays. Magic? I like to think everything that comes out of the
composition technology is magic, but there’s a more grounded reason.
Jones inserts a special display track in each Segment when it loads it.
This special Track gives Jones the ability to follow the Segments as they
play. The transition Segment is created on the fly within DirectMusic, so
there is no opportunity for Jones to intercept it. But, if you want to
believe it’s magic…

Enough of my foray into sound design. If nothing else, this exercise

furthers the point that I should stick to programming and leave
sound design to the pros. (And dream interpretation, too.)

If your project involves music and audio design that are done by
somebody other than the programmer (read: YOU!), it pays to use
scripting. It frees the participants to work more efficiently and faster,
and there’s no question that you will end up with a product that is
284 n Chapter 12

significantly more sophisticated and well honed. Not to mention it

will save you, as programmer, additional headaches!
On the rare occasion you are doing it all yourself, it might be a
toss up only because scripting is harder to debug than straight C++
in that you can’t single step through the code, but that’s a minor
point.
Chapter 13

Sound Effects

W e have talked a bit about how wonderful DirectX Audio is for

sound effects work, but we have yet to really focus on any-
thing outside of music making. Although the Loader, Segment,
AudioPath, and scripting are all important prerequisites for creating
immersive audio environments with DirectX Audio, our examples
have primarily focused on musical applications. Now we will spend
some time focusing specifically on using DirectX Audio for sound
effects.
There are some big reasons why you can benefit dramatically by
playing sound effects through AudioPaths instead of the traditional
approach of loading and playing sounds directly in DirectSound
Buffers:
n There is a performance advantage. 3D spatialization, whether in
software or hardware, is significantly more expensive than soft-
ware mixing of individual sounds on today’s CPUs. Even with
hardware acceleration, the overhead of shipping the audio data
over the bus can be more expensive than a simple CPU mix. The
AudioPath mechanism mixes one or more sounds in one or more
Segments and sends them all together to one 3D channel on the
sound card. In contrast, every single sound in the Buffer
approach has its own 3D channel. This approach ultimately costs
quite a bit more when sounds are layered, and to add insult to
injury, it is more complex to program.
n Using AudioPaths allows for more densely layered sound envi-
ronments because the limiting factor is not the number of sounds
that can play at once, but rather the number of 3D positions that
the sounds can play through.

285
286 n Chapter 13

n The AudioPath approach is content driven. The authoring format,

with scripting, variation, and more, is much richer than wave
files. Therefore, the sound designer can work on complex sound
schemes without tying up the programmer.
Naturally, working with AudioPaths is not without its unique chal-
lenges. I hope that we will address all of them in this chapter.
Jones is not the greatest place to demonstrate working with 3D
sound effects in a game-like environment, so our programming treat
this time, called Mingle, is a reproduction of a lively cocktail party
with some truly fascinating people milling around. We will start by
walking through the essentials of programming sound effects with
DirectX Audio and then spend some time with critical issues that can
make or break your audio design. Lastly, we will dig into Mingle.

Sound Effects Programming 101

In general, sound effects programming with DirectX Audio is very
straightforward. We already touched on it a bit in Chapter 8 when we
added 3D to our Hello World application, and the chapters on Seg-
ments and AudioPaths covered just about all the basics for loading
and playing sounds on AudioPaths. Here is a quick walk through the
specific things you must do to get sounds playing effectively with
real-time control of their 3D position. We start with the one assump-
tion that you created the Loader and Performance and have
initialized the Performance.

Allocating a Sound Effects AudioPath

Although any possible AudioPath configuration can be defined in an
AudioPath configuration file, for most sound effects work, three pre-
defined types do the job admirably.
n DMUS_APATH_DYNAMIC_3D: Use this for sounds that need
to be positioned in 3D space. All sounds played on the AudioPath
are mixed and then streamed through one DirectSound 3D
Buffer for 3D spatialization. The 3D controls of the DirectSound
Buffer can be directly manipulated by the application to move the
mix in space.
 Sound Effects n 287

n DMUS_APATH_DYNAMIC_MONO: Use this for mono

sounds that do not need 3D positioning. This mixes everything
played on the AudioPath into a Mono DirectSound Buffer. Note
that these can still be panned within the stereo spectrum.
n DMUS_APATH_DYNAMIC_STEREO: Use this for stereo
sounds. Again, everything is mixed down into one Buffer — this
time in stereo.
To create one of these AudioPaths, call IDirectMusicPerformance::
CreateStandardAudioPath() and pass the type of AudioPath you want.
Give it the number of pchannels you will need on the AudioPath. The
number of pchannels is really determined by how many pchannels
are used in the Segments that will play on the path. For sound effects
work, this is usually a small number. For example, if you are just II
playing wave files, only one pchannel is needed.

Unit
Here is the code to create a 3D AudioPath with four pchannels.
IDirectMusicAudioPath *pPath = NULL;
m_pPerformance->CreateStandardAudioPath(
DMUS_APATH_DYNAMIC_3D, // Create a 3D path
4, // Give it four pchannels
true, // Activate it for immediate use.
&pPath); // Returned path.

Later, when done with the AudioPath, just use Release() to get rid of
it:
pPath->Release();

Working with AudioPaths is quite different from programming

directly with DirectSound Buffers. Importantly, you can play any
number of sounds on the same AudioPath. This means that instead of
creating a 3D buffer for each individual sound, you can create one
AudioPath and play any number of Segments on it, and each Segment
can have one or more sounds of its own that play at once. This para-
digm shift is important to remember as you design your sound
system. Forgive me if I sound a little redundant with this fact, but it
is a good one to hammer home.
288 n Chapter 13

Playing Sounds on the AudioPath

First, load the sound into a Segment. The easiest way to do this is to
use the Loader’s LoadObjectFromFile() method, which takes a file
name and returns the loaded object (in this case a Segment):
IDirectMusicSegment8 *pSegment = NULL;
// Now, load the Segment.
m_pLoader->LoadObjectFromFile(
CLSID_DirectMusicSegment, // Class ID of Segment.
IID_IDirectMusicSegment8, // Segment interface.
pwzFileName, // File path.
(void **) &pSegment); // Returned Segment.

To play the Segment on the AudioPath, pass both the Segment and
AudioPath to the Performance’s PlaySegmentEx() method:
IDirectMusicSegmentState8 *pSegState;
hr = m_pPerformance->PlaySegmentEx(
pSegment, // The Segment
NULL,NULL, // Ignore these
DMUS_SEGF_SECONDARY, // Play as a secondary Segment
0, // No time stamp.
&pSegState, // Optionally, get a SegState.
NULL, // No prior Segment to stop.
pPath); // Use AudioPath, if supplied.

Remember that there is not a limit to how many Segments can play
on how many AudioPaths. One Segment can play on multiple
AudioPaths, and multiple Segments can play on one AudioPath.
If the Segment needs to be stopped for some reason, there are
three ways to do it:
n Stopping the Segment State: This stops just the one instance
of the playing Segment, so all other Segments playing on the
AudioPath continue. Use this to terminate one sound on an
object without affecting others.
n Stopping the Segment: This stops all instances of the Seg-
ment, no matter which AudioPaths they are playing on. This is
the simplest as well as the least useful option.
n Stopping everything in the AudioPath: This stops all Seg-
ments currently playing on the AudioPath. This is very useful for
sound effects work.
 Sound Effects n 289

In all three cases, call the Performance’s StopEx() method and pass
the Segment, Segment State, or AudioPath as the first parameter.
StopEx() figures out which parameter it has received and terminates
the appropriate sounds. In these two examples, we first pass the
Segment State and then the AudioPath:
// Stop just the one instance of a playing Segment.
m_pPerformance->StopEx(pSegState,NULL,0);
// Stop everything on the AudioPath.
m_pPerformance->StopEx(pPath,NULL,0);

Activating and Deactivating the AudioPath

Even when all Segments on an AudioPath have stopped playing, the
AudioPath continues to use system resources. This primarily II
involves the overhead of streaming empty data down to the hardware
buffer. When there are many hardware buffers, this can be a substan-

Unit
tial cost. It is a good idea to deactivate AudioPaths that are currently
not in use. Deactivation does not release the AudioPath resources. It
keeps them around so the AudioPath is ready to go the next time you
need it, but it does disconnect the CPU and I/O overhead. Given that
3D buffers can be somewhat expensive on some hardware configura-
tions, deactivation of the AudioPath is a good feature to use.
One method on IDirectMusicAudioPath, Activate(), handles both
activation and deactivation. Here is example code for first deactivat-
ing and then activating an AudioPath:
// Deactivate the AudioPath
pPath->Activate(false);
// Activate the AudioPath
pPath->Activate(true);

Controlling the 3D Parameters

Once sounds are playing on an AudioPath, it really gets fun because
now you can move them around in space by manipulating the 3D
interface of the DirectSound Buffer at the end of the AudioPath. To
do this, directly control the 3D Buffer itself via the IDirectSound-
3DBuffer8 interface, which can be retrieved via a call to the
AudioPath’s GetObjectInPath() method.
The IDirectSound3DBuffer8 interface has many powerful
options that you can use to manage the sound image. Of obvious
290 n Chapter 13

interest are the commands to set the position and velocity. You
should be acquainted with all of the features of the 3D Buffer, how-
ever. These include:
n Position: Position is the single most important feature. Without
it, you cannot possibly claim to have 3D sound in your applica-
tion. Get and set the position of the object in 3D space.
n Velocity: Every object can have a velocity, which is used to cal-
culate Doppler shift. DirectSound does not calculate velocity for
you, which it theoretically could by simply measuring the change
in distance over time because that would cause a delay in any
velocity change. Therefore, you must calculate it directly and set
it for each Buffer if you want to hear the Doppler effect.
n Max distance and min distance: These set the range of dis-
tance from the listener at which sounds are progressively attenu-
ated. Sounds closer than min distance cease to increase in
volume as they get closer. Sounds farther than max distance
cease to get quieter as they get farther away.
n Cone angle, orientation, and outside volume: A very sophis-
ticated feature is the ability to define how an object projects its
sound. You can specify a cone of sound that emits from an object.
Cone orientation sets the direction in 3D space. Angle sets the
width of an inner and outer cone. The inner cone wraps the
space that plays at full volume. The outer cone marks the bound-
ary where sound plays at outside volume. The area between the
inner and outer cones gradually attenuates from full volume to
outside volume.
n Mode: You can also specify whether the sound position is rela-
tive to the listener (more on that in a second) or absolute space
or centered inside the listener’s head. I’m not particularly fond of
voices inside my head, so I avoid that last choice.
The DirectX SDK has excellent documentation on working with the
DirectSound 3D Buffer parameters, so I am not going to spend a lot
of time on this subject beyond the important position and velocity.
Here is some example code that sets the 3D position of an AudioPath
(more on setting velocity in our application example later in this
chapter):
 Sound Effects n 291

// D3D vector needs to be initialized.

D3DVECTOR VPosition;
// We'll be using an IDirectSound3DBuffer interface.
IDirectSound3DBuffer *p3DBuffer = NULL;
// Use GetObjectInPath to retrieve the Buffer.
if (SUCCEEDED(m_pAudioPath->GetObjectInPath(
0,
DMUS_PATH_BUFFER,
0,
GUID_All_Objects,0,
IID_IDirectSound3DBuffer,
(void **)&p3DBuffer)))
{
// Okay, we got the 3D Buffer. Position it.
p3DBuffer->SetPosition(VPosition.x,VPosition.y,VPosition.z,DS3D_IMMEDIATE);
// Then let go of it.

}
p3DBuffer->Release(); II

Unit
Likewise, you can control any of the other 3D parameters on the
Buffer by using the IDirectSound3DBuffer8 interface. Remember
that you can adjust any of the regular DirectSound Buffer parame-
ters, like frequency, pan, and volume, in a similar way with the
IDirectSoundBuffer8 interface.

Controlling the Listener

In order for the 3D positioning of the sound objects to make any
sense, they need to be relative to an imaginary person whose view-
point corresponds with the picture we see on the monitor. In other
words, the ears of the viewer should be positioned and oriented in
much the same way as the eyes. This is called the “listener,” as
opposed to the visual “viewer.” Once the listener information is pro-
vided, DirectSound is able to correctly calculate the placement of
each 3D sound relative to the listener and render it appropriately.
The listener provides a series of parameters that you should
become acquainted with as you develop your 3D sound chops:
n Position: This determines the current position of the listener in
3D space.
n Orientation: This determines the direction the listener is fac-
ing. Orientation is described by two 3D vectors. One sets the
direction faced, and the second sets the rotation around that vec-
tor. Orientation addresses the question of which direction the
292 n Chapter 13

listener is facing and whether the listener is upside down, look-

ing sideways, or right side up.
n Velocity: This determines the speed at which the listener is
moving through space. This is combined with the velocities of
the 3D objects to determine their Doppler shift.
n Distance Factor: By default, positions are measured in meters.
The Distance Factor is simply a number multiplied against all
coordinates to translate into a larger or smaller number system.
For example, you might want to work in feet instead of meters.
Or, your game takes place in outer space and you are measuring
Aus instead of meters. Hmmm, for calculating Doppler, how long
does it take a sound to travel one light year in outer space? (I
sense that this is a trick question…)
n Doppler Factor: By default, Doppler is calculated technically
correctly based on the velocities provided for each object and the
listener. For effect, it might be desirable to exaggerate the Dopp-
ler. Doppler Factor multiplies the Doppler effect.
n Rolloff Factor: The Rolloff Factor controls the rate at which
sounds attenuate depending on their distance from the listener.
Use this to ignore the sound, roll it off, exaggerate it, or give it
the same effect as in the real world.
The listener provides methods for getting and setting each of these
parameters independently as well as two methods, SetAllPar-
ameters() and GetAllParameters(), for accessing them all at once via
a structure, DS3DLISTENER.
typedef struct {
DWORD dwSize;
D3DVECTOR vPosition;
D3DVECTOR vVelocity;
D3DVECTOR vOrientFront;
D3DVECTOR vOrientTop;
D3DVALUE flDistanceFactor;
D3DVALUE flRolloffFactor;
D3DVALUE flDopplerFactor;
} DS3DLISTENER, *LPDS3DLISTENER;

To access the Listener, use the AudioPath’s GetObjectInPath()

method. Since the Listener belongs to the primary Buffer, find it at
the DMUS_PATH_PRIMARY_BUFFER stage. The following
 Sound Effects n 293

example gets the Listener via an AudioPath and, for grins, adjusts
the Doppler Factor.
IDirectSound3DListener8 *pListener = NULL;
pPath->GetObjectInPath(0,
DMUS_PATH_PRIMARY_BUFFER, // Retrieve from primary Buffer.
0,GUID_All_Objects,0, // Ignore object type.
IID_IDirectSound3DListener8, // Request the listener interface.
(void **)&pListener);
if (pListener)
{
DS3DLISTENER Data;
Data.dwSize = sizeof(Data);
// Read all of the listener parameters.
pListener->GetAllParameters(&Data);
// Now change something for the sake of this example.
Data.flDopplerFactor = 10; // Really exagerate the Doppler. II
pListener->SetAllParameters(&Data,DS3D_IMMEDIATE);
pListener->Release();

Unit
}

Getting Serious
Okay, we have covered everything you need to know to get DirectX
Audio making sound. As you start to work with this, though, you will
find that there are some big picture issues that need to be sorted out
for optimal performance.
n Minimize latency: By default, the time between when you start
playing a sound and when you hear it can be too long for sudden
sound effects. Clearly, that needs fixing.
n Manage dynamic 3D resources: What do you do when you
have 100 objects flying in space and ten hardware AudioPaths to
share among them all?
n Keep music and sound effects separate: How to avoid clash-
ing volume, tempo, groove level, and more.

Minimize Latency
Latency is the number one concern for sound effects. For music, it
has not been as critical, since the responsiveness of a musical score
can be measured in beats and sometimes measures. Although the
intention was to provide as low latency as possible for both music
294 n Chapter 13

and sound effects, DX8 shipped with an average latency of 85 milli-

seconds. That’s fine for ambient sounds, but it simply doesn’t work
for sounds that are played in response to sudden actions from the
user. Obvious examples of these “twitch” sound effects would be
gunfire, car horns, and other sounds that are triggered by user
actions and so cannot be scheduled ahead of time in any way. These
need to respond at the frame rate, which is usually between 30 and
60 times a second, or between 33 and 16 milliseconds, respectively.
Fortunately, DX9 introduces dramatically improved latency. With
this release, the latency has dropped as low as 5ms, depending on
the sound card and driver. Suddenly, even the hair-trigger sound
effects work very well.
But there still are a few things you need to do to get optimal per-
formance. Although the latency is capable of dropping insanely low, it
is by default still kept pretty high — between 55 and 85ms, depend-
ing on the sound card. Why? In order to guarantee 100 percent
compatibility for all applications on all sound cards on all systems,
Microsoft decided that the very worse case setting must be used to
reliably produce glitch-free sound. We’re talking a five-year-old
Pentium I system with a buggy sound card running DX8 applications.
Fortunately, if you know your application is running on a half-
decent machine, you can override the settings and drive the latency
way, way down. There are two commands for doing this. Because the
audio is streamed from the synthesizer and through the effects
chains, the mechanism that does this needs to wake up at a regular
interval to process another batch of sound. You can determine both
how close to the output the write cursor sits and how frequently it
wakes up. The lower these numbers, the lower the overall latency.
But there’s a cost: If the write cursor is too close to the output time,
you can get glitches in the sound when it simply doesn’t wake up in
time. If the write period is too frequent, the CPU overhead goes up.
Indeed, these commands actually always existed with DirectMusic
from the very start, but they could not reliably deliver significant
results because it was very easy to drive the latency down too far
and get horrible glitching. But DX9 comes with a rewrite of the audio
sink system that borders on sheer genius. (I mean it; I am extremely
impressed with what the development team has done.) With that, we
can dial the numbers down as low as we want and still not glitch.
 Sound Effects n 295

Keep in mind, however, that there is the caveat that there’s an eso-
teric bad driver out there that could prove the exception.
The two commands set the write period and write latency and
are implemented via the IKsControl mechanism. IKsControl is a
general-purpose mechanism for talking to kernel modules and
low-level drivers. The two commands are represented by GUIDs:
n GUID_DMUS_PROP_WritePeriod: This sets how frequently
(in ms) the rendering process should wake up. By default, this is
10 milliseconds. The only cost in lowering this is increased CPU
overhead. Dropping to 5ms, though, is still very reasonable. The
latency contribution of the write period is, on average, half the
write period. So, dropping to 5ms is an average latency increase
of 2.5ms — worst case 5ms. II
n GUID_DMUS_PROP_WriteLatency: This sets the latency, in

Unit
milliseconds, to add to the sound card driver’s latency. For exam-
ple, if the sound card has a latency of 5ms and this is set to 5ms,
the real write latency ends up being 10ms.
So, total latency ends up being WritePeriod/2 + WriteLatency +
DriverLatency. Here’s the code to use this. This example sets the
write latency to 5ms and the write period to 5ms.
IDirectMusicAudioPath *pPath;
if (SUCCEEDED(m_pPerformance->GetDefaultAudioPath(&pPath))
{
IKsControl *pControl;
pPath->GetObjectInPath(0,
DMUS_PATH_PORT,0, // Talk to the synth
GUID_All_Objects,0, // Any type of synth
IID_IKsControl, // IKsControl interface
(void **)&pControl);
if (pControl)
{
KSPROPERTY ksp;
DWORD dwData;
ULONG cb;
dwData = 5; // Set the write period to 5ms.
ksp.Set = GUID_DMUS_PROP_WritePeriod ; // The command
ksp.Id = 0;
ksp.Flags = KSPROPERTY_TYPE_SET;
pControl->KsProperty(&ksp, sizeof(ksp),
&dwData, sizeof(dwData), &cb);
dwData = 5; // Now set the latency to 5ms.
ksp.Set = GUID_DMUS_PROP_WriteLatency ;
296 n Chapter 13

ksp.Id = 0;
ksp.Flags = KSPROPERTY_TYPE_SET;
pControl->KsProperty(&ksp, sizeof(ksp),
&dwData, sizeof(dwData), &cb);
pControl->Release();
}
pPath->Release();
}

Before you put these calls into your code, remember that it needs to
be running on DX9. If not, latency requests this low will definitely
cause glitches in the sound. Since you can ship your application with
a DX9 installer, this should be a moot point. But if your app needs to
be distributed in a lightweight way (i.e., via the web), then you might
not include the install. If so, you need to first verify on which version
you are running.
Unfortunately, there is no simple way to find out which version
of DirectX is installed. There was some religious reason for not
exposing such an obvious API call, but I can’t remember for the life
of me what it was. Fortunately, there is a sample piece of code,
GetDXVersion, that ships with the SDK. GetDXVersion sniffs around
making calls into the various DirectX APIs, looking for specific
features that would indicate the version. Mingle includes the Get-
DXVersion code, so it will run properly on DX8 as well as DX9.
Beware — this code won’t compile under DX8, so I’ve included a
separate project file, MingleDX8.dsp, that you should compile with if
you still have the DX8 SDK.

þ Note The low latency takes advantage of hardware acceleration in a

big way. As long as all of the buffers are allocated from hardware, the
worst-case driver latency stays low. Once hardware buffers are used up
and software emulation comes into play, additional latency is typically
added as the software emulation kicks in. However, today’s cards
usually have at least 64 3D buffers available, which, as we discuss, is
typically more than you’ll want. Nevertheless, the best low latency
strategy makes sure that the number of buffers allocated never exceeds
the hardware capabilities.

Manage Dynamic 3D Resources

So we get to the wedding scene, and there are well over 100 pigs fly-
ing overhead, each emitting a reliable stream of oinks, snorts, and
squeals over the continuous din of wing fluttering. Ooops, we only
 Sound Effects n 297

have 32 3D hardware buffers to work with. How do we manage the

voices? How do we ensure that the pig buzzing the camera is always
heard at the expense of silencing faraway pigs? And that trio of pigs
with guitars — we want to make sure we hear them regardless of
how near or far they may be.
DirectSound does have a dynamic voice management system
whereby each playing sound can be assigned a priority and voices
can swap out as defined by any combination of priority, time playing,
and distance from the listener. It’s not perfect, though. It only termi-
nates sounds. So, if a pig that is flying away from the listener is
terminated in order to make way for the hungry sow knocking over
the dessert table, it remains silent once it flies back toward the lis-
tener because there’s no mechanism to restart its sounds.
For better or worse, the AudioPath mechanism doesn’t even try
II
to handle this. You must roll your own. In some ways, this is a win

Unit
because you can completely define the algorithm to suit your needs,
and you have the extra flexibility of being able to restart with Seg-
ments or call into script routines or whatever is most appropriate for
your application. This does mean quite a bit of work. With that in
mind, the major thrust of the Mingle application deals with this exact
issue. Mingle includes an AudioPath management library that you
can rip out, revise, and replace in your app as you see fit. How does
it work? Let’s start with an overview of the design here, and then
let’s investigate in depth later when we look at the Mingle code.
There are two sets of items that we need to track:
n Things that emit sound: Each “thing” represents an independ-
ently movable object in the world that makes a noise and so
needs to be playing via one instance of an AudioPath. Each thing
is assigned a priority, which is a criteria for figuring out which
things can be heard, should there not be enough AudioPaths to
go around. There is no limit to how large the set of things can be.
There is no restriction to how priority should be calculated,
though distance from the listener tends to be a good one.
n 3D AudioPaths: This set is limited to the largest amount of
AudioPaths that can be running at one time. Each AudioPath is
assigned to one thing that plays through the AudioPath.
Since we have a finite set of AudioPaths, we need to match these up
with the things that have the highest priorities. But priorities can
298 n Chapter 13

change over time, especially as things move, so we need to scan

through the things every frame and make sure that the highest pri-
ority things are still the ones being heard. Figure 13-1 shows the
relationship of AudioPaths to things.
Once a frame, we do the following:
1. Scan through the list of things and recalculate their positions
and priorities. However, do not assign their positions yet.
2. Look for things with newly high priorities that outrank
things currently assigned to AudioPaths. Stop the older
things from playing and assign their AudioPaths to the new,
higher priority things.
3. Get the new 3D positions from the things assigned to
AudioPaths and set the AudioPath positions.
4. Start the new things, which have just been assigned
AudioPaths, to make sound.

Thing
Pri 1
3D AudioPath
Thing
Pri 3
3D AudioPath

Thing
3D AudioPath Pri 6

Thing
Pri 5

Figure 13-1: Three AudioPaths manage sounds for four things, sorted by priority.

Keep Music and Sound Effects Separate

If you think about it, sound effects and music follow very different
purposes as well as paradigms in entertainment, be it a movie, game,
or even a web site. Sound effects focus on recreating the reality of
the scene (albeit with plenty of artistic license). Music, on the other
 Sound Effects n 299

hand, clearly has nothing to do with reality and is added on top of the
story to manipulate our perception of it. This is all fine and good, but
if you have the same system generating both the music and the
sound effects, you can easily fall into situations where they are at
cross-purposes. For example, adjusting the tempo or intensity of the
music should not unintentionally alter the same for sound effects. In
a system as sophisticated as DirectX Audio, this conflict can happen
quite frequently and result in serious head scratching. Typical prob-
lems include:
n Groove level interference: Groove level is a great way to set
intensity for both music and sound effects, but you might want to
have separate intensity controls for music and sound effects. You
can accomplish this by having the groove levels on separate II
group IDs. But there has to be an easier way…

Unit
n Invalidation interference: When a primary or controlling Seg-
ment starts, it can automatically cause an invalidation of all play-
ing Segments. This is necessary for music because some
Segments may be relying on the control information and need to
regenerate with the new parameters. But sound effects couldn’t
care less if the underlying music chords changed. Worse, if a
wave is invalidated, it simply stops playing. You can avoid this
problem by playing controlling or primary Segments with the
DMUS_SEGF_INVALIDATE_PRI or DMUS_SEGF_AFTER-
PREPARETIME flags set. But there has to be an easier way…
n Volume interference: Games often offer separate music and
sound effects volume controls to the user. Global control of the
volume is most easily managed using the global volume control
on the Performance (GUID_PerfMasterVolume). That affects
everything, so it can’t be used. One solution is to call Set-
Volume() on every single AudioPath. Again, there has to be an
easier way…
Okay, I get the hint. Indeed, there is an easier way. It’s really quite
simple. Create two separate Performance objects, one for music and
one for sound effects. Suddenly, all the interference issues go out the
window. The only downside is that you end up with a little extra
work if the two worlds interconnect with each other, but even
scripting supports the concept of more than one Performance, so
that can be made to work if there is a need.
300 n Chapter 13

There are other bonuses as well:

n Each Performance has its own synth and effects architecture.
This means that they can run at different sample rates. The
downside is that any sounds or instruments shared by both are
downloaded twice, once to each synth.
n You can do some sound effects in music time, which has better
authoring support in some areas of DirectMusic Producer. Estab-
lish a tempo and write everything at that rate. This is how the
sound effects for Mingle were authored. Of course, Segments
authored in clock time continue to work well.
Mingle takes this approach. It creates two CAudio objects, each with
its own Performance. The sound effects Performance runs at the
default tempo of 120BPM without ever changing. 120BPM is conve-
nient because one measure equals one second. The music runs at a
sample rate of 48 kHz, while the sound effects run at 32 kHz. There
are separate volume controls for each. For efficiency, the two CAudio
instances share the same Loader.

Mingle
Okay, let’s have some programming fun. Imagine a cocktail party
with a decent complement of, uh, interesting people wandering
around: the snob with the running commentary, the hungry boor
noshing on every hors d’oeuvre in sight, the very irritating mosquito
person, and the guy who just can’t stop laughing. Gentle music plays
in the background, barely above the crowd noise. To experience all
this and more without inhaling any secondhand smoke, run Mingle
by selecting it from the Start menu (if you installed the CD). Other-
wise, double-click on its icon in the Unit II\Bin directory or compile
and run it from the 13_SoundEffects source directory. See Figure
13-2.
Taking up most of the Mingle window is a large square box with
a plus sign in the middle. The box is a large room, and you are the
plus marker in the middle. When Mingle first starts, there is general
party ambience, and some music starts playing quietly. Click in the
box. Ooops, you bumped into a partygoer, and he is letting you know
how he feels about that. Try again, but be more careful. Ouch!
 Sound Effects n 301

II

Unit
Figure 13-2: Mingle.

þ Note Regarding the munchkins, my two little boys walked into my

office when I was recording.

This is demonstrating low latency sound effects and 3D positioning.

Click around the box and hear how the sounds are positioned in
space.

þ Note If you are running under DX8, you should hear significant
buzzing or breakup of the sound. Mingle is really only meant to work
under DX9. Either install DX9 or rebuild Mingle with the low latency
code disabled (more on that in a bit).

Now let’s have some more serious fun. There are four boxes down
the left side, each representing a different person.
302 n Chapter 13

Click on the Talk button. This creates an instance of the person and
plops it in the party. You should immediately hear the person gabbing
and wandering around the room. Click the Talk button several more
times and additional instances of the person enter the room. The lit-
tle number box on the bottom right shows how many instances of
the particular person are yacking it up at the party. Click on the Shut
Up! button to remove a partygoer. Click on other participants, and
note that they are displayed with different colors.
Below the four boxes is an edit field that shows how many 3D
AudioPaths are allocated:

This displays how many AudioPaths are currently allocated for use
by all of the sound effects. This includes one AudioPath allocated for
the sounds that happen when you click the mouse. Mingle dynami-
cally reroutes the 3D AudioPaths to the people who are closest to
the center of the box (where the listener is). Continue to click on the
Talk buttons and create more people than there are AudioPaths.
Notice how the colored boxes turn white as they get farther from the
center. This indicates that a person is no longer being played through
an AudioPath.

To change the number of available AudioPaths, just edit the number

in the 3D Paths box. See what happens when you drop it down to just
 Sound Effects n 303

two or three. Change the number of people by clicking on the Talk

and Shut Up! buttons. How many AudioPaths does it take before you
can no longer track them individually by ear? How many before it’s
not noticeable when the dynamic swapping occurs? Keep experi-
menting, and try not to let the mosquito person irritate you too
much.
Across the bottom is a set of three sliders. These control three
of the listener parameters: Distance Factor, Doppler Factor, and
Rolloff Factor.

II

Unit
Drop the number of sounds to a reasonable number, so it’s easier to
track an individual sound, and then experiment with these sliders. If
you can handle it, Mosquito is particularly good for testing because it
emits a steady drone. Drag Distance Factor to the right. This
increases the distances that the people are moving. Notice that the
Doppler becomes more pronounced. That’s because the velocities
are much higher, since the distance traveled is greater. Drag Doppler
to the right. This increases the Doppler effect. Drag Rolloff to the
right. Notice how people get much quieter as they leave the center
area.
Finally, take a look at the two volume sliders:

These control the volumes for sound effects and music. Drag them
up or down to change the balance. Ah, much better. Drag them all
the way down and you no longer want to mangle Mingle.
That completes the tour. Mingle demonstrates several useful
things:
n Separate sound effects and music environments
304 n Chapter 13

n Volume control for sound effects and music

n Dynamic 3D resource management
n Low latency sound effects
n Background ambience
n Avoiding truncated waves
n Manipulating the listener properties
n Scripted control of sound effects
n Creating music with the composition engine
Now let’s see how they work.

Separate Sound Effects and Music Environments

Although Mingle doesn’t have the same UI as Jones, it does borrow
and continue to build on the CAudio class library for managing
DirectX Audio. In the previous projects, we created one instance of
CAudio, which managed the Performance, Loader, and all Segments,
scripts, and AudioPaths. For Mingle, we use two instances of
CAudio, which keeps the worlds of music and sound effects very sep-
arate. However, we’d like to share the Loader. Also, we’d like to set a
different sample rate and default AudioPath for each, since music and
sound effects have different requirements. So, CAudio::Init() takes
additional parameters for default AudioPath type and sample rate.
HRESULT CAudio::Init(
IDirectMusicLoader8 *pLoader, // Optionally provided Loader
DWORD dwSampleRate, // Sample rate for the Performance.
DWORD dwDefaultPath) // Optional default AudioPath.

{
// If a Loader was provided by the caller, use it.
HRESULT hr = S_OK;
if (pLoader)
{
m_pLoader = pLoader;
pLoader->AddRef();
}
// If not, call COM to create a new one.
else
{
hr = CoCreateInstance(
CLSID_DirectMusicLoader,
NULL,
 Sound Effects n 305

CLSCTX_INPROC,
IID_IDirectMusicLoader8,
(void**)&m_pLoader);
}
// Then, create the Performance.
if (SUCCEEDED(hr))
{
hr = CoCreateInstance(
CLSID_DirectMusicPerformance,
NULL,
CLSCTX_INPROC,
IID_IDirectMusicPerformance8,
(void**)&m_pPerformance);
}
if (SUCCEEDED(hr))
{
// Once the Performance is created, initialize it.
// Optionally, create a default AudioPath, as defined by
II
// dwDefaultPath, and give it 128 pchannels.

Unit
// Set the sample rate to the value passed in dwSampleRate.
// Also, get back the IDirectSound interface and store
// that in m_pDirectSound. This may come in handy later.
DMUS_AUDIOPARAMS Params;
Params.dwValidData = DMUS_AUDIOPARAMS_VOICES |
DMUS_AUDIOPARAMS_SAMPLERATE;
Params.dwSize = sizeof(Params);
Params.fInitNow = true;
Params.dwVoices = 100;
Params.dwSampleRate = dwSampleRate;
hr = m_pPerformance->InitAudio(NULL,&m_pDirectSound,NULL,
dwDefaultPath, // Default AudioPath type.
128,DMUS_AUDIOF_ALL,&Params);
}

CMingleApp has two instances of CAudio, m_Effects and m_Music.

It sets these up in its initialization. If both succeed, it opens the dia-
log window (which is the application) and closes down after that is
finished.
BOOL CMingleApp::InitInstance()
{
// Initialize COM.
CoInitialize(NULL);
// Create the sound effects CAudio. Give it a sample rate
// of 32K and have it create a default path with a stereo
// Buffer. We'll use that for the background ambience.
if (SUCCEEDED(m_Effects.Init(
NULL, // Create the Loader.
32000, // 32K sample rate.
306 n Chapter 13

DMUS_APATH_DYNAMIC_STEREO))) // Default AudioPath is stereo.

{
if (SUCCEEDED(m_Music.Init(
m_Effects.GetLoader(), // Use the Loader from FX.
48000, // Higher sample rate for music.
DMUS_APATH_SHARED_STEREOPLUSREVERB))) // Standard music path.
{
// Succeeded initializing both CAudio's, so run the window.
CMingleDlg dlg;
m_pMainWnd = &dlg;
dlg.DoModal();
// Done, time to close down music.
m_Music.Close();
}
// Close down effects.
m_Effects.Close();
}
// Done with COM.
CoUninitialize();
return FALSE;
}

Notice that the calls to CoInitialize() and CoUninitialize() were

yanked from CAudio and placed in CMingle::InitInstance(). As con-
venient as it was to put the code in CAudio, it was inappropriate,
since these should be called once.

Volume Control for Sound Effects and Music

Once we have two Performances for sound effects and music, it is
very easy to apply global volume control independently for each. Just
calculate the volume in units of 100 per decibel and use the
Performance’s SetGlobalParam() method to set it.
We add a method to CAudio to set the volume.
void CAudio::SetVolume(long lVolume)
{
m_pPerformance->SetGlobalParam(
GUID_PerfMasterVolume, // Command GUID for master volume.
&lVolume, // Volume parameter.
sizeof(lVolume)); // Size of the parameter.
}

Then, it’s trivial to add the sliders to the Mingle UI and connect
them to this method on the two instances of CAudio.
 Sound Effects n 307

Dynamic 3D Resource Management

This is the big one. We walked through the overall design earlier, so
now we can focus on the implementation in code. We add new fields
and functionality to the CAudioPath and CAudio classes and intro-
duce two new classes, CThing and CThingManager, which are used
to manage the sound-generating objects in Mingle.

CAudioPath
First, we need to add some fields to CAudioPath so it can track the
specific object (or “thing”) that it is rendering in 3D. These include a
pointer to the object as well as the object’s priority and whether a
change of object is pending. We also add a method for directly setting
the 3D position. Fields and flags are added for managing the priority
II
and status of a pending swap of objects, and an IDirectSound3D-

Unit
Buffer interface is added to provide a direct connect to the 3D
controls.
// Set the 3D position of the AudioPath. Optionally, the velocity.
bool Set3DPosition(D3DVECTOR *pvPosition,
D3DVECTOR *pvVelocity,DWORD dwApply);
// Get and set the 3D object attached to this AudioPath.
void * Get3DObject() { return m_p3DObject; };
void Set3DObject(void *p3DObject) { m_p3DObject = p3DObject; };
// Get and set the priority of the 3D object.
float GetPriority() { return m_flPriority; };
void SetPriority(float flPriority) { m_flPriority = flPriority; };
// Variables added for 3D object management.
void * m_p3DObject; // 3D tracking of an object in space.
float m_flPriority; // Priority of 3D object in space.
D3DVECTOR m_vLastPosition; // Store the last position set.
D3DVECTOR m_vLastVelocity; // And the last velocity.
IDirectSound3DBuffer *m_p3DBuffer; // Pointer to 3D Buffer interface.

m_p3DObject is the object that the AudioPath renders, and

m_flPriority is the object’s priority. The two D3D vectors are used to
cache the current position and velocity, so calls to set these can be a
little more efficient. Likewise, the m_p3DBuffer field provides a
quick way to access the 3D interface without calling GetObjectIn-
Path() every time.
308 n Chapter 13

þ Note The object stays intentionally unspecific to CAudioPath. It is just

a void pointer, so it could point to anything. (In Mingle, it points to a
CThing.) Why? We’d like to keep CAudioPath from having to know a
specific object design, which would tie it in much closer to the
application design and make this code a little less transportable.

CAudioPath::CAudioPath
The constructor has grown significantly to support all these new
parameters. In particular, it calls GetObjectInPath() to create a
pointer shortcut to the 3D Buffer interface at the end of the
AudioPath. This will be used to directly change the coordinates
every time the AudioPath moves.
CAudioPath::CAudioPath(IDirectMusicAudioPath *pAudioPath,WCHAR *pzwName)
{
m_pAudioPath = pAudioPath;
pAudioPath->AddRef();
wcstombs(m_szName,pzwName,sizeof(m_szName));
m_lVolume = 0;
// 3D AudioPath fields follow...
m_vLastPosition.x = 0;
m_vLastPosition.y = 0;
m_vLastPosition.z = 0;
m_vLastVelocity.x = 0;
m_vLastVelocity.y = 0;
m_vLastVelocity.z = 0;
m_flPriority = FLT_MIN;
m_p3DObject = NULL;
m_p3DBuffer = NULL;
// Try to get a 3D Buffer interface, if it exists.
pAudioPath->GetObjectInPath(
0,DMUS_PATH_BUFFER, // The DirectSound Buffer.
0,GUID_All_Objects,0, // Any Buffer (should only be one).
IID_IDirectSound3DBuffer,
(void **)&m_p3DBuffer);
}

CAudioPath::Set3DPosition()
Set3DPosition() is intended primarily for 3D AudioPaths in the 3D
pool. It is called on a regular basis (typically once per frame) to
update the 3D position of the AudioPath.
Set3DPosition() sports two optimizations. First, it stores the pre-
vious position and velocity of the 3D object in the m_vLastPosition
and m_vLastVelocity fields. It compares to see if either the position
or velocity has changed. If the 3D object’s position or velocity have
 Sound Effects n 309

not changed, Set3DPosition() returns without doing anything.

Secondly, Set3DPosition() keeps a pointer directly to the IDirect-
Sound3DBuffer interface so it doesn’t have to call GetObjectInPath()
every time. Set3DPosition() calls the appropriate IDirectSound3D-
Buffer methods to update the m_vLastPosition and m_vLastVelocity
fields if the position or velocity do change.
Set3DPosition() returns true for success and false for failure.
bool CAudioPath::Set3DPosition(
D3DVECTOR *pvPosition,
D3DVECTOR *pvVelocity,
DWORD dwApply)
{
// First, verify that this has changed since last time. If not, just
// return success.
if (!memcmp(&m_vLastPosition,pvPosition,sizeof(D3DVECTOR)))
II
{

Unit
// Position hasn't changed. What about velocity?
if (pvVelocity)
{
if (!memcmp(&m_vLastVelocity,pvVelocity,sizeof(D3DVECTOR)))
{
// No change to velocity. No need to do anything.
return true;
}
}
else return true;
}
// We'll be using the IDirectSound3DBuffer interface that
// we created in the constructor.
if (m_p3DBuffer)
{
// Okay, we have the 3D Buffer. Control it.
m_p3DBuffer->SetPosition(pvPosition->x,pvPosition->y,pvPosition->z,dwApply);
m_vLastPosition = *pvPosition;
// Velocity is optional.
if (pvVelocity)
{
m_p3DBuffer->SetVelocity(pvVelocity->x,pvVelocity->y,pvVelocity->z,dwApply);
m_vLastVelocity = *pvVelocity;
}
return true;
}
return false;
}
310 n Chapter 13

CAudio
CAudio needs to maintain a pool of AudioPaths. CAudio already has a
general-purpose list of AudioPaths, which we explored in full in
Chapter 11. However, the 3D pool needs to be separate since its
usage is significantly different. It carries a set of identical 3D
AudioPaths, intended specifically for swapping back and forth, as we
render. So, we create a second list.
CAudioPathList m_3DAudioPathList; // Pool of 3D AudioPaths.
DWORD m_dw3DPoolSize; // Size of 3D pool.

We provide a routine for setting the size of the pool, a routine for
allocating a 3D AudioPath, a routine for releasing one when done
with it, and a routine for finding the AudioPath with the lowest
priority.
// Methods for managing a pool of 3D AudioPaths.
DWORD Set3DPoolSize(DWORD dwPoolSize); // Set size of pool.
CAudioPath *Alloc3DPath(); // Allocate a 3D AudioPath.
void Release3DPath(CAudioPath *pPath); // Return a 3D AudioPath.
CAudioPath *GetLowestPriorityPath(); // Get lowest priority 3D AudioPath.

Let’s look at each one.

CAudio::Set3DPoolSize()
CAudio::Set3DPoolSize() sets the maximum size that the 3D pool is
allowed to grow to. Notice that it doesn’t actually allocate any
AudioPaths because they should still only be created when needed.
Optionally, the caller can pass POOLSIZE_USE_ALL_HARDWARE
instead of a pool size. This sets the pool size to the total number of
available hardware 3D buffers. This option allows the application to
automatically use the optimal number of 3D buffers. Set3DPoolSize()
accomplishes this by calling DirectSound’s GetCaps() method and
using the value stored in Caps.dwFreeHw3DAllBuffers.
DWORD CAudio::Set3DPoolSize(DWORD dwPoolSize)
{
// If the constant POOLSIZE_USE_ALL_HARDWARE was passed,
// call DirectSound's GetCaps method and get the total
// number of currently free 3D Buffers. Then, set that as
// the maximum.
if (dwPoolSize == POOLSIZE_USE_ALL_HARDWARE)
{
if (m_pDirectSound)
 Sound Effects n 311

{
DSCAPS DSCaps;
DSCaps.dwSize = sizeof(DSCAPS);
m_pDirectSound->GetCaps(&DSCaps);
m_dw3DPoolSize = DSCaps.dwFreeHw3DAllBuffers;
}
}
// Otherwise, use the passed value.
else
{
m_dw3DPoolSize = dwPoolSize;
}
// Return the PoolSize so the caller can know how many were
// allocated in the case of POOLSIZE_USE_ALL_HARDWARE.
return m_dw3DPoolSize;
}
II
CAudio::Alloc3DPath()

Unit
When the application does need a 3D AudioPath, it calls CAudio::
Alloc3DPath(). Alloc3DPath() first scans the list of 3D AudioPaths
already in the pool. Alloc3DPath() cannot take any paths that are
currently being used, which it tests by checking to see if the Audio-
Path’s Get3DObject() method returns anything. If there are no free
AudioPaths in the pool, Alloc3DPath() creates a new AudioPath.
CAudioPath *CAudio::Alloc3DPath()
{
DWORD dwCount = 0;
CAudioPath *pPath = NULL;
for (pPath = m_3DAudioPathList.GetHead();pPath;pPath = pPath->GetNext())
{
dwCount++;
// Get3DObject() returns whatever object this path is currently
// rendering. If NULL, the path is inactive, so take it.
if (!pPath->Get3DObject())
{
// Start the path running again and return it.
pPath->GetAudioPath()->Activate(true);
return pPath;
}
}
// Okay, no luck. Have we reached the pool size limit?
if (dwCount < m_dw3DPoolSize)
{
// No, so create a new AudioPath.
IDirectMusicAudioPath *pIPath = NULL;
m_pPerformance->CreateStandardAudioPath(
DMUS_APATH_DYNAMIC_3D, // Standard 3D AudioPath.
312 n Chapter 13

16, // 16 pchannels should be enough.

true, // Activate immediately.
&pIPath);
if (pIPath)
{
// Create a CAudioPath object to manage it.
pPath = new CAudioPath(pIPath,L"Dynamic 3D");
if (pPath)
{
// And stick in the pool.
m_3DAudioPathList.AddHead(pPath);
}
pIPath->Release();
}
}
return pPath;
}

CAudio::Release3DPath()
Conversely, CAudio::Release3DPath() takes an AudioPath that is cur-
rently being used to render something and stops it, freeing it up to
be used again by a different object (or thing).
void CAudio::Release3DPath(CAudioPath *pPath)
{
// Stop everything that is currently playing on this AudioPath.
m_pPerformance->StopEx(pPath->GetAudioPath(),0,0);
// Clear its object pointer.
pPath->Set3DObject(NULL);
// If we had more than we should (pool size was reduced), remove and delete.
if (m_3DAudioPathList.GetCount() > m_dw3DPoolSize)
{
m_3DAudioPathList.Remove(pPath);
// The CAudioPath destructor will take care of releasing
// the IDirectMusicAudioPath.
delete pPath;
}
// Otherwise, just deactivate so it won't eat resources.
else
{
pPath->GetAudioPath()->Activate(false);
}
}

CAudio::GetLowestPriorityPath()
GetLowestPriorityPath() scans through the list of AudioPaths and
finds the one with the lowest priority. This is typically done to find
 Sound Effects n 313

the AudioPath that would be the best candidate for swapping with an
object that has come into view and might be a higher priority. In
Mingle, GetLowestPriorityPath() is called by CThingManager when
it is reprioritizing AudioPaths.

þ Note Keep in mind that the highest priority possible is the lowest
number, or zero, not the other way around.

CAudioPath * CAudio::GetLowestPriorityPath()
{
float flPriority = 0.0; // Start with highest possible priority.
CAudioPath *pBest = NULL; // Haven't found anything yet.
CAudioPath *pPath = m_3DAudioPathList.GetHead();
for (;pPath;pPath = pPath->GetNext())
{
// Does this have a priority that is lower than best so far? II
if (pPath->GetPriority() > flPriority)

Unit
{
// Yes, so stick with it from now on.
flPriority = pPath->GetPriority();
pBest = pPath;
}
}
return pBest;
}

CThing
CThing manages a sound-emitting object in Mingle. It stores its cur-
rent position and velocity and updates these every frame. It also
maintains a pointer to the 3D AudioPath that it renders through and
uses that pointer to directly reposition the 3D coordinates of the
AudioPath.
For dynamic routing, CThing also stores a priority number. The
priority algorithm is simply the distance from the listener. The
shorter the distance, the higher the priority (lower the number). For
mathematical simplicity, the priority is calculated as the added
squares of the x and y coordinates.
The reassignment algorithm works in two stages. First, it marks
the things that need to be reassigned and forces them to stop play-
ing. Once all AudioPath reassignments have been made, it runs
through the list and starts the new things running. So, there is state
information that needs to be placed in CThing. The variables
314 n Chapter 13

m_fAssigned and m_fWasRunning are used to track the state of

CThing.
CThing uses script routines to start playback. There are five dif-
ferent types of things, as defined by the THING constants, and these
help determine which script variables to set and which script rou-
tines to call.
// All five types of thing:
#define THING_PERSON1 1 // Wandering snob
#define THING_PERSON2 2 // Wandering food moocher
#define THING_PERSON3 3 // Wandering mosquito person
#define THING_PERSON4 4 // Wandering laughing fool
#define THING_SHOUT 5 // Sudden shouts on mouse clicks

class CThing : public CMyNode

{
public:
CThing(CAudio *pAudio, DWORD dwType, CScript *pScript);
~CThing();
CThing *GetNext() { return (CThing *) CMyNode::GetNext(); };
bool Start();
bool Move();
bool Stop();
void CalcNewPosition(DWORD dwMils);
D3DVECTOR *GetPosition() { return &m_vPosition; };
void SetPosition (D3DVECTOR *pVector) { m_vPosition = *pVector; };
void SetAudioPath(CAudioPath *pPath) { m_pPath = pPath; };
CAudioPath *GetAudioPath() { return m_pPath; };
void SetType (DWORD dwType) { m_dwType = dwType; };
DWORD GetType() { return m_dwType; };
float GetPriority() { return m_flPriority; };
void MarkAssigned(CAudioPath *pPath);
bool IsAssigned() { return m_fAssigned; };
void ClearAssigned() { m_fAssigned = false; };
bool WasRunning() { return m_fWasRunning; };
bool NeedsPath() { return !m_pPath; };
void StopRunning() { m_pPath = NULL; m_fWasRunning = true; };
private:
CAudio * m_pAudio; // Keep pointer to CAudio for convenience.
DWORD m_dwType; // Which THING_ type this is.
bool m_fAssigned; // Was just assigned an AudioPath.
bool m_fWasRunning; // Was unassigned at some point in past.
float m_flPriority; // Distance from listener squared.
D3DVECTOR m_vPosition; // Current position.
D3DVECTOR m_vVelocity; // Direction it is currently going in.
CAudioPath *m_pPath; // AudioPath that manages the playback of this.
CScript * m_pScript; // Script to invoke for starting sound.
};
 Sound Effects n 315

CThing::Start()
CThing::Start() is called when a thing needs to start making sound.
This could occur when the thing is first invoked, or it could occur
when it has regained access to an AudioPath. First, Start() makes
sure that it indeed has an AudioPath. Once it does, it uses its 3D
position to set the 3D position on the AudioPath. It then sets the
AudioPath as the default AudioPath in the Performance. Next, Start()
calls a script routine to start playback. Although it is possible to hand
the AudioPath directly to the script and let it use it explicitly for
playback, it’s a little easier to just set it as the default, and then
there’s less work for the script to do.

þ Note There’s a more cynical reason for not passing the AudioPath as
a parameter. When an AudioPath is stored as a variable in the script, it
II
seems to cause an extra reference on the script, so the script never

Unit
completely goes away. This is a bug in DX9 that hopefully will get fixed
in the future.

bool CThing::Start()
{
if (m_pAudio)
{
if (!m_pPath)
{
m_pPath = m_pAudio->Alloc3DPath();
}
if (m_pPath )
{
m_pPath->Set3DObject(this);
m_pPath->Set3DPosition(&m_vPosition,&m_vVelocity,DS3D_IMMEDIATE);
if (m_pScript)
{
// Just set the AudioPath as the default path. Then,
// anything that gets played by
// the script will automatically play on this path.
m_pAudio->GetPerformance()->SetDefaultAudioPath(
m_pPath->GetAudioPath());
// Tell the script which of the four people
// (or the shouts) it should play.
m_pScript->GetScript()->SetVariableNumber(
L"PersonType",m_dwType,NULL);
// Then, call the StartTalking Routine, which
// will start something playing on the AudioPath.
m_pScript->GetScript()->CallRoutine(
L"StartTalking",NULL);
// No longer not running.
316 n Chapter 13

m_fWasRunning = false;
}
return true;
}
}
return false;
}

CThing::Stop()
CThing::Stop() is called when a thing should stop making sound.
Since it’s going to be quiet, there’s no need to hang on to an
AudioPath. So, Stop() calls CAudio::Release3DPath(), which kills the
sound and marks the AudioPath as free for the next taker.
bool CThing::Stop()
{
if (m_pAudio && m_pPath)
{
// Release3DPath() stops all audio on the path.
m_pAudio->Release3DPath(m_pPath);
// Don't point to the path any more cause it has moved on.
m_pPath = NULL;
return true;
}
return false;
}

CThing::CalcNewPosition()
CThing::CalcNewPosition() is called every frame to update the posi-
tion of the Thing. In order to update velocity, it’s necessary to
understand how much time has elapsed because velocity is really the
rate of distance changed over time. Since CThing doesn’t have its
own internal clock, it receives a time-elapsed parameter, dwMils,
which indicates how many milliseconds have elapsed since the last
call. It can then calculate a new position by using the velocity and
time elapsed. CalcNewPosition() also checks to see if the Thing
bumped into the edge of the box, in which case it reverses velocity,
causing the Thing to bounce back. When done setting the position
and velocity, CalcNewPosition() uses the new position to generate a
fresh new priority.
void CThing::CalcNewPosition(DWORD dwMils)
{
if (m_dwType < THING_SHOUT)
{
 Sound Effects n 317

// Velocity is in meters per second, so calculate the distance

// traveled in dwMils milliseconds.
m_vPosition.x += ((m_vVelocity.x * dwMils) / 1000);
if ((m_vPosition.x <= (float) -10.0) && (m_vVelocity.x < 0))
{
m_vVelocity.x = -m_vVelocity.x;
}
else if ((m_vPosition.x >= (float) 10.0) && (m_vVelocity.x > 0))
{
m_vVelocity.x = -m_vVelocity.x;
}
m_vPosition.y += ((m_vVelocity.y * dwMils) / 1000);
if ((m_vPosition.y <= (float) -10.0) && (m_vVelocity.y < 0))
{
m_vVelocity.y = -m_vVelocity.y;
}
else if ((m_vPosition.y >= (float) 10.0) && (m_vVelocity.y > 0))
{
II
m_vVelocity.y = -m_vVelocity.y;

Unit
}
// We actually track the square of the distance,
// since that's all we need for priority.
m_flPriority = m_vPosition.x * m_vPosition.x + m_vPosition.y * m_vPosition.y;
}
if (m_pPath)
{
m_pPath->SetPriority(m_flPriority);
}
}

CThing::Move()
Move() is called every frame. It simply transfers its own 3D position
to the 3D AudioPath. For efficiency, it uses the DS3D_DEFERRED
flag, indicating that the new 3D position command should be batched
up with all the other requests. Since CThingManager moves all of
the things at one time, it can make a call to the listener to commit all
the changes once it has moved all of the things.
bool CThing::Move()
{
// Are we currently active for sound?
if (m_pPath && m_pAudio)
{
m_pPath->Set3DPosition(&m_vPosition,&m_vVelocity,DS3D_DEFERRED);
return true;
318 n Chapter 13

}
return false;
}

CThingManager
CThingManager maintains the set of CThings that are milling around
the room. In addition to managing the list of CThings, it has routines
to create CThings, position them, and keep the closest CThings
mapped to active AudioPaths.
class CThingManager : public CMyList
{
public:
CThingManager() { m_pAudio = NULL; m_pScript = NULL; };
void Init(CAudio *pAudio,CScript *pScript,DWORD dwLimit);
CThing * GetHead() { return (CThing *) CMyList::GetHead(); };
CThing *RemoveHead() { return (CThing *) CMyList::RemoveHead(); };
void Clear();
CThing * CreateThing(DWORD dwType);
CThing * GetTypeThing(DWORD dwType); // Access first CThing of requested type.
DWORD GetTypeCount(DWORD dwType); // How many CThings of requested type?
void CalcNewPositions(DWORD dwMils); // Calculate new positions for all.
void MoveThings(); // Move CThings.
void ReassignAudioPaths(); // Maintain optimal AudioPath pairings.
void SetAudioPathLimit(DWORD dwLimit);
void EnforceAudioPathLimit();
private:
void StartAssignedThings();
CAudio * m_pAudio; // Keep pointer to CAudio for convenience.
CScript * m_pScript; // Script
DWORD m_dwAudioPathLimit; // Max number of AudioPaths.
};

CThingManager::CreateThing()
CreateThing() allocates a CThing class of the requested type and
installs the current CAudio and CScript in it. CreateThing() doesn’t
actually start the CThing playing, nor does it connect the CThing to
an AudioPath.
CThing *CThingManager::CreateThing(DWORD dwType)

{
CThing *pThing = new CThing(m_pAudio,dwType,m_pScript);
if (pThing)
{
 Sound Effects n 319

AddHead(pThing);
}
return pThing;
}

CThingManager::ReassignAudioPaths()
ReassignAudioPaths() is the heart of the dynamic resource system.
It makes sure that the highest priority things are matched up with
AudioPaths. To do so, it scans through the list of things, looking for
each thing that has no AudioPath but has attained a higher priority
than the lowest priority thing currently assigned to an AudioPath.
When it finds such a match, it steals the AudioPath from the lower
priority thing and reassigns the AudioPath to the higher priority
thing. At this point, it stops the old thing from making any noise by
killing all sounds on the AudioPath, but it doesn’t immediately start
II
the new sound playing. Instead, it sets a flag on the new thing, indi-

Unit
cating that it has just been assigned an AudioPath, and waits until the
next time around to start it. This is done to give the currently play-
ing sound a little time to clear out before the new one starts.

þ Note Under some circumstances, the approach used here of

delaying the sound’s start until the next frame is unacceptably slow. In
those cases, there is an alternate approach. Deactivate the AudioPath
by calling IDirectMusicAudioPath::Activate(false), and then reactivate it
by passing true to the same call. This causes the AudioPath to be
flushed of all sound. When this is done, you can start the new sound
immediately. However, there’s a greater chance of getting a click in the
audio because the old sound was stopped so suddenly.

void CThingManager::ReassignAudioPaths()
{
// First, start the things that were set up by the
// previous call to ReassignAudioPaths().
StartAssignedThings();

// Now, do a new round of swapping.

// Start by getting the lowest priority AudioPath.

// Use it to do the first swap.
CAudioPath *pLowestPath = m_pAudio->GetLowestPriorityPath();

CThing *pThing = GetHead();

for(; pThing && pLowestPath; pThing = pThing->GetNext())
{
if (pThing->NeedsPath() &&
320 n Chapter 13

(pThing->GetPriority() < pLowestPath->GetPriority()))

{
// We're closer, so steal the AudioPath.
CThing *pOtherThing = (CThing *) pLowestPath->Get3DObject();
if (pOtherThing)
{
pOtherThing->StopRunning();
}
// Stop the playback of all Segments on the path
// in case anything is still making sound.
m_pAudio->GetPerformance()->StopEx(pLowestPath->GetAudioPath(),0,0);
// Now, set the path to point to this thing.
pLowestPath->Set3DObject(pThing);
// Give it the new priority.
pLowestPath->SetPriority(pThing->GetPriority());
// Mark this thing to start playing on the path next time around.
// This gives a slight delay to allow any sounds in
// the old path to clear out.
pThing->MarkAssigned(pLowestPath);
// Okay, done. Now, get the new lowest priority path
// and use that to continue swapping...
pLowestPath = m_pAudio->GetLowestPriorityPath();
}
}
}

CThingManager::StartAssignedThings()
After canceling ReassignAudioPaths(), there may be one or more
things that have been assigned a new AudioPath but have not started
making any noise. This is intentional; it gives the AudioPath a
chance to drain whatever sound was in it before the thing newly
assigned to it starts making noise. So, StartAssignedThings() is
called on the next pass, and its job is simply to cause each newly
assigned thing to start making noise.
void CThingManager::StartAssignedThings()
{
// Scan through all of the CThings...
CThing *pThing = GetHead();
for(; pThing; pThing = pThing->GetNext())
{
// If this thing was assigned a new AudioPath, then start it.
if (pThing->IsAssigned())
{
// Get the newly assigned AudioPath.
CAudioPath *pPath;
 Sound Effects n 321

if (pPath = pThing->GetAudioPath())
{
// Was this previously making sound?
// It could have been an AudioPath that was assigned
// but was not played yet, in which case we do nothing.
if (pThing->WasRunning())
{
pThing->Start();
}
}
// Clear flag so we don't start it again.
pThing->ClearAssigned();
}
}
}

CThingManager::EnforceAudioPathLimit() II
EnforceAudioPathLimit() ensures that the number of AudioPaths

Unit
matches the number set in SetAudioPathLimit(). If the number went
down, it kills off low-priority AudioPaths to get back to where we
should be. EnforceAudioPathLimit() makes sure that any available
things that could be playing are given AudioPaths and turned on if
the number of AudioPaths increased.
void CThingManager::EnforceAudioPathLimit()
{
// First, count how many AudioPaths are currently assigned
// to things. This lets us know how many are currently in use.
DWORD dwCount = 0;
CThing *pThing = GetHead();
for(; pThing; pThing = pThing->GetNext())
{
if (pThing->GetAudioPath())
{
dwCount++;
}
}
// Do we have more AudioPaths in use than the limit? If so,
// we need to remove some of them.
if (dwCount > m_dwAudioPathLimit)
{
// Make sure we don't have any things or AudioPaths
// in a halfway state.
StartAssignedThings();
dwCount -= m_dwAudioPathLimit;
// dwCount now holds the number of AudioPaths that
// we can no longer use.
while (dwCount)
322 n Chapter 13

{
// Always knock off the lowest priority AudioPaths.
CAudioPath *pLowestPath = m_pAudio->GetLowestPriorityPath();
if (pLowestPath)
{
// Find the thing associated with the AudioPath.
pThing = (CThing *) pLowestPath->Get3DObject();
if (pThing)
{
// Disconnect the thing.
pThing->SetAudioPath(NULL);
}
// Release the AudioPath. This stops all playback
// on the AudioPath. Then, because the number of
// AudioPaths is above the limit, it removes the AudioPath
// from the pool.
m_pAudio->Release3DPath(pLowestPath);
}
dwCount--;
}
}
// On the other hand, if we have fewer AudioPaths in use than
// we are allowed, see if we can turn some more on.
else if (dwCount < m_dwAudioPathLimit)
{
// Do we have some turned off that could be turned on?
if (GetCount() > dwCount)
{
// If the total is under the limit,
// just allocate enough for all things.
if (GetCount() < m_dwAudioPathLimit)
{
dwCount = GetCount() - dwCount;
}
// Otherwise, allocate enough to meet the limit.
else
{
dwCount = m_dwAudioPathLimit - dwCount;
}
// Now, dwCount holds the amount of AudioPaths to turn on.
while (dwCount)
{
// Get the highest priority thing that is currently off.
pThing = GetHead();
float flPriority = FLT_MAX;
CThing *pHighest = NULL;
for (;pThing;pThing = pThing->GetNext())
{
// No AudioPath and higher priority?
if (!pThing->GetAudioPath() &&
 Sound Effects n 323

(pThing->GetPriority() < flPriority))

{
flPriority = pThing->GetPriority();
pHighest = pThing;
}
}
if (pHighest)
{
// We found a high priority AudioPath.
// Start it playing. That will cause it to
// allocate an AudioPath from the pool.
pHighest->Start();
}
dwCount--;
}
}

}
} II

Unit
CThingManager::CalcNewPositions()
CalcNewPositions() scans through the list of things and has each cal-
culate its position. This should be called prior to MoveThings(),
which sets the new 3D positions on the AudioPaths, and Reassign-
AudioPaths(), which uses the new positions to ensure that the
closest things are assigned to AudioPaths. CalcNewPositions()
receives the time elapsed since the last call as its only parameter.
This is used to calculate the velocity for each thing.
void CThingManager::CalcNewPositions(DWORD dwMils)
{
CThing *pThing = GetHead();
for (;pThing;pThing = pThing->GetNext())
{
pThing->CalcNewPosition(dwMils);
}
}

CThingManager::SetAudioPathLimit()
SetAudioPathLimit() is called whenever the maximum number of 3D
AudioPaths changes. In Mingle, this occurs when the user changes
the number of 3D AudioPaths with the edit box. First, SetAudioPath-
Limit() sets the internal parameter, m_dwAudioPathLimit, to the
new value. Then, it tells CAudio to do the same. Finally, it calls
EnforceAudioPathLimit(), which does the hard work of culling or
adding and activating 3D AudioPaths.
324 n Chapter 13

void CThingManager::SetAudioPathLimit(DWORD dwLimit)

{
m_dwAudioPathLimit = dwLimit;
m_pAudio->Set3DPoolSize(dwLimit);
EnforceAudioPathLimit();
}

CThingManager::MoveThings()
MoveThings() scans through the list of things and has each one
assign its new position to the AudioPath. Once all things have been
moved, MoveThings() calls the Listener’s CommitDeferredSet-
tings() method to cause the new 3D positions to take hold.
void CThingManager::MoveThings()
{
CThing *pThing = GetHead();
for (;pThing;pThing = pThing->GetNext())
{
pThing->Move();
}
m_pAudio->Get3DListener()->CommitDeferredSettings();
}

Mingle Implementation
The Mingle implementation of CThingManager is quite simple. You
can look at the source code in MingleDlg.cpp. Initialization occurs in
CMingleDlg::OnInitDialog(). In it, we load a script, which has rou-
tines that will be called by things when they start and stop playback.
m_pEffects points to the CAudio instance that manages sound
effects.
m_pScript = m_pEffects->LoadScript(L"..\\Media\\Mingle.spt");

Next, set the maximum number of 3D AudioPaths to eight. This is

intentionally a bit low so it can be easier to experiment with and hear
the results.
// Set the max number of 3D AudioPaths to 8.
m_dwMax3DPaths = m_pEffects->Set3DPoolSize(8);
// Initialize the thing manager with the CAudio for effects,
// the script for playing sounds, and the 3D AudioPath limit.
m_ThingManager.Init(m_pEffects,m_pScript,m_dwMax3DPaths);

That’s all the initialization needed.

 Sound Effects n 325

When one of the Talk buttons is clicked, a CThing object is cre-

ated and told to start making noise. The code is simple:
// Create a new CThing of the requested type.
CThing *pThing = m_ThingManager.CreateThing(dwType);
if (pThing)
{
// Successful, so start making sound.
pThing->Start();
}

Conversely, when one of the Shut Up! buttons is clicked, a CThing

object must be removed. This involves stopping the thing from play-
ing, removing it, deleting it, and then checking to see if there is a
lower priority thing waiting in the wings to play.
// Fing an instance of the requested type thing.
II
CThing *pThing = m_ThingManager.GetTypeThing(dwType);

Unit
if (pThing)
{
// Stop it. This will stop all sound on the
// AudioPath and release it.
pThing->Stop();
// Kill the thing!
m_ThingManager.Remove(pThing);
delete pThing;
// There might be another thing that now has the
// priority to be heard, so check and see.
m_ThingManager.EnforceAudioPathLimit();
UpdateCounts();
}

The dynamic scheduling occurs at regular intervals. Because Mingle

is a windowed app, it uses the Windows timer. In a non-windowed
application (like a game), this would more likely be once per frame.
The timer code executes a series of operations:
1. Calculate new positions for all CThings. This adjustment of
positions assigns new x, y, and z coordinates and assigns new
priorities.
2. Draw the things.
3. Use the priorities to make sure that the highest priority
things are being heard.
4. Update the 3D positions of the AudioPaths that have things
assigned to them.
326 n Chapter 13

void CMingleDlg::OnTimer(UINT nIDEvent)

{
// First, update the positions of all things.
// Indicate that this timer wakes up every 30 milliseconds.
m_ThingManager.CalcNewPositions(30);
// Draw them in their new positions.
CClientDC dc(this);
DrawThings(&dc);
// Now, make sure that all the highest priority
// things are being heard.
m_ThingManager.ReassignAudioPaths();
// Finally, update their 3D positions.
m_ThingManager.MoveThings();
CDialog::OnTimer(nIDEvent);
}

If at any point during the play the 3D AudioPath limit changes, one
call to SetAudioPathLimit() straightens things out.
m_ThingManager.SetAudioPathLimit(m_dwMax3DPaths);

That concludes dynamic 3D resource management!

Low-Latency Sound Effects

We already talked in some depth about how to drop the latency ear-
lier in this chapter, and the code sample for doing so was taken from
Mingle. Mingle demonstrates low-latency sound effects with the var-
ious complaints heard when you click on the party with your mouse.
First, create a CThing that is dedicated just to making these noises
whenever the mouse is clicked.
// Create the thing that makes a sound every time the mouse clicks.
m_pMouseThing = m_ThingManager.CreateThing(THING_SHOUT);
if (m_pMouseThing)
{
// Allocate an AudioPath up front and hang on to it because
// we always want it ready to run the moment the mouse click occurs.
m_pMouseThing->SetAudioPath(m_pEffects->Alloc3DPath());
m_pMouseThing->GetAudioPath()->Set3DObject((void *)m_pMouseThing);
}

When there is a mouse click, translate the mouse coordinates into

3D coordinates and set m_pMouseThing to these coordinates. Then,
immediately have it play a sound.
// Set the position of the AudioPath to the new position
// of the mouse.
 Sound Effects n 327

m_pMouseThing->SetPosition(&Vector);
// Start a sound playing immediately.
m_pMouseThing->Start();

Background Ambience
It’s not enough to have a handful of individuals wandering around us
mumbling, whining, munching, or laughing. To fill in the gaps and
make the party feel a lot larger, we’d like to play some background
ambience. The ambience track is a short Segment with a set of ste-
reo wave recordings set up to play as random variations. The
Segment is actually played by the script’s StartParty routine, so
there’s no special code in Mingle itself for playing background ambi-
ence other than the creation of a stereo dynamic AudioPath for the
initial default AudioPath, which StartParty uses to play the Segment
II
on. The Segment is set to loop forever, so nothing more needs to be

Unit
done.

Avoiding Truncated Waves

When a wave is played on an AudioPath that is pitch shifted down,
the duration of the wave can be prematurely truncated. This occurs
because the wave in the Wave Track has a specific duration assigned
to it. The intention is honorable; you can specify just a portion of a
wave to play by identifying the start time and duration. To work well
with the scheduling of other sounds in music or clock time, the dura-
tion is also defined in music or clock time. Therein lies a problem. If
you really want the wave to play through in its entirety, there’s no
way of specifying the duration in samples. This normally isn’t a prob-
lem, but if the wave gets pitch shifted down during playback (which
happens, for example, with Doppler shift), the wave sample duration
becomes longer than the duration imposed on it and it stops
prematurely.
The simplest solution involves writing some code to force the
wave durations to be long enough that they wrap the most dramatic
pitch bend you would throw at it. This is actually easy to do, and this
is what Mingle does. Through DirectMusic’s Tool mechanism, you
can write a filter that captures any event type on playback and alters
it as needed. So, we write a Tool that intercepts the DMUS_WAVE_
PMSG PMsg and multiplies the DMUS_WAVE_PMSG rtDuration
328 n Chapter 13

field by a reasonable number (say, 10). We place the Tool in the

Performance, so it intercepts everything played on all Segments.
Although Tools can be written as COM objects that are created
via CoCreateInstance() and authored into Segments or AudioPaths,
that level of sophistication isn’t necessary here. Instead, we declare
a class, CWaveTool, that supports the IDirectMusicTool interface,
which is actually very simple to write to. The heart of IDirectMusic-
Tool is one method, ProcessPMsg(), which takes a pmsg and works
its magic on it. In the case of CWaveTool, that magic is simply
increasing the duration field in the wave pmsg. There are a handful
of other methods in IDirectMusicTool that are used to set up what
pmsg types that the Tool handles and when the pmsg should be
delivered to the Tool; choices are immediately, at the time stamp on
the pmsg, or a little ahead of that.
Here are all the IDirectMusicTool methods, as used by
CWaveTool:
STDMETHODIMP CWaveTool::Init(IDirectMusicGraph* pGraph)
{
// Don't need to do anything at initialization.
return S_OK;
}

STDMETHODIMP CWaveTool::GetMsgDeliveryType(DWORD* pdwDeliveryType)

{
// This tool should process immediately and not wait for the time stamp.
*pdwDeliveryType = DMUS_PMSGF_TOOL_IMMEDIATE;
return S_OK;
}

STDMETHODIMP CWaveTool::GetMediaTypeArraySize(DWORD* pdwNumElements)

{
// We have exactly one media type to process: Waves.
*pdwNumElements = 1;
return S_OK;
}

STDMETHODIMP CWaveTool::GetMediaTypes(DWORD** padwMediaTypes, DWORD dwNumElements)

{
// Return the one type we handle.
**padwMediaTypes = DMUS_PMSGT_WAVE;
return S_OK;
}

STDMETHODIMP CWaveTool::ProcessPMsg(IDirectMusicPerformance* pPerf,

DMUS_PMSG* pPMSG)
 Sound Effects n 329

{
// Just checking to be safe...
if (NULL == pPMSG->pGraph)
{
return DMUS_S_FREE;
}
// Point to the next Tool after this one.
// Otherwise, it will just loop back here forever
// and lock up.
if (FAILED(pPMSG->pGraph->StampPMsg(pPMSG)))
{
return DMUS_S_FREE;
}
// This should always be a wave since we only allow
// that media type, but check anyway to be certain.
if( pPMSG->dwType == DMUS_PMSGT_WAVE)
{
// Okay, now the miracle code.
II
// Multiply the duration by ten to reach beyond the

Unit
// wildest pitch bend.
DMUS_WAVE_PMSG *pWave = (DMUS_WAVE_PMSG *) pPMSG;
pWave->rtDuration *= 10;
}
return DMUS_S_REQUEUE;
}

STDMETHODIMP CWaveTool::Flush(IDirectMusicPerformance* pPerf,

DMUS_PMSG* pPMSG,
REFERENCE_TIME rtTime)
{
// Nothing to do here.
return S_OK;
}

Once the CWaveTool is compiling nicely, we still need to stick it in

the Performance. Doing so is straightforward. We only need to cre-
ate one CWaveTool and place it directly in the Performance at
initialization time. The following code does so. The code uses an
AudioPath to access the Performance Tool Graph. Using an Audio-
Path is actually simpler than calling the GetGraph() method on the
Performance because GetGraph() returns NULL if there is no
Graph; DirectMusic doesn’t create one unless needed, for efficiency
reasons. GetObjectInPath() goes the extra step of creating the Graph
if it does not exist already.
// Use the default AudioPath to access the tool graph.
IDirectMusicAudioPath *pPath;
if (SUCCEEDED(m_pPerformance->GetDefaultAudioPath(&pPath)))
330 n Chapter 13

{
IDirectMusicGraph *pGraph = NULL;
pPath->GetObjectInPath(0,
DMUS_PATH_PERFORMANCE_GRAPH, // The Performance Tool Graph.
0,
GUID_All_Objects,0, // Only one object type.
IID_IDirectMusicGraph, // The Graph interface.
(void **)&pGraph);
if (pGraph)
{
// Create a CWaveTool
CWaveTool *pWaveTool = new CWaveTool;
if (pWaveTool)
{
// Insert it in the Graph. It will process all wave pmsgs that
// are played on this Performance.
pGraph->InsertTool(static_cast<IDirectMusicTool*>(pWaveTool),
NULL,0,0);
}
pGraph->Release();
}
pPath->Release();
}

That’s it. The Tool will stay in the Performance until the
Performance is shut down and released. At that point, the references
on the Tool will drop to zero, and its destructor will free the memory.

þ Note This technique has only one downside: If you actually do want
to set a duration shorter than the wavelength, it undoes your work. In
that scenario, the best thing to do is be more specific about where you
do this. You can control which pchannels the Tool modifies, or you can
place it in the Segments or AudioPaths where you want it to work, rather
than the entire Performance.

Manipulating the Listener Properties

Working with the Listener is critical for any 3D sound effects work,
so CAudio has a method, Get3DListener(), that retrieves the
DirectSound IDirectSound3DListener8 interface. This is used for
setting the position of the Listener as well as adjusting global 3D
sound parameters.

þ Note Get3DListener returns an AddRef()’d instance of the Listener, so

it’s important to remember to Release() it when done.
 Sound Effects n 331

IDirectSound3DListener8 *CAudio::Get3DListener()
{
IDirectSound3DListener8 *pListener = NULL;
IDirectMusicAudioPath *pPath;
// Any AudioPath will do because the listener hangs off the primary buffer.
if (m_pPerformance && SUCCEEDED(m_pPerformance->GetDefaultAudioPath(&pPath)))
{
pPath->GetObjectInPath(0,
DMUS_PATH_PRIMARY_BUFFER,0, // Access via the primary Buffer.
GUID_All_Objects,0,
IID_IDirectSound3DListener8,// Query for listener interface.
(void **)&pListener);
pPath->Release();
}
return pListener;
}

To facilitate the use of the Listener, the Mingle dialog code keeps a
II
copy of the Listener interface and the DS3DLISTENER data struc-

Unit
ture that can be filled with all of the Listener properties and set at
once.
DS3DLISTENER m_ListenerData; // Store current Listener params.
IDirectSound3DListener8 *m_pListener; // Use to update the Listener.

In the initialization of the Mingle window, the code gets access to the
Listener and then reads the initial parameters into m_ListenerData.
From then on, the window code can tweak any parameter and then
bulk-update the Listener with the changes.
m_pListener = m_pEffects->Get3DListener();
m_ListenerData.dwSize = sizeof(m_ListenerData);
if (m_pListener)
{
m_pListener->GetAllParameters(&m_ListenerData);
}

Later, when one or more Listener parameters need updating, just

the changed parameters can be altered and then the whole structure
is submitted to the Listener. For example, the following code updates
the Rolloff Factor in response to dragging a slider.
m_ListenerData.flRolloffFactor = (float) m_ctRollOff.GetPos() / 10;
m_pListener->SetAllParameters(&m_ListenerData,DS3D_IMMEDIATE );
332 n Chapter 13

Scripted Control of Sound Effects

Mingle uses scripting as much as possible. As we’ve already seen, all
sound effects are played via scripts. There are only three routines in
the script file mingle.spt:
n StartParty: This is called at the beginning of Mingle. It is
intended for any initialization that might be needed. It assigns
names for the four people, and it starts the background
ambience.
n StartTalking: This is called whenever anything starts making
sound. A variable, PersonType, is passed first, indicating which
thing type should make the sound.
n EndParty: This is called when the party is over and it’s time to
shut down. This isn’t entirely necessary, but it’s a little cleaner
to go ahead and turn off all the sounds before shutting down.
Here’s the script:
'Mingle Demo
'©2002 NewBlue Inc.

' Variable set by the application prior to calling StartTalking

dim PersonType ' Which object is making the sounds

' Variables retrieved by the application to find out the names

' of the people talking
dim Person1Name
dim Person2Name
dim Person3Name
dim Person4Name

sub StartTalking
if (PersonType = 1) then
Snob.play IsSecondary
elseif (PersonType = 2) then
Munch.play IsSecondary
elseif (PersonType = 3) then
Mosquito.play IsSecondary
elseif (PersonType = 4) then
HaHa.play IsSecondary
elseif (PersonType = 5) then
Shout.play IsSecondary
end if
end sub
 Sound Effects n 333

sub StartParty
Crowd.play IsSecondary
PersonType = 1
Person1Name = "Snob"
Person2Name = "HorsDeOoovers"
Person3Name = "Mosquito Person"
Person4Name = "Laughing Fool"
end sub

sub EndParty
Crowd.Stop
HaHa.Stop
Mosquito.Stop
Munch.Stop
Shout.Stop
Snob.Stop
end sub II
Here’s how the script is called at startup. Right after loading the

Unit
script, call StartParty to initialize the Person name variable and allow
the script to do anything else it might have to. Then, read the names.
Since these are strings, the only way we can read them is via
variants. Fortunately, there are some convenience routines that take
the pain out of working with variants. You must call VariantInit() to
initialize the variant. Then, use the variant to retrieve the string
from the script. Since the memory for a variant string is allocated
dynamically, it must be freed. So, call VariantClear(), and that cleans
it up appropriately.
// Call the initialization routine. This should set the Person
// name variables as well as start any background ambience.
if (SUCCEEDED(m_pScript->GetScript()->CallRoutine(L"StartParty",NULL)))
{
// We will use a variant to retrieve the name of each Person.
VARIANT Variant;
char szName[40];
VariantInit(&Variant);
if (SUCCEEDED(m_pScript->GetScript()->GetVariableVariant(
L"Person1Name",&Variant,NULL)))
{
// Copy the name to ASCII.
wcstombs(szName,Variant.bstrVal,40);
// Use it to set the box title.
SetDlgItemText(IDC_PERSON1_NAME,szName);
// Clear the variant to release the string memory.
VariantClear(&Variant);
}
// Do the same for the other three names...
334 n Chapter 13

We’ve already seen how StartTalking is called when we looked at

dynamic AudioPaths. EndParty is called when the app window is
being closed. If there’s anything that the script needs to do on shut-
down, it can do it there.
if (m_pScript)
{
m_pScript->GetScript()->CallRoutine(L"EndParty",NULL);
}

Creating Music with the Composition Engine

Although this chapter is all about sound effects, it’s important to
demonstrate how music and sound effects work together, so I
needed to put some music in this but not spend much time on it.
Enter DirectMusic’s Style and composition technologies.
The Composition Engine can write a piece of music using har-
monic (chord progression) guidelines authored in the form of a
ChordMap. For the overall shape of the music, the Composition
Engine can use a template Segment authored in Producer or actually
write a template on the fly using its own algorithm, which is defined
by a handful of predefined shapes. The shapes describe different
ideas for how the music should progress over time. Examples
include rising (increasing in intensity) and falling (decreasing in
intensity). A particularly useful shape is Song, which builds a struc-
ture with subsections for verse, chorus, etc., that it swaps back and
forth. For playback, the chords and phrasing created by the Composi-
tion Engine are interpreted by a Style.
As it happens, DX9 ships with almost 200 different Styles and
about two dozen ChordMaps. Not only are they a great set of exam-
ples for how to write style-based content, but they can be used as a
jumping-off point for writing your own Styles. Or, for really low-
budget apps like Mingle, you can always use them off the shelf. They
are also great for dropping in an application to sample different musi-
cal genres as well as making a great placeholder while waiting for the
real music to be authored.
It is really easy to use the Composition Engine to throw together
a piece of music to play in Mingle. We add ComposeSegment(),
which creates a CSegment as an alternative to LoadSegment(),
which reads an authored one from disk.
 Sound Effects n 335

CSegment *CAudio::ComposeSegment(
WCHAR *pwzStyleName, // File name of the Style.
WCHAR *pwzChordMapName, // File name of the ChordMap.
WORD wNumMeasures, // How many measures long?
WORD wShape, // Which shape?
WORD wActivity, // How busy are the chord changes?
BOOL fIntro, BOOL fEnd) // Do we want beginning and/or ending?
{
// Create a Composer object to build the Segment.
CSegment *pSegment = NULL;
if (!m_pComposer)
{
CoCreateInstance(
CLSID_DirectMusicComposer,
NULL,
CLSCTX_INPROC,
IID_IDirectMusicComposer,
(void**)&m_pComposer);
II
}

Unit
if (m_pComposer)
{
// First, load the Style.
IDirectMusicStyle *pStyle = NULL;
IDirectMusicChordMap *pChordMap = NULL;
if (SUCCEEDED(m_pLoader->LoadObjectFromFile(
CLSID_DirectMusicStyle,
IID_IDirectMusicStyle,
pwzStyleName,
(void **) &pStyle)))
{
// We have the Style, so load the ChordMap.
if (SUCCEEDED(m_pLoader->LoadObjectFromFile(
CLSID_DirectMusicChordMap,
IID_IDirectMusicChordMap,
pwzChordMapName,
(void **) &pChordMap)))
{
// Hooray, we have what we need. Call the Composition
// Engine and have it write a Segment.
IDirectMusicSegment8 *pISegment;
if (SUCCEEDED(m_pComposer->ComposeSegmentFromShape(
pStyle, wNumMeasures, wShape,
wActivity, fIntro, fEnd,
pChordMap,
(IDirectMusicSegment **) &pISegment)))
{
// Create a CSegment object to manage playback.
pSegment = new CSegment;
if (pSegment)
{
336 n Chapter 13

// Initialize
pSegment->Init(pISegment,m_pPerformance,SEGMENT_FILE);
m_SegmentList.AddTail(pSegment);
}
pISegment->Release();
}
pChordMap->Release();
}
pStyle->Release();
}
}
return pSegment;
}

Using this is simple. When Mingle first opens up, call Compose-
Segment() and have it write something long. Set it to repeat forever.
Play it and forget about it.
CSegment *pSegment = m_pMusic->ComposeSegment(
L"Layback.sty", // Layback style.
L"PDPTDIA.cdm", // Pop diatonic ChordMap.
100, // 100 measures.
DMUS_SHAPET_SONG, // Song shape.
2, // Relatively high chord activity,
true,true); // Need an intro and an ending.
if (pSegment)
{
// Play it over and over again.
pSegment->GetSegment()->SetRepeats(DMUS_SEG_REPEAT_INFINITE);
m_pMusic->PlaySegment(pSegment,NULL,NULL);
}

You might have noticed that this is the only sound-producing code in
Mingle that is not managed by the script. Unfortunately, scripting
does not support ComposeSegmentFromShape(), so we have to call
it directly in the program. However, one could have the script man-
age the selection of Style and ChordMap so that the music choice
could be iteratively changed without a recompile of the program.

DirectX Audio has the potential to create truly awesome sound envi-
ronments. Many of the techniques that apply well to music, like
scripting, variations, and even groove level, lend themselves to cre-
ating audio environments that are equally rich. The beauty is the
vast majority of the work is done on the authoring side with a mini-
mum amount of programming.
 Sound Effects n 337

The key is in understanding the technologies and mapping out a

good strategy. Hopefully this chapter points you in a good direction.
You are welcome to use the sample code in any way you’d like.

II

Unit
This page intentionally left blank.
Chapter 14

Notifications

S ometimes, an application would really like to know when a spe-

cific sound or musical event occurs. This is useful for many rea-
sons. For example, if the audio system can report when a Segment is
about to finish, the application can select and queue the appropriate
Segment to follow. There are plenty of creative paths to follow once
you have a robust, easy-to-use system that “notifies” the application
when specific audio events occur. Examples of things you can do
with a good notification system include:
n Placing signals in a Segment to indicate when specific sounds
occur to trigger visual elements
n Synchronizing animation to the musical beat so characters walk
and move with rhythm
Think about the corollary between film and games. In the film edit-
ing process, sounds often synchronize to visual happenings, and
sometimes visual elements cut to the sound or music. You might
think of an MTV video for the extreme example, but even action
sequences where the music is not center stage can be subtly
improved by letting sound drive some of the timing. Rhythm and
pacing are far more critical in the audible domain than visual. Think
about speech. It works very well to animate lips to words, but it is
invariably clumsy the other way around.
So far, with game sound, it’s been pretty much a one-way street;
sounds are triggered by visual events. But, with the proper tools and
techniques, you can change all that and make more compelling
environments.
DirectX Audio includes a rich set of options for synchronizing
visuals and actions to the sound track. Depending on the notification

339
340 n Chapter 14

requirements, DirectX Audio provides different approaches to

accomplish synchronization.
n Performance notifications: The Performance Engine provides
a standard system for handling notifications. Standard notifica-
tions range from Segment start and stop to groove level change.
n Lyrics: A Lyric Track in a Segment provides an easy way to pro-
vide notifications that are defined as text names. The beauty of
this approach is any text word can be placed in a Segment
timeline and then delivered to the application at the specific
point in time.
n Script track: A Script Track calls script routines at specific
points in the Segment playback.
In this chapter, we talk about all three and add features to Jones to
visualize both performance notifications and lyrics.

Performance Notifications
The standard mechanism for managing notifications is pretty com-
prehensive. DirectMusic’s Performance Engine provides different
categories of notifications, which you can turn on or off as needed.
These range from notifications that tell the application when differ-
ent sounds and music clips start and stop to notifications that
indicate when musical events, such as changes in time signature or
groove level, occur. The notifications pass through the Performance
in pmsg form, much like note or wave messages. However, instead
of sending these messages to the synthesizer, DirectMusic delivers
them to the application.
The application can even choose how to receive the notifications;
it can set up an event and be signaled when one arrives, or it can
simply poll for the notification messages.

Notification Categories
The notification pmsgs are broken into categories, defined via
GUIDs. Many of the categories have subtypes, which are in turn
defined by a constant DWORD. The categories include:
 Notifications n 341

n GUID_NOTIFICATION_CHORD: This notification indicates

that a chord has changed.
n GUID_NOTIFICATION_COMMAND: This notification indi-
cates whenever a groove level track broadcasts a new groove
level or embellishment command.
n GUID_NOTIFICATION_MEASUREANDBEAT: One of
these notifications is sent every beat. It carries the measure and
beat number with it.
n GUID_NOTIFICATION_PERFORMANCE: This notification
indicates a change in the status of all Segments in the Perfor-
mance. For example, it signals when sound or music first starts
as well as when the last Segment finishes playing.
n GUID_NOTIFICATION_RECOMPOSE: This notification
II
indicates when a Track is recomposed, which can automatically

Unit
occur on the fly. This is primarily used by authoring tools (in
other words, DirectMusic Producer).
n GUID_NOTIFICATION_SEGMENT: This notification indi-
cates a change in the playback status of a Segment, including
starting, looping, and stopping.
Of all of these, the Segment commands are by far the most useful.
These indicate when a Segment starts, stops, or loops and even
when it is stopped prematurely. The notification mechanism also
hands the application a pointer to the IDirectMusicSegmentState
interface associated with the playing Segment.

The Notification Pmsg

The DMUS_NOTIFICATION_PMSG starts with the DMUS_
PMSG_PART core, which provides the necessary infrastructure for
pmsg delivery and follows with several fields used to identify the
type of the notification:
typedef struct _DMUS_NOTIFICATION_PMSG
{
/* begin DMUS_PMSG_PART */
DMUS_PMSG_PART
/* end DMUS_PMSG_PART */
GUID guidNotificationType; // GUID indicating notification category.
342 n Chapter 14

DWORD dwNotificationOption; // Subtype of the notification.

DWORD dwField1; // Optional data field.
DWORD dwField2; // Second data field.
} DMUS_NOTIFICATION_PMSG;

guidNotificationType indicates the category of notification. These are

the notification categories listed earlier. A GUID is used to ensure
that new notification categories can be introduced by different par-
ties without any danger of clashing. Each category can optionally
supply more than one subtype, identified in dwNotificationOption.
The following two fields, dwField1 and dwField2, carry notification-
specific data. When used, their functionality is defined by the notifi-
cation category and option.
Additional relevant information is stored in the DMUS_PMSG_
PART portion. Of particular interest are:
REFERENCE_TIME rtTime; // Clock time this pmsg occurs.
MUSIC_TIME mtTime; // Music time of this pmsg.
IUnknown* punkUser; // Optional COM pointer to referenced object.

rtTime and mtTime carry the specific time that the notification
occurs. punkUser carries a pointer to a COM interface, allowing the
notification to drag along a pointer to an associated object, if appro-
priate. For example, the GUID_NOTIFICATION_SEGMENT
command uses punkUser to carry a pointer to the IDirectMusic-
SegmentState associated with the playing Segment.

Enabling a Notification
By default, DirectMusic does not generate any notifications. To
activate a notification, you must call the performance’s AddNotifica-
tionType() method:
// Enable Segment notifications.
pPerformance->AddNotificationType(GUID_NOTIFICATION_SEGMENT);

In this example, setting GUID_NOTIFICATION_SEGMENT tells

DirectMusic to generate a notification every time a Segment starts,
loops, or stops.
To disable a notification, call the inverse method:
// Disable time signature notifications
pPerformance->RemoveNotificationType(GUID_NOTIFICATION_MEASUREANDBEAT);
 Notifications n 343

Receiving a Notification
There are several ways to receive the notification pmsg. You can poll
for it. You can set up an event and wait to be signaled. Or you can
stick a Tool in the Performance and use the Tool to intercept it.

Polling for Notifications

Polling is the easiest way to receive a notification and typically the
most useful. If the application has a regular loop of things that it does
once per frame, then it can check for notifications as part of that
loop. The Performance provides a method, GetNotificationPMsg(),
that retrieves any due notification pmsgs.
DMUS_NOTIFICATION_PMSG *pPmsg; II
while (pPerformance->GetNotificationPMsg(&pPmsg) == S_OK)
{

Unit
// Do something here with the data from the notification, and then free it.
pPerformance->FreePMsg((DMUS_PMSG*)pPmsg);
}

GetNotificationPMsg() returns S_OK for every notification it has in

its queue; then it returns S_FALSE when there are none left. Be
careful: S_FALSE and S_OK are both considered success codes, so
don’t use the SUCCEEDED(hr) macro because it will always suc-
ceed. Instead, check specifically for S_OK.

þ Note Notifications don’t wait around forever. If the application does

not call GetNotificationPMsg() within a reasonable time (by default, two
seconds), DirectMusic discards the notification.

Waiting on an Event
If you would prefer, DirectMusic can signal your application exactly
when a notification is ready. This option uses the standard event sig-
naling Windows APIs.
First, create an event handle with a call to the Windows Create-
Event() function.
HANDLE hNotify = CreateEvent(NULL, FALSE, FALSE, NULL);

After creating the event, hand it to the Performance via its SetNotifi-
cationHandle() method. This tells the Performance to signal the
event whenever a notification is due.
344 n Chapter 14

pPerformance->SetNotificationHandle(hNotify, 4000);

The second parameter specifies how long, in milliseconds, the

Performance should hold onto the notification if it is not retrieved.
This keeps old notifications from piling up if the application is tempo-
rarily not listening to them. A value of 0 in this parameter indicates
that the default time of two seconds should be used.
Once the event is prepared, the application calls the Windows
API WaitForSingleObject() to wait for a notification. It’s important to
understand that this call stops (or “blocks”) the calling thread until a
notification arrives. Because of this, it doesn’t work to place this call
in a thread that needs to be doing anything while waiting, so be sure
to place it in a thread that can afford to wait. For example, it would
not work to call WaitForSingleObject() in the middle of a graphics/
UI loop.
WaitForSingleObject(hNotify, 1000);

In this code, the caller waits for a notification for up to one second.
Once the notification arrives, the caller retrieves the notification via
a call to GetNotificationPMsg().

Capturing Notifications with a Tool

There’s a third way to receive notification pmsgs. Create a Tool with
an IDirectMusicTool interface, set it up to handle DMUS_PMSGT_
NOTIFICATION message type, and place it in the Performance.
When a notification comes down the pike, the DirectMusic calls the
Tool’s ProcessPMsg() method, and it can then notify the application
in whatever way it wants to.
This approach requires a little more work, but it buys extra
flexibility:
n You have three options for delivery timing. Choose DMUS_
PMSGF_TOOL_IMMEDIATE, and the Tool gets called the
moment the notification is generated, which is typically 500ms
ahead of the time stamp. Choose DMUS_PMSGF_TOOL_
QUEUE, and the Tool is called a little early, typically about 50ms.
Choose DMUS_PMSGF_TOOL_ATTIME, and the Tool is called
as close to the exact time stamp as possible, which is what the
regular notification system does. As it turns out, DMUS_
PMSGF_TOOL_QUEUE is particularly useful because being
 Notifications n 345

slightly early (but still with the correct time stamp) results in a
response that might feel more accurate.
n You control where the notifications are sent by placing the Tool
in the appropriate object. You can insert Tools in Segments,
AudioPaths, and the Performance. A notification tool placed in a
Segment only receives notifications generated by that Segment,
which makes it very easy to understand the context of the
notification.
n If you plan to also use lyric notifications (we discuss that next),
you need to write a Tool anyway, and so you might as well use
the same mechanism for both and avoid the extra code.
For the last reason, we are using a Tool to capture notifications in
Jones.
II
Setting up a Tool is not as hard as it may seem. We learned this

Unit
in the last chapter when we created a Tool to lengthen wave pmsgs.
Define a C++ class based on the IDirectMusicTool interface. Set it
up to only receive DMUS_PMSGT_NOTIFICATION messages and
receive them at DMUS_PMSGF_TOOL_QUEUE time (which
means they arrive about 50ms or roughly a frame early). This chap-
ter’s Jones implementation covers installing such a Tool in detail, so
we can skip the code for defining and installing the Tool for now.
The active part of the Tool is the method ProcessPMsg(), which
receives the message. For processing notifications, it verifies that
the pmsg is of type DMUS_PMSGT_NOTIFICATION, reads the
parameters from the notification pmsg, does whatever it needs to do,
and then returns.
HRESULT CNotifyTool::ProcessPMsg(IDirectMusicPerformance* pPerf,
DMUS_PMSG* pPMSG)
{
if (pPMSG->dwType == DMUS_PMSGT_NOTIFICATION)
{
// Read the notification and do something.
}
// No need to send this on down the line.
return DMUS_S_FREE;
}

As you can see, the code to receive a notification via a Tool is

remarkably simple.
346 n Chapter 14

þ Note Be sure not to linger long inside the Tool’s ProcessPMsg()

method. This is a high-priority thread and should not be used for UI or
file I/O work, which could destroy the timing. Instead, set a flag or
place a message in a queue and then butt out immediately.

Interpreting the Notification

Once we have the notification in hand, we need to make sense of it.
This involves reading the dwType, guidNotificationType, and
dwNotificationOption fields to glean the specific notification.
When working with both lyrics and notifications, it’s important to
identify the pmsg type by checking the dwType field of the pmsg.
// Is this a notification? (Only applied to reading from a Tool.)
if (pMsg->dwType == DMUS_PMSGT_NOTIFICATION)
{
DMUS_NOTIFICATION_PMSG *pNotify = (DMUS_NOTIFICATION_PMSG *) pMsg;
// It’s a notification!
}

If it is a notification, check both guidNotificationType for the cate-

gory of notification and dwNotificationOption for the subtype. In this
example, the application wants to know when the music ends, so it
can fade out the scene.
DMUS_NOTIFICATION_PMSG *pNotify = (DMUS_NOTIFICATION_PMSG *) pMsg;
DWORD dwNotificationOption = pNotify->dwNotificationOption;
if (pNotify->guidNotificationType == GUID_NOTIFICATION_PERFORMANCE)
{
if (dwNotificationOption == DMUS_NOTIFICATION_MUSICSTOPPED)
{
// Music's over. Fade to black.
}
}

Lyric Notifications
Lyrics are a little simpler to talk about for several reasons:
n There’s only one way you can receive lyrics, and that is via a
Tool.
n There’s no enabling or disabling of lyrics. Regular notifications
need this feature because there are all kinds of messages that
you may not care about. With lyrics, the messages are always
 Notifications n 347

exactly what you want because they’ve been intentionally

authored into the segments.
n The lyric pmsg, DMUS_LYRIC_PMSG, is simpler too. It has
exactly one field, the lyric string:
typedef struct _DMUS_LYRIC_PMSG
{
/* begin DMUS_PMSG_PART */
DMUS_PMSG_PART
/* end DMUS_PMSG_PART */
WCHAR wszString[1]; /* NULL-terminated Unicode lyric */
} DMUS_LYRIC_PMSG;

Receiving a Lyric
There is exactly one way to receive a lyric — intercept it with a Tool.
II
Going this route does mean a little more work, but in truth we’re

Unit
talking an extra hour of programming that can pay off easily in the
long term.
Create a Tool with an IDirectMusicTool interface, set it up to
handle the DMUS_PMSGT_LYRIC message type, and place it in the
Performance. (We will provide example code later in this chapter
when we add notifications to Jones.) When a lyric comes down the
pike, DirectMusic calls the Tool’s ProcessPMsg() method, which can
then notify the application in whatever way it wants to.
HRESULT CLyricTool::ProcessPMsg(IDirectMusicPerformance* pPerf,
DMUS_PMSG* pPMSG)
{
if (pPMSG->dwType == DMUS_PMSGT_LYRIC)
{
// Read the lyric and do something based on what it says.
}
// No need to send this on down the line.
return DMUS_S_FREE;
}

Interpreting the Lyric

Once we have the lyric in hand, read it.
First, if working with both lyrics and notifications, check the
dwType field of the pmsg.
// Is this a lyric? (Only applied to reading from a Tool.)
if (pMsg->dwType == DMUS_PMSGT_LYRIC)
348 n Chapter 14

{
DMUS_LYRIC_PMSG *pLyric = (DMUS_LYRIC_PMSG *) pMsg;
// It's a lyric!
}

If it is a lyric, use a series of string comparisons to see what it is.

Remember that these are Unicode strings, so use the “L” to force
the comparison string to Unicode.
if (!wcscmp(pLyric->wszString,L"Boink!"))
{
// It's a Boink! Flatten Gruber's head momentarily!
}

That’s it. Since this code is probably called from within the Tool’s
ProcessPMsg() method, you do not even need to free the pmsg. Just
return DMUS_S_FREE and the Performance does it for you.

Case Study: Begonia’s Amazing Dance Moves

Although Jones has its own notification implementation, it doesn’t
actively do anything in response to the notifications. So, let’s write
something simpler that shows how to use them.
You are working on the sound design for the genre-shattering
new title, “Begonia’s Amazing Dance Moves.” This exciting new
adventure places Begonia’s writhing torso center stage. Your job is
to make it work. You decide to use notifications to communicate
state information back to the game engine.
First, define state variables for 1) Begonia’s mood, 2) whether
music is playing, and 3) her current dance move.
// Begonia's emotional state, placed in g_dwBegoniaMood.
#define BM_PLACID 1
#define BM_ANGUISHED 2
#define BM_WHO_ME 3
#define BM_UPBEAT 4
#define BM_DEAD 5
static DWORD g_dwBegoniaMood = BM_PLACID;

// Music status, tracked in g_dwMusicState.

#define MS_RUNNING 1
#define MS_STOPPED 2
#define MS_EXPIRING 3
static DWORD g_dwMusicState = MS_STOPPED;

// Begonia's dance moves, stored in g_dwBegoniaMove.

 Notifications n 349

#define BD_OFF 1 // Not dancing yet.

#define BD_START 2 // Dance command to start.
#define BD_JUMP 3 // Dance command to jump.
#define BD_WIGGLE 4 // Dance command to wiggle.
#define BD_NEXT 5 // Waiting for next dance command.
static DWORD g_dwBegoniaMove = BD_OFF;
// Also, track the music Segment.
static IDirectMusicSegmentState *g_pDanceMusic = NULL;

Now, write a routine that takes a pmsg, be it either lyric or notifica-

tion, and set the state variables appropriately. We assume there is
additional code (most likely a Tool, since this supports lyrics) to cap-
ture the pmsg and deliver it to the routine ProcessNotification():
void ProcessNotification(DMUS_PMSG *pMsg)
{
// Is this a lyric? (Only applied to reading from a Tool.) II
if (pMsg->dwType == DMUS_PMSGT_LYRIC)

Unit
{
DMUS_LYRIC_PMSG *pLyric = (DMUS_LYRIC_PMSG *) pMsg;
// Read the lyric and use to change game state, which will be
// picked up asynchronously by the main engine.
if (!wcscmp(pLyric->wszString,L"Agony Over"))
{
// Soul rendering theme is finished. Begonia can
// stop her tears now and be a little more upbeat.
g_dwBegoniaMood = BM_UPBEAT;
}
else if (!wcscmp(pLyric->wszString,L"Shout"))
{
// Someone shouts her name, Begonia reacts.
g_dwBegoniaMood = BM_WHO_ME;
}
}
else if (pMsg->dwType == DMUS_PMSGT_NOTIFICATION)
{
DMUS_NOTIFICATION_PMSG *pNotify = (DMUS_NOTIFICATION_PMSG *) pMsg;
DWORD dwNotificationOption = pNotify->dwNotificationOption;
if (pNotify->guidNotificationType == GUID_NOTIFICATION_PERFORMANCE)
{
if (dwNotificationOption == DMUS_NOTIFICATION_MUSICALMOSTEND)
{
// Uh oh, primary music Segment is about to end.
// Let main loop know to go ahead and schedule a new Segment.
g_dwMusicState = MS_EXPIRING;
}
}
else if (pNotify->guidNotificationType == GUID_NOTIFICATION_SEGMENT)
{
350 n Chapter 14

if (dwNotificationOption == DMUS_NOTIFICATION_SEGSTART)
{
// Is this the start of the dance music?
// It was scheduled to start on a measure, so we
// needed to wait and start once it was ready.
if (pMsg->punkUser == g_pDanceMusic)
{
// Yes! Begonia can jump up and start dancing.
g_dwBegoniaMove = BD_START;
}
}
}
else if (pNotify->guidNotificationType ==
GUID_NOTIFICATION_MEASUREANDBEAT)
{
// Begonia has rhythm. She moves in time with the music.
// Read the beat field to decide what to do.
// Each time g_dwBegoniaMove changes, the game engine
// sets her on a new path and returns g_dwBegoniaMove to BD_NEXT.
if (pNotify->dwField1 == 2)
{
// Begonia jumps on beat 3.
g_dwBegoniaMove = BD_JUMP;
}
else
{
// On any other beat, Begonia wiggles.
g_dwBegoniaMove = BD_WIGGLE;
}
}
}
// Return the pmsg to the Performance.
}

Depending on how you acquired the pmsg, you may need to free it
now:
// Return the pmsg to the Performance.
pPerf->FreePMsg(pMsg);

þ Note If this exercise has inspired you to name your first daughter
Begonia, please reconsider.
 Notifications n 351

Script Routines
Using scripting for notification is very different from performance
notifications or lyrics. Instead of writing C++ code to catch notifica-
tion or lyric messages, you write script routines that actively do
something at scheduled times in Segments.
Depending on what you intend to do with the notification, there
are different approaches:
n If the intention of the notification is to turn around and call a
script routine, then it clearly makes more sense to just call the
script in the first place! Likewise, if the intention is to play a
Segment, then it’s easy to write a script routine to do just that.
The beauty of these solutions is they never touch the C++ II
code.

Unit
n If the intention of the notification is to update a state variable,
then it’s just slightly more indirect. The script routine updates a
global variable in the script. Then, the main loop polls the vari-
able to read its value. This is actually very similar to the mecha-
nism we explored while choreographing Begonia’s performance.

Notifications in Jones
Let’s try the notification implementation in Jones and then see how
it’s done. If you installed the CD, run NotifyJones by selecting it
from the Start menu. Otherwise, double-click on its icon in the Unit
II\Bin directory or compile and run it from the 14_Notification
source directory.
Notice that a new display appears at the bottom-right corner.

The Notifications… button lets you select which notifications you’d

like to enable, and the box below it displays the notifications as the
Segments play.
352 n Chapter 14

By default, all notifications are disabled, so start by clicking on

the button. This opens the Notifications dialog.

Enable Performance, Segment, and Lyric and then click on OK.

Go to the Scripts section and click on the Open… button to read
in a script. Load SimpleScript.spt from the Media directory. It has
Segments that have been prepared with lyrics.
Double-click on StartLevel in the routine list to initiate play-
back of the primary Segment. Immediately, two notifications appear
in the display.

Seg Start indicates that a Segment started playing. Perf Start indi-
cates that it is also the very first Segment.
Double-click on Wasp. This triggers a call to Wasp() in the
script. Wasp() plays a secondary Segment, which also has a Lyric
Track.

Wasp() kicks off a flurry of notification activity. First, Seg Start indi-
cates that the new Segment has started. Then, Bzzzzz, Piff!, and
finally Zow! appear, as the Segment delivers each of these lyrics.
After a while, Seg Near End indicates that the Segment is almost fin-
ished, and finally Seg End appears as the Segment completes (if you
 Notifications n 353

look to the graphic display, you should see its rectangle moved com-
pletely to the left at this very moment).
Experiment a little more by turning on and off different notifica-
tions and playing the different script routines.

Jones Implementation
Jones doesn’t really use notifications in the same way that you would
for a game or other interactive application. Normally, you would want
to use notifications to trigger the application to run some code or set
a state variable. Instead, Jones just displays the text. But it’s impor-
tant that the notification system built into Jones as part of CAudio be
something that you can easily rip out and put to good use in a more II
useful way. So, the design focuses on making it easy to control, cap-

Unit
ture, and retrieve notifications. What you do with the notification
information is up to you.
We introduce the CNotificationManager class to control notifica-
tions. It handles all the busywork so that the tasks of enabling and
processing notifications are as simple as possible. CNotification-
Manager can:
n Enable and disable specific classes of notifications, including
lyrics
n Capture the notifications and store their information in an easy-
to-read-and-use format
n Provide easy retrieval of the notifications so the application can
act on them with a minimum of additional code
Let’s walk through the design.

Enabling Notifications
DirectMusic’s mechanism for classifying the notification categories
relies on GUIDs, which are a bit unwieldy. In order to avoid that
complexity throughout our code and also integrate lyric support in a
clean way, we represent each notification category with a bit flag. We
add one last bit flag to indicate a lyric. This allows us to have one
DWORD represent the on/off state of all notification categories,
including lyrics.
354 n Chapter 14

#define NOTIFYENABLE_PERFORMANCE 1 // GUID_NOTIFICATION_PERFORMANCE

#define NOTIFYENABLE_SEGMENT 2 // GUID_NOTIFICATION_SEGMENT
#define NOTIFYENABLE_TIMESIG 4 // GUID_NOTIFICATION_MEASUREANDBEAT
#define NOTIFYENABLE_CHORD 8 // GUID_NOTIFICATION_CHORD
#define NOTIFYENABLE_COMMAND 0x10 // GUID_NOTIFICATION_COMMAND
#define NOTIFYENABLE_RECOMPOSE 0x20 // GUID_NOTIFICATION_RECOMPOSE
#define NOTIFYENABLE_LYRIC 0x40 // Lyric pmsg

CNotificationManager has two routines that take these flags to

enable and disable notifications. They are called, appropriately,
EnableNotifications() and DisableNotifications().
EnableNotifications() takes one parameter, DWORD dwType-
Flags, which indicates which notification categories to turn on. For
each bit set, it calls the Performance’s AddNotificationType() method
with the appropriate GUID. Since lyrics are always on (there is no
Performance command to enable or disable lyric support), Enable-
Notifications() does not have to do anything for lyrics.
void CNotificationManager::EnableNotifications(DWORD dwTypeFlags)
{
if (dwTypeFlags & NOTIFYENABLE_PERFORMANCE)
{
m_pAudio->GetPerformance()->AddNotificationType(
GUID_NOTIFICATION_PERFORMANCE);
}
if (dwTypeFlags & NOTIFYENABLE_SEGMENT)
{
m_pAudio->GetPerformance()->AddNotificationType(
GUID_NOTIFICATION_SEGMENT);
}
... and so on for all notification categories, with the exception of
lyrics, which are always on…
// Now, set the flags so we remember what we enabled.
m_dwEnabledTypes |= dwTypeFlags;
}

Likewise, CNotificationManager::DisableNotifications() takes a

DWORD with bits for all the notifications to turn off, and it calls
RemoveNotificationType() with the appropriate GUID for each bit
set.
Along the same lines, we need a simpler system for tracking
each of the individual notification subtypes. We track the different
notifications with our own constants simply because this requires a
lot less work than the GUID + subfield organization of the Direct-
Music notifications. And, we can add lyric support under the same
umbrella.
 Notifications n 355

typedef enum _NOTIFICATION_TYPE

{
NOTIFICATION_NONE = 0, // Empty.
NOTIFICATION_SEGSTART = 1, // Segment started playback.
NOTIFICATION_SEGEND = 2, // Segment finished.
NOTIFICATION_SEGALMOSTEND = 3, // Segment about to finish.
NOTIFICATION_SEGLOOP = 4, // Segment looped.
NOTIFICATION_SEGABORT = 5, // Segment stopped prematurely.
NOTIFICATION_PERFSTARTED = 6, // First Segment started.
NOTIFICATION_PERFSTOPPED = 7, // Last Segment ended.
NOTIFICATION_PERFALMOSTEND = 8, // Current primary Segment almost ended.
NOTIFICATION_MEASUREBEAT = 9, // Beat marker.
NOTIFICATION_CHORD = 10, // Chord changed.
NOTIFICATION_GROOVE = 11, // Groove level changed.
NOTIFICATION_EMBELLISHMENT = 12, // Embellishment played.
NOTIFICATION_RECOMPOSE = 13, // Track recomposed.
NOTIFICATION_LYRIC
} NOTIFICATION_TYPE;
= 14 // Lyric. II

Unit
CNotificationManager has a method, TranslateType(), that reads a
pmsg and returns a NOTIFICATION_TYPE constant, which can
then be used to track the notification in a more friendly way.
NOTIFICATION_TYPE CNotificationManager::TranslateType(DMUS_PMSG *pMsg)
{
if (pMsg->dwType == DMUS_PMSGT_LYRIC)
{
return NOTIFICATION_LYRIC;
}
else if (pMsg->dwType == DMUS_PMSGT_NOTIFICATION)
{
DMUS_NOTIFICATION_PMSG *pNotify = (DMUS_NOTIFICATION_PMSG *) pMsg;
DWORD dwNotificationOption = pNotify->dwNotificationOption;
if (pNotify->guidNotificationType == GUID_NOTIFICATION_PERFORMANCE)
{
if (dwNotificationOption == DMUS_NOTIFICATION_MUSICSTARTED)
{
return NOTIFICATION_PERFSTARTED;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_MUSICALMOSTEND)
{
return NOTIFICATION_PERFALMOSTEND;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_MUSICSTOPPED)
{
return NOTIFICATION_PERFSTOPPED;
}
}
else if (pNotify->guidNotificationType == GUID_NOTIFICATION_SEGMENT)
{
356 n Chapter 14

if (dwNotificationOption == DMUS_NOTIFICATION_SEGSTART)
{
return NOTIFICATION_SEGSTART;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_SEGALMOSTEND)
{
return NOTIFICATION_SEGALMOSTEND;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_SEGEND)
{
return NOTIFICATION_SEGEND;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_SEGABORT)
{
return NOTIFICATION_SEGABORT;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_SEGLOOP)
{
return NOTIFICATION_SEGLOOP;
}
}
else if (pNotify->guidNotificationType ==
GUID_NOTIFICATION_MEASUREANDBEAT)
{
return NOTIFICATION_MEASUREBEAT;
}
else if (pNotify->guidNotificationType == GUID_NOTIFICATION_CHORD)
{
return NOTIFICATION_CHORD;
}
else if (pNotify->guidNotificationType == GUID_NOTIFICATION_COMMAND)
{
if (dwNotificationOption == DMUS_NOTIFICATION_GROOVE)
{
return NOTIFICATION_GROOVE;
}
else if (dwNotificationOption == DMUS_NOTIFICATION_EMBELLISHMENT)
{
return NOTIFICATION_EMBELLISHMENT;
}
}
else if (pNotify->guidNotificationType == GUID_NOTIFICATION_RECOMPOSE)
{
return NOTIFICATION_RECOMPOSE;
}
}
return NOTIFICATION_NONE;
}
 Notifications n 357

Capturing Notifications
To capture both notifications and lyrics, CNotificationManager imple-
ments an IDirectMusicTool interface so it can insert itself directly in
the Performance and intercept DMUS_NOTIFICATION_PMSG and
DMUS_LYRIC_PMSG messages. To make this possible, we base the
CNotificationManager class on the IDirectMusicTool interface.
CNotificationManager then provides its own implementation of all
the IDirectMusicTool methods.
class CNotificationManager : public CMyList, public IDirectMusicTool
{
public:
// IUnknown methods.
STDMETHODIMP QueryInterface(const IID &iid, void **ppv); II
STDMETHODIMP_(ULONG) AddRef();
STDMETHODIMP_(ULONG) Release();

Unit
// IDirectMusicTool methods.
STDMETHODIMP Init(IDirectMusicGraph* pGraph) ;
STDMETHODIMP GetMsgDeliveryType(DWORD* pdwDeliveryType ) ;
STDMETHODIMP GetMediaTypeArraySize(DWORD* pdwNumElements ) ;
STDMETHODIMP GetMediaTypes(DWORD** padwMediaTypes,
DWORD dwNumElements) ;
STDMETHODIMP ProcessPMsg(IDirectMusicPerformance* pPerf,
DMUS_PMSG* pPMSG) ;
STDMETHODIMP Flush(IDirectMusicPerformance* pPerf,
DMUS_PMSG* pPMSG,
REFERENCE_TIME rtTime) ;

Most of the methods handle the process of setting up the Tool. When
it is installing the Tool, DirectMusic calls Init(), GetMsgDelivery-
Type(), GetMediaTypeArraySize(), and GetMediaTypes(). Depending
on what the Tool needs to do, it can implement these as needed.
Init() allows the Tool to initialize its internal structures at the
time DirectMusic places the Tool in a Graph. For our purposes, there
is nothing to do here, so the method simply returns S_OK.
HRESULT CNotificationManager::Init(IDirectMusicGraph* pGraph)
{
// Don't need to do anything at initialization.
return S_OK;
}

GetMsgDeliveryType() tells DirectMusic when to deliver the mes-

sages. In this case, the Tool wants the messages delivered at queue
time.
358 n Chapter 14

HRESULT CNotificationManager::GetMsgDeliveryType(DWORD* pdwDeliveryType )

{
// This Tool should process just a tad before the time stamp.
*pdwDeliveryType = DMUS_PMSGF_TOOL_QUEUE;
return S_OK;
}

DirectMusic also needs to know which message types to send

through this Tool. Although a Tool can elect to simply ignore this and
receive all pmsgs, doing so is a bad idea for two reasons. First,
receiving all PMsgs incurs extra overhead (admittedly minimal, but
still…) to process everything and not just the messages we care
about. Second, it causes all messages to be held up by this delivery
time before they can proceed to the next stage. With a delivery of
DMUS_PMSGF_TOOL_QUEUE, the timing is right up against the
edge. If the delivery option becomes DMUS_PMSGF_TOOL_
ATTIME, we are in trouble because note and wave messages end up
delivered to the DirectMusic Synth too late, resulting in horribly
stuttered timing.
Therefore, it is best to implement the pair of methods GetMedia-
TypeArraySize(), which specifies the number of message types, and
GetMediaTypes(), which then copies over the list of message types.
HRESULT CNotificationManager::GetMediaTypeArraySize(DWORD* pdwNumElements )
{
// We have two media types to process: lyrics and notifications.
*pdwNumElements = 2;
return S_OK;
}

HRESULT CNotificationManager::GetMediaTypes(DWORD** padwMediaTypes,

DWORD dwNumElements)
{
// Return the two types we handle.
DWORD *pdwArray = *padwMediaTypes;
pdwArray[0] = DMUS_PMSGT_LYRIC;
pdwArray[1] = DMUS_PMSGT_NOTIFICATION;
return S_OK;
}

As we have seen several times, the heart of a Tool is really its

ProcessPMsg() method. This is the code that receives the pmsg and
does something with it. In this case, it doesn’t do too much. Our
code turns around and calls an internal method, InsertNotification(),
 Notifications n 359

and then tells DirectMusic to discard the pmsg. We discuss Insert-

Notification() next.
HRESULT CNotificationManager::ProcessPMsg(IDirectMusicPerformance* pPerf,
DMUS_PMSG* pPMSG)
{
// This should always be a lyric or notification,
// but check just to be safe.
if (( pPMSG->dwType == DMUS_PMSGT_LYRIC ) ||
( pPMSG->dwType == DMUS_PMSGT_NOTIFICATION ))
{
InsertNotification(pPMSG);
}
// No need to send this on down the line.
return DMUS_S_FREE;
}
II
InsertNotification() provides the glue to convert the message into a
notification event that can be retrieved asynchronously by the main

Unit
loop. It takes the pmsg and uses it to create a CNotification object,
which it places in its queue. Since the queue itself can be accessed
either from the Tool processing thread or the main loop thread, this
code must be protected with a critical section.
void CNotificationManager::InsertNotification(DMUS_PMSG *pMsg)
{
// TranslateType reads the pmsg and figures out the
// appropriate NOTATION_TYPE for it.
NOTIFICATION_TYPE ntType = TranslateType(pMsg);
// If this is a lyric, it still might not be enabled
// for capture, so check first.
if ((ntType != NOTIFICATION_LYRIC) ||
(m_dwEnabledTypes & NOTIFYENABLE_LYRIC))
{
// Allocate a CNotation structure to store the
// notification information.
CNotification *pNotification = new CNotification;
if (pNotification)
{
// The Init() routine fills the CNotification with
// name, type, and optional object pointer.
pNotification->Init(pMsg,ntType);
// Since this can be called from a different thread,
// we need to be safe with a critical section.
EnterCriticalSection(&m_CriticalSection);
AddTail(pNotification);
LeaveCriticalSection(&m_CriticalSection);
360 n Chapter 14

}
}
}

The CNotification class tracks one instance of a notification that has

been captured on playback. CNotification is based on the class
CMyList, so it can be managed in a linked list. CNotification stores
several parameters that it fills when it translates the original pmsg:
n A name, m_szName: If a lyric, this stores the actual lyric text.
If a performance notification, it stores a description of the
notification.
n The notification type, m_ntType: This is an identifier from
the enumerated set of NOTIFICATION_TYPEs.
n An optional pointer to an object, m_pObject: Some perfor-
mance notifications include an IUnknown pointer to an object.
This maintains its own internal reference to the object and
releases it when deleted.
n An optional data field, m_dwData: If a performance notifica-
tion uses one of the data fields in the message, it is copied here.
n The time stamp of the notification, m_rtTimeStamp: This
is the exact time that the action that the notification represents
will occur. Since the notifications are delivered slightly ahead of
time, retaining the exact intended time is important.
class CNotification : public CMyNode
{
public:
CNotification();
~CNotification();
CNotification* GetNext() { return (CNotification*)CMyNode::GetNext();}
void Init(DMUS_PMSG *pMsg,NOTIFICATION_TYPE ntType);
char *GetName() { return m_szName; };
void GetTimeStamp(REFERENCE_TIME *prtTime) { *prtTime = m_rtTimeStamp; };
NOTIFICATION_TYPE GetType() { return m_ntType; };
IUnknown * GetObject() { return m_pObject; }; // Note: This doesn’t AddRef!
DWORD GetData() { return m_dwData; };
private:
char m_szName[40]; // Name for notification.
NOTIFICATION_TYPE m_ntType; // Notification type.
REFERENCE_TIME m_rtTimeStamp; // Time the notification occured.
IUnknown * m_pObject; // Optional object pointer.
DWORD m_dwData; // Data field, borrowed from pmsg.
};
 Notifications n 361

The only substantial method, Init(), takes a DMUS_PMSG that is

either a lyric or notification and uses it to initialize its fields. All of
the other methods return data from the fields.

Retrieving Notifications
As the notifications are being captured by the CNotificationManager,
it stores them in its internal list. The application can then call
CNotificationManager::GetNextNotification() to retrieve the oldest
CNotification in the queue. When done with a retrieved
CNotification, the application must delete it.
CNotification *pNotification = pManager->GetNextNotification();
if (pNotification)
{ II
// Do something.
delete pNotification;

Unit
}

Now, let’s look and see how CAudio integrates CNotification-

Manager.

CAudio Integration
For notification support, CAudio adds a pointer to a CNotification-
Manager and two methods to access it:
public:
// Methods for managing notifications.
CNotification *GetNextNotification(); // Returns next notification in queue.
void EnableNotifications(DWORD dwType,bool fEnable);
private:
// CNotificationManager captures and queues notifications.
CNotificationManager * m_pNotifications;

CAudio does not create the CNotificationManager by default. Since it

is only needed in the event that the application wants to track notifi-
cations, CAudio creates its CNotificationManager when the applica-
tion calls EnableNotifications() the first time. At that point, it creates
an instance of CNotificationManager and places it as a Tool in the
Performance’s Tool Graph. The CNotificationManager object can be
manipulated as a Tool because it exposes the IDirectMusicTool
interface.
362 n Chapter 14

Once CAudio has installed CNotificationManager, calls to

CAudio::EnableNotifications() simply pass through to CNotification-
Manager::EnableNotifications() and CNotificationManager::
DisableNotifications().
Note that since CNotificationManager is managed as a COM
object, the last object to Release() it causes it to go away. Even
CAudio’s pointer to CNotificationManager is treated as a reference.
void CAudio::EnableNotifications(DWORD dwType, bool fEnable)
{
if (!m_pNotifications)
{
IDirectMusicAudioPath *pPath;
if (SUCCEEDED(m_pPerformance->GetDefaultAudioPath(&pPath)))
{
IDirectMusicGraph *pGraph = NULL;
pPath->GetObjectInPath(0,
DMUS_PATH_PERFORMANCE_GRAPH, // The Performance Tool Graph.
0,
GUID_All_Objects,0, // Only one object type.
IID_IDirectMusicGraph, // The Graph interface.
(void **)&pGraph);
if (pGraph)
{
m_pNotifications = new CNotificationManager(this);
if (m_pNotifications)
{
// Insert it in the Graph. It will process all lyric and
// notification pmsgs that are played on this Performance.
pGraph->InsertTool(
static_cast<IDirectMusicTool*>(m_pNotifications),
NULL,0,0);
}
pGraph->Release();
}
pPath->Release();
}
}
// Okay, hopefully we have a CNotificationManager.
if (m_pNotifications)
{
// If enabling or disabling, choose the appropriate routine.
if (fEnable)
{
m_pNotifications->EnableNotifications(dwType);
}
else
{
 Notifications n 363

m_pNotifications->DisableNotifications(dwType);
}
}
}

CAudio::GetNextNotification() simply passes through to CNotifica-

tionManager::GetNextNotification().
CNotification *CAudio::GetNextNotification()
{
if (m_pNotifications)
{
return m_pNotifications->GetNextNotification();
}
return NULL;
}

Finally, CAudio::Close(), which the application calls when it is fin-

II
ished with CAudio, releases the CNotificationManager.

Unit
if (m_pNotifications)
{
m_pNotifications->Release();
m_pNotifications = NULL;
}

To use the notifications, Jones works a little differently than a typical

application might. Jones displays notifications instead of using
notifications to trigger changes in behavior. Jones keeps the most
recent eight CNotifications in a queue so that it can continuously dis-
play them in its notification display.
The following code is called every frame (in Jones, this is every
30 milliseconds). It reads any new notifications from the Perfor-
mance and places them in its own queue. It then culls the queue to
keep it limited to eight. If the queue changed, it redraws the display.
bool fChanged = false;
if (m_pAudio)
{
CNotification *pNotif;
// Pull in new notifications and add them to the list.
while (pNotif = m_pAudio->GetNextNotification())
{
m_NotificationList.AddTail(pNotif);
fChanged = true;
}
// We want to keep a list of the last eight notifications for drawing.
// So, if the list is longer than eight, remove the items at the top.
364 n Chapter 14

while (m_NotificationList.GetCount() > 8)

{
pNotif = m_NotificationList.RemoveHead();
delete pNotif; // Just delete the CNotification when done with it.
fChanged = true;
}
}
if (fChanged)
{
// We can draw the new list in the display.

Case Study: Return to Begonia’s Amazing Dance Moves

Okay, since Jones’ use of CNotificationManager is a little nonstan-
dard, let’s take a look at how the Begonia dev team might use it.
This is closer to how you might use it in your application.
First, determine which notifications to allow with a call to
EnableNotifications().
m_pAudio->EnableNotifications(NOTIFYENABLE_LYRIC | // Enable lyrics
NOTIFYENABLE_PERFORMANCE | // Enable Performance
NOTIFYENABLE_SEGMENT | // Enable Segment
NOTIFYENABLE_TIMESIG,true);// Enable time signature

This activates capture of just the notification categories that we care

about.
Then, we need code to read and process notifications. This code
should be called once per frame. It reads the current notifications
and responds to them as appropriate by setting global state variables.
The code is pretty self explanatory:
CNotification *pNotif;
while (pNotif = m_pAudio->GetNextNotification())
{
switch (pNotif->GetType())
{
case NOTIFICATION_LYRIC :
if (!strcmp(pNotif->GetName(),"Agony Over"))
{
g_dwBegoniaMood = BM_UPBEAT;
}
else if (!strcmp(pNotif->GetName(),"Shout"))
{
g_dwBegoniaMood = BM_WHO_ME;
}
break;
case NOTIFICATION_PERFALMOSTEND :
 Notifications n 365

g_dwMusicState = MS_EXPIRING;
break;
case NOTIFICATION_SEGSTART :
if (pNotif->GetObject() == g_pDanceMusic)
{
g_dwBegoniaMove = BD_START;
}
break;
case NOTIFICATION_MEASUREBEAT :
if (pNotif->GetData() == 2)
{
g_dwBegoniaMove = BD_JUMP;
}
else
{
g_dwBegoniaMove = BD_WIGGLE;
}
break;
II
}

Unit
// Done with the notification.
delete pNotif;
}
}

Until now, we’ve focused on DirectX Audio techniques for building a

sound and music environment that responds effectively and realisti-
cally to user and game stimuli.
However, for the best possible experience, the control should go
both ways, and in some cases audio should drive gameplay and visu-
als. With notification support, we have the tools to do so.
This page intentionally left blank.
Unit III

DirectX Audio
Case Studies

Chapter 15: A Direct Music Case Study for Russian Squares for
Windows XP Plus Pack
Chapter 16: A DirectMusic Case Study for No One Lives Forever
Chapter 17: A DirectSound Case Study for Halo
Chapter 18: A DirectMusic Case Study for Interactive Music on
the Web
Chapter 19: A DirectMusic Case Study for Worms Blast
Chapter 20: A DirectMusic Case Study for Asheron’s Call 2:
The Fallen Kings
Chapter 21: Beyond Games: Bringing DirectMusic into the
Living Room

367
This page intentionally left blank.
 Chapter 15

A DirectMusic Case Study

for Russian Squares for
Windows XP Plus Pack

Introduction and Overview

R ussian Squares is a puzzle game involving rows and columns of

squares. The player eliminates rows of squares by matching the
square’s color or shape. The game’s difficulty increases as the player
eliminates the rows of squares; the time given decreases and
“blockers” are introduced to get in the player’s way. It sounds simple
enough, but the game is both challenging and extremely addictive!
The adaptive audio design of Russian Squares offers a wonderful
opportunity to demonstrate how an adaptive score works seamlessly
with core game design elements. When creating a musical score for a
game, start by determining what the game calls for. I approach each
game without any particular adaptive technique in mind and brain-
storm in the abstract. This way, creative ideas lead to technical
decisions. Once I have an abstract idea of how the score will work,
the engineering team and I figure out how to execute the adaptive
music system with the game engine. I believe that an adaptive audio
design document, which should complement or even be a part of the
game design document, is an important step in creating a highly
adaptive score.

369
370 n Chapter 15

Adaptive Elements
The adaptive elements of a game determine the form of an adaptive
score. An important first step is identifying potential adaptive ele-
ments that tie gameplay to the music. Start by asking questions
about the gameplay. What is the core gameplay element? How does
it function? What are secondary gameplay elements? In what ways
can I link the music to this gameplay?

Music Cells
The core gameplay of Russian Squares is the elimination and addi-
tion of rows of squares. We made this game design element the core
of the music functionality. As the game adds rows or the player elimi-
nates them, the music responds with a subtle change; the adaptive
music system adds or subtracts an instrument, the harmony or
rhythm changes, etc. The music follows the overall pace of the
player. To accomplish this, there are about 50 music cells per compo-
sition, which correlate to different groove levels. As the player
completes rows, the music incrementally transitions to the next
groove level. Logical transition boundaries make those transitions
musical and seamless. I used mainly measure boundaries for transi-
tions in Russian Squares.
I used music intros, ends, and breaks to kick off and end
gameplay. The music transitions to an ambient break section when
the player pauses the game. The adaptive music system plays the
appropriate intros and ends based on the current music cell.

Variation
Russian Squares uses instrument-level variation to keep the individ-
ual cells from getting monotonous. Each groove level is anywhere
from two to eight measures in length and repeats as the player
works on a given row. Within DirectMusic, each instrument can have
up to 32 variations. When combined with other instruments, these
variations increase the amount of variation logarithmically. Most
often, one to three instruments per cell use variation, and that gives
the music an organic, spontaneous feel and prevents that all-too-
familiar loopy feeling. Too much variation, however, can unglue the
music’s cohesion.
 A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 371

Music Layers
Layering short musical gestures over the currently playing groove
level accents many gameplay elements, such as the game clock, the
clearing and adding of rows, and blockers. I use DirectMusic motifs
and secondary Segments to accomplish this. These elements line up
rhythmically with the music (i.e., the tempo of the clocks synchro-
nizes with quarter note values of the music).

Sounds
DLS-2 banks allowed for maximum flexibility and variation in this
project. Some of the instrument variations control instrument filters
(i.e., low pass filter sweeps) in addition to notes, which are crucial
for the genre of music. The sounds used are a combination of Sonic
Implants (www.sonicimplants.com) and custom banks.

Cooperative Composing
Three other composers (Erik Aho, Mike Min, and Bryan Robbins)
assisted in creating the three arrangements. Working with nonlinear
cells made it easy to work simultaneously. Generally, we assigned a
range of cells to each composer. Often, someone would start an idea
and the next composer would elaborate on it or add variations. This
method of working made sharing musical ideas easy and kept every-
thing musically coherent.
III
Creating the Adaptive Audio Design
Part

Russian Squares: How the Game Plays

Russian Squares is a fast-paced puzzle game that grows with inten-
sity. The object is to eliminate rows of blocks by matching colors or
shapes faster than the clock. Each level starts with a square made up
of rows of blocks. The player controls an active square with a mouse
or arrow keys. The active block can move around the perimeter of
the square or push into the square, causing the square directly
across from it to pop out and become the new active square. The
first level’s square starts with blocks set up six by six. If time runs
out before the player matches a row, the game adds a row to the
372 n Chapter 15

square. As the player eliminates rows, the square decreases in size

(six by five, five by five, five by four, etc.) until the square disap-
pears. This constitutes the completion of the first level. Each
subsequent level begins with a larger square with more rows to
eliminate against a faster clock.
There is another obstacle thrown in as the game progresses.
Blockers are blocks within the square that cannot be moved.
Blockers appear in the square for a few seconds, get in the way of
the player’s actions, and then disappear or switch to another location
in the square. They definitely add a challenge and twist to the game.
There are nine levels in Russian Squares. A new game starts you
on level three; this is because there is the potential for the player to
move back a few levels! The progression of levels plays an important
part in how I created the music and integrated it into the game.

Aural and Visual Themes

There are three visual themes for Russian Squares: Neon, Candy,
and Shapes. The gameplay is identical in each, except that with
Shapes the player matches block shapes instead of color. Each theme
has a distinct aesthetic. Neon has a glowing green, high-tech feel.
Candy has a light-hearted, silver look. Shapes portrays a darker red
tone. My team wrote three pieces of adaptive music that correlate
with each visual theme. Spin Cycle sounds edgy and techy to compli-
ment Neon, Bubble Matter floats on light synthesizer pads over
drum and bass riffs for Candy, and the down-tempo groove of Gravity
Ride matches the darkness of its visual counterpart, Shapes.

The Musical Form

Form, in a nonlinear piece of music, abides by the same basic princi-
ples as linear music. Dynamics, harmony, orchestration, and the arc
of the form are critical elements for any composition. Linear music,
however, follows a singular path created by the composer, while non-
linear or adaptive music chooses one of many potential paths. This
does not mean that adaptive music is formless. Its form lies within
the boundaries set for the music. Creating these boundaries presents
a new set of challenges for game composers.
A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 373

For example, each potential outcome must make sense musically

and adhere to the overall consistency of the piece. It is important for
an adaptive piece of music to adhere to an overarching form, despite
its many paths. The composer builds a musical structure in which
the adaptive elements are contained. For example, a snowflake has a
form that describes its basic elements — size, temperature, weight,
and its crystalline form. Every snowflake differs slightly, yet they are
all snowflakes. Adaptive music functions in this way.
The overarching form of Russian Squares is one of increasing
intensity. Playing all of the music cells (DirectMusic patterns)
sequentially results in a gradual building of musical elements, yet the
performances that the gameplay produces vary greatly. The adaptive
element of the gameplay provides the dynamic ebb and flow of
music, while the instrument-level variation adds further nonlinearity.
The form takes into account the fact that the music cells will not
occur sequentially but will slide up and down the many groove
levels.
Each groove level/pattern usually has one or two musical differ-
ences from its closest neighbor. That difference is an instrument
addition/subtraction, an instrument change, a chord change, or a
rhythmic shift. We arranged them in such a way that the music grad-
ually moves from one large musical idea to another over the course
of several groove levels. This gave the music a sense of form.
The tempo of each composition remains constant. We scrapped
the whole idea of tempo changes for a couple of reasons. First, III
electronica does not usually feature gradual accelerando as a part of Part
the form, so it sounded ridiculous. Second, it added some technical
hurdles given our DirectMusic approach. The game would have to
call tempo changes directly (with “increase tempo” or “decrease
tempo” calls) or via controlling secondary Segments, which adds
another adaptive design layer. Each style/pattern would need to func-
tion well within a range of tempos, and some of the DLS banks (i.e.
drum grooves) were built around a specific tempo. We kept this
aspect simple by having only one tempo per composition. To prevent
it from sounding static, we used tricks such as rhythmic modulation
to increase the music’s sense of pulse. For example, the rhythm
changes dramatically over five groove levels in Spin Cycle. The
score achieves this change by introducing the new rhythm in one
374 n Chapter 15

groove level and then gradually emphasizing it (while

de-emphasizing the old rhythm) over the course of the next several
groove levels. Each player experiences the shift at a different rate,
depending on how well they play!

The Adaptive Goal and Core Objectives

The arc and progression to Russian Squares is one of growing inten-
sity. However, it is not a straight linear growth of intensity through
the levels. Each level begins calm and ends frantic, relatively speak-
ing, but each successive level begins and ends slightly more intense
than the last.

Level 1 Level 2 Level 3 Level 4 Level 5

Figure 15-1: Intensity Range

Our first objective for the adaptive score was to mirror the progres-
sion of game levels. This approach involved creating a score that
increased in intensity from the start to the end of each level (micro-
adaptability), while each successive level began at a higher intensity
than the last (macro-adaptability). Within the game levels, we focus
on the primary gameplay element of clearing rows. A musical change
occurs each time the player clears a row, a reward that lets the
player know he is on the right track. In addition, the change indicates
the growing intensity as the rows clear. Running out of time and hav-
ing a row added is the negative counterpart to clearing a row. The
score reflects this state as well.
 A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 375

Russian Squares uses an additive-subtractive approach to the

music, in which the music engine can add or peel away instrument
layers. Given that the time it takes to clear a row can be anywhere
from a few seconds to a few minutes, this is a very practical solution.
It allows for musical development while maintaining direction and
momentum. Implementing this technique has some complications,
however. In terms of aesthetics, simply moving along a linear curve
from low intensity to high intensity does not work, as adding layer
after layer and increasing the tempo is not very musical. The music
requires larger sections, instrumentation changes, rhythmic shifts,
and tonal changes to stay interesting. In this way, the music pro-
gresses in a multidimensional manner. This concept is the backbone
for the adaptive audio design of Russian Squares.
We mirror many other gameplay elements with musical ges-
tures. Most of these are musical accents indicating important events
as they occur in the game. DirectMusic motifs layer these accents
over the main body of music. These motifs include row clear, row
add, blocker, time running low, high score, and active square. We also
use musical gestures for start game, end game, and pause game.
Another objective is creating three compositions that share the
same core functionality. Three differing pieces of music react in the
same manner to the gameplay, and their common adaptive frame-
work allows them to be interchangeable. Having multiple
compositions underscores the need to establish a solid adaptive
framework prior to composing. Using quality instrument samples III
goes beyond aesthetics. Sounds also need to be flexible in an adap- Part
tive score. DLS instruments offer this flexibility because single note
samples can react more quickly than prerecorded phrases.

Inherent Obstacles
The biggest trick was creating a score that could gradually increase
or decrease in intensity at any time while creating a satisfying musi-
cal experience. How could we put such a score together? How would
the layers function? How would transitions operate? The overall
objective seemed simple enough, but just thinking about execution
for a few moments brought up all kinds of tough questions. More-
over, digging into the production of the score unearthed many more!
The music in Russian Squares needs to change quickly. Consider
that it may take the player a minute or two to clear a row at the start
376 n Chapter 15

of the game, yet toward the end of any level, the player can clear a
row or the game can add one every few seconds. This poses a curi-
ous problem. If music changes every minute or so, you should use
longer sections to better hold the listener’s interest and avoid repeti-
tion. However, when the music changes every few seconds, it must
not disrupt the aesthetic flow of the phrases. A successful adaptive
score strikes a balance between the length of phrases, their transi-
tion boundaries, and the music’s responsiveness to game changes.
In the modern electronica genre, nothing can make the music
cheesier than wimpy sounds. Electronica lives and dies by its
grooves, fat synthesizer patches, and filter sweeps. It depends
heavily on timbres. This demand for great sounds creates a challenge
for the audio producer facing severe memory constraints.
The last challenge we needed to overcome was repetition. Any
game composer understands the challenge of avoiding undue repeti-
tion in game scores. Russian Squares was no exception. A player can
easily spend a lot of time solving a given level. As the difficulty rises,
rows are cleared, added, cleared, and added again, triggering the
same music cells. Luckily, DirectMusic offered us many ways to deal
with this and the rest of the challenges we faced during production of
our adaptive score.

The DirectMusic Solution

After defining the music design goals for Russian Squares, I turned
to DirectMusic. I thought of the music in terms of small “cells,” or
phrases, that correlated to rows in a level — one cell per row. Given
my experience with DirectMusic, I saw three possible solutions to
the core design objective. We could use DirectMusic primary Seg-
ments as the main musical unit, so when a player cleared a row, a
transition to a new Segment occurred. We could build the composi-
tion out of secondary Segments that layered onto or peeled away
from the mix sequentially. Alternately, we could use Styles and their
groove levels as the core musical unit; this way, the game
transitioned up and down the groove levels as the player cleared
rows or the game added them.
I experimented with the primary Segment solution. I thought
Segments allowed for more flexibility with the form, such as tempo
A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 377

changes and longer, less repetitive phrases. However, our first few
tests showed some glaring problems with this approach. It worked
well when the music changed every minute or so, but when the
music changed more often, it sounded a bit stilted or unnatural. In
addition, the programmers told us that primary Segments would not
respond as quickly or efficiently in the game as groove levels.
Next, I tried layering secondary Segments but quickly learned
that this method was a logistical nightmare, both technically and
musically. Getting more than three or four secondary Segments to
line up musically and respond quickly to game calls was not practical.
I also experimented with a muting/unmuting system using control-
ling secondary Segments to get the same effect, but again it did not
turn out to be practical for this design.
We use Styles and groove levels in this application, as they
respond quickly. Each groove level associates with a row. If the
player clears a row, the game increases the groove level by one, and
if the game adds a row, it decreases the groove level by one. There-
fore, the game engine slides the set of groove levels up and down as
the game progresses. Each groove level consists of a musical phrase
that is two to 12 measures long and transitions easily to the next
groove level. We use measure boundaries as transition points
between groove levels, as they allow for relatively quick transitions
while remaining musical. There are 45 groove levels for each
composition.
Using the Style/groove level system, we take full advantage of III
DirectMusic’s intros, ends, and breaks. Anytime a new game or new Part
level begins, DirectMusic plays an appropriate intro leading into the
appropriate groove level. When the game ends or the player com-
pletes a level, the music seamlessly plays an ending. When the game
pauses, a DirectMusic break plays until the player resumes the
game. All of this gives the game score a polished and professional
feel.
We use secondary Segments and layer them over the groove lev-
els for the musical accents that emphasize specific game events: row
cleared, row added, blockers, and the clock running out of time. Even
though the groove level changes when the player clears a row or the
game adds one, we want a consistent underscore that occurs simul-
taneously with the row change. Another secondary Segment plays
every time a blocker appears in the game. These secondary
378 n Chapter 15

Segments play on the grid boundary (in this case, the next sixteenth
note) of the underlying groove level so that they sound virtually
instantaneous, yet rhythmically line up with the music. As time is
running out on the clock (ten seconds to go), a looping secondary
Segment plays until time runs out or the player clears the row. This
Segment begins on the beat boundary so that the “tick tock” sound
lines up with the beat of the groove level. Each composition uses a
different sound set for these secondary Segments. This way, all the
sounds are integral to each composition and add coherence. For
example, the clock for Spin Cycle is a buzzer sound; for Bubble Mat-
ter, it’s a wooden “tick tock”; Gravity Ride uses the clank of metal
spikes to mark time.

Harmonic Considerations
For this game, I chose not to take advantage of DirectMusic’s adap-
tive harmonic abilities, such as ChordMaps or Chord Tracks. The
simple harmonic structure of the music did not merit the extra pro-
duction time that it would have taken to make it function well within
the design. Frequent or complex harmonic and chordal shifts did not
fit the aesthetic of electronica music. I created the music’s harmonic
or chordal changes directly in the pattern editor. Harmonic changes
occur when moving from one groove level to the next, which corre-
lates to a row being cleared or added.

The Production Process

I kicked off the production process by gathering the initial DLS
sounds for each composition. Then, setting the tone and main
themes for the pieces, I divided the rest of the composing between
my team and myself. Rather than having one person assigned to each
composition, we all worked on every song. Each composer was
responsible for a range of groove levels in each piece. I periodically
merged all of our work so that we were aware of each other’s prog-
ress. Toward the end, I polished and mixed the pieces, ensuring
coherence.
 A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 379

Gathering, Creating, and Assembling DLS Banks

I find working in DirectMusic’s DLS creation environment to be
straightforward. Its interface, while not perfect, is similar to other
sample-bank creation software. The process consists of importing
wave samples, creating an instrument, and assigning samples to
regions across the keyboard. The DLS Level 2 standard allows for
the layering of samples and adds basic low pass filtering.
The trick to good sound is not in the tool but in the experimenta-
tion process. Many composers like to get right to the notes and use
whatever sounds are available. This can lead to a generic-sounding
piece of music. Creating sounds with a specific piece of music in
mind gives the sounds immediate relevance. There is a lot of back-
and-forth between tweaking a sound and trying it in the context of
the music. The process boils down to a few key points:
n Quality samples
n Instrument layers
n Real-time filters
n Good use of the ADSR amplitude envelope
n Efficient use of samples
n Real-time effects, such as reverb, delay, etc.
We created an individual sound set for each piece of music. We honed
each sound set to a specific composition, giving each piece a unique III
sound. The development team set aside 8MB for each song’s sample Part
set, for a total of 24MB of instruments. In the end, we kept the sam-
ple rate at 44.1 kHz for pristine sound. The game loads all 24MB of
instrument banks at the start so that the player can switch between
songs without pauses. We designed the sounds from scratch or used
licensed sounds from Sonic Network (the instruments are called
Sonic Implants) at www.sonicnetworkinc.com. Sonic Network has an
excellent electronica collection, which served as our point of sonic
departure.
Each composition has its own rhythmic feel and flavor that the
sound sets portray. There are live drum loops diced into quarter note
sections; these loops live in a DLS instrument, so they can play back
in their original form, or we can mix and match them in a variety of
380 n Chapter 15

rhythms. This loop-slicing technique is common to the electronica

genre and allows a live drum feel with added flexibility and variation.
For percussion we use standard drum machine sounds, such as
Roland 808, Yamaha R8, and other obscure beat boxes. Hollywood
Edge has a great collection of classic sounds called Beat Machine. I
got specific permission to use these and other Hollywood Edge
sounds in our DLS banks. I also used a Roland Jupiter synthesizer (a
little white noise and a filter gave an industrial flavor to Gravity
Ride) and some samples generated from percussive metal spike
sounds.
We used samples of synthesizers as an integral element of the
sound palette. We spent a lot of time building and layering synthetic
instrument sounds to sound thick, rich, and mean when they needed
to. Many of these instruments have three or four layers, which are
panned and slightly detuned to add density to the sounds. I also took
advantage of real-time filters as the primary musical gesture on both
the lead synthesizer and the rhythmic synthesizer of Spin Cycle.

Setting Up Styles
We set up each of the three compositions identically in DirectMusic
Producer. One style held all of the groove levels, the motifs, and the
Band for a given piece. We set up the Band (within the Style) with
the pchannels assigned to the custom DLS instrument patches after
we created the Style. We then set the instrument’s levels and pan-
ning for a rough mix, tweaking them during the composition process,
and created our patterns.

The Band and Mixing

Each composition uses only one DirectMusic Band. I find construct-
ing one Band, with all of the instruments assigned to different
pchannels, easier than using multiple Bands and switching between
them to change instrument sets. The unlimited number of available
pchannels makes this practical and avoids confusion over pattern to
Band assignments. It also simplifies the mixing process because it is
accomplished within one Band.
I maintain a rough mix via the Band while composing, making
adjustments as I go. One of the last stages of the production process
 A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 381

is to finalize the mix by fine-tuning the Bands’ parameters and set-

ting the AudioPath appropriately. This is far from mixing on a
hardware console. The process consists of listening through all the
Style/patterns and setting the Band’s pchannels when needed. This
accomplishes what I call the “general” or “relative” mix. Continuous
controller curves within pattern parts provide real-time dynamic
instrument motion.

AudioPaths and Effects

DirectMusic AudioPaths offer real-time effect processing and rout-
ing. Reverb, echo, and compression add a sense of polish to the
music of Russian Squares. Real-time reverb (as opposed to reverb
mixed with instrument samples) brings all of the instruments into
the same sonic space, creating a consistent-sounding mix. In addi-
tion, reverb helps smooth transitions from pattern to pattern. This is
because the reverb tails from one pattern carry over into the next
pattern, blurring the transition point. Echo enhances the feel of the
drum and percussion tracks. A delay time equivalent to a dotted
eighth note adds syncopation and helps the rhythms feel solid. Com-
pression punches up certain instruments, such as the synthesized
bass in Spin Cycle, making them sound more dense.
Each composition calls for different effect settings and therefore
uses its own custom AudioPath. The number of mix groups in an
AudioPath equals the number of effect settings needed. Using too III
many mix groups eats up processor power and takes longer to load.
AudioPaths in Russian Squares use three or four mix groups. All of
Part

the pchannels are assigned to one of those mix groups, depending on

the needed processing. One of the mix groups often remains dry (no
effects) for instruments, such as bass parts that do not call for
processing.
AudioPath settings for Spin Cycle:
n Mix group 1: no effects
n Mix group 2: Waves reverb 1 (mix –17dB, reverb time 780ms)
n Mix group 3: Waves reverb 2 (mix –5.7dB, reverb time 2640ms)
n Mix group 4: Compression/Echo 1 (wet dry mix 28, feedback 12,
left delay 510, right delay 525)
382 n Chapter 15

Pattern Creation and Groove Level Assignments

We assigned each pattern a single groove level number. Each compo-
sition has 35 patterns assigned to groove levels 10 through 45. We
assigned a range of groove levels to each level of the game. The
groove level ranges overlap from level to level. For example, we
assigned a range of 20 to 27 to the third level of the game (which is
where a “new game” starts), while we assigned a range of 24 to 32 to
game level four. The player does not hear the highest groove level
(i.e., 45) until the last rows of game level nine are being completed.
Patterns contained all the note and continuous controller infor-
mation of the music. We assigned each pattern part a pchannel via
the Band editor. We wrote, implemented, and arranged all of the
music for Russian Squares within DirectMusic Producer’s Pattern
editor. Composing outside the realm of DirectMusic Producer was
not practical in this case because we created the sounds specifically
for the DLS Level 2 format and Microsoft software synthesizer. We
did not have a good way to duplicate the sound set in another format.
Despite the 60- to 80-millisecond delay of the software synthesizer, I
performed some of the parts in real time using a MIDI keyboard that
triggered its own piano sound as reference during recording. I com-
posed much of the music via note-by-note entry using the piano roll
interface of the Pattern editor. Copying and pasting similar pattern
parts saved some time, but it was a slow process.

Continuous Controllers
Motion within instrument parts adds depth of emotion to an other-
wise static composition. Controllers 10 (pan), 11 (expression), and 74
(filter cutoff) breathe dynamic life into the Russian Squares music.
We manually entered control curves or points into Pattern Part Con-
tinuous Controller tracks. Performing CC data in real time using a
MIDI device was impractical and messy in DirectX 8. Performance
latency rendered the data inaccurate. In addition, thinning the CC
data after a Performance required tedious selecting and deleting.
Continuous controller curves, while painstaking to insert and adjust,
were much easier to edit than a multitude of individual CC data
points (which is the result of any CC Performance).
 A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 383

Continuous controller 11 (expression) handles any real-time

movement of an instrument’s volume. This allows the basic mix to
remain solid, while instrument volume moves dynamically relative to
its Band volume, which uses CC 7. In this way, CC 7 acts as a
pchannel’s master volume and CC 11 acts as its relative volume.
CC 74 controls an instrument’s filter cutoff, allowing for low pass
filter sweeps. DLS instrument filter parameters respond if set up
appropriately; you must check the Filter Enable box and turn the
resonance slider. Spin Cycle’s bass synthesizer settings are 18dB for
filter resonance, and its initial cutoff frequency is 8,728 cents. Useful
filter settings vary depending on the characteristics of the DLS
instrument. To ensure good results, audition while testing various
settings.

Creating DirectMusic Embellishments:

Intros, Ends, and Breaks
DirectMusic intros occur when the player starts a game, at the
beginning of each level, and when resuming a paused game. Ends
occur when the player completes a level, the player quits, or the
player wins a game. We use breaks whenever the player pauses the
game. DirectMusic embellishments are created just as other pat-
terns in a Style. They are simply tagged as an embellishment in the
Pattern Properties window. The groove range indicates when an
III
embellishment pattern is called. For instance, an intro pattern with a
Part

groove range of 1 to 17 plays if the music is to start from any of the

first 17 groove levels. Likewise, an end pattern with a groove range
of 10 to 20 will only play if the game calls an end while any groove
from 10 to 20 is playing. This allows you to tailor multiple intros and
ends for different patterns of music. For example, Spin Cycle utilizes
six intros and five ends. Their groove ranges span all of the composi-
tion’s groove levels from 10 to 45.
384 n Chapter 15

Segments
Each piece of music utilizes only one primary Segment. That Seg-
ment contains five tracks: Tempo, Time Signature, Style, Groove,
and Band. The Tempo Track is fixed, as are the Time Signature,
Style, and Band Tracks. The Style is inserted at beat one measure
one of the Style Track. When you are done with this, DirectMusic
automatically inserts that Style’s Band into the Band Track.
The game controls the groove level of the Segment Groove
Track by incrementing or decrementing the groove level when “clear
row” or “add row” occurs. The primary Segment plays continually as
its groove level moves up or down, thereby changing the Style/pat-
tern currently playing. For example, as the game starts, the Segment
plays groove level “Intro 20” and then moves to groove level 20. The
groove level increases to 21 as the player clears a row and moves to
22 when the player clears the next row. Then groove level “Break
22” plays if the player pauses the game. The player ends the game,
triggering groove level “End 22.”

Motifs and Secondary Segments

Motifs in Russian Squares layer short musical gestures over each
composition’s primary Segment. Five motifs accent relevant game
events: add row, clear row, blocker, clock, and high score. You create
motif patterns just as other Styles/patterns are created, but they are
stored within a Style’s Motif folder. Each motif in Russian Squares
uses only one or two pattern parts. This simple orchestration func-
tions well over the main body of music, which is denser. Creating
appropriate DLS instruments for each motif blends these gestures
coherently into the overall arrangement. The game triggers these
motifs frequently, so it is crucial that the timbres be convincing with-
out sounding weak or obnoxious. A Performance boundary of grid,
for clear row, add row, and blocker, ensures quick response while
synchronizing up rhythmically (to the nearest sixteenth note) with
the underlying music. Clock and high score motifs synchronize to
the beat of their primary Segment (Performance boundary set to
beat).
The clear row and add row motifs act as instant feedback to the
player. Each gesture’s distinctiveness becomes a keynote sound to
 A DirectMusic Case Study for Russian Squares for Windows XP Plus Pack n 385

the player. The clear row motif rewards the player’s progress, while
the add row gesture indicates the player’s regression. Each composi-
tion enlists a different set of sounds for these motifs. Synthesized
pitch bend gestures indicate clear row (upward pitch bend) and add
row (downward pitch bend) in Spin Cycle. Harp glissandos signal
clear row and add row in Bubble Matter, while Gravity Ride uses a
shaker and claves for those motifs. The blocker motif accents each
theme aggressively with single staccato gestures. As with the other
motifs, the instrument timbres mesh with the respective music
themes.
The clock motifs are the only looping motifs. They begin as the
game clock is running out of time (it turns red) and end when a row
is cleared or the clock runs out of time. This means that the clock
motif always ends as a clear row or add row motif plays, which cre-
ates a nice bookend. Each composition uses a unique instrument for
the sound of the clock — buzzer sound, wooden tick tock, and metal
clanks, respectively. The high score motif rewards the player when a
high score game is completed. These phrases are two or three mea-
sures long and overlap a composition’s end pattern. High score
motifs are simple one-instrument arpeggiated patterns that highlight
the player’s victory.
Motifs are inserted into secondary Segments for playback by the
game. (Motifs can be called independently of secondary Segments,
but the programmer requested they be called via secondary Seg-
ments due to the calls available in DirectX Audio Scripting.) Each of III
these secondary Segments contains a Time Signature Track and a Part
Segment Trigger Track. An insertion is made in the Segment Trigger
Track, and the properties window allows for selection of the
Style/motif. The game calls the secondary Segments, and they in
turn play the motifs.

Testing, Delivery, and Integration

We tested the DirectMusic components satisfactorily within Pro-
ducer. Transitions from pattern to pattern worked well using the A/B
Transition button. We mixed by listening to all of the patterns back to
back and making adjustments to individual pattern parts or the Band
as needed. We delivered the files to the DirectMusic programmer in
the run-time format, and no container files were used.
386 n Chapter 15

All of the DirectMusic code was written using DirectX Audio

Scripting. The game engine passed relevant cues to DirectMusic,
and the script handled the DirectMusic calls. We evaluated the initial
game build, made adjustments to the script and the DirectMusic
files, and created another build of the game. At this stage we discov-
ered that a motif tied to the active block was being triggered too
frequently and was thereby removed. We also honed the clear row
function so that the last three rows of a level did not increment the
groove level. We did this because the player can eliminate the last
three rows in a matter of seconds, undesirably triggering a quick
succession of patterns. So, with three rows to go, the current groove
level continues until the level is completed. I adjusted some instru-
ment levels after hearing the music within the context of the game.
Overall, the music functioned as designed and even exceeded expec-
tations. The pattern transitions sounded musical, and the embellish-
ments were timely and seamless.

Puzzle games offer unique opportunities to composers. Game states

change more often and quicker than in action/adventure or role-
playing games, thus the adaptive scoring must be more tightly inte-
grated with its game elements. An apt analogy: Scoring a puzzle
game is like scoring a Saturday morning cartoon, in which every
action on screen has a musical gesture associated with it, whereas
scoring an action game is more akin to scoring an epic movie where
broad musical gestures paint the soundscape and follow the overall
mood of the story. The score for Russian Squares demonstrates
DirectMusic’s ability to realize the adaptive music goals of this
highly active puzzle game. For those interested in learning adaptive
music techniques, I highly recommend scoring a puzzle game. Puz-
zle games allow the composer to focus on a few key gameplay
elements while utilizing a wealth of potential adaptive solutions.
 Chapter 16

A DirectMusic Case Study

for No One Lives Forever

An Overview of the Score

N o One Lives Forever (NOLF) is a first-person action adventure

game. It is a spy story set in the 1960s, complete with gadgets,
gizmos, and guns. The player assumes the role of Cate Archer, Brit-
ish spy extraordinaire! As shooters go, it is refreshingly lighthearted
and sprinkled with kitsch and humor.
Monolith Productions developed and Fox Interactive published
No One Lives Forever for the PC in 2000. I composed and produced
the adaptive score, and Rich Ragsdale contributed the title theme.
Eric Aho, Nathan Grigg, and Tobin Buttram created the DirectMusic
(DirectX 7) arrangements and composed additional music. Bryan
Bouwman programmed and integrated the game’s DirectMusic code,
and Sonic Network, Inc. provided many of the DLS instruments
(www.sonicnetworkinc.com).
For the soundtrack, I was asked to capture the flavor of the
’50s/’60s spy genre, without infringing on any existing copyrights.
Believe it or not, at first I was told to limit my use of brass instru-
ments (this directive came to me through the grapevine via the Bond
franchise). That is like being asked to produce a blues album without
guitars! The powers that be quickly got over the legal paranoia, how-
ever. I did have one theme refused because of a subtle P5, m6, M6
melodic progression (made famous by composer John Barry), even
though I thought it was the least “Bond-ish” of my themes. Actually,
I drew more influence from German composer Peter Thomas, whose

387
388 n Chapter 16

film scores have more of the lighthearted feel that we were after.
The Barbarella soundtrack was also required listening.
I began the pre-production process by writing five or six themes
and prototyping them. These themes became the backbone of the
adaptive score. The adaptive scoring techniques for NOLF came out
of the concepts and technology implemented for three previous
Monolith games — Shogo: Mobile Armor Division, Blood II: The
Chosen, and Sanity. Shogo: Mobile Armor Division was the first
game I scored that broke the music down into separate music states,
which matched the game action. Blood II: The Chosen and Sanity
each added to those concepts, creatively and technically.
NOLF built upon my adaptive scoring foundation, improving on
many aspects of my technique. This white paper describes the adap-
tive scoring concepts, the production process, and implementation
process used to create this game score. In this case study I describe
my intentions and the actual outcome, what worked and what didn’t.
I also describe what I’d like to achieve with future action scores.

The Adaptive Concept and Music States

NOLF gameplay has high points of action and ambient points —
times when the pace is furious, and times filled with suspense. In
many scenarios, the player may direct Cate in with guns blaring or
sneak her through the situation at hand. Obviously, the same music
cue wouldn’t be appropriate for both approaches. Also, there’s no
way of predetermining how long a firefight might last and what might
come immediately after it. These are the reasons that lead me to
break the music down into flexible music states.
After writing a thematic idea, I arrange it in a variety of music
states using subjective naming conventions that reflect their func-
tionality (or intensity) in the overall score. Some of the tags I’ve
employed are “ambient,” “suspenseful,” “action,” etc. Each of these
music states can play for an indeterminate amount of time. Typically,
each music state has about one and a half to three minutes of music
composed for it. It is sometimes difficult to calculate the exact
amount of time when considering variations. As a general rule, I
think in terms of how long a particular music state can hold a lis-
tener’s interest (more on this later). A music state can repeat as
 A DirectMusic Case Study for No One Lives Forever n 389

necessary until another music state is called. I could also define the
number of repeats. The music engine supported automatic transi-
tions from one music state to another or even to silence. This
prevents the music from repeating ad nauseum for moments when
player interaction is limited.

Music Sets
Each musical theme in NOLF is arranged using six basic music
states that make up a single music set. At the start of a level, one
music set is loaded along with the rest of the level assets. The six
standard music states are:
n Silence
n Super ambient
n Ambient
n Suspense/sneak
n Action 1
n Action 2
The key to composing music for any given music state is to give the
music enough musical ebb and flow to keep things interesting while
staying within the intensity range prescribed by the music state. For
example, the “ambient” music state may rise and fall a bit in musical
density but should not feel like it has moved up or down too dramati- III
cally. The goal is to maintain the musical state while holding interest.
One way that the music sets in NOLF achieve this level of sustained
Part

interest is through instrument-level variation.

Using variation on just a few instrument tracks of a given music
state was very effective and didn’t cut too deeply into the production
schedule. Instrument-level variation is used in the lower intensity
music states quite often. These music states start differently every
time they’re called up, giving the illusion of more music. In some
cases, a four- to eight-measure repeating music state feels like three
to five minutes of fresh music.
390 n Chapter 16

Transitions: Getting from A to B and Back Again

The ability to transition between the various music states provides
the flexibility needed for the soundtrack to adapt to the game state.
Seamless musical transitions facilitate this adaptability in a way that
sounds intentional and musically satisfying. In NOLF, any of the six
music states may be called at any time. This means that any given
music state must be able to modulate to any of the other five states.
This required transitions between states that made sense musi-
cally and did not interrupt the flow of the score. Sometimes simply
starting the next music state on a logical musical boundary was all
that was needed. Often, quickly ending one state and starting the
next was enough. However, the most satisfying transitions were the
ones that built up to a more intense music state or resolved down-
ward to a less intense music state (without missing a beat, so to
speak).

The Matrix
First conceived for the Shogo score, a transition matrix filled the
need to keep track of the myriad of possible movements between
music states. By defining the matrix in a simple script (more on the
script later), I was able to assign short sections of music to act as
specific transitions between states. When the game calls for a
change of music state, it knows which music state is currently play-
ing and which one it needs to move to and plays the appropriate
transition between them.
The transition matrix acts as a look-up chart for the music/game
engine. With six music states, there are 30 possible transitions.
Needless to say, I didn’t labor over 30 individual sections of music for
each theme. Many transitions did not need transition Segments, as
they sounded good cutting in on an appropriate boundary. Also, I
found that some transition Segments could be used for multiple tran-
sitions between music states. Transitions were generally divided
into two types to help clarify my thinking: transitions that move to a
higher or more intense music state and transitions that move to a
lower or less intense music state. Categorizing transitions in this
way made reusing transition material easier (i.e., transitioning from
music state three to music state two may be similar to “3 to 1,”
while “3 to 4” may be similar to “3 to 5” — but not always!).
 A DirectMusic Case Study for No One Lives Forever n 391

Performance Boundaries
Key to making the transitions work musically were performance
boundaries. Performance boundaries defined the points along a
music state where transitions could take place. Boundary types
included Immediate, Grid, Beat, Measure, and Segment. Each of
these boundary types proved useful for different situations in NOLF.
When a music state was rhythmically ambiguous, Immediate or Grid
worked fine, allowing for the quickest transitions. Beat and Measure
boundaries came in handy when the rhythmic pulse needed to stay
constant, and Segment boundaries allowed the currently playing
melodic phrase or harmony to resolve before transitioning.
Maintaining a balance between coherent musical transitions and
the need to move quickly between states challenged us as arrangers.
As you can hear if you play the game, some transitions work better
than others. When a new state is called, there is an acceptable win-
dow of time to move to the new state. We used a window of zero to
six seconds (eight seconds tops). This meant that at a tempo of
120 BPM, a four-measure phrase (in 4/4) was the absolute maximum
that a current music state could finish prior to transitioning to the
new state. One typical solution was to use two measure boundaries
(for quicker transitions) for most of a music state and four measure
boundaries in spots that called for them aesthetically.
Composing and arranging convincing musical transitions in an
adaptive score twists your brain and forces you to think nonlinearly. III
The interactive score used in NOLF only scratches the surface in Part
this regard; there is plenty of room for future innovation. I can say
for certain that having written a number of nonlinear scores, I’ll
never think about music the same way again. In a way, it’s freed my
thinking about how music is put together. Even when listening to lin-
ear music, I sometimes think “Hmmm… this piece could just as
easily start with this section instead of that one” or “I bet they
could’ve started that transition two measures earlier!,” etc. Music is
malleable and only frozen when we record it.
392 n Chapter 16

Stinger Motifs
Two or three of the music sets used in NOLF employed motifs. The
motifs were applied as short musical accents, or stingers, that played
over the top of the currently playing music state. Performance
boundaries were set so that the motifs would be in sync with the
underlying music, and Chord Tracks were used so that motifs would
play along with a functional harmony (this was the only use of
DirectMusic’s difficult-to-navigate harmonic features).
These motifs were composed of brass riffs, quick guitar licks,
and things that would easily fit over the music state Segments. More
flexibility would have been nice so that different motifs could be
assigned to specific music states (possible using DirectX 8 Audio
scripting). Five or six motifs were written for each music set. The
engine called a motif randomly when the player hit an enemy AI
square in the head (ouch!). A silent motif was employed to prevent a
motif from playing every single time.

Sounds and DLS Banks

All of the music in NOLF uses Microsoft’s software synthesizer in
conjunction with DLS. DLS banks are loaded into the software syn-
thesizer (using RAM) and played via the DirectMusic engine. Each
music set uses up to 8MB of DLS instruments (as 22 kHz samples),
which are loaded as each game level is loaded. These DLS banks are
selected, created, and optimized for each individual music set. This
gives each music set its own timbral character that coincides with
the aesthetic needs of each theme.
DirectX 7 doesn’t have Wave Tracks, and DirectX 8 and above
do; as a result, premixed tracks weren’t an option. The DLS+MIDI
approach provided the flexibility and practicality needed for features
such as instrument-level variation and motifs that respond to har-
monic information. My current projects use a combination of
Wave/Streaming Tracks and DLS+MIDI Tracks. This provides a bal-
ance of premixed CD-quality waves with the flexibility of MIDI.
However, as processor speeds continue to increase and better
real-time DSP comes about, professional production values will be
easier to attain via MIDI+DLS. The two approaches will likely
merge and simply be two tools in the same toolbox.
 A DirectMusic Case Study for No One Lives Forever n 393

Integration and Implementation

Even though we went with an off-the-shelf solution, namely Direct-
Music, there was still a good amount of programming needed to
successfully integrate the music engine with the LithTech engine
(Monolith’s game engine). Thankfully, much of the work was done on
previous games, and we simply needed to update the code for NOLF.
Perhaps the biggest leap in this area was in how the music states
were tied to the game. In Shogo, Blood II, and Sanity, music states
were called via location triggers placed strategically throughout the
levels. This was a laborious task (done in the LithTech level editor)
and a pain when enemy/NPC placement inevitably changed as the
ship date neared. Necessity was the mother of invention for NOLF,
as we didn’t have the production schedule to individually place music
triggers.
Bryan Bouwman and the fabulous programmers at Monolith
came up with the bright idea of tying global game states and NPC AI
directly to music states. This approach made perfect sense, as the
game knows when there is action on the screen; it knows when Cate
is sneaking around in “stealth mode,” and it knows when the player
is simply exploring a level. To add flexibility, the game state to music
state assignments are done individually for each game level, so dif-
ferent levels could have different assignments. In addition, more
than one music state could be assigned to a game state. For example,
music states 5 and 6 were often both assigned to the “combat” game III
state and music states 1 and 2 to “exploration,” etc. The game Part
chooses randomly between them at run time. This means that you
could play through the same level twice and have a somewhat differ-
ent score each time, yet the music would be appropriate to the action
in both cases.

Scripting
Monolith created a simple scripting method, which provided me with
some control over the music asset management and implementation.
Being a DirectX 7 game, we didn’t have the DirectX 8 Audio
Scripting that now comes with DirectMusic. In NOLF there is a
script for each music set that is called when a game level is loaded.
The script’s basic functions are:
394 n Chapter 16

n Load the music assets for the music set

• DLS instruments
• DirectMusic Styles, Bands, motifs, and Segments
n Assign DirectMusic Segments to the six music states
n Set up the transition matrix
• Assign transition Segments
• Assign transition boundaries
n Set the basic reverb settings
n Set up the motifs and their boundaries

The Test Player

Monolith also built a handy little LithTech DirectMusic player that
contained the adaptive functionality used in the game. It loads a
script and its music set and then plays the various music states using
the correct transitions between them (a simple selector allows you
to choose music states). Motifs can even be tested over the music
states. This player was a lifesaver, as it allowed me to debug the
music content in a game-like setting before implementing the
content.

DirectMusic and the Production Process

The Prototype and Pre-production
The first step in the whole production process was to zero in on the
musical direction and thematic material of the score. This began with
discussions about the style of the music with game designer Craig
Hubbard. Next, I was to bring these ideas to realization in the studio
in the form of a prototype. Each thematic prototype was created in
my MIDI studio using all appropriate synthesizer/sampler modules
and sounds available. The idea was to ignore the nonlinear aspects
that the music would take on and ignore the technical limitations that
the game machine would place on the music. In this way the focus of
the prototype was the thematic material itself, the musical style, and
an ideal set of production values.
 A DirectMusic Case Study for No One Lives Forever n 395

Each prototype was mixed to a stereo wave file and presented to

the designer and producer. Some themes were accepted on the first
take, some were sent back to the drawing board, and others were
rejected outright. By the end of the process, we had five or six
themes that we were all happy with, and DirectMusic production
could begin.

Composing and Sequencing

The sequences created for the prototypes (using Digital Performer
on a Mac) served as a starting point for production in DirectMusic
Producer. Some of the sequences were fairly complete, while others
required extensive work and additional sections once brought into
Producer. From Digital Performer I exported each prototype
sequence in the form of a standard MIDI file (.mid). This allowed
Producer to import the sequences for editing and arranging.
Due to its latency, interface issues, and some nagging bugs,
DirectMusic Producer can be a difficult program to use, clogging the
creative flow. This is especially true in terms of creating music
sequences. It’s slow going, and there’s no way around it (for now).
That being the case, I did as much sequencing as practical in Digital
Performer in conjunction with my samplers. One key piece of advice
that I can offer when using this approach is to have the instrument
samples from your studio match, as closely as possible, the instru-
ment samples of the DLS banks to be used in Producer. This is
becoming easier to do, as the DLS-2 format used by DirectMusic can
III
be easily translated to GigaStudio (.gig) and SoundFont formats. To
Part

create the game score for DieHard: Nakatomi Plaza, I had an exactly
mirrored set from DLS as the target format to GigaStudio as the pro-
duction format.

DLS Creation
The DLS Level 1 instruments for No One Lives Forever came from
two sources; first, we licensed many instrument collections from
Sonic Network (the sounds are called Sonic Implants — www.sonic-
implants.com). The other samples were homemade, including some
solo cello samples. Sounds are a composer’s palette, and having a
rich palette, despite memory constraints of the game, was key to
making the interactive score convincing.
396 n Chapter 16

There are some tricks to creating a rich instrument collection

within tough memory requirements.
n Layering sounds and resampling: When creating music in a
traditional MIDI studio, layering and stacking instrument
patches to create a thick-sounding timbre is commonplace. The
drawback to this technique in a game is memory usage and lim-
ited polyphony. The cure is to layer your patches and resample
them into a single set of samples to be assembled into DLS
instruments. For example, I created a brass staccato instrument
by stacking about five or six brass patches in unison (including
french horns, trombones, and trumpets). One sample from this
instrument using one voice sounded like the entire orchestral
brass section playing triple forte!
n The use of unique or interesting sounds: One interesting
timbre in a piece of music can carry the piece and make it memo-
rable. The low cello glissando in the H.A.R.M. theme is one such
example in NOLF. A generic cello patch and the pitch bend wheel
would never have cut it. Instead, I brought in cellist Lori
Goldston and had her record some short figures and motifs. The
ponticello glissando figures were then sampled and pitched down
about a perfect fourth. This became the central figure around
which the rest of the piece was composed. One “live”-sounding
instrument can trick our perception into hearing other parts as
performed live.
n Each instrument must sound convincing when soloed: If an
instrument sounds weak on its own, it most likely will not add
anything to your music. I resampled many instruments with a bit
of room or hall reverb on each sample. The real-time reverb in
DirectMusic AudioPaths is very useful, but it certainly isn’t your
Lexicon-quality algorithm. Adding a bit of high-quality process-
ing to the individual samples (be it reverb, compression, or EQ)
can go a long way to get that “professional” sound back into your
interactive score.
n The samples should match the individual composition:
Even within different orchestral arrangements, different sets of
samples are called for, depending on the pacing and mood of each
piece. The “one size fits all” mentality of General MIDI will fail
to give your score anything but a generic quality. Each theme of
 A DirectMusic Case Study for No One Lives Forever n 397

the NOLF score had some instrument sets that were built spe-
cifically for that theme — the vocal “BaDeDum” sample for its
theme and the horn “blatts” for the Ambush theme, in addition
to the cello samples already mentioned for the H.A.R.M. theme.
Creating looping samples posed a big challenge given the short
length of the samples. For many samples, a Mac program called
Infinity was used to create internal loops. The program has tools,
such as crossfade looping, that help smooth out harmonic thumps
and clicks common to short loops. That said, short loops are never
perfect, and compromises are always made for the sake of memory
constraints.

The DLS Editor

After creating and editing the individual samples, they were brought
into DirectMusic Producer for DLS creation. Each NOLF theme was
arranged within its own DirectMusic Producer project. Each project
has an Instruments folder to help organize the DLS Collections. My
tendency is to create several DLS Collections, each containing just a
few instruments and waves, rather than making one massive DLS
Collection containing all the instruments. This makes it more conve-
nient to share instruments between projects.
When creating a DLS Collection from scratch, a collection was
added to the project, and the edited waves were added to the collec-
tion’s wave folder. Next, an instrument is created and regions III
assigned across the instrument. The range of a region determines Part
how far a given sample will have to stretch up or down.
For most melodic instruments in NOLF, we found that two
regions per octave was adequate. This means that each sample will
be stretched up and down about a third in either direction. In crucial
or exposed ranges of certain instruments, more regions per octave
were necessary for a natural sound. Many instruments had various
versions, depending on the needs and constraints of a given theme.
For some instruments, extreme region ranges were used to get an
unnaturally low, yet purposeful effect. I always experiment with a
sample’s useful range and often find pleasant surprises, which make
their way into my music’s arrangements. This is a way to get extra
mileage out of your chosen samples.
398 n Chapter 16

Good use of instrument articulation can extend the variety of

your samples and adds dynamic expression to your instruments.
Copying an existing instrument, pasting it into the same collection
(as to not duplicate your wave data), and then altering the ADSR
articulation is the easiest way to add to your palette. Most common
in NOLF is one instrument with a short “attack” (0 to .1 seconds)
and its partner instrument with a longer swell attack.
Smart use of the release articulation can help smooth transitions
in an interactive score in a way that straight wave files could never
approach. When a transition occurs, there are often sustaining notes
that must stop or change to accommodate the new Segment. Instru-
ment release acts as a natural fade-out or decay, and each instrument
can have its own release time. Being a DirectX 7 game, NOLF used
DLS Level 1 as opposed to DLS Level 2 that came about in DirectX
8, meaning instruments didn’t use layering or filters.

Styles, Patterns, Bands, and Segments:

How They Work Together
Conceptualizing how DirectMusic and its components will work
together in your project should be your first task. Having read about
DirectMusic’s basic functionality, you may have a good understand-
ing of the various DirectMusic components, but different games have
different needs, and this is where DirectMusic’s open-ended
approach is both powerful and confusing in its complexity. The solu-
tions to these different needs are not spoon fed; they must be
thought out conceptually first and then applied using DirectMusic’s
toolset.
For my purposes in NOLF, Styles act as project bins for patterns,
Bands, and motifs. Patterns are anywhere from one to eight mea-
sures in length, corresponding to the length of logical musical
phrases. These patterns are assigned to Segments using groove
level assignments. Each pattern is assigned a unique groove level,
which a Segment can trigger by calling that number from its own
groove track. These Segments are then in turn assigned to LithTech
music states.
These layers of abstraction, patterns to Segments to music
states, while complex on the surface, give NOLF its nonlinear and
adaptive functionality. Picture the approach as such:
 A DirectMusic Case Study for No One Lives Forever n 399

n Patterns are generally short musical phrases.

n Segments use multiple or repeating patterns (triggered via
groove levels) each commonly one and a half to three minutes in
length (exceptions to be discussed further).
n The game’s AI ties into music states 1 through 6. One or more
Segments are assigned to each music state.

Figure 16-1: The Producer setup for the Ambush theme, showing project
structure and Style/pattern associations. III
Part

Style and Band Setup

Each project file contained at least one Style. A Style can only have
one time signature, so if a theme had changing time signatures, such
as the H.A.R.M. and Ambush themes, then a Style was needed for
each time signature! If this had been a DirectX 8 game, I may not
have used Styles at all but used Pattern Tracks within Segments
(which weren’t available in DirectX 7) instead, although I do like
using styles as a pattern bin, which makes mixing and matching pat-
terns across multiple Segments easier.
400 n Chapter 16

Each Style contains a default Band (called band1). This Band is

referenced while auditioning patterns and used during most of the
creative arranging process. I call this the core Band of the project.
For simplicity, I do my best to have one Band per theme and project.
This approach works well, as I find it easier to expand pchannels
within the Band than to create another Band for the sake of a patch
change. The volume and pan info in the core Band served as the rela-
tive starting point for most patterns with any dynamic changes done
using continuous controllers 7 (volume), 11 (expression), and 10
(pan). There was a continual adjusting of the core Band as an
arrangement evolved.

Figure 16-2: The Band setup for the Ambush theme.

Pattern Creation and Organization

When exporting the standard MIDI files from Digital Performer, I did
my best to save them out in condensed pattern length files. Dupli-
cate MIDI tracks were deleted and tracks of the same instrument
were combined in Digital Performer. Rather than save out one long
SMF file, individual SMF files of one to eight measures in length
were exported, so they would import more easily as patterns. Once
imported, we did have to adjust many of the parts’ pchannel assign-
ments. This is because DirectMusic assigns pchannels based on the
ordering of the imported SMF tracks — from top to bottom.
 A DirectMusic Case Study for No One Lives Forever n 401

Other patterns were created by copying material from existing

patterns and then altering it somewhat. Some patterns were created
by entering note data manually.
With respect to groove levels, NOLF patterns use a groove
range of one, meaning the “lo” and “hi” number are the same. This
one-to-one correlation between groove numbers and patterns makes
it easy to assign specific patterns to Segments. There are cases
where NOLF employs broader groove ranges to create pattern-level
variation; if two patterns have overlapping groove ranges or the
groove range is the same, DirectMusic randomly chooses one of the
patterns to play.

Instrument-level Variation
As you know, each pattern part can have up to 32 variations. Each
time a pattern is played, one variation per part is randomly chosen.
Multiple variations do not have to be utilized; in fact, many pattern
parts in NOLF had one “hard-wired” variation to play. We created
this “hard-wired” variation by disabling all other possible variations
for a pattern. I found that four or five variations on two or three parts
was enough variety for most patterns.
The ambient and sub-ambient patterns tended to get a deeper
variation treatment, with most parts having variations. It may be that
the moody, atmospheric, and arrhythmic nature of these music states
lent themselves to a truly nonlinear treatment.
Variations in the suspense or action music states entailed two or
III
three pattern parts with variation. This allowed the foundation of
Part

that music to remain consistent while providing some variety. There

are many instances where instrument-level variation is not used.
This was mainly due to a limited production schedule. A simple tech-
nique for creating subtle variation is to copy the original track
(variation 1), paste it into another variation, and then slightly alter it.
The new variation could have the same contour with different embel-
lishments. This way, it remains consistent with the intent of the
music, yet it adds interest. Another technique for using variations
effectively is composing unique melodies for one pattern’s variations
while the other parts may have subtle variation. This technique cre-
ates a “lead” instrument that riffs over a consistent, yet changing
bed of music. I ensure that each part has its own rhythmic space to
play in. For instance, all of the vibes’ variations may occur during
402 n Chapter 16

measures three and five, while the flute variations are given mea-
sures two and six. This ensures that the parts won’t step on each
other, despite its nonlinear nature.

Composing with the Pattern Editor

You have two choices for composing with the Pattern editor: by hand
editing the MIDI data in DirectMusic Producer or by recording with
a MIDI controller. The truth of the matter is that there is no easy
way to compose music using the Pattern editor. We did the best we
could with the tools at our disposal, however.
The first method is laborious but precise — inserting notes one
at a time using the mouse or the Enter and arrow keys. After a
phrase is inserted, each note’s duration has to be dragged or edited
to the desired length. Multiple notes can be selected, copied, and
pasted when a set of notes repeats. The most awkward aspect of this
approach is changing the velocities of individual notes by grabbing
and dragging the top edge of the selected notes. More precise
changes can be made in the note’s properties window. This method
is completely unglamorous and slow, but it gets the job done. If you
want to utilize DirectMusic’s variation abilities, you’ll certainly find
yourself doing this sort of editing.
Method two for composing content in the Pattern editor involves
interfacing a MIDI controller to DirectMusic Producer. This is done
using the MIDI/Performance Options tab. Once set up, the keyboard
is used to perform part variations in real time. Due to the latency of
the software synthesizer (80ms on average), I suggest monitoring
each performance with a piano sound from the keyboard controller.
Unfortunately, the MIDI timing and sync during real-time perfor-
mance is not ideal (in DirectX 7 and DirectX 8); therefore most
performances need hand editing in the pattern window. At least this
method allows for a certain amount of performance spontaneity.

Continuous Controllers
The use of continuous controllers is essential to breathing life into
any MIDI score, and a DirectMusic score is no exception. In
DirectMusic, each pattern part can have CC tracks inserted beneath
its piano roll. In NOLF, controller 7 (volume) is used to change a
 A DirectMusic Case Study for No One Lives Forever n 403

part’s general volume, while controller 11 (expression) creates the

dynamic ebb and flow.
Parts without dynamic expression can sound flat and monoto-
nous, no matter how well the part is written. Inserting expression
curves (CC 11) helps convey the dramatic intent of a musical phrase.
Expression curves are used in the NOLF score as crescendos, as
quick swells, as fade-ins/outs, and to emphasize certain phrases of a
part.
The interface in DirectMusic Producer for adding and editing CC
curves is easy to use conceptually, but actually moving and manipu-
lating curves may quickly tire your mouse hand. The CC curve
concept in DirectMusic Producer is much easier to use than placing
individual CC data points. I did experiment with “performing” curves
using an external controller (the EMU launch pad), but DirectMusic
Producer’s timing and latency problems prevented the data from
being recorded accurately.
This being said, there are two methods for applying CC data to
parts: the slow but accurate method of inserting curves by hand (and
mouse) or creating the CC data using another sequencer prior to
saving out the .mid files. The key in this case is to thin out the CC
data. This limits the use of processing power by the DirectMusic
engine and makes the data easier to edit once it is in DirectMusic
Producer.

III
þ Note Curves that do not reset during a transition can cause CC info
to “stick” at an unwanted level. This can occur when the music moves Part
from one Segment to the next during a CC curve. To prevent this, we
place CC data points at the beginning of every pattern part for those
pchannels that use CC data. This technique initializes the CC data for
each pattern, avoiding unwanted CC levels.

Segment Creation
Segments bring together all of the disparate patterns in a project into
unified arrangements. The patterns act as building blocks for the
Segments. Sometimes the order and arrangements of the Segments
were planned prior to their creation. At other times, we would
experiment with different orderings of patterns within Segments.
This flexibility resulted in some interesting outcomes, and we often
got more mileage out of our patterns. Creating short phrase-length
404 n Chapter 16

patterns gave us this flexibility to work with patterns as if they were

Lego blocks.
All NOLF Segments contain the following tracks: Style Track,
Tempo Track, Groove Track, Band Track, and Time Signature Track.
A few Segments also used Mute Tracks. Segments trigger patterns
via the Style and Groove Tracks. The Style is inserted into beat one
of the Style Track. This automatically creates a Band Track with that
Style’s Band inserted. From there, groove numbers are placed along
the Groove Track to create the Segment’s arrangement. Mute
Tracks are used to reveal a pattern gradually within a Segment. For
example, a Segment begins with groove level 10 with the Mute
Track muting most of the groove/pattern’s pchannels. Over the
course of 16 measures, the pchannels are individually unmuted,
slowly revealing the entire pattern.

Figure 16-3: A typical Segment from NOLF.

Setting up Performance boundaries longer than one measure was

tricky for NOLF because it was a DirectX 7 game. In DirectX 8 and
beyond, a Marker Track can be used to set up transition points
within a Segment, but DirectX 7 did not have that feature. In fact,
our work with NOLF may have influenced the addition of the Marker
Track by the DirectX team. There were two possible workarounds
for longer performance boundaries in NOLF. A music state can play
multiple Segments sequentially using the LithTech music script. The
Segment performance boundaries were set to “end of Segment.”
This allowed performance boundaries of varying length within a
music state. Another clever solution to the lack of a Marker Track
utilized the Time Signature Track. We would set the performance
boundary of a music state’s Segment to “measure” and place various
time signatures along the Segment where transitions were to occur.
In other words, the Time Signature Track had no direct correlation to
the underlying meter of the music. For example, if we wanted per-
formance boundaries every four measures of music, the time
signature would be 16/4. Workarounds have often inspired the
 A DirectMusic Case Study for No One Lives Forever n 405

creation of features, and features (such as the Marker Track) are not
created until there is a known need for them.

Motif Creation
Motifs are created within Styles. There is a folder within each Style
for motifs. Motifs are created and edited in the same manner as pat-
terns. The only difference is in functionality; motifs layer over
Segments at a Segment’s tempo, while patterns are triggered via
groove levels from within Segments. Motifs follow the harmonic
structure of a Segment (if set up to do so).
To keep things simple, NOLF motifs are short, simple phrases
that punctuate and accent the underlying music. As mentioned ear-
lier, NOLF motifs play when a direct hit is made on an enemy AI.
Often, one or two staccato brass chords accent the music, or a sus-
penseful string tremolo rings over the primary Segment. In all cases,
NOLF motifs are one measure or less in duration.
The instrument of a motif works best when it is different from
the instruments of the primary Segment. This prevents the motifs
from stepping on a part that is already playing. Using a different or
specific range also helps in this regard. For example, if the primary
Segment contains low- and mid-range brass instruments, using a
trumpet motif in the upper registers will fit well in the mix rather
than make it muddy. Even if the primary Segment contains a mid-
range trumpet, a solo trumpet motif will be heard because of its dis-
tinct timbre.
III
The performance boundary for motifs was always Grid or Beat,
Part

so they would play in sync with Segments, yet respond quickly to the
game. This means that a motif plays no more than a half second after
a direct hit and its gunshot sound. The timing coordinates perfectly
with the sound of the gun, responding to a perfect shot. A perfor-
mance boundary of Immediate might cause the motif to step on the
gun sound and would not play in sync with the primary Segment.
Performance boundaries of Measure or Segment would cause the
motif to respond too late, losing the desired timing. NOLF motifs uti-
lized DirectMusic’s Chord Tracks so they perform in harmony with
the underlying music. Motifs are created in a neutral key (C), and the
primary Segments’ Chord Track transposes the motifs according to
its chord symbols. The chord symbols inserted in the Chord Track
reflect the key or harmony of the primary Segment’s music. In most
406 n Chapter 16

cases, NOLF Chord Tracks contain one chord at the beginning of a

Segment setting its key center, and its patterns are transposed to
that key. The motifs, in turn, also perform in the key set by the
Chord Track. This use of Chord Tracks differs from the more com-
mon technique of placing chords at every harmonic or chordal
change within a Segment. The difference is that NOLF uses Chord
Tracks to set the overall harmony and key center rather than defin-
ing each chord change. This simplified approach works because the
motifs function well over any chord within its key.

Creating Transition Segments

Transitions link the six music states to one another musically. It
takes a puzzle-like logic to figure out how the transitions should
operate. NOLF uses specifically created Segment files for transitions
instead of DirectMusic embellishments. This is because DirectMusic
embellishment logic is only aware of one Segment/pattern at a time.
Its logic cannot take into account both the current music state and
the music state to which the music is transitioning. The transition
matrix architecture, set up in the script, allows specific Segments to
be assigned to each possible transition (between six music states,
there are 30 transitions). These transition Segments are one to four
measures in length. This duration allows enough time for convincing
transitions while being short enough to keep up with the game
action.
I ask myself two basic questions when composing a transition
Segment: Which music state am I moving from? And to which music
state am I transitioning? I write down all the possible transitions and
methodically check them off as they are created. I begin composing
transition Segments to and from silence — in other words, the intros
and ends of each music state (again, these are not DirectMusic
embellishments). The music composed for these intros and ends
provides the foundation for other transitions. This is because the
material written for a music state’s intro may become the basis for
transitioning to that music state from another music state. Also, a
music state’s end may also function well when transitioning from
that music state to other music states. (This is the puzzle logic that I
mentioned!) Sometimes I’ll use the end transition of one music state
and the intro of another music state as the transition between them.
The end material brings the music out of the current music state,
 A DirectMusic Case Study for No One Lives Forever n 407

and the intro brings the music into the next state. More often, this
end/intro transition is the basis for that transition, and further editing
and composing is done to make it work well.
Many transition Segments function well in multiple transitions,
and this saves production time. For example, the transition Segment
built for music states 2 to 3 may also work between music states 2
and 4, etc. I compose one music state’s transitions at a time. Thus,
when I’m working on music state 2, I sequentially create transitions
between states 2 to 1, 2 to 3, 2 to 4, 2 to 5, and 2 to 6. Again, this is
because there are bound to be similarities among these transitions
that I can reuse.
Simple transitions are often the most effective. If the music
needs to stop or transition quickly, a large percussive accent brings
the music to a halt. It’s as if the music hits a wall, blunt and jarring,
and that frequently works well within the game. There are also cases
when no transition Segment is needed between music states. In
these cases, the music flows directly from one music state to the
next, and the release times of the DLS instruments create a natural
blending or crossfade between music states.
Sometimes, one transition Segment is not adequate for a particu-
lar transition. If a music state contains a variety of musical sections,
more than one transition Segment may be needed. For instance,
music state 4 has an A section that is 16 measures in length and a B
section also 16 measures long. If the instruments used in each sec-
tion are different or the harmony and tempo vary between them, III
then a single end Segment may not work from both section A and Part
section B. In a case such as this, two transition Segments are cre-
ated, one for transitioning from the A section and a second for
transitioning from the B section. During gameplay, the transition
matrix calls the appropriate transition Segment, depending on the
current playback point of the music state (more on this in the section
titled “The Transition Matrix”).
When creating music states and their transitions, keep their har-
monic content in mind. Music states with disparate key centers
cause difficult-to-execute transitions because the drastic harmonic
modulation of the transition may sound unnatural or awkward. I rec-
ommend using key centers that are closely related (i.e., C major to G
major) to create convincing transitions. I also use chromatic modula-
tion in NOLF (i.e., B major to C major), and many transitions simply
408 n Chapter 16

stay in the same key. It is easy to back yourself into a corner har-
monically when creating an adaptive score. Be aware of the tonal
centers of the music states as you create them, and try to think
ahead about how they will transition harmonically.

The LithTech Player/Tester

Preparing DirectMusic Files for the Game
Run-time files are saved into the project’s Runtime folder after the
DirectMusic files are created and tested in Producer. Next, I
downsample the run-time DLS instrument waves to 22 kHz using
Awave Studio (www.fmjsoft.com). Awave is a fantastic utility for
sample format conversion and batch processing. There is no other
tool that I’m aware of that batch converts and downsamples DLS
files. From here, the files are ready for testing within the LithTech
player.

Figure 16-4: The LithTech DirectMusic test player.

 A DirectMusic Case Study for No One Lives Forever n 409

Using the Script

Monolith programmers and I developed a simple scripting system to
put more control into the hands of the composer. DirectX 7 did not
have the DirectX Audio Scripting now available in DirectX 8 and
after. We wanted an easy way for composers to set the basic
DirectMusic parameters for a given music set. Scripts are created
within a text document. A script template simplifies the task, and the
necessary fields are filled in for a given theme.
The script fields begin with the basic setup parameters.
NUMINTENSITIES indicates the number of music states to be
used. INITIALINTENSITY is the music state that begins playing
when the music set is loaded. PCHANNELS sets the number of
pchannels used, and VOICES programs the maximum number of
synthesizer voices. SYNTHSAMPLERATE sets the sample rate for
the DLS synthesizer. REVERB on/off, REVERBGAIN, REVERB-
MIX, REVERBTIME, and REVERBHIGHFREQRATIO allow the
composer to set the reverb parameters.
The next section of the script describes the specific DirectMusic
files to load into RAM. DLS banks, Styles, and Bands are listed here,
while Segments listed in the music state and transition sections of
the script are also loaded into RAM. The script loads all necessary
files for a music set at the beginning of a game level. The appropriate
script is set within the LithTech level editor for each game level and
is called as a level is loaded. Dynamic loading of DirectMusic files, III
while possible, is more complex to implement and slows the game Part
down during the unload/load cycles, which is why we chose to load
everything with the game levels.

Setting Up Music States in the Script

Setting up the music states entails listing the Segment or Segments
used for each music state. The format and its variables are INTEN-
SITY “music state number,” “times to loop” (–1 for infinite), “music
state to switch to when finished looping,” and then “Segment(s) to
be played.” A typical script lists them as such (note that LithTech
refers to music states as intensities):
410 n Chapter 16

INTENSITY 1 –1 (loop) 0 (go to) silence.sgt

INTENSITY 2 8 1 subambient.sgt
INTENSITY 3 –1 0 ambienttension.sgt
INTENSITY 4 –1 0 sneak1.sgt
INTENSITY 5 0 6 action1a.sgt
INTENSITY 6 0 5 action1b.sgt
INTENSITY 7 –1 0 action2.sgt
In the example above, Intensity 2 repeats eight times and then tran-
sitions to silence. This prevents that music state from repeating too
much. Also notice that Intensities 5 and 6 are actually one music
state that plays Intensity 5 and then 6 and repeats. It is split into two
parts so that different transition Segments can be assigned to each
part of that music state, as described earlier in the section titled
“Creating Transition Segments.”

The Transition Matrix

Each desired transition uses the following variables: “from music
state,” “to music state,” performance boundary, and transition Seg-
ment. A typical set of transitions is listed in the script as such:
TRANSITION 2 (from) 1 (to) BEAT Sub2Silence.sgt
TRANSITION 2 3 MEASURE Sub2Amb.sgt
TRANSITION 2 4 MEASURE Sub2Sneak.sgt
TRANSITION 2 5 MEASURE Sub2Action1.sgt
TRANSITION 2 6 SEGMENT Sub2Action2.sgt
Performance boundary can be set to Immediate, Grid, Beat, Mea-
sure, Segment, or Default, which uses the Segment’s internal
performance boundary.

Motifs and Secondary Segments

The final part of the script lists motifs and secondary Segments. The
motif format is style name, motif name, and performance boundary,
while secondary Segments simply list Segment name and perfor-
mance boundary.
 A DirectMusic Case Study for No One Lives Forever n 411

MOTIF style1.sty DirectHit1 (Motif) MEASURE

MOTIF style1.sty DirectHit2 BEAT
SECONDARY- fly.sgt MEASURE
SEGMENT

Using the LithTech Test Player

The LithTech DirectMusic test player allows composers to audition
their content with the functionality that it will have in the game. The
Base Directory field lists the path to the content to be tested. The
Rez File refers to the LithTech resource file if the game and music
content have been compiled. I avoid this and test the content
directly, as run-time files, before checking it into the game. Control
File Name names the script created for the content. Once those
fields are filled, the Init Level button loads the DirectMusic content.
The Intensity scroller plays the scripted music states with the appro-
priate transitions between them. There are windows for playing and
stopping secondary Segments and motifs over the music states. The
debug window shows any errors that may arise due to missing con-
tent or a typo in the script, etc.
This player is indispensable. I test all the transitions with the
LithTech player, make adjustments in Producer as needed, and test
them again. This process allows quick iteration between content cre-
ation and game environment testing. The player, the script, and the
Producer can be open at the same time, making the back and forth
process of creating and testing easy. The LithTech player is also a
III
fantastic demo tool for illustrating the content and its functionality to
Part

the game team before actual implementation. An adaptive music test

environment saves mountains of production time because the com-
poser can debug his/her content thoroughly, giving music creators
confidence that their content will work well in the game before the
big handoff.
412 n Chapter 16

Integration and Implementation

Integration of DirectMusic Technology
DirectMusic functionality had already been a part of the LithTech
engine, so integration of the technology was already complete. We
innovated in the area of AI, however. NOLF already integrated an
advanced state machine, which calculates the player’s state and
enemy AI. The programmers simply made it possible to trigger
music states via game states. These game state to music state asso-
ciations were made in the LithTech level editor so that different
game levels could have unique setups if desired. The level editor
also allowed two or more music states to be assigned to one game
state, one of which would be randomly chosen during run time. The
most intense music states, 5 and 6, were both assigned to the com-
bat game state. Also, music states 1 (silence) and 2 (sub-ambient)
often shared the quiet “investigate” game state. Assigning multiple
music states to a single game state cut down the repetition and pre-
dictability of music within a given level by adding variety to game
scenarios.

Implementation of the Music Content

The adaptive music state machine described above makes imple-
menting the DirectMusic content easy. The first steps include
checking the DirectMusic files into the game and properly setting up
each game level. The LithTech level editor selects the music theme
and script for a given game level. The music themes are thus
assigned to the various levels, each theme being used across an
average of three or four levels. Ninety percent of the music’s adapt-
ability is handled by the state machine. Location-based triggers
account for the other ten percent and override the state machine
when triggered. Location-based triggers come into play when a spe-
cific theme or music state is desired, regardless of the game state.
NOLF cinematics apply the same themes and music sets as the
adaptive game score. Music triggers are placed at key points in a cin-
ematic, where music is needed, and transitions between music states
automatically occur. Triggering music states from cinematics works
surprisingly well but certainly does not sound as good as custom
 A DirectMusic Case Study for No One Lives Forever n 413

cinematic scores. Music scored specifically to a scene matches the

events more precisely than music composed out of context. The
adaptive music sets and triggers are used because the NOLF
cinematics were not complete in time to score them individually.
The lesson is to reserve production time to custom score game
cinematics and demand that the developer finalize the timing of them
before the scoring begins.

Conclusion
The score for No One Lives Forever was a challenge to produce but
was also very rewarding. The first challenge was convincing the pro-
ducers at Fox Interactive that an adaptive score could have high
standards of production quality. The Monolith team helped make the
case by presenting demos of previous scores, such as Sanity. Also,
my prototype themes helped convince them of my abilities as a com-
poser. Putting together convincing DLS banks within tight memory
constraints also posed a big challenge. The optimization process was
time consuming and tedious but key to the sound of the game. Final
mixing and editing called for great attention to detail by going though
all the patterns (all their variations), motifs, and Segments, making
sure volume levels, panning, and instrumentation meshed well
together. NOLF’s game state/music state integration gave me the
greatest reward. It was fantastic to simply drop music into the game III
and hear the interactivity immediately. It was also gratifying to col- Part
laborate with a strong team of arrangers/composers. Having the help
of three other musicians produced more content for the game, and
sharing compositional ideas and techniques made us all better musi-
cians. Finally, spy music was just plain fun to compose. The game’s
sense of humor made it a delight to create its music.
Overall, the adaptive design functioned as planned or better. The
transitions reacted quickly and smoothly to the game calls, and the
mood of each music state matched the on-screen action very well.
The instrument variation, music state variation, and use of silence
alleviated the repetitiveness common to many games, and the motifs
made direct hits more satisfying to the player. At its best, the adap-
tive score draws the player deeper into the game experience. My
biggest criticism is that sometimes the game states change faster
414 n Chapter 16

than the music was intended to react. This makes the music seesaw
between music states unnaturally. Many of these instances are only
noticeable to me, but some are more obvious. I will be thinking of
solutions to this type of dilemma for my next adaptive score. Also,
the sonic quality of the music is limited due to the 22 kHz DLS
banks. A combination of Wave Tracks and DLS banks would have
allowed for longer samples, phrases, and premixed sections, which
can increase the overall fidelity of a score while maintaining
adaptability.
Each adaptive game score that I produce gives me ideas and con-
cepts for the next. The biggest lesson learned from NOLF is that
global music integration is hugely important to a successful score.
Good integration creates the logical lines of communication between
the music system and the game engine. If these lines are weak or
nonexistent, the music will not respond well to gameplay, no matter
how well the music functions out of context. Context is everything.

Addendum: A Guide to the NOLF Media Files

NOLF Quicktime Videos
n EarthOrbit: The Ambush theme starts in music state 5 (combat
1), transitions to music state 2 (ambient), and then transitions to
music state 6 (combat 2) with motifs.
n Hamburg Club: The BaDeDum theme starts with music state 3
(main theme) and transitions to music state 6 (combat 2); the
main theme returns and then transitions to music state 5 (com-
bat 1) and ends with the dialogue.
n MorroccoAmbushA and B: Each of these movies runs the
same scenario but with differing scores. AmbushA transitions to
the combat 1 music state, and AmbushB moves to the combat 2
music state.
n SniperB2: This scene exhibits the motifs of the Ambush theme
well, as the music transitions from music state 2 (sub-ambient)
to music state 3 (suspense).
 A DirectMusic Case Study for No One Lives Forever n 415

Ambush Music States Audio File

This music clip moves through all six of the music states for the
Ambush theme.
Time Music State
0:00 6 (combat 2)
1:22 transition
1:30 2 (sub-ambient)
2:14 transition
2:16 4 (suspense)
3:10 transition
3:12 3 (ambient)
4:08 transition
4:12 5 (combat 1)
5:33 transition
4:12 1 (silence)

III
Part
This page intentionally left blank.
 Chapter 17

A DirectSound
Case Study for Halo

Introduction
Audio the Bungie Way

T he evolution of the audio in Halo began with the first two install-
ments of the Myth series of games developed by Bungie in 1997
and 1998. A lot of time was spent laying the groundwork for the
audio engine that we developed further during the production of Oni
and utilized in Halo. Functionality was developed to allow for multi-
ple permutations, nonrepetitive loops, detail sounds, dynamic dialog,
and impacts reactive to velocity and distance, as well as cascading
sound effects. While Halo was rewritten from the ground up for the
Xbox (after Bungie was acquired by Microsoft), it is important to
note that many of the high-level sound design concepts and their
implementation into the Halo audio engine are the result of years of
development and refinement with Jason Jones and the Bungie pro-
gramming team, in particular Halo’s audio programmer Matt Segur.
We share a philosophy here at Bungie that we like to call “Audio
the Bungie Way,” which details fundamental audio design for any of
our games. At the core of this philosophy is the understanding that
repetitive audio is not only annoying but also unrealistic. This is why
we allow for so many permutations in every sound call, why music
does not play constantly throughout a gaming session, and why we
insist on the many customizations to our audio engines.

417
418 n Chapter 17

This core idea means that we spend a huge amount of our efforts
in the implementation stage. There are many games released every
year that have audio assets of the highest quality, but due to lacklus-
ter implementation, they fall short of having the maximum impact.
We believe firmly that half of what makes a game sound good is in
the implementation. This process begins with the technical design
and continues through pre-production, production, and post-pro-
duction. The process requires the full effort and support of the
production and programming teams.

History of Halo Audio

In July of 1999, our cinematic director, Joseph Staten, approached
Totalaudio (at that time Marty O’Donnell and Mike Salvatori) about
writing a soundtrack for a live game demo of Halo at the upcoming
MacWorld Expo to be shown by Bungie’s Jason Jones during Steve
Jobs’ keynote address. Bungie had a scripted demo running in real
time through OpenGL on the Macintosh but with no audio code. The
music composed for the demo was played in sync by hitting Play on a
CD player when the demo began.
Marty describes his approach to the music of Halo and establish-
ing a mood and feel for this ancient, mysterious ring artifact found by
humans 500 years hence in some unexplored corner of the galaxy:
I felt that I could evoke an ancient and mysterious feeling by start-
ing with some Gregorian monk-style chanting over a bed of strange
ambient sounds and then give the action sequences that followed an
epic and important feel by getting orchestral strings from the Chi-
cago Symphony to play over a somewhat rock ’n roll rhythm section.
I added an improvised Qawwali chant voice over the top to help
reinforce the “alien” nature of the environment. Whether these deci-
sions were the right ones or not doesn’t matter. I had two days to
write and produce this piece and there simply was no time to ponder
or experiment, which is sometimes a good thing.
Since this was also a venue that would feature a big screen, a
large auditorium, and a gigantic stereo sound system, I wanted to
not only capture the mood but also hook the audience. Anything that
sounded like “game music” was going to be a disappointment. Plus,
the track needed to be interesting enough in its own right so that the
 A DirectSound Case Study for Halo n 419

audience wouldn’t notice that they weren’t hearing any sound

effects. It seemed to work out pretty well.
For E3 2000, Bungie had ambitious plans. The team would show the
game being played and then follow it up with a ten-minute trailer
that hinted at the storyline of Halo and showed the technological
prowess of the Halo team. The hands-on demo had merely a rudi-
mentary sound engine, but the trailer was produced in full 5.1
surround sound and played from a DVD in an enclosed theater at E3.
Halo was one of the hits of the show, but at that point it was not
known that discussions had been ongoing for Microsoft to buy Bun-
gie and bring Halo to the Xbox. Given that the audio capabilities of
the Xbox would surpass any previous console/computer platform,
this was an exciting turning point in the audio production of Halo,
not only because the audio could be carefully directed toward one set
of audio hardware, but also because it greatly expanded the scope of
what could be accomplished.

Coming to the Xbox

All of the work that had been done on Halo leading up to E3 was cast
aside when Bungie arrived in Redmond, Washington, in July of 2000.
The team decided to rewrite all parts of the game to maximize the
power of the Xbox. This certainly included the audio engine. The
audio capabilities were such that we would have been foolish to not
take advantage of them. We focused the audio engine on the features
III
that we most wanted to take advantage of, which included 256 Part

voices, real-time 5.1 Dolby Digital encoding, 3D positioning, DSP

effects such as reverb, use of the hard drive for audio streaming, and
the Xbox ADPCM compression through which the audio chip can
play back without any CPU hit.
Before we break down the various areas of Halo’s audio as they
relate to DirectSound and the Xbox, here’s a word on our terminol-
ogy. The most basic building block of our audio engine is a soundfile,
which in our case is an AIFF file, stereo or mono, and 22.05 kHz or
44.1 kHz depending on usage. All soundfiles are grouped as permu-
tations into a soundtag (after ADPCM compression). A soundtag is a
file that contains not only the ADPCM audio data but also all relevant
information about the playback of the enclosed soundfile permuta-
tions such as variable pitch/volume, skip fractions (percentage
420 n Chapter 17

chance each permutation will play), and soundclass (e.g., “weapon

fire” or “unit footstep”). The audio engine does not recognize
soundfiles, only soundtags. Soundtags are attached to other types of
tags in the Halo engine, such as animation, effect, or particle tags.
The most complex building block in the Halo audio engine is the
soundlooping tag. Every element in a soundlooping tag is a soundtag,
which in turn contains a soundfile. These tags contain instructions
for how to loop the soundfiles including assignments, beginnings and
endings (which we call the “in” and “out” respectively), multiple
tracks, and detail sounds. These three types of files are all we need
to permeate a Bungie game with audio.
Although there are many more complex examples, if we want to
assign a gunshot to a weapon, we merely open the weapon tag, go to
the slot for the firing action (where we also see the smoke and flame
particle effects attached), and attach our soundtag (which has multi-
ple soundfile permutations of a gun firing with randomized pitch/
gain), and we are done. Another example would be creating sounds
for a character, such as the Hunter running. We make sound effects
based on an animation (AVI or MOV file) and then attach a soundtag
directly in the animation tag for the Hunter (which contains the doz-
ens of animations the Hunter might use) in the slot for the
appropriate animation.
Lastly, here’s a word on soundclasses; we can assign each
soundtag to a soundclass that groups similar types of soundfiles,
such as a weapon firing, music, unit dialog, or any of a dozen others
at our disposal. In essence, our soundclasses function in the same
way that sub-masters function on a mixing console. This is an impor-
tant divisor because it allows us some control over each area of the
audio in some key areas.
The soundclass attributes (delineated in the audio code) set such
things as the maximum number of soundfiles from that soundclass,
which can play simultaneously, whether the soundclass is of an
importance where it cannot be late (such as reloading a weapon
which needs to be synced to animation playback), or if it could be
delayed by a few CPU cycles (such as unit dialog), and also what
rolloff values should be used for that soundclass. This is important
for us because every soundclass needs to have a different rolloff. For
instance, footsteps needed to rolloff rather quickly. Distance rolloff
was considered as well. Marine dialog should be heard from a short
 A DirectSound Case Study for Halo n 421

distance away, while explosions such as grenades should be heard

from a great distance. The delay factor was important to our audio
programmer, as he was the one juggling our 12,300 soundfiles to
play back as necessary within 3MB of RAM. With constant loading
and purging of audio RAM, it’s crucial for the audio engine to know
whether it needs to play a sound immediately or if a 30- to 100-milli-
second delay is okay.

Production
Music
We utilized DirectSound for all of the music in Halo to provide a
dynamic musical score that is both constantly changing and reactive
to gameplay elements. Each piece of music during gameplay was
started and stopped in the scripting at various moments within each
level. When the gameplay design for a level was nearly completed,
we’d sit down with the designer and “spot” how music would play
during that level. When a piece was started, it began with the “in”
soundtag and continued into the main looping soundtag. This main
looping soundtag contained multiple permutations, which play back
randomly to create a constantly changing musical experience.
If you sit in one place in Halo listening to a piece of music, you
will notice that it never plays back exactly the same way twice due to III
the randomization of the permutations. The music was edited so that
the “in” soundfile plays seamlessly into any of the main loop permu-
Part

tations, and the main loop permutations can play seamlessly into the
“out” soundfile when the piece is stopped by the script. These main
loop permutations contain unique elements, so as they are randomly
played back, the piece takes on a new flavor each time you hear it.
The script can call upon an alternate track, which is used in reac-
tion to something going on during gameplay. This alternate track
(which can have its own unique “out”) might have a more or less
intense score, depending on the needs of that section of gameplay.
For instance, if the player is about to enter an encounter with a num-
ber of enemies, the script starts a piece of music beginning with the
“in” and proceeds to the “main” looping section. If during the course
of battle, the designer throws another onslaught of enemies at the
player, the script could trigger an alternate track that, in this case,
422 n Chapter 17

might be a more intense mix of the same piece of music. Then when
all enemies have been vanquished, the script could either stop the
piece of music (playing the “out” soundtag) or return to the “main”
looping section, indicating to the player that the battle was over.
This system means that the player gets a constantly changing musi-
cal score that reacts to gameplay elements as they appear.
Another example of this would be scripting an ambient piece of
music and then scripting a more rhythmic “alt” track that crossfades
when the player runs into enemies. For instance, we might begin a
piece of music when the player enters a new area. This music is
ambient in nature, without a clearly identifiable rhythm. When the
level designer decides to ambush the player with enemies, the alter-
nate track is triggered, which could either begin playing at the end of
the current loop or crossfade at the moment it is triggered. In this
case, the alternate track could be a more rhythmic version of the
ambient piece or depart from the ambient piece entirely. As when
the enemies have been vanquished, we can instruct the level
designer to script the tag to play the “alt out” or return to the
“main” loop to continue playing the ambient music.
Other elements that are useful, especially during ambient pieces
of music, are details and multiple tracks. Details are soundtags full of
soundfiles that could be triggered anytime during the playback of a
piece of music. We can set which group of files to use and a time
range in which to randomly trigger them. Additionally, we could layer
a second track on top of the first one, and having both “main” loops
playing back with multiple permutations of varying lengths would
provide an additional layer of variability. These techniques allow a lot
of flexibility and give the player variation and dynamic playback
while utilizing a traditional form of playback (streaming prerecorded
audio files). All music soundfiles were ADPCM compressed 44.1
kHz stereo AIFF files and loaded in 128K chunks from the hard drive
(sort of a faux streaming).

SFX
The background stereo ambiences in Halo were created in a similar
manner to the music. All ambient soundlooping tags were assembled
identically to the music files described above. We had multiple tracks
playing in many cases (such as an outdoor ambience plus a wind
track), each with multiple permutations in the “main” loop tag, as
 A DirectSound Case Study for Halo n 423

well as detail sounds that could be randomly triggered and were

placed randomly in 3D space.
One technique to highlight (which was used extensively in the
music tags as well) is permutation weighting. Permutation weighting
is the ability to assign different probabilities to each permutation to
control how often a given permutation is played. For example, there
are six permutations of a “main” loop in an outdoor ambience with
lengths varying from two to eight seconds and 27 seconds of mate-
rial. In one of those loops, there is a distinctive owl hoot, which
would be repeated every 27 seconds if the material was played back
in the same order each time.
Given the randomness of our permutation playback, you actually
might hear it on average every 27 seconds but sometimes a little
less and sometimes a little more frequently. If that distinctive hoot is
still heard too frequently, we can use permutation weighting to
assign a high skip fraction to that one permutation that tells the
audio engine to skip this permutation x-percentage of the time. We
adjust this number until we hear the distinctive hoot at a frequency
that seems natural.
We use this same technique extensively in the music (for unique
musical flourishes) and in SFX like the example above (or for an
annoying rattle in a piece of machinery we only want to hear every
so often), as well as in dialog permutations that should be heard
rarely (“I’d have been your daddy but that dog beat me over the
fence!” for example). As with the music, all stereo ambiences were III
44.1 kHz ADPCM compressed soundfiles. All other sound effects in Part
the game were 22.05 kHz mono ADPCM soundfiles and played back
through the best parts of DirectSound’s programming, such as 3D
positioning, DSP, including reverb, occlusion, Doppler effect, HRTF,
and of course real-time 5.1 Dolby Digital surround sound.
We could attach audio to anything in the game, and because of
that, the 3D positioning function was critically important to enable
the player to sort out where audio elements originate. Knowing that
a Marine “had your back” just over your right shoulder brought a
sense of security, just as hearing a Hunter’s metal armor behind you
would bring a sense of impending doom. There were many audio ele-
ments in the game that received 3D positioning, such as weapon
sound effects, bullet impacts, speech, mechanical objects, particles
such as flying dirt or sparks, and outdoor detail ambiences, such as
424 n Chapter 17

wildlife, rivers, and waterfalls, to name a few. In essence, everything

you hear in Halo that is not music or an ambient bed is 3D posi-
tioned. With the Xbox audio card, you can play back a maximum of 64
3D positioned sound effects simultaneously, leaving 192 2D voices
free.
We also utilize the DSP power that is built into the Xbox audio
chip. The onboard Motorola chip is identical to the chips that are
used in some Pro Tools DSP farm cards that give the Xbox DSP pro-
cessing power unlike any other platform. We can use as many
reverbs per level as we wish and can switch between them on the fly
without generating a lot of CPU overhead. It doesn’t matter if we use
a small room reverb or a huge airplane hangar reverb; the cost to the
engine is the same.
We use, on average, about six reverbs per level, embedding
information about reverb parameters at a given location directly into
the level geometry (right alongside the ambience soundtag that
should be played). There is not a single area in Halo where the 3D
audio is not being played with reverb, and it certainly adds a level of
realism to the environment. Standing inside the shaft on The Silent
Cartographer level and listening to the echo of reloading your pistol
helps underline the sheer size of the geometry that the artists cre-
ated for Halo.
Although we do not utilize other DSP functions extensively for
Halo, we do use a filter for those times in the game when the Master
Chief descends underwater or becomes immersed in pools of liquid
coolant. The filter simulates hearing sound through water very well.
We’re looking forward to using this and other functions of the DSP in
the future.
All 3D audio is subject to occlusion, obstruction, Doppler, and
HRTF. Anytime a solid piece of geometry gets between the player
and a sound source, it is occluded and obstructed. A good example of
this is in the hangar bay of the alien ship where there is a dropship
with the engine running. As the Master Chief steps behind a column,
the sound of the engine is occluded, rolling off both the gain and the
high end of the sound. Other geometry that occludes sound in Halo
includes walls and cliffs. A grenade explosion sounds muted when
you are around a corner as opposed to having it in an unobstructed
field of view.
 A DirectSound Case Study for Halo n 425

Sound sources emanating from objects that travel through space

in Halo, such as the Sentinels or vehicles, were subject to a Doppler
shift. A moment during
Halo’s development that
stands out is when we were
first testing our looping
sounds as they related to
Doppler and hooked up the
sound of a hot rod engine to
the rocket launcher.
We hooked up the hot
rod loop to the tag for the
launcher’s projectile, Figure 17-1: A rocket launcher in action.
walked the Master Chief
across a big field away from
the camera, and fired it back toward the camera. What ensued was of
great amusement as the rocket flew past, generating the sound of a
hot rod going about 100 miles an hour and exploding in the cliff
behind us. It worked like a charm, and we left it hooked up that way
for a week or so for everyone to enjoy. The Doppler shift in
DirectSound worked perfectly and added a lot of realism to sound
sources that move around.
HRTF (Head-Related Transfer Function) also add to the audio
experience by filtering sounds, depending on which direction the
character’s head is facing. You can hear it affect the dialog in the III
cinematics as well as in the sounds that play during combat. Probably Part
the best way to hear this effect, however, is listening to your own
footsteps. As you move the Master Chief around in Halo, listen to
the sound of his footsteps on various surfaces. Then run through the
same areas looking down toward his feet rather than straight ahead
and listen to those same footsteps; the difference is stunning.
Occlusion, obstruction, Doppler, and HRTF are all aspects of
DirectSound that highlight its capabilities to utilize the same source
filtered appropriately to add a sense of realism without having to
author content specifically to achieve the same effect.
426 n Chapter 17

Surround Sound
One built-in feature to the Xbox chip that is absolutely amazing is its
ability to generate real-time 5.1 Dolby Digital output. We encode and
decode Dolby Surround on the fly from dynamic game audio so that a
player gets a full surround audio experience throughout the game.
This is a first in console gaming — and a great first at that!
There are many advantages to implementing 5.1 audio, from
alerting the player to the presence of enemies to surrounding the
player with ambience and music. While we did not premix the music
or ambience in 5.1, we were able to surround the player with sound.
Stereo music and ambience were played 100 percent out of the front
speakers and 50 percent out of the rear speakers and also sent to the
LFE. Non-positional dialog was sent 100 percent to the center
speaker and 50 percent to each of the front speakers. All 3D posi-
tioned sound was played back in its appropriate position among the
four surround speakers and also sent to the LFE. The Halo audio
engine queries the Xbox at startup to see whether it is set to stereo
or Dolby Digital and plays back the audio appropriately to avoid dou-
bling up audio signals in the stereo fold down.

Attaching Sounds to Animations

In Halo, there were thou-
sands of animations created
for several dozen charac-
ters. The animator provided
us with movies of a group of
animations that needed
sound effects, and we would
then sound design in Pro
Tools, making multiple per-
mutations for each one as
necessary. For animations Figure 17-2: An Infection Form closes in on the
such as characters getting Master Chief.
in and out of vehicles, char-
acters moving around, or
the sound of the Flood Infection Forms skittering about, we gener-
ated an appropriate sound effect and attached the soundtag directly
to the animation tag for that character. We could also trigger it at a
 A DirectSound Case Study for Halo n 427

specific frame of the animation in order to ensure the tightest sync

possible.

Real-time Tracking of Velocity and Pitch

The engine can also trigger sound effects based on velocity. Sounds
such as vehicle or shell casing impacts will play louder or softer,
depending on the object’s velocity at the moment of impact. If you
watch a shell casing hit the ground, it will bounce several times with
each repetition softer than the one before. For a variety of different
sounds, including vehicle
engines, we had the ability
to vary pitch according to a
scale. In the case of a vehi-
cle engine, the scale would
be the RPM of the engine at
any given moment. Like
most driving games, we
used multi-layered looping
samples at various areas
throughout the RPM spec-
Figure 17-3: The jeep wins a game of chicken.
trum, so the samples were
never pitch bent very far.
We also use the scale for sounds other than engines, such as the
Banshee contrail or the human dropship hovering. For the contrail, III
we raise the pitch on the loop as it thickens, lower the pitch as it
thins, and fade it out when
Part

you fly straight. For the

human dropship, we actu-
ally use the scale to modify
the gain instead of the pitch
so that when the dropship
hovers, the player hears a
swirling, windy sound.
When the downdraft jets
are activated, the gain
ramps up on the wind and
then ramps down as the Figure 17-4: Dropships on the beach.
downdraft jets deactivate.
428 n Chapter 17

The scale can modify pitch, gain, and skip fraction, depending on the
requirements for the sound effect.

Cascading Sound Effects

Cascading sound effects include sounds such as breaking glass or
rocks and gravel that explode off cliffs or are kicked up from vehicles.
Glass breaking is the easiest example to picture. We attach an initial
breaking sound to the moment when a glass surface was broken.
After that point, however, there are several factors that would affect
how the remainder of the glass breaking should sound. Depending on
the size of the original piece of glass (how much glass would fall) and
the distance of the glass from a solid surface, we had the ability to
trigger glass breaking sounds in “cascades” attached to the impact of
each glass particle. Keeping in mind that there are often hundreds of
particles involved, we triggered soundtags in the following manner.
When the first particle impacts, we call “glass small,” which
would be a soundtag of a very small glass particle hitting a surface.
We set a number (eight, for example) of repeated calls within a spe-
cific time period after which the audio code triggers “glass medium,”
which would be a soundtag of multiple glass particles breaking. On
that tag, we set another number (five, for example) after which the
code would trigger “glass large,” which is a soundtag of the mother
of all glass smashes. After the final soundtag is triggered, we could
start the entire cascading sequence over again if there is still more
glass breaking. Each of the three levels would, of course, have many
permutations so that each time glass breaks, you do not hear the
same series of sound effects over and over. This gives us the ability
to sync to individual particles in combination with the “chorus”
effect achieved by recording a larger, more complex event.

Dialog
The dialog in Halo was one of the areas that helped to give Halo a
unique flavor. There are two types of dialog: cinematic dialog that is a
traditional linear script and dynamic dialog. As you play through
Halo, your Marines will dynamically alert you to what is going on
around you, warn each other and you of danger, chastise, apologize
for allies getting shot or killed by friendly fire, and generally amuse
you. This aspect of Halo’s audio could not have succeeded without
 A DirectSound Case Study for Halo n 429

the hard work of AI (artifi-

cial intelligence) program-
mer Chris Butcher who
gave us many possible AI
states in which a character
might say something and
controlled how much was
said to make it all seem
natural.
In addition to the seven
distinct Marine voices, we Figure 17-5: Shotgun-toting Marines are great
also created four unique to have around.
voices for the alien races.
To make the voices even more colorful, we used a different actor for
each character and utilized various dialects (that really is an Austra-
lian in there). We went out of our way to have as many permutations
as possible for each AI state (sometimes over 20) so that you can
play the game for a long time and rarely hear a repeated line. We also
used the same AI framework for all of the alien speech, some of
which was intelligible (grunts in particular) and some that was not.
With such a good speech
system, we merely had to
apply our production values
to the recording sessions.
We used professional III
(AFTRA/SAG) talent and Part
produced, recorded, and
edited the sessions our-
selves to ensure we got
exactly what we wanted.
Actors were cast out of both
Chicago and Seattle and Figure 17-6: A Jackal has an unpleasant
meeting with the front of the jeep.
came to the studio multiple
times to record initial takes
and pickups later on.
The cinematic dialog also works well due to both a great story-
line by the Bungie team and excellent writing by Joseph Staten.
Scripts for the cut scenes and other interior scenes during gameplay
were written and storyboarded so that we could clearly deliver the
430 n Chapter 17

plot to the player. There is about one hour and 15 minutes of cut
scenes in Halo, and that puts this part of Halo’s production on par
with a short feature film.

The Final Mix

In the commercial, film, and theater industries, the final mix is one of
the most important steps in post-production. Those of you who are
in the game industry are likely laughing right now, as it is rare that
we have the time necessary at the end of a project to do a final mix.
In Halo, we got a chance to do some broad stroke mixing but were
never able to get down to the minutiae that make up an ideal final
mix. It’s almost as if Halo is an interactive movie that changes every
time you play it, and while we need to be cognizant of that fact, we
can still do more to make sure that the mix “sits” nicely in all cases.
There are still elements of the mix that annoy us (as there will likely
be in every game we do), and all we can do is seek to minimize them
as we approach the finish line for each game.

What Worked, What Didn’t, and the Future of Both

The Good
Some of the most memorable audio experiences playing Halo came
about because of a lot of hard work and a little luck. The Marine
dialog, for instance, was a huge outpouring of effort in terms of
scripting, recording, editing, and implementing. We did not, however,
have any idea that it would work as well as it did until we got very
close to shipping. The combination of great AI coding by Chris
Butcher, an extensive script (153 AI categories), some top-notch
improvisation in the studio by our cadre of actors, and a basic prem-
ise that each category should have as many unique permutations as
possible (5,600 permutations in all) led to a tapestry of Marine dialog
that is constantly changing and a great source of enjoyment.
There were also a lot of other things that worked really well,
including Dolby 5.1, the performance of the reverbs, Doppler, 3D
positioning, and our dynamic music system. We were developing
content for the game at the same time that the hardware was being
developed for the Xbox. Because of this, some of these features
 A DirectSound Case Study for Halo n 431

were not a sure thing until late in the summer of 2001, mere months
before shipping. The Xbox audio team did a great job delivering what
they promised, making it a great platform on which to develop.

The Not So Good

There were some things that did not work so well that we will fix for
future games. Reverb morphing is something we had to hack in the
Bungie audio engine to prevent hearing the borders of reverb areas
pop, but it does not work perfectly in all cases. The HRTF function
and some 3D positioning were too heavy-handed during cinematics,
and we are looking at ways to disable or limit it in future titles. In a
film, character voices do not swing around to the rear speakers
because the camera angle changes. In Halo, there are actually a few
spots (the bridge in the first level, for instance) where the camera
position is so close to a character’s face that the audio engine thinks
the sound source is behind the player. Looking at a close-up of the
Captain’s eyes but hearing his voice coming from behind you is dis-
concerting. In Halo, everything but the music and ambiences were
stored at 22.05 kHz ADPCM compressed and result in quite a bit of
artifacting, depending on the source. It is most noticeable in the
speech. In future titles, we will be moving to 44.1 kHz for all of our
content, which will alleviate most artifacting.
The initial music we produced included some fine playing by
members of the Chicago Symphony and Chicago Lyric Opera III
Orchestra. Due to time constraints, we were forced to use sampled
instruments for the bulk of the production done during 2001. While
Part

there are quite a few good samples out there, they are not a replace-
ment for live musicians. For future titles, we will build time into our
schedule to record live musicians where appropriate.
Another time constraint issue was facial animation. We only had
the ability to open or close a character’s mouth in reaction to the
amplitude of the speech sample. For future games, we will be adapt-
ing a more robust facial animation system, which will make the
characters appear much more realistic when they are speaking.
There were several content areas in which we were constrained
to mono sound effects, such as first-person animations and all 3D
positional audio. Some 3D positional audio, such as the jeep engine,
should be stereo. There is work being done both at Bungie and
throughout Microsoft that will allow for stereo sound effects to be
432 n Chapter 17

3D positioned and will bring more life to sounds such as vehicles and
weapons.
Occlusion is another area we would like to refine. Our engine did
not allow for movable objects, such as doors, to occlude. For upcom-
ing games, we will be working to make sure that anything that
should occlude will occlude.
Our use of DSP was limited and is an area that has a lot of poten-
tial. We will be working with our programming team to allow us full
use of real-time EQ, LFO, and plug-ins such as limiters. We will also
be looking at the use of better data compression for high-quality,
low-memory audio in certain areas of the game.

Creating content for a game is merely half the battle. Implementing

that content in collaboration with a strong programming team is
absolutely essential, and in closing we would like to thank Jason
Jones and the Bungie programming team for giving us the tools and
support necessary to help make Halo a great-sounding game.
 Chapter 18

A DirectMusic Case
Study for Interactive
Music on the Web

Overview
Background

W eb content today consists of sophisticated graphics, animation,

and interactive visual content. By contrast, web sites use
sound only sporadically, often as the occasional button sound or
introductory flourish. Web sites use music even less frequently, and
this music is often repetitive and of poor sound quality. The differ-
ence that sound and music can make in a web presentation is enor-
mous, however — just as it is for other audiovisual media, such as
film or television. Music has the power to set a mood appropriate to
the content and enhance the overall impact of a web site. Appropri-
ately, for an interactive medium like the Internet, interactive music
can enhance a web site further still. It can contribute to the usability
of the site, assist navigation, and add emphasis to key content.
Bandwidth is the main cause for both the lack of quality and use
of audio content on the web. Standard uncompressed sound files (i.e.,
.wav and .aiff) are large and therefore take a long time to download
over slow connections. Compression reduces the size of sound files
but deducts from the quality of the file. Poor sound quality is incon-
sistent with the high production values expected of quality web
content. While short sound effects are usable, short, repeating music
loops become tedious to the listener very quickly.
433
434 n Chapter 18

The General MIDI (GM) standard is one solution to the band-

width problem, playing MIDI files using standard instrument
mappings to ensure that the sequence performs using the correct
instruments. The benefit of using General MIDI is that small MIDI
files are the only sound-related file required for downloads. The trou-
ble with General MIDI on the other hand is that the instruments are
limited to a defined set of 128, mostly variants of standards, such as
piano, guitar, and basic orchestral instruments, and tend to be poor
quality. General MIDI instruments can vary dramatically in timbre
and quality from one sound card to another, despite the intended
standardization. Any digital musician knows, for example, that piano
sounds on two different brands of sound cards can be drastically dif-
ferent from one another. This difference can wreak havoc on the
balance and feel of a piece of music.
Streaming music is one alternative to General MIDI synthesis.
Streaming consists of downloading sound files bit by bit as they play,
allowing playback to start relatively quickly, thus decreasing the per-
ceived loading time of the site. Fast streaming requires a significant
amount of compression, even with broadband access becoming more
commonplace. Also, streaming ties up valuable bandwidth for the
duration of the music file. This can slow down the loading of other
pages or cause the music to stop as other pages use bandwidth to
load.
Looking beyond the bandwidth problem, it is worth considering
that web content is interactive by definition. Standard music delivery
methods are designed to play a sound or MIDI file from start to
finish and do not allow for user interaction. As web content is inher-
ently interactive, linear music (regardless of how it is delivered) can
never truly do justice to the unpredictable nature of the user experi-
ence. For this, we need a system that is not only bandwidth efficient
and provides high sound quality but is also capable of responding to
user actions flexibly and in real time.
DirectMusic is ideal for delivering music over the web. Its use of
the DLS standard for synthesized instruments makes it bandwidth
efficient. For instance, a drumbeat using a DLS instrument may
consist of a kick drum sample, snare drum sample, and two hi-hat
samples. All of these samples together are likely to amount to less
than one measure of the whole beat but allow for endless permuta-
tions and fills, while treatments such as panning and filtering further
 A DirectMusic Case Study for Interactive Music on the Web n 435

add to the possibilities. The same amount of sample data required for
a single prerecorded drum pattern as part of a stream is therefore
able to produce countless variations. A sound designer can manipu-
late and reuse samples in new instruments without adding additional
material to the download. Furthermore, the sound designer can com-
press the samples on a case-by-case basis, allowing for the best
compromise between size and quality. In addition, since the user
downloads the actual DLS instrument as web content, the music
sounds consistent across the full range of playback hardware. The
ability to use real-time effects such as reverb, chorus, delay, and dis-
tortion further enhances the potential to make the most out of a few
samples, as well as adding depth to the final mix.
DirectMusic scripting adds the vital ingredient of interactivity.
DirectMusic content creators feed button clicks, mouse movements,
and other triggers into the script and use them to shape the music in
accordance with the user’s interactions with the site. It is also possi-
ble to give the user direct control over aspects of the music. This
control can range from something as simple as a volume control to
much more abstract control, say, over subtle harmonic changes; the
possibilities are limitless.
DirectMusic Styles and patterns allow for a great deal of control
over variation, making it perfect for background music that needs to
remain interesting over a long period of time (for instance, while the
user is reading the contents of the site). Within a pattern, you can
create up to 32 variations of each pattern part and control the play- III
back of these in a variety of ways. Parts, and even individual notes, Part
have parameters that can apply a degree of randomness to playback.
It is possible, with these features, to create self-varying musical
accompaniments guaranteed not to repeat in a predictable way.
DirectMusic’s efficiency, variability, and interactivity make it
uniquely suited to use on the web. When I first investigated the pos-
sibility of using it in this way, however, there was no available means
of communicating user interactions to a DirectMusic script. There-
fore, it became necessary to create an ActiveX control to do this.
436 n Chapter 18

The Wide Sounds DMX Control

Wide Sounds identified the potential of DirectMusic for music on the
web. We created the DMX control, an ActiveX control that accepts
various control messages and converts them into routine or set vari-
able calls and then passes these calls to DirectMusic. The DMX
control can also extract information from DirectMusic and pass it
back to the caller. The data for the control is stored in a .cab file on
the host’s web site and is downloaded and extracted when the con-
trol loads. Thus, a web page can use the DMX control as a generic
interface to DirectMusic’s capabilities.

Figure 18-1: The DMX control in action.

Prior to the development of the DMX control, using HTML+TIME

(Timed Interactive Multimedia Extensions — extensions to HTML
adding timing and media synchronization support) was the only way
to play DirectMusic content from a web page. This approach did not
provide the vitally important scripting functionality. In any case,
Internet Explorer 6 and later no longer supports HTML+TIME.
Creating the DMX control enabled us to have much greater
interactive control over the playback of DirectMusic content on the
web than was previously possible. By allowing a web designer to call
routines and set variables in a DirectMusic script from user interac-
tions, a vast amount of interactive functionality becomes available.

Functionality
The potential uses of scripted music for a web site are almost limit-
less, but here are a few of the key techniques and components that I
find useful when designing interactive music for a web site:
n Play Segment: Play a specific piece of music for a content area,
or start a new piece when clicking through to a different part of
the site.
n Transitions: Script changes to the primary Segment in such a
way that they provide a seamless transition in order to maintain
musical coherence and flow.
 A DirectMusic Case Study for Interactive Music on the Web n 437

n Motifs: Experiment with using motifs over the background

music.
n Chord tracks: Map out the harmony of a Style, allowing you to
fit secondary Segments such as motifs harmonically over the
background music.
n ChordMaps: Vary Styles harmonically using the branching func-
tionality of ChordMaps. Anything from simple key changes to
random chord progressions are possible.
n Mix effects: Use specially designed secondary Segments to
manipulate a wide range of mix parameters. You can adjust the
volume of different instruments, real-time effect values such as
delay feedback, DLS instrument filter cut-off, and many more
useful parameters for livening up a mix by using the continuous
controllers, the Pattern Tracks, or Parameter Control Tracks.
n Randomness: Use script routines to assign a random number to
a variable. This enables interactions to have an unpredictable
effect. You can control the probability of outcomes to keep the
randomness within reasonable limits. You can also give varia-
tions within a Style a degree of randomness, further avoiding
repetition.
n Timers: Use timers to lock and unlock features during transi-
tions, control queues of events to avoid clashes, or force varia-
tion after periods of inactivity. You can create timers by placing
script routine calls at a certain point in a Script Track within a
III
secondary Segment. Timers are a very useful under-the-hood
Part

element, especially in complex projects.

n Automatic variation: Embed routine calls in Script Tracks uti-
lizing any of the above-mentioned features. This is a useful way
of avoiding too much repetition.
Consider these techniques as a “toolbox” of features with which to
enhance the content of a web site.
438 n Chapter 18

The Web Site

We use the DirectMusic content from the Wide Sounds web site but
in a simplified HTML version for the purpose of this case study. We
designed the full site using Flash, and while you can control the
DMX control from Flash, it is easier to demonstrate the basics using
HTML. I removed some functionality and content in order to make
the HTML as clear and simple as possible.
The site provides information about Wide Sounds’ services, our
client roster, press releases, and contact info. The intended users of
the site are potential clients who have heard of the company and
want more information.

Figure 18-2: The front page of our site.

The structure of the site is very simple; it consists of a front page

that links to six sub-pages. I removed our technology demo from this
version as well as the link to its page. The Wide Sounds logo
remains present throughout and provides a link back to the front
page from each sub-page.
 A DirectMusic Case Study for Interactive Music on the Web n 439

The Adaptive Audio Design

We have looked at the interactive functionality available with
DirectMusic and the DMX control, and we are familiar with the
structure of the web site. The next step is to plan our music treat-
ment. We must make decisions about the style of the music that we
create and how it integrates interactively with the web site.

Style and Aesthetics — What Is Right for the Site?

When considering the style and functionality of the web site’s music,
avoid thinking in terms of your own particular taste. Similarly, do not
attempt to cram in as many tricks and clever interactions as you can
think of. Over-embellishment can annoy the user and obstruct or
confuse the purpose of the site. Consider the user’s point of view:
Who are they? What do they want from the site?
Visitors to the Wide Sounds site are most likely from the games
industry. Perhaps they are producers searching for an overview of
the company. They might want to read a bit about the services we
provide and find out about our background. As we are a music com-
pany, it is natural that we want our site to feature music, and as we
provide interactive music, it makes sense to reflect that in our site
music as well. However, the company information is the focus, so the
music should merely serve to enhance the user’s experience and
give a sense of quality.
III
For the purposes of the Wide Sounds web site, we use a gentle, Part

unobtrusive musical style that simply aims to provide a pleasant

backdrop. We have no idea what kind of musical style our visitors
want for their game projects, so it seems sensible to avoid an obvi-
ous game style. We do not want to pigeonhole ourselves as elec-
tronica artists, classical composers, or some other genre. Having a
very general style allows us to appeal to visitors from outside the
games industry, in our other markets of advertising, or on the web.
440 n Chapter 18

Form and Function

Finding the Hooks
Mouse clicks and rollovers drive the music script behind the Wide
Sounds web site. These interactions are very simple to implement
and give us the control we need in most circumstances. With this in
mind, we can identify which interactions should control our script.
We refer to these interactions as “hooks,” and it is important to
identify them before designing the music so that the musical content
matches the functionality of the site. You should look at the site that
you are creating music for in detail, become familiar with its struc-
ture, and think about how your music can complement and enhance
the content. Consider the amount of time users are likely to spend in
each area, which buttons will be used most frequently, and which
areas or features of the site should be highlighted musically.
We have six lozenge-shaped buttons on the front page used to
navigate the site. Navigating between content pages is clearly the
main interaction with the site, and therefore we utilize changes in
the primary Segment to reflect this priority. As a further embellish-
ment, when clicking through to a different page, an “opening” or
“closing” motif plays. This technique stems from the Flash site
where the new page visually expands or collapses from the button,
although the musical idea remains valid without the expanding/col-
lapsing animation. We also use button rollovers to play various notes
while forcing those notes to conform to the background harmony in a
variety of ways. This breaks down to a different note on each button,
all of which relate to the chord playing during the rollover. This turns
a row of buttons into a basic musical instrument that plays notes in
harmony with the background music! The constantly present Wide
Sounds logo links back to the front page from each of the content
pages. Because it is a logo, an “ident” style motif, as commonly used
in corporate branding, seems an appropriate embellishment. Finally,
we have added some simple controls specifically for the music:
Volume +
Volume –
Music Off
Music On
 A DirectMusic Case Study for Interactive Music on the Web n 441

Our web site is very simple, so there are only a small number of
hooks. Sites that are more complex may require more complex
music implementations. DirectMusic can certainly accommodate the
most demanding scripting requirements.
Once you have determined the hooks you will be using to drive
the music, you can begin to think about creating musical content to
match the site structure and interactive behavior.

Matching Music to Site Content

As our site is purely informational and has no distinctly themed con-
tent areas, such as pages dedicated to the different game titles that
we have worked on, there is no obvious requirement for distinctly
themed musical content. Therefore, the musical backdrop focuses on
atmosphere and variety. If the site had themed pages (say, historical
or geographical for instance), we could incorporate elements of the
respective themes in the musical backdrop. The backdrop could even
transition between different pieces of music as the user navigates to
different pages within the site.

Thinking as the User

Web site music should enhance a visitor’s experience. For the pur-
poses of the Wide Sounds site, the music should also encourage
them to retain our services. It is important to consider the user’s
expectations, needs, and experience when designing music for web III
sites.
What is the average length of time that a visitor views any given
Part

page on the site? Are there elements of the site that the user inter-
acts with quite often? How can the musical content aid navigation
and usability? Which recurring elements require consistency
throughout the site for clarity? Understanding how the site functions
helps in designing appropriate interactive audio.

Content Creation
While a detailed discussion of content creation techniques in
DirectMusic Producer is beyond the scope of this case study, there
are some techniques worth considering here. Download size and the
unpredictable nature of user interactions are particularly important
442 n Chapter 18

issues when designing interactive music for a web site. Unlike most
games, web sites can potentially be viewed for long periods with
very little interaction. Alternatively, the interactions may come thick
and fast, and the music must be able to respond without becoming
incoherent. Consider these issues carefully when creating content.
Make sure that DLS instruments make the most of a small amount
of sample data. Build variety into your patterns to maintain interest
when no script routines are being triggered. Create Segments that
can transition smoothly at any point.

DLS Instrumentation
Long loading times are a big turnoff for web site users, and although
our DirectMusic content loads separately from the rest of the site, it
is still important to keep the size of the download to a minimum. You
should pay particular attention to the DLS instrument sample data,
as this is the bulk of downloaded material.
It is important to minimize sample data, while maintaining the
highest possible quality. The target download size for this site is less
than 100KB. To meet this requirement, we choose instruments that
do not require large samples. Long multi-sampled orchestral instru-
ments require many more samples than anyone has the patience to
download. Use short looping source samples that spread across the
keyboard without suffering undesirable effects. For example, use
samples of analog style waveforms, such as sawtooth waves, as the
basis for DLS instruments. Only a few cycles are required and a wide
variety of sounds can be created from the same source. For drums
and percussion, create a small number of source hits instead of long
loops. Also, reuse samples at various pitches and with various enve-
lopes as the basis for different instruments to get the most mileage
out of them. Be creative and generate many rich, multi-layered
sounds all from the same source samples.

Sample Format and Compression

DirectMusic can use DLS instruments created from wave files
across a range of sample rates and bit depths. I find that 22.05 kHz,
16-bit samples usually strike the best balance between size and qual-
ity and often respond well to compression. It is possible to combine
 A DirectMusic Case Study for Interactive Music on the Web n 443

different sample rates within an instrument or DLS collection

though; this is sometimes necessary to get the best results.
DirectMusic in DirectX 8 has incomplete support for MP3 and
WMA compression, so Microsoft ADPCM is the best compression
type available in most cases. ADPCM compression reduces the bit
depth to 4 bit, so do not reduce a 16-bit sample to 8 bit if you plan to
compress the sample. The compression ratio achieved is typically
just short of one to four, so in some circumstances it is worth using a
44.1 kHz sample if it compresses well, rather than an uncompressed
22.05 kHz version.
It is important to consider how different timbres respond to com-
pression. For instance, when using ADPCM compression on 22.05
kHz drum sounds, you may find that a snare drum hardly suffers at
all, while a cymbal can become gritty and lose its shine or a rounded
bass drum may gain clearly audible compression artifacts. In some
cases, simply replacing one sample with another of the same type of
sound can be enough to get the quality you want without drastically
increasing the file size. Alternatively, you may need to use a higher
sample rate version of the source sample in order to achieve an
acceptable compressed version. For this reason, you should always
keep a backup of the highest quality version of every sample in your
project.

Instrument Choices
III
Considering what we know at this point, we decided that our instru-
mentation should lean toward analog style pads, basses and synths
Part

based on a small number of samples, and programmed rhythm parts

using a small palette of drum sounds. Of course, instrument choices
can change during the creative process. Our site uses two instru-
ments (Guitar1 and Marimba) that do not conform to these
instrument choices. We felt that the trade-off between variety and
download size was justified. You should consider the eventual cost in
download time of each instrument that you use from the very
beginning.
In our final DLS Collection, four samples make up the six non-
percussion instruments. The sample Pad1 is the source for three of
these instruments — Arpeggio1, Pad1, and Wibble.
444 n Chapter 18

Figure 18-3: The DLS Collection.

Look in the Waves folder in Flash.dlp, and observe that the final ver-
sion uses nine samples amounting to 207KB of storage space,
uncompressed. Of these, seven responded well to ADPCM compres-
sion, and the final run-time DLS file weighs in at 89KB — within our
target range even before we create the final downloadable archive, at
which point we can expect some further file compression.
By making the right instrument choices, reusing samples, and
selectively using compression, it is possible to create a good quality
sound palette and keep the download to an acceptable size.

The Styles and their Patterns

After identifying the required musical content and establishing the
resource limits that constrain it, the next step is composition and
creating the DirectMusic assets. The source material played back by
Segments is contained in Styles. A Style is like a collection of MIDI
tracks combined with many settings unique to DirectMusic that
influence the way they play back. You can import MIDI files created
in another sequencer or create content in DirectMusic Producer.
The MIDI content in a Style is organized into patterns. We must cre-
ate a pattern for each musical passage and define the conditions
under which they will play back.
 A DirectMusic Case Study for Interactive Music on the Web n 445

Styles for Each Content Zone

The music for our site is contained in one Style (see Figure 18-4),
since the music design avoids drastic stylistic variations within the
site. If the site had distinctly themed zones as discussed above, we
would almost certainly want to use a separate Style for each zone,
although the organization of DirectMusic assets is a matter of per-
sonal preference.

Figure 18-4: The Style, expanded

to show all the patterns.

Pattern Structure
III
All the patterns in the Style share more or less the same structure.
They are all eight bars long, and they all share the same chord
Part

sequence (as defined in the Chords for Composition track) and draw
on a main pool of pattern parts contained in “All” and “All2.” These
patterns serve as a source of linked Pattern Parts for the other pat-
terns and have a groove level of 100 so that they themselves never
actually play. Linking pattern parts reduces duplication of content
and further reduces the size of the final run-time files.
Our patterns fall into two groups. Patterns 101 to 106 are
derived from All, while 201 to 204 are derived from All2. Patterns
101 to 104 provide a progression and some variation for the main
page of the site, and Patterns 105 and 106 add a guitar motif with
varying accompaniment for half of the content pages. Patterns 201 to
204 provide the music for the remaining content pages, based on a
sparser arrangement of the same basic elements.
446 n Chapter 18

Figure 18-5: A Pattern Part using Variation Switch Points.

The percussion parts use variations and variation switch points to

avoid repetition and add interest to the rhythms. By adding variation
switch points at frequent intervals in every variation, the path
through the Pattern Part changes every time. This works particu-
larly well with percussion parts, where some amount of unpredict-
ability can greatly enhance the sense of a live performance.

Groove Levels
See the groove levels used by all the Patterns in Main.stp below.

Figure 18-6: The Style Designer window showing the Groove Range settings for
each Pattern.

Patterns 101 to 106 use groove levels one to seven roughly in order
of intensity. Groove level one is the most sparse, used only by the
Segment Intro.sgp when the site loads. The Segment Front.sgp, fea-
tured on the main page of the site, uses groove range two to four,
thus selecting from Patterns 102 to 104. Guitar.sgp uses groove
range five to seven or Patterns 105 and 106.
We assigned Patterns 201 to 204 to the groove range 11 to 16;
Minimal.sgp uses this groove range. The assignment of groove
 A DirectMusic Case Study for Interactive Music on the Web n 447

levels to patterns allows us to choose specific patterns or groups of

patterns for playback from a Segment. The Repeat Pattern menu in
the Segment’s Groove Track Properties dialog box adds a further
dimension to these playback choices. You can use the drop-down
menu shown below to select the method for choosing from multiple
patterns of the same groove level.

Figure 18-7: The Repeat Pattern menu.

Weighting groove ranges can influence the choice of patterns for

playback at run time. As the list of groove level assignments in Fig-
ure 18-6 shows, we assigned Patterns 201 to 203 to a single groove
level, while we assigned Pattern 204 to the range 14 to 16, or three
groove levels. If the Segment uses groove range 11 to 15 (as the III
final version does), Pattern 204 will be selected 40 percent of the
time, while each other pattern appears 20 percent of the time.
Part

In a more complex project with distinctly separate content zones

and Styles or where embellishments such as fills and breaks are
required, it is necessary to create custom patterns for those events.
You can assign these patterns to one of the various embellishment
types for playback from custom Segments. Our web site did not need
such patterns.

The Primary Segments

As you know, primary Segments play back music from a Style, and
secondary Segments play motifs and controller data or trigger timed
script routines. Our site uses four primary Segments. The Intro
plays when the web site loads using the lowest groove level. Intro’s
448 n Chapter 18

expression controller data fades in the Arpeggio and Wibble instru-

ments. Intro also has a Segment Trigger Track that triggers a
secondary Segment containing filter cutoff controller data and then
triggers the next primary Segment. Main, shown below, plays after
Intro and on every subsequent visit to the main page. Guitar triggers
when the user clicks through to the first, third, and fifth content
pages. Minimal triggers when the user clicks through to the second,
fourth, and sixth content pages. Please be aware that I disabled the
second content page in the HTML version of the site for this project.

Figure 18-8: Main.sgp.

Figure 18-8 shows Main.sgp as seen in the Segment Designer win-

dow. All of our primary Segments are constructed in the same way.
Each Track influences the selection and manipulation of Patterns in
different ways, as we can see by looking at Chord Tracks.

Chord Tracks and ChordMaps

There are many reasons to use Chord Tracks in a DirectMusic pro-
ject. Having a Chord Track in a primary Segment allows any second-
ary Segments to match the changes in harmony. This ensures
coherent playback, regardless of when the secondary Segments trig-
ger. DirectMusic can manipulate the harmony of properly mapped
Patterns in real time, enabling key changes and variable chord
sequences. The eight-bar chord sequence in our Patterns is
expanded into a 16-bar sequence in the Segments, simply by extend-
ing the Chord Track. No new note data is required.
The most important thing when working with chords in Direct-
Music is to map the Chords for Composition accurately. If done
correctly, harmonic changes work properly.
 A DirectMusic Case Study for Interactive Music on the Web n 449

Figure 18-9: The Chord Properties dialog.

After assigning chords to the relevant measures, set the playmode

for the different tracks so that, for instance, percussion parts do not
transpose. Inversion boundaries are also useful to restrict the range
across which a part can transpose.
The seven different playmodes provide much control over Pat-
tern Part responses to chords in a Chord Track. You can also
override the Pattern Part playmode setting on a note-by-note basis.
The rollover events on the main page of our site illustrate the
different playback modes. For instance, the lozenge-shaped buttons
each play a single note using the Chord/Scale playmode. These notes
are therefore transposed to the current chord and scale, maintaining
their harmonic functions. The Wide Sounds logo plays a motif that III
uses the Pedal Point playmode, playing the same notes every time Part
unless those notes don’t appear in the current scale. If the notes fall
outside the scale, they are transposed to the nearest appropriate
note.
It is possible to create variable branching chord sequences using
ChordMaps and Signpost Tracks. By using weighted connection lines
between the chords in a ChordMap and a set of signposts to deter-
mine the boundaries of the chord sequence, you can create anything
from simple alternating key changes to rambling, unpredictable free-
form chord patterns. This kind of sophistication is not necessary for
our simple site, but it can add a huge amount of variability while
barely increasing the size of the download.
450 n Chapter 18

The Secondary Segments

Now that we have our primary Segments, we can create the second-
ary Segments containing motifs and controller data that will be
triggered on top. In the planning stages of the Wide Sounds site, we
established that we needed a number of motifs. Loz1.sgp through
Loz6.sgp are individual notes used as rollover sound effects for each
of the six content buttons. WideLogo.sgp is a rollover sound effect
for the Wide Sounds logo. WipeUp.sgp is an opening motif for click-
ing through to a content page. WipeDown.sgp is a closing motif for
returning to the main page. Lastly, there is an extra motif, Scr.sgp,
that is unused in the HTML version of the site; it is a tuned clicking
loop used while scrolling text in the Flash version.

þ Note My preference is to create motifs as Pattern Tracks in Segments

rather than use the motif object type within a Style. This is because I find
them easier to control and more flexible than motifs.

It is very important to test motifs thoroughly in context, bearing in

mind that you can call the triggering routines at any time.
You can use the Secondary Segment toolbar in DirectMusic Pro-
ducer, shown below, to audition motifs while a primary Segment is
playing. Play every combination you can think of, and observe what
happens when multiple instances of the same motif trigger in quick
succession. This may cause overlapping notes to cut off, polyphony
to be exceeded, or the motif to sound extremely loud. If undesirable
effects occur, you will need to add functionality to your script to pre-
vent the playback of multiple instances.

Figure 18-10: The transport controls and Secondary Segment

toolbar.

Once you create your script, you can test the functionality of motifs
(and the script routines that drive them) further, but in the early
stages the Secondary Segment toolbar is a great help.
 A DirectMusic Case Study for Interactive Music on the Web n 451

There is one final secondary Segment in the project, triggered

upon loading and left to loop continuously from then on.
ClickFilt1.sgp contains a Pattern Track with filter cutoff controller
curves affecting the instrument Arpeggio.

Figure 18-11: Controller data in a CC Track, simulating

an LFO.

The purpose of this Segment is to act as an LFO, constantly tweak-

ing the instrument’s filter cutoff to add interest to the sound.

AudioPaths and Effects

Real-time effects can give depth to sounds made from small source
samples, turn mono into stereo, transform a sound into something
completely different, and enhance the depth of the mix. However,
when using many effects, it is important to consider the type of com-
puter the end user has. It is a good idea to test using a minimum
spec PC to make sure that you have glitch-free playback.
Look at FlashPath.aup and observe our AudioPath; it consists of
four mix groups. One is dry, two use different combinations of
Reverb and Echo, and the other uses the I3DL2 Reverb. A configura-
III
tion like this should not cause any playback problems. Using the Part

reverbs and stereo delays adds significant depth and atmosphere to

the overall sound, which is, after all, created entirely from tiny mono
samples.

Figure 18-12: FlashPath.aup in the AudioPath Designer window.

To ensure that any Segments you audition in DirectMusic Producer

play back on the correct AudioPath, select FlashPath.aup from the
default AudioPath drop-down menu located above the Project folder.
452 n Chapter 18

Testing the Functionality

Just as the Secondary Segment toolbar allows you to test the play-
back of motifs and other secondary Segments, the Transport List and
Controls allow you to select primary Segments for playback.
Right-clicking on the Transition (A/B) button brings up the Tran-
sition Options dialog (see Figure 18-13), allowing you to audition the
various transition methods between primary Segments. The avail-
able settings correspond to those in the Boundary tab of the
Segment Properties dialog, many of which can also be set in the
script. This is a good way to test how the transitions sound and
fine-tune your boundary settings before you create the script.

Figure 18-13: The Transition Options dialog.

At this point, we have created and tested all of the required content,
as defined in the planning stage. In summary, the project consists of
one Style, one DLS Collection, four primary Segments, seven sec-
ondary Segments, and one AudioPath. We are now ready to script the
project.
 A DirectMusic Case Study for Interactive Music on the Web n 453

Scripting
Scripting is the key to creating interactivity with DirectMusic. The
script drives the music in response to calls to its routines from the
DMX control. You have to create script routines for all of the hooks
defined previously. You may find “under-the-hood” routines useful in
controlling automated functions and in testing conditions for other
routines.
Getting the script right is vital for properly functioning music.
The slightest error can cause a routine, or even the whole script, to
fail. Having said this, AudioVBScript is a very simple scripting lan-
guage. Microsoft designed an easy-to-learn system for people with-
out a programming background. I learned how to use AudioVBScript
quickly, with no programming experience. This section provides
some useful pointers in relation to scripting a web site project.

Creating the Script

Consider in advance what routines your script requires. For our site,
we need routines to trigger the motifs Loz1.sgp to Loz6.sgp,
WideLogo.sgp, WipeUp.sgp, and WipeDown.sgp, the transitions
between the primary Segments, and the volume controls. Addi-
tionally, the DMX control must trigger the “start” routine automati-
cally once the content loads. Name these routines clearly early on;
this way, you can easily identify which routines the script uses in
III
performing various tasks when we add the calls into our HTML. I Part

like to go through the site, writing down the names of the routines I
will use for each interaction as I come across it.
You must make the content files available to the script before it
can use them. Do this by dragging the files from the project tree into
either the Embed Runtime or Reference Runtime containers in the
Script object. Each approach has advantages and disadvantages.
Dragging a Segment into the Embed Runtime container embeds the
Segment into the run-time version of the script file. In other words,
you do not need to include the run-time Segment file when you
deliver the final content files. This method is attractive, as it reduces
the number of files required for delivery. On the other hand, embed-
ding run-time files often involves complicated dependencies of
embedded files, especially where you use multiple scripts.
454 n Chapter 18

The Reference Runtime container does exactly what it says. You

must include the run-time Segment file along with the script file.
The script file references the Segment file at run time. The appeal of
this approach is that you do not need to worry about dependencies,
as long as all files are present. It also allows you to update a single
component file without changing any of the other files, which is often
useful in the final stages of testing.
Drag the content files into the Reference Runtime container;
then create the routines required to play them. A sensible starting
point is the start routine that the DMX control calls once everything
loads.
This script creates a basic start routine:
sub Start
Intro.play
end sub

Although it is not required for the site, it is always useful to have a

“stop” routine in the script while testing. The following stop routine
halts any looping Segments. You do not need to include the short,
non-looping motifs, as they never continue more than a second or so
after the primary and secondary Segments stop.
sub Stop
Front.stop AtImmediate
Intro.stop AtImmediate
Minimal.stop AtImmediate
Guitar.stop AtImmediate
ClickFilt1.stop AtImmediate
Scr.stop AtImmediate
end sub

The AtImmediate flag ensures that the Segments stop right away,
regardless of their boundary settings.
Any routines in the script appear in the list to the right of the
script (see Figure 18-14). Double-clicking on a routine in this list
executes that routine. By entering one routine at a time, you build
the script up gradually until all the required functionality is present.
 A DirectMusic Case Study for Interactive Music on the Web n 455

Figure 18-14: Routines and variables

displayed in the Script Designer window.

Creating an AudioPath Object

In DirectMusic Producer, Segments play back on the AudioPath
selected in the AudioPath list, shown below. If you want to use a cus-
tom AudioPath at run time, you must specify the AudioPath in your
script.

Figure 18-15: The AudioPath list.

Create an AudioPath object when the script starts and specify that III
object each time a Segment plays. This ensures that the run-time
version uses the correct custom AudioPath.
Part

sub CreateAudioPath
if AudioPathLoaded <> 1 then
AudioPathLoaded = 1
FlashPath.load
set AudioPath1 = FlashPath.create
end if
end sub

The routine above, when run for the first time, loads the AudioPath
FlashPath and defines the variable AudioPath1 as FlashPath.create.
Adding the parameter AudioPath1 to a play command now ensures
that the Segment plays back on the correct AudioPath.
456 n Chapter 18

sub Start
CreateAudioPath
Intro.play 0, AudioPath1
end sub

This version of the start routine initializes the AudioPath and plays
Intro back on the correct AudioPath by running CreateAudioPath.

Working with Variables

Scripts often require variables for retrieving, setting, and storing
integer values such as Master Volume or for conditional statements.
Variables exist within an individual routine, or they can be global and
thus available to any routine. Define global variables at the top of the
script by typing dim VariableName. Once you define the global vari-
ables, they appear in the variable list (below the routine list — see
Figure 18-14). This list displays the current value of global variables
and allows you to edit them manually for testing.
The Script Volume.spp, which is dedicated to the site’s volume
controls, uses the global variable Vol to track the master volume of
the Performance. Calling Down first uses GetMasterVolume to
retrieve the current master volume and then sets the variable Vol
accordingly. Down uses Vol in a conditional statement to determine
whether the master volume is at or above its minimum value. If the
master volume is not at the minimum value, Down reduces Vol by
500 and uses SetMasterVolume to set the master volume to the
newly reduced value of Vol (an attenuation of 5 dB).
sub Down
Vol = GetMasterVolume
if Vol > -9600 then
Vol = Vol - 500
end if
SetMasterVolume Vol
end sub

Volume exists as a separate script because I like to keep mechanisms

such as volume controls as easily accessible chunks of script. This is
a habit that I acquired from working on much more complex game
projects, and it is arguably unnecessary in such a small project. Com-
munication between the scripts is straightforward. Simply add the
name of the target script to the routine call in the parent script, as
shown in the following example.
 A DirectMusic Case Study for Interactive Music on the Web n 457

sub VolDown
Volume.Down
end sub

Variables can also be set and retrieved between scripts in the same
way.
Achieving desired functionality often requires complex interac-
tions between multiple routines and variables, especially in situa-
tions where the result of a particular routine differs depending on the
currently playing Segments. For example, when a transition occurs,
you may want to lock out certain motifs or prevent another transition
from occurring until the first one finishes. In these situations, you
may need to use several variables to track exactly what happens at
any given moment. You may also need to use conditional statements
to determine whether to start the new transition or motif or initiate
a queuing or locking mechanism. Our little web site happily lacks
such complexities, but DirectMusic scripting can cope with most
things that you care to throw at it if necessary!
Even with a simple project, keep scripts clear and tidy. There are
likely to be logical groupings of routines, and maintaining clarity
makes editing, testing, and bug fixing considerably quicker and eas-
ier. Any text that comes after an apostrophe is a comment (see
below). Comments are useful for explanatory notes or disabling bits
of script while testing.
sub Lozenge1
Loz1.play IsSecondary, AudioPath1' This is a comment
III
end sub Part

' sub Lozenge2

' Loz2.play IsSecondary, AudioPath1 This whole routine is commented out
' end sub

As with any interactive system, it is impossible to overemphasize

the value of thorough testing. Test constantly throughout the script-
ing process, and thoroughly check the script after any changes — no
matter how isolated they are.
458 n Chapter 18

Delivery
Once everything checks out in DirectMusic Producer, test the sys-
tem as part of the web site. First, create the run-time versions of the
DirectMusic files.

Figure 18-16: Saving run-time files from

DirectMusic Producer.

You can save all the run-time files at once from the File menu (see
Figure 18-16). The default destination is a folder called RuntimeFiles
created in the Project folder. If you embed the content files rather
than reference them, it may not be necessary to save run-time ver-
sions of every file. Avoid duplicating files or including obsolete files
that increase the total size of the download.

Packaging of Run-time Files

The DMX control downloads a compressed .cab archive file and
expands it to temporary files. This means only a single file needs to
download, and a final stage of compression applies to the content
files.

Figure 18-17: Compression settings in

WinAce.
 A DirectMusic Case Study for Interactive Music on the Web n 459

Using a compression utility such as WinAce (see Figure 18-17), add

the run-time files to an archive with “Normal” compression. Our
archive is WSMain1.cab and resides in the site structure in /music/.

Implementation
The DMX control works with HTML, Flash, and other web technol-
ogies. You can also use it in other applications that support ActiveX
controls, such as MS PowerPoint and Word. To use the DMX control
with HTML, you need a few lines of JavaScript to direct the routine
calls from the HTML to the DMX control, which is located in a dif-
ferent frame (refer to the discussion on framesets in the “Putting It
All Together” section) and identified as “Ctrl.” The JavaScript
audio.js is located in the Includes directory in the root of the site.
Reference audio.js in the head of each HTML document containing
routine calls as follows:
<!-- JavaScript to call into DMX Control -->

<script src="includes/audio.js" language="JavaScript"></script>

The triggers we use to call routines are those related to mouse

actions (i.e., user interactions):
OnClick
OnMouseOver
III
Our entire music implementation uses just these two triggers and Part
the following syntax:
OnClick="Call('RoutineName')"

Therefore, a call to the routine Section1 looks like this:

OnClick=”Call(‘Section1’)”

These calls are usually, but not exclusively, added to anchor tags.
460 n Chapter 18

Putting It All Together

The DMX control needs to remain loaded throughout the visit to the
site, so it is necessary to create a frameset containing two frames,
one for the pages of our site and the other for the control.
Index.htm, the default home page of our site, simply defines the
frameset:
<html>
<head>
<title>wide sounds :: pioneering and cutting edge interactive audio</title>
</head>
<frameset rows="445,*" framespacing="0" framepadding="0" frameborder="no"
frameborder="0">
<frame src="home.htm" name="main">
<frame src="control.htm" name="dmx_control">
</frameset>
</html>

Home.htm loads into main, and control.htm loads into dmx_control.

We have defined the layout so that main has a fixed height and
dmx_control sits beneath, keeping things tidy if the window resizes.
We hide the DMX control in the actual web site, but it is visible here
so that you can see the feedback that it provides.

The “Control” Object

Control.htm is a page containing nothing but the DMX control object.
In the object definition, the codebase parameter defines the location
of the control itself, and the name parameter identifies the control as
Ctrl.
<html>

<body bgcolor=#ffffff>

<div style="position: absolute; top: 0px; left: 265px; width:250px;">

<object classid="clsid:7A1B929A-67C2-40C9-85B2-748E007EB2F2"
codebase="http://www.widesounds.com/control/DMX.cab#version=1,0,1,5" id="Ctrl"
width="243" height="48" name="Ctrl">
<param name="_Version" value="65536">
<param name="_ExtentX" value="6429">
<param name="_ExtentY" value="1270">
<param name="_StockProps" value="0">
<param name="URL" value="http://www.widesounds.com/music/WSMain1.cab">
<param name="Start" value="Start">
<param name="SampleRate" value="44100">
 A DirectMusic Case Study for Interactive Music on the Web n 461

<param name="Background" value="26,162,192">

<param name="Control" value="26,162,192">
<param name="Text" value="0,0,0">
</object>
</div>

</body>
</html>

The various <param> tags define the appearance of the control, the
music source file, and the name of the Start routine that automati-
cally triggers when the content loads. When control.htm loads, the
DMX control either installs or launches, and loads the file defined in
the URL parameter. This URL must be absolute; relative URLs will
not work. The bottom half of the control displays text output describ-
ing its current state, such as Loading Script or Running. Right-click-
ing on the control and clicking on the Output tab displays all of the
output generated thus far, including details of routine calls. This out-
put is helpful in diagnosing scripting errors.

Attaching Triggers to Anchor Tags

Attaching triggers to the pages of the site is extremely easy. Simply
add the trigger definition as described above to the definition of an
anchor tag. For example, I define the volume controls in the follow-
ing piece of HTML:
<!-- Volume Controls --> III
<div style="position: absolute; top: 155px; left: 577px; width: 200px;"
Part

class="text_small">
<a href="#" OnClick="Call('VolUp')">Vol +</a> |
<a href="#" OnClick="Call('VolDown')">Vol -</a> |
<a href="#" OnClick="Call('VolOff')">Music Off</a> |
<a href="#" OnClick="Call('VolOn')">Music On</a>
</div>

Each anchor tag defines the text (e.g., Vol +) as a dummy link
(href=”#”) with the appropriate routine call added using the
OnClick trigger.
Multiple triggers are usable in the same tag, as in the following
example:
<a href=”whois.htm” OnMouseOver=”Call(‘Lozenge1’)” OnClick=”Call(‘Section1’)”>
462 n Chapter 18

This navigation button has one action for a rollover (the tuned single
note) and another for a click (‘Section1’), which triggers two routines
— one starting a new primary Segment at the next measure and the
other playing the motif WipeUp.sgt.
Going through the HTML and adding the relevant triggers
where necessary completes the music implementation. This is sim-
ple once the routine calls pass successfully into the DMX control,
and you test each piece of functionality as you add them.

Test, Test, Test!

The final step, as always, is testing the music in the context of the
site. As the permutations of interaction are vast and highly unpre-
dictable, it is sensible to spend some time using and abusing the site
to see how the music performs. Although users are likely to move at
a sensible pace around the site, stopping to read some sections, per-
haps skipping over others after a cursory glance, we need to be sure
that it stands up to anything that the user may do. Some time spent
trying as hard as possible to break it with frenetic button clicking is
very worthwhile.
 Chapter 19

A DirectMusic Case
Study for Worms Blast

Game Overview

W orms Blast was developed by Team17 Software Ltd. and pub-

lished by Ubi Soft Entertainment. I (Bjorn Lynne) composed
and arranged all of the music. That being said, let’s take a look at
how the music was created for this unique interactive game title.

Figure 19-1: Worms Blast.

463
464 n Chapter 19

Worms Blast looks simple at first but reveals its depths and surpris-
ing longevity as you play it, much in the same way as Tetris or the
predecessor of Worms Blast, Worms. The game features two play-
ers, each placed in a boat floating on water. The water has waves and
is difficult to move around in. The water level rises if objects fall in
the water. Overhead is a slowly descending landscape of colored
blocks, and between the two players is a wall. Occasionally, a window
opens in the wall between the two players, allowing the players to
fire directly at each other with a selection of wacky comedic weap-
ons. The landscape overhead moves slowly down toward the player.
By using a selection of weapons, the player can shoot color combina-
tions of the blocks that make up the overhead landscape, causing
them to disappear.
The game has several different modes, each with various goals,
such as outlasting your opponent, collecting certain combinations of
crates and colors, hitting certain targets, and so on. Worms Blast has
a large number of hidden weapons and comedy elements. The game
features characters already established in the long-running and
highly successful series of Worms games, and it has a fun, bright, and
colorful appearance. All in all, it is a game that looks very simple at
first, but the longer you play it, the more you discover its depth and
lasting playability.

Figure 19-2: Worms Blast.

 A DirectMusic Case Study for Worms Blast n 465

Music Preparation
In the early stages of trying out music content for this game, I used a
custom-made music system featuring a fairly straightforward back-
ground music track along with short sampled phrases of various
instruments, each belonging to one of the different characters in the
game. So if the Calvin character scored points or picked up a weapon,
his instrument would play a motif at the next possible entry point,
marked in the background music file with markers. The system
worked well enough, but because all phrases had to fit anywhere in
the background music, the music had to be very uniform, and it just
didn’t sound very inspiring or heartfelt. The music turned out rather
bland in comparison to what I had hoped for. I scratched my head for
a while and decided to give DirectMusic a go.
I had witnessed demonstrations of DirectMusic at a couple of
game development seminars. I was impressed with the technology
but less so with the resulting music. The concept of having a piece of
music written to a set of chords and then overlaying a new set of
chords to that music scared me. This was probably because the
examples I was hearing, while interesting from a technical point of
view, just didn’t sound very good. Based on this impression, I
decided early on that I would not use any of DirectMusic’s chord fea-
tures in this game. I was adamant that if I wrote a melody, I wanted
that melody played back exactly how I wrote it. III
I had never used DirectMusic Producer before, and as I loaded
up the application for the first time, I admit I was a little intimidated
Part

by the sheer number of screens, options, and features. Much of the

terminology was new to me: Segments, Bands, Styles, Groove
Levels, and ChordMaps. I’ve written music for games for the last ten
years and used just about every type of music and audio program
available, but this was the first time I encountered these terms.
Much of the first week was spent studying the supplied demo files
and reading the online help documentation. Things began falling into
place, and pretty soon I had my first sets of patterns and motifs up
and playing.
466 n Chapter 19

Making the Motifs Fit the Background Music

As the first bars of real game music came together, it occurred to me
that my decision to steer clear of all chord features could land me
with the same problem as with my earlier non-DirectMusic
attempts. Along with the background music, I was writing a number
of small motifs to be played on top of the background music. In order
for the motifs to work together with the background music at all
times, the background music needed to be uniform. After all, if the
motif was written in G7 and I refused to let DirectMusic alter it in
any way, when the background music played in say, B, we would have
some less-than-appealing note combinations.
I went for the middle ground, writing the music using one scale
and a wide variety of chords based on that scale. I found that my
motifs would work equally well on a C minor as a G minor or an
Fsus4, so long as I stuck to my scale. Even if the background music
was currently playing an Fsus4 (which doesn’t contain a G), if my
motif had a G in it, that was fine, so long as it didn’t include a note
that was outside of the scale.
I picked a scale to work with and made highly melodic music
using plenty of chords. My motifs played successfully on top of this
music without ever having DirectMusic change or tweak a single
note of music that I wrote. Now I could really get down to work and
write the real meat of the music content for Worms Blast.

Samples and Instruments

I created samples by recording my synthesizers, guitars, and other
instruments using SoundForge. I saved these recordings out as wave
files and imported them into DirectMusic Producer, where I sat for
hours and built my DLS instrument banks. I used DirectMusic Pro-
ducer to work out loop points, rather than SoundForge. I was always
very impressed with the “find best loop end” feature in DirectMusic,
which seems to be better at finding functional loop points than any-
thing else I’ve tried. In fact, I still use DirectMusic Producer to find
loop points in files when creating samples.
I decided early on not to use the preconfigured General MIDI
instruments. In my experience, music made with General MIDI
 A DirectMusic Case Study for Worms Blast n 467

instruments tends to get that “General MIDI” sound that gives off a
whiff of low budgets and shareware distribution. So I custom made
every sample in every instrument used in the game.
One of the most time-consuming tasks involved in this project
was the breaking down of premade drum loops. I split up each loop
into several very short loops, which I then rebuilt in DirectMusic
Producer. I would take a one-bar drum loop and slice it up into six-
teenths. This resulted in 16 stereo samples. Then I would import
these 16 stereo samples into DirectMusic Producer and design an
instrument with 32 regions — 16 left and 16 right. At the time I was
using DirectX 8.0, and DirectMusic Producer treated stereo samples
as two separate samples, so two separate regions had to be created
in the instrument designer. Luckily in version 8.1, this can be done
faster and easier, since the instrument designer now recognizes ste-
reo samples and automatically creates the two separate regions.
A useful tool in the above process is Square Circle Software’s
WaveSurgeon (www.wavesurgeon.com), which slices loops and saves
the individual short phrases as separate wave files and also exports a
MIDI “timing template.” This small MIDI file can be imported into
DirectMusic Producer as a pattern and, when played with the instru-
ment created from the individual loop slices, gives a reproduction of
the drum loop, which can be played back in different tempos. So if
the music increases or decreases in tempo, or the game experiences
a momentary system halt or slowdown, the drum loop will still be
locked to the rest of the music. III
Part

The Project Structure

Worms Blast was my first project using DirectMusic, so I was still a
little hung up on old ways of thinking about music as something that
has a beginning, a middle, and an end. I set out to create four differ-
ent in-game “music tracks,” each with a dedicated sample collection
and its own project folder. The game would then pick a random
project at the start of each round, load it, and start to play music from
it.
Half way through the project, I had some regrets about not creat-
ing all of the music in the entire game as a single project. Had I, it
would have been easier to share instruments, patterns, and motifs.
468 n Chapter 19

Looking back at it now, I’ve come to realize that perhaps it was for
the best after all. By separating the in-game music into four separate
projects, plus a fifth project for the front-end menu system, I had
less of a temptation to reuse the same instruments in too many of
the patterns. Since I was creating separate DLS instrument collec-
tions for each of the five music sets, I felt that I might as well make
all the instruments unique to that set. This gave the music and
instruments plenty of variety, an important issue for those wanting
to play the game for hours.
Creating five separate projects kept the number of Styles, Seg-
ments, and Patterns to a manageable level. Each of my five projects
contained about 60 different Patterns, 30 Segments, ten motifs, but
only one Band. This helped reduce the level of confusion that can
sometimes arise from the fact that, in a DirectMusic project, there
are different Bands all over the place that can affect the instruments
in unexpected ways. This isn’t helped by the fact that Bands some-
times have to be copied from a Style to a Segment or to a Script. If
you go and update one of those Bands, the other places where the
same Band exists do not get updated, leaving you with several copies
of a single Band with the same name, which you thought were all the
same but are in fact different from each other. The fact that I had
only one Band for each of my five different projects — even though
that band had to be copied to about 30 different Segments, and in
doing so were left independent of each other — helped ease the
chances of getting into a mess with the Bands.

Using Groove Levels to Set the Mood

In Worms Blast, the player must keep his character from being
squashed between the water under his boat and the blocks overhead.
If the water level rises too high, the space above the character’s
head will narrow, and he will soon find himself in trouble. I wanted to
use this in the music, to let the distance between the water and the
overhead blocks determine a “stress factor” or “danger level,” which
would be reflected in the music.
For each of my four different in-game music projects, I wrote
about six different main musical parts. I then created five different
versions of each of these parts with different intensities. For
 A DirectMusic Case Study for Worms Blast n 469

example, I had an eight-bar pattern called “main melodic part A —

level 1.” It was calm, laid-back, almost cozy. I copied it to a new pat-
tern and called it “main melodic part A — level 2.” In this version I
introduced a few more drum sounds, but it was still pretty laid-back.
I kept doing this until I had five different “mixes” of the same pat-
tern. The fifth one was very intense, with drums playing double-time
and some distorted instruments playing busy melodies along with
the main melody.
I then assigned groove levels to each of these patterns 11
through 20, 21 through 40, 41 through 60, and 61 through 80. I asked
the game programmer to use the distance between the water and the
overhead landscape to generate a number between 11 and 80. This
number fed into the DirectMusic engine as a groove level. With this
model in place, whenever the water gradually sank or rose or the
landscape overhead gradually caved in or withdrew, representative
patterns would play. The end result was that the music had a higher
stress level, according to how much danger the player’s character
was in.
I composed a few more patterns and assigned them to groove
levels one through ten. These patterns were very low intensity, basi-
cally just a subdued rhythm with a few occasional notes. I called this
“pause music” and asked the game programmer to set a groove level
of one when the game was in Pause mode. Pausing the game causes
the music to change to “pause music,” while keeping a steady
rhythm going. This worked really well; it sounded natural, slick, and III
cool. Part
In addition to the different pattern intensities, we also imple-
mented slow, subtle changes to the master tempo, which was tied to
the groove level. As the distance between the water and the blocks
decreased, the groove level increased and so did the tempo. We cre-
ated a tempo scale from about 100 percent to about 120 percent of
the original tempo, as set in the Segments, and had the tempo
change by one percent at a time. This combined with the different
patterns with varying intensity levels to further enhance the connec-
tion between the stress level in the game and the stress level in the
music.
470 n Chapter 19

Motifs for Minor Game Events

I decided to use “motifs” in two different ways in Worms Blast: one
for major game events, such as losing a life, triggering a firestorm,
triggering double-damage mode, etc., and another one for minor
game events, such as picking up a crate, gaining some extra health,
hitting a target, etc.
I used motifs to react to minor game events. We had nine differ-
ent motifs, each containing just a few notes with bright, sparkly
instruments so that they would cut through the mix and be heard
even if there was a lot of stuff going on in the background music. I
set the motifs’ boundary settings to Beat so that they would play in
time with the music, starting from the next quarter note. This
ensured that the motifs would play almost instantly but still not fall
out of the rhythm set in the background music.

Motifs for Major Game Events

There were important game events in Worms Blast for which simple
motifs were not sufficient to represent. These events included losing
a life, triggering a very big and powerful weapon, or entering some
special game mode. There were 18 of these major game events that
were important enough, not only to play a motif on top of the back-
ground music, but to actually come in and replace the background
music completely for a little while.
The way I solved this was to write 18 different patterns with
very distinct music, using dramatic effects and instruments that
stood out, such as gongs, sweeps, electro effects, etc. I had earlier
reserved groove levels 81 through 100, so I assigned these 18 pat-
terns to individual groove levels from 81 up to 98. For example, the
two-bar pattern for “Double Damage Mode” contained a Chinese
gong — a little Chinese style koto melody, and it was assigned
groove level 87. When “Double Damage” occurred in the game, the
game would input a groove level value of 87. This resulted in my
two-bar “Double Damage” tune being played from the next bar line,
replacing the background music instead of just playing over the top
of it. Obviously, the tempo remained unchanged, something which
 A DirectMusic Case Study for Worms Blast n 471

helped make this two-bar break fit into the soundtrack as a whole
and sound like it was meant to be there.
At the end of each two-bar “special event” pattern, I put a drum
fill-in/build-up to create a natural-sounding transition back to the reg-
ular background music.

Things I Would Have Done Differently

I can really think of only one major mistake that I made in the devel-
opment of the Worms Blast music, and that was to sample all my
instruments at 44.1 kHz, like I’ve always done. Through my years of
writing and producing music, I’ve gotten used to always starting out
with files of the highest possible fidelity and then downgrading as
and when necessary to save resources. I assumed that the music
would play back at 44.1 kHz in-game. My plan was to keep the most
“fidelity-demanding” instruments, such as sparkly keyboards, hi-
hats, shakers, transparent and open pads, etc., in 44.1 kHz but down-
grade other samples, such as bass sounds, etc., to a lower sample
rate to save system resources.
However, several months later I was told that the audio engine in
the game would only run at 22.05 kHz. So, having some samples in
44.1 kHz would be a waste of resources since it would, in effect, be
resampled to 22.05 kHz at the point of delivery. It turned out that I
might as well have done all of my samples in 22.05 kHz from the III
beginning, as I had to downgrade everything to 22.05 kHz in order Part
not to waste system resources with 44.1 kHz samples.
With DirectMusic Producer version 8.0, there was no easy way
to resample samples. I had to go back to my original sample sources,
use SoundForge to resample to 22.05 kHz, save each sample to a
new filename, and then go back to DirectMusic Producer and high-
light the sample, choose Replace Sample, and then find the newly
saved 22.05 kHz sample on my hard drive. This may not sound so
bad, but many of my instruments had many regions with separate
samples — not to mention my sliced loops, which could have up to
20 stereo samples (which DirectMusic Producer at the time treated
as two separate samples), making it necessary to perform this time-
consuming procedure to 40 separate samples, just for a single
instrument.
472 n Chapter 19

Actually, it was after this process that I managed to convince the

DirectMusic developers to put a “resample” feature in DirectMusic
Producer, something that was available from version 8.1 — too late
for me, but at least I feel I can take some credit for getting that fea-
ture included for others to benefit from.
I guess the moral of the story is when working with Direct-
Music, forget your old method of always working on a copy of your
music in the best possible fidelity and then downgrade as and when
necessary. Instead, find out the sample rate that the end product is
going to be delivered at before you do any sampling at all, and use
that as your highest frequency.

Conclusion
To sum up the Worms Blast music design, the game features the
following:
n Four different “sets” of background music
n About eight different musical parts in each set
n Five different intensity levels in each part
n Eighteen different two-bar patterns for major game events
n Nine different small motifs for minor game events
n Tempo changes to intensity
n Separate DirectMusic content for the front-end menu system,
the credits screen, and the world map screen
All in all, the game contains 338 musical patterns, most of which are
eight bars in length (some of which are two bars long), and 36 motifs.
It also has approximately 150 custom-made instruments, most of
which contain multiple regions.
Perhaps equally interesting is what this implementation doesn’t
have. First of all, it has no chords. Of course, the music has chords,
but as far as the DirectMusic engine is concerned, everything is in
the default chord of C major. It also doesn’t have ChordMaps, a con-
cept I admit I still haven’t really looked into. The game doesn’t use
any DirectMusic scripts. Instead, most of the interactive features are
implemented by the use of groove levels.
 Chapter 20

A DirectMusic Case Study

for Asheron’s Call 2:
The Fallen Kings

Introduction

T urbine Entertainment developed Asheron’s Call 2: The Fallen

Kings. Geoff Scott composed the music for the game. Dan Ogles
handled the DirectMusic programming. I designed the music system
and integration.

Music for Massively Multiplayer Games

Understanding the dynamics of the game is the first step to design-
ing a music system. It is important to ask yourself many questions
about the player’s experience that go beyond simple stylistic con-
cerns. Asheron’s Call 2 is an “online massively multiplayer role-
playing game,” or MMRPG for short. A massively multiplayer game
involves thousands of users playing together in one large fantasy
game world, hosted online. A role-playing game (RPG) is one in
which the player assumes the life of a fictional character that he or
she creates. We call this character an avatar. An avatar in Asheron’s
Call 2 is a three-dimensional character that can be one of a number of
fantasy races, be either gender, and have various customizable physi-
cal traits. In an MMRPG, gameplay is not clearly defined; players can
roam the game world as a loner, or they can choose to work with
other users to solve various quests and puzzles.

473
474 n Chapter 20

Massively multiplayer games (MMPs), in their current form,

have some unique challenges worth considering when designing a
music system. For instance, the average MMP game session is much
longer than most single-player game sessions. In addition, people
tend to play MMP games for many months or years before growing
tired of the game. Regardless of your development budget, chances
are you’re going to have a hard time making the music interesting
for an extremely long period.
The sheer size of an MMP is also daunting. Many of the tech-
niques used in a traditional interactive scoring approach may not be
viable simply because you cannot create music for every place or
place cues for every location. In our case, we are dealing with a
seamless world, which means we cannot rely on convenient divi-
sions to start and stop music or load in new data.
You also do not have control over in-game situations. In a tightly
scripted single-player game sequence, you only have to worry about
one person interacting with the game, and therefore it is possible to
set up musical changes based on what the listener expects to hap-
pen. With an MMP, an area that is supposed to be a hotbed of
excitement might be completely swarmed with players who cleared
out all the monsters, thus making it seem more like a safe haven
than a battleground.
Finally, when making an MMP, there is a strong desire to be able
to change data after the game’s release. In the case of Asheron’s
Call, we built the franchise around stories that propagate to the user
over small monthly patches. This has greatly influenced our data
structure choices, as we want to be able to make huge changes to
the game in very small amounts of data so our users don’t have to
suffer through large downloads. For us, being able to add new music
in a small amount of data was also important.
However, while there are a lot of new restrictions imposed by
the current MMP game designs, there are some unique opportuni-
ties as well. First, MMPs are primarily a social experience. While
you might think that you are building a certain type of game, what
you are really building is an experience for people with common
interests to socialize around. At any given time, a very large percent
of the user base is simply socializing around a meeting point or com-
mon interest within the game. Many of the systems in these games
are designed to encourage social interaction, as in the long run it will
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 475

be the social aspects of the game that keep many of your users sub-
scribed to your service.
Given this, it is necessary to look at the activities that friends
enjoy together, as well as those popular and universal throughout the
world’s culture. It is also important to look at the tools people use to
identify themselves as individuals while identifying themselves with
various groups within society. An MMP that successfully provides
these activities and tools will be one with a compelling social base
and therefore have a longer subscriber life.
In my experience, music has the potential to provide all of this
and more. Music is fundamentally a social experience. Whether you
are going to a concert or playing in a band, it is a social experience at
heart. Outside of downloading pornography, chat and music down-
loading are also the two largest activities on the web. This makes
some type of musical activity the perfect fit for an MMP, which is
why we decided to add some form of player-created music into the
game.

Project Goals
Before we move into the details of the music system, let’s define a
common set of terms, review the problems, and discuss potential
solutions to those problems:

Repetition III
This is probably the number one reason I turn the music off on so Part
many games. I simply get sick of hearing the same thing repeatedly.
It is important that the game’s music vary each time that it plays.
The wider the variance, the longer the ear can withstand the same
basic song playing.

Adaptive
When many people talk about “interactive” or “dynamic” music sys-
tems, they are often talking about the music system being able to
adapt to changes in gameplay. Much like a good film score, the music
should highlight key moments in the gameplay, making things more
exciting and supporting the mood.
476 n Chapter 20

Interactive
By my definitions, an interactive music system is one that the user
can directly control in some way. To accomplish this goal, AC2 allows
players to play music with each other using a simplified musical
interface. Additionally, the Tumeroks, a type of playable character (or
race, to be more accurate) in the game, use drums to cast magic
spells, which also work with the music system.

Informative
My final goal was that the music system be informative. It is my
belief that a passive music system does not hold the listener as long
as a system that is providing information interesting to the listener.
We spend an amazing amount of time making the visuals of a game
clue in players about their surroundings, and so should the music.
Many of these clues can be subtle, leaving the player with subcon-
scious information about situations and surroundings. For AC2, the
scenery around the player determines the background music, while
monsters bring in individual melodies that warn the player of their
presence.

Code Budget
We knew from the beginning of the project that our programming
time and support would be limited. The music system was very
much my pet project for AC2. Others treated it with less priority
than it deserved. The team passed it off from one programmer to
another, and it accrued bugs along the way. Given this, we tried to
design things to “just work” with as little custom code as possible.
We also knew that eventually we would work on other projects, so
others might not follow any rules that we established about how to
place music in the world in the future.
Our mechanism for triggering music was the same as triggering
sound. A music hook looked just like a sound hook to the system, so
anywhere that we could call a sound sample we could call a Segment.
This was useful because it meant we could attach music hooks to
character animations or spell effects through our “fx system” or
environments through our ambient sound system. Once triggered,
DirectMusic handles everything else.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 477

Project Overview
I hope that this section gives you a good idea of how we structured
the entire project, from both a technical and musical perspective.

Style and Feel

The feeling that we wanted in our music originally stemmed from
the style of playable races in Asheron’s Call 2. The Tumeroks, which
pull reference from tribal cultures, would play drums to channel
magic, rather than the traditional wand or scepter. Lugians, another
playable race in the game, pulled their reference from places like
Tibet and brought to mind the sounds of Tuvan throat singers and
Tibetan monks. The Humans would represent what people expect,
sonically, from an RPG, the traditional classical element, orchestral
scores, and church choirs.
Our first step was to create a piece of music for each race that
encompassed these elements and set the mood for each race. We
later used these pieces in the character creation screens to enhance
the mood of each race. Once we were happy with these pieces, we
created a score that combined the sounds and melodies of all three
pieces to set the style for the game itself.
With this settled, we began to work on the design for the
in-game music.
The system evolved over the course of many months until it
III
arrived at a point with which we were satisfied. A brief overview of Part

what we finally settled on is as follows:

n Location in the game world determines the current ambient
score. This sets basic things like tempo, key signature, Chord-
Maps, and a background musical score. This “master Segment”
is attached to a specific combination of a terrain and scene (such
as grass with willow trees) or on the physical geometry of a dun-
geon area.
n Each monster in the scene adds a musical element to the score.
Much like a subtler version of Sergey Prokofiev’s “Peter and the
Wolf,” each of these parts can have many variations.
n Players may hold a musical instrument in hand, such as a lute or
drum, and play various emotes, which emit a 3D-located melody
478 n Chapter 20

or rhythm from their instrument. If they do not have an instru-

ment, they can beat box.
n Each monster and avatar may add or subtract from the intensity
level of the music. Each monster or potentially aggressive avatar
in the scene adds two to the groove level, while each non-
aggressive avatar subtracts one. The default groove level with no
monsters or avatars in the scene is 30. This allows us to define a
relative intensity for the music based on the rough state of the
area. If the area contains mostly avatars, it is unlikely that any-
one is in any danger, and the music becomes calmer. Conversely,
if the area has a reasonable number of monsters in it, the groove
level will rise, and the music will become more intense.

Time Feel
Like many other projects, what we set out to do was quite a bit
smaller than what we ended up creating. Our original thinking cen-
tered on getting the Tumeroks to be able to cast spells using drum
rhythms and allowing them to play together in an interactive drum
circle over the background score. Our first step in this was to get a
series of bar-long rhythms working together with one-beat fills that
would sound pleasing while played during any beat of the measure.
Each part would have to be satisfactory by itself, yet add to the feel-
ing of the group.
One of the experiences that we really wanted to emulate was the
push and pull of time using polyrhythms. By incorporating the tradi-
tions of certain world music, such as African and Latin, we were able
to transition and modulate between the feel of multiple time signa-
tures while remaining in a single time signature. For instance,
playing in 6/8 using a rolling six feel, then playing four dotted eighth
notes against that to create the polyrhythm of “four over six,” and
then using those dotted eighth notes as the primary rhythmic pulse
creates the feeling of playing in 4/4 while remaining in 6/8. Using
polyrhythm like this can obscure the bar line and allude the ear as to
the placement of one. This allowed our Tumerok drummers to fire
off spells at any beat without disrupting the flow of the music.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 479

Figure 20-1: Tumeroks are deeply in tune with the spirits of the
world around them. They channel magical energy through
drums to attack their foe.

We chose a master tempo of 80 in the time signature of 6/8 because

it is slow and leaves plenty of space. Using 6/8, we incorporated mul-
tiple time feels, including 3/4, 4/4, 6/8, and 12/8. They can work by
themselves or against each other. Doubling the tempo to 160 beats
per second is still a comfortable pace and gives us an entirely new
palette of time feels to work with. We built even more complex feel
changes and polyrhythms from these fundamentals. A 9/8 feel con-
structed of triplets off of the quarter notes in the 3/4 rhythm is
III
difficult to make work as a fundamental, but it can be a nice
Part

polyrhythm if subtly used.

The Stack
Using these fundamentals as a base to work from, Geoff Scott, the
composer on the project, went into Fruity Loops and began to com-
pose the initial drum patterns. Using the sounds of a Djembe, Dun
Dun, and Todotzi, each drum would incorporate a different time feel.
As we created each pattern, we referenced it against the previous
patterns, creating a large “stack” of rhythms. Writing this way
ensured that everything worked against everything else and pro-
vided insight into the music’s architecture as a whole. It was
important to balance the timbres of the drums, spreading the thin
480 n Chapter 20

and thick or high- and low-pitched notes across the measure so that
no one beat became overly accented. We then reviewed each rhythm
to make sure it was satisfactory on its own as well as in the context
of the entire score.
Once we were happy with this, we moved the data into Direct-
Music. We then composed a series of small fills on each drum that
would represent the different types of spells in the game. We
grouped these spells into similar categories, such as attacks,
enchantments, heals, summons, buffs, and debuffs. We gave each
category its own rhythmic figure. When the user casts a spell using
a drum, the rhythmic phrase creates an accent. A call and response
event occurs when several players cast spells.

Background Score
Our goal for the background score was for it to sound almost as part
of the environment’s sound design itself. We wanted to keep things
relatively simple and ambient and avoid ear-catching resolutions and
stabs. We also knew there would potentially be a very large number
of melodic parts playing over a given background score and that
those melodies would be the primary force adding tension and reso-
lution to the composition.
We chose modal pads with ambiguous harmonic movement —
sweeps and swells of color that provide mood over traditional mel-
ody. Like our rhythmic scheme, the lack of harmonic resolution
would not tire the ear. Similar to the concept used in Indian music,
we stated the given scale as a raga (the tonic droned and then played
upon for an entire afternoon), creating a meditative space without
the repetition and destination of a cycling chord progression.
To give the music a sense of global awareness to the player’s
current predicament, we used various modes to color the music
accordingly. A player standing alone in an area would hear the back-
ground score in the Lydian mode, but as creatures fill the area, the
music transitioned to Phrygian, giving a darker tonality. If monsters
continue to outnumber the player, the music might transition to a
diminished mode while a quickening drum groove begins to pulse.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 481

Monster Melodies
After we had some basic background music working, we wrote a few
melodies, which we tied to individual monsters in the game. When
one of these monsters is in the area, the melody warns the player of
its presence. We wrote these melodies as if they were part of the
background score, but designed them so that they could play on any
measure in the score. We tried to capture the feel of the creature in
the sound of its DLS patch and used information about where differ-
ent creatures would appear to better mix the parts with each other.
While we could not rely on this information as being correct, it gave
us a good point to test and design from, as in most cases we knew
what creatures were likely to appear together.

Player Melodies
With a basic background score and our initial stack of rhythms estab-
lished, we started to create the player melodies for the game. These
would be two-bar melodies, which the player could trigger anywhere
in the game and would start playing on the next marker placed every
two bars in the music. When multiple users play multiple melodies,
they get the feeling of playing music together.
We started by composing against our rhythmic stack, using a
bass and lute sound. We created ten melodies for each instrument
and pulled from multiple time feels contained in the existing drum III
rhythms. Creating this stack was probably the most difficult aspect of
this project. Geoff and I went home with intense headaches on more
Part

than one occasion, as wrapping our heads around all of the data was
quite a task.
Geoff’s approach to the initial instrument, the lute, was to write
using counterpoint and harmony, while keeping the melodic range as
tight as possible to leave room for the other instruments. The initial
lute parts used two scalar melodies with three-part harmony coun-
tered against each other, while pulling rhythmically from the four and
six feels. We based the harmony on one chord structure and concen-
trated on the color of that chord’s chord tones and tensions, while
being aware that the melodies would transpose to new chords and
through chord progressions.
482 n Chapter 20

Like the drum rhythms, each part needed to be sustainable by

itself, as well as in a group. We maintained a careful balance between
making the parts interesting and not making it so interesting that it
became repetitive or dominated the mix.

Dimensions of Musical Space

One of the key concepts that allow this project to work
compositionally is the defining of musical space. We designed the
rhythmic feel for the game to allow as many avenues for musical
space within time as possible, providing multiple time feels within a
single time signature and tempo. We applied the same concepts to
the melodic space, as we needed to account for passing tones and
tensions across each beat of the music. While it’s natural to think
about the melodic content as following the same time space as the
rhythmic content does, melodic instruments have a quality that
many percussive instruments do not — the length of the note. In
many ways, this is yet another avenue of musical space to explore.
The final area of space is that of timbres. When the sounds of the
instruments are different enough, the ear has a much easier time
distinguishing them from other notes of different timbres.
These concepts of musical space became very important as we
began to add more instruments to the stack. As the stack became
larger, it was too unwieldy to manage as a single entity, and it was
necessary for us to write new parts with only subsets of the stack
available. Our ears could not truly hear 30 lines at once, let alone
over 200. However, once you understand how we wrote the original
melodies and the concepts of musical space, it no longer becomes
necessary to have the entire stack available for listening. You still
need to test against subsections of the stack, but eventually most of
it becomes something that you do not think about on a theoretical
level but just hear and do. In most cases, we tested each stack of ten
melodies against a selection of about 20 melodies from other
instruments.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 483

Expanding the Stack

Our first alternate instruments were simple repatchings of what we
created previously. The lute melodies were now available in a
marimba or harp sound, and we even added several new drums as
well. Simple repatchings gave us a wealth of variety with little work,
but we wanted to add new musical feelings to the mix. All the instru-
ments so far were of the same characteristic — a quick attack with a
fast decay.
Next, we added three instruments that filled a very different role
— long tone ambient sounds. We created the Ice Lute, Virinidi Lute,
and Barun Lute to give the player music long tones. In the case of
the Virindi Lute, each melody is three bars long and stresses upper
structure arpeggios and intervallic play. The Ice Lute uses a cold,
breathy, pad sound and uses the tensions of the chord and a forced
ambiguity of harmonic structure while maintaining the color of the
mode. This allows the instrument to float and weave through the
harmony, rather than define it. The Barun Lute follows this same for-
mula but with a very different timbre, more hollow and dry sounding
than the Ice Lute.
We also added flutes, which have a very different sound charac-
teristic that any of the instruments mentioned previously. I wrote
unique melodies for them, which stress the use of long held notes
and harmonies. We designed each melody to provide a contrast with
the original instruments. Each melody has a harmonized version. III
Modes and Chords
Part

Once again, I must stress the evolution of this project as a key moti-
vator in how we did things. Once every basic component was
working, we embarked on a long series of tweaks to make the overall
project more interesting and discovered ways in which we could
have done things better the first time. However, each of the restric-
tions that we placed on the project eventually became a motivator in
some new way to expand the project.
One such restriction was writing the background melodic score
in a single mode. While this made it much easier to write over and
provided the feel that we wanted, the player music suffered from the
484 n Chapter 20

lack of active chord changes. Individual parts would sound repetitive

if the players did not actively change the part that they were playing.
The solution was to write modal chord changes and map individual
parts to each of these changes. DirectMusic allows you to define up
to four versions of each chord, allowing us to have multiple layers of
chord changes and map each instrument to a different layer. The bot-
tom chord layer would never change, which meant that all of our
modal background score and monster melodies would play the same
as they always had. The second and third chord layers, however,
would be for the player music to follow.
There are, however, distinctly different ways for musicians to fol-
low chord changes. If we consider a basic blues as our example, a
bassist playing through the changes will often move with root
motion. That is, the bassist plays a line on the I chord and then plays
that same line on the IV chord. Meanwhile, a soloist might play
through those same changes without root motion by creating melo-
dies that adapt to the current chord scale but do not move with the
changing root.

Figure 20-2: Here is an example of one of our

chords. Chord track 1 is set to CMin7, the first chord
in the mode of Dorian. This is the mode for the
background music. Chord tracks 2 and 3 are also in
the mode of C Dorian, but their chords are set to an
EbMaj7 chord.

Look at Chord track three. Notice an EbMaj7 as it would be played

above C. Chord track two, however, contains the inversion of the
chord, which is closest to our root chord of C minor — in this case,
the third inversion of the chord.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 485

What we define here is how a bass part and a melodic part move
through the chord progressions. We map bass parts to chord track
three and follow root motion up to the Eb. Melodic parts, however,
choose the closest acceptable notes to what they originally play,
much like a guitar player going through changes on a solo.
This technique opened up a wealth of emergent behavior in the
player music. The player music follows chord changes that are not
present in the background score, yet work perfectly with the back-
ground score behind them. A user busy chatting while they play no
longer sounds repetitive but instead actively follows chord changes.
As we take the liberty of hiding interesting changes in parts of the
world that may not even have a background score, there is an invisi-
ble landscape of musical changes waiting for the players to discover.

Groove Level
In our first pass, we went through each background score and set
various styles to respond to different groove levels. Our default
groove level is 30; nearby monsters add two to the groove level,
while players subtract one. As monsters outnumber the player, the
groove level rises, and the system introduces new parts into the mix
to intensify the music.
This only provided a scope of density and did not create the
change in color and mood we desired. For this, we added additional
code to change the primary Segment based on the current groove III
level. As the player goes up in groove levels, the chords chosen for
the music become darker. For instance, an area in a base mode of C
Part

Dorian will move to C Phrygian, and finally to a C whole-half tone

scale, providing a much darker sound. Like all of these Segments, we
wrote secondary chord changes for the player music to follow.
486 n Chapter 20

Technical Considerations
Project Organization
Project organization, while not an exciting topic, is important none-
theless. Throughout the creation of our project, I took the time to
reorganize and rename any new data into standard naming conven-
tions, while cleaning up any old data that did not fit our paradigm. In
the end, solid organizational techniques will save you time and make
finding problems later easier.
Segment Trigger Tracks become a very useful organizational
tool, as I created a hierarchy of inherited data. A background score in
my project looks something like this:
n Master Segment
• Tempo/time
• Chords or ChordMap
• Markers
• Default Groove Level
• Band Segment
o Band
• Arrangement Segment
o Drum Segment
m Style Track for drums
o Strings Segment
m Style Track for strings
o Choir Segment
m Style Track for choir
One of the main reasons that I organize data in this fashion is to
speed up the process of creating variations, while keeping every-
thing neat and consistent. My master Segment contains all the
high-level controls, such as chords and tempo, while secondary Seg-
ments within the master Segment call Band data or group the
arrangement of multiple styles together. Making new arrangements,
mapping arrangements to new chord structures, or changing the
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 487

Bands for an entire area are all very easy to do because I have signif-
icantly less data to track in each Segment. Also of note is that I
always call Bands by creating a Segment for them and using a Seg-
ment Trigger Track. This saves a ton of time and frustration, as you
only have to update your Band in a single place when you want to
change it, instead of having to find all the Segments associated with
that instrument and edit their Band settings.
As a general practice, we created all note data in Styles, encapsu-
lated in a Segment, and called from a Segment Trigger Track or as a
secondary Segment. I never place note data directly into Segments
because I often find myself in the position of eventually needing
some function of styles and having to convert many Segments into
styles.

Optimizing Patch Changes

Because of the sheer number of potential Segments playing in our
music system, it was important to pay attention to optimization tech-
niques. In fact, at some point in the project, suboptimal data
arrangements led to some strange behaviors. I found out the hard
way that setting the Band is potentially a very expensive operation,
which if not optimized can cause small pauses, sync errors, and other
undesirable behaviors. Every time the engine called for a Band
change, DirectMusic was not filtering out the redundant Band
changes, which lead to hiccups since too many Bands were being set III
at once. My original Band Segment was mistakenly set to loop every
bar, and because it called as part of a looping master Segment, it
Part

accrued one extra Band change for every time the master Segment
looped. This essentially means that DirectMusic would upload five
instruments the first time through, ten the next, 15, then 20, and
then 25. At around 50 patch changes, the pauses and sync problems
start to become very noticeable until the music begins losing time.
Additionally, because the player music Segments must initialize a
Band with the 3D AudioPath, the player music Segment calls another
Band for each new melody that a user plays. Note that even when
resetting the same Band (such as on a loop), DirectMusic still
repatches that Band setting.
My first pass was to optimize the background score. I broke my
master Band into a series of Band Segments that contained the mini-
mum amount of Band data that each score needed. I gave each player
488 n Chapter 20

or monster melody its own Band Segment as well. However, while

this greatly optimized the background score, my two-bar looping
player music Segments were causing a lot of individual patch
changes, as each Segment would trigger a Band change on loop, and
it was possible to have upwards of 60 people playing music in an
area. My solution was to make each secondary Segment 999 bars
long, with the Style looping inside of it. This meant that a player
music Segment only called its Band when the Segment played ini-
tially, or when 1000 bars of music had passed by. While it is still
possible that 60 people could trigger a new Segment at once, causing
60 patch changes to happen, it is highly unlikely, especially given
Internet latencies.

Optimizing Voices and DLS Banks

It is also very important to optimize your voice counts and DLS
banks. It’s very easy to have a bloated voice count if you’re not care-
ful, thus slowing the performance of your entire game. The most
common cause of high voice counts is DLS banks, which have very
long release times. You want to trim the release times to be as short
as possible, and if you use a long release time, make sure you do not
create short patterns that repeat before that release time has passed.
For instance, if your Segment lasts for about four seconds and you
set your release time on the DLS bank to ten seconds, you end up
with notes from the original loop playing two loops later.
You should also be aware of the memory that you use, as it
directly accounts for much of the cost of changing bands. I do not go
into general audio data optimization techniques here. However, there
are things that you can do to make better-sounding DLS banks using
less data.

Cheap Stereo Separation

When creating synthesizer or orchestral patches, it is often desirable
to have a stereo effect. Ideally, your original samples are in stereo,
but this obviously chews up more data. A quick-and-dirty way to
achieve the same effect is to create two layers with different settings
applied to the same wave and pan those layers left and right.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 489

Note Scattering
On many percussion instruments, such as a thumb piano, I use a
technique I call note scattering to give them a more realistic sound.
Thumb pianos have individual metal bars for each note, and each of
these tend to have unique overtones, rattles, and other “signature
noises” associated with them. Instead of using six samples over a
three-octave range, spreading each sample over half an octave, I use
six samples and randomly alternate them on notes as I go up the
scale. This greatly reduces the chances of someone picking out the
repeating signature noises as a run plays up the scale.

Evolving Patches
Many of today’s synthesizers have very long patches that evolve and
change as they play. Such a patch would produce a very large sample
file, often unsuitable for the memory restrictions of game develop-
ment. However, it is quite possible to get this same effect using
several smaller, looping samples using the various parameters avail-
able in DirectMusic. To make a patch evolve, place the first sample
(say, a violin section) on layer one and set attack and delay times as
you normally would. Place another sample, perhaps a choir of voices,
on the second layer, and set the sample to ramp in very slowly over
time. When you trigger a note, you get the attack of the violins
slowly ramping into the voices. You can even use this trick with a III
single sample and play with its filter and modulation settings instead
or use its pan settings to move the sample from left to right.
Part

Custom Code
While we did not have support for full scripting, we did do some cus-
tom code in our interface to DirectMusic. The primary thing we
added was the concept of exclusivity, which we used to prevent mon-
ster melodies from playing over themselves. This concept would be
very useful to add to the core functionality of DirectMusic, as it
would allow users to weed out redundant Band changes by making
the Band Segment exclusive.
490 n Chapter 20

DirectMusic Script
We did the rest of our custom code in DirectMusic script and trig-
gered through script calls embedded in Segments. We used this code
primarily to handle the transition and selection of the primary Seg-
ment. The script intercepts calls to play new primary Segments,
places them in a queue, and triggers the new primary Segment when
the old primary Segment is about to end. The script also selects
which version of the primary Segment is going to play based on the
current groove level. We use this to change chord mappings based on
the groove level; higher groove levels get a more dangerous-
sounding mode, while lower ones get a more serene-sounding mode.
The entire script would be much simpler if DirectMusic had an
AtLoop flag for transitioning Segments, but currently that choice is
not available. If you set the transition to happen at the end of the
Segment and the Segment is set to loop indefinitely, you never reach
the end of the Segment and your transition never happens. Thus, we
needed custom code to handle this. A simplified version of this code
is listed below with an explanation of what each routine does.
First, we need to initialize the variables used in the script.
Dim finish ‘ used for determining flags to send with play command
Dim nextPlaying ‘ number of Segments in the queue
Dim nextSeg ‘ object of Segment in the queue
Dim groovereport ‘ current groove level
Dim segA ‘ used for storing temporary Segment object
Dim segB ‘ used for storing temporary Segment object
Dim segC ‘ used for storing temporary Segment object
Dim seg1 ‘ Primary Segment object name for groove offset 0-4
Dim seg1a ‘ Primary Segment object name for groove offset 5-9
Dim seg1b ‘ Primary Segment object name for groove offset 10+

The Init routine initializes the Segment variables to the Segments in

the DirectMusic project. I do this to make things easier to edit, as
now the Segment name only needs to be stored in one place, and the
variable can be used instead of hard coding the Segment names
everywhere they are referenced.
Sub Init

set seg1 = MI_Hum_C_Dorian

set seg1a = MI_Hum_C_Phrygian
set seg1b = MI_Hum_C_WholeHalf
End Sub
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 491

For each area in the game, there is a queue Segment that calls the
appropriate queue routine. This routine calls Init to initialize the
variables and then checks to see if the nextPlaying variable is set. By
default, DirectMusic initializes all variables to zero, so if nextPlaying
is zero, we know the music has not started. Once the script sets the
nextPlaying and finish flags, it calls the routine ChooseHuman1.
Sub Que_01
Init
If nextPlaying = 0 Then
finish = 0
Else
finish = 1
End If
nextPlaying = 1
ChooseHuman1
End Sub

This routine sets my temporary variables (segA, segB, and segC) to

be the primary Segments that I wish to choose between based on
groove level. It then calls the ChooseGroove3 routine.
Sub ChooseHuman1

Set segA = seg1

Set segB = seg1a
Set segC = seg1b
ChooseGroove3

End Sub III

This routine checks the master groove level offset and sets the
Part

nextSeg object to be the correct primary Segment based on that off-

set. It finally calls the TriggerPlay routine to play the Segment.
Sub ChooseGroove3

groovereport = GetMasterGrooveLevel()
If groovereport < 5 Then
Set nextSeg = segA
End If
If groovereport > 4 Then
Set nextSeg = segB
End If
If groovereport > 9 Then
Set nextSeg = segC
492 n Chapter 20

End If
TriggerPlay

End Sub

This routine checks the finish variable and triggers the Segment to
play as either AtMeasure.sgt or AtFinish.sgt. Playing the Segment
AtFinish.sgt essentially throws out the play command. If the primary
Segment is currently playing and set to loop indefinitely, it will never
reach AtFinish.sgt. This is necessary in case the primary Segment
does not start or stops for some unexplained reason. If this happens,
the new primary Segment will start playing immediately because no
primary Segment is currently playing. If the primary Segment is
already playing, then nothing happens until its Script Track calls the
next routine.
Sub TriggerPlay

If finish = 0 Then
nextSeg.play(AtMeasure)
else
finish = 0
nextSeg.play(AtFinish)
End If

End Sub

Figure 20-3: Our master Segment with its script call in the Script Track.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 493

All primary Segments call the final Segment just before they end. A
script call is placed in a Script Track on the second beat of the last
bar in the Segment. Once called, it checks to see which Segment
number nextPlaying is set to and calls the selection routine for that
primary Segment. In this example, there is only one choice, so it
calls the ChooseHuman1 routine explained above. By placing this on
the second beat of the last bar in the Segment, the routine triggers
the next primary Segment to play on the next measure, and the pri-
mary Segment transition happens at the end of the primary
Segment.
Sub PlayNext

Init

If nextPlaying = 1 Then
ChooseHuman1
End If

TriggerPlay
End Sub

The result is that the primary Segments only transition at their ends
while still being set to loop. Additionally, the control code allows us
to select new primary Segments based on the groove level, which
we use to recompose the music in this example from Dorian to Phry-
gian to a whole half-tone scale based on the current groove level.
This gives us a more reactive score, where darker tonalities are III
associated with being outnumbered, and lighter tonalities are associ- Part
ated with outnumbering the creatures around you.

Postmortem
Although we plan to expand the system that we created here as the
game evolves, it is always a good idea to go back and look at the vari-
ous things that went right and wrong in a project.
494 n Chapter 20

What Worked
Player Music as a Purely Social Game
If someone asked me what I am most proud of working on in
Asheron’s Call 2, it would be the addition of music as a purely social
game. No one has ever created a dynamic music system quite like it,
and we have already witnessed people getting to know each other
because of the music system. These types of systems are what build
friendships long after the core gameplay has become stale and pro-
vide people with new outlets for their creativity. I can only hope that
the music played in-game inspires someone to pick up an instrument
in real life, as music is something that fundamentally makes people’s
lives better.

Figure 20-4: During the last four hours of the beta test, users
gathered on top of the Deru Tree to play music together. Part of
this jam session is included as an MP3 track on the CD.

Layered Approach
I cannot express enough the importance of starting with something
simple and expanding it. Revision and the willingness to throw
things away is part of what makes a project work well in the end. At
any given stage in the project, we had something interesting work-
ing. As we added new layers to the mix, things just kept getting
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 495

more interesting. Many of the solutions and concepts presented

above were not part of the initial project and were continually added
or refined through the entire process.

Music as Sound Design

While style is not something you have, but rather something you
cannot get rid of, the choices we made within our style have worked
out very well. The combination of polyrhythmic structures and
modal melodic structures not only allow our system to work in inter-
esting ways but also do not tire the ear as much as other stylistic
choices would have. We also paid particular attention to blending
these stylistic choices with our background environments, treating
them as much as sound design as music.

Music Is Informative
The monster melodies and use of groove levels in the game provide
a kind of musical information that I believe positively enhance the
user’s experience of the game. They layer another kind of musical
tension into the mix based on the user’s situation and give arrange-
ments of monsters a unique musical sound. One of our designers
even realized that the game engine was not creating monsters in the
right locations just from hearing the music.

DirectMusic III
DirectMusic was truly the only music system that could accomplish Part
this type of holistic musical vision, and while the system is far from
perfect, it does some things amazingly well. The ability to adapt a
section of music to multiple key signatures allowed us to get a range
of moods from a single piece of music, while the synchronization and
small data rates of the file format allowed us to work within the net-
work-based parameters that an online game such as ours requires.
496 n Chapter 20

What Didn’t Work

Not What People Expect from an “Interactive” Music
System
In some ways, we did not do what people expected from an “interac-
tive” music system. When people hear this term, they usually think
that the music is going to kick in when they go into combat, play a
victory melody when they win, and go back to its normal state again.
Instead, our system is ambient in nature and does not provide the
painfully obvious kick at the right moments. While what we did was
more advanced (and I am very glad we did it), sometimes the simple
things are what people want. I do believe these types of situations
can assimilate into the current system, but this type of easily quanti-
fiable change is often what tires the ear of the music in the first
place. The trick is to achieve a balance between immediately notice-
able effects and the more subtle effects that do not tire the ear. The
color and density changes controlled by the groove level provide
some of this functionality, and I suspect that we will work more in
that direction in the future.

Learning DirectMusic while Creating the Project

Not fully understanding the functionality of DirectMusic made the
initial creation process much more painful than it needed to be. How-
ever, I do not know of a better way to learn anything than to do it,
and many of our “mistakes” lead to interesting benefits in the end.

Lack of Support
Most sound designers fight with a lack of support in their endeavors.
In many companies, it is hard to get the proper resources for sound,
as the industry tends to focus heavily on graphics. In addition, many
people have no idea how we create sound and music, let alone how
they affect the experience. The original coder assigned to the music
system worked on the user interface and localization of the game.
Both the music and UI require two huge systems. In addition, she
did not know much about sound and music, so explaining things
often took an exorbitant amount of time. Moreover, once someone
found a bug, verifying bug fixes always required Geoff or me to be
present. Additionally, the code proved very unstable at times, so
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 497

much so that during the last few weeks of development, we had to

rewrite the entire system from scratch.
Dan Ogles, one of our graphics programmers, rose to this monu-
mental task and did a fantastic job with it. However, I cannot imagine
how much easier it would have been had the focus and support been
there in the first place.

Future Work
In the future, I am interested in prototyping the two main concepts
of the system separate from each other before integrating them
together again. If we created a system of pure player music, we could
implement a host of new controls for users, allowing them to control
things like tempo, chord and color choices, effect properties, and
more. We are also very interested in pursuing a system that allows
users to create their own melodies through a very simple interface,
which would give them a much greater degree of musical control,
while still protecting them from bad note choices. I would like to see
how far we could push this system by itself, as there is potentially a
lot more room for control within the mix.
I would also like to spend some time prototyping a purely film
score-type approach, where the speed at which the music adapts to
the current situation is stressed. I would also like to try integrating
traditional Redbook audio tracks into the mix to increase the sonic
quality while still maintaining the interactivity. III
Once we have gained experience at pushing the boundaries of
these systems separately, I would like to take that knowledge and
Part

see what parts of the two systems can be combined without limiting
either system. Can we create a truly adaptive score, one that adapts
to any situation, with the sound quality of Redbook audio, while still
maintaining an interesting player music game? How much more con-
trol can we give to the users before they strangle their ears with it?
I also think that we need to spend some time learning other
areas of DirectMusic and exploring what other features lay hidden
under the hood. For instance, we did not make use of ChordMaps for
variation in the project, and we barely touched on the things that
scripting can provide. We also did not use chord changes to write our
background score and instead used them for player music and mood
changes. How would the system differ if our background score used
the more traditional style of writing in DirectMusic? Could we solve
498 n Chapter 20

less with composition and instead use DirectMusic’s functionality

better in the end, providing a higher degree of recomposition?

On the CD
On the CD included with this book, you will find a folder with a
greatly simplified, but quite functional, version of my project. Due to
licensing restrictions, I had to rewrite the music and use new DLS
banks for the music in this version, but the fundamental principals
used to create this project are identical to the ones used to create
the original score and should serve as a great utility to understanding
the techniques presented in this chapter.
In the project, you will find three folders containing the back-
ground score elements, monster melody elements, and player music
elements of the project. If you load the Demo bookmark in the pro-
ject, the Secondary Segment window will contain the necessary
Segments to preview all of the music in the project as well.
At the bottom of the Secondary Segment window, you will find
two queue Segments that start the background music through script-
ing or transition from one background score to the next. Above them
are three utility Segments used to raise, lower, or reset the groove

Figure 20-5: Secondary Segments in the example

project. Use the first secondary Segment to start
the project, the second and forth to change the
groove level, and the rest to sample various
instrument and monster melodies.
 A DirectMusic Case Study for Asheron’s Call 2: The Fallen Kings n 499

level of the project. You may see the current groove offset value by
looking at the groovereport variable in the Script window. The
remaining Segments in the Secondary Segment window are monster
melodies and player music, which you should trigger once the pri-
mary Segment starts to simulate what might be happening in the
game.
1. Start the primary Segment by playing the Que_Hum_C_
Dorian.sgt Segment in the Secondary Segment window. The
music may take a few seconds to begin.
2. Play several of the monster melodies (mn_drudge1.sgt, for
instance) to simulate the presence of monsters in the area.
3. Play several of the player music Segments, and notice how
they follow chord changes not originally heard in the back-
ground score, yet they still work with the background score.
4. Raise the groove level by playing the GrooveUp.sgt utility
Segment several times until you hear the sound of the drums
come in. After a few seconds, the key should change, and the
player music should adjust accordingly.
5. Raise the groove level several more times. The drums will
become louder, and the key will change again to become
even darker.

III
Part
This page intentionally left blank.
Chapter 21

Beyond Games: Bringing

DirectMusic into the
Living Room

What people are going to be selling more of in the future is not pieces
of music but systems by which people can customize listening experi-
ences for themselves… In that sense, musicians would be offering
unfinished pieces of music — pieces of raw material, but highly
evolved raw material, that has a strong flavor to it already. I can
also feel something evolving on the cusp between “music,” “game,”
and “demonstration”…. Such an experience falls in a nice new
place — between art and science and playing. This is where I expect
artists to be working more and more in the future.
Brian Eno in WIRED issue 3.05 — http://www.wired.com/wired/
3.05/eno.html?pg=4&topic=

A lthough DirectMusic provides many solutions to the challenge

of crafting a cinematic underscore that responds in sync with
the unpredictable unfolding of computer game events (as is evident
in the excellent work of composers like Guy Whitmore, Nathan
Grigg, and others), the greatest untapped potential of DirectMusic
resides in the very nature of its nonlinear playback architecture and
its widespread install base as part of the DirectX APIs. Nonlinear
music is currently at the far frontier of music production technology.
As such, it requires composers and designers to develop new
compositional, recording, and playback techniques, even new ways of
thinking about music. In return, it offers the possibility of

501
502 n Chapter 21

discovering and delivering qualitatively new musical experiences to

the worldwide community of tech-enabled music listeners.
It’s beyond the scope of this article to exhaustively document
why and how a nonlinear music format represents an important
“next step” in the evolution of music production. I only hope to con-
vey the challenges and the rewards that the current “state of the
nonlinear art” offers to both composers and listeners. In order to
demonstrate what nonlinearity offers in terms of musical experi-
ences I first recount a little of the history of early recording
technology and how the introduction of linear playback brought with
it an often overlooked paradigm shift that still affects how music is
both perceived and conceived today. We look at DirectMusic as the
basis for stand-alone nonlinear music content. We also look at the
example Segment included on the companion CD: a full bandwidth,
nonlinear music file made in collaboration with ProjeKct X (the
“R&D division” of the seminal rock group King Crimson: Pat
Mastelotto, Trey Gunn, Adrian Belew, and Robert Fripp). From
there, I present my view of the challenges unique to nonlinear com-
position that a composer must take into account when working in
this still very new and uncharted territory and present and explain
some “conceptual keys” that I’ve found useful in my work.

From “Sound Capture” to “Music Production…”

In a March 2002 article for The NY Times Magazine, author Kevin
Kelly recounts an anecdote of Indonesian gamelan musicians react-
ing to early recording technology. He sums up the nearly immediate
effect recording technology had on the development of musical
forms:
There is no music made today that has not been shaped by the fact
of recording and duplication. In fact, the ability to copy music has
been deeply disruptive ever since the invention of the gramophone.
When John D. Smoot, an engineer for the European company
Odeon, carted primitive recording equipment to the Indonesian
archipelago in 1904 to record the gamelan orchestras, local musi-
cians were perplexed. Why copy a performance? The popular local
tunes that circulated in their villages had a half-life of a few weeks.
 Beyond Games: Bringing DirectMusic into the Living Room n 503

Why would anyone want to listen to a stale rendition of an obsolete

piece when it was so easy to get fresh music?
As phonographs spread throughout the world, they had a
surprising effect; folk tunes, which had always been malleable,
changing with each performer and in each performance, were trans-
formed by the advent of recording into fixed songs that could be
endlessly and exactly repeated. Music became shorter, more melodic,
and more precise.
Early equipment could make recordings that contained no more
than four and a half minutes, so musicians truncated old works to
fit and created new music abbreviated to adapt to the phonograph.
Because the first sound recordings were of unamplified music,
recording emphasized the loud sounds of singers and de-emphasized
quiet instrumentals. The musicologist Timothy Day notes that once
pianists began recording they tried, for the first time, to “distinguish
carefully between every quaver and semiquaver — eighth note and
sixteenth note — throughout the piece.” Musicians played the way
technology listened. When the legendary recordist Frederick
Gaisberg arrived in Calcutta in 1902, only two decades after the
phonograph was invented, he found that Indian musicians were
already learning to imitate recorded music and lamented that there
was “no traditional music left to record.”
Kevin Kelly in The New York Times Magazine —
http://www.nytimes.com/2002/03/17/magazine/17ONLINE.html III
For most of us, the very idea of music is indistinguishable from the Part
experience of listening to the radio, LPs, CDs, MTV, MP3s, and the
like; so much so that the notion of even questioning the 100 percent
equivalency between recordings of music and the performance of
music itself can be hard to grasp. In the early days of recording, it
was inherently obvious that technology served only as a means of
preserving live sonic events, whether musical, oratorical, or other-
wise. For the first time in history, fleeting and transitory air
vibrations were frozen in wax to be reproduced and distributed
according to demand. As time passed and an industry grew around
the technology, desire for increased fidelity drove technological
development. As technology progressed exponentially, the pendulum
swung from more and more accurate sound reproduction over to
increasingly sophisticated sound production. This has led to the
504 n Chapter 21

current situation of a complete reversal of recording technology’s

original role.
Today, many big-budget “live” musical acts painstakingly stage
and choreograph their performances to replicate as closely as possi-
ble the sound of the studio recording. This studio recording itself
may consist primarily of only careful simulations and samples of some
long-ago live performance by a previous generation of performers.
We are at the point now where live concerts are often little more
than a lip-synced ballet set to the commercial release of a hit single
playing through a PA system, only barely supplemented by the sound
of the live musician’s actual in-the-moment stage performance. In
such cases, it is safe to say that sound production technology has
caused recording to replace live performance as the aim of music
making, even in a so-called live performance.
This is not to remotely imply that recordings are necessarily
inferior to live music or to devalue the education, enjoyment, and
entertainment that recorded music provides. In fact, the whole point
of this chapter is that technology can now allow even more musical
entertainment and enjoyment. The role of modern recording tech-
nology in serving the creative musical process is indisputable. It has
empowered some of the greatest musical artists of our time, as is
evident in the creation of iconic, never-performed-live masterworks,
like Sgt. Pepper’s Lonely Hearts Club Band and Bitches Brew, and it’s
worth acknowledging here that the recording studio with all its tools
may legitimately be viewed as a complicated sort of musical instru-
ment, which can be played skillfully by someone who has mastered
its various techniques.
The point is simply that recorded music is so ubiquitous and so
much a part of our cultural experience of music that we often forget
that before the advent of recording, the only way to hear music was
to participate in some sort of musical performance, either as a musi-
cian, an audience member, or, as in many cultures where the dividing
line between performer and audience is not so clear cut, a participant
in a socio-musical event like a religious service, the gamelan, African
call and response, and many, many more such traditions. Before
recording, all music was nonlinear, in that every performance was
free to respond to the unique necessities of the individual time,
place, and people involved.
 Beyond Games: Bringing DirectMusic into the Living Room n 505

While there is an incredible amount of technological genius and

artistry evident in the current tools for producing and recording
music, including nonlinear editing, all the results of this technology
are still presented to the end user through the inherent limitations of
linear playback that were dictated by early recording methods and
materials. All linear formats are by nature a closed system, in that
once a record is “in the can,” the creative process is done. The lis-
tener’s appreciation and familiarity may grow over time, but nothing
in the recording itself and the way it plays back is ever going to
change. Of course, this will always be a great and useful thing for
capturing the “perfect performance,” but linearity in music playback
no longer needs to be as irrevocable as a thermodynamic law; it can
instead be optional. It may always be the norm, but it needn’t be the
rule.
For the first time, the technology is widely available to overcome
the determining condition of linear playback and bring us back full
circle to the paraphrased question raised by the perplexed Indone-
sian musicians in the article quoted above: “Why would anyone want
to listen to a stale rendition of an obsolete piece when it’s so easy to
get fresh music?” As one answer to that question, I propose that
composers and music producers take advantage of existing alterna-
tives to the standard linear playback paradigm. DirectMusic is such
an alternative and already has a large install base. It provides a for-
mat based on nonlinear music architecture from its earliest
conception and, critically, it allows delivery of discrete nonlinear III
music files that can play on most any modern multimedia PC or any Part
other platform that supports the DirectX APIs, like the Xbox.

“…to Nonlinear Music Format”

This exploration of DirectMusic as an architectural basis for a con-
sumer-friendly nonlinear music format is based on more than just
bright ideas and speculation. Over the past few years, Guy Whitmore
and I composed and designed many examples of just such a format
using DirectMusic. These compositions proved concepts and
explored design methods blending various musical sources for demo
and instruction purposes. We used familiar artists’ material to dem-
onstrate the commercial possibilities of nonlinear playback to music
506 n Chapter 21

industry A&R people interested in new technology but understand-

ably new to the concept of nonlinear music. Some examples: I used
vocal phrase samples of Bjork in an original nonlinear composition to
show how a club DJ remix artist might approach nonlinear composi-
tion. Guy arranged different and variable instrumental backing tracks
with appropriate production value for a commercially released a
cappella version of a Sarah McLachlan song to demonstrate a more
traditional pop song approach within a nonlinear framework. He also
recreated tracks from Moby’s Play using DLS instruments and the
same public domain vocal samples he used from the Lomax/Smithso-
nian collection.
We’ve also been fortunate to work closely with bands keen to
explore the potential of nonlinear formats. One of them, ProjeKct X
(Pat Mastelotto, Trey Gunn, Adrian Belew, and Robert Fripp) pro-
vided multitracks of an improvised session to generate musical
material specifically for use in a nonlinear playback format. Not only
that, but they have also generously granted permission for me to
provide a small portion of that material here for demonstration and
educational purposes.
All of the aforementioned demos consisted of discrete, individual
files and a small accompanying player application designed to run on
most any up-to-date multimedia-capable Windows PC. Though usu-
ally larger than the “average” MP3 file, a nonlinear Segment file is
by definition not tied to the standard run-time/footprint ratio of a lin-
ear file. 10MB equals roughly ten minutes of run time for MP3,
whereas a 10MB DirectMusic Segment may be designed to play any-
where for a minute or two up to an hour or even indefinitely until
stopped by the user, if that’s what the designer wishes.
If you haven’t yet listened to the ProjeKct X Segment that
accompanies this book, copy the ProjeKct X directory on the com-
panion CD and copy it in its entirety to your C:\ drive. Open the
directory that contains both the ProjeKct.exe and ProjeKctX.sgt
files, and run ProjeKct.exe (they must be in the same directory for
the player to run). The player will automatically load the ProjeKct X
Segment on startup (allow some time for the files to decompress,
depending on your CPU speed). There is also a WMA file provided
as reference of the live linear capture of the piece from the studio.
It’s worth noting here that the material presented is only a fragment
 Beyond Games: Bringing DirectMusic into the Living Room n 507

of what was a more than a ten-minute linear piece with three distinct
movements.
As playing ProjeKctX.sgt demonstrates, the sound of Direct-
Music can be as full and clear as Redbook audio or better if the
composer wishes to use higher bandwidth source material. From the
user’s point of view a stand-alone nonlinear Segment appears to be
just another sound file that, with a registered player app installed,
only needs to be double-clicked to start. On first listen, it sounds like
most any other music file. However, on a second, third, fourth, and
even hundredth listen, the real difference between the DirectMusic
Segment and a linear file becomes more and more apparent — not in
sound quality of course, but in the actual experience of listening. In
fact, one could say that there never really is more than “one listen”
of a robust nonlinear music file because it can render a unique rendi-
tion with every play if the composer so desires. This is the essential
difference between linear and nonlinear playback: A robust nonlinear
music file continually rewards the engaged and active listener with
every listen.
Some people on hearing or reading about nonlinear music for the
first time form unrealistic assumptions and expect behavior along
the lines of “Mary Had a Little Lamb” somehow magically playing
itself as “Twinkle, Twinkle, Little Star.” A little listening experience
easily dispels such notions as most unlikely (not to mention undesir-
able). What generally happens with nonlinear playback is more like
what happens when listening to different performances of the same III
piece of music. The nonlinear performance can even sound like Part
different groups of musicians playing the same piece with genre-
appropriate stylistic variations if the composer provides enough
material. Even if the composer designs a huge piece with wildly
varying elements, the identity of any real musical composition stays
recognizable in the same way that we can recognize Coltrane’s ren-
dition of “My Favorite Things” as the “same tune” that Julie
Andrews sings in The Sound of Music. Simply put, a composition that
has no identifiable elements that can translate across different set-
tings and renditions is likely not a composition in the traditional
sense. Conversely, if a composition is recognized by an intelligent
listener regardless of arrangement or rendition, then the identity of
the piece remains intact as some sort of metadata, whether the for-
mat is linear or nonlinear. For instance, it will not take more than one
508 n Chapter 21

or two listenings of the ProjeKct X file or its original linear perfor-

mance for the listener to recognize it within a short moment. This is
true even though it was a group improvisation to start with and even
though the melodic lines, words, and textures play back in a different
arrangement every time in the DirectMusic version.

Nonlinear Music Design

What follows is a necessarily cursory, high-level look at the frame-
work behind some of the approaches that I’ve used in recent years
for designing stand-alone nonlinear music files using DirectMusic.
The ideas presented here are subjective and by no means exhaus-
tive. I hope they serve to spark other ideas or at least give
interested composers and designers a starting point and/or general
direction from which to proceed. Unfortunately, there is no way to
create a template or formulate a simple list of instructions for mak-
ing nonlinear music files. If there were, it would hardly be a creative
endeavor, and designing compelling nonlinear compositions requires
creativity or, at the very least, ingenuity to be done well.
One factor that makes a cookie-cutter approach impossible is
that the nature of the musical material used often dictates the most
desirable, or even feasible, design approach; even similar-sounding
pieces from the same genre can sometimes require different
approaches. Additionally, the nature of the DirectMusic architecture
is such that there are often several very different ways to accomplish
the same sounding outcome, which can be somewhat overwhelming
at first. Each approach may be more or less appropriate, depending
on the final application you design for and the strengths and weak-
nesses of your current skill set. Those with advanced MIDI chops
who are comfortable building DLS sample sets may like the flexibil-
ity that comes with that approach if the material allows. (The
ProjeKct X file was constructed with that approach, for instance.)
More traditional audio engineers may gravitate toward DirectMusic
Wave Tracks as their primary tool due to some similarity there with
nonlinear audio editing tools. Ultimately, each piece needs an
approach unique to it and to its designer. However, I’ve found it use-
ful to keep three high-level concepts in mind when approaching a
new DirectMusic project. They may seem obvious and simple to
 Beyond Games: Bringing DirectMusic into the Living Room n 509

grasp at first, but I’ve found that they become more useful the more
experienced I become.
First, in brief:
Your role in the project is the first key factor to consider. How
much freedom do you have to alter the material? If you are the sole,
original composer, the answer is perhaps easy. If you are working as
remixer, obviously a different approach is required. One role that is
unique to nonlinear music production is something like that of a
translator — bringing linear material written by another composer
(or yourself) into the nonlinear world while conveying the spirit of
the original material, which of course brings its own set of require-
ments and restrictions.
Next, what functionality do you need to design for, as in what
functionality does your player app expose to the end user? Will your
material play in a “start and stop” player like the ProjeKct X player?
Or are you required to expose more parameters, such as the ability
to change the mix by loading different AudioPaths or different Band
files to mute certain tracks, or allow control of purely musical param-
eters such as changing the tempo or harmonic character of the
music. Material based heavily on large linear audio tracks simply
won’t allow purely musical parameter changes, since these parame-
ters are, in effect, “baked in.” This leads naturally to the next
concept.
The most critical concept is what I call the granularity of the
material that you are working with. It is also the hardest concept to III
convey in the absence of nonlinear composition experience, since it Part
is unique to the field. Is the piece a Bach fugue for a keyboard instru-
ment, a jazz standard with vocals, an ultra-polished mega-hit pop
song, or a musically ambient sound bed for an art installment? Each
of these will have a different granularity, which will dictate your
approach and how much DirectMusic functionality you can use or
expose.
Now we can look at these concepts a little more in depth.

Role
Your role in the process is hopefully self-evident but still worth artic-
ulating to identify the boundaries within which you must work. Are
you the sole composer of the piece, working entirely from scratch
with the freedom to follow the piece wherever it leads? Are you
510 n Chapter 21

working with music from the classical canon in the public domain,
with a jazz standard, or with a traditional score composer and taking
his or her linear ideas and converting them to DirectMusic files (as
often happens in game music)? Or are you working with someone
else’s finished audio multitracks, using as much or as little as you
like in combination with your own material the way a remix artist
might? Or are you limiting yourself to recreating a variable playback
version of the original piece (“translating” from linear to nonlinear)?
The answers to these questions will help define your role.
Since the roles are not discrete stations, separate and isolated
from each other, we can arrange the various options on an axis or
continuum. For instance, remixers nearly always add their own
material to a remix, often using only a vocal track and nothing else
from the original. If working as translator, one might choose between
adhering to the exact form of the material, allowing variations within
the different parts, or one might make editorial decisions and
attempt to capture the spirit of the original while taking much liberty
with the original form. On a number line from 1 to 12, the continuum
might look something like this:

1 2 3 4 5 6 7 8 9 10 11 12
Composer----------------------------------------Remixer----------------------------------------Translator

My role in the creation of the ProjeKct X file would be somewhere

around 10, closer to translator than remixer. I didn’t have any new
material but still made some critical choices that affected the listen-
ing experience. My aim was to create a composition that did not just
sound like that particular piece of music playing but that sounded
more like that particular piece of music being played live, and to
accomplish this I made certain editorial/translation choices. For
instance, in the drum tracks I intentionally have them introduce the
piece the same way with every play, without variation. I also place
the tacit drum breaks on a similar timeline as the original live perfor-
mance, as well as have them progress in intensity the same way they
do in the original. In choosing the drum track to function as a sort of
formal template for the Segment, which provides cues and signposts
to the attentive listener, I gained more freedom to add variability to
the other instruments. I wasn’t forced to do it this way by technical
or musical constraints; these were merely editorial decisions based
on my reading of what would work best for the material in this
 Beyond Games: Bringing DirectMusic into the Living Room n 511

format. I also changed the mix quite a bit from the original and
brought in musical material from markedly different sections of the
same improvisational suite so that this small Segment of material
could stand a little better on its own as a demo. It is just one arrange-
ment of several that I did with the material, the most formally literal
with regard to the original. Had my role been further to the left of
the axis above, I might have added quite a bit of my own material to
act as setting for the ProjeKct X material or even just used samples
here and there in my own composition.

Functionality
Of course, game audio is currently the most common application to
compose nonlinear music for, but as the title of this article indicates,
the application for consideration here is a stand-alone player applica-
tion. Currently, the few stand-alone player apps available function
primarily as simple start and stop devices, but the APIs are there to
add specific DirectMusic functionality to allow a custom player to
expose values for different parameters to user input. The following
list of parameters is in the order of increasing granularity, as well as
increasing restriction on the type of source material that is compati-
ble with such functionality:
n Standard playlist functionality to play multiple Segment files in a
desired order
n Choice of different AudioPaths and different Band files within a
III
Segment to allow for different mixes Part

n Allow user to mute or unmute certain sub-tracks

n Allow user to choose approximate length of run time for each
Segment
n Allow user to choose level of “musical activity” or density
n Allow user to alter tempo
n Allow user to alter harmonic and melodic characteristics and/or
mode

1 2 3 4 5 6 7 8 9 10 11 12
Simple play/stop-----------------------Game engine-----------------------User-alterable parameters
512 n Chapter 21

The ProjeKct X player is simple in functionality, so the Segment I

provided here has a corresponding architecture; it sits squarely at 1
on the axis above. Another player I’ve worked with that was specifi-
cally designed to showcase this material allowed the user to choose
between various mixes and signal processing via alternate Audio-
Paths and Band files, as well as choose differing levels of musical
density and intensity via controlling Segments. Such a player would
have required me to ship corresponding assets and also led to differ-
ent design choices in the Segment’s construction. It’s important to
note that, due to the nature of the ProjeKct X source material, it
wouldn’t be possible to allow the user to alter the tempo or har-
monic/melodic content. This is due to the granularity of the material,
as explained below. In the middle of the axis is the game music
engine: It requires much more complex design and assets than a
simple Segment player, but it’s still a more “controlled environment”
than a player with musical parameter variables exposed to the user.
There is likely no direct user access to musical and sonic parameters
in a game. The playback parameters are more predictable by the
composer, therefore easier to plan for and work around as required.

Granularity
Anyone who has worked much with DirectMusic probably already
has an intuitive understanding of what I mean by “granularity.” It is
useful to think of it in the literal sense; sand is quite granular and
conforms naturally to the shape of most any vessel. Gravel is less
granular but still conforms to a mold, within rough limits. Bricks or
blocks are much less malleable, but in return maintain their own
shape and stability, which means we can use them to build a stable
vessel or container into which we can pour gravel and sand if we
wish. The analogy is simple: MIDI tracks in combination with DLS
instruments are like sand, phrase samples are like gravel, and Wave
Tracks are like bricks.

1 2 3 4 5 6 7 8 9 10 11 12
MIDI+DLS---------------------------------Phrase Samples--------------------------------Wave Tracks

The ProjeKct X file belongs at around 4 or 5 on this continuum;

phrase samples are built into DLS instruments. The left side allows
much more malleability with regard to musical parameters, such as
 Beyond Games: Bringing DirectMusic into the Living Room n 513

tempo, key, harmonic sequence, and phrasing. The right side is less
concerned with musical parameters and more concerned with audio
parameters. It is less musically malleable (in terms of key, tempo,
etc.) but as such retains much more of the critical performance char-
acteristics so crucial to popular music (i.e., the singer’s voice or an
instrumentalist’s tone). As such, its stability (or invariability) will
likely also determine the overall form of the Segment.
It is critical that a composer become familiar with how the basic
architectural units in DirectMusic relate to one another in order to
understand the concept of granularity. Roughly, from smallest to larg-
est, these units are notes, variations, patterns, Styles, and Segments.
These units tend to fit one inside the other, sort of like a series of
nested Russian boxes. Once you grasp the internal relationships of
the DirectMusic elements, the most critical step in designing a non-
linear Segment becomes somewhat easier — defining the smallest
possible unit you can reduce the source material to without losing
the essential quality you wish to convey, almost like discovering a
“musical molecule.” This analogy, of course, cannot be taken too
literally since your definition of the granularity will likely be deter-
mined somewhat by your role and not by some objective musical
quality.
For example, if you “translate” a hit song by a well-known artist
and you want your Segment to maintain a traditional song form, the
critical element is almost certainly the singer’s vocal performance
and the lyrics. The granularity in such a case is possibly as large as III
an entire verse and certainly no smaller than a full vocal/lyrical Part
phrase. This would tend to dictate a design based around Direct-
Music Wave Tracks, and the variations would likely be more in the
post-production elements, such as altering the mix via track levels
and differing DSP treatments, using alternate instrumental tracks, or
altering the song’s form. If you’re working closely with the artist,
you might be able to include alternate vocal takes. If, however, you
design for a dance remix of the same song and are not concerned
with maintaining pop song form, you can make the vocal line as gran-
ular as you want to take it. In such a case, cutting the vocal line into
phrase samples and making a DLS instrument with them might be
the best option, depending on the rest of your material.
In the case of instrumental music, the level of granularity may
also be as large as a verse or a chorus, especially if the critical
514 n Chapter 21

element you wish to keep is the player’s individual tone. If the

instrument is especially MIDI friendly though, high-quality DLS
instruments may be an option, in which case the level of granularity
may drop all the way down to the individual note level, again depend-
ing on your role.
For example, one could theoretically ship a well-captured key-
board performance in the form of MIDI-based DirectMusic Seg-
ments along with a Gigasampler quality DLS instrument. This would
be the digital equivalent of shipping a player piano, the piano rolls,
and the player. Rather than play the linear MIDI sequence the same
way every time, the Segments allow for subtle musical variation in
tempo, velocity, and duration, even within specifications derived
from analyzing the way a particular player varies its performances.
Imagine something like Keith Jarrett’s Sun concerts in a format that
sounded like he was playing your favorite Gigapiano in your listening
room and had the option for subtle or not-so-subtle “Jarrett-like”
variations that would differ with every playback. (“Hmm, shall I have
Mr. Jarrett perform on my Bosendorfer or the Steinway today?”) In
such a case, the DirectMusic engineer (i.e., the person who con-
verted the MIDI sequence to DirectMusic Segments and edited and
added the musically appropriate variation parameters) would be
working around 12 on the role axis above, but the material would be
near 1 on granularity. The most compatible player would be near 12
on the player functionality axis above.

Conclusion
In conclusion, I want to make clear that I view DirectMusic as the
basis for a nonlinear musical format, not as a finished and definitive
format. DirectMusic has the advantage of being part of the DirectX
APIs, which means there are many potential DirectMusic players (in
hardware terms) already in listeners’ homes right now, today. With
the resources in this book, anyone with enough interest could make
their own DirectMusic Segments and a simple player, make them
available via download or file sharing, and have millions of potential
listeners. Its disadvantage is that the authoring tools have never
benefited from the rigorous design/dev/user-test feedback cycle
required of commercial software in the marketplace. They have
 Beyond Games: Bringing DirectMusic into the Living Room n 515

always shipped free as part of the DirectX SDK, and it shows. How-
ever, dedicated users have proven that they can master the
challenges of the tools and turn the incredible complexity of the
architecture to great musical advantage in award-winning games.
Nonlinear music can become the next step in music production tech-
nology if music lovers and the listening public have access to quality
examples of nonlinear music, not only in games but also on their
home multimedia stations. If that happens and the demand for
“more” occurs, better tools and improvements in the architecture
will not be far behind. Then — who knows? — by the turn of the next
century, our current linear formats may seem as quaint and outdated
as one of Edison’s wax cylinders does now.

III
Part
This page intentionally left blank.
About the Authors

Jason Booth is currently employed as Creative

Director at Turbine Entertainment, and received his
formal training at the Berklee College of Music.
While his writings in this book are on music systems,
he’s equally comfortable working on game design,
tools design, or creating game art in Maya. When not
at work, he can usually be found at various concerts
and festivals. Send questions or comments to
[email protected].

Tobin Buttram is Audio Director — FASA/Microsoft

Game Studios. Tobin’s background in musical composi-
tion and performance includes a BA in music as well as
recording and touring extensively throughout North
America and Europe with Robert Fripp & The League
of Crafty Guitarists. He first started working with
interactive music applications as the lead content
designer of MSN’s Interactive Music Architecture
team in 1996. Working closely with the DirectMusic developers,
many of Tobin’s content design innovations for MSN’s early use of
Interactive Music Architecture were the basis for several design fea-
tures in DirectMusic.
Discography highlights include (as composer/audio director)
MechAssault (Xbox), Combat Flight Sim 3 (PC), and The Operative:
No One Lives Forever (PC) and (as composer/performer Robert
Fripp & The League of Crafty Guitarists): Intergalactic Boogie
Express — Live in Europe 1991 (DGM9502) and Show of Hands
(Editions EG).

517
About the Authors

Todd M. Fay (aka LAX) is a gun for hire in the

interactive entertainment, television, and music
industries, providing production, content devel-
opment, and business services to entertainment
and education professionals and companies. LAX
is an accomplished consultant, agent, composer,
producer, and journalist, having seen tours of
duty working with Creative Labs ATC, Blizzard
Entertainment, THQ, Cognitoy, G4 Media,
Tommy Tallarico Studios, and various private
parties. His work has appeared in Game Devel-
oper magazine, Gamasutra.com, and Music4Games.net, and on G4:
Television for Gamers, the 24-hour network dedicated to games and
the gaming lifestyle.
Todd is an interactive audio fanatic having grown up on games
and game music. He is the Director of Development for the Game
Audio Network Guild (www.audiogang.com), a professional organiza-
tion promoting excellence in interactive audio. Todd holds a bache-
lor’s degree in sound recording technology from the University of
Massachusetts Lowell. Feel free to contact him at
[email protected].

With more than 20 years of software development

experience, Todor J. Fay holds 22 audio technol-
ogy patents and has created more than two dozen
successful software products. A graduate of Cali-
fornia Institute of the Arts and Stanford Univer-
sity, with a master’s in computer science, Fay has
always strived to bring innovative technological
solutions to art. A member of G.A.N.G., he is a
respected expert on audio technologies and recently served on the
MIDI Manufacturers Association’s technology board.
Among his achievements is the co-authoring of the DLS specifi-
cation and creation of AudioActive, an interactive audio engine that
won Best of COMDEX for Multimedia in 1995 for his then-company,
The Blue Ribbon SoundWorks Ltd. Fay co-founded Blue Ribbon in
1988 and sold it to Microsoft Corporation in 1995.

518
 About the Authors

While at Microsoft Corporation, Fay led a team of developers to

create IMA (Interactive Music Architecture) and then DirectMusic.
He ultimately managed all of the DirectX Audio APIs before moving
to the Xbox division to work on new technologies. In 2002, Fay left
Microsoft to co-found NewBlue, Inc., an interactive technologies
developer and consultancy.

Bjorn Lynne is a composer and sound

designer working for PC games, video
games, TV, theater, video, and other media,
as well as publishing his own solo albums.
He started out as a game music composer
on the Amiga computer in 1990, and went
full time in 1995. Since then, he has been
the composer and sound designer for more
than 30 games, including all games in the Worms series, as well as
Seven Kingdoms, Seven Kingdoms 2, Stunt Car Racer, Phoenix —
Deep Space Resurrection, Addiction Pinball, Nightlong, Hotel Giant,
Siege of Avalon, Capitalism II, and many others. Bjorn recently gave
a speech about DirectMusic as part of a seminar for game developers
in Helsinki, Finland. He is currently composing for a sci-fi TV series
as well as his next DirectMusic project for an upcoming PC game.
He runs a fan newsletter and posts the latest news and projects on
his web site at http://www.lynnemusic.com.

Scott Morgan has a degree in music theory from the

University of North Texas (UNT), formerly known as
North Texas State University (NTSU). After college
he got the job as music editor of the cartoon “Dragon
Ball Z,” where he also arranged and performed music
for the show. After more than 100 episodes of
“Dragon Ball Z” he spent three months learning
DirectMusic and developing an interactive demo. His
demo landed him a contract at Microsoft testing DirectMusic. Now
he is going back and forth between composing music for games,
doing DirectMusic consulting and training for companies developing
interactive products, and doing audio test work for Microsoft.

519
About the Authors

Marty O’Donnell received his master’s degree

in music composition, with honors, from USC in
the early ’80s and since then he has written and
produced numerous original scores for TV, radio,
and film. His commercial work includes
Flintstones Vitamin’s “10 Million Strong and
Growing,” Mr. Clean, Principal Financial,
McDonald’s, Kellogg’s, and many others. In
1997 his company, TotalAudio, produced sound
design, Foley, and final mixes for Cyan’s Riven,
the sequel to Myst, and all the original music,
voices, and sound design for Bungie’s award-winning Myth: The
Fallen Lords. TotalAudio went on to produce the audio for Valkyrie
Studios’ Septerra Core: Legacy of the Creator and Bungie’s Myth II:
Soulblighter. In May of 2000, ten days after Marty accepted a
full-time position with Bungie Software focusing entirely on the
audio production for Oni and Halo, Microsoft purchased Bungie and
moved them to Redmond to develop games for the Xbox. The audio
for Halo received numerous industry awards including The Game
Developers Choice Award, The Edge Award, and Best Soundtrack of
2002 from Rolling Stone magazine. Although still co-owner of
TotalAudio with Mike Salvatori, who stayed in the Chicago studio,
Marty is currently Audio Director for Bungie Studios at Microsoft
and is working on Halo 2. He has been married for 25 years to
Marcie and they have two daughters, Alison and Christine.

Scott Selfon is the audio content consultant in the

Xbox Advanced Technology Group, providing educa-
tion and support for Xbox composers, sound
designers, and audio programmers. Previous roles
include acting as the test lead for DirectMusic Pro-
ducer and DirectX Audio Scripting and being
Program Manager focusing on the sonification of
MSN web sites using interactive soundtracks.
Scott has composed music for a wide range of media, including
film, television, games, and live performance for various ensembles.
He is the principal violist of several Seattle-area orchestras and
chamber groups, and is the artistic director of Hachi Pele, an a

520
 About the Authors

cappella group based at the University of Washington. Scott is an

alumnus of the University of Southern California, where he obtained
dual degrees in music composition (film scoring emphasis) and com-
puter engineering/computer science.

Ciaran Walsh began his musical career in his teens, writing songs
and playing bass in a number of bands and performing in and around
London, England. Fellow musicians introduced Ciaran to the world of
MIDI, sequencing, and sampling, and he soon bought his first com-
puter, an Amiga 500. Within two years Ciaran signed his first record
deal as a solo artist and between 1994 and 2000 he produced three
albums, ten singles, and a number of remixes and co-productions.
Under the name Deviant Electronics, he developed a reputation for
hard-hitting live shows that took him all over the world.
A long-standing interest in video games, and a belief that more
could be done with music to heighten the experience, led Ciaran to
begin working with Wide Games, an independent UK console games
developer. Together they set up Wide Sounds to specialize in interac-
tive audio for games and other media. Ciaran designed and scripted
the interactive music system for Prisoner of War, and continues to
work with DirectMusic on new titles. He has also created music and
interactive audio for television commercials, multimedia content, and
corporate branding for clients including Mercedes, Adidas, and Pepsi.

Jay Weinland is an eight-year industry veteran

who began his sound designing career with a small
Seattle area company named Manley and Associates
working on games for the Mac, PC, Sega Genesis,
and Sega Saturn. After Manley was acquired by
Electronic Arts, Jay worked on the FIFA series on
the PSX and Need For Speed on the PC and PSX.
Jay hooked up with Bungie after the Microsoft
acquisition and worked on Oni (PC, Mac, and PS2) and more recently
Halo (Xbox). After graduating from Oberlin College in 1991 (BM in
oboe, MMT in music education), Jay moved to Seattle with his wife,
Heather, where they still reside along with their two children,
Duncan (4) and Tessa (2).

521
About the Authors

Guy Whitmore is the owner and Creative

Director of Music Design Network, LLC. He has
composed over 25 game scores since 1994, as
well as writing for theater, film, and commercial
media. Recent titles include Tron 2.0, No One
Lives Forever 2, and Die Hard: Nakatomi Plaza.
He has garnered many accolades, including a
1996 nomination for Best Soundtrack from the
Academy of Interactive Arts & Sciences. He has
spoken at GDC, DigiPen, Cornish College, Sputnik (IGDA-Seattle),
and IMX (Interactive Music Expo). Guy serves on the advisory board
for G.A.N.G. (Game Audio Network Guild), and on Microsoft’s audio
advisory board. He heads the Adaptive Audio Now work group of the
IASIG (Interactive Audio Special Interest Group) and also
co-founded the SCA (Seattle Composers Alliance). Computer Games
magazine called his score for No One Lives Forever “the best inter-
active music seen in an action game!”

522
Index

3D, 145 articulation, 30

available Buffers, 297 in DLS instruments, 30
Buffer, 228 Asheron’s Call 2, 473
Channel, 285 avatar, 473
Cone, 290 project goals, 475
in Halo, 423 project overview, 477
Max Distance, 290 technical considerations, 486
Min Distance, 290 audio
Mode, 290 budget, 4
move position, 317 for the web, 437
Position, 290 on the Xbox, 419
resource management, 296, 307 producer, 3
spatialization, 285 AudioPath, 15, 137, 139, 223, 224
Velocity, 290 3D, 231, 236, 286, 302
5.1 surround sound, 419 3D position, 324
in Halo, 426 Activate, 238, 289
Buffer, 227, 247
A Configuration, 229
ActiveX control, 435 creating in script, 455
Activity, setting, 279 Deactivate, 289
adaptive audio, 3, 6, 369 DirectSound Phase, 227
additive-subtractive approach, 375 distance calculation, 321
design, 372, 376, 390, 439 DMO, 227
design document, 369 Download, 232
differences from interactive audio, 6 embedded, 212, 234
in Asheron’s Call, 475 in Russian Squares, 381
in NOLF, 338 in Wide Sounds, 451
in Wide Sounds, 439 loading, 229
macro-adaptability, 374 max 3D, 324
micro-adaptability, 374 max limit, 323
Additive Curve IDs, 16 mono, 231, 287
aesthetics in web audio design, 439 music, 231
Aho, Erik, 371, 387 Performance Phase, 225
AI in Halo, 429 Phases, 225
Aligned Cut In, 209 PlaySegmentEx(), 233
ambience, 4, 300 predefined, 231
anchor tags, 461 reassigning, 319
animation sounds, 426 Stage, 236
animation synchronization, 339 stereo, 231, 287
art and science, 501 stopping, 288

523
524 n Index

swapping, 310, 321 GetGroove(), 218

Tool, 246 GetLowestPriorityPath(), 312
Unload, 232 GetNextNotification(), 363
Volume, 237 GetObjectInfo(), 165
AudioPathItemInfo, 250 GetTempo(), 219
AudioPath Jones, 239 Init(), 164
AudioVBScript, 112, 271 LoadScript(), 274, 324
auditioning DLS instruments, 29 LoadSegment(), 197
avatar, 473 PlaySegment(), 259
Release3DPath(), 312
B ReleaseAll(), 168
Band, 34, 398 ReleaseItem(), 167
core, 400 ScanDirectory(), 168
Editor, 34 Set3DPoolSize(), 310, 324
in Russian Squares, 380 SetObjectType(), 165
setup in NOLF, 399 SetVolume(), 306
Transpose control, 34 StopSegment (), 216
Band Track, 22, 62, 178, 181 StopSegState(), 215
bandwidth on the web, 433 CAudioPath, 241, 307
Begonia, 348 Get3DParams(), 256
Belew, Adrian, 502 GetBufferParams(), 254
Bjork, 506 GetDMOCount(), 248
Bouwman, Bryan, 393 GetObjectInfo(), 249, 253
branching playback, 66 GetPChannelCount(), 245
Break Embellishment, 59 Set3DParams(), 256
Buffer, 228 Set3DPosition(), 308
3D, 228, 286 SetBufferParams(), 254
DirectSound, 287 SetVolume(), 259
Mix-in, 228, 247 Channel Group, 180
mono, 287 chord, 79
Sink-in, 248 in Asheron’s Call, 483
stereo, 287 inversion, 102
Synth-in, 228 level, 71, 81, 102
buffers and streaming, 19 progression, 11
Bumper Length, 192 properties, 79, 89, 102
Bungie, 417 rhythm, 59, 103
“Audio the Bungie Way,” 417 Chord Change note property, 101
Buttram, Tobin, 387 ChordLevel, 101
ChordMap, 87, 91, 186, 334
C composing with, 334
cadence, 96 Editor, 92
call and response, 480 in Wide Sounds, 448
cascading sound effects, 428 Recompose on Loop, 93
CAudio, 161, 162, 195 Recompose on Play, 93
Alloc3DPath(), 311 Chord playmode, 100
ChangeCacheStatus(), 168 Chord/Scale playmode, 99
ComposeSegment(), 335 Chord Track, 21, 76, 87, 181
CreateAudioPathFromConfig(), 243, 244 creating, 79, 88
EnableNotifications(), 362 in Asheron’s Call, 484
Get3DListener(), 330 in Wide Sounds, 448
GetChord(), 220 Chord Variation Choices window, 52
 Index n 525

Chords for Composition, 57, 96 Current Time, 191

CLSID, 140 CWaveTool, 328
CNotificationManager, 353
EnableNotifications(), 354 D
InsertNotification(), 359 D3DLISTENER, 292
TranslateType(), 355 debugging scripts, 119
CoCreateInstance(), 140, 142 Default playmode, 81
CoInitialize(), 142 design-time files, 18
COM, 135, 139 dialog in Halo, 428
compiler, 135 DirectMusic, 3, 135
composition chord, 96 in Worms Blast, 465
Composition Engine, 334 integration in NOLF, 412
compression, DirectMusic Producer, 3, 17, 75
in Wide Sounds, 442 DirectSound, 135
web, 433 in Halo, 421
Compression, 228 Listener, 291
conditional statement, 123 voice management, 297
Cone, 290 DirectX 7, 136
content-driven audio, 8 DLS, 8, 149
context, 176 articulations, 30
continuous controllers, 15 creating DLS banks, 379, 397
in NOLF, 400, 402 editor, 399
in Russian Squares, 382 for the web, 442
Control Track, 181, 184 in NOLF, 392
Controlling Segment, 11, 72, 183 in Wide Sounds, 442
Core Layer, 135, 136 in Worms Blast, 466
CreateEvent(), 343 Level 1, 27
crossfades, 12 Level 2, 8
CScript, 273 on the web, 434
Init(), 275 optimization in Asheron’s Call, 488
CSegment, 194 DLS Collection, 8, 9
EnumSegments(), 201 creating, 23
GetDefaultResolution(), 204 memory implications of using, 13
GetName(), 213 DLS Designer, 23
HasEmbeddedAudioPath(), 214 in NOLF, 397
Init(), 198 DLS instruments, 9
CSegState, 193, 214 auditioning, 29
CThing, 313 creating, 25, 27
CalcNewPosition(), 316 layers in, 27
Move(), 317 looping, 24
Stop(), 316 regions in, 27
Start(), 315 stereo, 32
CThingManager, 318 DMO, 228
CalcNewPositions(), 323 accessing in AudioPath, 248
CreateThing(), 318 AudioPath, 235
EnforceAudioPathLimit(), 321 AudioPath Stage, 251
MoveThings(), 324 Buffer, 228
ReassignAudioPaths(), 319 Compression, 228
SetAudioPathLimit(), 323 CPU, 237
StartAssignedThings(), 320 Distortion, 229
CuddlyMonster, 261 file configuration, 229
526 n Index

GUID, 236 Embellishment, 59

Jones, 238 in Russian Squares, 383
name, 249 Pattern, 59
programming interface, 257 End Embellishment, 59
property page, 241, 258 Eno, Brian, 501
Reverb, 228 Evil Donut, 161
Send, 228 evolving patches, 489
DMUS_CHORD_KEY, 219
DMUS_CHORD_PARAM, 219 F
DMUS_OBJECTDESC, 154, 156, 172 files, 149
DMUS_PCHANNEL_ALL, 235 compression, 151
DMUS_SEGF_AFTERLATENCYTIME, 207 design-time, 18
DMUS_SEGF_AFTERQUEUETIME, 207 encryption, 151
DMUS_SEGF_ALIGN, 210 MIDI, 8, 160
DMUS_SEGF_AUTOTRANSITION, 186, MOD, 8
208, 214 run-time, 18
DMUS_SEGF_BEAT, 205 Style, 55
DMUS_SEGF_INVALIDATE_PRI, 212 Wave, 8, 160
DMUS_SEGF_MARKER, 205 Fill Embellishment, 59
DMUS_SEGF_MEASURE, 205 Flash, 438
DMUS_SEGF_NOINVALIDATE, 212 Fripp, Robert, 502
DMUS_SEGF_REFTIME, 206 FrogCrunch, 264
DMUS_SEGF_SEGMENTEND, 206 Fruity Loops, 479
DMUS_SEGF_TIMESIG_ALWAYS, 208
DMUS_SEGF_USE_AUDIOPATH, 213, 234 G
DMUS_SEGF_VALID_START_BEAT, 210 gamelan, 502
DMUS_SEGF_VALID_START_GRID, 210 garbage collection, 158
DMUS_SEGF_VALID_START_MEASURE, General MIDI, 434
210 GetDXVersion, 296
DMUS_SEGF_VALID_START_TICK, 210 GetObjectInPath(), 235
DMUS_SEGMENT_QUEUE, 206 GetParam(), 181
DMX control, 436 GigaStudio, 395
Dolby Digital, 419 global Tools, 226
Don’t Cut Off note property, 101 gm.dls, 22, 23
Doppler, 292 Goldston, Lori, 396
in Halo, 425 granularity, 512
Download(), 232 grid, 204
Downloadable Sounds format, 8 Grigg, Nathan, 387, 501
downloading, 129 groove level, 7, 55, 84, 86, 299, 378, 382
dream interpretation, 283 in Asheron’s Call, 485, 490
drum loops, 467 in NOLF, 468
DS3DLISTENER, 331 in Russian Squares, 370
dxguid.lib, 142 in Wide Sounds, 446
in Worms Blast, 468
E scripted, 118
effects, 176, 285, 381 Groove Level Track, 181
in Halo, 422 groove range, 58
in Wide Sounds, 451 Groove Track, 61
electronica, 376 GUID, 140, 154
embedded AudioPath, 234 GUID_All_Objects, 236
embedding versus referencing content, 18 Gunn, Trey, 502
 Index n 527

H SetNotificationHandle(), 343
Halo, 417 SetPrepareTime(), 192, 211
dialog, 428 Stop(), 187
history of, 418 StopEx(), 184, 187
music production, 421 IDirectMusicScript, 262
sound effects production, 423 CallRoutine(), 263, 276
surround sound, 426 EnumRoutine(), 267, 276
Hollywood Edge, 380 EnumVariable(), 268, 277
HRESULT, 140 GetVariableNumber(), 265, 277
HRTF, 425 GetVariableObject(), 265
HTML+Time, 436 GetVariableVariant(), 265
Hubbard, Craig, 394 Init(), 275
Hysteria, setting, 279 SetVariableNumber(), 265
SetVariableObject(), 265
I SetVariableVariant(), 265
IActiveScript, 271 IDirectMusicSegment, 142, 177
IDirectMusicAudioPath, 232, 242 Download(), 144, 232
Activate(), 238, 289, 319 GetParam(), 181
ConvertPChannel(), 245 Unload(), 144, 232
CreateStandardAudioPath(), 231 IDirectMusicSegmentState, 186, 187, 214
GetObjectInPath(), 146, 235, 253 IDirectMusicSong, 185
Release(), 237 IDirectMusicTool, 328, 357, 361
SetVolume(), 237 Flush(), 329
IDirectMusicChordMap, 335 GetMediaTypeArraySize(), 328, 358
IDirectMusicComposer, 335 GetMediaTypes(), 328, 358
ComposeSegmentFromShape(), 336 GetMsgDeliveryType(), 328, 357
IDirectMusicGraph, 330 Init(), 328, 357
IDirectMusicLoader, 142 ProcessPMsg(), 328, 329, 345, 347, 359
ClearCache(), 157, 168 IDirectMusicTrack, 179
CollectGarbage(), 158 GetParam(), 182
EnableCache(), 157, 161, 164, 168 IDirectSound3DBuffer, 146, 289
EnumObject(), 157, 166 IKsControl, 295
GetObject(), 154, 171, 172 IMediaParams, 258
LoadObjectFromFile(), 144, 153, 288 informative music, 476
ReleaseObject(), 158, 167 intensity
ReleaseObjectByUnknown(), 158 in Asheron’s Call, 478
ScanDirectory(), 156, 161, 169 in Russian Squares, 374
SetObject(), 152, 155, 171 interaction “hooks,” 440
SetSearchDirectory(), 152 interactive audio, 3
IDirectMusicObject, 243 differences from adaptive audio, 6
IDirectMusicPerformance, 142 interactivity, 3
CloseDown(), 145 in Asheron’s Call, 476
CreateAudioPath(), 230, 243 on the web, 435
CreateStandardAudioPath(), 287 Intro Embellishment, 59
GetDefaultAudioPath(), 146 Invalidation, 211
GetGraph(), 329 inversion, 101
GetNotificationPMsg(), 343 IPersistStream, 150, 258
GetParam(), 181 Load(), 151
InitAudio(), 143, 145, 164 ISpecifyPropertyPages, 258
PlaySegmentEx(), 144, 146, 184, 187, 233 IUnknown, 140
SetBumperLength(), 192 AddRef(), 140
528 n Index

QueryInterface, 141 M
Release(), 140, 147 macro-adaptability, 374
marker, 205
J Marker Track, 37
Jarett, Keith, 514 massively multiplayer games, 473
Jones, 188, 272, 351 music for, 474
Mastelotto, Pat, 502
K McLachlan, Sarah, 506
Kelly, Kevin, 502 memory footprint, 4
King Crimson, 502 memory use in DirectMusic, 13
Message Window, 120
L MFC, 135, 161
latency, 138, 293 micro-adaptability, 374
sound effects, 326 MIDI, 226
Latency Time, 191, 207 DirectMusic and, 8, 9, 14, 15
layering, 19, 68 Channel, 180
in NOLF, 396 control over wave playback, 20
DLS instruments, 27 Controllers, 15, 16
in Asheron’s Call, 494 MIDI file, 8, 160
in Russian Squares, 371, 374 reading into Producer, 76
in Wave Tracks, 20 Min, Mike, 371
in Wide Sounds, 437 Mingle, 286, 297
LFO, 30 implementation, 324
linear music vs. nonlinear music, 505 Mix-in Buffer, 228
Listener, 291 Moby, 506
accessing, 292 MOD files, 8
Doppler, 292, 303 Monolith, 387
manipulating, 330 mood, 7
Orientation, 291 Mother of All Windows, 52, 104
Position, 291, 303 motif, 56, 197
Rolloff Factor, 292, 331 in NOLF, 392, 405
Velocity, 292 in Russian Squares, 384
LithTech DirectMusic player, 396, 410 in Wide Sounds, 437
LithTech engine, 393 in Worms Blast, 466, 470
LithTech Player/Tester, 408 looping, 384
Loader, 138, 139, 149 munchkin, 301
garbage collection, 158 music cells, 370, 377
LoaderView sample application, 159 music layers, 371
loading, 129 music sets, 389
Lock ID, 48 music states, 388, 391
loop-slicing, 380 instrument level variation, 403
looping DLS sounds, 24 MUSIC_TIME, 188
low-latency sound effects, 326 musical form, 372, 373
low-frequency oscillator (LFO), 30 musical space, 482
lyric, 340, 346 Mute Track, 181
interpreting, 347
notification, 346 N
Lyric Track, 340 No One Lives Forever (NOLF), 389
nonlinear music, 501, 505
 Index n 529

nonlinearity of games, 5 boundaries, 205, 391

design, 508 channel (PChannel), 14, 180
note, 98 Layer, 135, 136, 224
ChordLevel, 101 notifications, 340
functionality, 97 tempo, 10, 11
inversion, 101 phrase samples, 512
MIDI definition, 12 Pitch Bend Range, 35
piano roll editing, 78 pitch envelope, 30
playmodes, 98 pitch variation, 4
properties, 98 playmode, 98
range in DLS instrument, 27 pmsg, 181, 224, 225, 328, 341
scattering, 489 notification, 341
transpose by Band, 34 Prepare Time, 192, 207
variations, 42, 82 primary Segment, 11, 182
Note Pattern Track, 77 creating, 79
Note Sequence Track, 76 in Wide Sounds, 447
Note Style Editor, 84 producer, 3
notification, 339 production process, 394
capturing, 344, 357 programmer, 3
categories, 340 ProjeKct X, 502
command, 341 player, 512
disabling/enabling, 342, 354
interpreting, 346 Q
lyric, 346 Queue Time, 192, 207
Measure and Beat, 341
Performance, 341 R
pmsg, 341 Ragsdale, Rich, 387
retrieving, 361 reactivity, 176
Segment, 341 Recompose ChordMap, 93
types, 360 recorded music, 504
recording engineer, 3
P REFERENCE_TIME, 186, 188
part, 19 regions in DLS instruments, 27
patch change optimization, 487 remix, 510
Pattern, 55, 398 repetition, 4, 6, 376
changing the playing Pattern, 64 in Asheron’s Call, 475
creation in NOLF, 400 in games, 5
Editor, 57, 84 resource, 169
Embellishment, 59 resource management, 307
in Russian Squares, 382 Reverb, 228
in Style, 60 rhythm in Asheron’s Call, 478
in Wide Sounds, 445 Robbins, Bryan, 371
length, 61, 103 routine (script), 113, 262, 263
properties, 58 run-time files, 18
variability, 58 packaging, 458
Pattern Track, 49, 77 Russian Squares, 369, 372
creating, 77 adaptive elements, 370
properties, 81 Bubble Matter, 372, 378
pause/resume, 12 Gravity Ride, 372, 378
PedalPoint playmode, 100 musical form, 372
Performance, 138, 139 production obstacles, 375
530 n Index

Spin Cycle, 372, 378 stopping, 288

Transition, 37, 66, 213
S transposed playback, 70
script, 108, 262 Wave, 138
AudioPath, 116 Segment Trigger Track, 158
AudioVBScript, 112, 271 SegmentState, 184
comment, 115 self-reference loop, 158
content, 262, 266 Sequence Track, 21, 77, 178
creating, 109 Shogo, 390
debugging, 119 signpost, 91
embedded files, 110 cadence, 96
error handling, 270 chord, 94
file format, 267 in ChordMap Editor, 92
groove level, 118 list, 93
play flags, 454 Signpost Track, 93
properties, 113 in Wide Sounds, 449
routines, 113, 262, 263, 351 software synthesizer, 8
set activity, 280 Sonic Implants, 371
transition Segment, 116 sound designer, 3
variables, 112, 122, 262 sound effects, 176, 285
VBScript, 112, 271 low-latency, 326
Script Designer, 111 scripted, 332
Script Track, 128, 158, 270, 340 versus music, 298
scripting, 107, 261 soundclasses, 420
in Asheron’s Call, 490 soundfile, 419
in NOLF, 393 soundtag, 419
in Russian Squares, 386 details, 422
in Wide Sounds, 453 stereo DLS instruments, 32
sound effects, 332 stereo separation, 488
secondary Segment, 11, 70, 183 streaming, 18, 19
in NOLF, 410 in Halo, 422
in Wide Sounds, 450 on the web, 434
Segment, 17, 138, 160, 175, 398 wave, 19
alignment, 65, 67, 209 strips, 20
branching, 66 structure, 176
clock-time, 10 in Russian Squares, 372
Compose from Shape, 336 Style, 55, 84, 149, 380, 398
Controlling, 11, 72, 182, 202 Band, 56
creating from wave files, 17 creating, 56, 84
Delay, 206 in NOLF, 400
in Asheron’s Call, 492 in Wide Sounds, 444
in NOLF, 403 motif, 56, 197
in Russian Squares, 384 Pattern, 57, 60
layering, 66 Style Designer, 61
length, 63 Style Track, 61
loading, 196 surround sound, 426
music-time, 10 switch points, 37
primary, 11, 182, 202 synchronize video to music, 339
Resolution, 203 synthesizer, 8, 137
secondary, 11, 182, 202
SegmentState, 184
 Index n 531

T Lyric, 340
tempo, 3, 10, 217, 219 Mute, 181
DirectMusic restrictions, 10 Pattern, 49, 77, 179
in Asheron’s Call, 478 Script, 128, 158, 270, 271, 340
in Worms Blast, 467 Segment Trigger, 158
jitter, 299 Sequence, 21, 77, 178
Tempo Track, 21, 62, 76 SignPost, 93
Test Player, 394 Style, 61
Time, 191 Tempo, 21, 62, 76
Current, 191 Time Signature, 21, 62, 178
Latency, 191, 207 types of, 17
Prepare, 192, 207 Wave, 17, 178
Queue, 192, 207 transition, 36, 208
Time Signature, 217 boundary options, 37
Time Signature Track, 21, 62, 178 in Asheron’s Call, 493
Tool, 327 in NOLF, 390, 406
accessing in AudioPath, 246, 257 in Wide Sounds, 436
adjust timing, 191 matrix, 410
AudioPath, 225, 226, 345 Transition Segment, 66, 116
AudioPath Configuration, 229 transpose, 34
AudioPath Stage, 236 triggers, 461
capturing notifications, 343, 344 tutorial, 75
creating, 330
CWaveTool, 328, 329 U
delivery time, 344 Unload(), 232
Immediate Time, 344 unloading, 129
IPersistStream support, 258
Lyric, 345, 347 V
name, 251 variability, 4, 176
notification, 357 variable (script), 122, 125, 262
Performance, 226, 345 variable playback, 41
Prepare Time, 192 variant, 265, 333
property page, 258 VariantClear(), 333
Queue Time, 344, 358 variation, 4, 41, 42
Segment, 226, 345 auditioning, 44
setting up, 345 bar, 82
thread, 346 editing, 44
Wave processing, 327 empty, 43
ToolGraph, 226, 329 enable and disable, 43, 83
AudioPath, 329 in Halo, 423
creating, 329 in NOLF, 389, 401
inserting, 330 in ProjecKct X, 507, 510
Performance, 225, 330, 361 in Russian Squares, 370
trace statement, 121 in Wide Sounds, 437
Track, 17, 19, 177, 178 locking, 48
Band, 22, 62, 178, 181 on the web, 435
Chord, 21, 76, 87, 181 properties, 46
ChordMap, 186 Variation Choices window, 52, 104
Control, 181, 184 Variation Switch Points, 50, 446
Groove, 61, 181 VBScript, 112, 271
Group, 19 Velocity, 290
532 n Index

velocity tracking, 427 Segment, 138

version, 296 truncation, 327
DirectX, 296 Wave Editor, 25
Visual Studio, 135 Wave Track, 17, 178
voice optimization, 488 WaveSurgeon, 467
volume, 237 web control object, 460
3D, 290 Whitmore, Guy, 499, 501
AudioPath, 237, 259 Wide Sounds, 436
Buffer, 254, 255 DMX control, 436
control, 306 scripting, 453
envelope, 30 testing functionality of, 452
music, 306 web site, 438
music vs. sound effects, 299, 304, 306 Worms Blast, 463
pmsg, 180 Lynne, Bjorn, 463
sound effects, 306 music production, 465
project structure, 467
W samples and instruments, 466
wave, 177 Team17 Software, 463
Doppler shift, 327 Ubi Soft Entertainment, 463
file, 8, 160, 213 Write latency, 295
pmsg, 327 Write period, 295
Preload, 207
preroll, 207 X
Xbox, 419
 Looking for more?
Check out Wordware’s market-leading Game Developer’s
Library featuring the following new releases.

Learn FileMaker Pro 6

1-55622-974-7 • $39.95
6 x 9 • 504 pp.
FileMaker Pro 6 Developer’s Guide
to XML/XSL
1-55622-043-X • $49.95
6 x 9 • 416 pp.
Advanced FileMaker Pro 6 Web
Development Games That Sell! Advanced 3D Game Programming
1-55622-860-0 • $59.95 1-55622-950-X • $34.95 with DirectX 9.0
6 x 9 • 464 pp. 6 x 9 • 336 pp. 1-55622-968-2 • $59.95
6 x 9 • 552 pp.

ShadersX2: Introduction and ShaderX2: Shader Programming Strategy Game Programming

Tutorials with DirectX 9.0 Tips and Tricks with DirectX 9.0
1-55622-902-X • $59.95 1-55622-988-7 • $59.95 1-55622-922-4 • $59.95
6 x 9 • 500 pp. 6 x 9 • 500 pp. 6 x 9 • 560 pp.

Introduction to 3D Game Programming LightWave 3D 7.5 Lighting Essential LightWave 3D 7.5

with DirectX 9.0 1-55622-354-4 • $69.95 1-55622-226-2 • $44.95
1-55622-913-5 • $49.95 6 x 9 • 496 pp. 6 x 9 • 424 pp.
6 x 9 • 464 pp.

Visit us online at www.wordware.com for more information.

Use the following coupon code for online specials: audio-2882
 A Word About the Game Audio Network Guild
(G.A.N.G.)

Hundreds of music, audio, and interactive entertainment industry professionals are joined
together as members of the Game Audio Network Guild (G.A.N.G.). G.A.N.G. is an inter-
national, professional, non-profit organization dedicated to raising the bar for audio
development in interactive media. The individuals who make up G.A.N.G understand the
power of audio and its importance in making interactive media a gratifying and provoca-
tive experience. G.A.N.G. is determined to better the state of audio in interactive media.
As a guild, we change the way people experience interactive media. As a network, we
help each other achieve new successes in our own professional endeavors.

G.A.N.G. Members
G.A.N.G. boasts over 400 members in 22 different countries. Professionals, associates,
apprentices, and students alike make up the body of the organization. Members commu-
nicate via G.A.N.G.’s extensive online message board community, annual game industry
events, and through local chapter meetings. Members enjoy a slew of benefits and dis-
counts for belonging to G.A.N.G. but more important than all that is belonging to the
most prestigious organization in interactive audio!

G.A.N.G. Sponsors
G.A.N.G. sponsors support our initiative to promote excellence in audio for interactive
media:
n Audio Products International Corporation (www.miragespeakers.com
www.energyspeakers.com and www.athenaspeakers.com)
n Dolby Laboratories, Inc. (www.dolby.com)
n DTS (www.dtsonline.com)
n Native Instruments (www.nativeinstruments.com)
n Sound Ideas (www.sound-ideas.com)
Accomplishments and Long Term Goals
n GANG is working with the Recording Musicians Association to craft a new union
agreement specifically for videogames that allows game composers to use the fin-
est musicians in Los Angeles and other union cities
n GANG’s recently formed Grammy Committee is taking over the effort to create a
new Grammy category specifically for game music
n GANG is putting together a special committee to advise speaker manufacturer
Audio Products International Corporation in their endeavor to create the world’s
finest 5.1 speaker system specifically for gamers
n The 2003 Annual GANG Awards show hosted over 600 top industry profession-
als, presented 33 awards and 2 prizes for student and professional competitions,
and featured 4 live performances by GANG members!!

Join the G.A.N.G!

G.A.N.G. is an organization dedicated to promoting excellence in audio through educa-
tion, recognition, and support of its members, sponsors, and the game audio community
at large. Gatherings, networking, product discounts, and access to the growing library of
information on www.audiogang.org are just the icing on the cake. We’re changing the
landscape around the audio and interactive entertainment industries and we want you to
be a part of the renovations!
For more information and to join the organization visit www.audiogang.org or contact:
Todd M. Fay
Director of Development
Game Audio Network Guild
[email protected]
 into Be-Bop
When it comes to your audio and music
development needs, the experts at
NewBlue are playing your tune. Our
highly-respected team comprises years of
software and hardware experience to bring you
innovative, reactive technology and user paradigm design.

From consulting and custom development to proprietary

technology licensing and DirectX Audio expertise, NewBlue
can make the difference in your entertainment or multimedia
efforts. Contact us today. It’ll be music to your ears.

NewBlue: The sound of things to come.

Find us on the web at www.newblueinc.com
or email us at [email protected].
About the CD
The companion CD contains projects discussed in the book and the DirectX 9.0
SDK.
To use the DirectX 9 SDK, place the CD in your CD/DVD drive, browse to the
DirectX SDK Source Files folder, and follow these steps:
1. Unzip the files to a temporary folder on your hard drive; you will need xxx
free space for this.
2. From that folder, run the file in the root called Install.exe. This will bring
you to a splash screen menu.
3. Install the DirectX SDK first, following all instructions and rebooting if
necessary.
4. Install DirectMusic Producer next. You should now have all files needed to
use the book files found in the DirectX Audio Book Files folder.
The DirectX Audio Book Files folder is organized into three folders: Unit I, Unit II,
and Unit III. Each chapter in Unit I and Unit III has supporting material in an equiva-
lent unit/chapter folder on the CD. For example, if you’d like to run the ProjeKct X
sample from Chapter 21, go to Unit III\Chapter 21 to find it. Or, if you’d like to load
the tutorial from Chapter 5 into Producer, go to Unit I\Chapter 5\Tutorial Part One.
Unit II, which deals with programming, is slightly different. It too has folders
for each chapter, but they are identified by chapter number and programming pro-
ject, since there can be more than one programming project per chapter. Also, Unit
II has a few extra directories that are shared across all Unit II chapters: The Bin
directory contains the final executables of each project, the SourceMedia folder has
the DirectMusic Producer source files to create the content, and the Media direc-
tory includes the final run-time media files for the sample projects. You can either
copy these files by hand, or you can run the Setup program, also in the Unit II folder,
to automatically install these files on your hard drive. It includes an uninstaller, so
you can install, play with the materials, and when done wipe them out to make room
for more on your hard drive.
Unit III includes also sample media from various projects covered in the respec-
tive chapters. These samples offer an inside look into interactive audio production.

þ Warning: By opening the CD package, you accept the terms and

conditions of the CD/Source Code Usage License Agreement on the
following page.
Additionally, opening the CD package makes this book
nonreturnable.
 CD/Source Code Usage License Agreement
Please read the following CD/Source Code usage license agreement before opening the CD and
using the contents therein:
1. By opening the accompanying software package, you are indicating that you have read and
agree to be bound by all terms and conditions of this CD/Source Code usage license
agreement.
2. The compilation of code and utilities contained on the CD and in the book are copyrighted and
protected by both U.S. copyright law and international copyright treaties, and is owned by
Wordware Publishing, Inc. Individual source code, example programs, help files, freeware,
shareware, utilities, and evaluation packages, including their copyrights, are owned by the
respective authors.
3. No part of the enclosed CD or this book, including all source code, help files, shareware,
freeware, utilities, example programs, or evaluation programs, may be made available on a
public forum (such as a World Wide Web page, FTP site, bulletin board, or Internet news
group) without the express written permission of Wordware Publishing, Inc. or the author of
the respective source code, help files, shareware, freeware, utilities, example programs, or
evaluation programs.
4. You may not decompile, reverse engineer, disassemble, create a derivative work, or otherwise
use the enclosed programs, help files, freeware, shareware, utilities, or evaluation programs
except as stated in this agreement.
5. The software, contained on the CD and/or as source code in this book, is sold without war-
ranty of any kind. Wordware Publishing, Inc. and the authors specifically disclaim all other
warranties, express or implied, including but not limited to implied warranties of merchant-
ability and fitness for a particular purpose with respect to defects in the disk, the program,
source code, sample files, help files, freeware, shareware, utilities, and evaluation programs
contained therein, and/or the techniques described in the book and implemented in the exam-
ple programs. In no event shall Wordware Publishing, Inc., its dealers, its distributors, or the
authors be liable or held responsible for any loss of profit or any other alleged or actual private
or commercial damage, including but not limited to special, incidental, consequential, or other
damages.
6. One (1) copy of the CD or any source code therein may be created for backup purposes. The
CD and all accompanying source code, sample files, help files, freeware, shareware, utilities,
and evaluation programs may be copied to your hard drive. With the exception of freeware and
shareware programs, at no time can any part of the contents of this CD reside on more than
one computer at one time. The contents of the CD can be copied to another computer, as long
as the contents of the CD contained on the original computer are deleted.
7. You may not include any part of the CD contents, including all source code, example pro-
grams, shareware, freeware, help files, utilities, or evaluation programs in any compilation of
source code, utilities, help files, example programs, freeware, shareware, or evaluation pro-
grams on any media, including but not limited to CD, disk, or Internet distribution, without
the express written permission of Wordware Publishing, Inc. or the owner of the individual
source code, utilities, help files, example programs, freeware, shareware, or evaluation
programs.
8. You may use the source code, techniques, and example programs in your own commercial or
private applications unless otherwise noted by additional usage agreements as found on the
CD.

6 Warning By opening the CD package, you accept the terms and conditions
of the CD/Source Code Usage License Agreement.
Additionally, opening the CD package makes this book nonreturnable.