Clojure In Small Pieces

Rich Hickey

Juozas Baliuka Eric Bruneton Chas Emerick

Tom Faulhaber Stephen C. Gilardi Daniel Solano Gmez
Christophe Grand Stuart Halloway Mike Hinchey
Shawn Hoover Chris Houser Rajendra Inamdar
Eugene Kuleshov Robert Lachlan J. McConnell
Chris Nokleberg Allen Rohner Michel Salim
Jason Sankey Stuart Sierra Frantisek Sodomka
Vishal Vishnoi Jean Niklas Lorange Kyle Kingsbury
Kai Wu

Timothy Daly (Editor)

Based on Version 1.3.0-alpha4
November 14, 2013
Contents

Preface: Why Literate Programming i

Steps to build Clojure . . . . . . . . . . . . . . . . . . . . . . . . . i
Steps to change Clojure . . . . . . . . . . . . . . . . . . . . . . . . i
Why Bother? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

1 From The Ground Up (Kyle Kingsbury) 1

1.1 Clojure from the ground up . . . . . . . . . . . . . . . . . . . 1
1.1.1 Getting set up . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 The structure of programs . . . . . . . . . . . . . . . . 2
1.1.3 Review . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Clojure from the ground up: basic types . . . . . . . . . . . . 6
1.2.1 Types . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.2 Integers . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.3 Fractional numbers . . . . . . . . . . . . . . . . . . . . 9
1.2.4 Mathematical operations . . . . . . . . . . . . . . . . 9
1.2.5 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.6 Booleans and logic . . . . . . . . . . . . . . . . . . . . 13
1.2.7 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.2.8 Keywords . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.2.9 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.2.10 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.11 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.2.12 Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.2.13 Putting it all together . . . . . . . . . . . . . . . . . . 20

2 From ideas to implementation 23

2.1 Starting from Java . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2 Extends and Implements . . . . . . . . . . . . . . . . . . . . . 23
2.3 Understanding Persistent Vectors (Jean Lorange) . . . . . . . 23
2.3.1 Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.2 Popping . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3.3 O(1) != O(log n) . . . . . . . . . . . . . . . . . . . . . 31
2.3.4 Persistence . . . . . . . . . . . . . . . . . . . . . . . . 32
2.4 Red Black Trees . . . . . . . . . . . . . . . . . . . . . . . . . 39

i
ii CONTENTS

2.4.1 Persistence . . . . . . . . . . . . . . . . . . . . . . . . 39
2.4.2 Persistent Red Black Trees . . . . . . . . . . . . . . . 39
2.4.3 Node Structure . . . . . . . . . . . . . . . . . . . . . . 40
2.4.4 The Black Node implementation . . . . . . . . . . . . 42
2.4.5 Constructing a Black Node . . . . . . . . . . . . . . . 45
2.4.6 The Red Node implementation . . . . . . . . . . . . . 46
2.4.7 Constructing a Red Node . . . . . . . . . . . . . . . . 49
2.4.8 Okasakis balance cases . . . . . . . . . . . . . . . . . 50
2.4.9 Deleting a Node . . . . . . . . . . . . . . . . . . . . . 52
2.4.10 Clojures balance cases . . . . . . . . . . . . . . . . . . 53
2.4.11 Replacing a Node . . . . . . . . . . . . . . . . . . . . . 55
2.5 Immutable Data Structures . . . . . . . . . . . . . . . . . . . 55
2.6 Bit-Partitioned Hash Tries . . . . . . . . . . . . . . . . . . . . 55
2.6.1 Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6.2 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6.3 Hashmaps . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.6.4 Seqs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.7 Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.8 Java Interoperability . . . . . . . . . . . . . . . . . . . . . . . 62
2.8.1 Language primitives . . . . . . . . . . . . . . . . . . . 62
2.8.2 Clojure calling Java . . . . . . . . . . . . . . . . . . . 64
2.9 Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.10 Argument Deconstruction . . . . . . . . . . . . . . . . . . . . 65
2.11 The Read-Eval-Print Loop . . . . . . . . . . . . . . . . . . . . 65
2.12 Special Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.13 Reader Macros . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.14 Namespaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.15 Dynamic Comopilation . . . . . . . . . . . . . . . . . . . . . . 66
2.16 Ahead-Of-Time Comopilation . . . . . . . . . . . . . . . . . . 66
2.17 Lazy Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.18 Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.19 Concurrency . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
2.20 Identity and State . . . . . . . . . . . . . . . . . . . . . . . . 66
2.21 Software Transactional Memory . . . . . . . . . . . . . . . . . 67
2.22 Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.23 The Lisp Reader . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.23.1 Reader Syntax Macros . . . . . . . . . . . . . . . . . . 71
2.23.2 The String reader macro . . . . . . . . . . . . . . . . . 72
2.23.3 The Comment reader macro . . . . . . . . . . . . . . . 74
2.23.4 The Wrapping reader macro . . . . . . . . . . . . . . . 74
2.23.5 The Meta reader macro . . . . . . . . . . . . . . . . . 75
2.23.6 The SyntaxQuote reader macro . . . . . . . . . . . . . 76
2.23.7 The Unquote reader macro . . . . . . . . . . . . . . . 79
2.23.8 The List reader macro . . . . . . . . . . . . . . . . . . 80
2.23.9 The Unmatched Delimiter reader macro . . . . . . . . 80
2.23.10 The Vector reader macro . . . . . . . . . . . . . . . . 80
CONTENTS iii

2.23.11 The Map reader macro . . . . . . . . . . . . . . . . . . 81

2.23.12 The Character reader macro . . . . . . . . . . . . . . 81
2.23.13 The Arg reader macro . . . . . . . . . . . . . . . . . . 82
2.24 Reader Dispatch Macros . . . . . . . . . . . . . . . . . . . . . 83
2.24.1 The Var reader macro . . . . . . . . . . . . . . . . . . 84
2.24.2 The Regular Expression reader macro . . . . . . . . . 85
2.24.3 The Function reader macro . . . . . . . . . . . . . . . 86
2.24.4 The Set reader macro . . . . . . . . . . . . . . . . . . 87
2.24.5 The Eval reader macro . . . . . . . . . . . . . . . . . . 87
2.24.6 The Unreadable reader macro . . . . . . . . . . . . . . 89
2.24.7 The Discard reader macro . . . . . . . . . . . . . . . . 89
2.25 Vars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
2.26 Transients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
2.27 Atoms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
2.28 Refs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
2.29 Agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
2.30 Promise and Deliver . . . . . . . . . . . . . . . . . . . . . . . 94
2.31 Futures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
2.32 MultiMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
2.33 Deftype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
2.34 Defrecord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
2.35 Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
2.36 Prototypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.37 Genclass . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.37.1 Overriding protected methods . . . . . . . . . . . . . . 102
2.38 Proxies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
2.39 Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
2.40 Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
2.41 Generating Java Classes . . . . . . . . . . . . . . . . . . . . . 105
2.42 Type Hints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
2.43 Native Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . 105

3 Writing Idiomatic Clojure 107

4 The Ants Demo (Kai Wu) 109

4.1 The Simulation World . . . . . . . . . . . . . . . . . . . . . . 109
4.1.1 Initial setup of constants/magic-numbers . . . . . . . 109
4.1.2 The board: ready to mutate via transactions . . . . . 110
4.1.3 Ants as agents - doing asynchronous uncoordinated changes 112
4.1.4 Setting up the home, and ants . . . . . . . . . . . . . 113
4.1.5 Orientation and moving around the world . . . . . . . 115
4.1.6 Ant-agent behavior functions . . . . . . . . . . . . . . 116
4.1.7 World behavior: pheromone evaporation . . . . . . . . 124
4.2 The UI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.2.1 Using the Java AWT . . . . . . . . . . . . . . . . . . . 125
4.2.2 Functions to render the board and the ants . . . . . . 126
iv CONTENTS

4.2.3 Setting the scene, then updating it continually . . . . 128

4.3 Running the Program . . . . . . . . . . . . . . . . . . . . . . 130
4.3.1 Running the simulator . . . . . . . . . . . . . . . . . . 130

5 Parallel Processing 133

5.1 Natural Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5.2 Steeles parallel ideas . . . . . . . . . . . . . . . . . . . . . . . 133
5.3 Sequential mapreduce . . . . . . . . . . . . . . . . . . . . . . 137
5.3.1 Parallel mapreduce . . . . . . . . . . . . . . . . . . . . 139
5.4 MPI Message Passing . . . . . . . . . . . . . . . . . . . . . . 141
5.4.1 The N-colony Ant Demo . . . . . . . . . . . . . . . . . 141
5.5 MapReduce . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

6 The ant build sequence 143

6.1 ant.build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.2 The Execution . . . . . . . . . . . . . . . . . . . . . . . . . . 149

7 jvm/clojure/asm/ 163
7.1 AnnotationVisitor.java . . . . . . . . . . . . . . . . . . . . . . 163
7.2 AnnotationWriter.java . . . . . . . . . . . . . . . . . . . . . . 165
7.3 Attribute.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
7.4 ByteVector.java . . . . . . . . . . . . . . . . . . . . . . . . . . 176
7.5 ClassAdapter.java . . . . . . . . . . . . . . . . . . . . . . . . 183
7.6 ClassReader.java . . . . . . . . . . . . . . . . . . . . . . . . . 185
7.7 ClassVisitor.java . . . . . . . . . . . . . . . . . . . . . . . . . 229
7.8 ClassWriter.java . . . . . . . . . . . . . . . . . . . . . . . . . 233
7.9 Edge.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
7.10 FieldVisitor.java . . . . . . . . . . . . . . . . . . . . . . . . . 263
7.11 FieldWriter.java . . . . . . . . . . . . . . . . . . . . . . . . . . 264
7.12 Frame.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
7.13 Handler.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
7.14 Item.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
7.15 Label.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
7.16 MethodAdapter.java . . . . . . . . . . . . . . . . . . . . . . . 314
7.17 MethodVisitor.java . . . . . . . . . . . . . . . . . . . . . . . . 317
7.18 MethodWriter.java . . . . . . . . . . . . . . . . . . . . . . . . 326
7.19 Opcodes.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
7.20 Type.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394

8 jvm/clojure/asm/commons 413
8.1 AdviceAdapter.java . . . . . . . . . . . . . . . . . . . . . . . . 413
8.2 AnalyzerAdapter.java . . . . . . . . . . . . . . . . . . . . . . 426
8.3 CodeSizeEvaluator.java . . . . . . . . . . . . . . . . . . . . . 445
8.4 EmptyVisitor.java . . . . . . . . . . . . . . . . . . . . . . . . 449
8.5 GeneratorAdapter.java . . . . . . . . . . . . . . . . . . . . . . 453
8.6 LocalVariablesSorter.java . . . . . . . . . . . . . . . . . . . . 484
CONTENTS v

8.7 Method.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491

8.8 SerialVersionUIDAdder.java . . . . . . . . . . . . . . . . . . . 496
8.9 StaticInitMerger.java . . . . . . . . . . . . . . . . . . . . . . . 506
8.10 TableSwitchGenerator.java . . . . . . . . . . . . . . . . . . . 508

9 jvm/clojure/lang/ 509
9.1 AFn.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
9.2 AFunction.java . . . . . . . . . . . . . . . . . . . . . . . . . . 519
9.3 Agent.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
9.4 AMapEntry.java . . . . . . . . . . . . . . . . . . . . . . . . . 527
9.5 APersistentMap.java . . . . . . . . . . . . . . . . . . . . . . . 530
9.6 APersistentSet.java . . . . . . . . . . . . . . . . . . . . . . . . 538
9.7 APersistentVector.java . . . . . . . . . . . . . . . . . . . . . . 541
9.8 AReference.java . . . . . . . . . . . . . . . . . . . . . . . . . . 552
9.9 ARef.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 553
9.10 ArityException.java . . . . . . . . . . . . . . . . . . . . . . . 556
9.11 ArrayChunk.java . . . . . . . . . . . . . . . . . . . . . . . . . 556
9.12 ArraySeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 558
9.13 ASeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571
9.14 Associative.java . . . . . . . . . . . . . . . . . . . . . . . . . . 576
9.15 Atom.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577
9.16 ATransientMap.java . . . . . . . . . . . . . . . . . . . . . . . 579
9.17 ATransientSet.java . . . . . . . . . . . . . . . . . . . . . . . . 581
9.18 BigInt.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582
9.19 Binding.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
9.20 Box.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
9.21 ChunkBuffer.java . . . . . . . . . . . . . . . . . . . . . . . . . 586
9.22 ChunkedCons.java . . . . . . . . . . . . . . . . . . . . . . . . 586
9.23 Compile.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 588
9.24 Compiler.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 590
9.25 Cons.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 766
9.26 Counted.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 768
9.27 Delay.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 768
9.28 DynamicClassLoader.java . . . . . . . . . . . . . . . . . . . . 769
9.29 EnumerationSeq.java . . . . . . . . . . . . . . . . . . . . . . . 770
9.30 Fn.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772
9.31 IChunk.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772
9.32 IChunkedSeq.java . . . . . . . . . . . . . . . . . . . . . . . . . 773
9.33 IDeref.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 773
9.34 IEditableCollection.java . . . . . . . . . . . . . . . . . . . . . 774
9.35 IFn.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774
9.36 IKeywordLookup.java . . . . . . . . . . . . . . . . . . . . . . 796
9.37 ILookup.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 797
9.38 ILookupSite.java . . . . . . . . . . . . . . . . . . . . . . . . . 797
9.39 ILookupThunk.java . . . . . . . . . . . . . . . . . . . . . . . . 798
9.40 IMapEntry.java . . . . . . . . . . . . . . . . . . . . . . . . . . 798
vi CONTENTS

9.41 IMeta.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 799

9.42 Indexed.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 799
9.43 IndexedSeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . 799
9.44 IObj.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 800
9.45 IPersistentCollection.java . . . . . . . . . . . . . . . . . . . . 800
9.46 IPersistentList.java . . . . . . . . . . . . . . . . . . . . . . . . 801
9.47 IPersistentMap.java . . . . . . . . . . . . . . . . . . . . . . . 801
9.48 IPersistentSet.java . . . . . . . . . . . . . . . . . . . . . . . . 802
9.49 IPersistentStack.java . . . . . . . . . . . . . . . . . . . . . . . 802
9.50 IPersistentVector.java . . . . . . . . . . . . . . . . . . . . . . 802
9.51 IPromiseImpl.java . . . . . . . . . . . . . . . . . . . . . . . . 803
9.52 IProxy.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 803
9.53 IReduce.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 804
9.54 IReference.java . . . . . . . . . . . . . . . . . . . . . . . . . . 804
9.55 IRef.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805
9.56 ISeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805
9.57 IteratorSeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . 806
9.58 ITransientAssociative.java . . . . . . . . . . . . . . . . . . . . 807
9.59 ITransientCollection.java . . . . . . . . . . . . . . . . . . . . . 808
9.60 ITransientMap.java . . . . . . . . . . . . . . . . . . . . . . . . 808
9.61 ITransientSet.java . . . . . . . . . . . . . . . . . . . . . . . . 809
9.62 ITransientVector.java . . . . . . . . . . . . . . . . . . . . . . . 809
9.63 Keyword.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 810
9.64 KeywordLookupSite.java . . . . . . . . . . . . . . . . . . . . . 816
9.65 LazilyPersistentVector.java . . . . . . . . . . . . . . . . . . . 817
9.66 LazySeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 818
9.67 LineNumberingPushbackReader.java . . . . . . . . . . . . . . 823
9.68 LispReader.java . . . . . . . . . . . . . . . . . . . . . . . . . . 825
9.69 LockingTransaction.java . . . . . . . . . . . . . . . . . . . . . 836
9.70 MapEntry.java . . . . . . . . . . . . . . . . . . . . . . . . . . 850
9.71 MapEquivalence.java . . . . . . . . . . . . . . . . . . . . . . . 850
9.72 MethodImplCache.java . . . . . . . . . . . . . . . . . . . . . . 851
9.73 MultiFn.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 852
9.74 Named.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 861
9.75 Namespace.java . . . . . . . . . . . . . . . . . . . . . . . . . . 861
9.76 Numbers.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 867
9.77 Obj.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 947
9.78 PersistentArrayMap.java . . . . . . . . . . . . . . . . . . . . . 948
9.79 PersistentHashMap.java . . . . . . . . . . . . . . . . . . . . . 955
9.80 PersistentHashSet.java . . . . . . . . . . . . . . . . . . . . . . 980
9.81 PersistentList.java . . . . . . . . . . . . . . . . . . . . . . . . 982
9.82 PersistentQueue.java . . . . . . . . . . . . . . . . . . . . . . . 989
9.83 PersistentStructMap.java . . . . . . . . . . . . . . . . . . . . 995
9.84 PersistentTreeMap.java . . . . . . . . . . . . . . . . . . . . . 1000
9.85 PersistentTreeSet.java . . . . . . . . . . . . . . . . . . . . . . 1012
9.86 PersistentVector.java . . . . . . . . . . . . . . . . . . . . . . . 1014
CONTENTS vii

9.87 ProxyHandler.java . . . . . . . . . . . . . . . . . . . . . . . . 1030

9.88 Range.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1031
9.89 Ratio.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1033
9.90 Ref.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1034
9.91 Reflector.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 1044
9.92 Repl.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1054
9.93 RestFn.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055
9.94 Reversible.java . . . . . . . . . . . . . . . . . . . . . . . . . . 1094
9.95 RT.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1094
9.96 Script.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1138
9.97 SeqEnumeration.java . . . . . . . . . . . . . . . . . . . . . . . 1138
9.98 SeqIterator.java . . . . . . . . . . . . . . . . . . . . . . . . . . 1139
9.99 Seqable.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1140
9.100Sequential.java . . . . . . . . . . . . . . . . . . . . . . . . . . 1140
9.101Settable.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 1140
9.102Sorted.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1141
9.103StringSeq.java . . . . . . . . . . . . . . . . . . . . . . . . . . . 1141
9.104Symbol.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1143
9.105TransactionalHashMap.java . . . . . . . . . . . . . . . . . . . 1145
9.106Util.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1149
9.107Var.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1152
9.108XMLHandler.java . . . . . . . . . . . . . . . . . . . . . . . . . 1164

10 jvm/clojure 1167
10.1 main.java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1167

11 clj/clojure/ 1169
11.1 core.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1169
11.2 protocols.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1296
11.3 coredeftype.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1298
11.4 coreprint.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1315
11.5 coreproxy.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1323
11.6 data.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1332
11.7 genclass.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1334
11.8 gvec.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1351
11.9 inspector.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1361
11.10browse.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1365
11.11browseui.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1366
11.12io.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1367
11.13javadoc.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1376
11.14shell.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1378
11.15main.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1381
11.16parallel.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1389
11.17clformat.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1394
11.18columnwriter.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1436
11.19dispatch.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1438
viii CONTENTS

11.20pprintbase.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1448
11.21prettywriter.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1456
11.22printtable.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1467
11.23utilities.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1467
11.24pprint.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1470
11.24.1 java.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1471
11.25reflect.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1476
11.26repl.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1479
11.27set.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1485
11.28stacktrace.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1489
11.29string.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1490
11.30template.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1496
11.31junit.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1497
11.32tap.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1501
11.33test.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1503
11.34version.properties . . . . . . . . . . . . . . . . . . . . . . . . . 1519
11.35walk.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1519
11.36xml.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1522
11.37zip.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1524
11.38pom-template.xml . . . . . . . . . . . . . . . . . . . . . . . . 1531

12 test/clojure 1533
12.1 test/testclojure.clj . . . . . . . . . . . . . . . . . . . . . . . . 1533
12.2 test/testhelper.clj . . . . . . . . . . . . . . . . . . . . . . . . 1535
12.3 test/agents.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1537
12.4 test/annotations.clj . . . . . . . . . . . . . . . . . . . . . . . . 1541
12.5 test/atoms.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1541
12.6 test/clojureset.clj . . . . . . . . . . . . . . . . . . . . . . . . 1542
12.7 test/clojurexml.clj . . . . . . . . . . . . . . . . . . . . . . . . 1546
12.8 test/clojurezip.clj . . . . . . . . . . . . . . . . . . . . . . . . 1547
12.9 test/compilation.clj . . . . . . . . . . . . . . . . . . . . . . . . 1548
12.10test/control.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1550
12.11test/data.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1556
12.12test/datastructures.clj . . . . . . . . . . . . . . . . . . . . . . 1557
12.13test/def.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1574
12.14test/errors.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1575
12.15test/evaluation.clj . . . . . . . . . . . . . . . . . . . . . . . . 1576
12.16test/for.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1580
12.17test/genclass.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1583
12.18test/javainterop.clj . . . . . . . . . . . . . . . . . . . . . . . . 1585
12.19test/keywords.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1594
12.20test/logic.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1594
12.21test/macros.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1599
12.22test/main.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1599
12.23test/metadata.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1600
12.24test/multimethods.clj . . . . . . . . . . . . . . . . . . . . . . 1602
CONTENTS ix

12.25test/nslibs.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1605
12.26test/numbers.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1607
12.27test/otherfunctions.clj . . . . . . . . . . . . . . . . . . . . . . 1618
12.28test/parallel.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1620
12.29test/pprint.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1621
12.30test/predicates.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1621
12.31test/printer.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1624
12.32test/protocols.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1626
12.33test/reader.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1633
12.34test/reflect.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1640
12.35test/refs.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1640
12.36test/repl.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1641
12.37test/rt.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1642
12.38test/sequences.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1644
12.39test/serialization.clj . . . . . . . . . . . . . . . . . . . . . . . 1668
12.40test/special.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1671
12.41test/string.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1672
12.42test/test.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1674
12.43test/testfixtures.clj . . . . . . . . . . . . . . . . . . . . . . . . 1677
12.44test/transients.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1678
12.45test/vars.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1678
12.46test/vectors.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1680
12.47test/java5.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . 1686
12.48test/java6andlater.clj . . . . . . . . . . . . . . . . . . . . . . 1688
12.49test/examples.clj . . . . . . . . . . . . . . . . . . . . . . . . . 1690
12.50test/io.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1691
12.51test/javadoc.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1696
12.52test/shell.clj . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1696
12.53test/testclformat.clj . . . . . . . . . . . . . . . . . . . . . . . 1697
12.54test1/testhelper.clj . . . . . . . . . . . . . . . . . . . . . . . . 1714
12.55test/testpretty.clj . . . . . . . . . . . . . . . . . . . . . . . . 1715
12.56test1/examples.clj . . . . . . . . . . . . . . . . . . . . . . . . 1721
12.57test/moreexamples.clj . . . . . . . . . . . . . . . . . . . . . . 1722
12.58test/example.clj . . . . . . . . . . . . . . . . . . . . . . . . . . 1722

A External Java References 1723

B Copyright and Licenses 1725

C Building Clojure from this document 1733

C.1 The basic idea . . . . . . . . . . . . . . . . . . . . . . . . . . 1733
C.2 The tangle function in C . . . . . . . . . . . . . . . . . . . . . 1733
C.3 Makefile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1737
C.4 The tangle function in Clojure . . . . . . . . . . . . . . . . . 1744
C.4.1 Author and License . . . . . . . . . . . . . . . . . . . 1744
C.4.2 Abstract and Use Cases . . . . . . . . . . . . . . . . . 1744
x CONTENTS

C.4.3 The Latex Support Code . . . . . . . . . . . . . . . . 1746

C.4.4 Imports . . . . . . . . . . . . . . . . . . . . . . . . . . 1747
C.4.5 The Tangle Command . . . . . . . . . . . . . . . . . . 1748
C.4.6 The say function . . . . . . . . . . . . . . . . . . . . . 1748
C.4.7 The read-file function . . . . . . . . . . . . . . . . . . 1748
C.4.8 The ischunk function . . . . . . . . . . . . . . . . . . . 1749
C.4.9 The haschunks function . . . . . . . . . . . . . . . . . 1750
C.4.10 The expand function . . . . . . . . . . . . . . . . . . . 1751
C.4.11 The tangle function . . . . . . . . . . . . . . . . . . . 1751

Bibliography 1753

Index 1755
CONTENTS xi

Foreword

Rich Hickey invented Clojure. This is a fork of the project to experiment with
literate programming as a development and documentation technology.
Every effort is made to give credit for any and all contributions.
Clojure is a break with the past traditions of Lisp. This literate fork is a
break with the past traditions of code development. As such it is intended as
an experiment, not a replacement or competition with the official version of
Clojure.
Most programmers are still locked into the idea of making a program out of a
large pile of tiny files containing pieces of programs. They do not realize that
this organization was forced by the fact that machines like the PDP 11 only
had 8k of memory and a limit of 4k buffers in the editor. Thus there was a lot
of machinery built up, such as overlay linkers, to try to reconstruct the whole
program.
The time has come to move into a more rational means of creating and main-
taining programs. Knuth suggested we write programs like we write literature,
with the idea that we are trying to communicate the ideas to other people. The
fact that the machine can also run the programs is a useful side-effect but not
important.
Very few people have seen a literate program so this is intended as a complete
working example, published in book form. The intent is that you can sit and
read this book like any other novel. At the end of it you will be familiar with
the ideas and how those ideas are actually expressed in the code.
If programmers can read it and understand it then they can maintain and modify
it. The ideas will have been communicated. The code will be changed to match
changes in the idea. We will all benefit.
Ive tried to make it as simple as possible. Try it once, you might like it.

Tim Daly
December 28, 2010 ((iHy))
CONTENTS i

Preface: Why Literate Programming

This is a literate program, inspired by Donald Knuth [Knu84]. It is intended

to be read like a novel from cover to cover. The ideas are expressed clearly but
they are grounded in the actual source code.
The code in this documment is the executable source. The appendix gives the
procedure for building a running system from the enclosed sources.
Steps to build Clojure
Step 1
Basically you need the C program [1733] tangle.c which you can clip using a
text editor and save it as tangle.c.
Step 2
Compile tangle.c to create a function called tangle.

gcc -o tangle tangle.c

Step 3 Run tangle to extract the [1737] Makefile from this document.

tangle clojure.pamphlet Makefile >Makefile

Step 4

make

This will

create a new subdirectory called tpd containing all of the source code
test Clojure
create a running copy of Clojure
create the pdf
start a Clojure REPL

Steps to change Clojure

If you make changes to this document and want the new changes just type:

rm -rf tpd
tangle clojure.pamphlet Makefile >Makefile
make

or you can combine them into one line:

rm -rf tpd && tangle clojure.pamphlet Makefile >Makefile && make

ii CONTENTS

This will destroy the old source, extract the Makefile, and rebuild the system.
On a fast processor this takes about a minute.
Resist the urge to edit the files in the tpd directory. They are only there for the
compiler. Edit this file directly.
You can change where Clojure is built. In the [1737] Makefile there is a line
which defines the root of the directory build. You can change this or override
it on the command line to build Clojure elsewhere.

WHERE=tpd

To build a second copy of Clojure, or to work in some other directory, just type

make WHERE=newplace

Why Bother?
Why bother with such a difficult method of programming? Because worthwhile
programs should live.
Programs live because people maintain them. Maintaining and modifying
code correctly requires that you understand why the program is written as it
is, what the key ideas that make the program work, and why certain, not very
obvious, pieces of code exist. Programmers almost never write this information
down anywhere. Great ideas are invented, the code is written, a man page of
documentation is created, and the job is done.
Well, almost. What does is mean for a program to live? How does a program
survive once the original developers leave the project? There are many sources
of information but almost no source of knowledge. New programmers dont
know what the elders know. In order to live and continue to grow there
has to be a way to transmit this knowledge.
Literate programming is Knuths proposed idea for moving from the world of
ideas to the world of information. This is not simply another documentation
format. This is meant to be Literature. The ideas are presented, the impli-
cations are explored, the tradeoffs are discussed, and the code is motivated,
like characters in a novel.
You are encouraged to write or rewrite sections of this document to improve the
communication with the readers.
But I have to learn latex!. Well, for this document you do. But LATEXis
just more than a document markup language like HTML and it is no harder
to learn. It gives you the added advantage that you have a real language for
publishing real documents. Most books are typeset with this technology and
a lot of conferences and Journals require it. If you can learn Clojure, you can
learn LATEX. If youre a programmer you will always need to continue to learn,
at least until you retire into management.
CONTENTS iii

Having used literate programming for years I have collected some key quotes
that might stimulate your interest.

I believe that the time is ripe for significantly better documentation of pro-
grams, and that we can best achieve this by considering programs to be works
of literature. Hence, my title Literate Programming. Let us change our
traditional attitude to the construction of programs. Instead of imagining
that our main task is to instruct a computer what to do, let us concentrate
on explaining to human beings what we want a computer to do.
Donald Knuth Literate Programming (1984)

Step away from the machine. Literate programming has nothing to do with
tools or style. It has very little to do with programming. One of the hard
transitions to literate programming is literate thinking.
Timothy Daly in Lambda the Ultimate (2010)

The effect of this simple shift of emphasis can be so profound as to change

ones whole approach to programming. Under the literate programming
paradigm, the central activity of programming becomes that of conveying
meaning to other intelligent beings rather than merely convincing the com-
puter to behave in a particular way. It is the difference between performing
and exposing a magic trick.
Ross Williams, FunnelWeb Tutorial Manual

Another thing Ive been enjoying lately is literate programming. Amazingly

it turns out to be faster to write a literate program than an ordinary program
because debugging takes almost no time.
Bill Hart, SAGE Mailing list, May 3, 2010

The conversation is much more direct if the Design Concept per se, rather
than derivative representatives or partial details, is the focus.
Fred Brooks, The Design of Design

We are banning the old notion of literate programming that I used when
developing TEX82 because documentation has proven to be too much of a
pain.
Donald Knuth TUG 2010
iv CONTENTS

Once upon a time I took great care to ensure that TEX82 would be truly
archival so that results obtainable today would produce the same output 50
years from now but that was manifestly foolish. Lets face it, who is going
to care one whit for what I do today after even 5 years have elapsed, let
alone 50. Life is too short to re-read anything anymore in the internet age.
Nothing over 30 months old is trustworthy or interesting.
Donald Knuth TUG 2010
Chapter 1

From The Ground Up

(Kyle Kingsbury)

This is quoted from Kingbury. [King13]

1.1 Clojure from the ground up

1.1.1 Getting set up
When you have a JDK, youll need Leiningen, the Clojure build tool. If youre on
a Linux or OS X computer, these instructions should get you going right away.
If youre on Windows, see the Leiningen page for an installer. If you get stuck,
you might want to start with a primer on command line basics.

mkdir -p ~/bin
cd ~/bin
wget https://raw.github.com/technomancy/leiningen/stable/bin/lein
chmod a+x lein

Leiningen automatically handles installing Clojure, finding libraries from the

internet, and building and running your programs. Well create a new Leiningen
project to play around in:

cd
lein new scratch

This creates a new directory in your homedir, called scratch. If you see command
not found instead, it means the directory /bin isnt registered with your terminal
as a place to search for programs. To fix this, add the line

1
2 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

export PATH="$PATH":~/bin

to the file .bashprofile in your home directory, then run

source ~/.bash_profile.

Re-running

lein new scratch

should work.
Lets enter that directory, and start using Clojure itself:

cd scratch
lein repl

1.1.2 The structure of programs

boa$ lein repl

which responds with

nREPL server started on port 45413

REPL-y 0.2.0
Clojure 1.5.1
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
user=>

This is an interactive Clojure environment called a REPL, for Read, Execute,

Print Loop. Its going to read a program we enter, run that program, and print
the results. REPLs give you quick feedback, so theyre a great way to explore a
program interactively, run tests, and prototype new ideas.
Lets write a simple program. The simplest, in fact. Type nil, and hit enter.

user=> nil
nil

nil is the most basic value in Clojure. It represents emptiness, nothing-doing,

not-a-thing. The absence of information.

user=> true
true
user=> false
false
1.1. CLOJURE FROM THE GROUND UP 3

true and false are a pair of special values called Booleans. They mean exactly
what you think: whether a statement is true or false. true, false, and nil form
the three poles of the Lisp logical system.

user=> 0
0

This is the number zero. Its numeric friends are 1, -47, 1.2e-4, 1/3, and so
on. We might also talk about strings, which are chunks of text surrounded by
double quotes:

user=> "hi there!"

"hi there!"

nil, true, 0, and hi there! are all different types of values; the nouns of pro-
gramming. Just as one could say House. in English, we can write a program
like hello, world and it evaluates to itself: the string hello world. But most
sentences arent just about stating the existence of a thing; they involve action.
We need verbs.

user=> inc
#<core$inc clojure.core$inc@6f7ef41c>

This is a verb called incshort for increment. Specifically, inc is a symbol which
points to a verb: #<core$inc clojure.core$inc@6f7ef41c> just like the word run
is a name for the concept of running.
Theres a key distinction herethat a signifier, a reference, a label, is not the same
as the signified, the referent, the concept itself. If you write the word run on
paper, the ink means nothing by itself. Its just a symbol. But in the mind of a
reader, that symbol takes on meaning; the idea of running.
Unlike the number 0, or the string hi, symbols are references to other values.
when Clojure evaluates a symbol, it looks up that symbols meaning. Look up
inc, and you get #<core$inc clojure.core$inc@6f7ef41c>.
Can we refer to the symbol itself, without looking up its meaning?

user=> inc
inc

Yes. The single quote escapes an expression. It says Rather than evaluating
this text, simply return the text itself, unchanged. Quote a symbol, get a
symbol. Quote a number, get a number. Quote anything, and get it back
exactly as it came in.

user=> 123
4 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

123
user=> "foo"
"foo"
user=> (1 2 3)
(1 2 3)

A new kind of value, surrounded by parentheses: the list. LISP originally stood
for LISt Processing, and lists are still at the core of the language. This list
contains three elements: the numbers 1, 2, and 3. Lists can contain anything:
numbers, strings, even other lists:

user=> (nil "hi")

(nil "hi")

A list containing two elements: the number 1, and a second list. That list
contains two elements: the number 2, and another list. That list contains two
elements: 3, and an empty list.

user=> (1 (2 (3 ())))
(1 (2 (3)))

You could think of this structure as a treewhich is a provocative idea, because

languages are like trees too: sentences are comprised of clauses, which can be
nested, and each clause may have subjects modified by adjectives, and verbs
modified by adverbs, and so on. Lindsay, my best friend, took the dog which
we found together at the pound on fourth street, for a walk with her mother
Michelle.

Took
Lindsay
my best friend
the dog
which we found together
at the pound
on fourth street
for a walk
with her mother
Michelle

But lets try something simpler. Something we know how to talk about. Incre-
ment the number zero. As a tree:

Increment
the number zero

We have a symbol for incrementing, and we know how to write the number zero.
Lets combine them in a list:
1.1. CLOJURE FROM THE GROUND UP 5

clj=> (inc 0)
(inc 0)

A basic sentence. Remember, since its quoted, were talking about the tree,
the text, the expression, by itself. Absent interpretation. If we remove the
single-quote, Clojure will interpret the expression:

user=> (inc 0)
1

Incrementing zero yields one. And if we wanted to increment that value? In-
crement increment the number zero

user=> (inc (inc 0))

2

A sentence in Lisp is a list. It starts with a verb, and is followed by zero or

more objects for that verb to act on. Each part of the list can itself be another
list, in which case that nested list is evaluated first, just like a nested clause in
a sentence. When we type

(inc (inc 0))

Clojure first looks up the meanings for the symbols in the code:

(#<core$inc clojure.core$inc@6f7ef41c>
(#<core$inc clojure.core$inc@6f7ef41c>
0))

Then evaluates the innermost list (inc 0), which becomes the number 1:

(#<core$inc clojure.core$inc@6f7ef41c>
1)

Finally, it evaluates the outer list, incrementing the number 1:

Every list starts with a verb. Parts of a list are evaluated from left to right.
Innermost lists are evaluated before outer lists.

(+ 1 (- 5 2) (+ 3 4))
(+ 1 3 (+ 3 4))
(+ 1 3 7)
11
6 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

Thats it.
The entire grammar of Lisp: the structure for every expression in the language.
We transform expressions by substituting meanings for symbols, and obtain
some result. This is the core of the Lambda Calculus, and it is the theoretical
basis for almost all computer languages. Ruby, Javascript, C, Haskell; all lan-
guages express the text of their programs in different ways, but internally all
construct a tree of expressions. Lisp simply makes it explicit.

1.1.3 Review
We started by learning a few basic nouns: numbers like 5, strings like cat, and
symbols like inc and +. We saw how quoting makes the difference between an
expression itself and the thing it evaluates to. We discovered symbols as names
for other values, just like how words represent concepts in any other language.
Finally, we combined lists to make trees, and used those trees to represent a
program.
With these basic elements of syntax in place, its time to expand our vocabu-
lary with new verbs and nouns; learning to represent more complex values and
transform them in different ways.

1.2 Clojure from the ground up: basic types

Weve learned the basics of Clojures syntax and evaluation model. Now well
take a tour of the basic nouns in the language.

1.2.1 Types
Weve seen a few different values alreadyfor instance, nil, true, false, 1, 2.34, and
meow. Clearly all these things are different values, but some of them seem
more alike than others.
For instance, 1 and 2 are very similar numbers; both can be added, divided,
multiplied, and subtracted. 2.34 is also a number, and acts very much like 1
and 2, but its not quite the same. Its got decimal points. Its not an integer.
And clearly true is not very much like a number. What is true plus one? Or
false divided by 5.3? These questions are poorly defined.
We say that a type is a group of values which work in the same way. Its a
property that some values share, which allows us to organize the world into sets
of similar things. 1 + 1 and 1 + 2 use the same addition, which adds together
integers. Types also help us verify that a program makes sense: that you can
only add together numbers, instead of adding numbers to porcupines.
Types can overlap and intersect each other. Cats are animals, and cats are fuzzy
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 7

too. You could say that a cat is a member (or sometimes instance), of the fuzzy
and animal types. But there are fuzzy things like moss which arent animals,
and animals like alligators that arent fuzzy in the slightest.
Other types completely subsume one another. All tabbies are housecats, and
all housecats are felidae, and all felidae are animals. Everything which is true of
an animal is automatically true of a housecat. Hierarchical types make it easier
to write programs which dont need to know all the specifics of every value; and
conversely, to create new types in terms of others. But they can also get in the
way of the programmer, because not every useful classification (like fuzziness) is
purely hierarchical. Expressing overlapping types in a hierarchy can be tricky.
Every language has a type system; a particular way of organizing nouns into
types, figuring out which verbs make sense on which types, and relating types
to one another. Some languages are strict, and others more relaxed. Some
emphasize hierarchy, and others a more ad-hoc view of the world. We call
Clojures type system strong in that operations on improper types are simply
not allowed: the program will explode if asked to subtract a dandelion. We
also say that Clojures types are dynamic because they are enforced when the
program is run, instead of when the program is first read by the computer.
Well learn more about the formal relationships between types later, but for now,
keep this in the back of your head. Itll start to hook in to other concepts later.

1.2.2 Integers
Lets find the type of the number 3:

user=> (type 3)
java.lang.Long

So 3 is a java.lang.Long, or a Long, for short. Because Clojure is built on top

of Java, many of its types are plain old Java types.
Longs, internally, are represented as a group of sixty-four binary digits (ones
and zeroes), written down in a particular pattern called signed twos complement
representation. You dont need to worry about the specificsthere are only two
things to remember about longs. First, longs use one bit to store the sign:
whether the number is positive or negative. Second, the other 63 bits store the
size of the number. That means the biggest number you can represent with a
long is 263 - 1 (the minus one is because of the number 0), and the smallest
long is -263.
How big is 263 - 1?

user=> Long/MAX_VALUE
9223372036854775807
8 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

Thats a reasonably big number. Most of the time, you wont need anything
bigger, but what if you did? What happens if you add one to the biggest Long?

user=> (inc Long/MAX_VALUE)

ArithmeticException integer overflow clojure.lang.Numbers.throwIntOverflow
(Numbers.java:1388)

An error occurs! This is Clojure telling us that something went wrong. The
type of error was an ArithmeticException, and its message was integer overflow,
meaning this type of number cant hold a number that big. The error came from
a specific place in the source code of the program: Numbers.java, on line 1388.
Thats a part of the Clojure source code. Later, well learn more about how to
unravel error messages and find out what went wrong.
The important thing is that Clojures type system protected us from doing some-
thing dangerous; instead of returning a corrupt value, it aborted evaluation and
returned an error.
If you do need to talk about really big numbers, you can use a BigInt: an
arbitrary-precision integer. Lets convert the biggest Long into a BigInt, then
increment it:

user=> (inc (bigint Long/MAX_VALUE)) 9223372036854775808N

Notice the N at the end? Thats how Clojure writes arbitrary-precision integers.

user=> (type 5N)

clojure.lang.BigInt

There are also smaller numbers.

user=> (type (int 0))

java.lang.Integer
user=> (type (short 0))
java.lang.Short
user=> (type (byte 0))
java.lang.Byte

Integers are half the size of Longs; they store values in 32 bits. Shorts are 16
bits, and Bytes are 8. That means their biggest values are 2311 , 2151 , and
271 , respectively.

user=> Integer/MAX_VALUE
2147483647
user=> Short/MAX_VALUE
32767
user=> Byte/MAX_VALUE
127
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 9

1.2.3 Fractional numbers

To represent numbers between integers, we often use floating-point numbers,
which can represent small numbers with fine precision, and large numbers with
coarse precision. Floats use 32 bits, and Doubles use 64. Doubles are the default
in Clojure.

user=> (type 1.23)

java.lang.Double
user=> (type (float 1.23))
java.lang.Float

Floating point math is complicated, and we wont get bogged down in the details
just yet. The important thing to know is floats and doubles are approximations.
There are limits to their correctness:

user=> 0.99999999999999999
1.0

To represent fractions exactly, we can use the ratio type:

user=> (type 1/3)

clojure.lang.Ratio

1.2.4 Mathematical operations

The exact behavior of mathematical operations in Clojure depends on their
types. In general, though, Clojure aims to preserve information. Adding two
longs returns a long; adding a double and a long returns a double.

user=> (+ 1 2)
3
user=> (+ 1 2.0)
3.0

3 and 3.0 are not the same number; one is a long, and the other a double. But
for most purposes, theyre equivalent, and Clojure will tell you so:

user=> (= 3 3.0)
false
user=> (== 3 3.0)
true

= asks whether all the things that follow are equal. Since floats are approxi-
mations, = considers them different from integers. == also compares things,
10 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

but a little more loosely: it considers integers equivalent to their floating-point

representations.
We can also subtract with -, multiply with *, and divide with /.

user=> (- 3 1)
2
user=> (* 1.5 3)
4.5
user=> (/ 1 2)
1/2

Putting the verb first in each list allows us to add or multiply more than one
number in the same step:

user=> (+ 1 2 3)
6
user=> (* 2 3 1/5)
6/5

Subtraction with more than 2 numbers subtracts all later numbers from the
first. Division divides the first number by all the rest.

user=> (- 5 1 1 1)
2
user=> (/ 24 2 3)
4

By extension, we can define useful interpretations for numeric operations with

just a single number:

user=> (+ 2)
2
user=> (- 2)
-2
user=> (* 4)
4
user=> (/ 4)
1/4

We can also add or multiply a list of no numbers at all, obtaining the addi-
tive and multiplicative identities, respectively. This might seem odd, especially
coming from other languages, but well see later that these generalizations make
it easier to reason about higher-level numeric operations.

user=> (+)
0
user=> (*)
1
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 11

Often, we want to ask which number is bigger, or if one number falls between
two others. <= means less than or equal to, and asserts that all following values
are in order from smallest to biggest.

user=> (<= 1 2 3)
true
user=> (<= 1 3 2)
false

< means strictly less than, and works just like <=, except that no two values
may be equal.

user=> (<= 1 1 2)
true
user=> (< 1 1 2)
false

Their friends > and >= mean greater than and greater than or equal to, re-
spectively, and assert that numbers are in descending order.

user=> (> 3 2 1)
true
user=> (> 1 2 3)
false

Also commonly used are inc and dec, which add and subtract one to a number,
respectively:

user=> (inc 5)
6
user=> (dec 5)
4

One final note: equality tests can take more than 2 numbers as well.

user=> (= 2 2 2)
true
user=> (= 2 2 3)
false

1.2.5 Strings
We saw that strings are text, surrounded by double quotes, like foo. Strings
in Clojure are, like Longs, Doubles, and company, backed by a Java type:

user=> (type "cat")

java.lang.String
12 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

We can make almost anything into a string with str. Strings, symbols, numbers,
booleans; every value in Clojure has a string representation. Note that nils string
representation is ; an empty string.

user=> (str "cat")

"cat"
user=> (str cat)
"cat"
user=> (str 1)
"1"
user=> (str true)
"true"
user=> (str (1 2 3))
"(1 2 3)"
user=> (str nil)
""

str can also combine things together into a single string, which we call concate-
nation.

user=> (str "meow " 3 " times")

"meow 3 times"

To look for patterns in text, we can use a regular expression, which is a tiny
language for describing particular arrangements of text. re-find and re-matches
look for occurrences of a regular expression in a string. To find a cat:

user=> (re-find #"cat" "mystic cat mouse")

"cat"
user=> (re-find #"cat" "only dogs here")
nil

That #"..." is Clojures way of writing a regular expression.

With re-matches, you can extract particular parts of a string which match an
expression. Here we find two strings, separated by a :. The parentheses mean
that the regular expression should capture that part of the match. We get
back a list containing the part of the string that matched the first parentheses,
followed by the part that matched the second parentheses.

user=> (rest (re-matches #"(.+):(.+)" "mouse:treat"))

("mouse" "treat")

Regular expressions are a powerful tool for searching and matching text, es-
pecially when working with data files. Since regexes work the same in most
languages, you can use any guide online to learn more. Its not something you
have to master right away; just learn specific tricks as you find you need them.
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 13

1.2.6 Booleans and logic

Everything in Clojure has a sort of charge, a truth value, sometimes called
truthiness. true is positive and false is negative. nil is negative, too.

user=> (boolean true)

true
user=> (boolean false)
false
user=> (boolean nil)
false

Every other value in Clojure is positive.

user=> (boolean 0)
true
user=> (boolean 1)
true
user=> (boolean "hi there")
true
user=> (boolean str)
true

If youre coming from a C-inspired language, where 0 is considered false, this

might be a bit surprising. Likewise, in much of POSIX, 0 is considered success
and nonzero values are failures. Lisp allows no such confusion: the only negative
values are false and nil.
We can reason about truth values using and, or, and not. and returns the
first negative value, or the last value if all are truthy.

user=> (and true false true)

false
user=> (and true true true)
true
user=> (and 1 2 3)
3

Similarly, or returns the first positive value.

user=> (or false 2 3)

2
user=> (or false nil)
nil

And not inverts the logical sense of a value:

14 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

user=> (not 2)
false
user=> (not nil)
true

Well learn more about Boolean logic when we start talking about control flow;
the way we alter evaluation of a program and express ideas like if Im a cat, then
meow incessantly.

1.2.7 Symbols
We saw symbols in the previous chapter; theyre bare strings of characters, like
foo or +.

user=> (class str)

clojure.lang.Symbol

Every symbol actually has two names: one, a short name, is used to refer
to things locally. Another is the fully qualified name, which is used to refer
unambiguously to a symbol from anywhere. If I were a symbol, my name would
be Kyle, and my full name Kyle Kingsbury.
Symbol names are separated with a /. For instance, the symbol str actually
comes from a family called clojure.core, which means that its full name is clo-
jure.core/str

user=> (= str clojure.core/str)

true
user=> (name clojure.core/str)
"str"

When we talked about the maximum size of an integer, that was a fully-qualified
symbol, too.

user=> (type Integer/MAX_VALUE)

clojure.lang.Symbol

The job of symbols is to refer to things, to point to other values. When eval-
uating a program, symbols are looked up and replaced by their corresponding
values. Thats not the only use of symbols, but its the most common.

1.2.8 Keywords
Closely related to symbols and strings are keywords, which begin with a :.
Keywords are like strings in that theyre made up of text, but are specifically
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 15

intended for use as labels or identifiers. These arent labels in the sense of
symbols: keywords arent replaced by any other value. Theyre just names, by
themselves.

user=> (type :cat)

clojure.lang.Keyword
user=> (str :cat)
":cat"
user=> (name :cat)
"cat"

As labels, keywords are most useful when paired with other values in a collection,
like a map. Well come back to keywords shortly.

1.2.9 Lists
A collection is a group of values. Its a container which provides some structure,
some framework, for the things that it holds. We say that a collection con-
tains elements, or members. We saw one kind of collectiona listin the previous
chapter.

user=> (1 2 3)
(1 2 3)
user=> (type (1 2 3))
clojure.lang.PersistentList

Remember, we quote lists with a to prevent them from being evaluated. You
can also construct a list using list:

user=> (list 1 2 3)
(1 2 3)

Lists are comparable just like every other value:

user=> (= (list 1 2) (list 1 2))

true

You can modify a list by conjoining an element onto it:

user=> (conj (1 2 3) 4)
(4 1 2 3)

We added 4 to the listbut it appeared at the front. Why? Internally, lists are
stored as a chain of values: each link in the chain is a tiny box which holds the
value and a connection to the next link. This data structure, called a linked
list, offers immediate access to the first element.
16 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

user=> (first (list 1 2 3))

1

But getting to the second element requires an extra hop down the chain

user=> (second (list 1 2 3))

2

and the third element a hop after that, and so on.

user=> (nth (list 1 2 3) 2)

3

nth gets the element of an ordered collection at a particular index. The first
element is index 0, the second is index 1, and so on.
This means that lists are well-suited for small collections, or collections which
are read in linear order, but are slow when you want to get arbitrary elements
from later in the list. For fast access to every element, we use a vector.

1.2.10 Vectors
Vectors are surrounded by square brackets, just like lists are surrounded by
parentheses. Because vectors arent evaluated like lists are, theres no need to
quote them:

user=> [1 2 3]
[1 2 3]
user=> (type [1 2 3])
clojure.lang.PersistentVector

You can also create vectors with vector, or change other structures into vectors
with vec:

user=> (vector 1 2 3)
[1 2 3]
user=> (vec (list 1 2 3))
[1 2 3]

conj on a vector adds to the end, not the start:

user=> (conj [1 2 3] 4) [1 2 3 4]

Our friends first, second, and nth work here too; but unlike lists, nth is fast on
vectors. Thats because internally, vectors are represented as a very broad tree
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 17

of elements, where each part of the tree branches into 32 smaller trees. Even
very large vectors are only a few layers deep, which means getting to elements
only takes a few hops.
In addition to first, youll often want to get the remaining elements in a collection.
There are two ways to do this:

user=> (rest [1 2 3])

(2 3)
user=> (next [1 2 3])
(2 3)

rest and next both return everything but the first element. They differ only
by what happens when there are no remaining elements:

user=> (rest [1])

()
user=> (next [1])
nil

rest returns logical true, next returns logical false. Each has their uses, but
in almost every case theyre equivalentI interchange them freely.
We can get the final element of any collection with last:

user=> (last [1 2 3])

3

And figure out how big the vector is with count:

user=> (count [1 2 3])

3

Because vectors are intended for looking up elements by index, we can also use
them directly as verbs:

user=> ([:a :b :c] 1)

:b

So we took the vector containing three keywords, and asked Whats the element
at index 1? Index 1 is the second element, so this evaluates to :b.
Finally, note that vectors and lists containing the same elements are considered
equal in Clojure:

user=> (= (1 2 3) [1 2 3])
true

In almost all contexts, you can consider vectors, lists, and other sequences as
interchangeable. They only differ in their performance characteristics, and in a
few data-structure-specific operations.
18 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

1.2.11 Sets
Sometimes you want an unordered collection of values; especially when you plan
to ask questions like does the collection have the number 3 in it? Clojure, like
most languages, calls these collections sets.

user=> #{:a :b :c}

#{:a :c :b}

Sets are surrounded by #{...}. Notice that though we gave the elements :a, :b,
and :c, they came out in a different order. In general, the order of sets can shift
at any time. If you want a particular order, you can ask for it as a list or vector:

user=> (vec #{:a :b :c})

[:a :c :b]

Or ask for the elements in sorted order:

user=> (sort #{:a :b :c})

(:a :b :c)

conj on a set adds an element:

user=> (conj #{:a :b :c} :d)

#{:a :c :b :d}
user=> (conj #{:a :b :c} :a)
#{:a :c :b}

Sets never contain an element more than once, so conjing an element which is
already present does nothing. Conversely, one removes elements with disj:

user=> (disj #{"hornet" "hummingbird"} "hummingbird")

#{"hornet"}

The most common operation with a set is to check whether something is inside
it. For this we use contains?.

user=> (contains? #{1 2 3} 3)

true
user=> (contains? #{1 2 3} 5)
false

Like vectors, you can use the set itself as a verb. Unlike contains?, this expression
returns the element itself (if it was present), or nil.
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 19

user=> (#{1 2 3} 3)
3
user=> (#{1 2 3} 4)
nil

You can make a set out of any other collection with set.

user=> (set [:a :b :c])

#{:a :c :b}

1.2.12 Maps
The last collection on our tour is the map: a data structure which associates
keys with values. In a dictionary, the keys are words and the definitions are
the values. In a library, keys are call signs, and the books are values. Maps
are indexes for looking things up, and for representing different pieces of named
information together.

user=> {:name "spook" :weight 2 :color "black"}

{:weight 2, :name "spook", :color "black"}

Maps are surrounded by braces {...}, filled by alternating keys and values. In
this map, the three keys are :name, :color, and :weight, and their values are
spook, black, and 2, respectively. We can look up the corresponding value
for a key with get:

user=> (get {"cat" "meow" "dog" "woof"} "cat")

"meow"
user=> (get {:a 1 :b 2} :c)
nil

get can also take a default value to return instead of nil, if the key doesnt exist
in that map.

user=> (get {:glinda :good} :wicked :not-here)

:not-here

Since lookups are so important for maps, we can use a map as a verb directly:

user=> ({"amlodipine" 12 "ibuprofin" 50} "ibuprofin")

50

And conversely, keywords can also be used as verbs, which look themselves up
in maps:
20 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)

user=> (:raccoon {:weasel "queen" :raccoon "king"})

"king"

You can add a value for a given key to a map with assoc.

user=> (assoc {:bolts 1088} :camshafts 3)

{:camshafts 3 :bolts 1088}
user=> (assoc {:camshafts 3} :camshafts 2)
{:camshafts 2}

assoc adds keys if they arent present, and replaces values if theyre already
there. If you associate a value onto nil, it creates a new map.

user=> (assoc nil 5 2)

{5 2}

You can combine maps together using merge, which yields a map containing
all the elements of all given maps, preferring the values from later ones.

user=> (merge {:a 1 :b 2} {:b 3 :c 4})

{:c 4, :a 1, :b 3}

Finally, to remove a value, use dissoc.

user=> (dissoc {:potatoes 5 :mushrooms 2} :mushrooms)

{:potatoes 5}

1.2.13 Putting it all together

All these collections and types can be combined freely. As software engineers,
we model the world by creating a particular representation of the problem in
the program. Having a rich set of values at our disposal allows us to talk about
complex problems. We might describe a person:

{:name "Amelia Earhart"

:birth 1897
:death 1939
:awards {"US" #{"Distinguished Flying Cross"
"National Womens Hall of Fame"}
"World" #{"Altitude record for Autogyro"
"First to cross Atlantic twice"}}}

Or a recipe:
1.2. CLOJURE FROM THE GROUND UP: BASIC TYPES 21

{:title "Chocolate chip cookies"

:ingredients {"flour" [(+ 2 1/4) :cup]
"baking soda" [1 :teaspoon]
"salt" [1 :teaspoon]
"butter" [1 :cup]
"sugar" [3/4 :cup]
"brown sugar" [3/4 :cup]
"vanilla" [1 :teaspoon]
"eggs" 2
"chocolate chips" [12 :ounce]}

Or the Gini coefficients of nations, as measured over time:

{"Afghanistan" {2008 27.8}

"Indonesia" {2008 34.1 2010 35.6 2011 38.1}
"Uruguay" {2008 46.3 2009 46.3 2010 45.3}}

In Clojure, we compose data structures to form more complex values; to talk

about bigger ideas. We use operations like first, nth, get, and contains? to
extract specific information from these structures, and modify them using conj,
disj, assoc, dissoc, and so on.
We started this chapter with a discussion of types: groups of similar objects
which obey the same rules. We learned that bigints, longs, ints, shorts, and
bytes are all integers, that doubles and floats are approximations to decimal
numbers, and that ratios represent fractions exactly. We learned the differences
between strings for text, symbols as references, and keywords as short labels.
Finally, we learned how to compose, alter, and inspect collections of elements.
Armed with the basic nouns of Clojure, were ready to write a broad array of
programs.
Id like to conclude this tour with one last type of value. Weve inspected dozens
of types so farbut what what happens when you turn the camera on itself?

user=> (type type)

clojure.core$type

What is this type thing, exactly? What are these verbs weve been learning,
and where do they come from? This is the central question of chapter three:
functions.
22 CHAPTER 1. FROM THE GROUND UP (KYLE KINGSBURY)
Chapter 2

From ideas to
implementation

2.1 Starting from Java

2.2 Extends and Implements

2.3 Understanding Persistent Vectors (Jean Lo-

range)
This is quoted from Lorange [L13].
You may or may not heard about Clojures persistent vectors. It is a data
structure invented by Rich Hickey (influenced by Phil Bagwells paper on Ideal
Hash Trees) for Clojure, which gives practically O(1) runtime for insert, update,
lookups and subvec. As they are persistent, every modification creates a new
vector instead of changing the old one.
So, how do they work? Ill try to explain them through a series of blogposts, in
which we look at manageable parts each time. It will be a detailed explanation,
with all the different oddities around the implementation as well. Consequently,
this blog series may not be the perfect fit for people who want a summary on
how persistent vectors work.
For today, well have a look at a naive first attempt, and will cover updates,
insertion and popping (removal at the end).
Note that this blogpost does not represent how PersistentVector is implemented:
There are some speed optimizations, solutions for transients and other details
which we will cover later. However, this serves as a basis for understanding how

23
24 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

they work, and the general idea behind the vector implementation.
The Basic Idea
Mutable vectors and ArrayLists are generally just arrays which grows and
shrinks when needed. This works great when you want mutability, but is a
big problem when you want persistence. You get slow modification operations
because youll have to copy the whole array all the time, and it will use a lot of
memory. It would be ideal to somehow avoid redundancy as much as possible
without losing performance when looking up values, along with fast operations.
That is exactly what Clojures persistent vector does, and it is done through
balanced, ordered trees.
The idea is to implement a structure which is similar to a binary tree. The
only difference is that the interior nodes in the tree have a reference to at most
two subnodes, and does not contain any elements themselves. The leaf nodes
contain at most two elements. The elements are in order, which means that the
first element is the first element in the leftmost leaf, and the last element is the
rightmost element in the rightmost leaf. For now, we require that all leaf nodes
are at the same depth. As an example, take a look at the tree below: It has the
integers 0 to 8 in it, where 0 is the first element and 8 the last. The number 9
is the vector size:

vector size 9
vector pointer
node
leaf node

element

0 1 2 3 4 5 6 7 8
Visualization of a vector with 9 elements in it.

If we wanted to add a new element to the end of this vector and we were in the
mutable world, we would insert 9 in the rightmost leaf node:
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 25

10

0 1 2 3 4 5 6 7 8 9
Visualization of a vector with 10 elements in it.

But heres the issue: We cannot do that if we want to be persistent. And this
would obviously not work if we wanted to update an element! We would need
to copy the whole structure, or at least parts of it.
To minimize copying while retaining full persistence, we perform path copying:
We copy all nodes on the path down to the value were about to update or
insert, and replace the value with the new one when were at the bottom. A
result of multiple insertions is shown below. Here, the vector with 7 elements
share structure with a vector with 10 elements:

10

0 1 2 3 4 5 6 6 7 8 9
A visualization of two vectors, which use structural sharing.

The pink coloured nodes and edges are shared between the vectors, whereas the
brown and blue are separate. Other vectors not visualized may also share nodes
with these vectors. Update
The easiest modification operator to understand would be updates/replacing
values in a vector, so we will explain how updating works first. In Clojure,
thats an assoc modification, or an update-in.
To update an element, we would have to walk the tree down to the leaf node
where the element is place. While we walk down, we copy the nodes on our path
to ensure persistence. When weve gotten down to the leaf node, we copy it and
26 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

replace the value we wanted to replace with the new value. We then return the
new vector with the modified path.
As an example, lets assume we perform an assoc on the vector with the elements
from 0 to 8, like this:

(def brown [0 1 2 3 4 5 6 7 8])

(def blue (assoc brown 5 beef))

The internal structures, where the blue one has the copied path, is shown below:

9 9

0 1 2 3 4 5 4 beef 6 7 8
Two vectors, where weve updated a value.

Given that we have a way to know which node to go down, this seems easy
enough. Well go through how we find the path to a specific index in a later
part of this series.

2.3.1 Insertion
Insertion is not too much different from an update, except that we have some
edge cases where we have to generate nodes in order to fit in a value. We
essentially have three cases:
There is room for a new value in the rightmost leaf node. There is space in
the root, but not in the rightmost leaf node. There is not enough space in the
current root.
Well go through them all, as their solutions are not that difficult to grasp.

Just Like Assoc

Whenever there is enough space in the rightmost leaf node, we can just do as we
do when we perform an assoc: We just copy the path, and at the newly created
leaf node, we put in the value to the right of the rightmost element.
As an example, heres how we would do (conj [0 1 2 3 4] 5), and the internal
structures from doing so. black is the old:
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 27

5 6

0 1 2 3 4 4 5
Insertion with enough space in the leaf node.

Thats it. Theres no magic, just path copying and insertion in the leaf node.

Generate Nodes When You Need Them

So, what do we do when theres not enough space in the leftmost leaf node?
Luckily, well never end up in a position where we find out that were in the
wrong leaf node: We will always take the right path down to the leaf node.
Instead, we will realize that the node were trying to go down to doesnt yet
exist (the pointer is null). When a node doesnt exist, we generate one and set
that one as the copied node.

12 13

0 1 2 3 4 5 6 7 8 9 a b c

Insertion where we generate new nodes instead of copying.

In the figure above, the black nodes represent original nodes.

Root Overflow

The last case is the root overflow case. This happens when there isnt more
space in the tree with the current root node.
Its not that difficult to understand how we would solve this: We make a new
root node, and set the old root as the first child of the new root. From there
on, we perform the node generating, just as we did in the previous solution.
28 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

0 1 2 3 4 5 6 7 8
Insertion where we generate a new root.

One thing is to solve the problem, but detecting when it happens is also im-
portant. Luckily, this is also rather easy. When we have a two-way branching
vector, this happens when the old vectors size is a power of two. Generally
speaking, an n-way branching vector will have an overflow when the size is a
power of n.

2.3.2 Popping
The solutions for popping (removing the last element) isnt that difficult to
grasp either. Popping similar to inserting in that there are three cases:
The rightmost leaf node contains more than one element. The rightmost leaf
node contains exactly one element (zero after popping). The root node contains
exactly one element after popping.
Essentially, these are all ways of reverting 1, 2 and 3 in the previous section,
neither of which are extremely complex.

Dissoc to the Rescue

Again, we have a case where we can just do as we do when we update a structure:

We copy the path down to the rightmost leaf node, and remove the rightmost
element in the copied leaf node. As long as there is at least one element left in
the new leaf node, we dont have to do any magic.

6 5

0 1 2 3 4 5 4
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 29

Popping a value from a vector with more than one element in the
rightmost leaf node.

Keep in mind that popping multiple times on a vector will not yield identical
vectors: They are equal, but they dont share the root. For instance,

(def brown [0 1 2 3 4 5])

(def blue (pop brown))
(def green (pop brown))

will result in the following internal structure.

6 5 4

0 1 2 3 4 5 4 5
Performing pops on the same vector.

Removing Empty Nodes

Whenever we have a leaf node with only a single node, we have a different case.
We would like to avoid empty nodes in our tree at all cost. Therefore, whenever
we have an empty node, instead of returning it, we return null instead. The
parent node will then contain a null pointer, instead of a pointer to an empty
node:

6 7

0 1 2 3 4 5 6
Popping and removing a leaf node.

Here, the brown vector is the original, whereas the blue one is the popped one.
Unfortunately, it is not as easy as to just remove leaf nodes. You see, if we
return a null pointer to a node, which originally only had one child, we must
convert that one into a null pointer which we send back: The results of emptying
30 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

a node propagates upwards. This is a bit tricky to get right, but essentially it
works by looking at the new child, check if it is null and is supposed to be placed
at index 0, and return null if thats the case.
If this was implemented in Clojure, it may look something like this recursive
function:

(defn node-pop [idx depth cur-node]

(let [sub-idx (calculate-subindex idx depth)]
(if (leaf-node? depth)
(if (== sub-idx 0)
nil
(copy-and-remove cur-node sub-idx))
; not leaf node
(let [child (node-pop idx (- depth 1)
(child-at cur-node sub-idx))]
(if (nil? child)
(if (== sub-idx 0)
nil
(copy-and-remove cur-node sub-idx))
(copy-and-replace cur-node sub-idx child))))))

When such a function has been implemented, node removal has been taken care
of completely. As an example, see the graph below. Here, the popped (blue)
vector has removed two nodes: The leaf node containing c and its parent.

13 12

0 1 2 3 4 5 6 7 8 9 10 11 12
Popping and removing multiple nodes.

Root Killing

We have now covered all cases, except for one. With the current implementation,
we would get the following result if we popped a vector with nine elements:
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 31

8 9

0 1 2 3 4 5 6 7 8
Popping with bad root handling.

Thats right, wed have a root with a single pointer to a child node. Pretty
useless, as we would always move down into the child when we lookup or assoc
values, and inserting values would create a new root. What we would like to do
is to get rid of it.
This is possibly the easiest thing in this blogpost to actually do: After we have
finished popping, check if the root node contains only a single child (check that
the second child is null, for instance). If thats the case, and the root node is
not a leaf node, we can just replace the root node with its child.
The result is, as expected, a new vector (blue) with the first child of the original
vectors root node as root:

0 1 2 3 4 5 6 7 8
Popping with proper root handling.

2.3.3 O(1) != O(log n)

Some people out there are probably wondering how this can be said to be O(1)
at all. In fact, with only two children per node, this is O(log2 n), which is
(relatively) far from O(1).
However, no one has said that we have to only have two children per node (often
referred to as the branching factor). Clojure has 32 per node, which in result
turns into very shallow trees. In fact, the trees will be at most 6 nodes deep if
you have less than 1 billion elements in the vector. You need about 35 billion
elements to get up to a depth of 8 nodes. At that point, I would believe memory
consumption is a more serious issue.
32 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

To actually see the difference: Here is a 4-way branching tree with 14 elements,
which only is 2 levels deep. If you scroll up a bit, youll see a figure with two
vectors containing 13 and 12 elements, respectively. With two-way branching,
this is already 4 levels deep, double the height as this one.

14

0 1 2 3 4 5 6 7 8 9 a b c d
A 4-way branching vector.

As a result of the incredibly shallow trees, we tend to call modifications and

lookup of Clojure vectors to be effectively constant time, although they in
theory are O(log32 n). People with basic knowledge in big O notation know
that this is exactly the same as O(log n), but for marketing reasons people like
to add in the constant factor.
To understand how we pick the right branch, I think its good to give the proper
name of the structure and explain why it is named that. It sounds a bit weird
to explain branching through the name of a data structure, but it makes sense
when you consider that such a name may describe how it works. Naming
A more formal name for Clojures persistent vector structure is persistent bit-
partitioned vector trie. What I explained in the previous post was how persistent
digit-partitioned vector tries work. Dont worry, a bit-partitioned one is just an
optimized digit-partitioned trie, and in the previous post there is nothing differ-
ent about them. In this one, there is a small difference related to performance.
It has otherwise no practical differences.
I guess many of you dont know all of those words I mentioned in the above
paragraph, so lets describe them, one by one.

2.3.4 Persistence
In the last post, I used the word persistent. I said we want to be persistent,
but didnt really explain what persistence itself really means.
A persistent data structure doesnt modify itself: Strictly speaking they dont
have to be immutable internally, just have to be perceived as such. Whenever
you do updates, inserts and removals on a persistent data structure, you
get a new data structure back. The old version will always be consistent, and
whenever given some input in, it will always spit out the same output.
When we talk about a fully persistent data structure, all versions of a structure
should be updateable, meaning that all possible operations you can do on a
version can be performed on another. In early functional data structure time,
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 33

it was common to cheat with the structures and make the older versions
decay over time by mutating the internals, making them slower and slower
compared to the newer versions. However, Rich Hickey decided that all the
versions of Clojures persistent structures should have the same performance
guarantees, regardless of which version of the structure you are using.
Vector
A vector is a one-dimensional growable array. C++s std::vector and Javas
java.util.ArrayList are examples of mutable implementations. Theres not much
more to it than that, really. A vector trie is a trie which represents a vector. It
doesnt have to be persistent, but in our case, it is.
Trie
Tries are a specific type of trees, and I think its best to show the actual difference
by explaining the more known trees first.
In RB-trees and most other binary trees, mappings or elements are contained
in the interior nodes. Picking the right branch is done by comparing the ele-
ment/key at the current node: If the element is lower than the node element,
we branch left, and if it is higher, we branch right. Leaves are usually null
pointers/nil, and doesnt contain anything.

13

8 17

1 11 15 25

nil 6 22 27
nil nil nil nil

nil nil nil nil nil nil

A RB-tree

The RB-tree above is taken from Wikipedias article on RB-trees. Im not going
to explain how those work in detail, but let us take a tiny example on how we
check if 22 is contained in the RB-tree:

We start at the root, 13, and compare it with 22.

As 13 < 22, we go right.
The new node has 17 in it, and compare it with 22.
As 17 < 22, we still go right.
The next node weve walked into is 25.
As 25 > 22, we go left.
The next node is 22, so we know that 22 is contained in the tree.

If you want a good explanation on how RB-trees work, I would recommend

Julienne Walkers Red Black Tree Tutorial.
34 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

A trie, on the other hand, has all the values stored in its leaves. Picking the
right branch is done by using parts of the key as a lookup. Consequently, a trie
may have more than two branches. In our case, we may have as many as 32!

a b

a c a b

5 7 8 3

A trie

An example of a general trie is illustrated in the figure above. That specific trie
is a map: It takes a string of length two, and returns an integer represented by
that string if it exists in the trie. ac has the value 7, whereas ba has the value
8. Heres how the trie works:
For strings, we split the string into characters. We then take the first character,
find the edge represented by this value, and walk down that edge. If there is
no edge for that value, we stop, as it is not contained in the trie. If not, we
continue with the second character, and so on. Finally, when we are done, we
return the value if it exists.
As an example, consider ac. We do as follows:

We split ac up into [a, c], and start at the root node.

We check if there is an edge in the node for a,
and there is: We follow it.
We check if there is an edge in the node for c,
and there is. We follow that as well.
We have no more characters left, which means the current node contains
our value, 7. We therefore return 7.

Clojures Persistent Vector is a trie where the indices of elements are used as
keys. But, as you may guess, we must split up the index integers in some way.
To split up integers, we either use digit partitioning or its faster sibling, bit
partitioning.
Digit Partitioning
Digit partitioning means that we split up the key into digits, which we then use
as a basis for populating a trie. For instance, we can split up the key 9128 to
[9, 1, 2, 8], and put an element into a trie based on that. We may have to pad
with zeroes at the front of the list, if the depth of the trie is larger than the size
of the list.
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 35

We can also use whatever base we would like, not just base 10. We would then
have to convert the key to the base we wanted to use, and use the digits from
the conversion. As an example, consider 9128 yet again. 9128 is 35420 in base
7, so we would have to use the list [3, 5, 4, 2, 0] for lookup/insertion in the trie.

6
6 5 6
5 4 5
6 6 4 3 4
5 5 3 2 3
9457 4 4 2 1 2
3 3 1 0 1
2 2 0 0
1 1
0 0

Digit 3 5 4 2 0
Visualization of the 35420 lookup.

The trie (laying sideways, without the edges and nodes were not walking) above
shows how we traverse a digit-partitioned trie: We pick the most significant
digit, in this case 3, and walk that specific branch. We continue with the second
most significant digit in the same manner, until we have no digits left. When
weve walked the last branch, the object were standing with the first object in
the rightmost array in this case is the object we wanted to look up.
Implementing such a lookup scheme is not too hard, if you know how to find the
digits. Heres a Java version where everything not related to lookup is stripped
away:

public class DigitTrie {

public static final int RADIX = 7;

// Array of objects. Can itself contain an array of objects.

Object[] root;
// The maximal size/length of a child node (1 if leaf node)
int rDepth; // equivalent to RADIX ** (depth - 1)

public Object lookup(int key) {

Object[] node = this.root;

// perform branching on internal nodes here

for (int size = this.rDepth; size > 1; size /= RADIX) {
node = (Object[]) node[(key / size) % RADIX];
// If node may not exist, check if it is null here
}

// Last element is the value we want to lookup, return it.

36 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

return node[key % RADIX];

}
}

The rDepth value represents the maximal size of a child of the root node: A
number with n digits will have n to the power of RADIX possible values, and
we must be able to put them all in the trie without having collisions.
In the for loop within the lookup method, the value size represents the maximal
size a child of the current node can have. For each child we go over, that size is
decremented by the branching factor, i.e. the radix or base of the digit trie.
The reason were performing a modulo operation on the result is to ignore the
more significant digits digits weve branched on earlier. We could potentially
remove the higher digit from the key every time we branch into a child, but the
code would be a tiny bit more complicated in that case. Bit Partitioning
Digit-partitioned tries would generally have to do a couple of integer divisions
and modulo operations. Doing this is on every branch we must take is a bit
time consuming. We would therefore like to speed this part up if it is possible.
So, as you may guess, bit-partitioned tries are a subset of the digit-partitioned
tries. All digit-partitioned tries in a base which is a power of two (2, 4, 8, 16,
32, etc) can be turned into bit-partitioned ones. With some knowledge of bit
manipulation, we can remove those costly arithmetic operations.
Conceptually, it works in the same way as digit partitioning does. However,
instead of splitting the key into digits, we split it into chunks of bits with some
predefined size. For 32-way branching tries, we need 5 bits in each part, and for
4-way branching tries, we need 2. In general, we need as many bits as the size
of our exponent.
So, why is this faster? By using bit tricks, we can get rid of both integer division
and modulo. If power is two to the power of n, we can use that

x / power == x >>> n and

x % power == x & (power - 1).

These formulas are just identities related to how integers are represented inter-
nally, namely as sequences of bits.
If we use this result and combine it with the previous implementation, we end
up with the following code:

public class BitTrie {

public static final int BITS = 5,
WIDTH = 1 << BITS, // 2^5 = 32
MASK = WIDTH - 1; // 31, or 0x1f

// Array of objects. Can itself contain an array of objects.

Object[] root;
2.3. UNDERSTANDING PERSISTENT VECTORS (JEAN LORANGE) 37

// BITS times (the depth of this trie minus one).

int shift;

public Object lookup(int key) {

Object[] node = this.root;

// perform branching on internal nodes here

for (int level = this.shift; level > 0; level -= BITS) {
node = (Object[]) node[(key >>> level) & MASK];
// If node may not exist, check if it is null here
}

// Last element is the value we want to lookup, return it.

return node[key & MASK];
}
}

This is more or less exactly what Clojures implementation is doing! See these
lines of the Clojure code to verify it; The only difference is that it performs
boundary checks and a tail check as well.
The important thing to note here is that weve not only changed the operators,
but weve also replaced the rDepth value with a shift value. Instead of storing
the whole value, were only storing the exponent. This makes us able to use
bitshifting on the key, which we use in the (key level) part. The other
parts should be fairly straightforward to understand, given that one knows bit
operations well. However, lets take an example for the ones unfamiliar with such
tricks. The explanation is quite thorough, so feel to skip parts you understand.
Say we have only have 2 bit partitioning (4-way branching) instead of 5 bits
(32-way) for visualization purposes. If we want to look up a value in a trie with
887 elements, we would have a shift equal to 8: All the children of the root node
can contain at most 1 8 == 256 elements each. The width and mask is also
changed by the bit count: The mask will here be 3 instead of 31.

3 3
3 2 3 2 3
887 2 1 2 1 2
1 0 1 0 1
0 0 0

level 8 6 4 2 0

index 2 1 3 0 2

0...0 10 01 11 00 10
Visualization of the 626 lookup.

Say we want to look up the contents of the element with key 626. 626 in its
38 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

binary representation is 0000 0010 0111 0010. Following the algorithm, step by
step, both written above and within Clojures source code, we would have to do
the following:

node is set up to be the root note, and level is set up to be 8.

As level is over 0, we start the for loop.
We perform the operation key >>> level first. In this case,
this is 636 >>> 8, which cuts away the first 8 bits: Were left
with 0000 0010 or 2 in decimal.
We perform the masking: (key >>> level) & MASK == 2 & 3. The
masking sets all the bits except the first two to zero. This
yields no difference here: We still have 2.
We replace the current node with its child at index 2.
We decrement level by 2, and set it to 6.
As level is over 0, we continue the for loop.
We again perform the operation key >>> level == 636 >>> 6. This
cuts away the last 6 bits, and were left with 0000 0010 01,
or 9 in decimal.
We perform the masking: (key >>> level) & MASK == 9 & 3,
or 1001 & 0011 in binary. This shaves off the top 10,
and were left with 01, or 1 in decimal.
We replace the current node with its child at index 1.
We decrement level by 2, and set it to 4.
As level is over 0, we continue the for loop
Same trick here again. 636 >>> 4 leaves us with the bits 0010 0111.
The mask sets all but the 2 first bits to zero, and were left with 11.
We replace the current node with its child at index 0b11 == 3.
We decrement level by 2, and set it to 2.
As level is over 0, we continue the for loop
636 >>> 2 leaves us with 0010 0111 00.
The 2 first bits are 0b00 == 0.
We replace node with its first child, the one at index 0.
We decrement level by 2, and set it to 0.
As level is (finally) not over 0, we jump over the for loop.
We mask key with the mask, and get back the bits 10.
We return the contents at index 2 from node.

That is almost every single machine instruction you would have to perform to
lookup a value in a Clojure vector, with a depth of 5. Such a vector would
contain between 1 and 33 million elements. The fact that the shifts and masks
are some of the most efficient operations on a modern CPU makes the whole
deal even better. From a performance perspective, the only pain point left
on lookups are the cache misses. For Clojure, that is handled pretty well by the
JVM itself.
And thats how you do lookups in tries and in Clojures vector implementation.
I would guess the bit operations are the hardest one to grok, everything else
is actually very straightforward. You just need a rough understanding on how
tries work, and thats it!
2.4. RED BLACK TREES 39

2.4 Red Black Trees

There is a special kind of binary search tree called a red-black tree. It has the
property that the longest path from the root to any leaf is no more than twice
as long as the shortest path from the root to any other leaf.
In order to guarantee this property we construct a tree such that no two con-
nected nodes are red. The longest possible path will alternate red and black
nodes. The shortest path will be all black. Thus, the two paths will differ by at
most twice the length.
There are various rules defined (see Okasaki [Oka98] p25 and Wikipedia [Wiki]).
The Clojure implementation follows the Okasaki model, which we will use as
our background. Okasaki gives two rules that are always true in a properly
formed tree:

No red node has a red child

Every path from the root to an empty node contains the same number of
black nodes.

2.4.1 Persistence
Driscoll defines a persistent data structure as one that supports mutliple ver-
sions. A data structure that allows only a single version at a time is called
ephemeral [DSST89].
The basic technique of creating a persistent data structure is to make copies.
However this is expensive in time and space. Driscoll shows that data structures
can be defined using links between nodes where the nodes carry information
and the links connect the nodes. In such structures it is possible to only copy
nodes that are changed and update the links. Using this technique does not
modify the old data structure so the links can share portions of the old structure.
Since the old structure is not changed it is also available at the same time as
the new structure, albeit with different root nodes.

2.4.2 Persistent Red Black Trees

Okasaki has combined these two ideas to create Persistent Red Black Trees.
Each modification of the tree is done by making copies of changed nodes but
maintaining the red-black tree properties in the copy.
Since Clojure uses these persistent trees the actual tree is new copy, not a
modified tree. We will now look into the details of how this is done.
40 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

2.4.3 Node Structure

First we have to mirror Driscolls concept of a Node. This is created in the code
with a new Java abstract class called Node. Okasaki requires that a node have
4 pieces of information, described as T (Color, left, this, right). Thus
we find that Nodes have the ability to manipulate their left and right members.
Thus, for the left member we find:

left to return the left Node

addLeft to add a left Node
removeLeft to remove a left Node

and for the right member we find:

left to return the right Node

addLeft to add a right Node
removeLeft to remove a right Node

We have functions to set the color, blacken and redden. We can ask this node
for its key and val
Since every node has to maintain the Red-Black balance we have two methods,
balanceLeft and balanceRight. Note that these methods always return a black
node as the root of the red-black tree is always black.
We also have a method to replace this node, the replace method.
Some of the methods are abstract which means that any class that implements
a Node has to implement the following methods:

Node addLeft(Node ins);

Node addRight(Node ins);
Node removeLeft(Node del);
Node removeRight(Node del);
Node blacken();
Node redden();
Node replace(Object key, Object val, Node left, Node right);

Since the data structure is persistent we need to copy rather than modify the
structure. Each implementation of this abstract class has to handle the details.
Node assumes that it will return a Black node from its balance operations.
Nodes in a Clojure PersistentTreeMap have one of 8 possible subtypes of Node:

Black - a black leaf node with a null value

BlackVal - a black leaf node with a value
BlackBranch - a black interior node with children and a null value
2.4. RED BLACK TREES 41

BlackBranchVal - a black interior node with children and a value

Red - a red leaf node with a null value
RedVal - a red leaf node with a value
RedBranch - a red interior node with children and a null value
RedBranchVal - a red interior node with children and a value

(AMapEntry [527])
PersistentTreeMap Node Class

static abstract class Node extends AMapEntry{

final Object key;

Node(Object key){
this.key = key;
}

public Object key(){

return key;
}

public Object val(){

return null;
}

public Object getKey(){

return key();
}

public Object getValue(){

return val();
}

Node left(){
return null;
}

Node right(){
return null;
}

abstract Node addLeft(Node ins);

abstract Node addRight(Node ins);

abstract Node removeLeft(Node del);

abstract Node removeRight(Node del);

abstract Node blacken();

42 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

abstract Node redden();

Node balanceLeft(Node parent){

return black(parent.key, parent.val(), this, parent.right());
}

Node balanceRight(Node parent){

return black(parent.key, parent.val(), parent.left(), this);
}

abstract Node replace(Object key, Object val,

Node left, Node right);

2.4.4 The Black Node implementation

Since the Node class is abstract it has to have certain methods implemented in
the extending class.
We have two types of implementations, one for Red nodes and one for Black
nodes. Lets look at the Black node class. This is broken out into a set of
subclasses and we will drill down the chain for implemention Black nodes.
The class Black extends node and implements the abstract methods of Node.
This basically involves making sure that the subtrees maintain their balance
under the operations of adding and deleting the subtrees.
Black nodes assume they are black but they implement a method to change
their color. Since it is a copy operation it returns a new Red node as the result
of redden.
Black nodes are the default color for a node so the Black class does not have to
do anything special for color.
There are 4 possible Black node types

Black - a black leaf node with a null value

BlackVal - a black leaf node with a value
BlackBranch - a black interior node with children and a null value
BlackBranchVal - a black interior node with children and a value

A Black node is a leaf node with a null value. This is constructed by Persis-
tentTreeMaps [45] black method.
(Node [41])
PersistentTreeMap Black Class
2.4. RED BLACK TREES 43

static class Black extends Node{

public Black(Object key){
super(key);
}

Node addLeft(Node ins){

return ins.balanceLeft(this);
}

Node addRight(Node ins){

return ins.balanceRight(this);
}

Node removeLeft(Node del){

return balanceLeftDel(key, val(), del, right());
}

Node removeRight(Node del){

return balanceRightDel(key, val(), left(), del);
}

Node blacken(){
return this;
}

Node redden(){
return new Red(key);
}

Node replace(Object key, Object val, Node left, Node right){

return black(key, val, left, right);
}

A BlackVal node is a leaf node with a value. This is constructed by Persistent-

TreeMaps [45] black method.
(Black [42])
PersistentTreeMap BlackVal Class

static class BlackVal extends Black{

final Object val;

public BlackVal(Object key, Object val){

super(key);
this.val = val;
44 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

public Object val(){

return val;
}

Node redden(){
return new RedVal(key, val);
}

Interior nodes of Clojures Red Black trees can have a value. A BlackBranch
node is a leaf node with a null value and children, one of which could possibly
be null. This is constructed by PersistentTreeMaps [45] black method.
(Black [42])
PersistentTreeMap BlackBranch Class
static class BlackBranch extends Black{
final Node left;

final Node right;

public BlackBranch(Object key, Node left, Node right){

super(key);
this.left = left;
this.right = right;
}

public Node left(){

return left;
}

public Node right(){

return right;
}

Node redden(){
return new RedBranch(key, left, right);
}

A BlackBranchVal node is a leaf node with a value and children, one of which
2.4. RED BLACK TREES 45

could possibly be null. This is constructed by PersistentTreeMaps [45] black

method.
(BlackBranch [44])
PersistentTreeMap BlackBranchVal Class

static class BlackBranchVal extends BlackBranch{

final Object val;

public BlackBranchVal(Object key, Object val,

Node left, Node right){
super(key, left, right);
this.val = val;
}

public Object val(){

return val;
}

Node redden(){
return new RedBranchVal(key, val, left, right);
}

2.4.5 Constructing a Black Node

We can construct a new black node given 4 parameters:

key the TreeMap key

val the value stored in the TreeMap for this key
left a Node, possibly null
right a Node, possibly null

If both of the children are null we must be constructing a leaf node. If the value
is null we need only construct a naked Black node, otherwise we construct a
BlackVal node and store both the key and the value.
If either of the children exist but have no value for this node then construct a
BlackBranch internal tree node pointing at the children.
If we have all four parameters then we have an internal node that also has a
value associated with it. Save the value and the children in a BlackBranchVal
node.

PersistentTreeMap black method

46 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

static Black black(Object key, Object val, Node left, Node right){
if(left == null && right == null)
{
if(val == null)
return new Black(key);
return new BlackVal(key, val);
}
if(val == null)
return new BlackBranch(key, left, right);
return new BlackBranchVal(key, val, left, right);
}

2.4.6 The Red Node implementation

The Red Node class differs from the Black node in many ways. In particu-
lar, most of the operations return a new Red object whereas the Black node
implementation balances the subtrees.
Black nodes are the default color for a node so the Red class has to return Red
nodes for most operations.
There are 4 possible Red node types

Red - a black leaf node with a null value

RedVal - a black leaf node with a value
RedBranch - a black interior node with children and a null value
RedBranchVal - a black interior node with children and a value

A Red node is a leaf node with a null value. This is constructed by Persistent-
TreeMaps [50] red method.
(Node [41])
PersistentTreeMap Red Class

static class Red extends Node{

public Red(Object key){
super(key);
}

Node addLeft(Node ins){

return red(key, val(), ins, right());
}

Node addRight(Node ins){

return red(key, val(), left(), ins);
}
2.4. RED BLACK TREES 47

Node removeLeft(Node del){

return red(key, val(), del, right());
}

Node removeRight(Node del){

return red(key, val(), left(), del);
}

Node blacken(){
return new Black(key);
}

Node redden(){
throw new UnsupportedOperationException("Invariant violation");
}

Node replace(Object key, Object val, Node left, Node right){

return red(key, val, left, right);
}

A RedVal node is a leaf node with a value. This is constructed by Persistent-

TreeMaps [50] red method.
(Red [46])
PersistentTreeMap RedVal Class
static class RedVal extends Red{
final Object val;

public RedVal(Object key, Object val){

super(key);
this.val = val;
}

public Object val(){

return val;
}

Node blacken(){
return new BlackVal(key, val);
}

-
48 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

Interior nodes of Clojures Red Black trees can have a value. A RedBranch
node is a leaf node with a null value and children, one of which could possibly
be null. This is constructed by PersistentTreeMaps [50] red method.
(Red [46])
PersistentTreeMap RedBranch Class

static class RedBranch extends Red{

final Node left;

final Node right;

public RedBranch(Object key, Node left, Node right){

super(key);
this.left = left;
this.right = right;
}

public Node left(){

return left;
}

public Node right(){

return right;
}

Node balanceLeft(Node parent){

if(left instanceof Red)
return red(key, val(), left.blacken(),
black(parent.key, parent.val(),
right, parent.right()));
else if(right instanceof Red)
return red(right.key, right.val(),
black(key, val(), left, right.left()),
black(parent.key, parent.val(),
right.right(), parent.right()));
else
return super.balanceLeft(parent);

Node balanceRight(Node parent){

if(right instanceof Red)
return red(key, val(),
black(parent.key, parent.val(),
parent.left(), left),
right.blacken());
else if(left instanceof Red)
return red(left.key, left.val(),
black(parent.key, parent.val(),
2.4. RED BLACK TREES 49

parent.left(), left.left()),
black(key, val(), left.right(), right));
else
return super.balanceRight(parent);
}

Node blacken(){
return new BlackBranch(key, left, right);
}

A RedBranchVal node is a leaf node with a value and children, one of which
could possibly be null. This is constructed by PersistentTreeMaps [50] red
method.
(RedBranch [48])
PersistentTreeMap RedBranchVal Class

static class RedBranchVal extends RedBranch{

final Object val;

public RedBranchVal(Object key, Object val, Node left, Node right){

super(key, left, right);
this.val = val;
}

public Object val(){

return val;
}

Node blacken(){
return new BlackBranchVal(key, val, left, right);
}
}

2.4.7 Constructing a Red Node

We can construct a new red node given 4 parameters:

key the TreeMap key

val the value stored in the TreeMap for this key
left a Node, possibly null
right a Node, possibly null
50 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

If both of the children are null we must be constructing a leaf node. If the
value is null we need only construct a naked Red node, otherwise we construct
a RedVal node and store both the key and the value.
If either of the children exist but have no value for this node then construct a
RedBranch internal tree node pointing at the children.
If we have all four parameters then we have an internal node that also has a
value associated with it. Save the value and the children in a RedBranchVal
node.

PersistentTreeMap red method

static Red red(Object key, Object val, Node left, Node right){
if(left == null && right == null)
{
if(val == null)
return new Red(key);
return new RedVal(key, val);
}
if(val == null)
return new RedBranch(key, left, right);
return new RedBranchVal(key, val, left, right);
}

2.4.8 Okasakis balance cases

Okasaki distinguishes 4 cases of balancing. It turns out that all four cases
balance to the same tree, which is hardly a surprise given the constraints on the
red-black alternation.
ML [Wiki1, Har05] is a functional programming language and is used by Okasaki
to express his tree transformations.
Each case shows the ML code and the associated diagram. The ML code rep-
resents a node as a 4-tuple of type T. Thus we see a node as

Type ( Color, leftnode, thisnode, rightnode )

so we can read the first case as:

Node ( Black,
Node ( Red ,
Node ( Red, Node a, Node X, Node b ),
Node Y,
Node c
)
2.4. RED BLACK TREES 51

Node Z
Node d
)

Case 1: (B,T (R,T (R,a,x,b),y,c),z,d) = T (R,T (B,a,x,b),y,T (B,c,z,d))

Z => Y

Y d X Z

X c a b c d

a b

Case 2: (B, T (R,a,x,T (R,b,y,c)),z,d) = T (R,T (B,a,x,b),y,T (B,c,z,d))

Z => Y

X d X Z

a Y a b c d

b c
52 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

Case 3: (B,a,x,T (R,T (R,b,y,c),z,d)) = T (R,T (B,a,x,b),y,T (B,c,z,d))

X => Y

a Z X Z

Y d a b c d

b c

Case 4: (B,a,x,T (R,b,y,T (R,c,z,d))) = T (R,T (B,a,x,b),y,T (B,c,z,d))

X => Y

a Y X Z

b A a b c d

c d

2.4.9 Deleting a Node

Here we are being asked to remove the Node del. This request is either a call to
a black nodes [42] removeRight method or PersistentTreeMaps [1000] remove
method.

PersistentTreeMap balanceRightDel method

static Node balanceRightDel(Object key, Object val,

2.4. RED BLACK TREES 53

Node left, Node del){

if(del instanceof Red)
return red(key, val, left, del.blacken());
else if(left instanceof Black)
return leftBalance(key, val, left.redden(), del);
else if(left instanceof Red && left.right() instanceof Black)
return red(left.right().key, left.right().val(),
leftBalance(left.key, left.val(),
left.left().redden(),
left.right().left()),
black(key, val, left.right().right(), del));
else
throw new UnsupportedOperationException("Invariant violation");
}

PersistentTreeMap balanceLeftDel method

static Node balanceLeftDel(Object key, Object val,

Node del, Node right){
if(del instanceof Red)
return red(key, val, del.blacken(), right);
else if(right instanceof Black)
return rightBalance(key, val, del, right.redden());
else if(right instanceof Red && right.left() instanceof Black)
return red(right.left().key, right.left().val(),
black(key, val, del, right.left().left()),
rightBalance(right.key, right.val(),
right.left().right(),
right.right().redden()));
else
throw new UnsupportedOperationException("Invariant violation");
}

2.4.10 Clojures balance cases

We find the leftBalance handling the case where a red node has a red node as
its left child. There are three cases.
Case 1: The left child of a red node is red. We return new nodes X, Y, and Z
where

X Node ( Red,
Y
Z Node ( Black,
54 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

A => X

B b
Z Z

Case 2: The right child of a red node is red. We rewrite that as:

N => new

b
B Z Z

Case 3: Both children of a red node are black. Rewrite the node black.

N => A

B C B C

PersistentTreeMap leftBalance method

static Node leftBalance(Object key, Object val,

Node ins, Node right){
if(ins instanceof Red && ins.left() instanceof Red)
return red(ins.key, ins.val(),
ins.left().blacken(),
black(key, val, ins.right(), right));
else if(ins instanceof Red && ins.right() instanceof Red)
return red(ins.right().key, ins.right().val(),
black(ins.key, ins.val(),
ins.left(), ins.right().left()),
black(key, val, ins.right().right(), right));
else
return black(key, val, ins, right);
}

PersistentTreeMap rightBalance method

2.5. IMMUTABLE DATA STRUCTURES 55

static Node rightBalance(Object key, Object val, Node left, Node ins){
if(ins instanceof Red && ins.right() instanceof Red)
return red(ins.key, ins.val(),
black(key, val, left, ins.left()),
ins.right().blacken());
else if(ins instanceof Red && ins.left() instanceof Red)
return red(ins.left().key, ins.left().val(),
black(key, val, left, ins.left().left()),
black(ins.key, ins.val(),
ins.left().right(), ins.right()));
else
return black(key, val, left, ins);
}

2.4.11 Replacing a Node

PersistentTreeMap replace method

Node replace(Node t, Object key, Object val){

int c = doCompare(key, t.key);
return t.replace(t.key,
c == 0 ? val : t.val(),
c < 0 ? replace(t.left(), key, val) : t.left(),
c > 0 ? replace(t.right(), key, val) : t.right());
}

[1000] PersistentTreeMap implements several subclasses to support the interop-

erability with the rest of Clojure. These will not be explained here. Refer to the
other sections of this document for more details. In particular, see [103] Iterators
for methods of iterating across keys and values. See [60] Seqs for information
about manipulating sequences over tree maps.

2.5 Immutable Data Structures

2.6 Bit-Partitioned Hash Tries

A trie is a data structure that exploits the fact that a prefix or suffix has some
regular pattern[Bag00]. For instance, credit card numbers have a fixed format.
We can look at the set of 16 digit credit card numbers which have the pattern
AAAA-BBBB-CCCC-DDDD. Given this sequence of 4 digit fields we can create
a trie data structure for fast lookup.
56 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

If we created a tree structure where the first level of lookup was the first 4
digits AAAA we would have a maximum of 9999 possible first level branches.
If the second layer of the tree were based on BBBB then it would also have a
branching factor of 9999. We eventually get a tree at most 4 layers deep that
handle all 101 6 possible values.
Alternatively we can reduce the fan-out by choosing 2 digits, so AA gives a
fan-out of 99 and this happens at each level. Now the tree is 8 layers deep in
order to contain all possible nodes.
Clojure has chosen a strategy based a bit pattern. So if we look at the bits in
groups of five we get this kind of a trie:
2.6. BIT-PARTITIONED HASH TRIES 57

G G F F F F F E E E E E D D D D D C C C C C B B B B B A A A A A

B B x B x x

x x C x x C C

x D x x x x x

x x

Notice that the trie has a fan-out of 25 or 32 at each level except the last. To
contain all possible 32 bit values requires a trie of at most depth 7. This means
that a lookup will take at most 7 possible fetches. For any reasonable set of
numbers this is a near constant time lookup.
One advantage of this scheme is the ability to mask out a subset of the word,
fully right-shift the bits, and use the resulting number as an index into the array.
Any masked off value has to be between 0 and 31 because the mask is 5 bits
wide.
The 5 bit mask function is part of [969] PersistentHashMap. Given a number of
bits to shift right (a multiple of 5 appears to be used everywhere) this routine
returns a number between 0 and 31.

PersistentHashMap mask method

static int mask(int hash, int shift){

//return ((hash << shift) >>> 27);// & 0x01f;
return (hash >>> shift) & 0x01f;
}

This is used in the ArrayNode class. For instance, the find method
58 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

uses mask to get an index from 5 bits

uses that value to index into an array
if that entry is not set we didnt find it
shift 5 more bits right and search again.

ArrayNode find Object method

public Object find(int shift, int hash, Object key,

Object notFound){
int idx = mask(hash, shift);
INode node = array[idx];
if(node == null)
return notFound;
return node.find(shift + 5, hash, key, notFound);
}

Note that ArrayNode is a container class so it is actually recursively calling the

find method on whatever was found in that container slot. It might be anything
that implements the INode interface: (Serializable [1723])
PersistentHashMap INode interface

static interface INode extends Serializable {

INode assoc(int shift, int hash, Object key,
Object val, Box addedLeaf);

INode without(int shift, int hash, Object key);

IMapEntry find(int shift, int hash, Object key);

Object find(int shift, int hash, Object key, Object notFound);

ISeq nodeSeq();

INode assoc(AtomicReference<Thread> edit, int shift, int hash,

Object key, Object val, Box addedLeaf);

INode without(AtomicReference<Thread> edit, int shift, int hash,

Object key, Box removedLeaf);
}

There are 3 classes that implement INode.

[955] ArrayNode
2.6. BIT-PARTITIONED HASH TRIES 59

[959] BitmapIndexedNode
[965] HashCollisionNode

2.6.1 Lists

2.6.2 Vectors

2.6.3 Hashmaps
Maps are associations of keywords and data. Hashmaps are untyped data struc-
tures, as opposed to Records which carry their type.

> (def tim {:fname Tim

:lname Daly
:address {:street Long Ridge Road
:city Pittsburgh
:state PA
:zip 15022}})
#user/tim

We can look up by keyword since keywords are

> (:lname tim)

Daly

We can nest access using the thread first operator which threads arbitrary
pieces of code (as opposed to .. which threads Java classes).

> (-> tim :address :city)

Pittsburgh

We can create a new object with a change to a field.

> (assoc tim :fname Timothy)

{:lname Daly,
:address {:state PA,
:city Pittsburgh,
:street Long Ridge Road,
:zip 15022},
:fname Timothy}

The update-in function threads into a nested data structure and then applies
a function, in this case the inc function. This reduces the need to create inter-
mediate classes.
60 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

> (update-in tim [:address :zip] inc)

{:lname Daly,
:address {:state PA,
:city Pittsburgh,
:street Long Ridge Road,
:zip 15023},
:fname Tim}

2.6.4 Seqs
(ASeq [571])
PersistentTreeMap Seq Class

static public class Seq extends ASeq{

final ISeq stack;
final boolean asc;
final int cnt;

public Seq(ISeq stack, boolean asc){

this.stack = stack;
this.asc = asc;
this.cnt = -1;
}

public Seq(ISeq stack, boolean asc, int cnt){

this.stack = stack;
this.asc = asc;
this.cnt = cnt;
}

Seq(IPersistentMap meta, ISeq stack, boolean asc, int cnt){

super(meta);
this.stack = stack;
this.asc = asc;
this.cnt = cnt;
}

static Seq create(Node t, boolean asc, int cnt){

return new Seq(push(t, null, asc), asc, cnt);
}

static ISeq push(Node t, ISeq stack, boolean asc){

while(t != null)
{
stack = RT.cons(t, stack);
t = asc ? t.left() : t.right();
}
return stack;
2.7. RECORDS 61

public Object first(){

return stack.first();
}

public ISeq next(){

Node t = (Node) stack.first();
ISeq nextstack =
push(asc ? t.right() : t.left(), stack.next(), asc);
if(nextstack != null)
{
return new Seq(nextstack, asc, cnt - 1);
}
return null;
}

public int count(){

if(cnt < 0)
return super.count();
return cnt;
}

public Obj withMeta(IPersistentMap meta){

return new Seq(meta, stack, asc, cnt);
}
}

2.7 Records
Records carry their type, in this case Person, as opposed to hashmaps which
are untyped data structures.

> (defrecord Person [fname lname address])

user.Person

and in this case Address

> (defrecord Address [street city state zip])

user.Address

We can create a Var reference to a new record and set the address field to be a
new Address record.
62 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

> (def tim

(Person. Tim Daly
(Address. Long Ridge Road Pittsburgh PA 15022)))
#user/tim

Record field names are functions so we can use them as accessors.

> (:lname tim)

Daly

And we can access them with the thread first operator which threads through
the set of accessors and then applies the function, in this case, the inc function.
This is similar to the .. operator except that it applies to any kind of operator,
not just Java operators.

> (-> tim :address :city)

Pittsburgh

> (assoc tim :fname Timothy)

#:user.Person{:fname Timothy,
:lname Daly,
:address #:user.Address{
:street Long Ridge Road,
:city Pittsburgh,
:state PA,
:zip 15022}

> (update-in tim [:address :zip] inc)

#:user.Person{:fname Tim,
:lname Daly,
:address #:user.Address{
:street Long Ridge Road,
:city Pittsburgh,
:state PA,
:zip 15023}

2.8 Java Interoperability

2.8.1 Language primitives
Atomic data types[Hal11]
2.8. JAVA INTEROPERABILITY 63

type Clojure Java

string foo String

character \f Character
regular expression #"fo*" Pattern
integer 42 Long
big integer 42N BigInteger
double 3.14159 Double
big decimal 3.141519M BigDecimal
boolean true Boolean
nil nil null
ratio 22/7 N/A
symbol foo N/A
keyword :foo ::foo N/A
Data literals
These data structures can contain anything of any type.
type property example

list singly-linked, insert at front (1 2 3)

vector indexed, insert at rear [1 2 3]
map key-value pairs {:a 10 :b 20}
set key #{1 2 3}

People make claims that Lisp has no syntax . . . A more accurate thing to say
is that Lisp has the syntax of its data structures and that syntax has been
built upon to write code.
Stuart Halloway Clojure-Java Interp (Jan 19, 2011)

Function Calling

(println "Hello Wikileaks")

This is a list data structure with a first element of a Symbol and a second
element of a String. The first element is the function to be called and the
second element is the argument to that function.
Function Definition

(defn funname
"documentation string of the function"
[vector of arguments]
(function arg1 ... argN))

This is a list data structure with the elements:

64 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

defn, a Symbol, used as a function to define a function

funname, a Symbol, used as the name of the new function
docstring, a String, used to document the function
argvec, a Vector of arguments to the new function
function, a Symbol, used as the name of some other function to call
arg1. . .argN, a list of arguments to the function

For example,

> (defn say says the name and age [name age] (str name age))
#user/say

The defn function creates the new function called say which takes 2 arguments
and returns a string with the two arguments separated by a space as in:

> (say Tim 24)

Tim 24

You can add meta information to your programs to improve the performance or
simplify the runtime lookup of classes. The syntax for metadata is a Symbol or
hashmap prefixed by a caret ^. For example, the say function returns a String
so we could write

(defn ^String say

"says the name and age"
[name age]
(str name " " age))

If the item after the caret is a single Symbol it is expanded into a hashmap with
the keyword :tag and the Symbol. In our case, this is {:tag String}

2.8.2 Clojure calling Java

Java new in Clojure

new MyClass("arg1")

becomes

(MyClass. "arg1")

Accessing a Java static member

Math.PI

becomes
2.9. RECURSION 65

Math/PI

Accessing instance members

Since Clojure shifts the focus from the object to the function we call, the Clojure
syntax favors putting the function at the head of the list, the so-called function
position.

rnd.nextInt()

becomes

(.nextInt rnd)

Chaining function calls

person.getAddress().getZip()

becomes

(.. person getAddress getZip)

The doto macro uses the first form to create an object upon which subsequent
method calls can be applied in series.

(doto (JFrame. "argument")

(.add (proxy [JPanel] []))
(.setSize 640 480)
(.setVisible true))

2.9 Recursion

2.10 Argument Deconstruction

2.11 The Read-Eval-Print Loop

2.12 Special Forms

2.13 Reader Macros

2.14 Namespaces
Namespaces map to java packages.
66 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

2.15 Dynamic Comopilation

2.16 Ahead-Of-Time Comopilation

2.17 Lazy Evaluation

2.18 Metadata

2.19 Concurrency

2.20 Identity and State

Identity is a logical entity associated with a series of causally related values,
called states, over time. Identity is not a name but can be named (e.g. my
Mom is not your mom). Identity can be a composite object (e.g. the New
York Yankees). Programs that are processes need identity. [Hic10]
State is the value of an identity at a given time. Variables are not sufficient to
refer to a state because state could be composite. This leads to a situation where,
at a given time, the variable does not refer to a properly formed state but refers
to inconsistent composite information. There is a lack of time coordination.
In order to properly capture state we need to incorporate some notion of time.
Once we have time as a marker we can talk about state at a particular time.
Since Clojure deals with immutable objects that implies that a changed state
must be a different object since things dont change in place.
Since state is a combination of information and time it is possible for multiple
observers to query the state. Each will see a consistent view of the information
(the value) based on the time of the observation.
Clojure manages state by creating an intermediate object called a reference.
The reference object has atomic semantics. Changing a reference happens as
an all-or-nothing event. References are essentially a pointer to a stateful object.
The reference points at the current state. When a new state is being constructed
as a function of the old state there is no way to reference it until the function
completes. At that time the reference is atomically updated and the new state
can be observed as a wholly-formed object. This becomes the new state which
is a function of the current time and the new information. Observers of the
prior state will still see the old state unchanged.
There are four kinds of reference objects in Clojure which we will discuss in
detail below. These are vars, atoms, refs, and agents.
2.21. SOFTWARE TRANSACTIONAL MEMORY 67

2.21 Software Transactional Memory

2.22 Symbols
Internally, a Clojure Symbol contains four pieces of data

ns an interned string of the namespace for this symbol

name a interned string for this symbol name
hash a hash of the name and the namespace
meta meta data associated with this Symbol

Symbol private data

//these must be interned strings!

final String ns;
final String name;
final int hash;
final IPersistentMap _meta;

Javas String class has an intern method [Ora11] which returns a canonical
representation for the string object. The String class maintains a pool of strings.
When Strings intern method is invoked the pool is searched for an equals match.
If the search succeeds then a reference to the stored string is returned, otherwise
the new string is added to the pool and a reference to this String object is
returned. Thus, the String name of a Symbol is a unique reference.
As pointed out in [Web11], Clojure creates a new Symbol for each occurence it
reads. The Symbols are not unique despite having equal names:

node2: clj
Clojure 1.3.0-alpha4
user=> (identical? a a)
false
user=> (= a a)
true
user=> (identical? (clojure.lang.Symbol/intern "a")
(clojure.lang.Symbol/intern "a"))
false

Symbol intern method

static public Symbol intern(String nsname){

int i = nsname.lastIndexOf(/);
68 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

if(i == -1 || nsname.equals("/"))
return new Symbol(null, nsname.intern());
else
return
new Symbol(nsname.substring(0, i).intern(),
nsname.substring(i + 1).intern());
}

A new Symbol is automatically prepended with its namespace.

user=> (def a 1)
#user/a

Symbol intern method 2

static public Symbol intern(String ns, String name){

return new Symbol(ns == null ? null : ns.intern(), name.intern());
}

Since the namespace was null the current namespace user was used. We can
see this from the toString result.

Symbol method toString

public String toString(){

if(ns != null)
return ns + "/" + name;
return name;
}

Since Symbols optionally include a namespace user/a, which includes a names-

pace, and a which does not include a namespace, are different symbols but they
name the same variable.

user=> (= user/a a)
false
user=> (def a 1)
#user/a
user=> user/a
1
2.23. THE LISP READER 69

user=> (def user/a 2)

#user/a
user=> a
2

Symbol method equals

public boolean equals(Object o){

if(this == o)
return true;
if(!(o instanceof Symbol))
return false;

Symbol symbol = (Symbol) o;

//identity compares intended, names are interned

return name == symbol.name && ns == symbol.ns;
}

2.23 The Lisp Reader

When the LispReader read function is invoked it reads each character as an
integer. The read function is parsing S-expressions so it skips whitespace and
fails if it does not get a complete S-expression. Numbers are handled specially
by the reader.

LispReader read method

static public Object read(PushbackReader r,

boolean eofIsError,
Object eofValue,
boolean isRecursive)
throws Exception{

try
{
for(; ;)
{
int ch = r.read();

while(isWhitespace(ch))
ch = r.read();

if(ch == -1)
70 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

{
if(eofIsError)
throw new Exception("EOF while reading");
return eofValue;
}

if(Character.isDigit(ch))
{
Object n = readNumber(r, (char) ch);
if(RT.suppressRead())
return null;
return n;
}

IFn macroFn = getMacro(ch);

if(macroFn != null)
{
Object ret = macroFn.invoke(r, (char) ch);
if(RT.suppressRead())
return null;
//no op macros return the reader
if(ret == r)
continue;
return ret;
}

if(ch == + || ch == -)
{
int ch2 = r.read();
if(Character.isDigit(ch2))
{
unread(r, ch2);
Object n = readNumber(r, (char) ch);
if(RT.suppressRead())
return null;
return n;
}
unread(r, ch2);
}

String token = readToken(r, (char) ch);

if(RT.suppressRead())
return null;
return interpretToken(token);
}
}
catch(Exception e)
{
if(isRecursive || !(r instanceof LineNumberingPushbackReader))
throw e;
2.23. THE LISP READER 71

LineNumberingPushbackReader rdr =
(LineNumberingPushbackReader) r;
//throw new Exception(String.format("ReaderError:(%d,1) %s",
// rdr.getLineNumber(), e.getMessage()), e);
throw new ReaderException(rdr.getLineNumber(), e);
}
}

2.23.1 Reader Syntax Macros

Almost every special character is handled by looking up the special function
associated with that character in the macros array:

LispReader macros statement

static IFn[] macros = new IFn[256];

The default values are specified in the syntax macro table. These are all class
objects that implement the [774] IFn interface which requires that they have an
invoke method.
For the special case of the # character there is a separate table, the [84] Dispatch
Macro Table which reads the next character and calls another dispatch function.
All of the special characters get their meaning from these tables.
By default the special syntax characters are

" a [72] StringReader

; a [74] CommentReader
a [74] WrappingReader(QUOTE)
@ a [74] WrappingReader(DEREF)
^ a [75] MetaReader
a [76] SyntaxQuoteReader
~ a [79] UnquoteReader
( a [80] ListReader
) a [80] UnmatchedDelimiterReader
[ a [80] VectorReader
] a [80] UnmatchedDelimiterReader
{ a [81] MapReader
} a [80] UnmatchedDelimiterReader
\ a [81] CharacterReader
% a [82] ArgReader
72 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

# a [83] DispatchReader

LispReader Syntax Macro Table

macros["] = new StringReader();

macros[;] = new CommentReader();
macros[\] = new WrappingReader(QUOTE);
macros[@] = new WrappingReader(DEREF);//new DerefReader();
macros[^] = new MetaReader();
macros[] = new SyntaxQuoteReader();
macros[~] = new UnquoteReader();
macros[(] = new ListReader();
macros[)] = new UnmatchedDelimiterReader();
macros[[] = new VectorReader();
macros[]] = new UnmatchedDelimiterReader();
macros[{] = new MapReader();
macros[}] = new UnmatchedDelimiterReader();
// macros[|] = new ArgVectorReader();
macros[\\] = new CharacterReader();
macros[%] = new ArgReader();
\getchunk{LispReader sharpsign macro statement}

2.23.2 The String reader macro

(AFn [509])
LispReader StringReader class

public static class StringReader extends AFn{

public Object invoke(Object reader, Object doublequote)
throws Exception{
StringBuilder sb = new StringBuilder();
Reader r = (Reader) reader;

for(int ch = r.read(); ch != "; ch = r.read())

{
if(ch == -1)
throw new Exception("EOF while reading string");
if(ch == \\) //escape
{
ch = r.read();
if(ch == -1)
throw new Exception("EOF while reading string");
switch(ch)
{
case t:
2.23. THE LISP READER 73

ch = \t;
break;
case r:
ch = \r;
break;
case n:
ch = \n;
break;
case \\:
break;
case ":
break;
case b:
ch = \b;
break;
case f:
ch = \f;
break;
case u:
{
ch = r.read();
if (Character.digit(ch, 16) == -1)
throw new Exception(
"Invalid unicode escape: \\u" + (char) ch);
ch =
readUnicodeChar((PushbackReader) r,ch,16,4,true);
break;
}
default:
{
if(Character.isDigit(ch))
{
ch =
readUnicodeChar(
(PushbackReader) r,ch,8,3,false);
if(ch > 0377)
throw new Exception(
"Octal escape sequence must be in range [0, 377].");
}
else
throw new Exception(
"Unsupported escape character: \\" + (char) ch);
}
}
}
sb.append((char) ch);
}
return sb.toString();
}
}
74 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

2.23.3 The Comment reader macro

(AFn [509])
LispReader CommentReader class

public static class CommentReader extends AFn{

public Object invoke(Object reader, Object semicolon)
throws Exception{
Reader r = (Reader) reader;
int ch;
do
{
ch = r.read();
} while(ch != -1 && ch != \n && ch != \r);
return r;
}

2.23.4 The Wrapping reader macro

(AFn [509])
LispReader WrappingReader class

public static class WrappingReader extends AFn{

final Symbol sym;

public WrappingReader(Symbol sym){

this.sym = sym;
}

public Object invoke(Object reader, Object quote)

throws Exception{
PushbackReader r = (PushbackReader) reader;
Object o = read(r, true, null, true);
return RT.list(sym, o);
}

-
2.23. THE LISP READER 75

2.23.5 The Meta reader macro

(AFn [509])
LispReader MetaReader class

public static class MetaReader extends AFn{

public Object invoke(Object reader, Object caret)
throws Exception{
PushbackReader r = (PushbackReader) reader;
int line = -1;
if(r instanceof LineNumberingPushbackReader)
line = ((LineNumberingPushbackReader) r).getLineNumber();
Object meta = read(r, true, null, true);
if(meta instanceof Symbol || meta instanceof String)
meta = RT.map(RT.TAG_KEY, meta);
else if (meta instanceof Keyword)
meta = RT.map(meta, RT.T);
else if(!(meta instanceof IPersistentMap))
throw new IllegalArgumentException(
"Metadata must be Symbol,Keyword,String or Map");

Object o = read(r, true, null, true);

if(o instanceof IMeta)
{
if(line != -1 && o instanceof ISeq)
meta = ((IPersistentMap) meta).assoc(RT.LINE_KEY, line);
if(o instanceof IReference)
{
((IReference)o).resetMeta((IPersistentMap) meta);
return o;
}
Object ometa = RT.meta(o);
for(ISeq s = RT.seq(meta); s != null; s = s.next()) {
IMapEntry kv = (IMapEntry) s.first();
ometa = RT.assoc(ometa, kv.getKey(), kv.getValue());
}
return ((IObj) o).withMeta((IPersistentMap) ometa);
}
else
throw new IllegalArgumentException(
"Metadata can only be applied to IMetas");
}

-
76 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

2.23.6 The SyntaxQuote reader macro

(AFn [509])
LispReader SyntaxQuoteReader class

public static class SyntaxQuoteReader extends AFn{

public Object invoke(Object reader, Object backquote)
throws Exception{
PushbackReader r = (PushbackReader) reader;
try
{
Var.pushThreadBindings(
RT.map(GENSYM_ENV, PersistentHashMap.EMPTY));

Object form = read(r, true, null, true);

return syntaxQuote(form);
}
finally
{
Var.popThreadBindings();
}
}

static Object syntaxQuote(Object form) throws Exception{

Object ret;
if(Compiler.isSpecial(form))
ret = RT.list(Compiler.QUOTE, form);
else if(form instanceof Symbol)
{
Symbol sym = (Symbol) form;
if(sym.ns == null && sym.name.endsWith("#"))
{
IPersistentMap gmap =
(IPersistentMap) GENSYM_ENV.deref();
if(gmap == null)
throw new IllegalStateException(
"Gensym literal not in syntax-quote");
Symbol gs = (Symbol) gmap.valAt(sym);
if(gs == null)
GENSYM_ENV.set(
gmap.assoc(sym,
gs = Symbol.intern(null,
sym.name.substring(0, sym.name.length() - 1)
+ "__" + RT.nextID() + "__auto__")));
sym = gs;
}
else if(sym.ns == null && sym.name.endsWith("."))
{
Symbol csym =
Symbol.intern(null,
2.23. THE LISP READER 77

sym.name.substring(0, sym.name.length() - 1));

csym = Compiler.resolveSymbol(csym);
sym = Symbol.intern(null, csym.name.concat("."));
}
else if(sym.ns == null && sym.name.startsWith("."))
{
// Simply quote method names.
}
else
{
Object maybeClass = null;
if(sym.ns != null)
maybeClass = Compiler.currentNS().getMapping(
Symbol.intern(null, sym.ns));
if(maybeClass instanceof Class)
{
// Classname/foo ->
// package.qualified.Classname/foo
sym = Symbol.intern(
((Class)maybeClass).getName(), sym.name);
}
else
sym = Compiler.resolveSymbol(sym);
}
ret = RT.list(Compiler.QUOTE, sym);
}
else if(isUnquote(form))
return RT.second(form);
else if(isUnquoteSplicing(form))
throw new IllegalStateException("splice not in list");
else if(form instanceof IPersistentCollection)
{
if(form instanceof IPersistentMap)
{
IPersistentVector keyvals = flattenMap(form);
ret = RT.list(APPLY, HASHMAP,
RT.list(SEQ,
RT.cons(CONCAT, sqExpandList(keyvals.seq()))));
}
else if(form instanceof IPersistentVector)
{
ret = RT.list(APPLY, VECTOR,
RT.list(SEQ,
RT.cons(CONCAT,
sqExpandList(
((IPersistentVector) form).seq()))));
}
else if(form instanceof IPersistentSet)
{
ret = RT.list(APPLY, HASHSET,
78 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

RT.list(SEQ,
RT.cons(CONCAT,
sqExpandList(
((IPersistentSet) form).seq()))));
}
else if(form instanceof ISeq ||
form instanceof IPersistentList)
{
ISeq seq = RT.seq(form);
if(seq == null)
ret = RT.cons(LIST,null);
else
ret =
RT.list(SEQ, RT.cons(CONCAT, sqExpandList(seq)));
}
else
throw new UnsupportedOperationException(
"Unknown Collection type");
}
else if(form instanceof Keyword
|| form instanceof Number
|| form instanceof Character
|| form instanceof String)
ret = form;
else
ret = RT.list(Compiler.QUOTE, form);

if(form instanceof IObj && RT.meta(form) != null)

{
//filter line numbers
IPersistentMap newMeta =
((IObj) form).meta().without(RT.LINE_KEY);
if(newMeta.count() > 0)
return
RT.list(WITH_META, ret,
syntaxQuote(((IObj) form).meta()));
}
return ret;
}

private static ISeq sqExpandList(ISeq seq) throws Exception{

PersistentVector ret = PersistentVector.EMPTY;
for(; seq != null; seq = seq.next())
{
Object item = seq.first();
if(isUnquote(item))
ret = ret.cons(RT.list(LIST, RT.second(item)));
else if(isUnquoteSplicing(item))
ret = ret.cons(RT.second(item));
else
2.23. THE LISP READER 79

ret = ret.cons(RT.list(LIST, syntaxQuote(item)));

}
return ret.seq();
}

private static IPersistentVector flattenMap(Object form){

IPersistentVector keyvals = PersistentVector.EMPTY;
for(ISeq s = RT.seq(form); s != null; s = s.next())
{
IMapEntry e = (IMapEntry) s.first();
keyvals = (IPersistentVector) keyvals.cons(e.key());
keyvals = (IPersistentVector) keyvals.cons(e.val());
}
return keyvals;
}

2.23.7 The Unquote reader macro

(AFn [509])
LispReader UnquoteReader class

static class UnquoteReader extends AFn{

public Object invoke(Object reader, Object comma) throws Exception{
PushbackReader r = (PushbackReader) reader;
int ch = r.read();
if(ch == -1)
throw new Exception("EOF while reading character");
if(ch == @)
{
Object o = read(r, true, null, true);
return RT.list(UNQUOTE_SPLICING, o);
}
else
{
unread(r, ch);
Object o = read(r, true, null, true);
return RT.list(UNQUOTE, o);
}
}

-
80 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

2.23.8 The List reader macro

(AFn [509])
LispReader ListReader class

public static class ListReader extends AFn{

public Object invoke(Object reader, Object leftparen)
throws Exception{
PushbackReader r = (PushbackReader) reader;
int line = -1;
if(r instanceof LineNumberingPushbackReader)
line = ((LineNumberingPushbackReader) r).getLineNumber();
List list = readDelimitedList(), r, true);
if(list.isEmpty())
return PersistentList.EMPTY;
IObj s = (IObj) PersistentList.create(list);
// IObj s = (IObj) RT.seq(list);
if(line != -1)
return s.withMeta(RT.map(RT.LINE_KEY, line));
else
return s;
}

2.23.9 The Unmatched Delimiter reader macro

(AFn [509])
LispReader UnmatchedDelimiterReader class

public static class UnmatchedDelimiterReader extends AFn{

public Object invoke(Object reader, Object rightdelim)
throws Exception{
throw new Exception("Unmatched delimiter: " + rightdelim);
}

2.23.10 The Vector reader macro

(AFn [509])
LispReader VectorReader class
2.23. THE LISP READER 81

public static class VectorReader extends AFn{

public Object invoke(Object reader, Object leftparen)
throws Exception{
PushbackReader r = (PushbackReader) reader;
return LazilyPersistentVector
.create(readDelimitedList(], r, true));
}

2.23.11 The Map reader macro

(AFn [509])
LispReader MapReader class

public static class MapReader extends AFn{

public Object invoke(Object reader, Object leftparen)
throws Exception{
PushbackReader r = (PushbackReader) reader;
return RT.map(readDelimitedList(}, r, true).toArray());
}

2.23.12 The Character reader macro

(AFn [509])
LispReader CharacterReader class

public static class CharacterReader extends AFn{

public Object invoke(Object reader, Object backslash)
throws Exception{
PushbackReader r = (PushbackReader) reader;
int ch = r.read();
if(ch == -1)
throw new Exception("EOF while reading character");
String token = readToken(r, (char) ch);
if(token.length() == 1)
return Character.valueOf(token.charAt(0));
else if(token.equals("newline"))
return \n;
else if(token.equals("space"))
return ;
82 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

else if(token.equals("tab"))
return \t;
else if(token.equals("backspace"))
return \b;
else if(token.equals("formfeed"))
return \f;
else if(token.equals("return"))
return \r;
else if(token.startsWith("u"))
{
char c = (char) readUnicodeChar(token, 1, 4, 16);
if(c >= \uD800 && c <= \uDFFF) // surrogate code unit?
throw new Exception(
"Invalid character constant: \\u" +
Integer.toString(c, 16));
return c;
}
else if(token.startsWith("o"))
{
int len = token.length() - 1;
if(len > 3)
throw new Exception(
"Invalid octal escape sequence length: " + len);
int uc = readUnicodeChar(token, 1, len, 8);
if(uc > 0377)
throw new Exception(
"Octal escape sequence must be in range [0, 377].");
return (char) uc;
}
throw new Exception("Unsupported character: \\" + token);
}

2.23.13 The Arg reader macro

(AFn [509])
LispReader ArgReader class

static class ArgReader extends AFn{

public Object invoke(Object reader, Object pct) throws Exception{
PushbackReader r = (PushbackReader) reader;
if(ARG_ENV.deref() == null)
{
return interpretToken(readToken(r, %));
}
int ch = r.read();
2.24. READER DISPATCH MACROS 83

unread(r, ch);
//% alone is first arg
if(ch == -1 || isWhitespace(ch) || isTerminatingMacro(ch))
{
return registerArg(1);
}
Object n = read(r, true, null, true);
if(n.equals(Compiler._AMP_))
return registerArg(-1);
if(!(n instanceof Number))
throw new IllegalStateException(
"arg literal must be %, %& or %integer");
return registerArg(((Number) n).intValue());
}
}

2.24 Reader Dispatch Macros

The [825] LispReader read function, when it encounters a # character indexes
into the [72] Syntax Macro Table to find an entry initialized to a [83] Dis-
patchReader object.

LispReader sharpsign macro statement

macros[#] = new DispatchReader();

A DispatchReader reads the next character and uses it as an index into the [84]
dispatchMacros array. The reader function associated with the macro character,
in this case, is a [85] VarReader. This gets assigned to fn and then calls the
invoke method of the [85] VarReader class.
(AFn [509])
LispReader DispatchReader class

public static class DispatchReader extends AFn{

public Object invoke(Object reader, Object hash)
throws Exception{
int ch = ((Reader) reader).read();
if(ch == -1)
throw new Exception("EOF while reading character");
IFn fn = dispatchMacros[ch];
if(fn == null)
throw new Exception(
84 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

String.format("No dispatch macro for: %c", (char) ch));

return fn.invoke(reader, ch);
}
}

The dispatchMacros array is defined to be an array of IFn objects. The [774]

IFn interface requires that all implementations have to have an invoke method,
which VarReader has.
LispReader dispatchMacros statement

static IFn[] dispatchMacros = new IFn[256];

By default the special dispatch characters are:

^ a [75] MetaReader, for attaching meta information to objects

a [85] VarReader, returns a Var object
" a [85] RegexReader, returns a regular expression
( a [86] FnReader, returns a function object
{ a [87] SetReader, returns a set object
= a [87] EvalReader, returns an evaled object
! a [74] CommentReader, returns a commented object
< a [89] UnreadableReader, returns an unreadable object
_ a [89] DiscardReader, returns a discarded object

These are all defined in this table and they are explained in the following sec-
tions.
LispReader Dispatch Macro Table

dispatchMacros[^] = new MetaReader();

\getchunk{LispReader dispatchMacros VarReader statement}
dispatchMacros["] = new RegexReader();
dispatchMacros[(] = new FnReader();
dispatchMacros[{] = new SetReader();
dispatchMacros[=] = new EvalReader();
dispatchMacros[!] = new CommentReader();
dispatchMacros[<] = new UnreadableReader();
dispatchMacros[_] = new DiscardReader();

2.24.1 The Var reader macro

Vars have a special reader macro # so that the input form #foo expands into
(var foo). The Symbol foo must be a Var and the Var object itself, not its value,
2.24. READER DISPATCH MACROS 85

is returned.
The single-quote () array entry contains a [85] VarReader function.
LispReader dispatchMacros VarReader statement

dispatchMacros[\] = new VarReader();

The VarReader invoke method reads the next object and returns the Var refer-
ence. (AFn [509])
LispReader VarReader class

public static class VarReader extends AFn{

public Object invoke(Object reader, Object quote)
throws Exception{
PushbackReader r = (PushbackReader) reader;
Object o = read(r, true, null, true);
// if(o instanceof Symbol)
// {
// Object v =
// Compiler.maybeResolveIn(Compiler.currentNS(),
// (Symbol) o);
// if(v instanceof Var)
// return v;
// }
return RT.list(THE_VAR, o);
}
}

The # macro is useful for naming functions or variables in another namespace.

You can call function across namespaces by writing

(#mynamespace/functionname ...)

2.24.2 The Regular Expression reader macro

(AFn [509])
LispReader RegexReader class

public static class RegexReader extends AFn{

static StringReader stringrdr = new StringReader();

public Object invoke(Object reader, Object doublequote)

throws Exception{
StringBuilder sb = new StringBuilder();
Reader r = (Reader) reader;
86 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

for(int ch = r.read(); ch != "; ch = r.read())

{
if(ch == -1)
throw new Exception("EOF while reading regex");
sb.append( (char) ch );
if(ch == \\) //escape
{
ch = r.read();
if(ch == -1)
throw new Exception("EOF while reading regex");
sb.append( (char) ch ) ;
}
}
return Pattern.compile(sb.toString());
}
}

2.24.3 The Function reader macro

(AFn [509])
LispReader FnReader class

public static class FnReader extends AFn{

public Object invoke(Object reader, Object lparen)
throws Exception{
PushbackReader r = (PushbackReader) reader;
if(ARG_ENV.deref() != null)
throw new IllegalStateException(
"Nested #()s are not allowed");
try
{
Var.pushThreadBindings(
RT.map(ARG_ENV, PersistentTreeMap.EMPTY));
r.unread(();
Object form = read(r, true, null, true);

PersistentVector args = PersistentVector.EMPTY;

PersistentTreeMap argsyms =
(PersistentTreeMap) ARG_ENV.deref();
ISeq rargs = argsyms.rseq();
if(rargs != null)
{
int higharg =
(Integer) ((Map.Entry) rargs.first()).getKey();
if(higharg > 0)
{
for(int i = 1; i <= higharg; ++i)
2.24. READER DISPATCH MACROS 87

{
Object sym = argsyms.valAt(i);
if(sym == null)
sym = garg(i);
args = args.cons(sym);
}
}
Object restsym = argsyms.valAt(-1);
if(restsym != null)
{
args = args.cons(Compiler._AMP_);
args = args.cons(restsym);
}
}
return RT.list(Compiler.FN, args, form);
}
finally
{
Var.popThreadBindings();
}
}
}

2.24.4 The Set reader macro

(AFn [509])
LispReader SetReader class

public static class SetReader extends AFn{

public Object invoke(Object reader, Object leftbracket)
throws Exception{
PushbackReader r = (PushbackReader) reader;
return PersistentHashSet
.createWithCheck(readDelimitedList(}, r, true));
}

2.24.5 The Eval reader macro

(AFn [509])
LispReader EvalReader class

public static class EvalReader extends AFn{

88 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

public Object invoke(Object reader, Object eq) throws Exception{

if (!RT.booleanCast(RT.READEVAL.deref()))
{
throw new Exception(
"EvalReader not allowed when *read-eval* is false.");
}

PushbackReader r = (PushbackReader) reader;

Object o = read(r, true, null, true);
if(o instanceof Symbol)
{
return RT.classForName(o.toString());
}
else if(o instanceof IPersistentList)
{
Symbol fs = (Symbol) RT.first(o);
if(fs.equals(THE_VAR))
{
Symbol vs = (Symbol) RT.second(o);
//Compiler.resolve((Symbol) RT.second(o),true);
return RT.var(vs.ns, vs.name);
}
if(fs.name.endsWith("."))
{
Object[] args = RT.toArray(RT.next(o));
return
Reflector.invokeConstructor(
RT.classForName(
fs.name.substring(0, fs.name.length() - 1)),
args);
}
if(Compiler.namesStaticMember(fs))
{
Object[] args = RT.toArray(RT.next(o));
return
Reflector.invokeStaticMethod(
fs.ns, fs.name, args);
}
Object v =
Compiler.maybeResolveIn(Compiler.currentNS(), fs);
if(v instanceof Var)
{
return ((IFn) v).applyTo(RT.next(o));
}
throw new Exception("Cant resolve " + fs);
}
else
throw new IllegalArgumentException("Unsupported #= form");
}
}
2.25. VARS 89

2.24.6 The Unreadable reader macro

(AFn [509])
LispReader UnreadableReader class

public static class UnreadableReader extends AFn{

public Object invoke(Object reader, Object leftangle)
throws Exception{
throw new Exception("Unreadable form");
}
}

2.24.7 The Discard reader macro

(AFn [509])
LispReader DiscardReader class

public static class DiscardReader extends AFn{

public Object invoke(Object reader, Object underscore)
throws Exception{
PushbackReader r = (PushbackReader) reader;
read(r, true, null, true);
return r;
}
}

2.25 Vars

2.26 Transients
Transients are intended to make your code faster. Transients ignore structural
sharing so time and space is saved. The underlying data structure is different.
For example, we thread a persistent map through some associations.
transients example

(-> {} (assoc :a 1) (assoc :b 2) (assoc :c 3))

90 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

-
which results in the persistent map
transients example

{:c 3, :b 2, :a 1}

A transient version is similar but transients are not mutable.

transients example

(-> (transient {}) (assoc! :a 1) (assoc! :b 2) (assoc! :c 3) persistent!)

defn transient

(defn transient
"Alpha - subject to change.
Returns a new, transient version of the collection, in constant time."
{:added "1.1"
:static true}
[^clojure.lang.IEditableCollection coll]
(.asTransient coll))

The difference is that the transient call makes the persistent map is transformed
to a transient map. The assoc! call operates similar to assoc. The persistent!
call transforms the result to a persistent map.
There are several function that operate on transients, persistent!, conj!, as-
soc!, dissoc!, pop!, and disj!.

defn persistent!

(defn persistent!
"Alpha - subject to change.
Returns a new, persistent version of the transient collection, in
constant time. The transient collection cannot be used after this
call, any such use will throw an exception."
{:added "1.1"
:static true}
[^clojure.lang.ITransientCollection coll]
(.persistent coll))
2.26. TRANSIENTS 91

defn conj!

(defn conj!
"Alpha - subject to change.
Adds x to the transient collection, and return coll. The addition
may happen at different places depending on the concrete type."
{:added "1.1"
:static true}
[^clojure.lang.ITransientCollection coll x]
(.conj coll x))

defn assoc!

(defn assoc!
"Alpha - subject to change.
When applied to a transient map, adds mapping of key(s) to
val(s). When applied to a transient vector, sets the val at index.
Note - index must be <= (count vector). Returns coll."
{:added "1.1"
:static true}
([^clojure.lang.ITransientAssociative coll key val]
(.assoc coll key val))
([^clojure.lang.ITransientAssociative coll key val & kvs]
(let [ret (.assoc coll key val)]
(if kvs
(recur ret (first kvs) (second kvs) (nnext kvs))
ret))))

defn dissoc!

(defn dissoc!
"Alpha - subject to change.
Returns a transient map that doesnt contain a mapping for key(s)."
{:added "1.1"
:static true}
([^clojure.lang.ITransientMap map key] (.without map key))
([^clojure.lang.ITransientMap map key & ks]
(let [ret (.without map key)]
(if ks
92 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

(recur ret (first ks) (next ks))

ret))))

defn pop!

(defn pop!
"Alpha - subject to change.
Removes the last item from a transient vector. If
the collection is empty, throws an exception. Returns coll"
{:added "1.1"
:static true}
[^clojure.lang.ITransientVector coll]
(.pop coll))

defn disj!

(defn disj!
"Alpha - subject to change.
disj[oin]. Returns a transient set of the same (hashed/sorted) type,
that does not contain key(s)."
{:added "1.1"
:static true}
([set] set)
([^clojure.lang.ITransientSet set key]
(. set (disjoin key)))
([set key & ks]
(let [ret (disj set key)]
(if ks
(recur ret (first ks) (next ks))
ret))))

The imperative style will not work everywhere. For instance,

transients example

(let [a (transient {})]

(dotimes [i 20] (assoc! a i i))
(persistent! a))

-
2.26. TRANSIENTS 93

returns with the broken result

transients example

(0 0, 1 1, 2 2, 3 3, 4 4, 5 5, 6 6, 7 7}

-
which is only the first 8 values of the map.
However, if it is done in a recursive style, as in
transients example

(persistent!
(reduce (fn[m v] (assoc! m v v))
(transient {})
(range 1 21)))

-
yields all twenty pairs.
transients example

(0 0, 1 1, 2 2, 3 3, 4 4, 5 5, 6 6, 7 7, 8 8, 9 9, 10 10, 11 11, 12 12,

13 13, 14 14, 15 15, 16 16, 17 17, 18 18, 19 19, 20 20)

Transients can only be modified by the thread that creates them. Attemping to
access them will throw a java.util.concurrent.ExecutionException
transients example

@(let [c (transient [])] (future (conj! c :a)))

It is also not possible to modify a Transient after the call to persistent! will
throw a java.lang.IllegalAccessError exception. So this will not work:
transients example

(let [c (transient #{})] (persistent! c) (conj! c :a))

Transients do not work on every data structure. Lists, sets, and maps all throw
a java.lang.ClassCastException
transients example

(transient ())
(transient (sorted-set))
(transient (sorted-map))
94 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

The Transient data structure must support clojure.lang.IEditableCollection.

2.27 Atoms

2.28 Refs

2.29 Agents

2.30 Promise and Deliver

A Promise is a mechanism for storing data and reading this data across threads.
The promise function creates a promise that can be stored as in:
promise example

(def prom (promise))

defn promise

(defn promise
"Alpha - subject to change.
Returns a promise object that can be read with deref/@, and set,
once only, with deliver. Calls to deref/@ prior to delivery will
block. All subsequent derefs will return the same delivered value
without blocking."
{:added "1.1"
:static true}
[]
(let [d (java.util.concurrent.CountDownLatch. 1)
v (atom nil)]
(reify
clojure.lang.IDeref
(deref [_] (.await d) @v)
clojure.lang.IPromiseImpl
(hasValue [this]
(= 0 (.getCount d)))
clojure.lang.IFn
(invoke [this x]
(locking d
(if (pos? (.getCount d))
(do (reset! v x)
2.30. PROMISE AND DELIVER 95

(.countDown d)
this)
(throw (IllegalStateException.
"Multiple deliver calls to a promise"))))))))

To fulfill a promise we use the deliver function. So we would use

promise example

(deliver prom 1)

defn deliver

(defn deliver
"Alpha - subject to change.
Delivers the supplied value to the promise, releasing any pending
derefs. A subsequent call to deliver on a promise will throw an
exception."
{:added "1.1"
:static true}
[promise val] (promise val))

dereference @ promise To see the delivered value we deference the variable.

promise example

@prom

-
Note that dereferencing a promise before the value is delivered will block the
current thread. So you can have a future deliver on the promise from another
thread and wait on the result.
promise example

(do (future
(Thread/sleep 5000)
(deliver prom 1))
@prom)

IllegalStateException IllegalStateException promise Attemping to deliver a value

more than once will throw a java.lang.IllegalStateException
96 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

2.31 Futures
Futures are a mechanism for executing a function in a different thread and
retrieving the results later.
Futures are useful for long running tasks that should not block, such as a UI
thread, communicating across networks, or a task that has a side-effect that
does not affect the main computation, such as displaying an image or printing.
For example,
future example
(do
(future
(Thread/sleep 3000)
(sendToPrinter "resume.pdf"))
(messages/plain-message "file being printed"))

A future can return a value. We define a var that will hold the value of a future
as in:

defn future-call
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; futures (needs proxy);;;;;;;;;;;;;;;;;;
(defn future-call
"Takes a function of no args and yields a future object that will
invoke the function in another thread, and will cache the result and
return it on all subsequent calls to deref/@. If the computation has
not yet finished, calls to deref/@ will block."
{:added "1.1"
:static true}
[f]
(let [f (binding-conveyor-fn f)
fut (.submit clojure.lang.Agent/soloExecutor ^Callable f)]
(reify
clojure.lang.IDeref
(deref [_] (.get fut))
java.util.concurrent.Future
(get [_] (.get fut))
(get [_ timeout unit] (.get fut timeout unit))
(isCancelled [_] (.isCancelled fut))
(isDone [_] (.isDone fut))
(cancel [_ interrupt?] (.cancel fut interrupt?)))))

defmacro future
2.31. FUTURES 97

(defmacro future
"Takes a body of expressions and yields a future object that will
invoke the body in another thread, and will cache the result and
return it on all subsequent calls to deref/@. If the computation has
not yet finished, calls to deref/@ will block."
{:added "1.1"}
[& body] (future-call (^{:once true} fn* [] ~@body)))

future var

(def result
(future
(Thread/sleep 3000)
"The future has arrived"))

dereference @ future We can get the value of this var using the dereference
operator @.
future dereference

@result

CancellationException CancellationException future If we dereference the var

before the future completes then the dereference blocks until the result is avail-
able. The result retains its value so a second dereference will return immedi-
ately. If the future is cancelled (see below), then the dereference will throw a
java.util.concurrent.CancellationException.
future? We can test if a var is a future with the future? predicate.
future dereference

(future? result) ==> true

defn future?

(defn future?
"Returns true if x is a future"
{:added "1.1"
:static true}
[x] (instance? java.util.concurrent.Future x))
98 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

future-done? We can test if the future has completed witht he future-done?

predicate.
future done

(future-done? result) ==> true

defn future-done?

(defn future-done?
"Returns true if future f is done"
{:added "1.1"
:static true}
[^java.util.concurrent.Future f] (.isDone f))

A pending future can be cancelled with the future-cancel function

future cancel

(future-cancel result) ==> true

-
which, if it successfully cancels the future returns true.

defn future-cancel

(defn future-cancel
"Cancels the future, if possible."
{:added "1.1"
:static true}
[^java.util.concurrent.Future f] (.cancel f true))

We can test if the future was cancelled with the future-cancelled? function
future cancelled

(future-cancelled? result) ==> true

-
2.32. MULTIMETHODS 99

which, if the future was cancelled, will return true.

defn future-cancelled?

(defn future-cancelled?
"Returns true if future f is cancelled"
{:added "1.1"
:static true}
[^java.util.concurrent.Future f] (.isCancelled f))

2.32 MultiMethods

2.33 Deftype

2.34 Defrecord
Define a record object with slots named :a, :b, and :c

> (defrecord Foo [a b c])

user.Foo

This makes a Java class Foo in the Java package user.

Make a Var reference to a our new record object.

> (def f (Foo. 1 2 3))

#:user.Foo{:a 1, :b 2, :c 3}

Access the value of the :b slot in the record instance f

> (:b f)
2
or

> (.b f)
2

What is the Var f actually pointing to? It points to a class object.

> (class f)
user.Foo
100 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

We can apply normal functions to the query information about the class of f.

> (supers (class f))

#{clojure.lang.ILookup
java.lang.Object
java.util.Map
clojure.lang.Seqable
clojure.lang.IKeywordLookup
java.lang.Iterable
clojure.lang.IMeta
clojure.lang.IPersistentCollection
clojure.lang.IPersistentMap
clojure.lang.Counted
clojure.lang.IObj
java.io.Serializable
clojure.lang.Associative}

We see that the Var f references a record class which has a number of interesting
properties. In particular, given this record we know many things about its
properties.

[797] ILookup
Object Since records are Java Objects they can be compared with the
default equals method and can be stored in containers or passed as ar-
guments.
Map Since records implement Map they provide three collection views, as
a set of keys, as a set of values, and as a set of key-value mappings.
[1140] Seqable
[796] IKeywordLookup
Iterable Since records are Iterable they implement an iterator method
that allows them to be the target of a foreach statement.
[799] IMeta
[800] IPersistentCollection
[801] IPersistentMap
[768] Counted
[800] IObj
Serializable Since records are Serializable the implement writeObject and
readObject methods that make it possible to stream the state of the
object out and recreate it by reading it back.
[576] Associative
2.35. PROTOCOLS 101

2.35 Protocols
> (defprotocol coerce
coerce between things
(as-file [x] Coerce argument to a file)
(as-url [x] Coerce argument to a URL))
coerce

Defprotocol defines a named set of generic functions which dispatch on the type
of the first argument. The functions are defined in the same namespace (and
thus the same Java package) as the protocol.
Similar to Java interfaces but are functions, not associated with classes. In this
example, we mimic the Java idioms. A record is like class and coerce is like an
interface so we are saying that the class Name implements the as-file method
of interface coerce.
Extending inline

> (defrecord Name [n]

coerce
(as-file [] (File. n)))
user.Name

> (def n (Name. johndoe))

#user/n

> (as-file n)
#File johndoe

Extending protocol to classes that already exist

Protocols can implement methods without classes by associating them with
types of their first argument. So we extend the coerce protocol to work with
nil arguments and String arguments.

> (extend-protocol coerce

nil
(as-file [] nil)
(as-url [] nil)
String
(as-file [s] (File. s))
(as-url [s] (URL. s)))
nil
102 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

> (as-file data.txt)

#File data.txt

Extending existing classes to add protocols

> (defprotocol MyNewProtocol (myNewFunction [x] answer))

MyNewProtocol

> (extend-type String

coerce
(as-file [s] (File. s))
(as-url [s] (URL. s))
MyNewProtocol
(myNewMethod [x] (str answer: x)))
nil

> (myNewFunction test)

answer

> (extend-type Long MyNewProtocol (myNewFunction [x] (+ x 1)))

nil

> (myNewFunction 1)
2

2.36 Prototypes

2.37 Genclass
2.37.1 Overriding protected methods
In order to override a protected method and call the superclass you need to
make the protected method available under an alternate name. For examle, to
extend Java class Foo with fooMethod(bar) you would use

(ns.com.example.subclass-of-Foo
(:gen-class :extends com.example.Foo
:exposes-methods {fooMethod fooSuperMethod}))

(defn -fooMethod [this parameter]

(fooSuperMethod this parameter))
2.38. PROXIES 103

2.38 Proxies

2.39 Macros

2.40 Iterators
(Iterator [1723])
PersistentTreeMap NodeIterator Class

static public class NodeIterator implements Iterator{

Stack stack = new Stack();
boolean asc;

NodeIterator(Node t, boolean asc){

this.asc = asc;
push(t);
}

void push(Node t){

while(t != null)
{
stack.push(t);
t = asc ? t.left() : t.right();
}
}

public boolean hasNext(){

return !stack.isEmpty();
}

public Object next(){

Node t = (Node) stack.pop();
push(asc ? t.right() : t.left());
return t;
}

public void remove(){

throw new UnsupportedOperationException();
}
}

(Iterator [1723])
PersistentTreeMap KeyIterator Class

static class KeyIterator implements Iterator{

104 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION

NodeIterator it;

KeyIterator(NodeIterator it){
this.it = it;
}

public boolean hasNext(){

return it.hasNext();
}

public Object next(){

return ((Node) it.next()).key;
}

public void remove(){

throw new UnsupportedOperationException();
}
}

(Iterator [1723])
PersistentTreeMap ValIterator Class

static class ValIterator implements Iterator{

NodeIterator it;

ValIterator(NodeIterator it){
this.it = it;
}

public boolean hasNext(){

return it.hasNext();
}

public Object next(){

return ((Node) it.next()).val();
}

public void remove(){

throw new UnsupportedOperationException();
}
}

-
2.41. GENERATING JAVA CLASSES 105

2.41 Generating Java Classes

2.42 Type Hints

2.43 Native Arithmetic

106 CHAPTER 2. FROM IDEAS TO IMPLEMENTATION
Chapter 3

Writing Idiomatic Clojure

107
108 CHAPTER 3. WRITING IDIOMATIC CLOJURE
Chapter 4

The Ants Demo (Kai Wu)

This is adapted from Wu [1]

4.1 The Simulation World

The first part of ants.clj sets up the simulation world, where well be introduced
to some of Clojures powers.

4.1.1 Initial setup of constants/magic-numbers

After the copyright notice, the initial setup code of ants.clj is easy to under-
stand (for coders at least), even if youve never dealt with Lisp before. We see
parameters (aka constants and magic numbers) being defined for later use using
Clojures special form: def creates a var (a mutable storage location) which
connects a symbol to a value in the current namespace.

sim-world-setup

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; ant sim ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

;; Copyright (c) Rich Hickey. All rights reserved.
;;
;; The use and distribution terms for this software are covered by the
;; Common Public License 1.0 (http://opensource.org/licenses/cpl.php)
;; which can be found in the file CPL.TXT at the root of this distribution.
;; By using this software in any fashion, you are agreeing to be bound by
;; the terms of this license.
;;
;; You must not remove this notice, or any other, from this software.

109
110 CHAPTER 4. THE ANTS DEMO (KAI WU)

;; Set dimensions of the world, as a square 2-D board:

(def dim 80)
;; Number of ants = nants-sqrt^2
(def nants-sqrt 7)
;; Number of places with food:
(def food-places 35)
;; Range of amount of food at a place:
(def food-range 100)
;; Scale factor for pheromone drawing:
(def pher-scale 20.0)
;; Scale factor for food drawing:
(def food-scale 30.0)
;; Evaporation rate:
(def evap-rate 0.99)

(def animation-sleep-ms 100)

(def ant-sleep-ms 40)
(def evap-sleep-ms 1000)

(def running true)

4.1.2 The board: ready to mutate via transactions

Things get more interesting once the actual simulation environment needs defin-
ing:
cell

(defstruct cell :food :pher) ; May also have :ant and :home values

First, a call to defstruct (like a hashmap or dictionary in other languages)

defines a baseline cell.

defstruct is like a very lightweight class or constructor/template function,

and conveniently wraps Clojures create-struct.
Here, a cell has two keys to start, :food and :pher, to indicate the presence
of food and pheromones. A cell may also have keys of :ant and :home,
depending on whether an ant and/or the home-colony is present.

Next, the world function creates the 2-dimensional board of cells (here, a
square of 80x80 cells), represented as vectors (rows or the vertical y-dimension)
of a vector (the horizontal x-dimension columns in one row):
sim-world-board-creation
4.1. THE SIMULATION WORLD 111

;; World is a 2d vector of refs to cells

(def world
(apply vector
(map (fn [_]
(apply vector
(map (fn [_]
(ref (struct cell 0 0)))
(range dim))))
(range dim))))

-
Reading the above:

Start with the innermost map call, which uses an anonymous function to
create one column of 80 cells, per (range dim). The struct returns a
new structmap instance using the earlier cell as the basis, initializing the
:food and :pher values to zero.
But notice that struct is wrapped with a transactional ref, and heres
the first glimpse of Clojures concurrency powers. With each cell being
stateful (possibly time-varying values of :food, :pher, :ant, and :home
values) and with multiple threads updating the board and board elements,
wed typically think of using locks on each cell when updating its state.
But in Clojure with its software transactional memory (STM), we just
use ref for safe references to mutable collections (here, a struct) - all
changes to a cell will then be atomic, consistent, and isolated! 1 Like
using an RDBMS, you dont need to manually manage concurrency.
Once you understand the innermost (ref (struct cell 0 0 )) map call,
the rest of (def world...) is straightforward: apply uses vector as a
constructor function with the map function producing the vectors argu-
ments, creating a column in the 2-D board.
Then the pattern is repeated in the outermost (apply vector (map...))
call, creating all the columns of the 2-D board.
Note that as defined, each vector in world (again, a 2-D vector of vectors)
corresponds to an x-position, and of course, within that vector are the y-
positions (here, a total of 80 cells).

The place function is a selector function (think of place as the noun, not the
verb) returning particular cells in the 2-D world. Once we have a cell, we can
then mutate it to represent ants, food, and pheromones (or their absence):
place
1 STM is like a memory-only SQL database, thus the last property of being
durable/persistent wont be satisfied.
112 CHAPTER 4. THE ANTS DEMO (KAI WU)

(defn place [x y]
(-> world (nth x) (nth y)))

place takes a single vector argument (having two elements x and y), then
applies the thrush operator (the arrow-like >) on the world object, first
selecting the column (nth x) on world, then the row (nth y) on that
column.

Aside: the thrush operator

The thrush operator helps make code more concise, and arguably clearer: in-
stead of reading code inside-out to mentally evaluate it, we can read it left-
to-right. 2 Consider how the equivalent place function would look without
thrushing:

(defn place-verbose [[x y]]

(nth (nth world x) y))

4.1.3 Ants as agents - doing asynchronous uncoordinated

changes
Next well consider the active things in ants.clj, the ants themselves. As
before, we start with defstruct, defining an ant as having only one required
key, its direction. (An ant may temporarily have another key, :food.)

ants-defined

(defstruct ant :dir) ; Always has dir heading; may also have :food

(defn create-ant
"Create an ant at given location, returning an ant agent on the location."
[location direction]
(sync nil
(let [the-place (place location)
the-ant (struct ant direction)]
(alter the-place assoc :ant the-ant)
(agent location))))

-
2 Apparently Clojures thrush is not quite a true thrush, see Michael Fogus article at
blog.fogus.me/2010/09/28/thrush-in-clojure-redux
4.1. THE SIMULATION WORLD 113

To explain the above constructor function for ants, create-ant:

Takes two arguments, location and direction. location will be a vector

[x y], and as we saw, passed on to the place function as an argument;
direction is a number from 0-7 inclusive corresponding to one of the
eight cardinal directions.
More concurrency support: the sync function takes a flags argument
(as of Clojure 1.3, its still ignored so just pass nil), and then a list of
expressions that will be executed together atomically (all or nothing) as
a transaction.
The let special form binds pairs of symbols and expressions in its ar-
guments vector, providing local, lexical bindings within the scope of the
body following.
sync will ensure that any mutations of refs using the alter function will
be atomic. Previously we had used ref around each cell, so in the above
code where the-place is such a ref-wrapped cell, alter takes the-place
ref as its first argument, then assoc as the function to be applyed on the-
place, tying a new ant instance to it (remember that as a cell, the-place
is sure to have :food and :pher key-values already, now we add :ant).
Like the thrush operator earlier, the syntax of alter enables convenient
left-to-right reading.
Finally, the agent function. What are Clojure agents? To quote the
docs,
Agents provide shared access to mutable state. They allow non-
blocking (asynchronous as opposed to synchronous atoms) and
independent change of individual locations (unlike coordinated
change of multiple locations through refs).

Clojures agent function takes one required argument of state, returning an

agent object with initial value of that given state. Here, as the last line of
create-ant, agent effectively returns the ant object at its starting location.
Ants as agents make sense: we expect them to move around independently (i.e.
asynchronously) in the simulation world.

4.1.4 Setting up the home, and ants

The home of the ants is not a single cell on the world-board, but a square of cells,
with its top-left corner offset from the origin (0, 0). Its sides are proportional
to the number of ants because the home square will initially contain all the ants
- one ant per cell - before the simulation runs. We can see these two aspects of
the home-square in the two def calls for home-offset and home-range below.
114 CHAPTER 4. THE ANTS DEMO (KAI WU)

home-setup

(def home-offset (/ dim 4))

(def home-range (range home-offset (+ nants-sqrt home-offset)))

(defn setup
"Places initial food and ants, returns seq of ant agents."
[]
(sync nil
(dotimes [i food-places]
(let [p (place [(rand-int dim) (rand-int dim)])]
(alter p assoc :food (rand-int food-range))))
(doall
(for [x home-range y home-range]
(do
(alter (place [x y]) assoc :home true)
(create-ant [x y] (rand-int 8)))))))

The setup functions docstring tells us what its doing, so on to the details:

setup takes no arguments.

As we saw before in create-ant, the sync function wraps a sequence of
expressions that together should be executed atomically, all-or-nothing.
Setup initial food: The dotimes function takes two arguments, the first
a vector [name n] with n being the number of times that the body
(the second argument) will be repeatedly executed, usually for its side-
effects/mutations.
Here, the unused name i is bound to the integers from 0 to 34, since
we had specified food-places as 35 initially.
The body is clear enough: bind p to the randomly chosen place on
the world-board (using the rand-int function for (x, y). The already-
seen alter function modifies that p to have a random amount of food
value.
Placing the ants in their starting positions: The doall function forces
immediate evaluation of a lazy sequence - in this case the lazy sequence
produced by the for function.
Here, the for functions first argument is: two binding-form/collection-
expr pairs for every x and y position within the square of the ants
home.
4.1. THE SIMULATION WORLD 115

The for functions second argument is the body-expression, here

wrapped in the do special form which ensures order of evaluation
(usually, of expressions having side-effects): designate the place as a
home position, then create an ant on that place with a random initial
direction.

In sum, the setup function shows how to deal with state and its mutation in
Clojure: we started with a 2-D world-board of places (cells) as Clojure refs;
then we modify/mutate each place using alter. We can use various looping
functions such as dotimes and doall to process a batch of state-mutations (of
the world-board) atomically and consistently.

4.1.5 Orientation and moving around the world

Next, consider facing/orientation and moving to another place in the 2-D world.
Three functions below, followed by explanations:

world-wrapping

(defn bound
"Returns given n, wrapped into range 0-b"
[b n]
(let [n (rem n b)]
(if (neg? n)
(+ n b)
n)))

;; Directions are 0-7, starting at north and going clockwise. These are
;; the 2-D deltas in order to move one step in a given direction.
(def direction-delta {0 [0 -1]
1 [1 -1]
2 [1 0]
3 [1 1]
4 [0 1]
5 [-1 1]
6 [-1 0]
7 [-1 -1]})

(defn delta-location
"Returns the location one step in the given direction. Note the
world is a torus."
[[x y] direction]
(let [[dx dy] (direction-delta (bound 8 direction))]
[(bound dim (+ x dx)) (bound dim (+ y dy))]))

-
116 CHAPTER 4. THE ANTS DEMO (KAI WU)

With the 2-D world board, we have the 8 cardinal directions (North, North-East,
East, etc.), and board edges that wrap-around to the opposite side - like the old
arcade games of the 1980s, e.g. Pac-Man and Asteroids. The functions bound
and delta-location help enforce these world-behaviors, while the definition of
direction-delta maps a movement in a cardinal direction to the corresponding
change in x and y. A few comments on each:

The bound function using the built-in rem (i.e. remainder) function is
straightforward. Observe how bound is used in delta-location to ensure
wrap-around behavior in: 1) cardinal directions; 2) the world-board, at
its edges given by dim.
direction-delta maps the eight cardinal directions (0 is North) to the
corresponding changes in [x y]. Note the syntax: its an array-map literal,
where the order of insertion of key-value pairs (here, keys 0-7) will be
preserved.
delta-location takes the current [x y] location and a direction, returning
the new corresponding location on the world-board.

4.1.6 Ant-agent behavior functions

In Hickeys simulation, ants need to move (rotation and translation), pick up
and drop-off food, and make rudimentary decisions.

Ant movements

Our ants need two behaviors to get around their world: turning (or changing
the direction they face), and stepping forward. Lets deal with turning first:

ant-agent-turn

;ant agent functions

; An ant agent tracks the location of an ant, and controls the
; behavior of the ant at that location.

(defn turn
"Turns the ant at the location by the given amount."
[loc amt]
(dosync
(let [p (place loc)
ant (:ant @p)]
(alter p assoc :ant (assoc ant :dir (bound 8 (+ (:dir ant) amt))))))
loc)
4.1. THE SIMULATION WORLD 117

The turn function takes two arguments, location and the amount of turn.
Whats interesting is the usage of the dosync function, which ensures the ants
turn - the changes of state within the assoc function calls - is all-or-nothing.
The ant gets a new direction per the innermost assoc, then the outermost assoc
updates the place with the updated ant.
Now for actual movement to a new place:

ant-agent-move
(defn move
"Moves the ant in the direction it is heading. Must be called in a
transaction that has verified the way is clear."
[startloc]
(let [oldp (place startloc)
ant (:ant @oldp)
newloc (delta-location startloc (:dir ant))
newp (place newloc)]
;; move the ant
(alter newp assoc :ant ant)
(alter oldp dissoc :ant)
;; leave pheromone trail
(when-not (:home @oldp)
(alter oldp assoc :pher (inc (:pher @oldp))))
newloc))

The move function changes state of both the ant and board, thus the doc-string
note that it must be called in a transaction. The code is self-explanatory, though
if pheromone is a new term to you, youll want to learn about a dominant
form of chemical communication on Earth. Whenever our artificial ant is not
within its home, it will secrete pheromone (inc the :pher value by 1) at the
place it just left, making it easier (more likely) for it and other ants to travel
between home and food locations in the future (instead of doing a completely
random walk).

Ants and food

When an ant finds food, it picks up one unit of it; when it returns home with
a food unit, it will drop its food there. These two interactions (each having
two steps) change the board, and as with the move function, they need to occur
atomically (all-or-nothing) to ensure the world is in a consistent state.

ant-agent-food
118 CHAPTER 4. THE ANTS DEMO (KAI WU)

(defn take-food [loc]

"Takes one food from current location. Must be called in a
transaction that has verified there is food available."
(let [p (place loc)
ant (:ant @p)]
(alter p assoc
:food (dec (:food @p))
:ant (assoc ant :food true))
loc))

(defn drop-food [loc]

"Drops food at current location. Must be called in a
transaction that has verified the ant has food."
(let [p (place loc)
ant (:ant @p)]
(alter p assoc
:food (inc (:food @p))
:ant (dissoc ant :food))
loc))

Notice how similar the structure is for the two functions above; possibly theyre
candidates for macro refactoring.

Ant judgment

Our ants need some decision-making for their overall task of finding food and
bringing it home. As well see shortly, an ants behavior is based on two states,
either:

1. The ant does not have food, and is looking for it. In this mode, it weighs
the three map locations ahead of it (ahead, ahead-left, ahead-right) by the
presence of either food or pheromone.
2. The ant has food, and needs to bring it to the home box/location. Now
it weighs which of the three ahead-positions to take by the presence of
pheromone, or home.

So we need functions to express preference of the next location for an ant. The
functions rank-by and wrand help with that.

ant-agent-judgment-1

(defn rank-by
"Returns a map of xs to their 1-based rank when sorted by keyfn."
[keyfn xs]
4.1. THE SIMULATION WORLD 119

(let [sorted (sort-by (comp float keyfn) xs)]

(reduce (fn [ret i] (assoc ret (nth sorted i) (inc i)))
{} (range (count sorted)))))

The rank-by function gives weights to where an ant will move next in the
simulation world. It takes two arguments, keyfn and xs - but what do those
args look like, and where is rank-by used? In the behave function below;
youll see that the keyfn checks for the presence of :food, :pher, or :home
- in the three cells (board locations) of the xs vector of [ahead ahead-left
ahead-right]. 3

The (sort-by keyfn coll) function returns a sorted sequence of items in

coll, ordered by comparing (keyfn item). Here, for the local value sorted,
it will be ascending order of cells/places, by their :food/:home/:pher values
- each of those is valuable to an ant depending on whether its looking for
food, or bringing it home.
The (reduce f initial-val coll) functionn in its 3-arguments form here
has its 1st argument f as a function taking two arguments, the current/initial-
val value and the next/first item from coll. In this case, it will build-
up a map from the local sorted value, with the keys being the ranked
cells/places, and the values being integers 1, 2 and 3. To get a sense of
whats going on, try this on your Clojure REPL:

(let [sorted [0 0.7 1.0]]

(reduce (fn [ret i] (assoc ret (nth sorted i) (inc i)))
{}
(range (count sorted))))
;; You should see {1.0 3, 0.7 2, 0 1}
;;
;; Within the behave function below, the return value might be
;; like {<cell-ahead-left> 3, <cell-ahead-right> 2, <cell-ahead> 1}
;; or similar.

Next: The wrand function helps with the larger task of randomizing which
location/cell the ant moves to next in a weighted manner; i.e. the dice are
loaded with rank-by, then rolled here:

ant-agent-judgment-2
3 Remember that :food, :pher, and :home are mutually exclusive in a cell. When an ant
wants to go home with food, and the home cell(s) is ahead of it, it will always go home, there
wont be competing :pher presence.
120 CHAPTER 4. THE ANTS DEMO (KAI WU)

(defn wrand
"Given a vector of slice sizes, returns the index of a slice given a
random spin of a roulette wheel with compartments proportional to
slices."
[slices]
(let [total (reduce + slices)
r (rand total)]
(loop [i 0 sum 0]
(if (< r (+ (slices i) sum))
i
(recur (inc i) (+ (slices i) sum))))))

How is wrand used? Like rank-by, look in the behave function: its single
argument of slices is a vector of 3 integers (from rank-by above), corresponding
to the relative desirability of the 3 cells ahead of the ant. So if the slices argument
looked like [0 3 1], that would correspond to zero probability of moving ahead,
and 3/4 chance moving to the ahead-left cell over the ahead-right cell.

The let value total uses reduce to set the upper bound on the random
number; loosely like setting the maximum number of faces on the die to
be rolled (albeit that some die numbers are geometrically impossible).
The rand function returns a random floating point number from 0 (inclu-
sive) to n (exclusive).
Heres the only looping construct in the entire ants program: its analogous
to checking which compartment of the roulette wheel the ball fell in. The
if checks if r fell into the current pocket - the size of which is given by
(slices i). If yes, return the index corresponding to that pocket; if not,
check the next pocket/slice.

Tying it all together: the behave function for ants

The behave function below is the largest one, so it helps to keep in mind its
main parts while diving into details:

1. let values - help with readability.

2. Thread/sleep - helps slow down ants in the UI display.

3. dosync - ensures ants behavior is transactional, all-or-nothing.
4. if branch: main logic for an ant, if ant has :food take it home, otherwise
look for food.
4.1. THE SIMULATION WORLD 121

Also, consider the context of how behave is first used: within the main invo-
cation at the end, theres the expression:

(dorun (map #(send-off % behave) ants))

So the behave function is called on every ant agent via the send-off function,
which is how Clojure dispatches potentially blocking actions to agents. And
there certainly are potentially blocking actions when using behave, since ants
may try to move into the same cell, try to acquire the same food, etc.

ant-agent-behave

(defn behave
"The main function for the ant agent."
[loc]
(let [p (place loc)
ant (:ant @p)
ahead (place (delta-location loc (:dir ant)))
ahead-left (place (delta-location loc (dec (:dir ant))))
ahead-right (place (delta-location loc (inc (:dir ant))))
places [ahead ahead-left ahead-right]]
;; Old way of Java interop: (. Thread (sleep ant-sleep-ms))
;; New idiomatic way is,
(Thread/sleep ant-sleep-ms)
(dosync
(when running
(send-off *agent* #behave))
(if (:food ant)
;; Then take food home:
(cond
(:home @p)
(-> loc drop-food (turn 4))
(and (:home @ahead) (not (:ant @ahead)))
(move loc)
:else
(let [ranks (merge-with +
(rank-by (comp #(if (:home %) 1 0) deref) places)
(rank-by (comp :pher deref) places))]
(([move #(turn % -1) #(turn % 1)]
(wrand [(if (:ant @ahead) 0 (ranks ahead))
(ranks ahead-left) (ranks ahead-right)]))
loc)))
;; No food, go foraging:
(cond
(and (pos? (:food @p)) (not (:home @p)))
(-> loc take-food (turn 4))
(and (pos? (:food @ahead)) (not (:home @ahead)) (not (:ant @ahead)))
(move loc)
122 CHAPTER 4. THE ANTS DEMO (KAI WU)

:else
(let [ranks (merge-with +
(rank-by (comp :food deref) places)
(rank-by (comp :pher deref) places))]
(([move #(turn % -1) #(turn % 1)]
(wrand [(if (:ant @ahead) 0 (ranks ahead))
(ranks ahead-left) (ranks ahead-right)]))
loc)))))))

The let values

The let values: quite straightforward, just note the twist in how behave receives
a cell/location as its argument, not an ant (which an OO-centric design might
expect).
The only JVM/concurrency leakage: Thread/sleep
The

(. Thread (sleep ant-sleep-ms))

or

(Thread/sleep ant-sleep-ms)

call is our first encounter with Clojures Java Interop.

The first version uses the dot special form and in particular, the

(. Classname-symbol (method-symbol args*))

format, with Thread as the Classname-symbol, and sleep as the method-

symbol.
However, outside of macros, the idiomatic form for accessing method mem-
bers is the second form,

(Classname/staticMethod args*)

Beyond syntax, the point of this expression is to slow down an ant (one
ant-agent per thread) between their movements, so you can see in the UI
what theyre doing, and theyll appear more realistic.

But more interesting still: in this highly concurrent program, the sleep expres-
sion is about the *only explicit reference to threads* in the entire code, i.e. one
of the very few leaky abstractions hinting at Clojures use of underlying JVM
4.1. THE SIMULATION WORLD 123

concurrency constructs. Besides this call, there are no locks, and no explicit
thread allocations.
The main dosync call
Next, lets look at whats going on within the dosync transaction.
Repeating asynchronously, without looping
The first expression is:

(when running (send-off *agent* #behave))

Initially this may seem strange; arent we in the behave function because send-
off already called it before entering it? Wont this just loop uselessly, not hitting
the core if code below? Not quite:

Instead, send-off adds another execution of behave to the current agents

*queue* of work/functions, and immediately returns.
The current agent is referenced by the asterisk-surrounded *agent*
which Clojure dynamically binds to the current active agent on a
thread-local basis.
Thus after finishing this call of behave the ant will do another action
(execute behave again), and another, and so on. No explicit looping, just
*queue and repeat*.

Also, note the # sharp-quote, before behave; this is a Clojure Var, one of
Clojures mutable reference types. Its just syntactic sugar for (var behave).
Invoking a Var referring to a function is the same as invoking the function
itself...so why bother with it? I dont know; heres what I could find:

Besides Clojure docs, this SO thread also suggests theres no difference,

Apply a var is the same as applying the value store in the var.
Maybe the # prefix on behave causes the current threads value of
the function (with the current ant/location) to be sent to the queue?
NO/unlikely. If it was mean to be a dynamic var, it would have asterisks
around it like *agent*.
Another possibility is that the # maintains a runtime look-up so that
behave can be redefined on a running simulation while developing.

Why use send-off instead of send?

send vs. send-off - send uses threadpool of fixed size which has low switch-
ing overhead but blocking can dry up the threadpool. By contrast, send-
off uses a dynamic threadpool and blocking is tolerated - and thats the
right approach here as ant contention for the same location/food can cer-
tainly cause (temporary) blocking.
124 CHAPTER 4. THE ANTS DEMO (KAI WU)

Determining what the ant does next

Finally, the ants logic for what to do next is in the large if expression. The
code looks dense but at the top level its just a binary choice:

If the ant has food, take it home; the cond specifies 3 sub-cases:
1. At a home cell, drop the food and turn around 180 degrees, to exit
home for more food.
2. If a home cell is ahead, move to it.
3. Otherwise, do a ranking of cells ahead (places has the cells ahead,
ahead-left, ahead-right) per presence of pheromones, or home, and
then randomly select from those 3 cells per their ranking/weighting.

4.1.7 World behavior: pheromone evaporation

evaporate

(defn evaporate
"Causes all the pheromones to evaporate a bit."
[]
(dorun
(for [x (range dim) y (range dim)]
(dosync
(let [p (place [x y])]
(alter p assoc :pher (* evap-rate (:pher @p))))))))

For a bit of realism and a cleaner UI/visual, its useful to have the ants
pheromones diminish and evaporate from the world over time. The evapo-
rate function fulfills that requirement:

It takes no arguments, it will work over the entire world/board of cells,

accessed via the tuples of x and y.
The dorun function takes a lazy collection/sequence (here, that of the for
expression) and forces the realization of that collection for its side effects,
discarding any returned values.
Its unlike the similarly-named doall where we do care about the
values.
And its unlike doseq, which is like Clojures for but runs immedi-
ately and does not collect the results.
4.2. THE UI 125

dosync is used as before, for lock-free updating of a place cell. Here, the
desired side-effect/mutation is to update the :pher value at the place
cell with a lower number.

Well see shortly that evaporate will run every second, a process that (like the
ants) will be handled asynchronously using a Clojure agent.

4.2 The UI
The user interface for the ants relies heavily on Clojures Java inter-operation
capabilities. But as well see, its more than just wrapping calls to Java.

4.2.1 Using the Java AWT

clojureUI

;;;;;;;;;;;;;;;;;;;;;;;; UI ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

(import
(java.awt Color Graphics Dimension)
(java.awt.image BufferedImage)
(javax.swing JPanel JFrame))

The import pulls in classes from Javas Abstract Window Toolkit (AWT) pack-
age, and from the Java Swing package. Assuming unfamiliarity with Java Swing,
lets describe the classes used:

The Color class encapsulates a color in the standard RGB color space.
In the code below, its usage as a constructor for a color instance follows
several arities:
4 integer arguments: r, g, b, and a for the alpha/transparency (0
transparent, 255 opaque)
3 integer arguments: r g b
1 argument: not a constructor call, but an access of a predefined
static Color field by name, returning the color in the RGB color
space.
The Graphics class is an abstract base class for all graphics contexts, i.e.
a Graphics instance holds the current state data needed for rendering
it: the Component object on which to draw, the current clip, color, and
126 CHAPTER 4. THE ANTS DEMO (KAI WU)

font, etc. Below, well see that the Clojure functions that take a Graphics
instance as an argument:
fill-cell
render-ant
render-place
render

. . .all do some kind of rendering/drawing.

The Dimension class encapsulates the integer width and height of a com-
ponent. This class is used just once below, in setting the size of the panel
of the UI.
BufferedImage class is needed for raster image data; below, the render
function uses it to paint the background panel.

The JPanel class is the generic lightweight UI container in Java Swing

(seems like the div element in HTML). Below, its used just once for the
main display.
The JFrame class creates a top-level window (w/ title and border) in
Swing; its used just once below for the main ants UI window.

4.2.2 Functions to render the board and the ants

Each discrete cell on the world board is a square matrix of pixels; with an odd
number of pixels chosen, we can have a central position:
UI-scale

;pixels per world cell

(def scale 5)

By default, cells are empty; drawing cells having food or ant-deposited pheromones
is done by filling with symbolic colors - here by running the Java methods set-
Color and fillRect:
UI-fill-cell

(defn fill-cell [#^Graphics g x y c]

(doto g
(.setColor c)
(.fillRect (* x scale) (* y scale) scale scale)))
4.2. THE UI 127

-
Note the use of the doto function here and in many places below: in Java, pro-
cedural mutation of a newly constructed instance is common for initialization.
Clojures doto function is meant to be more concise in specifying the target
object just once, and then methods/setters acting on it and then returning it,
implicitly.
Drawing an ant: the graphical appearance of an ant is just a (5-pixel long) line
pointing in one of the 8 cardinal directions, of two different colors (having food
or not):
UI-render-ant

(defn render-ant [ant #^Graphics g x y]

(let [black (. (new Color 0 0 0 255) (getRGB))
gray (. (new Color 100 100 100 255) (getRGB))
red (. (new Color 255 0 0 255) (getRGB))
[hx hy tx ty] ({0 [2 0 2 4] ; Up/North pointing
1 [4 0 0 4]
2 [4 2 0 2]
3 [4 4 0 0]
4 [2 4 2 0] ; Down/South
5 [0 4 4 0]
6 [0 2 4 2]
7 [0 0 4 4]}
(:dir ant))]
(doto g
(.setColor (if (:food ant)
(new Color 255 0 0 255)
(new Color 0 0 0 255)))
(.drawLine (+ hx (* x scale)) (+ hy (* y scale))
(+ tx (* x scale)) (+ ty (* y scale))))))

-
Note the cleverly concise destructuring for the start and end drawing coordi-
nates, needed in AWTs drawLine method.
If a cell in the ants world is not empty, it has one or more of three things
present: pheromone, food, or an ant. The render-place function updates the
cells appearance accordingly:
UI-render-place

(defn render-place [g p x y]
(when (pos? (:pher p))
(fill-cell g x y (new Color 0 255 0
(int (min 255 (* 255 (/ (:pher p) pher-scale)))))))
(when (pos? (:food p))
(fill-cell g x y (new Color 255 0 0
(int (min 255 (* 255 (/ (:food p) food-scale)))))))
128 CHAPTER 4. THE ANTS DEMO (KAI WU)

(when (:ant p)
(render-ant (:ant p) g x y)))

Finally, the render function ties everything together: initializing the UI/window
appearance by applying render place to every cell, and also drawing the home
space of the ants. Note the heavy usage of the dot special form: the UI code
relies heavily on Java, though Clojures for and doto help us avoid Java boil-
erplate and stay concise:
UI-render

(defn render [g]

(let [v (dosync (apply vector (for [x (range dim) y (range dim)]
@(place [x y]))))
img (new BufferedImage (* scale dim) (* scale dim)
(. BufferedImage TYPE_INT_ARGB))
bg (. img (getGraphics))]
;; First paint everything white, on the bg instance:
(doto bg
(.setColor (. Color white))
(.fillRect 0 0 (. img (getWidth)) (. img (getHeight))))
(dorun
(for [x (range dim) y (range dim)]
(render-place bg (v (+ (* x dim) y)) x y)))
;; Draw the home space of the ants:
(doto bg
(.setColor (. Color blue))
(.drawRect (* scale home-offset) (* scale home-offset)
(* scale nants-sqrt) (* scale nants-sqrt)))
(. g (drawImage img 0 0 nil))
(. bg (dispose)))) ; Finished using Graphics object, release it.

4.2.3 Setting the scene, then updating it continually

Almost ready to begin our simulation; we need to setup some additional elements
per AWT conventions: the main UI panel where visual changes take place, the
top-level window frame, and an animator agent that continually updates the
visual elements:
UI-panel

(def panel (doto

(proxy [JPanel] [] (paint [g] (render g)))
(.setPreferredSize (new Dimension
4.2. THE UI 129

(* scale dim)
(* scale dim)))))

(def frame (doto (new JFrame) (.add panel) .pack .show))

Animation, panel-by-panel

Now for bringing the static starting picture to life - like the cartoons of old,
the animation function will draw the next state of the main panel displaying
the ants. Below, Hickey uses the queue-itself-then-run, again-and-again code
pattern weve seen before (above, in updating an ants state):
UI-animation

(def animator (agent nil))

(defn animation [x]

(when running
(send-off *agent* #animation))
(. panel (repaint))
(. Thread (sleep animation-sleep-ms))
nil)

Finally, we need another agent to handle one more time-track of changes: evap-
oration, using the evaporate function defined above.
UI-evaporation

(def evaporator (agent nil))

(defn evaporation [x]

(when running
(send-off *agent* #evaporation))
(evaporate)
(. Thread (sleep evap-sleep-ms))
nil)

-
130 CHAPTER 4. THE ANTS DEMO (KAI WU)

4.3 Running the Program

4.3.1 Running the simulator
At the REPL, you can enter the entire do expression below, or try each line
within it separately:
runtheprogram

(do
(load-file "./literate-ants.clj")
(def ants (setup))
(send-off animator animation)
(dorun (map #(send-off % behave) ants))
(send-off evaporator evaporation))

-
Either way youll see a new window appear with a white background, blue
square representing the ants home, red squares of food, black or red (w/ food)
moving lines representing each ant, and green squares for pheromones in various
concentrations. A lot happening concurrently, with no locks, and beautifully
concise code - welcome to Clojure!

The Ants Demo

\getchunk{sim-world-setup}
\getchunk{cell}
\getchunk{sim-world-board-creation}
\getchunk{place}
\getchunk{ants-defined}
\getchunk{home-setup}
\getchunk{world-wrapping}
\getchunk{ant-agent-judgment-2}
\getchunk{ant-agent-turn}
\getchunk{ant-agent-move}
\getchunk{ant-agent-food}
\getchunk{ant-agent-judgment-1}
\getchunk{ant-agent-behave}
\getchunk{evaporate}
\getchunk{clojureUI}
\getchunk{UI-scale}
\getchunk{UI-fill-cell}
\getchunk{UI-render-ant}
\getchunk{UI-render-place}
\getchunk{UI-render}
\getchunk{UI-panel}
\getchunk{UI-animation}
\getchunk{UI-evaporation}
4.3. RUNNING THE PROGRAM 131

\getchunk{runtheprogram}

-
132 CHAPTER 4. THE ANTS DEMO (KAI WU)
Chapter 5

Parallel Processing

5.1 Natural Graphs

Could we do massively parallel work in native lisp (i.e. not just a swig cover
for MPI). Could we develop a lisp graph data structure that was designed to be
run in parallel with multiple processes (aka closures, continuations) owning
subgraphs and using the subgraph edges leaving their clades as natural com-
munication paths? That way we could develop a natural map from the graph
to the available resources.
It raises interesting questions, such as how to dynamically allocate the graph
map to nodes transparently, similar to the way garbage collection is now trans-
parent. This would allow Lisp programs to be written against the graph and
run in parallel without knowing about parallel issues. The immutable data
structures and STM would help with the coordination and state issues.
The combination of a processor and a subgraph is called a clade, similar to
the idea of calling a processor and a program a process.

5.2 Steeles parallel ideas

From Steele [Ste09] we find the following ideas:

effective parallelism uses trees

associative combining operators are good
mapreduce is good
catamorphisms are good
there are systematic strategies for parallizing sequential code
we must lose the accumulator paradigm
we must emphasize divide and conquer

133
134 CHAPTER 5. PARALLEL PROCESSING

Contrast good sequential code and good parallel code

good sequential code minimizes total number of operations

clever tricks to reuse previously computed results
good parallel code often performs redundant operations to reduce
communication
good sequential algorithms minimize space usage
clever tricks to reuse storage
good parallel code often requires extra space to permit temporal de-
coupling
sequential idioms stress linear problem decomposition
process on thing at a time and accumulate results
good parallel code usually requires multiway problem

linearly linked lists are inherently sequential

compare peano arithmetic: 5 = ((((0+1)+1)+1)+1)+1
binary arithmetic is much more efficient than unayr
we need a multiway decomposition paradigm
length [ ] = 0
length [a] = 1
length (a++b) = (length a) + (length b)

This is just a summation problem: adding up a bunch of 1s!

Total work: (n)
Delay: (log n), O(n) depending on how a++b is split even worse if splitting
has worse than constant cost
Thoughts
What you want to do versus how you want to do it.
Give the compiler wiggle room. The wiggle room concept is based on the idea
that certain mathematical properties allow the compiler to rewrite a computa-
tion into a parallel form. So the idea of associativity, which allows terms of an
expression to be grouped in arbitrary ways without affecting the computation
allows us to rewrite

1 + 2 + 3 + 4 + 5 + 6 + 7 + 8

as

((1 + 2) + (3 + 4)) + ((5 + 6) + (7 + 8))

which then allows the compiler to add the subterms and then add their results.
Steele identifies several mathematical properties that the compiler can exploit:
5.2. STEELES PARALLEL IDEAS 135

associativity
commutativity

What are the parallel operators. Which of these parallel operators are thinking
vs implementation.
Could the add tree be log 32 since add can be multi-arg? Is there a mapping
from persistent trees to parallel code?
Clojure could partition tasks based on log32. A better, more fine grained alter-
native is to use [55] bit-partitioning to split tasks based on the size of the task
and the number of processors. That is,

(partition chunksize processors)

Concatenation lists give wiggle room. That is,

<> is the empty list

<23> is a singleton list
< a || b > is a concatenation of a and b

so

<<23 || 47> || <18 || 11>>

||

|| ||

23 47 18 11
but the same list could be:

<<<23 || 47> || 18> || 11>

136 CHAPTER 5. PARALLEL PROCESSING

||

|| 11

|| 18

23 47
or

<<23 || <47 || <>>> || <18 || <<> || 11>>>

||

|| ||

23 || 18 ||

47 / / 11
or

<23 || <47 || <18 || <11 || <>>>>>

5.3. SEQUENTIAL MAPREDUCE 137

||

23 ||

47 ||

18 ||

11 /

5.3 Sequential mapreduce

For the list form Steele defines three operators, map, reduce, and mapreduce.
They are:
Steele map

(define (map f xs) ; linear in (length xs)

(cond ((null? xs) ())
(else (cons (f (car xs)) (map f (cdr xs))))))

-
so, for example,
(map ( (x) (* x x)) (1 2 3)) (1 4 9)

Steele reduce

(define (reduce g id xs) ; linear in (length xs)

(cond ((null? xs) id)
(else (g (car xs) (reduce g id (cdr xs))))))

-
so, for example,
138 CHAPTER 5. PARALLEL PROCESSING

(reduce + 0 (1 4 9)) 14

Steele mapreduce

(define (mapreduce f g id xs) ; linear in (length xs)

(ocnd ((null? xs) id)
(else (g (f (car xs)) (mapreduce f g id (cdr xs))))))

-
so, for example,
(mapreduce ( (x) (* x x)) + 0 (1 2 3)) Rightarrow 14

Steele length

(define (length xs) ; linear in (length xs)

(mapreduce (lambda (q) 1) + 0 xs))

Using structural recursion, we can define filter as:

Steele filter

(define (filter p xs) ; linear in (length xs)

(cond ((null? xs) ())
((p (car xs)) (cons p (filter p (cdr xs))))
(else (filter p (cdr x)))))

Alternatively we could map the predicate down the list:

Steele filter 2

(define (filter p xs) ; linear in (length xs)??

(apply append
(map (lambda (x) (if (p x) (list x) ())) xs)))

Or, by using mapreduce

Steele filter 3

(define (filter p xs) ; linear in (length xs)!!

(mapreduce (lambda (x) (if (p x) (list x) ()))
append () xs)) ; each call to append is constant time

The reverse function could be defined structurally as:

Steele reverse 1
5.3. SEQUENTIAL MAPREDUCE 139

(define (reverse xs) ; quadratic in (length xs)

(cond ((null? xs) ())
(else (addright (reverse (cdr xs)) (car xs)))))

Or we could define a new function, revappend, by using the accumulator trick

to reduce the time complexity from quadratic to linear.
Steele revappend

(define (revappend xs ys) ; linear in (length xs)

(cond ((null? xs) ys)
(else (revappend (cdr xs) (cons (car xs) ys)))))

and use it to define reverse:

Steele reverse 2

(define (reverse xs) ; linear in (length xs)

(revappend xs ()))

5.3.1 Parallel mapreduce

Steele parallel mapreduce

(define (mapreduce f g id xs) ; logarithmic in (length xs)??

(ocnd ((null? xs) id)
((singleton? xs) (f (item xs)))
(else (split xs (lambda (ys zs)
(g (mapreduce f g id ys) ; opportunity for
(mapreduce f g id zs))))))) ; parallelism

Steele parallel map

(define (map f xs)

(mapreduce (lambda (x) (list (f x))) append () xs)) ; or conc

Steele parallel reduce

140 CHAPTER 5. PARALLEL PROCESSING

(define (reduce g id xs)

(mapreduce (lambda (x) x) g id xs))

Steele parallel length

(define (length xs) ; logarithmic in (length xs)??

(mapreduce (lambda (q) 1) + 0 xs))

Steele parallel filter

(define (filter p xs) ; logarithmic in (length xs)??

(mapreduce (lambda (x) (if (p x) (list x) ()))
append () xs))

Steele parallel reverse

(define (reverse xs) ; logarithmic in (length xs)??

(mapreduce list (lambda (ys zx) (append zs ys)) () xs))

for filter, unlike summation, we rely on maintaining the original order of

the elements in the input list. Thus || and + are associative, but only +
is commutative.
do not confuse the ordering of elements in the result list (which is a spatial
order) with the order in which they are computed (which is a temporal
order)
sequential programming often ties spatial order to temporal order
it is incorrect to say that since I dont pay attention to the order in which
things are done then we require commutativity. But commutativity has
to do with the spatial ordering in the result, not the temporal order in
which they are computed.

Steele defines Conjugate Transforms which, instead of mapping input items

directly to the output data type T:

map inputs (maybe by way of T) to a richer data type U

5.4. MPI MESSAGE PASSING 141

perform the computation in this richer space U (chosen to make compu-

tation simpler or faster)
finally, project the result from U back into T

Thought: he is just wrapping T with a structure for side information

Thought: much effort needs to be placed on the combining operator

5.4 MPI Message Passing

5.4.1 The N-colony Ant Demo

5.5 MapReduce
142 CHAPTER 5. PARALLEL PROCESSING
Chapter 6

The ant build sequence

6.1 ant.build

build.xml
<project name="clojure" default="all" xmlns:mvn="urn:maven-artifact-ant">

<description>
Build with "ant jar" and then start the
REPL with: "java -cp clojure.jar clojure.main".
You will need to install the Maven Ant
Tasks to ${ant.home}/lib in order to execute
the nightly-build or stable-build targets.
</description>

<property name="src" location="src"/>

<property name="test" location="test"/>
<property name="jsrc" location="${src}/jvm"/>
<property name="cljsrc" location="${src}/clj"/>
<property name="build" location="classes"/>
<property name="test-classes" location="test-classes"/>
<property name="dist" location="dist"/>
<!-- override this with
-Ddeployment.url=scp://build.clojure.org/srv/test-deploy
to test a deployment -->
<property name="deployment.url"
value="scp://build.clojure.org/srv/www/releases"/>

<target name="debug">
<echo message="${deployment.url}"/>
</target>

143
144 CHAPTER 6. THE ANT BUILD SEQUENCE

<!-- version related properties -->

<property file="${cljsrc}/clojure/version.properties"/>
<!-- ensures all version properties are present -->
<fail unless="clojure.version.major"/>
<fail unless="clojure.version.minor"/>
<fail unless="clojure.version.interim"/>

<condition property="clojure.version.incremental.label"
value=".${clojure.version.incremental}"
else="">
<length string="${clojure.version.incremental}"
when="greater" length="0" />
</condition>
<condition property="clojure.version.qualifier.label"
value="-${clojure.version.qualifier}"
else="">
<length string="${clojure.version.qualifier}" when="greater"
length="0" />
</condition>
<condition property="clojure.version.interim.label"
value="-SNAPSHOT"
else="">
<!-- We place -SNAPSHOT whenever interim is not set to false, not
only if interim is set to true (this is less typo prone in the
worst case -->
<not><equals arg1="${clojure.version.interim}" arg2="false"
trim="true"/></not>
</condition>

<property name="cvm" value="${clojure.version.major}"/>

<property name="cvm1" value="${clojure.version.minor}"/>
<property name="cvil" value="${clojure.version.incremental.label}"/>
<property name="cvql" value="${clojure.version.qualifier.label}"/>
<property name="cvil1" value="${clojure.version.interim.label}"/>
<property name="clojure.version.label"
value="${cvm}.${cvm1}${cvil}${cvql}${cvil}"/>

<!-- general filterset for use when clojure version must be copied -->
<filterset id="clojure-version-filterset">
<filter token="clojure-version" value="${clojure.version.label}"/>
</filterset>

<property name="clojure_noversion_jar" location="clojure.jar"/>

<property name="slim_noversion_jar" location="clojure-slim.jar"/>
<property name="src_noversion_jar" location="clojure-sources.jar"/>
<property name="clojure_jar"
location="clojure-${clojure.version.label}.jar"/>
<property name="slim_jar"
location="clojure-slim-${clojure.version.label}.jar"/>
<property name="src_jar"
6.1. ANT.BUILD 145

location="clojure-sources-${clojure.version.label}.jar"/>

<!-- These make sense for building on tapestry.formos.com -->

<property name="snapshot.repo.dir"
location="/var/www/maven-snapshot-repository"/>
<property name="stable.repo.dir" location="/var/www/maven-repository"/>

<target name="init" depends="clean">

<tstamp/>
<mkdir dir="${build}"/>
<antcall target="init-version"/>
</target>

<target name="init-version">
<copy file="pom-template.xml"
tofile="pom.xml">
<filterset refid="clojure-version-filterset"/>
</copy>
<!--prevents users from modifying accidentally the generated
pom.xml works only on linux.-->
<chmod file="pom.xml" perm="ugo-w"/>
</target>

<target name="compile-java" depends="init"

description="Compile Java sources.">
<javac srcdir="${jsrc}" destdir="${build}" includeJavaRuntime="yes"
debug="true" target="1.5"/>
</target>

<target name="compile-clojure" depends="compile-java"

description="Compile Clojure sources.">
<java classname="clojure.lang.Compile"
classpath="${build}:${cljsrc}"
failonerror="true"
fork="true">
<sysproperty key="clojure.compile.path" value="${build}"/>
<!-- <sysproperty key="clojure.compile.warn-on-reflection"
value="true"/> -->
<arg value="clojure.core"/>
<arg value="clojure.core.protocols"/>
<arg value="clojure.main"/>
<arg value="clojure.set"/>
<arg value="clojure.xml"/>
<arg value="clojure.zip"/>
<arg value="clojure.inspector"/>
<arg value="clojure.walk"/>
<arg value="clojure.stacktrace"/>
<arg value="clojure.template"/>
<arg value="clojure.test"/>
146 CHAPTER 6. THE ANT BUILD SEQUENCE

<arg value="clojure.test.tap"/>
<arg value="clojure.test.junit"/>
<arg value="clojure.pprint"/>
<arg value="clojure.java.io"/>
<arg value="clojure.repl"/>
<arg value="clojure.java.browse"/>
<arg value="clojure.java.javadoc"/>
<arg value="clojure.java.shell"/>
<arg value="clojure.java.browse-ui"/>
<arg value="clojure.string"/>
<arg value="clojure.data"/>
<arg value="clojure.reflect"/>
</java>
</target>

<target name="build"
description="Build Clojure (compilation only, no tests)."
depends="compile-java, compile-clojure"/>

<target name="compile-tests"
description="Compile the subset of tests that require compilation.">
<delete dir="${test-classes}"/>
<mkdir dir="${test-classes}"/>
<java classname="clojure.lang.Compile"
classpath="${test-classes}:${test}:${build}:${cljsrc}"
failonerror="true">
<sysproperty key="clojure.compile.path" value="${test-classes}"/>
<arg value="clojure.test-clojure.protocols.examples"/>
<arg value="clojure.test-clojure.genclass.examples"/>
</java>
</target>

<target name="test"
description="Run clojure tests without recompiling clojure."
depends="compile-tests">
<java classname="clojure.main" failonerror="true">
<classpath>
<path location="${test-classes}"/>
<path location="${test}"/>
<path location="${build}"/>
<path location="${cljsrc}"/>
</classpath>
<arg value="-e"/>
<arg value=
"(require (clojure [test-clojure :as main])) (main/run-ant)"/>
</java>
</target>

<target name="clojure-jar" depends="build"

description="Create clojure jar file.">
6.1. ANT.BUILD 147

<jar jarfile="${clojure_jar}" basedir="${build}">

<fileset dir="${cljsrc}">
<include name="**/*.clj"/>
<include name="clojure/version.properties"/>
</fileset>
<manifest>
<attribute name="Main-Class" value="clojure.main"/>
<attribute name="Class-Path" value="."/>
</manifest>
</jar>
<copy file="${clojure_jar}" tofile="${clojure_noversion_jar}" />
</target>

<target name="clojure-jar-slim" depends="build"

description=
"Create clojure-slim jar file (omits compiled Clojure code).">
<jar jarfile="${slim_jar}">
<fileset dir="${build}" includes="clojure/asm/**"/>
<fileset dir="${build}" includes="clojure/lang/**"/>
<fileset dir="${build}" includes="clojure/main.class"/>
<fileset dir="${cljsrc}">
<include name="**/*.clj"/>
<include name="clojure/version.properties"/>
</fileset>
<manifest>
<attribute name="Main-Class" value="clojure.main"/>
<attribute name="Class-Path" value="."/>
</manifest>
</jar>
<copy file="${slim_jar}" tofile="${slim_noversion_jar}" />
</target>

<target name="clojure-jar-sources" depends="build"

description="Create a JAR of Java sources.">
<jar jarfile="${src_jar}" basedir="${jsrc}" includes="**/*">
<fileset dir="${cljsrc}"
includes="clojure/version.properties"/>
</jar>
<copy file="${src_jar}" tofile="${src_noversion_jar}" />
</target>

<target name="all"
depends=
"build,test,clojure-jar,clojure-jar-slim,clojure-jar-sources"/>

<target name="clean"
description="Remove autogenerated files and directories.">
<delete dir="${build}"/>
<delete dir="${test-classes}"/>
<delete dir="${dist}"/>
148 CHAPTER 6. THE ANT BUILD SEQUENCE

<delete file="pom.xml"/>
<delete verbose="true">
<fileset dir="${basedir}" includes="*.jar"/>
<fileset dir="${basedir}" includes="*.zip"/>
</delete>
</target>

<target name="setup-maven">
<typedef resource="org/apache/maven/artifact/ant/antlib.xml"
uri="urn:maven-artifact-ant"/>
</target>

<target name="nightly-build" depends="ci-build"

description="Build and deploy to nightly (snapshot) repository.">
<sequential>
<typedef resource="org/apache/maven/artifact/ant/antlib.xml"
uri="urn:maven-artifact-ant"/>
<mvn:deploy file="${clojure_jar}">
<pom file="pom.xml"/>
<attach file="${src_jar}" classifier="sources"/>
<attach file="${slim_jar}" classifier="slim"/>
<remoteRepository url="file:${snapshot.repo.dir}"/>
</mvn:deploy>
</sequential>
</target>

<target name="release" depends="ci-build,dist"

description="Build and deploy to remote stable repository.">
<sequential>
<typedef resource="org/apache/maven/artifact/ant/antlib.xml"
uri="urn:maven-artifact-ant"/>
<mvn:install-provider artifactId="wagon-ssh" version="1.0-beta-2"/>
<echo message="Deploying to ${deployment.url}"/>
<mvn:deploy file="${clojure_jar}">
<pom file="pom.xml"/>
<attach file="${src_jar}" classifier="sources"/>
<attach file="${slim_jar}" classifier="slim"/>
<remoteRepository url="${deployment.url}">
<authentication username="root"
privateKey="${user.home}/.ssh/id_rsa"/>
</remoteRepository>
</mvn:deploy>
</sequential>
</target>

<target name="ci-build" depends="clean,all,setup-maven"

description=
"Continous integration build, installed to local repository.">
<mvn:install file="${clojure_jar}">
<pom file="pom.xml"/>
6.2. THE EXECUTION 149

<attach file="${src_jar}" classifier="sources"/>

<attach file="${slim_jar}" classifier="slim"/>
</mvn:install>
</target>

<target name="dist" depends="clean,clojure-jar"

description="Build distribution ZIP file.">
<property name="distdir"
value="${dist}/clojure-${clojure.version.label}"/>
<mkdir dir="${distdir}"/>
<copy todir="${distdir}" includeEmptyDirs="false">
<fileset dir="${basedir}">
<exclude name="pom.xml"/>
<exclude name="**/.git/**"/>
<exclude name="**/*.class"/>
<exclude name="**/*.iml"/>
<exclude name="**/*.ipr"/>
<exclude name="**/*.iws"/>
<exclude name="**/*.jar"/>
</fileset>
</copy>
<copy file="${clojure_noversion_jar}" todir="${distdir}"/>
<zip basedir="${dist}"
destfile="clojure-${clojure.version.label}.zip"/>
</target>
</project>

6.2 The Execution

ant -v build
Apache Ant version 1.7.1 compiled on November 10 2008
Buildfile: build.xml
Detected Java version: 1.6 in: /usr/lib/jvm/java-6-sun-1.6.0.13/jre
Detected OS: Linux
parsing buildfile BASE/build.xml
with URI = file:BASE/build.xml
Project base dir set to: BASE
[antlib:org.apache.tools.ant] Could not load definitions from resource
org/apache/tools/ant/antlib.xml. It could not be found.
[property] Loading BASE/src/clj/clojure/version.properties
Build sequence for target(s) build is
[clean, init, compile-java, compile-clojure, build]
Complete build sequence is
[clean, init, compile-java, compile-clojure, build, setup-maven,
init-version, compile-tests, test, clojure-jar, clojure-jar-slim,
clojure-jar-sources, all, ci-build, nightly-build, debug, dist,
150 CHAPTER 6. THE ANT BUILD SEQUENCE

release, ]

clean:
[delete] Could not find file BASE/pom.xml to delete.

init:
[mkdir] Created dir: BASE/classes
Project base dir set to: BASE
[antcall] calling target(s) [init-version] in build file
BASE/build.xml
parsing buildfile BASE/build.xml with
URI = file:BASE/build.xml
Project base dir set to: BASE
Override ignored for property "src"
Override ignored for property "test"
Override ignored for property "jsrc"
Override ignored for property "cljsrc"
Override ignored for property "build"
Override ignored for property "test-classes"
Override ignored for property "dist"
Override ignored for property "deployment.url"
[property] Loading BASE/src/clj/clojure/version.properties
Override ignored for property "clojure.version.qualifier"
Override ignored for property "clojure.version.major"
Override ignored for property "clojure.version.interim"
Override ignored for property "clojure.version.incremental"
Override ignored for property "clojure.version.minor"
Override ignored for property "clojure.version.incremental.label"
Override ignored for property "clojure.version.qualifier.label"
Override ignored for property "clojure.version.interim.label"
Override ignored for property "clojure.version.label"
Override ignored for property "clojure_noversion_jar"
Override ignored for property "slim_noversion_jar"
Override ignored for property "src_noversion_jar"
Override ignored for property "clojure_jar"
Override ignored for property "slim_jar"
Override ignored for property "src_jar"
Override ignored for property "snapshot.repo.dir"
Override ignored for property "stable.repo.dir"
Build sequence for target(s) init-version is [init-version]
Complete build sequence is
[init-version, setup-maven, clean, init, compile-java,
compile-clojure, build, compile-tests, test, clojure-jar,
clojure-jar-slim, clojure-jar-sources, all, ci-build,
nightly-build, debug, dist, release, ]
[antcall] Entering BASE/build.xml...
Build sequence for target(s) init-version is [init-version]
Complete build sequence is
[init-version, setup-maven, clean, init, compile-java,
compile-clojure, build, compile-tests, test, clojure-jar,
6.2. THE EXECUTION 151

clojure-jar-slim, clojure-jar-sources, all, ci-build,

nightly-build, debug, dist, release, ]

init-version:
[copy] Copying 1 file to BASE
[copy] Copying BASE/pom-template.xml to BASE/pom.xml
Replacing: @clojure-version@ -> 1.3.0-master-SNAPSHOT
[chmod] Current OS is Linux
[chmod] Executing chmod with arguments:
[chmod] ugo-w
[chmod] BASE/pom.xml
[chmod]
[chmod] The characters around the executable and arguments are
[chmod] not part of the command.
[chmod] Applied chmod to 1 file and 0 directories.
[antcall] Exiting BASE/build.xml.

compile-java:
[javac] clojure/asm/AnnotationVisitor.java added as
clojure/asm/AnnotationVisitor.class doesnt exist.
[javac] clojure/asm/AnnotationWriter.java added as
clojure/asm/AnnotationWriter.class doesnt exist.
[javac] clojure/asm/Attribute.java added as
clojure/asm/Attribute.class doesnt exist.
[javac] clojure/asm/ByteVector.java added as
clojure/asm/ByteVector.class doesnt exist.
[javac] clojure/asm/ClassAdapter.java added as
clojure/asm/ClassAdapter.class doesnt exist.
[javac] clojure/asm/ClassReader.java added as
clojure/asm/ClassReader.class doesnt exist.
[javac] clojure/asm/ClassVisitor.java added as
clojure/asm/ClassVisitor.class doesnt exist.
[javac] clojure/asm/ClassWriter.java added as
clojure/asm/ClassWriter.class doesnt exist.
[javac] clojure/asm/Edge.java added as
clojure/asm/Edge.class doesnt exist.
[javac] clojure/asm/FieldVisitor.java added as
clojure/asm/FieldVisitor.class doesnt exist.
[javac] clojure/asm/FieldWriter.java added as
clojure/asm/FieldWriter.class doesnt exist.
[javac] clojure/asm/Frame.java added as
clojure/asm/Frame.class doesnt exist.
[javac] clojure/asm/Handler.java added as
clojure/asm/Handler.class doesnt exist.
[javac] clojure/asm/Item.java added as
clojure/asm/Item.class doesnt exist.
[javac] clojure/asm/Label.java added as
clojure/asm/Label.class doesnt exist.
[javac] clojure/asm/MethodAdapter.java added as
clojure/asm/MethodAdapter.class doesnt exist.
152 CHAPTER 6. THE ANT BUILD SEQUENCE

[javac] clojure/asm/MethodVisitor.java added as

clojure/asm/MethodVisitor.class doesnt exist.
[javac] clojure/asm/MethodWriter.java added as
clojure/asm/MethodWriter.class doesnt exist.
[javac] clojure/asm/Opcodes.java added as
clojure/asm/Opcodes.class doesnt exist.
[javac] clojure/asm/Type.java added as
clojure/asm/Type.class doesnt exist.
[javac] clojure/asm/commons/AdviceAdapter.java added as
clojure/asm/commons/AdviceAdapter.class doesnt exist.
[javac] clojure/asm/commons/AnalyzerAdapter.java added as
clojure/asm/commons/AnalyzerAdapter.class doesnt exist.
[javac] clojure/asm/commons/CodeSizeEvaluator.java added as
clojure/asm/commons/CodeSizeEvaluator.class doesnt exist.
[javac] clojure/asm/commons/EmptyVisitor.java added as
clojure/asm/commons/EmptyVisitor.class doesnt exist.
[javac] clojure/asm/commons/GeneratorAdapter.java added as
clojure/asm/commons/GeneratorAdapter.class doesnt exist.
[javac] clojure/asm/commons/LocalVariablesSorter.java added as
clojure/asm/commons/LocalVariablesSorter.class doesnt exist.
[javac] clojure/asm/commons/Method.java added as
clojure/asm/commons/Method.class doesnt exist.
[javac] clojure/asm/commons/SerialVersionUIDAdder.java added as
clojure/asm/commons/SerialVersionUIDAdder.class
doesnt exist.
[javac] clojure/asm/commons/StaticInitMerger.java added as
clojure/asm/commons/StaticInitMerger.class doesnt exist.
[javac] clojure/asm/commons/TableSwitchGenerator.java added as
clojure/asm/commons/TableSwitchGenerator.class doesnt exist.
[javac] BASE/src/jvm/clojure/asm/commons/package.html
skipped - dont know how to handle it
[javac] BASE/src/jvm/clojure/asm/package.html
skipped - dont know how to handle it
[javac] clojure/lang/AFn.java added as
clojure/lang/AFn.class doesnt exist.
[javac] clojure/lang/AFunction.java added as
clojure/lang/AFunction.class doesnt exist.
[javac] clojure/lang/AMapEntry.java added as
clojure/lang/AMapEntry.class doesnt exist.
[javac] clojure/lang/APersistentMap.java added as
clojure/lang/APersistentMap.class doesnt exist.
[javac] clojure/lang/APersistentSet.java added as
clojure/lang/APersistentSet.class doesnt exist.
[javac] clojure/lang/APersistentVector.java added as
clojure/lang/APersistentVector.class doesnt exist.
[javac] clojure/lang/ARef.java added as
clojure/lang/ARef.class doesnt exist.
[javac] clojure/lang/AReference.java added as
clojure/lang/AReference.class doesnt exist.
[javac] clojure/lang/ASeq.java added as
6.2. THE EXECUTION 153

clojure/lang/ASeq.class doesnt exist.

[javac] clojure/lang/ATransientMap.java added as
clojure/lang/ATransientMap.class doesnt exist.
[javac] clojure/lang/ATransientSet.java added as
clojure/lang/ATransientSet.class doesnt exist.
[javac] clojure/lang/Agent.java added as
clojure/lang/Agent.class doesnt exist.
[javac] clojure/lang/ArityException.java added as
clojure/lang/ArityException.class doesnt exist.
[javac] clojure/lang/ArrayChunk.java added as
clojure/lang/ArrayChunk.class doesnt exist.
[javac] clojure/lang/ArraySeq.java added as
clojure/lang/ArraySeq.class doesnt exist.
[javac] clojure/lang/Associative.java added as
clojure/lang/Associative.class doesnt exist.
[javac] clojure/lang/Atom.java added as
clojure/lang/Atom.class doesnt exist.
[javac] clojure/lang/BigInt.java added as
clojure/lang/BigInt.class doesnt exist.
[javac] clojure/lang/Binding.java added as
clojure/lang/Binding.class doesnt exist.
[javac] clojure/lang/Box.java added as
clojure/lang/Box.class doesnt exist.
[javac] clojure/lang/ChunkBuffer.java added as
clojure/lang/ChunkBuffer.class doesnt exist.
[javac] clojure/lang/ChunkedCons.java added as
clojure/lang/ChunkedCons.class doesnt exist.
[javac] clojure/lang/Compile.java added as
clojure/lang/Compile.class doesnt exist.
[javac] clojure/lang/Compiler.java added as
clojure/lang/Compiler.class doesnt exist.
[javac] clojure/lang/Cons.java added as
clojure/lang/Cons.class doesnt exist.
[javac] clojure/lang/Counted.java added as
clojure/lang/Counted.class doesnt exist.
[javac] clojure/lang/Delay.java added as
clojure/lang/Delay.class doesnt exist.
[javac] clojure/lang/DynamicClassLoader.java added as
clojure/lang/DynamicClassLoader.class doesnt exist.
[javac] clojure/lang/EnumerationSeq.java added as
clojure/lang/EnumerationSeq.class doesnt exist.
[javac] clojure/lang/Fn.java added as
clojure/lang/Fn.class doesnt exist.
[javac] clojure/lang/IChunk.java added as
clojure/lang/IChunk.class doesnt exist.
[javac] clojure/lang/IChunkedSeq.java added as
clojure/lang/IChunkedSeq.class doesnt exist.
[javac] clojure/lang/IDeref.java added as
clojure/lang/IDeref.class doesnt exist.
[javac] clojure/lang/IEditableCollection.java added as
154 CHAPTER 6. THE ANT BUILD SEQUENCE

clojure/lang/IEditableCollection.class doesnt exist.

[javac] clojure/lang/IFn.java added as
clojure/lang/IFn.class doesnt exist.
[javac] clojure/lang/IKeywordLookup.java added as
clojure/lang/IKeywordLookup.class doesnt exist.
[javac] clojure/lang/ILookup.java added as
clojure/lang/ILookup.class doesnt exist.
[javac] clojure/lang/ILookupSite.java added as
clojure/lang/ILookupSite.class doesnt exist.
[javac] clojure/lang/ILookupThunk.java added as
clojure/lang/ILookupThunk.class doesnt exist.
[javac] clojure/lang/IMapEntry.java added as
clojure/lang/IMapEntry.class doesnt exist.
[javac] clojure/lang/IMeta.java added as
clojure/lang/IMeta.class doesnt exist.
[javac] clojure/lang/IObj.java added as
clojure/lang/IObj.class doesnt exist.
[javac] clojure/lang/IPersistentCollection.java added as
clojure/lang/IPersistentCollection.class doesnt exist.
[javac] clojure/lang/IPersistentList.java added as
clojure/lang/IPersistentList.class doesnt exist.
[javac] clojure/lang/IPersistentMap.java added as
clojure/lang/IPersistentMap.class doesnt exist.
[javac] clojure/lang/IPersistentSet.java added as
clojure/lang/IPersistentSet.class doesnt exist.
[javac] clojure/lang/IPersistentStack.java added as
clojure/lang/IPersistentStack.class doesnt exist.
[javac] clojure/lang/IPersistentVector.java added as
clojure/lang/IPersistentVector.class doesnt exist.
[javac] clojure/lang/IPromiseImpl.java added as
clojure/lang/IPromiseImpl.class doesnt exist.
[javac] clojure/lang/IProxy.java added as
clojure/lang/IProxy.class doesnt exist.
[javac] clojure/lang/IReduce.java added as
clojure/lang/IReduce.class doesnt exist.
[javac] clojure/lang/IRef.java added as
clojure/lang/IRef.class doesnt exist.
[javac] clojure/lang/IReference.java added as
clojure/lang/IReference.class doesnt exist.
[javac] clojure/lang/ISeq.java added as
clojure/lang/ISeq.class doesnt exist.
[javac] clojure/lang/ITransientAssociative.java added as
clojure/lang/ITransientAssociative.class doesnt exist.
[javac] clojure/lang/ITransientCollection.java added as
clojure/lang/ITransientCollection.class doesnt exist.
[javac] clojure/lang/ITransientMap.java added as
clojure/lang/ITransientMap.class doesnt exist.
[javac] clojure/lang/ITransientSet.java added as
clojure/lang/ITransientSet.class doesnt exist.
[javac] clojure/lang/ITransientVector.java added as
6.2. THE EXECUTION 155

clojure/lang/ITransientVector.class doesnt exist.

[javac] clojure/lang/Indexed.java added as
clojure/lang/Indexed.class doesnt exist.
[javac] clojure/lang/IndexedSeq.java added as
clojure/lang/IndexedSeq.class doesnt exist.
[javac] clojure/lang/IteratorSeq.java added as
clojure/lang/IteratorSeq.class doesnt exist.
[javac] clojure/lang/Keyword.java added as
clojure/lang/Keyword.class doesnt exist.
[javac] clojure/lang/KeywordLookupSite.java added as
clojure/lang/KeywordLookupSite.class doesnt exist.
[javac] clojure/lang/LazilyPersistentVector.java added as
clojure/lang/LazilyPersistentVector.class doesnt exist.
[javac] clojure/lang/LazySeq.java added as
clojure/lang/LazySeq.class doesnt exist.
[javac] clojure/lang/LineNumberingPushbackReader.java added as
clojure/lang/LineNumberingPushbackReader.class doesnt exist.
[javac] clojure/lang/LispReader.java added as
clojure/lang/LispReader.class doesnt exist.
[javac] clojure/lang/LockingTransaction.java added as
clojure/lang/LockingTransaction.class doesnt exist.
[javac] clojure/lang/MapEntry.java added as
clojure/lang/MapEntry.class doesnt exist.
[javac] clojure/lang/MapEquivalence.java added as
clojure/lang/MapEquivalence.class doesnt exist.
[javac] clojure/lang/MethodImplCache.java added as
clojure/lang/MethodImplCache.class doesnt exist.
[javac] clojure/lang/MultiFn.java added as
clojure/lang/MultiFn.class doesnt exist.
[javac] clojure/lang/Named.java added as
clojure/lang/Named.class doesnt exist.
[javac] clojure/lang/Namespace.java added as
clojure/lang/Namespace.class doesnt exist.
[javac] clojure/lang/Numbers.java added as
clojure/lang/Numbers.class doesnt exist.
[javac] clojure/lang/Obj.java added as
clojure/lang/Obj.class doesnt exist.
[javac] clojure/lang/PersistentArrayMap.java added as
clojure/lang/PersistentArrayMap.class doesnt exist.
[javac] clojure/lang/PersistentHashMap.java added as
clojure/lang/PersistentHashMap.class doesnt exist.
[javac] clojure/lang/PersistentHashSet.java added as
clojure/lang/PersistentHashSet.class doesnt exist.
[javac] clojure/lang/PersistentList.java added as
clojure/lang/PersistentList.class doesnt exist.
[javac] clojure/lang/PersistentQueue.java added as
clojure/lang/PersistentQueue.class doesnt exist.
[javac] clojure/lang/PersistentStructMap.java added as
clojure/lang/PersistentStructMap.class doesnt exist.
[javac] clojure/lang/PersistentTreeMap.java added as
156 CHAPTER 6. THE ANT BUILD SEQUENCE

clojure/lang/PersistentTreeMap.class doesnt exist.

[javac] clojure/lang/PersistentTreeSet.java added as
clojure/lang/PersistentTreeSet.class doesnt exist.
[javac] clojure/lang/PersistentVector.java added as
clojure/lang/PersistentVector.class doesnt exist.
[javac] clojure/lang/ProxyHandler.java added as
clojure/lang/ProxyHandler.class doesnt exist.
[javac] clojure/lang/RT.java added as
clojure/lang/RT.class doesnt exist.
[javac] clojure/lang/Range.java added as
clojure/lang/Range.class doesnt exist.
[javac] clojure/lang/Ratio.java added as
clojure/lang/Ratio.class doesnt exist.
[javac] clojure/lang/Ref.java added as
clojure/lang/Ref.class doesnt exist.
[javac] clojure/lang/Reflector.java added as
clojure/lang/Reflector.class doesnt exist.
[javac] clojure/lang/Repl.java added as
clojure/lang/Repl.class doesnt exist.
[javac] clojure/lang/RestFn.java added as
clojure/lang/RestFn.class doesnt exist.
[javac] clojure/lang/Reversible.java added as
clojure/lang/Reversible.class doesnt exist.
[javac] clojure/lang/Script.java added as
clojure/lang/Script.class doesnt exist.
[javac] clojure/lang/SeqEnumeration.java added as
clojure/lang/SeqEnumeration.class doesnt exist.
[javac] clojure/lang/SeqIterator.java added as
clojure/lang/SeqIterator.class doesnt exist.
[javac] clojure/lang/Seqable.java added as
clojure/lang/Seqable.class doesnt exist.
[javac] clojure/lang/Sequential.java added as
clojure/lang/Sequential.class doesnt exist.
[javac] clojure/lang/Settable.java added as
clojure/lang/Settable.class doesnt exist.
[javac] clojure/lang/Sorted.java added as
clojure/lang/Sorted.class doesnt exist.
[javac] clojure/lang/StringSeq.java added as
clojure/lang/StringSeq.class doesnt exist.
[javac] clojure/lang/Symbol.java added as
clojure/lang/Symbol.class doesnt exist.
[javac] clojure/lang/TransactionalHashMap.java added as
clojure/lang/TransactionalHashMap.class doesnt exist.
[javac] clojure/lang/Util.java added as
clojure/lang/Util.class doesnt exist.
[javac] clojure/lang/Var.java added as
clojure/lang/Var.class doesnt exist.
[javac] clojure/lang/XMLHandler.java added as
clojure/lang/XMLHandler.class doesnt exist.
[javac] clojure/main.java added as
6.2. THE EXECUTION 157

clojure/main.class doesnt exist.

[javac] Compiling 139 source files to BASE/classes
[javac] Using modern compiler
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/jre/jre/lib/rt.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/Classes/jce.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/Classes/jsse.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/core.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/graphics.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/security.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/server.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/xml.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/Classes/classes.jar from path
as it doesnt exist
dropping /usr/lib/jvm/java-6-sun-1.6.0.13/Classes/ui.jar from path
as it doesnt exist
[javac] Compilation arguments:
[javac] -d
[javac] BASE/classes
[javac] -classpath
[javac] BASE/classes:
/usr/share/ant/lib/ant-launcher.jar:
/usr/share/java/xmlParserAPIs.jar:
/usr/share/java/xercesImpl.jar:
/usr/share/ant/lib/ant-antlr.jar:
/usr/share/ant/lib/velocity.jar:
/usr/share/ant/lib/ant.jar:
/usr/share/ant/lib/commons-collections.jar:
/usr/share/ant/lib/werken.xpath.jar:
/usr/share/ant/lib/ant-jsch.jar:
/usr/share/ant/lib/ant-apache-regexp.jar:
/usr/share/ant/lib/jsch.jar:
/usr/share/ant/lib/ant-apache-oro.jar:
/usr/share/ant/lib/ant-jdepend.jar:
/usr/share/ant/lib/bcel.jar:
/usr/share/ant/lib/ant-jmf.jar:
/usr/share/ant/lib/ant-javamail.jar:
/usr/share/ant/lib/ant-commons-net.jar:
/usr/share/ant/lib/jdom0.jar:
/usr/share/ant/lib/ant-apache-bsf.jar:
/usr/share/ant/lib/ant-swing.jar:
/usr/share/ant/lib/logkit.jar:
/usr/share/ant/lib/ant-commons-logging.jar:
158 CHAPTER 6. THE ANT BUILD SEQUENCE

/usr/share/ant/lib/ant-apache-log4j.jar:
/usr/share/ant/lib/ant-apache-resolver.jar:
/usr/share/ant/lib/ant-trax.jar:
/usr/share/ant/lib/ant-apache-bcel.jar:
/usr/share/ant/lib/ant-junit.jar:
/usr/share/ant/lib/ant-nodeps.jar:
/usr/lib/jvm/java-6-sun-1.6.0.13/lib/tools.jar:
/usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/rt.jar:
/usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/jce.jar:
/usr/lib/jvm/java-6-sun-1.6.0.13/jre/lib/jsse.jar
[javac] -sourcepath
[javac] BASE/src/jvm
[javac] -target
[javac] 1.5
[javac] -g
[javac]
[javac] The characters around the executable and arguments are
[javac] not part of the command.
[javac] Files to be compiled:
BASE/src/jvm/clojure/asm/AnnotationVisitor.java
BASE/src/jvm/clojure/asm/AnnotationWriter.java
BASE/src/jvm/clojure/asm/Attribute.java
BASE/src/jvm/clojure/asm/ByteVector.java
BASE/src/jvm/clojure/asm/ClassAdapter.java
BASE/src/jvm/clojure/asm/ClassReader.java
BASE/src/jvm/clojure/asm/ClassVisitor.java
BASE/src/jvm/clojure/asm/ClassWriter.java
BASE/src/jvm/clojure/asm/Edge.java
BASE/src/jvm/clojure/asm/FieldVisitor.java
BASE/src/jvm/clojure/asm/FieldWriter.java
BASE/src/jvm/clojure/asm/Frame.java
BASE/src/jvm/clojure/asm/Handler.java
BASE/src/jvm/clojure/asm/Item.java
BASE/src/jvm/clojure/asm/Label.java
BASE/src/jvm/clojure/asm/MethodAdapter.java
BASE/src/jvm/clojure/asm/MethodVisitor.java
BASE/src/jvm/clojure/asm/MethodWriter.java
BASE/src/jvm/clojure/asm/Opcodes.java
BASE/src/jvm/clojure/asm/Type.java
BASE/src/jvm/clojure/asm/commons/AdviceAdapter.java
BASE/src/jvm/clojure/asm/commons/AnalyzerAdapter.java
BASE/src/jvm/clojure/asm/commons/CodeSizeEvaluator.java
BASE/src/jvm/clojure/asm/commons/EmptyVisitor.java
BASE/src/jvm/clojure/asm/commons/GeneratorAdapter.java
BASE/src/jvm/clojure/asm/commons/LocalVariablesSorter.java
BASE/src/jvm/clojure/asm/commons/Method.java
BASE/src/jvm/clojure/asm/commons/SerialVersionUIDAdder.java
BASE/src/jvm/clojure/asm/commons/StaticInitMerger.java
BASE/src/jvm/clojure/asm/commons/TableSwitchGenerator.java
BASE/src/jvm/clojure/lang/AFn.java
6.2. THE EXECUTION 159

BASE/src/jvm/clojure/lang/AFunction.java
BASE/src/jvm/clojure/lang/AMapEntry.java
BASE/src/jvm/clojure/lang/APersistentMap.java
BASE/src/jvm/clojure/lang/APersistentSet.java
BASE/src/jvm/clojure/lang/APersistentVector.java
BASE/src/jvm/clojure/lang/ARef.java
BASE/src/jvm/clojure/lang/AReference.java
BASE/src/jvm/clojure/lang/ASeq.java
BASE/src/jvm/clojure/lang/ATransientMap.java
BASE/src/jvm/clojure/lang/ATransientSet.java
BASE/src/jvm/clojure/lang/Agent.java
BASE/src/jvm/clojure/lang/ArityException.java
BASE/src/jvm/clojure/lang/ArrayChunk.java
BASE/src/jvm/clojure/lang/ArraySeq.java
BASE/src/jvm/clojure/lang/Associative.java
BASE/src/jvm/clojure/lang/Atom.java
BASE/src/jvm/clojure/lang/BigInt.java
BASE/src/jvm/clojure/lang/Binding.java
BASE/src/jvm/clojure/lang/Box.java
BASE/src/jvm/clojure/lang/ChunkBuffer.java
BASE/src/jvm/clojure/lang/ChunkedCons.java
BASE/src/jvm/clojure/lang/Compile.java
BASE/src/jvm/clojure/lang/Compiler.java
BASE/src/jvm/clojure/lang/Cons.java
BASE/src/jvm/clojure/lang/Counted.java
BASE/src/jvm/clojure/lang/Delay.java
BASE/src/jvm/clojure/lang/DynamicClassLoader.java
BASE/src/jvm/clojure/lang/EnumerationSeq.java
BASE/src/jvm/clojure/lang/Fn.java
BASE/src/jvm/clojure/lang/IChunk.java
BASE/src/jvm/clojure/lang/IChunkedSeq.java
BASE/src/jvm/clojure/lang/IDeref.java
BASE/src/jvm/clojure/lang/IEditableCollection.java
BASE/src/jvm/clojure/lang/IFn.java
BASE/src/jvm/clojure/lang/IKeywordLookup.java
BASE/src/jvm/clojure/lang/ILookup.java
BASE/src/jvm/clojure/lang/ILookupSite.java
BASE/src/jvm/clojure/lang/ILookupThunk.java
BASE/src/jvm/clojure/lang/IMapEntry.java
BASE/src/jvm/clojure/lang/IMeta.java
BASE/src/jvm/clojure/lang/IObj.java
BASE/src/jvm/clojure/lang/IPersistentCollection.java
BASE/src/jvm/clojure/lang/IPersistentList.java
BASE/src/jvm/clojure/lang/IPersistentMap.java
BASE/src/jvm/clojure/lang/IPersistentSet.java
BASE/src/jvm/clojure/lang/IPersistentStack.java
BASE/src/jvm/clojure/lang/IPersistentVector.java
BASE/src/jvm/clojure/lang/IPromiseImpl.java
BASE/src/jvm/clojure/lang/IProxy.java
BASE/src/jvm/clojure/lang/IReduce.java
160 CHAPTER 6. THE ANT BUILD SEQUENCE

BASE/src/jvm/clojure/lang/IRef.java
BASE/src/jvm/clojure/lang/IReference.java
BASE/src/jvm/clojure/lang/ISeq.java
BASE/src/jvm/clojure/lang/ITransientAssociative.java
BASE/src/jvm/clojure/lang/ITransientCollection.java
BASE/src/jvm/clojure/lang/ITransientMap.java
BASE/src/jvm/clojure/lang/ITransientSet.java
BASE/src/jvm/clojure/lang/ITransientVector.java
BASE/src/jvm/clojure/lang/Indexed.java
BASE/src/jvm/clojure/lang/IndexedSeq.java
BASE/src/jvm/clojure/lang/IteratorSeq.java
BASE/src/jvm/clojure/lang/Keyword.java
BASE/src/jvm/clojure/lang/KeywordLookupSite.java
BASE/src/jvm/clojure/lang/LazilyPersistentVector.java
BASE/src/jvm/clojure/lang/LazySeq.java
BASE/src/jvm/clojure/lang/LineNumberingPushbackReader.java
BASE/src/jvm/clojure/lang/LispReader.java
BASE/src/jvm/clojure/lang/LockingTransaction.java
BASE/src/jvm/clojure/lang/MapEntry.java
BASE/src/jvm/clojure/lang/MapEquivalence.java
BASE/src/jvm/clojure/lang/MethodImplCache.java
BASE/src/jvm/clojure/lang/MultiFn.java
BASE/src/jvm/clojure/lang/Named.java
BASE/src/jvm/clojure/lang/Namespace.java
BASE/src/jvm/clojure/lang/Numbers.java
BASE/src/jvm/clojure/lang/Obj.java
BASE/src/jvm/clojure/lang/PersistentArrayMap.java
BASE/src/jvm/clojure/lang/PersistentHashMap.java
BASE/src/jvm/clojure/lang/PersistentHashSet.java
BASE/src/jvm/clojure/lang/PersistentList.java
BASE/src/jvm/clojure/lang/PersistentQueue.java
BASE/src/jvm/clojure/lang/PersistentStructMap.java
BASE/src/jvm/clojure/lang/PersistentTreeMap.java
BASE/src/jvm/clojure/lang/PersistentTreeSet.java
BASE/src/jvm/clojure/lang/PersistentVector.java
BASE/src/jvm/clojure/lang/ProxyHandler.java
BASE/src/jvm/clojure/lang/RT.java
BASE/src/jvm/clojure/lang/Range.java
BASE/src/jvm/clojure/lang/Ratio.java
BASE/src/jvm/clojure/lang/Ref.java
BASE/src/jvm/clojure/lang/Reflector.java
BASE/src/jvm/clojure/lang/Repl.java
BASE/src/jvm/clojure/lang/RestFn.java
BASE/src/jvm/clojure/lang/Reversible.java
BASE/src/jvm/clojure/lang/Script.java
BASE/src/jvm/clojure/lang/SeqEnumeration.java
BASE/src/jvm/clojure/lang/SeqIterator.java
BASE/src/jvm/clojure/lang/Seqable.java
BASE/src/jvm/clojure/lang/Sequential.java
BASE/src/jvm/clojure/lang/Settable.java
6.2. THE EXECUTION 161

BASE/src/jvm/clojure/lang/Sorted.java
BASE/src/jvm/clojure/lang/StringSeq.java
BASE/src/jvm/clojure/lang/Symbol.java
BASE/src/jvm/clojure/lang/TransactionalHashMap.java
BASE/src/jvm/clojure/lang/Util.java
BASE/src/jvm/clojure/lang/Var.java
BASE/src/jvm/clojure/lang/XMLHandler.java
BASE/src/jvm/clojure/main.java
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

compile-clojure:
[java] Executing /usr/lib/jvm/java-6-sun-1.6.0.13/jre/bin/java
with arguments:
[java] -Dclojure.compile.path=BASE/classes
[java] -classpath
[java] BASE/classes:BASE/src/clj
[java] clojure.lang.Compile
[java] clojure.core
[java] clojure.core.protocols
[java] clojure.main
[java] clojure.set
[java] clojure.xml
[java] clojure.zip
[java] clojure.inspector
[java] clojure.walk
[java] clojure.stacktrace
[java] clojure.template
[java] clojure.test
[java] clojure.test.tap
[java] clojure.test.junit
[java] clojure.pprint
[java] clojure.java.io
[java] clojure.repl
[java] clojure.java.browse
[java] clojure.java.javadoc
[java] clojure.java.shell
[java] clojure.java.browse-ui
[java] clojure.string
[java] clojure.data
[java] clojure.reflect
[java]
[java] The characters around the executable and arguments are
[java] not part of the command.
[java] Compiling clojure.core to BASE/classes
[java] Compiling clojure.core.protocols to BASE/classes
[java] Compiling clojure.main to BASE/classes
[java] Compiling clojure.set to BASE/classes
[java] Compiling clojure.xml to BASE/classes
[java] Compiling clojure.zip to BASE/classes
162 CHAPTER 6. THE ANT BUILD SEQUENCE

[java] Compiling clojure.inspector to BASE/classes

[java] Compiling clojure.walk to BASE/classes
[java] Compiling clojure.stacktrace to BASE/classes
[java] Compiling clojure.template to BASE/classes
[java] Compiling clojure.test to BASE/classes
[java] Compiling clojure.test.tap to BASE/classes
[java] Compiling clojure.test.junit to BASE/classes
[java] Compiling clojure.pprint to BASE/classes
[java] Compiling clojure.java.io to BASE/classes
[java] Compiling clojure.repl to BASE/classes
[java] Compiling clojure.java.browse to BASE/classes
[java] Compiling clojure.java.javadoc to BASE/classes
[java] Compiling clojure.java.shell to BASE/classes
[java] Compiling clojure.java.browse-ui to BASE/classes
[java] Compiling clojure.string to BASE/classes
[java] Compiling clojure.data to BASE/classes
[java] Compiling clojure.reflect to BASE/classes

build:

BUILD SUCCESSFUL
Total time: 33 seconds
Chapter 7

jvm/clojure/asm/

7.1 AnnotationVisitor.java

AnnotationVisitor.java
\getchunk{France Telecom Copyright}
package clojure.asm;

/**
* A visitor to visit a Java annotation. The methods of this interface
* must be called in the following order: (<tt>visit<tt> |
* <tt>visitEnum<tt> | <tt>visitAnnotation<tt> | <tt>visitArray<tt>)*
* <tt>visitEnd<tt>.
*
* @author Eric Bruneton
* @author Eugene Kuleshov
*/
public interface AnnotationVisitor{

/**
* Visits a primitive value of the annotation.
*
* @param name the value name.
* @param value the actual value, whose type must be {@link Byte},
* {@link Boolean}, {@link Character}, {@link Short},
* {@link Integer}, {@link Long}, {@link Float},
* {@link Double}, {@link String} or {@link Type}. This
* value can also be an array of byte, boolean, short,
* char, int, long, float or double values (this is
* equivalent to using {@link #visitArray visitArray}
* and visiting each array element in turn, but is more
* convenient).

163
164 CHAPTER 7. JVM/CLOJURE/ASM/

*/
void visit(String name, Object value);

/**
* Visits an enumeration value of the annotation.
*
* @param name the value name.
* @param desc the class descriptor of the enumeration class.
* @param value the actual enumeration value.
*/
void visitEnum(String name, String desc, String value);

/**
* Visits a nested annotation value of the annotation.
*
* @param name the value name.
* @param desc the class descriptor of the nested annotation class.
* @return a visitor to visit the actual nested annotation value, or
* <tt>null</tt> if this visitor is not interested in visiting
* this nested annotation. <i>The nested annotation value must be
* fully visited before calling other methods on this annotation
* visitor</i>.
*/
AnnotationVisitor visitAnnotation(String name, String desc);

/**
* Visits an array value of the annotation. Note that arrays of primitive
* types (such as byte, boolean, short, char, int, long, float or double)
* can be passed as value to {@link #visit visit}. This is what
* {@link ClassReader} does.
*
* @param name the value name.
* @return a visitor to visit the actual array value elements, or
* <tt>null</tt> if this visitor is not interested in visiting
* these values. The name parameters passed to the methods of
* this visitor are ignored. <i>All the array values must be
* visited before calling other methods on this annotation
* visitor</i>.
*/
AnnotationVisitor visitArray(String name);

/**
* Visits the end of the annotation.
*/
void visitEnd();
}

-
7.2. ANNOTATIONWRITER.JAVA 165

7.2 AnnotationWriter.java
(AnnotationVisitor [163])
AnnotationWriter.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* An {@link AnnotationVisitor} that generates annotations in
* bytecode form.
*
* @author Eric Bruneton
* @author Eugene Kuleshov
*/
final class AnnotationWriter implements AnnotationVisitor{

/**
* The class writer to which this annotation must be added.
*/
private final ClassWriter cw;

/**
* The number of values in this annotation.
*/
private int size;

/**
* <tt>true<tt> if values are named, <tt>false</tt> otherwise. Annotation
* writers used for annotation default and annotation arrays use unnamed
* values.
*/
private final boolean named;

/**
* The annotation values in bytecode form. This byte vector only contains
* the values themselves, i.e. the number of values must be stored as a
* unsigned short just before these bytes.
*/
private final ByteVector bv;

/**
* The byte vector to be used to store the number of values of this
* annotation. See {@link #bv}.
*/
private final ByteVector parent;

/**
* Where the number of values of this annotation must be stored in
166 CHAPTER 7. JVM/CLOJURE/ASM/

* {@link #parent}.
*/
private final int offset;

/**
* Next annotation writer. This field is used to store annotation lists.
*/
AnnotationWriter next;

/**
* Previous annotation writer. This field is used to store annotation
* lists.
*/
AnnotationWriter prev;

// -------------------------------------------------------------------
// Constructor
// -------------------------------------------------------------------

/**
* Constructs a new {@link AnnotationWriter}.
*
* @param cw the class writer to which this annotation must be added.
* @param named <tt>true<tt> if values are named,
* <tt>false</tt> otherwise.
* @param bv where the annotation values must be stored.
* @param parent where the number of annotation values must be stored.
* @param offset where in <tt>parent</tt> the number of annotation values
* must be stored.
*/
AnnotationWriter(
final ClassWriter cw,
final boolean named,
final ByteVector bv,
final ByteVector parent,
final int offset){
this.cw = cw;
this.named = named;
this.bv = bv;
this.parent = parent;
this.offset = offset;
}

// -------------------------------------------------------------------
// Implementation of the AnnotationVisitor interface
// -------------------------------------------------------------------

public void visit(final String name, final Object value){

++size;
if(named)
7.2. ANNOTATIONWRITER.JAVA 167

{
bv.putShort(cw.newUTF8(name));
}
if(value instanceof String)
{
bv.put12(s, cw.newUTF8((String) value));
}
else if(value instanceof Byte)
{
bv.put12(B, cw.newInteger(((Byte) value).byteValue()).index);
}
else if(value instanceof Boolean)
{
int v = ((Boolean) value).booleanValue() ? 1 : 0;
bv.put12(Z, cw.newInteger(v).index);
}
else if(value instanceof Character)
{
bv.put12(C,
cw.newInteger(((Character) value).charValue()).index);
}
else if(value instanceof Short)
{
bv.put12(S, cw.newInteger(((Short) value).shortValue()).index);
}
else if(value instanceof Type)
{
bv.put12(c, cw.newUTF8(((Type) value).getDescriptor()));
}
else if(value instanceof byte[])
{
byte[] v = (byte[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(B, cw.newInteger(v[i]).index);
}
}
else if(value instanceof boolean[])
{
boolean[] v = (boolean[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(Z, cw.newInteger(v[i] ? 1 : 0).index);
}
}
else if(value instanceof short[])
{
short[] v = (short[]) value;
168 CHAPTER 7. JVM/CLOJURE/ASM/

bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(S, cw.newInteger(v[i]).index);
}
}
else if(value instanceof char[])
{
char[] v = (char[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(C, cw.newInteger(v[i]).index);
}
}
else if(value instanceof int[])
{
int[] v = (int[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(I, cw.newInteger(v[i]).index);
}
}
else if(value instanceof long[])
{
long[] v = (long[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(J, cw.newLong(v[i]).index);
}
}
else if(value instanceof float[])
{
float[] v = (float[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(F, cw.newFloat(v[i]).index);
}
}
else if(value instanceof double[])
{
double[] v = (double[]) value;
bv.put12([, v.length);
for(int i = 0; i < v.length; i++)
{
bv.put12(D, cw.newDouble(v[i]).index);
}
7.2. ANNOTATIONWRITER.JAVA 169

}
else
{
Item i = cw.newConstItem(value);
bv.put12(".s.IFJDCS".charAt(i.type), i.index);
}
}

public void visitEnum(

final String name,
final String desc,
final String value){
++size;
if(named)
{
bv.putShort(cw.newUTF8(name));
}
bv.put12(e, cw.newUTF8(desc)).putShort(cw.newUTF8(value));
}

public AnnotationVisitor visitAnnotation(

final String name,
final String desc){
++size;
if(named)
{
bv.putShort(cw.newUTF8(name));
}
// write tag and type, and reserve space for values count
bv.put12(@, cw.newUTF8(desc)).putShort(0);
return new AnnotationWriter(cw, true, bv, bv, bv.length - 2);
}

public AnnotationVisitor visitArray(final String name){

++size;
if(named)
{
bv.putShort(cw.newUTF8(name));
}
// write tag, and reserve space for array size
bv.put12([, 0);
return new AnnotationWriter(cw, false, bv, bv, bv.length - 2);
}

public void visitEnd(){

if(parent != null)
{
byte[] data = parent.data;
data[offset] = (byte) (size >>> 8);
data[offset + 1] = (byte) size;
170 CHAPTER 7. JVM/CLOJURE/ASM/

}
}

// -------------------------------------------------------------------
// Utility methods
// -------------------------------------------------------------------

/**
* Returns the size of this annotation writer list.
*
* @return the size of this annotation writer list.
*/
int getSize(){
int size = 0;
AnnotationWriter aw = this;
while(aw != null)
{
size += aw.bv.length;
aw = aw.next;
}
return size;
}

/**
* Puts the annotations of this annotation writer list into the given
* byte vector.
*
* @param out where the annotations must be put.
*/
void put(final ByteVector out){
int n = 0;
int size = 2;
AnnotationWriter aw = this;
AnnotationWriter last = null;
while(aw != null)
{
++n;
size += aw.bv.length;
aw.visitEnd(); // in case user forgot to call visitEnd
aw.prev = last;
last = aw;
aw = aw.next;
}
out.putInt(size);
out.putShort(n);
aw = last;
while(aw != null)
{
out.putByteArray(aw.bv.data, 0, aw.bv.length);
aw = aw.prev;
7.3. ATTRIBUTE.JAVA 171

}
}

/**
* Puts the given annotation lists into the given byte vector.
*
* @param panns an array of annotation writer lists.
* @param out where the annotations must be put.
*/
static void put(final AnnotationWriter[] panns, final ByteVector out){
int size = 1 + 2 * panns.length;
for(int i = 0; i < panns.length; ++i)
{
size += panns[i] == null ? 0 : panns[i].getSize();
}
out.putInt(size).putByte(panns.length);
for(int i = 0; i < panns.length; ++i)
{
AnnotationWriter aw = panns[i];
AnnotationWriter last = null;
int n = 0;
while(aw != null)
{
++n;
aw.visitEnd(); // in case user forgot to call visitEnd
aw.prev = last;
last = aw;
aw = aw.next;
}
out.putShort(n);
aw = last;
while(aw != null)
{
out.putByteArray(aw.bv.data, 0, aw.bv.length);
aw = aw.prev;
}
}
}
}

7.3 Attribute.java

Attribute.java

\getchunk{France Telecom Copyright}

172 CHAPTER 7. JVM/CLOJURE/ASM/

package clojure.asm;

/**
* A non standard class, field, method or code attribute.
*
* @author Eric Bruneton
* @author Eugene Kuleshov
*/
public class Attribute{

/**
* The type of this attribute.
*/
public final String type;

/**
* The raw value of this attribute, used only for unknown attributes.
*/
byte[] value;

/**
* The next attribute in this attribute list. May be <tt>null</tt>.
*/
Attribute next;

/**
* Constructs a new empty attribute.
*
* @param type the type of the attribute.
*/
protected Attribute(final String type){
this.type = type;
}

/**
* Returns <tt>true</tt> if this type of attribute is unknown.
* The default implementation of this method always returns
* <tt>true</tt>.
*
* @return <tt>true</tt> if this type of attribute is unknown.
*/
public boolean isUnknown(){
return true;
}

/**
* Returns <tt>true</tt> if this type of attribute is a code attribute.
*
* @return <tt>true</tt> if this type of attribute is a code attribute.
*/
7.3. ATTRIBUTE.JAVA 173

public boolean isCodeAttribute(){

return false;
}

/**
* Returns the labels corresponding to this attribute.
*
* @return the labels corresponding to this attribute, or
* <tt>null</tt> if this attribute is not a code attribute
* that contains labels.
*/
protected Label[] getLabels(){
return null;
}

/**
* Reads a {@link #type type} attribute. This method must return
* <i>new</i> {@link Attribute} object, of type {@link #type type},
* corresponding to the <tt>len</tt> bytes starting at the given offset,
* in the given class reader.
*
* @param cr the class that contains the attribute to be read.
* @param off index of the first byte of the attributes content
* in {@link ClassReader#b cr.b}. The 6 attribute header
* bytes, containing the type and the length of the
* attribute, are not taken into account
* here.
* @param len the length of the attributes content.
* @param buf buffer to be used to call
* {@link ClassReader#readUTF8 readUTF8},
* {@link ClassReader#readClass(int,char[]) readClass} or
* {@link ClassReader#readConst readConst}.
* @param codeOff index of the first byte of codes attribute content in
* {@link ClassReader#b cr.b}, or -1 if the attribute to
* be read is not a code attribute. The 6 attribute header
* bytes, containing the type and the length of the
* attribute, are not taken into account here.
* @param labels the labels of the methods code, or <tt>null</tt>
* if the attribute to be read is not a code attribute.
* @return a <i>new</i> {@link Attribute} object corresponding to the
* given bytes.
*/
protected Attribute read(
final ClassReader cr,
final int off,
final int len,
final char[] buf,
final int codeOff,
final Label[] labels){
Attribute attr = new Attribute(type);
174 CHAPTER 7. JVM/CLOJURE/ASM/

attr.value = new byte[len];

System.arraycopy(cr.b, off, attr.value, 0, len);
return attr;
}

/**
* Returns the byte array form of this attribute.
*
* @param cw the class to which this attribute must be added.
* This parameter can be used to add to the constant
* pool of this class the items that corresponds to
* this attribute.
* @param code the bytecode of the method corresponding to this code
* attribute, or <tt>null</tt> if this attribute is not
* a code attributes.
* @param len the length of the bytecode of the method
* corresponding to this code attribute, or
* <tt>null</tt> if this attribute is not a code
* attribute.
* @param maxStack the maximum stack size of the method corresponding to
* this code attribute, or -1 if this attribute is not
* a code attribute.
* @param maxLocals the maximum number of local variables of the method
* corresponding to this code attribute, or -1 if this
* attribute is not a code attribute.
* @return the byte array form of this attribute.
*/
protected ByteVector write(
final ClassWriter cw,
final byte[] code,
final int len,
final int maxStack,
final int maxLocals){
ByteVector v = new ByteVector();
v.data = value;
v.length = value.length;
return v;
}

/**
* Returns the length of the attribute list that begins with this
* attribute.
*
* @return the length of the attribute list that begins with this
* attribute.
*/
final int getCount(){
int count = 0;
Attribute attr = this;
while(attr != null)
7.3. ATTRIBUTE.JAVA 175

{
count += 1;
attr = attr.next;
}
return count;
}

/**
* Returns the size of all the attributes in this attribute list.
*
* @param cw the class writer to be used to convert the attributes
* into byte arrays, with the {@link #write write}
* method.
* @param code the bytecode of the method corresponding to these
* code attributes, or <tt>null</tt> if these
* attributes are not code attributes.
* @param len the length of the bytecode of the method
* corresponding to these code attributes, or
* <tt>null</tt> if these attributes are not code
* attributes.
* @param maxStack the maximum stack size of the method corresponding to
* these code attributes, or -1 if these attributes are
* not code attributes.
* @param maxLocals the maximum number of local variables of the method
* corresponding to these code attributes, or -1 if
* these attributes are not code attributes.
* @return the size of all the attributes in this attribute list.
* This size includes the size of the attribute headers.
*/
final int getSize(
final ClassWriter cw,
final byte[] code,
final int len,
final int maxStack,
final int maxLocals){
Attribute attr = this;
int size = 0;
while(attr != null)
{
cw.newUTF8(attr.type);
size +=
attr.write(cw,code,len,maxStack,maxLocals).length + 6;
attr = attr.next;
}
return size;
}

/**
* Writes all the attributes of this attribute list in the given byte
* vector.
176 CHAPTER 7. JVM/CLOJURE/ASM/

*
* @param cw the class writer to be used to convert the attributes
* into byte arrays, with the {@link #write write}
* method.
* @param code the bytecode of the method corresponding to these
* code attributes, or <tt>null</tt> if these attributes
* are not code attributes.
* @param len the length of the bytecode of the method
* corresponding to these code attributes, or
* <tt>null</tt> if these attributes are not code
* attributes.
* @param maxStack the maximum stack size of the method corresponding to
* these code attributes, or -1 if these attributes are
* not code attributes.
* @param maxLocals the maximum number of local variables of the method
* corresponding to these code attributes, or -1 if
* these attributes are not code attributes.
* @param out where the attributes must be written.
*/
final void put(
final ClassWriter cw,
final byte[] code,
final int len,
final int maxStack,
final int maxLocals,
final ByteVector out){
Attribute attr = this;
while(attr != null)
{
ByteVector b = attr.write(cw, code, len, maxStack, maxLocals);
out.putShort(cw.newUTF8(attr.type)).putInt(b.length);
out.putByteArray(b.data, 0, b.length);
attr = attr.next;
}
}
}

7.4 ByteVector.java

ByteVector.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
7.4. BYTEVECTOR.JAVA 177

* A dynamically extensible vector of bytes. This class is roughly

* equivalent to a DataOutputStream on top of a ByteArrayOutputStream,
* but is more efficient.
*
* @author Eric Bruneton
*/
public class ByteVector{

/**
* The content of this vector.
*/
byte[] data;

/**
* Actual number of bytes in this vector.
*/
int length;

/**
* Constructs a new {@link ByteVector ByteVector} with a default initial
* size.
*/
public ByteVector(){
data = new byte[64];
}

/**
* Constructs a new {@link ByteVector ByteVector} with the given initial
* size.
*
* @param initialSize the initial size of the byte vector to be
* constructed.
*/
public ByteVector(final int initialSize){
data = new byte[initialSize];
}

/**
* Puts a byte into this byte vector. The byte vector is automatically
* enlarged if necessary.
*
* @param b a byte.
* @return this byte vector.
*/
public ByteVector putByte(final int b){
int length = this.length;
if(length + 1 > data.length)
{
enlarge(1);
}
178 CHAPTER 7. JVM/CLOJURE/ASM/

data[length++] = (byte) b;
this.length = length;
return this;
}

/**
* Puts two bytes into this byte vector. The byte vector is automatically
* enlarged if necessary.
*
* @param b1 a byte.
* @param b2 another byte.
* @return this byte vector.
*/
ByteVector put11(final int b1, final int b2){
int length = this.length;
if(length + 2 > data.length)
{
enlarge(2);
}
byte[] data = this.data;
data[length++] = (byte) b1;
data[length++] = (byte) b2;
this.length = length;
return this;
}

/**
* Puts a short into this byte vector. The byte vector is automatically
* enlarged if necessary.
*
* @param s a short.
* @return this byte vector.
*/
public ByteVector putShort(final int s){
int length = this.length;
if(length + 2 > data.length)
{
enlarge(2);
}
byte[] data = this.data;
data[length++] = (byte) (s >>> 8);
data[length++] = (byte) s;
this.length = length;
return this;
}

/**
* Puts a byte and a short into this byte vector. The byte vector is
* automatically enlarged if necessary.
*
7.4. BYTEVECTOR.JAVA 179

* @param b a byte.
* @param s a short.
* @return this byte vector.
*/
ByteVector put12(final int b, final int s){
int length = this.length;
if(length + 3 > data.length)
{
enlarge(3);
}
byte[] data = this.data;
data[length++] = (byte) b;
data[length++] = (byte) (s >>> 8);
data[length++] = (byte) s;
this.length = length;
return this;
}

/**
* Puts an int into this byte vector. The byte vector is automatically
* enlarged if necessary.
*
* @param i an int.
* @return this byte vector.
*/
public ByteVector putInt(final int i){
int length = this.length;
if(length + 4 > data.length)
{
enlarge(4);
}
byte[] data = this.data;
data[length++] = (byte) (i >>> 24);
data[length++] = (byte) (i >>> 16);
data[length++] = (byte) (i >>> 8);
data[length++] = (byte) i;
this.length = length;
return this;
}

/**
* Puts a long into this byte vector. The byte vector is automatically
* enlarged if necessary.
*
* @param l a long.
* @return this byte vector.
*/
public ByteVector putLong(final long l){
int length = this.length;
if(length + 8 > data.length)
180 CHAPTER 7. JVM/CLOJURE/ASM/

{
enlarge(8);
}
byte[] data = this.data;
int i = (int) (l >>> 32);
data[length++] = (byte) (i >>> 24);
data[length++] = (byte) (i >>> 16);
data[length++] = (byte) (i >>> 8);
data[length++] = (byte) i;
i = (int) l;
data[length++] = (byte) (i >>> 24);
data[length++] = (byte) (i >>> 16);
data[length++] = (byte) (i >>> 8);
data[length++] = (byte) i;
this.length = length;
return this;
}

/**
* Puts an UTF8 string into this byte vector. The byte vector is
* automatically enlarged if necessary.
*
* @param s a String.
* @return this byte vector.
*/
public ByteVector putUTF8(final String s){
int charLength = s.length();
if(length + 2 + charLength > data.length)
{
enlarge(2 + charLength);
}
int len = length;
byte[] data = this.data;
// optimistic algorithm: instead of computing the byte length
// and then serializing the string (which requires two loops),
// we assume the byte length is equal to char length (which is
// the most frequent case), and we start serializing the string
// right away. During the serialization, if we find that this
// assumption is wrong, we continue with the general method.
data[len++] = (byte) (charLength >>> 8);
data[len++] = (byte) charLength;
for(int i = 0; i < charLength; ++i)
{
char c = s.charAt(i);
if(c >= \001 && c <= \177)
{
data[len++] = (byte) c;
}
else
{
7.4. BYTEVECTOR.JAVA 181

int byteLength = i;
for(int j = i; j < charLength; ++j)
{
c = s.charAt(j);
if(c >= \001 && c <= \177)
{
byteLength++;
}
else if(c > \u07FF)
{
byteLength += 3;
}
else
{
byteLength += 2;
}
}
data[length] = (byte) (byteLength >>> 8);
data[length + 1] = (byte) byteLength;
if(length + 2 + byteLength > data.length)
{
length = len;
enlarge(2 + byteLength);
data = this.data;
}
for(int j = i; j < charLength; ++j)
{
c = s.charAt(j);
if(c >= \001 && c <= \177)
{
data[len++] = (byte) c;
}
else if(c > \u07FF)
{
data[len++] =
(byte) (0xE0 | c >> 12 & 0xF);
data[len++] =
(byte) (0x80 | c >> 6 & 0x3F);
data[len++] =
(byte) (0x80 | c & 0x3F);
}
else
{
data[len++] =
(byte) (0xC0 | c >> 6 & 0x1F);
data[len++] =
(byte) (0x80 | c & 0x3F);
}
}
break;
182 CHAPTER 7. JVM/CLOJURE/ASM/

}
}
length = len;
return this;
}

/**
* Puts an array of bytes into this byte vector. The byte vector is
* automatically enlarged if necessary.
*
* @param b an array of bytes. May be <tt>null</tt> to put <tt>len</tt>
* null bytes into this byte vector.
* @param off index of the fist byte of b that must be copied.
* @param len number of bytes of b that must be copied.
* @return this byte vector.
*/
public ByteVector putByteArray(final byte[] b, final int off,
final int len){
if(length + len > data.length)
{
enlarge(len);
}
if(b != null)
{
System.arraycopy(b, off, data, length, len);
}
length += len;
return this;
}

/**
* Enlarge this byte vector so that it can receive n more bytes.
*
* @param size number of additional bytes that this byte vector should be
* able to receive.
*/
private void enlarge(final int size){
int length1 = 2 * data.length;
int length2 = length + size;
byte[] newData = new byte[length1 > length2 ? length1 : length2];
System.arraycopy(data, 0, newData, 0, length);
data = newData;
}
}

-
7.5. CLASSADAPTER.JAVA 183

7.5 ClassAdapter.java
(ClassVisitor [229])
ClassAdapter.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* An empty {@link ClassVisitor} that delegates to another
* {@link ClassVisitor}. This class can be used as a super class to
* quickly implement usefull class adapter classes, just by overriding
* the necessary methods.
*
* @author Eric Bruneton
*/
public class ClassAdapter implements ClassVisitor{

/**
* The {@link ClassVisitor} to which this adapter delegates calls.
*/
protected ClassVisitor cv;

/**
* Constructs a new {@link ClassAdapter} object.
*
* @param cv the class visitor to which this adapter must delegate calls.
*/
public ClassAdapter(final ClassVisitor cv){
this.cv = cv;
}

public void visit(

final int version,
final int access,
final String name,
final String signature,
final String superName,
final String[] interfaces){
cv.visit(version, access, name, signature, superName, interfaces);
}

public void visitSource(final String source, final String debug){

cv.visitSource(source, debug);
}

public void visitOuterClass(

final String owner,
final String name,
184 CHAPTER 7. JVM/CLOJURE/ASM/

final String desc){

cv.visitOuterClass(owner, name, desc);
}

public AnnotationVisitor visitAnnotation(

final String desc,
final boolean visible){
return cv.visitAnnotation(desc, visible);
}

public void visitAttribute(final Attribute attr){

cv.visitAttribute(attr);
}

public void visitInnerClass(

final String name,
final String outerName,
final String innerName,
final int access){
cv.visitInnerClass(name, outerName, innerName, access);
}

public FieldVisitor visitField(

final int access,
final String name,
final String desc,
final String signature,
final Object value){
return cv.visitField(access, name, desc, signature, value);
}

public MethodVisitor visitMethod(

final int access,
final String name,
final String desc,
final String signature,
final String[] exceptions){
return cv.visitMethod(access, name, desc, signature, exceptions);
}

public void visitEnd(){

cv.visitEnd();
}
}

-
7.6. CLASSREADER.JAVA 185

7.6 ClassReader.java

ClassReader.java

\getchunk{France Telecom Copyright}

package clojure.asm;

import java.io.InputStream;
import java.io.IOException;

/**
* A Java class parser to make a {@link ClassVisitor} visit an
* existing class. This class parses a byte array conforming to
* the Java class file format and calls the appropriate visit
* methods of a given class visitor for each field, method and
* bytecode instruction encountered.
*
* @author Eric Bruneton
* @author Eugene Kuleshov
*/
public class ClassReader{

/**
* Flag to skip method code. If this class is set <code>CODE</code>
* attribute wont be visited. This can be used, for example, to retrieve
* annotations for methods and method parameters.
*/
public final static int SKIP_CODE = 1;

/**
* Flag to skip the debug information in the class. If this flag is set
* the debug information of the class is not visited, i.e. the
* {@link MethodVisitor#visitLocalVariable visitLocalVariable} and
* {@link MethodVisitor#visitLineNumber visitLineNumber} methods
* will not be called.
*/
public final static int SKIP_DEBUG = 2;

/**
* Flag to skip the stack map frames in the class. If this flag is set
* the stack map frames of the class is not visited, i.e. the
* {@link MethodVisitor#visitFrame visitFrame} method will not be called.
* This flag is useful when the {@link ClassWriter#COMPUTE_FRAMES} option
* is used: it avoids visiting frames that will be ignored and
* recomputed from scratch in the class writer.
*/
public final static int SKIP_FRAMES = 4;
186 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Flag to expand the stack map frames. By default stack map frames are
* visited in their original format (i.e. "expanded" for classes whose
* version is less than V1_6, and "compressed" for the other classes).
* If this flag is set, stack map frames are always visited in
* expanded format (this option adds a decompression/recompression
* step in ClassReader and ClassWriter which degrades performances
* quite a lot).
*/
public final static int EXPAND_FRAMES = 8;

/**
* The class to be parsed. <i>The content of this array must not be
* modified. This field is intended for {@link Attribute} sub classes,
* and is normally not needed by class generators or adapters.</i>
*/
public final byte[] b;

/**
* The start index of each constant pool item in {@link #b b}, plus one.
* The one byte offset skips the constant pool item tag that indicates
* its type.
*/
private final int[] items;

/**
* The String objects corresponding to the CONSTANT_Utf8 items.
* This cache avoids multiple parsing of a given CONSTANT_Utf8
* constant pool item, which GREATLY improves performances (by a
* factor 2 to 3). This caching strategy could be extended to all
* constant pool items, but its benefit would not be so great for
* these items (because they are much less expensive to parse than
* CONSTANT_Utf8 items).
*/
private final String[] strings;

/**
* Maximum length of the strings contained in the constant pool of the
* class.
*/
private final int maxStringLength;

/**
* Start index of the class header information (access, name...) in
* {@link #b b}.
*/
public final int header;

// -------------------------------------------------------------------
// Constructors
7.6. CLASSREADER.JAVA 187

// ------------------------------------------------------------------

/**
* Constructs a new {@link ClassReader} object.
*
* @param b the bytecode of the class to be read.
*/
public ClassReader(final byte[] b){
this(b, 0, b.length);
}

/**
* Constructs a new {@link ClassReader} object.
*
* @param b the bytecode of the class to be read.
* @param off the start offset of the class data.
* @param len the length of the class data.
*/
public ClassReader(final byte[] b, final int off, final int len){
this.b = b;
// parses the constant pool
items = new int[readUnsignedShort(off + 8)];
int n = items.length;
strings = new String[n];
int max = 0;
int index = off + 10;
for(int i = 1; i < n; ++i)
{
items[i] = index + 1;
int size;
switch(b[index])
{
case ClassWriter.FIELD:
case ClassWriter.METH:
case ClassWriter.IMETH:
case ClassWriter.INT:
case ClassWriter.FLOAT:
case ClassWriter.NAME_TYPE:
size = 5;
break;
case ClassWriter.LONG:
case ClassWriter.DOUBLE:
size = 9;
++i;
break;
case ClassWriter.UTF8:
size = 3 + readUnsignedShort(index + 1);
if(size > max)
{
max = size;
188 CHAPTER 7. JVM/CLOJURE/ASM/

}
break;
// case ClassWriter.CLASS:
// case ClassWriter.STR:
default:
size = 3;
break;
}
index += size;
}
maxStringLength = max;
// the class header information starts just after the constant pool
header = index;
}

/**
* Returns the classs access flags (see {@link Opcodes}). This value may
* not reflect Deprecated and Synthetic flags when bytecode is before 1.5
* and those flags are represented by attributes.
*
* @return the class access flags
* @see ClassVisitor#visit(int,int,String,String,String,String[])
*/
public int getAccess(){
return readUnsignedShort(header);
}

/**
* Returns the internal name of the class (see
* {@link Type#getInternalName() getInternalName}).
*
* @return the internal class name
* @see ClassVisitor#visit(int,int,String,String,String,String[])
*/
public String getClassName(){
return readClass(header + 2, new char[maxStringLength]);
}

/**
* Returns the internal of name of the super class (see
* {@link Type#getInternalName() getInternalName}). For interfaces, the
* super class is {@link Object}.
*
* @return the internal name of super class, or <tt>null</tt> for
* {@link Object} class.
* @see ClassVisitor#visit(int,int,String,String,String,String[])
*/
public String getSuperName(){
int n = items[readUnsignedShort(header + 4)];
return n == 0 ? null : readUTF8(n, new char[maxStringLength]);
7.6. CLASSREADER.JAVA 189

/**
* Returns the internal names of the classs interfaces (see
* {@link Type#getInternalName() getInternalName}).
*
* @return the array of internal names for all implemented interfaces or
* <tt>null</tt>.
* @see ClassVisitor#visit(int,int,String,String,String,String[])
*/
public String[] getInterfaces(){
int index = header + 6;
int n = readUnsignedShort(index);
String[] interfaces = new String[n];
if(n > 0)
{
char[] buf = new char[maxStringLength];
for(int i = 0; i < n; ++i)
{
index += 2;
interfaces[i] = readClass(index, buf);
}
}
return interfaces;
}

/**
* Copies the constant pool data into the given {@link ClassWriter}.
* Should be called before the {@link #accept(ClassVisitor,int)} method.
*
* @param classWriter the {@link ClassWriter} to copy constant pool into.
*/
void copyPool(final ClassWriter classWriter){
char[] buf = new char[maxStringLength];
int ll = items.length;
Item[] items2 = new Item[ll];
for(int i = 1; i < ll; i++)
{
int index = items[i];
int tag = b[index - 1];
Item item = new Item(i);
int nameType;
switch(tag)
{
case ClassWriter.FIELD:
case ClassWriter.METH:
case ClassWriter.IMETH:
nameType = items[readUnsignedShort(index + 2)];
item.set(tag,
readClass(index, buf),
190 CHAPTER 7. JVM/CLOJURE/ASM/

readUTF8(nameType, buf),
readUTF8(nameType + 2, buf));
break;

case ClassWriter.INT:
item.set(readInt(index));
break;

case ClassWriter.FLOAT:
item.set(Float.intBitsToFloat(readInt(index)));
break;

case ClassWriter.NAME_TYPE:
item.set(tag,
readUTF8(index, buf),
readUTF8(index + 2, buf),
null);
break;

case ClassWriter.LONG:
item.set(readLong(index));
++i;
break;

case ClassWriter.DOUBLE:
item.set(Double.longBitsToDouble(readLong(index)));
++i;
break;

case ClassWriter.UTF8:
{
String s = strings[i];
if(s == null)
{
index = items[i];
s = strings[i] = readUTF(index + 2,
readUnsignedShort(index),
buf);
}
item.set(tag, s, null, null);
}
break;

// case ClassWriter.STR:
// case ClassWriter.CLASS:
default:
item.set(tag, readUTF8(index, buf), null, null);
break;
}
7.6. CLASSREADER.JAVA 191

int index2 = item.hashCode % items2.length;

item.next = items2[index2];
items2[index2] = item;
}

int off = items[1] - 1;

classWriter.pool.putByteArray(b, off, header - off);
classWriter.items = items2;
classWriter.threshold = (int) (0.75d * ll);
classWriter.index = ll;
}

/**
* Constructs a new {@link ClassReader} object.
*
* @param is an input stream from which to read the class.
* @throws IOException if a problem occurs during reading.
*/
public ClassReader(final InputStream is) throws IOException{
this(readClass(is));
}

/**
* Constructs a new {@link ClassReader} object.
*
* @param name the fully qualified name of the class to be read.
* @throws IOException if an exception occurs during reading.
*/
public ClassReader(final String name) throws IOException{
this(ClassLoader.getSystemResourceAsStream(name.replace(., /)
+ ".class"));
}

/**
* Reads the bytecode of a class.
*
* @param is an input stream from which to read the class.
* @return the bytecode read from the given input stream.
* @throws IOException if a problem occurs during reading.
*/
private static byte[] readClass(final InputStream is) throws IOException{
if(is == null)
{
throw new IOException("Class not found");
}
byte[] b = new byte[is.available()];
int len = 0;
while(true)
{
int n = is.read(b, len, b.length - len);
192 CHAPTER 7. JVM/CLOJURE/ASM/

if(n == -1)
{
if(len < b.length)
{
byte[] c = new byte[len];
System.arraycopy(b, 0, c, 0, len);
b = c;
}
return b;
}
len += n;
if(len == b.length)
{
byte[] c = new byte[b.length + 1000];
System.arraycopy(b, 0, c, 0, len);
b = c;
}
}
}

// -------------------------------------------------------------------
// Public methods
// -------------------------------------------------------------------

/**
* Makes the given visitor visit the Java class of this
* {@link ClassReader}. This class is the one specified in the
* constructor (see {@link #ClassReader(byte[]) ClassReader}).
*
* @param classVisitor the visitor that must visit this class.
* @param flags option flags that can be used to modify
* the default behavior of this class. See
* {@link #SKIP_DEBUG}, {@link #EXPAND_FRAMES}.
*/
public void accept(final ClassVisitor classVisitor, final int flags){
accept(classVisitor, new Attribute[0], flags);
}

/**
* Makes the given visitor visit the Java class of this
* {@link ClassReader}. This class is the one specified in the
* constructor (see {@link #ClassReader(byte[]) ClassReader}).
*
* @param classVisitor the visitor that must visit this class.
* @param attrs prototypes of the attributes that must be parsed
* during the visit of the class. Any attribute whose
* type is not equal to the type of one the
* prototypes will not be parsed: its byte array
* value will be passed unchanged to the ClassWriter.
* <i>This may corrupt it if this value contains
7.6. CLASSREADER.JAVA 193

* references to the constant pool, or has syntactic

* or semantic links with a class element that has
* been transformed by a class adapter between the
* reader and the writer</i>.
* @param flags option flags that can be used to modify the
* default behavior of this class. See
* {@link #SKIP_DEBUG}, {@link #EXPAND_FRAMES}.
*/
public void accept(
final ClassVisitor classVisitor,
final Attribute[] attrs,
final int flags){
byte[] b = this.b; // the bytecode array
char[] c = new char[maxStringLength]; // buffer used to read strings
int i, j, k; // loop variables
int u, v, w; // indexes in b
Attribute attr;

int access;
String name;
String desc;
String attrName;
String signature;
int anns = 0;
int ianns = 0;
Attribute cattrs = null;

// visits the header

u = header;
access = readUnsignedShort(u);
name = readClass(u + 2, c);
v = items[readUnsignedShort(u + 4)];
String superClassName = v == 0 ? null : readUTF8(v, c);
String[] implementedItfs = new String[readUnsignedShort(u + 6)];
w = 0;
u += 8;
for(i = 0; i < implementedItfs.length; ++i)
{
implementedItfs[i] = readClass(u, c);
u += 2;
}

boolean skipCode = (flags & SKIP_CODE) != 0;

boolean skipDebug = (flags & SKIP_DEBUG) != 0;
boolean unzip = (flags & EXPAND_FRAMES) != 0;

// skips fields and methods

v = u;
i = readUnsignedShort(v);
v += 2;
194 CHAPTER 7. JVM/CLOJURE/ASM/

for(; i > 0; --i)

{
j = readUnsignedShort(v + 6);
v += 8;
for(; j > 0; --j)
{
v += 6 + readInt(v + 2);
}
}
i = readUnsignedShort(v);
v += 2;
for(; i > 0; --i)
{
j = readUnsignedShort(v + 6);
v += 8;
for(; j > 0; --j)
{
v += 6 + readInt(v + 2);
}
}
// reads the classs attributes
signature = null;
String sourceFile = null;
String sourceDebug = null;
String enclosingOwner = null;
String enclosingName = null;
String enclosingDesc = null;

i = readUnsignedShort(v);
v += 2;
for(; i > 0; --i)
{
attrName = readUTF8(v, c);
// tests are sorted in decreasing frequency order
// (based on frequencies observed on typical classes)
if(attrName.equals("SourceFile"))
{
sourceFile = readUTF8(v + 6, c);
}
else if(attrName.equals("InnerClasses"))
{
w = v + 6;
}
else if(attrName.equals("EnclosingMethod"))
{
enclosingOwner = readClass(v + 6, c);
int item = readUnsignedShort(v + 8);
if(item != 0)
{
enclosingName = readUTF8(items[item], c);
7.6. CLASSREADER.JAVA 195

enclosingDesc = readUTF8(items[item] + 2, c);

}
}
else if(attrName.equals("Signature"))
{
signature = readUTF8(v + 6, c);
}
else if(attrName.equals("RuntimeVisibleAnnotations"))
{
anns = v + 6;
}
else if(attrName.equals("Deprecated"))
{
access |= Opcodes.ACC_DEPRECATED;
}
else if(attrName.equals("Synthetic"))
{
access |= Opcodes.ACC_SYNTHETIC;
}
else if(attrName.equals("SourceDebugExtension"))
{
int len = readInt(v + 2);
sourceDebug = readUTF(v + 6, len, new char[len]);
}
else if(attrName.equals("RuntimeInvisibleAnnotations"))
{
ianns = v + 6;
}
else
{
attr = readAttribute(attrs,
attrName,
v + 6,
readInt(v + 2),
c,
-1,
null);
if(attr != null)
{
attr.next = cattrs;
cattrs = attr;
}
}
v += 6 + readInt(v + 2);
}
// calls the visit method
classVisitor.visit(readInt(4),
access,
name,
signature,
196 CHAPTER 7. JVM/CLOJURE/ASM/

superClassName,
implementedItfs);

// calls the visitSource method

if(!skipDebug && (sourceFile != null || sourceDebug != null))
{
classVisitor.visitSource(sourceFile, sourceDebug);
}

// calls the visitOuterClass method

if(enclosingOwner != null)
{
classVisitor.visitOuterClass(enclosingOwner,
enclosingName,
enclosingDesc);
}

// visits the class annotations

for(i = 1; i >= 0; --i)
{
v = i == 0 ? ianns : anns;
if(v != 0)
{
j = readUnsignedShort(v);
v += 2;
for(; j > 0; --j)
{
v = readAnnotationValues(v + 2, c, true,
classVisitor.visitAnnotation(readUTF8(v, c), i != 0));
}
}
}

// visits the class attributes

while(cattrs != null)
{
attr = cattrs.next;
cattrs.next = null;
classVisitor.visitAttribute(cattrs);
cattrs = attr;
}

// calls the visitInnerClass method

if(w != 0)
{
i = readUnsignedShort(w);
w += 2;
for(; i > 0; --i)
{
classVisitor.visitInnerClass(
7.6. CLASSREADER.JAVA 197

readUnsignedShort(w) == 0
? null
: readClass(w, c), readUnsignedShort(w + 2) == 0
? null
: readClass(w + 2, c), readUnsignedShort(w + 4) == 0
? null
: readUTF8(w + 4, c),
readUnsignedShort(w + 6));
w += 8;
}
}

// visits the fields

i = readUnsignedShort(u);
u += 2;
for(; i > 0; --i)
{
access = readUnsignedShort(u);
name = readUTF8(u + 2, c);
desc = readUTF8(u + 4, c);
// visits the fields attributes and looks for a ConstantValue
// attribute
int fieldValueItem = 0;
signature = null;
anns = 0;
ianns = 0;
cattrs = null;

j = readUnsignedShort(u + 6);
u += 8;
for(; j > 0; --j)
{
attrName = readUTF8(u, c);
// tests are sorted in decreasing frequency order
// (based on frequencies observed on typical classes)
if(attrName.equals("ConstantValue"))
{
fieldValueItem = readUnsignedShort(u + 6);
}
else if(attrName.equals("Signature"))
{
signature = readUTF8(u + 6, c);
}
else if(attrName.equals("Deprecated"))
{
access |= Opcodes.ACC_DEPRECATED;
}
else if(attrName.equals("Synthetic"))
{
access |= Opcodes.ACC_SYNTHETIC;
198 CHAPTER 7. JVM/CLOJURE/ASM/

}
else if(attrName.equals("RuntimeVisibleAnnotations"))
{
anns = u + 6;
}
else if(attrName.equals("RuntimeInvisibleAnnotations"))
{
ianns = u + 6;
}
else
{
attr = readAttribute(attrs,
attrName,
u + 6,
readInt(u + 2),
c,
-1,
null);
if(attr != null)
{
attr.next = cattrs;
cattrs = attr;
}
}
u += 6 + readInt(u + 2);
}
// visits the field
FieldVisitor fv =
classVisitor.visitField(access, name, desc, signature,
fieldValueItem == 0 ? null : readConst(fieldValueItem, c));
// visits the field annotations and attributes
if(fv != null)
{
for(j = 1; j >= 0; --j)
{
v = j == 0 ? ianns : anns;
if(v != 0)
{
k = readUnsignedShort(v);
v += 2;
for(; k > 0; --k)
{
v = readAnnotationValues(v + 2, c, true,
fv.visitAnnotation(readUTF8(v, c), j != 0));
}
}
}
while(cattrs != null)
{
attr = cattrs.next;
7.6. CLASSREADER.JAVA 199

cattrs.next = null;
fv.visitAttribute(cattrs);
cattrs = attr;
}
fv.visitEnd();
}
}

// visits the methods

i = readUnsignedShort(u);
u += 2;
for(; i > 0; --i)
{
int u0 = u + 6;
access = readUnsignedShort(u);
name = readUTF8(u + 2, c);
desc = readUTF8(u + 4, c);
signature = null;
anns = 0;
ianns = 0;
int dann = 0;
int mpanns = 0;
int impanns = 0;
cattrs = null;
v = 0;
w = 0;

// looks for Code and Exceptions attributes

j = readUnsignedShort(u + 6);
u += 8;
for(; j > 0; --j)
{
attrName = readUTF8(u, c);
int attrSize = readInt(u + 2);
u += 6;
// tests are sorted in decreasing frequency order
// (based on frequencies observed on typical classes)
if(attrName.equals("Code"))
{
if(!skipCode)
{
v = u;
}
}
else if(attrName.equals("Exceptions"))
{
w = u;
}
else if(attrName.equals("Signature"))
{
200 CHAPTER 7. JVM/CLOJURE/ASM/

signature = readUTF8(u, c);

}
else if(attrName.equals("Deprecated"))
{
access |= Opcodes.ACC_DEPRECATED;
}
else if(attrName.equals("RuntimeVisibleAnnotations"))
{
anns = u;
}
else if(attrName.equals("AnnotationDefault"))
{
dann = u;
}
else if(attrName.equals("Synthetic"))
{
access |= Opcodes.ACC_SYNTHETIC;
}
else if(attrName.equals("RuntimeInvisibleAnnotations"))
{
ianns = u;
}
else if(attrName.equals(
"RuntimeVisibleParameterAnnotations"))
{
mpanns = u;
}
else if(attrName.equals(
"RuntimeInvisibleParameterAnnotations"))
{
impanns = u;
}
else
{
attr = readAttribute(attrs,
attrName,
u,
attrSize,
c,
-1,
null);
if(attr != null)
{
attr.next = cattrs;
cattrs = attr;
}
}
u += attrSize;
}
// reads declared exceptions
7.6. CLASSREADER.JAVA 201

String[] exceptions;
if(w == 0)
{
exceptions = null;
}
else
{
exceptions = new String[readUnsignedShort(w)];
w += 2;
for(j = 0; j < exceptions.length; ++j)
{
exceptions[j] = readClass(w, c);
w += 2;
}
}

// visits the methods code, if any

MethodVisitor mv = classVisitor.visitMethod(access,
name,
desc,
signature,
exceptions);

if(mv != null)
{
/*
* if the returned MethodVisitor is in fact a MethodWriter, it
* means there is no method adapter between the reader and the
* writer. If, in addition, the writers constant pool was
* copied from this reader (mw.cw.cr == this), and the
* signature and exceptions of the method have not been
* changed, then it is possible to skip all visit events and
* just copy the original code of the method to the writer
* (the access, name and descriptor can have been changed,
* this is not important since they are not copied as is from
* the reader).
*/
if(mv instanceof MethodWriter)
{
MethodWriter mw = (MethodWriter) mv;
if(mw.cw.cr == this)
{
if(signature == mw.signature)
{
boolean sameExceptions = false;
if(exceptions == null)
{
sameExceptions = mw.exceptionCount == 0;
}
else
202 CHAPTER 7. JVM/CLOJURE/ASM/

{
if(exceptions.length == mw.exceptionCount)
{
sameExceptions = true;
for(j = exceptions.length - 1; j >= 0; --j)
{
w -= 2;
if(mw.exceptions[j] !=
readUnsignedShort(w))
{
sameExceptions = false;
break;
}
}
}
}
if(sameExceptions)
{
/*
* we do not copy directly the code into
* MethodWriter to save a byte array copy
* operation. The real copy will be done in
* ClassWriter.toByteArray().
*/
mw.classReaderOffset = u0;
mw.classReaderLength = u - u0;
continue;
}
}
}
}

if(dann != 0)
{
AnnotationVisitor dv = mv.visitAnnotationDefault();
readAnnotationValue(dann, c, null, dv);
if(dv != null)
{
dv.visitEnd();
}
}
for(j = 1; j >= 0; --j)
{
w = j == 0 ? ianns : anns;
if(w != 0)
{
k = readUnsignedShort(w);
w += 2;
for(; k > 0; --k)
{
7.6. CLASSREADER.JAVA 203

w = readAnnotationValues(w + 2, c, true,
mv.visitAnnotation(readUTF8(w, c), j != 0));
}
}
}
if(mpanns != 0)
{
readParameterAnnotations(mpanns, c, true, mv);
}
if(impanns != 0)
{
readParameterAnnotations(impanns, c, false, mv);
}
while(cattrs != null)
{
attr = cattrs.next;
cattrs.next = null;
mv.visitAttribute(cattrs);
cattrs = attr;
}
}

if(mv != null && v != 0)

{
int maxStack = readUnsignedShort(v);
int maxLocals = readUnsignedShort(v + 2);
int codeLength = readInt(v + 4);
v += 8;

int codeStart = v;
int codeEnd = v + codeLength;

mv.visitCode();

// 1st phase: finds the labels

int label;
Label[] labels = new Label[codeLength + 1];
while(v < codeEnd)
{
int opcode = b[v] & 0xFF;
switch(ClassWriter.TYPE[opcode])
{
case ClassWriter.NOARG_INSN:
case ClassWriter.IMPLVAR_INSN:
v += 1;
break;
case ClassWriter.LABEL_INSN:
label = v - codeStart + readShort(v + 1);
if(labels[label] == null)
{
204 CHAPTER 7. JVM/CLOJURE/ASM/

labels[label] = new Label();

}
v += 3;
break;
case ClassWriter.LABELW_INSN:
label = v - codeStart + readInt(v + 1);
if(labels[label] == null)
{
labels[label] = new Label();
}
v += 5;
break;
case ClassWriter.WIDE_INSN:
opcode = b[v + 1] & 0xFF;
if(opcode == Opcodes.IINC)
{
v += 6;
}
else
{
v += 4;
}
break;
case ClassWriter.TABL_INSN:
// skips 0 to 3 padding bytes
w = v - codeStart;
v = v + 4 - (w & 3);
// reads instruction
label = w + readInt(v);
if(labels[label] == null)
{
labels[label] = new Label();
}
j = readInt(v + 8) - readInt(v + 4) + 1;
v += 12;
for(; j > 0; --j)
{
label = w + readInt(v);
v += 4;
if(labels[label] == null)
{
labels[label] = new Label();
}
}
break;
case ClassWriter.LOOK_INSN:
// skips 0 to 3 padding bytes
w = v - codeStart;
v = v + 4 - (w & 3);
// reads instruction
7.6. CLASSREADER.JAVA 205

label = w + readInt(v);
if(labels[label] == null)
{
labels[label] = new Label();
}
j = readInt(v + 4);
v += 8;
for(; j > 0; --j)
{
label = w + readInt(v + 4);
v += 8;
if(labels[label] == null)
{
labels[label] = new Label();
}
}
break;
case ClassWriter.VAR_INSN:
case ClassWriter.SBYTE_INSN:
case ClassWriter.LDC_INSN:
v += 2;
break;
case ClassWriter.SHORT_INSN:
case ClassWriter.LDCW_INSN:
case ClassWriter.FIELDORMETH_INSN:
case ClassWriter.TYPE_INSN:
case ClassWriter.IINC_INSN:
v += 3;
break;
case ClassWriter.ITFMETH_INSN:
v += 5;
break;
// case MANA_INSN:
default:
v += 4;
break;
}
}
// parses the try catch entries
j = readUnsignedShort(v);
v += 2;
for(; j > 0; --j)
{
label = readUnsignedShort(v);
Label start = labels[label];
if(start == null)
{
labels[label] = start = new Label();
}
label = readUnsignedShort(v + 2);
206 CHAPTER 7. JVM/CLOJURE/ASM/

Label end = labels[label];

if(end == null)
{
labels[label] = end = new Label();
}
label = readUnsignedShort(v + 4);
Label handler = labels[label];
if(handler == null)
{
labels[label] = handler = new Label();
}
int type = readUnsignedShort(v + 6);
if(type == 0)
{
mv.visitTryCatchBlock(start, end, handler, null);
}
else
{
mv.visitTryCatchBlock(start,
end,
handler,
readUTF8(items[type], c));
}
v += 8;
}
// parses the local variable, line number tables, and code
// attributes
int varTable = 0;
int varTypeTable = 0;
int stackMap = 0;
int frameCount = 0;
int frameMode = 0;
int frameOffset = 0;
int frameLocalCount = 0;
int frameLocalDiff = 0;
int frameStackCount = 0;
Object[] frameLocal = null;
Object[] frameStack = null;
boolean zip = true;
cattrs = null;
j = readUnsignedShort(v);
v += 2;
for(; j > 0; --j)
{
attrName = readUTF8(v, c);
if(attrName.equals("LocalVariableTable"))
{
if(!skipDebug)
{
varTable = v + 6;
7.6. CLASSREADER.JAVA 207

k = readUnsignedShort(v + 6);
w = v + 8;
for(; k > 0; --k)
{
label = readUnsignedShort(w);
if(labels[label] == null)
{
labels[label] = new Label(true);
}
label += readUnsignedShort(w + 2);
if(labels[label] == null)
{
labels[label] = new Label(true);
}
w += 10;
}
}
}
else if(attrName.equals("LocalVariableTypeTable"))
{
varTypeTable = v + 6;
}
else if(attrName.equals("LineNumberTable"))
{
if(!skipDebug)
{
k = readUnsignedShort(v + 6);
w = v + 8;
for(; k > 0; --k)
{
label = readUnsignedShort(w);
if(labels[label] == null)
{
labels[label] = new Label(true);
}
labels[label].line=readUnsignedShort(w + 2);
w += 4;
}
}
}
else if(attrName.equals("StackMapTable"))
{
if((flags & SKIP_FRAMES) == 0)
{
stackMap = v + 8;
frameCount = readUnsignedShort(v + 6);
}
/*
* here we do not extract the labels corresponding to
* the attribute content. This would require a full
208 CHAPTER 7. JVM/CLOJURE/ASM/

* parsing of the attribute, which would need to be

* repeated in the second phase (see below). Instead the
* content of the attribute is read one frame at a time
* (i.e. after a frame has been visited, the next frame
* is read), and the labels it contains are also
* extracted one frame at a time. Thanks to the ordering
* of frames, having only a "one frame lookahead" is not
* a problem, i.e. it is not possible to see an offset
* smaller than the offset of the current insn and for
* which no Label exist.
*/
// TODO true for frame offsets,
// but for UNINITIALIZED type offsets?
}
else if(attrName.equals("StackMap"))
{
if((flags & SKIP_FRAMES) == 0)
{
stackMap = v + 8;
frameCount = readUnsignedShort(v + 6);
zip = false;
}
/*
* IMPORTANT! here we assume that the frames are
* ordered, as in the StackMapTable attribute,
* although this is not guaranteed by the
* attribute format.
*/
}
else
{
for(k = 0; k < attrs.length; ++k)
{
if(attrs[k].type.equals(attrName))
{
attr = attrs[k].read(this,
v + 6,
readInt(v + 2),
c,
codeStart - 8,
labels);
if(attr != null)
{
attr.next = cattrs;
cattrs = attr;
}
}
}
}
v += 6 + readInt(v + 2);
7.6. CLASSREADER.JAVA 209

// 2nd phase: visits each instruction

if(stackMap != 0)
{
// creates the very first (implicit) frame from the
// method descriptor
frameLocal = new Object[maxLocals];
frameStack = new Object[maxStack];
if(unzip)
{
int local = 0;
if((access & Opcodes.ACC_STATIC) == 0)
{
if(name.equals("<init>"))
{
frameLocal[local++] = Opcodes.UNINITIALIZED_THIS;
}
else
{
frameLocal[local++] = readClass(header + 2, c);
}
}
j = 1;
loop:
while(true)
{
k = j;
switch(desc.charAt(j++))
{
caseZ:
caseC:
caseB:
caseS:
caseI:
frameLocal[local++] = Opcodes.INTEGER;
break;
caseF:
frameLocal[local++] = Opcodes.FLOAT;
break;
caseJ:
frameLocal[local++] = Opcodes.LONG;
break;
caseD:
frameLocal[local++] = Opcodes.DOUBLE;
break;
case[:
while(desc.charAt(j) == [)
{
++j;
210 CHAPTER 7. JVM/CLOJURE/ASM/

}
if(desc.charAt(j) == L)
{
++j;
while(desc.charAt(j) != ;)
{
++j;
}
}
frameLocal[local++] =
desc.substring(k, ++j);
break;
caseL:
while(desc.charAt(j) != ;)
{
++j;
}
frameLocal[local++] =
desc.substring(k + 1, j++);
break;
default:
break loop;
}
}
frameLocalCount = local;
}
/*
* for the first explicit frame the offset is not
* offset_delta + 1 but only offset_delta; setting the
* implicit frame offset to -1 allow the use of the
* "offset_delta + 1" rule in all cases
*/
frameOffset = -1;
}
v = codeStart;
Label l;
while(v < codeEnd)
{
w = v - codeStart;

l = labels[w];
if(l != null)
{
mv.visitLabel(l);
if(!skipDebug && l.line > 0)
{
mv.visitLineNumber(l.line, l);
}
}
7.6. CLASSREADER.JAVA 211

while(frameLocal != null
&& (frameOffset == w || frameOffset == -1))
{
// if there is a frame for this offset,
// makes the visitor visit it,
// and reads the next frame if there is one.
if(!zip || unzip)
{
mv.visitFrame(Opcodes.F_NEW,
frameLocalCount,
frameLocal,
frameStackCount,
frameStack);
}
else if(frameOffset != -1)
{
mv.visitFrame(frameMode,
frameLocalDiff,
frameLocal,
frameStackCount,
frameStack);
}

if(frameCount > 0)
{
int tag, delta, n;
if(zip)
{
tag = b[stackMap++] & 0xFF;
}
else
{
tag = MethodWriter.FULL_FRAME;
frameOffset = -1;
}
frameLocalDiff = 0;
if(tag <
MethodWriter.SAME_LOCALS_1_STACK_ITEM_FRAME)
{
delta = tag;
frameMode = Opcodes.F_SAME;
frameStackCount = 0;
}
else if(tag < MethodWriter.RESERVED)
{
delta = tag
- MethodWriter.SAME_LOCALS_1_STACK_ITEM_FRAME;
stackMap = readFrameType(frameStack,
0,
stackMap,
212 CHAPTER 7. JVM/CLOJURE/ASM/

c,
labels);
frameMode = Opcodes.F_SAME1;
frameStackCount = 1;
}
else
{
delta = readUnsignedShort(stackMap);
stackMap += 2;
if(tag ==
MethodWriter.SAME_LOCALS_1_STACK_ITEM_FRAME_EXTENDED)
{
stackMap = readFrameType(frameStack,
0,
stackMap,
c,
labels);
frameMode = Opcodes.F_SAME1;
frameStackCount = 1;
}
else if(tag >= MethodWriter.CHOP_FRAME &&
tag < MethodWriter.SAME_FRAME_EXTENDED)
{
frameMode = Opcodes.F_CHOP;
frameLocalDiff =
MethodWriter.SAME_FRAME_EXTENDED
- tag;
frameLocalCount -= frameLocalDiff;
frameStackCount = 0;
}
else if(tag ==
MethodWriter.SAME_FRAME_EXTENDED)
{
frameMode = Opcodes.F_SAME;
frameStackCount = 0;
}
else if(tag < MethodWriter.FULL_FRAME)
{
j = unzip ? frameLocalCount : 0;
for(k = tag
- MethodWriter.SAME_FRAME_EXTENDED;
k > 0; k--)
{
stackMap = readFrameType(frameLocal,
j++,
stackMap,
c,
labels);
}
frameMode = Opcodes.F_APPEND;
7.6. CLASSREADER.JAVA 213

frameLocalDiff = tag
- MethodWriter.SAME_FRAME_EXTENDED;
frameLocalCount += frameLocalDiff;
frameStackCount = 0;
}
else
{ // if (tag == FULL_FRAME) {
frameMode = Opcodes.F_FULL;
n = frameLocalDiff
= frameLocalCount
= readUnsignedShort(stackMap);
stackMap += 2;
for(j = 0; n > 0; n--)
{
stackMap = readFrameType(frameLocal,
j++,
stackMap,
c,
labels);
}
n = frameStackCount
= readUnsignedShort(stackMap);
stackMap += 2;
for(j = 0; n > 0; n--)
{
stackMap = readFrameType(frameStack,
j++,
stackMap,
c,
labels);
}
}
}
frameOffset += delta + 1;
if(labels[frameOffset] == null)
{
labels[frameOffset] = new Label();
}

--frameCount;
}
else
{
frameLocal = null;
}
}

int opcode = b[v] & 0xFF;

switch(ClassWriter.TYPE[opcode])
{
214 CHAPTER 7. JVM/CLOJURE/ASM/

case ClassWriter.NOARG_INSN:
mv.visitInsn(opcode);
v += 1;
break;
case ClassWriter.IMPLVAR_INSN:
if(opcode > Opcodes.ISTORE)
{
opcode -= 59; // ISTORE_0
mv.visitVarInsn(
Opcodes.ISTORE + (opcode >> 2),
opcode & 0x3);
}
else
{
opcode -= 26; // ILOAD_0
mv.visitVarInsn(
Opcodes.ILOAD + (opcode >> 2),
opcode & 0x3);
}
v += 1;
break;
case ClassWriter.LABEL_INSN:
mv.visitJumpInsn(opcode,
labels[w + readShort(v + 1)]);
v += 3;
break;
case ClassWriter.LABELW_INSN:
mv.visitJumpInsn(opcode - 33,
labels[w + readInt(v + 1)]);
v += 5;
break;
case ClassWriter.WIDE_INSN:
opcode = b[v + 1] & 0xFF;
if(opcode == Opcodes.IINC)
{
mv.visitIincInsn(readUnsignedShort(v + 2),
readShort(v + 4));
v += 6;
}
else
{
mv.visitVarInsn(opcode,
readUnsignedShort(v + 2));
v += 4;
}
break;
case ClassWriter.TABL_INSN:
// skips 0 to 3 padding bytes
v = v + 4 - (w & 3);
// reads instruction
7.6. CLASSREADER.JAVA 215

label = w + readInt(v);
int min = readInt(v + 4);
int max = readInt(v + 8);
v += 12;
Label[] table = new Label[max - min + 1];
for(j = 0; j < table.length; ++j)
{
table[j] = labels[w + readInt(v)];
v += 4;
}
mv.visitTableSwitchInsn(min,
max,
labels[label],
table);
break;
case ClassWriter.LOOK_INSN:
// skips 0 to 3 padding bytes
v = v + 4 - (w & 3);
// reads instruction
label = w + readInt(v);
j = readInt(v + 4);
v += 8;
int[] keys = new int[j];
Label[] values = new Label[j];
for(j = 0; j < keys.length; ++j)
{
keys[j] = readInt(v);
values[j] = labels[w + readInt(v + 4)];
v += 8;
}
mv.visitLookupSwitchInsn(labels[label],
keys,
values);
break;
case ClassWriter.VAR_INSN:
mv.visitVarInsn(opcode, b[v + 1] & 0xFF);
v += 2;
break;
case ClassWriter.SBYTE_INSN:
mv.visitIntInsn(opcode, b[v + 1]);
v += 2;
break;
case ClassWriter.SHORT_INSN:
mv.visitIntInsn(opcode, readShort(v + 1));
v += 3;
break;
case ClassWriter.LDC_INSN:
mv.visitLdcInsn(readConst(b[v + 1] & 0xFF, c));
v += 2;
break;
216 CHAPTER 7. JVM/CLOJURE/ASM/

case ClassWriter.LDCW_INSN:
mv.visitLdcInsn(
readConst(readUnsignedShort(v + 1), c));
v += 3;
break;
case ClassWriter.FIELDORMETH_INSN:
case ClassWriter.ITFMETH_INSN:
int cpIndex = items[readUnsignedShort(v + 1)];
String iowner = readClass(cpIndex, c);
cpIndex = items[readUnsignedShort(cpIndex + 2)];
String iname = readUTF8(cpIndex, c);
String idesc = readUTF8(cpIndex + 2, c);
if(opcode < Opcodes.INVOKEVIRTUAL)
{
mv.visitFieldInsn(opcode,iowner,iname,idesc);
}
else
{
mv.visitMethodInsn(opcode,iowner,iname,idesc);
}
if(opcode == Opcodes.INVOKEINTERFACE)
{
v += 5;
}
else
{
v += 3;
}
break;
case ClassWriter.TYPE_INSN:
mv.visitTypeInsn(opcode, readClass(v + 1, c));
v += 3;
break;
case ClassWriter.IINC_INSN:
mv.visitIincInsn(b[v + 1] & 0xFF, b[v + 2]);
v += 3;
break;
// case MANA_INSN:
default:
mv.visitMultiANewArrayInsn(readClass(v + 1, c),
b[v + 3] & 0xFF);
v += 4;
break;
}
}
l = labels[codeEnd - codeStart];
if(l != null)
{
mv.visitLabel(l);
}
7.6. CLASSREADER.JAVA 217

// visits the local variable tables

if(!skipDebug && varTable != 0)
{
int[] typeTable = null;
if(varTypeTable != 0)
{
k = readUnsignedShort(varTypeTable) * 3;
w = varTypeTable + 2;
typeTable = new int[k];
while(k > 0)
{
typeTable[--k]= w + 6; // signature
typeTable[--k]= readUnsignedShort(w + 8); //index
typeTable[--k]= readUnsignedShort(w); // start
w += 10;
}
}
k = readUnsignedShort(varTable);
w = varTable + 2;
for(; k > 0; --k)
{
int start = readUnsignedShort(w);
int length = readUnsignedShort(w + 2);
int index = readUnsignedShort(w + 8);
String vsignature = null;
if(typeTable != null)
{
for(int a = 0; a < typeTable.length; a += 3)
{
if(typeTable[a] == start
&& typeTable[a + 1] == index)
{
vsignature=readUTF8(typeTable[a + 2], c);
break;
}
}
}
mv.visitLocalVariable(readUTF8(w + 4, c),
readUTF8(w + 6, c),
vsignature,
labels[start],
labels[start + length],
index);
w += 10;
}
}
// visits the other attributes
while(cattrs != null)
{
attr = cattrs.next;
218 CHAPTER 7. JVM/CLOJURE/ASM/

cattrs.next = null;
mv.visitAttribute(cattrs);
cattrs = attr;
}
// visits the max stack and max locals values
mv.visitMaxs(maxStack, maxLocals);
}

if(mv != null)
{
mv.visitEnd();
}
}

// visits the end of the class

classVisitor.visitEnd();
}

/**
* Reads parameter annotations and makes the given visitor visit them.
*
* @param v start offset in {@link #b b} of the annotations to
* be read.
* @param buf buffer to be used to call {@link #readUTF8 readUTF8},
* {@link #readClass(int,char[]) readClass} or
* {@link #readConst readConst}.
* @param visible <tt>true</tt> if the annotations to be read are visible
* at runtime.
* @param mv the visitor that must visit the annotations.
*/
private void readParameterAnnotations(
int v,
final char[] buf,
final boolean visible,
final MethodVisitor mv){
int n = b[v++] & 0xFF;
for(int i = 0; i < n; ++i)
{
int j = readUnsignedShort(v);
v += 2;
for(; j > 0; --j)
{
v = readAnnotationValues(v + 2, buf, true,
mv.visitParameterAnnotation(i,readUTF8(v, buf),visible));
}
}
}

/**
* Reads the values of an annotation and makes the given visitor
7.6. CLASSREADER.JAVA 219

* visit them.
*
* @param v the start offset in {@link #b b} of the values to be
* read (including the unsigned short that gives the
* number of values).
* @param buf buffer to be used to call {@link #readUTF8 readUTF8},
* {@link #readClass(int,char[]) readClass} or
* {@link #readConst readConst}.
* @param named if the annotation values are named or not.
* @param av the visitor that must visit the values.
* @return the end offset of the annotation values.
*/
private int readAnnotationValues(
int v,
final char[] buf,
final boolean named,
final AnnotationVisitor av){
int i = readUnsignedShort(v);
v += 2;
if(named)
{
for(; i > 0; --i)
{
v = readAnnotationValue(v + 2, buf, readUTF8(v, buf), av);
}
}
else
{
for(; i > 0; --i)
{
v = readAnnotationValue(v, buf, null, av);
}
}
if(av != null)
{
av.visitEnd();
}
return v;
}

/**
* Reads a value of an annotation and makes the given visitor visit it.
*
* @param v the start offset in {@link #b b} of the value to be
* read (<i>not including the value name constant pool
* index</i>).
* @param buf buffer to be used to call {@link #readUTF8 readUTF8},
* {@link #readClass(int,char[]) readClass} or
* {@link #readConst readConst}.
* @param name the name of the value to be read.
220 CHAPTER 7. JVM/CLOJURE/ASM/

* @param av the visitor that must visit the value.

* @return the end offset of the annotation value.
*/
private int readAnnotationValue(
int v,
final char[] buf,
final String name,
final AnnotationVisitor av){
int i;
if(av == null)
{
switch(b[v] & 0xFF)
{
casee: // enum_const_value
return v + 5;
case@: // annotation_value
return readAnnotationValues(v + 3, buf, true, null);
case[: // array_value
return readAnnotationValues(v + 1, buf, false, null);
default:
return v + 3;
}
}
switch(b[v++] & 0xFF)
{
caseI: // pointer to CONSTANT_Integer
caseJ: // pointer to CONSTANT_Long
caseF: // pointer to CONSTANT_Float
caseD: // pointer to CONSTANT_Double
av.visit(name, readConst(readUnsignedShort(v), buf));
v += 2;
break;
caseB: // pointer to CONSTANT_Byte
av.visit(name,
new Byte((byte)
readInt(items[readUnsignedShort(v)])));
v += 2;
break;
caseZ: // pointer to CONSTANT_Boolean
av.visit(name, readInt(items[readUnsignedShort(v)]) == 0
? Boolean.FALSE
: Boolean.TRUE);
v += 2;
break;
caseS: // pointer to CONSTANT_Short
av.visit(name,
new Short((short)
readInt(items[readUnsignedShort(v)])));
v += 2;
break;
7.6. CLASSREADER.JAVA 221

caseC: // pointer to CONSTANT_Char

av.visit(name,
new Character((char)
readInt(items[readUnsignedShort(v)])));
v += 2;
break;
cases: // pointer to CONSTANT_Utf8
av.visit(name, readUTF8(v, buf));
v += 2;
break;
casee: // enum_const_value
av.visitEnum(name, readUTF8(v, buf), readUTF8(v + 2, buf));
v += 4;
break;
casec: // class_info
av.visit(name, Type.getType(readUTF8(v, buf)));
v += 2;
break;
case@: // annotation_value
v = readAnnotationValues(v + 2, buf, true,
av.visitAnnotation(name, readUTF8(v, buf)));
break;
case[: // array_value
int size = readUnsignedShort(v);
v += 2;
if(size == 0)
{
return readAnnotationValues(v - 2,
buf,
false,
av.visitArray(name));
}
switch(this.b[v++] & 0xFF)
{
caseB:
byte[] bv = new byte[size];
for(i = 0; i < size; i++)
{
bv[i] =
(byte) readInt(items[readUnsignedShort(v)]);
v += 3;
}
av.visit(name, bv);
--v;
break;
caseZ:
boolean[] zv = new boolean[size];
for(i = 0; i < size; i++)
{
zv[i] =
222 CHAPTER 7. JVM/CLOJURE/ASM/

readInt(items[readUnsignedShort(v)]) != 0;
v += 3;
}
av.visit(name, zv);
--v;
break;
caseS:
short[] sv = new short[size];
for(i = 0; i < size; i++)
{
sv[i] =
(short) readInt(items[readUnsignedShort(v)]);
v += 3;
}
av.visit(name, sv);
--v;
break;
caseC:
char[] cv = new char[size];
for(i = 0; i < size; i++)
{
cv[i] =
(char) readInt(items[readUnsignedShort(v)]);
v += 3;
}
av.visit(name, cv);
--v;
break;
caseI:
int[] iv = new int[size];
for(i = 0; i < size; i++)
{
iv[i] = readInt(items[readUnsignedShort(v)]);
v += 3;
}
av.visit(name, iv);
--v;
break;
caseJ:
long[] lv = new long[size];
for(i = 0; i < size; i++)
{
lv[i] = readLong(items[readUnsignedShort(v)]);
v += 3;
}
av.visit(name, lv);
--v;
break;
caseF:
float[] fv = new float[size];
7.6. CLASSREADER.JAVA 223

for(i = 0; i < size; i++)

{
fv[i] =
Float.intBitsToFloat(
readInt(items[readUnsignedShort(v)]));
v += 3;
}
av.visit(name, fv);
--v;
break;
caseD:
double[] dv = new double[size];
for(i = 0; i < size; i++)
{
dv[i] =
Double.longBitsToDouble(
readLong(items[readUnsignedShort(v)]));
v += 3;
}
av.visit(name, dv);
--v;
break;
default:
v = readAnnotationValues(v - 3,
buf,
false,
av.visitArray(name));
}
}
return v;
}

private int readFrameType(

final Object[] frame,
final int index,
int v,
final char[] buf,
final Label[] labels){
int type = b[v++] & 0xFF;
switch(type)
{
case 0:
frame[index] = Opcodes.TOP;
break;
case 1:
frame[index] = Opcodes.INTEGER;
break;
case 2:
frame[index] = Opcodes.FLOAT;
break;
224 CHAPTER 7. JVM/CLOJURE/ASM/

case 3:
frame[index] = Opcodes.DOUBLE;
break;
case 4:
frame[index] = Opcodes.LONG;
break;
case 5:
frame[index] = Opcodes.NULL;
break;
case 6:
frame[index] = Opcodes.UNINITIALIZED_THIS;
break;
case 7: // Object
frame[index] = readClass(v, buf);
v += 2;
break;
default: // Uninitialized
int offset = readUnsignedShort(v);
if(labels[offset] == null)
{
labels[offset] = new Label();
}
frame[index] = labels[offset];
v += 2;
}
return v;
}

/**
* Reads an attribute in {@link #b b}.
*
* @param attrs prototypes of the attributes that must be parsed during
* the visit of the class. Any attribute whose type is not
* equal to the type of one the prototypes is ignored
* (i.e. an empty {@link Attribute} instance is returned).
* @param type the type of the attribute.
* @param off index of the first byte of the attributes content in
* {@link #b b}. The 6 attribute header bytes, containing
* the type and the length of the attribute, are not taken
* into account here (they have already been read).
* @param len the length of the attributes content.
* @param buf buffer to be used to call {@link #readUTF8 readUTF8},
* {@link #readClass(int,char[]) readClass} or
* {@link #readConst readConst}.
* @param codeOff index of the first byte of codes attribute content in
* {@link #b b}, or -1 if the attribute to be read is not
* a code attribute. The 6 attribute header bytes,
* containing the type and the length of the attribute,
* are not taken into account here.
* @param labels the labels of the methods code, or <tt>null</tt> if
7.6. CLASSREADER.JAVA 225

* the attribute to be read is not a code attribute.

* @return the attribute that has been read, or <tt>null</tt> to skip
* this attribute.
*/
private Attribute readAttribute(
final Attribute[] attrs,
final String type,
final int off,
final int len,
final char[] buf,
final int codeOff,
final Label[] labels){
for(int i = 0; i < attrs.length; ++i)
{
if(attrs[i].type.equals(type))
{
return attrs[i].read(this, off, len, buf, codeOff, labels);
}
}
return new Attribute(type).read(this, off, len, null, -1, null);
}

// -------------------------------------------------------------------
// Utility methods: low level parsing
// -------------------------------------------------------------------

/**
* Returns the start index of the constant pool item in {@link #b b},
* plus one. <i>This method is intended for {@link Attribute} sub
* classes, and is normally not needed by class generators or
* adapters.</i>
*
* @param item the index a constant pool item.
* @return the start index of the constant pool item in {@link #b b},
* plus one.
*/
public int getItem(final int item){
return items[item];
}

/**
* Reads a byte value in {@link #b b}. <i>This method is intended for
* {@link Attribute} sub classes, and is normally not needed by class
* generators or adapters.</i>
*
* @param index the start index of the value to be read in {@link #b b}.
* @return the read value.
*/
public int readByte(final int index){
return b[index] & 0xFF;
226 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Reads an unsigned short value in {@link #b b}. <i>This method is
* intended for {@link Attribute} sub classes, and is normally not
* needed by class generators or adapters.</i>
*
* @param index the start index of the value to be read in {@link #b b}.
* @return the read value.
*/
public int readUnsignedShort(final int index){
byte[] b = this.b;
return ((b[index] & 0xFF) << 8) | (b[index + 1] & 0xFF);
}

/**
* Reads a signed short value in {@link #b b}. <i>This method is intended
* for {@link Attribute} sub classes, and is normally not needed by class
* generators or adapters.</i>
*
* @param index the start index of the value to be read in {@link #b b}.
* @return the read value.
*/
public short readShort(final int index){
byte[] b = this.b;
return (short) (((b[index] & 0xFF) << 8) | (b[index + 1] & 0xFF));
}

/**
* Reads a signed int value in {@link #b b}. <i>This method is intended
* for {@link Attribute} sub classes, and is normally not needed by
* class generators or adapters.</i>
*
* @param index the start index of the value to be read in {@link #b b}.
* @return the read value.
*/
public int readInt(final int index){
byte[] b = this.b;
return ((b[index] & 0xFF) << 24) | ((b[index + 1] & 0xFF) << 16)
| ((b[index + 2] & 0xFF) << 8) | (b[index + 3] & 0xFF);
}

/**
* Reads a signed long value in {@link #b b}. <i>This method is intended
* for {@link Attribute} sub classes, and is normally not needed by class
* generators or adapters.</i>
*
* @param index the start index of the value to be read in {@link #b b}.
* @return the read value.
*/
7.6. CLASSREADER.JAVA 227

public long readLong(final int index){

long l1 = readInt(index);
long l0 = readInt(index + 4) & 0xFFFFFFFFL;
return (l1 << 32) | l0;
}

/**
* Reads an UTF8 string constant pool item in {@link #b b}. <i>This
* method is intended for {@link Attribute} sub classes, and is normally
* not needed by class generators or adapters.</i>
*
* @param index the start index of an unsigned short value in
* {@link #b b}, whose value is the index of an UTF8
* constant pool item.
* @param buf buffer to be used to read the item. This buffer must be
* sufficiently large. It is not automatically resized.
* @return the String corresponding to the specified UTF8 item.
*/
public String readUTF8(int index, final char[] buf){
int item = readUnsignedShort(index);
String s = strings[item];
if(s != null)
{
return s;
}
index = items[item];
return strings[item] =
readUTF(index + 2, readUnsignedShort(index), buf);
}

/**
* Reads UTF8 string in {@link #b b}.
*
* @param index start offset of the UTF8 string to be read.
* @param utfLen length of the UTF8 string to be read.
* @param buf buffer to be used to read the string. This buffer must
* be sufficiently large. It is not automatically resized.
* @return the String corresponding to the specified UTF8 string.
*/
private String readUTF(int index, final int utfLen, final char[] buf){
int endIndex = index + utfLen;
byte[] b = this.b;
int strLen = 0;
int c, d, e;
while(index < endIndex)
{
c = b[index++] & 0xFF;
switch(c >> 4)
{
case 0:
228 CHAPTER 7. JVM/CLOJURE/ASM/

case 1:
case 2:
case 3:
case 4:
case 5:
case 6:
case 7:
// 0xxxxxxx
buf[strLen++] = (char) c;
break;
case 12:
case 13:
// 110x xxxx 10xx xxxx
d = b[index++];
buf[strLen++] = (char) (((c & 0x1F) << 6) | (d & 0x3F));
break;
default:
// 1110 xxxx 10xx xxxx 10xx xxxx
d = b[index++];
e = b[index++];
buf[strLen++] =
(char) (((c & 0x0F) << 12)
| ((d & 0x3F) << 6) | (e & 0x3F));
break;
}
}
return new String(buf, 0, strLen);
}

/**
* Reads a class constant pool item in {@link #b b}. <i>This method
* is intended for {@link Attribute} sub classes, and is normally not
* needed by class generators or adapters.</i>
*
* @param index the start index of an unsigned short value in
* {@link #b b}, whose value is the index of a class
* constant pool item.
* @param buf buffer to be used to read the item. This buffer must be
* sufficiently large. It is not automatically resized.
* @return the String corresponding to the specified class item.
*/
public String readClass(final int index, final char[] buf){
// computes the start index of the CONSTANT_Class item in b
// and reads the CONSTANT_Utf8 item designated by
// the first two bytes of this CONSTANT_Class item
return readUTF8(items[readUnsignedShort(index)], buf);
}

/**
* Reads a numeric or string constant pool item in {@link #b b}.
7.7. CLASSVISITOR.JAVA 229

* <i>This method is intended for {@link Attribute} sub classes,

* and is normally not needed by class generators or adapters.</i>
*
* @param item the index of a constant pool item.
* @param buf buffer to be used to read the item. This buffer must be
* sufficiently large. It is not automatically resized.
* @return the {@link Integer}, {@link Float}, {@link Long},
* {@link Double}, {@link String} or {@link Type} corresponding
* to the given constant pool item.
*/
public Object readConst(final int item, final char[] buf){
int index = items[item];
switch(b[index - 1])
{
case ClassWriter.INT:
return new Integer(readInt(index));
case ClassWriter.FLOAT:
return new Float(Float.intBitsToFloat(readInt(index)));
case ClassWriter.LONG:
return new Long(readLong(index));
case ClassWriter.DOUBLE:
return new Double(Double.longBitsToDouble(readLong(index)));
case ClassWriter.CLASS:
String s = readUTF8(index, buf);
return s.charAt(0) == [
? Type.getType(s)
: Type.getObjectType(s);
// case ClassWriter.STR:
default:
return readUTF8(index, buf);
}
}
}

7.7 ClassVisitor.java

ClassVisitor.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* A visitor to visit a Java class. The methods of this interface
* must be called in the following order: <tt>visit</tt>
* [ <tt>visitSource</tt> ] [ <tt>visitOuterClass</tt> ]
230 CHAPTER 7. JVM/CLOJURE/ASM/

* ( <tt>visitAnnotation</tt> | <tt>visitAttribute</tt> )*
* (<tt>visitInnerClass</tt> | <tt>visitField</tt> |
* <tt>visitMethod</tt> )* <tt>visitEnd</tt>.
*
* @author Eric Bruneton
*/
public interface ClassVisitor{

/**
* Visits the header of the class.
*
* @param version the class version.
* @param access the classs access flags (see {@link Opcodes}). This
* parameter also indicates if the class is deprecated.
* @param name the internal name of the class (see
* {@link Type#getInternalName() getInternalName}).
* @param signature the signature of this class. May be <tt>null</tt> if
* the class is not a generic one, and does not extend
* or implement generic classes or interfaces.
* @param superName the internal of name of the super class (see
* {@link Type#getInternalName() getInternalName}).
* For interfaces, the super class is {@link Object}.
* May be <tt>null</tt>, but only for the
* {@link Object} class.
* @param interfaces the internal names of the classs interfaces (see
* {@link Type#getInternalName() getInternalName}).
* May be <tt>null</tt>.
*/
void visit(
int version,
int access,
String name,
String signature,
String superName,
String[] interfaces);

/**
* Visits the source of the class.
*
* @param source the name of the source file from which the class was
* compiled. May be <tt>null</tt>.
* @param debug additional debug information to compute the
* correspondance between source and compiled elements of
* the class. May be <tt>null</tt>.
*/
void visitSource(String source, String debug);

/**
* Visits the enclosing class of the class. This method must be called
* only if the class has an enclosing class.
7.7. CLASSVISITOR.JAVA 231

*
* @param owner internal name of the enclosing class of the class.
* @param name the name of the method that contains the class, or
* <tt>null</tt> if the class is not enclosed in a method
* of its enclosing class.
* @param desc the descriptor of the method that contains the class, or
* <tt>null</tt> if the class is not enclosed in a method of
* its enclosing class.
*/
void visitOuterClass(String owner, String name, String desc);

/**
* Visits an annotation of the class.
*
* @param desc the class descriptor of the annotation class.
* @param visible <tt>true</tt> if the annotation is visible at runtime.
* @return a visitor to visit the annotation values, or <tt>null</tt> if
* this visitor is not interested in visiting this annotation.
*/
AnnotationVisitor visitAnnotation(String desc, boolean visible);

/**
* Visits a non standard attribute of the class.
*
* @param attr an attribute.
*/
void visitAttribute(Attribute attr);

/**
* Visits information about an inner class. This inner class is not
* necessarily a member of the class being visited.
*
* @param name the internal name of an inner class (see
* {@link Type#getInternalName() getInternalName}).
* @param outerName the internal name of the class to which the inner
* class belongs (see {@link Type#getInternalName()
* getInternalName}). May be <tt>null</tt> for not
* member classes.
* @param innerName the (simple) name of the inner class inside its
* enclosing class. May be <tt>null</tt> for anonymous
* inner classes.
* @param access the access flags of the inner class as originally
* declared in the enclosing class.
*/
void visitInnerClass(
String name,
String outerName,
String innerName,
int access);
232 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Visits a field of the class.
*
* @param access the fields access flags (see {@link Opcodes}).
* This parameter also indicates if the field is
* synthetic and/or deprecated.
* @param name the fields name.
* @param desc the fields descriptor (see {@link Type Type}).
* @param signature the fields signature. May be <tt>null</tt> if the
* fields type does not use generic types.
* @param value the fields initial value. This parameter, which may
* be <tt>null</tt> if the field does not have an
* initial value, must be an {@link Integer}, a
* {@link Float}, a {@link Long}, a {@link Double} or
* a {@link String} (for <tt>int</tt>, <tt>float</tt>,
* <tt>long</tt> or <tt>String</tt> fields
* respectively). <i>This parameter is only used for
* static fields</i>. Its value is ignored for non
* static fields, which must be initialized through
* bytecode instructions in constructors or methods.
* @return a visitor to visit field annotations and attributes, or
* <tt>null</tt> if this class visitor is not interested in
* visiting these annotations and attributes.
*/
FieldVisitor visitField(
int access,
String name,
String desc,
String signature,
Object value);

/**
* Visits a method of the class. This method <i>must</i> return a new
* {@link MethodVisitor} instance (or <tt>null</tt>) each time it is
* called, i.e., it should not return a previously returned visitor.
*
* @param access the methods access flags (see {@link Opcodes}).
* This parameter also indicates if the method is
* synthetic and/or deprecated.
* @param name the methods name.
* @param desc the methods descriptor (see {@link Type Type}).
* @param signature the methods signature. May be <tt>null</tt> if the
* method parameters, return type and exceptions do not
* use generic types.
* @param exceptions the internal names of the methods exception classes
* (see
* {@link Type#getInternalName() getInternalName}).
* May be <tt>null</tt>.
* @return an object to visit the byte code of the method, or
* <tt>null</tt> if this class visitor is not interested in
7.8. CLASSWRITER.JAVA 233

* visiting the code of this method.

*/
MethodVisitor visitMethod(
int access,
String name,
String desc,
String signature,
String[] exceptions);

/**
* Visits the end of the class. This method, which is the last one to be
* called, is used to inform the visitor that all the fields and methods
* of the class have been visited.
*/
void visitEnd();
}

7.8 ClassWriter.java
(ClassVisitor [229])
ClassWriter.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* A {@link ClassVisitor} that generates classes in bytecode form.
* More precisely this visitor generates a byte array conforming to
* the Java class file format. It can be used alone, to generate a
* Java class "from scratch", or with one or more
* {@link ClassReader ClassReader} and adapter class visitor
* to generate a modified class from one or more existing Java classes.
*
* @author Eric Bruneton
*/
public class ClassWriter implements ClassVisitor{

/**
* Flag to automatically compute the maximum stack size and the
* maximum number of local variables of methods. If this flag is set,
* then the arguments of the {@link MethodVisitor#visitMaxs visitMaxs}
* method of the {@link MethodVisitor} returned by the
* {@link #visitMethod visitMethod} method will be ignored, and
* computed automatically from the signature and the bytecode of
* each method.
*
234 CHAPTER 7. JVM/CLOJURE/ASM/

* @see #ClassWriter(int)
*/
public final static int COMPUTE_MAXS = 1;

/**
* Flag to automatically compute the stack map frames of methods from
* scratch. If this flag is set, then the calls to the
* {@link MethodVisitor#visitFrame} method are ignored, and the stack map
* frames are recomputed from the methods bytecode. The arguments of the
* {@link MethodVisitor#visitMaxs visitMaxs} method are also ignored and
* recomputed from the bytecode. In other words, computeFrames implies
* computeMaxs.
*
* @see #ClassWriter(int)
*/
public final static int COMPUTE_FRAMES = 2;

/**
* The type of instructions without any argument.
*/
final static int NOARG_INSN = 0;

/**
* The type of instructions with an signed byte argument.
*/
final static int SBYTE_INSN = 1;

/**
* The type of instructions with an signed short argument.
*/
final static int SHORT_INSN = 2;

/**
* The type of instructions with a local variable index argument.
*/
final static int VAR_INSN = 3;

/**
* The type of instructions with an implicit local variable index
* argument.
*/
final static int IMPLVAR_INSN = 4;

/**
* The type of instructions with a type descriptor argument.
*/
final static int TYPE_INSN = 5;

/**
* The type of field and method invocations instructions.
7.8. CLASSWRITER.JAVA 235

*/
final static int FIELDORMETH_INSN = 6;

/**
* The type of the INVOKEINTERFACE instruction.
*/
final static int ITFMETH_INSN = 7;

/**
* The type of instructions with a 2 bytes bytecode offset label.
*/
final static int LABEL_INSN = 8;

/**
* The type of instructions with a 4 bytes bytecode offset label.
*/
final static int LABELW_INSN = 9;

/**
* The type of the LDC instruction.
*/
final static int LDC_INSN = 10;

/**
* The type of the LDC_W and LDC2_W instructions.
*/
final static int LDCW_INSN = 11;

/**
* The type of the IINC instruction.
*/
final static int IINC_INSN = 12;

/**
* The type of the TABLESWITCH instruction.
*/
final static int TABL_INSN = 13;

/**
* The type of the LOOKUPSWITCH instruction.
*/
final static int LOOK_INSN = 14;

/**
* The type of the MULTIANEWARRAY instruction.
*/
final static int MANA_INSN = 15;

/**
* The type of the WIDE instruction.
236 CHAPTER 7. JVM/CLOJURE/ASM/

*/
final static int WIDE_INSN = 16;

/**
* The instruction types of all JVM opcodes.
*/
static byte[] TYPE;

/**
* The type of CONSTANT_Class constant pool items.
*/
final static int CLASS = 7;

/**
* The type of CONSTANT_Fieldref constant pool items.
*/
final static int FIELD = 9;

/**
* The type of CONSTANT_Methodref constant pool items.
*/
final static int METH = 10;

/**
* The type of CONSTANT_InterfaceMethodref constant pool items.
*/
final static int IMETH = 11;

/**
* The type of CONSTANT_String constant pool items.
*/
final static int STR = 8;

/**
* The type of CONSTANT_Integer constant pool items.
*/
final static int INT = 3;

/**
* The type of CONSTANT_Float constant pool items.
*/
final static int FLOAT = 4;

/**
* The type of CONSTANT_Long constant pool items.
*/
final static int LONG = 5;

/**
* The type of CONSTANT_Double constant pool items.
7.8. CLASSWRITER.JAVA 237

*/
final static int DOUBLE = 6;

/**
* The type of CONSTANT_NameAndType constant pool items.
*/
final static int NAME_TYPE = 12;

/**
* The type of CONSTANT_Utf8 constant pool items.
*/
final static int UTF8 = 1;

/**
* Normal type Item stored in the ClassWriter
* {@link ClassWriter#typeTable}, instead of the constant pool,
* in order to avoid clashes with normal constant pool items in
* the ClassWriter constant pools hash table.
*/
final static int TYPE_NORMAL = 13;

/**
* Uninitialized type Item stored in the ClassWriter
* {@link ClassWriter#typeTable}, instead of the constant pool, in
* order to avoid clashes with normal constant pool items in the
* ClassWriter constant pools hash table.
*/
final static int TYPE_UNINIT = 14;

/**
* Merged type Item stored in the ClassWriter
* {@link ClassWriter#typeTable}, instead of the constant pool, in
* order to avoid clashes with normal constant pool items in the
* ClassWriter constant pools hash table.
*/
final static int TYPE_MERGED = 15;

/**
* The class reader from which this class writer was constructed, if any.
*/
ClassReader cr;

/**
* Minor and major version numbers of the class to be generated.
*/
int version;

/**
* Index of the next item to be added in the constant pool.
*/
238 CHAPTER 7. JVM/CLOJURE/ASM/

int index;

/**
* The constant pool of this class.
*/
ByteVector pool;

/**
* The constant pools hash table data.
*/
Item[] items;

/**
* The threshold of the constant pools hash table.
*/
int threshold;

/**
* A reusable key used to look for items in the {@link #items} hash
* table.
*/
Item key;

/**
* A reusable key used to look for items in the {@link #items} hash
* table.
*/
Item key2;

/**
* A reusable key used to look for items in the {@link #items} hash
* table.
*/
Item key3;

/**
* A type table used to temporarily store internal names that will
* not necessarily be stored in the constant pool. This type table
* is used by the control flow and data flow analysis algorithm
* used to compute stack map frames from scratch. This array
* associates to each index <tt>i</tt> the Item whose index is
* <tt>i</tt>. All Item objects stored in this array are also stored
* in the {@link #items} hash table. These two arrays allow to retrieve
* an Item from its index or, conversly, to get the index of an Item
* from its value. Each Item stores an internal name in its
* {@link Item#strVal1} field.
*/
Item[] typeTable;

/**
7.8. CLASSWRITER.JAVA 239

* Number of elements in the {@link #typeTable} array.

*/
private short typeCount; // TODO int?

/**
* The access flags of this class.
*/
private int access;

/**
* The constant pool item that contains the internal name of this
* class.
*/
private int name;

/**
* The internal name of this class.
*/
String thisName;

/**
* The constant pool item that contains the signature of this class.
*/
private int signature;

/**
* The constant pool item that contains the internal name of the super
* class of this class.
*/
private int superName;

/**
* Number of interfaces implemented or extended by this class or
* interface.
*/
private int interfaceCount;

/**
* The interfaces implemented or extended by this class or interface.
* More precisely, this array contains the indexes of the constant
* pool items that contain the internal names of these interfaces.
*/
private int[] interfaces;

/**
* The index of the constant pool item that contains the name of the
* source file from which this class was compiled.
*/
private int sourceFile;
240 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* The SourceDebug attribute of this class.
*/
private ByteVector sourceDebug;

/**
* The constant pool item that contains the name of the enclosing class
* of this class.
*/
private int enclosingMethodOwner;

/**
* The constant pool item that contains the name and descriptor of the
* enclosing method of this class.
*/
private int enclosingMethod;

/**
* The runtime visible annotations of this class.
*/
private AnnotationWriter anns;

/**
* The runtime invisible annotations of this class.
*/
private AnnotationWriter ianns;

/**
* The non standard attributes of this class.
*/
private Attribute attrs;

/**
* The number of entries in the InnerClasses attribute.
*/
private int innerClassesCount;

/**
* The InnerClasses attribute.
*/
private ByteVector innerClasses;

/**
* The fields of this class. These fields are stored in a linked list
* of {@link FieldWriter} objects, linked to each other by their
* {@link FieldWriter#next} field. This field stores the first element
* of this list.
*/
FieldWriter firstField;
7.8. CLASSWRITER.JAVA 241

/**
* The fields of this class. These fields are stored in a linked list
* of {@link FieldWriter} objects, linked to each other by their
* {@link FieldWriter#next} field. This field stores the last element
* of this list.
*/
FieldWriter lastField;

/**
* The methods of this class. These methods are stored in a linked list
* of {@link MethodWriter} objects, linked to each other by their
* {@link MethodWriter#next} field. This field stores the first element
* of this list.
*/
MethodWriter firstMethod;

/**
* The methods of this class. These methods are stored in a linked list
* of {@link MethodWriter} objects, linked to each other by their
* {@link MethodWriter#next} field. This field stores the last element
* of this list.
*/
MethodWriter lastMethod;

/**
* <tt>true</tt> if the maximum stack size and number of local variables
* must be automatically computed.
*/
private boolean computeMaxs;

/**
* <tt>true</tt> if the stack map frames must be recomputed from scratch.
*/
private boolean computeFrames;

/**
* <tt>true</tt> if the stack map tables of this class are invalid. The
* {@link MethodWriter#resizeInstructions} method cannot transform
* existing stack map tables, and so produces potentially invalid
* classes when it is executed. In this case the class is reread
* and rewritten with the {@link #COMPUTE_FRAMES} option (the
* resizeInstructions method can resize stack map tables when this
* option is used).
*/
boolean invalidFrames;

// -------------------------------------------------------------------
// Static initializer
// -------------------------------------------------------------------
242 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Computes the instruction types of JVM opcodes.
*/
static
{
int i;
byte[] b = new byte[220];
String s =
"AAAAAAAAAAAAAAAABCKLLDDDDDEEEEEEEEEEEEEEEEEEEEAAAAAAAADD"
+ "DDDEEEEEEEEEEEEEEEEEEEEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
+ "AAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAIIIIIIIIIIIIIIIIDNOAA"
+ "AAAAGGGGGGGHAFBFAAFFAAQPIIJJIIIIIIIIIIIIIIIIII";
for(i = 0; i < b.length; ++i)
{
b[i] = (byte) (s.charAt(i) - A);
}
TYPE = b;

// code to generate the above string

//
// // SBYTE_INSN instructions
// b[Constants.NEWARRAY] = SBYTE_INSN;
// b[Constants.BIPUSH] = SBYTE_INSN;
//
// // SHORT_INSN instructions
// b[Constants.SIPUSH] = SHORT_INSN;
//
// // (IMPL)VAR_INSN instructions
// b[Constants.RET] = VAR_INSN;
// for (i = Constants.ILOAD; i <= Constants.ALOAD; ++i) {
// b[i] = VAR_INSN;
// }
// for (i = Constants.ISTORE; i <= Constants.ASTORE; ++i) {
// b[i] = VAR_INSN;
// }
// for (i = 26; i <= 45; ++i) { // ILOAD_0 to ALOAD_3
// b[i] = IMPLVAR_INSN;
// }
// for (i = 59; i <= 78; ++i) { // ISTORE_0 to ASTORE_3
// b[i] = IMPLVAR_INSN;
// }
//
// // TYPE_INSN instructions
// b[Constants.NEW] = TYPE_INSN;
// b[Constants.ANEWARRAY] = TYPE_INSN;
// b[Constants.CHECKCAST] = TYPE_INSN;
// b[Constants.INSTANCEOF] = TYPE_INSN;
//
// // (Set)FIELDORMETH_INSN instructions
// for (i = Constants.GETSTATIC; i <= Constants.INVOKESTATIC; ++i) {
7.8. CLASSWRITER.JAVA 243

// b[i] = FIELDORMETH_INSN;
// }
// b[Constants.INVOKEINTERFACE] = ITFMETH_INSN;
//
// // LABEL(W)_INSN instructions
// for (i = Constants.IFEQ; i <= Constants.JSR; ++i) {
// b[i] = LABEL_INSN;
// }
// b[Constants.IFNULL] = LABEL_INSN;
// b[Constants.IFNONNULL] = LABEL_INSN;
// b[200] = LABELW_INSN; // GOTO_W
// b[201] = LABELW_INSN; // JSR_W
// // temporary opcodes used internally by ASM - see Label and
// MethodWriter
// for (i = 202; i < 220; ++i) {
// b[i] = LABEL_INSN;
// }
//
// // LDC(_W) instructions
// b[Constants.LDC] = LDC_INSN;
// b[19] = LDCW_INSN; // LDC_W
// b[20] = LDCW_INSN; // LDC2_W
//
// // special instructions
// b[Constants.IINC] = IINC_INSN;
// b[Constants.TABLESWITCH] = TABL_INSN;
// b[Constants.LOOKUPSWITCH] = LOOK_INSN;
// b[Constants.MULTIANEWARRAY] = MANA_INSN;
// b[196] = WIDE_INSN; // WIDE
//
// for (i = 0; i < b.length; ++i) {
// System.err.print((char)(A + b[i]));
// }
// System.err.println();
}

// -------------------------------------------------------------------
// Constructor
// -------------------------------------------------------------------

/**
* Constructs a new {@link ClassWriter} object.
*
* @param flags option flags that can be used to modify the default
* behavior of this class. See {@link #COMPUTE_MAXS},
* {@link #COMPUTE_FRAMES}.
*/
public ClassWriter(final int flags){
index = 1;
pool = new ByteVector();
244 CHAPTER 7. JVM/CLOJURE/ASM/

items = new Item[256];

threshold = (int) (0.75d * items.length);
key = new Item();
key2 = new Item();
key3 = new Item();
this.computeMaxs = (flags & COMPUTE_MAXS) != 0;
this.computeFrames = (flags & COMPUTE_FRAMES) != 0;
}

/**
* Constructs a new {@link ClassWriter} object and enables optimizations
* for "mostly add" bytecode transformations. These optimizations are
* the following:
* <p/>
* <ul> <li>The constant pool from the original class is copied as is
* in the new class, which saves time. New constant pool entries will
* be added at the end if necessary, but unused constant pool entries
* <i>wont be removed</i>.</li> <li>Methods that are not transformed
* are copied as is in the new class, directly from the original class
* bytecode (i.e. without emitting visit events for all the method
* instructions), which saves a <i>lot</i> of time. Untransformed
* methods are detected by the fact that the {@link ClassReader}
* receives {@link MethodVisitor} objects that come from a
* {@link ClassWriter} (and not from a custom {@link ClassAdapter}
* or any other {@link ClassVisitor} instance).</li> </ul>
*
* @param classReader the {@link ClassReader} used to read the original
* class. It will be used to copy the entire
* constant pool from the original class and also
* to copy other fragments of original bytecode
* where applicable.
* @param flags option flags that can be used to modify the
* default behavior of this class. See
* {@link #COMPUTE_MAXS}, {@link #COMPUTE_FRAMES}.
*/
public ClassWriter(final ClassReader classReader, final int flags){
this(flags);
classReader.copyPool(this);
this.cr = classReader;
}

// -------------------------------------------------------------------
// Implementation of the ClassVisitor interface
// -------------------------------------------------------------------

public void visit(

final int version,
final int access,
final String name,
final String signature,
7.8. CLASSWRITER.JAVA 245

final String superName,

final String[] interfaces){
this.version = version;
this.access = access;
this.name = newClass(name);
thisName = name;
if(signature != null)
{
this.signature = newUTF8(signature);
}
this.superName = superName == null ? 0 : newClass(superName);
if(interfaces != null && interfaces.length > 0)
{
interfaceCount = interfaces.length;
this.interfaces = new int[interfaceCount];
for(int i = 0; i < interfaceCount; ++i)
{
this.interfaces[i] = newClass(interfaces[i]);
}
}
}

public void visitSource(final String file, final String debug){

if(file != null)
{
sourceFile = newUTF8(file);
}
if(debug != null)
{
sourceDebug = new ByteVector().putUTF8(debug);
}
}

public void visitOuterClass(

final String owner,
final String name,
final String desc){
enclosingMethodOwner = newClass(owner);
if(name != null && desc != null)
{
enclosingMethod = newNameType(name, desc);
}
}

public AnnotationVisitor visitAnnotation(

final String desc,
final boolean visible){
ByteVector bv = new ByteVector();
// write type, and reserve space for values count
bv.putShort(newUTF8(desc)).putShort(0);
246 CHAPTER 7. JVM/CLOJURE/ASM/

AnnotationWriter aw = new AnnotationWriter(this, true, bv, bv, 2);

if(visible)
{
aw.next = anns;
anns = aw;
}
else
{
aw.next = ianns;
ianns = aw;
}
return aw;
}

public void visitAttribute(final Attribute attr){

attr.next = attrs;
attrs = attr;
}

public void visitInnerClass(

final String name,
final String outerName,
final String innerName,
final int access){
if(innerClasses == null)
{
innerClasses = new ByteVector();
}
++innerClassesCount;
innerClasses.putShort(name == null ? 0 : newClass(name));
innerClasses.putShort(outerName == null ? 0 : newClass(outerName));
innerClasses.putShort(innerName == null ? 0 : newUTF8(innerName));
innerClasses.putShort(access);
}

public FieldVisitor visitField(

final int access,
final String name,
final String desc,
final String signature,
final Object value){
return new FieldWriter(this, access, name, desc, signature, value);
}

public MethodVisitor visitMethod(

final int access,
final String name,
final String desc,
final String signature,
final String[] exceptions){
7.8. CLASSWRITER.JAVA 247

return new MethodWriter(this,

access,
name,
desc,
signature,
exceptions,
computeMaxs,
computeFrames);
}

public void visitEnd(){

}

// -------------------------------------------------------------------
// Other public methods
// -------------------------------------------------------------------

/**
* Returns the bytecode of the class that was build with this class
* writer.
*
* @return the bytecode of the class that was build with this class
* writer.
*/
public byte[] toByteArray(){
// computes the real size of the bytecode of this class
int size = 24 + 2 * interfaceCount;
int nbFields = 0;
FieldWriter fb = firstField;
while(fb != null)
{
++nbFields;
size += fb.getSize();
fb = fb.next;
}
int nbMethods = 0;
MethodWriter mb = firstMethod;
while(mb != null)
{
++nbMethods;
size += mb.getSize();
mb = mb.next;
}
int attributeCount = 0;
if(signature != 0)
{
++attributeCount;
size += 8;
newUTF8("Signature");
}
248 CHAPTER 7. JVM/CLOJURE/ASM/

if(sourceFile != 0)
{
++attributeCount;
size += 8;
newUTF8("SourceFile");
}
if(sourceDebug != null)
{
++attributeCount;
size += sourceDebug.length + 4;
newUTF8("SourceDebugExtension");
}
if(enclosingMethodOwner != 0)
{
++attributeCount;
size += 10;
newUTF8("EnclosingMethod");
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
++attributeCount;
size += 6;
newUTF8("Deprecated");
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
&& (version & 0xffff) < Opcodes.V1_5)
{
++attributeCount;
size += 6;
newUTF8("Synthetic");
}
if(innerClasses != null)
{
++attributeCount;
size += 8 + innerClasses.length;
newUTF8("InnerClasses");
}
if(anns != null)
{
++attributeCount;
size += 8 + anns.getSize();
newUTF8("RuntimeVisibleAnnotations");
}
if(ianns != null)
{
++attributeCount;
size += 8 + ianns.getSize();
newUTF8("RuntimeInvisibleAnnotations");
}
if(attrs != null)
7.8. CLASSWRITER.JAVA 249

{
attributeCount += attrs.getCount();
size += attrs.getSize(this, null, 0, -1, -1);
}
size += pool.length;
// allocates a byte vector of this size, in order to
// avoid unnecessary arraycopy operations in the
// ByteVector.enlarge() method
ByteVector out = new ByteVector(size);
out.putInt(0xCAFEBABE).putInt(version);
out.putShort(index).putByteArray(pool.data, 0, pool.length);
out.putShort(access).putShort(name).putShort(superName);
out.putShort(interfaceCount);
for(int i = 0; i < interfaceCount; ++i)
{
out.putShort(interfaces[i]);
}
out.putShort(nbFields);
fb = firstField;
while(fb != null)
{
fb.put(out);
fb = fb.next;
}
out.putShort(nbMethods);
mb = firstMethod;
while(mb != null)
{
mb.put(out);
mb = mb.next;
}
out.putShort(attributeCount);
if(signature != 0)
{
out.putShort(newUTF8("Signature"))
.putInt(2)
.putShort(signature);
}
if(sourceFile != 0)
{
out.putShort(newUTF8("SourceFile"))
.putInt(2)
.putShort(sourceFile);
}
if(sourceDebug != null)
{
int len = sourceDebug.length - 2;
out.putShort(newUTF8("SourceDebugExtension"))
.putInt(len);
out.putByteArray(sourceDebug.data, 2, len);
250 CHAPTER 7. JVM/CLOJURE/ASM/

}
if(enclosingMethodOwner != 0)
{
out.putShort(newUTF8("EnclosingMethod"))
.putInt(4);
out.putShort(enclosingMethodOwner)
.putShort(enclosingMethod);
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
out.putShort(newUTF8("Deprecated")).putInt(0);
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
&& (version & 0xffff) < Opcodes.V1_5)
{
out.putShort(newUTF8("Synthetic")).putInt(0);
}
if(innerClasses != null)
{
out.putShort(newUTF8("InnerClasses"));
out.putInt(innerClasses.length + 2)
.putShort(innerClassesCount);
out.putByteArray(innerClasses.data,
0,innerClasses.length);
}
if(anns != null)
{
out.putShort(newUTF8("RuntimeVisibleAnnotations"));
anns.put(out);
}
if(ianns != null)
{
out.putShort(newUTF8("RuntimeInvisibleAnnotations"));
ianns.put(out);
}
if(attrs != null)
{
attrs.put(this, null, 0, -1, -1, out);
}
if(invalidFrames)
{
ClassWriter cw = new ClassWriter(COMPUTE_FRAMES);
new ClassReader(out.data).accept(cw, ClassReader.SKIP_FRAMES);
return cw.toByteArray();
}
return out.data;
}

// -------------------------------------------------------------------
// Utility methods: constant pool management
7.8. CLASSWRITER.JAVA 251

// -------------------------------------------------------------------

/**
* Adds a number or string constant to the constant pool of the class
* being build. Does nothing if the constant pool already contains
* a similar item.
*
* @param cst the value of the constant to be added to the constant
* pool. This parameter must be an {@link Integer},
* a {@link Float}, a {@link Long}, a {@link Double},
* a {@link String} or a {@link Type}.
* @return a new or already existing constant item with the given value.
*/
Item newConstItem(final Object cst){
if(cst instanceof Integer)
{
int val = ((Integer) cst).intValue();
return newInteger(val);
}
else if(cst instanceof Byte)
{
int val = ((Byte) cst).intValue();
return newInteger(val);
}
else if(cst instanceof Character)
{
int val = ((Character) cst).charValue();
return newInteger(val);
}
else if(cst instanceof Short)
{
int val = ((Short) cst).intValue();
return newInteger(val);
}
else if(cst instanceof Boolean)
{
int val = ((Boolean) cst).booleanValue() ? 1 : 0;
return newInteger(val);
}
else if(cst instanceof Float)
{
float val = ((Float) cst).floatValue();
return newFloat(val);
}
else if(cst instanceof Long)
{
long val = ((Long) cst).longValue();
return newLong(val);
}
else if(cst instanceof Double)
252 CHAPTER 7. JVM/CLOJURE/ASM/

{
double val = ((Double) cst).doubleValue();
return newDouble(val);
}
else if(cst instanceof String)
{
return newString((String) cst);
}
else if(cst instanceof Type)
{
Type t = (Type) cst;
return newClassItem(t.getSort() == Type.OBJECT
? t.getInternalName()
: t.getDescriptor());
}
else
{
throw new IllegalArgumentException("value " + cst);
}
}

/**
* Adds a number or string constant to the constant pool of the class
* being build. Does nothing if the constant pool already contains a
* similar item. <i>This method is intended for {@link Attribute}
* sub classes, and is normally not needed by class generators or
* adapters.</i>
*
* @param cst the value of the constant to be added to the constant pool.
* This parameter must be an {@link Integer}, a {@link Float},
* a {@link Long}, a {@link Double} or a {@link String}.
* @return the index of a new or already existing constant item with the
* given value.
*/
public int newConst(final Object cst){
return newConstItem(cst).index;
}

/**
* Adds an UTF8 string to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
* <i>This method is intended for {@link Attribute} sub classes, and
* is normally not needed by class generators or adapters.</i>
*
* @param value the String value.
* @return the index of a new or already existing UTF8 item.
*/
public int newUTF8(final String value){
key.set(UTF8, value, null, null);
Item result = get(key);
7.8. CLASSWRITER.JAVA 253

if(result == null)
{
pool.putByte(UTF8).putUTF8(value);
result = new Item(index++, key);
put(result);
}
return result.index;
}

/**
* Adds a class reference to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
* <i>This method is intended for {@link Attribute} sub classes, and is
* normally not needed by class generators or adapters.</i>
*
* @param value the internal name of the class.
* @return a new or already existing class reference item.
*/
Item newClassItem(final String value){
key2.set(CLASS, value, null, null);
Item result = get(key2);
if(result == null)
{
pool.put12(CLASS, newUTF8(value));
result = new Item(index++, key2);
put(result);
}
return result;
}

/**
* Adds a class reference to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
* <i>This method is intended for {@link Attribute} sub classes, and is
* normally not needed by class generators or adapters.</i>
*
* @param value the internal name of the class.
* @return the index of a new or already existing class reference item.
*/
public int newClass(final String value){
return newClassItem(value).index;
}

/**
* Adds a field reference to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
*
* @param owner the internal name of the fields owner class.
* @param name the fields name.
* @param desc the fields descriptor.
254 CHAPTER 7. JVM/CLOJURE/ASM/

* @return a new or already existing field reference item.

*/
Item newFieldItem(final String owner,
final String name,
final String desc){
key3.set(FIELD, owner, name, desc);
Item result = get(key3);
if(result == null)
{
put122(FIELD, newClass(owner), newNameType(name, desc));
result = new Item(index++, key3);
put(result);
}
return result;
}

/**
* Adds a field reference to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
* <i>This method is intended for {@link Attribute} sub classes, and is
* normally not needed by class generators or adapters.</i>
*
* @param owner the internal name of the fields owner class.
* @param name the fields name.
* @param desc the fields descriptor.
* @return the index of a new or already existing field reference item.
*/
public int newField(final String owner,
final String name,
final String desc){
return newFieldItem(owner, name, desc).index;
}

/**
* Adds a method reference to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
*
* @param owner the internal name of the methods owner class.
* @param name the methods name.
* @param desc the methods descriptor.
* @param itf <tt>true</tt> if <tt>owner</tt> is an interface.
* @return a new or already existing method reference item.
*/
Item newMethodItem(
final String owner,
final String name,
final String desc,
final boolean itf){
int type = itf ? IMETH : METH;
key3.set(type, owner, name, desc);
7.8. CLASSWRITER.JAVA 255

Item result = get(key3);

if(result == null)
{
put122(type, newClass(owner), newNameType(name, desc));
result = new Item(index++, key3);
put(result);
}
return result;
}

/**
* Adds a method reference to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
* <i>This method is intended for {@link Attribute} sub classes, and is
* normally not needed by class generators or adapters.</i>
*
* @param owner the internal name of the methods owner class.
* @param name the methods name.
* @param desc the methods descriptor.
* @param itf <tt>true</tt> if <tt>owner</tt> is an interface.
* @return the index of a new or already existing method reference item.
*/
public int newMethod(
final String owner,
final String name,
final String desc,
final boolean itf){
return newMethodItem(owner, name, desc, itf).index;
}

/**
* Adds an integer to the constant pool of the class being build. Does
* nothing if the constant pool already contains a similar item.
*
* @param value the int value.
* @return a new or already existing int item.
*/
Item newInteger(final int value){
key.set(value);
Item result = get(key);
if(result == null)
{
pool.putByte(INT).putInt(value);
result = new Item(index++, key);
put(result);
}
return result;
}

/**
256 CHAPTER 7. JVM/CLOJURE/ASM/

* Adds a float to the constant pool of the class being build. Does
* nothing if the constant pool already contains a similar item.
*
* @param value the float value.
* @return a new or already existing float item.
*/
Item newFloat(final float value){
key.set(value);
Item result = get(key);
if(result == null)
{
pool.putByte(FLOAT).putInt(key.intVal);
result = new Item(index++, key);
put(result);
}
return result;
}

/**
* Adds a long to the constant pool of the class being build. Does
* nothing if the constant pool already contains a similar item.
*
* @param value the long value.
* @return a new or already existing long item.
*/
Item newLong(final long value){
key.set(value);
Item result = get(key);
if(result == null)
{
pool.putByte(LONG).putLong(value);
result = new Item(index, key);
put(result);
index += 2;
}
return result;
}

/**
* Adds a double to the constant pool of the class being build. Does
* nothing if the constant pool already contains a similar item.
*
* @param value the double value.
* @return a new or already existing double item.
*/
Item newDouble(final double value){
key.set(value);
Item result = get(key);
if(result == null)
{
7.8. CLASSWRITER.JAVA 257

pool.putByte(DOUBLE).putLong(key.longVal);
result = new Item(index, key);
put(result);
index += 2;
}
return result;
}

/**
* Adds a string to the constant pool of the class being build. Does
* nothing if the constant pool already contains a similar item.
*
* @param value the String value.
* @return a new or already existing string item.
*/
private Item newString(final String value){
key2.set(STR, value, null, null);
Item result = get(key2);
if(result == null)
{
pool.put12(STR, newUTF8(value));
result = new Item(index++, key2);
put(result);
}
return result;
}

/**
* Adds a name and type to the constant pool of the class being build.
* Does nothing if the constant pool already contains a similar item.
* <i>This method is intended for {@link Attribute} sub classes, and
* is normally not needed by class generators or adapters.</i>
*
* @param name a name.
* @param desc a type descriptor.
* @return the index of a new or already existing name and type item.
*/
public int newNameType(final String name, final String desc){
key2.set(NAME_TYPE, name, desc, null);
Item result = get(key2);
if(result == null)
{
put122(NAME_TYPE, newUTF8(name), newUTF8(desc));
result = new Item(index++, key2);
put(result);
}
return result.index;
}

/**
258 CHAPTER 7. JVM/CLOJURE/ASM/

* Adds the given internal name to {@link #typeTable} and returns its
* index. Does nothing if the type table already contains this internal
* name.
*
* @param type the internal name to be added to the type table.
* @return the index of this internal name in the type table.
*/
int addType(final String type){
key.set(TYPE_NORMAL, type, null, null);
Item result = get(key);
if(result == null)
{
result = addType(key);
}
return result.index;
}

/**
* Adds the given "uninitialized" type to {@link #typeTable} and returns
* its index. This method is used for UNINITIALIZED types, made of an
* internal name and a bytecode offset.
*
* @param type the internal name to be added to the type table.
* @param offset the bytecode offset of the NEW instruction that created
* this UNINITIALIZED type value.
* @return the index of this internal name in the type table.
*/
int addUninitializedType(final String type, final int offset){
key.type = TYPE_UNINIT;
key.intVal = offset;
key.strVal1 = type;
key.hashCode =
0x7FFFFFFF & (TYPE_UNINIT + type.hashCode() + offset);
Item result = get(key);
if(result == null)
{
result = addType(key);
}
return result.index;
}

/**
* Adds the given Item to {@link #typeTable}.
*
* @param item the value to be added to the type table.
* @return the added Item, which a new Item instance with the same
* value as the given Item.
*/
private Item addType(final Item item){
++typeCount;
7.8. CLASSWRITER.JAVA 259

Item result = new Item(typeCount, key);

put(result);
if(typeTable == null)
{
typeTable = new Item[16];
}
if(typeCount == typeTable.length)
{
Item[] newTable = new Item[2 * typeTable.length];
System.arraycopy(typeTable, 0,
newTable, 0, typeTable.length);
typeTable = newTable;
}
typeTable[typeCount] = result;
return result;
}

/**
* Returns the index of the common super type of the two given types.
* This method calls {@link #getCommonSuperClass} and caches the result
* in the {@link #items} hash table to speedup future calls with the
* same parameters.
*
* @param type1 index of an internal name in {@link #typeTable}.
* @param type2 index of an internal name in {@link #typeTable}.
* @return the index of the common super type of the two given types.
*/
int getMergedType(final int type1, final int type2){
key2.type = TYPE_MERGED;
key2.longVal = type1 | (((long) type2) << 32);
key2.hashCode = 0x7FFFFFFF & (TYPE_MERGED + type1 + type2);
Item result = get(key2);
if(result == null)
{
String t = typeTable[type1].strVal1;
String u = typeTable[type2].strVal1;
key2.intVal = addType(getCommonSuperClass(t, u));
result = new Item((short) 0, key2);
put(result);
}
return result.intVal;
}

/**
* Returns the common super type of the two given types. The default
* implementation of this method <i>loads<i> the two given classes
* and uses the java.lang.Class methods to find the common super class.
* It can be overriden to compute this common super type in other ways,
* in particular without actually loading any class, or to take into
* account the class that is currently being generated by this
260 CHAPTER 7. JVM/CLOJURE/ASM/

* ClassWriter, which can of course not be loaded since it is under

* construction.
*
* @param type1 the internal name of a class.
* @param type2 the internal name of another class.
* @return the internal name of the common super class of the two given
* classes.
*/
protected String getCommonSuperClass(final String type1,
final String type2){
Class c, d;
try
{
c = Class.forName(type1.replace(/, .));
d = Class.forName(type2.replace(/, .));
}
catch(ClassNotFoundException e)
{
throw new RuntimeException(e);
}
if(c.isAssignableFrom(d))
{
return type1;
}
if(d.isAssignableFrom(c))
{
return type2;
}
if(c.isInterface() || d.isInterface())
{
return "java/lang/Object";
}
else
{
do
{
c = c.getSuperclass();
} while(!c.isAssignableFrom(d));
return c.getName().replace(., /);
}
}

/**
* Returns the constant pools hash table item which is equal to the
* given item.
*
* @param key a constant pool item.
* @return the constant pools hash table item which is equal to the
* given item, or <tt>null</tt> if there is no such item.
*/
7.8. CLASSWRITER.JAVA 261

private Item get(final Item key){

Item i = items[key.hashCode % items.length];
while(i != null && !key.isEqualTo(i))
{
i = i.next;
}
return i;
}

/**
* Puts the given item in the constant pools hash table. The hash
* table <i>must</i> not already contains this item.
*
* @param i the item to be added to the constant pools hash table.
*/
private void put(final Item i){
if(index > threshold)
{
int ll = items.length;
int nl = ll * 2 + 1;
Item[] newItems = new Item[nl];
for(int l = ll - 1; l >= 0; --l)
{
Item j = items[l];
while(j != null)
{
int index = j.hashCode % newItems.length;
Item k = j.next;
j.next = newItems[index];
newItems[index] = j;
j = k;
}
}
items = newItems;
threshold = (int) (nl * 0.75);
}
int index = i.hashCode % items.length;
i.next = items[index];
items[index] = i;
}

/**
* Puts one byte and two shorts into the constant pool.
*
* @param b a byte.
* @param s1 a short.
* @param s2 another short.
*/
private void put122(final int b, final int s1, final int s2){
pool.put12(b, s1).putShort(s2);
262 CHAPTER 7. JVM/CLOJURE/ASM/

}
}

7.9 Edge.java

Edge.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* An edge in the control flow graph of a method body. See
* {@link Label Label}.
*
* @author Eric Bruneton
*/
class Edge{

/**
* Denotes a normal control flow graph edge.
*/
final static int NORMAL = 0;

/**
* Denotes a control flow graph edge corresponding to an exception
* handler. More precisely any {@link Edge} whose {@link #info} is
* strictly positive corresponds to an exception handler. The actual
* value of {@link #info} is the index, in the {@link ClassWriter}
* type table, of the exception that is catched.
*/
final static int EXCEPTION = 0x7FFFFFFF;

/**
* Information about this control flow graph edge. If
* {@link ClassWriter#COMPUTE_MAXS} is used this field is the
* (relative) stack size in the basic block from which this edge
* originates. This size is equal to the stack size at the "jump"
* instruction to which this edge corresponds, relatively to the
* stack size at the beginning of the originating basic block.
* If {@link ClassWriter#COMPUTE_FRAMES} is used, this field is
* the kind of this control flow graph edge (i.e. NORMAL or EXCEPTION).
*/
int info;

/**
7.10. FIELDVISITOR.JAVA 263

* The successor block of the basic block from which this edge
* originates.
*/
Label successor;

/**
* The next edge in the list of successors of the originating basic
* block. See {@link Label#successors successors}.
*/
Edge next;
}

7.10 FieldVisitor.java

FieldVisitor.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* A visitor to visit a Java field. The methods of this interface
* must be called in the following order: ( <tt>visitAnnotation</tt> |
* <tt>visitAttribute</tt> )* <tt>visitEnd</tt>.
*
* @author Eric Bruneton
*/
public interface FieldVisitor{

/**
* Visits an annotation of the field.
*
* @param desc the class descriptor of the annotation class.
* @param visible <tt>true</tt> if the annotation is visible at runtime.
* @return a visitor to visit the annotation values, or <tt>null</tt> if
* this visitor is not interested in visiting this annotation.
*/
AnnotationVisitor visitAnnotation(String desc, boolean visible);

/**
* Visits a non standard attribute of the field.
*
* @param attr an attribute.
*/
void visitAttribute(Attribute attr);
264 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Visits the end of the field. This method, which is the last one to be
* called, is used to inform the visitor that all the annotations and
* attributes of the field have been visited.
*/
void visitEnd();
}

7.11 FieldWriter.java
(FieldVisitor [263])
FieldWriter.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* An {@link FieldVisitor} that generates Java fields in bytecode form.
*
* @author Eric Bruneton
*/
final class FieldWriter implements FieldVisitor{

/**
* Next field writer (see {@link ClassWriter#firstField firstField}).
*/
FieldWriter next;

/**
* The class writer to which this field must be added.
*/
private ClassWriter cw;

/**
* Access flags of this field.
*/
private int access;

/**
* The index of the constant pool item that contains the name of this
* method.
*/
private int name;

/**
* The index of the constant pool item that contains the descriptor of
7.11. FIELDWRITER.JAVA 265

* this field.
*/
private int desc;

/**
* The index of the constant pool item that contains the signature of
* this field.
*/
private int signature;

/**
* The index of the constant pool item that contains the constant value
* of this field.
*/
private int value;

/**
* The runtime visible annotations of this field. May be <tt>null</tt>.
*/
private AnnotationWriter anns;

/**
* The runtime invisible annotations of this field. May be <tt>null</tt>.
*/
private AnnotationWriter ianns;

/**
* The non standard attributes of this field. May be <tt>null</tt>.
*/
private Attribute attrs;

// -------------------------------------------------------------------
// Constructor
// -------------------------------------------------------------------

/**
* Constructs a new {@link FieldWriter}.
*
* @param cw the class writer to which this field must be added.
* @param access the fields access flags (see {@link Opcodes}).
* @param name the fields name.
* @param desc the fields descriptor (see {@link Type}).
* @param signature the fields signature. May be <tt>null</tt>.
* @param value the fields constant value. May be <tt>null</tt>.
*/
protected FieldWriter(
final ClassWriter cw,
final int access,
final String name,
final String desc,
266 CHAPTER 7. JVM/CLOJURE/ASM/

final String signature,

final Object value){
if(cw.firstField == null)
{
cw.firstField = this;
}
else
{
cw.lastField.next = this;
}
cw.lastField = this;
this.cw = cw;
this.access = access;
this.name = cw.newUTF8(name);
this.desc = cw.newUTF8(desc);
if(signature != null)
{
this.signature = cw.newUTF8(signature);
}
if(value != null)
{
this.value = cw.newConstItem(value).index;
}
}

// -------------------------------------------------------------------
// Implementation of the FieldVisitor interface
// -------------------------------------------------------------------

public AnnotationVisitor visitAnnotation(

final String desc,
final boolean visible){
ByteVector bv = new ByteVector();
// write type, and reserve space for values count
bv.putShort(cw.newUTF8(desc)).putShort(0);
AnnotationWriter aw = new AnnotationWriter(cw, true, bv, bv, 2);
if(visible)
{
aw.next = anns;
anns = aw;
}
else
{
aw.next = ianns;
ianns = aw;
}
return aw;
}

public void visitAttribute(final Attribute attr){

7.11. FIELDWRITER.JAVA 267

attr.next = attrs;
attrs = attr;
}

public void visitEnd(){

}

// -------------------------------------------------------------------
// Utility methods
// -------------------------------------------------------------------

/**
* Returns the size of this field.
*
* @return the size of this field.
*/
int getSize(){
int size = 8;
if(value != 0)
{
cw.newUTF8("ConstantValue");
size += 8;
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
&& (cw.version & 0xffff) < Opcodes.V1_5)
{
cw.newUTF8("Synthetic");
size += 6;
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
cw.newUTF8("Deprecated");
size += 6;
}
if(signature != 0)
{
cw.newUTF8("Signature");
size += 8;
}
if(anns != null)
{
cw.newUTF8("RuntimeVisibleAnnotations");
size += 8 + anns.getSize();
}
if(ianns != null)
{
cw.newUTF8("RuntimeInvisibleAnnotations");
size += 8 + ianns.getSize();
}
if(attrs != null)
268 CHAPTER 7. JVM/CLOJURE/ASM/

{
size += attrs.getSize(cw, null, 0, -1, -1);
}
return size;
}

/**
* Puts the content of this field into the given byte vector.
*
* @param out where the content of this field must be put.
*/
void put(final ByteVector out){
out.putShort(access).putShort(name).putShort(desc);
int attributeCount = 0;
if(value != 0)
{
++attributeCount;
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
&& (cw.version & 0xffff) < Opcodes.V1_5)
{
++attributeCount;
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
++attributeCount;
}
if(signature != 0)
{
++attributeCount;
}
if(anns != null)
{
++attributeCount;
}
if(ianns != null)
{
++attributeCount;
}
if(attrs != null)
{
attributeCount += attrs.getCount();
}
out.putShort(attributeCount);
if(value != 0)
{
out.putShort(cw.newUTF8("ConstantValue"));
out.putInt(2).putShort(value);
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
7.12. FRAME.JAVA 269

&& (cw.version & 0xffff) < Opcodes.V1_5)

{
out.putShort(cw.newUTF8("Synthetic")).putInt(0);
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
out.putShort(cw.newUTF8("Deprecated")).putInt(0);
}
if(signature != 0)
{
out.putShort(cw.newUTF8("Signature"));
out.putInt(2).putShort(signature);
}
if(anns != null)
{
out.putShort(cw.newUTF8("RuntimeVisibleAnnotations"));
anns.put(out);
}
if(ianns != null)
{
out.putShort(cw.newUTF8("RuntimeInvisibleAnnotations"));
ianns.put(out);
}
if(attrs != null)
{
attrs.put(cw, null, 0, -1, -1, out);
}
}
}

7.12 Frame.java

Frame.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* Information about the input and output stack map frames of a basic
* block.
*
* @author Eric Bruneton
*/
final class Frame{
270 CHAPTER 7. JVM/CLOJURE/ASM/

/*
* Frames are computed in a two steps process: during the visit
* of each instruction, the state of the frame at the end of
* current basic block is updated by simulating the action of
* the instruction on the previous state of this so called
* "output frame". In visitMaxs, a fix point algorithm is used
* to compute the "input frame" of each basic block, i.e. the
* stack mapframe at the begining of the basic block, starting
* from the input frameof the first basic block (which is
* computed from the method descriptor),and by using the
* previously computed output frames to compute the input
* state of the other blocks.
*
* All output and input frames are stored as arrays of
* integers. Reference and array types are represented by an
* index into a type table (which is not the same as the constant
* pool of the class, in order to avoid adding unnecessary
* constants in the pool - not all computed frames will end up
* being stored in the stack map table). This allows very fast
* type comparisons.
*
* Output stack map frames are computed relatively to the input
* frame of the basic block, which is not yet known when output
* frames are computed. It is therefore necessary to be able to
* represent abstract types such as "the type at position x in
* the input frame locals" or "the type at position x from the
* top of the input frame stack" or even "the type at position
* x in the input frame, with y more (or less) array dimensions".
* This explains the rather complicated type format used in
* output frames.
*
* This format is the following: DIM KIND VALUE (4, 4 and 24
* bits). DIM is a signed number of array dimensions (from -8 to
* 7). KIND is either BASE, LOCAL or STACK. BASE is used for
* types that are not relative to the input frame. LOCAL is used
* for types that are relative to the input local variable types.
* STACK is used for types that are relative to the input stack
* types. VALUE depends on KIND. For LOCAL types, it is an index
* in the input local variable types. For STACK types, it is a
* position relatively to the top of input frame stack. For BASE
* types, it is either one of the constants defined in
* FrameVisitor, or for OBJECT and UNINITIALIZED types, a tag
* and an index in the type table.
*
* Output frames can contain types of any kind and with a
* positive or negative dimension (and even unassigned types,
* represented by 0 - which does not correspond to any valid
* type value). Input frames can only contain BASE types of
* positive or null dimension. In all cases the type table
* contains only internal type names (array type descriptors
7.12. FRAME.JAVA 271

* are forbidden - dimensions must be represented through the

* DIM field).
*
* The LONG and DOUBLE types are always represented by using
* two slots (LONG + TOP or DOUBLE + TOP), for local variable
* types as well as in the operand stack. This is necessary to
* be able to simulate DUPx_y instructions, whose effect would
* be dependent on the actual type values if types were always
* represented by a single slot in the stack (and this is not
* possible, since actual type values are not always known -
* cf LOCAL and STACK type kinds).
*/

/**
* Mask to get the dimension of a frame type. This dimension is a signed
* integer between -8 and 7.
*/
final static int DIM = 0xF0000000;

/**
* Constant to be added to a type to get a type with one more dimension.
*/
final static int ARRAY_OF = 0x10000000;

/**
* Constant to be added to a type to get a type with one less dimension.
*/
final static int ELEMENT_OF = 0xF0000000;

/**
* Mask to get the kind of a frame type.
*
* @see #BASE
* @see #LOCAL
* @see #STACK
*/
final static int KIND = 0xF000000;

/**
* Mask to get the value of a frame type.
*/
final static int VALUE = 0xFFFFFF;

/**
* Mask to get the kind of base types.
*/
final static int BASE_KIND = 0xFF00000;

/**
* Mask to get the value of base types.
272 CHAPTER 7. JVM/CLOJURE/ASM/

*/
final static int BASE_VALUE = 0xFFFFF;

/**
* Kind of the types that are not relative to an input stack map frame.
*/
final static int BASE = 0x1000000;

/**
* Base kind of the base reference types. The BASE_VALUE of such types
* is an index into the type table.
*/
final static int OBJECT = BASE | 0x700000;

/**
* Base kind of the uninitialized base types. The BASE_VALUE of such
* types in an index into the type table (the Item at that index
* contains both an instruction offset and an internal class name).
*/
final static int UNINITIALIZED = BASE | 0x800000;

/**
* Kind of the types that are relative to the local variable types of
* an input stack map frame. The value of such types is a local
* variable index.
*/
private final static int LOCAL = 0x2000000;

/**
* Kind of the the types that are relative to the stack of an input
* stack map frame. The value of such types is a position relatively
* to the top of this stack.
*/
private final static int STACK = 0x3000000;

/**
* The TOP type. This is a BASE type.
*/
final static int TOP = BASE | 0;

/**
* The BOOLEAN type. This is a BASE type mainly used for array types.
*/
final static int BOOLEAN = BASE | 9;

/**
* The BYTE type. This is a BASE type mainly used for array types.
*/
final static int BYTE = BASE | 10;
7.12. FRAME.JAVA 273

/**
* The CHAR type. This is a BASE type mainly used for array types.
*/
final static int CHAR = BASE | 11;

/**
* The SHORT type. This is a BASE type mainly used for array types.
*/
final static int SHORT = BASE | 12;

/**
* The INTEGER type. This is a BASE type.
*/
final static int INTEGER = BASE | 1;

/**
* The FLOAT type. This is a BASE type.
*/
final static int FLOAT = BASE | 2;

/**
* The DOUBLE type. This is a BASE type.
*/
final static int DOUBLE = BASE | 3;

/**
* The LONG type. This is a BASE type.
*/
final static int LONG = BASE | 4;

/**
* The NULL type. This is a BASE type.
*/
final static int NULL = BASE | 5;

/**
* The UNINITIALIZED_THIS type. This is a BASE type.
*/
final static int UNINITIALIZED_THIS = BASE | 6;

/**
* The stack size variation corresponding to each JVM instruction.
* This stack variation is equal to the size of the values produced
* by an instruction, minus the size of the values consumed by this
* instruction.
*/
final static int[] SIZE;

/**
* Computes the stack size variation corresponding to each JVM
274 CHAPTER 7. JVM/CLOJURE/ASM/

* instruction.
*/
static
{
int i;
int[] b = new int[202];
String s =
"EFFFFFFFFGGFFFGGFFFEEFGFGFEEEEEEEEEEEEEEEEEEEEDEDEDDDDD"
+ "CDCDEEEEEEEEEEEEEEEEEEEEBABABBBBDCFFFGGGEDCDCDCDCDCDCDCDCD"
+ "CDCEEEEDDDDDDDCDCDCEFEFDDEEFFDEDEEEBDDBBDDDDDDCCCCCCCCEFED"
+ "DDCDCDEEEEEEEEEEFEEEEEEDDEEDDEE";
for(i = 0; i < b.length; ++i)
{
b[i] = s.charAt(i) - E;
}
SIZE = b;

// code to generate the above string

//
// int NA = 0; // not applicable (unused opcode or variable
// size opcode)
//
// b = new int[] {
// 0, //NOP, // visitInsn
// 1, //ACONST_NULL, // -
// 1, //ICONST_M1, // -
// 1, //ICONST_0, // -
// 1, //ICONST_1, // -
// 1, //ICONST_2, // -
// 1, //ICONST_3, // -
// 1, //ICONST_4, // -
// 1, //ICONST_5, // -
// 2, //LCONST_0, // -
// 2, //LCONST_1, // -
// 1, //FCONST_0, // -
// 1, //FCONST_1, // -
// 1, //FCONST_2, // -
// 2, //DCONST_0, // -
// 2, //DCONST_1, // -
// 1, //BIPUSH, // visitIntInsn
// 1, //SIPUSH, // -
// 1, //LDC, // visitLdcInsn
// NA, //LDC_W, // -
// NA, //LDC2_W, // -
// 1, //ILOAD, // visitVarInsn
// 2, //LLOAD, // -
// 1, //FLOAD, // -
// 2, //DLOAD, // -
// 1, //ALOAD, // -
// NA, //ILOAD_0, // -
7.12. FRAME.JAVA 275

// NA, //ILOAD_1, // -
// NA, //ILOAD_2, // -
// NA, //ILOAD_3, // -
// NA, //LLOAD_0, // -
// NA, //LLOAD_1, // -
// NA, //LLOAD_2, // -
// NA, //LLOAD_3, // -
// NA, //FLOAD_0, // -
// NA, //FLOAD_1, // -
// NA, //FLOAD_2, // -
// NA, //FLOAD_3, // -
// NA, //DLOAD_0, // -
// NA, //DLOAD_1, // -
// NA, //DLOAD_2, // -
// NA, //DLOAD_3, // -
// NA, //ALOAD_0, // -
// NA, //ALOAD_1, // -
// NA, //ALOAD_2, // -
// NA, //ALOAD_3, // -
// -1, //IALOAD, // visitInsn
// 0, //LALOAD, // -
// -1, //FALOAD, // -
// 0, //DALOAD, // -
// -1, //AALOAD, // -
// -1, //BALOAD, // -
// -1, //CALOAD, // -
// -1, //SALOAD, // -
// -1, //ISTORE, // visitVarInsn
// -2, //LSTORE, // -
// -1, //FSTORE, // -
// -2, //DSTORE, // -
// -1, //ASTORE, // -
// NA, //ISTORE_0, // -
// NA, //ISTORE_1, // -
// NA, //ISTORE_2, // -
// NA, //ISTORE_3, // -
// NA, //LSTORE_0, // -
// NA, //LSTORE_1, // -
// NA, //LSTORE_2, // -
// NA, //LSTORE_3, // -
// NA, //FSTORE_0, // -
// NA, //FSTORE_1, // -
// NA, //FSTORE_2, // -
// NA, //FSTORE_3, // -
// NA, //DSTORE_0, // -
// NA, //DSTORE_1, // -
// NA, //DSTORE_2, // -
// NA, //DSTORE_3, // -
// NA, //ASTORE_0, // -
// NA, //ASTORE_1, // -
276 CHAPTER 7. JVM/CLOJURE/ASM/

// NA, //ASTORE_2, // -
// NA, //ASTORE_3, // -
// -3, //IASTORE, // visitInsn
// -4, //LASTORE, // -
// -3, //FASTORE, // -
// -4, //DASTORE, // -
// -3, //AASTORE, // -
// -3, //BASTORE, // -
// -3, //CASTORE, // -
// -3, //SASTORE, // -
// -1, //POP, // -
// -2, //POP2, // -
// 1, //DUP, // -
// 1, //DUP_X1, // -
// 1, //DUP_X2, // -
// 2, //DUP2, // -
// 2, //DUP2_X1, // -
// 2, //DUP2_X2, // -
// 0, //SWAP, // -
// -1, //IADD, // -
// -2, //LADD, // -
// -1, //FADD, // -
// -2, //DADD, // -
// -1, //ISUB, // -
// -2, //LSUB, // -
// -1, //FSUB, // -
// -2, //DSUB, // -
// -1, //IMUL, // -
// -2, //LMUL, // -
// -1, //FMUL, // -
// -2, //DMUL, // -
// -1, //IDIV, // -
// -2, //LDIV, // -
// -1, //FDIV, // -
// -2, //DDIV, // -
// -1, //IREM, // -
// -2, //LREM, // -
// -1, //FREM, // -
// -2, //DREM, // -
// 0, //INEG, // -
// 0, //LNEG, // -
// 0, //FNEG, // -
// 0, //DNEG, // -
// -1, //ISHL, // -
// -1, //LSHL, // -
// -1, //ISHR, // -
// -1, //LSHR, // -
// -1, //IUSHR, // -
// -1, //LUSHR, // -
// -1, //IAND, // -
7.12. FRAME.JAVA 277

// -2, //LAND, // -
// -1, //IOR, // -
// -2, //LOR, // -
// -1, //IXOR, // -
// -2, //LXOR, // -
// 0, //IINC, // visitIincInsn
// 1, //I2L, // visitInsn
// 0, //I2F, // -
// 1, //I2D, // -
// -1, //L2I, // -
// -1, //L2F, // -
// 0, //L2D, // -
// 0, //F2I, // -
// 1, //F2L, // -
// 1, //F2D, // -
// -1, //D2I, // -
// 0, //D2L, // -
// -1, //D2F, // -
// 0, //I2B, // -
// 0, //I2C, // -
// 0, //I2S, // -
// -3, //LCMP, // -
// -1, //FCMPL, // -
// -1, //FCMPG, // -
// -3, //DCMPL, // -
// -3, //DCMPG, // -
// -1, //IFEQ, // visitJumpInsn
// -1, //IFNE, // -
// -1, //IFLT, // -
// -1, //IFGE, // -
// -1, //IFGT, // -
// -1, //IFLE, // -
// -2, //IF_ICMPEQ, // -
// -2, //IF_ICMPNE, // -
// -2, //IF_ICMPLT, // -
// -2, //IF_ICMPGE, // -
// -2, //IF_ICMPGT, // -
// -2, //IF_ICMPLE, // -
// -2, //IF_ACMPEQ, // -
// -2, //IF_ACMPNE, // -
// 0, //GOTO, // -
// 1, //JSR, // -
// 0, //RET, // visitVarInsn
// -1, //TABLESWITCH, // visiTableSwitchInsn
// -1, //LOOKUPSWITCH, // visitLookupSwitch
// -1, //IRETURN, // visitInsn
// -2, //LRETURN, // -
// -1, //FRETURN, // -
// -2, //DRETURN, // -
// -1, //ARETURN, // -
278 CHAPTER 7. JVM/CLOJURE/ASM/

// 0, //RETURN, // -
// NA, //GETSTATIC, // visitFieldInsn
// NA, //PUTSTATIC, // -
// NA, //GETFIELD, // -
// NA, //PUTFIELD, // -
// NA, //INVOKEVIRTUAL, // visitMethodInsn
// NA, //INVOKESPECIAL, // -
// NA, //INVOKESTATIC, // -
// NA, //INVOKEINTERFACE, // -
// NA, //UNUSED, // NOT VISITED
// 1, //NEW, // visitTypeInsn
// 0, //NEWARRAY, // visitIntInsn
// 0, //ANEWARRAY, // visitTypeInsn
// 0, //ARRAYLENGTH, // visitInsn
// NA, //ATHROW, // -
// 0, //CHECKCAST, // visitTypeInsn
// 0, //INSTANCEOF, // -
// -1, //MONITORENTER, // visitInsn
// -1, //MONITOREXIT, // -
// NA, //WIDE, // NOT VISITED
// NA, //MULTIANEWARRAY, // visitMultiANewArrayInsn
// -1, //IFNULL, // visitJumpInsn
// -1, //IFNONNULL, // -
// NA, //GOTO_W, // -
// NA, //JSR_W, // -
// };
// for (i = 0; i < b.length; ++i) {
// System.err.print((char)(E + b[i]));
// }
// System.err.println();
}

/**
* The label (i.e. basic block) to which these input and output stack map
* frames correspond.
*/
Label owner;

/**
* The input stack map frame locals.
*/
int[] inputLocals;

/**
* The input stack map frame stack.
*/
int[] inputStack;

/**
* The output stack map frame locals.
7.12. FRAME.JAVA 279

*/
private int[] outputLocals;

/**
* The output stack map frame stack.
*/
private int[] outputStack;

/**
* Relative size of the output stack. The exact semantics of this field
* depends on the algorithm that is used.
* <p/>
* When only the maximum stack size is computed, this field is the size
* of the output stack relatively to the top of the input stack.
* <p/>
* When the stack map frames are completely computed, this field is the
* actual number of types in {@link #outputStack}.
*/
private int outputStackTop;

/**
* Number of types that are initialized in the basic block.
*
* @see #initializations
*/
private int initializationCount;

/**
* The types that are initialized in the basic block. A constructor
* invocation on an UNINITIALIZED or UNINITIALIZED_THIS type must
* replace <i>every occurence</i> of this type in the local variables
* and in the operand stack. This cannot be done during the first
* phase of the algorithm since, during this phase, the local variables
* and the operand stack are not completely computed. It is therefore
* necessary to store the types on which constructors are invoked in
* the basic block, in order to do this replacement during the second
* phase of the algorithm, where the frames are fully computed. Note
* that this array can contain types that are relative to input locals
* or to the input stack (see below for the description of the
* algorithm).
*/
private int[] initializations;

/**
* Returns the output frame local variable type at the given index.
*
* @param local the index of the local that must be returned.
* @return the output frame local variable type at the given index.
*/
private int get(final int local){
280 CHAPTER 7. JVM/CLOJURE/ASM/

if(outputLocals == null || local >= outputLocals.length)

{
// this local has never been assigned in this basic block,
// so it is still equal to its value in the input frame
return LOCAL | local;
}
else
{
int type = outputLocals[local];
if(type == 0)
{
// this local has never been assigned in this
// basic block, so it is still equal to its value
// in the input frame
type = outputLocals[local] = LOCAL | local;
}
return type;
}
}

/**
* Sets the output frame local variable type at the given index.
*
* @param local the index of the local that must be set.
* @param type the value of the local that must be set.
*/
private void set(final int local, final int type){
// creates and/or resizes the output local variables
// array if necessary
if(outputLocals == null)
{
outputLocals = new int[10];
}
int n = outputLocals.length;
if(local >= n)
{
int[] t = new int[Math.max(local + 1, 2 * n)];
System.arraycopy(outputLocals, 0, t, 0, n);
outputLocals = t;
}
// sets the local variable
outputLocals[local] = type;
}

/**
* Pushes a new type onto the output frame stack.
*
* @param type the type that must be pushed.
*/
private void push(final int type){
7.12. FRAME.JAVA 281

// creates and/or resizes the output stack array if necessary

if(outputStack == null)
{
outputStack = new int[10];
}
int n = outputStack.length;
if(outputStackTop >= n)
{
int[] t = new int[Math.max(outputStackTop + 1, 2 * n)];
System.arraycopy(outputStack, 0, t, 0, n);
outputStack = t;
}
// pushes the type on the output stack
outputStack[outputStackTop++] = type;
// updates the maximun height reached by the output stack, if needed
int top = owner.inputStackTop + outputStackTop;
if(top > owner.outputStackMax)
{
owner.outputStackMax = top;
}
}

/**
* Pushes a new type onto the output frame stack.
*
* @param cw the ClassWriter to which this label belongs.
* @param desc the descriptor of the type to be pushed. Can also be a
* method descriptor (in this case this method pushes its
* return type onto the output frame stack).
*/
private void push(final ClassWriter cw, final String desc){
int type = type(cw, desc);
if(type != 0)
{
push(type);
if(type == LONG || type == DOUBLE)
{
push(TOP);
}
}
}

/**
* Returns the int encoding of the given type.
*
* @param cw the ClassWriter to which this label belongs.
* @param desc a type descriptor.
* @return the int encoding of the given type.
*/
private int type(final ClassWriter cw, final String desc){
282 CHAPTER 7. JVM/CLOJURE/ASM/

String t;
int index = desc.charAt(0) == ( ? desc.indexOf()) + 1 : 0;
switch(desc.charAt(index))
{
caseV:
return 0;
caseZ:
caseC:
caseB:
caseS:
caseI:
return INTEGER;
caseF:
return FLOAT;
caseJ:
return LONG;
caseD:
return DOUBLE;
caseL:
// stores the internal name, not the descriptor!
t = desc.substring(index + 1, desc.length() - 1);
return OBJECT | cw.addType(t);
// case [:
default:
// extracts the dimensions and the element type
int data;
int dims = index + 1;
while(desc.charAt(dims) == [)
{
++dims;
}
switch(desc.charAt(dims))
{
caseZ:
data = BOOLEAN;
break;
caseC:
data = CHAR;
break;
caseB:
data = BYTE;
break;
caseS:
data = SHORT;
break;
caseI:
data = INTEGER;
break;
caseF:
data = FLOAT;
7.12. FRAME.JAVA 283

break;
caseJ:
data = LONG;
break;
caseD:
data = DOUBLE;
break;
// case L:
default:
// stores the internal name, not the descriptor
t = desc.substring(dims + 1, desc.length() - 1);
data = OBJECT | cw.addType(t);
}
return (dims - index) << 28 | data;
}
}

/**
* Pops a type from the output frame stack and returns its value.
*
* @return the type that has been popped from the output frame stack.
*/
private int pop(){
if(outputStackTop > 0)
{
return outputStack[--outputStackTop];
}
else
{
// if the output frame stack is empty, pops from the input stack
return STACK | -(--owner.inputStackTop);
}
}

/**
* Pops the given number of types from the output frame stack.
*
* @param elements the number of types that must be popped.
*/
private void pop(final int elements){
if(outputStackTop >= elements)
{
outputStackTop -= elements;
}
else
{
// if the number of elements to be popped is greater than the
// number of elements in the output stack, clear it, and pops
// the remaining elements from the input stack.
owner.inputStackTop -= elements - outputStackTop;
284 CHAPTER 7. JVM/CLOJURE/ASM/

outputStackTop = 0;
}
}

/**
* Pops a type from the output frame stack.
*
* @param desc the descriptor of the type to be popped. Can also be a
* method descriptor (in this case this method pops the
* types corresponding to the method arguments).
*/
private void pop(final String desc){
char c = desc.charAt(0);
if(c == ()
{
pop((MethodWriter.getArgumentsAndReturnSizes(desc) >> 2) - 1);
}
else if(c == J || c == D)
{
pop(2);
}
else
{
pop(1);
}
}

/**
* Adds a new type to the list of types on which a constructor is
* invoked in the basic block.
*
* @param var a type on a which a constructor is invoked.
*/
private void init(final int var){
// creates and/or resizes the initializations array if necessary
if(initializations == null)
{
initializations = new int[2];
}
int n = initializations.length;
if(initializationCount >= n)
{
int[] t = new int[Math.max(initializationCount + 1, 2 * n)];
System.arraycopy(initializations, 0, t, 0, n);
initializations = t;
}
// stores the type to be initialized
initializations[initializationCount++] = var;
}
7.12. FRAME.JAVA 285

/**
* Replaces the given type with the appropriate type if it is one of
* the types on which a constructor is invoked in the basic block.
*
* @param cw the ClassWriter to which this label belongs.
* @param t a type
* @return t or, if t is one of the types on which a constructor is
* invoked in the basic block, the type corresponding to this
* constructor.
*/
private int init(final ClassWriter cw, final int t){
int s;
if(t == UNINITIALIZED_THIS)
{
s = OBJECT | cw.addType(cw.thisName);
}
else if((t & (DIM | BASE_KIND)) == UNINITIALIZED)
{
String type = cw.typeTable[t & BASE_VALUE].strVal1;
s = OBJECT | cw.addType(type);
}
else
{
return t;
}
for(int j = 0; j < initializationCount; ++j)
{
int u = initializations[j];
int dim = u & DIM;
int kind = u & KIND;
if(kind == LOCAL)
{
u = dim + inputLocals[u & VALUE];
}
else if(kind == STACK)
{
u = dim + inputStack[inputStack.length - (u & VALUE)];
}
if(t == u)
{
return s;
}
}
return t;
}

/**
* Initializes the input frame of the first basic block from the method
* descriptor.
*
286 CHAPTER 7. JVM/CLOJURE/ASM/

* @param cw the ClassWriter to which this label belongs.

* @param access the access flags of the method to which this
* label belongs.
* @param args the formal parameter types of this method.
* @param maxLocals the maximum number of local variables of this method.
*/
void initInputFrame(
final ClassWriter cw,
final int access,
final Type[] args,
final int maxLocals){
inputLocals = new int[maxLocals];
inputStack = new int[0];
int i = 0;
if((access & Opcodes.ACC_STATIC) == 0)
{
if((access & MethodWriter.ACC_CONSTRUCTOR) == 0)
{
inputLocals[i++] = OBJECT | cw.addType(cw.thisName);
}
else
{
inputLocals[i++] = UNINITIALIZED_THIS;
}
}
for(int j = 0; j < args.length; ++j)
{
int t = type(cw, args[j].getDescriptor());
inputLocals[i++] = t;
if(t == LONG || t == DOUBLE)
{
inputLocals[i++] = TOP;
}
}
while(i < maxLocals)
{
inputLocals[i++] = TOP;
}
}

/**
* Simulates the action of the given instruction on the output stack
* frame.
*
* @param opcode the opcode of the instruction.
* @param arg the operand of the instruction, if any.
* @param cw the class writer to which this label belongs.
* @param item the operand of the instructions, if any.
*/
void execute(
7.12. FRAME.JAVA 287

final int opcode,

final int arg,
final ClassWriter cw,
final Item item){
int t1, t2, t3, t4;
switch(opcode)
{
case Opcodes.NOP:
case Opcodes.INEG:
case Opcodes.LNEG:
case Opcodes.FNEG:
case Opcodes.DNEG:
case Opcodes.I2B:
case Opcodes.I2C:
case Opcodes.I2S:
case Opcodes.GOTO:
case Opcodes.RETURN:
break;
case Opcodes.ACONST_NULL:
push(NULL);
break;
case Opcodes.ICONST_M1:
case Opcodes.ICONST_0:
case Opcodes.ICONST_1:
case Opcodes.ICONST_2:
case Opcodes.ICONST_3:
case Opcodes.ICONST_4:
case Opcodes.ICONST_5:
case Opcodes.BIPUSH:
case Opcodes.SIPUSH:
case Opcodes.ILOAD:
push(INTEGER);
break;
case Opcodes.LCONST_0:
case Opcodes.LCONST_1:
case Opcodes.LLOAD:
push(LONG);
push(TOP);
break;
case Opcodes.FCONST_0:
case Opcodes.FCONST_1:
case Opcodes.FCONST_2:
case Opcodes.FLOAD:
push(FLOAT);
break;
case Opcodes.DCONST_0:
case Opcodes.DCONST_1:
case Opcodes.DLOAD:
push(DOUBLE);
push(TOP);
288 CHAPTER 7. JVM/CLOJURE/ASM/

break;
case Opcodes.LDC:
switch(item.type)
{
case ClassWriter.INT:
push(INTEGER);
break;
case ClassWriter.LONG:
push(LONG);
push(TOP);
break;
case ClassWriter.FLOAT:
push(FLOAT);
break;
case ClassWriter.DOUBLE:
push(DOUBLE);
push(TOP);
break;
case ClassWriter.CLASS:
push(OBJECT | cw.addType("java/lang/Class"));
break;
// case ClassWriter.STR:
default:
push(OBJECT | cw.addType("java/lang/String"));
}
break;
case Opcodes.ALOAD:
push(get(arg));
break;
case Opcodes.IALOAD:
case Opcodes.BALOAD:
case Opcodes.CALOAD:
case Opcodes.SALOAD:
pop(2);
push(INTEGER);
break;
case Opcodes.LALOAD:
case Opcodes.D2L:
pop(2);
push(LONG);
push(TOP);
break;
case Opcodes.FALOAD:
pop(2);
push(FLOAT);
break;
case Opcodes.DALOAD:
case Opcodes.L2D:
pop(2);
push(DOUBLE);
7.12. FRAME.JAVA 289

push(TOP);
break;
case Opcodes.AALOAD:
pop(1);
t1 = pop();
push(ELEMENT_OF + t1);
break;
case Opcodes.ISTORE:
case Opcodes.FSTORE:
case Opcodes.ASTORE:
t1 = pop();
set(arg, t1);
if(arg > 0)
{
t2 = get(arg - 1);
// if t2 is of kind STACK or LOCAL we cannot know
// its size!
if(t2 == LONG || t2 == DOUBLE)
{
set(arg - 1, TOP);
}
}
break;
case Opcodes.LSTORE:
case Opcodes.DSTORE:
pop(1);
t1 = pop();
set(arg, t1);
set(arg + 1, TOP);
if(arg > 0)
{
t2 = get(arg - 1);
// if t2 is of kind STACK or LOCAL we cannot know
// its size!
if(t2 == LONG || t2 == DOUBLE)
{
set(arg - 1, TOP);
}
}
break;
case Opcodes.IASTORE:
case Opcodes.BASTORE:
case Opcodes.CASTORE:
case Opcodes.SASTORE:
case Opcodes.FASTORE:
case Opcodes.AASTORE:
pop(3);
break;
case Opcodes.LASTORE:
case Opcodes.DASTORE:
290 CHAPTER 7. JVM/CLOJURE/ASM/

pop(4);
break;
case Opcodes.POP:
case Opcodes.IFEQ:
case Opcodes.IFNE:
case Opcodes.IFLT:
case Opcodes.IFGE:
case Opcodes.IFGT:
case Opcodes.IFLE:
case Opcodes.IRETURN:
case Opcodes.FRETURN:
case Opcodes.ARETURN:
case Opcodes.TABLESWITCH:
case Opcodes.LOOKUPSWITCH:
case Opcodes.ATHROW:
case Opcodes.MONITORENTER:
case Opcodes.MONITOREXIT:
case Opcodes.IFNULL:
case Opcodes.IFNONNULL:
pop(1);
break;
case Opcodes.POP2:
case Opcodes.IF_ICMPEQ:
case Opcodes.IF_ICMPNE:
case Opcodes.IF_ICMPLT:
case Opcodes.IF_ICMPGE:
case Opcodes.IF_ICMPGT:
case Opcodes.IF_ICMPLE:
case Opcodes.IF_ACMPEQ:
case Opcodes.IF_ACMPNE:
case Opcodes.LRETURN:
case Opcodes.DRETURN:
pop(2);
break;
case Opcodes.DUP:
t1 = pop();
push(t1);
push(t1);
break;
case Opcodes.DUP_X1:
t1 = pop();
t2 = pop();
push(t1);
push(t2);
push(t1);
break;
case Opcodes.DUP_X2:
t1 = pop();
t2 = pop();
t3 = pop();
7.12. FRAME.JAVA 291

push(t1);
push(t3);
push(t2);
push(t1);
break;
case Opcodes.DUP2:
t1 = pop();
t2 = pop();
push(t2);
push(t1);
push(t2);
push(t1);
break;
case Opcodes.DUP2_X1:
t1 = pop();
t2 = pop();
t3 = pop();
push(t2);
push(t1);
push(t3);
push(t2);
push(t1);
break;
case Opcodes.DUP2_X2:
t1 = pop();
t2 = pop();
t3 = pop();
t4 = pop();
push(t2);
push(t1);
push(t4);
push(t3);
push(t2);
push(t1);
break;
case Opcodes.SWAP:
t1 = pop();
t2 = pop();
push(t1);
push(t2);
break;
case Opcodes.IADD:
case Opcodes.ISUB:
case Opcodes.IMUL:
case Opcodes.IDIV:
case Opcodes.IREM:
case Opcodes.IAND:
case Opcodes.IOR:
case Opcodes.IXOR:
case Opcodes.ISHL:
292 CHAPTER 7. JVM/CLOJURE/ASM/

case Opcodes.ISHR:
case Opcodes.IUSHR:
case Opcodes.L2I:
case Opcodes.D2I:
case Opcodes.FCMPL:
case Opcodes.FCMPG:
pop(2);
push(INTEGER);
break;
case Opcodes.LADD:
case Opcodes.LSUB:
case Opcodes.LMUL:
case Opcodes.LDIV:
case Opcodes.LREM:
case Opcodes.LAND:
case Opcodes.LOR:
case Opcodes.LXOR:
pop(4);
push(LONG);
push(TOP);
break;
case Opcodes.FADD:
case Opcodes.FSUB:
case Opcodes.FMUL:
case Opcodes.FDIV:
case Opcodes.FREM:
case Opcodes.L2F:
case Opcodes.D2F:
pop(2);
push(FLOAT);
break;
case Opcodes.DADD:
case Opcodes.DSUB:
case Opcodes.DMUL:
case Opcodes.DDIV:
case Opcodes.DREM:
pop(4);
push(DOUBLE);
push(TOP);
break;
case Opcodes.LSHL:
case Opcodes.LSHR:
case Opcodes.LUSHR:
pop(3);
push(LONG);
push(TOP);
break;
case Opcodes.IINC:
set(arg, INTEGER);
break;
7.12. FRAME.JAVA 293

case Opcodes.I2L:
case Opcodes.F2L:
pop(1);
push(LONG);
push(TOP);
break;
case Opcodes.I2F:
pop(1);
push(FLOAT);
break;
case Opcodes.I2D:
case Opcodes.F2D:
pop(1);
push(DOUBLE);
push(TOP);
break;
case Opcodes.F2I:
case Opcodes.ARRAYLENGTH:
case Opcodes.INSTANCEOF:
pop(1);
push(INTEGER);
break;
case Opcodes.LCMP:
case Opcodes.DCMPL:
case Opcodes.DCMPG:
pop(4);
push(INTEGER);
break;
case Opcodes.JSR:
case Opcodes.RET:
throw new RuntimeException(
"JSR/RET are not supported with computeFrames option");
case Opcodes.GETSTATIC:
push(cw, item.strVal3);
break;
case Opcodes.PUTSTATIC:
pop(item.strVal3);
break;
case Opcodes.GETFIELD:
pop(1);
push(cw, item.strVal3);
break;
case Opcodes.PUTFIELD:
pop(item.strVal3);
pop();
break;
case Opcodes.INVOKEVIRTUAL:
case Opcodes.INVOKESPECIAL:
case Opcodes.INVOKESTATIC:
case Opcodes.INVOKEINTERFACE:
294 CHAPTER 7. JVM/CLOJURE/ASM/

pop(item.strVal3);
if(opcode != Opcodes.INVOKESTATIC)
{
t1 = pop();
if(opcode == Opcodes.INVOKESPECIAL
&& item.strVal2.charAt(0) == <)
{
init(t1);
}
}
push(cw, item.strVal3);
break;
case Opcodes.NEW:
push(UNINITIALIZED |
cw.addUninitializedType(item.strVal1, arg));
break;
case Opcodes.NEWARRAY:
pop();
switch(arg)
{
case Opcodes.T_BOOLEAN:
push(ARRAY_OF | BOOLEAN);
break;
case Opcodes.T_CHAR:
push(ARRAY_OF | CHAR);
break;
case Opcodes.T_BYTE:
push(ARRAY_OF | BYTE);
break;
case Opcodes.T_SHORT:
push(ARRAY_OF | SHORT);
break;
case Opcodes.T_INT:
push(ARRAY_OF | INTEGER);
break;
case Opcodes.T_FLOAT:
push(ARRAY_OF | FLOAT);
break;
case Opcodes.T_DOUBLE:
push(ARRAY_OF | DOUBLE);
break;
// case Opcodes.T_LONG:
default:
push(ARRAY_OF | LONG);
break;
}
break;
case Opcodes.ANEWARRAY:
String s = item.strVal1;
pop();
7.12. FRAME.JAVA 295

if(s.charAt(0) == [)
{
push(cw, "[" + s);
}
else
{
push(ARRAY_OF | OBJECT | cw.addType(s));
}
break;
case Opcodes.CHECKCAST:
s = item.strVal1;
pop();
if(s.charAt(0) == [)
{
push(cw, s);
}
else
{
push(OBJECT | cw.addType(s));
}
break;
// case Opcodes.MULTIANEWARRAY:
default:
pop(arg);
push(cw, item.strVal1);
break;
}
}

/**
* Merges the input frame of the given basic block with the input and
* output frames of this basic block. Returns <tt>true</tt> if the
* input frame of the given label has been changed by this operation.
*
* @param cw the ClassWriter to which this label belongs.
* @param frame the basic block whose input frame must be updated.
* @param edge the kind of the {@link Edge} between this label and
* label. See {@link Edge#info}.
* @return <tt>true</tt> if the input frame of the given label has been
* changed by this operation.
*/
boolean merge(final ClassWriter cw, final Frame frame, final int edge){
boolean changed = false;
int i, s, dim, kind, t;

int nLocal = inputLocals.length;

int nStack = inputStack.length;
if(frame.inputLocals == null)
{
frame.inputLocals = new int[nLocal];
296 CHAPTER 7. JVM/CLOJURE/ASM/

changed = true;
}

for(i = 0; i < nLocal; ++i)

{
if(outputLocals != null && i < outputLocals.length)
{
s = outputLocals[i];
if(s == 0)
{
t = inputLocals[i];
}
else
{
dim = s & DIM;
kind = s & KIND;
if(kind == LOCAL)
{
t = dim + inputLocals[s & VALUE];
}
else if(kind == STACK)
{
t = dim + inputStack[nStack - (s & VALUE)];
}
else
{
t = s;
}
}
}
else
{
t = inputLocals[i];
}
if(initializations != null)
{
t = init(cw, t);
}
changed |= merge(cw, t, frame.inputLocals, i);
}

if(edge > 0)
{
for(i = 0; i < nLocal; ++i)
{
t = inputLocals[i];
changed |= merge(cw, t, frame.inputLocals, i);
}
if(frame.inputStack == null)
{
7.12. FRAME.JAVA 297

frame.inputStack = new int[1];

changed = true;
}
changed |= merge(cw, edge, frame.inputStack, 0);
return changed;
}

int nInputStack = inputStack.length + owner.inputStackTop;

if(frame.inputStack == null)
{
frame.inputStack = new int[nInputStack + outputStackTop];
changed = true;
}

for(i = 0; i < nInputStack; ++i)

{
t = inputStack[i];
if(initializations != null)
{
t = init(cw, t);
}
changed |= merge(cw, t, frame.inputStack, i);
}
for(i = 0; i < outputStackTop; ++i)
{
s = outputStack[i];
dim = s & DIM;
kind = s & KIND;
if(kind == LOCAL)
{
t = dim + inputLocals[s & VALUE];
}
else if(kind == STACK)
{
t = dim + inputStack[nStack - (s & VALUE)];
}
else
{
t = s;
}
if(initializations != null)
{
t = init(cw, t);
}
changed |= merge(cw, t, frame.inputStack, nInputStack + i);
}
return changed;
}

/**
298 CHAPTER 7. JVM/CLOJURE/ASM/

* Merges the type at the given index in the given type array with
* the given type. Returns <tt>true</tt> if the type array has been
* modified by this operation.
*
* @param cw the ClassWriter to which this label belongs.
* @param t the type with which the type array element must be
* merged.
* @param types an array of types.
* @param index the index of the type that must be merged in types.
* @return <tt>true</tt> if the type array has been modified by this
* operation.
*/
private boolean merge(
final ClassWriter cw,
int t,
final int[] types,
final int index){
int u = types[index];
if(u == t)
{
// if the types are equal, merge(u,t)=u, so there is no change
return false;
}
if((t & ~DIM) == NULL)
{
if(u == NULL)
{
return false;
}
t = NULL;
}
if(u == 0)
{
// if types[index] has never been assigned, merge(u,t)=t
types[index] = t;
return true;
}
int v;
if((u & BASE_KIND) == OBJECT || (u & DIM) != 0)
{
// if u is a reference type of any dimension
if(t == NULL)
{
// if t is the NULL type, merge(u,t)=u, so there is no change
return false;
}
else if((t & (DIM | BASE_KIND)) == (u & (DIM | BASE_KIND)))
{
if((u & BASE_KIND) == OBJECT)
{
7.12. FRAME.JAVA 299

// if t is also a reference type, and if u and t have

// the same dimension merge(u,t) = dim(t) | common parent
// of the element types of u and t
v = (t & DIM) | OBJECT
| cw.getMergedType(t & BASE_VALUE, u & BASE_VALUE);
}
else
{
// if u and t are array types, but not with the same
// element type, merge(u,t)=java/lang/Object
v = OBJECT | cw.addType("java/lang/Object");
}
}
else if((t & BASE_KIND) == OBJECT || (t & DIM) != 0)
{
// if t is any other reference or array type,
// merge(u,t)=java/lang/Object
v = OBJECT | cw.addType("java/lang/Object");
}
else
{
// if t is any other type, merge(u,t)=TOP
v = TOP;
}
}
else if(u == NULL)
{
// if u is the NULL type, merge(u,t)=t,
// or TOP if t is not a reference type
v = (t & BASE_KIND) == OBJECT || (t & DIM) != 0 ? t : TOP;
}
else
{
// if u is any other type, merge(u,t)=TOP whatever t
v = TOP;
}
if(u != v)
{
types[index] = v;
return true;
}
return false;
}
}

-
300 CHAPTER 7. JVM/CLOJURE/ASM/

7.13 Handler.java

Handler.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* Information about an exception handler block.
*
* @author Eric Bruneton
*/
class Handler{

/**
* Beginning of the exception handlers scope (inclusive).
*/
Label start;

/**
* End of the exception handlers scope (exclusive).
*/
Label end;

/**
* Beginning of the exception handlers code.
*/
Label handler;

/**
* Internal name of the type of exceptions handled by this handler, or
* <tt>null</tt> to catch any exceptions.
*/
String desc;

/**
* Constant pool index of the internal name of the type of exceptions
* handled by this handler, or 0 to catch any exceptions.
*/
int type;

/**
* Next exception handler block info.
*/
Handler next;
}

-
7.14. ITEM.JAVA 301

7.14 Item.java

Item.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* A constant pool item. Constant pool items can be created with the
* newXXX methods in the {@link ClassWriter} class.
*
* @author Eric Bruneton
*/
final class Item{

/**
* Index of this item in the constant pool.
*/
int index;

/**
* Type of this constant pool item. A single class is used to represent
* all constant pool item types, in order to minimize the bytecode size
* of this package. The value of this field is one of
* {@link ClassWriter#INT},
* {@link ClassWriter#LONG}, {@link ClassWriter#FLOAT},
* {@link ClassWriter#DOUBLE}, {@link ClassWriter#UTF8},
* {@link ClassWriter#STR}, {@link ClassWriter#CLASS},
* {@link ClassWriter#NAME_TYPE}, {@link ClassWriter#FIELD},
* {@link ClassWriter#METH}, {@link ClassWriter#IMETH}.
* <p/>
* Special Item types are used for Items that are stored in the
* ClassWriter {@link ClassWriter#typeTable}, instead of the constant
* pool, in order to avoid clashes with normal constant pool items in
* the ClassWriter constant pools hash table. These special item types
* are {@link ClassWriter#TYPE_NORMAL}, {@link ClassWriter#TYPE_UNINIT}
* and {@link ClassWriter#TYPE_MERGED}.
*/
int type;

/**
* Value of this item, for an integer item.
*/
int intVal;

/**
* Value of this item, for a long item.
*/
302 CHAPTER 7. JVM/CLOJURE/ASM/

long longVal;

/**
* First part of the value of this item, for items that do not hold a
* primitive value.
*/
String strVal1;

/**
* Second part of the value of this item, for items that do not hold a
* primitive value.
*/
String strVal2;

/**
* Third part of the value of this item, for items that do not hold a
* primitive value.
*/
String strVal3;

/**
* The hash code value of this constant pool item.
*/
int hashCode;

/**
* Link to another constant pool item, used for collision lists in the
* constant pools hash table.
*/
Item next;

/**
* Constructs an uninitialized {@link Item}.
*/
Item(){
}

/**
* Constructs an uninitialized {@link Item} for constant pool element at
* given position.
*
* @param index index of the item to be constructed.
*/
Item(final int index){
this.index = index;
}

/**
* Constructs a copy of the given item.
*
7.14. ITEM.JAVA 303

* @param index index of the item to be constructed.

* @param i the item that must be copied into the item to be
* constructed.
*/
Item(final int index, final Item i){
this.index = index;
type = i.type;
intVal = i.intVal;
longVal = i.longVal;
strVal1 = i.strVal1;
strVal2 = i.strVal2;
strVal3 = i.strVal3;
hashCode = i.hashCode;
}

/**
* Sets this item to an integer item.
*
* @param intVal the value of this item.
*/
void set(final int intVal){
this.type = ClassWriter.INT;
this.intVal = intVal;
this.hashCode = 0x7FFFFFFF & (type + intVal);
}

/**
* Sets this item to a long item.
*
* @param longVal the value of this item.
*/
void set(final long longVal){
this.type = ClassWriter.LONG;
this.longVal = longVal;
this.hashCode = 0x7FFFFFFF & (type + (int) longVal);
}

/**
* Sets this item to a float item.
*
* @param floatVal the value of this item.
*/
void set(final float floatVal){
this.type = ClassWriter.FLOAT;
this.intVal = Float.floatToRawIntBits(floatVal);
this.hashCode = 0x7FFFFFFF & (type + (int) floatVal);
}

/**
* Sets this item to a double item.
304 CHAPTER 7. JVM/CLOJURE/ASM/

*
* @param doubleVal the value of this item.
*/
void set(final double doubleVal){
this.type = ClassWriter.DOUBLE;
this.longVal = Double.doubleToRawLongBits(doubleVal);
this.hashCode = 0x7FFFFFFF & (type + (int) doubleVal);
}

/**
* Sets this item to an item that do not hold a primitive value.
*
* @param type the type of this item.
* @param strVal1 first part of the value of this item.
* @param strVal2 second part of the value of this item.
* @param strVal3 third part of the value of this item.
*/
void set(
final int type,
final String strVal1,
final String strVal2,
final String strVal3){
this.type = type;
this.strVal1 = strVal1;
this.strVal2 = strVal2;
this.strVal3 = strVal3;
switch(type)
{
case ClassWriter.UTF8:
case ClassWriter.STR:
case ClassWriter.CLASS:
case ClassWriter.TYPE_NORMAL:
hashCode = 0x7FFFFFFF & (type + strVal1.hashCode());
return;
case ClassWriter.NAME_TYPE:
hashCode = 0x7FFFFFFF & (type + strVal1.hashCode()
* strVal2.hashCode());
return;
// ClassWriter.FIELD:
// ClassWriter.METH:
// ClassWriter.IMETH:
default:
hashCode = 0x7FFFFFFF &
(type + strVal1.hashCode()
* strVal2.hashCode() * strVal3.hashCode());
}
}

/**
* Indicates if the given item is equal to this one.
7.15. LABEL.JAVA 305

*
* @param i the item to be compared to this one.
* @return <tt>true</tt> if the given item if equal to this one,
* <tt>false</tt> otherwise.
*/
boolean isEqualTo(final Item i){
if(i.type == type)
{
switch(type)
{
case ClassWriter.INT:
case ClassWriter.FLOAT:
return i.intVal == intVal;
case ClassWriter.TYPE_MERGED:
case ClassWriter.LONG:
case ClassWriter.DOUBLE:
return i.longVal == longVal;
case ClassWriter.UTF8:
case ClassWriter.STR:
case ClassWriter.CLASS:
case ClassWriter.TYPE_NORMAL:
return i.strVal1.equals(strVal1);
case ClassWriter.TYPE_UNINIT:
return i.intVal == intVal && i.strVal1.equals(strVal1);
case ClassWriter.NAME_TYPE:
return i.strVal1.equals(strVal1)
&& i.strVal2.equals(strVal2);
// ClassWriter.FIELD:
// ClassWriter.METH:
// ClassWriter.IMETH:
default:
return i.strVal1.equals(strVal1)
&& i.strVal2.equals(strVal2)
&& i.strVal3.equals(strVal3);
}
}
return false;
}
}

7.15 Label.java

Label.java

\getchunk{France Telecom Copyright}

306 CHAPTER 7. JVM/CLOJURE/ASM/

package clojure.asm;

/**
* A label represents a position in the bytecode of a method. Labels
* are used for jump, goto, and switch instructions, and for try
* catch blocks.
*
* @author Eric Bruneton
*/
public class Label{

/**
* Indicates if this label is only used for debug attributes. Such a
* label is not the start of a basic block, the target of a jump
* instruction, or an exception handler. It can be safely ignored
* in control flow graph analysis algorithms (for optimization
* purposes).
*/
final static int DEBUG = 1;

/**
* Indicates if the position of this label is known.
*/
final static int RESOLVED = 2;

/**
* Indicates if this label has been updated, after instruction resizing.
*/
final static int RESIZED = 4;

/**
* Indicates if this basic block has been pushed in the basic block
* stack. See {@link MethodWriter#visitMaxs visitMaxs}.
*/
final static int PUSHED = 8;

/**
* Indicates if this label is the target of a jump instruction, or
* the start of an exception handler.
*/
final static int TARGET = 16;

/**
* Indicates if a stack map frame must be stored for this label.
*/
final static int STORE = 32;

/**
* Indicates if this label corresponds to a reachable basic block.
*/
7.15. LABEL.JAVA 307

final static int REACHABLE = 64;

/**
* Indicates if this basic block ends with a JSR instruction.
*/
final static int JSR = 128;

/**
* Indicates if this basic block ends with a RET instruction.
*/
final static int RET = 256;

/**
* Field used to associate user information to a label.
*/
public Object info;

/**
* Flags that indicate the status of this label.
*
* @see #DEBUG
* @see #RESOLVED
* @see #RESIZED
* @see #PUSHED
* @see #TARGET
* @see #STORE
* @see #REACHABLE
* @see #JSR
* @see #RET
*/
int status;

/**
* The line number corresponding to this label, if known.
*/
int line;

/**
* The position of this label in the code, if known.
*/
int position;

/**
* Number of forward references to this label, times two.
*/
private int referenceCount;

/**
* Informations about forward references. Each forward reference is
* described by two consecutive integers in this array: the first one
308 CHAPTER 7. JVM/CLOJURE/ASM/

* is the position of the first byte of the bytecode instruction that

* contains the forward reference, while the second is the position
* of the first byte of the forward reference itself. In fact the
* sign of the first integer indicates if this reference uses 2 or 4
* bytes, and its absolute value gives the position of the bytecode
* instruction.
*/
private int[] srcAndRefPositions;

// -------------------------------------------------------------------

/*
* Fields for the control flow and data flow graph analysis algorithms
* (used to compute the maximum stack size or the stack map frames).
* A control flow graph contains one node per "basic block", and one
* edge per "jump" from one basic block to another. Each node (i.e.,
* each basic block) is represented by the Label object that
* corresponds to the first instruction of this basic block. Each node
* also stores the list of its successors in the graph, as a linked
* list of Edge objects.
*
* The control flow analysis algorithms used to compute the maximum
* stack size or the stack map frames are similar and use two steps.
* The first step, during the visit of each instruction, builds
* information about the state of the local variables and the operand
* stack at the end of each basic block, called the "output frame",
* <i>relatively</i> to the frame state at the beginning of the basic
* block, which is called the "input frame", and which is <i>unknown</i>
* during this step. The second step, in
* {@link MethodWriter#visitMaxs}, is a fix point algorithm that
* icomputes information about the input frame of each basic block,
* from the nput state of the first basic block (known from the
* method signature), and by the using the previously computed
* relative output frames.
*
* The algorithm used to compute the maximum stack size only computes
* the relative output and absolute input stack heights, while the
* algorithm used to compute stack map frames computes relative
* output frames and absolute input frames.
*/

/**
* Start of the output stack relatively to the input stack. The exact
* semantics of this field depends on the algorithm that is used.
* <p/>
* When only the maximum stack size is computed, this field is the
* number of elements in the input stack.
* <p/>
* When the stack map frames are completely computed, this field is
* the offset of the first output stack element relatively to the top
7.15. LABEL.JAVA 309

* of the input stack. This offset is always negative or null. A

* null offset means that the output stack must be appended to the
* input stack. A -n offset means that the first n output stack
* elements must replace the top n input stack elements, and that
* the other elements must be appended to the input stack.
*/
int inputStackTop;

/**
* Maximum height reached by the output stack, relatively to the top
* of the input stack. This maximum is always positive or null.
*/
int outputStackMax;

/**
* Information about the input and output stack map frames of this
* basic block. This field is only used when
* {@link ClassWriter#COMPUTE_FRAMES} option is used.
*/
Frame frame;

/**
* The successor of this label, in the order they are visited. This
* linked list does not include labels used for debug info only. If
* {@link ClassWriter#COMPUTE_FRAMES} option is used then, in addition,
* it does not contain successive labels that denote the same
* bytecode position (in this case only the first label appears in
* this list).
*/
Label successor;

/**
* The successors of this node in the control flow graph. These
* successors are stored in a linked list of {@link Edge Edge}
* objects, linked to each other by their {@link Edge#next} field.
*/
Edge successors;

/**
* The next basic block in the basic block stack. This stack is used
* in the main loop of the fix point algorithm used in the second step
* of the control flow analysis algorithms.
*
* @see MethodWriter#visitMaxs
*/
Label next;

// -------------------------------------------------------------------
// Constructor
// -------------------------------------------------------------------
310 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Constructs a new label.
*/
public Label(){
}

/**
* Constructs a new label.
*
* @param debug if this label is only used for debug attributes.
*/
Label(final boolean debug){
this.status = debug ? DEBUG : 0;
}

// -------------------------------------------------------------------
// Methods to compute offsets and to manage forward references
// -------------------------------------------------------------------

/**
* Returns the offset corresponding to this label. This offset is
* computed from the start of the methods bytecode. <i>This method
* is intended for {@link Attribute} sub classes, and is normally
* not needed by class generators or adapters.</i>
*
* @return the offset corresponding to this label.
* @throws IllegalStateException if this label is not resolved yet.
*/
public int getOffset(){
if((status & RESOLVED) == 0)
{
throw new IllegalStateException(
"Label offset position has not been resolved yet");
}
return position;
}

/**
* Puts a reference to this label in the bytecode of a method. If the
* position of the label is known, the offset is computed and written
* directly. Otherwise, a null offset is written and a new forward
* reference is declared for this label.
*
* @param owner the code writer that calls this method.
* @param out the bytecode of the method.
* @param source the position of first byte of the bytecode
* instruction that contains this label.
* @param wideOffset <tt>true</tt> if the reference must be stored in 4
* bytes, or <tt>false</tt> if it must be stored with 2
7.15. LABEL.JAVA 311

* bytes.
* @throws IllegalArgumentException if this label has not been created by
* the given code writer.
*/
void put(
final MethodWriter owner,
final ByteVector out,
final int source,
final boolean wideOffset){
if((status & RESOLVED) != 0)
{
if(wideOffset)
{
out.putInt(position - source);
}
else
{
out.putShort(position - source);
}
}
else
{
if(wideOffset)
{
addReference(-1 - source, out.length);
out.putInt(-1);
}
else
{
addReference(source, out.length);
out.putShort(-1);
}
}
}

/**
* Adds a forward reference to this label. This method must be called
* only for a true forward reference, i.e. only if this label is not
* resolved yet. For backward references, the offset of the reference
* can be, and must be, computed and stored directly.
*
* @param sourcePosition the position of the referencing instruction.
* This position will be used to compute the
* offset of this forward reference.
* @param referencePosition the position where the offset for this
* forward reference must be stored.
*/
private void addReference(
final int sourcePosition,
final int referencePosition){
312 CHAPTER 7. JVM/CLOJURE/ASM/

if(srcAndRefPositions == null)
{
srcAndRefPositions = new int[6];
}
if(referenceCount >= srcAndRefPositions.length)
{
int[] a = new int[srcAndRefPositions.length + 6];
System.arraycopy(srcAndRefPositions,
0,
a,
0,
srcAndRefPositions.length);
srcAndRefPositions = a;
}
srcAndRefPositions[referenceCount++] = sourcePosition;
srcAndRefPositions[referenceCount++] = referencePosition;
}

/**
* Resolves all forward references to this label. This method must be
* called when this label is added to the bytecode of the method,
* i.e. when its position becomes known. This method fills in the
* blanks that where left in the bytecode by each forward reference
* previously added to this label.
*
* @param owner the code writer that calls this method.
* @param position the position of this label in the bytecode.
* @param data the bytecode of the method.
* @return <tt>true</tt> if a blank that was left for this label was
* to small to store the offset. In such a case the
* corresponding jump instruction is replaced with a pseudo
* instruction (using unused opcodes) using an unsigned two
* bytes offset. These pseudo instructions will need to be
* replaced with true instructions with wider offsets (4 bytes
* instead of 2). This is done in
* {@link MethodWriter#resizeInstructions}.
* @throws IllegalArgumentException if this label has already been
* resolved, or if it has not been
created by the given code writer.
*/
boolean resolve(
final MethodWriter owner,
final int position,
final byte[] data){
boolean needUpdate = false;
this.status |= RESOLVED;
this.position = position;
int i = 0;
while(i < referenceCount)
{
7.15. LABEL.JAVA 313

int source = srcAndRefPositions[i++];

int reference = srcAndRefPositions[i++];
int offset;
if(source >= 0)
{
offset = position - source;
if(offset < Short.MIN_VALUE || offset > Short.MAX_VALUE)
{
/*
* changes the opcode of the jump instruction, in order
* to be able to find it later (see resizeInstructions
* in MethodWriter). These temporary opcodes are similar
* to jump instruction opcodes, except that the 2 bytes
* offset is unsigned (and can therefore represent
* values from 0 to 65535, which is sufficient since
* the size of a method is limited to 65535 bytes).
*/
int opcode = data[reference - 1] & 0xFF;
if(opcode <= Opcodes.JSR)
{
// changes IFEQ ... JSR to opcodes 202 to 217
data[reference - 1] = (byte) (opcode + 49);
}
else
{
// changes IFNULL and IFNONNULL to opcodes 218
// and 219
data[reference - 1] = (byte) (opcode + 20);
}
needUpdate = true;
}
data[reference++] = (byte) (offset >>> 8);
data[reference] = (byte) offset;
}
else
{
offset = position + source + 1;
data[reference++] = (byte) (offset >>> 24);
data[reference++] = (byte) (offset >>> 16);
data[reference++] = (byte) (offset >>> 8);
data[reference] = (byte) offset;
}
}
return needUpdate;
}

/**
* Returns the first label of the series to which this label belongs.
* For an isolated label or for the first label in a series of
* successive labels, this method returns the label itself. For other
314 CHAPTER 7. JVM/CLOJURE/ASM/

* labels it returns the first label of the series.

*
* @return the first label of the series to which this label belongs.
*/
Label getFirst(){
return frame == null ? this : frame.owner;
}

// -------------------------------------------------------------------
// Overriden Object methods
// -------------------------------------------------------------------

/**
* Returns a string representation of this label.
*
* @return a string representation of this label.
*/
public String toString(){
return "L" + System.identityHashCode(this);
}
}

7.16 MethodAdapter.java
(MethodVisitor [317])
MethodAdapter.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* An empty {@link MethodVisitor} that delegates to another
* {@link MethodVisitor}. This class can be used as a super class to
* quickly implement usefull method adapter classes, just by overriding
* the necessary methods.
*
* @author Eric Bruneton
*/
public class MethodAdapter implements MethodVisitor{

/**
* The {@link MethodVisitor} to which this adapter delegates calls.
*/
protected MethodVisitor mv;

/**
7.16. METHODADAPTER.JAVA 315

* Constructs a new {@link MethodAdapter} object.

*
* @param mv the code visitor to which this adapter must delegate calls.
*/
public MethodAdapter(final MethodVisitor mv){
this.mv = mv;
}

public AnnotationVisitor visitAnnotationDefault(){

return mv.visitAnnotationDefault();
}

public AnnotationVisitor visitAnnotation(

final String desc,
final boolean visible){
return mv.visitAnnotation(desc, visible);
}

public AnnotationVisitor visitParameterAnnotation(

final int parameter,
final String desc,
final boolean visible){
return mv.visitParameterAnnotation(parameter, desc, visible);
}

public void visitAttribute(final Attribute attr){

mv.visitAttribute(attr);
}

public void visitCode(){

mv.visitCode();
}

public void visitFrame(

final int type,
final int nLocal,
final Object[] local,
final int nStack,
final Object[] stack){
mv.visitFrame(type, nLocal, local, nStack, stack);
}

public void visitInsn(final int opcode){

mv.visitInsn(opcode);
}

public void visitIntInsn(final int opcode, final int operand){

mv.visitIntInsn(opcode, operand);
}
316 CHAPTER 7. JVM/CLOJURE/ASM/

public void visitVarInsn(final int opcode, final int var){

mv.visitVarInsn(opcode, var);
}

public void visitTypeInsn(final int opcode, final String desc){

mv.visitTypeInsn(opcode, desc);
}

public void visitFieldInsn(

final int opcode,
final String owner,
final String name,
final String desc){
mv.visitFieldInsn(opcode, owner, name, desc);
}

public void visitMethodInsn(

final int opcode,
final String owner,
final String name,
final String desc){
mv.visitMethodInsn(opcode, owner, name, desc);
}

public void visitJumpInsn(final int opcode, final Label label){

mv.visitJumpInsn(opcode, label);
}

public void visitLabel(final Label label){

mv.visitLabel(label);
}

public void visitLdcInsn(final Object cst){

mv.visitLdcInsn(cst);
}

public void visitIincInsn(final int var, final int increment){

mv.visitIincInsn(var, increment);
}

public void visitTableSwitchInsn(

final int min,
final int max,
final Label dflt,
final Label labels[]){
mv.visitTableSwitchInsn(min, max, dflt, labels);
}

public void visitLookupSwitchInsn(

final Label dflt,
7.17. METHODVISITOR.JAVA 317

final int keys[],

final Label labels[]){
mv.visitLookupSwitchInsn(dflt, keys, labels);
}

public void visitMultiANewArrayInsn(final String desc, final int dims){

mv.visitMultiANewArrayInsn(desc, dims);
}

public void visitTryCatchBlock(

final Label start,
final Label end,
final Label handler,
final String type){
mv.visitTryCatchBlock(start, end, handler, type);
}

public void visitLocalVariable(

final String name,
final String desc,
final String signature,
final Label start,
final Label end,
final int index){
mv.visitLocalVariable(name, desc, signature, start, end, index);
}

public void visitLineNumber(final int line, final Label start){

mv.visitLineNumber(line, start);
}

public void visitMaxs(final int maxStack, final int maxLocals){

mv.visitMaxs(maxStack, maxLocals);
}

public void visitEnd(){

mv.visitEnd();
}
}

7.17 MethodVisitor.java

MethodVisitor.java

\getchunk{France Telecom Copyright}

318 CHAPTER 7. JVM/CLOJURE/ASM/

package clojure.asm;

/**
* A visitor to visit a Java method. The methods of this interface must
* be called in the following order:
* [ <tt>visitAnnotationDefault</tt> ] ( * <tt>visitAnnotation</tt> |
* <tt>visitParameterAnnotation</tt> | * <tt>visitAttribute</tt> )*
* [ <tt>visitCode</tt> ( <tt>visitFrame</tt> |
* <tt>visit</tt><i>X</i>Insn</tt> | <tt>visitLabel</tt> |
* <tt>visitTryCatchBlock</tt> | * <tt>visitLocalVariable</tt> |
* <tt>visitLineNumber</tt>)* <tt>visitMaxs</tt> ]
* <tt>visitEnd</tt>. In addition, the <tt>visit</tt><i>X</i>Insn</tt>
* and <tt>visitLabel</tt> methods must be called in the sequential
* order of the bytecode instructions of the visited code,
* <tt>visitTryCatchBlock</tt> must be called <i>before</i> the
* labels passed as arguments have been visited, and the
* <tt>visitLocalVariable</tt> and <tt>visitLineNumber</tt> methods
* must be called <i>after</i> the labels passed as arguments have
* been visited.
*
* @author Eric Bruneton
*/
public interface MethodVisitor{

// --------------------------------------------------------------------
// Annotations and non standard attributes
// --------------------------------------------------------------------

/**
* Visits the default value of this annotation interface method.
*
* @return a visitor to the visit the actual default value of this
* annotation interface method, or <tt>null</tt> if this
* visitor is not interested in visiting this default value.
* The name parameters passed to the methods of this
* annotation visitor are ignored. Moreover, exacly one visit
* method must be called on this annotation visitor, followed
* by visitEnd.
*/
AnnotationVisitor visitAnnotationDefault();

/**
* Visits an annotation of this method.
*
* @param desc the class descriptor of the annotation class.
* @param visible <tt>true</tt> if the annotation is visible at runtime.
* @return a visitor to visit the annotation values, or <tt>null</tt> if
* this visitor is not interested in visiting this annotation.
*/
AnnotationVisitor visitAnnotation(String desc, boolean visible);
7.17. METHODVISITOR.JAVA 319

/**
* Visits an annotation of a parameter this method.
*
* @param parameter the parameter index.
* @param desc the class descriptor of the annotation class.
* @param visible <tt>true</tt> if the annotation is visible at
* runtime.
* @return a visitor to visit the annotation values, or <tt>null</tt>
* if this visitor is not interested in visiting this annotation.
*/
AnnotationVisitor visitParameterAnnotation(
int parameter,
String desc,
boolean visible);

/**
* Visits a non standard attribute of this method.
*
* @param attr an attribute.
*/
void visitAttribute(Attribute attr);

/**
* Starts the visit of the methods code, if any (i.e. non abstract
* method).
*/
void visitCode();

/**
* Visits the current state of the local variables and operand stack
* elements. This method must(*) be called <i>just before</i> any
* instruction <b>i</b> that follows an unconditionnal branch
* instruction such as GOTO or THROW, that is the target of a jump
* instruction, or that starts an exception handler block. The visited
* types must describe the values of the local variables and of the
* operand stack elements <i>just before</i> <b>i</b> is executed.
* <br> <br> (*) this is mandatory only for classes whose version
* is greater than or equal to {@link Opcodes#V1_6 V1_6}. <br> <br>
* Packed frames are basically "deltas" from the state of the
* previous frame (very first frame is implicitly defined by the
* methods parameters and access flags): <ul> <li>
* {@link Opcodes#F_SAME} representing frame with exactly the same
* locals as the previous frame and with the empty stack.</li>
* <li>{@link Opcodes#F_SAME1}
* representing frame with exactly the same locals as the previous
* frame and with single value on the stack (<code>nStack</code> is 1
* and <code>stack[0]</code> contains value for the type of the
* stack item).</li> <li>{@link Opcodes#F_APPEND} representing frame
* with current locals are the same as the locals in the previous
320 CHAPTER 7. JVM/CLOJURE/ASM/

* frame, except that additional locals are defined

* (<code>nLocal</code> is 1, 2 or 3 and * <code>local</code> elements
* contains values representing added types).</li>
* <li>{@link Opcodes#F_CHOP} representing frame with current locals
* are the same as the locals in the previous frame, except that the
* last 1-3 locals are absent and with the empty stack
* (<code>nLocals</code> is 1, 2 or 3). </li> <li>
* {@link Opcodes#F_FULL} representing complete frame
* data.</li> </li> </ul>
*
* @param type the type of this stack map frame. Must be
* {@link Opcodes#F_NEW} for expanded frames, or
* {@link Opcodes#F_FULL}, {@link Opcodes#F_APPEND},
* {@link Opcodes#F_CHOP}, {@link Opcodes#F_SAME} or
* {@link Opcodes#F_APPEND}, {@link Opcodes#F_SAME1}
* for compressed frames.
* @param nLocal the number of local variables in the visited frame.
* @param local the local variable types in this frame. This array must
* not be modified. Primitive types are represented by
* {@link Opcodes#TOP}, {@link Opcodes#INTEGER},
* {@link Opcodes#FLOAT}, {@link Opcodes#LONG},
* {@link Opcodes#DOUBLE},{@link Opcodes#NULL} or
* {@link Opcodes#UNINITIALIZED_THIS} (long and double are
* represented by a single element). Reference types are
* represented by String objects (representing internal
* names, or type descriptors for array types), and
* uninitialized types by Label objects (this label
* designates the NEW instruction that created this
* uninitialized value).
* @param nStack the number of operand stack elements in the visited
* frame.
* @param stack the operand stack types in this frame. This array must
* not be modified. Its content has the same format as
* the "local" array.
*/
void visitFrame(
int type,
int nLocal,
Object[] local,
int nStack,
Object[] stack);

// --------------------------------------------------------------------
// Normal instructions
// --------------------------------------------------------------------

/**
* Visits a zero operand instruction.
*
* @param opcode the opcode of the instruction to be visited. This
7.17. METHODVISITOR.JAVA 321

* opcode is either NOP, ACONST_NULL, ICONST_M1, ICONST_0,

* ICONST_1, ICONST_2, ICONST_3, ICONST_4, ICONST_5,
* LCONST_0, LCONST_1, FCONST_0, FCONST_1, FCONST_2,
* DCONST_0, DCONST_1, IALOAD, LALOAD, FALOAD,
* DALOAD, AALOAD, BALOAD, CALOAD, SALOAD, IASTORE,
* LASTORE, FASTORE, DASTORE, AASTORE, BASTORE, CASTORE,
* SASTORE, POP, POP2, DUP, DUP_X1, DUP_X2, DUP2, DUP2_X1,
* DUP2_X2, SWAP, IADD, LADD, FADD, DADD, ISUB, LSUB, FSUB,
* DSUB, IMUL, LMUL, FMUL, DMUL, IDIV, LDIV, FDIV, DDIV,
* IREM, LREM, FREM, DREM, INEG, LNEG, FNEG, DNEG, ISHL,
* LSHL, ISHR, LSHR, IUSHR, LUSHR, IAND, LAND, IOR, LOR,
* IXOR, LXOR, I2L, I2F, I2D, L2I, L2F, L2D, F2I, F2L,
* F2D, D2I, D2L, D2F, I2B, I2C, I2S, LCMP, FCMPL, FCMPG,
* DCMPL, DCMPG, IRETURN, LRETURN, FRETURN, DRETURN,
* ARETURN, RETURN, ARRAYLENGTH, ATHROW, MONITORENTER,
* or MONITOREXIT.
*/
void visitInsn(int opcode);

/**
* Visits an instruction with a single int operand.
*
* @param opcode the opcode of the instruction to be visited. This
* opcode is either BIPUSH, SIPUSH or NEWARRAY.
* @param operand the operand of the instruction to be visited.<br> When
* opcode is BIPUSH, operand value should be between
* Byte.MIN_VALUE and Byte.MAX_VALUE.<br> When opcode
* is SIPUSH, operand value should be between
* Short.MIN_VALUE and Short.MAX_VALUE.<br> When opcode
* is NEWARRAY, operand value should be one of
* {@link Opcodes#T_BOOLEAN}, {@link Opcodes#T_CHAR},
* {@link Opcodes#T_FLOAT}, {@link Opcodes#T_DOUBLE},
* {@link Opcodes#T_BYTE}, {@link Opcodes#T_SHORT},
* {@link Opcodes#T_INT} or {@link Opcodes#T_LONG}.
*/
void visitIntInsn(int opcode, int operand);

/**
* Visits a local variable instruction. A local variable instruction is
* an instruction that loads or stores the value of a local variable.
*
* @param opcode the opcode of the local variable instruction to be
* visited. This opcode is either ILOAD, LLOAD, FLOAD,
* DLOAD, ALOAD, ISTORE, LSTORE, FSTORE, DSTORE, ASTORE
* or RET.
* @param var the operand of the instruction to be visited. This
* operand is the index of a local variable.
*/
void visitVarInsn(int opcode, int var);
322 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Visits a type instruction. A type instruction is an instruction that
* takes a type descriptor as parameter.
*
* @param opcode the opcode of the type instruction to be visited.
* This opcode is either NEW, ANEWARRAY, CHECKCAST or
* INSTANCEOF.
* @param desc the operand of the instruction to be visited. This
* operand is must be a fully qualified class name in
* internal form, or the type descriptor of an array
* type (see {@link Type Type}).
*/
void visitTypeInsn(int opcode, String desc);

/**
* Visits a field instruction. A field instruction is an instruction
* that loads or stores the value of a field of an object.
*
* @param opcode the opcode of the type instruction to be visited.
* This opcode is either GETSTATIC, PUTSTATIC, GETFIELD
* or PUTFIELD.
* @param owner the internal name of the fields owner class (see {@link
* Type#getInternalName() getInternalName}).
* @param name the fields name.
* @param desc the fields descriptor (see {@link Type Type}).
*/
void visitFieldInsn(int opcode, String owner, String name, String desc);

/**
* Visits a method instruction. A method instruction is an instruction
* that invokes a method.
*
* @param opcode the opcode of the type instruction to be visited.
* This opcode is either INVOKEVIRTUAL, INVOKESPECIAL,
* INVOKESTATIC or INVOKEINTERFACE.
* @param owner the internal name of the methods owner class (see
* {@link Type#getInternalName() getInternalName}).
* @param name the methods name.
* @param desc the methods descriptor (see {@link Type Type}).
*/
void visitMethodInsn(int opcode, String owner, String name, String desc);

/**
* Visits a jump instruction. A jump instruction is an instruction that
* may jump to another instruction.
*
* @param opcode the opcode of the type instruction to be visited. This
* opcode is either IFEQ, IFNE, IFLT, IFGE, IFGT, IFLE,
* IF_ICMPEQ, IF_ICMPNE, IF_ICMPLT, IF_ICMPGE, IF_ICMPGT,
* IF_ICMPLE, IF_ACMPEQ, IF_ACMPNE, GOTO, JSR, IFNULL or
7.17. METHODVISITOR.JAVA 323

* IFNONNULL.
* @param label the operand of the instruction to be visited. This
* operand is a label that designates the instruction
* to which the jump instruction may jump.
*/
void visitJumpInsn(int opcode, Label label);

/**
* Visits a label. A label designates the instruction that will be
* visited just after it.
*
* @param label a {@link Label Label} object.
*/
void visitLabel(Label label);

// --------------------------------------------------------------------
// Special instructions
// --------------------------------------------------------------------

/**
* Visits a LDC instruction.
*
* @param cst the constant to be loaded on the stack. This parameter
* must be a non null {@link Integer}, a {@link Float},
* a {@link Long}, a {@link Double} a {@link String} (or
* a {@link Type} for <tt>.class</tt> constants, for classes
* whose version is 49.0 or more).
*/
void visitLdcInsn(Object cst);

/**
* Visits an IINC instruction.
*
* @param var index of the local variable to be incremented.
* @param increment amount to increment the local variable by.
*/
void visitIincInsn(int var, int increment);

/**
* Visits a TABLESWITCH instruction.
*
* @param min the minimum key value.
* @param max the maximum key value.
* @param dflt beginning of the default handler block.
* @param labels beginnings of the handler blocks. <tt>labels[i]</tt>
* is the beginning of the handler block for the
* <tt>min + i</tt> key.
*/
void visitTableSwitchInsn(int min, int max, Label dflt, Label labels[]);
324 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Visits a LOOKUPSWITCH instruction.
*
* @param dflt beginning of the default handler block.
* @param keys the values of the keys.
* @param labels beginnings of the handler blocks. <tt>labels[i]</tt>
* is the beginning of the handler block for the
* <tt>keys[i]</tt> key.
*/
void visitLookupSwitchInsn(Label dflt, int keys[], Label labels[]);

/**
* Visits a MULTIANEWARRAY instruction.
*
* @param desc an array type descriptor (see {@link Type Type}).
* @param dims number of dimensions of the array to allocate.
*/
void visitMultiANewArrayInsn(String desc, int dims);

// ---------------------------------------------------------------
// Exceptions table entries, debug information, max stack and max
// locals
// ---------------------------------------------------------------

/**
* Visits a try catch block.
*
* @param start beginning of the exception handlers scope (inclusive).
* @param end end of the exception handlers scope (exclusive).
* @param handler beginning of the exception handlers code.
* @param type internal name of the type of exceptions handled by the
* handler, or <tt>null</tt> to catch any exceptions
* (for "finally" blocks).
* @throws IllegalArgumentException if one of the labels has already
* been visited by this visitor (by the
* {@link #visitLabel visitLabel} method).
*/
void visitTryCatchBlock(Label start,
Label end,
Label handler,
String type);

/**
* Visits a local variable declaration.
*
* @param name the name of a local variable.
* @param desc the type descriptor of this local variable.
* @param signature the type signature of this local variable. May be
* <tt>null</tt> if the local variable type does not
* use generic types.
7.17. METHODVISITOR.JAVA 325

* @param start the first instruction corresponding to the scope of

* this local variable (inclusive).
* @param end the last instruction corresponding to the scope of
* this local variable (exclusive).
* @param index the local variables index.
* @throws IllegalArgumentException if one of the labels has not already
* been visited by this visitor (by the
* {@link #visitLabel visitLabel} method).
*/
void visitLocalVariable(
String name,
String desc,
String signature,
Label start,
Label end,
int index);

/**
* Visits a line number declaration.
*
* @param line a line number. This number refers to the source file
* from which the class was compiled.
* @param start the first instruction corresponding to this line number.
* @throws IllegalArgumentException if <tt>start</tt> has not already
* been visited by this visitor (by the
* {@link #visitLabel visitLabel} method).
*/
void visitLineNumber(int line, Label start);

/**
* Visits the maximum stack size and the maximum number of local
* variables of the method.
*
* @param maxStack maximum stack size of the method.
* @param maxLocals maximum number of local variables for the method.
*/
void visitMaxs(int maxStack, int maxLocals);

/**
* Visits the end of the method. This method, which is the last one to
* be called, is used to inform the visitor that all the annotations and
* attributes of the method have been visited.
*/
void visitEnd();
}

-
326 CHAPTER 7. JVM/CLOJURE/ASM/

7.18 MethodWriter.java
(MethodVisitor [317])
MethodWriter.java

\getchunk{France Telecom Copyright}

package clojure.asm;

/**
* A {@link MethodVisitor} that generates methods in bytecode form.
* Each visit method of this class appends the bytecode corresponding
* to the visited instruction to a byte vector, in the order these
* methods are called.
*
* @author Eric Bruneton
* @author Eugene Kuleshov
*/
class MethodWriter implements MethodVisitor{

/**
* Pseudo access flag used to denote constructors.
*/
final static int ACC_CONSTRUCTOR = 262144;

/**
* Frame has exactly the same locals as the previous stack map frame
* and number of stack items is zero.
*/
final static int SAME_FRAME = 0; // to 63 (0-3f)

/**
* Frame has exactly the same locals as the previous stack map frame
* and number of stack items is 1
*/
final static int SAME_LOCALS_1_STACK_ITEM_FRAME = 64; // to 127 (40-7f)

/**
* Reserved for future use
*/
final static int RESERVED = 128;

/**
* Frame has exactly the same locals as the previous stack map frame
* and number of stack items is 1. Offset is bigger then 63;
*/
final static int SAME_LOCALS_1_STACK_ITEM_FRAME_EXTENDED = 247; // f7

/**
* Frame where current locals are the same as the locals in the previous
7.18. METHODWRITER.JAVA 327

* frame, except that the k last locals are absent. The value of k is
* given by the formula 251-frame_type.
*/
final static int CHOP_FRAME = 248; // to 250 (f8-fA)

/**
* Frame has exactly the same locals as the previous stack map frame
* and number of stack items is zero. Offset is bigger then 63;
*/
final static int SAME_FRAME_EXTENDED = 251; // fb

/**
* Frame where current locals are the same as the locals in the
* previous frame, except that k additional locals are defined.
* The value of k is given by the formula frame_type-251.
*/
final static int APPEND_FRAME = 252; // to 254 // fc-fe

/**
* Full frame
*/
final static int FULL_FRAME = 255; // ff

/**
* Indicates that the stack map frames must be recomputed from scratch.
* In this case the maximum stack size and number of local variables
* is also recomputed from scratch.
*
* @see #compute
*/
private final static int FRAMES = 0;

/**
* Indicates that the maximum stack size and number of local variables
* must be automatically computed.
*
* @see #compute
*/
private final static int MAXS = 1;

/**
* Indicates that nothing must be automatically computed.
*
* @see #compute
*/
private final static int NOTHING = 2;

/**
* Next method writer (see {@link ClassWriter#firstMethod firstMethod}).
*/
328 CHAPTER 7. JVM/CLOJURE/ASM/

MethodWriter next;

/**
* The class writer to which this method must be added.
*/
ClassWriter cw;

/**
* Access flags of this method.
*/
private int access;

/**
* The index of the constant pool item that contains the name of this
* method.
*/
private int name;

/**
* The index of the constant pool item that contains the descriptor
* of this method.
*/
private int desc;

/**
* The descriptor of this method.
*/
private String descriptor;

/**
* The signature of this method.
*/
String signature;

/**
* If not zero, indicates that the code of this method must be copied
* from the ClassReader associated to this writer in
* <code>cw.cr</code>. More precisely, this field gives the index of
* the first byte to copied from <code>cw.cr.b</code>.
*/
int classReaderOffset;

/**
* If not zero, indicates that the code of this method must be
* copied from the ClassReader associated to this writer in
* <code>cw.cr</code>. More precisely, this field gives the number
* of bytes to copied from <code>cw.cr.b</code>.
*/
int classReaderLength;
7.18. METHODWRITER.JAVA 329

/**
* Number of exceptions that can be thrown by this method.
*/
int exceptionCount;

/**
* The exceptions that can be thrown by this method. More precisely,
* this array contains the indexes of the constant pool items that
* contain the internal names of these exception classes.
*/
int[] exceptions;

/**
* The annotation default attribute of this method. May be
* <tt>null</tt>.
*/
private ByteVector annd;

/**
* The runtime visible annotations of this method. May be
* <tt>null</tt>.
*/
private AnnotationWriter anns;

/**
* The runtime invisible annotations of this method. May be
* <tt>null</tt>.
*/
private AnnotationWriter ianns;

/**
* The runtime visible parameter annotations of this method. May be
* <tt>null</tt>.
*/
private AnnotationWriter[] panns;

/**
* The runtime invisible parameter annotations of this method. May be
* <tt>null</tt>.
*/
private AnnotationWriter[] ipanns;

/**
* The non standard attributes of the method.
*/
private Attribute attrs;

/**
* The bytecode of this method.
*/
330 CHAPTER 7. JVM/CLOJURE/ASM/

private ByteVector code = new ByteVector();

/**
* Maximum stack size of this method.
*/
private int maxStack;

/**
* Maximum number of local variables for this method.
*/
private int maxLocals;

/**
* Number of stack map frames in the StackMapTable attribute.
*/
private int frameCount;

/**
* The StackMapTable attribute.
*/
private ByteVector stackMap;

/**
* The offset of the last frame that was written in the StackMapTable
* attribute.
*/
private int previousFrameOffset;

/**
* The last frame that was written in the StackMapTable attribute.
*
* @see #frame
*/
private int[] previousFrame;

/**
* Index of the next element to be added in {@link #frame}.
*/
private int frameIndex;

/**
* The current stack map frame. The first element contains the offset
* of the instruction to which the frame corresponds, the second
* element is the number of locals and the third one is the number
* of stack elements. The local variables start at index 3 and are
* followed by the operand stack values. In summary frame[0] = offset,
* frame[1] = nLocal, frame[2] = nStack, frame[3] = nLocal. All types
* are encoded as integers, with the same format as the one used in
* {@link Label}, but limited to BASE types.
*/
7.18. METHODWRITER.JAVA 331

private int[] frame;

/**
* Number of elements in the exception handler list.
*/
private int handlerCount;

/**
* The first element in the exception handler list.
*/
private Handler firstHandler;

/**
* The last element in the exception handler list.
*/
private Handler lastHandler;

/**
* Number of entries in the LocalVariableTable attribute.
*/
private int localVarCount;

/**
* The LocalVariableTable attribute.
*/
private ByteVector localVar;

/**
* Number of entries in the LocalVariableTypeTable attribute.
*/
private int localVarTypeCount;

/**
* The LocalVariableTypeTable attribute.
*/
private ByteVector localVarType;

/**
* Number of entries in the LineNumberTable attribute.
*/
private int lineNumberCount;

/**
* The LineNumberTable attribute.
*/
private ByteVector lineNumber;

/**
* The non standard attributes of the methods code.
*/
332 CHAPTER 7. JVM/CLOJURE/ASM/

private Attribute cattrs;

/**
* Indicates if some jump instructions are too small and need to be
* resized.
*/
private boolean resize;

/**
* Indicates if the instructions contain at least one JSR instruction.
*/
private boolean jsr;

// -------------------------------------------------------------------

/*
* Fields for the control flow graph analysis algorithm (used to
* compute the maximum stack size). A control flow graph contains
* one node per "basic block", and one edge per "jump" from one basic
* block to another. Each node (i.e., each basic block) is represented
* by the Label object that corresponds to the first instruction of
* this basic block. Each node also stores the list of its successors
* in the graph, as a linked list of Edge objects.
*/

/**
* Indicates what must be automatically computed.
*
* @see FRAMES
* @see MAXS
* @see NOTHING
*/
private int compute;

/**
* A list of labels. This list is the list of basic blocks in the
* method, i.e. a list of Label objects linked to each other by their
* {@link Label#successor} field, in the order they are visited by
* {@link visitLabel}, and starting with the first basic block.
*/
private Label labels;

/**
* The previous basic block.
*/
private Label previousBlock;

/**
* The current basic block.
*/
7.18. METHODWRITER.JAVA 333

private Label currentBlock;

/**
* The (relative) stack size after the last visited instruction. This
* size is relative to the beginning of the current basic block, i.e.,
* the true stack size after the last visited instruction is equal to
* the {@link Label#inputStackTop beginStackSize} of the current basic
* block plus <tt>stackSize</tt>.
*/
private int stackSize;

/**
* The (relative) maximum stack size after the last visited instruction.
* This size is relative to the beginning of the current basic block,
* i.e., the true maximum stack size after the last visited instruction
* is equal to the {@link Label#inputStackTop beginStackSize} of the
* current basic block plus <tt>stackSize</tt>.
*/
private int maxStackSize;

// -------------------------------------------------------------------
// Constructor
// -------------------------------------------------------------------

/**
* Constructs a new {@link MethodWriter}.
*
* @param cw the class writer in which the method must be
* added.
* @param access the methods access flags (see {@link Opcodes}).
* @param name the methods name.
* @param desc the methods descriptor (see {@link Type}).
* @param signature the methods signature. May be <tt>null</tt>.
* @param exceptions the internal names of the methods exceptions.
* May be <tt>null</tt>.
* @param computeMaxs <tt>true</tt> if the maximum stack size and
* number of local variables must be automatically
* computed.
* @param computeFrames <tt>true</tt> if the stack map tables must be
* recomputed from scratch.
*/
MethodWriter(
final ClassWriter cw,
final int access,
final String name,
final String desc,
final String signature,
final String[] exceptions,
final boolean computeMaxs,
final boolean computeFrames){
334 CHAPTER 7. JVM/CLOJURE/ASM/

if(cw.firstMethod == null)
{
cw.firstMethod = this;
}
else
{
cw.lastMethod.next = this;
}
cw.lastMethod = this;
this.cw = cw;
this.access = access;
this.name = cw.newUTF8(name);
this.desc = cw.newUTF8(desc);
this.descriptor = desc;
this.signature = signature;
if(exceptions != null && exceptions.length > 0)
{
exceptionCount = exceptions.length;
this.exceptions = new int[exceptionCount];
for(int i = 0; i < exceptionCount; ++i)
{
this.exceptions[i] = cw.newClass(exceptions[i]);
}
}
this.compute =
computeFrames ? FRAMES : (computeMaxs ? MAXS : NOTHING);
if(computeMaxs || computeFrames)
{
if(computeFrames && name.equals("<init>"))
{
this.access |= ACC_CONSTRUCTOR;
}
// updates maxLocals
int size = getArgumentsAndReturnSizes(descriptor) >> 2;
if((access & Opcodes.ACC_STATIC) != 0)
{
--size;
}
maxLocals = size;
// creates and visits the label for the first basic block
labels = new Label();
labels.status |= Label.PUSHED;
visitLabel(labels);
}
}

// -------------------------------------------------------------------
// Implementation of the MethodVisitor interface
// -------------------------------------------------------------------
7.18. METHODWRITER.JAVA 335

public AnnotationVisitor visitAnnotationDefault(){

annd = new ByteVector();
return new AnnotationWriter(cw, false, annd, null, 0);
}

public AnnotationVisitor visitAnnotation(

final String desc,
final boolean visible){
ByteVector bv = new ByteVector();
// write type, and reserve space for values count
bv.putShort(cw.newUTF8(desc)).putShort(0);
AnnotationWriter aw = new AnnotationWriter(cw, true, bv, bv, 2);
if(visible)
{
aw.next = anns;
anns = aw;
}
else
{
aw.next = ianns;
ianns = aw;
}
return aw;
}

public AnnotationVisitor visitParameterAnnotation(

final int parameter,
final String desc,
final boolean visible){
ByteVector bv = new ByteVector();
// write type, and reserve space for values count
bv.putShort(cw.newUTF8(desc)).putShort(0);
AnnotationWriter aw = new AnnotationWriter(cw, true, bv, bv, 2);
if(visible)
{
if(panns == null)
{
panns =
new AnnotationWriter[
Type.getArgumentTypes(descriptor).length];
}
aw.next = panns[parameter];
panns[parameter] = aw;
}
else
{
if(ipanns == null)
{
ipanns =
new AnnotationWriter[
336 CHAPTER 7. JVM/CLOJURE/ASM/

Type.getArgumentTypes(descriptor).length];
}
aw.next = ipanns[parameter];
ipanns[parameter] = aw;
}
return aw;
}

public void visitAttribute(final Attribute attr){

if(attr.isCodeAttribute())
{
attr.next = cattrs;
cattrs = attr;
}
else
{
attr.next = attrs;
attrs = attr;
}
}

public void visitCode(){

}

public void visitFrame(

final int type,
final int nLocal,
final Object[] local,
final int nStack,
final Object[] stack){
if(compute == FRAMES)
{
return;
}

if(type == Opcodes.F_NEW)
{
startFrame(code.length, nLocal, nStack);
for(int i = 0; i < nLocal; ++i)
{
if(local[i] instanceof String)
{
frame[frameIndex++] = Frame.OBJECT
| cw.addType((String) local[i]);
}
else if(local[i] instanceof Integer)
{
frame[frameIndex++] = ((Integer) local[i]).intValue();
}
else
7.18. METHODWRITER.JAVA 337

{
frame[frameIndex++] =
Frame.UNINITIALIZED
| cw.addUninitializedType("",
((Label) local[i]).position);
}
}
for(int i = 0; i < nStack; ++i)
{
if(stack[i] instanceof String)
{
frame[frameIndex++] = Frame.OBJECT
| cw.addType((String) stack[i]);
}
else if(stack[i] instanceof Integer)
{
frame[frameIndex++] = ((Integer) stack[i]).intValue();
}
else
{
frame[frameIndex++] =
Frame.UNINITIALIZED
| cw.addUninitializedType("",
((Label) stack[i]).position);
}
}
endFrame();
}
else
{
int delta;
if(stackMap == null)
{
stackMap = new ByteVector();
delta = code.length;
}
else
{
delta = code.length - previousFrameOffset - 1;
}

switch(type)
{
case Opcodes.F_FULL:
stackMap.putByte(FULL_FRAME)
.putShort(delta)
.putShort(nLocal);
for(int i = 0; i < nLocal; ++i)
{
writeFrameType(local[i]);
338 CHAPTER 7. JVM/CLOJURE/ASM/

}
stackMap.putShort(nStack);
for(int i = 0; i < nStack; ++i)
{
writeFrameType(stack[i]);
}
break;
case Opcodes.F_APPEND:
stackMap.putByte(SAME_FRAME_EXTENDED + nLocal)
.putShort(delta);
for(int i = 0; i < nLocal; ++i)
{
writeFrameType(local[i]);
}
break;
case Opcodes.F_CHOP:
stackMap.putByte(SAME_FRAME_EXTENDED - nLocal)
.putShort(delta);
break;
case Opcodes.F_SAME:
if(delta < 64)
{
stackMap.putByte(delta);
}
else
{
stackMap.putByte(SAME_FRAME_EXTENDED).putShort(delta);
}
break;
case Opcodes.F_SAME1:
if(delta < 64)
{
stackMap.putByte(
SAME_LOCALS_1_STACK_ITEM_FRAME + delta);
}
else
{
stackMap.putByte(
SAME_LOCALS_1_STACK_ITEM_FRAME_EXTENDED)
.putShort(delta);
}
writeFrameType(stack[0]);
break;
}

previousFrameOffset = code.length;
++frameCount;
}
}
7.18. METHODWRITER.JAVA 339

public void visitInsn(final int opcode){

// adds the instruction to the bytecode of the method
code.putByte(opcode);
// update currentBlock
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, 0, null, null);
}
else
{
// updates current and max stack sizes
int size = stackSize + Frame.SIZE[opcode];
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
// if opcode == ATHROW or xRETURN, ends current
// block (no successor)
if((opcode >= Opcodes.IRETURN && opcode <= Opcodes.RETURN)
|| opcode == Opcodes.ATHROW)
{
noSuccessor();
}
}
}

public void visitIntInsn(final int opcode, final int operand){

// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, operand, null, null);
}
else if(opcode != Opcodes.NEWARRAY)
{
// updates current and max stack sizes only for NEWARRAY
// (stack size variation = 0 for BIPUSH or SIPUSH)
int size = stackSize + 1;
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
340 CHAPTER 7. JVM/CLOJURE/ASM/

}
// adds the instruction to the bytecode of the method
if(opcode == Opcodes.SIPUSH)
{
code.put12(opcode, operand);
}
else
{ // BIPUSH or NEWARRAY
code.put11(opcode, operand);
}
}

public void visitVarInsn(final int opcode, final int var){

// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, var, null, null);
}
else
{
// updates current and max stack sizes
if(opcode == Opcodes.RET)
{
// no stack change, but end of current block
// (no successor)
currentBlock.status |= Label.RET;
// save stackSize here for future use
// (see {@link #findSubroutineSuccessors})
currentBlock.inputStackTop = stackSize;
noSuccessor();
}
else
{ // xLOAD or xSTORE
int size = stackSize + Frame.SIZE[opcode];
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
}
}
if(compute != NOTHING)
{
// updates max locals
int n;
if(opcode == Opcodes.LLOAD || opcode == Opcodes.DLOAD
|| opcode == Opcodes.LSTORE || opcode == Opcodes.DSTORE)
7.18. METHODWRITER.JAVA 341

{
n = var + 2;
}
else
{
n = var + 1;
}
if(n > maxLocals)
{
maxLocals = n;
}
}
// adds the instruction to the bytecode of the method
if(var < 4 && opcode != Opcodes.RET)
{
int opt;
if(opcode < Opcodes.ISTORE)
{
/* ILOAD_0 */
opt = 26 + ((opcode - Opcodes.ILOAD) << 2) + var;
}
else
{
/* ISTORE_0 */
opt = 59 + ((opcode - Opcodes.ISTORE) << 2) + var;
}
code.putByte(opt);
}
else if(var >= 256)
{
code.putByte(196 /* WIDE */).put12(opcode, var);
}
else
{
code.put11(opcode, var);
}
if(opcode >= Opcodes.ISTORE && compute == FRAMES && handlerCount > 0)
{
visitLabel(new Label());
}
}

public void visitTypeInsn(final int opcode, final String desc){

Item i = cw.newClassItem(desc);
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, code.length, cw, i);
342 CHAPTER 7. JVM/CLOJURE/ASM/

}
else if(opcode == Opcodes.NEW)
{
// updates current and max stack sizes only if opcode == NEW
// (no stack change for ANEWARRAY, CHECKCAST, INSTANCEOF)
int size = stackSize + 1;
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
}
// adds the instruction to the bytecode of the method
code.put12(opcode, i.index);
}

public void visitFieldInsn(

final int opcode,
final String owner,
final String name,
final String desc){
Item i = cw.newFieldItem(owner, name, desc);
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, 0, cw, i);
}
else
{
int size;
// computes the stack size variation
char c = desc.charAt(0);
switch(opcode)
{
case Opcodes.GETSTATIC:
size = stackSize + (c == D || c == J ? 2 : 1);
break;
case Opcodes.PUTSTATIC:
size = stackSize + (c == D || c == J ? -2 : -1);
break;
case Opcodes.GETFIELD:
size = stackSize + (c == D || c == J ? 1 : 0);
break;
// case Constants.PUTFIELD:
default:
size = stackSize + (c == D || c == J ? -3 : -2);
break;
7.18. METHODWRITER.JAVA 343

}
// updates current and max stack sizes
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
}
// adds the instruction to the bytecode of the method
code.put12(opcode, i.index);
}

public void visitMethodInsn(

final int opcode,
final String owner,
final String name,
final String desc){
boolean itf = opcode == Opcodes.INVOKEINTERFACE;
Item i = cw.newMethodItem(owner, name, desc, itf);
int argSize = i.intVal;
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, 0, cw, i);
}
else
{
/*
* computes the stack size variation. In order not to
* recompute several times this variation for the same
* Item, we use the intVal field of this item to store
* this variation, once it has been computed. More
* precisely this intVal field stores the sizes of the
* arguments and of the return value corresponding to desc.
*/
if(argSize == 0)
{
// the above sizes have not been computed yet,
// so we compute them...
argSize = getArgumentsAndReturnSizes(desc);
// ... and we save them in order
// not to recompute them in the future
i.intVal = argSize;
}
int size;
if(opcode == Opcodes.INVOKESTATIC)
{
344 CHAPTER 7. JVM/CLOJURE/ASM/

size = stackSize - (argSize >> 2) + (argSize & 0x03) + 1;

}
else
{
size = stackSize - (argSize >> 2) + (argSize & 0x03);
}
// updates current and max stack sizes
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
}
// adds the instruction to the bytecode of the method
if(itf)
{
if(argSize == 0)
{
argSize = getArgumentsAndReturnSizes(desc);
i.intVal = argSize;
}
code.put12(Opcodes.INVOKEINTERFACE, i.index)
.put11(argSize >> 2, 0);
}
else
{
code.put12(opcode, i.index);
}
}

public void visitJumpInsn(final int opcode, final Label label){

Label nextInsn = null;
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(opcode, 0, null, null);
// label is the target of a jump instruction
label.getFirst().status |= Label.TARGET;
// adds label as a successor of this basic block
addSuccessor(Edge.NORMAL, label);
if(opcode != Opcodes.GOTO)
{
// creates a Label for the next basic block
nextInsn = new Label();
}
}
else
7.18. METHODWRITER.JAVA 345

{
if(opcode == Opcodes.JSR)
{
jsr = true;
currentBlock.status |= Label.JSR;
addSuccessor(stackSize + 1, label);
// creates a Label for the next basic block
nextInsn = new Label();
/*
* note that, by construction in this method, a JSR block
* has at least two successors in the control flow graph:
* the first one leads the next instruction after the
* JSR, while the second one leads to the JSR target.
*/
}
else
{
// updates current stack size (max stack size unchanged
// because stack size variation always negative in this
// case)
stackSize += Frame.SIZE[opcode];
addSuccessor(stackSize, label);
}
}
}
// adds the instruction to the bytecode of the method
if((label.status & Label.RESOLVED) != 0
&& label.position - code.length < Short.MIN_VALUE)
{
/*
* case of a backward jump with an offset < -32768. In this case
* we automatically replace GOTO with GOTO_W, JSR with JSR_W
* and IFxxx <l> with IFNOTxxx <l> GOTO_W <l>, where IFNOTxxx
* is the "opposite" opcode of IFxxx (i.e., IFNE for IFEQ) and
* where <l> designates the instruction just after the GOTO_W.
*/
if(opcode == Opcodes.GOTO)
{
code.putByte(200); // GOTO_W
}
else if(opcode == Opcodes.JSR)
{
code.putByte(201); // JSR_W
}
else
{
// if the IF instruction is transformed into IFNOT GOTO_W the
// next instruction becomes the target of the IFNOT
// instruction
if(nextInsn != null)
346 CHAPTER 7. JVM/CLOJURE/ASM/

{
nextInsn.status |= Label.TARGET;
}
code.putByte(opcode <= 166
? ((opcode + 1) ^ 1) - 1
: opcode ^ 1);
code.putShort(8); // jump offset
code.putByte(200); // GOTO_W
}
label.put(this, code, code.length - 1, true);
}
else
{
/*
* case of a backward jump with an offset >= -32768, or of a
* forward jump with, of course, an unknown offset. In these
* cases we store the offset in 2 bytes (which will be
* increased in resizeInstructions, if needed).
*/
code.putByte(opcode);
label.put(this, code, code.length - 1, false);
}
if(currentBlock != null)
{
if(nextInsn != null)
{
// if the jump instruction is not a GOTO, the next
// instruction is also a successor of this instruction.
// Calling visitLabel adds the label of this next
// instruction as a successor of the current block,
// and starts a new basic block
visitLabel(nextInsn);
}
if(opcode == Opcodes.GOTO)
{
noSuccessor();
}
}
}

public void visitLabel(final Label label){

// resolves previous forward references to label, if any
resize |= label.resolve(this, code.length, code.data);
// updates currentBlock
if((label.status & Label.DEBUG) != 0)
{
return;
}
if(compute == FRAMES)
{
7.18. METHODWRITER.JAVA 347

if(currentBlock != null)
{
if(label.position == currentBlock.position)
{
// successive labels, do not start a new basic block
currentBlock.status |= (label.status & Label.TARGET);
label.frame = currentBlock.frame;
return;
}
// ends current block (with one new successor)
addSuccessor(Edge.NORMAL, label);
}
// begins a new current block
currentBlock = label;
if(label.frame == null)
{
label.frame = new Frame();
label.frame.owner = label;
}
// updates the basic block list
if(previousBlock != null)
{
if(label.position == previousBlock.position)
{
previousBlock.status |= (label.status & Label.TARGET);
label.frame = previousBlock.frame;
currentBlock = previousBlock;
return;
}
previousBlock.successor = label;
}
previousBlock = label;
}
else if(compute == MAXS)
{
if(currentBlock != null)
{
// ends current block (with one new successor)
currentBlock.outputStackMax = maxStackSize;
addSuccessor(stackSize, label);
}
// begins a new current block
currentBlock = label;
// resets the relative current and max stack sizes
stackSize = 0;
maxStackSize = 0;
// updates the basic block list
if(previousBlock != null)
{
previousBlock.successor = label;
348 CHAPTER 7. JVM/CLOJURE/ASM/

}
previousBlock = label;
}
}

public void visitLdcInsn(final Object cst){

Item i = cw.newConstItem(cst);
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(Opcodes.LDC, 0, cw, i);
}
else
{
int size;
// computes the stack size variation
if(i.type == ClassWriter.LONG ||
i.type == ClassWriter.DOUBLE)
{
size = stackSize + 2;
}
else
{
size = stackSize + 1;
}
// updates current and max stack sizes
if(size > maxStackSize)
{
maxStackSize = size;
}
stackSize = size;
}
}
// adds the instruction to the bytecode of the method
int index = i.index;
if(i.type == ClassWriter.LONG || i.type == ClassWriter.DOUBLE)
{
code.put12(20 /* LDC2_W */, index);
}
else if(index >= 256)
{
code.put12(19 /* LDC_W */, index);
}
else
{
code.put11(Opcodes.LDC, index);
}
}
7.18. METHODWRITER.JAVA 349

public void visitIincInsn(final int var, final int increment){

if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame.execute(Opcodes.IINC, var, null, null);
}
}
if(compute != NOTHING)
{
// updates max locals
int n = var + 1;
if(n > maxLocals)
{
maxLocals = n;
}
}
// adds the instruction to the bytecode of the method
if((var > 255) || (increment > 127) || (increment < -128))
{
code.putByte(196 /* WIDE */)
.put12(Opcodes.IINC, var)
.putShort(increment);
}
else
{
code.putByte(Opcodes.IINC).put11(var, increment);
}
}

public void visitTableSwitchInsn(

final int min,
final int max,
final Label dflt,
final Label labels[]){
// adds the instruction to the bytecode of the method
int source = code.length;
code.putByte(Opcodes.TABLESWITCH);
code.length += (4 - code.length % 4) % 4;
dflt.put(this, code, source, true);
code.putInt(min).putInt(max);
for(int i = 0; i < labels.length; ++i)
{
labels[i].put(this, code, source, true);
}
// updates currentBlock
visitSwitchInsn(dflt, labels);
}
350 CHAPTER 7. JVM/CLOJURE/ASM/

public void visitLookupSwitchInsn(

final Label dflt,
final int keys[],
final Label labels[]){
// adds the instruction to the bytecode of the method
int source = code.length;
code.putByte(Opcodes.LOOKUPSWITCH);
code.length += (4 - code.length % 4) % 4;
dflt.put(this, code, source, true);
code.putInt(labels.length);
for(int i = 0; i < labels.length; ++i)
{
code.putInt(keys[i]);
labels[i].put(this, code, source, true);
}
// updates currentBlock
visitSwitchInsn(dflt, labels);
}

private void visitSwitchInsn(final Label dflt, final Label[] labels){

// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame
.execute(Opcodes.LOOKUPSWITCH, 0, null, null);
// adds current block successors
addSuccessor(Edge.NORMAL, dflt);
dflt.getFirst().status |= Label.TARGET;
for(int i = 0; i < labels.length; ++i)
{
addSuccessor(Edge.NORMAL, labels[i]);
labels[i].getFirst().status |= Label.TARGET;
}
}
else
{
// updates current stack size (max stack size unchanged)
--stackSize;
// adds current block successors
addSuccessor(stackSize, dflt);
for(int i = 0; i < labels.length; ++i)
{
addSuccessor(stackSize, labels[i]);
}
}
// ends current block
noSuccessor();
}
7.18. METHODWRITER.JAVA 351

public void visitMultiANewArrayInsn(final String desc, final int dims){

Item i = cw.newClassItem(desc);
// Label currentBlock = this.currentBlock;
if(currentBlock != null)
{
if(compute == FRAMES)
{
currentBlock.frame
.execute(Opcodes.MULTIANEWARRAY, dims, cw, i);
}
else
{
// updates current stack size (max stack size unchanged
// because stack size variation always negative or null)
stackSize += 1 - dims;
}
}
// adds the instruction to the bytecode of the method
code.put12(Opcodes.MULTIANEWARRAY, i.index).putByte(dims);
}

public void visitTryCatchBlock(

final Label start,
final Label end,
final Label handler,
final String type){
++handlerCount;
Handler h = new Handler();
h.start = start;
h.end = end;
h.handler = handler;
h.desc = type;
h.type = type != null ? cw.newClass(type) : 0;
if(lastHandler == null)
{
firstHandler = h;
}
else
{
lastHandler.next = h;
}
lastHandler = h;
}

public void visitLocalVariable(

final String name,
final String desc,
final String signature,
352 CHAPTER 7. JVM/CLOJURE/ASM/

final Label start,

final Label end,
final int index){
if(signature != null)
{
if(localVarType == null)
{
localVarType = new ByteVector();
}
++localVarTypeCount;
localVarType.putShort(start.position)
.putShort(end.position - start.position)
.putShort(cw.newUTF8(name))
.putShort(cw.newUTF8(signature))
.putShort(index);
}
if(localVar == null)
{
localVar = new ByteVector();
}
++localVarCount;
localVar.putShort(start.position)
.putShort(end.position - start.position)
.putShort(cw.newUTF8(name))
.putShort(cw.newUTF8(desc))
.putShort(index);
if(compute != NOTHING)
{
// updates max locals
char c = desc.charAt(0);
int n = index + (c == J || c == D ? 2 : 1);
if(n > maxLocals)
{
maxLocals = n;
}
}
}

public void visitLineNumber(final int line, final Label start){

if(lineNumber == null)
{
lineNumber = new ByteVector();
}
++lineNumberCount;
lineNumber.putShort(start.position);
lineNumber.putShort(line);
}

public void visitMaxs(final int maxStack, final int maxLocals){

if(compute == FRAMES)
7.18. METHODWRITER.JAVA 353

{
// completes the control flow graph with exception handler blocks
Handler handler = firstHandler;
while(handler != null)
{
Label l = handler.start.getFirst();
Label h = handler.handler.getFirst();
Label e = handler.end.getFirst();
// computes the kind of the edges to h
String t = handler.desc == null
? "java/lang/Throwable"
: handler.desc;
int kind = Frame.OBJECT | cw.addType(t);
// h is an exception handler
h.status |= Label.TARGET;
// adds h as a successor of labels between start
// and end
while(l != e)
{
// creates an edge to h
Edge b = new Edge();
b.info = kind;
b.successor = h;
// adds it to the successors of l
b.next = l.successors;
l.successors = b;
// goes to the next label
l = l.successor;
}
handler = handler.next;
}

// creates and visits the first (implicit) frame

Frame f = labels.frame;
Type[] args = Type.getArgumentTypes(descriptor);
f.initInputFrame(cw, access, args, this.maxLocals);
visitFrame(f);

/*
* fix point algorithm: mark the first basic block as changed
* (i.e. put it in the changed list) and, while there are
* changed basic blocks, choose one, mark it as unchanged,
* and update its successors (which can be changed in the
* process).
*/
int max = 0;
Label changed = labels;
while(changed != null)
{
// removes a basic block from the list of changed basic
354 CHAPTER 7. JVM/CLOJURE/ASM/

// blocks
Label l = changed;
changed = changed.next;
l.next = null;
f = l.frame;
// a reacheable jump target must be stored in the stack map
if((l.status & Label.TARGET) != 0)
{
l.status |= Label.STORE;
}
// all visited labels are reacheable, by definition
l.status |= Label.REACHABLE;
// updates the (absolute) maximum stack size
int blockMax = f.inputStack.length + l.outputStackMax;
if(blockMax > max)
{
max = blockMax;
}
// updates the successors of the current basic block
Edge e = l.successors;
while(e != null)
{
Label n = e.successor.getFirst();
boolean change = f.merge(cw, n.frame, e.info);
if(change && n.next == null)
{
// if n has changed and is not already in the
// changed list, adds it to this list
n.next = changed;
changed = n;
}
e = e.next;
}
}
this.maxStack = max;

// visits all the frames that must be stored in the stack map
Label l = labels;
while(l != null)
{
f = l.frame;
if((l.status & Label.STORE) != 0)
{
visitFrame(f);
}
if((l.status & Label.REACHABLE) == 0)
{
// finds start and end of dead basic block
Label k = l.successor;
int start = l.position;
7.18. METHODWRITER.JAVA 355

int end = (k == null ? code.length : k.position) - 1;

// if non empty basic block
if(end >= start)
{
// replaces instructions with NOP ... NOP ATHROW
for(int i = start; i < end; ++i)
{
code.data[i] = Opcodes.NOP;
}
code.data[end] = (byte) Opcodes.ATHROW;
// emits a frame for this unreachable block
startFrame(start, 0, 1);
frame[frameIndex++] =
Frame.OBJECT
| cw.addType("java/lang/Throwable");
endFrame();
}
}
l = l.successor;
}
}
else if(compute == MAXS)
{
// completes the control flow graph with exception handler blocks
Handler handler = firstHandler;
while(handler != null)
{
Label l = handler.start;
Label h = handler.handler;
Label e = handler.end;
// adds h as a successor of labels between start
// and end
while(l != e)
{
// creates an edge to h
Edge b = new Edge();
b.info = Edge.EXCEPTION;
b.successor = h;
// adds it to the successors of l
if((l.status & Label.JSR) != 0)
{
// if l is a JSR block, adds b after the first two
// edges to preserve the hypothesis about JSR block
// successors order (see {@link #visitJumpInsn})
b.next = l.successors.next.next;
l.successors.next.next = b;
}
else
{
b.next = l.successors;
356 CHAPTER 7. JVM/CLOJURE/ASM/

l.successors = b;
}
// goes to the next label
l = l.successor;
}
handler = handler.next;
}

if(jsr)
{
// completes the control flow graph with the RET successors
/*
* first step: finds the subroutines. This step determines,
* for each basic block, to which subroutine(s) it belongs,
* and stores this set as a bit set in the
* {@link Label#status} field. Subroutines are numbered
* with powers of two, from 0x1000 to 0x80000000 (so there
* must be at most 20 subroutines in a method).
*/
// finds the basic blocks that belong to the "main"
// subroutine
int id = 0x1000;
findSubroutine(labels, id);
// finds the basic blocks that belong to the real subroutines
Label l = labels;
while(l != null)
{
if((l.status & Label.JSR) != 0)
{
// the subroutine is defined by ls TARGET, not by l
Label subroutine = l.successors.next.successor;
// if this subroutine does not have an id yet...
if((subroutine.status & ~0xFFF) == 0)
{
// ...assigns it a new id and finds its
// basic blocks
id = id << 1;
findSubroutine(subroutine, id);
}
}
l = l.successor;
}
// second step: finds the successors of RET blocks
findSubroutineSuccessors(0x1000, new Label[10], 0);
}

/*
* control flow analysis algorithm: while the block stack is not
* empty, pop a block from this stack, update the max stack size,
* compute the true (non relative) begin stack size of the
7.18. METHODWRITER.JAVA 357

* successors of this block, and push these successors onto the

* stack (unless they have already been pushed onto the stack).
* Note: by hypothesis, the {@link Label#inputStackTop} of the
* blocks in the block stack are the true (non relative)
* beginning stack sizes of these blocks.
*/
int max = 0;
Label stack = labels;
while(stack != null)
{
// pops a block from the stack
Label l = stack;
stack = stack.next;
// computes the true (non relative) max stack size of this
// block
int start = l.inputStackTop;
int blockMax = start + l.outputStackMax;
// updates the global max stack size
if(blockMax > max)
{
max = blockMax;
}
// analyses the successors of the block
Edge b = l.successors;
if((l.status & Label.JSR) != 0)
{
// ignores the first edge of JSR blocks
// (virtual successor)
b = b.next;
}
while(b != null)
{
l = b.successor;
// if this successor has not already been pushed...
if((l.status & Label.PUSHED) == 0)
{
// computes its true beginning stack size...
l.inputStackTop = b.info == Edge.EXCEPTION
? 1 : start + b.info;
// ...and pushes it onto the stack
l.status |= Label.PUSHED;
l.next = stack;
stack = l;
}
b = b.next;
}
}
this.maxStack = max;
}
else
358 CHAPTER 7. JVM/CLOJURE/ASM/

{
this.maxStack = maxStack;
this.maxLocals = maxLocals;
}
}

public void visitEnd(){

}

// -------------------------------------------------------------------
// Utility methods: control flow analysis algorithm
// -------------------------------------------------------------------

/**
* Computes the size of the arguments and of the return value of a
* method.
*
* @param desc the descriptor of a method.
* @return the size of the arguments of the method (plus one for the
* implicit this argument), argSize, and the size of its return
* value, retSize, packed into a single int i =
* <tt>(argSize << 2) | retSize</tt> (argSize is therefore equal
* to <tt>i >> 2</tt>, and retSize to <tt>i & 0x03</tt>).
*/
static int getArgumentsAndReturnSizes(final String desc){
int n = 1;
int c = 1;
while(true)
{
char car = desc.charAt(c++);
if(car == ))
{
car = desc.charAt(c);
return n << 2
| (car == V ? 0 : (car == D || car == J ? 2 : 1));
}
else if(car == L)
{
while(desc.charAt(c++) != ;)
{
}
n += 1;
}
else if(car == [)
{
while((car = desc.charAt(c)) == [)
{
++c;
}
if(car == D || car == J)
7.18. METHODWRITER.JAVA 359

{
n -= 1;
}
}
else if(car == D || car == J)
{
n += 2;
}
else
{
n += 1;
}
}
}

/**
* Adds a successor to the {@link #currentBlock currentBlock} block.
*
* @param info information about the control flow edge to be added.
* @param successor the successor block to be added to the current block.
*/
private void addSuccessor(final int info, final Label successor){
// creates and initializes an Edge object...
Edge b = new Edge();
b.info = info;
b.successor = successor;
// ...and adds it to the successor list of the currentBlock block
b.next = currentBlock.successors;
currentBlock.successors = b;
}

/**
* Ends the current basic block. This method must be used in the case
* where the current basic block does not have any successor.
*/
private void noSuccessor(){
if(compute == FRAMES)
{
Label l = new Label();
l.frame = new Frame();
l.frame.owner = l;
l.resolve(this, code.length, code.data);
previousBlock.successor = l;
previousBlock = l;
}
else
{
currentBlock.outputStackMax = maxStackSize;
}
currentBlock = null;
360 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Finds the basic blocks that belong to a given subroutine, and marks
* these blocks as belonging to this subroutine (by using
* {@link Label#status} as a bit set (see {@link #visitMaxs}). This
* recursive method follows the control flow graph to find all the
* blocks that are reachable from the given block WITHOUT following
* any JSR target.
*
* @param block a block that belongs to the subroutine
* @param id the id of this subroutine
*/
private void findSubroutine(final Label block, final int id){
// if block is already marked as belonging to subroutine id,
// returns
if((block.status & id) != 0)
{
return;
}
// marks block as belonging to subroutine id
block.status |= id;
// calls this method recursively on each successor, except
// JSR targets
Edge e = block.successors;
while(e != null)
{
// if block is a JSR block, then block.successors.next
// leads to the JSR target (see {@link #visitJumpInsn}) and
// must therefore not be followed
if((block.status & Label.JSR) == 0 || e != block.successors.next)
{
findSubroutine(e.successor, id);
}
e = e.next;
}
}

/**
* Finds the successors of the RET blocks of the specified subroutine,
* and of any nested subroutine it calls.
*
* @param id id of the subroutine whose RET block successors must
* be found.
* @param JSRs the JSR blocks that were followed to reach this
* subroutine.
* @param nJSRs number of JSR blocks in the JSRs array.
*/
private void findSubroutineSuccessors(
final int id,
7.18. METHODWRITER.JAVA 361

final Label[] JSRs,

final int nJSRs){
// iterates over all the basic blocks...
Label l = labels;
while(l != null)
{
// for those that belong to subroutine id...
if((l.status & id) != 0)
{
if((l.status & Label.JSR) != 0)
{
// finds the subroutine to which l leads by
// following the second edge of l.successors (see
// {@link #visitJumpInsn})
int nId = l.successors.next.successor.status & ~0xFFF;
if(nId != id)
{
// calls this method recursively with l pushed onto
// the JSRs stack to find the successors of the RET
// blocks of this nested subroutine nId
JSRs[nJSRs] = l;
findSubroutineSuccessors(nId, JSRs, nJSRs + 1);
}
}
else if((l.status & Label.RET) != 0)
{
/*
* finds the JSR block in the JSRs stack that corresponds
* to this RET block, and updates the successors of this
* RET block accordingly. This corresponding JSR is the
* one that leads to the subroutine to which the RET
* block belongs. But the RET block can belong to
* several subroutines (if a nested subroutine returns
* to its parent subroutine implicitely, without a RET).
* So, in fact, the JSR that corresponds to this RET is
* the first block in the JSRs stack, starting from the
* bottom of the stack, that leads to a subroutine to
* which the RET block belongs.
*/
for(int i = 0; i < nJSRs; ++i)
{
int JSRstatus =
JSRs[i].successors.next.successor.status;
if(((JSRstatus & ~0xFFF) &
(l.status & ~0xFFF)) != 0)
{
Edge e = new Edge();
e.info = l.inputStackTop;
e.successor = JSRs[i].successors.successor;
e.next = l.successors;
362 CHAPTER 7. JVM/CLOJURE/ASM/

l.successors = e;
break;
}
}
}
}
l = l.successor;
}
}

// -------------------------------------------------------------------
// Utility methods: stack map frames
// -------------------------------------------------------------------

/**
* Visits a frame that has been computed from scratch.
*
* @param f the frame that must be visited.
*/
private void visitFrame(final Frame f){
int i, t;
int nTop = 0;
int nLocal = 0;
int nStack = 0;
int[] locals = f.inputLocals;
int[] stacks = f.inputStack;
// computes the number of locals (ignores TOP types that are just
// after a LONG or a DOUBLE, and all trailing TOP types)
for(i = 0; i < locals.length; ++i)
{
t = locals[i];
if(t == Frame.TOP)
{
++nTop;
}
else
{
nLocal += nTop + 1;
nTop = 0;
}
if(t == Frame.LONG || t == Frame.DOUBLE)
{
++i;
}
}
// computes the stack size (ignores TOP types that are just after
// a LONG or a DOUBLE)
for(i = 0; i < stacks.length; ++i)
{
t = stacks[i];
7.18. METHODWRITER.JAVA 363

++nStack;
if(t == Frame.LONG || t == Frame.DOUBLE)
{
++i;
}
}
// visits the frame and its content
startFrame(f.owner.position, nLocal, nStack);
for(i = 0; nLocal > 0; ++i, --nLocal)
{
t = locals[i];
frame[frameIndex++] = t;
if(t == Frame.LONG || t == Frame.DOUBLE)
{
++i;
}
}
for(i = 0; i < stacks.length; ++i)
{
t = stacks[i];
frame[frameIndex++] = t;
if(t == Frame.LONG || t == Frame.DOUBLE)
{
++i;
}
}
endFrame();
}

/**
* Starts the visit of a stack map frame.
*
* @param offset the offset of the instruction to which the frame
* corresponds.
* @param nLocal the number of local variables in the frame.
* @param nStack the number of stack elements in the frame.
*/
private void startFrame(final int offset,
final int nLocal,
final int nStack){
int n = 3 + nLocal + nStack;
if(frame == null || frame.length < n)
{
frame = new int[n];
}
frame[0] = offset;
frame[1] = nLocal;
frame[2] = nStack;
frameIndex = 3;
}
364 CHAPTER 7. JVM/CLOJURE/ASM/

/**
* Checks if the visit of the current frame {@link #frame} is finished,
* and if yes, write it in the StackMapTable attribute.
*/
private void endFrame(){
if(previousFrame != null)
{ // do not write the first frame
if(stackMap == null)
{
stackMap = new ByteVector();
}
writeFrame();
++frameCount;
}
previousFrame = frame;
frame = null;
}

/**
* Compress and writes the current frame {@link #frame} in the
* StackMapTable attribute.
*/
private void writeFrame(){
int clocalsSize = frame[1];
int cstackSize = frame[2];
if((cw.version & 0xFFFF) < Opcodes.V1_6)
{
stackMap.putShort(frame[0]).putShort(clocalsSize);
writeFrameTypes(3, 3 + clocalsSize);
stackMap.putShort(cstackSize);
writeFrameTypes(3 + clocalsSize, 3 + clocalsSize + cstackSize);
return;
}
int localsSize = previousFrame[1];
int type = FULL_FRAME;
int k = 0;
int delta;
if(frameCount == 0)
{
delta = frame[0];
}
else
{
delta = frame[0] - previousFrame[0] - 1;
}
if(cstackSize == 0)
{
k = clocalsSize - localsSize;
switch(k)
7.18. METHODWRITER.JAVA 365

{
case-3:
case-2:
case-1:
type = CHOP_FRAME;
localsSize = clocalsSize;
break;
case 0:
type = delta < 64 ? SAME_FRAME : SAME_FRAME_EXTENDED;
break;
case 1:
case 2:
case 3:
type = APPEND_FRAME;
break;
}
}
else if(clocalsSize == localsSize && cstackSize == 1)
{
type = delta < 63
? SAME_LOCALS_1_STACK_ITEM_FRAME
: SAME_LOCALS_1_STACK_ITEM_FRAME_EXTENDED;
}
if(type != FULL_FRAME)
{
// verify if locals are the same
int l = 3;
for(int j = 0; j < localsSize; j++)
{
if(frame[l] != previousFrame[l])
{
type = FULL_FRAME;
break;
}
l++;
}
}
switch(type)
{
case SAME_FRAME:
stackMap.putByte(delta);
break;
case SAME_LOCALS_1_STACK_ITEM_FRAME:
stackMap.putByte(SAME_LOCALS_1_STACK_ITEM_FRAME + delta);
writeFrameTypes(3 + clocalsSize, 4 + clocalsSize);
break;
case SAME_LOCALS_1_STACK_ITEM_FRAME_EXTENDED:
stackMap.putByte(SAME_LOCALS_1_STACK_ITEM_FRAME_EXTENDED)
.putShort(delta);
writeFrameTypes(3 + clocalsSize, 4 + clocalsSize);
366 CHAPTER 7. JVM/CLOJURE/ASM/

break;
case SAME_FRAME_EXTENDED:
stackMap.putByte(SAME_FRAME_EXTENDED).putShort(delta);
break;
case CHOP_FRAME:
stackMap.putByte(SAME_FRAME_EXTENDED + k).putShort(delta);
break;
case APPEND_FRAME:
stackMap.putByte(SAME_FRAME_EXTENDED + k).putShort(delta);
writeFrameTypes(3 + localsSize, 3 + clocalsSize);
break;
// case FULL_FRAME:
default:
stackMap.putByte(FULL_FRAME)
.putShort(delta)
.putShort(clocalsSize);
writeFrameTypes(3, 3 + clocalsSize);
stackMap.putShort(cstackSize);
writeFrameTypes(3+clocalsSize, 3+clocalsSize+cstackSize);
}
}

/**
* Writes some types of the current frame {@link #frame} into the
* StackMapTableAttribute. This method converts types from the format
* used in {@link Label} to the format used in StackMapTable attributes.
* In particular, it converts type table indexes to constant pool
* indexes.
*
* @param start index of the first type in {@link #frame} to write.
* @param end index of last type in {@link #frame} to write
* (exclusive).
*/
private void writeFrameTypes(final int start, final int end){
for(int i = start; i < end; ++i)
{
int t = frame[i];
int d = t & Frame.DIM;
if(d == 0)
{
int v = t & Frame.BASE_VALUE;
switch(t & Frame.BASE_KIND)
{
case Frame.OBJECT:
stackMap.putByte(7)
.putShort(cw.newClass(cw.typeTable[v].strVal1));
break;
case Frame.UNINITIALIZED:
stackMap.putByte(8).putShort(cw.typeTable[v].intVal);
break;
7.18. METHODWRITER.JAVA 367

default:
stackMap.putByte(v);
}
}
else
{
StringBuffer buf = new StringBuffer();
d >>= 28;
while(d-- > 0)
{
buf.append([);
}
if((t & Frame.BASE_KIND) == Frame.OBJECT)
{
buf.append(L);
buf.append(cw.typeTable[t & Frame.BASE_VALUE].strVal1);
buf.append(;);
}
else
{
switch(t & 0xF)
{
case 1:
buf.append(I);
break;
case 2:
buf.append(F);
break;
case 3:
buf.append(D);
break;
case 9:
buf.append(Z);
break;
case 10:
buf.append(B);
break;
case 11:
buf.append(C);
break;
case 12:
buf.append(S);
break;
default:
buf.append(J);
}
}
stackMap.putByte(7).putShort(cw.newClass(buf.toString()));
}
}
368 CHAPTER 7. JVM/CLOJURE/ASM/

private void writeFrameType(final Object type){

if(type instanceof String)
{
stackMap.putByte(7).putShort(cw.newClass((String) type));
}
else if(type instanceof Integer)
{
stackMap.putByte(((Integer) type).intValue());
}
else
{
stackMap.putByte(8).putShort(((Label) type).position);
}
}

// -------------------------------------------------------------------
// Utility methods: dump bytecode array
// -------------------------------------------------------------------

/**
* Returns the size of the bytecode of this method.
*
* @return the size of the bytecode of this method.
*/
final int getSize(){
if(classReaderOffset != 0)
{
return 6 + classReaderLength;
}
if(resize)
{
// replaces the temporary jump opcodes introduced by
// Label.resolve.
resizeInstructions();
}
int size = 8;
if(code.length > 0)
{
cw.newUTF8("Code");
size += 18 + code.length + 8 * handlerCount;
if(localVar != null)
{
cw.newUTF8("LocalVariableTable");
size += 8 + localVar.length;
}
if(localVarType != null)
{
cw.newUTF8("LocalVariableTypeTable");
7.18. METHODWRITER.JAVA 369

size += 8 + localVarType.length;
}
if(lineNumber != null)
{
cw.newUTF8("LineNumberTable");
size += 8 + lineNumber.length;
}
if(stackMap != null)
{
boolean zip = (cw.version & 0xFFFF) >= Opcodes.V1_6;
cw.newUTF8(zip ? "StackMapTable" : "StackMap");
size += 8 + stackMap.length;
}
if(cattrs != null)
{
size += cattrs.getSize(cw,
code.data,
code.length,
maxStack,
maxLocals);
}
}
if(exceptionCount > 0)
{
cw.newUTF8("Exceptions");
size += 8 + 2 * exceptionCount;
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
&& (cw.version & 0xffff) < Opcodes.V1_5)
{
cw.newUTF8("Synthetic");
size += 6;
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
cw.newUTF8("Deprecated");
size += 6;
}
if(signature != null)
{
cw.newUTF8("Signature");
cw.newUTF8(signature);
size += 8;
}
if(annd != null)
{
cw.newUTF8("AnnotationDefault");
size += 6 + annd.length;
}
if(anns != null)
370 CHAPTER 7. JVM/CLOJURE/ASM/

{
cw.newUTF8("RuntimeVisibleAnnotations");
size += 8 + anns.getSize();
}
if(ianns != null)
{
cw.newUTF8("RuntimeInvisibleAnnotations");
size += 8 + ianns.getSize();
}
if(panns != null)
{
cw.newUTF8("RuntimeVisibleParameterAnnotations");
size += 7 + 2 * panns.length;
for(int i = panns.length - 1; i >= 0; --i)
{
size += panns[i] == null ? 0 : panns[i].getSize();
}
}
if(ipanns != null)
{
cw.newUTF8("RuntimeInvisibleParameterAnnotations");
size += 7 + 2 * ipanns.length;
for(int i = ipanns.length - 1; i >= 0; --i)
{
size += ipanns[i] == null ? 0 : ipanns[i].getSize();
}
}
if(attrs != null)
{
size += attrs.getSize(cw, null, 0, -1, -1);
}
return size;
}

/**
* Puts the bytecode of this method in the given byte vector.
*
* @param out the byte vector into which the bytecode of this method must
* be copied.
*/
final void put(final ByteVector out){
out.putShort(access).putShort(name).putShort(desc);
if(classReaderOffset != 0)
{
out.putByteArray(cw.cr.b, classReaderOffset, classReaderLength);
return;
}
int attributeCount = 0;
if(code.length > 0)
{
7.18. METHODWRITER.JAVA 371

++attributeCount;
}
if(exceptionCount > 0)
{
++attributeCount;
}
if((access & Opcodes.ACC_SYNTHETIC) != 0
&& (cw.version & 0xffff) < Opcodes.V1_5)
{
++attributeCount;
}
if((access & Opcodes.ACC_DEPRECATED) != 0)
{
++attributeCount;
}
if(signature != null)
{
++attributeCount;
}
if(annd != null)
{
++attributeCount;
}
if(anns != null)
{
++attributeCount;
}
if(ianns != null)
{
++attributeCount;
}
if(panns != null)
{
++attributeCount;
}
if(ipanns != null)
{
++attributeCount;
}
if(attrs != null)
{
attributeCount += attrs.getCount();
}
out.putShort(attributeCount);
if(code.length > 0)
{
int size = 12 + code.length + 8 * handlerCount;
if(localVar != null)
{
size += 8 + localVar.length;
372 CHAPTER 7. JVM/CLOJURE/ASM/

}
if(localVarType != null)
{
size += 8 + localVarType.length;
}
if(lineNumber != null)
{
size += 8 + lineNumber.length;
}
if(stackMap != null)
{
size += 8 + stackMap.length;
}
if(cattrs != null)
{
size += cattrs.getSize(cw,
code.data,
code.length,
maxStack,
maxLocals);
}
out.putShort(cw.newUTF8("Code")).putInt(size);
out.putShort(maxStack).putShort(maxLocals);
out.putInt(code.length).putByteArray(code.data, 0, code.length);
out.putShort(handlerCount);
if(handlerCount > 0)
{
Handler h = firstHandler;
while(h != null)
{
out.putShort(h.start.position)
.putShort(h.end.position)
.putShort(h.handler.position)
.putShort(h.type);
h = h.next;
}
}
attributeCount = 0;
if(localVar != null)
{
++attributeCount;
}
if(localVarType != null)
{
++attributeCount;
}
if(lineNumber != null)
{
++attributeCount;
}
7.18. METHODWRITER.JAVA 373

if(stackMap != null)
{
++attributeCount;
}
if(cattrs != null)
{
attributeCount += cattrs.getCount();
}
out.putShort(attributeCount);
if(localVar != null)
{
out.putShort(cw.newUTF8("LocalVariableTable"));
out.putInt(localVar.length + 2).putShort(localVarCount);
out.putByteArray(localVar.data, 0, localVar.length);
}
if(localVarType != null)
{
out.putShort(cw.newUTF8("LocalVariableTypeTable"));
out.putInt(localVarType.length + 2)
.putShort(localVarTypeCount);
out.putByteArray(localVarType.data, 0, localVarType.length);
}
if(lineNumber != null)
{
out.putShort(cw.newUTF8("LineNumberTable"));
out.putInt(lineNumber.length + 2).putShort(lineNumberCount);
out.putByteArray(lineNumber.data, 0, lineNumber.length);
}
if(stackMap != null)
{
boolean zip = (cw.version & 0xFFFF) >= Opcodes.V1_6;
out.putShort(cw.newUTF8(zip ? "StackMapTable" : "StackMap"));
out.putInt(stackMap.length + 2).putShort(frameCount);
out.putByteArray(stackMap.data, 0, stackMap.length);
}
if(cattrs != null)
{
cattrs.put(cw, code.data, code.length,
maxLocals, maxStack, out);
}
}
if(exceptionCount