CHAPMAN & HALL/CRC COMPUTER and INFORMATION SCIENCE SERIES

Handbook of

DATA STRUCTURES and APPLICATIONS

2005 by Chapman & Hall/CRC

CHAPMAN & HALL/CRC COMPUTER and INFORMATION SCIENCE SERIES

Series Editor: Sartaj Sahni

PUBLISHED TITLES HANDBOOK OF SCHEDULING: ALGORITHMS, MODELS, AND PERFORMANCE ANALYSIS Joseph Y-T. Leung THE PRACTICAL HANDBOOK OF INTERNET COMPUTING Munindar P. Singh HANDBOOK OF DATA STRUCTURES AND APPLICATIONS Dinesh P. Mehta and Sartaj Sahni

FORTHCOMING TITLES DISTRIBUTED SENSOR NETWORKS S. Sitharama Iyengar and Richard R. Brooks SPECULATIVE EXECUTION IN HIGH PERFORMANCE COMPUTER ARCHITECTURES David Kaeli and Pen-Chung Yew

2005 by Chapman & Hall/CRC

CHAPMAN & HALL/CRC COMPUTER and INFORMATION SCIENCE SERIES

Handbook of

DATA STRUCTURES and APPLICATIONS

Edited by

Dinesh P. Mehta
Colorado School of Mines Golden

and

Sartaj Sahni
University of Florida Gainesville

CHAPMAN & HALL/CRC

A CRC Press Company Boca Raton London New York Washington, D.C.
2005 by Chapman & Hall/CRC

For Chapters 7, 20, and 23 the authors retain the copyright.

Library of Congress Cataloging-in-Publication Data

Handbook of data structures and applications / edited by Dinesh P. Mehta and Sartaj Sahni. p. cm. (Chapman & Hall/CRC computer & information science) Includes bibliographical references and index. ISBN 1-58488-435-5 (alk. paper) 1. System designHandbooks, manuals, etc. 2. Data structures (Computer science)Handbooks, manuals, etc. I. Mehta, Dinesh P. II. Sahni, Sartaj. III. Chapman & Hall/CRC computer and information science series QA76.9.S88H363 2004 005.7'3dc22

2004055286

This book contains information obtained from authentic and highly regarded sources. Reprinted material is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable efforts have been made to publish reliable data and information, but the author and the publisher cannot assume responsibility for the validity of all materials or for the consequences of their use. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval system, without prior permission in writing from the publisher. All rights reserved. Authorization to photocopy items for internal or personal use, or the personal or internal use of specific clients, may be granted by CRC Press, provided that $1.50 per page photocopied is paid directly to Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923 USA. The fee code for users of the Transactional Reporting Service is ISBN 1-58488-435-5/04/$0.00+$1.50. The fee is subject to change without notice. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. The consent of CRC Press does not extend to copying for general distribution, for promotion, for creating new works, or for resale. Specific permission must be obtained in writing from CRC Press for such copying. Direct all inquiries to CRC Press, 2000 N.W. Corporate Blvd., Boca Raton, Florida 33431. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation, without intent to infringe.

Visit the CRC Press Web site at www.crcpress.com

2005 by Chapman & Hall/CRC No claim to original U.S. Government works International Standard Book Number 1-58488-435-5 Library of Congress Card Number 2004055286 Printed in the United States of America 1 2 3 4 5 6 7 8 9 0 Printed on acid-free paper

2005 by Chapman & Hall/CRC

Dedication

To our wives, Usha Mehta and Neeta Sahni

2005 by Chapman & Hall/CRC

Preface
In the late sixties, Donald Knuth, winner of the 1974 Turing Award, published his landmark book The Art of Computer Programming: Fundamental Algorithms. This book brought together a body of knowledge that dened the data structures area. The term data structure, itself, was dened in this book to be A table of data including structural relationships. Niklaus Wirth, the inventor of the Pascal language and winner of the 1984 Turing award, stated that Algorithms + Data Structures = Programs. The importance of algorithms and data structures has been recognized by the community and consequently, every undergraduate Computer Science curriculum has classes on data structures and algorithms. Both of these related areas have seen tremendous advances in the decades since the appearance of the books by Knuth and Wirth. Although there are several advanced and specialized texts and handbooks on algorithms (and related data structures), there is, to the best of our knowledge, no text or handbook that focuses exclusively on the wide variety of data structures that have been reported in the literature. The goal of this handbook is to provide a comprehensive survey of data structures of dierent types that are in existence today. To this end, we have subdivided this handbook into seven parts, each of which addresses a dierent facet of data structures. Part I is a review of introductory material. Although this material is covered in all standard data structures texts, it was included to make the handbook self-contained and in recognition of the fact that there are many practitioners and programmers who may not have had a formal education in Computer Science. Parts II, III, and IV discuss Priority Queues, Dictionary Structures, and Multidimensional structures, respectively. These are all well-known classes of data structures. Part V is a catch-all used for well-known data structures that eluded easy classication. Parts I through V are largely theoretical in nature: they discuss the data structures, their operations and their complexities. Part VI addresses mechanisms and tools that have been developed to facilitate the use of data structures in real programs. Many of the data structures discussed in previous parts are very intricate and take some eort to program. The development of data structure libraries and visualization tools by skilled programmers are of critical importance in reducing the gap between theory and practice. Finally, Part VII examines applications of data structures. The deployment of many data structures from Parts I through V in a variety of applications is discussed. Some of the data structures discussed here have been invented solely in the context of these applications and are not well-known to the broader community. Some of the applications discussed include Internet Routing, Web Search Engines, Databases, Data Mining, Scientic Computing, Geographical Information Systems, Computational Geometry, Computational Biology, VLSI Floorplanning and Layout, Computer Graphics and Image Processing. For data structure and algorithm researchers, we hope that the handbook will suggest new ideas for research in data structures and for an appreciation of the application contexts in which data structures are deployed. For the practitioner who is devising an algorithm, we hope that the handbook will lead to insights in organizing data that make it possible to solve the algorithmic problem more cleanly and eciently. For researchers in specic application areas, we hope that they will gain some insight from the ways other areas have handled their data structuring problems. Although we have attempted to make the handbook as complete as possible, it is impossible to undertake a task of this magnitude without some omissions. For this, we apologize in advance and encourage readers to contact us with information about signicant data

2005 by Chapman & Hall/CRC

structures or applications that do not appear here. These could be included in future editions of this handbook. We would like to thank the excellent team of authors, who are at the forefront of research in data structures, that have contributed to this handbook. The handbook would not have been possible without their painstaking eorts. We are extremely saddened by the untimely demise of a prominent data structures researcher, Professor G sli R. Hjaltason, who was to write a chapter for this handbook. He will be missed greatly by the Computer Science community. Finally, we would like to thank our families for their support during the development of the handbook.

Dinesh P. Mehta Sartaj Sahni

2005 by Chapman & Hall/CRC

About the Editors

Dinesh P. Mehta Dinesh P. Mehta received the B.Tech. degree in computer science and engineering from the Indian Institute of Technology, Bombay, in 1987, the M.S. degree in computer science from the University of Minnesota in 1990, and the Ph.D. degree in computer science from the University of Florida in 1992. He was on the faculty at the University of Tennessee Space Institute from 1992-2000, where he received the Vice Presidents Award for Teaching Excellence in 1997. He was a Visiting Professor at Intels Strategic CAD Labs in 1996 and 1997. He has been an Associate Professor in the Mathematical and Computer Sciences department at the Colorado School of Mines since 2000. Dr. Mehta is a co-author of the text Fundamentals of Data Structures in C + +. His publications and research interests are in VLSI design automation, parallel computing, and applied algorithms and data structures. His data structures-related research has involved the development or application of diverse data structures such as directed acyclic word graphs (DAWGs) for strings, corner stitching for VLSI layout, the Q-sequence oorplan representation, binary decision trees, Voronoi diagrams and TPR trees for indexing moving points. Dr. Mehta is currently an Associate Editor of the IEEE Transactions on Circuits and Systems-I. Sartaj Sahni Sartaj Sahni is a Distinguished Professor and Chair of Computer and Information Sciences and Engineering at the University of Florida. He is also a member of the European Academy of Sciences, a Fellow of IEEE, ACM, AAAS, and Minnesota Supercomputer Institute, and a Distinguished Alumnus of the Indian Institute of Technology, Kanpur. Dr. Sahni is the recipient of the 1997 IEEE Computer Society Taylor L. Booth Education Award, the 2003 IEEE Computer Society W. Wallace McDowell Award and the 2003 ACM Karl Karlstrom Outstanding Educator Award. Dr. Sahni received his B.Tech. (Electrical Engineering) degree from the Indian Institute of Technology, Kanpur, and the M.S. and Ph.D. degrees in Computer Science from Cornell University. Dr. Sahni has published over two hundred and fty research papers and written 15 texts. His research publications are on the design and analysis of ecient algorithms, parallel computing, interconnection networks, design automation, and medical algorithms. Dr. Sahni is a co-editor-in-chief of the Journal of Parallel and Distributed Computing, a managing editor of the International Journal of Foundations of Computer Science, and a member of the editorial boards of Computer Systems: Science and Engineering, International Journal of High Performance Computing and Networking, International Journal of Distributed Sensor Networks and Parallel Processing Letters. He has served as program committee chair, general chair, and been a keynote speaker at many conferences. Dr. Sahni has served on several NSF and NIH panels and he has been involved as an external evaluator of several Computer Science and Engineering departments.

2005 by Chapman & Hall/CRC

Contributors
Srinivas Aluru
Iowa State University Ames, Iowa

Arne Andersson
Uppsala University Uppsala, Sweden

Lars Arge
Duke University Durham, North Carolina

Sunil Arya
Hong Kong University of Science and Technology Kowloon, Hong Kong

Surender Baswana
Indian Institute of Technology, Delhi New Delhi, India

Mark de Berg
Technical University, Eindhoven Eindhoven, The Netherlands

Gerth Stlting Brodal

University of Aarhus Aarhus, Denmark

Bernard Chazelle
Princeton University Princeton, New Jersey

Chung-Kuan Cheng
University of California, San Diego San Diego, California

Siu-Wing Cheng
Hong Kong University of Science and Technology Kowloon, Hong Kong

Camil Demetrescu
Universit a di Roma Rome, Italy

Narsingh Deo
University of Central Florida Orlando, Florida

Sumeet Dua
Louisiana Tech University Ruston, Louisiana

Christian A. Duncan
University of Miami Miami, Florida

Peter Eades
University of Sydney and NICTA Sydney, Australia

Andrzej Ehrenfeucht
University of Colorado, Boulder Boulder, Colorado

Rolf Fagerberg
University of Southern Denmark Odense, Denmark

Zhou Feng
Fudan University Shanghai, China

Irene Finocchi
Universit a di Roma Rome, Italy

Michael L. Fredman
Rutgers University, New Brunswick New Brunswick, New Jersey

Teolo F. Gonzalez
University of California, Santa Barbara Santa Barbara, California

Michael T. Goodrich
University of California, Irvine Irvine, California

Leonidas Guibas
Stanford University Palo Alto, California

S. Gunasekaran
Louisiana State University Baton Rouge, Louisiana

Pankaj Gupta
Cypress Semiconductor San Jose, California

Prosenjit Gupta
International Institute of Information Technology Hyderabad, India

Joachim Hammer
University of Florida Gainesville, Florida

Monika Henzinger
Google, Inc. Mountain View, California

Seok-Hee Hong
University of Sydney and NICTA Sydney, Australia

Wen-Lian Hsu
Academia Sinica Taipei, Taiwan

Giuseppe F. Italiano
Universit a di Roma Rome, Italy

S. S. Iyengar
Louisiana State University Baton Rouge, Louisiana

Ravi Janardan
University of Minnesota Minneapolis, Minnesota

2005 by Chapman & Hall/CRC

Haim Kaplan
Tel Aviv University Tel Aviv, Israel

Kun Suk Kim

University of Florida Gainesville, Florida

Vipin Kumar
University of Minnesota Minneapolis, Minnesota

Stefan Kurtz
University of Hamburg Hamburg, Germany

Kim S. Larsen
University of Southern Denmark Odense, Denmark

D. T. Lee
Academia Sinica Taipei, Taiwan

Sebastian Leipert
Center of Advanced European Studies and Research Bonn, Germany

Scott Leutenegger
University of Denver Denver, Colorado

Ming C. Lin
University of North Carolina Chapel Hill, North Carolina

Stefano Lonardi
University of California, Riverside Riverside, California

Mario A. Lopez
University of Denver Denver, Colorado

Haibin Lu
University of Florida Gainesville, Florida

S. N. Maheshwari
Indian Institute of Technology, Delhi New Delhi, India

Dinesh Manocha
University of North Carolina Chapel Hill, North Carolina

Ross M. McConnell
Colorado State University Fort Collins, Colorado

Dale McMullin
Colorado School of Mines Golden, Colorado

Dinesh P. Mehta
Colorado School of Mines Golden, Colorado

Mark Moir
Sun Microsystems Laboratories Burlington, Massachusetts

Pat Morin
Carleton University Ottawa, Canada

David M. Mount
University of Maryland College Park, Maryland

J. Ian Munro
University of Waterloo Ontario, Canada

Stefan Naeher
University of Trier Trier, Germany

Bruce F. Naylor
University of Texas, Austin Austin, Texas

Chris Okasaki
United States Military Academy West Point, New York

C. Pandu Rangan
Indian Institute of Technology, Madras Chennai, India

Alex Pothen
Old Dominion University Norfolk, Virginia

Alyn Rockwood
Colorado School of Mines Golden, Colorado

S. Srinivasa Rao
University of Waterloo Ontario, Canada

Rajeev Raman
University of Leicester Leicester, United Kingdom

Wojciech Rytter
New Jersey Institute of Technology Newark, New Jersey & Warsaw University Warsaw, Poland

Sartaj Sahni
University of Florida Gainesville, Florida

Hanan Samet
University of Maryland College Park, Maryland

Sanjeev Saxena
Indian Institute of Technology, Kanpur Kanpur, India

2005 by Chapman & Hall/CRC

Markus Schneider
University of Florida Gainesville, Florida

Bernhard Seeger
University of Marburg Marburg, Germany

Sandeep Sen
Indian Institute of Technology, Delhi New Delhi, India

Nir Shavit
Sun Microsystems Laboratories Burlington, Massachusetts

Michiel Smid
Carleton University Ottawa, Canada

Bettina Speckmann
Technical University, Eindhoven Eindhoven, The Netherlands

John Stasko
Georgia Institute of Technology Atlanta, Georgia

Michael Steinbach
University of Minnesota Minneapolis, Minnesota

Roberto Tamassia
Brown University Providence, Rhode Island

Pang-Ning Tang
Michigan State University East Lansing, Michigan

Sivan Toledo
Tel Aviv University Tel Aviv, Israel

Luca Vismara
Brown University Providence, Rhode Island

V. K. Vaishnavi
Georgia State University Atlanta, Georgia

Jerey Scott Vitter

Purdue University West Lafayette, Indiana

Mark Allen Weiss

Florida International University Miami, Florida

Peter Widmayer
ETH Z urich, Switzerland

Bo Yao
University of California, San Diego San Diego, California

Donghui Zhang
Northeastern University Boston, Massachusetts

2005 by Chapman & Hall/CRC

Contents
Part I: 1 2 3 4 Fundamentals
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-1 2-1 3-1 4-1

Analysis of Algorithms Sartaj Sahni Basic Structures Dinesh P. Mehta . Trees Dinesh P. Mehta . . . . . . . . Graphs Narsingh Deo . . . . . . . . .

Part II: 5 6 7 8

Priority Queues
. . . . . . . . . . . . . . . . . . . .
5-1 6-1 7-1 8-1

Leftist Trees Sartaj Sahni . . . . . . . . . . . . . . . . . . . . . Skew Heaps C. Pandu Rangan . . . . . . . . . . . . . . . . . . Binomial, Fibonacci, and Pairing Heaps Michael L. Fredman Double-Ended Priority Queues Sartaj Sahni . . . . . . . . . .

Part III: 9 10 11 12 13 14 15

Dictionary Structures
9-1 10-1 11-1 12-1 13-1 14-1 15-1

Hash Tables Pat Morin . . . . . . . . . . . . . . . . . . . . . . . . . . . . Balanced Binary Search Trees Arne Andersson, Rolf Fagerberg, and Kim S. Larsen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Finger Search Trees Gerth Stlting Brodal . . . . . . . . . . . . . . . . Splay Trees Sanjeev Saxena . . . . . . . . . . . . . . . . . . . . . . . . . Randomized Dictionary Structures C. Pandu Rangan . . . . . . . . . Trees with Minimum Weighted Path Length Wojciech Rytter . . . . B Trees Donghui Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Part IV: 16 17 18 19 20 21 22 23 24 25 26 27

Multidimensional and Spatial Structures

Multidimensional Spatial Data Structures Hanan Samet . . . . . . . 16-1 Planar Straight Line Graphs Siu-Wing Cheng . . . . . . . . . . . . . . 17-1 Interval, Segment, Range, and Priority Search Trees D. T. Lee . . . . 18-1 Quadtrees and Octrees Srinivas Aluru . . . . . . . . . . . . . . . . . . . 19-1 Bruce F. Naylor . . . . . . . . . . . . 20-1 Binary Space Partitioning Trees R-trees Scott Leutenegger and Mario A. Lopez . . . . . . . . . . . . . 21-1 Managing Spatio-Temporal Data Sumeet Dua and S. S. Iyengar . . 22-1 Kinetic Data Structures Leonidas Guibas . . . . . . . . . . . . . . . . . 23-1 Online Dictionary Structures Teolo F. Gonzalez . . . . . . . . . . . . 24-1 Bernard Chazelle . . . . . . . . . . . . . . . . . . . . . . . . . . 25-1 Cuttings Approximate Geometric Query Structures Christian A. Duncan and Michael T. Goodrich . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26-1 Geometric and Spatial Data Structures in External Memory Jerey Scott Vitter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27-1

2005 by Chapman & Hall/CRC

Part V: 28 29 30 31 32 33 34 35 36 37 38 39

Miscellaneous Data Structures

Tries Sartaj Sahni . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28-1 Sux Trees and Sux Arrays Srinivas Aluru . . . . . . . . . . . . . . 29-1 String Searching Andrzej Ehrenfeucht and Ross M. McConnell . . . . 30-1 Persistent Data Structures Haim Kaplan . . . . . . . . . . . . . . . . . 31-1 PQ Trees, PC Trees, and Planar Graphs Wen-Lian Hsu and Ross M. McConnell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32-1 Data Structures for Sets Rajeev Raman . . . . . . . . . . . . . . . . . . 33-1 Cache-Oblivious Data Structures Lars Arge, Gerth Stlting Brodal, and Rolf Fagerberg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34-1 Dynamic Trees Camil Demetrescu, Irene Finocchi, and Giuseppe F. Italiano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35-1 Dynamic Graphs Camil Demetrescu, Irene Finocchi, and Giuseppe F. Italiano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36-1 Succinct Representation of Data Structures J. Ian Munro and S. Srinivasa Rao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37-1 Randomized Graph Data-Structures for Approximate Shortest Paths Surender Baswana and Sandeep Sen . . . . . . . . . . . . . . . . . . . . . . . . . 38-1 Searching and Priority Queues in o(log n) Time Arne Andersson . . 39-1

Part VI: 40 41 42 43 44 45 46 47

Data Structures in Languages and Libraries

40-1 41-1 42-1 43-1 44-1 45-1 46-1 47-1

Functional Data Structures Chris Okasaki . . . . . . . . . . . . . . . . LEDA, a Platform for Combinatorial and Geometric Computing Stefan Naeher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Structures in C++ Mark Allen Weiss . . . . . . . . . . . . . . . . Data Structures in JDSL Michael T. Goodrich, Roberto Tamassia, and Luca Vismara . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data Structure Visualization John Stasko . . . . . . . . . . . . . . . . . Drawing Trees Sebastian Leipert . . . . . . . . . . . . . . . . . . . . . . Drawing Graphs Peter Eades and Seok-Hee Hong . . . . . . . . . . . . Concurrent Data Structures Mark Moir and Nir Shavit . . . . . . . .

Part VII: 48 49 50 51 52 53 54 55 56

Applications
48-1 49-1 50-1 51-1 52-1 53-1 54-1 55-1 56-1

IP Router Tables Sartaj Sahni, Kun Suk Kim, and Haibin Lu . . . Multi-Dimensional Packet Classication Pankaj Gupta . . . . . . . . Data Structures in Web Information Retrieval Monika Henzinger . . The Web as a Dynamic Graph S. N. Maheshwari . . . . . . . . . . . . Layout Data Structures Dinesh P. Mehta . . . . . . . . . . . . . . . . . Floorplan Representation in VLSI Zhou Feng, Bo Yao, and ChungKuan Cheng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Computer Graphics Dale McMullin and Alyn Rockwood . . . . . . . Geographic Information Systems Bernhard Seeger and Peter Widmayer Collision Detection Ming C. Lin and Dinesh Manocha . . . . . . . . .

2005 by Chapman & Hall/CRC

57 58 59 60 61 62 63 64

Image Data Structures S. S. Iyengar, V. K. Vaishnavi, and S. Gunasekaran . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57-1 Computational Biology Stefan Kurtz and Stefano Lonardi . . . . . . 58-1 Elimination Structures in Scientic Computing Alex Pothen and Sivan Toledo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59-1 Data Structures for Databases Joachim Hammer and Markus Schneider 60-1 Data Mining Vipin Kumar, Pang-Ning Tan, and Michael Steinbach 61-1 Computational Geometry: Fundamental Structures Mark de Berg and Bettina Speckmann . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62-1 Computational Geometry: Proximity and Location Sunil Arya and David M. Mount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63-1 Computational Geometry: Generalized Intersection Searching Prosenjit Gupta, Ravi Janardan, and Michiel Smid . . . . . . . . . . . . . . . . . . 64-1

2005 by Chapman & Hall/CRC

I
Fundamentals
1 Analysis of Algorithms Sartaj Sahni . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-1
Introduction Operation Counts Step Counts Counting Cache Misses Asymptotic Complexity Recurrence Equations Amortized Complexity Practical Complexities

2 Basic Structures
Introduction

Dinesh P. Mehta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2-1 3-1

Arrays

Linked Lists

Stacks and Queues Binary Trees and Properties Binary Search Trees Heaps

3 Trees

Dinesh P. Mehta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Introduction Tree Representation Traversals Threaded Binary Trees Trees

Binary Tree Tournament

4 Graphs

Narsingh Deo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4-1

Introduction Graph Representations Connectivity, Distance, and Spanning Trees Searching a Graph Simple Applications of DFS and BFS Minimum Spanning Tree Shortest Paths Eulerian and Hamiltonian Graphs

2005 by Chapman & Hall/CRC

1
Analysis of Algorithms
1.1 1.2 1.3 1.4 1.5 1.6 1.7 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Operation Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Step Counts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Counting Cache Misses . . . . . . . . . . . . . . . . . . . . . . . . . . . .
A Simple Computer Model Eect of Cache Misses on Run Time Matrix Multiplication

1-1 1-2 1-4 1-6 1-9 1-12 1-14

Asymptotic Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . .
Big Oh Notation (O ) Omega () and Theta () Notations Little Oh Notation (o)

Recurrence Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Substitution Method

Table-Lookup Method

Amortized Complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . .
What is Amortized Complexity? Maintenance Contract The McWidget Company Subset Generation

Sartaj Sahni
University of Florida

1.8

Practical Complexities . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1-23

1.1

Introduction

The topic Analysis of Algorithms is concerned primarily with determining the memory (space) and time requirements (complexity) of an algorithm. Since the techniques used to determine memory requirements are a subset of those used to determine time requirements, in this chapter, we focus on the methods used to determine the time complexity of an algorithm. The time complexity (or simply, complexity) of an algorithm is measured as a function of the problem size. Some examples are given below. 1. The complexity of an algorithm to sort n elements may be given as a function of n. 2. The complexity of an algorithm to multiply an m n matrix and an n p matrix may be given as a function of m, n, and p. 3. The complexity of an algorithm to determine whether x is a prime number may be given as a function of the number, n, of bits in x. Note that n = log2 (x + 1) . We partition our discussion of algorithm analysis into the following sections. 1. Operation counts. 2. Step counts. 3. Counting cache misses.

1-1

2005 by Chapman & Hall/CRC

1-2

Handbook of Data Structures and Applications Asymptotic complexity. Recurrence equations. Amortized complexity. Practical complexities.

4. 5. 6. 7.

See [1, 35] for additional material on algorithm analysis.

1.2

Operation Counts

One way to estimate the time complexity of a program or method is to select one or more operations, such as add, multiply, and compare, and to determine how many of each is done. The success of this method depends on our ability to identify the operations that contribute most to the time complexity.
Example 1.1

[Max Element] Figure 1.1 gives an algorithm that returns the position of the largest element in the array a[0:n-1]. When n > 0, the time complexity of this algorithm can be estimated by determining the number of comparisons made between elements of the array a. When n 1, the for loop is not entered. So no comparisons between elements of a are made. When n > 1, each iteration of the for loop makes one comparison between two elements of a, and the total number of element comparisons is n-1. Therefore, the number of element comparisons is max{n-1, 0}. The method max performs other comparisons (for example, each iteration of the for loop is preceded by a comparison between i and n) that are not included in the estimate. Other operations such as initializing positionOfCurrentMax and incrementing the for loop index i are also not included in the estimate.

int max(int [] a, int n) { if (n < 1) return -1; // no max int positionOfCurrentMax = 0; for (int i = 1; i < n; i++) if (a[positionOfCurrentMax] < a[i]) positionOfCurrentMax = i; return positionOfCurrentMax; } FIGURE 1.1: Finding the position of the largest element in a[0:n-1].

The algorithm of Figure 1.1 has the nice property that the operation count is precisely determined by the problem size. For many other problems, however, this is not so. Figure 1.2 gives an algorithm that performs one pass of a bubble sort. In this pass, the largest element in a[0:n-1] relocates to position a[n-1]. The number of swaps performed by this algorithm depends not only on the problem size n but also on the particular values of the elements in the array a. The number of swaps varies from a low of 0 to a high of n 1.

2005 by Chapman & Hall/CRC

Analysis of Algorithms void bubble(int [] a, int n) { for (int i = 0; i < n - 1; i++) if (a[i] > a[i+1]) swap(a[i], a[i+1]); } FIGURE 1.2: A bubbling pass.

1-3

Since the operation count isnt always uniquely determined by the problem size, we ask for the best, worst, and average counts.
Example 1.2

[Sequential Search] Figure 1.3 gives an algorithm that searches a[0:n-1] for the rst occurrence of x. The number of comparisons between x and the elements of a isnt uniquely determined by the problem size n. For example, if n = 100 and x = a[0], then only 1 comparison is made. However, if x isnt equal to any of the a[i]s, then 100 comparisons are made. A search is successful when x is one of the a[i]s. All other searches are unsuccessful. Whenever we have an unsuccessful search, the number of comparisons is n. For successful searches the best comparison count is 1, and the worst is n. For the average count assume that all array elements are distinct and that each is searched for with equal frequency. The average count for a successful search is 1 n
n

i = (n + 1)/2
i=1

int sequentialSearch(int [] a, int n, int x) { // search a[0:n-1] for x int i; for (i = 0; i < n && x != a[i]; i++); if (i == n) return -1; // not found else return i; } FIGURE 1.3: Sequential search.

Example 1.3

[Insertion into a Sorted Array] Figure 1.4 gives an algorithm to insert an element x into a sorted array a[0:n-1]. We wish to determine the number of comparisons made between x and the elements of a. For the problem size, we use the number n of elements initially in a. Assume that n 1. The best or minimum number of comparisons is 1, which happens when the new element x

2005 by Chapman & Hall/CRC

1-4

Handbook of Data Structures and Applications

void insert(int [] a, int n, int x) { // find proper place for x int i; for (i = n - 1; i >= 0 && x < a[i]; i--) a[i+1] = a[i]; a[i+1] = x; } FIGURE 1.4: Inserting into a sorted array. is to be inserted at the right end. The maximum number of comparisons is n, which happens when x is to be inserted at the left end. For the average assume that x has an equal chance of being inserted into any of the possible n+1 positions. If x is eventually inserted into position i+1 of a, i 0, then the number of comparisons is n-i. If x is inserted into a[0], the number of comparisons is n. So the average count is 1 n(n + 1) n n 1 1 ( ( ( + n) = + (n i) + n) = j + n) = n + 1 i=0 n + 1 j =1 n+1 2 2 n+1 This average count is almost 1 more than half the worst-case count.
n1 n

// insert x

1.3

Step Counts

The operation-count method of estimating time complexity omits accounting for the time spent on all but the chosen operations. In the step-count method, we attempt to account for the time spent in all parts of the algorithm. As was the case for operation counts, the step count is a function of the problem size. A step is any computation unit that is independent of the problem size. Thus 10 additions can be one step; 100 multiplications can also be one step; but n additions, where n is the problem size, cannot be one step. The amount of computing represented by one step may be dierent from that represented by another. For example, the entire statement return a+b+b*c+(a+b-c)/(a+b)+4; can be regarded as a single step if its execution time is independent of the problem size. We may also count a statement such as x = y; as a single step. To determine the step count of an algorithm, we rst determine the number of steps per execution (s/e) of each statement and the total number of times (i.e., frequency) each statement is executed. Combining these two quantities gives us the total contribution of each statement to the total step count. We then add the contributions of all statements to obtain the step count for the entire algorithm.

2005 by Chapman & Hall/CRC

Analysis of Algorithms
Statement int sequentialSearch( ) { int i; for (i = 0; i < n && x != a[i]; i++); if (i == n) return -1; else return i; } Total s/e 0 0 1 1 1 1 0 Frequency 0 0 1 1 1 1 0 Total steps 0 0 1 1 1 1 0 4

1-5

TABLE 1.1

Best-case step count for Figure 1.3

Statement int sequentialSearch( ) { int i; for (i = 0; i < n && x != a[i]; i++); if (i == n) return -1; else return i; } Total

s/e 0 0 1 1 1 1 0

Frequency 0 0 1 n+1 1 0 0

Total steps 0 0 1 n+1 1 0 0 n+3

TABLE 1.2

Worst-case step count for Figure 1.3

Statement int sequentialSearch( ) { int i; for (i = 0; i < n && x != a[i]; i++); if (i == n) return -1; else return i; } Total

s/e 0 0 1 1 1 1 0

Frequency 0 0 1 j+1 1 1 0

Total steps 0 0 1 j+1 1 1 0 j+4

TABLE 1.3

Step count for Figure 1.3 when x = a[j]

Example 1.4

[Sequential Search] Tables 1.1 and 1.2 show the best- and worst-case step-count analyses for sequentialSearch (Figure 1.3). For the average step-count analysis for a successful search, we assume that the n values in a are distinct and that in a successful search, x has an equal probability of being any one of these values. Under these assumptions the average step count for a successful search is the sum of the step counts for the n possible successful searches divided by n. To obtain this average, we rst obtain the step count for the case x = a[j] where j is in the range [0, n 1] (see Table 1.3). Now we obtain the average step count for a successful search: 1 n
n1

(j + 4) = (n + 7)/2
j =0

This value is a little more than half the step count for an unsuccessful search. Now suppose that successful searches occur only 80 percent of the time and that each a[i] still has the same probability of being searched for. The average step count for sequentialSearch is .8 (average count for successful searches) + .2 (count for an unsuccessful search) = .8(n + 7)/2 + .2(n + 3) = .6n + 3.4

2005 by Chapman & Hall/CRC

1-6

Handbook of Data Structures and Applications

1.4
1.4.1

Counting Cache Misses

A Simple Computer Model

Traditionally, the focus of algorithm analysis has been on counting operations and steps. Such a focus was justied when computers took more time to perform an operation than they took to fetch the data needed for that operation. Today, however, the cost of performing an operation is signicantly lower than the cost of fetching data from memory. Consequently, the run time of many algorithms is dominated by the number of memory references (equivalently, number of cache misses) rather than by the number of operations. Hence, algorithm designers focus on reducing not only the number of operations but also the number of memory accesses. Algorithm designers focus also on designing algorithms that hide memory latency. Consider a simple computer model in which the computers memory consists of an L1 (level 1) cache, an L2 cache, and main memory. Arithmetic and logical operations are performed by the arithmetic and logic unit (ALU) on data resident in registers (R). Figure 1.5 gives a block diagram for our simple computer model.

ALU R L1 L2

main memory

FIGURE 1.5: A simple computer model.

Typically, the size of main memory is tens or hundreds of megabytes; L2 cache sizes are typically a fraction of a megabyte; L1 cache is usually in the tens of kilobytes; and the number of registers is between 8 and 32. When you start your program, all your data are in main memory. To perform an arithmetic operation such as an add, in our computer model, the data to be added are rst loaded from memory into registers, the data in the registers are added, and the result is written to memory. Let one cycle be the length of time it takes to add data that are already in registers. The time needed to load data from L1 cache to a register is two cycles in our model. If the required data are not in L1 cache but are in L2 cache, we get an L1 cache miss and the required data are copied from L2 cache to L1 cache and the register in 10 cycles. When the required data are not in L2 cache either, we have an L2 cache miss and the required data are copied from main memory into L2 cache, L1 cache, and the register in 100 cycles. The write operation is counted as one cycle even when the data are written to main memory because we do not wait for the write to complete before proceeding to the next operation. For more details on cache organization, see [2].

1.4.2

Eect of Cache Misses on Run Time

For our simple model, the statement a = b + c is compiled into the computer instructions load a; load b; add; store c;

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-7

where the load operations load data into registers and the store operation writes the result of the add to memory. The add and the store together take two cycles. The two loads may take anywhere from 4 cycles to 200 cycles depending on whether we get no cache miss, L1 misses, or L2 misses. So the total time for the statement a = b + c varies from 6 cycles to 202 cycles. In practice, the variation in time is not as extreme because we can overlap the time spent on successive cache misses. Suppose that we have two algorithms that perform the same task. The rst algorithm does 2000 adds that require 4000 load, 2000 add, and 2000 store operations and the second algorithm does 1000 adds. The data access pattern for the rst algorithm is such that 25 percent of the loads result in an L1 miss and another 25 percent result in an L2 miss. For our simplistic computer model, the time required by the rst algorithm is 2000 2 (for the 50 percent loads that cause no cache miss) + 1000 10 (for the 25 percent loads that cause an L1 miss) + 1000 100 (for the 25 percent loads that cause an L2 miss) + 2000 1 (for the adds) + 2000 1 (for the stores) = 118,000 cycles. If the second algorithm has 100 percent L2 misses, it will take 2000 100 (L2 misses) + 1000 1 (adds) + 1000 1 (stores) = 202,000 cycles. So the second algorithm, which does half the work done by the rst, actually takes 76 percent more time than is taken by the rst algorithm. Computers use a number of strategies (such as preloading data that will be needed in the near future into cache, and when a cache miss occurs, the needed data as well as data in some number of adjacent bytes are loaded into cache) to reduce the number of cache misses and hence reduce the run time of a program. These strategies are most eective when successive computer operations use adjacent bytes of main memory. Although our discussion has focused on how cache is used for data, computers also use cache to reduce the time needed to access instructions.

1.4.3

Matrix Multiplication

The algorithm of Figure 1.6 multiplies two square matrices that are represented as twodimensional arrays. It performs the following computation:
n

c[i][j] =
k=1

a[i][k] b[k][j], 1 i n, 1 j n

(1.1)

void squareMultiply(int [][] a, int [][] b, int [][] c, int n) { for (int i = 0; i < n; i++) for (int j = 0; j < n; j++) { int sum = 0; for (int k = 0; k < n; k++) sum += a[i][k] * b[k][j]; c[i][j] = sum; } } FIGURE 1.6: Multiply two n n matrices.

2005 by Chapman & Hall/CRC

1-8

Handbook of Data Structures and Applications

Figure 1.7 is an alternative algorithm that produces the same two-dimensional array c as is produced by Figure 1.6. We observe that Figure 1.7 has two nested for loops that are not present in Figure 1.6 and does more work than is done by Figure 1.6 with respect to indexing into the array c. The remainder of the work is the same. void fastSquareMultiply(int [][] a, int [][] b, int [][] c, int n) { for (int i = 0; i < n; i++) for (int j = 0; j < n; j++) c[i][j] = 0; for (int i = 0; i < n; i++) for (int j = 0; j < n; j++) for (int k = 0; k < n; k++) c[i][j] += a[i][k] * b[k][j]; } FIGURE 1.7: Alternative algorithm to multiply square matrices.

You will notice that if you permute the order of the three nested for loops in Figure 1.7, you do not aect the result array c. We refer to the loop order in Figure 1.7 as ijk order. When we swap the second and third for loops, we get ikj order. In all, there are 3! = 6 ways in which we can order the three nested for loops. All six orderings result in methods that perform exactly the same number of operations of each type. So you might think all six take the same time. Not so. By changing the order of the loops, we change the data access pattern and so change the number of cache misses. This in turn aects the run time. In ijk order, we access the elements of a and c by rows; the elements of b are accessed by column. Since elements in the same row are in adjacent memory and elements in the same column are far apart in memory, the accesses of b are likely to result in many L2 cache misses when the matrix size is too large for the three arrays to t into L2 cache. In ikj order, the elements of a, b, and c are accessed by rows. Therefore, ikj order is likely to result in fewer L2 cache misses and so has the potential to take much less time than taken by ijk order. For a crude analysis of the number of cache misses, assume we are interested only in L2 misses; that an L2 cache-line can hold w matrix elements; when an L2 cache-miss occurs, a block of w matrix elements is brought into an L2 cache line; and that L2 cache is small compared to the size of a matrix. Under these assumptions, the accesses to the elements of a, b and c in ijk order, respectively, result in n3 /w, n3 , and n2 /w L2 misses. Therefore, the total number of L2 misses in ijk order is n3 (1 + w + 1/n)/w. In ikj order, the number of L2 misses for our three matrices is n2 /w, n3 /w, and n3 /w, respectively. So, in ikj order, the total number of L2 misses is n3 (2 + 1/n)/w. When n is large, the ration of ijk misses to ikj misses is approximately (1 + w)/2, which is 2.5 when w = 4 (for example when we have a 32-byte cache line and the data is double precision) and 4.5 when w = 8 (for example when we have a 64-byte cache line and double-precision data). For a 64-byte cache line and single-precision (i.e., 4 byte) data, w = 16 and the ratio is approximately 8.5. Figure 1.8 shows the normalized run times of a Java version of our matrix multiplication algorithms. In this gure, mult refers to the multiplication algorithm of Figure 1.6. The

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-9

normalized run time of a method is the time taken by the method divided by the time taken by ikj order.
1.2 1.1 1

n = 500 mult

n = 1000 ijk

n = 2000 ikj

FIGURE 1.8: Normalized run times for matrix multiplication.

Matrix multiplication using ikj order takes 10 percent less time than does ijk order when the matrix size is n = 500 and 16 percent less time when the matrix size is 2000. Equally surprising is that ikj order runs faster than the algorithm of Figure 1.6 (by about 5 percent when n = 2000). This despite the fact that ikj order does more work than is done by the algorithm of Figure 1.6.

1.5
1.5.1

Asymptotic Complexity
Big Oh Notation (O)

Let p(n) and q (n) be two nonnegative functions. p(n) is asymptotically bigger (p(n) asymptotically dominates q (n)) than the function q (n) i q (n) =0 n p(n) lim (1.2)

q (n) is asymptotically smaller than p(n) i p(n) is asymptotically bigger than q (n). p(n) and q (n) are asymptotically equal i neither is asymptotically bigger than the other.
Example 1.5

Since lim

n 3n2

10n + 7 10/n + 7/n2 = = 0/3 = 0 + 2n + 6 3 + 2/n + 6/n2

3n2 + 2n + 6 is asymptotically bigger than 10n + 7 and 10n + 7 is asymptotically smaller than 3n2 + 2n + 6. A similar derivation shows that 8n4 + 9n2 is asymptotically bigger than 100n3 3, and that 2n2 + 3n is asymptotically bigger than 83n. 12n + 6 is asymptotically equal to 6n + 2. In the following discussion the function f (n) denotes the time or space complexity of an algorithm as a function of the problem size n. Since the time or space requirements of

2005 by Chapman & Hall/CRC

1-10

Handbook of Data Structures and Applications

a program are nonnegative quantities, we assume that the function f has a nonnegative value for all values of n. Further, since n denotes an instance characteristic, we assume that n 0. The function f (n) will, in general, be a sum of terms. For example, the terms of f (n) = 9n2 + 3n + 12 are 9n2 , 3n, and 12. We may compare pairs of terms to determine which is bigger. The biggest term in the example f (n) is 9n2 . Figure 1.9 gives the terms that occur frequently in a step-count analysis. Although all the terms in Figure 1.9 have a coecient of 1, in an actual analysis, the coecients of these terms may have a dierent value.

Term 1 log n n n log n n2 n3 2n n!

Name constant logarithmic linear n log n quadratic cubic exponential factorial

FIGURE 1.9: Commonly occurring terms.

We do not associate a logarithmic base with the functions in Figure 1.9 that include log n because for any constants a and b greater than 1, loga n = logb n/ logb a. So loga n and logb n are asymptotically equal. The denition of asymptotically smaller implies the following ordering for the terms of Figure 1.9 (< is to be read as is asymptotically smaller than): 1 < log n < n < n log n < n2 < n3 < 2n < n! Asymptotic notation describes the behavior of the time or space complexity for large instance characteristics. Although we will develop asymptotic notation with reference to step counts alone, our development also applies to space complexity and operation counts. The notation f (n) = O(g (n)) (read as f (n) is big oh of g (n)) means that f (n) is asymptotically smaller than or equal to g (n). Therefore, in an asymptotic sense g (n) is an upper bound for f (n).
Example 1.6

From Example 1.5, it follows that 10n + 7 = O(3n2 + 2n + 6); 100n3 3 = O(8n4 + 9n2 ). We see also that 12n + 6 = O(6n + 2); 3n2 + 2n + 6 = O(10n + 7); and 8n4 + 9n2 = O(100n3 3). Although Example 1.6 uses the big oh notation in a correct way, it is customary to use g (n) functions that are unit terms (i.e., g (n) is a single term whose coecient is 1) except when f (n) = 0. In addition, it is customary to use, for g (n), the smallest unit term for which the statement f (n) = O(g (n)) is true. When f (n) = 0, it is customary to use g (n) = 0.
Example 1.7

The customary way to describe the asymptotic behavior of the functions used in Example 1.6 is 10n + 7 = O(n); 100n3 3 = O(n3 ); 12n + 6 = O(n); 3n2 + 2n + 6 = O(n); and 8n4 + 9n2 = O(n3 ).

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-11

In asymptotic complexity analysis, we determine the biggest term in the complexity; the coecient of this biggest term is set to 1. The unit terms of a step-count function are step-count terms with their coecients changed to 1. For example, the unit terms of 3n2 + 6n log n + 7n + 5 are n2 , n log n, n, and 1; the biggest unit term is n2 . So when the step count of a program is 3n2 + 6n log n + 7n + 5, we say that its asymptotic complexity is O(n2 ). Notice that f (n) = O(g (n)) is not the same as O(g (n)) = f (n). In fact, saying that O(g (n)) = f (n) is meaningless. The use of the symbol = is unfortunate, as this symbol commonly denotes the equals relation. We can avoid some of the confusion that results from the use of this symbol (which is standard terminology) by reading the symbol = as is and not as equals.

1.5.2

Omega () and Theta () Notations

Although the big oh notation is the most frequently used asymptotic notation, the omega and theta notations are sometimes used to describe the asymptotic complexity of a program. The notation f (n) = (g (n)) (read as f (n) is omega of g (n)) means that f (n) is asymptotically bigger than or equal to g (n). Therefore, in an asymptotic sense, g (n) is a lower bound for f (n). The notation f (n) = (g (n)) (read as f (n) is theta of g (n)) means that f (n) is asymptotically equal to g (n).
Example 1.8

10n + 7 = (n) because 10n + 7 is asymptotically equal to n; 100n3 3 = (n3 ); 12n + 6 = (n); 3n3 +2n +6 = (n); 8n4 +9n2 = (n3 ); 3n3 +2n +6 = (n5 ); and 8n4 +9n2 = (n5 ). 10n + 7 = (n) because 10n + 7 is asymptotically equal to n; 100n3 3 = (n3 ); 12n + 6 = (n); 3n3 + 2n + 6 = (n); 8n4 + 9n2 = (n3 ); 3n3 + 2n + 6 = (n5 ); and 8n4 + 9n2 = (n5 ). The best-case step count for sequentialSearch (Figure 1.3) is 4 (Table 1.1), the worstcase step count is n +3, and the average step count is 0.6n +3.4. So the best-case asymptotic complexity of sequentialSearch is (1), and the worst-case and average complexities are (n). It is also correct to say that the complexity of sequentialSearch is (1) and O(n) because 1 is a lower bound (in an asymptotic sense) and n is an upper bound (in an asymptotic sense) on the step count. When using the notation, it is customary to use, for g (n), the largest unit term for which the statement f (n) = (g (n)) is true. At times it is useful to interpret O(g (n)), (g (n)), and (g (n)) as being the following sets: O(g (n)) = {f (n)|f (n) = O(g (n))} (g (n)) = {f (n)|f (n) = (g (n))} (g (n)) = {f (n)|f (n) = (g (n))} Under this interpretation, statements such as O(g1 (n)) = O(g2 (n)) and (g1 (n)) = (g2 (n)) are meaningful. When using this interpretation, it is also convenient to read f (n) = O(g (n)) as f of n is in (or is a member of) big oh of g of n and so on.

2005 by Chapman & Hall/CRC

1-12

Handbook of Data Structures and Applications

1.5.3

Little Oh Notation (o)

The little oh notation describes a strict upper bound on the asymptotic growth rate of the function f . f (n) is little oh of g (n) i f (n) is asymptotically smaller than g (n). Equivalently, f (n) = o(g (n)) (read as f of n is little oh of g of n) i f (n) = O(g (n)) and f (n) = (g (n)).
Example 1.9

[Little oh] 3n + 2 = o(n2 ) as 3n + 2 = O(n2 ) and 3n + 2 = (n2 ). However, 3n + 2 = o(n). Similarly, 10n2 + 4n + 2 = o(n3 ), but is not o(n2 ). The little oh notation is often used in step-count analyses. A step count of 3n + o(n) would mean that the step count is 3n plus terms that are asymptotically smaller than n. When performing such an analysis, one can ignore portions of the program that are known to contribute less than (n) steps.

1.6

Recurrence Equations

Recurrence equations arise frequently in the analysis of algorithms, particularly in the analysis of recursive as well as divide-and-conquer algorithms.
Example 1.10

[Binary Search] Consider a binary search of the sorted array a[l : r], where n = r l + 1 0, for the element x. When n = 0, the search is unsuccessful and when n = 1, we compare x and a[l]. When n > 1, we compare x with the element a[m] (m = (l + r)/2 ) in the middle of the array. If the compared elements are equal, the search terminates; if x < a[m], we search a[l : m 1]; otherwise, we search a[m + 1 : r]. Let t(n) be the worst-case complexity of binary search. Assuming that t(0) = t(1), we obtain the following recurrence. t(n) = where c is a constant.
Example 1.11

t(1) n1 t( n/2 ) + c n > 1

(1.3)

[Merge Sort] In a merge sort of a[0 : n 1], n 1, we consider two cases. When n = 1, no work is to be done as a one-element array is always in sorted order. When n > 1, we divide a into two parts of roughly the same size, sort these two parts using the merge sort method recursively, then nally merge the sorted parts to obtain the desired sorted array. Since the time to do the nal merge is (n) and the dividing into two roughly equal parts takes O(1) time, the complexity, t(n), of merge sort is given by the recurrence: t(n) = where c is a constant. Solving recurrence equations such as Equations 1.3 and 1.4 for t(n) is complicated by the presence of the oor and ceiling functions. By making an appropriate assumption on the permissible values of n, these functions may be eliminated to obtain a simplied recurrence. t(1) n=1 t( n/2 ) + t( n/2 ) + cn n > 1 (1.4)

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-13

In the case of Equations 1.3 and 1.4 an assumption such as n is a power of 2 results in the simplied recurrences: t(1) n1 t(n) = (1.5) t(n/2) + c n > 1 and t(n) = t(1) 2t(n/2) + cn n=1 n>1 (1.6)

Several techniquessubstitution, table lookup, induction, characteristic roots, and generating functionsare available to solve recurrence equations. We describe only the substitution and table lookup methods.

1.6.1

Substitution Method

In the substitution method, recurrences such as Equations 1.5 and 1.6 are solved by repeatedly substituting right-side occurrences (occurrences to the right of =) of t(x), x > 1, with expressions involving t(y ), y < x. The substitution process terminates when the only occurrences of t(x) that remain on the right side have x = 1. Consider the binary search recurrence of Equation 1.5. Repeatedly substituting for t() on the right side, we get t(n) = = = = . . . = = t(n/2) + c (t(n/4) + c) + c t(n/4) + 2c t(n/8) + 3c

t(1) + c log2 n (log n)

For the merge sort recurrence of Equation 1.6, we get t(n) = 2t(n/2) + cn = 2(2t(n/4) + cn/2) + cn = 4t(n/4) + 2cn = 4(2t(n/8) + cn/4) + 2cn = 8t(n/8) + 3cn . . . = nt(1) + cn log2 n = (n log n)

1.6.2

Table-Lookup Method

The complexity of many divide-and-conquer algorithms is given by a recurrence of the form t(n) = t(1) n=1 a t(n/b) + g (n) n > 1 (1.7)

2005 by Chapman & Hall/CRC

1-14

Handbook of Data Structures and Applications

h(n) O(nr ), r < 0 ((log n)i ), i 0 (n ), r > 0
r

f (n) O(1) (((log n)i+1 )/(i + 1)) (h(n))

TABLE 1.4

f (n) values for various h(n) values

where a and b are known constants. The merge sort recurrence, Equation 1.6, is in this form. Although the recurrence for binary search, Equation 1.5, isnt exactly in this form, the n 1 may be changed to n = 1 by eliminating the case n = 0. To solve Equation 1.7, we assume that t(1) is known and that n is a power of b (i.e., n = bk ). Using the substitution method, we can show that t(n) = nlogb a [t(1) + f (n)] (1.8)

j logb a . where f (n) = k j =1 h(b ) and h(n) = g (n)/n Table 1.4 tabulates the asymptotic value of f (n) for various values of h(n). This table allows us to easily obtain the asymptotic value of t(n) for many of the recurrences we encounter when analyzing divide-and-conquer algorithms. Let us solve the binary search and merge sort recurrences using this table. Comparing Equation 1.5 with n 1 replaced by n = 1 with Equation 1.7, we see that a = 1, b = 2, and g (n) = c. Therefore, logb (a) = 0, and h(n) = g (n)/nlogb a = c = c(log n)0 = ((log n)0 ). From Table 1.4, we obtain f (n) = (log n). Therefore, t(n) = nlogb a (c + (log n)) = (log n). For the merge sort recurrence, Equation 1.6, we obtain a = 2, b = 2, and g (n) = cn. So logb a = 1 and h(n) = g (n)/n = c = ((log n)0 ). Hence f (n) = (log n) and t(n) = n(t(1) + (log n)) = (n log n).

1.7
1.7.1

Amortized Complexity
What is Amortized Complexity?

The complexity of an algorithm or of an operation such as an insert, search, or delete, as dened in Section 1.1, is the actual complexity of the algorithm or operation. The actual complexity of an operation is determined by the step count for that operation, and the actual complexity of a sequence of operations is determined by the step count for that sequence. The actual complexity of a sequence of operations may be determined by adding together the step counts for the individual operations in the sequence. Typically, determining the step count for each operation in the sequence is quite dicult, and instead, we obtain an upper bound on the step count for the sequence by adding together the worst-case step count for each operation. When determining the complexity of a sequence of operations, we can, at times, obtain tighter bounds using amortized complexity rather than worst-case complexity. Unlike the actual and worst-case complexities of an operation which are closely related to the step count for that operation, the amortized complexity of an operation is an accounting artifact that often bears no direct relationship to the actual complexity of that operation. The amortized complexity of an operation could be anything. The only requirement is that the

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-15

sum of the amortized complexities of all operations in the sequence be greater than or equal to the sum of the actual complexities. That is amortized(i)
1in 1in

actual(i)

(1.9)

where amortized(i) and actual(i), respectively, denote the amortized and actual complexities of the ith operation in a sequence of n operations. Because of this requirement on the sum of the amortized complexities of the operations in any sequence of operations, we may use the sum of the amortized complexities as an upper bound on the complexity of any sequence of operations. You may view the amortized cost of an operation as being the amount you charge the operation rather than the amount the operation costs. You can charge an operation any amount you wish so long as the amount charged to all operations in the sequence is at least equal to the actual cost of the operation sequence. Relative to the actual and amortized costs of each operation in a sequence of n operations, we dene a potential function P (i) as below P (i) = amortized(i) actual(i) + P (i 1) (1.10)

That is, the ith operation causes the potential function to change by the dierence between the amortized and actual costs of that operation. If we sum Equation 1.10 for 1 i n, we get P (i) =
1in

(amortized(i) actual(i) + P (i 1))

1in

or (P (i) P (i 1)) =
1in

(amortized(i) actual(i))
1in

or P (n) P (0) =
1in

(amortized(i) actual(i))

From Equation 1.9, it follows that P (n) P (0) 0 (1.11)

When P (0) = 0, the potential P (i) is the amount by which the rst i operations have been overcharged (i.e., they have been charged more than their actual cost). Generally, when we analyze the complexity of a sequence of n operations, n can be any nonnegative integer. Therefore, Equation 1.11 must hold for all nonnegative integers. The preceding discussion leads us to the following three methods to arrive at amortized costs for operations: 1. Aggregate Method In the aggregate method, we determine an upper bound for the sum of the actual costs of the n operations. The amortized cost of each operation is set equal to this upper bound divided by n. You may verify that this assignment of amortized costs satises Equation 1.9 and is, therefore, valid.

2005 by Chapman & Hall/CRC

1-16

Handbook of Data Structures and Applications

2. Accounting Method In this method, we assign amortized costs to the operations (probably by guessing what assignment will work), compute the P (i)s using Equation 1.10, and show that P (n) P (0) 0. 3. Potential Method Here, we start with a potential function (probably obtained using good guess work) that satises Equation 1.11 and compute the amortized complexities using Equation 1.10.

1.7.2

Maintenance Contract

Problem Denition

In January, you buy a new car from a dealer who oers you the following maintenance contract: $50 each month other than March, June, September and December (this covers an oil change and general inspection), $100 every March, June, and September (this covers an oil change, a minor tune-up, and a general inspection), and $200 every December (this covers an oil change, a major tune-up, and a general inspection). We are to obtain an upper bound on the cost of this maintenance contract as a function of the number of months.
Worst-Case Method

We can bound the contract cost for the rst n months by taking the product of n and the maximum cost incurred in any month (i.e., $200). This would be analogous to the traditional way to estimate the complexitytake the product of the number of operations and the worst-case complexity of an operation. Using this approach, we get $200n as an upper bound on the contract cost. The upper bound is correct because the actual cost for n months does not exceed $200n.
Aggregate Method

To use the aggregate method for amortized complexity, we rst determine an upper bound on the sum of the costs for the rst n months. As tight a bound as is possible is desired. The sum of the actual monthly costs of the contract for the rst n months is 200 n/12 + = = = = 100 ( n/3 n/12 ) + 50 (n n/3 ) 100 n/12 + 50 n/3 + 50 n 100 n/12 + 50 n/3 + 50 n 50n(1/6 + 1/3 + 1) 50n(3/2) 75n

The amortized cost for each month is set to $75. Table 1.5 shows the actual costs, the amortized costs, and the potential function value (assuming P (0) = 0) for the rst 16 months of the contract. Notice that some months are charged more than their actual costs and others are charged less than their actual cost. The cumulative dierence between what the operations are charged and their actual costs is given by the potential function. The potential function satises Equation 1.11 for all values of n. When we use the amortized cost of $75 per month, we get $75n as an upper bound on the contract cost for n months. This bound is tighter than the bound of $200n obtained using the worst-case monthly cost.

2005 by Chapman & Hall/CRC

Analysis of Algorithms
month actual cost amortized cost P() 1 50 75 25 2 50 75 50 3 100 75 25 4 50 75 50 5 50 75 75 6 100 75 50 7 50 75 75 8 50 75 100 9 100 75 75 10 50 75 100 11 50 75 125 12 200 75 0 13 50 75 25 14 50 75 50

1-17
15 100 75 25 16 50 75 50

TABLE 1.5

Maintenance contract

Accounting Method

When we use the accounting method, we must rst assign an amortized cost for each month and then show that this assignment satises Equation 1.11. We have the option to assign a dierent amortized cost to each month. In our maintenance contract example, we know the actual cost by month and could use this actual cost as the amortized cost. It is, however, easier to work with an equal cost assignment for each month. Later, we shall see examples of operation sequences that consist of two or more types of operations (for example, when dealing with lists of elements, the operation sequence may be made up of search, insert, and remove operations). When dealing with such sequences we often assign a dierent amortized cost to operations of dierent types (however, operations of the same type have the same amortized cost). To get the best upper bound on the sum of the actual costs, we must set the amortized monthly cost to be the smallest number for which Equation 1.11 is satised for all n. From the above table, we see that using any cost less than $75 will result in P (n) P (0) < 0 for some values of n. Therefore, the smallest assignable amortized cost consistent with Equation 1.11 is $75. Generally, when the accounting method is used, we have not computed the aggregate cost. Therefore, we would not know that $75 is the least assignable amortized cost. So we start by assigning an amortized cost (obtained by making an educated guess) to each of the dierent operation types and then proceed to show that this assignment of amortized costs satises Equation 1.11. Once we have shown this, we can obtain an upper bound on the cost of any operation sequence by computing f (i) amortized(i)
1ik

where k is the number of dierent operation types and f (i) is the frequency of operation type i (i.e., the number of times operations of this type occur in the operation sequence). For our maintenance contract example, we might try an amortized cost of $70. When we use this amortized cost, we discover that Equation 1.11 is not satised for n = 12 (for example) and so $70 is an invalid amortized cost assignment. We might next try $80. By constructing a table such as the one above, we will observe that Equation 1.11 is satised for all months in the rst 12 month cycle, and then conclude that the equation is satised for all n. Now, we can use $80n as an upper bound on the contract cost for n months.
Potential Method

We rst dene a potential function for the analysis. The only guideline you have in dening this function is that the potential function represents the cumulative dierence between the amortized and actual costs. So, if you have an amortized cost in mind, you may be able to use this knowledge to develop a potential function that satises Equation 1.11, and then use the potential function and the actual operation costs (or an upper bound on these actual costs) to verify the amortized costs. If we are extremely experienced, we might start with the potential function

2005 by Chapman & Hall/CRC

1-18

Handbook of Data Structures and Applications

0 25 50 t(n) = 75 100 125

n n n n n n

mod mod mod mod mod mod

12 = 0 12 = 1 or 3 12 = 2, 4, or 6 12 = 5, 7, or 9 12 = 8 or 10 12 = 11

Without the aid of the table (Table 1.5) constructed for the aggregate method, it would take quite some ingenuity to come up with this potential function. Having formulated a potential function and veried that this potential function satises Equation 1.11 for all n, we proceed to use Equation 1.10 to determine the amortized costs. From Equation 1.10, we obtain amortized(i) = actual(i) + P (i) P (i 1). Therefore, amortized(1) = amortized(2) = amortized(3) = actual(1) + P (1) P (0) = 50 + 25 0 = 75 actual(2) + P (2) P (1) = 50 + 50 25 = 75 actual(3) + P (3) P (2) = 100 + 25 50 = 75

and so on. Therefore, the amortized cost for each month is $75. So, the actual cost for n months is at most $75n.

1.7.3

The McWidget Company

Problem Denition

The famous McWidget company manufactures widgets. At its headquarters, the company has a large display that shows how many widgets have been manufactured so far. Each time a widget is manufactured, a maintenance person updates this display. The cost for this update is $c + dm, where c is a xed trip charge, d is a charge per display digit that is to be changed, and m is the number of digits that are to be changed. For example, when the display is changed from 1399 to 1400, the cost to the company is $c + 3d because 3 digits must be changed. The McWidget company wishes to amortize the cost of maintaining the display over the widgets that are manufactured, charging the same amount to each widget. More precisely, we are looking for an amount $e = amortized(i) that should levied against each widget so that the sum of these charges equals or exceeds the actual cost of maintaining/updating the display ($e n actual total cost incurred for rst n widgets for all n 1). To keep the overall selling price of a widget low, we wish to nd as small an e as possible. Clearly, e > c + d because each time a widget is made, at least one digit (the least signicant one) has to be changed.
Worst-Case Method

This method does not work well in this application because there is no nite worst-case cost for a single display update. As more and more widgets are manufactured, the number of digits that need to be changed increases. For example, when the 1000th widget is made, 4 digits are to be changed incurring a cost of c + 4d, and when the 1,000,000th widget is made, 7 digits are to be changed incurring a cost of c + 7d. If we use the worst-case method, the amortized cost to each widget becomes innity.

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-19

widget actual cost amortized cost P() widget actual cost amortized cost P()

1 1 1.12 0.12 15 1 1.12 0.80

2 1 1.12 0.24 16 1 1.12 0.92

3 1 1.12 0.36 17 1 1.12 1.04

4 1 1.12 0.48 18 1 1.12 1.16

5 1 1.12 0.60 19 1 1.12 1.28

6 1 1.12 0.72 20 2 1.12 0.40

7 1 1.12 0.84 21 1 1.12 0.52

8 1 1.12 0.96 22 1 1.12 0.64

9 1 1.12 1.08 23 1 1.12 0.76

10 2 1.12 0.20 24 1 1.12 0.88

11 1 1.12 0.32 25 1 1.12 1.00

12 1 1.12 0.44 26 1 1.12 1.12

13 1 1.12 0.56 27 1 1.12 1.24

14 1 1.12 0.68 28 1 1.12 1.36

TABLE 1.6

Data for widgets

Aggregate Method

Let n be the number of widgets made so far. As noted earlier, the least signicant digit of the display has been changed n times. The digit in the tens place changes once for every ten widgets made, that in the hundreds place changes once for every hundred widgets made, that in the thousands place changes once for every thousand widgets made, and so on. Therefore, the aggregate number of digits that have changed is bounded by n(1 + 1/10 + 1/100 + 1/1000 + ...) = (1.11111...)n So, the amortized cost of updating the display is $c + d(1.11111...)n/n < c + 1.12d. If the McWidget company adds $c + 1.12d to the selling price of each widget, it will collect enough money to pay for the cost of maintaining the display. Each widget is charged the cost of changing 1.12 digits regardless of the number of digits that are actually changed. Table 1.6 shows the actual cost, as measured by the number of digits that change, of maintaining the display, the amortized cost (i.e., 1.12 digits per widget), and the potential function. The potential function gives the dierence between the sum of the amortized costs and the sum of the actual costs. Notice how the potential function builds up so that when it comes time to pay for changing two digits, the previous potential function value plus the current amortized cost exceeds 2. From our derivation of the amortized cost, it follows that the potential function is always nonnegative.
Accounting Method

We begin by assigning an amortized cost to the individual operations, and then we show that these assigned costs satisfy Equation 1.11. Having already done an amortized analysis using the aggregate method, we see that Equation 1.11 is satised when we assign an amortized cost of $c + 1.12d to each display change. Typically, however, the use of the accounting method is not preceded by an application of the aggregate method and we start by guessing an amortized cost and then showing that this guess satises Equation 1.11. Suppose we assign a guessed amortized cost of $c + 2d for each display change. P (n) P (0) = = = (amortized(i) actual(i))
1in

(c + 2d)n
1in

actual(i)

(c + 2d)n (c + (1 + 1/10 + 1/100 + ...)d)n (c + 2d)n (c + 1.12d)n 0

This analysis also shows us that we can reduce the amortized cost of a widget to $c +1.12d.

2005 by Chapman & Hall/CRC

1-20

Handbook of Data Structures and Applications

An alternative proof method that is useful in some analyses involves distributing the excess charge P (i) P (0) over various accounting entities, and using these stored excess charges (called credits) to establish P (i + 1) P (0) 0. For our McWidget example, we use the display digits as the accounting entities. Initially, each digit is 0 and each digit has a credit of 0 dollars. Suppose we have guessed an amortized cost of $c + (1.111...)d. When the rst widget is manufactured, $c + d of the amortized cost is used to pay for the update of the display and the remaining $(0.111...)d of the amortized cost is retained as a credit by the least signicant digit of the display. Similarly, when the second through ninth widgets are manufactured, $c + d of the amortized cost is used to pay for the update of the display and the remaining $(0.111...)d of the amortized cost is retained as a credit by the least signicant digit of the display. Following the manufacture of the ninth widget, the least signicant digit of the display has a credit of $(0.999...)d and the remaining digits have no credit. When the tenth widget is manufactured, $c + d of the amortized cost are used to pay for the trip charge and the cost of changing the least signicant digit. The least signicant digit now has a credit of $(1.111...)d. Of this credit, $d are used to pay for the change of the next least signicant digit (i.e., the digit in the tens place), and the remaining $(0.111...)d are transferred to the tens digit as a credit. Continuing in this way, we see that when the display shows 99, the credit on the tens digit is $(0.999...)d and that on the ones digit (i.e., the least signicant digit) is also $(0.999...)d. When the 100th widget is manufactured, $c + d of the amortized cost are used to pay for the trip charge and the cost of changing the least signicant digit, and the credit on the least signicant digit becomes $(1.111...)d. Of this credit, $d are used to pay for the change of the tens digit from 9 to 0, the remaining $(0.111...)d credit on the ones digit is transferred to the tens digit. The credit on the tens digit now becomes $(1.111...)d. Of this credit, $d are used to pay for the change of the hundreds digit from 0 to 1, the remaining $(0.111...)d credit on the tens digit is transferred to the hundreds digit. The above accounting scheme ensures that the credit on each digit of the display always equals $(0.111...)dv , where v is the value of the digit (e.g., when the display is 206 the credit on the ones digit is $(0.666...)d, the credit on the tens digit is $0, and that on the hundreds digit is $(0.222...)d. From the preceding discussion, it follows that P (n) P (0) equals the sum of the digit credits and this sum is always nonnegative. Therefore, Equation 1.11 holds for all n.
Potential Method

We rst postulate a potential function that satises Equation 1.11, and then use this function to obtain the amortized costs. From the alternative proof used above for the accounting method, we can see that we should use the potential function P (n) = (0.111...)d i vi , where vi is the value of the ith digit of the display. For example, when the display shows 206 (at this time n = 206), the potential function value is (0.888...)d. This potential function satises Equation 1.11. Let q be the number of 9s at the right end of j (i.e., when j = 12903999, q = 3). When the display changes from j to j + 1, the potential change is (0.111...)d(1 9q ) and the actual cost of updating the display is $c + (q + 1)d. From Equation 1.10, it follows that the amortized cost for the display change is

actual cost + potential change = c + (q + 1)d + (0.111...)d(1 9q ) = c + (1.111...)d

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-21

1.7.4

Subset Generation

Problem Denition

The subsets of a set of n elements are dened by the 2n vectors x[1 : n], where each x[i] is either 0 or 1. x[i] = 1 i the ith element of the set is a member of the subset. The subsets of a set of three elements are given by the eight vectors 000, 001, 010, 011, 100, 101, 110, and 111, for example. Starting with an array x[1 : n] has been initialized to zeroes (this represents the empty subset), each invocation of algorithm nextSubset (Figure 1.10) returns the next subset. When all subsets have been generated, this algorithm returns null. public int [] nextSubset() {// return next subset; return null if no next subset // generate next subset by adding 1 to the binary number x[1:n] int i = n; while (i > 0 && x[i] == 1) {x[i] = 0; i--;} if (i == 0) return null; else {x[i] = 1; return x;} } FIGURE 1.10: Subset enumerator. We wish to determine how much time it takes to generate the rst m, 1 m 2n subsets. This is the time for the rst m invocations of nextSubset.
Worst-Case Method

The complexity of nextSubset is (c), where c is the number of x[i]s that change. Since all n of the x[i]s could change in a single invocation of nextSubset, the worst-case complexity of nextSubset is (n). Using the worst-case method, the time required to generate the rst m subsets is O(mn).
Aggregate Method

The complexity of nextSubset equals the number of x[i]s that change. When nextSubset is invoked m times, x[n] changes m times; x[n 1] changes m/2 times; x[n 2] changes m/4 times; x[n 3] changes m/8 times; and so on. Therefore, the sum of the actual costs of the rst m invocations is 0i log2 m (m/2i ) < 2m. So, the complexity of generating the rst m subsets is actually O(m), a tighter bound than obtained using the worst-case method. The amortized complexity of nextSubset is (sum of actual costs)/m < 2m/m = O(1).
Accounting Method

We rst guess the amortized complexity of nextSubset, and then show that this amortized complexity satises Equation 1.11. Suppose we guess that the amortized complexity is 2. To verify this guess, we must show that P (m) P (0) 0 for all m. We shall use the alternative proof method used in the McWidget example. In this method, we distribute the excess charge P (i) P (0) over various accounting entities, and use these

2005 by Chapman & Hall/CRC

1-22

Handbook of Data Structures and Applications

stored excess charges to establish P (i + 1) P (0) 0. We use the x[j ]s as the accounting entities. Initially, each x[j ] is 0 and has a credit of 0. When the rst subset is generated, 1 unit of the amortized cost is used to pay for the single x[j ] that changes and the remaining 1 unit of the amortized cost is retained as a credit by x[n], which is the x[j ] that has changed to 1. When the second subset is generated, the credit on x[n] is used to pay for changing x[n] to 0 in the while loop, 1 unit of the amortized cost is used to pay for changing x[n 1] to 1, and the remaining 1 unit of the amortized cost is retained as a credit by x[n 1], which is the x[j ] that has changed to 1. When the third subset is generated, 1 unit of the amortized cost is used to pay for changing x[n] to 1, and the remaining 1 unit of the amortized cost is retained as a credit by x[n], which is the x[j ] that has changed to 1. When the fourth subset is generated, the credit on x[n] is used to pay for changing x[n] to 0 in the while loop, the credit on x[n 1] is used to pay for changing x[n 1] to 0 in the while loop, 1 unit of the amortized cost is used to pay for changing x[n 2] to 1, and the remaining 1 unit of the amortized cost is retained as a credit by x[n 2], which is the x[j ] that has changed to 1. Continuing in this way, we see that each x[j ] that is 1 has a credit of 1 unit on it. This credit is used to pay the actual cost of changing this x[j ] from 1 to 0 in the while loop. One unit of the amortized cost of nextSubset is used to pay for the actual cost of changing an x[j ] to 1 in the else clause, and the remaining one unit of the amortized cost is retained as a credit by this x[j ]. The above accounting scheme ensures that the credit on each x[j ] that is 1 is exactly 1, and the credit on each x[j ] that is 0 is 0. From the preceding discussion, it follows that P (m) P (0) equals the number of x[j ]s that are 1. Since this number is always nonnegative, Equation 1.11 holds for all m. Having established that the amortized complexity of nextSubset is 2 = O(1), we conclude that the complexity of generating the rst m subsets equals m amortized complexity = O(m).
Potential Method

We rst postulate a potential function that satises Equation 1.11, and then use this function to obtain the amortized costs. Let P (j ) be the potential just after the j th subset is generated. From the proof used above for the accounting method, we can see that we should dene P (j ) to be equal to the number of x[i]s in the j th subset that are equal to 1. By denition, the 0th subset has all x[i] equal to 0. Since P (0) = 0 and P (j ) 0 for all j , this potential function P satises Equation 1.11. Consider any subset x[1 : n]. Let q be the number of 1s at the right end of x[] (i.e., x[n], x[n 1], , x[n q + 1], are all 1s). Assume that there is a next subset. When the next subset is generated, the potential change is 1 q because q 1s are replaced by 0 in the while loop and a 0 is replaced by a 1 in the else clause. The actual cost of generating the next subset is q + 1. From Equation 1.10, it follows that, when there is a next subset, the amortized cost for nextSubset is actual cost + potential change = q + 1 + 1 q = 2 When there is no next subset, the potential change is q and the actual cost of nextSubset is q . From Equation 1.10, it follows that, when there is no next subset, the amortized cost for nextSubset is actual cost + potential change = q q = 0 Therefore, we can use 2 as the amortized complexity of nextSubset. Consequently, the actual cost of generating the rst m subsets is O(m).

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-23

1.8

Practical Complexities

We have seen that the time complexity of a program is generally some function of the problem size. This function is very useful in determining how the time requirements vary as the problem size changes. For example, the run time of an algorithm whose complexity is (n2 ) is expected to increase by a factor of 4 when the problem size doubles and by a factor of 9 when the problem size triples. The complexity function also may be used to compare two algorithms P and Q that perform the same task. Assume that algorithm P has complexity (n) and that algorithm Q has complexity (n2 ). We can assert that algorithm P is faster than algorithm Q for suciently large n. To see the validity of this assertion, observe that the actual computing time of P is bounded from above by cn for some constant c and for all n, n n1 , while that of Q is bounded from below by dn2 for some constant d and all n, n n2 . Since cn dn2 for n c/d, algorithm P is faster than algorithm Q whenever n max{n1 , n2 , c/d}. One should always be cautiously aware of the presence of the phrase suciently large in the assertion of the preceding discussion. When deciding which of the two algorithms to use, we must know whether the n we are dealing with is, in fact, suciently large. If algorithm P actually runs in 106 n milliseconds while algorithm Q runs in n2 milliseconds and if we always have n 106 , then algorithm Q is the one to use. To get a feel for how the various functions grow with n, you should study Figures 1.11 and 1.12 very closely. These gures show that 2n grows very rapidly with n. In fact, if a algorithm needs 2n steps for execution, then when n = 40, the number of steps needed is approximately 1.1 1012 . On a computer performing 1,000,000,000 steps per second, this algorithm would require about 18.3 minutes. If n = 50, the same algorithm would run for about 13 days on this computer. When n = 60, about 310.56 years will be required to execute the algorithm, and when n = 100, about 4 1013 years will be needed. We can conclude that the utility of algorithms with exponential complexity is limited to small n (typically n 40).

log n 0 1 2 3 4 5

n 1 2 4 8 16 32

n log n 0 2 8 24 64 160

n2 1 4 16 64 256 1024

n3 1 8 64 512 4096 32,768

2n 2 4 16 256 65,536 4,294,967,296

FIGURE 1.11: Value of various functions. Algorithms that have a complexity that is a high-degree polynomial are also of limited utility. For example, if an algorithm needs n10 steps, then our 1,000,000,000 steps per second computer needs 10 seconds when n = 10; 3171 years when n = 100; and 3.17 1013 years when n = 1000. If the algorithms complexity had been n3 steps instead, then the computer would need 1 second when n = 1000, 110.67 minutes when n = 10,000, and 11.57 days when n = 100,000. Figure 1.13 gives the time that a 1,000,000,000 instructions per second computer needs to execute an algorithm of complexity f (n) instructions. One should note that currently only the fastest computers can execute about 1,000,000,000 instructions per second. From a

2005 by Chapman & Hall/CRC

1-24

Handbook of Data Structures and Applications

2n 60 n2

50

40 n logn 30 f 20

10

n logn

4 n

10

FIGURE 1.12: Plot of various functions. practical standpoint, it is evident that for reasonably large n (say n > 100) only algorithms of small complexity (such as n, n log n, n2 , and n3 ) are feasible. Further, this is the case even if we could build a computer capable of executing 1012 instructions per second. In this case the computing times of Figure 1.13 would decrease by a factor of 1000. Now when n = 100, it would take 3.17 years to execute n10 instructions and 4 1010 years to execute 2n instructions.

n 10 20 30 40 50 100 103 104 105 106

n .01 s .02 s .03 s .04 s .05 s .10 s 1 s 10 s 100 s 1 ms

n log2 n .03 s .09 s .15 s .21 s .28 s .66 s 9.96 s 130 s 1.66 ms 19.92 ms

n2 .1 s .4 s .9 s 1.6 s 2.5 s 10 s 1 ms 100 ms 10 s 16.67 m

n3 1 s 8 s 27 s 64 s 125 s 1 ms 1s 16.67 m 11.57 d 31.71 y

f (n)

n4 10 s 160 s 810 s 2.56 ms 6.25 ms 100 ms 16.67 m 115.7 d 3171 y 3.17 107 y

n10 10 s 2.84 h 6.83 d 121 d 3.1 y 3171 y 3.17 1013 y 3.17 1023 y 3.17 1033 y 3.17 1043 y

2n 1 s 1 ms 1s 18 m 13 d 4 1013 y 32 10283 y

s = microsecond = 106 seconds; ms = milliseconds = 103 seconds s = seconds; m = minutes; h = hours; d = days; y = years FIGURE 1.13: Run times on a 1,000,000,000 instructions per second computer.

Acknowledgment
This work was supported, in part, by the National Science Foundation under grant CCR9912395.

2005 by Chapman & Hall/CRC

Analysis of Algorithms

1-25

References
[1] T. Cormen, C. Leiserson, and R. Rivest, Introduction to Algorithms, McGraw-Hill, New York, NY, 1992. [2] J. Hennessey and D. Patterson, Computer Organization and Design, Second Edition, Morgan Kaufmann Publishers, Inc., San Francisco, CA, 1998, Chapter 7. [3] E. Horowitz, S. Sahni, and S. Rajasekaran, Fundamentals of Computer Algorithms, W. H. Freeman and Co., New York, NY, l998. [4] G. Rawlins, Compared to What: An Introduction to the Analysis of Algorithms, W. H. Freeman and Co., New York, NY, 1992. [5] S. Sahni, Data Structures, Algorithms, and Applications in Java, McGraw-Hill, NY, 2000.

2005 by Chapman & Hall/CRC

2
Basic Structures
2.1 2.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Operations on an Array Sorted Arrays Array Doubling Multiple Lists in a Single Array Heterogeneous Arrays Multidimensional Arrays Sparse Matrices

2-1 2-1

2.3

Linked Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Chains Circular Lists Doubly Linked Circular Lists Generalized Lists

2-7 2-12

Dinesh P. Mehta
Colorado School of Mines

2.4

Stacks and Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Stack Implementation

Queue Implementation

2.1

Introduction

In this chapter, we review several basic structures that are usually taught in a rst class on data structures. There are several text books that cover this material, some of which are listed here [14]. However, we believe that it is valuable to review this material for the following reasons: 1. In practice, these structures are used more often than all of the other data structures discussed in this handbook combined. 2. These structures are used as basic building blocks on which other more complicated structures are based. Our goal is to provide a crisp and brief review of these structures. For a more detailed explanation, the reader is referred to the text books listed at the end of this chapter. In the following, we assume that the reader is familiar with arrays and pointers.

2.2

Arrays

An array is conceptually dened as a collection of <index,value> pairs. The implementation of the array in modern programming languages such as C++ and Java uses indices starting at 0. Languages such as Pascal permitted one to dene an array on an arbitrary range of integer indices. In most applications, the array is the most convenient method to store a collection of objects. In these cases, the index associated with a value is unimportant. For example, if an array city is being used to store a list of cities in no particular order, it doesnt really matter whether city[0] is Denver or Mumbai. If, on the other hand, an array name is being used to store a list of student names in alphabetical order, then,

2-1

2005 by Chapman & Hall/CRC

2-2

Handbook of Data Structures and Applications

although the absolute index values dont matter, the ordering of names associated with the ordering of the index does matter: i.e., name[i] must precede name[j] in alphabetical order, if i < j . Thus, one may distinguish between sorted arrays and unsorted arrays. Sometimes arrays are used so that the index does matter. For example, suppose we are trying to represent a histogram: we want to maintain a count of the number of students that got a certain score on an exam from a scale of 0 to 10. If score[5] = 7, this means that 7 students received a score of 5. In this situation, it is possible that the desired indices are not supported by the language. For example, C++ does not directly support using indices such as blue, green, and red. This may be rectied by using enumerated types to assign integer values to the indices. In cases when the objects in the array are large and unwieldy and have to be moved around from one array location to another, it may be advantageous to store pointers or references to objects in the array rather than the objects themselves. Programming languages provide a mechanism to retrieve the value associated with a supplied index or to associate a value with an index. Programming languages like C++ do not explicitly maintain the size of the array. Rather, it is the programmers responsibility to be aware of the array size. Further, C++ does not provide automatic range-checking. Again, it is the programmers responsibility to ensure that the index being supplied is valid. Arrays are usually allocated contiguous storage in the memory. An array may be allocated statically (i.e., during compile-time) or dynamically (i.e., during program execution). For example, in C++, a static array is dened as: int list[20]; while a dynamic one is dened as: int* list; . . list = new int[25]; An important dierence between static and dynamic arrays is that the size of a static array cannot be changed during run time, while that of a dynamic array can (as we will see in Section 2.2.3).

2.2.1

Operations on an Array

1. Retrieval of an element: Given an array index, retrieve the corresponding value. This can be accomplished in O(1) time. This is an important advantage of the array relative to other structures. If the array is sorted, this enables one to compute the minimum, maximum, median (or in general, the ith smallest element) essentially for free in O(1) time. 2. Search: Given an element value, determine whether it is present in the array. If the array is unsorted, there is no good alternative to a sequential search that iterates through all of the elements in the array and stops when the desired element is found: int SequentialSearch(int* array, int n, int x) // search for x in array[n] { for (int i = 0; i < n; i++) if (array[i] == x) return i; // search succeeded

2005 by Chapman & Hall/CRC

Basic Structures return -1; // search failed } In the worst case, this requires O(n) time. If, however, the array is sorted, binary search can be used. int BinarySearch(int* array, int n, int x) { int first = 0, mid, last = n-1; while (first < last) { mid = (first + last)/2; if (array[mid] == x) return mid; // search succeeded if (x < array[mid]) last = mid-1; else first = mid+1; } return -1; // search failed } Binary search only requires O(log n) time. 3. Insertion and Deletion: These operations can be the arrays Achilles heel. First, consider a sorted array. It is usually assumed that the array that results from an insertion or deletion is to be sorted. The worst case scenario presents itself when an element that is smaller than all of the elements currently in the array is to be inserted. This element will be placed in the leftmost location. However, to make room for it, all of the existing elements in the array will have to be shifted one place to the right. This requires O(n) time. Similarly, a deletion from the leftmost element leaves a vacant location. Actually, this location can never be vacant because it refers to a word in memory which must contain some value. Thus, if the program accesses a vacant location, it doesnt have any way to know that the location is vacant. It may be possible to establish some sort of code based on our knowledge of the values contained in the array. For example, if it is known that an array contains only positive integers, then one could use a zero to denote a vacant location. Because of these and other complications, it is best to eliminate vacant locations that are interspersed among occupied locations by shifting elements to the left so that all vacant locations are placed to the right. In this case, we know which locations are vacant by maintaining an integer variable which contains the number of locations starting at the left that are currently in use. As before, this shifting requires O(n) time. In an unsorted array, the eciency of insertion and deletion depends on where elements are to be added or removed. If it is known for example that insertion and deletion will only be performed at the right end of the array, then these operations take O(1) time as we will see later when we discuss stacks.

2-3

2.2.2

Sorted Arrays

We have already seen that there are several benets to using sorted arrays, namely: searching is faster, computing order statistics (the ith smallest element) is O(1), etc. This is the rst illustration of a key concept in data structures that will be seen several times in this handbook: the concept of preprocessing data to make subsequent queries ecient. The idea is that we are often willing to invest some time at the beginning in setting up a data structure so that subsequent operations on it become faster. Some sorting algorithms such

2005 by Chapman & Hall/CRC

2-4

Handbook of Data Structures and Applications

as heap sort and merge sort require O(n log n) time in the worst case, whereas other simpler sorting algorithms such as insertion sort, bubble sort and selection sort require O(n2 ) time in the worst case. Others such as quick sort have a worst case time of O(n2 ), but require O(n log n) on the average. Radix sort requires (n) time for certain kinds of data. We refer the reader to [5] for a detailed discussion. However, as we have seen earlier, insertion into and deletion from a sorted array can take (n) time, which is large. It is possible to merge two sorted arrays into a single sorted array in time linear in the sum of their sizes. However, the usual implementation needs additional (n) space. See [6] for an O(1)-space merge algorithm.

2.2.3

Array Doubling

To increase the length of a (dynamically allocated) one-dimensional array a that contains elements in positions a[0..n 1], we rst dene an array of the new length (say m), then copy the n elements from a to the new array, and nally change the value of a so that it references the new array. It takes (m) time to create an array of length m because all elements of the newly created array are initialized to a default value. It then takes an additional (n) time to copy elements from the source array to the destination array. Thus, the total time required for this operation is (m + n). This operation is used in practice to increase the array size when it becomes full. The new array is usually twice the length of the original; i.e., m = 2n. The resulting complexity ((n)) would normally be considered to be expensive. However, when this cost is amortized over the subsequent n insertions, it in fact only adds (1) time per insertion. Since the cost of an insertion is (1), this does not result in an asymptotic increase in insertion time. In general, increasing array size by a constant factor every time its size is to be increased does not adversely aect the asymptotic complexity. A similar approach can be used to reduce the array size. Here, the array size would be reduced by a constant factor every time.

2.2.4

Multiple Lists in a Single Array

The array is wasteful of space when it is used to represent a list of objects that changes over time. In this scenario, we usually allocate a size greater than the number of objects that have to be stored, for example by using the array doubling idea discussed above. Consider a completely-lled array of length 8192 into which we are inserting an additional element. This insertion causes the array-doubling algorithm to create a new array of length 16,384 into which the 8192 elements are copied (and the new element inserted) before releasing the original array. This results in a space requirement during the operation which is almost three times the number of elements actually present. When several lists are to be stored, it is more ecient to store them all in a single array rather than allocating one array for each list. Although this representation of multiple lists is more space-ecient, insertions can be more expensive because it may be necessary to move elements belonging to other lists in addition to elements in ones own list. This representation is also harder to implement.

10 11 12 List 2

25 26 27 28 29 List 3

List 1

FIGURE 2.1: Multiple lists in a single array.

2005 by Chapman & Hall/CRC

Basic Structures

2-5

2.2.5

Heterogeneous Arrays

The denition of an array in modern programming languages requires all of its elements to be of the same type. How do we then address the scenario where we wish to use the array to store elements of dierent types? In earlier languages like C, one could use the union facility to articially coalesce the dierent types into one type. We could then dene an array on this new type. The kind of object that an array element actually contains is determined by a tag. The following denes a structure that contains one of three types of data. struct Animal{ int id; union { Cat c; Dog d; Fox f; } } The programmer would have to establish a convention on how the id tag is to be used: for example, that id = 0 means that the animal represented by the struct is actually a cat, etc. The union allocates memory for the largest type among Cat, Dog, and Fox. This is wasteful of memory if there is a great disparity among the sizes of the objects. With the advent of object-oriented languages, it is now possible to dene the base type Animal. Cat, Dog, and Fox may be implemented using inheritance as derived types of Animal. An array of pointers to Animal can now be dened. These pointers can be used to refer to any of Cat, Dog, and Fox.

2.2.6

Multidimensional Arrays

Row- or Column Major Representation

Earlier representations of multidimensional arrays mapped the location of each element of the multidimensional array into a location of a one- dimensional array. Consider a twodimensional array with r rows and c columns. The number of elements in the array n = rc. The element in location [i][j ], 0 i < r and 0 j < c, will be mapped onto an integer in the range [0, n 1]. If this is done in row-major order the elements of row 0 are listed in order from left to right followed by the elements of row 1, then row 2, etc. the mapping function is ic + j . If elements are listed in column-major order, the mapping function is jr + i. Observe that we are required to perform a multiplication and an addition to compute the location of an element in an array.
Array of Arrays Representation

In Java, a two-dimensional array is represented as a one-dimensional array in which each element is, itself, a one-dimensional array. The array int [][] x = new int[4][5]; is actually a one-dimensional array whose length is 4. Each element of x is a one-dimensional array whose length is 5. Figure 2.2 shows an example. This representation can also be used in C++ by dening an array of pointers. Each pointer can then be used to point to a

2005 by Chapman & Hall/CRC

2-6

Handbook of Data Structures and Applications

[0] [1] [2] [3] [4] x[0] x[1] x[2] x[3]

FIGURE 2.2: The Array of Arrays Representation.

dynamically-created one-dimensional array. The element x[i][j ] is found by rst retrieving the pointer x[i]. This gives the address in memory of x[i][0]. Then x[i][j ] refers to the element j in row i. Observe that this only requires the addition operator to locate an element in a one-dimensional array.
Irregular Arrays

A two-dimensional array is regular in that every row has the same number of elements. When two or more rows of an array have dierent number of elements, we call the array irregular. Irregular arrays may also be created and used using the array of arrays representation.

2.2.7

Sparse Matrices

A matrix is sparse if a large number of its elements are 0. Rather than store such a matrix as a two-dimensional array with lots of zeroes, a common strategy is to save space by explicitly storing only the non-zero elements. This topic is of interest to the scientic computing community because of the large sizes of some of the sparse matrices it has to deal with. The specic approach used to store matrices depends on the nature of sparsity of the matrix. Some matrices, such as the tridiagonal matrix have a well-dened sparsity pattern. The tridiagonal matrix is one where all of the nonzero elements lie on one of three diagonals: the main diagonal and the diagonals above and below it. See Figure 2.3(a).

2 1 0 0 0

1 3 1 0 0

0 4 1 4 0

0 0 2 7 3

0 0 0 4 5

1 2 4 3 2

0 3 1 3 4

0 0 2 2 1

0 0 0 1 3

0 0 0 0 3

1 0 0 0 0

3 1 0 0 0

1 3 2 0 0

5 2 3 1 0

8 4 4 2 4

Tridiagonal Matrix (a)

Lower Triangular Matrix (b)

Upper Triangular Matrix (c)

FIGURE 2.3: Matrices with regular structures.

2005 by Chapman & Hall/CRC

Basic Structures

2-7

There are several ways to represent this matrix as a one-dimensional array. We could order elements by rows giving [2,1,1,3,4,1,1,2,4,7,4,3,5] or by diagonals giving [1,1,4,3,2,3,1,7,5,1,4,2,4]. Figure 2.3 shows other special matrices: the upper and lower triangular matrices which can also be represented using a one-dimensional representation. Other sparse matrices may have an irregular or unstructured pattern. Consider the matrix in Figure 2.4(a). We show two representations. Figure 2.4(b) shows a one-dimensional array of triples, where each triple represents a nonzero element and consists of the row, column, and value. Figure 2.4(c) shows an irregular array representation. Each row is represented by a one-dimensional array of pairs, where each pair contains the column number and the corresponding nonzero value.

6 4 0 0

0 4 1 0

0 0 0 0 (0,6) (0,4) (1,1) (3,1)

2 0 0 1

0 0 2 1 (3,2) (1,4) (4,2) (4,1)

5 1 0 0 (5,5) (5,1) row col val 0 0 6 0 3 2 0 5 5 1 0 4 1 1 4 1 5 1 2 1 1 2 4 2 3 3 1 3 4 1

(a)

(b)

(c)
FIGURE 2.4: Unstructured matrices.

2.3

Linked Lists

The linked list is an alternative to the array when a collection of objects is to be stored. The linked list is implemented using pointers. Thus, an element (or node) of a linked list contains the actual data to be stored and a pointer to the next node. Recall that a pointer is simply the address in memory of the next node. Thus, a key dierence from arrays is that a linked list does not have to be stored contiguously in memory.

List first

ListNode data link .... .... 0

FIGURE 2.5: The structure of a linked list.

2005 by Chapman & Hall/CRC

2-8

Handbook of Data Structures and Applications

The code fragment below denes a linked list data structure, which is also illustrated in Figure 2.5: class ListNode { friend class List; private: int data; ListNode *link; } class List { public: // List manipulation operations go here ... private: ListNode *first; } A chain is a linked list where each node contains a pointer to the next node in the list. The last node in the list contains a null (or zero) pointer. A circular list is identical to a chain except that the last node contains a pointer to the rst node. A doubly linked circular list diers from the chain and the circular list in that each node contains two pointers. One points to the next node (as before), while the other points to the previous node.

2.3.1

Chains

The following code searches for a key k in a chain and returns true if the key is found and false, otherwise. bool List::Search(int k) { for (ListNode *current = first; current; current = current->next) if (current->data == k) then return true; return false; } In the worst case, Search takes (n) time. In order to insert a node newnode in a chain immediately after node current, we simply set newnodes pointer to the node following current (if any) and currents pointer to newnode as shown in the Figure 2.6.

current first .... .... 0

newnode
FIGURE 2.6: Insertion into a chain. The dashed links show the pointers after newnode has been inserted.

2005 by Chapman & Hall/CRC

Basic Structures

2-9

To delete a node current, it is necessary to have a pointer to the node preceding current. This nodes pointer is then set to current->next and node current is freed. Both insertion and deletion can be accomplished in O(1) time provided that the required pointers are initially available. Whether this is true or not depends on the context in which these operations are called. For example, if you are required to delete the node with key 50, if it exists, from a linked list, you would rst have to search for 50. Your search algorithm would maintain a trailing pointer so that when 50 is found, a pointer to the previous node is available. Even though, deletion takes (1) time, deletion in this context would require (n) time in the worst case because of the search. In some cases, the context depends on how the list is organized. For example, if the list is to be sorted, then node insertions should be made so as to maintain the sorted property (which could take (n) time). On the other hand, if the list is unsorted, then a node insertion can take place anywhere in the list. In particular, the node could be inserted at the front of the list in (1) time. Interestingly, the author has often seen student code in which the insertion algorithm traverses the entire linked list and inserts the new element at the end of the list! As with arrays, chains can be sorted or unsorted. Unfortunately, however, many of the benets of a sorted array do not extend to sorted linked lists because arbitrary elements of a linked list cannot be accessed quickly. In particular, it is not possible to carry out binary search in O(log n) time. Nor is it possible to locate the ith smallest element in O(1) time. On the other hand, merging two sorted lists into one sorted list is more convenient than merging two sorted arrays into one sorted array because the traditional implementation requires space to be allocated for the target array. A code fragment illustrating the merging of two sorted lists is shown below. This is a key operation in mergesort: void Merge(List listOne, List listTwo, List& merged) { ListNode* one = listOne.first; ListNode* two = listTwo.first; ListNode* last = 0; if (one == 0) {merged.first = two; return;} if (two == 0) {merged.first = one; return;} if (one->data < two->data) last = merged.first = one; else last = merged.first = two; while (one && two) if (one->data < two->data) { last->next = one; last= one; one = one->next; } else { last->next = two; last = two; two = two->next; } if (one) last->next = one; else last->next = two; } The merge operation is not dened when lists are unsorted. However, one may need to combine two lists into one. This is the concatenation operation. With chains, the best approach is to attach the second list to the end of the rst one. In our implementation of the linked list, this would require one to traverse the rst list until the last node is encountered and then set its next pointer to point to the rst element of the second list. This requires

2005 by Chapman & Hall/CRC

2-10

Handbook of Data Structures and Applications

time proportional to the size of the rst linked list. This can be improved by maintaining a pointer to the last node in the linked list. It is possible to traverse a singly linked list in both directions (i.e., left to right and a restricted right-to-left traversal) by reversing links during the left-to-right traversal. Figure 2.7 shows a possible conguration for a list under this scheme.

FIGURE 2.7: Illustration of a chain traversed in both directions.

As with the heterogeneous arrays described earlier, heterogeneous lists can be implemented in object-oriented languages by using inheritance.

2.3.2

Circular Lists

In the previous section, we saw that to concatenate two unsorted chains eciently, one needs to maintain a rear pointer in addition to the rst pointer. With circular lists, it is possible to accomplish this with a single pointer as follows: consider the circular list in Figure 2.8. The second node in the list can be accessed through the rst in O(1) time.

Circlist first

ListNode data link .... ....

FIGURE 2.8: A circular list. Now, consider the list that begins at this second node and ends at the rst node. This may be viewed as a chain with access pointers to the rst and last nodes. Concatenation can now be achieved in O(1) time by linking the last node of one chain to the rst node of the second chain and vice versa.

2.3.3

Doubly Linked Circular Lists

A node in a doubly linked list diers from that in a chain or a singly linked list in that it has two pointers. One points to the next node as before, while the other points to the previous node. This makes it possible to traverse the list in both directions. We observe that this is possible in a chain as we saw in Figure 2.7. The dierence is that with a doubly linked list, one can initiate the traversal from any arbitrary node in the list. Consider the following problem: we are provided a pointer x to a node in a list and are required to delete

2005 by Chapman & Hall/CRC

Basic Structures

2-11

it as shown in Figure 2.9. To accomplish this, one needs to have a pointer to the previous node. In a chain or a circular list, an expensive list traversal is required to gain access to this previous node. However, this can be done in O(1) time in a doubly linked circular list. The code fragment that accomplishes this is as below:
x

first

first

FIGURE 2.9: Deletion from a doubly linked list.

void DblList::Delete(DblListNode* x) { x->prev->next = x->next; x->next->prev = x->prev; delete x; } An application of doubly linked lists is to store a list of siblings in a Fibonacci heap (Chapter 7).

2.3.4

Generalized Lists

A generalized list A is a nite sequence of n 0 elements, e0 , e1 , ..., en1 , where ei is either an atom or a generalized list. The elements ei that are not atoms are said to be sublists of A. Consider the generalized list A = ((a, b, c), ((d, e), f ), g ). This list contains three elements: the sublist (a, b, c), the sublist ((d, e), f ) and the atom g . The generalized list may be implemented by employing a GenListNode type as follows: private: GenListNode* next; bool tag; union { char data;

2005 by Chapman & Hall/CRC

2-12

Handbook of Data Structures and Applications GenListNode* down;

}; If tag is true, the element represented by the node is a sublist and down points to the rst node in the sublist. If tag is false, the element is an atom whose value is contained in data. In both cases, next simply points to the next element in the list. Figure 2.10 illustrates the representation.

FIGURE 2.10: Generalized List for ((a,b,c),((d,e),f),g).

2.4

Stacks and Queues

The stack and the queue are data types that support insertion and deletion operations with well-dened semantics. Stack deletion deletes the element in the stack that was inserted the last, while a queue deletion deletes the element in the queue that was inserted the earliest. For this reason, the stack is often referred to as a LIFO (Last In First Out) data type and the queue as an FIFO (First In First out) data type. A deque (double ended queue) combines the stack and the queue by supporting both types of deletions. Stacks and queues nd a lot of applications in Computer Science. For example, a system stack is used to manage function calls in a program. When a function f is called, the system creates an activation record and places it on top of the system stack. If function f calls function g , the local variables of f are added to its activation record and an activation record is created for g . When g terminates, its activation record is removed and f continues executing with the local variables that were stored in its activation record. A queue is used to schedule jobs at a resource when a rst-in rst-out policy is to be implemented. Examples could include a queue of print-jobs that are waiting to be printed or a queue of packets waiting to be transmitted over a wire. Stacks and queues are also used routinely to implement higher-level algorithms. For example, a queue is used to implement a breadthrst traversal of a graph. A stack may be used by a compiler to process an expression such as (a + b) (c + d).

2.4.1

Stack Implementation

Stacks and queues can be implemented using either arrays or linked lists. Although the burden of a correct stack or queue implementation appears to rest on deletion rather than

2005 by Chapman & Hall/CRC

Basic Structures

2-13

insertion, it is convenient in actual implementations of these data types to place restrictions on the insertion operation as well. For example, in an array implementation of a stack, elements are inserted in a left-to-right order. A stack deletion simply deletes the rightmost element. A simple array implementation of a stack class is shown below: class Stack { public: Stack(int maxSize = 100); // 100 is default size void Insert(int); int* Delete(int&); private: int *stack; int size; int top; // highest position in array that contains an element }; The stack operations are implemented as follows: Stack::Stack(int maxSize): size(maxSize) { stack = new int[size]; top = -1; } void Stack::Insert(int x) { if (top == size-1) cerr << "Stack Full" << endl; else stack[++top] = x; } int* Stack::Delete(int& x) { if (top == -1) return 0; // stack empty else { x = stack[top--]; return &x; } } The operation of the following code fragment is illustrated in Figure 2.11. Stack s; int x; s.Insert(10); s.Insert(20); s.Insert(30); s.Delete(x); s.Insert(40); s.Delete(x); It is easy to see that both stack operations take O(1) time. The stack data type can also be implemented using linked lists by requiring all insertions and deletions to be made at the front of the linked list.

2005 by Chapman & Hall/CRC

2-14

Handbook of Data Structures and Applications

2 1 0 top = 1 10 top = 0 20 10 top = 1

30 20 10 top = 2 20 10 top = 1

40 20 10 top = 2

2 20 1 10 0 top = 1

FIGURE 2.11: Stack operations.

2.4.2

Queue Implementation

An array implementation of a queue is a bit trickier than that of a stack. Insertions can be made in a left-to-right fashion as with a stack. However, deletions must now be made from the left. Consider a simple example of an array of size 5 into which the integers 10, 20, 30, 40, and 50 are inserted as shown in Figure 2.12(a). Suppose three elements are subsequently deleted (Figure 2.12(b)).

10 20 30 40 50 front rear

40 50 front rear

(a)

(b)

FIGURE 2.12: Pitfalls of a simple array implementation of a queue.

What if we are now required to insert the integer 60. On one hand, it appears that we are out of room as there is no more place to the right of 50. On the other hand, there are three locations available to the left of 40. This suggests that we use a circular array implementation of a queue, which is described below. class Queue { public: Queue(int maxSize = 100); // 100 is default size void Insert(int); int* Delete(int&); private: int *queue; int size; int front, rear; }; The queue operations are implemented below: Queue::Queue(int maxSize): size(maxSize) { queue= new int[size];

2005 by Chapman & Hall/CRC

Basic Structures front = rear = 0; } void Queue::Insert(int x) { int k = (rear + 1) % size; if (front == k) cerr << "Queue Full!" <, endl; else queue[rear = k] = x; } int* Queue::Delete(int& x) { if (front == rear) return 0; // queue is empty x = queue[++front %= size]; return &x; }

2-15

Figure 2.13 illustrates the operation of this code on an example. The rst gure shows an empty queue with rst = rear = 0. The second gure shows the queue after the integer 10 is inserted. The third gure shows the queue when 20, 30, 40, 50, and 60 have been inserted. The fourth gure shows the queue after 70 is inserted. Notice that, although one slot remains empty, the queue is now full because Queue::Insert will not permit another element to be inserted. If it did permit an insertion at this stage, rear and front would be the same. This is the condition that Queue:Delete checks to determine whether the queue is empty! This would make it impossible to distinguish between the queue being full and being empty. The fth gure shows the queue after two integers (10 and 20) are deleted. The last gure shows a full queue after the insertion of integers 80 and 90.
3 2 1 front rear 0 3 2 20 1 10 70 front 0 7 rear 0 rear 7 4 30 40 50 5 60 6 front 2 1 70 7 4 5 6 rear 2 1 10 0 3 7 4 30 40 50 front 5 2 1 90 rear 80 0 70 7 3 4 5 6 2 20 1 10 front 0 3 7 4 30 40 50 5 3 30 4 40 50 5 60 6 rear

front

60 6

60 6

FIGURE 2.13: Implementation of a queue in a circular array.

2005 by Chapman & Hall/CRC

2-16

Handbook of Data Structures and Applications

It is easy to see that both queue operations take O(1) time. The queue data type can also be implemented using linked lists by requiring all insertions and deletions to be made at the front of the linked list.

Acknowledgments
The author would like to thank Kun Gao, Dean Simmons and Mr. U. B. Rao for proofreading this chapter. This work was supported, in part, by the National Science Foundation under grant CCR-9988338.

References
[1] D. Knuth, The Art of Computer Programming, Vol. 1, Fundamental Algorithms, Third edition. Addison-Wesley, NY, 1997. [2] S. Sahni, Data Structures, Algorithms, and Applications in C++, McGraw-Hill, NY, 1998. [3] S. Sahni, Data Structures, Algorithms, and Applications in Java, McGraw-Hill, NY, 2000. [4] E. Horowitz, S. Sahni, and D. Mehta, Fundamentals of Data Structures in C++, W. H. Freeman, NY, 1995. [5] D. Knuth, The Art of Computer Programming, Vol. 3, Sorting and Searching, Second edition. Addison-Wesley, NY, 1997. [6] Practical in-place merging, B. Huang and M. Langston, Communications of the ACM, 31:3, 1988, pp. 348-352.

2005 by Chapman & Hall/CRC

3
Trees
3.1 3.2 3.3 3.4 3.5 3.6 3.7 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tree Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
List Representation Left Child-Right Sibling Representation Binary Tree Representation

3-1 3-3 3-4 3-7 3-9 3-10 3-13 3-16

Binary Trees and Properties . . . . . . . . . . . . . . . . . . . . . .

Properties

Binary Tree Representation

Binary Tree Traversals . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Inorder Traversal Preorder Traversal Traversal Level Order Traversal Threads Tree

Postorder

Threaded Binary Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Inorder Traversal of a Threaded Binary

Binary Search Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Denition

Search

Insert

Delete

Miscellaneous

Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Priority Queues Denition of a Max-Heap Insertion Deletion

Dinesh P. Mehta
Colorado School of Mines

3.8

Tournament Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Winner Trees

Loser Trees

3.1

Introduction

The tree is a natural representation for hierarchical information. Thus, trees are used to represent genealogical information (e.g., family trees and evolutionary trees), organizational charts in large companies, the directory structure of a le system on a computer, parse trees in compilers and the structure of a knock-out sports tournament. The Dewey decimal notation, which is used to classify books in a library, is also a tree structure. In addition to these and other applications, the tree is used to design fast algorithms in computer science because of its eciency relative to the simpler data structures discussed in Chapter 2. Operations that take linear time on these structures often take logarithmic time on an appropriately organized tree structure. For example, the average time complexity for a search on a key is linear on a linked list and logarithmic on a binary search tree. Many of the data structures discussed in succeeding chapters of this handbook are tree structures. Several kinds of trees have been dened in the literature: 1. Free or unrooted tree: this is dened as a graph (a set of vertices and a set of edges that join pairs of vertices) such that there exists a unique path between any two vertices in the graph. The minimum spanning tree of a graph is a well-known example of a free tree. Graphs are discussed in Chapter 4. 2. Rooted tree: a nite set of one or more nodes such that

3-1

2005 by Chapman & Hall/CRC

3-2

Handbook of Data Structures and Applications (a) There is a special node called the root. (b) The remaining nodes are partitioned into n 0 disjoint sets T1 , ..., Tn , where each of these sets is a tree. T1 , ..., Tn are called the subtrees of the root. If the order in which the subtrees are arranged is not important, then the tree is a rooted, unordered (or oriented) tree. If the order of the subtrees is important, the tree is rooted and ordered. Figure 3.1 depicts the relationship between the three types of trees. We will henceforth refer to the rooted, ordered tree simply as tree.

FIGURE 3.1: The three trees shown are distinct if they are viewed as rooted, ordered trees. The rst two are identical if viewed as oriented trees. All three are identical if viewed as free trees.

3. k -ary tree: a nite set of nodes that is either empty or consists of a root and the elements of k disjoint k -ary trees called the 1st, 2nd, ..., k th subtrees of the root. The binary tree is a k -ary tree with k = 2. Here, the rst and second subtrees are respectively called the left and right subtrees of the root. Note that binary trees are not trees. One dierence is that a binary tree can be empty, whereas a tree cannot. Second, the two trees shown in Figure 3.2 are dierent binary trees but would be dierent drawings of the same tree.

FIGURE 3.2: Dierent binary trees.

Figure 3.3 shows a tree with 11 nodes. The number of subtrees of a node is its degree. Nodes with degree 0 are called leaf nodes. Thus, node A has degree 3, nodes B , D, and I have degree 2, node E has degree 1, and nodes C , F , G, H , J , and K have degree 0 (and are leaves of the tree). The degree of a tree is the maximum of the degree of the nodes in the tree. The roots of the subtrees of a node X are its children. X is the parent of

2005 by Chapman & Hall/CRC

Trees

3-3

its children. Children of the same parent are siblings. In the example, B , C , and D are each others siblings and are all children of A. The ancestors of a node are all the nodes excluding itself along the path from the root to that node. The level of a node is dened by letting the root be at level zero. If a node is at level l, then its children are at level l + 1. The height of a tree is the maximum level of any node in the tree. The tree in the example has height 4. These terms are dened in the same way for binary trees. See [16] for more information on trees.

LEVEL A B E I J K
FIGURE 3.3: An example tree.

0 D G H 1 2 3 4

C F

3.2
3.2.1

Tree Representation
List Representation

The tree of Figure 3.3 can be written as the generalized list (A (B (E (I (J, K)), F), C, D(G, H))). The information in the root node comes rst followed by a list of subtrees of the root. This enables us to represent a tree in memory using generalized lists as discussed in Chapter 2.

3.2.2

Left Child-Right Sibling Representation

Figure 3.4(a) shows the node structure used in this representation. Each node has a pointer to its leftmost child (if any) and to the sibling on its immediate right (if any). The tree in Figure 3.3 is represented by the tree in Figure 3.4(b).

3.2.3

Binary Tree Representation

Observe that the left child-right sibling representation of a tree (Figure 3.4(b)) may be viewed as a binary tree by rotating it clockwise by 45 degrees. This gives the binary tree

2005 by Chapman & Hall/CRC

3-4

Handbook of Data Structures and Applications

data

left child

right sibling

A B E I F C G D H I K J K E F G B C

(a)

(b)

(c)
FIGURE 3.4: Tree Representations.

representation shown in Figure 3.4(c). This representation can be extended to represent a forest, which is dened as an ordered set of trees. Here, the roots of the trees are viewed as siblings. Thus, a roots right pointer points to the next tree root in the set. We have
LEMMA 3.1

There is a one-to-one correspondence between the set of forests and the set of binary trees.

3.3

Binary Trees and Properties

Binary trees were dened in Section 3.1. For convenience, a binary tree is sometimes extended by adding external nodes. External nodes are imaginary nodes that are added wherever an empty subtree was present in the original tree. The original tree nodes are known as internal nodes. Figure 3.5(a) shows a binary tree and (b) the corresponding extended tree. Observe that in an extended binary tree, all internal nodes have degree 2 while all external nodes have degree 0. (Some authors use the term full binary tree to denote a binary tree whose nodes have 0 or two children.) The external path length of a tree is the sum of the lengths of all root-to-external node paths in the tree. In the example, this is 2 + 2 + 3 + 3 + 2 = 12. The internal path length is similarly dened by adding lengths of all root-to-internal node paths. In the example, this quantity is 0 + 1 + 1 + 2 = 4.

3.3.1

Properties

LEMMA 3.2

A binary tree with n internal nodes has n + 1 external nodes.

2005 by Chapman & Hall/CRC

Trees
A B D C 1 B 2 3 D 4 A C 5

3-5

(a)

(b)

FIGURE 3.5: (b) shows the extended binary tree corresponding to the binary tree of (a). External nodes are depicted by squares.

Each internal node in the extended tree has branches leading to two children. Thus, the total number of branches is 2n. Only n 1 internal nodes have a single incoming branch from a parent (the root does not have a parent). Thus, each of the remaining n + 1 branches points to an external node.
Proof LEMMA 3.3 For any non-empty binary tree with n0 leaf nodes and n2 nodes of degree 2, n0 = n2 + 1.

Let n1 be the number of nodes of degree 1 and n = n0 + n1 + n2 (Eq. 1) be the total number of nodes. The number of branches in a binary tree is n 1 since each non-root node has a branch leading into it. But, all branches stem from nodes of degree 1 and 2. Thus, the number of branches is n1 + 2n2 . Equating the two expressions for number of branches, we get n = n1 + 2n2 + 1 (Eq. 2). From Eqs. 1 and 2, we get n0 = n2 + 1.
Proof LEMMA 3.4

The external path length of any binary tree with n internal nodes is 2n greater than its internal path length.

Proof The proof is by induction. The lemma clearly holds for n = 0 when the internal and external path lengths are both zero. Consider an extended binary tree T with n internal nodes. Let ET and IT denote the external and internal path lengths of T . Consider the extended binary tree S that is obtained by deleting an internal node whose children are both external nodes (i.e., a leaf) and replacing it with an external node. Let the deleted internal node be at level l. Thus, the internal path length decreases by l while the external path length decreases by 2(l + 1) l = l + 2. From the induction hypothesis, ES = IS + 2(n 1). But, ET = ES + l + 2 and IT = IS + l. Thus, ET IT = 2n. LEMMA 3.5 Proof

The maximum number of nodes on level i of a binary tree is 2i , i 0.

This is easily proved by induction on i.

2005 by Chapman & Hall/CRC

3-6

Handbook of Data Structures and Applications The maximum number of nodes in a binary tree of height k is 2k+1 1.
k i=0

LEMMA 3.6 Proof

k+1

1.

Each level i, 0 i k , has 2i nodes. Summing over all i results in

2i =

LEMMA 3.7 The height of a binary tree with n internal nodes is at least log2 (n + 1) and at most n 1.

The worst case is a skewed tree (Figure 3.6(a)) and the best case is a tree with 2i nodes at every level i except possibly the bottom level (Figure 3.6(b)). If the height is h, then n + 1 2h , where n + 1 is the number of external nodes.
Proof

1 2 3 4 5 4 D 2 B 5 E

1 A 3 6 F

C
7 G

H
8

I
9

J
10

K
11

L
12

(a)

(b)

FIGURE 3.6: (a) Skewed and (b) complete binary trees.

LEMMA 3.8

The number of distinct binary trees with n nodes is

2n 1 n+1 n

Proof

For a detailed proof, we refer the reader to [7]. However, we note that Cn = are known as the Catalan numbers, which occur frequently in combinatorial problems. The Catalan number Cn also describes the number of trees with n + 1 nodes and the number of binary trees with 2n + 1 nodes all of which have 0 or 2 children.
2n 1 n+1 n

3.3.2

Binary Tree Representation

Binary trees are usually represented using nodes and pointers. A TreeNode class may be dened as: class TreeNode { TreeNode* LeftChild;

2005 by Chapman & Hall/CRC

Trees TreeNode* RightChild; KeyType data; };

3-7

In some cases, a node might also contain a parent pointer which facilitates a bottom-up traversal of the tree. The tree is accessed by a pointer root of type TreeNode* to its root. When the binary tree is complete (i.e., there are 2i nodes at every level i, except possibly the last level which has nodes lled in from left to right), it is convenient to use an array representation. The complete binary tree in Figure 3.6(b) can be represented by the array 1 [ A 2 B 3 C 4 D 5 E 6 F 7 G 8 H 9 I 10 11 12 J K L ]

Observe that the children (if any) of a node located at position i of the array can be found at positions 2i and 2i + 1 and its parent at i/2 .

3.4

Binary Tree Traversals

Several operations on trees require one to traverse the entire tree: i.e., given a pointer to the root of a tree, process every node in the tree systematically. Printing a tree is an example of an operation that requires a tree traversal. Starting at a node, we can do one of three things: visit the node (V ), traverse the left subtree recursively (L), and traverse the right subtree recursively (R). If we adopt the convention that the left subtree will be visited before the right subtree, we have three types of traversals LV R, V LR, and LRV which are called inorder, preorder, and postorder, respectively, because of the position of V with respect to L and R. In the following, we will use the expression tree in Figure 3.7 to illustrate the three traversals, which result in inx, prex, and postx forms of the expression. A fourth traversal, the level order traversal, is also studied.

* A B C

FIGURE 3.7: An expression tree.

3.4.1

Inorder Traversal

The following is a recursive algorithm for an inorder traversal that prints the contents of each node when it is visited. The recursive function is invoked by the call inorder(root). When run on the example expression tree, it returns A*B+C*D. inorder(TreeNode* currentNode) {

2005 by Chapman & Hall/CRC

3-8

Handbook of Data Structures and Applications if (currentNode) { inorder(currentNode->LeftChild); cout << currentNode->data; inorder(currentNode->RightChild); }

3.4.2

Preorder Traversal

The following is a recursive algorithm for a preorder traversal that prints the contents of each node when it is visited. The recursive function is invoked by the call preorder(root). When run on the example expression tree, it returns +*AB*CD. preorder(TreeNode* currentNode) { if (currentNode) { cout << currentNode->data; preorder(currentNode->LeftChild); preorder(currentNode->RightChild); } }

3.4.3

Postorder Traversal

The following is a recursive algorithm for a postorder traversal that prints the contents of each node when it is visited. The recursive function is invoked by the call postorder(root). When run on the example expression tree, it prints AB*CD*+. postorder(TreeNode* currentNode) { if (currentNode) { postorder(currentNode->LeftChild); postorder(currentNode->RightChild); cout << currentNode->data; } } The complexity of each of the three algorithms is linear in the number of tree nodes. Nonrecursive versions of these algorithms may be found in [6]. Both versions require (implicitly or explicitly) a stack.

3.4.4

Level Order Traversal

The level order traversal uses a queue. This traversal visits the nodes in the order suggested in Figure 3.6(b). It starts at the root and then visits all nodes in increasing order of their level. Within a level, the nodes are visited in left-to-right order. LevelOrder(TreeNode* root) { Queue q<TreeNode*>; TreeNode* currentNode = root;

2005 by Chapman & Hall/CRC

Trees while (currentNode) { cout << currentNode->data; if (currentNode->LeftChild) q.Add(currentNode->LeftChild); if (currentNode->RightChild) q.Add(currentNode->RightChild); currentNode = q.Delete(); //q.Delete returns a node pointer } }

3-9

3.5
3.5.1

Threaded Binary Trees

Threads

Lemma 3.2 implies that a binary tree with n nodes has n + 1 null links. These null links can be replaced by pointers to nodes called threads. Threads are constructed using the following rules: 1. A null right child pointer in a node is replaced by a pointer to the inorder successor of p (i.e., the node that would be visited after p when traversing the tree inorder). 2. A null left child pointer in a node is replaced by a pointer to the inorder predecessor of p. Figure 3.8 shows the binary tree of Figure 3.7 with threads drawn as broken lines. In order

* A B C

FIGURE 3.8: A threaded binary tree.

to distinguish between threads and normal pointers, two boolean elds LeftThread and RightThread are added to the node structure. If p->LeftThread is 1, then p->LeftChild contains a thread; otherwise it contains a pointer to the left child. Additionally, we assume that the tree contains a head node such that the original tree is the left subtree of the head node. The LeftChild pointer of node A and the RightChild pointer of node D point to the head node.

3.5.2

Inorder Traversal of a Threaded Binary Tree

Threads make it possible to perform an inorder traversal without using a stack. For any node p, if ps right thread is 1, then its inorder successor is p->RightChild. Otherwise the inorder successor is obtained by following a path of left-child links from the right child of p until a node with left thread 1 is reached. Function Next below returns the inorder

2005 by Chapman & Hall/CRC

3-10

Handbook of Data Structures and Applications

successor of currentNode (assuming that currentNode is not 0). It can be called repeatedly to traverse the entire tree in inorder in O(n) time. The code below assumes that the last node in the inorder traversal has a threaded right pointer to a dummy head node. TreeNode* Next(TreeNode* currentNode) { TreeNode* temp = currentNode->RightChild; if (currentNode->RightThread == 0) while (temp->LeftThread == 0) temp = temp->LeftChild; currentNode = temp; if (currentNode == headNode) return 0; else return currentNode; } Threads simplify the algorithms for preorder and postorder traversal. It is also possible to insert a node into a threaded tree in O(1) time [6].

3.6
3.6.1

Binary Search Trees

Denition

A binary search tree (BST) is a binary tree that has a key associated with each of its nodes. The keys in the left subtree of a node are smaller than or equal to the key in the node and the keys in the right subtree of a node are greater than or equal to the key in the node. To simplify the discussion, we will assume that the keys in the binary search tree are distinct. Figure 3.9 shows some binary trees to illustrate the denition.

18 10 7 9 19 16 2 4 6

12 16 14 18 5

10 15 20 25

(a)

(b)

(c)

FIGURE 3.9: Binary trees with distinct keys: (a) is not a BST. (b) and (c) are BSTs.

3.6.2

Search

We describe a recursive algorithm to search for a key k in a tree T : rst, if T is empty, the search fails. Second, if k is equal to the key in T s root, the search is successful. Otherwise,

2005 by Chapman & Hall/CRC

Trees

3-11

we search T s left or right subtree recursively for k depending on whether it is less or greater than the key in the root. bool Search(TreeNode* b, KeyType k) { if (b == 0) return 0; if (k == b->data) return 1; if (k < b->data) return Search(b->LeftChild,k); if (k > b->data) return Search(b->RightChild,k); }

3.6.3

Insert

To insert a key k , we rst carry out a search for k . If the search fails, we insert a new node with k at the null branch where the search terminated. Thus, inserting the key 17 into the binary search tree in Figure 3.9(b) creates a new node which is the left child of 18. The resulting tree is shown in Figure 3.10(a).

12 4 2 6 14 17 (a) 16 18 2 4 6

14 16 18

(b)

FIGURE 3.10: Tree of Figure 3.9(b) with (a) 18 inserted and (b) 12 deleted.

typedef TreeNode* TreeNodePtr; Node* Insert(TreeNodePtr& b, KeyType k) { if (b == 0) {b = new TreeNode; b->data= k; return b;} if (k == b->data) return 0; // dont permit duplicates if (k < b->data) Insert(b->LeftChild, k); if (k > b->data) Insert(b->RightChild, k); }

3.6.4

Delete

The procedure for deleting a node x from a binary search tree depends on its degree. If x is a leaf, we simply set the appropriate child pointer of xs parent to 0 and delete x. If x has one child, we set the appropriate pointer of xs parent to point directly to xs child and

2005 by Chapman & Hall/CRC

3-12

Handbook of Data Structures and Applications

then delete x. In Figure 3.9(c), node 20 is deleted by setting the right child of 15 to 25. If x has two children, we replace its key with the key in its inorder successor y and then delete node y . The inorder successor contains the smallest key greater than xs key. This key is chosen because it can be placed in node x without violating the binary search tree property. Since y is obtained by rst following a RightChild pointer and then following LeftChild pointers until a node with a null LeftChild pointer is encountered, it follows that y has degree 0 or 1. Thus, it is easy to delete y using the procedure described above. Consider the deletion of 12 from Figure 3.9(b). This is achieved by replacing 12 with 14 in the root and then deleting the leaf node containing 14. The resulting tree is shown in Figure 3.10(b).

3.6.5

Miscellaneous

Although Search, Insert, and Delete are the three main operations on a binary search tree, there are others that can be dened which we briey describe below. Minimum and Maximum that respectively nd the minimum and maximum elements in the binary search tree. The minimum element is found by starting at the root and following LeftChild pointers until a node with a 0 LeftChild pointer is encountered. That node contains the minimum element in the tree. Another operation is to nd the k th smallest element in the binary search tree. For this, each node must contain a eld with the number of nodes in its left subtree. Suppose that the root has m nodes in its left subtree. If k m, we recursively search for the k th smallest element in the left subtree. If k = m + 1, then the root contains the k th smallest element. If k > m +1, then we recursively search the right subtree for the k m 1st smallest element. The Join operation takes two binary search trees A and B as input such that all the elements in A are smaller than all the elements of B . The objective is to obtain a binary search tree C which contains all the elements originally in A and B . This is accomplished by deleting the node with the largest key in A. This node becomes the root of the new tree C . Its LeftChild pointer is set to A and its RightChild pointer is set to B . The Split operation takes a binary search tree C and a key value k as input. The binary search tree is to be split into two binary search trees A and B such that all keys in A are less than or equal to k and all keys in B are greater than k . This is achieved by searching for k in the binary search tree. The trees A and B are created as the search proceeds down the tree as shown in Figure 3.11. An inorder traversal of a binary search tree produces the elements of the binary search tree in sorted order. Similarly, the inorder successor of a node with key k in the binary search tree yields the smallest key larger than k in the tree. (Note that we used this property in the Delete operation described in the previous section.) All of the operations described above take O(h) time, where h is the height of the binary search tree. The bounds on the height of a binary tree are derived in Lemma 3.7. It has been shown that when insertions and deletions are made at random, the height of the binary search tree is O(log n) on the average.

2005 by Chapman & Hall/CRC

Trees
20 10 5 8 15 23 26 25 27 30 35 38 5 8 10 15 23 20 25 26 27 30 35

3-13

38

FIGURE 3.11: Splitting a binary search tree with k = 26.

3.7
3.7.1

Heaps
Priority Queues

Heaps are used to implement priority queues. In a priority queue, the element with highest (or lowest) priority is deleted from the queue, while elements with arbitrary priority are inserted. A data structure that supports these operations is called a max(min) priority queue. Henceforth, in this chapter, we restrict our discussion to a max priority queue. A priority queue can be implemented by a simple, unordered linked list. Insertions can be performed in O(1) time. However, a deletion requires a search for the element with the largest priority followed by its removal. The search requires time linear in the length of the linked list. When a max heap is used, both of these operations can be performed in O(log n) time.

3.7.2

Denition of a Max-Heap

A max heap is a complete binary tree such that for each node, the key value in the node is greater than or equal to the value in its children. Observe that this implies that the root contains the largest value in the tree. Figure 3.12 shows some examples of max heaps.

22 16 15 12 3 6 20 19 4 8

10 6 1

FIGURE 3.12: Max heaps.

We dene a class Heap with the following data members.

2005 by Chapman & Hall/CRC

3-14

Handbook of Data Structures and Applications

private: Element *heap; int n; // current size of max heap int MaxSize; // Maximum allowable size of the heap The heap is represented using an array (a consequence of the complete binary tree property) which is dynamically allocated.

3.7.3

Insertion

Suppose that the max heap initially has n elements. After insertion, it will have n + 1 elements. Thus, we need to add a node so that the resulting tree is a complete binary tree with n + 1 nodes. The key to be inserted is initially placed in this new node. However, the key may be larger than its parent resulting in a violation of the max property with its parent. In this case, we swap keys between the two nodes and then repeat the process at the next level. Figure 3.13 demonstrates two cases of an insertion into a max heap.

20 Insert x 15 4 3 12 4 15 3

20 12 x

x=8
15 4 3

20 12 8

20

x=16
15 4 3 16 12

FIGURE 3.13: Insertion into max heaps.

The algorithm is described below. In the worst case, the insertion algorithm moves up the heap from leaf to root spending O(1) time at each level. For a heap with n elements, this takes O(log n) time. void MaxHeap::Insert(Element x) { if (n == MaxSize) {HeapFull(); return;} n++; for (int i = n; i > 1; i = i/2 ) { if (x.key <= heap[i/2].key) break; heap[i] = heap[i/2]; } heap[i] = x; }

2005 by Chapman & Hall/CRC

Trees

3-15

3.7.4

Deletion

The element to be deleted (i.e., the maximum element in the heap) is removed from the root node. Since the binary tree must be restructured to become a complete binary tree on n 1 elements, the node in position n is deleted. The element in the deleted node is placed in the root. If this element is less than either of the roots (at most) two children, there is a violation of the max property. This is xed by swapping the value in the root with its larger child. The process is repeated at the other levels until there is no violation. Figure 3.14 illustrates deletion from a max heap.

20 DeleteMax 15 4 3 12 4 15 3 12 4 15

3 12

15 3 4 12 3 4

15 12

FIGURE 3.14: Deletion from max heaps.

The deletion algorithm is described below. In the worst case, the deletion algorithm moves down the heap from root to leaf spending O(1) time at each level. For a heap with n elements, this takes O(log n) time. Element* MaxHeap::DeleteMax(Element& x) { if (n == 0) {HeapEmpty(); return 0;} x = heap[1]; Element last = heap[n]; n--; for (int i = 1, j = 2; j <= n; i = j, j *= 2) { if (j < n) if (heap[j].key < heap[j+1].key) j++; // j points to the larger child if (last.key >= heap[j].key) break; heap[i] = heap[j]; // move child up } heap[i] = last; return &x; }

2005 by Chapman & Hall/CRC

3-16

Handbook of Data Structures and Applications

3.8

Tournament Trees

Consider the following problem: suppose we have k sequences, each of which is sorted in nondecreasing order, that are to be merged into one sequence in nondecreasing order. This can be achieved by repeatedly transferring the element with the smallest key to an output array. The smallest key has to be found from the leading elements in the k sequences. Ordinarily, this would require k 1 comparisons for each element transferred. However, with a tournament tree, this can be reduced to log2 k comparisons per element.

3.8.1

Winner Trees

A winner tree is a complete binary tree in which each node represents the smaller of its two children. The root represents the smallest node in the tree. Figure 3.15 illustrates a winner tree with k = 8 sequences. The winner of the tournament is the value 8 from sequence 0. The winner of the tournament is the smallest key from the 8 sequences and is transferred

8 8 8 8 19 20 22 25 30 15 20 26 15 45 50 62 37 40 50 37 41 42 43 18 21 36 18 18 26 31 38

S0

S1

S2

S3

S4

S5

S6

S7

FIGURE 3.15: A winner tree for k = 8. Three keys in each of the eight sequences are shown. For example, sequence 2 consists of 15, 20, and 26.

to an output array. The next element from sequence 0 is now brought into play and a tournament is played to determine the next winner. This is illustrated in Figure 3.16. It is easy to see that the tournament winner can be computed in (log n) time.

3.8.2

Loser Trees

The loser tree is an alternative representation that stores the loser of a match at the corresponding node. The loser tree corresponding to Figure 3.15 is shown in Figure 3.17. An advantage of the loser tree is that to restructure the tree after a winner has been output, it is sucient to examine nodes on the path from the leaf to the root rather than the siblings of nodes on this path.

2005 by Chapman & Hall/CRC

Trees

3-17

15 15 19 19 20 22 25 30 15 20 26 15 45 50 62 37 40 50 37 41 42 43 18 21 36 18 18 26 31 38

S0

S1

S2

S3

S4

S5

S6

S7

FIGURE 3.16: Winner tree of Figure 3.15 after the next element of sequence 0 plays the tournament. Matches are played at the shaded nodes.

8 18 15 22 8 19 20 22 25 30 15 20 26 45 45 50 62

tournament winner

37 41 37 40 50 41 42 43 18 21 36 26 26 31 38

S0

S1

S2

S3

S4

S5

S6

S7

FIGURE 3.17: Loser tree corresponding to the winner tree of Figure 3.15.

2005 by Chapman & Hall/CRC

3-18

Handbook of Data Structures and Applications

Acknowledgments
The author would like to thank Kun Gao and Mr. U. B. Rao for proofreading this chapter. This work was supported, in part, by the National Science Foundation under grant CCR9988338.

References
[1] D. Knuth, The Art of Computer Programming, Vol. 1, Fundamental Algorithms, Third edition. Addison Wesley, NY, 1997. [2] T. Cormen, C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms, Second edition. McGraw-Hill, New York, NY, 2001. [3] S. Sahni, Data Structures, Algorithms, and Applications in C++, McGraw-Hill, NY, 1998. [4] S. Sahni, Data Structures, Algorithms, and Applications in Java, McGraw-Hill, NY, 2000. [5] R. Sedgewick, Algorithms in C++, Parts 1-4, Third edition. Addison-Wesley, NY, 1998. [6] E. Horowitz, S. Sahni, and D. Mehta Fundamentals of Data Structures in C++, W. H. Freeman, NY, 1995. [7] D. Stanton and D. White, Constructive Combinatorics, Springer-Verlag, NY, 1986.

2005 by Chapman & Hall/CRC

4
Graphs
4.1 4.2 4.3 4.4 4.5 4.6 4.7 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Graph Representations . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Weighted Graph Representation

4-1 4-3 4-6 4-8 4-12 4-14

Connectivity, Distance, and Spanning Trees . . .

Spanning Trees

Searching a Graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Depth-First Search

Breadth-First Search

Simple Applications of DFS and BFS . . . . . . . . . . .

Depth-First Search on a Digraph Sorting Topological

Minimum Spanning Tree . . . . . . . . . . . . . . . . . . . . . . . . . .

Kruskals MST Algorithm Prims MST Algorithm Boruvkas MST Algorithm Constrained MST

Shortest Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Single-Source Shortest Paths, Nonnegative Weights Single-Source Shortest Paths, Arbitrary Weights All-Pairs Shortest Paths

4-19

Narsingh Deo
University of Central Florida

4.8

Eulerian and Hamiltonian Graphs . . . . . . . . . . . . . . .

4-24

4.1

Introduction

Trees, as data structures, are somewhat limited because they can only represent relations of a hierarchical nature, such as that of parent and child. A generalization of a tree so that a binary relation is allowed between any pair of elements would constitute a graphformally dened as follows: A graph G = (V, E ) consists of a nite set of vertices V = {1 , 2 , . . . , n } and a nite set E of edges E = {e1 , e2 , . . . , em } (see Figure 4.1). To each edge e there corresponds a pair of vertices (u, ) which e is said to be incident on. While drawing a graph we represent each vertex by a dot and each edge by a line segment joining its two end vertices . A graph is said to be a directed graph (or digraph for short) (see Figure 4.2) if the vertex pair (u, ) associated with each edge e (also called arc ) is an ordered pair. Edge e is then said to be directed from vertex u to vertex , and the direction is shown by an arrowhead on the edge. A graph is undirected if the end vertices of all the edges are unordered (i.e., edges have no direction). Throughout this chapter we use the letters n and m to denote the number of vertices |V | and number of edges |E | respectively, in a graph. A vertex is often referred to as a node (a term more popular in applied elds). Two or more edges having the same pair of end vertices are called parallel edges or multi edges , and a graph with multi edges is sometimes referred to as a multigraph . An edge whose two end vertices are the same is called a self-loop (or just loop ). A graph in which neither

4-1

2005 by Chapman & Hall/CRC

4-2

Handbook of Data Structures and Applications

FIGURE 4.1: Undirected graph with 5 vertices and 6 edges.

FIGURE 4.2: Digraph with 6 vertices and 11 edges.

parallel edges nor self-loops are allowed is often called a simple graph . If both self-loops and parallel edges are allowed we have a general graph (also referred to as pseudograph ). Graphs in Figure 4.1 and Figure 4.2 are both simple but the graph in Figure 4.3 is pseudograph. If the graph is simple we can refer to each edge by its end vertices. The number of edges incident on a vertex v , with self-loops counted twice, is called the degree , deg (v ), of vertex v . In directed graphs a vertex has in-degree (number of edges going into it) and out-degree (number of edges going out of it). In a digraph if there is a directed edge (x, y ) from x to y , vertex y is called a successor of x and vertex x is called a predecessor of y . In case of an undirected graph two vertices are said to be adjacent or neighbors if there is an edge between them. A weighted graph is a (directed or undirected) graph in which a real number is assigned to each edge. This number is referred to as the weight of that edge. Weighted directed graphs are often referred to as networks . In a practical network this number (weight) may represent the driving distance, the construction cost, the transit time, the reliability, the transition probability, the carrying capacity, or any other such attribute of the edge [1, 4, 18, 20]. Graphs are the most general and versatile data structures. Graphs have been used to model and solve a large variety of problems in the discrete domain. In their modeling and problem solving ability graphs are to the discrete world what dierential equations are to

2005 by Chapman & Hall/CRC

Graphs

4-3

FIGURE 4.3: A pseudograph of 6 vertices and 10 edges.

the world of the continuum.

4.2

Graph Representations

For a given graph a number of dierent representations are possible. The ease of implementation, as well as the eciency of a graph algorithm depends on the proper choice of the graph representation. The two most commonly used data structures for representing a graph (directed or undirected) are adjacency lists and adjacency matrix . In this section we discuss these and other data structures used in representing graphs. Adjacency Lists: The adjacency lists representation of a graph G consists of an array Adj of n linked lists, one for each vertex in G, such that Adj [ ] for vertex consists of all vertices adjacent to . This list is often implemented as a linked list. (Sometimes it is also represented as a table, in which case it is called the star representation [18].) Adjacency Matrix: The adjacency matrix of a graph G = (V, E ) is an n n matrix A = [aij ] in which aij = 1 if there is an edge from vertex i to vertex j in G; otherwise aij = 0. Note that in an adjacency matrix a self-loop can be represented by making the corresponding diagonal entry 1. Parallel edges could be represented by allowing an entry to be greater than 1, but doing so is uncommon, since it is usually convenient to represent each element in the matrix by a single bit. The adjacency lists and adjacency matrix of an undirected graph are shown in Figure 4.4, and the corresponding two representations for a digraph are shown in Figure 4.5.

2005 by Chapman & Hall/CRC

4-4

Handbook of Data Structures and Applications

(a) 0 1 1 1 1 0 0 0 1 0 0 1 (c) 1 0 1 0

(b)

FIGURE 4.4: An undirected graph (a) with four vertices and four edges; (b) its adjacency lists representation, and (c) its adjacency matrix representation.

(a) 0 1 0 0 0 1 0 1 0 1 0 0 1 0 0 (c) 1 1 1 0 0 0 0 0 0 0

(b)

FIGURE 4.5: Two representations: (a) A digraph with ve vertices and eight edges; (b) its adjacency lists representation, and (c) its adjacency matrix representation.

2005 by Chapman & Hall/CRC

Graphs

4-5

Clearly the memory required to store a graph of n vertices in the form of adjacency matrix is O(n2 ), whereas for storing it in the form of its adjacency lists is O(m + n). In general if the graph is sparse, adjacency lists are used but if the graph is dense, adjacency matrix is preferred. The nature of graph processing is an important factor in selecting the data structure. There are other less frequently used data structures for representing graphs, such as forward or backward star , the edge-list , and vertex-edge incidence matrix [1, 4, 15, 18, 20].

4.2.1

Weighted Graph Representation

Both adjacency lists and adjacency matrix can be adapted to take into account the weights associated with each edge in the graph. In the former case an additional eld is added in the linked list to include the weight of the edge; and in the latter case the graph is represented by a weight matrix in which the (i, j )th entry is the weight of edge (i, j ) in the weighted graph. These two representations for a weighted graph are shown in Figure 4.6. The boxed numbers next to the edges in Figure 4.6(a) are the weights of the corresponding edges. It should be noted that in a weight matrix, W , of a weighted graph, G, if there is no edge (i, j ) in G, the corresponding element wij is usually set to (in practice, some very large number). The diagonal entries are usually set to (or to some other value depending on the application and algorithm). It is easy to see that the weight matrix of an undirected graph (like the adjacency matrix) is symmetric.

(a) W = 19 18 43 16 35 85 11 77 43

(b)

(c)

FIGURE 4.6: Two representations: (a) A weighted digraph with ve vertices and nine edges; (b) its adjacency lists, and (c) its weight matrix.

2005 by Chapman & Hall/CRC

4-6

Handbook of Data Structures and Applications

4.3

Connectivity, Distance, and Spanning Trees

Just as two vertices x and y in a graph are said to be adjacent if there is an edge joining them, two edges are said to be adjacent if they share (i.e., are incident on) a common vertex. A simple path , or path for short, is a sequence of adjacent edges (1 , 2 ), (2 , 3 ), . . ., (k2 , k1 ), (k1 , k ), sometimes written (1 , 2 , . . . , k ), in which all the vertices 1 , 2 , . . . , k are distinct except possibly 1 = k . In a digraph this path is said to be directed from 1 to k ; in an undirected graph this path is said to be between 1 and k . The number of edges in a path, in this case, k 1, is called the length of the path. In Figure 4.3 sequence (6 , 4 ), (4 , 1 ), (1 , 2 ) = (6 , 4 , 1 , 2 ) is a path of length 3 between 6 and 2 . In the digraph in Figure 4.6 sequence (3, 1), (1, 5), (5, 2), (2, 4) = (3, 1, 5, 2, 4) is a directed path of length 4 from vertex 3 to vertex 4. A cycle or circuit is a path in which the rst and the last vertices are the same. In Figure 4.3 (3 , 6 , 4 , 1 , 3 ) is a cycle of length 4. In Figure 4.6 (3, 2, 1, 3) is a cycle of length 3. A graph that contains no cycle is called acyclic . A subgraph of a graph G = (V, E ) is a graph whose vertices and edges are in G. A subgraph g of G is said to be induced by a subset of vertices S V if g results when the vertices in V S and all the edges incident on them are removed from G. For example, in Figure 4.3, the subgraph induced by {1 , 3 , 4 } would consists of these three vertices and four edges {e3 , e5 , e6 , e7 }. An undirected graph G is said to be connected if there is at least one path between every pair of vertices i and j in G. Graph G is said to be disconnected if it has at least one pair of distinct vertices u and v such that there is no path between u and v . Two vertices x and y in an undirected graph G = (V, E ) are said to be connected if there exists a path between x and y . This relation of being connected is an equivalence relation on the vertex set V , and therefore it partitions the vertices of G into equivalence classes. Each equivalence class of vertices induces a subgraph of G. These subgraphs are called connected components of G. In other words, a connected component is a maximal connected subgraph of G. A connected graph consists of just one component, whereas a disconnected graph consists of several (connected) components. Each of the graphs in Figures 4.1, 4.3, and 4.4 is connected. But the graph given in Figure 4.7 is disconnected, consisting of four components.

FIGURE 4.7: A disconnected graph of 10 vertices, 8 edges, and 4 components.

2005 by Chapman & Hall/CRC

Graphs

4-7

Notice that a component may consist of just one vertex such as j in Figure 4.7 with no edges. Such a component (subgraph or graph) is called an isolated vertex . Equivalently, a vertex with zero degree is called an isolated vertex. Likewise, a graph (subgraph or component) may consist of just one edge, such as edge (i, d) in Figure 4.7. One of the simplest and often the most important and useful questions about a given graph G is: Is G connected? And if G is not connected what are its connected components? This question will be taken up in the next section, and an algorithm for determining the connected components will be provided; but rst a few more concepts and denitions. Connectivity in a directed graph G is more involved. A digraph is said to be connected if the undirected graph obtained by ignoring the edge directions in G is connected. A directed graph is said to be strongly connected if for every pair of vertices i and j there exists at least one directed path from i to j and at least one from j to i . A digraph which is connected but not strongly connected is called weakly connected . A disconnected digraph (like a disconnected undirected graph) consists of connected components; and a weaklyconnected digraph consists of strongly-connected components. For example, the connected digraph in Figure 4.5 consists of four strongly-connected componentsinduced by each of the following subsets of vertices {1, 2},{3},{4}, and {5}. Another important question is that of distance from one vertex to another. The distance from vertex a to b is the length of the shortest path (i.e., a path of the smallest length) from a to b, if such a path exists. If no path from a to b exists, the distance is undened and is often set to . Thus, the distance from a vertex to itself is 0; and the distance from a vertex to an adjacent vertex is 1. In an undirected graph distance from a to b equals the distance from b to a, i.e., it is symmetric. It is also not dicult to see that the distances in a connected undirected graph (or a strongly connected digraph) satisfy the triangle inequality. In a connected, undirected (unweighted) graph G, the maximum distance between any pair of vertices is called the diameter of G.

4.3.1

Spanning Trees

A connected, undirected, acyclic (without cycles) graph is called a tree , and a set of trees is called a forest . We have already seen rooted trees and forests of rooted trees in the preceding chapter, but the unrooted trees and forests discussed in this chapter are graphs of a very special kind that play an important role in many applications. In a connected undirected graph G there is at least one path between every pair of vertices and the absence of a cycle implies that there is at most one such path between any pair of vertices in G. Thus if G is a tree, there is exactly one path between every pair of vertices in G. The argument is easily reversed, and so an undirected graph G is a tree if and only if there is exactly one path between every pair of vertices in G. A tree with n vertices has exactly (n 1) edges. Since (n 1) edges are the fewest possible to connect n points, trees can be thought of as graphs that are minimally connected . That is, removing any edge from a tree would disconnect it by destroying the only path between at least one pair of vertices. A spanning tree for a connected graph G is a subgraph of G which is a tree containing every vertex of G. If G is not connected, a set consisting of one spanning tree for each component is called a spanning forest of G. To construct a spanning tree (forest) of a given undirected graph G, we examine the edges of G one at a time and retain only those that do not not form a cycle with the edges already selected. Systematic ways of examining the edges of a graph will be discussed in the next section.

2005 by Chapman & Hall/CRC

4-8

Handbook of Data Structures and Applications

4.4

Searching a Graph

It is evident that for answering almost any nontrivial question about a given graph G we must examine every edge (and in the process every vertex) of G at least once. For example, before declaring a graph G to be disconnected we must have looked at every edge in G; for otherwise, it might happen that the one edge we had decided to ignore could have made the graph connected. The same can be said for questions of separability, planarity, and other properties [15, 16]. There are two natural ways of scanning or searching the edges of a graph as we move from vertex to vertex: (i) once at a vertex v we scan all edges incident on v and then move to an adjacent vertex w, then from w we scan all edges incident on w. This process is continued till all the edges reachable from v are scanned. This method of fanning out from a given vertex v and visiting all vertices reachable from v in order of their distances from v (i.e. rst visit all vertices at a distance one from v , then all vertices at distances two from v , and so on) is referred to as the breadth-rst search (BFS) of the graph. (ii) An opposite approach would be, instead of scanning every edge incident on vertex v , we move to an adjacent vertex w (a vertex not visited before) as soon as possible, leaving v with possibly unexplored edges for the time being. In other words, we trace a path through the graph going on to a new vertex whenever possible. This method of traversing the graph is called the depth-rst search (DFS). Breadth-rst and depth-rst searches are fundamental methods of graph traversal that form the basis of many graph algorithms [7, 15, 16, 19]. The details of these two methods follow.

4.4.1

Depth-First Search

Depth-rst search on an undirected graph G = (V, E ) explores the graph as follows. When we are visiting a vertex v V , we follow one of the edges (v, w) incident on v . If the vertex w has been previously visited, we return to v and choose another edge. If the vertex w (at the other end of edge (v, w) from v ) has not been previously visited, we visit it and apply the process recursively to w. If all the edges incident on v have been thus traversed, we go back along the edge (u, v ) that had rst led to the current vertex v and continue exploring the edges incident on u. We are nished when we try to back up from the vertex at which the exploration began. Figure 4.8 illustrates how depth-rst search examines an undirected graph G represented as an adjacency lists. We start with a vertex a. From a we traverse the rst edge that we encounter, which is (a, b). Since b is a vertex never visited before, we stay at b and traverse the rst untraversed edge encountered at b, which is (b, c). Now at vertex c, the rst untraversed edge that we nd is (c, a). We traverse (c, a) and nd that a has been previously visited. So we return to c, marking the edge (c, a) in some way (as a dashed line in Figure 4.8(c)) to distinguish it from edges like (b, c), which lead to new vertices and shown as the thick lines. Back at vertex c, we look for another untraversed edge and traverse the rst one that we encounter, which is (c, d). Once again, since d is a new vertex, we stay at d and look for an untraversed edge. And so on. The numbers next to the vertices in Figure 4.8(c) show the order in which they were visited; and the numbers next to the edges show the order in which they were traversed.

2005 by Chapman & Hall/CRC

Graphs

4-9

(a)

(b)

(c) FIGURE 4.8: A graph (a); its adjacency lists (b); and its depth-rst traversal (c). The numbers are the order in which vertices were visited and edges traversed. Edges whose traversal led to new vertices are shown with thick lines, and edges that led to vertices that were already visited are shown with dashed lines.

DepthFirstSearch(G) for each vertex x V do num[x] 0 end for T reeEdges 0 i0 for each vertex x V do if num[x] = 0 then DFS-Visit(x) end if end for

2005 by Chapman & Hall/CRC

4-10

Handbook of Data Structures and Applications

DFS-Visit(v ) ii+1 num[v ] i for each vertex w Adj [v ] do if num[w] = 0 then {// w is new vertex //} T reeEdges T reeEdges (v, w) {// (v, w) is a tree edge //} DFS-Visit(w) end if end for FIGURE 4.9: Algorithm for depth-rst search on an undirected graph G.

Depth-rst search performed on a connected undirected graph G = (V, E ), partitions the edge set into two types: (i) Those that led to new vertices during the search constitute the branches of a spanning tree of G and (ii) the remaining edges in E are called back edges because their traversal led to an already visited vertex from which we backed down to the current vertex. A recursive depth-rst search algorithm is given in Figure 4.9. Initially, every vertex x is marked unvisited by setting num[x] to 0. Note that in the algorithm shown in Figure 4.9, only the tree edges are kept track of. The time complexity of the depth-rst search algorithm is O(m + n), provided the input is in the form of an adjacency matrix.

4.4.2

Breadth-First Search

In breadth-rst search we start exploring from a specied vertex s and mark it visited. All other vertices of the given undirected graph G are marked as unvisited by setting num[] = 0. Then we visit all vertices adjacent to s (i.e., in the adjacency list of s). Next, we visit all unvisited vertices adjacent to the rst vertex in the adjacency list of s. Unlike the depth-rst search, in breadth-rst search we explore (fan out) from vertices in order in which they themselves were visited. To implement this method of search, we maintain a queue (Q) of visited vertices. As we visit a new vertex for the rst time, we place it in (i.e., at the back of) the queue. We take a vertex v from front of the queue and traverse all untraversed edges incident at v adding to the list of tree edges those edges that lead to unvisited vertices from v ignoring the rest. Once a vertex v has been taken out of the queue, all the neighbors of v are visited and v is completely explored. Thus, during the execution of a breadth-rst search we have three types of vertices: (i) unvisited, those that have never been in the queue; (ii) completely explored, those that have been in the queue but are not now in the queue; and (iii) visited but not completely explored, i.e., those that are currently in the queue. Since every vertex (reachable from the start vertex s) enters and exits the queue exactly once and every edge in the adjacency list of a vertex is traversed exactly once, the time complexity of the breadth-rst search is O(n + m).

2005 by Chapman & Hall/CRC

Graphs BreadthFirstSearch(G, s) for each vertex x V {s} do visited[x] 0 {// all vertices unvisited except s //} end for T reeEdges null Q {// queue of vertices is initially empty //} visited[s] 1 {// mark s as visited //} enqueue(Q, s) {// place s in the queue //} while Q = do {// queue is not empty //} v dequeue(Q) for each w Adj [v ] do if visited[w] = 0 then {// w is a new vertex //} visited[w] 1 T reeEdges T reeEdges {(v, w)} enqueue(Q, w) end if end for end while

4-11

FIGURE 4.10: Algorithm for breadth-rst search on an undirected graph G from vertex s.

An algorithm for performing a breadth-rst search on an undirected connected graph G from a specied vertex s is given in Figure 4.10. It produces a breadth-rst tree in G rooted at vertex s. For example, the spanning tree produced by BFS conducted on the graph in Figure 4.8 starting at vertex a, is shown in Figure 4.11. The numbers next to the vertices show the order in which the vertices were visited during the BFS.

FIGURE 4.11: Spanning tree produced by breadth-rst search on graph in Figure 4.8 starting from vertex a. The numbers show the order in which vertices were visited.

2005 by Chapman & Hall/CRC

4-12

Handbook of Data Structures and Applications

4.5

Simple Applications of DFS and BFS

In the preceding section we discussed two basic but powerful and ecient techniques for systematically searching a graph such that every edge is traversed exactly once and every vertex is visited once. With proper modications and embellishments these search techniques can be used to solve a variety of graph problems. Some of the simple ones are discussed in this section. Cycle Detection: The existence of a back edge (i.e., a nontree edge) during a depthrst search indicates the existence of cycle. To test this condition we just add an else clause to the if num[w] = 0 statement in DFS-Visit(v ) procedure in Figure 4.9. That is, if num[w] = 0, (v, w) is a back edge, which forms a cycle with tree edges in the path from w to v . Spanning Tree: If the input graph G for the depth-rst (or breadth-rst) algorithm is connected, the set TreeEdges at the termination of the algorithm in Figure 4.9 (or in Figure 4.10, for breadth-rst) produces a spanning tree of G. Connected Components: If, on the other hand, the input graph G = (V, E ) is disconnected we can use depth-rst search to identify each of its connected components by assigning a unique component number compnum[v ] to every vertex belonging to one component. The pseudocode of such an algorithm is given below (Figure 4.12) for each vertex v V do compnum[v ] 0 end for for each vertex v V do if compnum[v ] = 0 then cc+1 COMP(v ) end if end for COMP(x) compnum[x] c for each w Adj [x] do if compnum[w] = 0 then COMP(w) end if end for FIGURE 4.12: Depth-rst search algorithm for nding connected components of a graph.

4.5.1

Depth-First Search on a Digraph

Searching a digraph is somewhat more involved because the direction of the edges is an additional feature that must be taken into account. In fact, a depth-rst search on a digraph produces four kinds of edges (rather than just two types for undirected graphs):

2005 by Chapman & Hall/CRC

Graphs

4-13

(i) Tree edgeslead to an unvisited vertex (ii) Back edgeslead to an (visited) ancestor vertex in the tree (iii) Down-edges (also called forward edges) lead to a (visited) descendant vertex in the tree, and (iv) Cross edges, lead to a visited vertex, which is neither ancestor nor descendant in the tree [3, 15, 16, 18, 19].

4.5.2

Topological Sorting

The simplest use of the depth-rst search technique on digraphs is to determine a labeling of the vertices of an acyclic digraph G = (V, E ) with integers 1, 2, . . . , |V |, such that if there is a directed edge from vertex i to vertex j , then i < j ; such a labeling is called topological sort of the vertices of G. For example, the vertices of the digraph in Figure 4.13(a) are topologically sorted but those of Figure 4.13(b) are not. Topological sorting can be viewed as the process of nding a linear order in which a given partial order can be embedded. It is not dicult to show that it is possible to topologically sort the vertices of a digraph if and only if it is acyclic. Topological sorting is useful in the analysis of activity networks where a large, complex project is represented as a digraph in which the vertices correspond to the goals in the project and the edges correspond to the activities. The topological sort gives an order in which the goals can be achieved [1, 9, 18].

(a) Topologically sorted.

(b) Not topologically sorted.

FIGURE 4.13: Acyclic digraphs.

Topological sorting begins by nding a vertex of G = (V, E ) with no outgoing edge (such a vertex must exist if G is acyclic) and assigning this vertex the highest numbernamely, |V |. This vertex is then deleted from G, along with all its incoming edges. Since the remaining digraph is also acyclic, we can repeat the process and assign the next highest number, namely |V | 1, to a vertex with no outgoing edges, and so on. To keep the algorithm O(|V | + |E |), we must avoid searching the modied digraph for a vertex with no outgoing edges. We do so by performing a single depth-rst search on the given acyclic digraph G. In addition to the usual array num, we will need another array, label, of size |V | for recording the topologically sorted vertex labels. That is, if there is an edge (u, v ) in G, then label[u] < label[v ]. The complete search and labeling procedure TOPSORT is given in Figure 4.14. Use the acyclic digraph in Figure 4.13(a) with vertex set V = {a, b, c, d, e, f, g } as the input

2005 by Chapman & Hall/CRC

4-14

Handbook of Data Structures and Applications

to the topological sort algorithm in Figure 4.14; and verify that the vertices get relabeled 1 to 7, as shown next to the original namesin a correct topological order.

Topological-Sort(G) for each vertex x V do num[x] 0 label[x] 0 end for j n+1 i0 for each vertex x V do if num[x] = 0 then {// x has no labeled ancestor //} TOPSORT(x) end if end for TOPSORT(v ) ii+1 num[v ] i for each w Adj [v ] do {// examine all descendants of w //} if num[w] = 0 then TOPSORT(w) else if label[w] = 0 then Error {// cycle detected //} end if j j1 label[v ] j end for FIGURE 4.14: Algorithm for topological sorting.

4.6

Minimum Spanning Tree

How to connect a given set of points with lowest cost is a frequently-encountered problem, which can be modeled as the problem of nding a minimum-weight spanning tree T in a weighted, connected, undirected graph G = (V, E ). Methods for nding such a spanning tree, called a minimum spanning tree (MST), have been investigated in numerous studies and have a long history [8]. In this section we will discuss the bare essentials of the two commonly used MST algorithmsKruskals and Primsand briey mention a third one.

4.6.1

Kruskals MST Algorithm

An algorithm due to J. B. Kruskal, which employs the smallest-edge-rst strategy, works as follows: First we sort all the edges in the given network by weight, in nondecreasing order. Then one by one the edges are examined in order, smallest to the largest. If an edge ei , upon examination, is found to form a cycle (when added to edges already selected)

2005 by Chapman & Hall/CRC

Graphs

4-15

it is discarded. Otherwise, ei is selected to be included in the minimum spanning tree T . The construction stops when the required n 1 edges have been selected or when all m edges have been examined. If the given network is disconnected, we would get a minimum spanning forest (instead of tree). More formally, Kruskals method may be stated as follows: T while |T | < (n 1) and E = do e smallest edge in E E E { e} if T {e} has no cycle then T T { e} end if end while if |T | < (n 1) then write network disconnected end if

Although the algorithm just outlined is simple enough, we do need to work out some implementation details and select an appropriate data structure for achieving an ecient execution. There are two crucial implementational details that we must consider in this algorithm. If we initially sort all m edges in the given network, we may be doing a lot of unnecessary work. All we really need is to be able to to determine the next smallest edge in the network at each iteration. Therefore, in practice, the edges are only partially sorted and kept as a heap with smallest edge at the root of a min heap. In a graph with m edges, the initial construction of the heap would require O(m) computational steps; and the next smallest edge from a heap can be obtained in O(log m) steps. With this improvement, the sorting cost is O(m + p log m), where p is the number of edges examined before an MST is constructed. Typically, p is much smaller than m. The second crucial detail is how to maintain the edges selected (to be included in the MST) so far, such that the next edge to be examined can be eciently tested for a cycle formation. As edges are examined and included in T , a forest of disconnected trees (i.e., subtrees of the nal spanning tree) is produced. The edge e being examined will form a cycle if and only if both its end vertices belong to the same subtree in T . Thus to ensure that the edge currently being examined does not form a cycle, it is sucient to check if it connects two dierent subtrees in T . An ecient way to accomplish this is to group the n vertices of the given network into disjoint subsets dened by the subtrees (formed by the edges included in T so far). Thus if we maintain the partially constructed MST by means of subsets of vertices, we can add a new edge by forming the UNION of two relevant subsets, and we can check for cycle formation by FINDing if the two end vertices of the edge, being examined, are in the same subset. These subsets can themselves be kept as rooted trees. The root is an element of the subset and is used as a name to identify that subset. The FIND subprocedure is called twiceonce for each end vertex of edge eto determine the sets to which the two end vertices belong. If they are dierent, the UNION subprocedure will merge the two subsets. (If they are the same subset, edge e will be discarded.) The subsets, kept as rooted trees, are implemented by keeping an array of parent pointers

2005 by Chapman & Hall/CRC

4-16

Handbook of Data Structures and Applications

for each of the n elements. Parent of a root, of course, is null. (In fact, it is useful to assign parent[root] = -number of vertices in the tree.) While taking the UNION of two subsets, we merge the smaller subset into the larger one by pointing the parent pointer in the root of the smaller subset to the root of the larger subset. Some of these details are shown in Figure 4.15. Note that r1 and r2 are the roots identifying the sets to which vertices u and v belong. INITIALIZATION: set parent array to -1 {// n vertices from singleton sets //} form initial heap of m edges ecount 0 {// number of edges examined so far //} tcount 0 {// number of edges in T so far //} T ITERATION: while tcount < (n 1) and ecount < m do e edge(u, v ) from top of heap ecount ecount + 1 remove e from heap restore heap r1 FIND(u) r2 FIND(v ) if r1 = r2 then T T { e} tcount tcount + 1 UNION(r1, r2) end if end while if tcount < (n 1) then write network disconnected end if FIGURE 4.15: Kruskals minimum spanning tree algorithm.

When algorithm in Figure 4.15 is applied to the weighted graph in Figure 4.16, the order in which edges are included one by one to form the MST are (3, 5), (4, 6), (4, 5), (4, 2), (6, 7), (3, 1). After the rst ve smallest edges are included in the MST, the 6th and 7th and 8th smallest edges are rejected. Then the 9th smallest edge (1, 3) completes the MST and the last two edges are ignored.

4.6.2

Prims MST Algorithm

A second algorithm, discovered independently by several people (Jarnik in 1936, Prim in 1957, Dijkstra in 1959) employs the nearest neighbor strategy and is commonly referred to as Prims algorithm. In this method one starts with an arbitrary vertex s and joins it to its nearest neighbor, say y . That is, of all edges incident on vertex s, edge (s, y ), with the smallest weight, is made part of the MST. Next, of all the edges incident on s or y we

2005 by Chapman & Hall/CRC

Graphs

4-17

choose one with minimum weight that leads to some third vertex, and make this edge part of the MST. We continue this process of reaching out from the partially constructed tree (so far) and bringing in the nearest neighbor until all vertices reachable from s have been incorporated into the tree.

FIGURE 4.16: A connected weighted graph for MST algorithm.

As an example, let us use this method to nd the minimum spanning tree of the weighted graph given in Figure 4.16. Suppose that we start at vertex 1. The nearest neighbor of vertex 1 is vertex 3. Therefore, edge (1, 3) becomes part of the MST. Next, of all the edges incident on vertices 1 and 3 (and not included in the MST so far) we select the smallest, which is edge (3, 5) with weight 14. Now the partially constructed tree consists of two edges (1, 3) and (3, 5). Among all edges incident at vertices 1,3, and 5, edge (5, 4) is the smallest, and is therefore included in the MST. The situation at this point is shown in Figure 4.17. Clearly, (4, 6), with weight 18 is the next edge to be included. Finally, edges (4, 2) and (6, 7) will complete the desired MST.

FIGURE 4.17: Partially constructed MST for the network of Figure 4.16.

2005 by Chapman & Hall/CRC

4-18

Handbook of Data Structures and Applications

The primary computational task in this algorithm is that of nding the next edge to be included into the MST in each iteration. For each ecient execution of this task we will maintain an array near[u] for each vertex u not yet in the tree (i.e., u V VT ). near[u] is that vertex in VT which is closest to u. (Note that V is the set of all vertices in the network and VT is the subset of V included in MST thus far.) Initially, we set near[s] 0 to indicate that s is in the tree, and for every other vertex v , near[v ] s.

For convenience, we will maintain another array dist[u] of the actual distance (i.e., edge weight) to that vertex in VT which is closest to u. In order to determine which vertex is to be added to the set VT next, we compare all nonzero values in dist array and pick the smallest. Thus n i comparisons are sucient to identify the ith vertex to be added. Initially, since s is the only vertex in VT , dist[u] is set to wsu . As the algorithm proceeds, these two arrays are updated in each iteration (see Figure 4.17 for an illustration).

A formal description of the nearest-neighbor algorithm is given in Figure 4.18. It is assumed that the input is given in the form of an n n weight matrix W (in which nonexistent edges have weights). Set V = {1, 2, . . . , n} is the set of vertices of the graph. VT and ET are the sets of vertices and edges of the partially formed (minimum spanning) tree. Vertex set VT is identied by zero entries in array near. INITIALIZATION: choose starting vertex s arbitrarily for every vertex i other than s do near[i] s dist[i] wsi end for VT {s} {// set of vertices in MST so far //} ET {// set of edges in MST so far //} ITERATION: while |VT | < n do u vertex in (V VT ) with smallest value of dist(u) if dist[u] then write graph disconnected and exit end if ET ET {(u, near[u])} VT VT {u} for x (V VT ) do if wux < dist[x] then dist[x] wux near[x] u end if end for end while FIGURE 4.18: Prims minimum spanning tree algorithm.

2005 by Chapman & Hall/CRC

Graphs

4-19

4.6.3

Boruvkas MST Algorithm

There is yet a third method for computing a minimum spanning tree, which was rst proposed by O. Boruvka in 1926 (but rediscovered by G. Chouqet in 1938 and G. Sollin in 1961). It works as follows: First, the smallest edge incident on each vertex is found; these edges form part of the minimum spanning tree. There are at least n/2 such edges. The connected components formed by these edges are collapsed into supernodes. (There are no more than n/2 such vertices at this point.) The process is repeated on supernodes and then on the resulting supersupernodes, and so on, until only a single vertex remains. This will require at most log2 n steps, because at each step the number of vertices is reduced at least by a factor of 2. Because of its inherent parallelism the nearest-neighborfrom-each-vertex approach is particularly appealing for parallel implementations. These three greedy algorithms and their variations have been implemented with dierent data structures and their relative performanceboth theoretical as well as empirical have been studied widely. The results of some of these studies can be found in [2, 13, 14, 16].

4.6.4

Constrained MST

In many applications, the minimum spanning tree is required to satisfy an additional constraint, such as (i) the degree of each vertex in the MST should be equal to or less than a specied value; or (ii) the diameter of the MST should not exceed a specied value; or (iii) the MST must have at least a specied number of leaves (vertices of degree 1 in a tree); and the like. The problem of computing such a constrained minimum spanning tree is usually NP-complete. For a discussion of various constrained MST problems and some heuristics solving them see [6].

4.7

Shortest Paths

In the preceding section we dealt with the problem of connecting a set of points with smallest cost. Another commonly encountered and somewhat related problem is that of nding the lowest-cost path (called shortest path) between a given pair of points. There are many types of shortest-path problems. For example, determining the shortest path (i.e., the most economical path or fastest path, or minimum-fuel-consumption path) from one specied vertex to another specied vertex; or shortest paths from a specied vertex to all other vertices; or perhaps shortest path between all pairs of vertices. Sometimes, one wishes to nd a shortest path from one given vertex to another given vertex that passes through certain specied intermediate vertices. In some applications, one requires not only the shortest but also the second and third shortest paths. Thus, the shortest-path problems constitute a large class of problems; particularly if we generalize it to include related problems, such as the longest-path problems, the most-reliable-path problems, the largest-capacity-path problems, and various routing problems. Therefore, the number of papers, books, reports, dissertations, and surveys dealing with the subject of shortest paths runs into hundreds [5]. Here we will discuss two very basic and important shortest-path problems: (i) how to determine the shortest distance (and a shortest path) from a specied vertex s to another specied vertex t, and (ii) how to determine shortest distances (and paths) from every vertex to every other vertex in the network. Several other problems can be solved using these two basic algorithms.

2005 by Chapman & Hall/CRC

4-20

Handbook of Data Structures and Applications

4.7.1

Single-Source Shortest Paths, Nonnegative Weights

Let us rst consider a classic algorithm due to Dijkstra for nding a shortest path (and its weight) from a specied vertex s (source or origin ) to another specied vertex t (target or sink ) in a network G in which all edge weights are nonnegative. The basic idea behind Dijkstras algorithm is to fan out from s and proceed toward t (following the directed edges), labeling the vertices with their distances from s obtained so far. The label of a vertex u is made permanent once we know that it represents the shortest possible distance from s (to u). All vertices not permanently labeled have temporary labels. We start by giving a permanent label 0 to source vertex s, because zero is the distance of s from itself. All other vertices get labeled , temporarily, because they have not been reached yet. Then we label each immediate successor v of source s, with temporary labels equal to the weight of the edge (s, v ). Clearly, the vertex, say x, with smallest temporary label (among all its immediate successors) is the vertex closest to s. Since all edges have nonnegative weights, there can be no shorter path from s to x. Therefore, we make the label of x permanent. Next, we nd all immediate successors of vertex x, and shorten their temporary labels if the path from s to any of them is shorter by going through x (than it was without going through x). Now, from among all temporarily labeled vertices we pick the one with the smallest label, say vertex y , and make its label permanent. This vertex y is the second closest vertex from s. Thus, at each iteration, we reduce the values of temporary labels whenever possible (by selecting a shorter path through the most recent permanently labeled vertex), then select the vertex with the smallest temporary label and make it permanent. We continue in this fashion until the target vertex t gets permanently labeled. In order to distinguish the permanently labeled vertices from the temporarily labeled ones, we will keep a Boolean array f inal of order n. When the ith vertex becomes permanently labeled, the ith element of this array changes from f alse to true. Another array, dist, of order n will be used to store labels of vertices. A variable recent will be used to keep track of most recent vertex to be permanently labeled. Assuming that the network is given in the form of a weight matrix W = [wij ], with weights for nonexistent edges, and vertices s and t are specied, this algorithm (which is called Dijkstras shortest-path or the label-setting algorithm ) may be described as follows (Figure 4.19):

INITIALIZATION: for all v V do dist[v ] f inal[v ] f alse pred[v ] 1 end for dist[s] 0 f inal[s] true recent s {// vertex s is permanently labeled with 0. All other vertices are temporarily labeled with . Vertex s is the most recent vertex to be permanently labeled //}

2005 by Chapman & Hall/CRC

Graphs

4-21

ITERATION: while f inal[t] = f alse do for every immediate successor v of recent do if not f inal[v ] then {// update temporary labels //} newlabel dist[recent] + wrecent,v if newlabel < dist[v ] then dist[v ] newlabel pred[v ] recent {// relabel v if there is a shorter path via vertex recent and make recent the predecessor of v on the shortest path from s //} end if end if end for let y be the vertex with the smallest temporary label, which is = f inal[y ] true recent y {// y , the next closest vertex to s gets permanently labeled //} end while FIGURE 4.19: Dijkstras shortest-path algorithm.

4.7.2

Single-Source Shortest Paths, Arbitrary Weights

In Dijkstras shortest-path algorithm (Figure 4.19), it was assumed that all edge weights wij were nonnegative numbers. If some of the edge weights are negative, Dijkstras algorithm will not work. (Negative weights in a network may represent costs and positive ones, prot.) The reason for the failure is that once the label of a vertex is made permanent, it cannot be changed in future iterations. In order to handle a network that has both positive and negative weights, we must ensure that no label is considered permanent until the program halts. Such an algorithm (called a label-correcting method , in contrast to Dijkstras labelsetting method ) is described as below. Like Dijkstras algorithm, the label of the starting vertex s is set to zero and that of every other vertex is set to , a very large number. That is, the initialization consists of dist(s) 0 for all v = s do dist(v ) end for In the iterative step, dist(v ) is always updated to the currently known distance from s to v , and the predecessor pred(v ) of v is also updated to be the predecessor vertex of v on the currently known shortest path from s to v . More compactly, the iteration may be expressed as follows: while an edge (u, v ) such that dist(u) + wuv < dist(v ) do dist(v ) dist(u) + wuv pred(v ) u end while Several implementations of this basic iterative step have been studied, experimented with, and reported in the literature. One very ecient implementation, works as follows. We maintain a queue of vertices to be examined. Initially, this queue, Q, contains only the starting vertex s. The vertex u from the front of the queue is examined (as follows)

2005 by Chapman & Hall/CRC

4-22

Handbook of Data Structures and Applications

and deleted. Examining u consists of considering all edges (u, v ) going out of u. If the length of the path to vertex v (from s) is reduced by going through u, that is, if dist(u) + wuv < dist(v ) then dist(v ) dist(u) + wuv {// dist(v ) is reset to the smaller value //} pred(v ) u end if Moreover, this vertex v is added to the queue (if it is not already in the queue) as a vertex to be examined later. Note that v enters the queue only if dist(v ) is decremented as above and if v is currently not in the queue. Observe that unlike in Dijkstras method (the label-setting method) a vertex may enter (and leave) the queue several timeseach time a shorter path is discovered. It is easy to see that the label-correcting algorithm will not terminate if the network has a cycle of negative weight.

4.7.3

All-Pairs Shortest Paths

We will now consider the problem of nding a shortest path between every pair of vertices in the network. Clearly, in an n-vertex directed graph there are n(n 1) such pathsone for each ordered pair of distinct verticesand n(n 1)/2 paths in an undirected graph. One could, of course, solve this problem by repeated application of Dijkstras algorithm, once for each vertex in the network taken as the source vertex s. We will instead consider a dierent algorithm for nding shortest paths between all pairs of vertices, which is known as Warshall-Floyd algorithm. It requires computation time proportional to n3 , and allows some of the edges to have negative weights, as long as no cycles of net negative weight exist. The algorithm works by inserting one or more vertices into paths, whenever it is advantageous to do so. Starting with n n weight matrix W = [wij ] of direct distances between the vertices of the given network G, we construct a sequence of n matrices W (1) , W (2) , . . . , W (n) . (l) Matrix W (1) , 1 l n, may be thought of as the matrix whose (i, j )th entry w ij gives the length of the shortest path among all paths from i to j with vertices 1, 2, . . . , l allowed (l) as intermediate vertices. Matrix W (l) = w ij is constructed as follows: w w
(l) ij (0) ij

= wij
(l1) } lj

= min{w

(l1) (l1) , w il ij

+w

for l = 1, 2, . . . , n

(4.1)

In other words, in iteration 1, vertex 1 is inserted in the path from vertex i to vertex j if wij > wi1 + w1j . In iteration 2, vertex 2 can be inserted, and so on. For example, in Figure 4.6 the shortest path from vertex 2 to 4 is 2134; and the following replacements occur: Iteration 1 : w Iteration 2 : w
(0) 23 (2) 24

is replaced by (w is replaced by (w
(3)

(0) 21 (2) 23

+w +w

(0) 13 ) (2) 34 )

Once the shortest distance is obtained in w 23 , the value of this entry will not be altered in subsequent operations. We assume as usual that the weight of a nonexistent edge is , that x + = , and that min{x, } = x for all x. It can easily be seen that all distance matrices W (l) calculated from (4.1) can be overwritten on W itself. The algorithm may be stated as follows:

2005 by Chapman & Hall/CRC

Graphs for l 1 to n do for i 1 to n do if wil = then for j 1 to n do wij min{wij , wil + wlj } end for end if end for end for FIGURE 4.20: All-pairs shortest distance algorithm.

4-23

If the network has no negative-weight cycle, the diagonal entries w

(n) ii

represent the length

(n)

of shortest cycles passing through vertex i. The o-diagonal entries w ij are the shortest distances. Notice that negative weight of an individual edge has no eect on this algorithm as long as there is no cycle with a net negative weight. Note that the algorithm in Figure 4.20 does not actually list the paths, it only produces their costs or weights. Obtaining paths is slightly more involved than it was in algorithm in Figure 4.19 where a predecessor array pred was sucient. Here the paths can be constructed from a path matrix P = [pij ] (also called optimal policy matrix ), in which pij is the second to the last vertex along the shortest path from i to j the last vertex being j . The path matrix P is easily calculated by adding the following steps in Figure 4.20. Initially, we set pij i, if wij = , and pij 0, if wij = . In the lth iteration if vertex l is inserted between i and j ; that is, if wil + wlj < wij , then we set pij plj . At the termination of the execution, the shortest path (i, v1 , v2 , . . . , vq , j ) from i to j can be obtained from matrix P as follows:

vq = pij vq1 = pi,vq vq2 = pi,vq1 . . . i = pi,v1

The storage requirement is n2 , no more than for storing the weight matrix itself. Since all the intermediate matrices as well as the nal distance matrix are overwritten on W itself. Another n2 storage space would be required if we generated the path matrix P also. The computation time for the algorithm in Figure 4.20 is clearly O(n3 ), regardless of the number of edges in the network.

2005 by Chapman & Hall/CRC

4-24

Handbook of Data Structures and Applications

4.8

Eulerian and Hamiltonian Graphs

A path when generalized to include visiting a vertex more than once is called a trail. In other words, a trail is a sequence of edges (v1 , v2 ), (v2 , v3 ),. . ., (vk2 , vk1 ), (vk1 , vk ) in which all the vertices (v1 , v2 , . . . , vk ) may not be distinct but all the edges are distinct. Sometimes a trail is referred to as a (non-simple) path and path is referred to as a simple path. For example in Figure 4.8(a) (b, a), (a, c), (c, d), (d, a), (a, f ) is a trail (but not a simple path because vertex a is visited twice. If the rst and the last vertex in a trail are the same, it is called a closed trail , otherwise an open trail . An Eulerian trail in a graph G = (V, E ) is one that includes every edge in E (exactly once). A graph with a closed Eulerian trail is called a Eulerian graph . Equivalently, in an Eulerian graph, G, starting from a vertex one can traverse every edge in G exactly once and return to the starting vertex. According to a theorem proved by Euler in 1736, (considered the beginning of graph theory), a connected graph is Eulerian if and only if the degree of its every vertex is even. Given a connected graph G it is easy to check if G is Eulerian. Finding an actual Eulerian trail of G is more involved. An ecient algorithm for traversing the edges of G to obtain an Euler trail was given by Fleury. The details can be found in [20]. A cycle in a graph G is said to be Hamiltonian if it passes through every vertex of G. Many families of special graphs are known to be Hamiltonian, and a large number of theorems have been proved that give sucient conditions for a graph to be Hamiltonian. However, the problem of determining if an arbitrary graph is Hamiltonian is NP-complete. Graph theory, a branch of combinatorial mathematics, has been studied for over two centuries. However, its applications and algorithmic aspects have made enormous advances only in the past fty years with the growth of computer technology and operations research. Here we have discussed just a few of the better-known problems and algorithms. Additional material is available in the references provided. In particular, for further exploration the Stanford GraphBase [10], the LEDA [12], and the Graph Boost Library [17] provide valuable and interesting platforms with collection of graph-processing programs and benchmark databases.

Acknowledgment
The author gratefully acknowledges the help provided by Hemant Balakrishnan in preparing this chapter.

References
[1] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin, Network Flows: Theory, Algorithms, and Applications, Prentice Hall, 1993. [2] B. Chazelle, A minimum spanning tree algorithm with inverse Ackermann type complexity, Journal of the ACM, Vol. 47, pp. 1028-1047, 2000. [3] T. H. Cormen, C. L. Leiserson, and R. L. Rivest, Introduction to Algorithms, MIT Press and McGraw-Hill, 1990. [4] N. Deo, Graph Theory with Applications in Engineering and Computer Science, Prentice Hall, 1974. [5] N. Deo and C. Pang, Shortest Path Algorithms: Taxonomy and Annotation, Networks, Vol. 14, pp. 275-323, 1984. [6] N. Deo and N. Kumar, Constrained spanning tree problem: approximate methods and parallel computation, American Math Society, Vol. 40, pp. 191-217, 1998

2005 by Chapman & Hall/CRC

Graphs
[7] H. N. Gabow,Path-based depth-rst search for strong and biconnected components, Information Processing, Vol. 74, pp. 107-114, 2000. [8] R. L. Graham and P. Hell, On the history of minimum spanning tree problem, Annals of the History of Computing, Vol. 7, pp. 43-57, 1985. [9] E. Horowitz, S. Sahni, and B. Rajasekaran, Computer Algorithms/C++, Computer Science Press, 1996. [10] D. E. Knuth, The Stanford GraphBase: A Platform for Combinatorial Computing, Addison-Wesley, 1993. [11] K. Mehlhorn, Data Structures and Algorithms 2: NP-Completeness and Graph Algorithms, Springer-Verlag, 1984. [12] K. Mehlhorn and S. Naher, LEDA: A platform for combinatorial and Geometric Computing, Cambridge University Press, 1999. [13] B. M. E. Moret and H. D. Shapiro, An empirical analysis of algorithms for constructing minimum spanning tree, Lecture Notes in Computer Science, Vol. 519, pp. 400-411, 1991. [14] C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, Prentice-Hall, 1982. [15] E. M. Reingold, J. Nievergelt, and N. Deo, Combinatorial Algorithms: Theory and Practice, Prentice-Hall, 1977. [16] R. Sedgewick, Algorithms in C: Part 5 Graph Algorithms, Addison-Wesley, third edition, 2002. [17] J. G. Siek, L. Lee, and A. Lumsdaine, The Boost Graph Library - User Guide and Reference Manual, Addison Wesley, 2002. [18] M. M. Syslo, N. Deo, and J. S. Kowalik, Discrete Optimization Algorithms : with Pascal Programs, Prentice-hall, 1983. [19] R. E. Tarjan, Data Structures and Network Algorithms, Society for Industrial and Applied Mathematics, 1983. [20] K. Thulasiraman and M. N. S. Swamy, Graphs: Theory and Algorithms, WileyInterscience, 1992.

4-25

2005 by Chapman & Hall/CRC

II
Priority Queues
5 Leftist Trees
Introduction

Sartaj Sahni . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Height-Biased Leftist Trees

5-1 6-1

Weight-Biased Leftist Trees

6 Skew Heaps

C. Pandu Rangan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Meldable Priority Queues and Skew

Introduction Basics of Amortized Analysis Heaps Bibliographic Remarks

7 Binomial, Fibonacci, and Pairing Heaps

Michael L. Fredman . . . . . .

7-1

Introduction Binomial Heaps Fibonacci Heaps Pairing Heaps Summaries of the Algorithms Related Developments

Pseudocode

8 Double-Ended Priority Queues

Sartaj Sahni . . . . . . . . . . . . . . . . . . . . . . . .

8-1

Denition and an Application Symmetric Min-Max Heaps Interval Heaps Max Heaps Deaps Generic Methods for DEPQs Meldable DEPQs

Min-

2005 by Chapman & Hall/CRC

5
Leftist Trees
5.1 5.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Height-Biased Leftist Trees . . . . . . . . . . . . . . . . . . . . . . .
Denition Insertion into a Max HBLT Deletion of Max Element from a Max HBLT Melding Two Max HBLTs Initialization Deletion of Arbitrary Element from a Max HBLT

5-1 5-2

Sartaj Sahni
University of Florida

5.3

Weight-Biased Leftist Trees . . . . . . . . . . . . . . . . . . . . . . .

Denition

5-8

Max WBLT Operations

5.1

Introduction

A single-ended priority queue (or simply, a priority queue) is a collection of elements in which each element has a priority. There are two varieties of priority queuesmax and min. The primary operations supported by a max (min) priority queue are (a) nd the element with maximum (minimum) priority, (b) insert an element, and (c) delete the element whose priority is maximum (minimum). However, many authors consider additional operations such as (d) delete an arbitrary element (assuming we have a pointer to the element), (e) change the priority of an arbitrary element (again assuming we have a pointer to this element), (f) meld two max (min) priority queues (i.e., combine two max (min) priority queues into one), and (g) initialize a priority queue with a nonzero number of elements. Several data structures: e.g., heaps (Chapter 3), leftist trees [2, 5], Fibonacci heaps [7] (Chapter 7), binomial heaps [1] (Chapter 7), skew heaps [11] (Chapter 6), and pairing heaps [6] (Chapter 7) have been proposed for the representation of a priority queue. The dierent data structures that have been proposed for the representation of a priority queue dier in terms of the performance guarantees they provide. Some guarantee good performance on a per operation basis while others do this only in the amortized sense. Max (min) heaps permit one to delete the max (min) element and insert an arbitrary element into an n element priority queue in O(log n) time per operation; a nd max (min) takes O(1) time. Additionally, a heap is an implicit data structure that has no storage overhead associated with it. All other priority queue structures are pointer-based and so require additional storage for the pointers. Max (min) leftist trees also support the insert and delete max (min) operations in O(log n) time per operation and the nd max (min) operation in O(1) time. Additionally, they permit us to meld pairs of priority queues in logarithmic time. The remaining structures do not guarantee good complexity on a per operation basis. They do, however, have good amortized complexity. Using Fibonacci heaps, binomial queues, or skew heaps, nd max (min), inserts and melds take O(1) time (actual and amortized) and a delete max (min) takes O(log n) amortized time. When a pairing heap is

5-1

2005 by Chapman & Hall/CRC

5-2

Handbook of Data Structures and Applications

a b (a) A binary tree 2 1 0 0 1 0 0 1 0 (d) w values

FIGURE 5.1: s and w values.

f c d e

(b) Extended binary tree 5

1 0

2 1 1

(c) s values

used, the amortized complexity is O(1) for nd max (min) and insert (provided no decrease key operations are performed) and O(logn) for delete max (min) operations [12]. Jones [8] gives an empirical evaluation of many priority queue data structures. In this chapter, we focus on the leftist tree data structure. Two varieties of leftist trees height-biased leftist trees [5] and weight-biased leftist trees [2] are described. Both varieties of leftist trees are binary trees that are suitable for the representation of a single-ended priority queue. When a max (min) leftist tree is used, the traditional single-ended priority queue operations nd max (min) element, delete/remove max (min) element, and insert an elementtake, respectively, O(1), O(log n) and O(log n) time each, where n is the number of elements in the priority queue. Additionally, an n-element max (min) leftist tree can be initialized in O(n) time and two max (min) leftist trees that have a total of n elements may be melded into a single max (min) leftist tree in O(log n) time.

5.2
5.2.1

Height-Biased Leftist Trees

Denition

Consider a binary tree in which a special node called an external node replaces each empty subtree. The remaining nodes are called internal nodes. A binary tree with external nodes added is called an extended binary tree. Figure 5.1(a) shows a binary tree. Its corresponding extended binary tree is shown in Figure 5.1(b). The external nodes appear as shaded boxes. These nodes have been labeled a through f for convenience. Let s(x) be the length of a shortest path from node x to an external node in its subtree. From the denition of s(x), it follows that if x is an external node, its s value is 0.

2005 by Chapman & Hall/CRC

Leftist Trees Furthermore, if x is an internal node, its s value is min{s(L), s(R)} + 1

5-3

where L and R are, respectively, the left and right children of x. The s values for the nodes of the extended binary tree of Figure 5.1(b) appear in Figure 5.1(c).
DEFINITION 5.1

[Crane [5]] A binary tree is a height-biased leftist tree (HBLT) i at every internal node, the s value of the left child is greater than or equal to the s value of the right child. The binary tree of Figure 5.1(a) is not an HBLT. To see this, consider the parent of the external node a. The s value of its left child is 0, while that of its right is 1. All other internal nodes satisfy the requirements of the HBLT denition. By swapping the left and right subtrees of the parent of a, the binary tree of Figure 5.1(a) becomes an HBLT.

THEOREM 5.1

Let x be any internal node of an HBLT.

(a) The number of nodes in the subtree with root x is at least 2s(x) 1. (b) If the subtree with root x has m nodes, s(x) is at most log2 (m + 1). (c) The length, rightmost(x), of the right-most path from x to an external node (i.e., the path obtained by beginning at x and making a sequence of right-child moves) is s(x).
Proof From the denition of s(x), it follows that there are no external nodes on the s(x) 1 levels immediately below node x (as otherwise the s value of x would be less). The subtree with root x has exactly one node on the level at which x is, two on the next level, four on the next, , and 2s(x)1 nodes s(x) 1 levels below x. The subtree may have additional nodes at levels more than s(x) 1 below x. Hence the number of nodes in the s(x)1 subtree x is at least i=0 2i = 2s(x) 1. Part (b) follows from (a). Part (c) follows from the denition of s and the fact that, in an HBLT, the s value of the left child of a node is always greater than or equal to that of the right child. DEFINITION 5.2 A max tree (min tree) is a tree in which the value in each node is greater (less) than or equal to those in its children (if any).

Some max trees appear in Figure 5.2, and some min trees appear in Figure 5.3. Although these examples are all binary trees, it is not necessary for a max tree to be binary. Nodes of a max or min tree may have an arbitrary number of children.
DEFINITION 5.3 A max HBLT is an HBLT that is also a max tree. A min HBLT is an HBLT that is also a min tree.

The max trees of Figure 5.2 as well as the min trees of Figure 5.3 are also HBLTs; therefore, the trees of Figure 5.2 are max HBLTs, and those of Figure 5.3 are min HBLTs. A max priority queue may be represented as a max HBLT, and a min priority queue may be represented as a min HBLT.

2005 by Chapman & Hall/CRC

5-4

Handbook of Data Structures and Applications

14 12 10 8 (a) 6 7 5 (b) 6

9 25

30

(c)

FIGURE 5.2: Some max trees.

2 7 10 8 (a) 6 4 50 20

10 21

11

(b)
FIGURE 5.3: Some min trees.

(c)

5.2.2

Insertion into a Max HBLT

The insertion operation for max HBLTs may be performed by using the max HBLT meld operation, which combines two max HBLTs into a single max HBLT. Suppose we are to insert an element x into the max HBLT H . If we create a max HBLT with the single element x and then meld this max HBLT and H , the resulting max HBLT will include all elements in H as well as the element x. Hence an insertion may be performed by creating a new max HBLT with just the element that is to be inserted and then melding this max HBLT and the original.

5.2.3

Deletion of Max Element from a Max HBLT

The max element is in the root. If the root is deleted, two max HBLTs, the left and right subtrees of the root, remain. By melding together these two max HBLTs, we obtain a max HBLT that contains all elements in the original max HBLT other than the deleted max element. So the delete max operation may be performed by deleting the root and then melding its two subtrees.

5.2.4

Melding Two Max HBLTs

Since the length of the right-most path of an HBLT with n elements is O(log n), a meld algorithm that traverses only the right-most paths of the HBLTs being melded, spending O(1) time at each node on these two paths, will have complexity logarithmic in the number of elements in the resulting HBLT. With this observation in mind, we develop a meld algorithm that begins at the roots of the two HBLTs and makes right-child moves only. The meld strategy is best described using recursion. Let A and B be the two max HBLTs that are to be melded. If one is empty, then we may use the other as the result. So assume that neither is empty. To perform the meld, we compare the elements in the two roots. The root with the larger element becomes the root of the melded HBLT. Ties may be broken

2005 by Chapman & Hall/CRC

Leftist Trees

5-5

arbitrarily. Suppose that A has the larger root and that its left subtree is L. Let C be the max HBLT that results from melding the right subtree of A and the max HBLT B . The result of melding A and B is the max HBLT that has A as its root and L and C as its subtrees. If the s value of L is smaller than that of C , then C is the left subtree. Otherwise, L is.
Example 5.1

Consider the two max HBLTs of Figure 5.4(a). The s value of a node is shown outside the node, while the element value is shown inside. When drawing two max HBLTs that are to be melded, we will always draw the one with larger root value on the left. Ties are broken arbitrarily. Because of this convention, the root of the left HBLT always becomes the root of the nal HBLT. Also, we will shade the nodes of the HBLT on the right. Since the right subtree of 9 is empty, the result of melding this subtree of 9 and the tree with root 7 is just the tree with root 7. We make the tree with root 7 the right subtree of 9 temporarily to get the max tree of Figure 5.4(b). Since the s value of the left subtree of 9 is 0 while that of its right subtree is 1, the left and right subtrees are swapped to get the max HBLT of Figure 5.4(c). Next consider melding the two max HBLTs of Figure 5.4(d). The root of the left subtree becomes the root of the result. When the right subtree of 10 is melded with the HBLT with root 7, the result is just this latter HBLT. If this HBLT is made the right subtree of 10, we get the max tree of Figure 5.4(e). Comparing the s values of the left and right children of 10, we see that a swap is not necessary. Now consider melding the two max HBLTs of Figure 5.4(f). The root of the left subtree is the root of the result. We proceed to meld the right subtree of 18 and the max HBLT with root 10. The two max HBLTs being melded are the same as those melded in Figure 5.4(d). The resultant max HBLT (Figure 5.4(e)) becomes the right subtree of 18, and the max tree of Figure 5.4(g) results. Comparing the s values of the left and right subtrees of 18, we see that these subtrees must be swapped. Swapping results in the max HBLT of Figure 5.4(h). As a nal example, consider melding the two max HBLTs of Figure 5.4(i). The root of the left max HBLT becomes the root of the result. We proceed to meld the right subtree of 40 and the max HBLT with root 18. These max HBLTs were melded in Figure 5.4(f). The resultant max HBLT (Figure 5.4(g)) becomes the right subtree of 40. Since the left subtree of 40 has a smaller s value than the right has, the two subtrees are swapped to get the max HBLT of Figure 5.4(k). Notice that when melding the max HBLTs of Figure 5.4(i), we rst move to the right child of 40, then to the right child of 18, and nally to the right child of 10. All moves follow the right-most paths of the initial max HBLTs.

5.2.5

Initialization

It takes O(n log n) time to initialize a max HBLT with n elements by inserting these elements into an initially empty max HBLT one at a time. To get a linear time initialization algorithm, we begin by creating n max HBLTs with each containing one of the n elements. These n max HBLTs are placed on a FIFO queue. Then max HBLTs are deleted from this queue in pairs, melded, and added to the end of the queue until only one max HBLT remains.
Example 5.2

We wish to create a max HBLT with the ve elements 7, 1, 9, 11, and 2. Five singleelement max HBLTs are created and placed in a FIFO queue. The rst two, 7 and 1,

2005 by Chapman & Hall/CRC

5-6

Handbook of Data Structures and Applications

9 7 1 1 7

(a) 1 1 5 (d) 2 1 6 1 5 (g) 2 1 1 20 1 5 (j) 30 2 10 7 1 2 10 2 7 1 1 5 2 10 7 (h) 1 1 2

(b) 2 7 (e) 2 6 1 1 1 20 30 1 5 (i) 2 18 2 6 1 1 5 2 2 10 7 1 (k)

FIGURE 5.4: Melding max HBLTs.

(c) 1 7 1 1 (f) 2 7 1 5

10

7 1 5

10

18

10

18

18

40 10 1 1 6

18

40 18 6

40 30 1

1 1

20

are deleted from the queue and melded. The result (Figure 5.5(a)) is added to the queue. Next the max HBLTs 9 and 11 are deleted from the queue and melded. The result appears in Figure 5.5(b). This max HBLT is added to the queue. Now the max HBLT 2 and that of Figure 5.5(a) are deleted from the queue and melded. The resulting max HBLT (Figure 5.5(c)) is added to the queue. The next pair to be deleted from the queue consists of the max HBLTs of Figures Figure 5.5 (b) and (c). These HBLTs are melded to get the max HBLT of Figure 5.5(d). This max HBLT is added to the queue. The queue now has just one max HBLT, and we are done with the initialization.

2005 by Chapman & Hall/CRC

Leftist Trees

5-7

7 1 9

11 1

7 2 1 7

11 9 2 (d)

(a)

(b)

(c)

FIGURE 5.5: Initializing a max HBLT. For the complexity analysis of of the initialization operation, assume, for simplicity, that n is a power of 2. The rst n/2 melds involve max HBLTs with one element each, the next n/4 melds involve max HBLTs with two elements each; the next n/8 melds are with trees that have four elements each; and so on. The time needed to meld two leftist trees with 2i elements each is O(i + 1), and so the total time for the initialization is O(n/2 + 2 (n/4) + 3 (n/8) + ) = O(n i ) = O(n) 2i

5.2.6

Deletion of Arbitrary Element from a Max HBLT

Although deleting an element other than the max (min) element is not a standard operation for a max (min) priority queue, an ecient implementation of this operation is required when one wishes to use the generic methods of Cho and Sahni [3] and Chong and Sahni [4] to derive ecient mergeable double-ended priority queue data structures from ecient singleended priority queue data structures. From a max or min leftist tree, we may remove the element in any specied node theN ode in O(log n) time, making the leftist tree a suitable base structure from which an ecient mergeable double-ended priority queue data structure may be obtained [3, 4]. To remove the element in the node theN ode of a height-biased leftist tree, we must do the following: 1. Detach the subtree rooted at theN ode from the tree and replace it with the meld of the subtrees of theN ode. 2. Update s values on the path from theN ode to the root and swap subtrees on this path as necessary to maintain the leftist tree property. To update s on the path from theN ode to the root, we need parent pointers in each node. This upward updating pass stops as soon as we encounter a node whose s value does not change. The changed s values (with the exception of possibly O(log n) values from moves made at the beginning from right children) must form an ascending sequence (actually, each must be one more than the preceding one). Since the maximum s value is O(log n) and since all s values are positive integers, at most O(log n) nodes are encountered in the updating pass. At each of these nodes, we spend O(1) time. Therefore, the overall complexity of removing the element in node theN ode is O(log n).

2005 by Chapman & Hall/CRC

5-8

Handbook of Data Structures and Applications

5.3
5.3.1

Weight-Biased Leftist Trees

Denition

We arrive at another variety of leftist tree by considering the number of nodes in a subtree, rather than the length of a shortest root to external node path. Dene the weight w(x) of node x to be the number of internal nodes in the subtree with root x. Notice that if x is an external node, its weight is 0. If x is an internal node, its weight is 1 more than the sum of the weights of its children. The weights of the nodes of the binary tree of Figure 5.1(a) appear in Figure 5.1(d)
DEFINITION 5.4 [Cho and Sahni [2]] A binary tree is a weight-biased leftist tree (WBLT) i at every internal node the w value of the left child is greater than or equal to the w value of the right child. A max (min) WBLT is a max (min) tree that is also a WBLT.

Note that the binary tree of Figure 5.1(a) is not a WBLT. However, all three of the binary trees of Figure 5.2 are WBLTs.
THEOREM 5.2

Let x be any internal node of a weight-biased leftist tree. The length, rightmost(x), of the right-most path from x to an external node satises rightmost(x) log2 (w(x) + 1).

Proof The proof is by induction on w(x). When w(x) = 1, rightmost(x) = 1 and log2 (w(x) + 1) = log2 2 = 1. For the induction hypothesis, assume that rightmost(x) log2 (w(x)+1) whenever w(x) < n. Let RightChild(x) denote the right child of x (note that this right child may be an external node). When w(x) = n, w(RightChild(x)) (n 1)/2 and rightmost(x) = 1 + rightmost(RightChild(x)) 1 + log2 ((n 1)/2 + 1) = 1 + log2 (n + 1) 1 = log2 (n + 1).

5.3.2

Max WBLT Operations

Insert, delete max, and initialization are analogous to the corresponding max HBLT operation. However, the meld operation can be done in a single top-to-bottom pass (recall that the meld operation of an HBLT performs a top-to-bottom pass as the recursion unfolds and then a bottom-to-top pass in which subtrees are possibly swapped and s-values updated). A single-pass meld is possible for WBLTs because we can determine the w values on the way down and so, on the way down, we can update w-values and swap subtrees as necessary. For HBLTs, a nodes new s value cannot be determined on the way down the tree. Since the meld operation of a WBLT may be implemented using a single top-to-bottom pass, inserts and deletes also use a single top-to-bottom pass. Because of this, inserts and deletes are faster, by a constant factor, in a WBLT than in an HBLT [2]. However, from a WBLT, we cannot delete the element in an arbitrarily located node, theN ode, in O(log n) time. This is because theN ode may have O(n) ancestors whose w value is to be updated. So, WBLTs are not suitable for mergeable double-ended priority queue applications [3, 8]. C++ and Java codes for HBLTs and WBLTs may be obtained from [9] and [10], respectively.

2005 by Chapman & Hall/CRC

Leftist Trees

5-9

Acknowledgment
This work was supported, in part, by the National Science Foundation under grant CCR9912395.

References
[1] M. Brown, Implementation and analysis of binomial queue algorithms, SIAM Jr. on Computing, 7, 3, 1978, 298-319. [2] S. Cho and S. Sahni, Weight biased leftist trees and modied skip lists, ACM Jr. on Experimental Algorithmics, Article 2, 1998. [3] S. Cho and S. Sahni, Mergeable double-ended priority queues. International Journal on Foundations of Computer Science, 10, 1, 1999, 1-18. [4] K. Chong and S. Sahni, Correspondence based data structures for double ended priority queues. ACM Jr. on Experimental Algorithmics, Volume 5, 2000, Article 2, 22 pages. [5] C. Crane, Linear Lists and Priority Queues as Balanced Binary Trees, Tech. Rep. CS-72-259, Dept. of Comp. Sci., Stanford University, 1972. [6] M. Fredman, R. Sedgewick, D. Sleator, and R.Tarjan, The pairing heap: A new form of self-adjusting heap. Algorithmica, 1, 1986, 111-129. [7] M. Fredman and R. Tarjan, Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms, JACM, 34, 3, 1987, 596-615. [8] D. Jones, An empirical comparison of priority-queue and event-set implementations, Communications of the ACM, 29, 4, 1986, 300-311. [9] S. Sahni, Data Structures, Algorithms, and Applications in C++, McGraw-Hill, NY, 1998, 824 pages. [10] S. Sahni, Data Structures, Algorithms, and Applications in Java, McGraw-Hill, NY, 2000, 846 pages. [11] D. Sleator and R. Tarjan, Self-adjusting heaps, SIAM Jr. on Computing, 15, 1, 1986, 52-69. [12] J. Stasko and J. Vitter, Pairing heaps: Experiments and analysis, Communications of the ACM, 30, 3, 1987, 234-249.

2005 by Chapman & Hall/CRC

6
Skew Heaps
6.1 6.2 6.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basics of Amortized Analysis . . . . . . . . . . . . . . . . . . . . . Meldable Priority Queues and Skew Heaps . . . . .
Meldable Priority Queue Operations Cost of Meld Operation

6-1 6-2 6-5 6-9

Amortized

C. Pandu Rangan
Indian Institute of Technology, Madras

6.4

Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6.1

Introduction

Priority Queue is one of the most extensively studied Abstract Data Types (ADT) due to its fundamental importance in the context of resource managing systems, such as operating systems. Priority Queues work on nite subsets of a totally ordered universal set U . Without any loss of generality we assume that U is simply the set of all non-negative integers. In its simplest form, a Priority Queue supports two operations, namely, insert(x, S ) : update S by adding an arbitrary x U to S . delete-min(S ) : update S by removing from S the minimum element of S . We will assume for the sake of simplicity, all the items of S are distinct. Thus, we assume that x S at the time of calling insert(x, S ). This increases the cardinality of S , denoted usually by |S |, by one. The well-known data structure Heaps, provide an elegant and ecient implementation of Priority Queues. In the Heap based implementation, both insert(x, S ) and delete-min(S ) take O(log n) time where n = |S |. Several extensions for the basic Priority Queues were proposed and studied in response to the needs arising in several applications. For example, if an operating system maintains a set of jobs, say print requests, in a priority queue, then, always, the jobs with high priority are serviced irrespective of when the job was queued up. This might mean some kind of unfairness for low priority jobs queued up earlier. In order to straighten up the situation, we may extend priority queue to support delete-max operation and arbitrarily mix delete-min and delete-max operations to avoid any undue stagnation in the queue. Such priority queues are called Double Ended Priority Queues. It is easy to see that Heap is not an appropriate data structure for Double Ended Priority Queues. Several interesting alternatives are available in the literature [1] [3] [4]. You may also refer Chapter 8 of this handbook for a comprehensive discussion on these structures. In another interesting extension, we consider adding an operation called melding. A meld operation takes two disjoint sets, S1 and S2 , and produces the set S = S1 S2 . In terms of an implementation, this requirement translates to building a data structure for S , given

6-1

2005 by Chapman & Hall/CRC

6-2

Handbook of Data Structures and Applications

the data structures of S1 and S2 . A Priority Queue with this extension is called a Meldable Priority Queue. Consider a scenario where an operating system maintains two dierent priority queues for two printers and one of the printers is down with some problem during operation. Meldable Priority Queues naturally model such a situation. Again, maintaining the set items in Heaps results in very inecient implementation of Meldable Priority Queues. Specically, designing a data structure with O(log n) bound for each of the Meldable Priority Queue operations calls for more sophisticated ideas and approaches. An interesting data structure called Leftist Trees, implements all the operations of Meldable Priority Queues in O(log n) time. Leftist Trees are discussed in Chapter 5 of this handbook. The main objective behind the design of a data structure for an ADT is to implement the ADT operations as eciently as possible. Typically, eciency of a structure is judged by its worst-case performance. Thus, when designing a data structure, we seek to minimize the worst case complexity of each operation. While this is a most desirable goal and has been theoretically realized for a number of data structures for key ADTs, the data structures optimizing worst-case costs of ADT operations are often very complex and pretty tedious to implement. Hence, computer scientists were exploring alternative design criteria that would result in simpler structures without losing much in terms of performance. In Chapter 13 of this handbook, we show that incorporating randomness provides an attractive alternative avenue for designers of the data structures. In this chapter we will explore yet another design goal leading to simpler structural alternatives without any degrading in overall performance. Since the data structures are used as basic building blocks for implementing algorithms, a typical execution of an algorithm might consist of a sequence of operations using the data structure over and again. In the worst case complexity based design, we seek to reduce the cost of each operation as much as possible. While this leads to an overall reduction in the cost for the sequence of operations, this poses some constraints on the designer of data structure. We may relax the requirement that the cost of each operation be minimized and perhaps design data structures that seek to minimize the total cost of any sequence of operations. Thus, in this new kind of design goal, we will not be terribly concerned with the cost of any individual operations, but worry about the total cost of any sequence of operations. At rst thinking, this might look like a formidable goal as we are attempting to minimize the cost of an arbitrary mix of ADT operations and it may not even be entirely clear how this design goal could lead to simpler data structures. Well, it is typical of a novel and deep idea; at rst attempt it may puzzle and bamboozle the learner and with practice one tends to get a good intuitive grasp of the intricacies of the idea. This is one of those ideas that requires some getting used to. In this chapter, we discuss about a data structure called Skew heaps. For any sequence of a Meldable Priority Queue operations, its total cost on Skew Heaps is asymptotically same as its total cost on Leftist Trees. However, Skew Heaps are a bit simpler than Leftist Trees.

6.2

Basics of Amortized Analysis

We will now clarify the subtleties involved in the new design goal with an example. Consider a typical implementation of Dictionary operations. The so called Balanced Binary Search Tree structure (BBST) implements these operations in O(m log n) worst case bound. Thus, the total cost of an arbitrary sequence of m dictionary operations, each performed on a tree of size at most n, will be O(log n). Now we may turn around and ask: Is there a data structure on which the cost of a sequence of m dictionary operations is O(m log n) but

2005 by Chapman & Hall/CRC

Skew Heaps

6-3

individual operations are not constrained to have O(log n) bound? Another more pertinent question to our discussion - Is that structure simpler than BBST, at least in principle? An armative answer to both the questions is provided by a data structure called Splay Trees. Splay Tree is the theme of Chapter 12 of this handbook. Consider for example a sequence of m dictionary operations S1 , S2 , ..., Sm , performed using a BBST. Assume further that the size of the tree has never exceeded n during the sequence of operations. It is also fairly reasonable to assume that we begin with an empty tree and this would imply n m. Let the actual cost of executing Si be Ci . Then the total cost of the sequence of operations is C1 + C2 + + Cm . Since each Ci is O(log n) we easily conclude that the total cost is O(m log n). No big arithmetic is needed and the analysis is easily nished. Now, assume that we execute the same sequence of m operations but employ a Splay Tree in stead of a BBST. Assuming that ci is the actual cost of Si in a Splay Tree, the total cost for executing the sequence of operation turns out to be c1 + c2 + . . . + cm . This sum, however, is tricky to compute. This is because a wide range of values are possible for each of ci and no upper bound other than the trivial bound of O(n) is available for ci . Thus, a naive, worst case cost analysis would yield only a weak upper bound of O(nm) whereas the actual bound is O(m log n). But how do we arrive at such improved estimates? This is where we need yet another powerful tool called potential function. The potential function is purely a conceptual entity and this is introduced only for the sake of computing a sum of widely varying quantities in a convenient way. Suppose there is a function f : D R+ {0}, that maps a conguration of the data structure to a non-negative real number. We shall refer to this function as potential function. Since the data type as well as data structures are typically dynamic, an operation may change the conguration of data structure and hence there may be change of potential value due to this change of conguration. Referring back to our sequence of operations S1 , S2 , . . . , Sm , let Di1 denote the conguration of data structure before the executing the operation Si and Di denote the conguration after the execution of Si . The potential dierence due to this operation is dened to be the quantity f (Di ) f (Di1 ). Let ci denote the actual cost of Si . We will now introduce yet another quantity, ai , dened by ai = ci + f (Di ) f (Di1 ). What is the consequence of this denition?
m m

Note that
i=1

ai =
i=1

ci + f (Dm ) f (D0 ).

Let us introduce one more reasonable assumption that f (D0 ) = f () = 0. Since f (D) 0 for all non empty structures, we obtain, ai = ci + f (Dm ) ci

If we are able to choose cleverly a good potential function so that ai s have tight, uniform bound, then we can evaluate the sum ai easily and this bounds the actual cost sum ci . In other words, we circumvent the diculties posed by wide variations in ci by introducing new quantities ai which have uniform bounds. A very neat idea indeed! However, care must be exercised while dening the potential function. A poor choice of potential function will result in ai s whose sum may be a trivial or useless bound for the sum of actual costs. In fact, arriving at the right potential function is an ingenious task, as you will understand by the end of this chapter or by reading the chapter on Splay Trees.

2005 by Chapman & Hall/CRC

6-4

Handbook of Data Structures and Applications

The description of the data structures such as Splay Trees will not look any dierent from the description of a typical data structures - it comprises of a description of the organization of the primitive data items and a bunch of routines implementing ADT operations. The key dierence is that the routines implementing the ADT operations will not be analyzed for their individual worst case complexity. We will only be interested in the the cumulative eect of these routines in an arbitrary sequence of operations. Analyzing the average potential contribution of an operation in an arbitrary sequence of operations is called amortized analysis. In other words, the routines implementing the ADT operations will be analyzed for their amortized cost. Estimating the amortized cost of an operation is rather an intricate task. The major diculty is in accounting for the wide variations in the costs of an operation performed at dierent points in an arbitrary sequence of operations. Although our design goal is inuenced by the costs of sequence of operations, dening the notion of amortized cost of an operation in terms of the costs of sequences of operations leads one nowhere. As noted before, using a potential function to o set the variations in the actual costs is a neat way of handling the situation. In the next denition we formalize the notion of amortized cost. [Amortized Cost] Let A be an ADT with basic operations O = {O1 , O2 , , Ok } and let D be a data structure implementing A. Let f be a potential function dened on the congurations of the data structures to non-negative real number. Assume further that f () = 0. Let D denote a conguration we obtain if we perform an operation Ok on a conguration D and let c denote the actual cost of performing Ok on D. Then, the amortized cost of Ok operating on D, denoted as a(Ok , D), is given by
DEFINITION 6.1

a(Ok , D) = c + f (D ) f (D) If a(Ok , D) c g (n) for all conguration D of size n, then we say that the amortized cost of Ok is O(g (n)). Let D be a data structure implementing an ADT and let s1 , s2 , , sm denote an arbitrary sequence of ADT operations on the data structure starting from an empty structure D0 . Let ci denote actual cost of the operation si and Di denote the conguration obtained which si operated on Di1 , for 1 i m. Let ai denote the amortized cost of si operating on Di1 with respect to an arbitrary potential function. Then,
THEOREM 6.1
m m

ci
i=1 i=1

ai .

Proof

Since ai is the amortized cost of si working on the conguration Di1 , we have ai = a(si , Di1 ) = ci + f (Di ) f (Di1 )

Therefore,

2005 by Chapman & Hall/CRC

Skew Heaps

6-5

ai
i=1

=
i=1

ci + (f (Dm ) f (D0 ))
m

= f (Dm ) +
i=1 m

ci (since f (D0 ) = 0)

i=1

ci

REMARK 6.1 The potential function is common to the denition of amortized cost of m m all the ADT operations. Since i=1 ai i=1 ci holds good for any potential function, a clever choice of the potential function will yield tight upper bound for the sum of actual cost of a sequence of operations.

6.3

Meldable Priority Queues and Skew Heaps

DEFINITION 6.2 [Skew Heaps] A Skew Heap is simply a binary tree. Values are stored in the structure, one per node, satisfying the heap-order property: A value stored at a node is larger than the value stored at its parent, except for the root (as root has no parent). REMARK 6.2 Throughout our discussion, we handle sets with distinct items. Thus a set of n items is represented by a skew heap of n nodes. The minimum of the set is always at the root. On any path starting from the root and descending towards a leaf, the values are in increasing order.

6.3.1

Meldable Priority Queue Operations

Recall that a Meldable Priority queue supports three key operations: insert, delete-min and meld. We will rst describe the meld operation and then indicate how other two operations can be performed in terms of the meld operation. Let S1 and S2 be two sets and H1 and H2 be Skew Heaps storing S1 and S2 respectively. Recall that S1 S2 = . The meld operation should produce a single Skew Heap storing the values in S1 S2 . The procedure meld (H1 , H2 ) consists of two phases. In the rst phase, the two right most paths are merged to obtain a single right most path. This phase is pretty much like the merging algorithm working on sorted sequences. In this phase, the left subtrees of nodes in the right most paths are not disturbed. In the second phase, we simply swap the children of every node on the merged path except for the lowest. This completes the process of melding. Figures 6.1, 6.2 and 6.3 clarify the phases involved in the meld routine. Figure 6.1 shows two Skew Heaps H1 and H2 . In Figure 6.2 we have shown the scenario after the completion of the rst phase. Notice that right most paths are merged to obtain the right most path of a single tree, keeping the respective left subtrees intact. The nal

2005 by Chapman & Hall/CRC

6-6

Handbook of Data Structures and Applications

7 5

33 35 10

20

15

43

23

11

25

40

H1

H2
FIGURE 6.1: Skew Heaps for meld operation.

33

43

35

23

10

20

11

25

15

40

FIGURE 6.2: Rightmost paths are merged. Left subtrees of nodes in the merged path are intact.

Skew Heap is obtained in Figure 6.3. Note that left and right child of every node on the right most path of the tree in Figure 6.2 (except the lowest) are swapped to obtain the nal Skew Heap.

2005 by Chapman & Hall/CRC

Skew Heaps
5

6-7

33

35

43

10

23

11

20

15

25

40

FIGURE 6.3: Left and right children of nodes (5), (7), (9), (10), (11) of Figure 2 are swapped. Notice that the children of (15) which is the lowest node in the merged path, are not swapped.

It is easy to implement delete-min and insert in terms of the meld operation. Since minimum is always found at the root, delete-min is done by simply removing the root and melding its left subtree and right subtree. To insert an item x in a Skew Heap H1 , we create a Skew Heap H2 consisting of only one node containing x and then meld H1 and H2 . From the above discussion, it is clear that cost of meld essentially determines the cost of insert and delete-min. In the next section, we analyze the amortized cost of meld operation.

6.3.2

Amortized Cost of Meld Operation

At this juncture we are left with the crucial task of identifying a suitable potential function. Before proceeding further, perhaps one should try the implication of certain simple potential functions and experiment with the resulting amortized cost. For example, you may try the function f (D) = number of nodes in D( and discover how ineective it is!). We need some denitions to arrive at our potential function.
DEFINITION 6.3 For any node x in a binary tree, the weight of x, denoted wt(x), is the number of descendants of x, including itself. A non-root node x is said to be heavy if wt(x) > wt(parent(x))/2. A non-root node that is not heavy is called light. The root is neither light nor heavy.

2005 by Chapman & Hall/CRC

6-8

Handbook of Data Structures and Applications

The next lemma is an easy consequence of the denition given above. All logarithms in this section have base 2.
LEMMA 6.1 For any node, at most one of its children is heavy. Furthermore, any root to leaf path in a n-node tree contains at most log n light nodes.

[Potential Function] A non-root is called right if it is the right child of its parent; it is called left otherwise. The potential of a skew heap is the number of right heavy node it contains. That is, f (H ) = number of right heavy nodes in H . We extend the denition of potential function to a collection of skew heaps as follows: f (H1 , H2 , , Ht ) = t i=1 f (Hi ).
DEFINITION 6.4

Here is the key result of this chapter.

THEOREM 6.2

Let H1 and H2 be two heaps with n1 and n2 nodes respectively. Let n = n1 + n2 . The amortized cost of meld (H1 , H2 ) is O(log n).

Proof

Let h1 and h2 denote the number of heavy nodes in the right most paths of H1 and H2 respectively. The number of light nodes on them will be at most log n1 and log n2 respectively. Since a node other than root is either heavy or light, and there are two root nodes here that are neither heavy or light, the total number of nodes in the right most paths is at most 2 + h1 + h2 + log n1 + log n2 2 + h1 + h2 + 2 log n Thus we get a bound for actual cost c as c 2 + h1 + h2 + 2 log n

(6.1)

In the process of swapping, the h1 + h2 nodes that were right heavy, will lose their status as right heavy. While they remain heavy, they become left children for their parents hence they do not contribute for the potential of the output tree and this means a drop in potential by h1 + h2 . However, the swapping might have created new heavy nodes and let us say, the number of new heavy nodes created in the swapping process is h3 . First, observe that all these h3 new nodes are attached to the left most path of the output tree. Secondly, by Lemma 6.1, for each one of these right heavy nodes, its sibling in the left most path is a light node. However, the number of light nodes in the left most path of the output tree is less than or equal to log n by Lemma 6.1. Thus h3 log n . Consequently, the net change in the potential is h3 h1 h2 log n h1 h2 . The amortized cost = c + potential dierence 2 + h1 + h2 + 2 log n + log n h1 h2 = 3 log n + 2. Hence, the amortized cost of meld operation is O(log n) and this completes the proof.

2005 by Chapman & Hall/CRC

Skew Heaps

6-9

Since insert and delete-min are handled as special cases of meld operation, we conclude
THEOREM 6.3 The amortized cost complexity of all the Meldable Priority Queue operations is O(log n) where n is the number of nodes in skew heap or heaps involved in the operation.

6.4

Bibliographic Remarks

Skew Heaps were introduced by Sleator and Tarjan [7]. Leftist Trees have O(log n) worst case complexity for all the Meldable Priority Queue operations but they require heights of each subtree to be maintained as additional information at each node. Skew Heaps are simpler than Leftist Trees in the sense that no additional balancing information need to be maintained and the meld operation simply swaps the children of the right most path without any constraints and this results in a simpler code. The bound 3 log2 n + 2 for melding was signicantly improved to log n( here denotes the well-known golden ratio ( 5 + 1)/2 which is roughly 1.6) by using a dierent potential function and an intricate analysis in [6]. Recently, this bound was shown to be tight in [2]. Pairing Heap, introduced by Fredman et al. [5], is yet another self-adjusting heap structure and its relation to Skew Heaps is explored in Chapter 7 of this handbook.

References
[1] A. Aravind and C. Pandu Rangan, Symmetric Min-Max heaps: A simple data structure for double-ended priority queue, Information Processing Letters, 69:197-199, 1999. [2] B. Schoenmakers, A tight lower bound for top-down skew heaps, Information Processing Letters, 61:279-284, 1997. [3] S. Carlson, The Deap - A double ended heap to implement a double ended priority queue, Information Processing Letters, 26: 33-36, 1987. [4] S. Chang and M. Du, Diamond dequeue: A simple data structure for priority dequeues, Information Processing Letters, 46:231-237, 1993. [5] M. L. Fredman, R. Sedgewick, D. D. Sleator, and R. E. Tarjan, The pairing heap: A new form of self-adjusting heap, Algorithmica, 1:111-129, 1986. [6] A. Kaldewaij and B. Schoenmakers, The derivation of a tighter bound for top-down skew heaps, Information Processing Letters, 37:265-271, 1991. [7] D. D. Sleator and R. E. Tarjan, Self-adjusting heaps, SIAM J Comput., 15:52-69, 1986.

2005 by Chapman & Hall/CRC

7
Binomial, Fibonacci, and Pairing Heaps
7.1 7.2 7.3 7.4 7.5 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Binomial Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fibonacci Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pairing Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pseudocode Summaries of the Algorithms . . . . . .
Link and Insertion Algorithms Binomial Heap-Specic Algorithms Fibonacci Heap-Specic Algorithms Pairing Heap-Specic Algorithms

7-1 7-2 7-6 7-12 7-14

Michael L. Fredman
Rutgers University at New Brunswick

7.6

Related Developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7-17

7.1

Introduction

This chapter presents three algorithmically related data structures for implementing meldable priority queues: binomial heaps, Fibonacci heaps, and pairing heaps. What these three structures have in common is that (a) they are comprised of heap-ordered trees, (b) the comparisons performed to execute extractmin operations exclusively involve keys stored in the roots of trees, and (c) a common side eect of a comparison between two root keys is the linking of the respective roots: one tree becomes a new subtree joined to the other root. A tree is considered heap-ordered provided that each node contains one item, and the key of the item stored in the parent p(x) of a node x never exceeds the key of the item stored in x. Thus, when two roots get linked, the root storing the larger key becomes a child of the other root. By convention, a linking operation positions the new child of a node as its leftmost child. Figure 7.1 illustrates these notions. Of the three data structures, the binomial heap structure was the rst to be invented (Vuillemin [13]), designed to eciently support the operations insert, extractmin, delete, and meld. The binomial heap has been highly appreciated as an elegant and conceptually simple data structure, particularly given its ability to support the meld operation. The Fibonacci heap data structure (Fredman and Tarjan [6]) was inspired by and can be viewed as a generalization of the binomial heap structure. The raison d etre of the Fibonacci heap structure is its ability to eciently execute decrease-key operations. A decrease-key operation replaces the key of an item, specied by location, by a smaller value: e.g. decreasekey(P,knew ,H). (The arguments specify that the item is located in node P of the priority queue H, and that its new key value is knew .) Decrease-key operations are prevalent in many network optimization algorithms, including minimum spanning tree, and shortest path. The pairing heap data structure (Fredman, Sedgewick, Sleator, and Tarjan [5]) was

2005 by Chapman & Hall/CRC

7-2

Handbook of Data Structures and Applications

6
(a) before linking.

6
(b) after linking.

FIGURE 7.1: Two heap-ordered trees and the result of their linking.

devised as a self-adjusting analogue of the Fibonacci heap, and has proved to be more ecient in practice [11]. Binomial heaps and Fibonacci heaps are primarily of theoretical and historical interest. The pairing heap is the more ecient and versatile data structure from a practical standpoint. The following three sections describe the respective data structures. Summaries of the various algorithms in the form of pseudocode are provided in section 7.5.

7.2

Binomial Heaps

We begin with an informal overview. A single binomial heap structure consists of a forest of specially structured trees, referred to as binomial trees. The number of nodes in a binomial tree is always a power of two. Dened recursively, the binomial tree B0 consists of a single node. The binomial tree Bk , for k > 0, is obtained by linking two trees Bk1 together; one tree becomes the leftmost subtree of the other. In general Bk has 2k nodes. Figures 7.2(a-b) illustrate the recursion and show several trees in the series. An alternative and useful way to view the structure of Bk is depicted in Figure 7.2(c): Bk consists of a root and subtrees (in order from left to right) Bk1 , Bk2 , , B0 . The root of the binomial tree Bk has k children, and the tree is said to have rank k . We also observe that the height of Bk (maximum number of edges on any path directed away from the root) is k . The k descendants name binomial heap is inspired by the fact that the root of Bk has j at distance j .

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-3

(a) Recursion for binomial trees.

(b) Several binomial trees.

(c) An alternative recursion.

FIGURE 7.2: Binomial trees and their recursions. Because binomial trees have restricted sizes, a forest of trees is required to represent a priority queue of arbitrary size. A key observation, indeed a motivation for having tree sizes being powers of two, is that a priority queue of arbitrary size can be represented as a union of trees of distinct sizes. (In fact, the sizes of the constituent trees are uniquely determined and correspond to the powers of two that dene the binary expansion of n, the size of the priority queue.) Moreover, because the tree sizes are unique, the number of trees in the forest of a priority queue of size n is at most lg(n + 1). Thus, nding the minimum key in the priority queue, which clearly lies in the root of one of its constituent trees (due to the heap-order condition), requires searching among at most lg(n + 1) tree roots. Figure 7.3 gives an example of binomial heap. Now lets consider, from a high-level perspective, how the various heap operations are performed. As with leftist heaps (cf. Chapter 6), the various priority queue operations are to a large extent comprised of melding operations, and so we consider rst the melding of two heaps. The melding of two heaps proceeds as follows: (a) the trees of the respective forests are combined into a single forest, and then (b) consolidation takes place: pairs of trees having common rank are linked together until all remaining trees have distinct ranks. Figure 7.4 illustrates the process. An actual implementation mimics binary addition and proceeds in much the same was as merging two sorted lists in ascending order. We note that insertion is a special case of melding.

2005 by Chapman & Hall/CRC

7-4

Handbook of Data Structures and Applications

FIGURE 7.3: A binomial heap (showing placement of keys among forest nodes).

(a) Forests of two heaps Q1 and Q2 to be melded.

(b) Linkings among trees in the combined forest.

(c) Forest of meld(Q1 ,Q2 ).

FIGURE 7.4: Melding of two binomial heaps. The encircled objects reect trees of common rank being linked. (Ranks are shown as numerals positioned within triangles which in turn represent individual trees.) Once linking takes place, the resulting tree becomes eligible for participation in further linkings, as indicated by the arrows that identify these linking results with participants of other linkings.

The extractmin operation is performed in two stages. First, the minimum root , the node containing the minimum key in the data structure, is found by examining the tree roots of the appropriate forest, and this node is removed. Next, the forest consisting of the subtrees of this removed root, whose ranks are distinct (see Figure 7.2(c)) and thus viewable as

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-5

constituting a binomial heap, is melded with the forest consisting of the trees that remain from the original forest. Figure 7.5 illustrates the process.

(a) Initial forest.

(b) Forests to be melded.

FIGURE 7.5: Extractmin Operation: The location of the minimum key is indicated in (a). The two encircled sets of trees shown in (b) represent forests to be melded. The smaller trees were initially subtrees of the root of the tree referenced in (a).

Finally, we consider arbitrary deletion. We assume that the node containing the item to be deleted is specied. Proceeding up the path to the root of the tree containing , we permute the items among the nodes on this path, placing in the root the item x originally in , and shifting each of the other items down one position (away from the root) along the path. This is accomplished through a sequence of exchange operations that move x towards the root. The process is referred to as a sift-up operation. Upon reaching the root r, r is then removed from the forest as though an extractmin operation is underway. Observe that the re-positioning of items in the ancestors of serves to maintain the heap-order property among the remaining nodes of the forest. Figure 7.6 illustrates the re-positioning of the item being deleted to the root. This completes our high-level descriptions of the heap operations. For navigational purposes, each node contains a leftmost child pointer and a sibling pointer that points to the next sibling to its right. The children of a node are thus stored in the linked list dened by sibling pointers among these children, and the head of this list can be accessed by the leftmost child pointer of the parent. This provides the required access to the children of

2005 by Chapman & Hall/CRC

7-6

Handbook of Data Structures and Applications

root to be deleted

4 initial location of item to be deleted 5 7 9

(a) initial item placement.

3 4 7 9
(b) after movement to root.

FIGURE 7.6: Initial phase of deletion sift-up operation.

a node for the purpose of implementing extractmin operations. Note that when a node obtains a new child as a consequence of a linking operation, the new child is positioned at the head of its list of siblings. To facilitate arbitrary deletions, we need a third pointer in each node pointing to its parent. To facilitate access to the ranks of trees, we maintain in each node the number of children it has, and refer to this quantity as the node rank. Node ranks are readily maintained under linking operations; the node rank of the root gaining a child gets incremented. Figure 7.7 depicts these structural features. As seen in Figure 7.2(c), the ranks of the children of a node form a descending sequence in the childrens linked list. However, since the melding operation is implemented by accessing the tree roots in ascending rank order, when deleting a root we rst reverse the list order of its children before proceeding with the melding. Each of the priority queue operations requires in the worst case O(log n) time, where n is the size of the heap that results from the operation. This follows, for melding, from the fact that its execution time is proportional to the combined lengths of the forest lists being merged. For extractmin, this follows from the time for melding, along with the fact that a root node has only O(log n) children. For arbitrary deletion, the time required for the sift-up operation is bounded by an amount proportional to the height of the tree containing the item. Including the time required for extractmin, it follows that the time required for arbitrary deletion is O(log n). Detailed code for manipulating binomial heaps can be found in Weiss [14].

7.3

Fibonacci Heaps

Fibonacci heaps were specically designed to eciently support decrease-key operations. For this purpose, the binomial heap can be regarded as a natural starting point. Why? Consider the class of priority queue data structures that are implemented as forests of heapordered trees, as will be the case for Fibonacci heaps. One way to immediately execute a

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-7

(a) elds of a node.

(b) a node and its three children.

FIGURE 7.7: Structure associated with a binomial heap node. Figure (b) illustrates the positioning of pointers among a node and its three children.

decrease-key operation, remaining within the framework of heap-ordered forests, is to simply change the key of the specied data item and sever its link to its parent, inserting the severed subtree as a new tree in the forest. Figure 7.8 illustrates the process. (Observe that the link to the parent only needs to be cut if the new key value is smaller than the key in the parent node, violating heap-order.) Fibonacci heaps accomplish this without degrading the asymptotic eciency with which other priority queue operations can be supported. Observe that to accommodate node cuts, the list of children of a node needs to be doubly linked. Hence the nodes of a Fibonacci heap require two sibling pointers. Fibonacci heaps support ndmin, insertion, meld, and decrease-key operations in constant amortized time, and deletion operations in O(log n) amortized time. For many applications, the distinction between worst-case times versus amortized times are of little signicance. A Fibonacci heap consists of a forest of heap-ordered trees. As we shall see, Fibonacci heaps dier from binomial heaps in that there may be many trees in a forest of the same rank, and there is no constraint on the ordering of the trees in the forest list. The heap also includes a pointer to the tree root containing the minimum item, referred to as the min-pointer , that facilitates ndmin operations. Figure 7.9 provides an example of a Fibonacci heap and illustrates certain structural aspects. The impact of severing subtrees is clearly incompatible with the pristine structure of the binomial tree that is the hallmark of the binomial heap. Nevertheless, the tree structures that can appear in the Fibonacci heap data structure must suciently approximate binomial trees in order to satisfy the performance bounds we seek. The linking constraint imposed by binomial heaps, that trees being linked must have the same size, ensures that the number of children a node has (its rank), grows no faster than the logarithm of the size of the subtree rooted at the node. This rank versus subtree size relation is key to obtaining the O(log n) deletion time bound. Fibonacci heap manipulations are designed with this in mind. Fibonacci heaps utilize a protocol referred to as cascading cuts to enforce the required rank versus subtree size relation. Once a node has had two of its children removed as a result of cuts, s contribution to the rank of its parent is then considered suspect in terms of rank versus subtree size. The cascading cut protocol requires that the link to s parent

2005 by Chapman & Hall/CRC

7-8

Handbook of Data Structures and Applications

(a) Initial tree.

(b) Subtree to be severed.

(c) Resulting changes

FIGURE 7.8: Immediate decrease-key operation. The subtree severing (Figures (b) and (c)) is necessary only when k < j .

be cut, with the subtree rooted at then being inserted into the forest as a new tree. If s parent has, as a result, had a second child removed, then it in turn needs to be cut, and the cuts may thus cascade. Cascading cuts ensure that no non-root node has had more than one child removed subsequent to being linked to its parent. We keep track of the removal of children by marking a node if one of its children has been cut. A marked node that has another child removed is then subject to being cut from its parent. When a marked node becomes linked as a child to another node, or when it gets cut from its parent, it gets unmarked. Figure 7.10 illustrates the protocol of cascading cuts. Now the induced node cuts under the cascading cuts protocol, in contrast with those primary cuts immediately triggered by decrease-key operations, are bounded in number by the number of primary cuts. (This follows from consideration of a potential function dened to be the total number of marked nodes.) Therefore, the burden imposed by cascading cuts can be viewed as eectively only doubling the number of cuts taking place in the absence of the protocol. One can therefore expect that the performance asymptotics are not degraded as a consequence of proliferating cuts. As with binomial heaps, two trees in a Fibonacci heap can only be linked if they have equal rank. With the cascading cuts protocol in place,

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-9

(a) a heap

(b) elds of a node.

(c) pointers among a node and its three children.

FIGURE 7.9: A Fibonacci heap and associated structure. we claim that the required rank versus subtree size relation holds, a matter which we address next. Lets consider how small the subtree rooted at a node having rank k can be. Let be the mth child of from the right. At the time it was linked to , had at least m 1 other children (those currently to the right of were certainly present). Therefore had rank at least m 1 when it was linked to . Under the cascading cuts protocol, the rank of could have decreased by at most one after its linking to ; otherwise it would have been removed as a child. Therefore, the current rank of is at least m 2. We minimize the size of the subtree rooted at by minimizing the sizes (and ranks) of the subtrees rooted at

2005 by Chapman & Hall/CRC

7-10

Handbook of Data Structures and Applications

(a) Before decrease-key.

(b) After decrease-key.

FIGURE 7.10: Illustration of cascading cuts. In (b) the dashed lines reect cuts that have taken place, two nodes marked in (a) get unmarked, and a third node gets marked.

s children. Now let Fj denote the minimum possible size of the subtree rooted at a node of rank j , so that the size of the subtree rooted at is Fk . We conclude that (for k 2) Fk = Fk2 + Fk3 + + F0 + 1 +1
k

terms

where the nal term, 1, reects the contribution of to the subtree size. Clearly, F0 = 1 and F1 = 2. See Figure 7.11 for an illustration of this construction. Based on the preceding recurrence, it is readily shown that Fk is given by the (k + 2)th Fibonacci number (from whence the name Fibonacci heap was inspired). Moreover, since the Fibonacci numbers grow exponentially fast, we conclude that the rank of a node is indeed bounded by the logarithm of the size of the subtree rooted at the node. We proceed next to describe how the various operations are performed. Since we are not seeking worst-case bounds, there are economies to be exploited that could also be applied to obtain a variant of Binomial heaps. (In the absence of cuts, the individual trees generated by Fibonacci heap manipulations would all be binomial trees.) In particular we shall adopt a lazy approach to melding operations: the respective forests are simply combined by concatenating their tree lists and retaining the appropriate min-pointer. This requires only constant time. An item is deleted from a Fibonacci heap by deleting the node that originally contains it, in contrast with Binomial heaps. This is accomplished by (a) cutting the link to the nodes parent (as in decrease-key) if the node is not a tree root, and (b) appending the list of children of the node to the forest. Now if the deleted node happens to be referenced by the min-pointer, considerable work is required to restore the min-pointer the work previously deferred by the lazy approach to the operations. In the course of searching among the roots

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-11

FIGURE 7.11: Minimal tree of rank k . Node ranks are shown adjacent to each node.

of the forest to discover the new minimum key, we also link trees together in a consolidation process. Consolidation processes the trees in the forest, linking them in pairs until there are no longer two trees having the same rank, and then places the remaining trees in a new forest list (naturally extending the melding process employed by binomial heaps). This can be accomplished in time proportional to the number of trees in forest plus the maximum possible node rank. Let max-rank denote the maximum possible node rank. (The preceding discussion implies that max-rank = O(log heap-size).) Consolidation is initialized by setting up an array A of trees (initially empty) indexed by the range [0,max-rank]. A non-empty position A[d] of A contains a tree of rank d. The trees of the forest are then processed using the array A as follows. To process a tree T of rank d, we insert T into A[d] if this position of A is empty, completing the processing of T. However, if A[d] already contains a tree U, then T and U are linked together to form a tree W, and the processing continues as before, but with W in place of T, until eventually an empty location of A is accessed, completing the processing associated with T. After all of the trees have been processed in this manner, the array A is scanned, placing each of its stored trees in a new forest. Apart from the nal scanning step, the total time necessary to consolidate a forest is proportional to its number of trees, since the total number of tree pairings that can take place is bounded by this number (each pairing reduces by one the total number of trees present). The time required for the nal scanning step is given by max-rank = log(heap-size). The amortized timing analysis of Fibonacci heaps considers a potential function dened as the total number of trees in the forests of the various heaps being maintained. Ignoring consolidation, each operation takes constant actual time, apart from an amount of time proportional to the number of subtree cuts due to cascading (which, as noted above, is only constant in amortized terms). These cuts also contribute to the potential. The children of a deleted node increase the potential by O(log heap-size). Deletion of a minimum heap node additionally incurs the cost of consolidation. However, consolidation reduces our potential, so that the amortized time it requires is only O(log heap-size). We conclude therefore that all non-deletion operations require constant amortized time, and deletion requires O(log n) amortized time. An interesting and unresolved issue concerns the protocol of cascading cuts. How would the performance of Fibonacci heaps be aected by the absence of this protocol? Detailed code for manipulating Fibonacci heaps can found in Knuth [9].

2005 by Chapman & Hall/CRC

7-12

Handbook of Data Structures and Applications

7.4

Pairing Heaps

The pairing heap was designed to be a self-adjusting analogue of the Fibonacci heap, in much the same way that the skew heap is a self-adjusting analogue of the leftist heap (See Chapters 5 and 6). The only structure maintained in a pairing heap node, besides item information, consists of three pointers: leftmost child, and two sibling pointers. (The leftmost child of a node uses it left sibling pointer to point to its parent, to facilitate updating the leftmost child pointer its parent.) See Figure 7.12 for an illustration of pointer structure.

FIGURE 7.12: Pointers among a pairing heap node and its three children.

The are no cascading cuts only simple cuts for decrease-key and deletion operations. With the absence of parent pointers, decrease-key operations uniformly require a single cut (removal from the sibling list, in actuality), as there is no ecient way to check whether heap-order would otherwise be violated. Although there are several varieties of pairing heaps, our discussion presents the two-pass version (the simplest), for which a given heap consists of only a single tree. The minimum element is thus uniquely located, and melding requires only a single linking operation. Similarly, a decrease-key operation consists of a subtree cut followed by a linking operation. Extractmin is implemented by removing the tree root and then linking the roots subtrees in a manner described below. Other deletions involve (a) a subtree cut, (b) an extractmin operation on the cut subtree, and (c) linking the remnant of the cut subtree with the original root. The extractmin operation combines the subtrees of the root using a process referred to as two-pass pairing. Let x1 , , xk be the subtrees of the root in left-to-right order. The rst pass begins by linking x1 and x2 . Then x3 and x4 are linked, followed by x5 and x6 , etc., so that the odd positioned trees are linked with neighboring even positioned trees. Let y1 , , yh , h = k/2 , be the resulting trees, respecting left-to-right order. (If k is odd, then y k/2 is xk .) The second pass reduces these to a single tree with linkings that proceed from right-to-left. The rightmost pair of trees, yh and yh1 are linked rst, followed by the linking of yh2 with the result of the preceding linking etc., until nally we link y1 with the structure formed from the linkings of y2 , , yh . See Figure 7.13. Since two-pass pairing is not particularly intuitive, a few motivating remarks are oered. The rst pass is natural enough, and one might consider simply repeating the process on the remaining trees, until, after logarithmically many such passes, only one tree remains. Indeed, this is known as the multi-pass variation. Unfortunately, its behavior is less understood than that of the two-pass pairing variation. The second (right-to-left) pass is also quite natural. Let H be a binomial heap with exactly 2k items, so that it consists of a single tree. Now suppose that an extractmin followed by

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-13

...

(a) rst pass.

...

(b) second pass.

FIGURE 7.13: Two-pass Pairing. The encircled trees get linked. For example, in (b) trees A and B get linked, and the result then gets linked with the tree C, etc.

an insertion operation are executed. The linkings that take place among the subtrees of the deleted root (after the new node is linked with the rightmost of these subtrees) entail the right-to-left processing that characterizes the second pass. So why not simply rely upon a single right-to-left pass, and omit the rst? The reason, is that although the second pass preserves existing balance within the structure, it doesnt improve upon poorly balanced situations (manifested when most linkings take place between trees of disparate sizes). For example, using a single-right-to-left-pass version of a pairing heap to sort an increasing sequence of length n (n insertions followed by n extractmin operations), would result in an n2 sorting algorithm. (Each of the extractmin operations yields a tree of height 1 or less.) See Section 7.6, however, for an interesting twist. In actuality two-pass pairing was inspired [5] by consideration of splay trees (Chapter 12). If we consider the child, sibling representation that maps a forest of arbitrary trees into a binary tree, then two-pass pairing can be viewed as a splay operation on a search tree path with no bends [5]. The analysis for splay trees then carries over to provide an amortized analysis for pairing heaps. The asymptotic behavior of pairing heaps is an interesting and unresolved matter. Reecting upon the tree structures we have encountered in this chapter, if we view the binomial trees that comprise binomial heaps, their structure highly constrained, as likened to perfectly spherical masses of discrete diameter, then the trees that comprise Fibonacci heaps can be viewed as rather rounded masses, but rarely spherical, and of arbitrary (non-discrete) size. Applying this imagery to the trees that arise from pairing heap manipulations, we can aptly liken these trees to chunks of clay subject to repeated tearing and compaction, typically irregular in form. It is not obvious, therefore, that pairing heaps should be asymptotically ecient. On the other hand, since the pairing heap design dispenses with the rather complicated, carefully crafted constructs put in place primarily to facilitate proving the time bounds enjoyed by Fibonacci heaps, we can expect eciency gains at the level of elemen-

2005 by Chapman & Hall/CRC

7-14

Handbook of Data Structures and Applications

tary steps such as linking operations. From a practical standpoint the data structure is a success, as seen from the study of Moret and Shapiro [11]. Also, for those applications for which decrease-key operations are highly predominant, pairing heaps provably meet the optimal asymptotic bounds characteristic of Fibonacci heaps [3]. But despite this, as well as empirical evidence consistent with optimal eciency in general, pairing heaps are in fact asymptotically sub-optimal for certain operation sequences [3]. Although decrease-key requires only constant worst-case time, its execution can asymptotically degrade the eciency of extractmin operations, even though the eect is not observable in practice. On the positive side, it has been demonstrated [5] that under all circumstances the operations require only O(log n) amortized time. Additionally, Iacono [7] has shown that insertions require only constant amortized time; signicant for those applications that entail many more insertions than deletions. The reader may wonder whether some alternative to two-pass pairing might provably attain the asymptotic performance bounds satised by Fibonacci heaps. However, for information-theoretic reasons no such alternative exists. (In fact, this is how we know the two-pass version is sub-optimal.) A precise statement and proof of this result appears in Fredman [3]. Detailed code for manipulating pairing heaps can be found in Weiss [14].

7.5

Pseudocode Summaries of the Algorithms

This section provides pseudocode reecting the above algorithm descriptions. The procedures, link and insert, are suciently common with respect to all three data structures, that we present them rst, and then turn to those procedures having implementations specic to a particular data structure.

7.5.1

Link and Insertion Algorithms

Function link(x,y){ // x and y are tree roots. The operation makes the root with the // larger key the leftmost child of the other root. For binomial and // Fibonacci heaps, the rank field of the prevailing root is // incremented. Also, for Fibonacci heaps, the node becoming the child // gets unmarked if it happens to be originally marked. The function // returns a pointer to the node x or y that becomes the root. } Algorithm Insert(x,H){ //Inserts into heap H the item x I = Makeheap(x) // Creates a single item heap I containing the item x. H = Meld(H,I). }

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-15

7.5.2

Binomial Heap-Specic Algorithms

Function Meld(H,I){ // The forest lists of H and I are combined and consolidated -- trees // having common rank are linked together until only trees of distinct // ranks remain. (As described above, the process resembles binary // addition.) A pointer to the resulting list is returned. The // original lists are no longer available. } Function Extractmin(H){ //Returns the item containing the minimum key in the heap H. //The root node r containing this item is removed from H. r = find-minimum-root(H) if(r = null){return "Empty"} else{ x = item in r H = remove(H,r) // removes the tree rooted at r from the forest of H I = reverse(list of children of r) H = Meld(H,I) return x } } Algorithm Delete(x,H) //Removes from heap H the item in the node referenced by x. r = sift-up(x) // r is the root of the tree containing x. As described above, // sift-up moves the item information in x to r. H = remove(H,r) // removes the tree rooted at r from the forest of H I = reverse(list of children of r) H = Meld(H,I) }

7.5.3

Fibonacci Heap-Specic Algorithms

Function Findmin(H){ //Return the item in the node referenced by the min-pointer of H //(or "Empty" if applicable) } Function Meld(H,I){ // The forest lists of H and I are concatenated. The keys referenced // by the respective min-pointers of H and I are compared, and the // min-pointer referencing the larger key is discarded. The concatenation // result and the retained min-pointer are returned. The original // forest lists of H and I are no longer available. }

2005 by Chapman & Hall/CRC

7-16

Handbook of Data Structures and Applications

Algorithm Cascade-Cut(x,H){ //Used in decrease-key and deletion. Assumes parent(x) != null y = parent(x) cut(x,H) // The subtree rooted at x is removed from parent(x) and inserted into // the forest list of H. The mark-field of x is set to FALSE, and the // rank of parent(x) is decremented. x = y while(x is marked and parent(x) != null){ y = parent(x) cut(x,H) x = y } Set mark-field of x = TRUE } Algorithm Decrease-key(x,k,H){ key(x) = k if(key of min-pointer(H) > k){ min-pointer(H) = x} if(parent(x) != null and key(parent(x)) > k){ Cascade-Cut(x,H)} } Algorithm Delete(x,H){ If(parent(x) != null){ Cascade-Cut(x,H) forest-list(H) = concatenate(forest-list(H), leftmost-child(x)) H = remove(H,x) // removes the (single node) tree rooted at x from the forest of H } else{ forest-list(H) = concatenate(forest-list(H), leftmost-child(x)) H = remove(H,x) if(min-pointer(H) = x){ consolidate(H) // trees of common rank in the forest list of H are linked // together until only trees having distinct ranks remain. The // remaining trees then constitute the forest list of H. // min-pointer is reset to reference the root with minimum key. } } }

7.5.4

Pairing Heap-Specic Algorithms

Function Findmin(H){ // Returns the item in the node referenced by H (or "empty" if applicable) } Function Meld(H,I){ return link(H,I) }

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-17

Function Decrease-key(x,k,H){ If(x != H){ Cut(x) // The node x is removed from the child list in which it appears key(x) = k H = link(H,x) } else{ key(H) = k} } Function Two-Pass-Pairing(x){ // x is assumed to be a pointer to the first node of a list of tree // roots. The function executes two-pass pairing to combine the trees // into a single tree as described above, and returns a pointer to // the root of the resulting tree. } Algorithm Delete(x,H){ y = Two-Pass-Pairing(leftmost-child(x)) if(x = H){ H = y} else{ Cut(x) // The subtree rooted at x is removed from its list of siblings. H = link(H,y) } }

7.6

Related Developments

In this section we describe some results pertinent to the data structures of this chapter. First, we discuss a variation of the pairing heap, referred to as the skew-pairing heap. The skew-pairing heap appears as a form of missing link in the landscape occupied by pairing heaps and skew heaps (Chapter 6). Second, we discuss some adaptive properties of pairing heaps. Finally, we take note of soft heaps, a new shoot of activity emanating from the primordial binomial heap structure that has given rise to the topics of this chapter.
Skew-Pairing Heaps

There is a curious variation of the pairing heap which we refer to as a skew-pairing heap the name will become clear. Aside from the linking process used for combining subtrees in the extractmin operation, skew-pairing heaps are identical to two-pass pairing heaps. The skew-pairing heap extractmin linking process places greater emphasis on right-to-left linking than does the pairing heap, and proceeds as follows. First, a right-to-left linking of the subtrees that fall in odd numbered positions is executed. Let Hodd denote the result. Similarly, the subtrees in even numbered positions are linked in right-to-left order. Let Heven denote the result. Finally, we link the two trees, Hodd and Heven . Figure 7.14 illustrates the process. The skew-pairing heap enjoys O(log n) time bounds for the usual operations. Moreover, it has the following curious relationship to the skew heap. Suppose a nite sequence S of

2005 by Chapman & Hall/CRC

7-18

Handbook of Data Structures and Applications

(a) subtrees before linking.

(b) linkings.

FIGURE 7.14: Skew-pairing heap: linking of subtrees performed by extractmin. As described in Figure 7.13, encircled trees become linked.

meld and extractmin operations is executed (beginning with heaps of size 1) using (a) a skew heap and (b) a skew-pairing heap. Let Cs and Csp be the respective sets of comparisons between keys that actually get performed in the course of the respective executions (ignoring the order of the comparison executions). Then Csp Cs [4]. Moreover, if the sequence S terminates with the heap empty, then Csp = Cs . (This inspires the name skew-pairing.) The relationship between skew-pairing heaps and splay trees is also interesting. The child, sibling transformation, which for two-pass pairing heaps transforms the extractmin operation into a splay operation on a search tree path having no bends, when applied to the skew-pairing heap, transforms extractmin into a splay operation on a search tree path having a bend at each node. Thus, skew-pairing heaps and two-pass pairing heaps demarcate opposite ends of a spectrum.
Adaptive Properties of Pairing Heaps

Consider the problem of merging k sorted lists of respective lengths n1 , n2 , , nk , with ni = n. The standard merging strategy that performs lg k rounds of pairwise list merges requires n lg k time. However, a merge pattern based upon the binary Human tree, having minimal external path length for the weights n1 , n2 , , nk , is more ecient when the lengths ni are non-uniform, and provides a near optimal solution. Pairing heaps can be utilized to provide a rather dierent solution as follows. Treat each sorted list as a

2005 by Chapman & Hall/CRC

Binomial, Fibonacci, and Pairing Heaps

7-19

linearly structured pairing heap. Then (a) meld these k heaps together, and (b) repeatedly execute extractmin operations to retrieve the n items in their sorted order. The number of comparisons that take place is bounded by O(log n n 1 , , nk )

Since the above multinomial coecient represents the number of possible merge patterns, the information-theoretic bound implies that this result is optimal to within a constant factor. The pairing heap thus self-organizes the sorted list arrangement to approximate an optimal merge pattern. Iacono has derived a working-set theorem that quanties a similar adaptive property satised by pairing heaps. Given a sequence of insertion and extractmin operations initiated with an empty heap, at the time a given item x is deleted we can attribute to x a contribution bounded by O(log op(x)) to the total running time of the sequence, where op(x) is the number of heap operations that have taken place since x was inserted (see [8] for a slightly tighter estimate). Iacono has also shown that this same bound applies for skew and skew-pairing heaps [8]. Knuth [10] has observed, at least in qualitative terms, similar behavior for leftist heaps . Quoting Knuth: Leftist trees are in fact already obsolete, except for applications with a strong tendency towards last-in-rst-out behavior.
Soft Heaps

An interesting development (Chazelle [1]) that builds upon and extends binomial heaps in a dierent direction is a data structure referred to as a soft heap . The soft heap departs from the standard notion of priority queue by allowing for a type of error, referred to as corruption , which confers enhanced eciency. When an item becomes corrupted, its key value gets increased. Findmin returns the minimum current key, which might or might not be corrupted. The user has no control over which items become corrupted, this prerogative belonging to the data structure. But the user does control the overall amount of corruption in the following sense. The user species a parameter, 0 < 1/2, referred to as the error rate, that governs the behavior of the data structure as follows. The operations ndmin and deletion are supported in constant amortized time, and insertion is supported in O(log 1/ ) amortized time. Moreover, no more than an fraction of the items present in the heap are corrupted at any given time. To illustrate the concept, let x be an item returned by ndmin, from a soft heap of size n. Then there are no more than n items in the heap whose original keys are less than the original key of x. Soft heaps are rather subtle, and we wont attempt to discuss specics of their design. Soft heaps have been used to construct an optimal comparison-based minimum spanning tree algorithm (Pettie and Ramachandran [12]), although its actual running time has not been determined. Soft heaps have also been used to construct a comparison-based algorithm with known running time m(m, n) on a graph with n vertices and m edges (Chazelle [2]), where (m, n) is a functional inverse of the Ackermann function. Chazelle [1] has also observed that soft heaps can be used to implement median selection in linear time; a signicant departure from previous methods.

2005 by Chapman & Hall/CRC

7-20

Handbook of Data Structures and Applications

References
[1] B. Chazelle, The Soft Heap: An Approximate Priority Queue with Optimal Error Rate, Journal of the ACM , 47 (2000), 10121027. [2] B. Chazelle, A Faster Deterministic Algorithm for Minimum Spanning Trees, Journal of the ACM, 47 (2000), 10281047. [3] M. L. Fredman, On the Eciency of Pairing Heaps and Related Data Structures, Journal of the ACM , 46 (1999), 473501. [4] M. L. Fredman, A Priority Queue Transform, WAE: International Workshop on Algorithm Engineering LNCS 1668 (1999), 243257. [5] M. L. Fredman, R. Sedgewick, D. D. Sleator, and R. E. Tarjan, The Pairing Heap: A New Form of Self-adjusting Heap, Algorithmica , 1 (1986), 111129. [6] M. L. Fredman and R. E. Tarjan, Fibonacci Heaps and Their Uses in Improved Optimization Algorithms, Journal of the ACM , 34 (1987), 596615. [7] J. Iacono, New upper bounds for pairing heaps, Scandinavian Workshop on Algorithm Theory , LNCS 1851 (2000), 3542. [8] J. Iacono, Distribution sensitive data structures, Ph.D. Thesis, Rutgers University , 2001. [9] D. E. Knuth, The Stanford Graph Base , ACM Press, New York, N.Y., 1994. [10] D. E. Knuth, Sorting and Searching 2d ed., Addison-Wesley, Reading MA., 1998. [11] B. M. E. Moret and H. D. Shapiro, An Empirical Analysis of Algorithms for Constructing a Minimum Spanning Tree, Proceedings of the Second Workshop on Algorithms and Data Structures (1991), 400411. [12] S. Pettie and V. Ramachandran, An Optimal Minimum Spanning Tree Algorithm, Journal of the ACM 49 (2002), 1634. [13] J. Vuillemin, A Data Structure for Manipulating Priority Queues, Communications of the ACM , 21 (1978), 309314. [14] M. A. Weiss, Data Structures and Algorithms in C , 2d ed., Addison-Wesley, Reading MA., 1997.

2005 by Chapman & Hall/CRC

8
Double-Ended Priority Queues
8.1 8.2 8.3 Denition and an Application . . . . . . . . . . . . . . . . . . . . Symmetric Min-Max Heaps . . . . . . . . . . . . . . . . . . . . . . . Interval Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Inserting an Element Removing the Min Element Initializing an Interval Heap Complexity of Interval Heap Operations The Complementary Range Search Problem

8-1 8-2 8-5

8.4 8.5 8.6

Min-Max Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Inserting an Element Inserting an Element Dual Priority Queues Correspondence

8-11 8-16 8-19 8-21

Removing the Min Element Removing the Min Element Total Correspondence

Deaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Generic Methods for DEPQs . . . . . . . . . . . . . . . . . . . . .

Leaf

Sartaj Sahni
University of Florida

8.7

Meldable DEPQs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8.1

Denition and an Application

A double-ended priority queue (DEPQ) is a collection of zero or more elements. Each element has a priority or value. The operations performed on a double-ended priority queue are: 1. 2. 3. 4. getM in() ... return element with minimum priority getM ax() ... return element with maximum priority put(x) ... insert the element x into the DEPQ removeM in() ... remove an element with minimum priority and return this element 5. removeM ax() ... remove an element with maximum priority and return this element One application of a DEPQ is to the adaptation of quick sort, which has the the best expected run time of all known internal sorting methods, to external sorting. The basic idea in quick sort is to partition the elements to be sorted into three groups L, M , and R. The middle group M contains a single element called the pivot, all elements in the left group L are the pivot, and all elements in the right group R are the pivot. Following this partitioning, the left and right element groups are sorted recursively. In an external sort, we have more elements than can be held in the memory of our computer. The elements to be sorted are initially on a disk and the sorted sequence is to be left on the disk. When the internal quick sort method outlined above is extended to an

8-1

2005 by Chapman & Hall/CRC

8-2

Handbook of Data Structures and Applications

external quick sort, the middle group M is made as large as possible through the use of a DEPQ. The external quick sort strategy is: 1. Read in as many elements as will t into an internal DEPQ. The elements in the DEPQ will eventually be the middle group of elements. 2. Read in the remaining elements. If the next element is the smallest element in the DEPQ, output this next element as part of the left group. If the next element is the largest element in the DEPQ, output this next element as part of the right group. Otherwise, remove either the max or min element from the DEPQ (the choice may be made randomly or alternately); if the max element is removed, output it as part of the right group; otherwise, output the removed element as part of the left group; insert the newly input element into the DEPQ. 3. Output the elements in the DEPQ, in sorted order, as the middle group. 4. Sort the left and right groups recursively. In this chapter, we describe four implicit data structuressymmetric min-max heaps, interval heaps, min-max heaps, and deapsfor DEPQs. Also, we describe generic methods to obtain ecient DEPQ data structures from ecient data structures for single-ended priority queues (PQ).1

8.2

Symmetric Min-Max Heaps

Several simple and ecient implicit data structures have been proposed for the representation of a DEPQ [1, 2, 4, 5, 16, 17, 21]. All of these data structures are adaptations of the classical heap data structure (Chapter 2) for a PQ. Further, in all of these DEPQ data structures, getM ax and getM in take O(1) time and the remaining operations take O(log n) time each (n is the number of elements in the DEPQ). The symmetric min-max heap structure of Arvind and Pandu Rangan [1] is the simplest of the implicit data structures for DEPQs. Therefore, we describe this data structure rst. A symmetric min-max heap (SMMH) is a complete binary tree in which each node other than the root has exactly one element. The root of an SMMH is empty and the total number of nodes in the SMMH is n + 1, where n is the number of elements. Let x be any node of the SMMH. Let elements(x) be the elements in the subtree rooted at x but excluding the element (if any) in x. Assume that elements(x) = . x satises the following properties: 1. The left child of x has the minimum element in elements(x). 2. The right child of x (if any) has the maximum element in elements(x). Figure 8.1 shows an example SMMH that has 12 elements. When x denotes the node with 80, elements(x) = {6, 14, 30, 40}; the left child of x has the minimum element 6 in elements(x); and the right child of x has the maximum element 40 in elements(x). You may verify that every node x of this SMMH satises the stated properties. Since an SMMH is a complete binary tree, it is stored as an implicit data structure using the standard mapping of a complete binary tree into an array. When n = 1, the minimum and maximum elements are the same and are in the left child of the root of the SMMH.

1 A minPQ supports the operations getmin(), put(x), and removeM in() while a maxP Q supports the operations getM ax(), put(x), and removeM ax().

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-3

4 8 12 20 10 60 16 14 6 30

80 40

FIGURE 8.1: A symmetric min-max heap.

4 8 12 20 10 60 16 14 6 30

80 40 A

FIGURE 8.2: The SMMH of Figure 8.1 with a node added.

When n > 1, the minimum element is in the left child of the root and the maximum is in the right child of the root. So the getM in and getM ax operations take O(1) time. It is easy to see that an n + 1-node complete binary tree with an empty root and one element in every other node is an SMMH i the following are true: P1: For every node x that has a right sibling, the element in x is less than or equal to that in the right sibling of x. P2: For every node x that has a grandparent, the element in the left child of the grandparent is less than or equal to that in x. P3: For every node x that has a grandparent, the element in the right child of the grandparent is greater than or equal to that in x. Notice that if property P1 is satised at node x, then at most one of P 2 and P 3 may be violated at x. Using properties P1 through P3 we arrive at simple algorithms to insert and remove elements. These algorithms are simple adaptations of the corresponding algorithms for min and max heaps. Their complexity is O(log n). We describe only the insert operation. Suppose we wish to insert 2 into the SMMH of Figure 8.1. Since an SMMH is a complete binary tree, we must add a new node to the SMMH in the position shown in Figure 8.2; the new node is labeled A. In our example, A will denote an empty node. If the new element 2 is placed in node A, property P2 is violated as the left child of the grandparent of A has 6. So we move the 6 down to A and obtain Figure 8.3. Now we see if it is safe to insert the 2 into node A. We rst notice that property P1 cannot be violated, because the previous occupant of node A was greater than 2. Similarly, property P3 cannot be violated. Only P2 can be violated only when x = A. So we check P2 with x = A. We see that P2 is violated because the left child of the grandparent of A

2005 by Chapman & Hall/CRC

8-4

Handbook of Data Structures and Applications

4 8 12 20 10 60 16 14 A 30

80 40 6

FIGURE 8.3: The SMMH of Figure 8.2 with 6 moved down.

A 8 12 20 10 60 16 14 4 30

80 40 6

FIGURE 8.4: The SMMH of Figure 8.3 with 4 moved down.

2 8 12 20 10 60 16 14 4 30

80 40 6

FIGURE 8.5: The SMMH of Figure 8.4 with 2 inserted.

has the element 4. So we move the 4 down to A and obtain the conguration shown in Figure 8.4. For the conguration of Figure 8.4 we see that placing 2 into node A cannot violate property P1, because the previous occupant of node A was greater than 2. Also properties P2 and P3 cannot be violated, because node A has no grandparent. So we insert 2 into node A and obtain Figure 8.5. Let us now insert 50 into the SMMH of Figure 8.5. Since an SMMH is a complete binary tree, the new node must be positioned as in Figure 8.6. Since A has a right child of its parent, we rst check P1 at node A. If the new element (in this case 50) is smaller than that in the left sibling of A, we swap the new element and the element in the left sibling. In our case, no swap is done. Then we check P2 and P3.

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-5

2 8 12 20 10 60 16 14 4 30

80 40 6 A

FIGURE 8.6: The SMMH of Figure 8.5 with a node added.

2 8 12 20 10 60 16 14 4 30

80 A 6 40

FIGURE 8.7: The SMMH of Figure 8.6 with 40 moved down.

We see that placing 50 into A would violate P3. So the element 40 in the right child of the grandparent of A is moved down to node A. Figure 8.7 shows the resulting conguration. Placing 50 into node A of Figure 8.7 cannot create a P1 violation because the previous occupant of node A was smaller. Also, a P2 violation isnt possible either. So only P3 needs to be checked at A. Since there is no P3 violation at A, 50 is placed into A. The algorithm to remove either the min or max element is a similar adaptation of the trickle-down algorithm used to remove an element from a min or max heap.

8.3

Interval Heaps

The twin heaps of [21], the min-max pair heaps of [17], the interval heaps of [11, 16], and the diamond deques of [5] are virtually identical data structures. In each of these structures, an n element DEPQ is represented by a min heap with n/2 elements and a max heap with the remaining n/2 elements. The two heaps satisfy the property that each element in the min heap is the corresponding element (two elements correspond if they occupy the same position in their respective binary trees) in the max heap. When the number of elements in the DEPQ is odd, the min heap has one element (i.e., element n/2 ) that has no corresponding element in the max heap. In the twin heaps of [21], this is handled as a special case and one element is kept outside of the two heaps. In min-max pair heaps, interval heaps, and diamond deques, the case when n is odd is handled by requiring element n/2 of the min heap to be element n/4 of the max heap. In the twin heaps of [21], the min and max heaps are stored in two arrays min and max

2005 by Chapman & Hall/CRC

8-6

Handbook of Data Structures and Applications

2,30 3,17 4,12 4,10 5,11 5,9 3,11 4,7 8,8 5,10 7,9 4,15 6,15

FIGURE 8.8: An interval heap. using the standard array representation of a complete binary tree2 [15]. The correspondence property becomes min[i] max[i], 1 i n/2 . In the min-max pair heaps of [17] and the interval heaps of [16], the two heaps are stored in a single array minmax and we have minmax[i].min being the ith element of the min heap, 1 i n/2 and minmax[i].max being the ith element of the max heap, 1 i n/2 . In the diamond deque [5], the two heaps are mapped into a single array with the min heap occupying even positions (beginning with position 0) and the max heap occupying odd positions (beginning with position 1). Since this mapping is slightly more complex than the ones used in twin heaps, min-max pair heaps, and interval heaps, actual implementations of the diamond deque are expected to be slightly slower than implementations of the remaining three structures. Since the twin heaps of [21], the min-max pair heaps of [17], the interval heaps of [16], and the diamond deques of [5] are virtually identical data structures, we describe only one of theseinterval heapsin detail. An interval heap is a complete binary tree in which each node, except possibly the last one (the nodes of the complete binary tree are ordered using a level order traversal), contains two elements. Let the two elements in node P be a and b, where a b. We say that the node P represents the closed interval [a, b]. a is the left end point of the interval of P , and b is its right end point. The interval [c, d] is contained in the interval [a, b] i a c d b. In an interval heap, the intervals represented by the left and right children (if they exist) of each node P are contained in the interval represented by P . When the last node contains a single element c, then a c b, where [a, b] is the interval of the parent (if any) of the last node. Figure 8.8 shows an interval heap with 26 elements. You may verify that the intervals represented by the children of any node P are contained in the interval of P . The following facts are immediate: 1. The left end points of the node intervals dene a min heap, and the right end points dene a max heap. In case the number of elements is odd, the last node has a single element which may be regarded as a member of either the min or max heap. Figure 8.9 shows the min and max heaps dened by the interval heap of Figure 8.8. 2. When the root has two elements, the left end point of the root is the minimum element in the interval heap and the right end point is the maximum. When

2 In a full binary tree, every non-empty level has the maximum number of nodes possible for that level. Number the nodes in a full binary tree 1, 2, beginning with the root level and within a level from left to right. The nodes numbered 1 through n dene the unique complete binary tree that has n nodes.

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

2 3 4 4 5 5 3 4 8 5 7 4 6 10 12 11 9 17 11 7 8 10 9 30 15 15

8-7

(a) min heap

(b) max heap

FIGURE 8.9: Min and max heaps embedded in Figure 8.8.

2,30 3,17 4,12 4,10 5,11 5,9 3,11 4,7 8,8 5,10 7,9 A 4,15 6,15

FIGURE 8.10: Interval heap of Figure 8.8 after one node is added.

the root has only one element, the interval heap contains just one element. This element is both the minimum and maximum element. 3. An interval heap can be represented compactly by mapping into an array as is done for ordinary heaps. However, now, each array position must have space for two elements. 4. The height of an interval heap with n elements is (log n).

8.3.1

Inserting an Element

Suppose we are to insert an element into the interval heap of Figure 8.8. Since this heap currently has an even number of elements, the heap following the insertion will have an additional node A as is shown in Figure 8.10. The interval for the parent of the new node A is [6, 15]. Therefore, if the new element is between 6 and 15, the new element may be inserted into node A. When the new element is less than the left end point 6 of the parent interval, the new element is inserted into the min heap embedded in the interval heap. This insertion is done using the min heap insertion procedure starting at node A. When the new element is greater than the right end point 15 of the parent interval, the new element is inserted into the max heap embedded in the interval heap. This insertion is done using the max heap insertion procedure starting at node A. If we are to insert the element 10 into the interval heap of Figure 8.8, this element is put into the node A shown in Figure 8.10. To insert the element 3, we follow a path from node A towards the root, moving left end points down until we either pass the root or reach a node whose left end point is 3. The new element is inserted into the node that now has

2005 by Chapman & Hall/CRC

8-8

Handbook of Data Structures and Applications

2,30 3,17 4,12 4,10 5,11 5,9 3,11 4,7 8,8 5,10 7,9 6 3,15 4,15

FIGURE 8.11: The interval heap of Figure 8.8 with 3 inserted.

2,40 3,17 4,12 4,10 5,11 5,9 3,11 4,7 8,8 5,10 7,9 15 4,30 6,15

FIGURE 8.12: The interval heap of Figure 8.8 with 40 inserted.

no left end point. Figure 8.11 shows the resulting interval heap. To insert the element 40 into the interval heap of Figure 8.8, we follow a path from node A (see Figure 8.10) towards the root, moving right end points down until we either pass the root or reach a node whose right end point is 40. The new element is inserted into the node that now has no right end point. Figure 8.12 shows the resulting interval heap. Now, suppose we wish to insert an element into the interval heap of Figure 8.12. Since this interval heap has an odd number of elements, the insertion of the new element does not increase the number of nodes. The insertion procedure is the same as for the case when we initially have an even number of elements. Let A denote the last node in the heap. If the new element lies within the interval [6, 15] of the parent of A, then the new element is inserted into node A (the new element becomes the left end point of A if it is less than the element currently in A). If the new element is less than the left end point 6 of the parent of A, then the new element is inserted into the embedded min heap; otherwise, the new element is inserted into the embedded max heap. Figure 8.13 shows the result of inserting the element 32 into the interval heap of Figure 8.12.

8.3.2

Removing the Min Element

The removal of the minimum element is handled as several cases: 1. When the interval heap is empty, the removeM in operation fails. 2. When the interval heap has only one element, this element is the element to be returned. We leave behind an empty interval heap. 3. When there is more than one element, the left end point of the root is to be

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

2,40 B 3,17 4,12 4,10 5,11 5,9 3,11 C 4,7 D 8,8 5,10 7,9 4,32 6,30 15,15

8-9

FIGURE 8.13: The interval heap of Figure 8.12 with 32 inserted.

3,40 3,17 4,12 4,10 5,11 5,9 4,15 7,11 8,8 5,10 7,9 15 4,32 6,30

FIGURE 8.14: The interval heap of Figure 8.13 with minimum element removed.

returned. This point is removed from the root. If the root is the last node of the interval heap, nothing more is to be done. When the last node is not the root node, we remove the left point p from the last node. If this causes the last node to become empty, the last node is no longer part of the heap. The point p removed from the last node is reinserted into the embedded min heap by beginning at the root. As we move down, it may be necessary to swap the current p with the right end point r of the node being examined to ensure that p r. The reinsertion is done using the same strategy as used to reinsert into an ordinary heap. Let us remove the minimum element from the interval heap of Figure 8.13. First, the element 2 is removed from the root. Next, the left end point 15 is removed from the last node and we begin the reinsertion procedure at the root. The smaller of the min heap elements that are the children of the root is 3. Since this element is smaller than 15, we move the 3 into the root (the 3 becomes the left end point of the root) and position ourselves at the left child B of the root. Since, 15 17 we do not swap the right end point of B with the current p = 15. The smaller of the left end points of the children of B is 3. The 3 is moved from node C into node B as its left end point and we position ourselves at node C . Since p = 15 > 11, we swap the two and 15 becomes the right end point of node C . The smaller of left end points of C s children is 4. Since this is smaller than the current p = 11, it is moved into node C as this nodes left end point. We now position ourselves at node D. First, we swap p = 11 and Ds right end point. Now, since D has no children, the current p = 7 is inserted into node D as Ds left end point. Figure 8.14 shows the result. The max element may removed using an analogous procedure.

2005 by Chapman & Hall/CRC

8-10

Handbook of Data Structures and Applications

8.3.3

Initializing an Interval Heap

Interval heaps may be initialized using a strategy similar to that used to initialize ordinary heapswork your way from the heap bottom to the root ensuring that each subtree is an interval heap. For each subtree, rst order the elements in the root; then reinsert the left end point of this subtrees root using the reinsertion strategy used for the removeM in operation, then reinsert the right end point of this subtrees root using the strategy used for the removeM ax operation.

8.3.4

Complexity of Interval Heap Operations

The operations isEmpty (), size(), getM in(), and getM ax() take O(1) time each; put(x), removeM in(), and removeM ax() take O(log n) each; and initializing an n element interval heap takes O(n) time.

8.3.5

The Complementary Range Search Problem

In the complementary range search problem, we have a dynamic collection (i.e., points are added and removed from the collection as time goes on) of one-dimensional points (i.e., points have only an x-coordinate associated with them) and we are to answer queries of the form: what are the points outside of the interval [a, b]? For example, if the point collection is 3, 4, 5, 6, 8, 12, the points outside the range [5, 7] are 3, 4, 8, 12. When an interval heap is used to represent the point collection, a new point can be inserted or an old one removed in O(log n) time, where n is the number of points in the collection. Note that given the location of an arbitrary element in an interval heap, this element can be removed from the interval heap in O(log n) time using an algorithm similar to that used to remove an arbitrary element from a heap. The complementary range query can be answered in (k ) time, where k is the number of points outside the range [a, b]. This is done using the following recursive procedure: 1. If the interval tree is empty, return. 2. If the root interval is contained in [a, b], then all points are in the range (therefore, there are no points to report), return. 3. Report the end points of the root interval that are not in the range [a, b]. 4. Recursively search the left subtree of the root for additional points that are not in the range [a, b]. 5. Recursively search the right subtree of the root for additional points that are not in the range [a, b]. 6. return Let us try this procedure on the interval heap of Figure 8.13. The query interval is [4, 32]. We start at the root. Since the root interval is not contained in the query interval, we reach step 3 of the procedure. Whenever step 3 is reached, we are assured that at least one of the end points of the root interval is outside the query interval. Therefore, each time step 3 is reached, at least one point is reported. In our example, both points 2 and 40 are outside the query interval and so are reported. We then search the left and right subtrees of the root for additional points. When the left subtree is searched, we again determine that the root interval is not contained in the query interval. This time only one of the root interval points (i.e., 3) is to be outside the query range. This point is reported and we proceed to search the left and right subtrees of B for additional points outside the query range. Since

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-11

the interval of the left child of B is contained in the query range, the left subtree of B contains no points outside the query range. We do not explore the left subtree of B further. When the right subtree of B is searched, we report the left end point 3 of node C and proceed to search the left and right subtrees of C . Since the intervals of the roots of each of these subtrees is contained in the query interval, these subtrees are not explored further. Finally, we examine the root of the right subtree of the overall tree root, that is the node with interval [4, 32]. Since this nodes interval is contained in the query interval, the right subtree of the overall tree is not searched further. The complexity of the above six step procedure is (number of interval heap nodes visited). The nodes visited in the preceding example are the root and its two children, node B and its two children, and node C and its two children. Thus, 7 nodes are visited and a total of 4 points are reported. We show that the total number of interval heap nodes visited is at most 3k + 1, where k is the number of points reported. If a visited node reports one or two points, give the node a count of one. If a visited node reports no points, give it a count of zero and add one to the count of its parent (unless the node is the root and so has no parent). The number of nodes with a nonzero count is at most k . Since no node has a count more than 3, the sum of the counts is at most 3k . Accounting for the possibility that the root reports no point, we see that the number of nodes visited is at most 3k + 1. Therefore, the complexity of the search is (k ). This complexity is asymptotically optimal because every algorithm that reports k points must spend at least (1) time per reported point. In our example search, the root gets a count of 2 (1 because it is visited and reports at least one point and another 1 because its right child is visited but reports no point), node B gets a count of 2 (1 because it is visited and reports at least one point and another 1 because its left child is visited but reports no point), and node C gets a count of 3 (1 because it is visited and reports at least one point and another 2 because its left and right children are visited and neither reports a point). The count for each of the remaining nodes in the interval heap is 0.

8.4

Min-Max Heaps

In the min-max heap structure [2], all n DEPQ elements are stored in an n-node complete binary tree with alternating levels being min levels and max levels (Figure 8.15, nodes at max levels are shaded). The root level of a min-max heap is a min level. Nodes on a min level are called min nodes while those on a max level are max nodes. Every min (max) node has the property that its value is the smallest (largest) value in the subtree of which it is the root. Since 5 is in a min node of Figure 8.15, it is the smallest value in its subtree. Also, since 30 and 26 are in max nodes, these are the largest values in the subtrees of which they are the root. The following observations are a direct consequence of the denition of a min-max heap. 1. When n = 0, there is no min nor max element. 2. When n = 1, the element in the root is both the min and the max element. 3. When n > 1, the element in the root is the min element; the max element is one of the up to two children of the root. From these observations, it follows that getM in() and getM ax() can be done in O(1) time each.

2005 by Chapman & Hall/CRC

8-12

Handbook of Data Structures and Applications

5 30 10 12 14 25 16 21 22 20 26 18

min max min max

FIGURE 8.15: A 12-element min-max heap.

5 30 10 12 14 25 16 21 22 20 26 18

min max min max

FIGURE 8.16: A 13-node complete binary tree.

8.4.1

Inserting an Element

When inserting an element newElement into a min-max heap that has n elements, we go from a complete binary tree that has n nodes to one that has n +1 nodes. So, for example, an insertion into the 12-element min-max heap of Figure 8.15 results in the 13-node complete binary tree of Figure 8.16. When n = 0, the insertion simply creates a min-max heap that has a single node that contains the new element. Assume that n > 0 and let the element in the parent, parentN ode, of the new node j be parentElement. If newElement is placed in the new node j , the min- and max-node property can be violated only for nodes on the path from the root to parentN ode. So, the insertion of an element need only be concerned with ensuring that nodes on this path satisfy the required min- and max-node property. There are three cases to consider. 1. parentElement = newElement In this case, we may place newElement into node j . With such a placement, the min- and max-node properties of all nodes on the path from the root to parentN ode are satised. 2. parentN ode > newElement

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-13

2 30 10 12 14 25 16 21 22 5 20 26 18

min max min max

FIGURE 8.17: Min-max heap of Figure 8.15 following insertion of 2.

If parentN ode is a min node, we get a min-node violation. When a min-node violation occurs, we know that all max nodes on the path from the root to parentN ode are greater than newElement. So, a min-node violation may be xed by using the trickle-up process used to insert into a min heap; this trickleup process involves only the min nodes on the path from the root to parentN ode. For example, suppose that we are to insert newElement = 2 into the min-max heap of Figure 8.15. The min nodes on the path from the root to parentN ode have values 5 and 20. The 20 and the 5 move down on the path and the 2 trickles up to the root node. Figure 8.17 shows the result. When newElement = 15, only the 20 moves down and the sequence of min nodes on the path from the root to j have values 5, 15, 20. The case when parentN ode is a max node is similar. 3. parentN ode < newElement When parentN ode is a min node, we conclude that all min nodes on the path from the root to parentN ode are smaller than newElement. So, we need be concerned only with the max nodes (if any) on the path from the root to parentN ode. A trickle-up process is used to correctly position newElement with respect to the elements in the max nodes of this path. For the example of Figure 8.16, there is only one max node on the path to parentN ode. This max node has the element 26. If newElement > 26, the 26 moves down to j and newElement trickles up to the former position of 26 (Figure 8.18 shows the case when newElement = 32). If newElement < 26, newElement is placed in j . The case when parentN ode is a max node is similar. Since the height of a min-max heap is (log n) and a trickle-up examines a single element at at most every other level of the min-max heap, an insert can be done in O(log n) time.

8.4.2

Removing the Min Element

When n = 0, there is no min element to remove. When n = 1, the min-max heap becomes empty following the removal of the min element, which is in the root. So assume that n > 1. Following the removal of the min element, which is in the root, we need to go from an n-element complete binary tree to an (n 1)-element complete binary tree. This causes the element in position n of the min-max heap array to drop out of the min-max

2005 by Chapman & Hall/CRC

8-14

Handbook of Data Structures and Applications

5 30 10 12 14 25 16 21 22 20 26 32 18

min max min max

FIGURE 8.18: The min-max heap of Figure 8.15 following the insertion of 32.

22 30 10 12 14 25 16 21 20 26 18

min max min max

FIGURE 8.19: Situation following a remove min from Figure 8.15.

heap. Figure 8.17 shows the situation following the removal of the min element, 5, from the min-max heap of Figure 8.15. In addition to the 5, which was the min element and which has been removed from the min-max heap, the 22 that was in position n = 12 of the min-max heap array has dropped out of the min-max heap. To get the dropped-out element 22 back into the min-max heap, we perform a trickle-down operation that begins at the root of the min-max heap. The trickle-down operation for min-max heaps is similar to that for min and max heaps. The root is to contain the smallest element. To ensure this, we determine the smallest element in a child or grandchild of the root. If 22 is the smallest of these children and grandchildren, the 22 is placed in the root. If not, the smallest of the children and grandchildren is moved to the root; the trickle-down process is continued from the position vacated by the just moved smallest element. In our example, examining the children and grandchildren of the root of Figure 8.19, we determine that the smallest of these is 10. Since 10 < 22, the 10 moves to the root and the 22 trickles down (Figure 8.20). A special case arises when this trickle down of the 22 by 2 levels causes the 22 to trickle past a smaller element (in our example, we trickle past a larger element 30). When this special case arises, we simply exchange the 22 and the smaller element being trickled past. The trickle-down process applied at the vacant node

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-15

10 30 22 12 14 25 16 21 20 26 18

min max min max

FIGURE 8.20: Situation following one iteration of the trickle-down process. of Figure 8.20 results in the 22 being placed into the vacant node. Suppose that droppedElement is the element dropped from minmaxHeap[n] when a remove min is done from an n-element min-max heap. The following describes the trickledown process used to reinsert the dropped element. 1. The root has no children. In this case droppedElement is inserted into the root and the trickle down terminates. 2. The root has at least one child. Now the smallest key in the min-max heap is in one of the children or grandchildren of the root. We determine which of these nodes has the smallest key. Let this be node k . The following possibilities need to be considered: (a) droppedElement minmaxHeap[k ]. droppedElement may be inserted into the root, as there is no smaller element in the heap. The trickle down terminates. (b) droppedElement > minmaxHeap[k ] and k is a child of the root. Since k is a max node, it has no descendants larger than minmaxHeap[k ]. Hence, node k has no descendants larger than droppedElement. So, the minmaxHeap[k ] may be moved to the root, and droppedElement placed into node k . The trickle down terminates. (c) droppedElement > minmaxHeap[k ] and k is a grandchild of the root. minmaxHeap[k ] is moved to the root. Let p be the parent of k . If droppedElement > minmaxHeap[p], then minmaxHeap[p] and droppedElement are interchanged. This interchange ensures that the max node p contains the largest key in the subheap with root p. The trickle down continues with k as the root of a min-max (sub) heap into which an element is to be inserted. The complexity of the remove-min operation is readily seen to be O(log n). The removemax operation is similar to the remove-min operation, and min-max heaps may be initialized in (n) time using an algorithm similar to that used to initialize min and max heaps [15].

2005 by Chapman & Hall/CRC

8-16

Handbook of Data Structures and Applications

3 7 9 15 11 5 12 10 18

20 16

FIGURE 8.21: An 11-element deap.

8.5

Deaps

The deap structure of [4] is similar to the two-heap structures of [5, 16, 17, 21]. At the conceptual level, we have a min heap and a max heap. However, the distribution of elements between the two is not n/2 and n/2 . Rather, we begin with an (n + 1)-node complete binary tree. Its left subtree is the min heap and its right subtree is the max heap (Figure 8.21, max-heap nodes are shaded). The correspondence property for deaps is slightly more complex than that for the two-heap structures of [5, 16, 17, 21]. A deap is a complete binary tree that is either empty or satises the following conditions: 1. The root is empty. 2. The left subtree is a min heap and the right subtree is a max heap. 3. Correspondence property. Suppose that the right subtree is not empty. For every node x in the left subtree, dene its corresponding node y in the right subtree to be the node in the same position as x. In case such a y doesnt exist, let y be the corresponding node for the parent of x. The element in x is the element in y . For the example complete binary tree of Figure 8.21, the corresponding nodes for the nodes with 3, 7, 5, 9, 15, 11, and 12, respectively, have 20, 18, 16, 10, 18, 16, and 16. Notice that every node y in the max heap has a unique corresponding node x in the min heap. The correspondence property for max-heap nodes is that the element in y be the element in x. When the correspondence property is satised for all nodes in the min heap, this property is also satised for all nodes in the max heap. We see that when n = 0, there is no min or max element, when n = 1, the root of the min heap has both the min and the max element, and when n > 1, the root of the min heap is the min element and the root of the max heap is the max element. So, both getM in() and getM ax() may be implemented to work in O(1) time.

8.5.1

Inserting an Element

When an element is inserted into an n-element deap, we go form a complete binary tree that has n + 1 nodes to one that has n + 2 nodes. So, the shape of the new deap is well dened. Following an insertion, our 11-element deap of Figure 8.21 has the shape shown in Figure 8.22. The new node is node j and its corresponding node is node i.

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-17

3 7 9 15 11 i 5 12 10 18

20 16 j

FIGURE 8.22: Shape of a 12-element deap.

2 3 9 7 11 5 12 10 18 15

20 16

FIGURE 8.23: Deap of Figure 8.21 with 2 inserted.

To insert newElement, temporarily place newElement into the new node j and check the correspondence property for node j . If the property isnt satised, swap newElement and the element in its corresponding node; use a trickle-up process to correctly position newElement in the heap for the corresponding node i. If the correspondence property is satised, do not swap newElement; instead use a trickle-up process to correctly place newElement in the heap that contains node j . Consider the insertion of newElement = 2 into Figure 8.22. The element in the corresponding node i is 15. Since the correspondence property isnt satised, we swap 2 and 15. Node j now contains 15 and this swap is guaranteed to preserve the max-heap properties of the right subtree of the complete binary tree. To correctly position the 2 in the left subtree, we use the standard min-heap trickle-up process beginning at node i. This results in the conguration of Figure 8.23. To insert newElement = 19 into the deap of Figure 8.22, we check the correspondence property between 15 and 19. The property is satised. So, we use the trickle-up process for max heaps to correctly position newElement in the max heap. Figure 8.24 shows the result. Since the height of a deap is (log n), the time to insert into a deap is O(log n).

2005 by Chapman & Hall/CRC

8-18

Handbook of Data Structures and Applications

3 7 9 15 11 5 12 10 19 18

20 16

FIGURE 8.24: Deap of Figure 8.21 with 19 inserted.

5 7 9 15 11 10 12 18

20 16

FIGURE 8.25: Deap of Figure 8.21 following a remove min operation.

8.5.2

Removing the Min Element

Assume that n > 0. The min element is in the root of the min heap. Following its removal, the deap size reduces to n 1 and the element in position n + 1 of the deap array is dropped from the deap. In the case of our example of Figure 8.21, the min element 3 is removed and the 10 is dropped. To reinsert the dropped element, we rst trickle the vacancy in the root of the min heap down to a leaf of the min heap. This is similar to a standard min-heap trickle down with as the reinsert element. For our example, this trickle down causes 5 and 11 to, respectively, move to their parent nodes. Then, the dropped element 10 is inserted using a trickle-up process beginning at the vacant leaf of the min heap. The resulting deap is shown in Figure 8.25. Since a removeM in requires a trickle-down pass followed by a trickle-up pass and since the height of a deap is (log n), the time for a removeM in is O(log n). A removeM ax is similar. Also, we may initialize a deap in (n) time using an algorithm similar to that used to initialize a min or max heap [15].

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-19

2 5 6 7 4 5 6 4

7 2

min heap

max heap

FIGURE 8.26: Dual heap.

8.6
8.6.1

Generic Methods for DEPQs

Dual Priority Queues

General methods [8] exist to arrive at ecient DEPQ data structures from single-ended priority queue data structures that also provide an ecient implementation of the remove (theN ode) operation (this operation removes the node theN ode from the PQ). The simplest of these methods, dual structure method, maintains both a min PQ (called minP Q) and a max PQ (called maxP Q) of all the DEPQ elements together with correspondence pointers between the nodes of the min PQ and the max PQ that contain the same element. Figure 8.26 shows a dual heap structure for the elements 6, 7, 2, 5, 4. Correspondence pointers are shown as double-headed arrows. Although Figure 8.26 shows each element stored in both the min and the max heap, it is necessary to store each element in only one of the two heaps. The minimum element is at the root of the min heap and the maximum element is at the root of the max heap. To insert an element x, we insert x into both the min and the max heaps and then set up correspondence pointers between the locations of x in the min and max heaps. To remove the minimum element, we do a removeM in from the min heap and a remove(theN ode), where theN ode is the corresponding node for the removed element, from the max heap. The maximum element is removed in an analogous way.

8.6.2

Total Correspondence

The notion of total correspondence borrows heavily from the ideas used in a twin heap [21]. In the twin heap data structure n elements are stored in a min heap using an array minHeap[1 : n] and n other elements are stored in a max heap using the array maxHeap[1 : n]. The min and max heaps satisfy the inequality minHeap[i] maxHeap[i], 1 i n. In this way, we can represent a DEPQ with 2n elements. When we must represent a DEPQ with an odd number of elements, one element is stored in a buer, and the remaining elements are divided equally between the arrays minHeap and maxHeap. In total correspondence, we remove the positional requirement in the relationship between pairs of elements in the min heap and max heap. The requirement becomes: for each element a in minP Q there is a distinct element b in maxP Q such that a b and vice versa. (a, b) is a corresponding pair of elements. Figure 8.27(a) shows a twin heap with 11

2005 by Chapman & Hall/CRC

8-20
2 10

Handbook of Data Structures and Applications

2 10

buffer = 9 (a) Twin heap

buffer = 9 (b) Total correspondence heap

FIGURE 8.27: Twin heap and total correspondence heap.

elements and Figure 8.27(b) shows a total correspondence heap. The broken arrows connect corresponding pairs of elements. In a twin heap the corresponding pairs (minHeap[i], maxHeap[i]) are implicit, whereas in a total correspondence heap these pairs are represented using explicit pointers. In a total correspondence DEPQ, the number of nodes is either n or n 1. The space requirement is half that needed by the dual priority queue representation. The time required is also reduced. For example, if we do a sequence of inserts, every other one simply puts the element in the buer. The remaining inserts put one element in maxP Q and one in minP Q. So, on average, an insert takes time comparable to an insert in either maxP Q or minP Q. Recall that when dual priority queues are used the insert time is the sum of the times to insert into maxP Q and minP Q. Note also that the size of maxP Q and minP Q together is half that of a dual priority queue. If we assume that the complexity of the insert operation for priority queues as well as 2 remove(theN ode) operations is no more than that of the delete max or min operation (this is true for all known priority queue structures other than weight biased leftist trees [6]), then the complexity of removeM ax and removeM in for total correspondence DEPQs is the same as for the removeM ax and removeM in operation of the underlying priority queue data structure. Using the notion of total correspondence, we trivially obtain ecient DEPQ structures starting with any of the known priority queue structures (other than weight biased leftist trees [6]). The removeM ax and removeM in operations can generally be programmed to run faster than suggested by our generic algorithms. This is because, for example, a removeM ax() and put(x) into a max priority queue can often be done faster as a single operation changeM ax(x). Similarly a remove(theN ode) and put(x) can be programmed as a change (theN ode, x) operation.

8.6.3

Leaf Correspondence

In leaf correspondence DEPQs, for every leaf element a in minP Q, there is a distinct element b in maxP Q such that a b and for every leaf element c in maxP Q there is a distinct element d in minP Q such that d c. Figure 8.28 shows a leaf correspondence heap. Ecient leaf correspondence DEPQs may be constructed easily from PQs that satisfy the following requirements [8]: (a) The PQ supports the operation remove(Q, p) eciently. (b) When an element is inserted into the PQ, no nonleaf node becomes a leaf node (except possibly the node for the newly inserted item).

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

8-21

10

buffer = 9
FIGURE 8.28: Leaf correspondence heap. (c) When an element is deleted (using remove, removeM ax or removeM in) from the PQ, no nonleaf node (except possibly the parent of the deleted node) becomes a leaf node. Some of the PQ structures that satisfy these requirements are height-biased leftist trees (Chapter 5) [9, 15, 20], pairing heaps (Chapter 7) [12, 19], and Fibonacci heaps [13] (Chapter 7). Requirements (b) and (c) are not satised, for example, by ordinary heaps and the FMPQ structure of [3]. Although heaps and Brodals FMPQ structure do not satisfy the requirements of our generic approach to build a leaf correspondence DEPQ structure from a priority queue, we can nonetheless arrive at leaf correspondence heaps and leaf correspondence FMPQs using a customized approach.

8.7

Meldable DEPQs

A meldable DEPQ (MDEPQ) is a DEPQ that, in addition to the DEPQ operations listed above, includes the operation meld(x, y ) ... meld the DEPQs x and y into a single DEPQ The result of melding the double-ended priority queues x and y is a single double-ended priority queue that contains all elements of x and y . The meld operation is destructive in that following the meld, x and y do not remain as independent DEPQs. To meld two DEPQs in less than linear time, it is essential that the DEPQs be represented using explicit pointers (rather than implicit ones as in the array representation of a heap) as otherwise a linear number of elements need to be moved from their initial to their nal locations. Olariu et al. [17] have shown that when the min-max pair heap is represented in such a way, an n element DEPQ may be melded with a k element one (k n) in O(log(n/k ) log k ) time. When k = n, this is O(log2 n). Hasham and Sack [14] have shown that the complexity of melding two min-max heaps of size n and k , respectively, is (n + k ). Brodal [3] has developed an MDEPQ implementation that allows one to nd the min and max elements, insert an element, and meld two priority queues in O(1) time. The time needed to delete the minimum or maximum element is O(log n). Although the asymptotic complexity provided by this data structure are the best one can hope for [3], the data structure has practical limitations. First, each element is represented twice using

2005 by Chapman & Hall/CRC

8-22

Handbook of Data Structures and Applications

a total of 16 elds per element. Second, even though the delete operations have O(log n) complexity, the constant factors are very high and the data structure will not perform well unless nd, insert, and meld are the primary operations. Cho and Sahni [7] have shown that leftist trees [9, 15, 20] may be adapted to obtain a simple representation for MDEPQs in which meld takes logarithmic time and the remaining operations have the same asymptotic complexity as when any of the aforementioned DEPQ representations is used. Chong and Sahni [8] study MDEPQs based on pairing heaps [12, 19], Binomial and Fibonacci heaps [13], and FMPQ [3]. Since leftist heaps, pairing heaps, Binomial and Fibonacci heaps, and FMPQs are meldable priority queues that also support the remove(theN ode) operation, the MDEPQs of [7, 8] use the generic methods of Section 8.6 to construct an MDEPQ data structure from the corresponding MPQ (meldable PQ) structure. It is interesting to note that if we use the FMPQ structure of [3] as the base MPQ structure, we obtain a total correspondence MDEPQ structure in which removeM ax and removeM in take logarithmic time, and the remaining operations take constant time. This adaptation is superior to the dual priority queue adaptation proposed in [3] because the space requirements are almost half. Additionally, the total correspondence adaptation is faster. Although Brodals FMPQ structure does not satisfy the requirements of the generic approach to build a leaf correspondence MDEPQ structure from a priority queue, we can nonetheless arrive at leaf correspondence FMPQs using a customized approach.

Acknowledgment
This work was supported, in part, by the National Science Foundation under grant CCR9912395.

References
[1] A. Arvind and C. Pandu Rangan, Symmetric min-max heap: A simpler data structure for double-ended priority queue, Information Processing Letters, 69, 1999, 197-199. [2] M. Atkinson, J. Sack, N. Santoro, and T. Strothotte, Min-max heaps and generalized priority queues, Communications of the ACM, 29, 996-1000, 1986. [3] G. Brodal, Fast meldable priority queues, Workshop on Algorithms and Data Structures, 1995. [4] S. Carlsson, The deap A double ended heap to implement double ended priority queues, Information Processing Letters, 26, 33-36, 1987. [5] S. Chang and M. Du, Diamond deque: A simple data structure for priority deques, Information Processing Letters, 46, 231-237, 1993. [6] S. Cho and S. Sahni, Weight biased leftist trees and modied skip lists, ACM Jr. on Experimental Algorithms, Article 2, 1998. [7] S. Cho and S. Sahni, Mergeable double ended priority queue, International Journal on Foundation of Computer Sciences, 10, 1, 1999, 1-18. [8] K. Chong and S. Sahni, Correspondence based data structures for double ended priority queues, ACM Jr. on Experimental Algorithmics, Volume 5, 2000, Article 2, 22 pages. [9] C. Crane, Linear lists and priority queues as balanced binary trees, Technical Report CS-72-259, Computer Science Department, Stanford University, [10] Y. Ding and M. Weiss, The Relaxed Min-Max Heap: A Mergeable Double-Ended Priority Queue, Acta Informatica, 30, 215-231, 1993. [11] Y. Ding and M. Weiss, On the Complexity of Building an Interval Heap, Information Processing Letters, 50, 143-144, 1994.

2005 by Chapman & Hall/CRC

Double-Ended Priority Queues

[12] M. L. Fredman, R. Sedgewick, D. D. Sleator, and R. E. Tarjan, The paring heap : A new form of self-adjusting heap, Algorithmica, 1:111-129, 1986. [13] M. Fredman and R. Tarjan, Fibonacci heaps and their uses in improved network optimization algorithms, JACM, 34:3, 596-615, 1987. [14] A. Hasham and J. Sack, Bounds for min-max heaps, BIT, 27, 315-323, 1987. [15] E. Horowitz, S. Sahni, D. Mehta, Fundamentals of Data Structures in C++, Computer Science Press, NY, 1995. [16] J. van Leeuwen and D. Wood, Interval heaps, The Computer Journal, 36, 3, 209-216, 1993. [17] S. Olariu, C. Overstreet, and Z. Wen, A mergeable double-ended priority queue, The Computer Journal, 34, 5, 423-427, 1991. [18] D. Sleator and R. Tarjan, Self-adjusting binary search trees, JACM, 32:3, 652-686, 1985. [19] J. T. Stasko and J. S. Vitter, Pairing heaps : Experiments and Analysis, Communication of the ACM, 30:3, 234-249, 1987. [20] R. Tarjan, Data structures and network algorithms, SIAM, Philadelphia, PA, 1983. [21] J. Williams, Algorithm 232, Communications of the ACM, 7, 347-348, 1964.

8-23

2005 by Chapman & Hall/CRC

III
Dictionary Structures
9 Hash Tables Pat Morin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9-1

Introduction Hash Tables for Integer Keys Other Developments

Random Probing

Historical Notes

10 Balanced Binary Search Trees Arne Andersson, Rolf Fagerberg, and Kim S. Larsen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1
Introduction Basic Denitions Generic Discussion of Balancing Classic Balancing Schemes Rebalancing a Tree to Perfect Balance Schemes with no Balance Information Low Height Schemes Relaxed Balance

11 Finger Search Trees

Gerth Stlting Brodal . . . . . . . . . . . . . . . . . . . . . . . . . . . 11-1

Finger Searching Dynamic Finger Search Trees domized Finger Search Trees Applications

Level Linked (2,4)-Trees

Ran-

12 Splay Trees

Sanjeev Saxena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12-1

Introduction Splay Trees Analysis Optimality of Splay Trees Linking and Cutting Trees Case Study: Application to Network Flows Implementation Without Linking and Cutting Trees FIFO: Dynamic Tree Implementation Variants of Splay Trees and Top-Down Splaying

13 Randomized Dictionary Structures

C. Pandu Rangan . . . . . . . . . . . . . . 13-1

Introduction Preliminaries Skip Lists Structural Properties of Skip Lists Dictionary Operations Analysis of Dictionary Operations Randomized Binary Search Trees Bibliographic Remarks

14 Trees with Minimum Weighted Path Length

Wojciech Rytter . . . . 14-1

Introduction Human Trees Height Limited Human Trees Optimal Binary Search Trees Optimal Alphabetic Tree Problem Optimal Lopsided Trees Parallel Algorithms

15 B Trees

Donghui Zhang . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15-1

Introduction Discussions

The Disk-Based Environment

The B-tree

The B+-tree

Further

2005 by Chapman & Hall/CRC

9
Hash Tables
9.1 9.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hash Tables for Integer Keys . . . . . . . . . . . . . . . . . . . . .
Hashing by Division Hashing by Multiplication Universal Hashing Static Perfect Hashing Dynamic Perfect Hashing

9-1 9-2

9.3

Random Probing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hashing with Chaining Hashing with Open Addressing Linear Probing Quadratic Probing Double Hashing Brents Method Multiple-Choice Hashing Asymmetric Hashing LCFS Hashing Robin-Hood Hashing Cuckoo Hashing

9-8

Pat Morin
Carleton University

9.4 9.5

Historical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Other Developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9-15 9-15

9.1

Introduction

A set abstract data type (set ADT) is an abstract data type that maintains a set S under the following three operations: 1. Insert(x): Add the key x to the set. 2. Delete(x): Remove the key x from the set. 3. Search(x): Determine if x is contained in the set, and if so, return a pointer to x. One of the most practical and widely used methods of implementing the set ADT is with hash tables. Note that the three set ADT operations can easily be implemented to run in O(log n) time per operation using balanced binary search trees (See Chapter 10). If we assume that the input data are integers in the set U = {0, . . . , u 1} then they can even be implemented to run in sub-logarithmic time using data structures for integer searching (Chapter 39). However, these data structures actually do more than the three basic operations we require. In particular if we search for an element x that is not present in S then these data structures can report the smallest item in S that is larger than x (the successor of x) and/or the largest item in S that is smaller than x (the predecessor of x). Hash tables do away with this extra functionality of nding predecessors and successors and only perform exact searches. If we search for an element x in a hash table and x is not present then the only information we obtain is that x / S . By dropping this extra functionality hash tables can give better performance bounds. Indeed, any reasonable hash table implementation performs each of the three set ADT operations in O(1) expected time.

9-1

2005 by Chapman & Hall/CRC

9-2

Handbook of Data Structures and Applications

The main idea behind all hash table implementations discussed in this chapter is to store a set of n = |S | elements in an array (the hash table) A of length m n. In doing this, we require a function that maps any element x to an array location. This function is called a hash function h and the value h(x) is called the hash value of x. That is, the element x gets stored at the array location A[h(x)]. The occupancy of a hash table is the ratio = n/m of stored elements to the length of A. The study of hash tables follows two very dierent lines. Many implementations of hash tables are based on the integer universe assumption : All elements stored in the hash table come from the universe U = {0, . . . , u 1}. In this case, the goal is to design a hash function h : U {0, . . . , m 1} so that for each i {0, . . . , m 1}, the number of elements x S such that h(x) = i is as small as possible. Ideally, the hash function h would be such that each element of S is mapped to a unique value in {0, . . . , m 1}. Most of the hash functions designed under the integer universe assumption are number-theoretic constructions. Several of these are described in Section 9.2. Historically, the integer universe assumption seems to have been justied by the fact that any data item in a computer is represented as a sequence of bits that can be interpreted as a binary number. However, many complicated data items require a large (or variable) number of bits to represent and this make u the size of the universe very large. In many applications u is much larger than the largest integer that can t into a single word of computer memory. In this case, the computations performed in number-theoretic hash functions become inecient. This motivates the second major line of research into hash tables. This research work is based on the random probing assumption random probing assumption: Each element x that is inserted into a hash table is a black box that comes with an innite random probe sequence x0 , x1 , x2 , . . . where each of the xi is independently and uniformly distributed in {0, . . . , m 1}. Hash table implementations based on the random probing assumption are described in Section 9.3. Both the integer universe assumption and the random probing assumption have their place in practice. When there is an easily computing mapping of data elements onto machine word sized integers then hash tables for integer universes are the method of choice. When such a mapping is not so easy to compute (variable length strings are an example) it might be better to use the bits of the input items to build a good pseudorandom sequence and use this sequence as the probe sequence for some random probing data structure. To guarantee good performance, many hash table implementations require that the occupancy be a constant strictly less than 1. Since the number of elements in a hash table changes over time, this requires that the array A be resized periodically. This is easily done, without increasing the amortized cost of hash table operations by choosing three constants 0 < 1 < 2 < 3 < 1 so that, whenever n/m is not the interval (1 , 3 ) the array A is resized so that its size is n/2 . A simple amortization argument (Chapter 1) shows that the amortized cost of this resizing is O(1) per update (Insert/Delete) operation.

9.2

Hash Tables for Integer Keys

In this section we consider hash tables under the integer universe assumption, in which the key values x come from the universe U = {0, . . . , u 1}. A hash function h is a function whose domain is U and whose range is the set {0, . . . , m 1}, m u. A hash function h is said to be a perfect hash function for a set S U if, for every x S , h(x) is unique. A perfect hash function h for S is minimal if m = |S |, i.e., h is a bijection between S and {0, . . . , m 1}. Obviously a minimal perfect hash function for S is desirable since it

2005 by Chapman & Hall/CRC

Hash Tables

9-3

allows us to store all the elements of S in a single array of length n. Unfortunately, perfect hash functions are rare, even for m much larger than n. If each element of S is mapped independently and uniformly to a random element of {0, . . . , m 1} then the birthday paradox (See, for example, Feller [27]) states that, if m is much less than n2 then there will almost surely exist two elements of S that have the same hash value. We begin our discussion with two commonly used hashing schemes that are heuristic in nature. That is, we can not make any non-trivial statements about the performance of these schemes when storing an arbitrary set S . We then discuss several schemes that have provably good performance.

9.2.1

Hashing by Division

In hashing by division, we use the hash function h(x) = x mod m . To use this hash function in a data structure, we maintain an array A[0], . . . , A[m 1] where each element of this array is a pointer to the head of a linked list (Chapter 2). The linked list Li pointed to by the array element A[i] contains all the elements x such that h(x) = i. This technique of maintaining an array of lists is called hashing with chaining . In such a hash table, inserting an element x takes O(1) time; we compute i = h(x) and append (or prepend) x to the list Li . However, searching for and/or deleting an element x is not so easy. We have to compute i = h(x) and then traverse the list Li until we either nd x or reach the end of the list. The cost of this is proportional to the length of Li . Obviously, if our set S consists of the elements 0, m, 2m, 3m, . . . , nm then all elements are stored in the list L0 and searches and deletions take linear time. However, one hopes that such pathological cases do not occur in practice. For example, if the elements of S are uniformly and independently distributed in U and u is a multiple of m then the expected size of any list Li is only n/m. In this case, searches and deletions take O(1 + ) expected time. To help avoid pathological cases, the choice of m is important. In particular, m a power of 2 is usually avoided since, in a binary computer, taking the remainder modulo a power of 2 means simply discarding some high-order bits. Taking m to be a prime not too close to a power of 2 is recommended [37].

9.2.2

Hashing by Multiplication

The implementation of a hash table using hashing by multiplication is exactly the same as that of hashing by division except that the hash function h(x) = mxA mod m is used. Here A is a real-valued constant whose choice we discuss below. The advantage of the multiplication method is that the value of m is not critical. We can take m to be a power of 2, which makes it convenient for use on binary computers. Although any value of A gives a hash function, some values of A are better than others. (Setting A = 0 is clearly not a good idea.) Knuth [37] suggests using the golden ratio for A, i.e., setting A = ( 5 1)/2 = 0.6180339887 . . .

2005 by Chapman & Hall/CRC

9-4

Handbook of Data Structures and Applications

This choice of A is motivated by a theorem, rst conjectured by Oderfeld and later proven by Swierczkowski [59]. This theorem states that the sequence mA mod m, 2mA mod m, 3mA mod m, . . . , nmA mod m partitions the interval (0, m) into n + 1 intervals having only three distinct lengths. Furthermore, the next element (n + 1)mA mod m in the sequence is always contained in one of the largest intervals.1 Of course, no matter what value of A we select, the pigeonhole principle implies that for u nm then there will always exist some hash value i and some S U of size n such that h(x) = i for all x S . In other words, we can always nd a set S all of whose elements get stored in the same list Li . Thus, the worst case of hashing by multiplication is as bad as hashing by division.

9.2.3

Universal Hashing

The argument used at the end of the previous section applies equally well to any hash function h. That is, if the table size m is much smaller than the universe size u then for any hash function there is some large (of size at least u/m ) subset of U that has the same hash value. To get around this diculty we need a collection of hash functions from which we can choose one that works well for S . Even better would be a collection of hash functions such that, for any given S , most of the hash functions work well for S . Then we could simply pick one of the functions at random and have a good chance of it working well. Let H be a collection of hash functions, i.e., functions from U onto {0, . . . , m 1}. We say that H is universal if, for each x, y U the number of h H such that h(x) = h(y ) is at most |H|/m. Consider any S U of size n and suppose we choose a random hash function h from a universal collection of hash functions. Consider some value x U . The probability that any key y S has the same hash value as x is only 1/m. Therefore, the expected number of keys in S , not equal to x, that have the same hash value as x is only n h (x ) = (n 1)/m if x S n/m if x /S

Therefore, if we store S in a hash table using the hash function h then the expected time to search for, or delete, x is O(1 + ). From the preceding discussion, it seems that a universal collection of hash functions from which we could quickly select one at random would be very handy indeed. With such a collection at our disposal we get an implementation of the set ADT that has O(1) insertion time and O(1) expected search and deletion time. Carter and Wegman [8] describe three dierent collections of universal hash functions. If the universe size u is a prime number2 then H = {hk1 ,k2 ,m (x) = ((k1 x + k2 ) mod u)) mod m : 1 k1 < u, 0 k2 < u}

1 In fact, any irrational number has this property [57]. The golden ratio is especially good because it is not too close to a whole number. 2 This is not a major restriction since, for any u > 1, there always exists a prime number in the set {u, u + 1, . . . , 2u}. Thus we can enforce this assumption by increasing the value of u by a constant factor.

2005 by Chapman & Hall/CRC

Hash Tables

9-5

is a collection of universal hash functions. Clearly, choosing a function uniformly at random from H can be done easily by choosing two random values k1 {1, . . . , u 1} and k2 {0, . . . , u 1}. Thus, we have an implementation of the set ADT with O(1) expected time per operation.

9.2.4

Static Perfect Hashing

The result of Carter and Wegman on universal hashing is very strong, and from a practical point of view, it is probably the strongest result most people will ever need. The only thing that could be improved about their result is to make it deterministic, so that the running times of all operations are O(1) worst-case. Unfortunately, this is not possible, as shown by Dietzfelbinger et al. [23]. Since there is no hope of getting O(1) worst-case time for all three set ADT operations, the next best thing would be to have searches that take O(1) worst-case time. In this section we describe the method of Fredman, Koml os and Szemer edi [28]. This is a static data structure that takes as input a set S U and builds a data structure of size O(n) that can test if an element x is in S in O(1) worst-case time. Like the universal hash functions from the previous section, this method also requires that u be a prime number. This scheme uses hash functions of the form hk,m (x) = (kx mod u)) mod m .3 Let Bk,m (S, i) be the number of elements x S such that hk,m (x) = i, i.e., the number of elements of S that have hash value i when using the hash function hk,m . The function Bk,m gives complete information about the distribution of hash values of S . The main lemma used by Fredman et al. is that, if we choose k U uniformly at random then
m 1

E
i=0

Bk,m (S, i) 2

<

n2 . m

(9.1)

There are two important special cases of this result. In the sparse case we take m = n2 /, for some constant 0 < < 1. In this case, the expectation in (9.1) is less than . Therefore, by Markovs inequality, the probability that this sum is greater than or equal to 1 is at most . But, since this sum is a non-negative integer, then with probability at least 1 it must be equal to 0. In other words, with probability at least 1 , Bk,m (S, i) 1 for all 0 i m 1, i.e., the hash function hk,m is perfect for S . Of course this implies that we can nd a perfect hash function very quickly by trying a small number of random elements k U and testing if they result in perfect hash functions. (The expected number of elements that we will have to try is only 1/(1 ).) Thus, if we are willing to use quadratic space then we can perform searches in O(1) worst-case time. In the dense case we assume that m is close to n and discover that, for many values of k , the hash values are distributed fairly evenly among the set 1, . . . , m. More precisely, if we use a table of size m = n, then
m 1

E
i=0

Bk,m (S, i)2 3n .

3 Actually,

it turns out that any universal hash function also works in the FKS scheme [16, Section 11.5].

2005 by Chapman & Hall/CRC

9-6

Handbook of Data Structures and Applications

By Markovs inequality this means that

m 1

Pr
i=0

Bk,m (S, i)2 3n/

1 .

(9.2)

Again, we can quickly nd a value of k satisfying (9.2) by testing a few randomly chosen values of k . These two properties are enough to build a two-level data structure that uses linear space and executes searches in worst-case constant time. We call the following data structure the FKS- data structure, after its inventors Fredman, Koml os and Szemer edi. At the top level, the data structure consists of an array A[0], . . . , A[m 1] where m = n. The elements of this array are pointers to other arrays A0 , . . . , Am1 , respectively. To decide what will be stored in these other arrays, we build a hash function hk,m that satises the conditions of (9.2). This gives us the top-level hash function hk,m (x) = (kx mod u) mod m. Each element x S gets stored in the array pointed to by A[hk,m (x)]. What remains is to describe how we use the arrays A0 , . . . , Am1 . Let Si denote the set of elements x S such that hk,m (s) = i. The elements of Si will be stored in Ai . The size of Si is ni = Bk,m (S, i). To store the elements of Si we set the size of Ai to mi = ni 2 / = Bk,n (S, i)2 /. Observe that, by (9.2), all the Ai s take up a total space of m 1 O(n), i.e., i=0 mi = O(n). Furthermore, by trying a few randomly selected integers we can quickly nd a value ki such that the hash function hki ,mi is perfect for Si . Therefore, we store the element x Si at position Ai [hki ,mi (x)] and x is the unique element stored at that location. With this scheme we can search for any value x U by computing two hash values i = hk,m (x) and j = hki ,mi (x) and checking if x is stored in Ai [j ]. Building the array A and computing the values of n0 , . . . , nm1 takes O(n) expected time since for a given value k we can easily do this in O(n) time and the expected number of values of k that we must try before nding one that satises (9.2) is O(1). Similarly, building each subarray Ai takes O(ni 2 ) expected time, resulting in an overall expected running time of O(n). Thus, for any constant 0 < < 1, an FKS- data structure can be constructed in O(n) expected time and this data structure can execute a search for any x U in O(1) worst-case time.

9.2.5

Dynamic Perfect Hashing

The FKS- data structure is nice in that it allows for searches in O(1) time, in the worst case. Unfortunately, it is only static; it does not support insertions or deletions of elements. In this section we describe a result of Dietzfelbinger et al. [23] that shows how the FKS- data structure can be made dynamic with some judicious use of partial rebuilding (Chapter 10). The main idea behind the scheme is simple: be lazy at both the upper and lower levels of the FKS- data structure. That is, rebuild parts of the data structure only when things go wrong. At the top level, we relax the condition that the size m of the upper array A is exactly n and allow A to have size anywhere between n and 2n. Similarly, at the lower level we allow the array Ai to have a size mi anywhere between ni 2 / and 2ni 2 /. Periodically, we will perform a global rebuilding operation in which we remove all n elements from the hash table. Some elements which have previously been marked as deleted will be discarded, thereby reducing the value of n. We put the remaining elements in a list, and recompute a whole new FKS-(/2) data structure for the elements in the list. This data structure is identical to the standard FKS-(/2) data structure except that, at the top level we use an array of size m = 2n.

2005 by Chapman & Hall/CRC

Hash Tables

9-7

Searching in this data structure is exactly the same as for the static data structure. To search for an element x we compute i = hk,m (x) and j = hki ,mi (x) and look for x at location Ai [j ]. Thus, searches take O(1) worst-case time. Deleting in this data structure is done in the laziest manner possible. To delete an element we only search for it and then mark it as deleted. We will use the convention that this type of deletion does not change the value of n since it does not change the number of elements actually stored in the data structure. While doing this, we also keep track of the number of elements that are marked as deleted. When this number exceeds n/2 we perform a global rebuilding operation. The global rebuilding operation takes O(n) expected time, but only occurs during one out of every n/2 deletions. Therefore, the amortized cost of this operation is O(1) per deletion. The most complicated part of the data structure is the insertion algorithm and its analysis. To insert a key x we know, because of how the search algorithm works, that we must ultimately store x at location Ai [j ] where i = hk,m (x) and j = hki ,mi (x). However, several things can go wrong during the insertion of x: 1. The value of n increases by 1, so it may be that n now exceeds m. In this case we perform a global rebuilding operation and we are done. 1 2 2. We compute i = hk,m (x) and discover that m i=0 ni > 3n/. In this case, the hash function hk,m used at the top level is no longer any good since it is producing an overall hash table that is too large. In this case we perform a global rebuilding operation and we are done. 3. We compute i = hk,m (x) and discover that, since the value of ni just increased by one, ni 2 / > mi . In this case, the array Ai is too small to guarantee that we can quickly nd a perfect hash function. To handle this, we copy the elements of Ai into a list L and allocate a new array Ai with the new size mi = 2ni 2 /. We then nd a new value ki such that hki ,mi is a perfect hash function for the elements of L and we are done. 4. The array location Ai [j ] is already occupied by some other element y . But in this case, we know that Ai is large enough to hold all the elements (otherwise we would already be done after Case 3), but the value ki being used in the hash function hki ,mi is the wrong one since it doesnt give a perfect hash function for Si . Therefore we simply try new values for ki until we nd a nd a value ki that yields a perfect hash function and we are done. If none of the preceding 4 cases occurs then we can simply place x at location Ai [j ] and we are done. Handling Case 1 takes O(n) expected time since it involves a global rebuild of the entire data structure. However, Case 1 only happens during one out of every (n) insertions, so the amortized cost of all occurrences of Case 1 is only O(1) per insertion. Handling Case 2 also takes O(n) expected time. The question is: How often does Case 2 occur? To answer this question, consider the phase that occurs between two consecutive occurrences of Case 1. During this phase, the data structure holds at most m distinct elements. Call this set of elements S . With probability at least (1 ) the hash function hk,m selected at the beginning of the phase satises (9.2) so that Case 2 never occurs during the phase. Similarly, the probability that Case 2 occurs exactly once during the phase is at most (1 ). In general, the probability that Case 2 occurs exactly i times during a phase is at most i (1 ). Thus, the expected cost of handling all occurrences of Case 2

2005 by Chapman & Hall/CRC

9-8

Handbook of Data Structures and Applications

during the entire phase is at most

i (1 )i O(n) = O(n) .
i=0

But since a phase involves (n) insertions this means that the amortized expected cost of handling Case 2 is O(1) per insertion. Next we analyze the total cost of handling Case 3. Dene a subphase as the period of time between two global rebuilding operations triggered either as a result of a deletion, Case 1 or Case 2. We will show that the total cost of handling all occurrences of Case 3 during a subphase is O(n) and since a subphase takes (n) time anyway this does not contribute to the cost of a subphase by more than a constant factor. When Case 3 occurs at the array Ai it takes O(mi ) time. However, while handling Case 3, mi increases by a constant factor, so the total cost of handling Case 3 for Ai is dominated by the value of mi at the end m 1 of the subphase. But we maintain the invariant that i=0 mi = O(n) during the entire subphase. Thus, handling all occurrences of Case 3 during a subphase only requires O(n) time. Finally, we consider the cost of handling Case 4. For a particular array Ai , consider the subsubphase between which two occurrences of Case 3 cause Ai to be rebuilt or a global rebuilding operation takes place. During this subsubphase the number of distinct elements that occupy Ai is at most mi . Therefore, with probability at least 1 any randomly chosen value of ki U is a perfect hash function for this set. Just as in the analysis of Case 2, this implies that the expected cost of handling all occurrences of Case 3 at Ai during a subsubphase is only O(mi ). Since a subsubphase ends with rebuilding all of Ai or a global rebuilding, at a cost of (mi ) all the occurrences of Case 4 during a subsubphase do not contribute to the expected cost of the subsubphase by more than a constant factor. To summarize, we have shown that the expected cost of handling all occurrences of Case 4 is only a constant factor times the cost of handling all occurrences of Case 3. The cost of handling all occurrences of Case 3 is no more than a constant factor times the expected cost of all global rebuilds. The cost of handling all the global rebuilds that occur as a result of Case 2 is no more than a constant factor times the cost of handling all occurrences of global rebuilds that occur as a consequence of Case 1. And nally, the cost of all global rebuilds that occur as a result of Case 1 or of deletions is O(n) for a sequence of n update operations. Therefore, the total expected cost of n update operation is O(n).

9.3

Random Probing

Next we consider hash table implementations under the random probing assumption: Each element x stored in the hash table comes with a random sequence x0 , x1 , x2 , . . . where each of the xi is independently and uniformly distributed in {1, . . . , m}.4 We begin with a discussion of the two basic paradigms: hashing with chaining and open addressing. Both these paradigms attempt to store the key x at array position A[x0 ]. The dierence between these two algorithms is their collision resolution strategy , i.e., what the algorithms do when a user inserts the key value x but array position A[x0 ] already contains some other key.

4 A variant of the random probing assumption, referred to as the uniform hashing assumption, assumes that x0 , . . . , xm1 is a random permutation of 0, . . . , m 1.

2005 by Chapman & Hall/CRC

Hash Tables

9-9

9.3.1

Hashing with Chaining

In hashing with chaining, a collision is resolved by allowing more than one element to live at each position in the table. Each entry in the array A is a pointer to the head of a linked list. To insert the value x, we simply append it to the list A[x0 ]. To search for the element x, we perform a linear search in the list A[x0 ]. To delete the element x, we search for x in the list A[x0 ] and splice it out. It is clear that insertions take O(1) time, even in the worst case. For searching and deletion, the running time is proportional to a constant plus the length of the list stored at A[x0 ]. Notice that each of the at most n elements not equal to x is stored in A[x0 ] with probability 1/m, so the expected length of A[x0 ] is either = n/m (if x is not contained in the table) or 1 + (n 1)/m (if x is contained in the table). Thus, the expected cost of searching for or deleting an element is O(1 + ). The above analysis shows us that hashing with chaining supports the three set ADT operations in O(1) expected time per operation, as long as the occupancy, , is a constant. It is worth noting that this does not require that the value of be less than 1. If we would like more detailed information about the cost of searching, we might also ask about the worst-case search time dened as W = max{length of the list stored at A[i] : 0 i m 1} . It is very easy to prove something quite strong about W using only the fact that the length of each list A[i] is a binomial(n, 1/m) random variable. Using Chernos bounds on the tail of the binomial distribution [13], this immediately implies that Pr{length of A[i] c ln n} n(c) . Combining this with Booles inequality (Pr{A or B } Pr{A} + Pr{B }) we obtain Pr{W c ln n} n n(c) = n(c) . Thus, with very high probability, the worst-case search time is logarithmic in n. This also implies that E[W ] = O(log n). The distribution of W has been carefully studied and it is known that, with high probability , i.e., with probability 1 o(1), W = (1 + o(1)) ln n/ ln ln n [33, 38].5 Gonnet has proven a more accurate result that W = 1 (n) 3/2 + o(1) with high probability. Devroye [18] shows that similar results hold even when the distribution of x0 is not uniform.

9.3.2

Hashing with Open Addressing

Hashing with open addressing diers from hashing with chaining in that each table position A[i] is allowed to store only one value. When a collision occurs at table position i, one of the two elements involved in the collision must move on to the next element in its probe sequence. In order to implement this eciently and correctly we require a method of marking elements as deleted. This method could be an auxiliary array that contains one bit for each element of A, but usually the same result can be achieved by using a special key value del that does not correspond to any valid key.

5 Here, and throughout this chapter, if an asymptotic notation does not contain a variable then the variable that tends to innity is implicitly n. Thus, for example, o(1) is the set of non-negative functions of n that tend to 0 as n .

2005 by Chapman & Hall/CRC

9-10

Handbook of Data Structures and Applications

To search for an element x in the hash table we look for x at positions A[x0 ], A[x1 ], A[x2 ], and so on until we either (1) nd x, in which case we are done or (2) nd an empty table position A[xi ] that is not marked as deleted, in which case we can be sure that x is not stored in the table (otherwise it would be stored at position xi ). To delete an element x from the hash table we rst search for x. If we nd x at table location A[xi ] we then simply mark A[xi ] as deleted. To insert a value x into the hash table we examine table positions A[x0 ], A[x1 ], A[x2 ], and so on until we nd a table position A[xi ] that is either empty or marked as deleted and we store the value x in A[xi ]. Consider the cost of inserting an element x using this method. Let ix denote the smallest value i such that xix is either empty or marked as deleted when we insert x. Thus, the cost of inserting x is a constant plus ix . The probability that the table position x0 is occupied is at most so, with probability at least 1 , ix = 0. Using the same reasoning, the probability that we store x at position xi is at most Pr{ix = i} i (1 ) (9.3)

since the table locations x0 , . . . , xi1 must be occupied, the table location xi must not be occupied and the xi are independent. Thus, the expected number of steps taken by the insertion algorithm is

i Pr{ix = i} = (1 )
i=1 i=1

ii1 = 1/(1 )

for any constant 0 < < 1. The cost of searching for x and deleting x are both proportional to the cost of inserting x, so the expected cost of each of these operations is O(1/(1 )).6 We should compare this with the cost of hashing with chaining. In hashing with chaining,the occupancy has very little eect on the cost of operations. Indeed, any constant , even greater than 1 results in O(1) time per operation. In contrast, open addressing is very dependent on the value of . If we take > 1 then the expected cost of insertion using open addressing is innite since the insertion algorithm never nds an empty table position. Of course, the advantage of hashing with chaining is that it does not require lists at each of the A[i]. Therefore, the overhead of list pointers is saved and this extra space can be used instead to maintain the invariant that the occupancy is a constant strictly less than 1. Next we consider the worst case search time of hashing with open addressing. That is, we study the value W = max{ix : x is stored in the table at location ix }. Using (9.3) and Booles inequality it follows almost immediately that Pr{W > c log n} n(c) . Thus, with very high probability, W , the worst case search time, is O(log n). Tighter bounds on W are known when the probe sequences x0 , . . . , xm1 are random permutations of 0, . . . , m 1. In this case, Gonnet[29] shows that E[W ] = log1/ n log1/ (log1/ n) + O(1).

6 Note

that the expected cost of searching for or deleting an element x is proportional to the value of at the time x was inserted. If many deletions have taken place, this may be quite dierent than the current value of .

2005 by Chapman & Hall/CRC

Hash Tables

9-11

Open addressing under the random probing assumption has many nice theoretical properties and is easy to analyze. Unfortunately, it is often criticized as being an unrealistic model because it requires a long random sequences x0 , x1 , x2 , . . . for each element x that is to be stored or searched for. Several variants of open addressing discussed in the next few sections try to overcome this problem by using only a few random values.

9.3.3

Linear Probing

Linear probing is a variant of open addressing that requires less randomness. To obtain the probe sequence x0 , x1 , x2 , . . . we start with a random element x0 {0, . . . , m 1}. The element xi , i > 0 is given by xi = (i + x0 ) mod m. That is, one rst tries to nd x at location x0 and if that fails then one looks at (x0 + 1) mod m, (x0 + 2) mod m and so on. The performance of linear probing is discussed by Knuth [37] who shows that the expected number of probes performed during an unsuccessful search is at most (1 + 1/(1 )2 )/2 and the expected number of probes performed during a successful search is at most (1 + 1/(1 ))/2 . This is not quite as good as for standard hashing with open addressing, especially in the unsuccessful case. Linear probing suers from the problem of primary clustering . If j consecutive array entries are occupied then a newly inserted element will have probability j/m of hashing to one of these entries. This results in j + 1 consecutive array entries being occupied and increases the probability (to (j + 1)/m) of another newly inserted element landing in this cluster. Thus, large clusters of consecutive elements have a tendency to grow larger.

9.3.4

Quadratic Probing

Quadratic probing is similar to linear probing; an element x determines its entire probe sequence based on a single random choice, x0 . Quadratic probing uses the probe sequence x0 , (x0 + k1 + k2 ) mod m, (x0 + 2k1 + 22 k2 ) mod m, . . .. In general, the ith element in the probe sequence is xi = (x0 + ik1 + i2 k2 ) mod m. Thus, the nal location of an element depends quadratically on how many steps were required to insert it. This method seems to work much better in practice than linear probing, but requires a careful choice of m, k1 and k2 so that the probe sequence contains every element of {0, . . . , m 1}. The improved performance of quadratic probing is due to the fact that if there are two elements x and y such that xi = yj then it is not necessarily true (as it is with linear probing) that xi+1 = yj +1 . However, if x0 = y0 then x and y will have exactly the same probe sequence. This lesser phenomenon is called secondary clustering . Note that this secondary clustering phenomenon implies that neither linear nor quadratic probing can hope to perform any better than hashing with chaining. This is because all the elements that have the same initial hash x0 are contained in an implicit chain. In the case of linear probing, this chain is dened by the sequence x0 , x0 + 1, x0 + 2, . . . while for quadratic probing it is dened by the sequence x0 , x0 + k1 + k2 , x0 + 2k1 + 4k2 , . . .

9.3.5

Double Hashing

Double hashing is another method of open addressing that uses two hash values x0 and x1 . Here x0 is in the set {0, . . . , m 1} and x1 is in the subset of {1, . . . , m 1} that is

2005 by Chapman & Hall/CRC

9-12

Handbook of Data Structures and Applications

relatively prime to m. With double hashing, the probe sequence for element x becomes x0 , (x0 + x1 ) mod m, (x0 + 2x1 ) mod m, . . .. In general, xi = (x0 + ix1 ) mod m, for i > 0. The expected number of probes required by double hashing seems dicult to determine exactly. Guibas has proven that, asymptotically, and for occupancy .31, the performance of double hashing is asymptotically equivalent to that of uniform hashing. Empirically, the performance of double hashing matches that of open addressing with random probing regardless of the occupancy [37].

9.3.6

Brents Method

Brents method [5] is a heuristic that attempts to minimize the average time for a successful search in a hash table with open addressing. Although originally described in the context of double hashing (Section 9.3.5) Brents method applies to any open addressing scheme. The age of an element x stored in an open addressing hash table is the minimum value i such that x is stored at A[xi ]. In other words, the age is one less than the number of locations we will probe when searching for x. Brents method attempts to minimize the total age of all elements in the hash table. To insert the element x we proceed as follows: We nd the smallest value i such that A[xi ] is empty; this is where standard open-addressing would insert x. Consider the element y stored at location A[xi2 ]. This element is stored there because yj = xi2 , for some j 0. We check if the array location A[yj +1 ] is empty and, if so, we move y to location A[yj +1 ] and store x at location A[xi2 ]. Note that, compared to standard open addressing, this decreases the total age by 1. In general, Brents method checks, for each 2 k i the array entry A[xik ] to see if the element y stored there can be moved to any of A[yj +1 ], A[yj +2 ], . . . , A[yj +k1 ] to make room for x. If so, this represents a decrease in the total age of all elements in the table and is performed. Although Brents method seems to work well in practice, it is dicult to analyze theoretically. Some theoretical analysis of Brents method applied to double hashing is given by Gonnet and Munro [31]. Lyon [44], Munro and Celis [49] and Poblete [52] describe some variants of Brents method.

9.3.7

Multiple-Choice Hashing

It is worth stepping back at this point and revisiting the comparison between hash tables and binary search trees. For balanced binary search trees, the average cost of searching for an element is O(log n). Indeed, it easy to see that for at least n/2 of the elements, the cost of searching for those elements is (log n). In comparison, for both the random probing schemes discussed so far, the expected cost of search for an element is O(1). However, there are a handful of elements whose search cost is (log n/ log log n) or (log n) depending on whether hashing with chaining or open addressing is used, respectively. Thus there is an inversion: Most operations on a binary search tree cost (log n) but a few elements (close to the root) can be accessed in O(1) time. Most operations on a hash table take O(1) time but a few elements (in long chains or with long probe sequences) require (log n/ log log n) or (log n) time to access. In the next few sections we consider variations on hashing with chaining and open addressing that attempt to reduce the worst-case search time W . Multiple-choice hashing is hashing with chaining in which, during insertion, the element x has the choice of d 2 dierent lists in which it can be stored. In particular, when we insert x we look at the lengths of the lists pointed to by A[x0 ], . . . , A[xd1 ] and append x to A[xi ], 0 i < d such that the length of the list pointed to by A[xi ] is minimum. When searching for x, we search for x in each of the lists A[x0 ], . . . , A[xd1 ] in parallel. That is, we

2005 by Chapman & Hall/CRC

Hash Tables

9-13

look at the rst elements of each list, then the second elements of each list, and so on until we nd x. As before, to delete x we rst search for it and then delete it from whichever list we nd it in. It is easy to see that the expected cost of searching for an element x is O(d) since the expected length of each the d lists is O(1). More interestingly, the worst case search time is bounded by O(dW ) where W is the length of the longest list. Azar et al. [3] show that E[W ] = ln ln n + O(1) . ln d (9.4)

Thus, the expected worst case search time for multiple-choice hashing is O(log log n) for any constant d 2.

9.3.8

Asymmetric Hashing

Asymmetric hashing is a variant of multiple-choice hashing in which the hash table is split into d blocks, each of size n/d. (Assume, for simplicity, that n is a multiple of d.) The probe value xi , 0 i < d is drawn uniformly from {in/d, . . . , (i + 1)n/d 1}. As with multiple-choice hashing, to insert x the algorithm examines the lengths of the lists A[x0 ], A[x1 ], . . . , A[xd1 ] and appends x to the shortest of these lists. In the case of ties, it appends x to the list with smallest index. Searching and deletion are done exactly as in multiple-choice hashing. V ocking [64] shows that, with asymmetric hashing the expected length of the longest list is ln ln n E[W ] + O(1) . d ln d The function d is a generalization of the golden ratio , so that 2 = (1 + 5)/2. Note that this improves signicantly on standard multiple-choice hashing (9.4) for larger values of d.

9.3.9

LCFS Hashing

LCFS hashing is a form of open addressing that changes the collision resolution strategy.7 Reviewing the algorithm for hashing with open addressing reveals that when two elements collide, priority is given to the rst element inserted into the hash table and subsequent elements must move on. Thus, hashing with open addressing could also be referred to as FCFS (rst-come rst-served) hashing . With LCFS (last-come rst-served) hashing, collision resolution is done in exactly the opposite way. When we insert an element x, we always place it at location x0 . If position x0 is already occupied by some element y because yj = x0 then we place y at location yj +1 , possibly displacing some element z , and so on. Poblete and Munro [53] show that, after inserting n elements into an initially empty table, the expected worst case search time is bounded above by E[W ] 1 + 1 (n) 1 + ln ln(1/(1 )) +O ln 1 (n) 1 ln
2

1 (n)

7 Amble and Knuth [1] were the rst to suggest that, with open addressing, any collision resolution strategy could be used.

2005 by Chapman & Hall/CRC

9-14

Handbook of Data Structures and Applications

where is the gamma function and 1 (n) = ln n ln ln n 1+ ln ln ln n +O ln ln n 1 ln ln n .

Historically, LCFS hashing is the rst version of open addressing that was shown to have an expected worst-case search time that is o(log n).

9.3.10

Robin-Hood Hashing

Robin-Hood hashing [9, 10, 61] is a form of open addressing that attempts to equalize the search times of elements by using a fairer collision resolution strategy. During insertion, if we are trying to place element x at position xi and there is already an element y stored at position yj = xi then the younger of the two elements must move on. More precisely, if i j then we will try to insert x at position xi+1 , xi+2 and so on. Otherwise, we will store x at position xi and try to to insert y at positions yj +1 , yj +2 and so on. Devroye et al. [20] show that, after performing n insertions on an initially empty table of size m = n using the Robin-Hood insertion algorithm, the worst case search time has expected value E[W ] = (log log n) and this bound is tight. Thus, Robin-Hood hashing is a form of open addressing that has doubly-logarithmic worst-case search time. This makes it competitive with the multiplechoice hashing method of Section 9.3.7.

9.3.11

Cuckoo Hashing

Cuckoo hashing [50] is a form of multiple choice hashing in which each element x lives in one of two tables A or B , each of size m = n/. The element x will either be stored at location A[xA ] or B [xB ]. There are no other options. This makes searching for x an O(1) time operation since we need only check two array locations. The insertion algorithm for cuckoo hashing proceeds as follows:8 Store x at location A[xA ]. If A[xA ] was previously occupied by some element y then store y at location B [yB ]. If B [yB ] was previously occupied by some element z then store z at location A[zA ], and so on. This process ends when we place an element into a previously empty table slot or when it has gone on for more than c log n steps. In the former case, the insertion of x completes successfully. In the latter case the insertion is considered a failure, and the entire hash table is reconstructed from scratch using a new probe sequence for each element in the table. That is, if this reconstruction process has happened i times then the two hash values we use for an element x are xA = x2i and xB = x2i+1 . Pagh and Rodler [50] (see also Devroye and Morin [19]) show that, during the insertion of n elements, the probability of requiring a reconstruction is O(1/n). This, combined with the fact that the expected insertion time is O(1) shows that the expected cost of n insertions in a Cuckoo hashing table is O(n). Thus, Cuckoo hashing oers a somewhat simpler alternative to the dynamic perfect hashing algorithms of Section 9.2.5.

8 The algorithm takes its name from the large but lazy cuckoo bird which, rather than building its own nest, steals the nest of another bird forcing the other bird to move on.

2005 by Chapman & Hall/CRC

Hash Tables

9-15

9.4

Historical Notes

In this section we present some of the history of hash tables. The idea of hashing seems to have been discovered simultaneously by two groups of researchers. Knuth [37] cites an internal IBM memorandum in January 1953 by H. P. Luhn that suggested the use of hashing with chaining. Building on Luhns work, A. D. Linh suggested a method of open addressing that assigns the probe sequence x0 , x0 /10 , x0 /100 , x0 /1000 , . . . to the element x. At approximately the same time, another group of researchers at IBM: G. M. Amdahl, E. M. Boehme, N. Rochester and A. L. Samuel implemented hashing in an assembly program for the IBM 701 computer. Amdahl is credited with the idea of open addressing with linear probing. The rst published work on hash tables was by A. I. Dumey [24], who described hashing with chaining and discussed the idea of using remainder modulo a prime as a hash function. Ershov [25], working in Russia and independently of Amdahl, described open addressing with linear probing. Peterson [51] wrote the rst major article discussing the problem of searching in large les and coined the term open addressing. Buchholz [7] also gave a survey of the searching problem with a very good discussion of hashing techniques at the time. Theoretical analyses of linear probing were rst presented by Konheim and Weiss [39] and Podderjugin. Another, very inuential, survey of hashing was given by Morris [47]. Morris survey is the rst published use of the word hashing although it was already in common use by practitioners at that time.

9.5

Other Developments

The study of hash tables has a long history and many researchers have proposed methods of implementing hash tables. Because of this, the current chapter is necessarily incomplete. (At the time of writing, the hash.bib bibliography on hashing contains over 800 entries.) We have summarized only a handful of the major results on hash tables in internal memory. In this section we provide a few references to the literature for some of the other results. For more information on hashing, Knuth [37], Vitter and Flajolet [63], Vitter and Chen [62], and Gonnet and Baeza-Yates [30] are useful references. Brents method (Section 9.3.6) is a collision resolution strategy for open addressing that reduces the expected search time for a successful search in a hash table with open addressing. Several other methods exist that either reduce the expected or worst-case search time. These include binary tree hashing [31, 45], optimal hashing [31, 54, 55], Robin-Hood hashing (Section 9.3.10), and min-max hashing [9, 29]. One interesting method, due to Celis [9], applies to any open addressing scheme. The idea is to study the distribution of the ages of elements in the hash table, i.e., the distribution give by Di = Pr{x is stored at position xi } and start searching for x at the locations at which we are most likely to nd it, rather than searching the table positions x0 , x1 , x2 . . . in order. Perfect hash functions seem to have been rst studied by Sprugnoli [58] who gave some heuristic number theoretic constructions of minimal perfect hash functions for small data sets. Sprugnoli is responsible for the terms perfect hash function and minimal perfect hash function. A number of other researchers have presented algorithms for discovering minimal and near-minimal perfect hash functions. Examples include Anderson and Anderson [2], Cichelli [14, 15], Chang [11, 12], Gori and Soda [32], and Sager [56]. Berman et al. [4]

2005 by Chapman & Hall/CRC

9-16

Handbook of Data Structures and Applications

and K orner and Marton [40] discuss the theoretical limitations of perfect hash functions. A comprehensive, and recent, survey of perfect hashing and minimal perfect hashing is given by Czech et al. [17]. Tarjan and Yao [60] describe a set ADT implementation that gives O(log u/ log n) worstcase access time. It is obtained by combining a trie (Chapter 28) of degree n with a compression scheme for arrays of size n2 that contain only n non-zero elements. (The trie has O(n) nodes each of which has n pointers to children, but there are only a total of O(n) children.) Although their result is superseded by the results of Fredman et al. [28] discussed in Section 9.2.4, they are the rst theoretical results on worst-case search time for hash tables. Dynamic perfect hashing (Section 9.2.5) and cuckoo hashing (Section 9.3.11) are methods of achieving O(1) worst case search time in a dynamic setting. Several other methods have been proposed [6, 21, 22]. Yao [65] studies the membership problem . Given a set S U , devise a data structure that can determine for any x U whether x is contained in S . Yao shows how, under various conditions, this problem can be solved using a very small number of memory accesses per query. However, Yaos algorithms sometimes derive the fact that an element x is in S without actually nding x. Thus, they dont solve the set ADT problem discussed at the beginning of this chapter since they can not recover a pointer to x. The power of two random choices, as used in multiple-choice hashing, (Section 9.3.7) has many applications in computer science. Karp, Luby and Meyer auf der Heide [34, 35] were the rst to use this paradigm for simulating PRAM computers on computers with fewer processors. The book chapter by Mitzenmacher et al. [46] surveys results and applications of this technique. A number of table implementations have been proposed that are suitable for managing hash tables in external memory. Here, the goal is to reduce the number of disk blocks that must be accessed during an operation, where a disk block can typically hold a large number of elements. These schemes include linear hashing [43], dynamic hashing [41], virtual hashing [42], extendible hashing [26], cascade hashing [36], and spiral storage [48]. In terms of hashing, the main dierence between internal memory and external memory is that, in internal memory, an array is allocated at a specic size and this can not be changed later. In contrast, an external memory le may be appended to or be truncated to increase or decrease its size, respectively. Thus, hash table implementations for external memory can avoid the periodic global rebuilding operations used in internal memory hash table implementations.

Acknowledgment
The author is supported by a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC).

2005 by Chapman & Hall/CRC

Hash Tables

9-17

References
[1] O. Amble and D. E. Knuth. Ordered hash tables. The Computer Journal, 17(2):135 142, 1974. [2] M. R. Anderson and M. G. Anderson. Comments on perfect hashing functions: A single probe retrieving method for static sets. Communications of the ACM, 22(2):104, 1979. [3] Y. Azar, A. Z. Broder, A. R. Karlin, and E. Upfal. Balanced allocations. SIAM Journal on Computing, 29(1):180200, 1999. [4] F. Berman, M. E. Bock, E. Dittert, M. J. ODonnell, and D. Plank. Collections of functions for perfect hashing. SIAM Journal on Computing, 15(2):604618, 1986. [5] R. P. Brent. Reducing the storage time of scatter storage techniques. Communications of the ACM, 16(2):105109, 1973. [6] A. Brodnik and J. I. Munro. Membership in constant time and almost minimum space. SIAM Journal on Computing, 28:16271640, 1999. [7] W. Buchholz. File organization and addressing. IBM Systems Journal, 2(1):86111, 1963. [8] J. L. Carter and M. N. Wegman. Universal classes of hash functions. Journal of Computer and System Sciences, 18(2):143154, 1979. [9] P. Celis. Robin Hood hashing. Technical Report CS-86-14, Computer Science Department, University of Waterloo, 1986. [10] P. Celis, P.- A. Larson, and J. I. Munro. Robin Hood hashing. In Proceedings of the 26th Annual IEEE Symposium on Foundations of Computer Science (FOCS85), pages 281288. IEEE Press, 1985. [11] C. C. Chang. An ordered minimal perfect hashing scheme based upon Eulers theorem. Information Sciences, 32(3):165172, 1984. [12] C. C. Chang. The study of an ordered minimal perfect hashing scheme. Communications of the ACM, 27(4):384387, 1984. [13] H. Cherno. A measure of the asymptotic ecient of tests of a hypothesis based on the sum of observations. Annals of Mathematical Statistics, 23:493507, 1952. [14] R. J. Cichelli. Minimal perfect hash functions made simple. Communications of the ACM, 23(1):1719, 1980. [15] R. J. Cichelli. On Cichellis minimal perfect hash functions method. Communications of the ACM, 23(12):728729, 1980. [16] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein. Introduction to Algorithms. MIT Press, Cambridge, Massachussetts, 2nd edition, 2001. [17] Z. J. Czech, G. Havas, and B. S. Majewski. Perfect hashing. Theoretical Computer Science, 182(1-2):1143, 1997. [18] L. Devroye. The expected length of the longest probe sequence when the distribution is not uniform. Journal of Algorithms, 6:19, 1985. [19] L. Devroye and P. Morin. Cuckoo hashing: Further analysis. Information Processing Letters, 86(4):215219, 2002. [20] L. Devroye, P. Morin, and A. Viola. On worst case Robin-Hood hashing. SIAM Journal on Computing. To appear. [21] M. Dietzfelbinger and F. Meyer auf der Heide. A new universal class of hash functions and dynamic hashing in real time. In Proceedings of the 17th International Colloquium on Automata, Languages, and Programming (ICALP90), pages 619, 1990. [22] M. Dietzfelbinger, J. Gil, Y. Matias, and N. Pippenger. Polynomial hash functions are reliable. In Proceedings of the 19th International Colloquium on Automata, Languages, and Programming (ICALP92), pages 235246, 1992.

2005 by Chapman & Hall/CRC

9-18

Handbook of Data Structures and Applications

[23] M. Dietzfelbinger, A. R. Karlin, K. Mehlhorn, F. Meyer auf der Heide, H. Rohnert, and R. E. Tarjan. Dynamic perfect hashing: Upper and lower bounds. SIAM Journal on Computing, 23(4):738761, 1994. [24] A. I. Dumey. Indexing for rapid random access memory systems. Computers and Automation, 5(12):69, 1956. [25] A. P. Ershov. On programming of arithmetic operations. Doklady Akademii Nauk SSSR, 118(3):427430, 1958. [26] R. Fagin, J. Nievergelt, N. Pippenger, and H. R. Strong. Extendible hashing a fast access method for dynamic les. ACM Transactions on Database Systems, 4(3):315 344, 1979. [27] W. Feller. An Introduction to Probability Theory and its Applications. John Wiley & Sons, New York, 1968. [28] M. L. Fredman, J. Koml os, and E. Szemer edi. Storing a sparse table with O(1) worst case access time. Journal of the ACM, 31(3):538544, 1984. [29] G. H. Gonnet. Expected length of the longest probe sequence in hash code searching. Journal of the ACM, pages 289304, 1981. [30] G. H. Gonnet and R. Baeza-Yates. Handbook of Algorithms and Data Structures: in Pascal and C. Addison-Wesley, Reading, MA, USA, 2nd edition, 1991. [31] G. H. Gonnet and J. I. Munro. Ecient ordering of hash tables. SIAM Journal on Computing, 8(3):463478, 1979. [32] M. Gori and G. Soda. An algebraic approach to Cichellis perfect hashing. Bit, 29(1):2 13, 1989. [33] N. L. Johnson and S. Kotz. Urn Models and Their Applications. John Wiley & Sons, New York, 1977. [34] R. Karp, M. Luby, and F. Meyer auf der Heide. Ecient PRAM simulation on a distributed memory machine. Technical Report TR-93-040, International Computer Science Institute, Berkeley, CA, USA, 1993. [35] R. M. Karp, M. Luby, and F. Meyer auf der Heide. Ecient PRAM simulation on a distributed memory machine. In Proceedings of the 24th ACM Symposium on the Theory of Computing (STOC92), pages 318326. ACM Press, 1992. [36] P. Kjellberg and T. U. Zahle. Cascade hashing. In Proceedings of the 10th International Conference on Very Large Data Bases (VLDB80), pages 481492. Morgan Kaufmann, 1984. [37] D. E. Knuth. The Art of Computer Programming, volume 3. Addison-Wesley, 2nd edition, 1997. [38] V. F. Kolchin, B. A. Sevastyanov, and V. P. Chistyakov. Random Allocations. John Wiley & Sons, New York, 1978. [39] A. G. Konheim and B. Weiss. An occupancy discipline and its applications. SIAM Journal of Applied Mathematics, 14:12661274, 1966. [40] J. K orner and K. Marton. New bounds for perfect hashing via information theory. European Journal of Combinatorics, 9(6):523530, 1988. [41] P.- A. Larson. Dynamic hashing. Bit, 18(2):184201, 1978. [42] W. Litwin. Virtual hashing: A dynamically changing hashing. In Proceedings of the 4th International Conference on Very Large Data Bases (VLDB80), pages 517523. IEEE Computer Society, 1978. [43] W. Litwin. Linear hashing: A new tool for le and table addressing. In Proceedings of the 6th International Conference on Very Large Data Bases (VLDB80), pages 212223. IEEE Computer Society, 1980. [44] G. E. Lyon. Packed scatter tables. Communications of the ACM, 21(10):857865, 1978.

2005 by Chapman & Hall/CRC

Hash Tables
[45] E. G. Mallach. Scatter storage techniques: A unifying viewpoint and a method for reducing retrieval times. The Computer Journal, 20(2):137140, 1977. [46] M. Mitzenmacher, A. W. Richa, and R. Sitaraman. The power of two random choices: A survey of techniques and results. In P. Pardalos, S. Rajasekaran, and J. Rolim, editors, Handbook of Randomized Computing, volume 1, chapter 9. Kluwer, 2001. [47] R. Morris. Scatter storage techniques. Communications of the ACM, 11(1):3844, 1968. [48] J. K. Mullin. Spiral storage: Ecient dynamic hashing with constant performance. The Computer Journal, 28(3):330334, 1985. [49] J. I. Munro and P. Celis. Techniques for collision resolution in hash tables with open addressing. In Proceedings of 1986 Fall Joint Computer Conference, pages 601610. ACM Press, 1999. [50] R. Pagh and F. F. Rodler. Cuckoo hashing. In Proceedings of the 9th Annual European Symposium on Algorithms (ESA 2001), volume 2161 of Lecture Notes in Computer Science, pages 121133. Springer-Verlag, 2001. [51] W. W. Peterson. Addressing for random-access storage. IBM Journal of Research and Development, 1(2):130146, 1957. [52] P. V. Poblete. Studies on hash coding with open addressing. M. Math Essay, University of Waterloo, 1977. [53] P. V. Poblete and J. Ian Munro. Last-come-rst-served hashing. Journal of Algorithms, 10:228248, 1989. [54] G. Poonan. Optimal placement of entries in hash tables. In ACM Computer Science Conference (Abstract Only), volume 25, 1976. (Also DEC Internal Tech. Rept. LRD1, Digital Equipment Corp. Maynard Mass). [55] R. L. Rivest. Optimal arrangement of keys in a hash table. Journal of the ACM, 25(2):200209, 1978. [56] T. J. Sager. A polynomial time generator for minimal perfect hash functions. Communications of the ACM, 28(5):523532, 1985. [57] V. T. S os. On the theory of diophantine approximations. i. Acta Mathematica Budapest, 8:461471, 1957. [58] R. Sprugnoli. Perfect hashing functions: A single probe retrieving method for static sets. Communications of the ACM, 20(11):841850, 1977. [59] S. Swierczkowski. On successive settings of an arc on the circumference of a circle. Fundamenta Mathematica, 46:187189, 1958. [60] R. E. Tarjan and A. C.-C. Yao. Storing a sparse table. Communications of the ACM, 22(11):606611, 1979. [61] A. Viola and P. V. Poblete. Analysis of linear probing hashing with buckets. Algorithmica, 21:3771, 1998. [62] J. S. Vitter and W.-C. Chen. The Design and Analysis of Coalesced Hashing. Oxford University Press, Oxford, UK, 1987. [63] J. S. Vitter and P. Flajolet. Analysis of algorithms and data structures. In J. van Leeuwen, editor, Handbook of Theoretical Computer Science, volume A: Algorithms and Complexity, chapter 9, pages 431524. North Holland, 1990. [64] B. V ocking. How asymmetry helps load balancing. In Proceedings of the 40th Annual IEEE Symposium on Foundations of Computer Science (FOCS99), pages 131140. IEEE Press, 1999. [65] A. C.-C. Yao. Should tables be sorted? Journal of the ACM, 28(3):615628, 1981.

9-19

2005 by Chapman & Hall/CRC

10
Balanced Binary Search Trees
10.1 10.2 10.3 10.4 10.5 10.6 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basic Denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Trees Binary Trees as Dictionaries Implementation of Binary Search Trees

10-1 10-2 10-4 10-7

Generic Discussion of Balancing . . . . . . . . . . . . . . . . .

Balance Denitions Rebalancing Algorithms Complexity Results

Classic Balancing Schemes . . . . . . . . . . . . . . . . . . . . . . . .

AVL-Trees Weight-Balanced Trees Balanced Binary Trees Based on Multi-Way Trees.

Arne Andersson
Uppsala University

Rebalancing a Tree to Perfect Balance . . . . . . . . . . 10-11 Schemes with no Balance Information . . . . . . . . . . 10-12
Implicit Representation of Balance Information General Balanced Trees Application to Multi-Dimensional Search Trees

Rolf Fagerberg
University of Southern Denmark

10.7 10.8

Low Height Schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-17 Relaxed Balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10-20

Red-Black Trees Other Results

Kim S. Larsen
University of Southern Denmark

AVL-Trees

Multi-Way Trees

10.1

Introduction

Balanced binary search trees are among the most important data structures in Computer Science. This is because they are ecient, versatile, and extensible in many ways. They are used as a black-box in numerous algorithms and even other data structures. The main virtue of balanced binary search trees is their ability to maintain a dynamic set in sorted order, while supporting a large range of operations in time logarithmic in the size of the set. The operations include search, insertion, deletion, predecessor/successor search, range search, rank search, batch update, split, meld, and merge. These operations are described in more detail in Section 10.2 below. Data structures supporting the operations search, insertion, deletion, and predecessor (and/or successor) search are often denoted ordered dictionaries. In the comparison based model, the logarithmic performance of balanced binary search trees is optimal for ordered dictionaries, whereas in the RAM model, faster operations are possible [13, 18]. If one considers unordered dictionaries, i.e., only the operations search, insertion, and deletion, expected constant time is possible by hashing.

10-1

2005 by Chapman & Hall/CRC

10-2

Handbook of Data Structures and Applications

10.2
10.2.1

Basic Denitions
Trees

There are many ways to dene trees. In this section, we dene a tree as a hierarchical organization of a collection of nodes. For alternatives to our exposition, see the chapter on trees. A tree can be empty. If it is not empty, it consists of one node, which is referred to as the root of the tree, and a collection of trees, referred to as subtrees. Thus, a tree consists of many smaller trees, each with their own root. We use r to denote the single node which is the root of the entire tree. We only consider nite trees, i.e., every collection of subtrees is nite, and there are no innite chains of nonempty subtrees. Furthermore, we only consider ordered trees, meaning that the collection of subtrees of a node is an ordered sequence rather than just a set. If every nonempty tree has exactly two subtrees, then the tree is called binary. In this case, we refer to the two subtrees as the left and right subtrees. We use u, v , w, etc. to denote nodes and T to denote trees, applying apostrophes, index, etc. to increase the name space. For a node u, we use u.l and u.r to denote the left and right subtree, respectively, of the tree rooted by u. However, when no confusion can occur, we do not necessarily distinguish between nodes and subtrees. Thus, by the subtree v , we mean the subtree rooted at the node v and by T we mean the entire tree or the root of the tree. We use the standard genealogical terminology to denote nodes in the vicinity of a designated node. Thus, if u is the root of a tree and v is the root of a subtree of u, then v is referred to as a child of u. By analogy, this denes grandchildren, parent, grandparent, and sibling. The set of nodes belonging to a nonempty tree is its root, along with all the nodes belonging to its subtrees. For an empty tree, this set is of course empty. If a node v belongs to the subtree of u, then v is a descendant of u, and u is an ancestor of v . An ancestor or descendant v of a node u is proper if u = v . Quite often, it is convenient to refer to empty subtrees as real nodes, in which case they are referred to as external nodes (or leaves). The remaining nodes are then referred to as internal nodes. It is easy to prove by induction that the number of external nodes is always one larger than the number of internal nodes. The number of nodes belonging to a tree is referred to as its size (or its weight). In some applications, we dene the size of the tree to be the number of internal nodes in the tree, but more often it is convenient to dene the size of the tree to be the number of external nodes. We use n to denote the size of the tree rooted by r, and |u| to denote the size of the subtree rooted by u. A path in a tree is a sequence of nodes u1 , u2 , . . . , uk , k 1, such that for i {1, . . . , k 1}, ui+1 is a child of ui . Note that the length of such a path is k 1. The depth of a node u in the tree T is the length of the path from the root of T to u, and the height of a tree T is the maximal depth of any external node.

10.2.2

Binary Trees as Dictionaries

When trees are used to implement the abstract data type dictionary, nodes have associated values. A dictionary basically organizes a set of keys, which must be elements drawn from a total ordering, and must usually supply at least the operations search, insertion, and deletion. There may be additional information associated with each key, but this does not

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees lead to any conceptual complications, so here we simply focus on the keys.

10-3

When a tree is used as a dictionary, each node stores one key, and we impose the following ordering invariant (the in-order invariant): for each node u in the tree, every key in u.l is strictly smaller than u.k , and every key in u.r is strictly larger than u.k . A tree organized according to this invariant is referred to as a binary search tree. An important implication of this ordering invariant is that a sorted list of all the keys in the tree can be produced in linear time using an in-order traversal dened recursively as follows. On an empty tree, do nothing. Otherwise, recurs on the left subtree, report the root key, and then recurs on the right subtree. Many dierent operations can be supported by binary search tree implementations. Here, we discuss the most common. Using the ordering invariant, we can devise a searching procedure of asymptotic time complexity proportional to the height of the tree. Since searching turns out to be at the heart of most of the operations of interest to us, unless we stipulate otherwise, all the operations in the following inherit the same complexity.
Simple Searching

To search for x in a tree rooted by u, we rst compare x to u.k . If they are equal, a positive response is given. Otherwise, if x is smaller than u.k , we search recursively in u.l, and if x is larger, we search in u.r. If we arrive at an empty tree, a negative response is given. In this description, we have used ternary comparisons, in that our decisions regarding how to proceed depend on whether the search key is less than, equal to, or greater than the root key. For implementation purposes, it is possible to use the more ecient binary comparisons [12]. A characteristic feature of search trees is that when a searching fails, a nearest neighbor can be provided eciently. Dictionaries supporting predecessor/successor queries are referred to as ordered. This is in contrast to hashing (described in a chapter of their own) which represents a class of unordered dictionaries. A predecessor search for x must return the largest key less than or equal to x. This operation as well as the similar successor search are simple generalizations of the search strategy outlined above. The case where x is found on the way is simple, so assume that x is not in the tree. Then the crucial observation is that if the last node encountered during the search is smaller than x, then this node is the predecessor. Otherwise, the predecessor key is the largest key in the left subtree of the last node on the search path containing a key smaller than x. A successor search is similar.
Simple Updates

An insertion takes a tree T and a key x not belonging to T as arguments and adds a node containing x and two empty subtrees to T . The node replaces the empty subtree in T where the search for x terminates. A deletion takes a tree T and a key x belonging to T as arguments and removes the node u containing x from the tree. If us children are empty trees, u is simply replaced by an empty tree. If u has exactly one child which is an internal node, then this child is replacing u. Finally, if u has two internal nodes as children, us predecessor node v is used. First, the key in u is overwritten by the key of v , after which v is deleted. Note that because of the choice of v , the ordering invariant is not violated. Note also that v has at most one child which is an internal node, so one of the simpler replacing strategies described above can be used to remove v .

2005 by Chapman & Hall/CRC

10-4

Handbook of Data Structures and Applications

More Searching Procedures

A range search takes a tree T and two key values k1 k2 as arguments and returns all keys x for which k1 x k2 . A range search can be viewed as an in-order traversal, where we do not recurs down the left subtree and do not report the root key if k1 should be in the right subtree; similarly, we do not recurs down the right subtree and do not report the root key if k2 should be in the left subtree. The complexity is proportional to the height of the tree plus the size of the output. A useful technique for providing more complex operations eciently is to equip the nodes in the tree with additional information which can be exploited in more advanced searching, and which can also be maintained eciently. A rank search takes a tree T and an integer d between one and n as arguments, and returns the dth smallest key in T . In order to provide this functionality eciently, we store in each node the size of the subtree in which it is the root. Using this information during a search down the tree, we can at each node determine in which subtree the node must be located and we can appropriately adjust the rank that we search for recursively. If the only modications made to the tree are small local changes, this extra information can be kept up-to-date eciently, since it can always be recomputed from the information in the children.
Operations Involving More Trees

The operation split takes a key value x and tree T as arguments and returns two trees; one containing all keys from T less than or equal to x and one with the remaining keys. The operations is destructive, meaning that the argument tree T will not be available after the operation. The operation meld takes two trees as arguments, where all keys in one tree are smaller than all keys in the other, and combines the trees into one containing all the keys. This operation is also destructive. Finally, merge combines the keys from two argument trees, with no restrictions on keys, into one. Also this operation is destructive.

10.2.3

Implementation of Binary Search Trees

In our discussion of time and space complexities, we assume that some standard implementation of trees are used. Thus, in analogy with the recursive denition, we assume that a tree is represented by information associated with its root, primarily the key, along with pointers (references) to its left and right subtrees, and that this information can be accessed in constant time. In some situations, we may assume that additional pointers are present, such as parentpointers, giving a reference from a node to its parent. We also sometimes use level-pointers. A level consists of all nodes of the same depth, and a level-pointer to the right from a node with key k points to the node at the same level with the smallest key larger than k . Similar for level-pointers to the left.

10.3

Generic Discussion of Balancing

As seen in Section 10.2, the worst case complexity of almost all operations on a binary search tree is proportional to its height, making the height its most important single characteristic. Since a binary tree of height h contains at most 2h 1 nodes, a binary tree of n nodes has a height of at least log(n + 1) . For static trees, this lower bound is achieved by a tree where all but one level is completely lled. Building such a tree can be done in linear time (assuming that the sorted order of the keys is known), as discussed in Section 10.5 below. In the dynamic case, however, insertions and deletions may produce a very unbalanced

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-5

treefor instance, inserting elements in sorted order will produce a tree of height linear in the number of elements. The solution is to rearrange the tree after an insertion or deletion of an element, if the operation has made the tree unbalanced. For this, one needs a denition of balance and a rebalancing algorithm describing the rearrangement leading to balance after updates. The combined balance denition and rebalancing algorithm we denote a rebalancing scheme. In this section, we discuss rebalancing schemes at a generic level. The trivial rebalancing scheme consists of dening a balanced tree as one having the optimal height log(n + 1) , and letting the rebalancing algorithm be the rebuilding of the entire tree after each update. This costs linear time per update, which is exponentially larger than the search time of the tree. It is one of the basic results of Computer Science, rst proved by Adelson-Velski and Landis in 1962 [1], that logarithmic update cost can be achieved simultaneously with logarithmic search cost in binary search trees. Since the appearance of [1], many other rebalancing schemes have been proposed. Almost all reproduce the result of [1] in the sense that they, too, guarantee a height of c log(n) for some constant c > 1, while handling updates in O(log n) time. The schemes can be grouped according to the ideas used for denition of balance, the ideas used for rebalancing, and the exact complexity results achieved.

10.3.1

Balance Denitions

The balance denition is a structural constraint on the tree ensuring logarithmic height. Many schemes can viewed as belonging to one of the following three categories: schemes with a constraint based on the heights of subtrees, schemes with a constraint based on the sizes of subtrees, and schemes which can be seen as binarizations of multi-way search tree schemes and which have a constraint inherited from these. The next section will give examples of each. For most schemes, balance information is stored in the nodes of the tree in the form of single bits or numbers. The structural constraint is often expressed as an invariant on this information, and the task of the rebalancing algorithm is to reestablish this invariant after an update.

10.3.2

Rebalancing Algorithms

The rebalancing algorithm restores the structural constraint of the scheme if it is violated by an update. It uses the balance information stored in the nodes to guide its actions. The general form of the algorithm is the same in almost all rebalancing schemesbalance violations are removed by working towards the root along the search path from the leaf where the update took place. When removing a violation at one node, another may be introduced at its parent, which is then handled, and so forth. The process stops at the root at the latest. The violation at a node is removed in O(1) time by a local restructuring of the tree and/or a change of balance information, giving a total worst case update time proportional to the height of the tree. The fundamental restructuring operation is the rotation, shown in Figure 10.1. It was introduced in [1]. The crucial feature of a rotation is that it preserves the in-order invariant of the search tree while allowing one subtree to be moved upwards in the tree at the expense of another. A rotation may be seen as substituting a connected subgraph T consisting of two nodes with a new connected subgraph T on the same number of nodes, redistributing the keys (here x and y ) in T according to in-order, and redistributing the subtrees rooted at leaves

2005 by Chapman & Hall/CRC

10-6
y x C A B

Handbook of Data Structures and Applications

x

A B

y C

FIGURE 10.1: Rotation. of T by attaching them as leaves of T according to in-order. Described in this manner, it is clear that in-order will be preserved for any two subgraphs T and T having an equal number of nodes. One particular case is the double rotation shown in Figure 10.2, so named because it is equivalent to two consecutive rotations.

z x y A B C D

x B C

z D

FIGURE 10.2: Double rotation. Actually, any such transformation of a connected subgraph T to another T on the same number of nodes can be executed through a series of rotations. This can be seen by noting that any connected subgraph can be converted into a right-path, i.e., a tree where all left children are empty trees, by repeated rotations (in Figure 10.1, if y but not x is on the rightmost path in the tree, the rotation will enlarge the rightmost path by one node). Using the right-path as an intermediate state and running one of the conversions backwards will transform T into T . The double rotation is a simple case of this. In a large number of rebalancing schemes, the rebalancing algorithm performs at most one rotation or double rotation per node on the search path. We note that rebalancing schemes exist [34] where the rebalancing along the search path is done in a top-down fashion instead of the bottom-up fashion described above. This is useful when several processes concurrently access the tree, as discussed in Section 10.8. In another type of rebalancing schemes, the restructuring primitive used is the rebuilding of an entire subtree to perfect balance, where perfect balance means that any node is the median among the nodes in its subtree. This primitive is illustrated in Figure 10.3. In these rebalancing schemes, the restructuring is only applied to one node on the search path for the update, and this resolves all violations of the balance invariant. The use of this rebalancing technique is sometimes termed local or partial rebuilding (in contrast to global rebuilding of data structures, which designates a periodically rebuilding of the entire structure). In Section 10.5, we discuss linear time algorithms for rebalancing a (sub-)tree to perfect balance.

10.3.3

Complexity Results

Rebalancing schemes can be graded according to several complexity measures. One such measure is how much rebalancing work is needed after an update. For this measure, typical

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-7

11111 00000 00000 11111 00000 11111 00000 11111 00000 11111

11111 00000 00000 11111 00000 11111 00000 11111 00000 11111

FIGURE 10.3: Rebuilding a subtree.

values include amortized O(log n), worst case O(log n), amortized O(1), and worst case O(1). Values below logarithmic may at rst sight seem useless due to the logarithmic search time of balanced search trees, but they are relevant in a number of settings. One setting is nger search trees (described in a chapter of their own in this book), where the search for the update point in the tree does not start at the root and hence may take sub-logarithmic time. Another setting is situations where the nodes of the tree are annotated with information which is expensive to update during restructuring of the tree, such that rotations may take non-constant time. This occurs in Computational Geometry, for instance. A third setting is concurrent access to the tree by several processes. Searching the tree concurrently is not a problem, whereas concurrent updates and restructuring may necessitate lockings of nodes in order to avoid inconsistencies. This makes restructuring more expensive than searches. Another complexity measure is the exact height maintained. The majority of schemes maintain a height bounded by c log n for some constant c > 1. Of other results, splay trees [70] have no sub-linear bound on the height, but still perform searches in amortized O(log n) time. Splay trees are described in a chapter of their own in this book. In the other direction, a series of papers investigate how close c can get to the optimal value one, and at what rebalancing cost. We discuss these results in Section 10.7. One may also consider the exact amount of balance information stored in each node. Some schemes store an integer, while some only need one or two bits. This may eect the space consumption of nodes, as a single bit may be stored implicitly, e.g., as the sign bit of a pointer, or by storing subtrees out of order when the bit is set. Schemes even exist which do not need to store any information at all in nodes. We discuss these schemes in Section 10.6 Finally, measures such as complexity of implementation and performance in practice can also be considered. However, we will not discuss these here, mainly because these measures are harder to quantify.

10.4
10.4.1

Classic Balancing Schemes

AVL-Trees

AVL-trees where introduced in 1962 in [1], and are named after their inventors AdelsonVelski and Landis. They proposed the rst dictionary structure with logarithmic search and update times, and also introduced the rebalancing technique using rotations. The balance denition in AVL-trees is based on the height of subtrees. The invariant is that for any node, the heights of its two subtrees dier by at most one. Traditionally, the balance information maintained at each node is +1, 0, or 1, giving the dierence in heights between the right subtree and the left subtree. This information can be represented by two bits. Another method is to mark a node when its height is larger than its siblings. This requires only one bit per node, but reading the balance of a node now involves visiting

2005 by Chapman & Hall/CRC

10-8

Handbook of Data Structures and Applications

its children. In the other direction, storing the height of each node requires log log n bits of information per node, but makes the rebalancing algorithms simpler to describe and analyze. By induction on h, it is easily proved that for an AVL-tree of height h, the minimum number of nodes is Fh+2 1, where Fi denotes the ith Fibonacci number, dened by F1 = F2 = 1 and Fj +2 = Fj +1 + Fj . A well-known fact for Fibonacci numbers is that Fi i2 , where is the golden ratio ( 5 + 1)/2 1.618. This shows that the height of an AVL-tree with n nodes is at most log (n + 1), i.e., AVL-trees have a height bound of the type c log n with c = 1/ log 1.440. After an update, violations of the balance invariant can only occur at nodes on the search path from the root to the update point, as only these nodes have subtrees changed. The rebalancing algorithm resolves these in a bottom-up fashion. At each node, it either performs a rotation, performs a double rotation, or just updates balance information, with the choice depending on the balance of its child and grandchild on the search path. The algorithm stops when it can guarantee that no ancestor has a balance problem, or when the root is reached. In AVL-trees, the rebalancing algorithm has the following properties: After an insertion, change of balance information may take place any number of steps towards the root, but as soon as a rotation or double rotation takes place, no further balance problems remain. Hence, only O(1) structural change is made. In contrast, after a deletion it may happen that rotations are performed at all nodes on the search path. If only insertions take place, the amortized amount of rebalancing work, including updating of balance information, can be shown [58] to be O(1). The same is true if only deletions take place [75]. It is not true in the fully dynamic case, as it is easy to nd an AVL-tree where alternating insertions and deletions of the same key require rebalancing along the entire search path after each update.

10.4.2

Weight-Balanced Trees

Weight-balanced trees were proposed in 1973 by Nievergelt and Reingold [62], and have a balance denition based on the sizes of subtrees. Here, the size of a subtree is most conveniently dened as the number of external nodes (empty trees) in the subtree, and the size, also denoted the weight, of a node is the size of its subtree. The balance invariant of weight-balanced trees states that for any node, the ratio between its own weight and the weight of its right child (or left) is in the interval [ , 1 ] for some xed value > 0. This ratio is denoted the balance of the node. Since a node of weight three must have subtrees of weight two and one, we must have 1/3. Weight-balanced trees are also called BB[]-trees, which stands for trees of bounded balance with parameter . By the balance criterion, for any node v the weight of the parent of v is at least a factor 1/(1 ) larger than the weight of v . A tree of height k therefore has a root of weight at least 1/(1 )k , which shows that the height of a weight-balanced tree with n nodes is at most log1/(1) (n + 1), i.e., weight-balanced trees have a height bound of the type c log n with c = 1/ log(1 ) > 1.709. The balance information stored in each node is its weight, for which log n bits are needed. After an update, this information must be updated for all nodes on the search path from the root to the update point. Some of these nodes may now violate the balance criterion. The rebalancing algorithm proposed in [62] resolves this unbalance in a bottom-up fashion along the search path using either a rotation or a double rotation at each violating node. The choice of rotation depends on the weight of the children and the grandchildren of the node.

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-9

In [62], the rebalancing algorithm was claimed to work for in the interval [ 0 , 1 1/2 ], but Blum and Mehlhorn [20] later observed that the correct interval is (2/11 , 1 1/ 2 ]. They also showed that for strictly inside this interval, the rebalancing of an unbalanced node restores its balance to a value in [ (1 + ) , 1 (1 + ) ], where depends on the choice of . This implies that when the node becomes unbalanced again, the number of updates which have taken place below it since it was last rebalanced is at least a fraction (depending on ) of its current weight. This feature, unique to weight-balanced trees, has important applications, e.g., for data structures in Computational Geometry. A number of these structures are binary search trees where each node has an associated secondary structure built on the elements in the subtree of the node. When a rotation takes place, the structures of the nodes taking part in the rotation will have to be rebuilt. If we attribute the cost of this rebuilding evenly to the updates which have taken place below the node since it was last involved in a rotation, then, as an example, a linear rebuilding cost of the secondary structure will amount to a constant attribution to each of these updates. As the search path for an update contains O(log n) nodes, any single update can at most receive this many attributions, which implies an amortized O(log n) update complexity for the entire data structure. The same analysis allows BB[]-trees to be maintained by local rebuilding instead of rotations in amortized O(log n) time, as rst noted by Overmars and van Leeuwen [69]: After an update, the subtree rooted at the highest unbalanced node (if any) on the search path is rebuilt to perfect balance. Since a rebuilding of a subtree leaves all nodes in it with balance close to 1/2, the number of updates which must have taken place below the node since it was last part of a rebuilding is a constant fraction of its current weight. The rebuilding uses work linear in this weight, which can be covered by attributing a constant amount of work to each of these updates. Again, each update is attributed O(log n) work. This scheme will work for any 1/3. For the original rebalancing algorithm using rotations, a better analysis can be made for chosen strictly inside the interval (2/11 , 1 1/ 2 ]: The total work per rebalancing operation is now O(1), so the work to be attributed to each update below a node is O(1/w), where w is the weight of the node. As noted above in the proof of the height bound of weight-balanced trees, w is exponentially increasing along the search path from the update point to the root. This implies that each update is attributed only O(1) work in total, and also that the number of rotations taking place at a given height decreases exponentially with the height. This result from [20] seems to be the rst on O(1) amortized rebalancing in binary search trees. The actual time spent after an update is still logarithmic in weightbalanced trees, though, as the balance information needs to be updated along the entire search path, but this entails no structural changes. Recently, the idea of balancing by weight has been applied to multi-way search trees [14], leading to trees ecient in external memory which posses the same feature as weightbalanced binary trees, namely that between each rebalancing at a node, the number of updates which have taken place below the node is proportional to the weight of the node.

10.4.3

Balanced Binary Trees Based on Multi-Way Trees.

The B-tree [17], which is treated in another chapter of this book, is originally designed to handle data stored on external memory. The basic idea is to associate a physical block with a high-degree node in a multi-way tree. A B-tree is maintained by merging and splitting nodes, and by increasing and decreasing the number of layers of multi-way nodes. The smallest example of a B-tree is the 2-3-tree [2], where the nodes have degree 2 or 3. In a typical B-tree implementation, the degree of a node is much larger, and it varies roughly

2005 by Chapman & Hall/CRC

10-10

Handbook of Data Structures and Applications

within a factor of 2. The concept of multi-way nodes, splitting, and merging, has also proven to be very fruitful in the design of balancing schemes for binary trees. The rst such example is the binary B-tree [15], a binary implementation of 2-3-trees. Here, the idea is to organize binary nodes into larger chunks of nodes, here called pseudo-nodes. In the binary version of a 2-3-tree, a node of degree 2 is represented by one binary node, while a node of degree 3 is represented as two binary nodes (with the additional constraint that one of the two nodes is the right child of the other). In the terms of binary nodes grouped into pseudo-nodes, it is convenient to say that edges within a pseudo-node are horizontal while edges between pseudo-nodes are vertical. As a natural extension of binary B-trees, Bayer invented Symmetric Binary Trees, or SBBtrees [16]. The idea was that, instead of only allowing a binary node to have one horizontal outgoing edge to its right child, we can allow both left- and right-edges to be horizontal. For both binary B-trees and Symmetric Binary B-trees, Bayer designed maintenance algorithms, where the original B-tree operations split, merge, and increase/decrease number of levels were implemented for the pseudo-nodes. Today, SBB-trees mostly appear under the name red-black trees [34]. Here, the horizontal and vertical edges are represented by one color per node. (Both notations can be represented by one bit per node.) SBB/red-black trees are binary implementations of B-trees where each node has degree between 2 and 4. One advantage with SBB-trees/red-black trees is that a tree can be updated with only a constant number of rotations per insertion or deletion. This property is important for example when maintaining priority search trees [56] where each rotation requires (log n) time. The rst binary search tree with O(1) rotations per update was the half-balanced trees by Olivi e [66]. Olivi es idea was to use path-balancing, where the quotient between the shortest and longest path from each node is restricted to be at most 1/2, and he showed that this path-balance could be maintained with O(1) rotations per update. It turns out to be the case that half-balanced trees and SBB/red-black trees are structurally equivalent, although their maintenance algorithms are dierent. It has also been proven by Tarjan [73] that SBB/red-black trees can be maintained by O(1) rotations. These algorithms can also be generalized to maintain pseudo-nodes of higher degree, resulting in binary B-tree implementations with lower height [8], still requiring O(1) rotations per update. The mechanism behind the constant number of rotations per update can be explained in a simple way by examining three cases of what can happen during insertion and deletion in a binary B-tree representation. When a pseudo-node becomes too large, it can be split into two pseudo-nodes without any rotation; we just need to change the balance information. Also, when a pseudo-node becomes too small and its sibling has minimal size, these two nodes can be merged without any rotation; we just change balance information. In all other cases, when a pseudo-node becomes too small or too large, this will be resolved by moving nodes between the pseudo-node and its sibling and no splitting or merging will take place. From these three basic facts, it can be shown that as soon as the third case above occurs, no more rebalancing will be done during the same update. Hence, the third case, requiring rotations, will only occur once per update. For details, we refer to the literature [8, 73]. Binary B-trees can also be used to design very simple maintenance algorithms that are

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-11

easy to code. This is illustrated by AA-trees [5, 77]. AA-trees are actually the same as Bayers binary version of 2-3-trees, but with design focused on simplicity. Compared with normal red-black tree implementations, AA-trees require very few dierent cases in the algorithm and much less code for implementation. While binary B-trees and SBB/red-black trees deal with small pseudo-nodes, the stratied trees by van Leeuwen and Overmars [76] use large pseudo-nodes arranged in few layers. The concept of stratication does not imply that all pseudo-nodes have similar size; it is mainly a way to conceptually divide the tree into layers, using the notion of merging and splitting.

10.5

Rebalancing a Tree to Perfect Balance

A basic operation is the rebalancing operation, which takes a binary tree as input and produces a balanced tree. This operation is important in itself, but it is also used as a subroutine in balancing schemes (see Section 10.6). It is quite obvious that one can construct a perfectly balanced tree from an ordered tree, or a sorted list, in linear time. The most straightforward way is to put the elements in sorted order into an array, take the median as the root of the tree, and construct the left and right subtrees recursively from the upper and lower halves of the array. However, this is unnecessarily cumbersome in terms of time, space, and elegance. A number of restructuring algorithms, from the type mentioned above to more elegant and ecient ones based on rotations, can be found in the literature [26, 27, 33, 54, 72]. Of these, the one by Stout and Warren [72] seems to be most ecient. It uses the following principle: 1. Skew. Make right rotations at the root until no left child remains. Continue down the right path making right rotations until the entire tree becomes one long rightmost path (a vine). 2. Split. Traverse down the vine a number of times, each time reducing the length of the vine by left rotations. If we start with a vine of length 2p 1, for some integer p, and make one rotation per visited node, the resulting vine will be of length 2p1 1 after the rst pass, 2p2 1 after the second pass, etc., until the vine is reduced to a single node; the resulting tree is a perfectly balanced tree. If the size of the tree is 2p 1 , this will work without any problem. If, however, the size is not a power of two, we have to make some special arrangements during the rst pass of left rotations. Stout and Warren solved the problem of how to make evenly distributed rotations along the vine in a rather complicated way, but there is a simpler one. It has never before been published in itself, but has been included in demo software and in published code [6, 11]. The central operation is a split operation that takes as parameters two numbers p1 and p2 and compresses a right-skewed path of p1 nodes into a path of p2 nodes (2p2 p1 ). The simple idea is to use a counter stepping from p1 p2 to p2 (p1 p2 ) with increment p1 p2 . Every time this counter reaches or exceeds a multiple of p2 , a rotation is performed. In eect, the operation will make p1 p2 evenly distributed left rotations. With this split operation available, we can do as follows to rebalance a tree of size n (n internal nodes): First, skew the tree. Next, nd the largest integer b such that b is an even power of 2 and b 1 n. Then, if b 1 < n, call Split with parameters n and b 1. Now, the vine will have proper length and we can traverse it repeatedly, making a left rotation at each visited node, until only one node remains.

2005 by Chapman & Hall/CRC

10-12

Handbook of Data Structures and Applications

In contrast to the Stout-Warren algorithm, this algorithm is straightforward to implement. We illustrate it in Figure 10.4. We describe the ve trees, starting with the topmost: 1. A tree with 12 internal nodes to be balanced. 2. After Skew. 3. With n = 12 and b = 8, we call split with parameters 12 and 7, which implies that ve evenly distributed rotations will be made. As the result, the vine will be of length 7, which fullls the property of being 2p 1. 4. The next split can be done by traversing the vine, making one left rotation at each node. As a result, we get a vine of length 3 (nodes 3, 6, and 10). 5. After the nal split, the tree is perfectly balanced.

10.6

Schemes with no Balance Information

As discussed above, a balanced binary search tree is typically maintained by local constraints on the structure of the tree. By keeping structure information in the nodes, these constraints can be maintained during updates. In this section, we show that a plain vanilla tree, without any local balance information, can be maintained eciently. This can be done by coding the balance information implicitly (Section 10.6.1) or by using global instead of local balance criteria, hereby avoiding the need for balance information (Section 10.6.2). Splay trees [70] also have no balance information. They do not have a sub-linear bound on their height, but still perform searches in amortized O(log n) time. Splay trees are described in a chapter of their own in this book.

10.6.1

Implicit Representation of Balance Information

One idea of how to remove the need for local balance information is to store the information implicitly. There are two main techniques for this: coding information in the way empty pointers are located or coding information by changing the order between left and right children. In both cases, we can easily code one bit implicitly at each internal node, but not at external nodes. Therefore, we weed to use balance schemes that can do with only one bit per internal node and no balance information at external nodes. As an example, we may use the AVL-tree. At each node, we need to keep track of whether the two subtrees have the same height or if one of them is one unit higher than its sibling. We can do this with one bit per internal node by letting the bit be 1 if and only if the node is higher than its sibling. For external nodes we know the height, so no balance information is needed there. The assumption that we only need one bit per internal node is used in the two constructions below.
Using Empty Pointers

As pointed out by Brown [24, 25], the explicitly stored balance information may in some classes of balanced trees be eliminated by coding the information through the location of empty pointers. We use a tree of pseudo-nodes, where a pseudo-node contains two consecutive elements, stored in two binary nodes. The pseudo-node will have three outgoing pointers, and since the two binary nodes are consecutive, one of the three pointers will be

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

8 3 1 2 4 6 5 1 7 10 9 11 12

10-13

10

11

12

1 3 2 4 5 6 8 7 9 11 10 12

3 1 2 4 7 5 8 9 11 6 10 12

6 3 1 2 4 5 7 8 9 11 10 12

FIGURE 10.4: Rebalancing a binary search tree.

empty. By varying which of the two nodes become parent, we can arrange the pseudonode in two ways. These two dierent structures is used to represent bit values 0 and 1, respectively; by checking the position of the empty pointer, we can compute the bit value. In order for this to work, we allow the pseudo-nodes at the bottom of the tree to contain one or two binary nodes. During insertion, we traverse down the tree. If the inserted element lies between the two keys in a visited pseudo-node, we replace it by one of the elements in the pseudo-node and

2005 by Chapman & Hall/CRC

10-14

Handbook of Data Structures and Applications

continue down the tree with that element instead. At the bottom of the tree, if we nd a pseudo-node with only one key, we just add the new key. If, on the other hand, we nd a pseudo-node with two keys, we split it into two pseudo-nodes which will cause an insertion in the tree of pseudo-nodes. Rotations etc. can be done with pseudo-nodes instead of ordinary binary nodes. (If a rotation involves the lowest level of the tree of pseudo-nodes, some care has to be taken in order to maintain the invariant that only the lowest pseudo-nodes may contain a single node.) Deletions are handled correspondingly. If the deleted element is contained in an internal pseudo-node, we replace it by its predecessor or successor, which resides at the bottom of the tree; in this way we ensure that the deletion occurs at the bottom. If the deletion occurs at a pseudo-node with two binary nodes, we just remove the node, if the pseudo-node contains only one node, a deletion occurs in the tree of pseudo-nodes. Despite the pseudo-nodes, the tree is really just a binary search tree where no balance information is explicitly stored. Since each pseudo-node has internal height 2, and the number of pseudo-nodes is less than n, the height of the binary tree is O(log n). A drawback is that the height of the underlying binary tree will become higher by the use of pseudonodes. Instead of n internal nodes we will have roughly n/2 pseudo-nodes, each of height 2. In the worst case, the height of the binary tree will be doubled.
Swapping Pointers

Another possibility for coding information into a structure is to use the ordering of nodes. If we redene binary search trees, such that the left and right subtree of a node are allowed to change place, we can use this possibility to encode one bit per node implicitly. By comparing the keys of the two children of a node, the one-bit information can be extracted. During search, we have to make one comparison extra at each node. This idea has been used by Munro and Suwanda [5961] to achieve implicit implementation of binary search trees, but it can of course also be used for traditional pointer-based tree structures.

10.6.2

General Balanced Trees

In the following, we use |T | to denote the weight (number of leaves) in a tree T . We also use |v | to denote the weight of a subtree rooted at node v . It should be noted that for a tree T storing n keys in internal nodes, |T | = n + 1 Instead of coding balance information into the structure of the tree, we can let the tree take any shape, as long as its height is logarithmic. Then, there is no local balance criterion to maintain, and we need no balance information in the nodes, not even implicitly coded. As we show below, the tree can still be maintained eciently. When maintaining trees this way, we use the technique of partial rebuilding. This technique was rst introduced by Overmars and van Leeuwen [68, 69] for maintaining weightbalanced trees. By making a partial rebuilding at node v , we mean that the subtree rooted at v is rebuilt into a perfectly balanced tree. The cost of such rebalancing is (|v |). In Section 10.5, we discuss linear time algorithms for rebalancing a (sub-)tree to perfect balance. Apart from the advantage of requiring no balance information in the nodes, it can be shown [7] that the constant factor for general balanced trees is lower than what has been shown for the maintenance of weight-balanced trees by partial rebuilding. The main idea in maintaining a general balanced tree is to let the tree take any shape as long as its height does not exceed log |T | by more than a specied constant factor. The key observation is that whenever the tree gets too high by an insertion, we can nd a node where partial rebuilding can be made at a low amortized cost. (Since deletions do not

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-15

increase the height of the tree, we can handle deletions eciently by rebuilding the entire tree after a large number of elements have been deleted.) We use two constants c > 1, and b > 0, and we maintain a balanced tree T with maximum height c log |T | + b . No balance information is used, except two global integers, containing |T |, the number of leaves in T , and d(T ), the number of deletions made since the last time the entire tree T was rebalanced. Updates are performed in the following way: Insertion: If the depth of the new leaf exceeds c log(|T | + d(T )) , we back up along the insertion path until we nd the lowest node v , such that h(v ) > c log |v | . The subtree v is then rebuilt to perfect balance. The node v is found by explicitly traversing the subtrees below the nodes on the path from the inserted leaf to v , while counting the number of leaves. The cost for this equals the cost for traversing the subtree below v once, which is O(|v |). Deletion: d(T ) increases by one. If d(T ) (2b/c 1)|T |, we rebuild T to perfect balance and set d(T ) = 0. First, we show that the height is low enough. Since deletions do not increase the height of T , we only need to show that the height is not increased too much by an insertion. We prove this by induction. Assume that h(T ) c log(|T | + d(T )) (10.1)

holds before an insertion. (Note that the height of an empty tree is zero.) During the insertion, the height condition can only be violated by the new node. However, if such a violation occurs, the partial rebuilding will ensure that Inequality 10.1 holds after the insertion. Hence, Inequality 10.1 holds by induction. Combining this with the fact that d(T ) < (2b/c 1)|T |, we get that h(T ) c log |T | + b . Next, we show that the maintenance cost is low enough. Since the amortized cost for the rebuilding of the entire tree caused by deletions is obviously O(1) per deletion, we only need to consider insertions. In fact, by the way we choose where to perform rebuilding, we can guarantee that when a partial rebuilding occurs at node v , (v ) updates have been made below v since the last time v was involved in a partial rebuilding. Indeed, this observation is the key observation behind general balanced trees. Let vH be v s child on the path to the inserted node. By the way v is selected by the algorithm, we know the following about v and vh : h(v ) > h(vH ) h(v ) = Combining these, we get c log |v | < h(v ) = h(vH ) + 1 c log |vH | + 1 and, thus log |v | < |vH | > log |vH | + 1/c 21/c |v | (10.6) (10.5) c log |v | c log |vH | h(vh ) + 1 (10.2) (10.3) (10.4)

2005 by Chapman & Hall/CRC

10-16
8 5 1 4 3 2 6 7 9

Handbook of Data Structures and Applications

16 14 15 12 11 10 13 17 18 19

8 5 1 4 3 2 6 9 7 10 11 13 12 14 15 17 18 16 19

FIGURE 10.5: Upper tree: A GB(1.2)-tree which requires rebalancing. Lower tree: After partial rebuilding.

Since 21/c > 1/2, we conclude that the weight of vH is (v ) larger than the weight of v s other child. The only way this dierence in weight between the two children can occur is by insertions or deletion below v . Hence, (v ) updates must have been made below v since the last time v was involved in a partial rebuilding. In order for the amortized analysis to hold, we need to reserve a constant cost at v for each update below v . At each update, updates are made below O(log n) nodes, so the total reservation per update is O(log n). Since the tree is allowed to take any shape as long as its height is low enough, we call this type of balanced tree general balanced trees [7]. We use the notation GB-trees or GB(c)trees, where c is the height constant above. (The constant b is omitted in this notation.) (The idea of general balanced trees have also been rediscovered under the name scapegoat trees [33].) Example. The upper tree in Figure 10.5 illustrates a GB(1.2)-tree where ve deletions and some insertions have been made since the last global rebuilding. When inserting 10, the height becomes 7, which is too high, since 7 > c log(|T | + d(T )) = 1.2 log(20+5) = 6. We back up along the path until we nd the node 14. The height of this node is 5 and the weight is 8. Since 5 > 1.2 log 8 , we can make a partial rebuilding at that node. The resulting tree is shown as the lower tree in Figure 10.5.

10.6.3

Application to Multi-Dimensional Search Trees

The technique of partial rebuilding is an attractive method in the sense that it is useful not only for ordinary binary search trees, but also for more complicated data structures, such as multi-dimensional search trees, where rotations cannot be used eciently. For example, partial rebuilding can be used to maintain logarithmic height in k -d trees [19]

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-17

under updates [57, 68, 69]. A detailed study of the use of partial rebuilding can be found in Mark Overmars Ph.D. thesis [68]. For the sake of completeness, we just mention that if the cost of rebalancing a subtree v is O(P (|v |)), the amortized cost of an update will be O
2 P (n) n

log n . For example, applied to k -d trees, we get an amortized update cost of

O(log n).

10.7

Low Height Schemes

Most rebalancing schemes reproduce the result of AVL-trees [1] in the sense that they guarantee a height of c log(n) for some constant c > 1, while doing updates in O(log n) time. Since the height determines the worst-case complexity of almost all operations, it may be reasonable to ask exactly how close to the best possible height log(n + 1) a tree can be maintained during updates. Presumably, the answer depends on the amount of rebalancing work we are willing to do, so more generally the question is: given a function f , what is the best possible height maintainable with O(f (n)) rebalancing work per update? This question is of practical interestin situations where many more searches than updates are performed, lowering the height by factor of (say) two will improve overall performance, even if it is obtained at the cost of a larger update time. It is also of theoretical interest, since we are asking about the inherent complexity of maintaining a given height in binary search trees. In this section, we review the existing answers to the question. Already in 1976, Maurer et al. [55] proposed the k -neighbor trees, which guarantee a height of c log(n), where c can be chosen arbitrarily close to one. These are unarybinary trees, with all leaves having the same depth and with the requirement that between any two unary nodes on the same level, at least k 1 binary nodes appear. They may be viewed as a type of (1, 2)-trees where the rebalancing operations exchange children, not only with neighboring nodes (as in standard (a, b)-tree or B -tree rebalancing), but with nodes a horizontal distance k away. Since at each level, at most one out of k nodes is unary, the number of nodes increases by a factor of (2(k 1) + 1)/k = 2 1/k for each level. This implies a height bound of log21/k n = log(n)/ log(2 1/k ). By rst order approximation, log(1 + x) = (x) and 1/(1 + x) = 1 (x) for x close to zero, so 1/ log(2 1/k ) = 1/(1 + log(1 1/2k )) = 1 + (1/k ). Hence, k -trees maintain a height of (1 + (1/k )) log n in time O(k log n) per update. Another proposal [8] generalizes the red-black method of implementing (2, 4)-trees as binary trees, and uses it to implement (a, b)-trees as binary trees for a = 2k and b = 2k+1 . Each (a, b)-tree node is implemented as a binary tree of perfect balance. If the underlying (a, b)-tree has t levels, the binary tree has height at most t(k +1) and has at least (2k )t = 2kt nodes. Hence, log n tk , so the height is at most (k + 1)/k log n = (1 + 1/k ) log n. As in red-black trees, a node splitting or fusion in the (a, b)-tree corresponds to a constant amount of recoloring. These operations may propagate along the search path, while the remaining rebalancing necessary takes place at a constant number of (a, b)-tree nodes. In the binary formulation, these operations involve rebuilding subtrees of size (2k ) to perfect balance. Hence, the rebalancing cost is O(log(n)/k + 2k ) per update. Choosing k = log log n gives a tree with height bound log n + log(n)/ log log(n) and update time O(log n). Note that the constant for the leading term of the height bound is now one. To accommodate a non-constant k , the entire tree is rebuilt when log log n changes. Amortized this is O(1) work, which can be made a worst case bound by using incremental rebuilding [68]. Returning to k -trees, we may use the method of non-constant k also there. One possibility

2005 by Chapman & Hall/CRC

10-18

Handbook of Data Structures and Applications

is k = (log n), which implies a height bound as low as log n + O(1), maintained with O(log2 n) rebalancing work per update. This height is O(1) from the best possible. A similar result can be achieved using the general balanced trees described in Section 10.6: In the proof of complexity in that section, the main point is that the cost |v | of a rebuilding at a node v can be attributed to at least (21/c 1/2)|v | updates, implying that each update is attributed at most (1/(21/c 1/2)) cost at each of the at most O(log n) nodes on the search path. The rebalancing cost is therefore O(1/(21/c 1/2) log n) for maintaining height c log n. Choosing c = 1 + 1/ log n gives a height bound of log n + O(1), maintained in O(log2 n) amortized rebalancing work per update, since (21/(1+1/ log n)) 1/2) can be shown to be (1/ log n) using the rst order approximations 1/(1 + x) = 1 (x) and 2x = 1 + (x) for x close to zero. We note that a binary tree with a height bound of log n + O(1) in a natural way can be embedded in an array of length O(n): Consider a tree T with a height bound of log n + k for an integer k , and consider n ranging over the interval [2i ; 2i+1 [ for an integer i. For n in this interval, the height of T never exceeds i + k , so we can think of T as embedded in a virtual binary tree T with i + k completely full levels. Numbering nodes in T by an in-order traversal and using these numbers as indexes in an array A of size 2i+k 1 gives an embedding of T into A. The keys of T will appear in sorted order in A, but empty array entries may exist between keys. An insertion into T which violates the height bound corresponds to an insertion into the sorted array A at a non-empty position. If T is maintained by the algorithm based on general balanced trees, rebalancing due to the insertion consists of rebuilding some subtree in T to perfect balance, which in A corresponds to an even redistribution of the elements in some consecutive segment of the array. In particular, the redistribution ensures an empty position at the insertion point. In short, the tree rebalancing algorithm can be used as a maintenance algorithm for a sorted array of keys supporting insertions and deletions in amortized O(log2 n) time. The requirement is that the array is never lled to more than some xed fraction of its capacity (the fraction is 1/2k1 in the example above). Such an amortized O(log2 n) solution, phrased directly as a maintenance algorithm for sorted arrays, rst appeared in [38]. By the converse of the embedding just described, [38] implies a rebalancing algorithm for low height trees with bounds as above. This algorithm is similar, but not identical, to the one arising from general balanced trees (the criteria for when to rebuild/redistribute are similar, but dier in the details). A solution to the sorted array maintenance problem with worst case O(log2 n) update time was given in [78]. Lower bounds for the problem appear in [28, 29], with one of the bounds stating that for algorithms using even redistribution of the elements in some consecutive segment of the array, O(log2 n) time is best possible when the array is lled up to some constant fraction of its capacity. We note that the correspondence between the tree formulation and the array formulation only holds when using partial rebuilding to rebalance the treeonly then is the cost of the redistribution the same in the two versions. In contrast, a rotation in the tree will shift entire subtrees up and down at constant cost, which in the array version entails cost proportional to the size of the subtrees. Thus, for pointer based implementation of trees, the above (log2 n) lower bound does not hold, and better complexities can be hoped for. Indeed, for trees, the rebalancing cost can be reduced further. One method is by applying the idea of bucketing : The subtrees on the lowest (log K ) levels of the tree are changed into buckets holding (K ) keys. This size bound is maintained by treating the buckets as (a, b)-tree nodes, i.e., by bucket splitting, fusion, and sharing. Updates in the top tree only happen when a bucket is split or fused, which only happens for every (K ) updates in the bucket. Hence, the amortized update time for the top tree drops by a factor K . The buckets themselves can be implemented as well-balanced binary treesusing the schemes

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-19

above based on k -trees or general balanced trees for both top tree and buckets, we arrive at a height bound of log n + O(1), maintained with O(log log2 n) amortized rebalancing work. Applying the idea recursively inside the buckets will improve the time even further. This line of rebalancing schemes was developed in [3, 4, 9, 10, 42, 43], ending in a scheme [10] maintaining height log(n + 1) + 1 with O(1) amortized rebalancing work per update. This rather positive result is in contrast to an observation made in [42] about the cost of maintaining exact optimal height log(n + 1) : When n = 2i 1 for an integer i, there is only one possible tree of height log(n + 1) , namely a tree of i completely full levels. By the ordering of keys in a search tree, the keys of even rank are in the lowest level, and the keys of odd rank are in the remaining levels (where the rank of a key k is dened as the number of keys in the tree that are smaller than k ). Inserting a new smallest key and removing the largest key leads to a tree of same size, but where all elements previously of odd rank now have even rank, and vice versa. If optimal height is maintained, all keys previously in the lowest level must now reside in the remaining levels, and vice versain other words, the entire tree must be rebuilt. Since the process can be repeated, we obtain a lower bound of (n), even with respect to amortized complexity. Thus, we have the intriguing situation that a height bound of log(n +1) has amortized complexity (n) per update, while raising the height bound a trie to log(n + 1) + 1 reduces the complexity to (1). Actually, the papers [3, 4, 9, 10, 42, 43] consider a more detailed height bound of the form log(n + 1) + , where is any real number greater than zero. For less than one, this expression is optimal for the rst integers n above 2i 1 for any i, and optimal plus one for the last integers before 2i+1 1. In other words, the smaller an , the closer to the next power of two is the height guaranteed to be optimal. Considering tangents to the graph of the logarithm function, it is easily seen that is proportional to the fraction of integers n for which the height is non-optimal. Hence, an even more detailed formulation of the question about height bound versus rebalancing work is the following: Given a function f , what is the smallest possible such that the height bound log(n + 1) + is maintainable with O(f (n)) rebalancing work per update? In the case of amortized complexity, the answer is known. In [30], a lower bound is given, stating that no algorithm using o(f (n)) amortized rebuilding work per update can guarantee a height of log(n + 1) + 1/f (n) for all n. The lower bound is proved by mapping trees to arrays and exploiting a fundamental lemma on density from [28]. In [31], a balancing scheme was given which maintains height log(n + 1) + 1/f (n) in amortized O(f (n)) time per update, thereby matching the lower bound. The basic idea of the balancing scheme is similar to k -trees, but a more intricate distribution of unary nodes is used. Combined, these results show that for amortized complexity, the answer to the question above is (n) (1/f (n)). We may view this expression as describing the inherent amortized complexity of rebalancing a binary search tree, seen as a function of the height bound maintained. Using the observation above that for any i, log(n + 1) + is equal to log(n + 1) for n from 2i 1 to (1 ())2i+1 , the result may alternatively be viewed as the cost of maintaining optimal height when n approaches the next power of two: for n = (1 )2i+1 , the cost is (1/). A graph depicting this cost appears in Figure 10.6. This result holds for the fully dynamic case, where one may keep the size at (1 )2i+1 by alternating between insertions and deletions. In the semi-dynamic case where only insertions take place, the amortized cost is smalleressentially, it is the integral of the function in Figure 10.6, which gives (n log n) for n insertions, or (log n) per insertion.

2005 by Chapman & Hall/CRC

10-20

Handbook of Data Structures and Applications

Rebalancing Cost Per Update

60 50 40 30 20 10 0 0 10 20 30 40 Size of Tree 50 60 70

FIGURE 10.6: The cost of maintaining optimal height as a function of tree size.

More concretely, we may divide the insertions causing n to grow from 2i to 2i+1 into i segments, where segment one is the rst 2i1 insertions, segment two is the next 2i2 insertions, and so forth. In segment j , we employ the rebalancing scheme from [31] with f (n) = (2j ), which will keep optimal height in that segment. The total cost of insertions is O(2i ) inside each of the i segments, for a combined cost of O(i2i ), which is O(log n) amortized per insertion. By the same reasoning, the lower bound from [30] implies that this is best possible for maintaining optimal height in the semi-dynamic case. Considering worst case complexity for the fully dynamic case, the amortized lower bound stated above of course still applies. The best existing upper bound is height log(n + 1) + min{1/ f (n), log(n)/f (n)} , maintained in O(f (n)) worst case time, by a combination of results in [4] and [30]. For the semi-dynamic case, a worst case cost of (n) can be enforced when n reaches a power of two, as can be seen by the argument above on odd and even ranks of nodes in a completely full tree.

10.8

Relaxed Balance

In the classic search trees, including AVL-trees [1] and red-black trees [34], balancing is tightly coupled to updating. After an insertion or deletion, the updating procedure checks to see if the structural invariant is violated, and if it is, the problem is handled using the balancing operations before the next operation may be applied to the tree. This work is carried out in a bottom-up fashion by either solving the problem at its current location using rotations and/or adjustments of balance variables, or by carrying out a similar operation which moves the problem closer to the root, where, by design, all problems can be solved. In relaxed balancing, the tight coupling between updating and balancing is removed. Basically, any restriction on when rebalancing is carried out and how much is done at a time is removed, except that the smallest unit of rebalancing is typically one single or double rotation. The immediate disadvantage is of course that the logarithmic height guarantee disappears, unless other methods are used to monitor the tree height. The advantage gained is exibility in the form of extra control over the combined process of updating and balancing. Balancing can be turned o during periods with frequent searching and updating (possibly from an external source). If there is not too much correlation between updates, the tree would likely remain fairly balanced during that time. When the frequency drops, more time can be spend on balancing. Furthermore, in multi-processor

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-21

environments, balancing immediately after an update is a problem because of the locking strategies with must be employed. Basically, the entire search path must be locked because it may be necessary to rebalance all the way back up to the root. This problem is discussed as early as in [34], where top-down balancing is suggested as a means of avoiding having to traverse the path again bottom-up after an update. However, this method generally leads to much more restructuring than necessary, up to (log n) instead of O(1). Additionally, restructuring, especially in the form of a sequence of rotations, is generally signicantly more time-consuming than adjustment of balance variables. Thus, it is worth considering alternative solutions to this concurrency control problem. The advantages outlined above are only fully obtained if balancing is still ecient. That is the challenge: to dene balancing constraints which are exible enough that updating without immediate rebalancing can be allowed, yet at the same time suciently constrained that balancing can be handled eciently at any later time, even if path lengths are constantly super-logarithmic. The rst partial result, dealing with insertions only, is from [41]. Below, we discuss the results which support insertion as well as deletion.

10.8.1

Red-Black Trees

In standard red-black trees, the balance constraints require that no two consecutive nodes are red and that for any node, every path to a leaf has the same number of black nodes. In the relaxed version, the rst constraint is abandoned and the second is weakened in the following manner: Instead of a color variable, we use an integer variable, referred to as the weight of a node, in such a way that zero can be interpreted as red and one as black. The second constraint is then changed to saying that for any node, every path to a leaf has the same sum of weights. Thus, a standard red-black tree is also a relaxed tree; in fact, it is the ideal state of a relaxed tree. The work on red-black trees with relaxed balance was initiated in [64, 65]. Now, the updating operations must be dened so that an update can be performed in such a way that updating will leave the tree in a well-dened state, i.e., it must be a relaxed tree, without any subsequent rebalancing. This can be done as shown in Fig. 10.7. The operations are from [48]. The trees used here, and depicted in the gure, are assumed to be leaf-oriented. This terminology stems from applications where it is convenient to treat the external nodes dierently from the remaining nodes. Thus, in these applications, the external nodes are not empty trees, but real nodes, possibly of another type than the internal nodes. In database applications, for instance, if a sequence of sorted data in the form of a linked list is already present, it is often desirable to build a tree on top of this data to facilitate faster searching. In such cases, it is often convenient to allow copies of keys from the leaves to also appear in the tree structure. To distinguish, we then refer to the key values in the leaves as keys, and refer to the key values in the tree structure as routers, since they merely guide the searching procedure. The ordering invariant is then relaxed, allowing keys in the left subtree of a tree rooted by u to be smaller than or equal to u.k , and the size of the tree is often dened as the number of leaves. When using the terminology outlined here, we refer to the trees as leaf-oriented trees. The balance problems in a relaxed tree can now be specied as the relations between balance variables which prevent the tree from being a standard red-black tree, i.e., consecutive red nodes (nodes of weight zero) and weights greater than one. Thus, the balancing scheme must be targeted at removing these problems. It is an important feature of the design that the global constraint on a standard red-black tree involving the number of black nodes is

2005 by Chapman & Hall/CRC

10-22

Handbook of Data Structures and Applications

FIGURE 10.7: Update operations.

not lost after an update. Instead, the information is captured in the second requirement and as soon as all weight greater than one has been removed, the standard constraint holds again. The strategy for the design of balancing operations is the same as for the classical search trees. Problems are removed if this is possible, and otherwise, the problem is moved closer to the root, where all problems can be resolved. In Fig. 10.8, examples are shown of how consecutive red nodes and weight greater than one can be eliminated, and in Fig. 10.9, examples are given of how these problems may be moved closer to the root, in the case where they cannot be eliminated immediately.

FIGURE 10.8: Example operations eliminating balance problems.

FIGURE 10.9: Example operations moving balance problems closer to the root.

It is possible to show complexity results for relaxed trees which are similar to the ones which can be obtained in the classical case. A logarithmic bound on the number of balancing operations required to balance the tree in response to an update was established in [23]. Since balancing operations can be delayed any amount of time, the usual notion of n as the number of elements in the tree at the time of balancing after an update is not really meaningful, so the bound is logarithmic in N , which is the maximum number of elements in the tree since it was last in balance. In [22], amortized constant bounds were obtained and in [45], a version is presented which has fewer and smaller operations, but meets the same bounds. Also, restructuring of the tree is worst-case constant per update. Finally, [48] extends the set of operations with a group insertion, such that an entire search tree

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-23

can be inserted in between two consecutive keys in amortized time O(log m), where m is the size of the subtree. The amortized bounds as well as the worst case bounds are obtained using potential function techniques [74]. For group insertion, the results further depend on the fact that trees with low total potential can build [40], such that the inserted subtree does not increase the potential too dramatically.

10.8.2

AVL-Trees

The rst relaxed version of AVL-trees [1] is from [63]. Here, the standard balance constraint of requiring that the heights of any two subtrees dier by at most one is relaxed by introducing a slack parameter, referred to as a tag value. The tag value, tu , of any node u must be an integer greater than or equal to 1, except that the tag value of a leaf must be greater than or equal to zero. The constraint that heights may dier by at most one is then imposed on the relaxed height instead. The relaxed height rh(u) of a node u is dened as rh(u) = tu , max(rh(u.l), rh(u.r)) + 1 + tu , if u is a leaf otherwise

As for red-black trees, enough exibility is introduced by this denition that updates can be made without immediate rebalancing while leaving the tree in a well-dened state. This can be done by adjusting tag values appropriately in the vicinity of the update location. A standard AVL-tree is the ideal state of a relaxed AVL-tree, which is obtained when all tag values are zero. Thus, a balancing scheme aiming at this is designed. In [44], it is shown that a scheme can be designed such that the complexities from the sequential case are met. Thus, only a logarithmic number of balancing operations must be carried out in response to an update before the tree is again in balance. As opposed to red-black trees, the amortized constant rebalancing result does not hold in full generality for AVL-trees, but only for the semi-dynamic case [58]. This result is matched in [46]. A dierent AVL-based version was treated in [71]. Here, rotations are only performed if the subtrees are balanced. Thus, violations of the balance constraints must be dealt with bottom-up. This is a minimalistic approach to relaxed balance. When a rebalancing operation is carried out at a given node, the children do not violate the balance constraints. This limits the possible cases, and is asymptotically as ecient as the structure described above [52, 53].

10.8.3

Multi-Way Trees

Multi-way trees are usually described either as (a, b)-trees or B -trees, which are treated in another chapter of this book. An (a, b)-tree [37, 57] consists of nodes with at least a and at most b children. Usually, it is required that a 2 to ensure logarithmic height, and in order to make the rebalancing scheme work, b must be at least 2a 1. Searching and updating including rebalancing is O(log a n). If b 2a, then rebalancing becomes amortized O(1). The term B -trees [17] is often used synonymously, but sometimes refers to the variant where b = 2a 1 or the variant where b = 2a. For (a, b)-trees, the standard balance constraints for requiring that the number of children of each node is between a and b and that every leaf is at the same depth are relaxed as follows. First, nodes are allowed to have fewer than a children. This makes it possible to perform a deletion without immediate rebalancing. Second, nodes are equipped with a tag value, which is a non-positive integer value, and leaves are only required to have the same

2005 by Chapman & Hall/CRC

10-24

Handbook of Data Structures and Applications

relaxed depth, which is the usual depth, except that all tag values encountered from the root to the node in question are added. With this relaxation, it becomes possible to perform an insertion locally and leave the tree in a well-dened state. Relaxed multi-way trees were rst considered in [63], and complexity results matching the standard case were established in [50]. Variations with other properties can be found in [39]. Finally, a group insertion operation with a complexity of amortized O(loga m), where m is the size of the group, can be added while maintaining the already achieved complexities for the other operations [47, 49]. The amortized result is a little stronger than usual, where it is normally assumed that the initial structure is empty. Here, except for very small values of a and b, zero-potential trees of any size can be constructed such the amortized results starting from such a tree hold immediately [40].

10.8.4

Other Results

Even though there are signicant dierences between the results outlined above, it is possible to establish a more general result giving the requirements for when a balanced search tree scheme can be modied to give a relaxed version with corresponding complexity properties [51]. The main requirements are that rebalancing operations in the standard scheme must be local constant-sized operations which are applied bottom-up, but in addition, balancing operation must also move the problems of imbalance towards the root. See [35] for an example of how these general ideas are expressed in the concrete setting of red-black trees. In [32], it is demonstrated how the ideas of relaxed balance can be combined with methods from search trees of near-optimal height, and [39] contains complexity results made specically for the reformulation of red-black trees in terms of layers based on black height from [67]. Finally, performance results from experiments with relaxed structures can be found in [21, 36].

References
[1] G. M. Adelson-Velski and E. M. Landis. An algorithm for the organisation of information. Doklady Akadamii Nauk SSSR, 146:263266, 1962. In Russian. English translation in Soviet Math. Doklady, 3:12591263, 1962. [2] A. V. Aho, J. E. Hopcroft, and J. D. Ullman. Data Structures and Algorithms. Addison-Wesley, Reading, Massachusetts, 1983. [3] A. Andersson. Optimal bounds on the dictionary problem. In Proceeding of the Symposium on Optimal Algorithms, volume 401 of Lecture Notes in Computer Science, pages 106114. Springer-Verlag, 1989. [4] A. Andersson. Ecient Search Trees. PhD thesis, Department of Computer Science, Lund University, Sweden, 1990. [5] A. Andersson. Balanced search trees made simple. In Proceedings of the Third Workshop on Algorithms and Data Structures, volume 709 of Lecture Notes in Computer Science, pages 6071. Springer-Verlag, 1993. [6] A. Andersson. A demonstration of balanced trees. Algorithm animation program (for macintosh) including users guide. Can be downloaded from authors homepage, 1993. [7] A. Andersson. General balanced trees. Journal of Algorithms, 30:128, 1999. [8] A. Andersson, C. Icking, R. Klein, and T. Ottmann. Binary search trees of almost optimal height. Acta Informatica, 28:165178, 1990. [9] A. Andersson and T. W. Lai. Fast updating of well-balanced trees. In Proceedings of

2005 by Chapman & Hall/CRC

Balanced Binary Search Trees

10-25

[10]

[11] [12] [13]

[14] [15] [16] [17] [18] [19] [20] [21]

[22]

[23] [24] [25] [26] [27] [28]

[29]

the Second Scandinavian Workshop on Algorithm Theory, volume 447 of Lecture Notes in Computer Science, pages 111121. Springer-Verlag, 1990. A. Andersson and T. W. Lai. Comparison-ecient and write-optimal searching and sorting. In Proceedings of the Second International Symposium on Algorithms, volume 557 of Lecture Notes in Computer Science, pages 273282. Springer-Verlag, 1991. A. Andersson and S. Nilsson. An ecient list implementation. In JavaOne Conference, 1999. Code can be found at Stefan Nilssons home page. Arne Andersson. A note on searching in a binary search tree. Software Practice & Experience, 21(10):11251128, 1991. Arne A. Andersson and Mikkel Thorup. Tight(er) worst-case bounds on dynamic searching and priority queues. In Proceedings of the Thirty Second Annual ACM Symposium on Theory of Computing, pages 335342. ACM Press, 2000. Lars Arge and Jerey Scott Vitter. Optimal external memory interval management. SIAM Journal on Computing, 32(6):14881508, 2003. R. Bayer. Binary B-trees for virtual memory. In Proceedings of the ACM SIGIFIDET Workshop on Data Description, Access and control, pages 219235, 1971. R. Bayer. Symmetric binary B-trees: Data structure and maintenance algorithms. Acta Informatica, 1(4):290306, 1972. R. Bayer and E. McCreight. Organization and maintenance of large ordered indexes. Acta Informatica, 1:173189, 1972. Paul Beame and Faith E. Fich. Optimal bounds for the predecessor problem and related problems. Journal of Computer and System Sciences, 65(1):3872, 2002. J. L. Bentley. Multidimensional binary search trees used for associative searching. Communications of the ACM, 18(9):509517, 1975. N. Blum and K. Mehlhorn. On the average number of rebalancing operations in weight-balanced trees. Theoretical Computer Science, 11:303320, 1980. Luc Boug e, Joaquim Gabarr o, Xavier Messeguer, and Nicolas Schabanel. Concurrent rebalancing of AVL trees: A ne-grained approach. In Proceedings of the Third Annual European Conference on Parallel Processing, volume 1300 of Lecture Notes in Computer Science, pages 421429. Springer-Verlag, 1997. Joan Boyar, Rolf Fagerberg, and Kim S. Larsen. Amortization results for chromatic search trees, with an application to priority queues. Journal of Computer and System Sciences, 55(3):504521, 1997. Joan F. Boyar and Kim S. Larsen. Ecient rebalancing of chromatic search trees. Journal of Computer and System Sciences, 49(3):667682, 1994. M. R. Brown. A storage scheme for height-balanced trees. Information Processing Letters, 7(5):231232, 1978. M. R. Brown. Addendum to A storage scheme for height-balanced trees. Information Processing Letters, 8(3):154156, 1979. H. Chang and S. S. Iynegar. Ecient algorithms to globally balance a binary search tree. Communications of the ACM, 27(7):695702, 1984. A. C. Day. Balancing a binary tree. Computer Journal, 19(4):360361, 1976. P. F. Dietz, J. I. Seiferas, and J. Zhang. A tight lower bound for on-line monotonic list labeling. In Proceedings of the Fourth Scandinavian Workshop on Algorithm Theory, volume 824 of Lecture Notes in Computer Science, pages 131142. SpringerVerlag, 1994. P. F. Dietz and J. Zhang. Lower bounds for monotonic list labeling. In Proceedings of the Second Scandinavian Workshop on Algorithm Theory, volume 447 of Lecture Notes in Computer Science, pages 173180. Springer-Verlag, 1990.

2005 by Chapman & Hall/CRC

10-26

Handbook of Data Structures and Applications

[30] Rolf Fagerberg. Binary search trees: How low can you go? In Proceedings of the Fifth Scandinavian Workshop on Algorithm Theory, volume 1097 of Lecture Notes in Computer Science, pages 428439. Springer-Verlag, 1996. [31] Rolf Fagerberg. The complexity of rebalancing a binary search tree.