Gilles Pesant (Ed.

)
LNCS 9255

Principles and Practice

of Constraint Programming
21st International Conference, CP 2015
Cork, Ireland, August 31 – September 4, 2015
Proceedings

123
Lecture Notes in Computer Science 9255
Commenced Publication in 1973
Founding and Former Series Editors:
Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board
David Hutchison
Lancaster University, Lancaster, UK
Takeo Kanade
Carnegie Mellon University, Pittsburgh, PA, USA
Josef Kittler
University of Surrey, Guildford, UK
Jon M. Kleinberg
Cornell University, Ithaca, NY, USA
Friedemann Mattern
ETH Zurich, Zürich, Switzerland
John C. Mitchell
Stanford University, Stanford, CA, USA
Moni Naor
Weizmann Institute of Science, Rehovot, Israel
C. Pandu Rangan
Indian Institute of Technology, Madras, India
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Demetri Terzopoulos
University of California, Los Angeles, CA, USA
Doug Tygar
University of California, Berkeley, CA, USA
Gerhard Weikum
Max Planck Institute for Informatics, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/7408
Gilles Pesant (Ed.)

Principles and Practice

of Constraint Programming
21st International Conference, CP 2015
Cork, Ireland, August 31 – September 4, 2015
Proceedings

123
Editor
Gilles Pesant
École Polytechnique de Montréal
Montréal, Québec
Canada

ISSN 0302-9743 ISSN 1611-3349 (electronic)

Lecture Notes in Computer Science
ISBN 978-3-319-23218-8 ISBN 978-3-319-23219-5 (eBook)
DOI 10.1007/978-3-319-23219-5

Library of Congress Control Number: 2015946574

Springer Cham Heidelberg New York Dordrecht London

© Springer International Publishing Switzerland 2015
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the
material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book are
believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors
give a warranty, express or implied, with respect to the material contained herein or for any errors or
omissions that may have been made.

Printed on acid-free paper

Springer International Publishing AG Switzerland is part of Springer Science+Business Media

(www.springer.com)
 Preface

This volume contains the proceedings of the 21st International Conference on the
Principles and Practice of Constraint Programming (CP 2015), which was held in Cork,
Ireland, from August 31 to September 4, 2015. Detailed information about the con-
ference is available at http://cp2015.a4cp.org. The CP conference is the annual inter-
national conference on constraint programming. It is concerned with all aspects of
computing with constraints, including theory, algorithms, environments, languages,
models, systems, and applications such as decision making, resource allocation,
scheduling, configuration, and planning.
For the purpose of the conference’s scientific programming, we invited submissions
to the technical, application, and published journal tracks. We received 80, 25, and 14
submissions to these tracks respectively. Authors chose to submit either long (15 pages)
or short (8 pages) papers to the technical and application tracks. The review process for
the technical track relied on a two-level Program Committee and on additional reviewers
recruited by Program Committee members. Each submission to the technical track was
assigned to one member of the Senior Program Committee and three members of the
track’s Program Committee. Submissions to the application track were each assigned to
three members of its Program Committee. Every paper received at least three reviews.
Once the initial reviews were in, authors were given an opportunity to respond before a
detailed discussion was undertaken at the level of the Program Committees, overseen by
the Program Chair and the Senior Program Committee member or the Application Track
Chair, as appropriate. For the first time this year the Senior Program Committee did not
meet in person but deliberated by video conference instead. The published journal track
gives an opportunity to discuss important results in the area of constraint programming
that appeared recently in relevant journals, but had not been previously presented to the
community at conferences. Submissions were evaluated by a separate Program Com-
mittee for relevance and significance.
At the end of the reviewing process, we accepted 39 papers from the technical track,
15 papers from the application track, and 10 papers from the published journal paper
track. The Senior Program Committee awarded the Best Paper Prize to John N. Hooker
for “Projection, Consistency, and George Boole” and the Best Student Paper Prize to
Alexander Ivrii, Sharad Malik, Kuldeep S. Meel, and Moshe Vardi for “On Computing
Minimal Independent Support and Its Applications to Sampling and Counting.” The
Application Track Program Committee awarded the Best Application Paper Prize to
Tommaso Urli and Philip Kilby for “Long-Haul Fleet Mix and Routing Optimisation
with Constraint Programming and Large Neighbourhood Search.” The Program Chair
and the Constraints journal editor-in-chief, Michela Milano, also invited six papers
from the technical and application tracks for direct publication in that journal. These
were presented at the conference like any other paper and they appear in the pro-
ceedings as a one-page abstract.
VI Preface

This edition of the conference was part of George Boole 200, a celebration of the
life and work of George Boole who was born in 1815 and worked at the University
College of Cork. It was also co-located with the 31st International Conference on Logic
Programming (ICLP 2015). The conference program featured three invited talks by
Claire Bagley, Gerhard Friedrich (joint with ICLP), and Douglas R. Smith. This vol-
ume includes one-page abstracts of their talks. The conference also featured four
tutorials and six satellite workshops, whose topics are listed in this volume. The
Doctoral Program gave PhD. students an opportunity to present their work to more
senior researchers, to meet with an assigned mentor for advice on their research and
early career, to attend special tutorials, and to interact with one another. The winners
of the 2015 ACP Research Excellence Award and Doctoral Research Award presented
their award talks. The results of the annual MiniZinc Challenge were announced and a
joint ICLP/CP Programming Competition was held. For the first time an Industry
Outreach effort, headed by Helmut Simonis, provided formal opportunities for CP
researchers and representatives from industry to interact: The former were presented
with industrial problems to solve in a friendly competition and the latter were presented
with samples of industrial problems solved using CP technology. Following the con-
ference, a CSPlib sprint event took place to update this library of constraint satisfaction
problems. All these contributed to a very exciting program.
I am grateful to many people who made this conference such a success. First of all, to
the authors who provided excellent material to select from. Then to the members of the
Program Committees and additional reviewers who worked hard to provide constructive,
high-quality reviews. To members of the Senior Program Committee who helped me
ensure that each paper was adequately discussed, wrote meta-reviews for their assigned
papers, and participated in live remote deliberations — for some, quite early or late in the
day. To Christian Schulte in particular, who agreed to step in where I had a conflict of
interest. Of course there is a whole team standing with me, who chaired various aspects
of the conference: Ken Brown, Barry O’Sullivan (Conference Chairs) and their own local
team (Barry Hurley as webmaster; Kathy Bunney who provided coordinating support; the
technical and administrative support staff of the Insight Centre for Data Analytics at
UCC), Louis-Martin Rousseau (Application Track Chair), Thomas Schiex (Published
Journal Track Chair), David Bergman and Marie Pelleau (Doctoral Program Chairs),
Willem-Jan van Hoeve (Workshop and Tutorial Chair), Helmut Simonis (Industry Out-
reach Chair), and Ian Miguel (Publicity Chair). Thank you for your dedication! I
acknowledge and thank our sponsors for their generous support: they include, at the time
of writing, the Association for Constraint Programming, the Association for Logic Pro-
gramming, ECCAI — the European Coordinating Committee for Artificial Intelligence,
the Insight Centre for Data Analytics, Science Foundation Ireland, Springer, and Uni-
versity College Cork. Finally, I thank the ACP Executive Committee for the trust they
showed me in asking me to serve as Program Chair. It has been an honor.

June 2015 Gilles Pesant

 Tutorials and Workshops

Tutorials

Constraints and Bioinformatics: Results and Challenges

Agostino Dovier
Lagrangian Relaxation for Domain Filtering
Hadrien Cambazard
Towards Embedded Answer Set Solving
Torsten Schaub
XCSP3
Frédéric Boussemart, Christophe Lecoutre, and Cédric Piette

Workshops

Workshop on Constraint-Based Methods in Bioinformatics (WCB 2015)

Alessandro Dal Palù and Agostino Dovier
6th International Workshop on Bin Packing and Placement Constraints (BPPC 2015)
Nicolas Beldiceanu and François Fages
5th International Workshop on the Cross-Fertilization Between CSP and SAT
(CSPSAT 2015)
Yael Ben-Haim, Valentin Mayer-Eichberger, and Yehuda Naveh
14th International Workshop on Constraint Modelling and Reformulation
(ModRef 2015)
Ozgur Akgun and Peter Nightingale
Workshop on Teaching Constraint Programming
Alan Frisch, Ciaran McCreesh, Karen Petrie, and Patrick Prosser
CP and Analytics
Youssef Hamadi and Willem-Jan van Hoeve
 Conference Organization

Conference Chairs
Ken Brown University College Cork, Ireland
Barry O’Sullivan University College Cork, Ireland

Program Chair
Gilles Pesant École Polytechnique de Montréal, Canada

Application Track Chair

Louis-Martin Rousseau École Polytechnique de Montréal, Canada

Published Journal Track Chair

Thomas Schiex INRA Toulouse, France

Doctoral Program Chairs

David Bergman University of Connecticut, USA
Marie Pelleau Université de Montréal, Canada

Workshop and Tutorial Chair

Willem-Jan van Hoeve Carnegie Mellon University, USA

Industry Outreach Chair

Helmut Simonis Insight Centre for Data Analytics, UCC, Ireland

Publicity Chair
Ian Miguel University of St Andrews, UK

Senior Program Committee

Chris Beck University of Toronto, Canada
Nicolas Beldiceanu TASC (CNRS/Inria), Mines Nantes, France
Christian Bessiere CNRS, France
Yves Deville UCLouvain, Belgium
X Conference Organization

Pierre Flener Uppsala University, Sweden

John Hooker Carnegie Mellon University, USA
Peter Jeavons University of Oxford, UK
Christophe Lecoutre CRIL, University of Artois, France
Jimmy Lee The Chinese University of Hong Kong, Hong Kong,
SAR China
Michela Milano DISI Università di Bologna, Italy
Jean-Charles Régin University of Nice Sophia Antipolis / CNRS, France
Christian Schulte KTH Royal Institute of Technology, Sweden
Peter J. Stuckey NICTA and the University of Melbourne, Australia
Pascal Van Hentenryck NICTA and ANU, Australia
Willem-Jan van Hoeve Carnegie Mellon University, USA
Toby Walsh NICTA and UNSW, Australia
Roland Yap National University of Singapore, Singapore

Technical Track Program Committee

Carlos Ansótegui Universitat de Lleida, Spain
Fahiem Bacchus University of Toronto, Canada
Pedro Barahona Universidade Nova de Lisboa, Portugal
Roman Bartak Charles University in Prague, Czech Republic
David Bergman University of Connecticut, USA
Hadrien Cambazard Grenoble INP, CNRS, Joseph Fourier University, France
Hubie Chen Universidad del País Vasco and Ikerbasque, Spain
Geoffrey Chu NICTA and the University of Melbourne, Australia
David Cohen Royal Holloway, University of London, UK
Remi Coletta University of Montpellier, France
Martin Cooper IRIT - Universitíé Paul Sabatier, France
Sophie Demassey CMA, MINES ParisTech, France
François Fages Inria Paris-Rocquencourt, France
Alan Frisch University of York, UK
Maria Garcia De La Monash University, Australia
Banda
Arnaud Gotlieb SIMULA Research Laboratory, Norway
Stefano Gualandi Università di Pavia, Italy
Emmanuel Hebrard LAAS, CNRS, France
Philippe Jégou LSIS - UMR CNRS 7296 - Aix-Marseille University,
France
George Katsirelos INRA, Toulouse, France
Zeynep Kiziltan Università di Bologna, Italy
Lars Kotthoff University of British Columbia, Canada
Philippe Laborie IBM, France
Michele Lombardi DISI Università di Bologna, Italy
Xavier Lorca Ecole des Mines de Nantes, France
Inês Lynce Inst. Superior Técnico INESC-ID Lisboa, Portugal
Arnaud Malapert University of Nice Sophia Antipolis / CNRS, France
 Conference Organization XI

Joao Marques-Silva University College Dublin, Ireland

Ian Miguel University of St. Andrews, UK
Justin Pearson Uppsala University, Sweden
Justyna Petke University College London, UK
Steve Prestwich Insight, UCC, Ireland
Patrick Prosser Glasgow University, UK
Claude-Guy Quimper Université Laval, Canada
Andrea Rendl NICTA and Monash University, Australia
Michel Rueher University of Nice Sophia Antipolis / CNRS, France
Marius Silaghi Florida Institute of Technology, USA
Stephen Smith Carnegie Mellon University, USA
Christine Solnon LIRIS CNRS UMR 5205 / INSA Lyon, France
Kostas Stergiou University of Western Macedonia, Greece
Guido Tack Monash University, Australia
Charlotte Truchet LINA, UMR 6241, Université de Nantes, France
Nic Wilson Insight, UCC, Ireland

Application Track Program Committee

Claire Bagley Oracle Corporation, USA
Thierry Benoist Innovation 24 - LocalSolver, France
Mats Carlsson SICS, Sweden
Jean-Guillaume Fages COSLING S.A.S., France
Carmen Gervet Université de Savoie, LISTIC, France
Laurent Perron Google, France
Ashish Sabharwal AI2, USA
Pierre Schaus UCLouvain, Belgium
Paul Shaw IBM, France
Helmut Simonis Insight, UCC, Ireland
Peter van Beek University of Waterloo, Canada
Mark Wallace Monash University, Australia
Tallys Yunes University of Miami, USA

Published Journal Track Program Committee

Simon de Givry MIAT-INRA, France
Yves Deville UCLouvain, Belgium
John Hooker Carnegie Mellon University, USA
Michela Milano DEIS Università di Bologna, Italy
Nina Narodytska Carnegie Mellon University, USA
Patrick Prosser Glasgow University, UK
Francesca Rossi University of Padova, Italy
Roland Yap Hock Chuan National University of Singapore, Singapore
Invited Talks
 Constraint-based Problems and Solutions
in the Global Enterprise

Claire Bagley

Advanced Constraint Technology

Oracle Corporation, Burlington, MA 01803, USA
{[email protected]}

Oracle is a large global technology organization, whose product offerings have grown
organically and through the acquisition of first class companies. Its leadership has
expanded to the entire technology stack, to span the full range of computer hardware,
operating systems, programming languages, databases, enterprise software, collabora-
tion management tools, and into the cloud.
With thousands of products and tools running the full imaginable breadth and depth
of a technology stack, it is then not surprising that Oracle is exposed to a vast number
of complex combinatorial problems, as well as the different types of technology to
solve them. Indeed, many of the classical applications and variations of constraint
problems are represented: planning, scheduling, rostering, vehicle routing, configura-
tion, networking, grid optimization, logistics, analytics, and cloud management. As
expected with the development of products and the acquisition of companies operating
in these domains, a large number of technologies come into play including Constraint
Programming (CP), Mathematical Programming (MP), local search, heuristics,
knowledge-based reasoning, genetic algorithms, machine learning, and many more.
The Advanced Constraint Technology (ACT) group at Oracle is tasked with
identifying, understanding and solving the complex combinatorial problems that arise
in a diverse field of application environments. Our expertise and industry proven ACT
products are available to assist Oracle development teams on how to best model and
solve their problems using CP, MP, Heuristics, and Hybrid solutions.
In this talk we examine some of the successful solutions to constrained problems
within such a large corporation. We discuss at a high level the many opportunities for
further integration and unification of constraint technologies into more products and
tools, including the challenge of modeling and solving in highly interactive scenarios.
Most importantly, we open the discussion about various challenges faced by large
front-end centered companies, with many degrees of separation between organizations,
who must balance resources to focus over immediate and long-term deliverables.
 Industrial Success Stories of ASP and CP:
What’s Still Open?

Gerhard Friedrich

Institut für Angewandte Informatik

Alpen-Adria Universität Klagenfurt, Austria
e-mail: [email protected]

Abstract More than 25 years ago together with Siemens we started to inves-
tigate the possibility of substituting the classical rule-based configuration
approach by model- based techniques. It turned out that in those days only
constrained programming (CP) had any real chance of meeting the application
demands. By exploiting CP we were able to significantly improve the produc-
tivity of highly trained employees (by more than 300 %) and to substantially
reduce software development and maintenance costs (by more than 80 %) [4].
Consequently, CP has been our method of choice for problem solving in
industrial projects since 1989 [3].

Some years ago, we started to investigate answer set programming (ASP) techniques
[2], mainly because of the possibility to apply a very expressive logical first-order
language for specifying problems. It emerged that, by using simply problem encoding,
we were able to solve difficult real world problem instances witnessing the enormous
improvements of logic programming over the last decades [1].
Although ASP and CP have proven their practical applicability, we will point out
challenges of large problems of the electronic and the semiconductor industry. In
particular, we will stress the power of problem-specific heuristics [5, 9] which turned
out to be the key in many applications of problem solvers.
Looking at the famous equation “algorithm = logic + control” [6] most of the current
work in the AI community assumes that control should be problem- independent and
only the logical specification depends on the problem to be solved, i.e. “algorithm =
logic(problem) + control”. It is not surprising that for the current problem solving
technology this is a practical approach up to a certain size of the problem instances, since
we deal with NP-hard problems in many cases. However, it is observed (and examples
are given [10, 7]) that problem-specific heuristics allow enormous run-time improve-
ments. This success is based on problem-specific control, i.e. “algorithm = logic
(problem) + control(problem)”. Unfortunately, the design of such problem-specific
heuristics is very time-consuming and redesigns are frequently required because of
recurrent changes of the problem. Interestingly, humans are very successful at devel-
oping such problem-specific heuristics. Therefore, we argue that the automation of
generating problem-specific heuristics with satisfying quality is still an important basic
AI research goal with high practical impact that should be achievable [8].
 Industrial Success Stories of ASP and CP XVII

References
1. Aschinger, M., Drescher, C., Friedrich, G., Gottlob, G., Jeavons, P., Ryabokon, A., Thor-
stensen, E.: Optimization methods for the partner units problem. In: Achterberg, T., Beck, J.C.
(eds.) CPAIOR 2011. LNCS, vol. 6697, pp. 4–19. Springer, Heidelberg (2011). http://dx.doi.
org/10.1007/978-3-642-21311-3
2. Brewka, G., Eiter, T., Truszczynski, M.: Answer set programming at a glance. Commun.
ACM 54(12), 92–103 (2011). http://doi.acm.org/10.1145/2043174.2043195
3. Falkner, A., Haselboeck, A., Schenner, G., Schreiner, H.: Benefits from three configurator
generations. In: Blecker, T., Edwards, K., Friedrich, G., Hvam, L., Salvodor, F. (eds.)
Innovative Processes and Products for Mass Customization, vol. 3, pp. 89–103 (2007)
4. Fleischanderl, G., Friedrich, G., Haselböck, A., Schreiner, H., Stumptner, M.: Configuring
large systems using generative constraint satisfaction. IEEE Intell. Syst. 13(4), 59–68 (1998)
5. Gebser, M., Kaufmann, B., Romero, J., Otero, R., Schaub, T., Wanko, P.: Domain-specific
heuristics in answer set programming. In: desJardins, M., Littman, M.L. (eds.) Proceedings
of the Twenty-Seventh AAAI Conference on Artificial Intelligence, July 14–18, 2013,
Bellevue, Washington, USA. AAAI Press (2013). http://www.aaai.org/ocs/index.php/AAAI/
AAAI13/paper/view/6278
6. Kowalski, R.A.: Algorithm = logic + control. Commun. ACM 22(7), 424–436 (1979). http://
doi.acm.org/10.1145/359131.359136
7. Mersheeva, V., Friedrich, G.: Multi-uav monitoring with priorities and limited energy
resources. In: Brafman, R.I., Domshlak, C., Haslum, P., Zilberstein, S. (eds.) Proceedings
of the Twenty-Fifth International Conference on Automated Planning and Scheduling,
ICAPS 2015, Jerusalem, Israel, June 7-11, 2015, ICAPS 2015. pp. 347–356. AAAI Press
(2015). http://www.aaai.org/ocs/index.php/ICAPS/ICAPS15/paper/view/10460
8. Pearl, J.: On the discovery and generation of certain heuristics. AI Mag. 4(1), 23–33 (1983).
http://www.aaai.org/ojs/index.php/aimagazine/article/view/385
9. Schrijvers, T., Tack, G., Wuille, P., Samulowitz, H., Stuckey, P.J.: Search combinators.
Constraints 18(2), 269–305 (2013). http://dx.doi.org/10.1007/s10601-012- 9137-8
10. Teppan, E.C., Friedrich, G., Falkner, A.A.: Quickpup: A heuristic backtracking algorithm for
the partner units configuration problem. In: Fromherz, M.P.J., Muñoz- Avila, H. (eds.)
Proceedings of the Twenty-Fourth Conference on Innovative Applications of Artificial
Intelligence, July 22–26, 2012, Toronto, Ontario, Canada. AAAI (2012). http://www.aaai.
org/ocs/index.php/IAAI/IAAI-12/paper/view/4793
 Synthesis of Constraint Solvers

Douglas R. Smith and Stephen J. Westfold

Kestrel Institute, Palo Alto, CA 94034 USA

{smith,westfold}@kestrel.edu

In [2], we present a mathematical framework for specifying and formally designing

high-performance constraint solving algorithms. The framework is based on concepts
from abstract interpretation which generalizes earlier work on a Galois
Connection-based model of Global Search algorithms. The main focus is on how to use
the framework to automate the calculations necessary to construct correct, efficient
problem-specific constraint solvers.
It is common practice in the constraint-solving community to solve a new problem
P by building a reduction to a well-studied problem Q that has a well-engineered
solver. One problem with this approach is that the reduction of P to Q often loses some
key structure which cannot then be exploited by the Q-solver. Our thesis is that a native
solver can always be generated for a constraint problem that outperforms a reduction to
an existing solver.
This talk focuses on three main results from [2]:
1. Algorithm theory – We develop and prove an algorithm theory for constraint solving
with propagation, conflict detection and analysis, backjumping, and learning that is
parametric on the constraint logic.
2. Design Method for Constraint Propagation – We prove that Arc Consistency is a
best-possible constraint propagation mechanism for arbitrary CSPs, and then showed
how to calculate optimal code for propagation. From Arc Consistency formula
schemes we calculate simple Definite Constraints that can be instantiated into the
optimal Definite Constraint Solver scheme [1].
3. Theory of Conflict Analysis – There are several mathematical formalisms for gen-
eralizing conflict analysis to arbitrary logics. We present a general pattern for cal-
culating resolution rules in a given logic, and prove how resolution can be iterated to
soundly infer a new constraint for backjumping and learning purposes.

References
1. Rehof, J., Mogenson, T.: Tractable constraints finite semilattices. Sci. Comput. Program. 35,
191–221 (1999)
2. Smith, D.R., Westfold, S.: Toward Synthesis Constraint Solvers. Tech. rep., Kestrel Institute
(2013). http://www.kestrel.edu/home/people/smith/pub/CW- report.pdf
 Contents

Technical Track

Encoding Linear Constraints with Implication Chains to CNF. . . . . . . . . . . . 3

Ignasi Abío, Valentin Mayer-Eichberger, and Peter J. Stuckey

Anytime Hybrid Best-First Search with Tree Decomposition

for Weighted CSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
David Allouche, Simon de Givry, George Katsirelos, Thomas Schiex,
and Matthias Zytnicki

Improved Constraint Propagation via Lagrangian Decomposition. . . . . . . . . . 30

David Bergman, Andre A. Cire, and Willem-Jan van Hoeve

Strengthening Convex Relaxations with Bound Tightening for Power

Network Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Carleton Coffrin, Hassan L. Hijazi, and Pascal Van Hentenryck

Broken Triangles Revisited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

Martin C. Cooper, Aymeric Duchein, and Guillaume Escamocher

A Microstructure-Based Family of Tractable Classes for CSPs . . . . . . . . . . . 74

Martin C. Cooper, Philippe Jégou, and Cyril Terrioux

The Unary Resource with Transition Times . . . . . . . . . . . . . . . . . . . . . . . . 89

Cyrille Dejemeppe, Sascha Van Cauwelaert, and Pierre Schaus

A Global Constraint for a Tractable Class of Temporal

Optimization Problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
Alban Derrien, Jean-Guillaume Fages, Thierry Petit,
and Charles Prud’homme

Exploiting GPUs in Solving (Distributed) Constraint Optimization

Problems with Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Ferdinando Fioretto, Tiep Le, Enrico Pontelli, William Yeoh,
and Tran Cao Son

Conflict Ordering Search for Scheduling Problems . . . . . . . . . . . . . . . . . . . 140

Steven Gay, Renaud Hartert, Christophe Lecoutre, and Pierre Schaus

Simple and Scalable Time-Table Filtering for the Cumulative Constraint . . . . 149
Steven Gay, Renaud Hartert, and Pierre Schaus
XX Contents

General Bounding Mechanism for Constraint Programs . . . . . . . . . . . . . . . . 158

Minh Hoàng Hà, Claude-Guy Quimper, and Louis-Martin Rousseau

Smallest MUS Extraction with Minimal Hitting Set Dualization . . . . . . . . . . 173

Alexey Ignatiev, Alessandro Previti, Mark Liffiton,
and Joao Marques-Silva

Upper and Lower Bounds on the Time Complexity

of Infinite-Domain CSPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
Peter Jonsson and Victor Lagerkvist

Generalized Totalizer Encoding for Pseudo-Boolean Constraints . . . . . . . . . . 200

Saurabh Joshi, Ruben Martins, and Vasco Manquinho

Smaller Selection Networks for Cardinality Constraints Encoding . . . . . . . . . 210

Michał Karpiński and Marek Piotrów

PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining . . . . 226

Amina Kemmar, Samir Loudni, Yahia Lebbah, Patrice Boizumault,
and Thierry Charnois

On Tree-Preserving Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

Shufeng Kong, Sanjiang Li, Yongming Li, and Zhiguo Long

Modeling and Solving Project Scheduling with Calendars . . . . . . . . . . . . . . 262

Stefan Kreter, Andreas Schutt, and Peter J. Stuckey

Deterministic Estimation of the Expected Makespan of a POS Under

Duration Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Michele Lombardi, Alessio Bonfietti, and Michela Milano

A Parallel, Backjumping Subgraph Isomorphism Algorithm

Using Supplemental Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
Ciaran McCreesh and Patrick Prosser

Automated Auxiliary Variable Elimination Through On-the-Fly

Propagator Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
Jean-Noël Monette, Pierre Flener, and Justin Pearson

Automatically Improving SAT Encoding of Constraint Problems

Through Common Subexpression Elimination in Savile Row . . . . . . . . . . . . 330
Peter Nightingale, Patrick Spracklen, and Ian Miguel

Exact Sampling for Regular and Markov Constraints

with Belief Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341
Alexandre Papadopoulos, François Pachet, Pierre Roy,
and Jason Sakellariou
 Contents XXI

Randomness as a Constraint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351

Steven D. Prestwich, Roberto Rossi, and S. Aramagan Tarim

Quasipolynomial Simulation of DNNF by a Non-determinstic

Read-Once Branching Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
Igor Razgon

MiniSearch: A Solver-Independent Meta-Search Language for MiniZinc . . . . 376

Andrea Rendl, Tias Guns, Peter J. Stuckey, and Guido Tack

Two Clause Learning Approaches for Disjunctive Scheduling. . . . . . . . . . . . 393

Mohamed Siala, Christian Artigues, and Emmanuel Hebrard

Bounding an Optimal Search Path with a Game of Cop and Robber

on Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
Frédéric Simard, Michael Morin, Claude-Guy Quimper,
François Laviolette, and Josée Desharnais

Restricted Path Consistency Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

Kostas Stergiou

Machine Learning of Bayesian Networks Using Constraint Programming. . . . 429

Peter van Beek and Hella-Franziska Hoffmann

Hybridization of Interval CP and Evolutionary Algorithms

for Optimizing Difficult Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
Charlie Vanaret, Jean-Baptiste Gotteland, Nicolas Durand,
and Jean-Marc Alliot

A General Framework for Reordering Agents Asynchronously

in Distributed CSP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463
Mohamed Wahbi, Younes Mechqrane, Christian Bessiere,
and Kenneth N. Brown

Automatically Generating Streamlined Constraint Models with ESSENCE

and CONJURE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
James Wetter, Özgür Akgün, and Ian Miguel

Application Track

Constraint-Based Local Search for Finding Node-Disjoint Bounded-Paths

in Optical Access Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499
Alejandro Arbelaez, Deepak Mehta, and Barry O’Sullivan

Open Packing for Facade-Layout Synthesis Under a General

Purpose Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508
Andrés Felipe Barco, Jean-Guillaume Fages, Elise Vareilles,
Michel Aldanondo, and Paul Gaborit
XXII Contents

Power Capping in High Performance Computing Systems . . . . . . . . . . . . . . 524

Andrea Borghesi, Francesca Collina, Michele Lombardi,
Michela Milano, and Luca Benini

A Constraint-Based Approach to the Differential Harvest Problem . . . . . . . . 541

Nicolas Briot, Christian Bessiere, and Philippe Vismara

Constrained Minimum Sum of Squares Clustering by Constraint

Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557
Thi-Bich-Hanh Dao, Khanh-Chuong Duong, and Christel Vrain

A Constraint Programming Approach for Non-preemptive

Evacuation Scheduling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574
Caroline Even, Andreas Schutt, and Pascal Van Hentenryck

Solving Segment Routing Problems with Hybrid Constraint

Programming Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
Renaud Hartert, Pierre Schaus, Stefano Vissicchio,
and Olivier Bonaventure

Modeling Universal Instruction Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 609

Gabriel Hjort Blindell, Roberto Castañeda Lozano, Mats Carlsson,
and Christian Schulte

Optimizing the Cloud Service Experience Using Constraint Programming . . . 627

Serdar Kadioglu, Mike Colena, Steven Huberman, and Claire Bagley

Find Your Way Back: Mobility Profile Mining with Constraints . . . . . . . . . . 638
Lars Kotthoff, Mirco Nanni, Riccardo Guidotti, and Barry O’Sullivan

Joint Vehicle and Crew Routing and Scheduling . . . . . . . . . . . . . . . . . . . . . 654

Edward Lam, Pascal Van Hentenryck, and Philip Kilby

Constructing Sailing Match Race Schedules: Round-Robin Pairing Lists . . . . 671

Craig Macdonald, Ciaran McCreesh, Alice Miller, and Patrick Prosser

Design and Evaluation of a Constraint-Based Energy Saving and

Scheduling Recommender System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 687
Seán Óg Murphy, Óscar Manzano, and Kenneth N. Brown

Scheduling Running Modes of Satellite Instruments

Using Constraint-Based Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704
Cédric Pralet, Solange Lemai-Chenevier, and Jean Jaubert
 Contents XXIII

Abstracts of Papers Fast Tracked to Constraints Journal

Using Finite Transducers for Describing and Synthesising Structural

Time-Series Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723
Nicolas Beldiceanu, Mats Carlsson, Rémi Douence, and Helmut Simonis

Projection, Consistency, and George Boole. . . . . . . . . . . . . . . . . . . . . . . . . 724

J.N. Hooker

On computing Minimal Independent Support and its applications

to sampling and counting (Extended Abstract) . . . . . . . . . . . . . . . . . . . . . . 725
Alexander Ivrii, Sharad Malik, Kuldeep S. Meel, and Moshe Y. Vardi

General Game Playing with Stochastic CSP . . . . . . . . . . . . . . . . . . . . . . . . 726

Frédéric Koriche, Sylvain Lagrue, Éric Piette, and Sébastien Tabary

Visual Search Tree Profiling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 728

Maxim Shishmarev, Christopher Mears, Guido Tack,
and Maria Garcia de la Banda

Long-Haul Fleet Mix and Routing Optimisation with Constraint

Programming and Large Neighbourhood Search . . . . . . . . . . . . . . . . . . . . . 729
Philip Kilby and Tommaso Urli

Abstracts of Published Journal Track Papers

On the Reification of Global Constraints (Abstract) . . . . . . . . . . . . . . . . . . . 733

Nicolas Beldiceanu, Mats Carlsson, Pierre Flener, and Justin Pearson

MDD Propagation for Sequence Constraints . . . . . . . . . . . . . . . . . . . . . . . . 734

David Bergman, Andre A. Cire, and Willem-Jan van Hoeve

Discrete Optimization with Decision Diagrams . . . . . . . . . . . . . . . . . . . . . . 735

David Bergman, Andre A. Cire, Willem-Jan van Hoeve,
and John Hooker

A Hybrid Approach Combining Local Search and Constraint Programming

for a Large Scale Energy Management Problem . . . . . . . . . . . . . . . . . . . . . 736
Haris Gavranović and Mirsad Buljubašić

Representing and Solving Finite-Domain Constraint Problems

using Systems of Polynomials (Extended Abstract) . . . . . . . . . . . . . . . . . . . 737
Chris Jefferson, Peter Jeavons, Martin J. Green,
and M.R.C. van Dongen

A Quadratic Extended Edge-Finding Filtering Algorithm for Cumulative

Resource Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
Roger Kameugne, Laure Pauline Fotso, and Joseph Scott
XXIV Contents

Achieving Domain Consistency and Counting Solutions for Dispersion

Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
Gilles Pesant

meSAT: Multiple Encodings of CSP to SAT . . . . . . . . . . . . . . . . . . . . . . . 741

Mirko Stojadinović and Filip Maric

Constraint programming for LNG ship scheduling and inventory

management (Abstract) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
Willem-Jan van Hoeve

Revisiting the Limits of MAP Inference by MWSS on Perfect Graphs . . . . . . 743

Adrian Weller

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745

Technical Track
 Encoding Linear Constraints with Implication
Chains to CNF

Ignasi Abı́o1(B) , Valentin Mayer-Eichberger1,2 , and Peter J. Stuckey1,3

1
NICTA, Canberra, Australia
{ignasi.abio,valentin.mayer-eichberger,peterj.stuckey}@nicta.com.au
2
University of New South Wales, Sydney, Australia
3
University of Melbourne, Melbourne, Australia

Abstract. Linear constraints are the most common constraints occur-

ring in combinatorial problems. For some problems which combine lin-
ear constraints with highly combinatorial constraints, the best solving
method is translation to SAT. Translation of a single linear constraint to
SAT is a well studied problem, particularly for cardinality and pseudo-
Boolean constraints. In this paper we describe how we can improve
encodings of linear constraints by taking into account implication chains
in the problem. The resulting encodings are smaller and can propagate
more strongly than separate encodings. We illustrate benchmarks where
the encoding improves performance.

1 Introduction

In this paper we study linear integer constraints (LI constraints), that is, con-
straints of the form a1 x1 +· · ·+an xn # a0 , where the ai are integer given values,
the xi are finite-domain integer variables, and the relation operator # belongs
to {<, >, ≤, ≥, =, =}. We will assume w.l.o.g that # is ≤, the ai are positive and
all the domains of the variables are {0, 1..di }, since other cases can be reduced to
this one.1 Special case of linear constraints are: pseudo-Boolean (PB) constraints
where the domain of each variable is {0..1}, cardinality (CARD) constraints
where additionally ai = 1, 1 ≤ i ≤ n, and at-most-one (AMO) constraints where
additionally a0 = 1.
Linear integer constraints appear in many combinatorial problems such as
scheduling, planning or software verification, and, therefore, many different SMT
solvers [9,14] and encodings [4,6,11] have been suggested for handling them.
There are two main approaches to encoding linear constraints: cardinality con-
straints are encoded as some variation of a sorting network [3]; multi-decision
diagrams (MDDs) are used to encode more general linear constraints [5], which
in the special case of pseudo-Boolean constraints collapse to binary decision
diagrams (BDDs).

1
See [5] for details. Note that propagation is severely hampered by replacing equalities
by a pair of inequalities.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 3–11, 2015.
DOI: 10.1007/978-3-319-23219-5 1
4 I. Abı́o et al.

Any form of encoding linear constraints to SAT introduces many intermediate

Boolean variables, and breaks the constraint up into many parts. This gives us
the opportunity to improve the encoding if we can recognize other constraints in
the problem that help tighten the encoding of some part of the linear constraint.
Example 1. Consider a pseudo-Boolean constraint x1 + 2x2 + 2x3 + 4x4 + 5x5 +
6x6 + 8x7 ≤ 14 where we also have that x2 + x3 + x5 ≤ 1 we can rewrite the
constraints as x1 +2x235 +4x4 +3x5 +6x6 +8x7 ≤ 14 and x235 ≡ (x2 +x3 +x5 = 1),
where x235 is a new Boolean variable. Notice, that x235 can be used to encode
the at-most-one constraint. 
Example 2. Consider a pseudo-Boolean constraint 4x1 + 2x2 + 5x3 + 4x4 ≤ 9
and the implications x1 ← x2 (i.e. x1 ∨ ¬x2 ) and x2 ← x3 . Separately they do
not propagate, but considered together we can immediately propagate ¬x3 . 
In this paper we show how to encode pseudo-Boolean constraints taking into
account implication chains, as seen in Example 2. The resulting encodings are
no larger than the separate encoding, but result in strictly stronger propagation.
The approach also allows us to encode general linear integer constraints, and
is a strict generalization of the MDD encoding of linear integer constraints [5].
We show how these new combined encodings are effective in practice on a set of
hard real-life sports scheduling examples, and that the combination of pseudo-
Booleans with implication chains arises in a wide variety of models.

2 Preliminaries
2.1 SAT Solving
Let X = {x1 , x2 , . . .} be a fixed set of propositional variables. If x ∈ X then x
and ¬x are positive and negative literals, respectively. The negation of a literal
l, written ¬l, denotes ¬x if l is x, and x if l is ¬x. A clause is a disjunction of
literals l1 ∨ · · · ∨ ln . An implication x1 → x2 is notation for the clause ¬x1 ∨ x2 ,
similarly x1 ← x2 denotes x1 ∨ ¬x2 . A CNF formula is a conjunction of clauses.
A (partial) assignment A is a set of literals such that {x, ¬x} ⊆ A for any
x ∈ X , i.e., no contradictory literals appear. A literal l is true () in A if l ∈ A, is
false (⊥) in A if ¬l ∈ A, and is undefined in A otherwise. A clause C is true in A
if at least one of its literals is true in A. A formula F is true in A if all its clauses
are true in A. In that case, A is a model of F . Systems that decide whether a
formula F has any model are called SAT-solvers, and the main inference rule
they implement is unit propagation: given a CNF F and an assignment A, find
a clause in F such that all its literals are false in A except one, say l, which
is undefined, add l to A and repeat the process until reaching a fix-point. A
detailed explanation can be found in [7].
Let [l..u] where l and u are integers represent the set {l, . . . , u}. Let y be
an integer variable with domain [0..d]. The order encoding introduces Boolean
variables yi for 1 ≤ i ≤ d. A variable yi is true iff y < i. The encoding also
introduces the clauses yi → yi+1 for 1 ≤ i < d.
 Encoding Linear Constraints with Implication Chains to CNF 5

2.2 Multi Decision Diagrams

A directed acyclic graph is called an ordered Multi Decision Diagram (MDD) if

it satisfies the following properties:

– It has two terminal nodes, namely T (true) and F (false).

– Each non-terminal node is labeled by an array of Booleans [xi1 , . . . , xidi ]
representing the order encoding of integer variable yi where yi ranges from
[0..di ]. The variable yi is called the selector variable.
– Every node labeled by yi has di + 1 outgoing edges, labelled xi1 , ¬xi1 , . . . ,
¬xidi .
– Each edge goes from a node with selector yi to a node with selector variable
yj has i < j.

The MDD is quasi-reduced if no isomorphic subgraphs exist. It is reduced if,

moreover, no nodes with only one child exist. A long edge is an edge connecting
two nodes with selector variables yi and yj such that j > i + 1. In the following
we only consider quasi-reduced ordered MDDs without long edges, and we just
refer to them as MDDs for simplicity. A Binary Decision Diagram (BDD) is an
MDD where ∀i, di = 1.
An MDD represents a function f : {0, 1, . . . , d1 } × {0, 1, . . . , d2 } × · · · ×
{0, 1, . . . , dn } → {⊥, } in the obvious way. Moreover, given a fixed variable
ordering, there is only one MDD representing that function. We refer to [17] for
further details about MDDs.
A function f is anti-monotonic in argument yi if vi ≥ vi implies that
f (v1 , . . . , vi−1 , vi , vi+1 , . . . , vn ) =  ⇒ f (v1 , . . . , vi−1 , vi , vi+1 , . . . , vn ) =  for
all values v1 , . . . , vn , vi . An MDD is anti-monotonic if it encodes a function f
that is anti-monotonic in all arguments. We shall only consider anti-monotonic
MDDs in this paper.
Given an anti-monotonic MDD M, we can encode it into CNF by intro-
ducing a new Boolean variable bo to represent each node o in the MDD M;
unary clauses {bT , ¬bF , br } where r is the root node of the MDD; and clauses
{¬bo ∨ bo0 } ∪ {¬bo ∨ xij ∨ boj | j ∈ [1..di ]} for each node o of the form
mdd([xi1 , . . . , xidi ], [o0 , o1 , . . . , odi ]). See [5] for more details.
We can encode arbitrary MDDs to SAT using Tseitin transformation but the
encoding is substantially more complicated.

3 Pseudo Boolean Constraints and Chains

A chain x1 ⇐ x2 ⇐ · · · ⇐ xn is a constraint requiring x1 ← x2 , x2 ← x3 , . . . ,

xn−1 ← xn . A unary chain x1 is the trivial case that imposes no constraint. A
chain is compatible with an ordered list of Boolean variables x1 , . . . , xn if the
chain is of the form xl ⇐ xl+1 ⇐ · · · ⇐ xk , l ≤ k. Given an ordered list L
of Boolean variables x1 , . . . , xn a chain coverage S is a set of variable-disjoint
compatible chains such that each variable appears in exactly one chain. We
will sometimes treat a chain coverage S as a Boolean formula equivalent to the
6 I. Abı́o et al.

constraints of the chains appearing in S. Given a variable ordering and a set of

disjoint compatible chains we can always construct a chain coverage by adding
in unary chains.

Example 3. Given list L of variables x1 , . . . , x9 and chains x1 ⇐ x2 ⇐ x3 , x5 ⇐

x6 , x7 ⇐ x8 , then a chain coverage S of L is {x1 ⇐ x2 ⇐ x3 , x4 , x5 ⇐ x6 , x7 ⇐
x8 , x9 }. S represents the constraint x1 ← x2 ∧ x2 ← x3 ∧ x5 ← x6 ∧ x7 ← x8 . 

Given a Boolean variable ordering L, a PB constraint C and chain coverage

S of L we will demonstrate how to build an MDD to encode the constraint C
taking into account the chain constraints in S. First lets examine how chains
arise in models.

At-most-one constraints. Given PB C ≡ a1 x1 + . . . + an xn ≤ a0 and AMO

A ≡ x1 + x2 + . . . + xk ≤ 1. We can reformulate A using new variables xj
where x1 + . . . + xk = x1 + . . . + xk using the ladder encoding [12] which gives
xi → xi for i = 1 . . . k, xi → ¬xi+1 and xi+1 → xi for i = 1 . . . k − 1. If
[a1 , . . . , ak ] is the sorted array of coefficients [a1 , . . . , ak ] then C can be written
as a1 x1 + (a2 − a1 )x2 + · · · + (ak − ak−1 )xk + ak+1 xk+1 + · · · an xn ≤ a0 . The
chain comes from the auxiliary variables: x1 ⇐ x2 ⇐ · · · ⇐ xk .

General linear integer constraints. A general LI C ≡ a1 y1 + · · · am ym ≤ a0 can

be expressed as a PB with chains. We encode each integer yi with domain [0..di ]
by the order encoding [xi1 , . . . , xidi ] and then C can be rewritten as a1 (¬x11 ) +
· · · + a1 (¬x1d1 ) + · · · + am (¬xm1 ) + · · · + am (¬xmdm ) ≤ a0 with chains ¬xi1 ⇐
¬xi2 ⇐ · · · ⇐ ¬xidi .

Shared coefficients. Frequently, PB constraints contain a large number of coeffi-

cients that are the same. This structure can be exploited. A similar technique of
grouping shared coefficients is described in [5] which in our context is restated
using chains. Given a PB C ≡ ax1 + · · · + axk + ak+1 xk+1 + · · · + an xn ≤ a0
where the first k variables share the same coefficient. We introduce new vari-
ables x1 , . . . , xk to encode the sum x1 + · · · + xk so x1 + . . . + xk = x1 + . . . + xk
and encode this constraint (usually using some form of sorting network [3]).
This ensures that x1 ⇐ x2 ⇐ · · · ⇐ xk . Then C can be rewritten as
ax1 + · · · + axk + ak+1 xk+1 + · · · + an xn ≤ a0 . There are several advantages
with this rewritten version. The sorting network can be represented more com-
pactly than the same logic in an MDD (O(k · log2 k) vs O(k 2 )). Secondly, the
introduced variables xj are meaningful for the constraint and are likely to be
useful for branching and in conflict clause learning during the search. Moreover,
the sorted variables may be reusable for rewriting other constraints.

Binary implicants. Finally, a more general method is to automatically extract

chains from the global problem description. There are a number of methods
to detect binary implicants of CNF encodings [10,13]. Given a set of binary
implicants B and a PB constraint C we can search for a chain coverage S implied
 Encoding Linear Constraints with Implication Chains to CNF 7

by B, and an ordering L of the variables in C with which S is compatible, and

then encode the reordered constraint C making use of the chain coverage S.
In the experimental section of this paper we have only considered the first
three types of chains.

4 Translating Through MDDs with Chains

The main algorithm in this section generalizes the construction of an MDD in

[5]. We first restate definitions of the original algorithm and then show how
to take advantage of chains in the new construction. The CNF decomposition
has desirable properties, i.e. we show that the encoding is more compact and
propagates stronger.

4.1 Preliminaries for the Construction

Let M be the MDD of pseudo-Boolean C and let ν be a node of M with selector

variable xi . We define the interval of ν as the set of values α such that the MDD
rooted at ν represents the pseudo-Boolean constraint ai xi + · · · + an xn ≤ α. It is
easy to see that this definition corresponds in fact to an interval. The key point
in constructing the MDD is to label each node of the MDD with its interval
[β, γ].
In the following, for every i ∈ {1, 2, . . . , n + 1}, we use a set Li consisting of
pairs ([β, γ], M), where M is the MDD of the constraint ai xi + · · · + an xn ≤ a0
for every a0 ∈ [β, γ] (i.e., [β, γ] is the interval of M). All these sets are kept in
a tuple L = (L1 , L2 , . . . , Ln+1 ).
Note that by definition of the MDD’s intervals, if both ([β1 , γ1 ], M1 ) and
([β2 , γ2 ], M2 ) belong to Li then either [β1 , γ1 ] = [β2 , γ2 ] or [β1 , γ1 ] ∩ [β2 , γ2 ] = ∅.
Moreover, the first case holds if and only if M1 = M2 . Therefore, Li can be
represented with a binary search tree-like data structure, where insertions and
searches can be done in logarithmic time. The function search(K, Li ) searches
whether there exists a pair ([β, γ], M) ∈ Li with K ∈ [β, γ]. Such a tuple is
returned if it exists, otherwise an empty interval is returned in the first com-
ponent of the pair. Similarly, we also use function insert(([β, γ], M), Li ) for
insertions.

4.2 Algorithm and Properties of the Construction

In this section we show how to translate a PB C ≡ a1 x1 + . . . an xn ≤ a0

and a chain coverage S for variable order x1 , . . . , xn . Algorithm 1 describes
the construction of the MDD. The initial call is MDDChain(1, C, S). The call
MDDChain(i, C  , S) recursively builds an MDD for C  ∧ S by building the ith
level. If the chain including xi is xi ⇐ · · · ⇐ xk it builds an MDD node that
has child nodes with selector xk+1 . If the chain for xi is unary this is the usual
MDD (BDD) construction.
8 I. Abı́o et al.

Algorithm 1. Procedure MDDChain

Require: i ∈ {1, 2, . . . , n + 1} and pseudo-Boolean constraint C  : ai xi + . . . + an xn ≤
a0 and chain coverage S on [x1 , . . . , xn ]
Ensure: returns [β, γ] interval of C  and M its MDD
1: ([β, γ], M) ← search(a0 , Li ).
2: if [β, γ] = ∅ then
3: return ([β, γ], M).
4: else
5: δ0 ← 0
6: let {xi ⇐ xi+1 ⇐ · · · ⇐ xk } ∈ S % including unary chain xi
7: u←k−i+1
8: for all j such that 0 ≤ j ≤ u do
9: ([βj , γj ], Mj ) ← MDDChain(k + 1, ak+1 xk+1 + · · · + an xn ≤ a0 − δj , S).
10: δj+1 ← δj + ai+j
11: end for
12: M ← mdd([xi , . . . , xk ], M0 , . . . , Mu )
13: [β, γ] ← [β0 , γ0 ] ∩ [β1 + δ1 , γ1 + δ1 ] ∩ · · · ∩ [βu + δu , γu + δu ].
14: insert(([β, γ], M), Li ).
15: return ([β, γ], M).
16: end if

Example 4. The MDDs that result from MDDChain(1, C, S) where C ≡

4x1 + 2x2 + 5x3 + 4x4 ≤ 9 of Example 2 encoded with chain coverage (a)
S = {x1 , x2 , x3 , x4 } (no chains) and (b) S = {x1 ⇐ x2 ⇐ x3 , x4 } are shown
in Figure 1. The diagrams show [β, γ] for each node with the remainder of the
constraint at the left. Unit propagation of the CNF of (b) sets x3 = ⊥ immedi-
ately since 4x4 ≤ −1 is ⊥.

We can prove that the algorithm returns a correct MDD, that is no larger
than the MDD (BDD) encoding of C, and that the resulting CNF encoding is
domain consistent on the original variables x1 , . . . , xn . Proofs are available at [1].

Theorem 1. Given a pseudo-Boolean constraint C ≡ a1 x1 +· · ·+an xn ≤ a0 and

chain coverage S on [x1 , . . . , xn ] then MDDChain(1, C, S) returns an MDD M
representing function f such that constraint C ∧ S |= f . The running time of the
algorithm is O(n · a0 · log a0 ). 

Theorem 2. Given a pseudo-Boolean constraint C ≡ a1 x1 + · · · + an xn ≤ a0

and chain coverage S on [x1 , . . . , xn ] then the MDD MDDChain(1, C, S) has
no more nodes than MDDChain(1, C, {x1 , . . . , xn }), the BDD for C. 

Theorem 3. Given a pseudo-Boolean constraint C ≡ a1 x1 + · · · + an xn ≤ a0

and chain coverage S on [x1 , . . . , xn ] then unit propagation on the CNF encoding
of MDDChain(1, C, S) ∧ S enforces domain consistency of C ∧ S on variables
x1 , . . . , xn . 
 Encoding Linear Constraints with Implication Chains to CNF 9

4x1 + 2x2 + 5x3 + 4x4 ≤ [9, 9] [9, 9]

EE ¬x1  ,,
x1
ttt EE
y tt
t E"   ,,
,,
2x2 + 5x3 + 4x4 ≤ [5, 5]
JJJ¬x2
[9, 9]
 ,,
x2 yy x3  x
JJJ y  ,,x1 ,¬x1
|yy
x2 ¬x2 2
 %  ,,
5x3 + 4x4 ≤ [0, 3] [5, 8] [9, 9]   ,,
JJJ¬x3 EE ¬x3 
JJJx3 EE ¬x3 ,x3  ,,
x3
 %  E"   
4x4 ≤ [−∞, −1] [0, 3] [4, 9] [−∞, −1] [0, 3] [4, 9]
x4 ss FFF¬x4 ¬x ,x x4 ss F FF¬x4 ¬x ,x
s FF  4 4 s FF  4 4
 yssss  yssss
¬x4 ,x4 ¬x4 ,x4
# #
F T F T

(a) (b)

Fig. 1. The MDDs that result from 4x1 + 2x2 + 5x3 + 4x4 ≤ 9 encoded (a) without
and (b) with the chain x1 ⇐ x2 ⇐ x3 .

5 Experiments

To illustrate the advantage of combined compilation we consider a challenging

combinatorial optimization problem where both AMO and shared coefficients
chains arise.
Sports league scheduling is a challenging combinatorial optimization problem.
We consider scheduling a double round-robin sports league of N teams. All teams
meet each other once in the first N − 1 weeks and again in the second N − 1
weeks, with exactly one match per team each week. A given pair of teams must
play at the home of one team in one half, and at the home of the other in the
other half, and such matches must be spaced at least a certain minimal number
of weeks apart. Additional constraints include, e.g., that no team ever plays at
home (or away) three times in a row, other (public order, sportive, TV revenues)
constraints, blocking given matches on given days, etc.
Additionally, the different teams can propose a set of constraints with some
importance (low, medium or high). We aim not only to maximize the number of
these constraints satisfied, but also to assure that at least some of the constraints
of every team are satisfied. More information can be found in [2].
Low-importance constraints are given a weight of 1; medium-importance,
5, and high-importance, 10. For every constraint proposed by a team i, a new
Boolean variable xi,j is created. This variable is set to true  if the constraint
is violated. For every team, a pseudo-Booleanconstraint
 j wi,j xi,j ≤ Ki is
imposed. The objective function to minimize is i j wi,j xi,j . The data is based
on real-life instances.
Desired constraints typically refer to critical weeks in the schedule, e.g.
around Christmas, or other key dates, and preferences of different teams almost
always clash. Double round-robin tournaments contain a lot of AMO and EO
constraints (for instance, each week each team meets exactly one team). These
AMO constraints can be used to simplify the desired constraints.
10 I. Abı́o et al.

Table 1. Results for sports league scheduling, showing the number of runs that find a
solution of different quality after different time limits (seconds).

Quality Some solution cost ≤ 30 + best cost ≤ 20 + best cost ≤ 10 + best

Timelimit 300 900 3600 300 900 3600 300 900 3600 300 900 3600
MDD1 148 190 199 21 55 107 17 35 74 6 25 51
MDD2 151 194 199 27 59 115 19 38 81 12 25 43
MDD3 160 191 200 56 107 162 45 72 121 41 52 87
LCG 69 123 172 21 29 51 18 21 35 14 20 27
Gurobi 0 0 0 0 0 0 0 0 0 0 0 0

The benchmark consists of 10 instances and each method is run 20 times

with different seeds, for 200 total runs. Compared methods are: MDD1, the usual
MDD (in fact, BDD) method to encode PBs [4]; MDD2, the method of [5] using
sorting networks for the identical coefficients and then using an MDD; MDD3,
the method defined herein; LCG, using lazy clause generation [15]; and Gurobi,
using the MIP solver Gurobi. Barcelogic [8] SAT Solver was used in methods
MDD1, MDD2, MDD3 and LCG.
The results can be shown at Table 1. The number of times a solution has
been found within the time limit can be found at columns 2-4. Columns 5-7
present the number of times (within the timelimit) a method finds a solution
of cost at most best + 30, where best is the cost of the best solution found by
any method. Similarly, columns 8-10 and 11-13 contain the number of times a
solution of cost at most best + 20 and best + 10 has been found.
As we can see the new encoding substantially improves on previous encodings
of the problem. For these sports leagues scheduling problems it is well known
that other solving approaches do not compete with SAT encoding [2].

6 Conclusion and Future Work

We demonstrate a new method for encoding pseudo-Boolean constraints taking
into account implications chains. The improved encoding is beneficial on hard
benchmark problems. The approach is an extension on earlier work on encoding
linear constraints to SAT [5]. The approach is related to the propagator for
the increasing sum constraint y = a1 y1 + · · · + an yn ∧ y1 ≤ y2 ≤ · · · ≤ yn
described in [16], which combines a linear constraint with a “chain” of integer
inequalities. Interestingly, increasing sum is not directly encodable as an MDD
using the method herein, but it does suggest that the methods can be extended
to arbitrary sets of chains all compatible with a global variable order. Another
interesting direction for future work is to consider combining chains with the
sorting networks encodings of linear constraints (e.g. [11]).

Acknowledgments. NICTA is funded by the Australian Government as represented

by the Department of Broadband, Communications and the Digital Economy and the
Australian Research Council through the ICT Centre of Excellence program.
 Encoding Linear Constraints with Implication Chains to CNF 11

References
1. Abio, I., Mayer-Eichberge, V., Stuckey, P.: Encoding linear constraints with impli-
cation chains to CNF. Tech. rep., University of Melbourne (2015). http://www.
people.eng.unimelb.edu.au/pstuckey/papers/cp2015a.pdf
2. Abı́o, I.: Solving hard industrial combinatorial problems with SAT. Ph.D. thesis,
Technical University of Catalonia (UPC) (2013)
3. Abı́o, I., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E.: A parametric
approach for smaller and better encodings of cardinality constraints. In: Schulte,
C. (ed.) CP 2013. LNCS, vol. 8124, pp. 80–96. Springer, Heidelberg (2013)
4. Abı́o, I., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E., Mayer-
Eichberger, V.: A New Look at BDDs for Pseudo-Boolean Constraints. J. Artif.
Intell. Res. (JAIR) 45, 443–480 (2012)
5. Abı́o, I., Stuckey, P.J.: Encoding linear constraints into SAT. In: O’Sullivan, B.
(ed.) CP 2014. LNCS, vol. 8656, pp. 75–91. Springer, Heidelberg (2014)
6. Bailleux, O., Boufkhad, Y., Roussel, O.: New encodings of pseudo-boolean con-
straints into CNF. In: Kullmann, O. (ed.) SAT 2009. LNCS, vol. 5584, pp. 181–194.
Springer, Heidelberg (2009)
7. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability,
Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press (2009)
8. Bofill, M., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E., Rubio, A.: The
barcelogic SMT solver. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123,
pp. 294–298. Springer, Heidelberg (2008)
9. Dutertre, B., de Moura, L.: The YICES SMT Solver. Tech. rep., Computer Science
Laboratory, SRI International (2006). http://yices.csl.sri.com
10. Eén, N., Biere, A.: Effective preprocessing in SAT through variable and clause elim-
ination. In: Bacchus, F., Walsh, T. (eds.) SAT 2005. LNCS, vol. 3569, pp. 61–75.
Springer, Heidelberg (2005)
11. Eén, N., Sörensson, N.: Translating Pseudo-Boolean Constraints into SAT. Journal
on Satisfiability, Boolean Modeling and Computation 2(1–4), 1–26 (2006)
12. Gent, I.P., Prosser, P., Smith, B.M.: A 0/1 encoding of the GACLex constraint for
pairs of vectors. In: ECAI 2002 workshop W9: Modelling and Solving Problems
with Constraints. University of Glasgow (2002)
13. Heule, M.J.H., Järvisalo, M., Biere, A.: Efficient CNF simplification based on
binary implication graphs. In: Sakallah, K.A., Simon, L. (eds.) SAT 2011. LNCS,
vol. 6695, pp. 201–215. Springer, Heidelberg (2011)
14. de Moura, L., Bjorner, N.: Z3: An Efficient SMT Solver. Tech. rep., Microsoft
Research, Redmond (2007). http://research.microsoft.com/projects/z3
15. Ohrimenko, O., Stuckey, P., Codish, M.: Propagation via lazy clause generation.
Constraints 14(3), 357–391 (2009)
16. Petit, T., Régin, J.-C., Beldiceanu, N.: A Θ(n) bound-consistency algorithm for
the increasing sum constraint. In: Lee, Jimmy (ed.) CP 2011. LNCS, vol. 6876, pp.
721–728. Springer, Heidelberg (2011)
17. Srinivasan, A., Ham, T., Malik, S., Brayton, R.: Algorithms for discrete function
manipulation. In: 1990 IEEE International Conference on Computer-Aided Design,
ICCAD 1990, pp. 92–95. Digest of Technical Papers (1990)
 Anytime Hybrid Best-First Search with Tree
Decomposition for Weighted CSP

David Allouche, Simon de Givry(B) , George Katsirelos,

Thomas Schiex, and Matthias Zytnicki

MIAT, UR-875, INRA, F-31320 Castanet Tolosan, France

{david.allouche,simon.degivry,george.katsirelos,
thomas.schiex,matthias.zytnicki}@toulouse.inra.fr

Abstract. We propose Hybrid Best-First Search (HBFS), a search strat-

egy for optimization problems that combines Best-First Search (BFS)
and Depth-First Search (DFS). Like BFS, HBFS provides an anytime
global lower bound on the optimum, while also providing anytime upper
bounds, like DFS. Hence, it provides feedback on the progress of search
and solution quality in the form of an optimality gap. In addition, it
exhibits highly dynamic behavior that allows it to perform on par with
methods like limited discrepancy search and frequent restarting in terms
of quickly finding good solutions.
We also use the lower bounds reported by HBFS in problems with
small treewidth, by integrating it into Backtracking with Tree Decompo-
sition (BTD). BTD-HBFS exploits the lower bounds reported by HBFS in
individual clusters to improve the anytime behavior and global pruning
lower bound of BTD.
In an extensive empirical evaluation on optimization problems from
a variety of application domains, we show that both HBFS and BTD-
HBFS improve both anytime and overall performance compared to their
counterparts.

Keywords: Combinatorial optimization · Anytime algorithm ·

Weighted constraint satisfaction problem · Cost function networks ·
Best-first search · Tree decomposition

1 Introduction

Branch and Bound search is a fundamental tool in exact combinatorial optimiza-

tion. For minimization, in order to prune the search tree, all variants of Branch
and Bound rely on a local lower bound on the cost of the best solution below a
given node.
Depth-First Search (DFS) always develops a deepest unexplored node. When
the gap between the local lower bound and a global upper bound on the cost of an
optimal solution – usually provided by the best known solution – becomes empty,
backtrack occurs. DFS is often used in Constraint Programming because it offers


c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 12–29, 2015.
DOI: 10.1007/978-3-319-23219-5 2
 Anytime Hybrid Best-First Search with Tree Decomposition 13

polyspace complexity, it takes advantage of the incrementality of local consisten-

cies and it has a reasonably good anytime behavior that can be further enhanced
by branching heuristics. This anytime behavior is however largely destroyed in
DFS variants targeted at solving problems with a reasonable treewidth such as
BTD [7] or AND/OR search [6].
Best-First Search (BFS) instead always develops the node with the lowest
lower bound first. It offers a running global lower bound and has been proved to
never develop more nodes than DFS for the same lower bound [22]. But it has a
worst-case exponential space complexity and the optimal solution is always the
only solution produced.
An ideal Branch and Bound algorithm would combine the best of all
approaches. It would have a bearable space complexity, benefit from the incre-
mentality of local consistencies and offer both updated global upper and lower
bounds as the problem is solved. It would also not loose all its anytime qualities
when used in the context of treewidth sensitive algorithms such as BTD.
With updated global lower and upper bounds, it becomes possible to com-
pute a current global optimality gap. This gap can serve as a meaningful indi-
cator of search progress, providing a direct feedback in terms of the criteria
being optimized. This gap also becomes of prime importance in the context of
tree-decomposition based Branch and Bound algorithms such as BTD [7] as
global bounds for each cluster can typically be used to enhance pruning in other
clusters.
In this paper, we introduce HBFS, an hybrid, easy to implement, anyspace
Branch and Bound algorithm combining the qualities of DFS and BFS. The only
limitation of HBFS is that it may require to compromise the anytime updating
of the global lower bound for space. This can be achieved dynamically dur-
ing search. HBFS can also be combined with a tree-decomposition to define the
more complex BTD-HBFS, a BTD variant offering anytime solutions and updated
global optimality gap.
On a set of more than 3,000 benchmark problems from various sources
(MaxCSP, WCSP, Markov Random Fields, Partial Weighted MaxSAT) includ-
ing resource allocation, bioinformatics, image processing and uncertain reasoning
problems, we observe that HBFS improves DFS in term of efficiency, while being
able to quickly provide good solutions – on par with LDS and Luby restarts –
and a global running optimality gap. Similarly, HBFS is able to improve the
efficiency and anytime capacities of BTD.

2 Background
Our presentation is restricted to binary problems for simplicity. Our implemen-
tation does not have such restriction. A binary Cost Function Network (CFN) is
a triplet (X, D, W ). X = {1, . . . , n} is a set of n variables. Each variable i ∈ X
has a finite domain Di ∈ D of values than can be assigned to it. The maximum
domain size is d. W is a set of cost functions. A binary cost function wij ∈ W is
a function wij : Di × Dj → [0, k] where k is a given maximum integer cost cor-
responding to a completely forbidden assignment (expressing hard constraints).
14 D. Allouche et al.

If they do not exist, we add to W one unary cost function for every variable such
that wi : Di → [0, k] and a zero arity constraint w∅ (a constant cost payed by
any assignment, defining a lower bound on the optimum). All these additional
cost functions will have initial value 0, leaving the semantics of the problem
unchanged.
The Weighted Constraint Satisfaction Problem (WCSP)  is to find a
 n
minimum
 cost complete assignment: min (a1 ,...,an )∈ i Di {w∅ + i=1 wi (ai ) +
w (a
wij ∈W ij i j , a )}, an optimization problem with an associated NP-complete
decision problem.
The WCSP can be solved exactly using Branch and Bound maintaining some
lower bound: at each node ν of a tree, we use the local non naive lower bound
ν.lb = w∅ provided by a given soft arc consistency [5]. Each node corresponds to
a sequence of decisions ν.δ. The root node has an empty decision sequence. When
a node is explored, an unassigned variable is chosen and a branching decision
to either assign the variable to a chosen value (left branch, positive decision)
or remove the value from the domain (right branch, negative decision) is taken.
The number of decisions taken to reach a given node ν is the depth of the node,
ν.depth. A node of the search tree that corresponds to a complete assignment is
called a leaf. At this point, ν.lb is assumed to be equal to the node cost (which
is guaranteed by all soft arc consistencies).
The graph G = (X, E) of a CFN has
one vertex for each variable and one edge
(i, j) for every binary cost function wij ∈
W . A tree decomposition of this graph is
defined by a tree (C, T ). The set of nodes
of the tree is C = {C1 , . . . , Cm } where Ce
is a set of variables (Ce ⊆ X) called a clus-
ter. T is a set of edges connecting clusters
and forming a tree (a connected acyclic
graph). The set ofclusters C must cover
all the variables ( Ce ∈C Ce = X) and all
the cost functions (∀{i, j} ∈ E, ∃Ce ∈
C s.t. i, j ∈ Ce ). Furthermore, if a vari-
able i appears in two clusters Ce and Cg ,
i must also appear in all the clusters Cf on
the unique path from Ce to Cg in (C, T ). Fig. 1. A tree-decomposition of the
If the cardinality of the largest cluster in a CELAR06 radio frequency assignment
tree decomposition is ω + 1 then the width problem, rooted in C1 with subproblem
of the decomposition is ω. The treewidth P5 highlighted.
of a graph is the minimum width among
all its decompositions [24].

3 Hybrid Best-First Search

Classical BFS explores the search tree by keeping a list open of open nodes
representing unexplored subproblems. Initially, this list is reduced to the root
 Anytime Hybrid Best-First Search with Tree Decomposition 15

node at depth 0. Iteratively, a best node is explored: the node is removed and
replaced by its two left and right children with updated decisions, lower bound
and depth. In this paper we always choose as best node a node with the smallest
ν.lb, breaking ties by selecting a node with maximum ν.depth. The first leaf of the
tree explored is then guaranteed to be an optimal solution [14,22]. The list open
may reach a size in O(dn ) and, if incrementality in the lower bound computation
is sought, each node should hold the minimum data-structures required for soft
arc consistency enforcing (in O(ed) per node).
The pseudocode for Hybrid BFS is described as Algorithm 1. HBFS starts
with the empty root node in the list of open nodes. It then iteratively picks a
best node ν from the open list as above, replays all the decisions in ν.δ leading to
an assignment Aν , while maintaining consistency. It then performs a depth-first
search probe starting from that node for a limited number Z of backtracks. The
DFS algorithm is a standard DFS algorithm except for the fact that, when the
bound on the number of backtracks is reached, it places all the nodes corre-
sponding to open right branches of its current search state in the open list (see
Figure 2).
At the price of increased memory usage, this
1
hybrid maintains the advantages of depth-first
search. Since it spends a significant amount of its
time in a DFS subroutine, it can exploit the incre- 2
mentality of arc consistency filtering during DFS
search without any extra space cost: nodes in the 3 6
open list will just contain decisions δ and lower
bound lb, avoiding the extra O(ed) space required 4 5 7
for incrementality during BFS. However, each time
a node is picked up in open, the set of ν.depth 8
decisions must be “replayed” and local consistency
reinforced from the root node state, leading to Fig. 2. A tree that is par-
redundant propagation. This cost can be mitigated tially explored by DFS with
to some degree by merging all decisions into one. backtrack limit = 3. Nodes
Hence, a single fixpoint has to be computed rather with a bold border are
than ν.depth. Additionally, the cost can be fur- leaves, nodes with no border
ther reduced using other techniques employed by are placed in the open list
copying solvers [26]. Regardless of these mitigation after the backtrack bound is
techniques, some redundancy is unavoidable, hence exceeded. Nodes are num-
the number of backtracks performed at each DFS bered in the order they are
visited.
probe should be large enough to avoid excessive
redundancy.
Second, as it is allowed to perform Z backtracks in a depth-first manner
before it picks a new node, it may find new and better incumbent solutions, thus
it is anytime. The number of backtracks of each DFS probe should be sufficiently
small to offer quick diversification: by exploring a new best node, we are offered
the opportunity to reconsider early choices, similarly to what LDS [8] and Luby
16 D. Allouche et al.

randomized restarts [18] may offer. Additionally, with early upper bounds, we
can also prune the open node list and remove all nodes such that ν.lb ≥ ub.
To balance the conflicting objectives of reducing repeated propagation and
diversification, we dynamically adjust the amount of backtracks Z that can be
performed during one DFS probe by trying to keep the observed rate of redun-
dantly propagated decisions between reasonable bounds (α and β). In all the
algorithms here, we assume that the number of nodes (Nodes) and backtracks
(Backtracks) are implicitly maintained during search.

Function HBFS(clb,cub) : pair(integer,integer)

open := ν(δ = ∅, lb = clb) ;
while (open = ∅ and clb < cub) do
ν :=pop(open) /* Choose a node with minimum lower bound and maximum
depth */;
Restore state ν.δ, leading to assignment Aν , maintaining local consistency ;
NodesRecompute := NodesRecompute + ν.depth ;
cub :=DFS(Aν ,cub,Z)/* puts all right open branches in open */ ;
clb := max(clb, lb(open)) ;
if (NodesRecompute > 0) then
if (NodesRecompute/Nodes > β and Z ≤ N ) then Z := 2 × Z;
else if (NodesRecompute/Nodes < α and Z ≥ 2) then Z := Z/2;
return (clb, cub);

Algorithm 1. Hybrid Best-First Search. Initial call: HBFS(w∅ ,k) with Z = 1.

This hybrid does not preserve the polyspace complexity of DFS. However, it
can easily be made anyspace. If memory is exhausted (or a memory upper bound
is reached, with the same effect), the algorithm can switch from bounded DFS
to complete DFS. This means that for every node it picks from the open list, it
explores the entire subtree under that node. Hence, it will not generate any new
open nodes. It can continue in this mode of operation until memory pressure is
relieved.
Finally, this method computes stronger global lower bounds than DFS, as the
cost of a best node in the open list defines a global lower bound, as in BFS. DFS
instead cannot improve on the global lower bound computed at the root until it
finally visits the first right branch. In the context of a single instance this is only
important in the sense that it provides a better estimation of the optimality gap.
However, we will see that this can improve performance in decomposition-based
methods.

3.1 Related Work

Alternate search space exploration schemes have been proposed in the field of
heuristic search, as variations of A* search. These schemes can be applied to dis-
 Anytime Hybrid Best-First Search with Tree Decomposition 17

crete optimization, yielding other variants of best-first search. However, depth-

first search is not effective or even feasible in domains where A* search is used:
for example, it is possible in planning to have exponentially long sequences of
actions when short plans exist. Hence, methods like BRFSL(k) [27] can only do
bounded-depth DFS probes. Also, in contrast to HBFS, they do not insert the
open nodes of the DFS probes into the open list of BFS. Other methods like
Weighted best-first search [23], ARA* [16] and ANA* [2] weigh future assign-
ments more heavily in order to bias the search towards solutions. We do not
need to modify the branching heuristic in any way in HBFS.
Stratification [1], which solves a weighted MaxSAT instance by iteratively
considering larger subsets of its clauses, starting with those that have the high-
est weight, provides similar benefits to HBFS, as provides solutions quickly and
produces lower bounds. This techniques, however, can be viewed as a wrapper
over an optimization method and is therefore orthogonal to HBFS.
Alternate heuristics for choosing the next node to explore may yield different
algorithms. When we can identify a preferred value to assign at each choice point,
the discrepancy of a node ν is the number of right branches in the path from the
root to ν. If we always open the node with the smallest discrepancy, set Z = 1
and disable the adaptive heuristic, HBFS is identical to Limited Discrepancy
Search (LDS)1 [8].
In ILP, a closely related approach is so-called BFS with diving heuristics [3].
Such heuristics perform a single depth-first probe trying to find a feasible solu-
tion. Although the idea is quite close to that of HBFS, it is typically restricted
to a single branch, the open nodes it leaves are not added to the open node
file and is treated separately from the rest of the search process. This is in part
motivated by the fact that DFS is considered impractical in ILP [17] and by the
fact that the lower bounding method (LP) used is not as lightweight as those
used in WCSP.

4 Hybrid Best-First Search and Tree Decompositions

When the graph of a CFN has bounded treewidth, the O(dn ) worst-case com-
plexity of DFS can be improved using a tree decomposition of the CFN graph.
We can trivially observe that the tree decomposition can be rooted by selecting
a root cluster denoted C1 . The separator of a non root cluster Ce is Ce ∩ pa(Ce ),
where pa(Ce ) is the parent of Ce in T . Local consistency can be enforced on
e
the problem and provide a cluster-localized lower-bound w∅ for each cluster Ce .
The sum of these cluster-wise lower bounds is a lower bound for the complete
problem. Beyond this trivial observation, Terrioux and Jégou [28] and de Givry
et al. [7] have extended BTD [9] (which we call BTD-DFS here for clarity) from
pure satisfaction problems to the case of optimization (WCSP), in a way simi-
lar to AND/OR search [19]. Next, we briefly describe BTD-DFS, as given by de
Givry et al, as we base our own algorithm on this.
1
In WCSP optimization, we always have a non naive value heuristic that selects a
value (i, a) with minimum unary marginal cost wi (a) or better, the EAC support [13].
18 D. Allouche et al.

In BTD-DFS, by always assigning the variables of a cluster before the variables

of its descendant clusters, it is possible to exploit the fact that assigning a cluster
Ce separates all its child clusters children(Ce ). Each child cluster Cf is the root
of a subproblem Pf defined by the subtree rooted in Cf which becomes indepen-
dent of others. So, each subproblem Pf conditioned by the current assignment
Af of its separator, can be independently and recursively solved to optimality.
If we memoize the optimum cost of every solved conditioned subproblem Pe |Ae
in a cache, then Pe |Ae will never be solved again and an overall O(ndω+1 ) time
complexity can be guaranteed.
Although this simple strategy offers an attractive worst case theoretical
bound, it may behave poorly in practice. Indeed, each conditioned subprob-
lem Pe |Ae is always solved from scratch to optimality. This ignores additional
information that can be extracted from already solved clusters. Imagine Ce has
been assigned and that we have an upper bound ub (a solution) for the problem
Pe |Ae . Assume that Ce has two children Cf and Cf  and that we have solved the
e
first subproblem Pf |Af to optimality. By subtracting the lower bound w∅ and
the optimum of Pf |Af from ub, we obtain the maximum cost that a solution of
Pf  |Af  may have in order to be able to improve over ub. Instead of solving it
from scratch, we can solve Pf  |Af  with this initial upper bound and either find
an optimal solution – which can be cached – or fail. If we fail, we have proved
a global lower bound on the cost of an optimal solution of Pf  |Af  . This lower
bound can be cached and prevent repeated search if Pf  |Af  is revisited with
the same or a lower initial upper bound. Otherwise, the problem will be solved
again and again either solved to optimality or fail and provide an improved global
lower bound. This has been shown to improve search in practice while offering
a theoretical bound on time complexity in O(kn.dω+1 ) (each time a subproblem
Pf |Af is solved again, the global lower bound increases at least by 1).
In practice, we therefore cache two values, LB Pe |Ae and UB Pe |Ae , for every
visited assignment Ae of the separator of every cluster Ce . We always assume
caching is done implicitly: LB Pe |Ae is updated every time a stronger lower bound
is proved for Pe |Ae and UB Pe |Ae when an updated upper bound is found. When
an optimal solution is found and proved to be optimal, we will therefore have
LB Pe |Ae = UB Pe |Ae . Thanks to these cached bounds and to the cluster-wise local
e
lower bounds w∅ , an improved local lower bound lb(Pe |Ae ) for the subproblem
Pe |Ae can be computed by recursively summing the maximum of the cached and
local bound (see [7]).
We show pseudocode for the resulting algorithm combining BTD and DFS in
Algorithm 2. Grayed lines in this code are not needed for the DFS variant and
should be ignored. The algorithm is called on root cluster C1 , with an assignment
A1 = ∅, a set of unassigned variables V = C1 and initial lower and upper bound
clb and cub set respectively to lb(P1 |∅) and k (the maximum cost). The last
argument, RecCall is a functional argument that denotes which function will be
used to recurse inside BTD-DFS. Here, RecCall will be initially equal to BTD-
DFS itself. The algorithm always returns two identical values equal to the current
 Anytime Hybrid Best-First Search with Tree Decomposition 19

Function BTD-DFS(A,Ce ,V ,clb,cub,RecCall ) : pair(integer,integer)

if (V = ∅) then
i :=pop(V ) /* Choose an unassigned variable in Ce */ ;
a :=pop(Di ) /* Choose a value */ ;
Assign a to i, maintaining local consistency on subproblem lb(Pe |A ∪ {(i = a)}) ;
clb  := max(clb, lb(Pe |A ∪ {(i = a)})) ;
if (clb < cub) then
(cub, cub) := BTD-DFS(A ∪ {(i = a)}, Ce , V − {i}, clb , cub,RecCall );
Ce .backtracks := Ce .backtracks + 1;
if (max(clb, lb(Pe |A)) < cub) then
Remove a from i, maintaining local consistency on subproblem
lb(Pe |A ∪ {(i = a)}) ;
clb  := max(clb, lb(Pe |A ∪ {(i = a)})) ;
if (clb < cub) then
if (Ce .backtracks < Ce .limit and Backtracks < Pe .limit) then
(cub, cub) := BTD-DFS(A ∪ {(i = a)}, Ce , V , clb , cub,RecCall );
else /* Stop depth-first search */
Push current search node in open list of Pe |A at position clb ;
else
S := Children(Ce ) ;
/* Solve all clusters with non-zero optimality gap and unchanged lb or ub */ ;
while (S = ∅ and lb(Pe |A) < cub) do
Cf :=pop(S) /* Choose a child cluster */ ;
if (LB Pf |A < UB Pf |A ) then
cub  := min(UB Pf |A , cub − [lb(Pe |A) − lb(Pf |Af )]) ;
(clb  , cub  ) := RecCall (A, Cf , Cf , lb(Pf |Af ), cub ,RecCall );
Update LB Pf |A and UB Pf |A using clb  and cub  ;
e 
cub := min(cub, w∅ + Cf ∈Children(Ce ) UB Pf |A );
if max(clb, lb(Pe |A)) < cub then
Push current search node in open list of Pe |A at position max(clb, lb(Pe |A)) ;
Ce .limit := Ce .backtracks /* Stop depth-first search */ ;
return (cub, cub)

Algorithm 2. BTD using depth-first search

upper bound.2 Caches are initially empty and return naive values LB Pe /A = 0
and UB Pe /A = k for all clusters and separator assignments.

4.1 Using HBFS in BTD

BTD-DFS has two main disadvantages: first, it has very poor anytime behavior,
as it cannot produce a solution in a decomposition with k leaves until k − 1 leaf
clusters have been completely solved. This affects the strength of pruning, as
2
This is clearly redundant for BTD-DFS, but allows a more uniform presentation with
BTD-HBFS.
20 D. Allouche et al.

values are only pruned if the current lower bound added to the marginal cost of
the value exceeds the upper bound. Second, because child clusters are examined
in order, only the lower bounds of siblings earlier than Cf in that order can
contribute to pruning in Cf . For example, consider a cluster Ce with 3 child
e
clusters Cf1 , Cf2 , Cf3 . Assume that ub = 31 and under an assignment A, w∅ has
known cost 10 while Pf1 |Af1 , Pf2 |Af2 and Pf3 |Af3 all have optimal cost 10, and
lb(Pf1 |Af1 ) = lb(Pf2 |Af2 ) = lb(Pf3 |Af3 ) = 0. Clearly the subproblem under Ce
cannot improve on the upper bound, but when we solve Cf1 and Cf2 , BTD-DFS
does not reduce the effective upper bound at all. However, it may be relatively
easy to prove a lower bound of 7 for each of the child clusters. If we had this
information, we could backtrack.
HBFS has the ability to quickly provide good lower and upper bounds and inter-
rupts itself as soon as the limit number of backtracks is reached. Using HBFS instead
of DFS in BTD should allow to quickly probe each subproblem to obtain interme-
diate upper and lower bounds for each of them. The upper bounds can be used to
quickly build a global solution, giving anytime behavior to BTD. The lower bounds
of all subproblems can be used to improve pruning in all other clusters.
The pseudocode of BTD-HBFS is described as Algorithm 3. It takes the same
arguments as BTD-DFS but ignores the last one (used only to pass informa-
tion from BTD-HBFS to BTD-DFS). BTD-HBFS relies on BTD-DFS pseudocode,
assuming that all grayed lines of BTD-DFS are active. These reactivated lines
in Algorithm 2 impose per-cluster and per-subproblem backtrack limits. Every
cluster Ce has a counter Ce .backtracks for number of backtracks performed inside
the cluster and an associated limit Ce .limit. Every subproblem Pe has a limit
Pe .limit on the number of backtracks N performed inside the subproblem. Ini-
tially, Ce .limit = Pe .limit = ∞ for all Ce ∈ C.
Every subproblem Pe |Ae has its own list of open nodes Pe |Ae .open for each
upper bound which it is given. The value of the upper bound participates in
the definition of the actual search space that needs to be explored. If the same
subproblem is revisited later with a lower upper bound, then the search space
shrinks and we can just copy the open list associated with the higher bound and
prune all nodes ν such that ν.lb ≥ ub. But if the upper bound is more relaxed
than any previous upper bound then we need to create a new open list starting
with the root node.
Finally, the loop in line 1 is interrupted as soon as the optimality gap reduces
or the number of backtracks reaches the subproblem limit, making the search
more dynamic. If subproblems quickly update their bound, the remaining back-
tracks can be used in a higher cluster. However, the subproblem under each
child cluster is guaranteed to get at least Z backtracks. The result is that we
spend most of the time in leaf clusters. When one cluster exhausts its budget,
the search quickly returns to the root cluster.

Example 1. Consider the example in figure 3. We have a CFN with the tree
decomposition given in the box labeled (C, T ) and the search in each cluster is
shown in a box labeled by that cluster’s name. Let N = 2 and Z = 1 in this
example. The search visits nodes as they are numbered in the figure. When it
 Anytime Hybrid Best-First Search with Tree Decomposition 21

reaches node 4, cluster C1 is completely instantiated and hence it descends into

C2 and after node 7 it descends into C4 . After node 10, we have performed a
backtrack in this cluster, and since Z = 1 we end this DFS probe and return
control to BTD-HBFS. The limit on the number of backtracks in P4 is still not
exceeded, so we choose a new node from the open list, node 11, a conflict. Again
we return control to BTD-HBFS and, having exceeded the backtrack limit on
P4 , exit this cluster, but with an improved lower bound. Since C4 exceeded its
backtrack limit before improving its lower bound, no more search is allowed in
parent clusters. The search is allowed, however, to visit sibling clusters, hence it
explores C5 (nodes 12–14), which it exits with an improved upper bound before
exceeding its backtrack limit, and C3 (nodes 15–18). Once it returns to node 7
after cluster C5 , that node is not closed, because one of the child clusters is not
closed. It is instead put back on the open list. Similarly node 4 is put back on
the open list of C1 . At that point, best-first search picks another node from the
open list of C1 , node 19, and continues from there. 


Function BTD-HBFS(A,Ce ,V ,clb,cub, ) : pair(integer,integer)

open := open list of Pe |A(cub) ;
if (open = ∅) then
if exists minimum cub s.t. cub > cub and open(Pe |A(cub )) = ∅ then
open = {ν ∈ open(Pe |A(cub )) | ν.lb < cub}
else
open = {∅} /* Contains only the root node at position clb */
Pe .limit := Backtracks + N /* Set a global backtrack limit for the subproblem */ ;
clb  := max(clb, lb(open)) ;
cub  := cub ;
1 while (open = ∅ and clb  < cub  and (Ce = C1 or (clb  = clb and cub  =
cub and Backtracks < Pe .limit))) do
ν :=pop(open) /* Choose a node with minimum lower bound and maximum
depth */ ;
Restore state ν.δ, leading to assignment Aν , maintaining local consistency ;
NodesRecompute := NodesRecompute + ν.depth ;
Ce .limit := Ce .backtracks + Z /* Set a depth-first search backtrack limit */ ;
(cub  , cub  ) :=BTD-DFS(Aν ,Ce ,Vν ,max(clb  , lb(ν), lb(Pe |Aν )),cub  ,BTD-HBFS) ;
clb  := max(clb  , lb(open)) ;
if (NodesRecompute > 0) then
if (NodesRecompute/Nodes > β and Z ≤ N ) then Z := 2 × Z;
else if (NodesRecompute/Nodes < α and Z ≥ 2) then Z := Z/2;
return (clb  , cub  ) /* invariant clb  ≥ clb and cub  ≤ cub */ ;

Algorithm 3. Hybrid Best-First Search with Tree Decomposition.

BTD-HBFS addresses both the issues of BTD that we identified above. First, it is
anytime, because as soon as UB Pe |Ae < k for all subproblems, we can combine the
assignments that gave these upper bounds to produce a global solution. Second, it
constantly updates lower bounds for all active subproblems, so the search effort in
each subproblem immediately exploits all other lower bounds.
22 D. Allouche et al.

(C, T ) C1
C1 1

C2 C3 2 19

C4 C5 3

C2 /A1 C3 /A1
5 15

6 16 17

7 18

C4 /A2 C5 /A2
8 12

9 11 13

10 14

Fig. 3. An example of a run of BTD-HBFS. White nodes are open, nodes with a black
border are conflicts (square) or solutions (circle). Grey nodes with a white border are
explored but put back in the open list. Grey nodes with no border are closed.

Like HBFS, BTD-HBFS can be made anyspace, i.e., its memory usage can be
limited to any amount beyond what is needed for BTD-DFS, including zero. The
cache of bounds can also be limited, at the expense of additional subproblem
recomputations, leading to worst-case complexity exponential in the tree decom-
position height.

Theorem 1. Given a CFN P with treewidth ω, BTD-HBFS computes the opti-

mum in time O(kndω+1 ) and space O(knd2ω ).

Proof (Sketch). For correctness, observe that BTD-DFS solves independent sub-
problems separately using DFS, hence using HBFS or any other solution method
does not affect correctness. Each leaf node in an internal cluster is closed only
 Anytime Hybrid Best-First Search with Tree Decomposition 23

when all child clusters are solved, hence all bounds for each subproblem and
open node list are correct. Finally, exploration at the root cluster continues until
the optimality gap is closed. Complexity stems from the complexity of BTD and
the additional overhead of storing open lists for each separator assignment and
upper bound. 


We implemented a simpler version of this algorithm with better space com-

plexity: each time BTD-HBFS is called on Pe |A with a higher upper bound than
previously stored, we wipe the open list and replace it with the root node of Ce .
This removes theoretical guarantees on the performance of the algorithm, but
does not hurt practical performance, as we will see.

4.2 Related Work

AND/OR Branch and Bound search has already been combined with BFS [20].
The resulting AOBF algorithm, has good worst-case time complexity similar to
BTD-DFS, but otherwise has the space-intensive non anytime behavior of BFS.
The poor anytime ability of BTD has been addressed by breadth-rotating
AND/OR search (BRAO) [21]. BRAO interleaves DFS on all components, so
it can combine the incumbents of all components to produce a global solution.
However, as it performs DFS on each component, it does not produce better
lower bounds.
OR-decomposition [12] is an anytime method that exploits lower bounds
produced by other clusters by performing DFS in which it interleaves variable
choices from all components, and uses caching to achieve the same effect as BTD.
However, the global lower bound it computes depends on the partial assignments
of all components. Thus it may revisit the same partial assignment of one com-
ponent many times. This may also inhibit its anytime behavior, as a high cost
partial assignment in one component will prevent other components from reach-
ing good solutions. Moreover, the local lower bound for each component is only
updated by visiting the right branch at its root.
Russian Doll Search [25], uses DFS to solve each cluster of a rooted tree
decomposition in topological order. This method is not anytime, as it cannot
produce a solution until it starts solving the root cluster. Moreover, it computes
lower bounds that are independent of the separator assignment, hence can be
lower than their true value.

5 Experimental Results

We used benchmark instances including stochastic graphical models from the

UAI evaluation in 2008 and 2010, the Probabilistic Inference Challenge 2011,
the Weighted Partial Max-SAT Evaluation 2013, the MiniZinc Challenge 2012
and 2013, Computer Vision and Pattern Recognition problems from OpenGM23

3
http://hci.iwr.uni-heidelberg.de/opengm2/
24 D. Allouche et al.

and additional instances from the CostFunctionLib4 . This is a total of more than
3,000 instances that can be encoded as Cost Function Networks, available at
http://genoweb.toulouse.inra.fr/∼degivry/evalgm, with domain sizes that range
from d = 2 to 503, n = 2 to 903, 884 variables, and e = 3 to 2, 912, 880 cost
functions of arity from r = 2 to 580. We used toulbar2 version 0.9.8.0-dev5 on
a cluster of 48-core Opteron 6176 nodes at 2.3 GHz with 378 GB RAM, with a
limit of 24 simultaneous jobs per node.
In all cases, the local lower bound is provided by maintaining EDAC [13].
The variable ordering includes both weighted-degree [4] and last-conflict [15]
heuristics. The value ordering is to select the EAC support value first [13]. All
executions used a min-fill variable ordering for DAC preprocessing. For HBFS, we
set the node recomputation parameters to [α, β] = [5%, 10%] and the backtrack
limit N to 10, 000.
The methods based on BTD use a different value ordering heuristic: if a
solution is known for a cluster, it keeps the same value if possible and if not
uses EAC support values as the previous methods. A min-fill ordering is used
for building a tree decomposition. Children of a cluster are statically sorted by
minimum separator size first and smallest number of subproblem variables next.
Our aim is to determine whether HBFS is able to improve over DFS both in
terms of number of problems solved (including the optimality proof) and in its
anytime behavior. Similarly, we compare BTD-HBFS to BTD-DFS. We include in
our comparison two methods that are known to significantly improve the upper
bound anytime behavior of DFS: Limited Discrepancy Search [8] and DFS with
Luby restarts [18].
We also include results from BRAO [21] using the daoopt solver6 with static
mini-bucket lower bounds of different strength (i-bound set to 15 and 35) and
without local search nor iterative min-fill preprocessing. daoopt is restricted to
cost functions expressed by complete tables, hence we exclude most MaxSAT
families (except MIPLib and MaxClique) in tests where we use it.

5.1 Proving Optimality

We show in figure 4 a cactus plot comparing all the algorithms that do not use
tree decompositions, but also include BTD-HBFSrk for reference (see below). We
see that HBFS is the best performing decomposition-unaware algorithm. It out-
performs DFS and DFS with Luby restarts significantly, and slightly outperforms
LDS.
Although our benchmark set includes very large instances, and our HBFS
implementation does not include automatic control of space usage, no instance
required more than 32 GB. The median memory usage was 36.8 MB for DFS
and 38.2 MB for hybrid BFS. The worst-case largest ratio between HBFS and

4
https://mulcyber.toulouse.inra.fr/projects/costfunctionlib
5
Available in the git repository at https://mulcyber.toulouse.inra.fr/projects/
toulbar2/ in the bfs branch.
6
https://github.com/lotten/daoopt
 Anytime Hybrid Best-First Search with Tree Decomposition 25

Fig. 4. Number of solved instances within a given time. Methods in the legend are
sorted at time=20min.

B
12.1M B = 31.4 on MRF Grid instance grid20x20.f15 (unsolved in
DFS was 379.8M
one hour by both methods).
In figure 5, we compare algorithms exploiting a tree decomposition (BTD-
like) or a pseudo-tree (BRAO).We see that BTD-HBFS slightly outperforms BTD-
DFS, both outperforming BRAO. However, many of these instances have large
treewidth and BTD-like methods are not ideal for these. Even for instances with
small treewidth, the decomposition is often a deep tree in which each cluster
shares all but one variables with its parent. In these cases, following the tree
decomposition imposes a static variable ordering on BTD, while HBFS degrades
to DFS. Finding good tree decompositions is not straightforward [10,11]. A sim-
ple way to improve one is to merge clusters until no separator has size greater
than k, even if this increases width. We call the algorithms that apply this
BTD-DFSrk and BTD-HBFSrk . Figure 5 includes results for BTD-DFSr4 and BTD-
HBFSr4 . BTD-DFSr4 is significantly better than BTD-DFS and BTD-HBFSr4 out-
performs BTD-DFSr4 by an even greater margin. BTD-HBFSr4 is also the overall
best performer as shown in figure 4.

5.2 Anytime Behavior

To analyze the algorithms’ anytime behavior, we first show in figure 6 the evo-
lution of the lower and upper bounds for two instances: the SPOT5 404 (left)
and the RLFAP CELAR06 instances (right). We solve both instances using DFS,
LDS, DFS with Luby restarts, HBFS, BTD-DFS and BTD-HBFS. In both instances,
we see that HBFS and BTD-HBFS improve significantly on the upper bound any-
time ability of DFS and BTD-DFS, respectively. Moreover, the lower bound that
they report increases quickly in the beginning and keeps increasing with time.
For all other algorithms, the lower bound increases by small amounts and infre-
quently, when the left branch of the root node is closed. The HBFS variants are
as fast as the base algorithms in proving optimality.
26 D. Allouche et al.

Fig. 5. Number of solved instances as time passes on a restricted benchmark set

(without MaxSAT). Methods in the legend are sorted at time=20min.
130
HBFS
DFS-Luby
120 DFS-LDS
DFS
110

100

90

80

70

60
0.1 1 10 100 1000

Fig. 6. Evolution of the lower and upper bounds (Y axis, in cost units) as time (X axis,
in seconds) passes for HBFS, Luby restart, LDS, and DFS on SPOT5 404 instance (left)
and also BTD, and BTD-HBFS for the RLFAP CELAR06 instance (right). Methods are
sorted in increasing time to find the optimum. For each curve, the first point represents
the time where the optimum is found and the second point the time (if any) of proof
of optimality.

In figure 7, we summarize the evolution of lower and upper bounds for each
algorithm over all instances that required more than 5 sec to be solved by DFS.
Specifically, for each instance I we normalize all costs as follows: the initial
lower bound produced by EDAC (which is common to all algorithms) is 0; the
best – but potentially suboptimal – solution found by any algorithm is 1; the
worst solution is 2. This normalization is invariant to translation and scaling.
Additionally, we normalize time from 0 to 1 for each pair of algorithm A and
instance I, so that preprocessing ends at time 0 and each run finishes at time
1. This time normalization is different for different instances and for different
algorithms on the same instance. A point x, y on the lower bound line for
algorithm A in figure 7 means that after normalized runtime x, algorithm A has
proved on average over all instances a normalized lower bound of y and similarly
for the upper bound. We show both the upper and lower bound curves for all
 Anytime Hybrid Best-First Search with Tree Decomposition 27

Fig. 7. Average evolution of normalized upper and lower bounds for each algorithm.

algorithms evaluated here. In order for the last point of each curve to be visible,
we extend all curves horizontally after 1.0.
This figure mostly ignores absolute performance in order to illustrate the
evolution of upper and lower bounds with each algorithm, hence cannot be inter-
preted without the additional information provided by the cactus plots in figures
4 and 5. It confirms that HBFS improves on DFS in terms of both upper and
lower bound anytime behavior and similarly for BTD-HBFSr4 over BTD-DFSr4
and BRAO, with the latter being especially dramatic. The two HBFS variants
are, as expected, significantly better than all other algorithms in terms of the
lower bounds they produce. HBFS and BTD-HBFSr4 produce solutions of the
same quality as LDS, while DFS-Luby is slightly better than this group on this
restricted benchmark set (without MaxSAT).
Despite the fact that time to solve an instance is normalized away in figure
7, it does give some information that is absent from the cactus plots and that
is the average normalized lower and upper bounds at time 1. Figure 7 tells us
that DFS-Luby finds the best solution most often, as its upper bound curve is
the lowest at time 1. It is followed closely by the HBFS variants and LDS, while
DFS and BTD-DFSr4 are significantly worse. On the other hand, DFS-Luby is
significantly worse than the HBFS variants in the cactus plot. HBFS and BTD-
HBFSr4 give better lower bounds in those instances that they failed to solve, so
their lower bound curves are higher at point 1.

6 Conclusions
Hybrid BFS is an easily implemented variant of the Branch and Bound algo-
rithm combining advantages of BFS and DFS. While being a generic strategy,
applicable to essentially any combinatorial optimization framework, we used it
to improve Depth-First Branch and Bound maintaining soft arc consistency and
28 D. Allouche et al.

tested it on a large benchmark set of problems from various formalisms, includ-

ing Cost Function Networks, Markov Random Field, Partial Weighted MaxSAT
and CP instances representing a variety of application domains in bioinformat-
ics, planning, resource allocation, image processing and more. We showed that
HBFS improves on DFS or DFS equipped with LDS or restarts in terms of num-
ber of problems solved within a deadline but also in terms of anytime quality
and optimality gap information.
HBFS is also able to improve Tree Decomposition aware variants of DFS such
as BTD, being able to solve more problems than the previous DFS based BTD
on the same set of benchmarks. BTD is targeted at problems with relatively
low treewidth and has been instrumental in solving difficult radio-link frequency
assignment problems. On such problems, BTD-HBFS provides to BTD the same
improvements as to DFS.
Its ability to provide feedback on the remaining search effort, to describe the
current remaining search space in a list of open nodes and to decompose search
in self-interrupted DFS probes makes it a very dynamic search method, very
attractive for implementing multi-core search.

Acknowledgments. We are grateful to the Genotoul (Toulouse) Bioinformatic plat-

form for providing us computational support for this work.

References
1. Ansótegui, C., Bonet, M.L., Gabàs, J., Levy, J.: Improving sat-based weighted
maxsat solvers. In: Proc. of CP 2012, Québec City, Canada, pp. 86–101 (2012)
2. van den Berg, J., Shah, R., Huang, A., Goldberg, K.: ANA*: Anytime nonparamet-
ric A*. In: Proceedings of Twenty-Fifth AAAI Conference on Artificial Intelligence
(AAAI 2011) (2011)
3. Berthold, T.: Primal heuristics for mixed integer programs. Master’s thesis, Tech-
nischen Universität Berlin (2006). urn:nbn:de:0297-zib-10293
4. Boussemart, F., Hemery, F., Lecoutre, C., Sais, L.: Boosting systematic search by
weighting constraints. In: ECAI, vol. 16, p. 146 (2004)
5. Cooper, M., de Givry, S., Sanchez, M., Schiex, T., Zytnicki, M., Werner, T.: Soft
arc consistency revisited. Artificial Intelligence 174(7), 449–478 (2010)
6. Dechter, R., Mateescu, R.: And/or search spaces for graphical models. Artificial
Intelligence 171(2), 73–106 (2007)
7. de Givry, S., Schiex, T., Verfaillie, G.: Exploiting tree decomposition and soft local
consistency in weighted CSP. In: Proc. of the National Conference on Artificial
Intelligence, AAAI 2006, pp. 22–27 (2006)
8. Harvey, W.D., Ginsberg, M.L.: Limited discrepency search. In: Proc. of the 14th
IJCAI, Montréal, Canada (1995)
9. Jégou, P., Terrioux, C.: Hybrid backtracking bounded by tree-decomposition of
constraint networks. Artif. Intell. 146(1), 43–75 (2003)
10. Jégou, P., Terrioux, C.: Combining restarts, nogoods and decompositions for solv-
ing csps. In: Proc. of ECAI 2014, Prague, Czech Republic, pp. 465–470 (2014)
11. Jégou, P., Terrioux, C.: Tree-decompositions with connected clusters for solving
constraint networks. In: Proc. of CP 2014, Lyon, France, pp. 407–423 (2014)
 Anytime Hybrid Best-First Search with Tree Decomposition 29

12. Kitching, M., Bacchus, F.: Exploiting decomposition in constraint optimization

problems. In: Stuckey, P.J. (ed.) CP 2008. LNCS, vol. 5202, pp. 478–492. Springer,
Heidelberg (2008)
13. Larrosa, J., de Givry, S., Heras, F., Zytnicki, M.: Existential arc consistency: get-
ting closer to full arc consistency in weighted CSPs. In: Proc. of the 19th IJCAI,
pp. 84–89, Edinburgh, Scotland (August 2005)
14. Lawler, E., Wood, D.: Branch-and-bound methods: A survey. Operations Research
14(4), 699–719 (1966)
15. Lecoutre, C., Saı̈s, L., Tabary, S., Vidal, V.: Reasoning from last conflict(s) in
constraint programming. Artificial Intelligence 173, 1592–1614 (2009)
16. Likhachev, M., Gordon, G.J., Thrun, S.: ARA*: Anytime A* with provable bounds
on sub-optimality. In: Advances in Neural Information Processing Systems, p. None
(2003)
17. Linderoth, J.T., Savelsbergh, M.W.: A computational study of search strategies
for mixed integer programming. INFORMS Journal on Computing 11(2), 173–187
(1999)
18. Luby, M., Sinclair, A., Zuckerman, D.: Optimal speedup of las vegas algorithms. In:
Proceedings of the 2nd Israel Symposium on the Theory and Computing Systems,
pp. 128–133. IEEE (1993)
19. Marinescu, R., Dechter, R.: AND/OR branch-and-bound for graphical models. In:
Proc. of IJCAI 2005, Edinburgh, Scotland, UK, pp. 224–229 (2005)
20. Marinescu, R., Dechter, R.: Best-first AND/OR search for graphical models. In:
Proceedings of the National Conference on Artificial Intelligence, pp. 1171–1176.
AAAI Press, MIT Press, Menlo Park, Cambridge (1999, 2007)
21. Otten, L., Dechter, R.: Anytime and/or depth-first search for combinatorial opti-
mization. AI Communications 25(3), 211–227 (2012)
22. Pearl, J.: Heuristics – Intelligent Search Strategies for Computer Problem Solving.
Addison-Wesley Publishing Comp. (1985)
23. Pohl, I.: Heuristic search viewed as path finding in a graph. Artificial Intelligence
1(3), 193–204 (1970)
24. Robertson, N., Seymour, P.D.: Graph minors. II. Algorithmic aspects of tree-width.
Journal of Algorithms 7(3), 309–322 (1986)
25. Sanchez, M., Allouche, D., de Givry, S., Schiex, T.: Russian doll search with tree
decomposition. In: IJCAI, pp. 603–608 (2009)
26. Schulte, C.: Comparing trailing and copying for constraint programming. In: Logic
Programming, Las Cruces, New Mexico, USA, pp. 275–289 (1999)
27. Stern, R., Kulberis, T., Felner, A., Holte, R.: Using lookaheads with optimal best-
first search. In: AAAI (2010)
28. Terrioux, C., Jégou, P.: Bounded backtracking for the valued constraint satisfaction
problems. In: Rossi, F. (ed.) CP 2003. LNCS, vol. 2833, pp. 709–723. Springer,
Heidelberg (2003)
 Improved Constraint Propagation via
Lagrangian Decomposition

David Bergman1 , Andre A. Cire2 , and Willem-Jan van Hoeve3(B)

1
School of Business, University of Connecticut, Mansfield, USA
[email protected]
2
University of Toronto Scarborough, Toronto, Canada
[email protected]
3
Tepper School of Business, Carnegie Mellon University, Pittsburgh, USA
[email protected]

Abstract. Constraint propagation is inherently restricted to the local

information that is available to each propagator. We propose to improve
the communication between constraints by introducing Lagrangian
penalty costs between pairs of constraints, based on the Lagrangian
decomposition scheme. The role of these penalties is to force variable
assignments in each of the constraints to correspond to one another. We
apply this approach to constraints that can be represented by decision
diagrams, and show that propagating Lagrangian cost information can
help improve the overall bound computation as well as the solution time.

1 Introduction
Modern finite-domain constraint programming (CP) solvers employ a constraint
propagation process in which domain changes for the variables are propagated
between constraints. To allow for more communication and knowledge sharing
between constraints, several techniques have been proposed. One possibility is to
propagate more structural information than variable domains, such as (relaxed)
decision diagrams [1,10]. Another option, in the context of optimization prob-
lems, is to combine constraints with the objective function, and utilize mathe-
matical programming relaxations for stronger cost-based filtering [7,15]. These
approaches, however, have in common that consistency checks are done sepa-
rately and independently for each constraint. Higher-order consistencies, such as
pairwise consistency [12] can consider multiple constraints simultaneously, but
may suffer from a relatively high computational cost.
We propose an alternative, and generic, approach to improve the propaga-
tion between constraints based on Lagrangian decomposition [9]. In Lagrangian
decomposition, the constraint set of a given problem is partitioned into struc-
tured subproblems, each of which is defined on a duplicate copy of the original
variables. To link the subproblems, constraints are added to ensure that each
of the duplicates is equal to the original variable. These latter constraints are
then relaxed with an associated Lagrangian multiplier, and moved to the objec-
tive. This results in independent subproblems that can be separately optimized.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 30–38, 2015.
DOI: 10.1007/978-3-319-23219-5 3
 Improved Constraint Propagation via Lagrangian Decomposition 31

Intuitively, the idea is to force the variables to take the same value in each con-
straint, via the Lagrangian penalties, which are iteratively updated. This will
somehow synchronize the consistency checks for each of the constraints; instead
of allowing each constraint to check its consistency w.r.t. an arbitrary tuple, we
iteratively arrive at tuples with minimal disagreement.
Since constraint programming has been designed to work with (global) con-
straints that capture a specific structure of the problem, the application of
Lagrangian decomposition in this context seems natural and promising. Indeed,
we show that the Lagrangian decomposition is not only useful to improve the
bound on the objective, but can also be applied for cost-based domain filtering.
The structure of the paper is as follows. We first provide an overview of the
most relevant related work in Section 2. In Section 3 we recall the Lagrangian
decomposition scheme. We apply this to constraint programming models in
Section 4. Experimental results on instances with multiple alldiff constraints
are given in Section 5, while Section 6 provides results on set covering problems.
We conclude in Section 7.

2 Related Work
Lagrangian relaxations have been widely applied in operations research as well
as constraint programming. One of the first applications in CP is the work by
Benoist et al. [2] on the Traveling Tournament Problem. A formal treatment
was provided by Sellmann [16] who showed that optimal Lagrangian multipliers
may not result in the most effective domain filtering. Recently, [8] introduced
a framework for automated Lagrangian relaxation in a constraint programming
context. That work explicitly generalizes Lagrangian relaxations to CP problems
using measures of constraint violations, or degrees of satisfiability.
Adapting weights for improving propagation has also been applied in the
context of Valued Constraint Satisfaction Problems [6]. In that work, a lin-
ear programming model is proposed for computing Optimal Soft Arc Consis-
tency, but Lagrangian relaxations are not used. Khemmoudj et al. [13] combine
arc consistency with Lagrangian relaxation for filtering constraint satisfaction
problems (CSPs). They consider binary CSPs (i.e., each constraint has at most
two variables in its scope) in extensional form. Lastly, Bergman et al. [3] intro-
duce Lagrangian relaxations in the context of propagating (relaxed) decision
diagrams.

3 Lagrangian Decomposition
Lagrangian decomposition has been introduced to strengthen Lagrangian bounds
for integer linear optimization problems [9]. Consider an integer linear program
of the form:
(P ) max{f x | Ax ≤ b, Cx ≤ d, x ∈ X},
for some feasible set X, where x ∈ Rn is a vector of variables, f ∈ Rn represents
a ‘weight’ vector, A and C represent constraint coefficient matrices, and b and
32 D. Bergman et al.

c are constant right-hand size vectors. This is equivalent to the reformulated

program
max{f x | Ay ≤ b, Cx ≤ d, x = y, x ∈ X, y ∈ Y },
for any set Y containing X.
The Lagrangian decomposition of P consists in dualizing the equality con-
straints x = y with Lagrangian multipliers λ ∈ Rn :

LP (λ) := max{f x + λ(y − x) | Cx ≤ d, x ∈ X, Ay ≤ b, y ∈ Y }

= max{(f − λ)x | Cx ≤ d, x ∈ X} + max{λy | Ay ≤ by ∈ Y }

The Lagrangian dual is to find those Lagrangian multipliers λ that provide the
best bound:
min LP (λ).
λ

Guignard and Kim [9] show that the optimal bound obtained from this
Lagrangian decomposition is at least as strong as the standard Lagrangian
bounds from dualizing either Ax ≤ b or Cx ≤ d. Lagrangian decomposition may
be particularly useful when the problem is composed of several well-structured
subproblems, such as those defined by (global) constraints in CP models.

4 Application to Constraint Programming

We apply Lagrangian decomposition to constraint optimization problems

(COPs), which include constraint satisfaction problems (CSPs) as special case.
It is important to note that this approach will transform each of the original
constraints into an ‘optimization constraint’; instead of representing a witness
for feasibility the constraint now has to represent a witness for optimality, even
if the constraint is not directly linked to the objective function, or in case of
feasibility problems.
When the variables have numeric domains, the method from Section 3 can
be directly applied. In general, however, domains need not be numeric, and we
will therefore focus our discussion on this more general case. Consider a COP
with variables x1 , . . . , xn that have given finite domains xi ∈ Di :

max f (x1 , . . . , xn )
s.t. Cj (x1 , . . . , xn ) j ∈ {1, . . . , m} (1)
xi ∈ Di i ∈ {1, . . . , n}

For simplicity we assume here that all variables appear in all constraints, but
we can allow a different subset of variables for each constraint. Also, Cj may
represent any substructure, for example a global constraint, a table constraint, or
a collection of constraints. We introduce for each variable xi and each constraint
j = 1, . . . , m a duplicate variable yij with domain Di . We let the set yi1 represent
 Improved Constraint Propagation via Lagrangian Decomposition 33

our ‘base’ variables, to which we will compare the variables yij for j = 2, . . . , m.
The reformulated COP is as follows:
max f (y11 , . . . , yn1 )
s.t. Cj (y1j , . . . , ynj ) j ∈ {1, . . . , m}
yi1 = yij i ∈ {1, . . . , n}, j ∈ {2, . . . , m}
yij ∈ Di i ∈ {1, . . . , n}, j ∈ {1, . . . , m}

To establish the Lagrangian decomposition, we relax the constraints yi1 = yij

and move these into the objective as yi1 = yij with associated Lagrangian multi-
pliers. To measure its violation, we propose to represent yi1 = yij with the set of
constraints ((yij = v) − (yi1 = v)) for all v ∈ Di , where (yij = v) is interpreted as
a binary value representing the truth value of the expression. Lastly, we define
a Lagrangian multiplier for each i, j (j ≥ 2) and each v ∈ D(xi ) as a vector
j
λi := λji [v].

The Lagrangian objective function can then be written as:

m 
 n 
max f (y11 , . . . , yn1 ) + λji [v]((yij = v) − (yi1 = v))
j=2 i=1 v∈D(xi )
n 
m 
 
= f (y11 , . . . , yn1 ) + λji [yij ] − λji [yi1 ]
j=2 i=1

j
This leads to the following decomposition (for any given set of multipliers λi ):
⎧ ⎫
⎨ m  n ⎬
max f (y11 , . . . , yn1 ) − λji [yi1 ] | C1 (y11 , . . . , yn1 )
⎩ ⎭
m
  n j=2 i=1 
  j j
+ max λi [yi ] | Cj (y1j , . . . , ynj )
j=2 i=1

which are m independent subproblems.

mLet zj be the optimal objective value for
subproblem j ∈ {1, . . . , m}. Then j=1 zj is a valid bound on f (x1 , . . . , xn ).
Note that the duplicate variables have only been introduced for the formal
description of the method. In practice, all constraints Cj use the original vari-
ables.

Design Choices. The Lagrangian decomposition scheme can be adapted by

allocating parts of original objective to different subproblems. Moreover, we can
introduce equality constraints between any pair of subproblems. We will illus-
trate the latter in the following example.
34 D. Bergman et al.

Example 1. Consider the following CSP:

C1 : alldiff(x1 , x2 , x3 ) C2 : alldiff(x2 , x4 , x5 ) C3 : alldiff(x3 , x5 )

x1 ∈ {a, b}, x2 ∈ {b, c}, x3 ∈ {a, c}, x4 ∈ {a, b}, x5 ∈ {a, b, c}

This CSP is domain consistent as well as pairwise consistent, and has one solution
(x1 , x2 , x3 , x4 , x5 ) = (b, c, a, a, b).
We construct a Lagrangian decomposition based on the constraints
C1 , C2 , C3 . To link these, we only need to introduce the constraints y22 = y21 ,
y33 = y31 , y53 = y52 , and their associated multipliers. This yields the following
three subproblems, with respective objective values z1 , z2 , z3 :
 2 3

z1 = max −λ2 [y21 ] − λ3 [y31 ] | alldiff(y11 , y21 , y31 )
 2 3

z2 = max λ2 [y22 ] − λ5 [y52 ] | alldiff(y22 , y42 , y52 )
 3 3

z3 = max λ3 [y33 ] + λ5 [y53 ] | alldiff(y33 , y53 )

This CSP can be considered as a COP with a zero-valued objective function

so that the value z1 + z2 + z3 is an upper bound on the satisfiability of this
problem, for any Lagrangian multipliers; if the bound is below zero, the problem
is unsatisfiable. And so, the optimal Lagrangian decomposition bound is 0.  

Cost-Based Domain Filtering. In addition to pruning the search based on

the overall bound LP (λ) and a given lower bound B, we can apply cost-based
domain filtering. The difference with existing cost-based filtering methods is that
the bounds from the different subproblems can all be conditioned on a specific
variable/value pair. To this end, let zj |xi =v be the optimal objective value for
subproblem j ∈ {1, . . . , m} in which yij = v. We have the following result:

Proposition 1. If j zj |xi =v < B then v can be removed from Di .
This result may be particularly effective when there is no single subproblem that
collects all variables. We continue our example to give an illustration.

Example 2. Continuing Example 1, consider the following Lagrangian multi-

pliers (all others are zero): λ22 [b] = 0.5, λ33 [a] = 0.5, λ35 [a] = 0.5. This yields
z1 = −0.5, z2 = 0.5, z3 = 0.5, and a total bound of 0.5. Even though this is not
optimal, when we condition x2 = b or x5 = c, the bound becomes −0.5 in both
cases, and by Proposition 1 we can remove those values from their respective
domains. We can similarly remove values a from D1 , b from D2 , and c from D3
using the multipliers λ22 [c] = −0.5, λ33 [c] = 0.5, λ35 [c] = 0.5. 


Example 2 implies the following result:

Proposition 2. Cost-based filtering based on Lagrangian decomposition can be
stronger than pairwise consistency.
 Improved Constraint Propagation via Lagrangian Decomposition 35

Implementation Issues. To apply Lagrangian propagation efficiently, it is

important that each constraint is optimized efficiently. For many constraints
optimization versions are already available [11], to which the Lagrangian costs
can be immediately added. For example, in our experiments we represent con-
straints by decision diagrams, which permit to find the optimal solution quickly
via a shortest (or longest) path calculation. Also, cost-based propagators are
available that filter sub-optimal arcs from the decision diagram. Second, the
search for optimal Lagrangian multipliers can be done with different methods
[14]. Regardless, any set of multipliers results in a valid relaxation, and we do
not necessarily need to solve the Lagrangian dual to optimality. In our imple-
mentation, we compute the multipliers once at the root node and reuse them
during the CP search process.

5 Application: Multiple Alldifferent Constraints

As first application, we consider systems of multiple overlapping alldiff con-
straints, as in [1]. These are defined on a set X = {x1 , . . . , xn } of variables with
domain {1, . . . , n}. Each alldiff constraint is defined on a subset of variables
Sj ⊂ X, for j = 1, . . . , k. We then consider the following COP:
 n

max wi xi | alldiff(Sj ) ∀j ∈ {1, . . . , k}
i=1

We generated instances with n = 10, 11, 12, 13, 14, k = 4, and |Sj | = n − 2 for
all j = 1, . . . , 5. For the Lagrangian decomposition, we partition the alldiff
constraints into two arbitrary subsets of size two, and define one multi-valued
decision diagram (MDD) for each subset. In other words, we apply MDD prop-
agation to these subsets of alldiff constraints. The two MDDs thus formed
are the basis for the Lagrangian decomposition, which follows the description in
Section 4 (where the j-th MDD represents constraint set Cj ).
We implemented the MDD propagation as well as the Lagrangian decom-
position as a global constraint in IBM ILOG CPO 12.6, similar to [5]. The
(near-)optimal Lagrangian multipliers were computed using the Kelly-Cheney-
Goldstein method [14], using IBM ILOG CPLEX 12.6 as the linear programming
solver. We fix the CP search to be lexicographic in the order of the variables, to
ensure the search tree is the same across all instances. We compare the perfor-
mance with and without Lagrangian multipliers.
In Figure 1.a we show the root node percent gap for the 25 instances (where
the optimal value was obtained by formulating an integer linear program and
solving the instances using CPLEX). The reduction in the gap can be sub-
stantial, in some case several orders of magnitude. This reduction in the opti-
mality gap and additional cost-based filtering due to the Lagrangian multipliers
enables more instances to be solved in shorter computational time, as depicted in
Figure 1.b. Depending on the configuration of the alldiff systems the improve-
ment can be marginal and in some cases negligible.
36 D. Bergman et al.

a. Root node gap comparison b. Performance plot

Fig. 1. Evaluating the impact of the Lagrangian decomposition on systems of mul-

tiple alldiff constraints. (a) compares the root node gap obtained with Lagrangian
decomposition (Mdd Lag Gap) and without (Mdd Gap) and (b) depicts a performance
profile comparing the number of instances solved (N Solved) within a given time limit
(horizontal axis) with Lagrangian decomposition (With Lag) and without (No Lag).

6 Application: Set Covering

The set covering problem is defined on a universe of n elements U = {1, . . . , n}.
Given a collection of subsets C1 , . . . , Cm ⊆ U and weights wi (i = 1, . . . , n), the
problem is to find a set of elements S ⊂ U of minimum total weight such that all
S ∩ Cj is not empty for all j = 1, . . . , m. Using a binary variable xi to represent
whether element i is in S, the problem can be formulated as the following COP:
⎧ ⎫
⎨ n  ⎬
min wi xi | xi ≥ 1 ∀j ∈ {1, . . . , m}, xi ∈ {0, 1} ∀i ∈ {1, . . . , n}
⎩ ⎭
i=1 i∈Cj

Instead of defining a subproblem for each separate constraint, we create exact

binary decision diagram (BDD) representations for collections of them. That
is, using the construction method described in [4], we create a BDD by adding
constraints one at the time, until the exact width exceeds a given limit (in our
case 100 nodes on any layer). We then create the next BDD, and so forth. This
forms a partition of the constraint set, each of which is represented by an exact
BDD. For the instances we considered, we construct 10 or 11 BDDs per instance.
We also slightly modify the Lagrangian decomposition method by represent-
ing the original objective function in each of the BDDs, and dualizing constraints

xji = xji for every pair (j, j  ). Hence, the Lagrangian bound is no longer the sum
of the bounds of the respective BDDs, rather the average over the objectives.
In previous work [4], it was shown that the bounds from BDDs were most
effective when the constraint matrix has a relatively small bandwidth. We there-
fore used the same benchmark generator to evaluate the impact of the Lagrangian
decomposition for increasing bandwidths. We generated instances with n = 150
variables, randomly generated costs, and uniform-randomly selected subsets Cj
from within a given bandwidth of size 55 to 75 (five instances for each band-
width). To generate the costs, we let c(i) represent the number of subsets Cj that
 Improved Constraint Propagation via Lagrangian Decomposition 37

a. Impact of Lagrangian decomposition b. Comparison with single relaxed BDD

Fig. 2. Evaluating the bound from Lagrangian decomposition for set covering problems
of varying bandwidth.

contain element i. Then the cost for variable xi is taken uniform randomly in
[0.75∗c(i), 1.25∗c(i)]. The results are shown in Figure 2, showing the average over
the five instances per bandwidth. Figure 2.a depicts four lines: the optimal solu-
tion (found by CPLEX), the average bound without using Lagrangian decom-
position, the maximum bound without using Lagrangian decomposition, and
lastly the average bound when using the Lagrangian decomposition. Lagrangian
decomposition generates bounds of much better quality than the independent
BDDs. For example, for bandwidth 65 the average bound of 27.53 is improved
to 77.80 using the Lagrangian decomposition, on average.
We also compare the Lagrangian decomposition to the original BDD relax-
ation from [4] that represents all constraints in a single BDD respecting a given
maximum width. A larger width leads to a stronger relaxation and better bounds.
Figure 2.b compares the percent gap (between the lower bound and the optimal
solution) of the BDD relaxation for maximum widths 2,000 and 20,000 with
that of the Lagrangian decomposition. We note that the BDD relaxation with
maximum width 2,000 has about the same memory requirements as the separate
BDDs for the Lagrangian decomposition. As the bandwidth increases, the qual-
ity of BDD relaxation rapidly declines, while the Lagrangian decomposition is
much more stable and outperforms the BDD relaxation (decreasing the gap from
117.6% to 33% for bandwidth 75 and maximum width 2,000). This demonstrates
that Lagrangian decompositions can be used to improve BDD-based optimiza-
tion when a single BDD relaxation can no longer provide sufficient power to
represent the entire problem. We do note, however, that the Lagrangian decom-
position takes more time to compute (on average 60s) compared to the single
BDD relaxation (on average 1.4s for width 2,000 and 17s for width 20,000).

7 Conclusion
We have introduced Lagrangian decomposition in the context of constraint pro-
gramming as a generic approach to improve the constraint propagation process.
The key idea is that we penalize variables in different constraints to take differ-
ent assignments. We have shown how this approach can be utilized for stronger
38 D. Bergman et al.

cost-based domain filtering, and that it leads to improved bounds for systems of
alldiff constraints and set covering problems.

References
1. Andersen, H.R., Hadzic, T., Hooker, J.N., Tiedemann, P.: A constraint store based
on multivalued decision diagrams. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741,
pp. 118–132. Springer, Heidelberg (2007)
2. Benoist, T., Laburthe, F., Rottembourg, B.: Lagrange relaxation and con-
straint programming collaborative schemes for traveling tournament problems. In:
Proceedings of the International Workshop on Integration of Artificial Intelligence
and Operations Research Techniques in Constraint Programming for Combinato-
rial Optimization Problems (CPAIOR 2001) (2001)
3. Bergman, D., Cire, A.A., van Hoeve, W.J.: Lagrangian Bounds from Decision Dia-
grams. Constraints 20(3), 346–361 (2015)
4. Bergman, D., van Hoeve, W.-J., Hooker, J.N.: Manipulating MDD relaxations for
combinatorial optimization. In: Achterberg, T., Beck, J.C. (eds.) CPAIOR 2011.
LNCS, vol. 6697, pp. 20–35. Springer, Heidelberg (2011)
5. Cire, A.A., van Hoeve, W.J.: Multivalued Decision Diagrams for Sequencing Prob-
lems. Operations Research 61(6), 1411–1428 (2013)
6. Cooper, M.C., de Givry, S., Sanchez, M., Schiex, T., Zytnicki, M., Werner, T.: Soft
arc consistency revisited. Artificial Intelligence 174(7–8), 449–478 (2010)
7. Focacci, F., Lodi, A., Milano, M.: Cost-based domain filtering. In: Jaffar, J. (ed.)
CP 1999. LNCS, vol. 1713, pp. 189–203. Springer, Heidelberg (1999)
8. Fontaine, D., Michel, L., Van Hentenryck, P.: Constraint-based lagrangian relax-
ation. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 324–339. Springer,
Heidelberg (2014)
9. Guignard, M., Kim, S.: Lagrangian Decomposition: A Model Yielding Stronger
Lagrangian Bounds. Mathematical Programming 39, 215–228 (1987)
10. Hoda, S., van Hoeve, W.-J., Hooker, J.N.: A systematic approach to MDD-
based constraint programming. In: Cohen, D. (ed.) CP 2010. LNCS, vol. 6308,
pp. 266–280. Springer, Heidelberg (2010)
11. van Hoeve, W.J., Katriel, I.: Global constraints. In: Handbook of Constraint Pro-
gramming, pp. 169–208. Elsevier (2006)
12. Janssen, P., Jégou, P., Nouguier, B., Vilarem, M.C.: A filtering process for general
constraint-satisfaction problems: achieving pairwise-consistency using an associ-
ated binary representation. In: IEEE International Workshop on Tools for Arti-
ficial Intelligence, Architectures, Languages and Algorithms, pp. 420–427. IEEE
(1989)
13. Khemmoudj, M.O.I., Bennaceur, H., Nagih, A.: Combining arc-consistency and
dual lagrangean relaxation for filtering CSPS. In: Barták, R., Milano, M. (eds.)
CPAIOR 2005. LNCS, vol. 3524, pp. 258–272. Springer, Heidelberg (2005)
14. Lemaréchal, C.: Lagrangian relaxation. In: Jünger, M., Naddef, D. (eds.) Com-
putational Combinatorial Optimization. LNCS, vol. 2241, pp. 112–156. Springer,
Heidelberg (2001)
15. Régin, J.-C.: Arc consistency for global cardinality constraints with costs. In:
Jaffar, J. (ed.) CP 1999. LNCS, vol. 1713, pp. 390–404. Springer, Heidelberg (1999)
16. Sellmann, M.: Theoretical foundations of cp-based lagrangian relaxation. In:
Wallace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 634–647. Springer, Heidelberg
(2004)
 Strengthening Convex Relaxations with Bound
Tightening for Power Network Optimization

Carleton Coffrin1,2(B) , Hassan L. Hijazi1,2 , and Pascal Van Hentenryck1,2

1
NICTA - Optimisation Research Group, Canberra, Australia
2
College of Engineering and Computer Science, Australian National University,
Canberra, Australia
{carleton.coffrin,hassan.hijazi,pvh}@nicta.com.au

Abstract. Convexification is a fundamental technique in (mixed-

integer) nonlinear optimization and many convex relaxations are
parametrized by variable bounds, i.e., the tighter the bounds, the
stronger the relaxations. This paper studies how bound tightening can
improve convex relaxations for power network optimization. It adapts
traditional constraint-programming concepts (e.g., minimal network and
bound consistency) to a relaxation framework and shows how bound
tightening can dramatically improve power network optimization. In par-
ticular, the paper shows that the Quadratic Convex relaxation of power
flows, enhanced by bound tightening, almost always outperforms the
state-of-the-art Semi-Definite Programming relaxation on the optimal
power flow problem.

Keywords: Continuous constraint networks · Minimal network · Bound

consistency · Convex relaxation · AC power flow · QC relaxation · AC
optimal power flow

1 Introduction
In (mixed-integer) nonlinear optimization, convexification is used to obtain dual
bounds, complementing primal heuristics. In many cases, these convex relax-
ations are parametrized by variable bounds and the tighter the bounds are, the
stronger the relaxations. There is thus a strong potential for synergies between
convex optimization and constraint programming. This paper explores these
synergies in the context of power system optimization.
The power industry has been undergoing a fundamental transformation in
recent years. Deregulation, the emergence of power markets, pressure for reduced
capital investment, and the need to secure a clean sustainable energy supply all
stress the importance of efficiency and reliability in the design and operation of
power networks. As a result, optimization has become a critical component of
NICTA—NICTA is funded by the Australian Government through the Department
of Communications and the Australian Research Council through the ICT Centre
of Excellence Program.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 39–57, 2015.
DOI: 10.1007/978-3-319-23219-5 4
40 C. Coffrin et al.

the emerging smart-grid [28] and has resulted in millions of dollars in annual
savings [32].
Power network applications range from long-term network design and invest-
ment tasks [7,12,21] to minute-by-minute operation tasks [14,16,17,19,23]. All
of these optimization problems share a common core, the Alternating Current
(AC) power flow equations, which model the steady-state physics of power flows.
These equations form a system of continuous non-convex nonlinear equations
that prove to be a significant challenge for existing general-purpose optimiza-
tion tools. It is thus not surprising that, in the last decade, significant attention
has been devoted to developing computationally efficient convex relaxations.
The main contribution of this paper is to show that constraint program-
ming can substantially improve the quality of convex relaxations for power flow
applications. To obtain this result, the paper defines the concept of constraint
relaxation networks and generalizes traditional consistency notions to these net-
works, including minimal network and bound consistency. These concepts, and
the associated algorithms, are then applied to optimal power flow applications
with and without load uncertainty. The experimental results demonstrate the
significant value of bound tightening for power flow applications. In particular,

1. Bound tightening reduces the domains of the variables by as much as 90%

in many cases.
2. In over 90% of the test cases considered, propagation over the convex relax-
ation was sufficient to close the optimality gap within 1%. Only 4 of the test
cases considered remain open.
3. The network consistency algorithm improves the quality of the Quadratic
Convex (QC) relaxation [18] considerably. The QC relaxation now outper-
forms, in the vast majority of the cases, the established state-of-the-art Semi-
Definite Programming (SDP) relaxation on the optimal power flow problem.
4. Parallelization can significantly reduce the runtime requirements of bound
tightening, making the proposed algorithms highly practical.

The rest of the paper is organized as follows. Section 2 reviews the AC power flow
feasibility problem and introduces the notations. Section 3 reviews the state-of-
the-art QC power flow relaxation, which is essential for building efficient consis-
tency algorithms. Section 4 formalizes the idea of constraint relaxation networks
and Section 5 applies this formalism to AC power flows. Section 6 studies the
quality of bound tightening in this application domain and Section 7 evalu-
ates the proposed methods on the ubiquitous AC Optimal Power Flow problem.
Section 8 illustrates the potential of the proposed methods on power flow appli-
cations incorporating uncertainty and Section 9 concludes the paper.

2 AC Power Flow
A power network is composed of a variety of components such as buses, lines,
generators, and loads. The network can be interpreted as a graph (N, E) where
the set of buses N represent the nodes and the set of lines E represent the edges.
 Strengthening Convex Relaxations with Bound Tightening 41

Note that E is a set of directed arcs and E R will be used to indicate those arcs in
the reverse direction. To break numerical symmetries in the model and to allow
easy comparison of solutions, a reference node r ∈ N is also specified.
Every node i ∈ N in the network has three properties, voltage Vi = vi ∠θi ,
power generation Sig = pgi + iqig , and power consumption Sid = pdi + iqid , all of
which are complex numbers due to the oscillating nature of AC power. Each line
(i, j) ∈ E has an admittance Yij = gij + ibij , also a complex number. These
network values are connected by two fundamental physical laws, Kirchhoff’s
Current Law (KCL),

Sig − Sid = Sij ∀ i ∈ N (1)
(i,j)∈E∪E R

and Ohm’s Law,

Sij = Yij∗ (Vi Vi∗ − Vi Vj∗ ) ∀ (i, j) ∈ E ∪ E R . (2)

Note that bold values indicate parameters that are constant in the classic AC
power flow problem and non-bold values are the decision variables.
In addition to these physical laws, the following operational constraints are
required in AC power flows. Generator output limitations on S g ,

Sigl ≤ Sig ≤ Sigu ∀i ∈ N . (3)

Line thermal limits on Sij ,

R
|Sij | ≤ su
ij ∀(i, j) ∈ E ∪ E . (4)

Bus voltage limits on Vi ,

vil ≤ |Vi | ≤ viu ∀i ∈ N (5)

and line phase angle difference limits on Vi Vj∗ ,

 
θij
Δl
≤ ∠ Vi Vj∗ ≤ θij
Δu
∀(i, j) ∈ E (6)

Note that power networks are designed and operated so that −π/3 ≤ θ Δl ≤
θ Δu ≤ π/3 [22] and values as low as π/18 are common in practice [33]. Addi-
tionally the values of v l , v u , su must be positive as they are bounds on the
magnitudes of complex numbers.
Combining all of these constraints and expanding them into their real-number
representation yields the AC Power Flow Feasibility Problem (AC-PF) presented
in Model 1. The input data is indicated by bold values and a description of
the decision variables is given in the model. Constraint (7a) sets the reference
angle. Constraints (7b)–(7c) capture KCL and constraints (7d)–(7e) capture
Ohm’s Law. Constraints (7f) link the phase angle differences on the lines to the
bus variables and constraints (7g) enforce the thermal limit on the lines. This
particular formulation of AC-PF is advantageous as the auxiliary variables θΔ , p,
42 C. Coffrin et al.

Model 1. The AC Power Flow Feasibility Problem (AC-PF)

variables:
pgi ∈ (pgl
i , pi ) ∀i ∈ N - active power generation
gu

qig ∈ (qigl , qigu ) ∀i ∈ N - reactive power generation

vi ∈ (vil , viu ) ∀i ∈ N - bus voltage magnitude
θi ∈ (−∞, ∞) ∀i ∈ N - bus voltage angle
Δ
θij Δl
∈ (θij Δu
, θij ) ∀(i, j) ∈ E - angle difference on a line (aux.)
R
pij ∈ (−su
ij , sij ) ∀(i, j) ∈ E ∪ E
u
- active power flow on a line (aux.)
R
qij ∈ (−su u
ij , sij ) ∀(i, j) ∈ E ∪ E - reactive power flow on a line (aux.)
subject to:
θr = 0 (7a)

pgi − pdi = pij ∀i ∈ N (7b)
(i,j)∈E∪E R

qig − qid = qij ∀i ∈ N (7c)
(i,j)∈E∪E R

pij = gij vi2 − gij vi vj cos(θij

Δ Δ
) − bij vi vj sin(θij ) (i, j) ∈ E ∪ E R (7d)
qij = −bij vi2 + Δ
bij vi vj cos(θij ) − Δ
gij vi vj sin(θij ) (i, j) ∈ E ∪ E R
(7e)
Δ
θij = θi − θj ∀(i, j) ∈ E (7f)
p2ij + 2
qij ≤ (su
ij )
2
∀(i, j) ∈ E ∪ E R
(7g)

and q isolate the problem’s non-convexities in constraints (7d)–(7e) and enable

all but one of the operational constraints to be captured by the variable bounds.
This continuous constraint satisfaction problem is NP-Hard in general [24,40]
and forms a core sub-problem that underpins a wide variety of power network
optimization tasks.
To address the computational difficulties of AC-PF, convex relaxations (i.e.
polynomial time) have attracted significant interest in recent years. Such relax-
ations include the Semi-Definite Programming (SDP) [2], Second-Order Cone
(SOC) [20], Convex-DistFlow (CDF) [13], and the recent Quadratic Convex (QC)
[18] relaxations. To further improve these relaxations, this paper proposes con-
sistency notions and associated propagation algorithms for AC power flows. A
detailed evaluation on 57 AC transmission system test cases demonstrates that
the propagation algorithms enable reliable and efficient methods for improving
these relaxations on a wide variety of power network optimization tasks via
industrial-strength convex optimization solvers (e.g., Gurobi, Cplex, Mosek).
The next section reviews the QC relaxation in detail, which forms the core of
the proposed propagation algorithms.
 Strengthening Convex Relaxations with Bound Tightening 43

3 The Quadratic Convex (QC) Relaxation

The QC relaxation [18] was introduced to utilize the bounds on the voltage vari-
ables v and θΔ , which are ignored by the other relaxations. The key idea is to use
the variable bounds to derive convex envelopes around the non-convex aspects of
the AC-PF problem. The derivation begins by lifting the voltage product terms
in to the higher dimensional W-space using the following equalities:

wi = vi2 i ∈ N (8a)
R Δ
wij = vi vj cos(θij ) ∀(i, j) ∈ E (8b)
I Δ
wij = vi vj sin(θij ) ∀(i, j) ∈ E (8c)

When Model 1 is lifted into this W-space, all of the remaining constraints are
convex. On its own, this lifted model is a weak relaxation but the QC relaxation
strengthens it by developing convex relaxations of the nonlinear equations (8a)–
(8c) for the operational bounds on variables v and θΔ . The convex envelopes for
the square and bilinear functions are well-known [27], i.e.,

2 T
x̌ ≥ x2
x  ≡ (T-CONV)
x̌ ≤ (xu + xl )x − xu xl
⎧
⎪
⎪ ˇ ≥ xl y + y l x − xl y l
xy
⎪
⎪
⎨xy
ˇ ≥ xu y + y u x − x u y u
xyM ≡ (M-CONV)
⎪
⎪ ˇ ≤ xl y + y u x − x l y u
xy
⎪
⎪
⎩
ˇ ≤ xu y + y l x − x u y l
xy
Under the assumption that the phase angle difference bound is within −π/2 ≤
θ Δl ≤ θ Δu ≤ π/2, relaxations for sine and cosine are given by

ˇ ≤ cos(x)
cx
cos(x)C ≡ cos(xl )−cos(xu )
(C-CONV)
ˇ ≥
cx (xl −xu )
(x − xl ) + cos(xl )
⎧
m

m
⎪
⎪sxˇ ≤ cos x
x − xm
+ sin x
if xl < 0 ∧ xu > 0
⎪
⎪ 2 2 2
⎪
⎪
m

m
⎪
⎪ m
⎪
⎪ ˇ ≥ cos 2
sx x
x + 2 − sin 2
x x
if xl < 0 ∧ xu > 0
⎪
⎪
⎪
⎨sxˇ ≤ sin(x) if xl ≥ 0
sin(x)S ≡ (S-CONV)
⎪
⎪
l
)−sin(xu )
⎪
⎪ ˇ ≥ sin(x
sx (x − xl ) + sin(xl ) if xl ≥ 0
⎪
⎪
(xl −xu )
⎪
⎪
⎪
l
)−sin(xu )
⎪
⎪ ˇ ≤ sin(x
sx l −xu ) (x − xl ) + sin(xl ) if xu ≤ 0
⎪
⎪
(x
⎩
ˇ ≥ sin(x)
sx if xu ≤ 0
where xm = max(−xl , xu ). These are a generalization of the relaxations pro-
posed in [18] to support asymmetrical bounds on x. Utilizing these building
44 C. Coffrin et al.

Model 2. The QC Power Flow Feasibility Problem (QC-PF)

variables: Variables of Model 1
stij ∈ (−1, 1) ∀(i, j) ∈ E - relaxation of the sine (aux.)
ctij ∈ (0, 1) ∀(i, j) ∈ E - relaxation of the cosine (aux.)
vvij ∈ (vil vjl , viu vju ) ∀(i, j) ∈ E - relaxation of the voltage product (aux.)


wi ∈ (vil )2 , (viu )2 ∀i ∈ N - relaxation of the voltage square (aux.)
R
wij ∈ (0, ∞) ∀(i, j) ∈ E - relaxation of the voltage and cosine product (aux.)
I
wij ∈ (−∞, ∞) ∀(i, j) ∈ E - relaxation of the voltage and sine product (aux.)
subject to: (7a)–(7c),(7f)–(7g)
CONV(wi = vi2 ∈ (vil , viu )) ∀i ∈ N (10a)
Δ
CONV(ctij = cos(θij ) ∈ Δl
(θij , θij
Δu
)) ∀(i, j) ∈ E (10b)
Δ
CONV(stij = sin(θij ) ∈ Δl
(θij , θij
Δu
)) ∀(i, j) ∈ E (10c)
CONV(vvij = vi vj ∈ (vil , viu ) × (vjl , vju )) ∀(i, j) ∈ E (10d)
R
CONV(wij = vvij ctij ∈ l
(vvij u
, vvij ) × (ctlij , ctu
ij ) ∀(i, j) ∈ E (10e)
I
CONV(wij = vvij stij ∈ l
(vvij u
, vvij ) × (stlij , stu
ij ) ∀(i, j) ∈ E (10f)
R 2 I 2
(wij ) + (wij ) ≤ wi wj ∀(i, j) ∈ E (10g)
R I
pij = gij wi − gij wij − bij wij ∀(i, j) ∈ E (10h)
R I
qij = −bij wi + bij wij − gij wij ∀(i, j) ∈ E (10i)
R I
pji = gij wj − gij wij + bij wij ∀(i, j) ∈ E (10j)
R I
qji = −bij wj + bij wij + gij wij ∀(i, j) ∈ E (10k)

blocks, convex relaxations for equations (8a)–(8c) can be obtained by composing

R
relaxations of the subexpressions, for example, wij ≡ vi vj M cos(θi − θj )C M .
Lastly, the QC relaxation proposes to strengthen these convex relaxations with
a valid second-order cone constraint [11,18,20],
R 2 I 2
(wij ) + (wij ) ≤ wi wj ∀(i, j) ∈ E (9)

The complete QC relaxation of the AC-PF problem is presented in Model

2 (QC-PF), which incorporates many of the components of Model 1. In the
model, the constraint CONV(y = f (x) ∈ D) is used to indicate that y lies in a
convex relaxation of function f within the domain D. Constraints (10a)–(10f)
implement the convex relaxations and constraints (10g) further strengthen these
relaxations. Constraints (10h)–(10k) capture the line power flow in terms of the
W-space variables.

The Impact of Tight Bounds in the QC Relaxation: Constraints (10a)–(10f)

in Model 2 highlight the critical role that the bounds on v and θΔ play in
 Strengthening Convex Relaxations with Bound Tightening 45

1.0

1.0

0.8
0.5

0.5

0.4
Cosine
0.0

0.0

0.0
0.8
−0.5

−0.5
Sine

0.4
Variable Bound
Constraint
−1.0

−1.0
Feasible Region Sine Cosine

0.0
−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

Domain (radians) Domain (radians) Domain (radians)

Fig. 1. The Impact of Variable Bounds on the Convex Relaxations.

the strength of the QC relaxation. Figure 1 illustrates this point by showing

the convex relaxations for sine and cosine over the domains θΔ ∈ (−π/3, π/3)
and θΔ ∈ (−π/3, 0). This figure indicates two key points: (1) Although the
reduction in the size of the bound is 50% in this case, the area inside of the
convex relaxations have been reduced even more significantly; (2) Both the sine
and cosine functions are monotonic when the sign of θΔ is known, which produces
tight convex relaxations.

4 Consistency of Constraint Relaxation Networks

As discussed in Section 2, the core of many applications in power network opti-
mization is a continuous constraint network. Moreover, the QC relaxation of
this continuous constraint network depends on the bounds of the variables. As
a result, constraint propagation now has two benefits: On one hand, it reduces
the domains of the variables while, on the other hand, it strengthens the relax-
ation. These two processes reinforce each other, since tighter constraints generate
tighter bounds creating a virtuous cycle. This section generalizes traditional con-
sistency notions to this new context through the concepts of constraint schemes
and constraint relaxations. Since solutions to continuous constraint networks
are real numbers and computer implementations typically rely on floating-point
numbers, some care must be exercised in formalizing these notions. The formal-
ization also assumes that only bound reasoning is of interest, since these are
continuous constraint networks. However, the concepts generalize naturally to
domain reasoning.

Continuous Constraint Networks: Constraint networks are defined in terms of

a set of variables X = {x1 , . . . , xn } ranging over intervals I = {I1 , . . . , In }
and a set of constraints. An interval I = [l, u] denotes the set of real numbers
{r ∈  | l ≤ r ≤ u}. This paper only considers floating-point intervals, i.e.,
intervals whose bounds are floating-point numbers. If r is a real number, [r]
denotes the smallest floating-point interval containing r, [r]− the largest floating-
point number no greater than r, and [r]+ the smallest floating-point number no
smaller than r. A variable assignment assigns to each variable xi a value from its
46 C. Coffrin et al.

interval Ii . A constraint is a function (X → ) → Bool which, given a variable

assignment, returns a truth value denoting whether the assignment satisfies the
constraint.

Definition 1 (Continuous Constraint Network (CCN)). A continuous

constraint network is a triple (X, I, C) where X = (x1 , . . . , xn ) is a collection of
variables ranging over I = (I1 , . . . , In ) and C is a set of constraints.

Definition 2 (Solution to a CCN). A solution to a CCN (X, I, C), where

X = (x1 , . . . , xn ) and I = (I1 , . . . , In ), is an assignment σ = {x1 ← v1 ; . . . ; xn ←
vn } such that vi ∈ Ii and for all c ∈ C: c(σ) holds. The set of solutions to a
CCN P is denoted by Σ(P).

In the following we use max(x, Σ) to denote the maximum value of variable x

in the assignments Σ, i.e., max(x, Σ) = maxσ∈Σ σ(x), where σ(x) denotes the
value of variable x in assignment σ. The value min(x, Σ) is defined similarly.
The following definition adapts the traditional concept of minimal constraint
network [31] to continuous constraint networks.

Definition 3 (Minimal CCN). A CCN P = (X, I, C), where X =

(x1 , . . . , xn ) and I = (I1 , . . . , In ), is minimal if, for each variable xi , the interval
Ii = [li , ui ] satisfies li = [min(xi , Σ(P))]− ∧ ui = [max(xi , Σ(P))]+ .

Note that the bounds are not necessarily solutions themselves but are as tight
as the floating-point accuracy allows for. Given a CCN P = (X, I, C), its largest
minimal network P  = (X , I , C) (Im ⊆ I) always exists and is unique since
there are only finitely many floating-point intervals.
The concept of bound consistency [39] captures a relaxation of the mini-
mal network: It only requires the variable bounds to be tight locally for each
constraint.

Definition 4 (Bound Consistency for CCNs). A CCN P = (X, I, C),

where X = (x1 , . . . , xn ) and I = (I1 , . . . , In ), is bound-consistent if each con-
straint c is bound-consistent with respect to I. A constraint c is bound-consistent
with respect to I if the continuous constraint network (X, I, {c}) is minimal.

Once again, given a CCN P = (X, I, C), its largest bound-consistent network
P = (X, Im , C) (Im ⊆ I) always exists and is unique. In the following, we use
minCCN (X, I, C) and bcCCN (X, I, C) to denote these networks, i.e.,

minCCN(X, I, C) = max{Im ⊆ I | (X, Im , C) is minimal},

bcCCN(X, I, C) = max{Im ⊆ I | (X, Im , C) is bound-consistent}.

Constraint Relaxation Networks: The convex relaxations used in the QC relax-

ation depend on the variable bounds, i.e., the stronger the bounds the stronger
the relaxations. Since the relaxations change over time, it is necessary to intro-
duce new consistency notions: constraint schemes and constraint relaxations.
 Strengthening Convex Relaxations with Bound Tightening 47

Definition 5 (Continuous Constraint Scheme). A constraint scheme r is a

function I → (X → ) → Bool which, given a collection of intervals, returns a
constraint. Moreover, the scheme r satisfies the following monotonicity property:
I ⊆ I  ⇒ (r(I  )(σ) ⇒ r(I)(σ))
for all collections of intervals I and I  , and variable assignment σ.
The monotonicity property ensures that tighter bounds produce tighter con-
straints. Traditional constraints are constraint schemes that just ignore the ini-
tial bounds. A constraint relaxation is a constraint scheme that preserves the
solutions to the original constraint.
Definition 6 (Constraint Relaxation). A constraint scheme r is a relaxation
of constraint c if, for all assignment σ and bounds I = ([σ(x1 )], . . . , [σ(xn )]), we
have r(I)(σ) ⇒ c(σ).1
Example 1. Consider the constraint c(x, y, z) which holds if z = xy. Given
bounds [xl , xu ] and [y l , y u ] for variables x and y, the McCormick relaxation
[27] is a constraint scheme specified by the collection of constraints in M-CONV.
Note that this envelope ignores the bound on variable z. Additionally this con-
straint scheme is also a constraint relaxation of c because it is known to be the
convex envelope of z = xy for any bounds on x and y [27].
Definition 7 (Continuous Constraint Relaxation Network (CCRN)).
A constraint relaxation network is a triple (X, I, R) where X is a collection of
variables ranging over I and R is a set of constraint relaxations.
In the following, we use R(I) to denote {r(I) | r ∈ R} if R is a set of relaxations.

Consistency of Constraint Relaxation Networks: We now generalize the con-

cepts of minimal and bound-consistent networks to CCRNs. The definitions cap-
ture the fact that no additional bound tightening is possible for the relaxations
induced by the bounds.
Definition 8 (Minimal CCRN). A CCRN P = (X, I, R), X = (x1 , . . . , xn )
and I = (I1 , . . . , In ), is minimal if the CCN network (X, I, R(I)) is.
Definition 9 (Bound-Consistent CCRN). A CCRN P = (X, I, R), where
X = (x1 , . . . , xn ) and I = (I1 , . . . , In ), is bound-consistent if the CCN network
(X, I, R(I)) is.
Once again, the largest minimal or bound-consistent network of a CCRN exists
and is unique by monotonicity of constraint relaxations. In the following, we use
minCCRN (X, I, C) and bcCCRN (X, I, C) to denote these networks, i.e.,
minCCRN(X, I, R) = max{Im ⊆ I | (X, Im , R) is minimal},
bcCCRN(X, I, R) = max{Im ⊆ I | (X, Im , R) is bound-consistent}.
1
Note that some of the convex relaxations used in the QC relaxation are only valid
within some bounds. This is easily captured by assuming that the constraint itself
imposes these bounds.
48 C. Coffrin et al.

minCCRN(X, I, R) bcCCN(X, I, C)
I n := I; I n := I;
repeat repeat
I o := I n ; I o := I n ;
I n := minCCN(X, I o , R(I o )); for all c ∈ C
until I o = I n Icn 
:= minCCN(X, I o , {c});
return I n ; I n := c∈C Icn ;
until I o = I n
return I n ;
Fig. 2. Computing the Minimal Fig. 3. Computing the Largest
Continuous Constraint Relaxation Bound-Consistent Constraint Net-
Networks work.

The following property establishes the soundness of bound tightenings in

CCRNs.

Proposition 1. Let (X, I, C) be a CCN and let (X, I, R) be a CCRN such that
R = {r | c ∈ C ∧ r is a relaxation of c}. Then,

minCCN(X, I, C) ⊆ minCCRN(X, I, R),

bcCCN(X, I, C) ⊆ bcCCRN(X, I, R).

The minimal and bound-consistent relaxation networks can be computed by

a simple fixpoint algorithm that iterates the consistency algorithm over the
increasingly tighter relaxation networks. Figure 2 depicts the algorithm for
computing a minimal network. The algorithm is similar for bound consistency.
Observe that the bound-consistency algorithm has a fixpoint algorithm embed-
ded inside the top-level fixpoint.

4.1 Relation to Concepts in Global Optimization

The idea of bounds propagation for global optimization goes as far back as
[6]: It was subsequently implemented in the Numerica system which also per-
forms bound propagation on a linearization of the nonlinear constraints [37,38].
The notion of using bound reductions for improving convex relaxations of non-
convex programs was first widely recognized in the Branch-and-Reduce (BnR)
algorithm [34]. BnR is a natural extension of Branch-and-Bound over continuous
domains, which includes additional steps to reduce the domains of the variables
at each search node. This line of work has developed into two core bound reduc-
tion ideas: (1) Feasibility-Based Range Reduction (FBRR), which is concerned
with pruning techniques based on feasibility information and (2) Optimality
Based Range Reduction (OBRR), which develops bound reductions based on
Lagrangian-duality arguments [35]. A variety of methods have been developed
for FBRR and OBRR with various pruning strength and computational time
tradeoffs [5,25,34]. However, all these methods are non-global bound reduction
techniques and may be iterated until a desired level of consistency is achieved.
 Strengthening Convex Relaxations with Bound Tightening 49

CCRNs and the associated consistency notions (minCCRN, bcCCRN) devel-

oped herein are examples of FBRR methods. The idea of computing minCCRN
is discussed informally in [4] for the special case where the relaxation is a system
of linear or convex equations (note that the algorithm in Figure 2 applies for any
kind of CSP). It is often noted in the FBRR literature that just one iteration of
the minCCRN is too costly to compute [4,35], let alone the full fixpoint. The pre-
ferred approach is to perform some bound propagation (not always to the fixpoint)
on linear relaxations of the non-convex problem [5,25,35]. In fact, specialized algo-
rithms have been proposed for computing bound consistency on purely linear sys-
tems for this purpose [4]. The linear bound-consistency computations discussed in
[4,5] are weaker forms of the bcCCRN notion considered here since it does not
explicitly mention re-linearizing the relaxation after bound propagation is com-
plete and re-computing bounds consistency. It is important to note that the algo-
rithm in Figure 2 seamlessly hybridizes the FBRR ideas from global optimization to
CP systems, which include arbitrary global constraints that are outside the scope
of purely mathematical programs. This advantage is utilized in the next section.

5 Constraint Relaxation Networks for Power Flows

This section discusses how to compute the largest minimal and bound-consistent
networks for the relaxation model (X, I, R) defined by Model 2. Observe first
that the convex relaxations used in Model 2 are all monotonic.
Proposition 2. The convex relaxations T-CONV, M-CONV, C-CONV, and S-
CONV are monotonic.

Minimal Network: The largest minimal network is computed by Algorithm QC-N

which applies the fixpoint algorithm minCCRN shown in Figure 2 to Model 2.
The underlying minCCN networks are computed by optimizing each variable
independently, i.e.,
Ixn := [ min σ(x), max σ(x)];
σ:R(σ) σ:R(σ)

Observe that this computation is inherently parallel, since all the optimizations
are independent.

Bound-Consistent Network: The largest bound-consistent network is computed

by Algorithm QC-B which applies the bound-consistency counterpart of algo-
rithm minCCRN to Model 2. The bcCCN networks needed in this fixpoint
algorithm are computed by the algorithm shown in Figure 3. Algorithm bcCCN
computes the intervals Icn that are bound-consistent for each constraint c ∈ C
before taking the intersection of these intervals. The process is iterated until a
fixpoint is obtained. This algorithm was selected because the bound-consistency
computations can be performed in parallel.
Observe also that the QC-B algorithm is applied to a version of Model 2
using a global constraint
line power qc(pij , qij , vi , θi , pji , qji , vj , θj )
50 C. Coffrin et al.

Fig. 4. QC Consistency Algorithms – Quality Analysis.

Fig. 5. QC Consistency Algorithms – Runtime Analysis.

that captures constraints (7f)–(7g), (10a)–(10k) for each line (i, j) ∈ E. The
use of this global constraint means that QC-B computes a stronger form of
bounds consistency than one based purely on Model 2. This stronger level of
consistency is necessary to obtain reasonable bound tightenings. Note that all the
optimizations in algorithms QC-N and QC-B are convex optimization problems
which can be solved in polynomial time.

6 Strength and Performance of the Bound Tightening

This section evaluates the benefits of QC-N and QC-B on the general feasibility
problem in Model 1. Algorithms QC-N and QC-B were implemented in AMPL
[15] using IPOPT 3.12 [41] to solve the convex nonlinear programs. The prop-
agation algorithms were executed on Dell PowerEdge R415 servers with Dual
2.8GHz AMD 6-Core Opteron 4184 CPUs and 64GB of memory with a conver-
gence tolerance of  = 0.001. Their performance is evaluated on 57 transmission
system test cases from the NESTA 0.3.0 archive [10] ranging from 3 to 300 buses.
Figure 4 summarizes the results of QC-B and QC-N on three key metrics: the
phase angle difference domains (θΔ ), the voltage domains (v), and the number of
lines where the sign of θΔ is determined. Each plot summarizes the distribution
of 57 values as a standard box-and-whisker plot, where the width of the box
reflects the first and third quartiles, the black line inside the box is the median,
and the whiskers reflect min and max values up to 1.5 IQR with the remaining
data points plotted as outliers. In these plots values to the left are preferable.
The domain reduction of the QC-N approach is substantial, typically pruning
the domain θΔ by 90% and the domain of v by 30% and determining the sign
of θΔ for about half of the lines. Across all of the metrics, it is clear that QC-N
has significant benefits over QC-B.
Figure 5 summarizes the runtime performance of QC-B and QC-N on three
key metrics: Total CPU time (T1 ), fully parallel CPU wall-clock time (T∞ ), and
 Strengthening Convex Relaxations with Bound Tightening 51

the number of fixpoint iterations. The total runtimes of the QC-B and QC-N
algorithms vary widely based on the size of the network under consideration
and can range from seconds to hours. Fortunately, regardless of the size of the
network, the number of iterations in the fixpoint computation is small (often
less than 10). As a result, the parallel runtime of the algorithms scale well with
the size of the network and rarely exceeds 1 minute, which is well within the
runtime requirements of the majority of network optimization applications.2

7 Application to AC Optimal Power Flow

This section assesses the benefits of QC-B and QC-N on the ubiquitous AC
Optimal Power Flow problem (AC-OPF) [29,30]. The goal of the AC-OPF is
to find the cheapest way to satisfy the loads given the network flow constraints
and generator costs functions, which are typically quadratic. If c2i , c1i , c0i are
the cost coefficients for generating power at bus i ∈ N , the AC-OPF objective
function is given by,

minimize: c2i (pgi )2 + c1i (pgi ) + c0i (11)
i∈N

The complete non-convex AC-OPF problem is Model 1 with objective (11) and
the QC relaxation of this problem is Model 2 with objective (11).
The goal is to compare five AC-OPF relaxations for bounding primal AC-
OPF solutions produced by IPOPT, which only guarantees local optimality. The
five relaxations under consideration are as follows:
1. QC - as defined in Model 2.
2. QC-B - bcCCRN for Model 2.
3. QC-N - minCCRN for Model 2.
4. SDP - a state-of-the-art relaxation based on semi-definite programming [26].
5. SDP-N - the SDP relaxation strengthened with bounds from QC-N.
There is no need to consider other existing relaxations as the QC and SDP
dominate them [11]. The computational environment and test cases are those of
Section 6. SDPT3 4.0 [36] was used to solve the SDP models.
Table 1 presents the detailed performance and runtime results on all 57 test
cases. They can be summarized as follows: (1) The optimality gaps of the QC
relaxation are significantly reduced by both QC-N and QC-B; (2) QC-N closes
the AC-OPF optimality gap to below 1% in 90% of the cases considered and
closes 10 open test cases; (3) QC-N almost always outperforms the SDP relax-
ation in quality with comparable parallel runtimes; (4) For the test cases with
significant optimality gaps, QC-N outperforms the SDP relaxation most often,
even when the SDP relaxation is strengthened with QC-N bounds (i.e., SDP-N).
2
Dedicated high performance computational resources are commonplace in power
system operation centers. The T∞ runtime is realistic in these settings where high-
level of reliability is critical.
52 C. Coffrin et al.

Table 1. Quality and Runtime Results of Convex Relaxations on the AC-OPF Problem
(bold - best in row (runtime used to break ties in quality), — - solving error)

$/h Optimality Gap (%) T∞ Runtime (sec.)

Test Case AC SDP-N SDP QC-N QC-B QC AC SDP-N SDP QC-N QC-B QC
case3 lmbd 5812 0.1 0.4 0.1 1.0 1.2 0.2 6.8 4.7 0.5 0.4 0.1
case4 gs 156 0.0 0.0 0.0 0.0 0.0 0.2 7.2 4.8 0.4 0.8 0.1
case5 pjm 17551 5.2 5.2 9.3 14.5 14.5 0.1 6.4 5.1 0.9 0.3 0.2
case6 c 23 0.0 0.0 0.0 0.3 0.3 0.0 6.9 5.4 1.3 0.4 0.1
case6 ww 3143 0.0 0.0 0.0 0.1 0.6 0.3 5.4 5.4 0.8 2.7 0.1
case9 wscc 5296 0.0 0.0 0.0 0.0 0.0 0.2 6.2 4.9 1.5 0.7 0.1
case14 ieee 244 0.0 0.0 0.0 0.1 0.1 0.1 4.8 5.2 2.0 0.4 0.1
case24 ieee rts 63352 0.0 0.0 0.0 0.0 0.0 0.2 8.5 6.0 3.2 0.5 0.2
case29 edin 29895 0.0 0.0 0.0 0.1 0.1 0.4 8.2 7.8 15.8 1.4 1.1
case30 as 803 0.0 0.0 0.0 0.1 0.1 0.3 6.9 5.4 2.3 0.5 0.1
case30 fsr 575 0.0 0.0 0.1 0.3 0.4 0.2 5.5 6.1 2.2 1.0 0.2
case30 ieee 205 0.0 0.0 0.0 5.3 15.4 0.4 7.8 6.3 0.7 0.7 0.3
case39 epri 96505 0.0 0.0 0.0 0.0 0.0 0.2 6.5 7.1 2.1 0.6 0.2
case57 ieee 1143 0.0 0.0 0.0 0.1 0.1 0.1 11.4 9.1 5.1 0.9 0.4
case73 ieee rts 189764 0.0 0.0 0.0 0.0 0.0 0.5 12.5 8.5 4.7 0.7 0.5
case118 ieee 3720 0.1 0.1 0.4 1.0 1.7 0.3 18.2 12.0 21.1 6.0 0.8
case162 ieee dtc 4237 1.0 1.1 0.7 3.8 4.2 0.7 57.6 34.2 25.9 7.0 1.5
case189 edin 849 — 0.1 0.1 — 0.2 0.9 12.3 13.3 6.5 59.9 1.6
case300 ieee 16894 0.1 0.1 0.1 1.0 1.2 0.9 40.8 25.5 48.2 14.4 2.4
case3 lmbd api 367 0.0 1.3 0.0 0.5 1.8 0.2 4.0 4.0 0.5 1.5 0.1
case4 gs api 767 0.0 0.0 0.0 0.2 0.7 0.9 6.9 3.9 0.8 0.3 0.1
case5 pjm api 2994 0.0 0.0 0.0 0.4 0.4 0.0 6.9 7.0 0.2 0.4 0.1
case6 c api 807 0.0 0.0 0.0 0.5 0.5 0.6 5.3 5.4 0.3 0.4 0.1
case6 ww api 273 — 0.0 0.0 2.1 13.1 0.2 4.5 15.0 0.4 0.4 0.1
case9 wscc api 656 0.0 0.0 0.0 0.0 0.0 0.4 5.4 6.4 0.8 0.9 0.1
case14 ieee api 323 0.0 0.0 0.2 1.3 1.3 0.1 6.4 4.7 0.5 0.4 0.1
case24 ieee rts api 6421 0.7 1.4 0.3 3.3 13.8 0.2 8.6 7.2 1.4 1.6 0.2
case29 edin api 295764 — — 0.1 0.4 0.4 0.3 12.6 7.8 28.4 1.1 3.2
case30 as api 571 0.0 0.0 0.0 2.4 4.8 0.4 7.6 6.0 3.7 0.8 0.2
case30 fsr api 372 3.6 11.1 2.7 42.8 46.0 0.2 7.9 6.7 1.4 0.4 0.2
case30 ieee api 411 0.0 0.0 0.0 0.9 1.0 0.3 8.9 6.5 1.0 0.5 0.2
case39 epri api 7466 0.0 0.0 0.0 0.8 3.0 0.1 9.2 6.5 4.9 1.9 0.2
case57 ieee api 1430 0.0 0.1 0.0 0.2 0.2 0.4 8.8 8.1 3.2 0.6 0.4
case73 ieee rts api 20123 0.9 4.3 0.1 3.6 12.0 0.6 15.4 9.5 11.4 2.0 0.6
case118 ieee api 10258 16.7 31.5 11.8 38.9 44.0 0.6 14.2 14.6 11.3 4.7 0.8
case162 ieee dtc api 6095 0.6 1.0 0.1 1.4 1.5 0.4 51.9 32.8 25.5 2.1 1.5
case189 edin api 1971 — 0.1 0.0 — 5.6 0.3 14.3 13.5 8.3 67.1 1.1
case300 ieee api 22825 0.0 0.0 0.2 0.6 0.8 0.9 47.5 28.5 71.1 3.6 2.6
case3 lmbd sad 5992 0.1 2.1 0.0 0.2 1.2 0.1 5.1 4.2 0.2 0.9 0.1
case4 gs sad 324 0.0 0.0 0.0 0.5 0.8 0.1 4.4 3.9 0.1 1.3 0.1
case5 pjm sad 26423 0.0 0.0 0.0 0.7 1.1 0.1 5.7 5.3 0.2 0.4 0.1
case6 c sad 24 0.0 0.0 0.0 0.4 0.4 0.1 6.7 4.6 0.2 0.3 0.1
case6 ww sad 3149 0.0 0.0 0.0 0.1 0.3 0.1 5.9 5.4 0.2 0.2 0.1
case9 wscc sad 5590 0.0 0.0 0.0 0.2 0.4 0.3 5.5 4.4 0.1 0.5 0.1
case14 ieee sad 244 0.0 0.0 0.0 0.1 0.1 0.1 7.5 4.6 0.5 0.3 0.1
case24 ieee rts sad 79804 1.4 6.1 0.1 3.4 3.9 0.3 9.3 5.7 0.6 0.4 0.3
case29 edin sad 46933 5.8 28.4 0.9 20.0 20.6 0.5 7.1 8.5 15.5 0.3 1.6
case30 as sad 914 0.1 0.5 0.0 2.9 3.1 0.1 6.4 6.8 2.3 0.3 0.2
case30 fsr sad 577 0.1 0.1 0.1 0.5 0.6 0.1 6.2 6.8 1.9 0.3 0.2
case30 ieee sad 205 0.0 0.0 0.0 2.0 4.0 0.3 7.0 6.0 0.6 0.6 0.1
case39 epri sad 97219 0.0 0.1 0.0 0.0 0.0 0.1 7.1 6.0 1.0 0.9 0.2
case57 ieee sad 1143 0.0 0.0 0.0 0.1 0.1 0.4 8.8 7.6 1.9 0.8 0.3
case73 ieee rts sad 235241 2.4 4.1 0.1 3.1 3.5 0.3 9.7 8.4 3.6 0.6 0.8
case118 ieee sad 4323 4.0 7.6 1.4 7.6 8.3 0.4 15.4 13.8 5.9 0.6 1.0
case162 ieee dtc sad 4368 1.7 3.6 0.4 5.9 6.9 0.9 46.8 37.7 27.3 2.1 1.4
case189 edin sad 914 — 1.2 0.5 — 2.2 0.6 11.4 17.4 12.2 49.6 1.1
case300 ieee sad 16912 0.1 0.1 0.1 0.8 1.2 0.9 25.2 30.8 45.6 5.5 2.4
 Strengthening Convex Relaxations with Bound Tightening 53

Overall, these results clearly establish QC-N is the new state-of-the-art con-
vex relaxation of the AC-OPF. General purpose global optimization solvers (e.g.,
Couenne 0.4 [3] and SCIP 3.1.1 [1,8]) were also considered for comparison. Pre-
liminary results indicated that these general purpose solvers are much slower
than the dedicated power flow relaxations considered here and cannot produce
competitive lower bounds on these networks with in 10 hours of computation.

8 Propagation with Load Uncertainty

Loads in power systems are highly predictable. In transmission systems, it is
commonplace for minute-by-minute load forecasts to be within 5% of the true
values [9]. This high degree of predictability can be utilized by the bound tight-
ening algorithms proposed here. Indeed, if the feasible set of Model 1 is increased
to include a range of possible load values, determined by the forecast, then the
algorithms compute a description of all possible future power flows. This section
studies the power of bound propagation in this setting.
Model 3 presents an extension of Model 1 to incorporate load uncertainty.
New decision variables for the possible load values are introduced (i.e., pd , q d )
and their bounds come from the extreme values of the load forecasting model.
The lower bounds on active power generation (pg ) are also increased to include
0, as generators may become inactive at some point in the future (e.g., due to
scheduled maintenance or market operations). Constraints (12a)–(12b) incorpo-
rate the load variables into KCL. The other constraints remain the same as in
Model 1. Because only the KCL constraints are modified in this formulation,
the QC relaxation of Model 3 (QC-U) is similar to Model 1, as described in
Section 3. For the experimental evaluation, the 57 deterministic test cases were
extended into uncertain load cases by adopting a forecast model of ±5% of the
deterministic load value.
Figure 6 compares the quality of minCCRN on the QC-U model (QC-U-N)
to minCCRN in the deterministic case (QC-N) in order to illustrate the pruning

Model 3. The AC-PF Program with Load Uncertainty (AC-PF-U)

variables: Variables of Model 1

pdi ∈ (pdl du
i , pi ) ∀i ∈ N - active power load interval

qid ∈ (qidl , qidu ) ∀i ∈ N - reactive power load interval

pgi ∈ (0, pdu
i ) ∀i ∈ N - active power generation interval

subject to: (7a), (7d)–(7g)


pgi − pdi = pij ∀i ∈ N (12a)
(i,j)∈E∪E R

qig − qid = qij ∀i ∈ N (12b)
(i,j)∈E∪E R
54 C. Coffrin et al.

QC−N

QC−U−N

0 20 40 60 80 100
Angle Diff. Sign Unknown (% of lines)

Fig. 6. QC Consistency Algorithms with Load Uncertainty – Quality Analysis.

Fig. 7. Comparison of AC-OPF Bound Improvements of QC Variants.

loss due to uncertainty. The results indicate that, even when load uncertainty is
incorporated, minCCRN still prunes the variable domains significantly, typically
reducing the voltage angle domains by 80% and the voltage magnitude domains
by 10%, and determining the sign of θΔ for about 30% of the lines. The domain
reduction on θΔ in QC-U-N is particularly significant.
Figure 7 considers the AC-OPF and summarizes the optimality gaps pro-
duced under load certainty and uncertainty. QC-U-N produces significant
improvement in optimality gaps, moving from < 5% (QC) to less than < 1%.
Obviously, load certainty (QC-N) closes the remaining 1%.

9 Conclusion
This paper studied how bound tightening can improve convex relaxations by
adapting traditional constraint-programming concepts (e.g., minimal network
and bound consistency) to a relaxation framework. It showed that, on power
flow applications, bound tightening over the QC relaxation can dramatically
reduce variable domains. Moreover, on the ubiquitous AC-OPF problem, the QC
relaxation, enhanced by bound tightening, almost always outperforms the state-
of-the-art SDP relaxation on the optimal power flow problem. The paper also
showed that bound tightening yields significant benefits under load uncertainty,
demonstrating a breadth of applicability. These results highlight the significant
potential synergies between constraint programming and convex optimization
for complex engineering problems.

References
1. Achterberg, T.: Scip: solving constraint integer programs. Mathematical Program-
ming Computation 1(1), 1–41 (2009). http://dx.doi.org/10.1007/s12532-008-0001-1
 Strengthening Convex Relaxations with Bound Tightening 55

2. Bai, X., Wei, H., Fujisawa, K., Wang, Y.: Semidefinite programming for optimal
power flow problems. International Journal of Electrical Power & Energy Systems
30(67), 383–392 (2008)
3. Belotti, P.: Couenne: User manual (2009). https://projects.coin-or.org/Couenne/
(accessed April 10, 2015)
4. Belotti, P., Cafieri, S., Lee, J., Liberti, L.: On feasibility based bounds tightening
(2012). http://www.optimization-online.org/DB HTML/2012/01/3325.html
5. Belotti, P., Lee, J., Liberti, L., Margot, F., Wachter, A.: Branching and bounds
tightening techniques for non-convex minlp. Optimization Methods Software 24(4–
5), 597–634 (2009)
6. Benhamou, F., McAllester, D., Van Hentenryck, P.: Clp (intervals) revisited.Tech.
rep., Brown University, Providence, RI, USA (1994)
7. Bent, R., Coffrin, C., Gumucio, R., Van Hentenryck, P.: Transmission network
expansion planning: Bridging the gap between ac heuristics and dc approximations.
In: Proceedings of the 18th Power Systems Computation Conference (PSCC 2014),
Wroclaw, Poland (2014)
8. Berthold, T., Heinz, S., Vigerske, S.: Extending a cip framework to solvemiqcps.
In: Lee, J., Leyffer, S. (eds.) Mixed Integer Nonlinear Programming. The IMA
Volumes in Mathematics and its Applications, vol. 154, pp. 427–444. Springer
New York (2012)
9. Chen, Y., Luh, P., Guan, C., Zhao, Y., Michel, L., Coolbeth, M., Friedland, P.,
Rourke, S.: Short-term load forecasting: Similar day-based wavelet neural networks.
IEEE Transactions on Power Systems 25(1), 322–330 (2010)
10. Coffrin, C., Gordon, D., Scott, P.: NESTA, The Nicta Energy System Test Case
Archive. CoRR abs/1411.0359 (2014). http://arxiv.org/abs/1411.0359
11. Coffrin, C., Hijazi, H., Van Hentenryck, P.: The QC Relaxation: Theoretical and
Computational Results on Optimal Power Flow. CoRR abs/1502.07847 (2015).
http://arxiv.org/abs/1502.07847
12. Coffrin, C., Van Hentenryck, P.: Transmission system restoration: Co-optimization
of repairs, load pickups, and generation dispatch. International Journal of Electrical
Power & Energy Systems (2015) (forthcoming)
13. Farivar, M., Clarke, C., Low, S., Chandy, K.: Inverter var control for distribution
systems with renewables. In: 2011 IEEE International Conference on Smart Grid
Communications (SmartGridComm), pp. 457–462, October 2011
14. Fisher, E., O’Neill, R., Ferris, M.: Optimal transmission switching. IEEE Transac-
tions on Power Systems 23(3), 1346–1355 (2008)
15. Fourer, R., Gay, D.M., Kernighan, B.: AMPL: a mathematical programming lan-
guage. In: Wallace, S.W. (ed.) Algorithms and Model Formulations in Mathemat-
ical Programming, pp. 150–151. Springer-Verlag New York Inc., New York (1989)
16. Fu, Y., Shahidehpour, M., Li, Z.: Security-constrained unit commitment with ac
constraints*. IEEE Transactions on Power Systems 20(3), 1538–1550 (2005)
17. Hedman, K., Ferris, M., O’Neill, R., Fisher, E., Oren, S.: Co-optimization of gen-
eration unit commitment and transmission switching with n-1 reliability. In: 2010
IEEE Power and Energy Society General Meeting, pp. 1–1, July 2010
18. Hijazi, H., Coffrin, C., Van Hentenryck, P.: Convex quadratic relaxations
of mixed-integer nonlinear programs in power systems (2013). http://www.
optimization-online.org/DB HTML/2013/09/4057.html
56 C. Coffrin et al.

19. Hijazi, H., Thiebaux, S.: Optimal ac distribution systems reconfiguration. In:
Proceedings of the 18th Power Systems Computation Conference (PSCC 2014),
Wroclaw, Poland (2014)
20. Jabr, R.: Radial distribution load flow using conic programming. IEEE Transac-
tions on Power Systems 21(3), 1458–1459 (2006)
21. Jabr, R.: Optimization of ac transmission system planning. IEEE Transactions on
Power Systems 28(3), 2779–2787 (2013)
22. Kundur, P.: Power System Stability and Control. McGraw-Hill Professional (1994)
23. Lavaei, J., Low, S.: Zero duality gap in optimal power flow problem. IEEE Trans-
actions on Power Systems 27(1), 92–107 (2012)
24. Lehmann, K., Grastien, A., Van Hentenryck, P.: AC-Feasibility on Tree Networks
is NP-Hard. IEEE Transactions on Power Systems (2015) (to appear)
25. Liberti, L.: Writing global optimization software. In: Liberti, L., Maculan, N.
(eds.) Global Optimization, Nonconvex Optimization and Its Applications, vol. 84,
pp. 211–262. Springer, US (2006). http://dx.doi.org/10.1007/0-387-30528-9 8
26. Madani, R., Ashraphijuo, M., Lavaei, J.: Promises of conic relaxation for
contingency-constrained optimal power flow problem (2014). http://www.ee.
columbia.edu/lavaei/SCOPF 2014.pdf (accessed February 22, 2015)
27. McCormick, G.: Computability of global solutions to factorable nonconvex pro-
grams: Part i convex underestimating problems. Mathematical Programming 10,
146–175 (1976)
28. Miller, J.: Power system optimization smart grid, demand dispatch, and micro-
grids, September 2011. http://www.netl.doe.gov/smartgrid/referenceshelf/
presentations/SE%20Dist%20Apparatus%20School Final 082911 rev2.pdf
(accessed April 22, 2012)
29. Momoh, J., Adapa, R., El-Hawary, M.: A review of selected optimal power flow lit-
erature to 1993. i. nonlinear and quadratic programming approaches. IEEE Trans-
actions on Power Systems 14(1), 96–104 (1999)
30. Momoh, J., El-Hawary, M., Adapa, R.: A review of selected optimal power flow
literature to 1993. ii. newton, linear programming and interior point methods.
IEEE Transactions on Power Systems 14(1), 105–111 (1999)
31. Montanari, U.: Networks of Constraints : Fundamental Properties and Applications
to Picture Processing. Information Science 7(2), 95–132 (1974)
32. Ott, A.: Unit commitment in the pjm day-ahead and real-time mar-
kets, June 2010. http://www.ferc.gov/eventcalendar/Files/20100601131610-Ott,
%20PJM.pdf (accessed April 22, 2012)
33. Purchala, K., Meeus, L., Van Dommelen, D., Belmans, R.: Usefulness of DC power
flow for active power flow analysis. In: Power Engineering Society General Meeting,
pp. 454–459 (2005)
34. Ryoo, H., Sahinidis, N.: A branch-and-reduce approach to global optimization.
Journal of Global Optimization 8(2), 107–138 (1996)
35. Sahinidis, N.: Global optimization and constraint satisfaction: the branch-and-
reduce approach. In: Bliek, C., Jermann, C., Neumaier, A. (eds.) Global Optimiza-
tion and Constraint Satisfaction. LNCS, vol. 2861, pp. 1–16. Springer, Heidelberg
(2003)
36. Toh, K.C., Todd, M., Ttnc, R.H.: Sdpt3 - a matlab software package for semidefi-
nite programming. Optimization Methods and Software 11, 545–581 (1999)
37. Van Hentenryck, P., McAllister, D., Kapur, D.: Solving Polynomial Systems Using
a Branch and Prune Approach. SIAM Journal on Numerical Analysis 34(2) (1997)
 Strengthening Convex Relaxations with Bound Tightening 57

38. Van Hentenryck, P., Michel, L., Deville, Y.: Numerica: a Modeling Language for
Global Optimization. The MIT Press, Cambridge (1997)
39. Van Hentenryck, P., Saraswat, V., Deville, Y.: The design, implementation, and
evaluation of the constraint language cc(FD). In: Podelski, A. (ed.) Constraint Pro-
gramming: Basics and Trends. LNCS, vol. 910, pp. 293–316. Springer, Heidelberg
(1995)
40. Verma, A.: Power grid security analysis: An optimization approach. Ph.D. thesis,
Columbia University (2009)
41. Wächter, A., Biegler, L.T.: On the implementation of a primal-dual interior point
filter line search algorithm for large-scale nonlinear programming. Mathematical
Programming 106(1), 25–57 (2006)
 Broken Triangles Revisited

Martin C. Cooper1(B) , Aymeric Duchein1 , and Guillaume Escamocher2

1
IRIT, University of Toulouse III, 31062 Toulouse, France
{cooper,Aymeric.Duchein}@irit.fr
2
INSIGHT Centre for Data Analytics, University College Cork, Cork, Ireland
[email protected]

Abstract. A broken triangle is a pattern of (in)compatibilities between

assignments in a binary CSP (constraint satisfaction problem). In
the absence of certain broken triangles, satisfiability-preserving domain
reductions are possible via merging of domain values. We investigate the
possibility of maximising the number of domain reduction operations by
the choice of the order in which they are applied, as well as their inter-
action with arc consistency operations. It turns out that it is NP-hard
to choose the best order.

1 Introduction
The notion of broken triangle has generated a certain amount of interest in the
constraints community: it has led to the definition of novel tractable classes [3,7],
variable elimination rules [1] and domain reduction rules [4,5]. The merging of
pairs of values in the same variable domain which do not belong to a broken
triangle has been shown to lead to considerable reduction of search space size
for certain benchmark instances of binary CSP [4]. The corresponding reduction
operation, known as BTP-merging, is satisfiability-preserving and is therefore
worthy of a deeper theoretical analysis as a potentially useful preprocessing
operation. An obvious question is whether the order in which BTP-merging
operations, and other domain-reduction operations such as arc consistency, are
performed has an effect on the number of possible merges.
Definition 1. A binary CSP instance I consists of
– a set X of n variables,
– a domain D(x) of possible values for each variable x ∈ X, with d the maxi-
mum domain size,
– a relation Rxy ⊆ D(x) × D(y), for each pair of distinct variables x, y ∈ X,
which consists of the set of compatible pairs of values (a, b) for variables
(x, y).
A partial solution to I on Y = {y1 , . . . , yr } ⊆ X is a set {y1 , a1 , . . . , yr , ar }
such that ∀i, j ∈ [1, r], (ai , aj ) ∈ Ryi yj . A solution to I is a partial solution on
X.
M.C. Cooper—supported by ANR Project ANR-10-BLAN-0210 and EPSRC grant
EP/L021226/1.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 58–73, 2015.
DOI: 10.1007/978-3-319-23219-5 5
 Broken Triangles Revisited 59

For simplicity of presentation, Definition 1 assumes that there is exactly one

constraint relation for each pair of variables. An instance I is arc consistent if
for each pair of distinct variables x, y ∈ X, for each value a ∈ D(x), there is a
value b ∈ D(y) such that (a, b) ∈ Rxy .



• a
d •hhhh 
A h
h• b

y A   
A  x
e A•

z

Fig. 1. A broken triangle on two values a, b ∈ D(x).

We now formally define the value-merging operation based on absence of

certain broken triangles. A broken triangle on values a, b is shown in Figure 1.
In all figures in this paper, the pairs of values joined by a solid line are exactly
those belonging to the corresponding constraint relation.
Definition 2. A broken triangle on the pair of variable-value assignments a, b ∈
D(x) consists of a pair of assignments d ∈ D(y), e ∈ D(z) to distinct variables
y, z ∈ X \ {x} such that (a, d) ∈ / Rxy , (b, d) ∈ Rxy , (a, e) ∈ Rxz , (b, e) ∈
/ Rxz
and (d, e) ∈ Ryz . The pair of values a, b ∈ D(x) is BT-free if there is no broken
triangle on a, b.
BTP-merging values a, b ∈ D(x) in a binary CSP consists in replacing a, b
in D(x) by a new value c which is compatible with all variable-value assignments
compatible with at least one of the assignments x, a or x, b (i.e. ∀y ∈ X \ {x},
∀d ∈ D(y), (c, d) ∈ Rxy iff (a, d) ∈ Rxy or (b, d) ∈ Rxy )
When a, b ∈ D(x) are BT-free in a binary CSP instance I, the instance I 
obtained from I by merging a, b ∈ D(x) is satisfiable if and only if I is satisfiable.
Furthermore, given a solution to the instance resulting from the merging of two
values, we can find a solution to the original instance in linear time [4].
The paper is structured as follows. In Section 2 we investigate the interaction
between arc consistency and BTP-merging. In Section 3 we show that finding the
best order in which to apply BTP-mergings is NP-hard, even for arc-consistent
instances. In Section 4 we prove that this remains true even if we only perform
merges at a single variable. In Section 5 we take this line of work one step further
by showing that it is also NP-hard to find the best sequence of merges by a weaker
property combing virtual interchangeability and neighbourhood substitutability.

2 Mixing Arc Consistency and BTP-merging

BTP-merging can be seen as a generalisation of neighbourhood substitutabil-
ity [6], since if a ∈ D(x) is neighbourhood substitutable for b ∈ D(x) then a, b
60 M.C. Cooper et al.

can be BTP-merged. The possible interactions between arc consistency (AC) and
neighbourhood substitution (NS) are relatively simple and can be summarised
as follows:
1. The fact that a ∈ D(x) is AC-supported or not at variable y remains invariant
after the elimination of any other value b (in D(x) \ {a} or in the domain
D(z) of any variable z = x) by neighbourhood substitution.
2. An arc-consistent value a ∈ D(x) that is neighbourhood substitutable
remains neighbourhood substitutable after the elimination of any other value
by arc consistency.
3. On the other hand, a value a ∈ D(x) may become neighbourhood substi-
tutable after the elimination of a value c ∈ D(y) (y = x) by arc consistency.
Indeed, it has been shown that the maximum cumulated number of elimina-
tions by arc consistency and neighbourhood substitution can be achieved by
first establishing arc consistency and then applying any convergent sequence
of NS eliminations (i.e. any valid sequence of eliminations by neighbourhood
substitution until no more NS eliminations are possible) [2].

   
• • • •
 

 
a • c •
b • @
•l • @
 @ l  @ @
@ll @ @
@ l @ @
@ l  @ @ 
(a) @• l• (b) @• @•
 

Fig. 2. (a) An instance in which applying AC leads to the elimination of all values
(starting with the values a and b), but applying BTP merging leads to just one elimi-
nation, namely the merging of a with b (with the resulting instance shown in (b)).

The interaction between arc consistency and BTP-merging is not so simple

and can be summarised as follows:
1. The fact that a ∈ D(x) is AC-supported or not at variable y remains invariant
after the BTP-merging of any other pair of other values b, c (in D(x)\{a} or in
the domain D(z) of any variable z = x). However, after the BTP-merging of
two arc-inconsistent values the resulting merged value may be arc consistent.
An example is given in Figure 2(a). In this 3-variable instance, the two values
a, b ∈ D(x) can be eliminated by arc consistency (which in turn leads to
 Broken Triangles Revisited 61

the elimination of all values), or alternatively they can be BTP-merged (to

produce the value c) resulting in the instance shown in Figure 2(b) in which
no more eliminations are possible by AC or BTP-merging.
2. A single elimination by AC may prevent a sequence of several BTP-mergings.
An example is given in Figure 3(a). In this 4-variable instance, if the value b
is eliminated by AC, then no other eliminations are possible by AC or BTP-
merging in the resulting instance (shown in Figure 3(b)), whereas if a and b
are BTP-merged into a new value d (as shown in Figure 3(c)) this destroys
a broken triangle thus allowing c to be BTP-merged with d (as shown in
Figure 3(d)).
3. On the other hand, two values in the domain of a variable x may become
BTP-mergeable after an elimination of a value c ∈ D(y) (y = x) by arc
consistency.

3 The Order of BTP-mergings

It is known that BTP-merging can both create and destroy broken triangles [4].
This implies that the choice of the order in which BTP-mergings are applied
may affect the total number of merges that can be performed. Unfortunately,
maximising the total number of merges in a binary CSP instance turns out to be
NP-hard, even when bounding the maximum size of the domains d by a constant
as small as 3. For simplicity of presentation, we first prove this for the case in
which the instance is not necessarily arc consistent. We will then prove a tighter
version, namely NP-hardness of maximising the total number of merges even in
arc-consistent instances.
Theorem 1. The problem of determining if it is possible to perform k BTP-
mergings in a boolean binary CSP instance is NP-complete.
Proof. For a given sequence of k BTP-mergings, verifying if this sequence is
correct can be performed in O(kn2 d2 ) time because looking for broken triangles
for a given couple of values takes O(n2 d2 ) [4]. As we can verify a solution in
polynomial time, the problem of determining if it is possible to perform k BTP-
mergings in a binary CSP instance is in NP. So to complete the proof of NP-
completeness it suffices to give a polynomial-time reduction from the well-known
3-SAT problem. Let I3SAT be an instance of 3-SAT (SAT in which each clause
contains exactly 3 literals) with variables X1 , . . . , XN and clauses C1 , . . . , CM .
We will create a boolean binary CSP instance ICSP which has a sequence of
k = 3 × M mergings if and only if I3SAT is satisfiable.
For each variable Xi of I3SAT , we add a new variable zi to ICSP . For each
occurrence of Xi in the clause Cj of I3SAT , we add two more variables xij and
yij to ICSP . Each D(zi ) contains only one value ci and each D(xij ) (resp. D(yij ))
contains only two values ai and bi (resp. ai and bi ). The roles of variables xij
and yij are the following:
Xi = true ⇔ ∀j, ai , bi can be merged in D(xij ) (1)
Xi = f alse ⇔ ∀j, ai , bi can be merged in D(yij ) (2)
62 M.C. Cooper et al.

   
• • • •

%@ @ @ @
% @ @ @ @
% @ @ @ @
 % @ @   @ @ 
a •h% h @•
@ ( a •hh @•
@ (
@ hh( h( h(((( hhh
h( (((
b •% ( h( h h @ @
( (h(
( h h @
l (( hhh (( hhh
(@
c •e l@
@• c •(@ @•
 @e   @ @ 
l
@e @l @ @
@e @ l @ @
@e@l  @ @ 
(a) l•
e• @
@ (b) @• @•
 

   
• • • •
 @ @ %@,@

@ @ % ,@ @
@ @ %,, @ @
 @ @  %, @ @ 
d •hhh @ (@• % 
, @ @•
hh( ((((
@ ( h(
h h @ •,
% @
((( hhh l
c •(@ h
@•  e
l
@•
 @ @  e 
@ @ ell
@ @ e l
@ @  el 
(c) @• @• (d) e• l•
 

Fig. 3. (a) An instance in which applying AC leads to one elimination (the value b)
(as shown in (b)), but applying BTP merging leads to two eliminations, namely a with
b (shown in (c)) and then d with c (shown in (d)).

In order to prevent the possibility of merging both (ai , bi ) and (ai , bi ), we
define the following constraints for zi , xij and yij : ∀j Rxij zi = {(bi , ci )} and
Ryij zi = {(bi , ci )}; ∀j ∀ k Rxij yik = {(ai , ai )}. These constraints are shown
in Figure 4(a) for a single j (where a pair of points not joined by a solid line
are incompatible). By this gadget, we create a broken triangle on each yij when
merging values in the xij and vice versa.
Then for each clause Ci = (Xj , Xk , Xl ), we add the following constraints in
order to have at least one of the literals Xj , Xk , Xl true: Ryji yki = {(aj , bk )},
Ryki yli = {(ak , bl )} and Ryli yji = {(al , bj )}. This construction, shown in
Figure 4(b), is such that it allows two mergings on the variables yji , yki , yli
before a broken triangle is created. For example, merging aj , bj and then ak , bk
creates a broken triangle on ai , bi . So a third merging is not possible.
 Broken Triangles Revisited 63

   
ai • • ai aj • • bk

bi • • bi bj • • ak

 @  S    

xij @  yij yji S  yki
@ S • bl
@• ci S
 S• al
zi 
(a) (b) yli

Fig. 4. (a) Representation of the variable Xi and its negation (by the possibility of
performing a merge in D(xij ) or D(yij ), respectively, according to rules (1),(2)). (b)
Representation of the clause (Xj ∨ Xk ∨ Xl ). Pairs of points joined by a solid line are
compatible and incompatible otherwise.

If the clause Ci contains a negated literal Xj instead of Xj , it suffices to

replace yji by xji . Indeed, Figure 5 shows the construction for the clause (Xj ∨
Xk ∨ Xl ) together with the gadgets for each variable. The maximum number
of mergings that can be performed are one per occurrence of each variable in
a clause, which is exactly 3 × M . Given a sequence of 3 × M mergings in the
CSP instance, there is a corresponding solution to I3SAT given by (1) and (2).
The above reduction allows us to code I3SAT as the problem of testing the
existence of a sequence of k = 3 × M mergings in the corresponding instance
ICSP . This reduction being polynomial, we have proved the NP-completeness
of the problem of determining whether k BTP merges are possible in a boolean
binary CSP instance.

zj zk
 
cj • ck •
@  @
  @   @ 

bj • @• bj 
bk • @• bk
T  
•Taj
aj • T ak • • ak
  T 
 
yji
 xji
T  yki xki
T 
al •

• al

bl • • bl
 e 
%
yli e   %x
e % li
e%
cl •

zl

Fig. 5. Gadget representing the clause (Xj ∨ Xk ∨ Xl ).

64 M.C. Cooper et al.

The reduction given in the proof of Theorem 1 supposes that no arc consis-
tency operations are used. We will now show that it is possible to modify the
reduction so as to prevent the elimination of any values in the instance ICSP
by arc consistency, even when the maximum size of the domains d is bounded
by a constant as small as 3. Recall that an arc-consistent instance remains arc-
consistent after any number of BTP-mergings.

Theorem 2. The problem of determining if it is possible to perform k BTP-

mergings in an arc-consistent binary CSP instance is NP-complete, even when
only considering binary CSP instances where the size of the domains is bounded
by 3.

Proof. In order to ensure arc consistency of the instance ICSP , we add a new
value di to the domain of each of the variables xij , yij , zi . However, we can-
not simply make di compatible with all values in all other domains, because
this would allow all values to be merged with di , destroying in the process the
semantics of the reduction.
In the three binary constraints concerning the triple of variables xij , yij , zi ,
we make di compatible with all values in the other two domains except di . In
other words, we add the following tuples to constraint relations, as illustrated in
Figure 6:

– ∀i∀j, (ai , di ), (bi , di ), (di , ci ) ∈ Rxij zi

– ∀i∀j, (ai , di ), (bi , di ), (di , ci ) ∈ Ryij zi
– ∀i∀j, (ai , di ), (bi , di ), (di , ai ), (di , bi ) ∈ Rxij yij

This ensures arc consistency, without creating new broken triangles on ai , bi or

ai , bi , while at the same time preventing BTP-merging with the new value di .
It is important to note that even after BTP-merging of one of the pairs ai , bi or
ai , bi , no BTP-merging is possible with di in D(xij ), D(yij ) or D(zi ) due to the
presence of broken triangles on this triple of variables. For example, the pair of
values ai , di ∈ D(xij ) belongs to a broken triangle on ci ∈ D(zi ) and di ∈ D(yij ),
and this broken triangle still exists if the values ai , bi ∈ D(yij ) are merged.
We can then simply make di compatible with all values in the domain of all
variables outside this triple of variables. This ensures arc consistency, and does
i z
 
• ci

AS

• AS

 QA Sdi
 

 @Q S 
@AQ



bi •P @
AQ
S• bi
P 
ai •h
hP 
P @
A
( • ai

(h
(h(P h(
(P
hP
(
di • hA• di
 
xij yij

Fig. 6. Ensuring arc consistency between the variables zi , yij , xij by addition of new
values di .
 Broken Triangles Revisited 65

not introduce any broken triangles on ai , bi or ai , bi . With these constraints we
ensure arc consistency without changing any of the properties of ICSP used in
the reduction from 3-SAT described in the proof of Theorem 1. For each pair of
values ai , bi ∈ D(xij ) and ai , bi ∈ D(yij ), no new broken triangle is created since
these two values always have the same compatibility with all the new values dk .
As we have seen, the constraints shown in Figure 6 prevent any merging of the
new values dk .

Corollary 1. The problem of determining if it is possible to perform k value

eliminations by arc consistency and BTP-merging in a binary CSP instance is
NP-complete, even when only considering binary CSP instances where the size
of the domains is bounded by 3.

From a practical point of view, an obvious question concerns the existence of

a non-optimal but nevertheless useful heuristic for choosing the order of BTP-
merges. Imagine that a heuristic concerning the order in which to apply BTP-
merges involves first finding all possible merges before choosing between them.
If this is the case, then once a merge has been chosen and performed, the list of
possible merges has to be recalculated. This process is thus already an order of
magnitude slower than the simple technique (applied in the experiments in [4])
consisting of performing a merge as soon as it is detected.

4 Optimal Sequence of BTP-mergings at a Single

Variable
We now show that even when only considering a single domain, finding the
optimal order of BTP-mergings is NP-Complete. For simplicity of presentation,
we first prove this for the case in which the instance is not necessarily arc-
consistent. We then prove a tighter version for arc-consistent instances.

Theorem 3. The problem of determining if it is possible to perform k BTP-

mergings within a same domain in a binary CSP instance is NP-Complete.

Proof. For a given sequence of k BTP-mergings within a domain of a binary CSP

instance I, verifying if this sequence is correct can be performed in O(kN 2 d2 )
time, with N being the number of variables in I and d being the size of the largest
domain in I, because looking for broken triangles for a given couple of values
takes O(N 2 d2 ). As we can verify a solution in polynomial time, the problem of
determining if it is possible to perform k BTP-mergings within a single domain in
a binary CSP instance is in NP. So to complete the proof of NP-Completeness, it
suffices to give a polynomial-time reduction from the well-known SAT problem.
Let ISAT be a SAT instance with n variables X1 , X2 , . . . , Xn and m clauses
C1 , C2 , . . . , Cm . We reduce ISAT to a binary CSP instance ICSP containing a
variable v0 such that ISAT is satisfiable if and only if we can make k = n + m
BTP-mergings within D(v0 ). ICSP is defined in the following way:
1. ICSP contains 1+n(2+4m+9(n−1)) variables v0 , v1 , v2 , . . . , vn(2+4m+9(n−1)) .
66 M.C. Cooper et al.

2. D(v0 ) contains 3×n+m values with the following names: x1 , x2 , . . . , xn , x1 T,

x2 T, . . . , xn T, x1 F, x2 F, . . . , xn F, c1 , c2 , . . . , cm . All other domains in ICSP
only contain one value.
3. ∀i ∈ [1, n], xi T and xi F can never be BTP-merged. The idea here is to allow
exactly one BTP-merging among the three values xi , xi T and xi F : xi and
xi T if Xi is assigned True in the SAT instance ISAT , xi and xi F if Xi is
assigned False instead.
4. ∀(i, j) ∈ [1, n] × [1, m] such that Cj does not contain Xi (respectively Xi ),
cj can never be BTP-merged with xi T (respectively xi F ).
5. ∀(i, j) ∈ [1, n]×[1, m] such that Cj contains Xi (respectively Xi ), cj can only
be BTP-merged with xi T (respectively xi F ) if either cj or xi T (respectively
xi F ) has been previously BTP-merged with xi .
6. ∀i, j with 1 ≤ i < j ≤ n, the nine following couples can never be BTP-
merged: xi and xj , xi and xj T , xi and xj F , xi T and xj , xi T and xj T , xi T
and xj F , xi F and xj , xi F and xj T , xi F and xj F . The idea here is to prevent
any BTP-merging between two CSP values corresponding to two different
SAT variables.

When we say that two values a and b can never be BTP-merged, it means that
we add two variables va and vb , with only one value a in D(va ), and only one
value b in D(vb ), such that a is compatible with a and incompatible with b , b
is compatible with b and incompatible with a , a is compatible with b and all
other edges containing either a or b are incompatible. The purpose of making
a (respectively b ) incompatible will all values in the instance except a and b
(respectively b and a ) is twofold. First, it ensures that no future BTP-merging
can establish a compatibility between a (respectively b ) and b (respectively a)
and thus destroy the broken triangle. Second, it ensures that the only broken
triangle introduced by a and b is on a and b, so that the addition of a and b
does not prevent any other BTP-merging than the one between a and b.
When we say that two values a and b can only be BTP-merged if either a
or b has been previously BTP-merged with some third value c, it means that
we add two variables va and vb , with only one value a in D(va ), and only one
value b in D(vb ), such that a is compatible with a and incompatible with b ,
b is compatible with b and incompatible with a , c is compatible with both a
and b , a is compatible with b and all other edges containing either a or b are
incompatible. Here again, the purpose of making a (respectively b ) incompatible
will all values in the instance except a, b and c (respectively b, a and c) is
twofold. First, it ensures that no future BTP-merging that does not include c
can establish a compatibility between a (respectively b ) and b (respectively a)
and thus destroy the broken triangle. Second, it ensures that the only broken
triangle introduced by a and b is on a and b, so that the addition of a and b
does not prevent any other BTP-merging than the one between a and b.
For every couple of values that can never be BTP-merged, and for every
couple of values that can only be BTP-merged when one of them has been
previously BTP-merged with some third value, we add two new single-valued
variables to ICSP . Therefore, the third point in the definition of ICSP adds 2n
 Broken Triangles Revisited 67

variables to ICSP , the fourth and fifth points in the definition of ICSP add 4nm
variables to ICSP and the sixth point in the definition of ICSP adds 9n(n − 1)
variables to ICSP . Therefore, the total number of single-valued variables added
to ICSP is n(2 + 4m + 9(n − 1)), as expected from the first point in the definition
of ICSP .

– The number of BTP-mergings is limited by n + m:

From the third point in the definition of ICSP , for all i ∈ [1, n], we can BTP-
merge at most once within the triple {xi , xi T, xi F }. From the sixth point
in the definition of ICSP , we cannot BTP-merge two values within D(v0 ) if
they are associated to two different SAT variables Xi and Xj . Therefore, we
have at most m BTP-mergings remaining, one for each cj for 1 ≤ j ≤ m.
– If we can BTP-merge n + m times, then we have a solution for ISAT :
Since we have done the maximum number of BTP-mergings, we know that
for all i ∈ [1, n], xi has been BTP-merged with either xi T or xi F , but not
both. So we create the following solution for ISAT : ∀i ∈ [1, n], we assign T rue
to Xi if xi and xi T have been BTP-merged, and F alse otherwise. From the
fourth and fifth points in the definition of ICSP , we know that for each j
in [1, m], Cj is satisfied by the literal associated with the value Cj has been
BTP-merged with.
– If we have a solution for ISAT , then we can BTP-merge n + m times:
∀i ∈ [1, n], we BTP-merge xi with xi T if T rue has been assigned to Xi , with
xi F otherwise. ∀(i, j) ∈ [1, n] × [1, m], we BTP-merge cj and the value that
xi has been BTP-merged with if Xi is satisfied in Cj and cj has not been
BTP-merged yet. From the fifth point in the definition of ICSP , we know we
can BTP-merge each cj once.

Therefore ISAT is satisfiable if and only if we can perform k = n + m BTP-

mergings within D(v0 ), and we have the result.

We now generalise the result of Theorem 3 to arc-consistent binary CSP

instances.

Theorem 4. The problem of determining if it is possible to perform k BTP-

mergings within a same domain in an arc-consistent binary CSP instance is
NP-Complete.

Proof. We transform the binary CSP instance ICSP from the proof of Theorem 3

into an arc-consistent binary CSP instance ICSP . To do so, we add a new value
di in D(vi ) for 1 ≤ i ≤ n(2 + 4m + 9(n − 1)) such that all di are incompatible

with each other and compatible with all other points in ICSP . This ensures arc
consistency. It remains to show that:

1. For any couple of values (a, b) ∈ D(v0 ), adding the values di does not create
the broken triangle on a and b, even if a or b is the result of a previous
BTP-merging:
Suppose that we have two values a, b ∈ D(v0 ) such that adding the values di
68 M.C. Cooper et al.

creates a broken triangle on a and b. Let a ∈ D(va ) and b ∈ D(vb ) be the

other two values forming the broken triangle. Since it was the new values di
that created this particular broken triangle, either a or b is one of the di .
Without loss of generality, we assume that a is one of the di . But since the
di are compatible with all values from D(v0 ), both a and b are compatible
with a , even if a or b is the result of a previous BTP-merging. Therefore,
there cannot be any broken triangle on a and b caused by the new values di .
2. For all i ∈ [1, n(2 + 4m + 9(n − 1))], it is never possible to BTP-merge the
two values in D(vi ):
We assume, for simplicity of presentation and without loss of generality,
that the SAT instance ISAT has more than one variable and that no clause
contains both a variable and its negation. Let i ∈ [1, n(2 + 4m + 9(n − 1))].
Let a and di be the two points in D(vi ). From the proof of Theorem 3, we
know that a is compatible with only one value from D(v0 ). Let b be this
value. If b is associated with one of the SAT variables from ISAT , then from
the sixth point in the definition of ICSP in the proof of Theorem 3 we know
that there is at least one value c ∈ D(v0 ) that can never be BTP-merged
with b, and therefore will always be incompatible with a. If on the other
hand c is associated with one of the SAT clauses from ISAT , then from the
fourth point in the definition of ICSP in the proof of Theorem 3 we know
that there is at least one value c ∈ D(v0 ) that can never be BTP-merged
with b, and therefore will always be incompatible with a. Therefore, we have
a value c ∈ D(v0 ) that is always incompatible with a, even if c is the result
of a previous BTP-merging. Let j ∈ [1, n(2 + 4m + 9(n − 1))], such that
j = i. Since the di are incompatible with each other, and compatible with

all other values in ICSP , then dj is compatible with both a and c, and di
is compatible with c and incompatible with dj . Therefore we have a broken
triangle on a and di that can never be destroyed. Therefore a and di can
never be BTP-merged and we have the result.

One motivation for studying the single-variable version of the problem was
that if all values in D(x) can be BTP-merged, then the variable x can be elim-
inated since its domain becomes a singleton. Our proof of NP-hardness in the
single-variable case relied on a large domain which was not actually reduced
to a singleton. There remains therefore an interesting open question concerning
the complexity of eliminating the largest number of variables by sequences of
BTP-merging operations.

5 Virtual Interchangeability and Neighbourhood

Substitution

Since testing whether two values a, b ∈ D(x) are BTP-mergeable requires testing
all pairs of assignments to all pairs of distinct variables y, z = x, it is natural
to investigate weaker versions which are less costly to test. Two such weaker
versions are neighbourhood substitutability [6] and virtual interchangeability [8].
 Broken Triangles Revisited 69

Given two values a, b ∈ D(x), a is neighbourhood substitutable by b (NS), and

so can be merged with b, if for all variables y = x, ∀c ∈ D(y), (a, c) ∈ Rxy ⇒
(b, c) ∈ Rxy . Two values a, b ∈ D(x) are virtual interchangeable (VI), and so
can be merged, if for all variables y = x except at most one, ∀c ∈ D(y), (a, c) ∈
Rxy ⇔ (b, c) ∈ Rxy . Applying both VI and neighbourhood substitution (NS)
operations until convergence provides a weaker and less time-costly alternative
to applying BTP-merging operations until convergence. An interesting question
is therefore whether it is possible to find in polynomial time an optimal (i.e.
longest) sequence of VI and NS operations. The following theorem shows that
this problem is in fact also NP-hard.

Theorem 5. Determining whether there exists a sequence of VI and NS opera-

tions of length k that can be applied to a binary CSP instance is NP-complete.

Proof. Since checking the validity of a sequence of VI and NS operations can be

achieved in polynomial time, the problem is in NP. To complete the proof we
demonstrate a polynomial reduction from 3SAT. Let I3SAT be an instance of
3SAT. We will show how to construct a binary CSP instance ICSP and a value
k so that it is possible to perform k merges by VI and NS if and only if I3SAT
is satisfiable.
For each variable x in I3SAT , we introduce Boolean variables called x and x
in ICSP : variable x in I3SAT is assigned true (respectively, false) if and only if the
two domain values in D(x) (respectively, D(x)) are merged in the corresponding
sequence of merges in ICSP . Figure 7(a) shows the gadget for choosing the truth
value for x. The variables x and x are both connected to another variable, not
shown in this figure: this prevents the values in their domains being merged
before the values in the domains of t or t are merged. Initially, the only merges
(by VI and NS) which can occur are in the domain of variable s: we can either
merge values 1 and 2 or values 2 and 3. Once one of these merges is performed,
the other it not possible. Figure 7 shows the propagation of merges which occurs
in the second of these cases. Figure 7(b) shows the result of this merge in D(s).
In Figure 7(b) the only merge that is possible is the merging of values 1 and
3 (by NS) in D(t). The result is shown in Figure 7(c). Now, the two values in
the domain of x can be merged (by VI) since x is constrained by a single other
variable (not shown in the figure). It is important to note that no other merges
are possible. By a similar chain of propagations, if we had chosen to merge 1
and 2 in D(s), then the values 1 and 3 in D(t ) would have been merged, and
finally the two values in D(x). This gadget therefore allows us to choose a truth
value for the corresponding variable x of I3SAT .
In order to code the instance I3SAT , we need to be able to have several
copies of each variable. This is achieved by the gadget shown in Figure 8(a).
The variables x1 and x2 are each assumed to be constrained by a single other
variable not shown in the figure. The variable x is the variable in the gadget of
Figure 7. If the values in D(x) are merged then this allows the merging (by VI)
of the pair of values 0, 1 ∈ D(u) and the merging of the pair of values 2,3. In the
resulting instance, the two values in the domain of xi (i = 1, 2) can be merged.
70 M.C. Cooper et al.

 x
t  •

  

1 •   •
QQ   

Q
2  •
s  Q Q
 
Q Q 
1 •  • Q Q•
S
@  3  Q  
 Q

2 •
PPS
@  Q •
SP
@P  t   

(a) P   
• S@ P•Q
3  PP 
S@ P
Q P  x
S @• PP
P Q •
S PPQ PQ
S PQP•
• 
 x
 
•
   
t
1 •Q 


•
Q

  
 Q

2 •Q Q
s  
Q Q 

1 •  • Q Q•
S 
@ 3   Q 
 
Q


•P
{2, 3}  S
@  Q•
PS@
PP  t   

(b)  
S@PP•P  

S @ QQ PP
 PP  x
S @•PP Q P•
Q
S  PPQ
S PQP•
•
 
 x
•

 •
 
t
 
 

2 •
Q 

s  
Q
  
1 •  • Q •
S  
@{1, 3} Q 

 
Q
S

•P
{2, 3}  @  Q•

PS@
PP  t   

(c)  
S@PP•P  

S @ QQ PP
 PP  x
S @•P Q P•
S PPQ PQ
S PQP•
• 

Fig. 7. (a) Gadget for choosing a truth value for x: true if the two values in D(x) are
merged; false if the two values in D(x) are merged. This same gagdet (b) after merging
the values 2 and 3 in D(s), then (c) after merging the values 1 and 3 in D(t) .
 Broken Triangles Revisited 71

u
  x1
• •

x  0

•P • •
PP 1 
(a) PP 

x2
•P P• •
 PP 2
PP
P• •
 3 


x •
Q
Q 
•Q QQ •
 S
v 
 Q Q  v 

y • Q Q• 
•P • S
Q  
PP
 S S 
(b) 
Q  P PP
•  Q 
• •


P S
PP S •
S
   PP
z •  S•
P
 

•

Fig. 8. (a) Gadget for making copies of a variable x: if the two values in D(x) can be
merged, then the two values in D(x1 ) and the two values in D(x2 ) can be merged. (b)
Gadget used in the simulation of a clause: the variable v is true (i.e. the two values in
its domain can be merged) if and only if x, y, z are all true.

This gadget therefore allows us to make multiple copies of the variable x all with
the same truth value.
To complete the proof, we now show how to code each clause c of I3SAT . There
are exactly seven assignments to the variables in c which satisfy this clause. For
each of these assignments, we add a gadget of the form shown in Figure 8(b). The
variables x, y, z are the output of the gadgets introduced above and correspond
to the variables x, y, z occurring in the clause c in I3SAT . In the example shown
in the figure, the satisfying assignment is x = y = z = true. When the two
values in each of the domains of these three variables can be merged (and only
in this case), the values in the domain of v can also be merged. The triangle of
variables to the right of v in Figure 8(b) prevents the merging of the values in
D(v) when only two of the three variables x, y, z are assigned true.
In order to have the same number of copies of x and x in our construction,
we also add a gadget similar to Figure 8(b) for the one non-satisfying assignment
to the three variables of the clause c: in this case, the variable v is constrained
by two other variables (as is the variable v  in Figure 8(b)) which prevents the
merging of the values in D(v).
Suppose that there are n variables and m clauses in I3SAT . The maximum
total number of merges which can be performed in ICSP is 3 per gadget shown in
72 M.C. Cooper et al.

Figure 7, 4 per gadget shown in Figure 8(a) and 1 per gadget shown in Figure 8(b)
(provided the gadget corresponds to a truth assignment which satisfies the clause
c). Each clause c requires four copies of each of the three variables x occurring
in c (as well as four copies of x). For each copy of each literal assigned the value
true, there are 4 merges in the gadget of Figure 8(a). For the first occurrence
of each variable, produced by the gadget of Figure 7, there is one less merge (3
instead of 4). Finally, for each satisfied clause there is one merge. This implies
that we can perform a total of k = 48m − n + m = 49m − n merges in ICSP if
and only if I3SAT is satisfiable. Since this reduction is clearly polynomial, this
completes the proof.

6 Conclusion

We have investigated the possibility of maximising the number of domain reduc-

tion operations in binary CSP instances by choosing an optimal order in which
to apply them. Whereas for consistency and neighbourhood-substitution oper-
ations, the number of domain reduction operations can be maximised in poly-
nomial time, the problem becomes NP-hard when we allow merging operations,
such as virtual interchangeability or BTP-merging. We emphasise that this does
not detract from the possible utility of such value-merging operations in practice,
which is an independent question.
Different tractable subproblems of binary CSP have been defined based on
the absence of certain broken triangles [3,5,7]. Instances can be solved by elim-
inating variables one by one and, in each case, the recognition of instances in
the tractable class can be achieved in polynomial time by a greedy algorithm
since the elimination of one variable cannot prevent the elimination of another
variable. BTP-merging, on the other hand, performs reduction operations on a
lower level than the elimination of variables. Given the NP-completeness results
in this paper, recognizing those instances which can be reduced to a trivial prob-
lem with only singleton domains by some sequence of BTP-merges is unlikely to
be tractable, but this remains an open problem.

References
1. Cohen, D.A., Cooper, M.C.: Guillaume Escamocher and Stanislav Živný, Variable
and Value Elimination in Binary Constraint Satisfaction via Forbidden Patterns. J.
Comp. Systems Science (2015). http://dx.doi.org/10.1016/j.jcss.2015.02.001
2. Martin, C.: Cooper, Fundamental Properties of Neighbourhood Substitution in Con-
straint Satisfaction Problems. Artif. Intell. 90(1–2), 1–24 (1997)
3. Cooper, M.C., Jeavons, P.G., Salamon, A.Z.: Generalizing constraint satisfaction on
trees: Hybrid tractability and variable elimination. Artif. Intell. 174(9–10), 570–584
(2010)
4. Cooper, M.C., El Mouelhi, A., Terrioux, C., Zanuttini, B.: On broken triangles.
In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 9–24. Springer, Heidelberg
(2014)
 Broken Triangles Revisited 73

5. Cooper, M.C.: Beyond consistency and substitutability. In: O’Sullivan, B. (ed.) CP

2014. LNCS, vol. 8656, pp. 256–271. Springer, Heidelberg (2014)
6. Freuder, E.C.: Eliminating interchangeable values in constraint satisfaction prob-
lems. In: Proceedings AAAI 1991, pp. 227–233 (1991)
7. Jégou, P., Terrioux, C.: The extendable-triple property: a new CSP tractable class
beyond BTP. In: AAAI (2015)
8. Likitvivatanavong, C., Yap, R.H.C.: Eliminating redundancy in CSPs through merg-
ing and subsumption of domain values. ACM SIGAPP Applied Computing Review
13(2) (2013)
 A Microstructure-Based Family of Tractable
Classes for CSPs

Martin C. Cooper1(B) , Philippe Jégou2 , and Cyril Terrioux2

1
IRIT, University of Toulouse III, 31062 Toulouse, France
[email protected]
2
Aix-Marseille Université, CNRS, ENSAM, Université de Toulon, LSIS UMR 7296,
Avenue Escadrille Normandie-Niemen, 13397 Marseille Cedex 20, France
{philippe.jegou,cyril.terrioux}@lsis.org

Abstract. The study of tractable classes is an important issue in Arti-

ficial Intelligence, especially in Constraint Satisfaction Problems. In
this context, the Broken Triangle Property (BTP) is a state-of-the-art
microstructure-based tractable class which generalizes well-known and
previously-defined tractable classes, notably the set of instances whose
constraint graph is a tree. In this paper, we propose to extend and to
generalize this class using a more general approach based on a param-
eter k which is a given constant. To this end, we introduce the k-BTP
property (and the class of instances satisfying this property) such that
we have 2-BTP = BTP, and for k > 2, k-BTP is a relaxation of BTP in
the sense that k-BTP  (k + 1)-BTP. Moreover, we show that if k-TW
is the class of instances having tree-width bounded by a constant k, then
k-TW  (k + 1)-BTP. Concerning tractability, we show that instances
satisfying k-BTP and which are strong k-consistent are tractable, that
is, can be recognized and solved in polynomial time. We also study the
relationship between k-BTP and the approach of Naanaa who proposed
a set-theoretical tool, known as the directional rank, to extend tractable
classes in a parameterized way. Finally we propose an experimental study
of 3-BTP which shows the practical interest of this class, particularly
w.r.t. the practical solving of instances satisfying 3-BTP and for other
instances, w.r.t. to backdoors based on this tractable class.

1 Introduction
Finding islands of tractability, generally called tractable classes is an important
issue in Artificial Intelligence, especially in Constraint Satisfaction Problems
(CSPs [1]). Many studies have addressed this issue, from the very beginnings
of Artificial Intelligence. These results are often theoretical in nature with, in
certain cases, tractable classes which can be considered as somewhat artifi-
cial. But some tractable classes have actually been used in practice, such as
the classes defined by constraint networks with bounded tree-width [2,3]. More
recently, the concept of hybrid class has been defined, for example with the
Supported by ANR Project ANR-10-BLAN-0210 and EPSRC grant EP/L021226/1.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 74–88, 2015.
DOI: 10.1007/978-3-319-23219-5 6
 A Microstructure-Based Family of Tractable Classes for CSPs 75

class BTP [4]. This class strictly contains both structural tractable classes (such
as tree-structured CSPs) and tractable classes defined by language restrictions.
One major advantage of this class, in addition to its generalization of already-
known tractable classes, is related to its practical interest. Indeed, instances of
this class can be solved in polynomial time using algorithms, such as MAC (Main-
taining Arc-Consistency [5]) and RFL (Real Full Look-ahead [6]), implemented
in efficient solvers which allows it to be used directly in practice. In addition,
it may also help to explain theoretically the practical efficiency of solvers, even
though the theoretical complexity of the algorithms employed by the solvers is
exponential in the worst case.
In this paper, we return to this type of approach by generalizing the tractable
class BTP which is defined by a property excluding certain patterns (called
Broken Triangles) in the microstructure graph associated with a binary CSP
instance. Very recent work in this same direction introduced the class ETP
[7] which generalizes BTP by relaxing some of its conditions, since it tolerates
some broken triangles which are forbidden by BTP . Here we propose a broader
generalization called k-BTP which extends this previous work along two axes.
First, in the spirit of ETP , the new class allows the presence of a larger number of
broken triangles, generalizing strictly ETP (and thus BTP ). Secondly, the class
k-BTP is parameterized by a constant k, thus providing a generic version, which
may prove of theoretical interest for general values of k, although in practice we
consider the case k = 3 to be of most interest. Thus, while BTP is defined
for sets of three variables and ETP for sets of four variables, k-BTP is defined
on the basis of sets of k + 1 variables where k is a fixed constant. According
to this approach, BTP = 2-BTP while ETP  3-BTP . Thus, this approach
makes it possible to strictly generalize these two classes. Furthermore, k-BTP
retains some of their interesting properties and practical advantages mentioned
above. Notably, we show that classical algorithms such as MAC or RFL can solve
instances belonging to k-BTP in polynomial time, assuming that these instances
verify Strong k-Consistency [8]. Moreover, we highlight the relationships of this
class with known structural and hybrid classes. We show in particular that the
class of constraint networks whose tree-width is bounded by k is strictly included
in the class k-BTP . This result gives a first answer to a question recently asked by
M. Vardi about the relationships between ETP and the tractable class induced
by instances of bounded tree-width. We also highlight a recent but relatively
unknown result that was proposed by Naanaa [9] whose relationships with k-
BTP we investigate.
In Section 2 we recall the definitions of the tractable classes BTP and ETP . In
Section 3 we define the new class k-BTP and show that instances from this class
can be detected in polynomial time even when the variable order is not known
in advance. Furthermore, we show that, under the extra hypothesis of strong
k-consistency, such instances can be solved in polynomial time, and, in fact,
standard algorithms will solve them. In Section 4 we investigate relationships
between k-BTP and several known tractable classes and in Section 5 we report
results of experimental trials on benchmark problems.
76 M.C. Cooper et al.

2 Background
Formally, a constraint satisfaction problem also called constraint network is
a triple (X, D, C), where X = {x1 , . . . , xn } is a set of n variables, D =
(Dx1 , . . . , Dxn ) is a list of finite domains of values, one per variable, and
C = {c1 , . . . , ce } is a finite set of e constraints. Each constraint ci is a pair
(S(ci ), R(ci )), where S(ci ) = {xi1 , . . . , xik } ⊆ X is the scope of ci , and
R(ci ) ⊆ Dxi1 × · · · × Dxik is its compatibility relation. The arity of ci is |S(ci )|.
In this paper, we only deal with the case of binary CSPs, that is CSPs for which
all the constraints are of arity 2. Hence, we will denote by cij the constraints
involving xi and xj . The structure of a constraint network is represented by a
graph, called the constraint graph, whose vertices correspond to variables and
edges to the constraint scopes. An assignment to a subset Y of X is said to be
consistent if it does not violate any constraint whose scope is included in Y .
We use the notation R(cij )[a] to represent the set of values in Dxj compatible
with a ∈ Dxi . Thus, if there is a constraint with scope {i, j}, then R(cij )[a] =
{b ∈ Dxj |(a, b) ∈ R(cij )}; if there is no constraint with scope {i, j}, then, by
default, R(cij )[a] = Dxj . We recall the BTP property presented in [4].
Definition (BTP). A binary CSP instance (X, D, C) satisfies the Broken Tri-
angle Property (BTP) w.r.t. the variable ordering < if, for all triples of vari-
ables (xi , xj , xk ) s.t. i < j < k, if (vi , vj ) ∈ R(cij ), (vi , vk ) ∈ R(cik ) and
(vj , vk ) ∈ R(cjk ), then either (vi , vk ) ∈ R(cik ) or (vj , vk ) ∈ R(cjk ). If neither
of these two tuples exist, (vi , vj , vk , vk ) is called a broken triangle on xk w.r.t.
xi and xj .
If there exists at least one broken triangle on xk w.r.t. xi and xj , (xi , xj , xk ) is
called a broken triple on xk w.r.t. xi and xj . Let BTP be the set of the instances
for which BTP holds w.r.t. some variable ordering. The BTP property is related
to the compatibility between domain values, which can be graphically visualized
(Figure 1) on the microstructure graph. For example, in Figure 1 (a), there is
a broken triangle on x3 with respect to the variables x1 and x2 since we have
(v1 , v3 ) ∈
/ R(c13 ) and (v2 , v3 ) ∈
/ R(c23 ) while (v1 , v2 ) ∈ R(c12 ), (v1 , v3 ) ∈ R(c13 )
and (v2 , v3 ) ∈ R(c23 ) hold. So (x1 , x2 , x3 ) is a broken triple on x3 w.r.t. x1
and x2 . In contrast, in Figure 1 (b), if one of the two dashed edges (that is
binary tuples) appears in the microstructure, the BTP property holds for all
variable orderings.
Very recently, the property BTP has been relaxed to the Extendable-Triple
Property [7] by considering four variables rather than three, and allowing some
broken triangles.
Definition (ETP). A binary CSP instance P satisfies the Extendable-Triple
Property (ETP) with respect to the variable ordering < if, and only if, for all
subsets of four variables (xi , xj , xk , xl ) such that i < j < k < l, there is at most
one broken triple on xl among (xi , xj , xl ), (xi , xk , xl ) and (xj , xk , xl ).
In this way, a binary CSP can satisfy the ETP property while it contains two
broken triples among (xi , xj , xk , xl ), one on xk , and another one on xl , while
 A Microstructure-Based Family of Tractable Classes for CSPs 77

x1 x3 x1 x3
v3 v3
v1 v’3 v1 v’3

x2 v2 x2 v2
(a) (b)
Fig. 1. A non-BTP instance (a) and a BTP one (b) w.r.t. the order x1 < x2 < x3 if
one of the dashed lines occurs.

none is possible with BTP. So, ETP strictly generalizes BTP since each instance
satisfying BTP satisfies ETP while the reverse is false. So the class of instances
satisfying BTP (denoted BTP ) is strictly included in the class of instances sat-
isfying ETP (denoted ETP ) as indicated in Theorem 1 of [7] (BTP  ETP ).
As in the case of BTP, ETP allows us to define a tractable class but we need to
impose an additional property related to the level of local consistency which must
be verified. While the set of instances satisfying BTP define a tractable class, the
set of instances satisfying ETP must also satisfy Strong-Path-Consistency [8],
that is arc and path-consistency. Nevertheless, such instances have some of the
desirable properties of instances satisfying BTP, e.g. they can be solved in poly-
nomial time by usual algorithms such as MAC or RFL. In the next section, we
introduce a new property which generalizes BTP and ETP.

3 k-BTP: Definition and Properties

In this section, we introduce a new property k-BTP which generalizes previous

work along two axes. First, the property ETP is relaxed in the sense that we
allow more broken triangles than ETP when considering subsets of four variables.
But we also introduce a parameter k ≥ 2 allowing us to consider subsets of k + 1
variables, with k = 2 corresponding to BTP and k = 3 corresponding to a strict
generalization of ETP.
Definition (k-BTP). A binary CSP instance P satisfies the property k-BTP
for a given k (2 ≤ k < n) and with respect to the variable ordering < if, and
only if, for all subsets of k + 1 variables xi1 , xi2 , . . . xik+1 such that i1 < i2 <
· · · < ik−1 < ik < ik+1 , there is at least one triple of variables (xij , xij  , xik+1 )
with 1 ≤ j = j  ≤ k such that there is no broken triangle on xik+1 w.r.t. xij and
xij  . Let k-BTP be the set of the instances for which k-BTP holds w.r.t. some
variable ordering.
One can observe that 2-BTP is exactly BTP while 3-BTP includes ETP. So,
we can immediately extend Theorem 1 of [7] since BTP  ETP  3-BTP . But
above all, a more general result holds, which is an immediate consequence of the
definition of k-BTP:
78 M.C. Cooper et al.

Theorem 1. For all k ≥ 2, k-BTP  (k+1)-BTP

To analyze the tractability of k-BTP, we now show that the instances of this
class can be recognized in polynomial time:
Theorem 2. Given a binary CSP instance P = (X, D, C) and a constant k
with 2 ≤ k < n, there is a polynomial time algorithm to find a variable ordering
< such that P satisfies k-BTP w.r.t. <, or to determine that no such ordering
exists.
Proof: As in the corresponding proofs for BTP [4] and ETP [7], we define a
CSP instance Po which is consistent if and only if a possible ordering exists.
More precisely, this instance has a variable oi with domain {1, . . . , n} per vari-
able xi of X. The value of oi represents the position of the variable xi in
the ordering. We add a constraint involving {oi1 , oi2 , . . . oik , oik+1 } and impos-
ing the condition oik+1 < max(oi1 , oi2 , . . . oik ) for each k+1-tuple of variables
(xi1 , xi2 , . . . xik , xik+1 ) such that each triple of variables (xij , xij  , xik+1 ) with
1 ≤ j = j  ≤ k has at least one broken triangle on xik+1 w.r.t. xij and xij  .
If Po has a solution, then let < be any total ordering of the variables which is
a completion of the partial ordering given by the values of the variables oi . Then
for each k+1-tuple of variables (xi1 , xi2 , . . . xik , xik+1 ), with i1 < . . . < ik+1 ,
we have at least one triple of variables (xij , xij  , xik+1 ) with 1 ≤ j = j  ≤ k
which has no broken triangle on xik+1 w.r.t. xij and xij  . Indeed, if this were
not the case, then the constraint oik+1 < max(oi1 , oi2 , . . . oik ) would have been
imposed, which is in contradiction with i1 < . . . < ik+1 . So, if Po has a solution,
we have an ordering satisfying the k-BTP property. Conversely, let us consider
an ordering satisfying the k-BTP property and assume that Po has no solution.
It means that at least one constraint oik+1 < max(oi1 , oi2 , . . . oik ) is violated.
So each triple of variables (xij , xij  , xik+1 ) with 1 ≤ j = j  ≤ k has at least
one broken triangle on xik+1 , which is impossible since this ordering satisfies the
k-BTP property. Hence Po has a solution if and only if P admits an ordering
satisfying the k-BTP property.
We now prove that Po can be built and solved in polynomial time. Find-
ing all the broken triples can be achieved in O(n3 .d4 ) time, while defining the
constraints oik+1 < max(oi1 , oi2 , . . . oik ) can be performed in O(nk+1 ). So Po
can be computed in O(n3 .d4 + nk+1 ). Moreover, Po can be solved in polyno-
mial time by establishing generalized arc-consistency since its constraints are
max-closed [10]. 2
We analyze now the complexity of solving instances of the class k-BTP
(k ≥ 3). The following theorem shows that this is NP-hard since this is true
even for the smaller class ETP ⊂ 3-BTP .
Theorem 3. Deciding whether an instance of the class ETP is satisfiable is
NP-complete.
Proof: It sufficies to exhibit a polynomial reduction from binary CSP to its sub-
problem ETP . Given any binary CSP instance I, we can construct an equivalent
instance I  by
 A Microstructure-Based Family of Tractable Classes for CSPs 79

1. adding a new variable xij (with domain Dxij = Dxi ) for each constraint cij
2. adding a new equality constraint between each xi and xij
3. replacing each constraint ({xi , xj }, R) by the constraint ({xij , xj }, R).
Let < be any variable order in I  in which all the new variables xij occur after
all the original variables xk . Since each variable is constrained by at most two
variables which precede it in this order, we can easily deduce that I  satisfies
ETP. It follows from this polynomial reduction that deciding whether an instance
of the class ETP is satisfiable is NP-complete. 2
To ensure the tractability of the class k-BTP , we consider an additional
condition which is that instances satisfy Strong k-Consistency [8].
Definition (Strong k-Consistency). A binary CSP instance P satisfies i-
Consistency if any consistent assignment to i − 1 variables can be extended to
a consistent assignment on any ith variable. A binary CSP instance P satisfies
Strong k-Consistency if it satisfies i-Consistency for all i such that 1 < i ≤ k.
Strong k-Consistency and k-BTP allow us to define a new tractable class:
Theorem 4. Let P be a binary CSP instance P such that there exists a constant
k with 2 ≤ k < n for which P satisfies both Strong k-Consistency and k-BTP
w.r.t. the variable ordering <. Then P is consistent and a solution can be found
in polynomial time.
Proof: We consider an ordering for variable assignments corresponding to the
ordering <. As the instance satisfies Strong k-Consistency, it satisfies arc-
consistency and thus, no domain is empty and each value has a support in
each other domain. Moreover, as the instance satisfies Strong k-Consistency, we
have a consistent assignment on the k first variables. Now, and more generally,
suppose that we have a consistent assignment (u1 , u2 , . . . ul−1 , ul ) for the l first
variables x1 , x2 , . . . xl−1 , xl in the ordering, with k ≤ l < n. We show that this
assignment can be consistently extended to the variable xl+1 . To show this, we
must prove that ∩1≤i≤l R(cil+1 )[ui ] = ∅, that is there is at least one value in the
domain of xl+1 which is compatible with the assignment (u1 , u2 , . . . ul−1 , ul ).
We first prove this for l = k. Consider the consistent assignment
(u1 , u2 , . . . uk−1 , uk ) on the k first variables. Consider a k + 1th variable xk+1
appearing later in the ordering. Since P satisfies k-BTP , there exists at least
one triple of variables (xj , xj  , xk+1 ) with 1 ≤ j = j  ≤ k such that there is no
broken triangle on xk+1 w.r.t. xj and xj  . By Lemma 2.4 given in [4], we have:
(R(cjk+1 )[uj ] ⊆ R(cj  k+1 )[uj  ])
or
(R(cj  k+1 )[uj  ] ⊆ R(cjk+1 )[uj ])
Without loss of generality, assume that we have R(cjk+1 )[uj ] ⊆ R(cj  k+1 )[uj  ]
and j < j  . Since P satisfies Strong k-Consistency, we know that the sub-
assignment of (u1 , u2 , . . . , uj , . . . uk−1 , uk ) on k−1 variables excluding the assign-
ment uj  for xj  can be consistently extended to xk+1 . Moreover, we know that
80 M.C. Cooper et al.

R(cjk+1 )[uj ] ⊆ R(cj  k+1 )[uj  ] and by arc-consistency, R(cij ik+1 )[uj ] = ∅. Thus,
(u1 , u2 , . . . , uj , . . . , uj  , . . . , uk , uk+1 ) is a consistent assignment to the k + 1 first
variables.
Note that this proof holds for all subsets of k + 1 variables such that
xk+1 appears later in the ordering <, not only for the k + 1 first variables
x1 , x2 , . . . xk−1 , xk and xk+1 .
Now, we prove the property for l with k < l < n. That is, we show that a
consistent assignment (u1 , u2 , . . . ul−1 , ul ) can be extended to a (l +1)th variable.
As induction hypothesis, we assume that every consistent assignment on l − 1
variables can be extended to a lth variable, which appears later in the considered
ordering <.
Consider a consistent assignment (u1 , u2 , . . . ul−1 , ul ) on the l first vari-
ables. Let (ui1 , ui2 , . . . uik ) be a sub-assignment on k variables of the assign-
ment (u1 , u2 , . . . ul−1 , ul ). As P satisfies k-BTP, and as k < l < n, for all
subsets of k variables xi1 , xi2 , . . . xik , we know that there is a triangle which
is not broken in xl+1 w.r.t. xij and xij  , with xij and xij  appearing in the
variables xi1 , xi2 , . . . xik . So, without loss of generality, we can consider that
i1 ≤ ij < ij  ≤ ik ≤ l and we have R(cij l+1 )[uij ] ⊆ R(cij  l+1 )[uij  ]. Note that
xij and xij  can be interchanged in the ordering if necessary.
Now, consider the consistent assignment (u1 , u2 , . . . ul−1 , ul ) on the l
first variables. By the induction hypothesis, each partial assignment of
(u1 , u2 , . . . ul−1 , ul ) on l − 1 variables can be extended to a consistent assign-
ment on xl+1 with a compatible value ul+1 . So, consider the partial assign-
ment on l − 1 variables where uij  does not appear. This assignment is
for example (u1 , u2 , . . . uij , . . . ul−1 , ul , ul+1 ). As we have R(cij l+1 )[uij ] ⊆
R(cij  l+1 )[uij  ], the value uij  is also compatible with ul+1 , and thus the assign-
ment (u1 , u2 , . . . uij , . . . uij  , . . . ul−1 , ul , ul+1 ) on the l + 1 first variables is a con-
sistent assignment.
So, every consistent assignment (u1 , u2 , . . . ul−1 , ul ) on (x1 , x2 , . . . xl−1 , xl )
can be extended to a (l + 1)th variable, for all l with k < l < n. And more gener-
ally, we have shown that every consistent assignment on l variables, not necessar-
ily consecutive in the ordering (as are the l first variables), can be extended to a
consistent assignment for every (l+1)th variable which appears after these l vari-
ables in the ordering < associated with k-BTP. Thus, the induction hypothesis
holds for the next step.
Note that this proof also shows that an instance which satisfies Strong
k-Consistency and k-BTP (with respect to the variable ordering <) is consistent.
Finally, given the ordering <, we show that finding a solution can be per-
formed in polynomial time. Given a consistent assignment (u1 , u2 , . . . ul ) with
l < n, finding a compatible value ul+1 for the next variable xl+1 is feasible by
searching in its domain whose size is at most d. For each value, we need to ver-
ify the constraints connecting the variable xl+1 which can be done in O(el+1 )
if the next variable xl+1 has el+1 neighbors in the previous variables. Since
Σ1≤l<n el+1 = e, the total cost to find a solution is O((n + e).d). 2
 A Microstructure-Based Family of Tractable Classes for CSPs 81

In the sequel, we denote k-BTP -SkC , the class of instances satisfying k-

BTP and Strong k-Consistency. One of the most interesting properties of the
tractable class BTP is the fact that the instances of this class can be solved in
polynomial time using classical algorithms (such as MAC or RFL) implemented
in most solvers. The next property establishes that a similar result holds for
k-BTP -SkC . Indeed, the proof of Theorem 4 allows us to show that algorithms
such as BT (Backtracking), MAC and RFL can solve any instance of the class
k-BTP -SkC in polynomial time:
Theorem 5. Given a binary CSP instance P = (X, D, C) and a variable order-
ing < such that P satisfies k-BTP w.r.t. <, and is Strongly-k-Consistent, the
algorithms BT, MAC and RFL find a solution of the instance P in polynomial
time.
Proof: As the instance satisfies Strong k-Consistency, BT using the ordering <
for the variable assignment can find a consistent assignment on x1 , x2 , . . . xk−1
and xk . Moreover, given l, with k < l < n, it is shown in the proof of Theorem 4
that a consistent assignment (u1 , u2 , . . . ul−1 , ul ) on x1 , x2 , . . . xl−1 and xl can be
extended to a (l + 1)th variable, that is on xl+1 . To find the assignment of xl+1 ,
we need to look for a compatible value in its domain. This is feasible in O(el+1 .d)
assuming that xl+1 has el+1 neighbors in the previous variables. So, as for the
proof of Theorem 4, finding a solution of P is globally feasible in O((n + e).d).
If we consider now algorithms such as MAC or RFL, by the same reasoning, we
show that their complexity is bounded by O(n.(n + e).d2 ) due to the additional
cost of the arc-consistency filtering performed after each variable assignment. 2
In Section 5, we discuss the interest of the class k-BTP from a practical
viewpoint. In the next section, we study the relationships between k-BTP and
some tractable classes.

4 Relationship with Some Tractable Classes

We consider the important tractable class based on the notion of tree-
decomposition of graphs [11].
Definition (Tree-Decomposition). Given a graph G = (X, C), a tree-
decomposition of G is a pair (E, T ) with T = (I, F ) a tree and E = {Ei : i ∈ I}
a family of subsets of X, such that each subset (called cluster or bag in Graph
Theory) Ei is a node of T and satisfies:
1. ∪i∈I Ei = X,
2. for each edge {x, y} ∈ C, there exists i ∈ I with {x, y} ⊆ Ei , and
3. for all i, j, k ∈ I, if k is in a path from i to j in T , then Ei ∩ Ej ⊆ Ek .
The width of a tree-decomposition (E, T ) is equal to maxi∈I |Ei | − 1. The tree-
width w of G is the minimal width over all the tree-decompositions of G.
Let k-TW be the class of binary CSPs instances such that their tree-width
is less than or equal to a constant k. Recently, M. Vardi asked a question about
82 M.C. Cooper et al.

the relationships between k-TW and ETP or other generalizations of BTP . The
next theorems give a first partial answer to this question.
Theorem 6. k-TW  (k + 1)-BTP .
Proof: We show firstly that k-TW ⊆ (k + 1)-BTP . It is well known that if the
tree-width of a binary instance of CSP is bounded by k, there is an ordering
< on variables, such that for xi ∈ X, |{xj ∈ X : j < i and cji ∈ C}| ≤ k [2].
Now, consider a subset of k + 2 variables xi1 , xi2 , . . . xik , xik+1 , xik+2 such that
i1 < i2 < · · · < ik−1 < ik < ik+1 < ik+2 . Since the tree-width is bounded
by k, we know that there are at most k constraints cij ik+2 ∈ C. So, there is
at least one triple of variables (xij , xij  , xik+2 ) with 1 ≤ j = j  ≤ k such that
cij ik+2 ∈
/ C or cij  ik+2 ∈
/ C. Without loss of generality, assume that there is no
constraint cij ik+2 ∈ C. Thus, there is no broken triangle on xik+2 w.r.t. xij and
xij  because all the values of Dxij are compatible with all the values of Dxik+2 .
So, the considered instance of CSP satisfies the property (k + 1)-BTP . Finally,
it is easy to define instances whose tree-width is strictly greater than k which
satisfy the property (k + 1)-BTP . For example, we can consider an instance of
CSP with domains of size one, with the complete constraint graph, and with one
solution. The tree-width of this instance is n − 1 while it satisfies k-BTP for all
possible values of k. 2
The cost of checking for satisfiability of instances in k-TW has a similar
cost to that of achieving Strong (k+1)-Consistency, that is O(nk+1 dk+1 ). Nev-
ertheless, this does not allow us to establish a formal inclusion of k-TW in
(k+1)-BTP -S(k+1)C which is tractable while (k+1)-BTP is NP-complete for
k ≥ 2 by Theorem 3. But if we denote k-TW -S(k+1)C, the class of binary CSPs
instances belonging to k-TW and which satisfy Strong (k+1)-Consistency, the
next result holds:
Theorem 7. k-TW -S(k+1)C  (k + 1)-BTP -S(k+1)C.
The tractable class BTP has also recently been generalized in a different
way to that proposed in this paper, again by noticing that not all broken trian-
gles need to be forbidden [12]. We will show that these two generalizations are
orthogonal.
Definition (∀∃-BTP). A binary CSP instance P satisfies the property ∀∃-BTP
w.r.t. the variable ordering < if, and only if, for each pair of variables xi , xk such
that i < k, for all vi ∈ Dxi , ∃vk ∈ Dxk such that (vi , vk ) ∈ R(cik ) and for all xj
with j < k and j = i, and for all vj ∈ Dxj and for all vk ∈ Dxk , (vi , vj , vk , vk )
is not a broken triangle on xk w.r.t. xi and xj . Let ∀∃-BTP be the set of the
instances for which ∀∃-BTP holds w.r.t. some variable ordering.
The class ∀∃-BTP can be solved and recognized in polynomial time [12]. It
represents a tractable class which strictly includes BTP since it does not forbid
all broken triangles. Since k-BTP also does not forbid all broken triangles, it is
natural to compare these two classes. We do this for the special case k = 3, but
the same argument applies for any value of k ≥ 3.
 A Microstructure-Based Family of Tractable Classes for CSPs 83

Theorem 8. Even for sets of binary CSP instances which are strong path con-
sistent, the properties 3-BTP and ∀∃-BTP are incomparable.

Proof: Consider an instance P ∗ in which each domain Dxk contains a value a∗

such that for all other variables xi , for all values vi ∈ Dxi , (vi , a∗ ) ∈ R(cik ).
Then P ∗ satisfies ∀∃-BTP since there can be no broken triangle of the form
(vi , vj , a∗ , vk ), the value a∗ being compatible with all assignments to all other
variables. It is easy to complete such an instance P ∗ so that it does not satisfy
3-BTP for any variable ordering by adding broken triangles on domain elements
other than a∗ .
Consider a 3-variable binary CSP instance P3 with domains {0, 1, 2} and
the following three constraints: x1 = x2 , x1 = x3 , x2 = x3 , i.e. a 3-colouring
problem on a complete graph on three vertices. Then P3 is strong path consistent
and trivially satisfies 3-BTP (since there are only 3 variables), but P3 does not
satisfy ∀∃-BTP for any ordering i < j < k of the variables (due to the existence
of broken triangles on assignments (xi , a), (xj , b), (xk , a), (xk , b) for all pairs of
distinct colours a, b). 2
We now consider a very general tractable class recently discovered by
Naanaa [9] and which undoubtedly deserves to be better known.
Let E be a finite set and let {Ei }i∈I be a finite family of subsets of E. The
family {Ei }i∈I is said to be independent if and only if for all J ⊂ I,
 
Ei ⊂ Ej
i∈I j∈J

(where the notation A ⊂ B means that A is a proper subset of B). Observe that
{Ei }i∈I cannot be independent if ∃j = j  ∈ I such that Ej ⊆ Ej  , since in this
case and with J = I \ {j  } we would have
 
Ei = Ej .
i∈I j∈J

Definition (Directional Rank). Let P be a binary CSP instance whose vari-

ables are totally ordered by <. The directional rank of variable xm is the size k
of the largest consistent assignment (a1 , . . . , ak ) to a set of variables xi1 , . . . , xik
(with i1 < . . . < ik < m) such that the family of sets {R(cij m )[aj ]}j=1,...,k is
independent. The directional rank of P (w.r.t the ordering < of its variables) is
the maximum directional rank over all its variables.
Naanaa has shown that if P is a binary CSP instance which has directional
rank no greater than k and is directional strong (k + 1)-consistent then I is
globally consistent [9]. We denote DR-k, the set of these instances. Naanaa
points out that some known tractable classes, such as binary CSP instances
with connected row convex constraints [13], have bounded directional rank.
If a binary CSP instance P is (k + 1)-BTP, then no variable can have
a directional rank greater than k. This is because for any variable xm and
any assignments (a1 , . . . , ak+1 ) to any set of variables xi1 , . . . , xik+1 with
84 M.C. Cooper et al.

i1 < . . . < ik+1 < m, by the definition of (k+1)-BTP, we must have R(cij m )[aj ] ⊆
R(cij  m )[aj  ] for some j = j  ∈ {1, . . . , k + 1}; hence, as observed above, the sets
{R(cij m )[aj ]}j=1,...,k+1 cannot be independent. It follows that the tractability of
(k+1)-BTP -S(k+1)C is also a corollary of the result of Naanaa [9]. On the other
hand, the property (k + 1)-BTP, although subsumed by DR-k, can be detected in
time complexity O(nk dk + n3 d4 ) compared to O(nk+1 dk+1 ) for DR-k.

5 Experiments
In this section, we compare the tractable classes BTP , ETP -SPC , k-BTP -SkC
and DR-k-1 (where SPC stands for Strong Path Consistency) from a practical
viewpoint. We only consider the case k = 3, since strong k-consistency becomes
too expensive in time for k > 3 and may add constraints of arity k − 1.
Tractable classes are often critized for being artificial in the sense that their
underlying properties seldom occur in real instances. So, here, we first highlight
the existence of instances belonging to some of these classes among the bench-
mark instances classically exploited for solver evaluations and comparisons. More
precisely, our experiments involve 2,373 binary benchmarks from the third CSP
Solver Competition1 and cover all the benchmarks exploited in [7].
Then we will investigate the possible link between efficient solving and
belonging to these tractable classes.

5.1 Instances Belonging to Tractable Classes

Since the tractable classes ETP -SPC , 3-BTP -SPC and DR-2 require strong
path-consistency, we first achieve SPC on each instance before checking whether
it belongs to the considered classes, in the same spirit as [14,15]. In so doing, 628
instances are detected as inconsistent and so they trivially belong to all of these
tractable classes. 85 of the remaining instances belong to 3-BTP -SPC while 87
have directional rank at most two. Among these instances, we have respectively
71 and 76 instances in BTP -SPC and ETP -SPC . These differences between
these tractable classes are well highlighted by some instances of the bqwh-
15-106 family since we can observe all the possible configurations of the rela-
tions BTP -SPC  ETP -SPC  3-BTP -SPC  DR-2. For example, instance
bqwh-15-106-13 belongs to all the considered tractable classes while instances
bqwh-15-106-28, bqwh-15-106-16 and bqwh-15-106-76 only belong respectively
to three, two or one of these tractable classes. Table 1 presents some instances
belonging to classes ETP -SPC , 3-BTP -SPC or DR-2. It also provides the
tree-width w of these instances and their tree-width w once SPC is enforced.
When the exact tree-width is unknown (recall that computing an optimal tree-
decomposition is an NP-hard problem), we give a range. We can note the diver-
sity of these instances (academic, random or real-world instances). Some of these
instances belong to 3-BTP -SPC or DR-2 thanks to their structure. For instance,

1
See http://www.cril.univ-artois.fr/CPAI08.
 A Microstructure-Based Family of Tractable Classes for CSPs 85

Table 1. Some instances belonging to BTP -SPC , ETP -SPC , 3-BTP -SPC or DR-2
after the application of SPC with their tree-width w and the tree-width w of the
instances once SPC is enforced.

Instance n w w BTP -SPC ETP -SPC 3-BTP -SPC DR-2

bqwh-15-106-13 106 [7, 48] 104 yes yes yes yes
bqwh-15-106-16 106 [6, 45] 99 no no yes yes
bqwh-15-106-28 106 [7, 52] 105 no yes yes yes
bqwh-15-106-76 106 [6, 44] 100 no no no yes
bqwh-15-106-77 106 [7, 50] 100 no no yes yes
bqwh-18-141-33 141 [7, 64] 134 yes yes yes yes
bqwh-18-141-57 141 [7, 66] 137 yes yes yes yes
domino-100-100 100 2 2 yes yes yes yes
domino-5000-500 5000 2 2 yes yes yes yes
driverlogw-04c-sat 272 [19, 56] [214, 221] no no no yes
driverlogw-09-sat 650 [39, 108] 629 yes yes yes yes
fapp17-0300-10 300 [6, 153] [6, 154] yes yes yes yes
fapp18-0350-10 350 [5, 192] [12, 199] yes yes yes yes
fapp23-1800-9 1800 [6, 1325] [41, 1341] yes yes yes yes
graph12-w0 680 1 1 yes yes yes yes
graph13-w0 916 1 1 yes yes yes yes
hanoi-7 126 1 1 yes yes yes yes
langford-2-4 8 7 7 yes yes yes yes
lard-83-83 83 82 82 no no yes yes
lard-91-91 91 90 90 no no yes yes
os-taillard-4-100-0 16 [3, 9] 15 yes yes yes yes
os-taillard-4-100-9 16 [3, 9] 15 yes yes yes yes
scen5 400 [11, 32] [167, 188] no no yes yes

graph12-w0 and hanoi-7 have an acyclic constraint graph while the tree-width of
domino-100-100 and crossword-m1-uk-puzzle01 is two. However, most instances
have a tree-width greater than two. Moreover, in most cases, the application
of SPC may significantly increase the original tree-width of these instances. For
example, the tree-width of instance driverlogw-09-sat is initially bounded by 108
and is equal to 629 after the application of SPC. This increase is explained by
the pairs of values which are forbidden by SPC. When SPC forbids a pair of
values (vi , vj ) for a given pair of variables (xi , xj ), it removes (vi , vj ) from the
relation R(cij ) if the constraint cij exists. However, if the constraint cij does not
exist yet, SPC must first add it to the problem. In such a case, depending on the
added constraints and their number, the tree-width may significantly increase.
Note that the considered instances whose tree-width is initially at most two have
a tree-width unchanged by the application of SPC.

5.2 Link Between Efficient Solving and Belonging

to Tractable Classes
In this subsection, our aim is not to provide a new module based on tractable
classes in order to improve the efficiency of solvers but to see whether we
86 M.C. Cooper et al.

can exploit some tractable classes to explain the efficiency of solvers on some
instances. Indeed, we think that tractable classes are more useful from a practi-
cal viewpoint if they are implicitly handled by classical solvers than by ad-hoc
methods (as is generally the case). For instance, it is well kwown that MAC
can solve in backtrack-free manner any binary CSP whose constraint network is
acyclic without knowing that the instance has this particular feature [16].
Most state-of-the-art solvers rely on variants of MAC or RFL algorithms. In
the following, we focus our study on MAC but we have observed similar results
for RFL.
As far as solving is concerned, all the instances belonging to 3-BTP -SPC
or DR-2 are solved in a backtrack-free manner by MAC except the instance
driverlogw-04c-sat which needs one backtrack. Note that MAC has no knowledge
about the variable ordering used to satisfy 3-BTP or to obtain a directional rank
of at most two. In most cases, we have observed that the ordering CSP instance
built in the proof of Theorem 2 in order to compute a suitable variable ordering
has no constraints. So any variable ordering is suitable. In contrast, for about
a dozen instances, this CSP has several constraints but remains clearly under-
constrained and the constraint network has several connected components. This
ensues that the ordering CSP in general a huge number of solutions. So it is very
likely that MAC exploits implicitly one of these suitable variable orderings. For
example, the ordering CSP for checking whether the bqwh-15-106-76 instance
(which has 106 variables) has a directional rank at most two has 65 connected
components and admits more than 33 million solutions.
Some of the instances are solved efficiently by MAC in a backtrack-free man-
ner even though they do not belong to one of the studied tractable classes. Hence,
we now consider the notion of backdoor [17] with the aim in view to provide some
explanation about this efficiency in the same spirit as [7]. A backdoor is a set of
variables defined with respect to a class such that once the backdoor variables
are assigned, the problem falls in the class. Here, we are interested in back-
doors which are discovered implicitly by MAC when it assigns some variables.
Indeed, after some assignments and the associated filtering, the remaining part
of the problem may become tractable. So we assess the number of variables which
must be assigned before MAC finds implicitly a backdoor w.r.t. one of the studied
classes. In practice, over the 50 considered instances, we observe that MAC finds
a backdoor w.r.t. BTP after having assigned more variables than for the other
considered classes. The numbers of assigned variables required to find a backdoor
respectively for ETP and 3-BTP are very close, and even equal in most cases.
By considering DR-2, we save a few variables compared to ETP and 3-BTP . For
example, MAC needs to assign at most five variables before finding a backdoor
w.r.t. to 3-BTP or DR-2 for 14 instances compared to 12 and 4 instances, respec-
tively, for ETP and BTP 2 . Of course, the resulting instances do not necessarily
satisfy strong path-consistency and so we cannot exploit directly Theorem 5 to
explain the efficiency of MAC. Nevertheless, when the instance is 3-BTP and

2
Note that these instances do not include all the instances mentioned in [7] since some
of them belong to 3-BTP -SPC and/or DR-2.
 A Microstructure-Based Family of Tractable Classes for CSPs 87

strong path-consistent after having assigned some variables, MAC may exploit
implicitly a suitable variable ordering since, as evoked above, the corresponding
ordering CSP often admits a large number of solutions. Furthermore Theorem 5
provides sufficient conditions so that MAC solves some instances in polynomial
time, but these conditions are not always necessary. For instance, MAC solves
the instances which belong to BTP in polynomial time without requiring a suit-
able variable ordering or the satisfaction of strong path-consistency. Hence, one
part of the explanation of the practical efficiency of MAC may lie in its ability
to exploit implicitly different tractable classes.

6 Conclusion
This paper introduces a novel family of tractable classes for binary CSPs, denoted
k-BTP whose tractability is associated with a given level of strong k-consistency.
It is based on a hierarchy of classes of instances with the BTP class as the
base case. While BTP is defined on subsets of 3 variables, the k-BTP class is
defined on sets of k+1 variables, while relaxing the restrictive conditions imposed
by BTP which is the class 2-BTP . We showed that k-BTP inherits some of
the desirable properties of BTP, such as polynomial solvability using standard
algorithms such as MAC. We also showed that k-BTP strictly generalizes the
class of instances whose tree-width is bounded by a constant and we analyzed
the relationships with the class based on the notion of directional rank recently
introduced by Naanaa. To assess the practical interest of the k-BTP class, an
experimental analysis is presented focusing on the particular case of 3-BTP .
This analysis shows a significant advantage of 3-BTP compared to BTP and to
CSPs of bounded tree-width.
Further research is required to determine if the condition corresponding to
strong k-consistency is actually necessary or whether a weaker condition would
suffice. Indeed, experiments showed that MAC can solve without backtracking
certain instances belonging to 3-BTP even when they do not verify the cor-
responding level of consistency. From a practical point of view, an interesting
challenge is to find the minimum (generally) required level of consistency among
different kinds of local consistencies such as PIC [18], maxRPC [19] or SAC [20].
Note that, from a theoretical point of view, we can easily deduce from Theo-
rem 3 that any local consistency that only performs domain filtering (e.g. PIC,
maxRPC, SAC) cannot be sufficient (assuming P=NP) since ETP is invariant
under domain filtering operations.
Moreover, studying a relaxation of the k-BTP condition needs to be
addressed so as to further expand the class of instances that can be solved in
polynomial time, but along different avenues to the one proposed in [9], even if
further theoretical and experimental research are clearly required to fully appre-
ciate all the consequences of Naanaa’s result. Finally, it could be interesting
to investigate a similar approach to the one introduced in [21] which provides
a novel polynomial-time reduction operation based on the merging of domain
values.
88 M.C. Cooper et al.

References
1. Rossi, F., van Beek, P., Walsh, T.: Handbook of Constraint Programming. Elsevier
(2006)
2. Dechter, R., Pearl, J.: Tree-Clustering for Constraint Networks. Artificial Intelli-
gence 38, 353–366 (1989)
3. Gottlob, G., Leone, N., Scarcello, F.: A Comparison of Structural CSP Decompo-
sition Methods. Artificial Intelligence 124, 243–282 (2000)
4. Cooper, M.C., Jeavons, P., Salamon, A.: Generalizing constraint satisfaction on
trees: hybrid tractability and variable elimination. Artificial Intelligence 174,
570–584 (2010)
5. Sabin, D., Freuder, E.: Contradicting conventional wisdom in constraint satisfac-
tion. In: Proceedings of ECAI, pp. 125–129 (1994)
6. Nadel, B.: Tree Search and Arc Consistency in Constraint-Satisfaction Algorithms.
Search in Artificial Intelligence, pp. 287–342. Springer-Verlag (1988)
7. Jégou, P., Terrioux, C.: The extendable-triple property: a new CSP tractable class
beyond BTP. In: Proceedings of AAAI, pp. 3746–3754 (2015)
8. Freuder, E.: A Sufficient Condition for Backtrack-Free Search. Journal of the ACM
29(1), 24–32 (1982)
9. Naanaa, W.: Unifying and extending hybrid tractable classes of csps. Journal of
Experimental and Theoretical Artificial Intelligence 25(4), 407–424 (2013)
10. Jeavons, P., Cooper, M.: Tractable constraints on ordered domains. Artificial Intel-
ligence 79(2), 327–339 (1995)
11. Robertson, N., Seymour, P.D.: Graph minors II: Algorithmic aspects of treewidth.
Algorithms 7, 309–322 (1986)
12. Cooper, M.C.: Beyond consistency and substitutability. In: O’Sullivan, B. (ed.) CP
2014. LNCS, vol. 8656, pp. 256–271. Springer, Heidelberg (2014)
13. Deville, Y., Barette, O., van Hentenryck, P.: Constraint satisfaction over connected
row convex constraints. Artificial Intelligence 109(1–2), 243–271 (1999)
14. El Mouelhi, A., Jégou, P., Terrioux, C.: Hidden tractable classes: from theory to
practice. In: Proceedings of ICTAI, pp. 437–445 (2014)
15. El Mouelhi, A., Jégou, P., Terrioux, C.: Hidden Tractable Classes: from Theory to
Practice. Constraints (2015)
16. Sabin, D., Freuder, E.: Understanding and Improving the MAC Algorithm. In:
Smolka, G. (ed.) CP 1997. LNCS, vol. 1330, pp. 167–181. Springer, Heidelberg
(1997)
17. Williams, R., Gomes, C.P., Selman, B.: Backdoors to typical case complexity. In:
Proceedings of IJCAI, pp. 1173–1178 (2003)
18. Freuder, E., Elfe, C.D.: Neighborhood inverse consistency preprocessing. In:
Proceedings of AAAI, pp. 202–208 (1996)
19. Debruyne, R., Bessière, C.: From restricted path consistency to max-restricted
path consistency. In: Smolka, G. (ed.) CP 1997. LNCS, vol. 1330, pp. 312–326.
Springer, Heidelberg (1997)
20. Debruyne, R., Bessière, C.: Domain Filtering Consistencies. Journal of Artificial
Intelligence Research 14, 205–230 (2001)
21. Cooper, M.C., El Mouelhi, A., Terrioux, C., Zanuttini, B.: On broken triangles.
In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 9–24. Springer, Heidelberg
(2014)
 The Unary Resource with Transition Times

Cyrille Dejemeppe, Sascha Van Cauwelaert(B) , and Pierre Schaus

UCLouvain, ICTEAM, Place Sainte Barbe 2, 1348 Louvain-la-Neuve, Belgium

{cyrille.dejemeppe,sascha.cauwelaert,pierre.schaus}@uclouvain.be

Abstract. Transition time constraints are ubiquitous in scheduling

problems. They are said to be sequence-dependent if their durations
depend on both activities between which they take place. In this context,
we propose to extend the Θ-tree and Θ-Λ-tree data structures introduced
by Vilı́m in order to strengthen the bound computation of the earliest
completion time of a set of activities, by taking into account the sequence
dependent transition time constraints. These extended structures can
be substituted seamlessly in the state-of-the-art Vilı́m’s filtering algo-
rithms for unary resource constraints (Overload Checking, Detectable
Precedences, Not-First/Not-Last and Edge-Finding algorithms) with-
out changing their O(n log(n)) time complexities. Furthermore, this new
propagation procedure is totally independent from additional constraints
or the objective function to optimize. The proposed approach is able to
reduce the number of nodes by several order of magnitudes on some
instances of the job-shop with transition times problem, without intro-
ducing too much overhead on other instances for which it is less effective.

Keywords: Scheduling · Transition times · Global constraints ·

Constraint Programming

1 Introduction
This work extends the classic unary/disjunctive resource propagation algorithms
to include propagation over sequence-dependent transition times between activ-
ities. A wide range of real-world scheduling problems from the industry involves
transition times between activities. An example is the quay crane scheduling
problem in container terminals [21] where the crane is modeled as a unary
resource and transition times represent the moves of the crane on the rail to
move from one position to another along the vessel to load/unload containers.
We introduce filtering algorithms to tighten the bounds of (non-preemptive)
activities while taking into account the transition times between them. These
filtering algorithms are extensions of the unary resource propagation algo-
rithms (Overload Checking, Detectable Precedences, Not-First/Not-Last, Edge-
Finding) introduced in [18]. All these algorithms rely on an efficient computation
of the earliest completion time (ect) of a group of activities using the so-called
Theta tree and Theta-Lambda tree data structures. We demonstrate the effi-
ciency of the filtering on job-shop with transition times problem instances.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 89–104, 2015.
DOI: 10.1007/978-3-319-23219-5 7
90 C. Dejemeppe et al.

In Section 2, we give an overview of the tackled problems and of current

state-of-the-art techniques to solve them. In Section 3, we explain the require-
ments needed to integrate transition times propagation. Section 4 explains how
to obtain lower bounds for the time spent by transitions between activities from a
set. Then, Section 5 describes how to integrate this bound to efficiently compute
the ect of a set of activities with extended Θ-tree structures. Section 6 then
explains how classic unary algorithms can consider transition times by using
the extended Θ-tree structures. Finally, we report results obtained by the new
propagation procedure in Section 7.

2 Background
In Constraint Programming (CP), a scheduling problem is modeled by associat-
ing three variables to each activity Ai : starti , endi , and durationi representing
respectively the starting time, ending time and processing time of Ai . These
variables are linked together by the following relation:

starti + durationi = endi

Depending on the considered problem, global constraints linking the activity

variables are added to the model. In this work, we are interested in the unary
resource constraint. A unary resource, sometimes referred to as a machine, is a
resource allowing only a single activity to use it at any point in time. As such,
all activities demanding the same unary resource cannot overlap in time:

∀i, j i = j : (endi ≤ startj ) ∨ (endj ≤ starti )

The unary resource can be generalized by requiring transition times between

activities. A transition time tt i,j is a minimal amount of time that must occur
between two activities Ai and Aj if Ai ≺ Aj (precedes). These transition times
are described in a matrix M in which the entry at line i and column j represents
the minimum transition time between Ai and Aj , tt i,j . We assume that transition
times respect the triangular inequality. That is, inserting an activity between two
activities always increases the time between these activities:

∀i, j, k i = j = k : tt i,j ≤ tt i,k + tt k,j

The unary resource with transition times imposes the following relation:

∀i, j : (endi + tt i,j ≤ startj ) ∨ (endj + tt j,i ≤ starti ) (1)

2.1 Related Work

As described in a recent survey [2], scheduling problems with transition times
can be classified in different categories. First the activities can be in batch
(i.e. a machine allows several activities of the same batch to be processed
simultaneously) or not. Transition times may exist between successive batches.
 The Unary Resource with Transition Times 91

A CP approach for batch problems with transition times is described in [18]. Sec-
ondly the transition times may be sequence-dependent or sequence-independent.
Transition times are said to be sequence-dependent if their durations depend on
both activities between which they occur. On the other hand, transition times
are sequence-independent if their duration only depend on the activity after
which it takes place. The problem category we study in this article is non-batch
sequence-dependent transition time problems.
Several methods have been proposed to solve such problems. Ant Colony
Optimization (ACO) approaches were used in [9] and [15] while [6], [4], [13] and
[10] propose Local Search and Genetic Algorithm based methods. [13] proposes
a propagation procedure with the Iterative Flattening Constraint-Based Local
Search technique. The existing CP approaches for solving sequence-dependent
problems are [8], [3], [20] and [11].
Focacci et al [8] introduce a propagator for job-shop problems involving
alternative resources with non-batch sequence-dependent transition times. In
this approach a successor model is used to compute lower-bounds on the total
transition time. The filtering procedures are based on a minimum assignment
algorithm (a well known lower bound for the Travelling Salesman Problem). In
this approach the total transition time is a constrained variable involved in the
objective function (the makespan).
In [3], a Travelling Salesman Problem with Time Window (TSPTW) relax-
ation is associated to each resource. The activities used by a resource are rep-
resented as vertices in a graph and edges between vertices are weighted with
corresponding transition times. The TSPTW obtained by adding time windows
to vertices from bounds of corresponding activities is then resolved. If one of the
TSPTW is found un-satisfiable, then the corresponding node of the search tree
is pruned. A similar technique is used in [5] with additional propagation.
In [20], an equivalent model of multi-resource scheduling problem is proposed
to solve sequence-dependent transition times problems. Finally, in [11], a model
with a reified constraint for transition times is associated to a specific search to
solve job-shop with sequence-dependent transition times problems.
To the best of our knowledge, the CP filtering introduced in this article is
the first one proposing to extend all the classic filtering algorithms for unary
resources (Overload Checking [7], Detectable Precedences [17], Not-First/Not-
Last [19] and Edge Finding [19]) by integrating transition times, independently of
the objective function of the problem. This filtering can be used in any problem
involving a unary resource with sequence-dependent transition times.

2.2 Unary Resource Propagators in CP

The earliest starting time of an activity Ai denoted est i , is the time before which
Ai cannot start. The latest starting time of Ai , lst i , is the time after which Ai
cannot start. The domain of starti is thus the interval [est i ; lst i ]. Similarly the
earliest completion time of Ai , ect i , is the time before which Ai cannot end and
the latest completion time of Ai , lct i , is the time after which Ai cannot end.
92 C. Dejemeppe et al.

The domain of endi is thus the interval [ect i ; lct i ]. These definitions can be
extended to a set of activity Ω. For example, est Ω is defined as follows:

est Ω = min {est j |j ∈ Ω}

The propagation procedure for the unary resource constraint introduced

in [18] contains four different propagation algorithms all running with time com-
plexity in O(n log(n)) : Overload Checking (OC), Detectable Precedences (DP),
Not-First/Not-Last (NF/NL) and Edge Finding (EF). These propagation algo-
rithms all rely on an efficient computation of the earliest completion time of a
set of activities Ω using data structures called Theta Tree and Theta-Lambda
Tree introduced in [18]. Our contribution is a tighter computation of the lower
bound ect Ω taking into account the transition times between activities.

3 Transition Times Extension Requirements

The propagation procedure we introduce in this article relies on the computation
of ect Ω , the earliest completion time of a set of activities. This value depends on
the transition times between activities inside Ω. Let ΠΩ be the set of all possible
permutations of activities in Ω. For a given permutation π ∈ ΠΩ (where π(i)
is the activity taking place at position i), we can define the total time spent by
transition times, tt π , as follows:
|Ω|−1

tt π = tt π(i),π(i+1)
i=1

A lower bound for the earliest completion time of Ω can then defined as:
 
ect NP
Ω = max

est Ω  + pΩ  + min tt π (2)
Ω ⊆Ω π∈ΠΩ 

Unfortunately, computing this value is NP-hard. Indeed, computing the optimal

permutation π ∈ Π minimizing tt π is equivalent to solving a TSP. Since embed-
ding an exponential algorithm in a propagator is generally impractical, a looser
lower bound can be used instead:

ect Ω = max

{est Ω  + pΩ  + tt(Ω  )}
Ω ⊆Ω

where tt(Ω  ) is a lower bound of the total time consumed by transition times
between activities in Ω  :
tt(Ω  ) ≤ min tt π
π∈ΠΩ 

Our goal is to keep the overall O(n log(n)) time complexity of Vilı́m’s algo-
rithms. The lower bound tt(Ω  ) must therefore be available in constant time for
a given set Ω  . Our approach to obtain constant time lower-bounds for a given
set Ω  during search is to base its computation solely on the cardinality |Ω  |.
 The Unary Resource with Transition Times 93

More precisely, for each possible subset of cardinality k ∈ {1, . . . , n}, we pre-
compute the smallest transition time permutation of size k on Ω:
 
tt(k) =  min  min tt π
{Ω ⊆Ω: |Ω |=k} π∈ΠΩ 

For each k, the lower bound computation thus requires to solve a resource con-
strained shortest path problem (also NP-hard) with a fixed number of edges k
and with a free origin and destination. The next section proposes several ways
of pre-computing efficient lower bounds tt(k) for k ∈ {1, . . . , n}. Our formula
to compute a lower bound for the earliest completion time of a set of activities
(making use of pre-computed lower-bounds of transition times) becomes:

ectΩ = max

{est Ω  + pΩ  + tt(|Ω  |)} (3)
Ω ⊆Ω

4 Lower Bound of Transitions Times

The computation of tt(k) for all k ∈ {1, . . . , n} is NP-hard. This is a constrained
shortest path problem (for k = n it amounts to solving a TSP) in a graph
where each node corresponds to an activity and directed edges between nodes
represent the transition time between corresponding activities. Although these
computations are achieved at the initialization of the constraint, we propose to
use polynomial lower bounding procedures instead. Several approaches are used
and since none of them is dominated any other one, we simply take the maximum
of the computed lower bounds.

Minimum Weight Forest. A lower bound for tt(k) is a minimal subset of edges
of size k taken from this graph. We propose to strengthen this bound by using
Kruskal’s algorithm [12] to avoid selecting edges forming a cycle. We stop this
algorithm as soon as we have collected k edges. The result is a set of edges
forming a minimum weight forest (i.e. a set of trees) with exactly k edges.

Dynamic Programming. We can build the layered graph with exactly k lay-
ers and each layer containing all the activities. Arcs are only defined between
two successive layers with the weights corresponding to the transition times. A
shortest path on this graph between the first and last layer can be obtain with
Dijkstra. By construction this shortest path will use exactly k transitions, the
relaxation being that a same activity or transition can be used several times.

Minimum Cost Flow. Another relaxation is to keep the degree constraint but
relax the fact that selected edges must form a contiguous connected path. This
relaxation reduces to solving a minimum cost flow problem of exactly k units on
the complete bipartite graph formed by the transitions.
94 C. Dejemeppe et al.

Lagrangian Relaxation. As explained in Chapter 16 of [1], the Lagrangian relax-

ation (a single Lagrangian multiplier is necessary for the exactly k transitions
constraint) combined with a sub-gradient optimization technique can easily be
applied to compute a lower bound on the constrained shortest path problem.

5 Extending the Θ-tree with Transition Times

As introduced in [17], the O(n log n) propagation algorithms for unary resource
use the so-called Θ-tree data structure. We propose to extend it in order to
integrate transition times while keeping the same time complexities for all its
operations.
A Θ-tree is a balanced binary tree in which each leaf represents an activity
from a set Θ and internal nodes gather information about the set of (leaf) activ-
ities under this node. For an internal node v, we denote by Leaves(v), the leaf
activities under v. Leaves are ordered in non-decreasing order of the est of the
activities. That is, for two activities i and j, if est i < est j , then i is represented
by a leaf node that is at the left of the leaf node representing j. This ensures the
property:
∀i ∈ Left(v), ∀j ∈ Right(v) : est i ≤ est j
where left(v) and right(v) are respectively the left and right children of v, and
Left(v) and Right(v) denote Leaves(left(v)) and Leaves(right(v)).
A node v contains precomputed values about Leaves(v): ΣPv represents the
sum of the durations of activities in Leaves(v) and ect v is the ect of Leaves(v).
More formally, the values maintained in an internal node v are defined as follows:

ΣPv = pj
j∈Leaves(v)

ect v = ect Leaves(v) = max {est Θ + pΘ }

Θ  ⊆Leaves(v)

For a given leaf l representing an activity i, the values of ΣPl and ect l are pi
and ect i , respectively. In [18] Vilı́m has shown that for a node v these values
only depends on the values defined in both its left(v) and right(v) child. The
incremental update rules introduced in [18] are:

ΣPv = ΣPleft(v) + ΣPright(v)

 
ect v = max ect right(v) , ect left(v) + ΣPright(v)

An example of a classic Θ-tree is given in Figure 1.

When transition times are considered, the ect v value computed in the internal
nodes of the Θ-tree may only be a loose lower-bound since it is only based on
the earliest start times and the processing times. We strengthen the estimation
of the earliest computation times (denoted ect ∗ ) by also considering transition
times. We add another value inside the nodes: nv is the cardinality of Leaves(v)
(nv = |Leaves(v)|). The new update rules for the internal nodes of a Θ-tree are:
 The Unary Resource with Transition Times 95

Fig. 1. Classic Θ-tree as described in [18].

ΣPv = ΣPleft(v) + ΣPright(v)

nv = nleft(v) + nright(v)
 ∗ ∗
max{ectright(v ) , ectleft(v ) + ΣPright(v) + tt(nright(v) + 1)} : v internal
ectv∗ =
ect v : v leaf

As an example, let us consider the set of four activities used in the Θ-tree
example of Figure 1. Let us assume that the associated transition times are as
defined in the matrix M of Figure 2. The lower bounds for set of activities of
different cardinality are reported next to the matrix. With the new update rules
defined above, we obtain the extended Θ-tree presented in Figure 3. Note that
the values of ect ∗ in the internal nodes are larger than the values of ect reported
in the classic Θ-tree (Figure 1).

Fig. 2. Example of transition time matrix and associated lower bounds of transition
times permutations.

Lemma 1. ect v ≤ ectv∗ ≤ ectLeaves(v

 
) = maxΘ  ⊆Leaves(v) {est Θ +pΘ + tt(|Θ |)}
 

Proof. The proof is similar to the proof of Proposition 7 in [18], by also inte-
grating the inequality tt(|Θ |) ≥ tt(|Θ | ∩ Left(v)|) + tt(|Θ | ∩ Right(v)|), which
is itself a direct consequence of the fact that tt(k) is monotonic in k.

Since the new update rules are also executed in constant time for one node, we
keep the time complexities of the initial Θ-tree structure from [18] which are at
worst O(n log(n)) for the insertion of all activities inside the tree.
96 C. Dejemeppe et al.

Fig. 3. Extended Θ-tree for transition times. The ect ∗ values reported in the internal
nodes have been computed using the update rule of the extended Θ-tree.

Extending the Θ-Λ-tree with Transition Times

The Edge-Finding (EF) algorithm requires an extension of the original Θ-tree,
called Θ-Λ-tree [18]. This extension is used to obtain an efficient EF algorithm.
In this extension, in addition to the activities included in a Θ-tree, activities
can be marked as gray nodes. Gray nodes represent activities that are not really
in the set Θ. However, they allow to easily compute ect Θ if one of the gray
activities were included in Θ. If we consider the set of gray activities Λ such that
Λ ∩ Θ = ∅, we are interested in computing the largest ect obtained by including
one activity from Λ into Θ:
ect (Θ,Λ) = max ect Θ∪{i}
i∈Λ

In addition to ΣPv , ect v , the Θ-Λ-tree structure also maintains ΣPv and ect v ,
respectively corresponding to ΣPv and ect v , if the single gray activity in the
sub-tree rooted by v maximizing ect v were included:
  
∗ ∗ ∗
ect (Θ,Λ) = max ectΘ , max ectΘ∪{i}
i∈Λ

The update rule for ΣPv remains the same as the one described in [18]. However,
following a similar reasoning as the one used for the extended Θ-tree, we add
the nv value, and update rules are modified for ect v and n v . The rules become:
 
ΣPv = max ΣPleft(v) + ΣPright(v) , ΣPleft(v) + ΣPright(v)
⎧ ∗ ⎫
⎪ ect
⎨ right(v )
, ⎪
⎬
∗ ∗
ect v = max ect left(v ) + ΣP right(v) + tt(nright(v) + 1),
⎪
⎩ ∗ ⎪
⎭
ectleft(v ) + ΣPright(v) + tt(n right(v) + 1)

nv + 1 if the subtree rooted in v contains a gray node
nv =
nv otherwise
 The Unary Resource with Transition Times 97

This extended Θ-Λ-tree allows us to efficiently observe how the ect ∗ of a set of
activities is impacted if a single activity is added to this set. This information
allows the EF algorithm to perform propagation efficiently1 . An example of Θ-
Λ-tree based on the example from Figure 3 and Figure 2 is displayed in Figure 4.

Fig. 4. Extended Θ-Λ-tree with modified update rules.

Similarly to the reasoning applied for the Θ-tree, the time complexities
remain the same as the ones for the original Θ-Λ-tree structure from [18], which
are at worst O(n log(n)).

6 Disjunctive Propagation Algorithms with Transition

Times
In [18], a propagation procedure for the unary resource constraint is defined.
This propagation procedure consists of a propagation loop including Overload
Checking (OC), Detectable Precedences (DP), Not-First/Not-Last (NF/NL) and
Edge Finding (EF) propagation algorithms. The first three rely on the Θ-tree
while the latter employs the Θ-Λ-tree. Some small modifications can be done
to these algorithms to obtain an efficient propagation procedure making use of
knowledge about transition times.
1
Finding the “responsible” activity arg maxi ect Θ∪{i} (required by EF) is done simi-
larly to [18].
98 C. Dejemeppe et al.

6.1 Extension of Classic Unary Resource Propagation Algorithms

The four mentioned propagation algorithms use a Θ-tree or a Θ-Λ-tree to com-

pute ect Θ on a set of activities Θ. OC checks if ect Θ > lct Θ . DP, NF/NL and
EF rely on a set of rules that potentially allow to update the est or lct of an
activity. They all incrementally add/remove activities to a set of activities Θ
while maintaining the value ect Θ . When a rule is triggered by the consideration
of a given activity, the est or lct of this activity can be updated according to
the current value of ect Θ .
These four propagation algorithms can be used for the propagation of the
transition time constraints. To do so, we propose to substitute in the filtering
algorithms the Θ-tree and the Θ-Λ-tree structures by their extended versions.
∗
In the presence of transition times, ect ∗ /ect is indeed a stronger bound than
ect /ect . Furthermore, the update rules can be slightly modified to obtain an even
stronger propagation. When one of these algorithms detects that an activity i is
after all activities in a set Θ, the following update rule can be applied:
∗
est i ← max {est i , ectΘ }

In addition to all the transitions between activities of Θ - already taken into

∗
account in ectΘ - there must be a transition between one activity and i (not
necessarily from Θ, as we do not know which activity will be just before i in
the final schedule). It is therefore correct to additionally consider the minimal
transition from any activity to i. The update rule becomes:
 
∗
est i ← max est i , ectΘ + min tt j,i
j =i

An analogous reasoning can be applied to the update rule of the lct of an activity.
Similarly to the fix point propagation loop proposed in [18] for the unary
resource constraint, the four extended propagation algorithms are combined to
achieve an efficient propagation on transition time constraints. This allows to
obtain a global propagation procedure instead of the conjunction of pairwise
transition constraints described by Equation 1. The approach has however the
∗
disadvantage that the computation of ectΘ integrates a lower bound. This pre-
vents having the guarantee that sufficient propagation is achieved. The loop
must thus also integrate the conjunction of pairwise transition constraints given
in Equation 1. However, experimental results provided in Section 7 exhibits that
the supplementary global constraint reasoning can provide a substantial filtering
gain.

6.2 Detectable Precedences Propagation Example

Let us consider a small example (inspired from an example of [18]) with 3 activ-
ities, A, B and C whose domains are illustrated in Figure 5. The corresponding
transition matrix and lower bounds are given in Figure 6.
 The Unary Resource with Transition Times 99

Fig. 5. Example of extended Detectable Precedences with transition times. The

extended version updates est C from 11 to 17, while the joint use of transition time
binary constraints with the original unary constraint is not able to make this deduction.

Lower Bound k=1 k=2

⎛ ⎞ Min Weight Forest 2 5
046
Dynamic Programming 2 5
M = ⎝2 0 5⎠
Min Cost Flow 2 5
430
Lagrangian Relaxation 2 5
tt(k) 2 5

Fig. 6. Transition times for activities from Figure 5

Fig. 7. Comparison of classic and extended Θ-tree on the example described in Fig-
ures 5 and 6.

From this example, the Detectable Precedences algorithm will eventually

build a Θ-tree containing activities A and B. Figures 7a and 7b respectively
show the classic and the extended Θ-trees.
As one can see, ect ∗ is larger than ect as it is not agnostic about the transition
time constraints. Furthermore, the update rule of est C also includes the minimal
transition time from any activity to C. This leads to the following update of est C :
 
∗
est C = max est C , ectΘ + min tt i,C
i =C
= max {11, 12 + 5} = 17
100 C. Dejemeppe et al.

We finally obtain an updated est C , as shown by the red bold bracket in Figure 5.
Notice that the joint use of the constraints given in Equation 1 with the original
unary constraint of [18] would not make this deduction.

7 Evaluation
To evaluate our constraint, we used the OscaR solver [14] and ran instances on
AMD Opteron processors (2.7 GHz). For each considered instance, we used the
3 following filterings for the unary constraint with transition times:

1. Binary constraints2 (φb ) given in Equation 1.

2. Binary constraints given in Equation 1 with the Unary global constraint of
[18] (φb+u ).
3. The constraint introduced in this article (φuTT ). Based on our experience,
we slightly changed the propagation loop order: Edge-finding is put in first
position.

Considered Benchmarks. We constructed instances considering transition times

from famous JobShop benchmarks. For a given benchmark B, in each instance,
we added generated transition times between activities, while ensuring that tri-
angular inequality always hold. From B, we generated new benchmarks B(a,b)
inside which the instances are expanded by transition times uniformly picked
between a% and b% of D, where D is the average duration of all activities in
the original instance.
We generated instances from the well-known Taillard’s instances3 . From each
instance, we generated 2 instances for a given pair (a, b), where the following pairs
were used: (50, 100), (50, 150), (50, 200), (100, 150), (100, 200) and (150, 200).
This allowed us to create 960 new instances4 .

Comparison of the 3 models

In order to present fair results regarding the benefits that are only provided by
our constraint, we first followed the methodology introduced in [16]. Afterwards,
we made measurements using a static search strategy, as it cannot be influenced
by the additional pruning provided by our constraint.

Potential of the Constraint. In brief, the approach presented in [16] proposes to

pre-compute a search tree using the filtering that prunes the less - the baseline

2
For efficiency reason, dedicated propagators have been implemented instead of post-
ing reified constraint.
3
Available at http://mistic.heig-vd.ch/taillard/problemes.dir/ordonnancement.dir/
ordonnancement.html.
4
Available at http://becool.info.ucl.ac.be/resources/benchmarks-unary-resource-
transition-times
 The Unary Resource with Transition Times 101

propagator - and then to replay this search tree using the different studied fil-
tering procedures. The point is to only measure the time gain provided by the
propagation, by decoupling the gain provided by the search strategy (while still
being able to use dynamic ones) from the one provided by the propagation. We
used φb as the baseline filtering, and the SetTimes (st) search strategy to con-
struct the search tree, as this strategy is recognized to provide good performances
in Scheduling. The search tree construction time was limited to 600 seconds. We
then constructed performance profiles as described in [16]. Basically, those are
cumulative distribution functions of a performance metric τ . Here, τ is the ratio
between the solution time (or number of backtracks) of a target approach (i.e.
φb+u or φuTT ) and that of the baseline (i.e. φb ). For time (similar for number of
backtracks), the function is defined as:
 
1  t(replay(st), M ∪ φi ) 

Fφi (τ ) =  M ∈ M : ≤ τ  (4)
|M| t(replay(st), M )

where M is the set of considered instances while t(replay(st), M ∪ φi ) and

t(replay(st), M ) are respectively the time required to replay the generated search
tree with the studied model (model using φi , i.e. φb+u or φuTT ) and with the
baseline model.
Figures 8a and 8b respectively provide the profiles for time and backtrack
for all the 960 instances5 . Figure 8c provides a “long-term” view of Figure 8a.
From Figure 8a, we can first conclude that φb+u is clearly worse than φuTT
and φb from a time perspective. Moreover, Figure 8b shows that φb+u rarely
offers more pruning than φb .
In comparison, we can see from Figure 8a that for ∼ 20% of the instances,
φuTT is about 10 times faster than φb , and that we solve ∼ 35% of the instances
faster (see FφuTT (1)). Moreover, it offers more pruning for ∼ 75% of the instances
(see Figure 8b).
From Figure 8c, we can see that the constraint does not have too much
overhead, as φuTT is at worst about 7.5 times slower than φb for ∼ 45% percent
of the instances (FφuTT (7.5) − FφuTT (1)). It is a bit slower for the remaining

1.00
φuTT
0.75 φb+u
% instances

0.50

0.25

0.00
0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 2.5 5.0 7.5 10.0 12.5
τ τ τ

(a) τ is a time ratio. (b) τ is a backtrack ratio. (c) “Long-term” profile (τ

is a time ratio).

Fig. 8. Performance profiles for the 960 generated instances.

5
When instances were separated by number of jobs, the profiles had similar shapes.
102 C. Dejemeppe et al.

Table 1. Best time results for φuTT compared to φb . The problem is to find the given
makespan m using a binary static search strategy. Time is in seconds.
φuTT φb φb+u
Instance m
Time #Fails Time #Fails Time #Fails
15 15-3 225 50 100-1 2,344 1.12 2,442 117.92 980,330 432.07 911,894
50 15-8 750 50 100-2 6,682 2.11 744 182.27 1,127,272 999.79 1,127,272
20 15-7 300 150 200-2 4,784 0.24 449 17.63 168,466 62.27 168,466
15 15-6 225 50 100-1 2,398 3.90 5,593 187.93 889,079 534.20 602,591
50 20-3 1000 50 150-2 7,387 2.96 1,709 126.61 584,407 829.25 584,407
100 20-4 2000 150 200-1 18,595 11.59 885 340.32 332,412 1225.44 206,470
30 15-3 450 50 200-1 4,643 1.97 1,178 39.23 226,700 314.34 226,700
15 15-5 225 100 150-2 3,320 0.91 2,048 16.40 119,657 63.38 119,657
50 20-2 1000 50 100-1 6,979 3.79 1,680 63.16 878,162 4.63 1,695
30 15-10 450 100 200-1 5,586 0.74 687 9.24 106,683 41.25 106,683

Table 2. Worst time results for φuTT compared to φb . The problem is to find the given
makespan m using a binary static search strategy. Time is in seconds.
φuTT φb φb+u
Instance m
Time #Fails Time #Fails Time #Fails
15 15-10 225 50 200-2 2,804 645.26 546,803 127.38 546,803 572.81 546,803
50 15-9 750 50 200-1 6,699 954.77 164,404 174.63 164,437 1,108.43 164,437
20 20-5 400 100 150-2 4,542 213.54 78,782 38.26 78,968 180.20 78,968
20 20-8 400 100 150-2 4,598 147.55 164,546 26.42 164,576 175.69 164,576
15 15-2 225 50 100-2 2,195 178.37 96,821 31.23 96,821 139.84 96,821
20 20-6 400 100 200-1 4,962 11.15 8,708 1.94 8,745 11.87 8,745
30 20-8 600 50 200-1 5,312 18.63 6,665 3.15 6,687 19.93 6,687
20 15-10 300 50 200-2 3,571 85.84 61,185 14.24 61,185 65.12 61,185
50 20-8 1000 100 200-1 9,186 286.61 88,340 46.17 88,340 180.23 88,340
20 15-1 300 100 150-1 3,557 189.37 208,003 29.55 209,885 157.33 209,885

∼ 20%, which roughly corresponds to the percentage of instances for which

φuTT provides no extra pruning (see FφuTT (1) in Figure 8b).

Evaluation Over a Static Search Strategy. We here present results in a more

“traditional” fashion. We compute the best makespan m that can be obtained
with φb within 600 seconds, using the following binary static search strategy:
fixed variable order, left branch assigns starti to est i , right branch removes est i
from the domain of starti . Then, the time and number of failures required by
each model to find this solution are computed. We filtered out instances for
which the solution was found by φb in less than 1 seconds and we computed the
time ratio between φuTT and φb . From this perspective, the 10 best and worst
results are reported in tables 1 and 2, respectively. On the 10 best instances, the
gains (the number of failures and time) are significant (sometimes two orders
of magnitude). On the 10 worst instances, the times obtained with φuTT are
 The Unary Resource with Transition Times 103

similar to the results using the classical unary resource (i.e. φb+u ), while they are
at worst around 6.4 times slower than the simple binary decomposition (i.e. φb ).

8 Conclusion

In this paper, we proposed to extend classic unary resource propagation algo-

rithms such that they consider transition times. We first stated that a lower
bound of the time taken by transitions between activities from a set Ω is required
to have a tighter bound of ect Ω . We described several possible methods to com-
pute these lower bounds. We then proposed to extend the Θ-tree and Θ-Λ-tree
structures to integrate these lower bounds. These extended structures can then
be used in unary propagation algorithms: OC, DP, NF/NL and EF. The new
obtained propagation procedure has the advantage that it can be used conjointly
with any other constraint and that it is completely independent from the objec-
tive to optimize. We have demonstrated that the additional pruning achieved
by this propagation can dramatically reduce the number of nodes (and thus the
time taken to solve the problem) on a wide range of instances.
Future work would analyze the possibility to integrate tighter incremental
lower bounds in Θ-tree and Θ-Λ-tree structures. The order and real usefulness
of the propagators (OC, DP, NF/NL, EF) should also be studied in order to
acquire the most efficient fixpoint propagation loop. Finally, we would like to
experiment on a new update rule in Θ-tree and Θ-Λ-tree to be able to obtain
tighter lower bounds for ect Ω .

References
1. Ahuja, R.K., Magnanti, T.L., Orlin, J.B.: Network Flows: Theory, Algorithms, and
Applications. Prentice-Hall Inc, Upper Saddle River (1993)
2. Allahverdi, A., Ng, C., Cheng, T.E., Kovalyov, M.Y.: A survey of scheduling prob-
lems with setup times or costs. European Journal of Operational Research 187(3),
985–1032 (2008)
3. Artigues, C., Belmokhtar, S., Feillet, D.: A new exact solution algorithm for the job
shop problem with sequence-dependent setup times. In: Régin, J.-C., Rueher, M.
(eds.) CPAIOR 2004. LNCS, vol. 3011, pp. 37–49. Springer, Heidelberg (2004)
4. Artigues, C., Buscaylet, F., Feillet, D.: Lower and upper bound for the job shop
scheduling problem with sequence-dependent setup times. In: Proceedings of the
Second Multidisciplinary International Conference on Scheduling: Theory and
Applications, MISTA 2005 (2005)
5. Artigues, C., Feillet, D.: A branch and bound method for the job-shop problem with
sequence-dependent setup times. Annals of Operations Research 159(1), 135–159
(2008)
6. Balas, E., Simonetti, N., Vazacopoulos, A.: Job shop scheduling with setup times,
deadlines and precedence constraints. Journal of Scheduling 11(4), 253–262 (2008)
7. Baptiste, P., Le Pape, C., Nuijten, W.: Constraint-based scheduling: applying con-
straint programming to scheduling problems, vol. 39. Springer Science & Business
Media (2001)
104 C. Dejemeppe et al.

8. Focacci, F., Laborie, P., Nuijten, W.: Solving scheduling problems with setup times
and alternative resources. In: AIPS, pp. 92–101 (2000)
9. Gagné, C., Price, W.L., Gravel, M.: Scheduling a single machine with sequence
dependent setup time using ant colony optimization. Faculté des sciences de
l’administration de l’Université Laval, Direction de la recherche (2001)
10. González, M.A., Vela, C.R., Varela, R.: A new hybrid genetic algorithm for the job
shop scheduling problem with setup times. In: ICAPS, pp. 116–123 (2008)
11. Grimes, D., Hebrard, E.: Job shop scheduling with setup times and maximal time-
lags: a simple constraint programming approach. In: Lodi, A., Milano, M., Toth, P.
(eds.) CPAIOR 2010. LNCS, vol. 6140, pp. 147–161. Springer, Heidelberg (2010)
12. Kruskal, J.B.: On the shortest spanning subtree of a graph and the traveling
salesman problem. Proceedings of the American Mathematical society 7(1), 48–50
(1956)
13. Oddi, A., Rasconi, R., Cesta, A., Smith, S.F.: Exploiting iterative flattening search
to solve job shop scheduling problems with setup times. PlanSIG2010, p. 133 (2010)
14. OscaR Team: OscaR: Scala in OR (2012). https://bitbucket.org/oscarlib/oscar
15. Tahar, D.N., Yalaoui, F., Amodeo, L., Chu, C.: An ant colony system minimizing
total tardiness for hybrid job shop scheduling problem with sequence dependent
setup times and release dates. In: Proceedings of the International Conference on
Industrial Engineering and Systems Management, pp. 469–478 (2005)
16. Van Cauwelaert, S., Lombardi, M., Schaus, P.: Understanding the potential of
propagators. In: Proceedings of the Twelfth International Conference on Integra-
tion of Artificial Intelligence and Operations Research techniques in Constraint
Programming (2015)
17. Vilı́m, P.: O (nlog n) filtering algorithms for unary resource constraint. In:
Régin, J.-C., Rueher, M. (eds.) CPAIOR 2004. LNCS, vol. 3011, pp. 335–347.
Springer, Heidelberg (2004)
18. Vilım, P.: Global constraints in scheduling. Ph.D. thesis, Charles University in
Prague, Faculty of Mathematics and Physics, Department of Theoretical Computer
Science and Mathematical Logic, KTIML MFF, Universita Karlova, Praha (2007)
19. Vilı́m, P., Barták, R., Čepek, O.: Extension of o (n log n) filtering algorithms for
the unary resource constraint to optional activities. Constraints 10(4), 403–425
(2005)
20. Wolf, A.: Constraint-based task scheduling with sequence dependent setup times,
time windows and breaks. GI Jahrestagung 154, 3205–3219 (2009)
21. Zampelli, S., Vergados, Y., Van Schaeren, R., Dullaert, W., Raa, B.: The berth
allocation and quay crane assignment problem using a CP approach. In: Schulte, C.
(ed.) CP 2013. LNCS, vol. 8124, pp. 880–896. Springer, Heidelberg (2013)
 A Global Constraint for a Tractable Class
of Temporal Optimization Problems

Alban Derrien1 , Jean-Guillaume Fages2 , Thierry Petit1,3 ,

and Charles Prud’homme1(B)
1
TASC (CNRS/INRIA), Mines Nantes, 44307 Nantes, France
{alban.derrien,charles.prudhomme}@mines-nantes.fr
2
COSLING S.A.S., 44307 Nantes, France
[email protected]
3
Foisie School of Business, WPI, Worcester, MA 01609, USA
[email protected]

Abstract. This paper is originally motivated by an application where

the objective is to generate a video summary, built using intervals
extracted from a video source. In this application, the constraints used
to select the relevant pieces of intervals are based on Allen’s algebra.
The best state-of-the-art results are obtained with a small set of ad hoc
solution techniques, each specific to one combination of the 13 Allen’s
relations. Such techniques require some expertise in Constraint Program-
ming. This is a critical issue for video specialists. In this paper, we design
a generic constraint, dedicated to a class of temporal problems that cov-
ers this case study, among others. ExistAllen takes as arguments a vector
of tasks, a set of disjoint intervals and any of the 213 combinations of
Allen’s relations. ExistAllen holds if and only if the tasks are ordered
according to their indexes and for any task at least one relation is satis-
fied, between the task and at least one interval. We design a propagator
that achieves bound-consistency in O(n + m), where n is the number of
tasks and m the number of intervals. This propagator is suited to any
combination of Allen’s relations, without any specific tuning. Therefore,
using our framework does not require a strong expertise in Constraint
Programming. The experiments, performed on real data, confirm the
relevance of our approach.

1 Introduction
The study of temporal relations between elements of a process is a very active
topic of research, with a wide range of applications: biomedical informatics [5,20],
law [17], media [2,4,8] etc. Temporal reasoning enables to analyze the content of
a document in order to infer high-level information. In particular, a summary of
a tennis match may be generated from the match recording with Artificial Intel-
ligence techniques [2,4]. The summarization requires to extract some noticeable
time intervals and annotate them with qualitative attributes using signal recog-
nition techniques. Then, the summary generation may be formulated as a Con-
straint Satisfaction Problem (CSP) where variables are the video segments to

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 105–120, 2015.
DOI: 10.1007/978-3-319-23219-5 8
106 A. Derrien et al.

be displayed whereas constraints stem from different considerations: displaying

relevant information, balancing the selected content, having nice transitions etc.
Then, the CSP may be solved quite efficiently with a Constraint-Programming
(CP) solver. At first sight, this seems an easy task.
Unfortunately, things get harder when it comes to practice. Designing a CP
model requires some expert knowledge, in order to achieve good performances
on hard problems. This is particularly true when one has to design a global
constraint that would be missing in the solver. For instance, the summariza-
tion model of [2] relied on many disjunctions that are poorly propagated by
constraint engines and thus lead to unsatisfiable performances. Therefore, the
authors collaborated with CP experts to design ad hoc global constraints, which
lead to significant speedups [4]. Alternatively, one may have used temporal logic
models, to benefit from solution techniques and solvers dedicated the tempo-
ral CSP [6,7,9,18]. As the video summarization gets more sophisticated, all
these approaches suffer from the need of specific and often intricate propaga-
tors/models. This is a critical issue for video specialists, who are rarely CP
experts. Furthermore, one may need to include in her model other features avail-
able in state-of-the-art constraint solvers, such as standard global constraints and
predefined search strategies. What is missing is a both expressive and efficient
global constraint for modeling a relevant class of problems on time intervals.
We introduce the ExistAllen constraint, defined on a vector of tasks T and
a set of disjoint intervals I, respectively of size n and m. Given a subset R
of Allen’s relations [1], ExistAllen is satisfied if and only if the two following
properties are satisfied:

1. For any task in T at least one relation in R is satisfied between this task
and at least one interval in I.
2. Tasks in T are ordered according to their indexes in the vector given as
argument, i.e., for any integer i, 1 ≤ i < n, the task Ti should end before or
at the starting time of Ti+1 .

In the context of video-summarization, tasks in T are the video segments

that compose the summary. Fixed video sequences in I are extracted from the
source according to some precise features. In this way, it is possible to constrain
the content of the summary with qualitative information.
Considering the invariability of task processing times, we introduce a bound-
consistency propagator for this constraint, suited to any of the 213 subsets of
Allen’s relations. The time complexity of the most natural algorithm for this
propagator is O(n×m). We propose an improved algorithm, running in O(n+m)
time. While ExistAllen may be used in different contexts, e.g., online schedul-
ing, this paper is motivated by video-summarization. Our experiments on the
Boukadida et al.’s application [4] demonstrate that using our generic constraint
and its linear propagator is significantly better than the models built with stan-
dard constraints of the solver, and competitive with the ad hoc global constraint
approach.
 A Global Constraint for a Tractable Class 107

Table 1. Allen’s temporal algebra relations.

Symbol Relation
T (1)
T (2)
T (1) p T (2) (T (2) pi T (1) ) T (1) precedes T (2)
T (1)
T (2)
T (1) m T (2) (T (2) mi T (1) ) T (1) meets T (2)
T (1)
T (2)
T (1) o T (2) (T (2) oi T (1) ) T (1) overlaps T (2)
T (1)
T (2)
T (1) s T (2) (T (2) si T (1) ) T (1) starts T (2)
T (1)
T (2)
T (1) d T (2) (T (2) di T (1) ) T (1) during T (2)
T (1)
(2)
T
T (1) f T (2) (T (2) f i T (1) ) T (1) finishes T (2)
T (1)
T (2)
T (1) eq T (2) T (1) equal to T (2)

2 Background

In this section we give some background and fix the notations used in this paper.

2.1 Temporal Constraint Networks

Temporal reasoning has been an important research topic for the last thirty
years. One may distinguish qualitative temporal reasoning, based on relations
between intervals and/or time points, from quantitative reasoning where dura-
tion of a given event is represented in a numerical fashion. Allen’s algebra [1]
represents qualitative temporal knowledge by interval constraint networks. An
interval constraint network is a directed graph where nodes represent intervals
and edges are labelled with disjunctions of Allen’s relations. Table 1 details those
relations. Many state-of-the-art papers deal with generic solving techniques and
tractability of temporal networks [6,7,9,18], including temporal problems with
quantified formulas [10,12]. Most of these methods make no strong restriction
on the constraint network to be solved.
A few techniques, more specialized, focus on optimization problems. The two
most related to this paper are the following. Kumar et al. [11] consider temporal
problems with “taboo” regions, minimizing the number of intersections between
108 A. Derrien et al.

tasks and a set of fixed intervals. These problems occur in scheduling applica-
tions, among others. In the context of video summarization, a recent paper [4]
proposes the idea of using global constraints involving a set of ordered intervals.
Each constraint is restricted to one specific relation in the 213 combinations of
Allen’s relations. The propagators are not described.

2.2 Constraint Programming (CP)

CP is a problem solving framework where relations between variables are stated
in the form of constraints, which together form a constraint network. Each vari-
able x has a domain D(x), whose minimum value is x and maximum value is x.
A task T in a set T is an object represented by three integer variables: sT , its
starting time, eT , its ending time, and pT , its processing time. The task should
satisfy the constraint sT +pT = eT . An interval I in a set I is a fixed task, defined
by integer values instead of variables. A propagator is an algorithm associated
with a constraint, stated on a set of variables. This propagator removes from
domains values that cannot be part of a solution to that constraint. The notion
of consistency characterizes propagator effectiveness. In this paper, we consider
bound (Z)-consistency [3]. When domains are exclusively represented by their
bounds (i.e., have no holes), bound (Z)-consistency ensures that for each variable
x, x and x can be part of a solution of the constraint.

3 The ExistAllen Constraint

This section introduces the ExistAllen constraint and its propagator. Let T =
{T1 , T2 , . . . , Tn } be a set of tasks, such that any task Ti+1 must be scheduled
at or after the end of task Ti . Similarly, we define a set of ordered Intervals
I = {I1 , I2 , . . . , Im }. From a subset R of Allen’s relations, ExistAllen ensures
that any task in T is related to at least one interval in I.
Definition 1 (ExistAllen). ExistAllen(T , R, I) ⇔
 
∀T ∈ T , T RI
R∈R I∈I

∧ (∀i, 1 ≤ i < n, sTi+1 ≥ eTi )

Example 1. When summarizing the video of a tennis match, it may be required
that each segment (task) selected to be part the summary contains an applause.
Applause intervals can be preprocessed [4]. Such requirement can then be for-
mulated with a ExistAllen constraint, where T are the selected segments, I are
the applause segments and R is set to {f i, di, eq, si}.
A symmetry can be used, where the problem is seen in a mirror: starting
variables become ending variables and the lower bounds filtering of the mirror
relation is applied to the upper bounds (e.g. starts is propagated onto upper
bounds using finishes, see Table 1). Therefore, we put the focus on the algorithms
for the lower bounds of starting/ending task variables in T . We consider here
that processing times are exclusively updated by the constraints sT + pT = eT .
 A Global Constraint for a Tractable Class 109

3.1 Basic Filtering: One Relation, One Task, One Interval

The basic filtering rule specific to each Allen’s relation can be derived from time-
point logic relations [19], in order to state lower and upper bounds of starting and
ending times of tasks. For instance, consider a task T ∈ T , an interval I ∈ I, and
the relation starts. The relation is satisfied if and only if two conditions are met:

T s I ⇔ sT = sI ∧ eT < eI

The only filtering on the lower bounds of task T induced by the two conditions of
relation starts is sT ≥ sI . On the same basis, we define in Table 2 the conditions
and filtering rules for the 13 Allen’s algebra relations between a task T and an
interval I.

Table 2. Allen’s algebra lower bound filtering on each variable of a task T .

Relation Conditions Filtering Relation Conditions Filtering

T pI eT < sI sT = sI sT ≥ sI
T si I
T pi I eI < sT sT > eI eT > eI eT > eI
T mI eT = sI eT ≥ sI sT > sI sT > sI
T dI
T mi I sT = eI sT ≥ eI eT < eI
sT < sI sT < sI
T di I
T oI eT < eI eT > eI eT > eI
eT > sI eT > sI sT > sI sT > sI
T f I
sT < eI eT = eI eT ≥ eI
T oi I sT > sI sT > sI sT < sI
T fi I
eT > eI eT > eI eT = eI eT ≥ eI
sT = sI sT ≥ sI sT = sI sT ≥ sI
T sI T eq I
eT < eI eT = eI eT ≥ eI

3.2 A First Propagator

We propose a propagator based on two procedures. Again, we only present the

algorithms adjusting lower bounds of starting times and ending times.
The main procedure, ExistAllenQuadratic (Algorithm 1), takes as argu-
ments the sets T and I and a subset R of Allen’s relations. Algorithm 1 considers
all tasks in T and checks the relations according to intervals in I. At the end of
the procedure, the bounds of variables in T are updated according to the earliest
support. If no support has been found, a domain is emptied and the solver raises
a failure exception. The order between the tasks is maintained by the procedure
propagate at line 10, whose propagation is obvious.
Algorithm 1 calls the procedure AllenAllRelation(Algorithm 2, line 6),
which performs the check for one task and one interval, with all the relations
in R. In order to define a procedure instead of a function returning a pair of
bounds, we use two global variables s∗ and e∗ , storing permanently the two
lowest adjustments, respectively for the lower bounds of the starting and ending
110 A. Derrien et al.

Require: Global Variable : s∗ , e∗

1: procedure ExistAllenQuadratic(T , R, I)
2: for i = 1 to n do  Loop over tasks
3: s∗ ← sTi + 1  Intialize to a value out of the domain
4: e∗ ← eTi + 1
5: for j = 1 to m do  Loop over Intervals
6: AllenAllRelation(Ti , R, Ij )
7: end for
8: sTi ← s∗
9: eTi ← e∗
10: if i < n − 1 then propagate(eTi , ≤, sTi+1 ) end if
11: end for
12: end procedure
Algorithm 1: Main procedure.

time of the current task. An adjustment of the bound of one such variable is
made in Algorithm 2 if and only if the current relation gives a support which is
less than the lowest previously computed support.

Require: Global Variable : s∗ , e∗

1: procedure AllenAllRelation(T, R, I)
2: for all r ∈ R do
3: if checkCondition(T, r, I) then
4: s∗ ← min(s∗ , seekSupportStart(T, r, I))
5: e∗ ← min(e∗ , seekSupportEnd(T, r, I))
6: end if
7: end for
8: end procedure
Algorithm 2: Update of s∗ and e∗ .

The function checkCondition(T, r, I) (line 3) returns true if and only if a

support can be found. Consider again the example of relation starts. From the
condition induced by Table 2, col. 2, we have:

checkCondition(T , s, I) ⇔ sT ≤ sI ∧ sT ≥ sI ∧ eT < eI .

If the conditions are met then a support exists. seekSupportStart(T , s, I)

returns the lowest support for the starting time variable, that is, sI . As no fil-
tering is directly induced for the ending time variable, the minimal support
returned, for the relation s by seekSupportEnd(T , s, I) is max(eT , sI +pT ).
For each Allen’s algebra relation, the three functions are similarly derived from
Table 2.

Lemma 1. The time complexity of Algorithm 1 is in O(n × m).

 A Global Constraint for a Tractable Class 111

Proof. As the number of Allen’s relations is constant (at most 13), and the call
of the three functions (lines 3, 4 and 5 in Algorithm 2) are in O(1), the whole
filtering of lower bounds is performed in O(n × m) time complexity.
Theorem 1. The propagator based on Algorithm 1 and its symmetric calls for
upper bounds of starting/ending variables in T , ensure bounds(Z)-consistency
if processing times are fixed integers.
Proof. After the execution of Algorithm 1, the optimal update has been done for
each lower bound, according to the current upper bounds. The filtering of upper
bounds is symmetric. Therefore, providing that durations of tasks are fixed, the
propagator ensures bound (Z)-consistency when a fixpoint is reached.
A fixpoint is not necessarily reached after running Algorithm 1 twice, once
to filter lower bounds and once to filter upper bounds. Indeed, the pass on
upper bounds can filter values which were previously supports for lower bounds.
Let’s consider ExistAllen(T , R, I) depicted in Figure 1 wherein: T = {T1 =
sT1 = [0, 2], pT1 = [2, 4], eT1 = [4,6] , T2 = sT2 = [5, 6], pT2 = [4, 5], eT2 = [10, 10] },
R = {d, di} and I = {I 1 = [1, 5], I 2 = [6, 8]}.

T1 = sT1 = [0, 2], eT1 = [4,6] T1 for T1 on di = [0, 6[

T1 for T1 on d = [2, 4[

T2 = sT2 = [5, 6], eT2 = [10, 10] T2

I 1 = [1, 5], I 2 = [6, 8] I1 I2

0 1 2 3 4 5 6 7 8 9 10

Fig. 1. Several phases may be required to get a fixpoint when the processing times of
tasks are not fixed.

We now simulate the calls of Algorithm 1 required to reach a fixpoint. No

values are filtered during the run of Algorithm 1 on lower bounds, since di
provides the minimal support for sT1 , d provides the minimal support for eT1
and di provides the minimal supports for T2 . On the run of Algorithm 1 on
upper bounds, since no relation provides support for 6 from sT2 , the value is
removed. The ordering constraint (Algorithm 1, line 10) removes 6 from eT1 .
Thus, di is no longer valid to provide a support for T1 . Consequently, a second
call to Algorithm 1 on lower bounds has to be done and the minimal value for
sT1 will then be 2, given by relation d.
However, when task processing times are constants, the minimum support of
the ending date of an activity is the minimum support of its starting date plus the
constant. Therefore, the issue mentioned in the previous example cannot occur.
The fixpoint can be reached in two passes. One may note that, in this case, we
could improve the algorithm by ordering Allen’s relations. As our target appli-
cation involves variable processing times, we do not detail this simplification.
112 A. Derrien et al.

3.3 A Linear Propagator

This section introduces an improved propagator, running in O(n + m) time
complexity.
First, one may observe that the satisfaction of the relation precedes can be
done in constant time. Indeed, if a task T can precede the last interval Im ,
the lower bounds are a support. And if not, task T cannot precede any inter-
val. The same way relation precedes inverse can be symmetrically checked with
the first intervals. Therefore, to simplify the presentation and without loss of
generality, we now consider that relations p and pi can be isolated and treated
separately. For each task, they can be checked in constant time. In this section
we exclude them from R.
Second, as the sets T and I are chronologically ordered, we can exploit
dominance properties that lead to a linear propagator. We now provide those
properties and their proof, as well as the ExistAllenLinear procedure. As in
Section 3.2, we focus on lower bounds of starting/ending variables of tasks in T .

Property 1. An interval I which starts after a support y for the ending time of
a task T cannot provide a support for eT lower than y.

Proof. Consider a task T , R ⊆ {m, mi, o, oi, s, si, d, di, f, f i, eq}, y a support for
eT and I an interval, such that y < sI . From filtering rules in Table 2, col. 3, no
relation in R provides a support lower than y for eT . 


Property 2. For any of the relations in {mi, oi, s, si, d, f, eq}, an interval I which
starts after a support y for the ending time of a task T cannot provide a support
for sT lower than y.

Proof. Similar to proof of Property 1. 



Property 3. For any of the relations in {m, o, di, f i}, an interval I which starts
after a support for the ending time of a task T can provide a support for sT . The
lowest support for T is then given by the interval I in I having the lowest sI .

Proof. Let T be a task and r be a relation, r ∈ R = {m, o, di, f i}, x be a support

for sT , y be a support for eT and I be the interval in I with the lowest sI , such
that sI > y. For each r ∈ R, we distinguish two cases.
Case 1 (T is related to I with relation r, T r I). As the support for eT is at
least sI and no rule on starting time is explicitly defined by r (Table 2, col. 3),
then the support for sT is at least sI − pT . Given that all intervals from I are
chronologically ordered, no interval greater than I can provide a lower x.

Case 2 (T is not related to I with relation r: ¬(T r I)). Consider the m relation.
Task T can meet interval I only if sI ∈ [eT , eT ] (Table 2 col. 2). As it exists a
support y with a value e∗ < sI and the relation m is not satisfied, we have
eT < sI . Given that all intervals are chronologically ordered, no interval with a
greater index can meet the task T . A similar reasoning can be applied to the
other relations in R. 

 A Global Constraint for a Tractable Class 113

Thanks to Properties 1, 2 and 3, we can improve the Algorithm 1 by stopping

the iteration over intervals in I for a given task T (Line 5) if e∗ < sI . We now
provide two properties from which, for a given task T, the iteration over intervals
in I does not have to always start at the first interval of I.

Property 4. An interval I which ends before the ending time of a task Ti cannot
provide a support for the next task Ti+1 .

Proof. Let Ti be a task and I an interval such that eI < eTi . Then eI < sTi+1 .
Ti+1 cannot be in relation with I. 


Property 5. Given a task T , there exists at most one interval between the interval
I i with the highest ending time such that eI i < eT and the interval I j with the
lowest starting time such that sI j > eT .

Proof. Let T be a task, let I i be the interval with the highest eI such that
eI < eT , we have then eI i+1 > eT , and let I j be the interval with the lowest
sI j such that sI j > eT , we have then sI j−1 < eT . Given that all intervals are
chronologically ordered, and that sI j−1 < eI i+1 , we have that j − 1 ≤ i + 1, that
is j − i ≤ 2. As the difference between indices i and j is at most 2, there is at
most one interval between i and j. 


Thanks to Properties 4 and 5, we know that the next task cannot be in

relation with any interval whose index is lower than or equal to j − 2. We can
improve the Algorithm 1 by starting the iteration over intervals for a given task
at j − 1.

Require: Global Variable : s∗ , e∗

1: procedure ExistAllenLinear(T , R, I)
2: j←0
3: for i = 1 to n do  Loop over Tasks
4: s∗ ← sTi + 1  Intialize to a value out of the domain
5: e∗ ← eTi + 1
6: repeat  Loop over Intervals
7: j ←j+1
8: AllenAllRelation(Ti , R, Ij )
9: until j < m and e∗ < sIj  Stop iteration, see Properties 1, 2 and 3
10: sTi ← s∗
11: eTi ← e∗
12: if i < n − 1 then propagate(eTi , ≤, sTi+1 ) end if
13: j ← max(0, j − 2)  Set next interval index, see Properties 4 and 5
14: end for
15: end procedure
Algorithm 3: Linear Algorithm for Main Procedure.

By construction, Algorithms 3 and 1 do exactly the same filtering.

Theorem 2. The time complexity of Algorithm 3 is in O(n + m).

114 A. Derrien et al.

Proof. The number of evaluated intervals is equal to m+2×(n−1): every time a

new task is evaluated, the algorithm goes two intervals back. The new algorithm
is then in O(n + m). 


3.4 Improvements

In this section, we describe three improvements brought to the ExistAllen con-

straint in order to improve its efficiency in practice.
First, the Algorithm 2 can be adapted to store, for a given task, the index
of the first interval which satisfied the conditions of a relation. Indeed, intervals
located before that interval do not provide a support for the task (and they will
never do in the current search sub-tree). By doing so, useless calls to check-
Condition can be avoided since they will always return false. In practice, an
operation is added after the line 7 in Algorithm 3 to put in j the maximum
between j and the first satisfying interval for the task evaluated. These indices
are automatically updated upon backtrack.
Similarly, the tasks whose variables have been modified since the last call
of the procedure have to be memorized. Thus, the for-loop (line 3-14 in Algo-
rithm 3) can start from the index of the first modified task. Moreover, following
tasks that have not been modified can be skipped safely.
Finally, our generic framework enables to replace some remarkable combina-
tions of the Allen’s algebra relations with meta-relations. By doing so, even if
the complexity of the Algorithm 3 remains the same, the number of operations
made to check conditions and seek supports for the combined relations may be
reduced. For instance, a “contains” meta-relation, as described in Example 1,
which expresses {f i, di, eq, si}, can save up to three calls of the methods in the
for-loop in Algorithm 2, lines 2-7. Note that since {p, pi} are handled in a par-
ticular way by the constraint, it is even more efficient to limit the combinations
to relations in {m, mi, o, oi, s, si, d, di, f, f i, eq}. Adding meta-relations is easy in
our implementation since we use a facade design pattern to define the 13 rela-
tions. Some meta-relations may require to define their inverse, in order to filter
on upper bounds. This is not the case for “contains”, though. Indeed, the mirror
relation of f i is si, the mirror relation of si is f i, while the mirror relation of di
is di and the mirror relation of eq is eq.

4 Evaluation

The main contribution of this work is an “expressivity gain”, which leads to

reducing the investment necessary to build and maintain a model. Nevertheless,
it is important to check if this gain does not come at the price of efficiency.
In this section, we empirically evaluate the impact of the proposed filtering
algorithm. First, we recall the video summarization problem. Second, we show
that the expressive ExistAllen constraint we introduced is very competitive with
the state-of-the-art dedicated approach.
 A Global Constraint for a Tractable Class 115

4.1 Problem Description

The video summarization problem of [4] consists in extracting audio-visual fea-

tures and computing segments from an input video. The goal is to provide a
summary of the video. More precisely, they consider tennis match records.
Several features (games, applause, speech, dominant color, etc.) are extracted
as a preprocessing step, in order to compute time intervals that describe the
video. Then, the problem is to compute segments of the video that will consti-
tute the summary. The number of segments to compute and their minimal and
maximal duration are given as parameter, as well as the summary duration. In
this case-study, the purpose is to build a tennis match summary with a duration
between four and five minutes, composed of ten segments, whose duration varies
between 10 and 120 seconds.
In order to obtain a good summary (from the end-user point of view), this
process is subject to constraints, such as:

– (1a) a segment should not cut a speech interval,

– (1b) a segment should not cut a game interval,
– (2) each selected segment must contain an applause interval,
– (3) the cardinality of the intersection between the segments and the dominant
color intervals must be at least one third of the summary,

On the assumption that an applause indicates an interesting action, the pres-

ence of applause in the summary must be maximized, i.e., the cardinality of the
intersection between the computed segments and the applause time intervals
must be as large as possible.

Table 3. Match features.

Name Total duration # Speech # Applause # Dominant color # Games

M1 2h08 571 271 1323 156
M2 1h22 332 116 101 66
M3 3h03 726 383 223 194

4.2 Benchmark

We consider the model implementation as well as a 3-instance dataset (see

Table 3) kindly provided by Boukadida et. al. [4].
The model is encoded using integer variables. A segment is represented by
three variables to indicate its start, duration and end. If constraints (1a) and (1b)
are easily ensured by forbidding values for the segment start and end variables,
most constraints have been encoded using ad hoc propagators. This holds on
constraint (2), whereas it could be handled with a single ExistAllen constraint
wherein T is the set of selected segments, R is equal to {f i, di, eq, si} and I is
the set of applause time intervals. Therefore, to evaluate the practical impact of
the linear-time ExistAllen propagator, four models are considered.
116 A. Derrien et al.

1. decomp: constraint (2) is explicitly represented by the disjunction depicted

in Section 3, Definition 1, using primitive constraints of the solver.
2. allen(n.m): constraint (2) is represented by an ExistAllen constraint, using
the quadratic propagator presented in section 3.2,
3. allen(n+m): constraint (2) is represented by an ExistAllen constraint, using
the linear propagator described in sections 3.3 and 3.4.
4. ad hoc: constraint (2) is represented with the dedicated constraints of [4].
Such model is given for reference only as neither its complexity nor its con-
sistency level are known.

100 M1

95

90

85

102
100 M2
98
objective

96
94
92
90

110
105 M3
100
95
90 ad hoc
85 allen(n+m)
80 allen(n.m)
75 decomp

1 10 100 1000
time (sec)

Fig. 2. Comparative evaluation of decomp, ad hoc, allen(n.m) and allen(n+m) on

the three matches M 1, M 2 and M 3 with a static search heuristic. The plots report
the evolution of the objective value to be maximized with respect to the cpu time in
seconds. The x-axis are in logscale.
 A Global Constraint for a Tractable Class 117

Each of the four models has been implemented in Choco-3.3.0 [16]. Each of
the instances was executed with a 15 minutes timeout, on a Macbook Pro with
8-core Intel Xeon E5 at 3Ghz running a MacOS 10.10, and Java 1.8.0 25. Each
instance was run on its own core, each with up to 4096MB of memory.
In order to compare the efficiency on the four models, we first consider a static
search heuristic: the variables representing the segment bounds are selected in a
lexicographic order and assigned to their lower bounds. In this way, we ensure
that the same search tree is explored, finding the same solutions in the same

220
190
160
130 M1
100
70
40
10

180
170
160
objective

150 M2
140
130
120

240
210
180
150
M3
120 ad hoc
90 allen(n+m)
allen(n.m)
60
decomp
30
1 10 100 1000
time (sec)

Fig. 3. Comparative evaluation of decomp, ad hoc,allen(n.m) and allen(n+m) on the

three matches M 1, M 2 and M 3 with ABS and PGLNS. The plots report the evolution
of the objective value to be maximized with respect to the cpu time in seconds. The
x-axis are in logscale.
118 A. Derrien et al.

order, and that only the running time is evaluated. The comparative evaluations
of the four models are in reported in Figure 2. Each plot is associated with
an instance and indicates the improvement of the objective function over time.
Recall that the x-axis are in logscale. The three plots are similar.
First of all, a strict reformulation (decomp) is clearly not competitive with
the models with specific constraints: decomp is always the slowest. This has to
do with the huge number of constraints and additional variables it requires to
express the ExistAllen constraint. As an example in the match M 1 where there
are 10 tasks, 271 intervals and 4 relations, each triplet T, R, I is expressed by
three new binary constraints, each of them reified by a new boolean variable.
Second, the quadratic propagator improves the filtering of a strict reformulation.
Third, as expected, the performances are even better with the linear propaga-
tor. For information purpose, the results of the ad hoc model (as used in [4])
are reported. allen(n+m), our generic constraint is shown to be the most effec-
tive model, mildly faster than ad hoc. This confirms that our generic linear
propagator, besides being expressive and flexible and offering guarantees on its
complexity and consistency level, is very efficient in practice.
The four models have also been evaluated with the configuration described
in [4], that is, using Activity-based search [13] combined with Propagation-
Guided Large Neighborhood Search [14].1 Due to the intrinsic randomness of
ABS and PGLNS, each resolution was run 30 times. Thus, the average objec-
tive values are reported in Figure 3. Recall that the x-axis are in logscale. The
dynamic strategy offered by the combination of ABS and PGLNS enables to
reduce the differences between the various models. Although to a slightly lesser
extent, the order between efficiency of the four models is preserved when apply-
ing a more aggressive search strategy heuristic.

5 Conclusion
We introduced ExistAllen, a generic constraint defined on a vector of tasks
and a set of disjoint intervals, which applies on any of the 213 combinations
of Allen’s algebra relations. This constraint is useful to tackle many problems
related to time intervals, such as the video-summarization problem [4], used as a
case study. From a technical viewpoint, we proposed a generic propagator that
achieves bound-consistency in O(n + m) worst-case time, where n is the number
of tasks and m the number of intervals, whereas the most natural implemen-
tation requires O(n × m) worst-case time. Our experiments demonstrate that
using our technique is very competitive with the best ad hoc approach, specific
to one particular combination of relations, while being much more expressive.
Future work includes the extension of this approach to several task sets,
in order to tackle problems beyond the context of video-summarization. In the
Satellite Data Download Management Problem [15], Earth observation satellites
acquire data that need to be downloaded to the ground. The download of data
is subject to temporal constraints, such as fitting in visibility windows. Using
1
The configuration for ABS and PGLNS are the default one described in [13] and [14].
 A Global Constraint for a Tractable Class 119

ExistAllen constraint, the visibility windows are the intervals, the amount of
acquired data fixes the duration of the download tasks, and the relation required
is during. The tasks can be ordered with respect to the type of acquired data
and their need to be scheduled.

Acknowledgments. The authors thank Haykel Boukadida, Sid-Ahmed Berrani and

Patrick Gros for helping us modeling the tennis match summarization problem and for
having providing us their dataset.

References
1. Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM
26(11), 832–843 (1983)
2. Berrani, S.-A., Boukadida, H., Gros, P.: Constraint satisfaction programming for
video summarization. In: Proceedings of the 2013 IEEE International Symposium
on Multimedia, ISM 2013, Washington, DC, USA, pp. 195–202. IEEE Computer
Society (2013)
3. Bessière, C.: Constraint propagation. Research report 06020. In: Rossi, F.,
van Beek, P., Walsh, T. (eds.) Handbook of Constraint Programming, LIRMM,
chapter 3. Elsevier (2006)
4. Boukadida, H., Berrani, S.-A., Gros, P.: A novel modeling for video summarization
using constraint satisfaction programming. In: Bebis, G., et al. (eds.) ISVC 2014,
Part II. LNCS, vol. 8888, pp. 208–219. Springer, Heidelberg (2014)
5. Bramsen, P., Deshp, P., Lee, Y.K., Barzilay, R.: Finding temporal order in dis-
charge summaries, pp. 81–85 (2006)
6. Choueiry, B.Y., Lin, X.: An efficient consistency algorithm for the temporal con-
straint satisfaction problem. AI Commun. 17(4), 213–221 (2004)
7. Dechter, R., Meiri, I., Pearl, J.: Temporal constraint networks. Artif. Intell.
49(1–3), 61–95 (1991)
8. Ibrahim, Z.A.A., Ferrane, I., Joly, P.: Temporal relation analysis in audiovi-
sual documents for complementary descriptive information. In: Detyniecki, M.,
Jose, J.M., Nürnberger, A., van Rijsbergen, C.J.K. (eds.) AMR 2005. LNCS,
vol. 3877, pp. 141–154. Springer, Heidelberg (2006)
9. Koubarakis, M.: From local to global consistency in temporal constraint networks.
Theor. Comput. Sci. 173(1), 89–112 (1997)
10. Koubarakis, M., Skiadopoulos, S.: Querying temporal and spatial constraint
networks in PTIME. Artif. Intell. 123(1–2), 223–263 (2000)
11. Satish Kumar, T.K., Cirillo, M., Koenig, S.: Simple temporal problems with taboo
regions. In: des Jardins, M., Littman, M.L. (eds.) Proceedings of the Twenty-
Seventh AAAI Conference on Artificial Intelligence, July 14–18, 2013, Bellevue,
Washington, USA. AAAI Press (2013)
12. Ladkin, P.B.: Satisfying first-order constraints about time intervals. In:
Shrobe, H.E., Mitchell, T.M., Smith, R.G. (eds.) Proceedings of the 7th
National Conference on Artificial Intelligence. St. Paul, MN, August 21–26, 1988,
pp. 512–517. AAAI Press/The MIT Press (1988)
13. Michel, L., Van Hentenryck, P.: Activity-based search for black-box constraint
programming solvers. In: Beldiceanu, N., Jussien, N., Pinson, É. (eds.) CPAIOR
2012. LNCS, vol. 7298, pp. 228–243. Springer, Heidelberg (2012)
120 A. Derrien et al.

14. Perron, L., Shaw, P., Furnon, V.: Propagation guided large neighborhood search.
In: Proceedings of the Principles and Practice of Constraint Programming - CP
2004, 10th International Conference, CP 2004, Toronto, Canada, pp. 468–481,
September 27–October 1, 2004
15. Pralet, C., Verfaillie, G., Maillard, A., Hebrard, E., Jozefowiez, N., Huguet, M.-J.,
Desmousceaux, T., Blanc-Paques, P., Jaubert, J.: Satellite data download man-
agement with uncertainty about the generated volumes. In: Chien, S., Do, M.B.,
Fern, A., Ruml, W. (eds.) Proceedings of the Twenty-Fourth International Con-
ference on Automated Planning and Scheduling, ICAPS 2014, Portsmouth, New
Hampshire, USA, June 21–26, 2014. AAAI (2014)
16. Prud’homme, C., Fages, J.-G., Lorca, X.: Choco3 Documentation. http://www.
choco-solver.org. TASC, INRIA Rennes, LINA CNRS UMR 6241, COSLING
S.A.S. (2014)
17. Schilder, F.: Event extraction and temporal reasoning in legal documents. In:
Schilder, F., Katz, G., Pustejovsky, J. (eds.) Annotating, Extracting and Rea-
soning about Time and Events. LNCS (LNAI), vol. 4795, pp. 59–71. Springer,
Heidelberg (2007)
18. van Beek, P., Cohen, R.: Exact and approximate reasoning about temporal rela-
tions. Computational Intelligence 6, 132–144 (1990)
19. Vilain, M., Kautz, H., van Beek, P.: Readings in qualitative reasoning about
physical systems. In: Constraint Propagation Algorithms for Temporal Reasoning:
A Revised Report, pp. 373–381. Morgan Kaufmann Publishers Inc., San Francisco
(1990)
20. Zhou, L., Hripcsak, G.: Methodological review: Temporal reasoning with medical
data-a review with emphasis on medical natural language processing. J. of Biomed-
ical Informatics 40(2), 183–202 (2007)
 Exploiting GPUs in Solving (Distributed)
Constraint Optimization Problems
with Dynamic Programming

Ferdinando Fioretto1,2(B) , Tiep Le1 , Enrico Pontelli1 , William Yeoh1 ,

and Tran Cao Son1
1
Department of Computer Science, New Mexico State University,
Las Cruces, NM, USA
{ffiorett,tile,epontell,wyeoh,tson}@cs.nmsu.edu
2
Department of Mathematics and Computer Science,
University of Udine, Udine, Italy

Abstract. This paper proposes the design and implementation of a

dynamic programming based algorithm for (distributed) constraint opti-
mization, which exploits modern massively parallel architectures, such as
those found in modern Graphical Processing Units (GPUs). The paper
studies the proposed algorithm in both centralized and distributed opti-
mization contexts. The experimental analysis, performed on unstruc-
tured and structured graphs, shows the advantages of employing GPUs,
resulting in enhanced performances and scalability.

1 Introduction

The importance of constraint optimization is outlined by the impact of its appli-

cation in a range of Constraint Optimization Problems (COPs), such as sup-
ply chain management (e.g., [15,27]) and roster scheduling (e.g., [1,8]). When
resources are distributed among a set of autonomous agents and communication
among the agents are restricted, COPs take the form of Distributed Constraint
Optimization Problems (DCOPs) [21,33]. In this context, agents coordinate their
value assignments to maximize the overall sum of resulting constraint utilities.
DCOPs are suitable to model problems that are distributed in nature, and where
a collection of agents attempts to optimize a global objective within the confines
of localized communication. They have been employed to model various dis-
tributed optimization problems, such as meeting scheduling [20,32,35], resources
allocation [13,36], and power network management problems [17].
Dynamic Programming (DP) based approaches have been adopted to solve
COPs and DCOPs. The Bucket Elimination (BE) procedure [10] iterates over
the variables of the COP, reducing the problem at each step by replacing a

This research is partially supported by the National Science Foundation under grant
number HRD-1345232. The views and conclusions contained in this document are
those of the authors and should not be interpreted as representing the official policies,
either expressed or implied, of the sponsoring organizations, agencies, or the U.S.
government.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 121–139, 2015.
DOI: 10.1007/978-3-319-23219-5 9
122 F. Fioretto et al.

variable and its related utility functions with a single new function, derived
by optimizing over the possible values of the replaced variable. The Dynamic
Programming Optimization Protocol (DPOP) [25] is one of the most efficient
DCOP solvers, and it can be seen as a distributed version of BE, where agents
exchange newly introduced utility functions via messages.
The importance of DP-based approaches arises in several optimization fields
including constraint programming [2,28]. For example, several propagators adopt
DP-based techniques to establish constraint consistency; for instance, (1) the
knapsack constraint propagator proposed by Trick applies DP techniques to
establish arc consistency on the constraint [31]; (2) the propagator for the regular
constraint establishes arc consistency using a specific digraph representation of
the DFA, which has similarities to dynamic programming [24]; (3) the context
free grammar constraint makes use of a propagator based on the CYK parser
that uses DP to enforce generalized arc consistency [26].
While DP approaches may not always be appropriate to solve (D)COPs, as
their time and space requirements may be prohibitive, they may be very effective
in problems with particular structures, such as problems where their underlying
constraint graphs have small induced widths or distributed problems where the
number of messages is crucial for performance, despite the size of the messages.
The structure exploited by DP-based approaches in constructing solutions makes
it suitable to exploit a novel class of massively parallel platforms that are based
on the Single Instruction Multiple Thread (SIMT) paradigm—where multiple
threads may concurrently operate on different data, but are all executing the
same instruction at the same time. The SIMT-based paradigm is widely used
in modern Graphical Processing Units (GPUs) for general purpose parallel com-
puting. Several libraries and programming environments (e.g., Compute Unified
Device Architecture (CUDA)) have been made available to allow programmers
to exploit the parallel computing power of GPUs.
In this paper, we propose a design and implementation of a DP-based algo-
rithm that exploits parallel computation using GPUs to solve (D)COPs. Our
proposal aims at employing GPU hardware to speed up the inference process of
DP-based methods, representing an alternative way to enhance the performance
of DP-based constraint optimization approaches. This paper makes the following
contributions: (1) We propose a novel design and implementation of a central-
ized and a distributed DP-based algorithm to solve (D)COPs, which harnesses
the computational power offered by parallel platforms based on GPUs; (2) We
enable the use of concurrent computations between CPU(s) and GPU(s), during
(D)COP resolution; and (3) We report empirical results that show significant
improvements in performance and scalability.

2 Background
2.1 Centralized Constraint Optimization Problems (COPs)
A (centralized) Constraint Optimization Problem (COP) is defined as (X, D, C)
where: X = {x1 , . . . , xn } is a set of variables; D = {D1 , . . . , Dn } is a set of
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 123

x1 x2 Utilities x3
x1 for i < j
x1 0 0 max(5+5, 8+8) = 16 1
xi x j Utilities 0 1 max(5+20, 8+3) = 25 0
0 0 5 1 0 max(20+5, 3+8) = 25 0
x2 x2 0 1 8 1 1 max(20+20, 3+3) = 40 0
1 0 20
x1 Utilities x2 x3
1 1 3
x3 0 max(5+16, 8+25) = 33 1 0
x3
1 max(20+25, 3+40) = 45 0 0
(a) (b) (c) (d)

Fig. 1. Example (D)COP (a-c) and UTIL phase computations in DPOP (d).

domains for the variables in X, where Di is the set of possible values for the
variable xi ; C is a finite set of utility functions on variables in X, with fi :
xj ∈xi Dj → R+ ∪ {−∞}, where xi ⊆ X is the set of variables relevant to fi ,
referred to as the scope of fi , and −∞ is used to denote that a given combination
of values for the variables in xi is not allowed.1 A solution is a value assignment
for a subset of variables from X that is consistent
n with their respective domains;
i.e., it is a partial function θ : X → i=1 Di such that, for each xj ∈ X, if
θ(xj ) is defined, then θ(xj ) ∈ Dj . A solution is complete if it assigns a value to
each variable in X. We will use the notation σ to denote a complete solution,
and, for a set of variables V = {xi1 , . . . , xih } ⊆ X, σV = σ(xi1 ), . . . , σ(xih ),
where i1 < · · · < ih . The goal for a COP is to find a complete solution σ ∗
that maximizes the  total problem utility expressed by its utility functions, i.e.,
σ ∗ = argmaxσ∈Σ fi ∈C fi (σxi ), where Σ is the state space, defined as the set of
all possible complete solutions.
Given a COP P , GP = (X, EC ) is the constraint graph of P , where {x, y} ∈
EC iff ∃fi ∈ C such that {x, y} ⊆ xi . Fig. 1(a) shows the constraint graph of a
simple COP with three variables, x1 , x2 , and x3 . The domain of each variable is
the set {0, 1}. Fig. 1(c) describes the utility functions of the COP.

Definition 1 (Projection). The projection of a utility function fi on a set of

variables V ⊆ xi is a new utility function fi|V : V → R+ ∪ {−∞}, such that for
each possible assignment θ ∈ xj ∈V Dj , fi|V (θ) = max fi (σxi ).
σ∈Σ,σV =θ

In other words, fi|V is constructed from the tuples of fi , removing the values of
the variable that do not appear in V and removing duplicate values by keeping
the maximum utility of the original tuples in fi .
Definition 2 (Concatenation). Let us consider two assignments θ , defined
for variables V , and θ , defined for variables W , such that for each x ∈ V ∩ W
we have that θ (x) = θ (x). Their concatenation is an assignment θ · θ defined
for V ∪ W , such as for each x ∈ V (resp. x ∈ W ) we have that θ · θ (x) = θ (x)
(resp. θ · θ (x) = θ (x)).
1
For simplicity, we assume that tuples of variables are built according to a predefined
ordering.
124 F. Fioretto et al.

Algorithm 1. BE
1 for i ← n downto 1 do
2 Bi ← {fj ∈ C | xi ∈ xj ∧ i = max{k | xk ∈ xj }}
 
3 fˆi ← π−xi fj
fj ∈Bi
4 X ← X \ {xi }
5 C ← (C ∪ {fˆi }) \ Bi

We define two operations on utility functions:

• The aggregation of two functions fi and fj , is a function fi + fj : xi ∪ xj →
R+ ∪ {−∞}, such that ∀θ ∈ xk ∈xi Dk and ∀θ ∈ xk ∈xj Dk , if θ · θ is
defined, then we have that (fi + fj )(θ · θ ) = fi (θ ) + fj (θ ).
• Projecting out a variable xj ∈ xi from a function fi , denoted as π−xj (fi ),
produces a new function with scope xi \ {xj }, and defined as the projection
of fi on xi \ {xj }, i.e., π−xj (fi ) = fi|xi {xj } .
Bucket Elimination (BE): BE [10,11] is a dynamic programming based pro-
cedure that can be used to solve COPs. Algorithm 1 illustrates its pseudocode.
Given a COP (X, D, C) and an ordering o = x1 , . . . , xn  on the variables in X,
we say that a variable xi has a higher priority with respect to variable xj if xi
appears after xj in o. BE operates from the highest to lowest priority variable.
When operating on variable xi , it creates a bucket Bi , which is the set of all
utility functions that involve xi as the highest priority variable in their scope
(line 2). The algorithm then computes a new utility function fˆi by aggregating
the functions in Bi and projecting out xi (line 3). Thus, xi can be removed from
the set of variables X to be processed (line 4) and the new function fˆi replaces
in C all the utility functions that appear in Bi (line 5). In our example, BE
operates, in order, on the variables x3 , x2 , and x1 . When x3 is processed, the
bucket B3 is {f13 , f23 }, and the fˆ3 utility function is shown in Fig. 1(d) top.
The rightmost column shows the values for x3 after its projection. BE updates
the sets X = {x1 , x2 } and C = {f12 , fˆ3 }. When x2 is processed, B2 = {f12 , fˆ3 }
and fˆ2 is shown in Fig. 1(d) bottom. Thus, X = {x1 } and C = {fˆ2 }. Lastly, the
algorithm processes x1 , sets B1 = {fˆ2 }, and fˆ1 contains one value combination
σ ∗ = 1, 0, 0, which corresponds to an optimal solution to the problem.
The complexity of the algorithm is bounded by the time needed to process
a bucket (line 3), which is exponential in number of variables in the bucket.

2.2 Distributed Constraint Optimization Problems (DCOPs)

In a Distributed Constraint Optimization Problem (DCOP) [21,25,33], the vari-

ables, domains, and utility functions of a COP are distributed among a collection
of agents. A DCOP is defined as (X, D, C, A, α), where X, D, and C are defined
as in a COP, A = {a1 , . . . , ap } is a set of agents, and α : X → A maps each
variable to one agent. Following common conventions, we restrict our attention
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 125

to binary utility functions and assume that α is a bijection: Each agent con-
trols exactly one variable. Thus, we will use the terms “variable” and “agent”
interchangeably and assume that α(xi ) = ai . This is a common assumption in
the DCOP literature as there exist pre-processing techniques that transform a
general DCOP into this more restrictive DCOP [7,34]. In DCOPs, solutions are
defined as for COPs, and many solution approaches emulate those proposed in
the COP literature. For example, ADOPT [21] is a distributed version of Itera-
tive Deepening Depth First Search, and DPOP [25] is a distributed version of BE.
The main difference is in the way the information is shared among agents. Typi-
cally, a DCOP agent knows exclusively its domain and the functions involving its
variable. It can communicate exclusively with its neighbors (i.e., agents directly
connected to it in the constraint graph2 ), and the exchange of information takes
the form of messages. Given a DCOP P , a DFS pseudo-tree arrangement for GP
is a spanning tree T = X, ET  of GP such that if fi ∈ C and {x, y} = xi , then x
and y appear in the same branch of T . Edges of GP that are in (resp. out of)
ET are called tree edges (resp. backedges). The tree edges connect parent-child
nodes, while backedges connect a node with its pseudo-parents and its pseudo-
children. We use N (ai ) = {aj ∈ A|{xi , xj } ∈ ET } to denote the neighbors of agent
ai ; Ci , P Ci , Pi , and PPi to denote the set of children, pseudo-children, parent,
and pseudo-parents of agent ai ; and sep(ai ) to denote the separator of agent
ai , which is the set of ancestor agents that are constrained (i.e., they are linked
in GP ) with agent ai or with one of its descendant agents in the pseudo-tree.
Fig. 1(b) shows one possible pseudo-tree for the problem, where the agent a1
has one pseudo-child a3 (the dotted line is a backedge).
Dynamic Programming Optimization Protocol (DPOP): DPOP [25] is a
dynamic programming based DCOP algorithm that is composed of three phases.
(1) Pseudo-tree generation: Agents coordinate to build a pseudo-tree, realized
through existing distributed pseudo-tree construction algorithms [16]. (2) UTIL
propagation: Each agent, starting from the leaves of the pseudo-tree, computes
the optimal sum of utilities in its subtree for each value combination of variables
in its separator. The agent does so by aggregating the utilities of its functions
with the variables in its separator and the utilities in the UTIL messages received
from its child agents, and then projecting out its own variable. In our example
problem, agent a3 computes the optimal utility for each value combination of
variables x1 and x2 (Fig. 1(d) top), and sends the utilities to its parent agent
a2 in a UTIL message. When the root agent a1 receives the UTIL message from
each of its children, it computes the maximum utility of the entire problem.
(3) VALUE propagation: Each agent, starting from the root of the pseudo-
tree, determines the optimal value for its variable. The root agent does so by
choosing the value of its variable from its UTIL computations—selecting the
value with the maximal utility. It sends the selected value to its children in a
VALUE message. Each agent, upon receiving a VALUE message, determines
the value for its variable that results in the maximum utility given the variable

2
The constraint graph of a DCOP is equivalent to that of the corresponding COP.
126 F. Fioretto et al.

assignments (of the agents in its separator) indicated in the VALUE message.
Such assignment is further propagated to the children via VALUE messages.
The complexity of DPOP is dominated by the UTIL propagation phase, which
is exponential in the size of the largest separator set sep(ai ) for all ai ∈ A. The
other two phases require a polynomial number of linear size messages, and the
complexity of the local operations is at most linear in the size of the domain.
Observe that the UTIL propagation phase of DPOP emulates the BE process
in a distributed context [6]. Given a pseudo-tree and its preorder listing o, the
UTIL message generated by each DPOP agent ai is equivalent to the aggregated
and projected function fˆi in BE when xi is processed according to the ordering o.

2.3 Graphical Processing Units (GPUs)

Modern GPUs are multiprocessor devices, offering hundreds of computing cores
and a rich memory hierarchy to support graphical processing. We consider the
NVIDIA CUDA programming model [29], which enables the use of the mul-
tiple cores of a graphics card to accelerate general (non-graphical) applica-
tions. The underlying model of parallelism is Single-Instruction Multiple-Thread
(SIMT), where the same instruction is executed by different threads that run
on identical cores, grouped in Streaming Multiprocessors (SMs), while data and
operands may differ from thread to thread.
A typical CUDA program is a C/C++ program. The functions in the pro-
gram are distinguished based on whether they are meant for execution on the
CPU (referred to as the host) or in parallel on the GPU (referred as the device).
The functions executed on the device are called kernels, and are executed by sev-
eral threads. To facilitate the mapping of the threads to the data structures being
processed, threads are grouped in blocks, and have access to several memory lev-
els, each with different properties in terms of speed, organization, and capacity.
CUDA maps blocks (coarse-grain parallelism) to the SMs for execution. Each
SM schedules the threads in a block (fine-grain parallelism) on its computing
cores in chunks of 32 threads (warps) at a time. Threads in a block can com-
municate by reading and writing a common area of memory (shared memory).
Communication between blocks and communication between the blocks and the
host is realized through a large slow global memory. The development of CUDA
programs that efficiently exploit SIMT parallelism is a challenging task. Several
factors are critical in gaining performance. Memory levels have significantly dif-
ferent sizes (e.g., registers are in the order of dozens per thread, shared memory is
in the order of a few kilobytes per block) and access times, and various optimiza-
tion techniques are available (e.g., coalesced of memory accesses to contiguous
locations into a single memory transaction).

3 GPU-Based (Distributed) Bucket Elimination

(GPU-(D)BE)
Our GPU-based (Distributed) Bucket Elimination framework, extends BE (resp.
DPOP) by exploiting GPU parallelism within the aggregation and projection
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 127

Algorithm 2. GPU-(D)BE
(1) Generate pseudo-tree
2 GPU-Initialize( )
3 if Ci = ∅ then
4 UTILxi ⇔ ParallelCalcUtils( )
(5) Send UTIL message (xi , UTILxi ) to Pi
6 else
7 Activate UTILMessageHandler(·)
(8) Activate VALUEMessageHandler(·)

operations. These operations are responsible for the creation of the functions fˆi
in BE (line 3 of Algorithm 1) and the UTIL tables in DPOP (UTIL propagation
phase), and they dominate the complexity of the algorithms. Thus, we focus on
the details of the design and the implementation relevant to such operations.
Due to the equivalence of BE and DPOP, we will refer to the UTIL tables and
to the aggregated and projected functions fˆ of Algorithm 1, as well as variables
and agents, interchangeably. Notice that the computation of the utility for each
value combination in a UTIL table is independent of the computation in the
other combinations. The use of a GPU architecture allows us to exploit such
independence, by concurrently exploring several combinations of the UTIL table,
computed by the aggregation operator, as well as concurrently projecting out
variables.
Algorithm 2 illustrates the pseudocode, where we use the following notations:
Line numbers in parenthesis denote those instructions required exclusively in the
distributed case. Starred line numbers denote those instructions executed concur-
rently by both the CPU and the GPU. The symbols ← and ⇔ denote sequential
and parallel (multiple GPU-threads) operations, respectively. If a parallel oper-
D←H H←D
ation requires a copy from host (device) to device (host), we write ⇔ ( ⇔ ).
Host to device (resp. device to host) memory transfers are performed immedi-
ately before (resp. after) the execution of the GPU kernel. Algorithm 2 shows
the pseudocode of GPU-(D)BE for an agent ai . Like DPOP, also GPU-(D)BE
is composed of three phases; the first and third phase are executed exclusively
in the distributed version. The first phase is identical to that of DPOP (line 1).
In the second phase:
• Each agent ai calls GPU-Initialize() to set up the GPU kernel. For example,
it determines the amount of global memory to be assigned to each UTIL table
and initializes the data structures on the GPU device memory (line 2).
• Each agent ai aggregates the utilities for the functions between its variables
and its separator, projects its variable out (line 4), and sends them to its
parent (line 5). The MessageHandlers of lines 7 and 8 are activated for each
new incoming message.
By the end of the second phase (line 11), the root agent knows the overall utility
for each values of its variable xi . It chooses the value that results in the maximum
128 F. Fioretto et al.

Procedure UTILMessageHandler(ak , UTILak )

(9) Store UTILak
10 if received UTIL message from each child ac ∈ Ci then
11 UTILai ⇔ ParallelCalcUtils( )
12 if Pi = NULL then
13 d∗i ← ChooseBestValue(∅)
(14) foreach ac ∈ Ci do
(15) VALUEai ← (xi , d∗i )
(16) Send VALUE message (ai , VALUEai ) to ac
(17) else Send UTIL message (ai , UTILai ) to Pi

Procedure VALUEMessageHandler(ak , VALUEak )

(18) VALUEai ← VALUEak
(19) d∗i ← ChooseBestValue(VALUEai )
(20) foreach ac ∈ Ci do
(21) VALUEai ← {(xi , d∗i )} ∪ {(xk , d∗k ) ∈ VALUEak | xk ∈ sep(ac )}
(22) Send VALUE message (ai , VALUEai ) to ac

utility (line 13). Then, in the distributed version, it starts the third phase by
sending to each child agent ac the value of its variable xi (lines 14-16). These
operations are repeated by every agent receiving a VALUE message (lines 18-22).
In contrast, in the centralized version, the value assignment for each variable is
set by the root agent directly.

3.1 GPU Data Structures

In order to fully capitalize on the parallel computational power of GPUs, the

data structures need to be designed in such a way to limit the amount of infor-
mation exchanged between the CPU host and the GPU device, and in order to
minimize the accesses to the (slow) device global memory (and ensure that they
are coalesced). To do so, each agent identifies the set of relevant static entities,
i.e., information required during the GPU computation, which does not mutate
during the resolution process. The static entities are communicated to the GPU
once at the beginning of the computation. This allows each agent running on
a GPU device to communicate with the CPU host exclusively to exchange the
results of the aggregation and projection processes. The complete set of utility
functions, the constraint graph, and the agents ordering, all fall in such category.
Thus, each agent ai stores:
• The set of utility functions involving exclusively xi and a variable in ai ’s
separator set: Si = {fj ∈ C | xi ∈ xj ∧ sep(ai ) ∩ xj = ∅}. For a given function
fj ∈ Si , its utility values are stored in an array named gFunc j .
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 129

• The domain Di of its variable (for simplicity assumed to be all of equal car-
dinality).
• The set Ci of ai ’s children.
• The separator sets sep(ai ), and sep(ac ), for each ac ∈ Ci .
The GPU-Initialize() procedure of line 2, invoked after the pseudo-tree
construction, stores the data structures above for each agent on the GPU device.
As a technical detail, all the data stored on the GPU global memory is organized
in mono-dimensional arrays, so as to facilitate coalesced memory accesses. In
particular, the identifier and scope of the functions in Si as well as identifiers and
separator sets of child agents in Ci are stored within a single mono-dimensional
array. The utility values stored in the rows of each function are padded to ensures
that a row is aligned to a memory word—thus minimizing the number of memory
accesses.
GPU-Initialize() is also responsible for reserving a portion of the GPU
global memory to store the values for the agent’s UTIL table, denoted by gUtils i ,
and those of its children, denoted by gChUtils c , for each ac ∈ Ci . As a techni-
cal note, an agent’s UTIL table is mapped onto the GPU device to store only
the utility values, not the associated variables values. Its j-th entry is associated
with the j-th permutation of the variable values in sep(ai ), in lexicographic order.
This strategy allows us to employ a simple perfect hashing to efficiently associate
row numbers with variables’ values and vice versa. Note that the agent’s UTIL
table size grows exponentially with the size of its separator set; more precisely,
after projecting out xi , it has |Di |sep(ai ) entries. However, the GPU global mem-
ory is typically limited to a few GB (e.g., in our experiments it is 2GB). Thus,
each agent, after allocating its static entities, checks if it has enough space to
allocate its children’s UTIL tables and a consistent portion (see next subsection
for details) of its own UTIL table. In this case, it sets the project on device flag
to true, which signals that both aggregate and project operations can be done
on the GPU device.3 Otherwise it sets the flag to false and bounds the device
UTIL size table to the maximum storable space on the device. In this case, the
aggregation operations are performed only partially on the GPU device.

3.2 Parallel Aggregate and Project Operations

The ParallelCalcUtils procedure (executed in lines 4 and 11) is responsible
for performing the aggregation and projection operations, harnessing the paral-
lelism provided by the GPU. Due to the possible large size of the UTIL tables,
we need to separate two possible cases and devise specific solutions accordingly:
(a) When the device global memory is sufficiently large to store all ai ’s children
UTIL tables as well as a significant portion of ai ’s UTIL table4 (i.e., when
3
If the UTIL table of agent ai does not fit in the global memory, we partition such
table in smaller chunks, and iteratively execute the GPU kernel until all rows of the
table are processed.
4
In our experiments, we require that at least 1/10 of the UTIL table can be stored
in the GPU. We experimentally observed that a partitioning of the table in at most
130 F. Fioretto et al.

Procedure ParallelCalcUtils( )
23 if project on device then
D←H
24 gChUTILac ⇔ UTILac for all ac ∈ Ci
25 R ← 0 ; U T ILai ← ∅
26 while R < |Di |sep(ai ) do
27 if project on device then
H←D
28* UTILai ⇔ GPU-Aggregate-Project(R)
29 else
H←D
30* UTILai ⇔ GPU-Aggregate(R)
31* UTILai ← AggregateCh-Project(ai , UTILai , UTILac ) for all
ac ∈ Ci
32* UTILai ← U T ILai ∪ Compress(UTILai )
33 R ← R + |UTILai |
34 return UTILai

project on device = true), both aggregation and projection of the agent’s

UTIL table are performed in parallel on the GPU. The procedure first stores
the UTIL tables received from the children of ai into their assigned locations
in the GPU global memory (lines 23-24). It then iterates through successive
GPU kernel calls (line 28) until the UTILai table is fully computed (lines 26-
33). Each iterations computes a certain number of rows of the UTILai table
(R serves as counter).
(b) When the device global memory is insufficiently large to store all ai ’s chil-
dren UTIL tables as well as a significant portion of ai ’s UTIL table (i.e.,
when project on device = false), the agent alternates the use of the GPU
and the CPU to compute UTILai . The GPU is in charge of aggregating the
functions in Si (line 30), while the CPU aggregates the children UTIL table,5
projecting out xi . Note that, in this case, the UTILai storage must include
all combinations of values for the variables in sep(xi ) ∪ {xi }, thus the pro-
jection operation is performed on the CPU host. As in the previous case,
the UTILai is computed incrementally, given the amount of available GPU
global memory.
To fully capitalize on the use of the GPU, we exploit an additional level of
parallelism, achieved by running GPU kernels and CPU computations concur-
rently; this is possible when the UTILai table is computed in multiple chunks.
Fig. 2 illustrates the concurrent computations between the CPU and GPU. After
transferring the children UTIL tables into the device memory (Init)—in case
10 chunks provides a good time balance between memory transfers and actual com-
putation.
5
The CPU aggregates only those child UTIL table that could not fit in the GPU
memory. Those that fit in memory are integrated through the GPU computation as
done in the previous point.
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 131

(Init) Compress U 1 Compress U 2

CPU
(Host) Copy Execute K 1 Copy Execute K 2 Copy

GPU
(Device)
H D Compute U 1 D H Compute U 2 D H

Update Global Mem. Update Global Mem.

Fig. 2. Concurrent computation between host and device.

(a) only—the execution of kernel K1 produces the update of the first chunk of
UTILai , denoted by U1 in Fig. 2, which is transferred to the CPU host. The
successive parallel operations are performed asynchronously with respect to the
GPU, that is, the execution of the j-th CUDA kernel Kj (j > 1), returns the con-
trol immediately to the CPU, which concurrently operates a compression opera-
tion on the previously computed UTILai chunk (line 32), referred to as Uk−1 in
Fig. 2. For case (b), the CPU also executes concurrently the AggregateCh-
Project of line 31. We highlight the concurrent operations by marking with a
∗
symbol their respective lines in the procedure ParallelCalcUtils.
Technical Details: We now describe in more detail how we divide the workload
among parallel blocks, i.e., the mapping between the UTIL table rows and the
CUDA blocks. A total of T = 64 · k (1 ≤ k ≤ 16) threads (a block) are associ-
ated to the computation of T permutations of values for sep(ai ). The value k
depends on the architecture and it is chosen to maximize the number of con-
current threads running at the same time. In our experiments, we set k = 3.
The number of blocks is chosen so that the corresponding aggregate number
of threads does not exceed the total number of UTILai permutations currently
stored in the device. Let h be the number of stream multiprocessors of the GPU.
Then, the maximum number of UTIL permutations that can be computed con-
currently is M = h · T . In our experiments h = 14, and thus, M = 2688. Fig. 3
provides an illustration of the UTIL permutations computed in parallel on GPU.
The blocks Bi in each row are executed in parallel on different SMs. Within each
block, a total of (at most) 192 threads operate on as many entries of the UTIL
table.
The GPU kernel procedure is shown in lines 35-49. We surround line numbers
with | · | to denote parts of the procedure executed by case (b). The kernel takes
as input the number R of the UTIL table permutations computed during the
previous kernel calls. Each thread identifies its entry index rid within the table
chunk UTILai (line 35). It then assigns the shared memory allocated to local
arrays to store the static entities Si , Ci , and sep(ac ), for each ac ∈ Ci . In addi-
tion it reserves the space θ to store the assignments corresponding to the UTIL
permutation being computed by each thread, which is retrieved using the thread
entry index and the offset R (line 38). Decode implements a minimal perfect
hash function to convert the entry index of the UTIL table to its associated vari-
ables value permutation. Each thread aggregates the functions in Si (lines 42-44)
and the UTIL tables of ai ’s children (lines 45-47), for each element of its domain
132 F. Fioretto et al.

Fig. 3. GPU kernel parallel computations.

Procedure GPU-Aggregate-Project(R)
|35| rid ← the thread’s entry index of UTILi
|36| did ← the thread’s value index of Di
|37| |θ, Si |, Ci , sep(xc ) ← AssignSharedMem() for all xc ∈ Ci
|38| θ ← decode(R + rid )
|39| util ← −∞
40 foreach did ∈ Di do
|41| util did ← 0
|42| foreach fj ∈ Si do
|43| ρj ← encode(θxj | xi = did )
|44| util did ← util did + gFunc j [ρj ]
45 foreach ac ∈ Ci do
46 ρc ← encode(θsep(ac ) | xi = did )
47 util did ← util did + gChUtils c [ρc ]
|48| util ← max(util , util did )
|49| gUtils i [rid ] ← util

(lines 40-48). The Encode routine converts a given assignments for the variables
in the scope of a function fj (line 43), or in the separator set of child ac (line 46),
to the corresponding array index, sorted in lexicographic order. The value for
the variable xi within each input, is updated at each iteration of the for loop.
The projection operation is executed in line 48. Finally, the thread stores the
best utility in the corresponding position of the array gUtils i
The GPU-Aggregate procedure (called in line 30), is illustrated in
lines 35-49—line numbers surrounded by | · |. Each thread is in charge of
a value combination in sep(ai ) ∪ {xi }, thus, the foreach loop of lines 40-
48 is operated in parallel by |Di | threads. Lines 45-47 are not executed. The
AggregateCh-Project procedure (line 31), which operates on the CPU, is
similar to the GPU-Aggregate-Project procedure, except that lines 36-37,
and 42-44, are not executed.
The proposed kernel has been the result of several investigations. We exper-
imented with other levels of parallelism, e.g., by unrolling the for-loops among
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 133

groups of threads. However, these modifications create divergent branches, which

degrade the parallel performance. We experimentally observed that such degra-
dation worsen consistently as the size of the domain increases.

3.3 General Observations

Observation 1. GPU-DBE requires the same number of messages as those

required by DPOP, and it requires messages of the same size as those required
by DPOP.
Observation 2. The UTIL messages constructed by each GPU-DBE agent are
identical to those constructed by each corresponding DPOP agent.
The above observations follow from the pseudo-tree construction and VALUE
propagation GPU-DBE phases, which are identical to those of DPOP. Thus, their
corresponding messages and message sizes are identical in both algorithms. More-
over, given a pseudo-tree, each DPOP/GPU-DBE agent computes the UTIL
table containing each combination of values for the variables in its separator set.
Thus, the UTIL messages of GPU-DBE and DPOP are identical.
Observation 3. The memory requirements of GPU-(D)BE is, in the worst case,
exponential in the induced width of the problem (for each agent).
This observation follows from the equivalence of the UTIL propagation phase of
DPOP and BE [6] and from Observation 2.
Observation 4. GPU-(D)BE is complete and correct.
The completeness and correctness of GPU-(D)BE follow from the completeness
and correctness of BE [10] and DPOP [25].

4 Related Work

The use of GPUs to solve difficult combinatorial problems has been explored
by several proposals in different areas of constraint optimization. For instance,
Meyer et al. [18] proposed a multi-GPU implementation of the simplex tableau
algorithm which relies on a vertical problem decomposition to reduce communi-
cation between GPUs. In constraint programming, Arbelaez and Codognet [3]
proposed a GPU-based version of the Adaptive Search that explores several large
neighborhoods in parallel, resulting in a speedup factor of 17. Campeotto et al. [9]
proposed a GPU-based framework that exploits both parallel propagation and
parallel exploration of several large neighborhoods using local search techniques,
leading to a speedup factor of up to 38. The combination of GPUs with dynamic
programming has also been explored to solve different combinatorial optimiza-
tion problems. For instance, Boyer et al. [5] proposed the use of GPUs to compute
the classical DP recursion step for the knapsack problem, which led to a speedup
factor of 26. Pawlowski et al. [23] presented a DP-based solution for the coalition
structure formation problem on GPUs, reporting up to two orders of magnitude
134 F. Fioretto et al.

of speedup. Differently from other proposals, our approach aims at using GPUs
to exploit SIMT-style parallelism from DP-based methods to solve general COPs
and DCOPs.

5 Experimental Results
We compare our centralized and distributed versions of GPU-(D)BE with BE [10]
and DPOP [25] on binary constraint networks with random, scale-free, and reg-
ular grid topologies. The instances for each topology are generated as follows:
Random: We create an n-node network, whose density p1 produces n (n−1) p1 
edges in total. We do not bound the tree-width, which is based on the underlying
graph.
Scale-free: We create an n-node network based on the Barabasi-Albert
model [4]: Starting from a connected 2-node network, we repeatedly add a new
node, randomly connecting it to two existing nodes. In turn, these two nodes
are selected with probabilities that are proportional to the numbers of their
connected edges. The total number of edges is 2 (n − 2) + 1.
Regular grid: We create an n-node network arranged as a rectangular grid,
where each internal node is connected to four neighboring nodes, while nodes
on the grid edges (resp. corners) are connected to two (resp. three) neighboring
nodes.
We generate 30 instances for each topology, ensuring that the underlying graph
is connected. The utility functions are generated using random integer costs in
[0, 100], and the constraint tightness (i.e., ratio of entries in the utility table
different from −∞) p2 is set to 0.5 for all experiments. We set as default param-
eters, |A| = |X| = 10, |Di | = 5 for all variables, and p1 = 0.3 for random networks,
and |A| = |X| = 9 for regular grids. Experiments for GPU-DBE are conducted
using a multi-agent DCOP simulator, that simulates the concurrent activities of
multiple agents, whose actions are activated upon receipt of a message. We use
the publicly-available implementation of DPOP available in the FRODO frame-
work v.2.11 [19], and we use the same framework to run the BE algorithm, in a
centralized setting.
Since all algorithms are complete, our focus is on runtime. Performance of
the centralized algorithms are evaluated using the algorithm’s wallclock runtime,
while distributed algorithms’ performances are evaluated using the simulated
runtime metric [30]. We imposed a timeout of 300s of wallclock (or simulated)
time and a memory limit of 32GB. Results are averaged over all instances and
are statistically significant with p-values < 1.638 e−12 .6 These experiment are
performed on an AMD Opteron 6276, 2.3GHz, 128GB of RAM, which is equipped
with a GPU device GeForce GTX TITAN with 14 multiprocessors, 2688 cores,
and a clock rate of 837MHz.
6
t-test performed with null hypothesis: GPU-based algorithms are faster than non-
GPU ones.
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 135

Fig. 4. Runtimes for COPs (top) and DCOPs (bottom) at varying number of vari-
ables/agents.

Fig. 4 illustrates the runtime, in seconds, for random (a), scale-free (b), and
regular grid (c) topologies, varying the number of variables (resp. agents) for the
centralized (resp. distributed) algorithms. The centralized algorithms (BE and
GPU-BE) are shown at the top of the figure, while the distributed algorithms
(DPOP and GPU-DBE) are illustrated at the bottom. All plots are in log-scale.
We make the following observations:
• The GPU-based DP-algorithms (for both centralized and distributed cases)
are consistently faster than the non-GPU-based ones. The speedups obtained
by GPU-BE vs. BE are, on average, and minimum (showed in parenthesis)
69.3 (16.1), 34.9 (9.5), and 125.1 (42.6), for random, scale-free, and regular grid
topologies, respectively. For the distributed algorithms, the speedups obtained
by GPU-DBE vs. DPOP are on average (minimum) 44.7 (14.7), 22.3 (8.2), and
124.2 (38.8), for random, scale-free, and regular grid topologies, respectively.
• In terms of scalability, the GPU-based algorithms scale better than the non-
GPU-based ones. In addition, their scalability increases with the level of struc-
ture exposed by each particular topology. On random graphs, which have
virtually no structure, the GPU-based algorithms reach a timeout for instances
with small number of variables (25 variables—compared to 20 variables for the
non-GPU-based algorithms). On scale-free networks, the GPU-(D)BE algo-
rithms can solve instances up to 50 variables,7 while BE and DPOP reach a
timeout for instances greater than 40 variables. On regular grids, the GPU-
based algorithms can solve instances up to 100 variables, while the non-GPU-
based ones, fail to solve any instance with 36 or more variables.
We relate these observations to the size of the separator sets and, thus, the size
of the UTIL tables that are constructed in each problem. In our experiments,

7
With 60 variables, we reported 12/30 instances solved for GPU-(D)BE.
136 F. Fioretto et al.

Fig. 5. Runtimes for COPs (top) and DCOPs (bottom) at varying number of vari-
ables/agents.

we observe that the average sizes of the separator sets are consistently larger
in random graphs, followed by scale-free networks, followed by regular grids.
• Finally, the trends of the centralized algorithms are similar to those of the
distributed algorithms: The simulated runtimes of the DCOP algorithms are
consistently smaller than the wallclock runtimes of the COP ones.
Fig. 5 illustrates the behavior of the algorithms when varying the graph den-
sity p1 for the random graphs (a), and the domains size for random graphs (b)
and regular grids (c). As for the previous experiments, the centralized (resp. dis-
tributed) algorithms are shown on the top (resp. bottom) of the figure. We can
observe:
• The trends for the algorithms runtime, when varying both p1 and domains
size, are similar to those observed in the previous experiments.
• GPU-(D)BE achieves better speed-up for smaller p1 (Fig. 4 (a)). The result
is explained by observing that small p1 values correspond to smaller induced
width of the underlying constraint graph. In turn, for small p1 values, GPU-
(D)BE agents construct smaller UTIL tables, which increases the probabil-
ity of performing the complete inference process on the GPU, through the
GPU-Aggregate-Project procedure. This observation is also consistent
with what observed in the previous experiments in terms of scalability.
• GPU-(D)BE achieves greater speedups in presence of large domains. This is
due to the fact that large domains correspond to large UTIL tables, enabling
the GPU-based algorithms to exploit a greater amount of parallelism, provided
that the UTIL tables can be stored in the global memory of the GPU.

6 Conclusions and Discussions

In this paper, we presented an investigation of the use of GPUs to exploit
SIMT-style parallelism from DP-based methods to solve COPs and DCOPs.
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 137

We proposed a procedure, inspired by BE (for COPs) and DPOP (for DCOPs),

that makes use of multiple threads to parallelize the aggregation and projection
phases. Experimental results show that the use of GPUs may provide signifi-
cant advantages in terms of runtime and scalability. The proposed results are
significant—the wide availability of GPUs provides access to parallel computing
solutions that can be used to improve efficiency of (D)COP solvers. Furthermore,
GPUs are renowned for their complex architectures (multiple memory levels with
very different size and speed characteristics; relatively slow cores), which often
create challenges to the effective exploitation of parallelism from irregular appli-
cations; the strong experimental results indicate that the proposed algorithms
are well-suited to GPU architectures. While envisioning further research in this
area, we anticipate several challenges:
• In terms of implementation, GPU programming can be more demanding when
compared to a classical sequential implementation. One of the current limita-
tions for (D)COP-based GPU approaches is the absence of solid abstractions
that allow component integration, modularly, without restructuring the whole
program.
• Exploiting the integration of CPU and GPU computations is a key factor to
obtain competitive solvers performance. Complex and repeated calculations
should be delegated to GPUs, while simpler and memory intensive operations
should be assigned to CPUs. It is however unclear how to determine good
tradeoffs of such integrations. For instance, repeatedly invoking many mem-
ory demanding GPU kernels could be detrimental to the overall performance,
due to the high cost of allocating the device memory (e.g., shared memory).
Creating lightweight communication mechanisms between CPU and GPU (for
instance, by taking advantage of the asynchronism of CUDA streams) to allow
active GPU kernels to be used in multiple instances could be a possible solu-
tion to investigate.
• While this paper describes the applicability of our approach to BE and DPOP,
we believe that analogous techniques can be derived and applied to other
DP-based approaches to solve (D)COPs—e.g., to implement the logic of DP-
based propagators. We also envision that such technology could open the door
to efficiently enforcing higher form of consistencies than domain consistency
(e.g., path consistency [22], adaptive consistency [12], or the more recently
proposed branch consistency for DCOPs [14]), especially when the constraints
need to be represented explicitly.

References
1. Abdennadher, S., Schlenker, H.: Nurse scheduling using constraint logic program-
ming. In: Proceedings of the Conference on Innovative Applications of Artificial
Intelligence (IAAI), pp. 838–843 (1999)
2. Apt, K.: Principles of constraint programming. Cambridge University Press (2003)
3. Arbelaez, A., Codognet, P.: A GPU implementation of parallel constraint-based
local search. In: Proceedings of the Euromicro International Conference on Parallel,
Distributed and network-based Processing (PDP), pp. 648–655 (2014)
138 F. Fioretto et al.

4. Barabási, A.-L., Albert, R.: Emergence of scaling in random networks. Science

286(5439), 509–512 (1999)
5. Boyer, V., El Baz, D., Elkihel, M.: Solving knapsack problems on GPU. Computers
& Operations Research 39(1), 42–47 (2012)
6. Brito, I., Meseguer, P.: Improving DPOP with function filtering. In: Proceedings
of the International Conference on Autonomous Agents and Multiagent Systems
(AAMAS), pp. 141–158 (2010)
7. Burke, D., Brown, K.: Efficiently handling complex local problems in distributed
constraint optimisation. In: Proceedings of the European Conference on Artificial
Intelligence (ECAI), pp. 701–702 (2006)
8. Burke, E.K., De Causmaecker, P., Berghe, G.V., Van Landeghem, H.: The state of
the art of nurse rostering. Journal of scheduling 7(6), 441–499 (2004)
9. Campeotto, F., Dovier, A., Fioretto, F., Pontelli, E.: A GPU implementation of
large neighborhood search for solving constraint optimization problems. In: Pro-
ceedings of the European Conference on Artificial Intelligence (ECAI), pp. 189–194
(2014)
10. Dechter, R.: Bucket elimination: a unifying framework for probabilistic inference.
In: Learning in graphical models, pp. 75–104. Springer (1998)
11. Dechter, R.: Constraint Processing. Morgan Kaufmann Publishers Inc., San
Francisco (2003)
12. Dechter, R., Pearl, J.: Network-based heuristics for constraint-satisfaction prob-
lems. Springer (1988)
13. Farinelli, A., Rogers, A., Petcu, A., Jennings, N.: Decentralised coordination of
low-power embedded devices using the Max-Sum algorithm. In: Proceedings of
the International Conference on Autonomous Agents and Multiagent Systems
(AAMAS), pp. 639–646 (2008)
14. Fioretto, F., Le, T., Yeoh, W., Pontelli, E., Son, T.C.: Improving DPOP with
branch consistency for solving distributed constraint optimization problems. In:
O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 307–323. Springer, Heidelberg
(2014)
15. Gaudreault, J., Frayret, J.-M., Pesant, G.: Distributed search for supply chain
coordination. Computers in Industry 60(6), 441–451 (2009)
16. Hamadi, Y., Bessière, C., Quinqueton, J.: Distributed intelligent backtracking.
In: Proceedings of the European Conference on Artificial Intelligence (ECAI),
pp. 219–223 (1998)
17. Kumar, A., Faltings, B., Petcu, A.: Distributed constraint optimization with struc-
tured resource constraints. In: Proceedings of the International Conference on
Autonomous Agents and Multiagent Systems (AAMAS), pp. 923–930 (2009)
18. Lalami, M.E., El Baz, D., Boyer, V.: Multi GPU implementation of the simplex
algorithm. Proceedings of the International Conference on High Performance Com-
puting and Communication (HPCC) 11, 179–186 (2011)
19. Léauté, T., Faltings, B.: Distributed constraint optimization under stochastic
uncertainty. In: Proceedings of the AAAI Conference on Artificial Intelligence
(AAAI), pp. 68–73 (2011)
20. Maheswaran, R., Tambe, M., Bowring, E., Pearce, J., Varakantham, P.: Taking
DCOP to the real world: Efficient complete solutions for distributed event schedul-
ing. In: Proceedings of the International Conference on Autonomous Agents and
Multiagent Systems (AAMAS), pp. 310–317 (2004)
21. Modi, P., Shen, W.-M., Tambe, M., Yokoo, M.: ADOPT: Asynchronous distributed
constraint optimization with quality guarantees. Artificial Intelligence 161(1–2),
149–180 (2005)
Exploiting GPUs in Solving (Distributed) Constraint Optimization Problems 139

22. Montanari, U.: Networks of constraints: Fundamental properties and applications

to picture processing. Information sciences 7, 95–132 (1974)
23. Pawlowski, K., Kurach, K., Michalak, T., Rahwan, T.: Coalition structure genera-
tion with the graphic processor unit. Technical Report CS-RR-13-07, Department
of Computer Science, University of Oxford (2014)
24. Pesant, G.: A regular language membership constraint for finite sequences of vari-
ables. In: Proceedings of the International Conference on Principles and Practice
of Constraint Programming (CP), pp. 482–495 (2004)
25. Petcu, A., Faltings, B.: A scalable method for multiagent constraint optimiza-
tion. In: Proceedings of the International Joint Conference on Artificial Intelligence
(IJCAI), pp. 1413–1420 (2005)
26. Quimper, C.-G., Walsh, T.: Global grammar constraints. In: Benhamou, F. (ed.)
CP 2006. LNCS, vol. 4204, pp. 751–755. Springer, Heidelberg (2006)
27. Rodrigues, L.C.A., Magatão, L.: Enhancing supply chain decisions using constraint
programming: a case study. In: Gelbukh, A., Kuri Morales, Á.F. (eds.) MICAI 2007.
LNCS (LNAI), vol. 4827, pp. 1110–1121. Springer, Heidelberg (2007)
28. Rossi, F., van Beek, P., Walsh, T. (eds.) Handbook of Constraint Programming.
Elsevier (2006)
29. Sanders, J., Kandrot, E.: CUDA by Example. An Introduction to General-Purpose
GPU Programming. Addison Wesley (2010)
30. Sultanik, E., Modi, P.J., Regli, W.C.: On modeling multiagent task scheduling as
a distributed constraint optimization problem. In: Proceedings of the International
Joint Conference on Artificial Intelligence (IJCAI), pp. 1531–1536 (2007)
31. Trick, M.A.: A dynamic programming approach for consistency and propagation
for knapsack constraints. Annals of Operations Research 118(1–4), 73–84 (2003)
32. Yeoh, W., Felner, A., Koenig, S.: BnB-ADOPT: An asynchronous branch-and-
bound DCOP algorithm. Journal of Artificial Intelligence Research 38, 85–133
(2010)
33. Yeoh, W., Yokoo, M.: Distributed problem solving. AI Magazine 33(3), 53–65
(2012)
34. Yokoo, M. (ed.): Distributed Constraint Satisfaction: Foundation of Cooperation
in Multi-agent Systems. Springer (2001)
35. Zivan, R., Okamoto, S., Peled, H.: Explorative anytime local search for distributed
constraint optimization. Artificial Intelligence 212, 1–26 (2014)
36. Zivan, R., Yedidsion, H., Okamoto, S., Glinton, R., Sycara, K.: Distributed con-
straint optimization for teams of mobile sensing agents. Journal of Autonomous
Agents and Multi-Agent Systems 29(3), 495–536 (2015)
 Conflict Ordering Search
for Scheduling Problems

Steven Gay1(B) , Renaud Hartert1 , Christophe Lecoutre2 , and Pierre Schaus1

1
UCLouvain, ICTEAM, Place Sainte Barbe 2, 1348 Louvain-la-Neuve, Belgium
{steven.gay,renaud.hartert,pierre.schaus}@uclouvain.be
2
CRIL-CNRS UMR 8188, Université d’Artois, 62307 Lens, France
[email protected]

Abstract. We introduce a new generic scheme to guide backtrack

search, called Conflict Ordering Search (COS), that reorders variables
on the basis of conflicts that happen during search. Similarly to gen-
eralized Last Conflict (LC), our approach remembers the last variables
on which search decisions failed. Importantly, the initial ordering behind
COS is given by a specified variable ordering heuristic, but contrary to
LC, once consumed, this first ordering is forgotten, which makes COS
conflict-driven. Our preliminary experiments show that COS – although
simple to implement and parameter-free – is competitive with special-
ized searches on scheduling problems. We also show that our approach
fits well within a restart framework, and can be enhanced with a value
ordering heuristic that selects in priority the last assigned values.

1 Introduction
Backtracking search is a central complete algorithm used to solve combinato-
rial constrained problems. Unfortunately, it suffers from thrashing – repeat-
edly exploring the same fruitless subtrees – during search. Restarts, adaptive
heuristics, and strong consistency algorithms are typical Constraint Program-
ming (CP) techniques used to cope with thrashing.
Last Conflicts (LC) [9] has been shown to be highly profitable to complete
search algorithms, both in constraint satisfaction and in automated artificial
intelligence planning. The principle behind LC is to select in priority the last
conflicting variables as long as they cannot be instantiated without leading to
a failure. Interestingly enough, last conflict search can be combined with any
underlying variable ordering heuristic. In normal mode, the underlying heuristic
selects the variables to branch on, whereas in conflict mode, variables are directly
selected in a conflict set built by last conflict.
While last conflict uses conflicts to repair the search heuristic, we show in this
paper that conflicts can also be used to drive the search process by progressively
replacing the initial variable heuristic. Basically, the idea behind our approach –
namely, Conflict Ordering Search – is to reorder variables according to the most
recent conflict they were involved in. Our experiments highlight that this simple
reordering scheme, while being generic, can outperform domain specific heuristics
for scheduling problems.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 140–148, 2015.
DOI: 10.1007/978-3-319-23219-5 10
 Conflict Ordering Search for Scheduling Problems 141

2 Related Works
We start by providing a quick overview of general-purpose search heuristics and
schemes since our approach is definitively one of them. The simple variable
ordering heuristic dom [5] – which selects variables by their domain size – has
long been considered as the most robust backtrack search heuristic. However,
a decade ago, modern adaptive heuristics were introduced. Such heuristics take
into account information related to the part of the search space already explored.
The two first proposed generic adaptive heuristics are impact [16] and wdeg [1].
The former relies on a measure of the effect of any assignment, and the latter
associates a counter with each constraint (and indirectly, with each variable)
indicating how many times any constraint led to a domain wipe-out. Counting-
based heuristics [14] and activity-based search [11] are two recently introduced
additional adaptive techniques to guide the search process.
Interestingly, Last Conflict (LC) [9] is a search mechanism that can be applied
on top of any variable ordering heuristic. Precisely, the generalized form LC(k)
works by recording and assigning first the k variables involved in the k last deci-
sions that provoked a failure after propagation. The underlying search heuristic
is used when all the last k conflicting variables have been assigned. While the
ability of relying on an underlying search heuristic is a strong point of LC, setting
the parameter k can be problematic, as we shall see later. The related scheme
introduced in this paper goes further and orders permanently all conflicting vari-
ables using the time of their last conflicts, eventually becoming independent of
the helper variable ordering heuristic.

3 Guiding Search by Timestamping Conflicts

We first introduce basic concepts. Then, we introduce Conflict Ordering Search

and highlight its benefits within a context of restarts. We conclude this section
by discussing the differences between LC(k) and COS.

3.1 Background

CSP. A Constraint Satisfaction Problem (CSP) P is a pair (X , C), where X is

a finite set of variables and C is a finite set of constraints. Each variable x ∈ X
has a domain dom(x) that contains the allowed values for x. A valuation on a
subset X ⊆ X of variables maps each variable x ∈ X with a value in dom(x).
Each constraint c ∈ C has a scope scp(c) ⊆ X , and is semantically defined by a
set of allowed valuations on scp(c); the valuations that satisfy c. A valuation on
X is a solution of P iff it satisfies each constraint of P . A CSP is satisfiable iff
it admits at least one solution.

Tree-Search. One can solve CSPs by using backtrack search, a complete depth-
first exploration of the search space, with backtracking when a dead-end occurs.
At each search node, a filtering process φ can be performed on domains by
142 S. Gay et al.

soliciting propagators associated with constraints. A CSP is in failure, denoted ⊥,

when unsatisfiability is detected by φ. A branching heuristic is a function that
maps a non-failed CSP to an ordered sequence of constraints, called decisions.
In this work, we only consider binary variable-based branching heuristics, i.e.,
heuristics that always generate sequences of decisions of the form x ∈ D, x ∈/ D,
where x is a variable of X and D a strict subset of dom(x). A search tree
is the structure explored by backtrack search through its filtering capability
and its branching heuristic. A failed node in the search tree is a node where
unsatisfiability has been detected by φ.

3.2 Conflict Ordering

Considering a variable-based branching heuristic, we can associate a failed search
node with the variable involved in the decision leading to it. This allows us to
timestamp variables with the number of the last conflict they caused (see Fig. 1).
The basic idea behind Conflict Ordering Search is to leverage this timestamping
mechanism to reorder the variables during search.

Fig. 1. Conflict numbering and timestamps associated with each variable. Variables
are stamped with the number of their latest conflict (or 0 by default).

Algorithm 1 describes Conflict Ordering Search. For simplicity, we only con-

sider classical binary branching with decisions of the form x ≤ v and x > v.
We use an integer nConflicts to count the number of conflicts and a reference
lastVar to the last variable involved in a decision (initially null at the root of the
search-tree). We also consider a one-dimensional array stamps that associates
with each variable x ∈ X the last time variable x was involved in a conflict.
They are all initialized to 0. We suppose that φ corresponds to a domain filter-
ing consistency, which is at least as strong as the partial form of arc consistency
ensured by the forward checking algorithm [5].
If the resulting CSP at line 1 is trivially inconsistent (⊥), false is returned
(line 6). If the failure is due to a previous decision (line 3), the number of conflicts
is incremented and the conflicting variable timestamped with this number (lines 4
and 5). Otherwise, COS returns true if a solution has been found, i.e., the domain
of each variable in X is a singleton (lines 7 and 8). The selection of the next
variable to branch on is performed between lines 9 and 13. Here, the timestamps
are used to select the unbound variable involved in the latest conflict. If no
unbound variable ever conflicted, the search falls back to the bootstrapping
 Conflict Ordering Search for Scheduling Problems 143

heuristic varHeuristic. When a new value has been selected by the heuristic
valHeuristic[x], we recursively call COS . One can observe that the complexity
of selecting a variable is linear in time and space, hence scaling well.1

Algorithm 1. COS (P = (X , C): CSP)

Output: true iff P is satisfiable
1 P ← φ(P )
2 if P = ⊥ then
3 if lastVar = null then
4 nConflicts ← nConflicts + 1
5 stamps[lastVar] ← nConflicts
6 return false
7 if ∀x ∈ X , |dom(x)| = 1 then
8 return true
9 failed ← {x ∈ X : stamps[x] > 0 ∧ |dom(x)| > 1}
10 if failed = ∅ then
11 lastVar ← varHeuristic.select()
12 else
13 lastVar ← argmax x∈failed {stamps[x]}
14 v ← valHeuristic[lastVar].select()
15 return COS (P|lastVar≤v ) ∨ COS (P|lastVar>v )

Example 1. Let us consider a toy CSP with n “white” variables, and m “black”
variables. White variables have a binary domain while black variables have
{1, 2, . . . , m − 1} as domain. We also add a binary difference constraint on each
pair of black variables (thus making the CSP unsatisfiable), but no constraint
at all on the white variables. Let us also assume a variable ordering heuristic
that selects the white variables first, then the black variables. Hence, proving
unsatisfiability using this heuristic requires to prove the “black conflict” for the
2n valuations of the white variables (see left part of Fig. 2). Using COS on top
of this heuristic allows one to detect unsatisfiability quickly. Indeed, the m − 2
first conflicting black variables will be prioritized as a white variable cannot be
involved in a conflict. The number of times the “black conflict” as to be proven
thus becomes linear in the number of white variables n (see right part of Fig. 2).

COSPhase: a Variant. Because it is known that remembering last assigned val-

ues for later priority uses can be worthwhile (see for example phase saving [15]
in SAT), we propose such a variant for COS. So, when a positive decision x ≤ v
succeeds, we record its value v. Then, when branching is performed on a times-
tamped variable x, we exploit the associated recorded value v. If v is still in
1
The time complexity could be improved if an ordered linked-list is used instead of
the array stamps.
144 S. Gay et al.

Without Conflict Ordering With Conflict Ordering

. . .

Fig. 2. Conflict Ordering Search reorders the variables to reduce the number of times
the inconsistent black subtree has to be explored.

the domain of x, we use interval splitting on it, i.e., we branch with decisions
x ≤ v and x > v, otherwise the value heuristic is solicited. Observe that this
mechanism follows the first-fail/best-first principle.

3.3 Restarts and Timestamp Ordering

Depth-first search is far from always being the most efficient way to solve a CSP.
In some cases, it may suffer from heavy-tailed distributions [4]. While restarting
the search with a randomized heuristic is a typical way to avoid the worst parts
of a long-tailed curve, nogood learning mitigates the effect of cutting a search
process short by remembering parts of the search space already explored [8].2
In the context of COS, we observe that our approach not only remembers the
sources of conflicts but also produce a variable ordering that yields a behavior
similar to randomization. We thus propose to use no additional randomization
when restarting, only using the natural randomizing effect of conflict ordering
instead. The rationale is that while conflict ordering is good at finding a set of
conflicting variables in a given context – i.e., a sequence of previous decisions –
restarting with conflict ordering has the effect of trying the latest conflict set in
other contexts.

Example 2. Let us consider the toy CSP described in Example 1. The time
required to prove unfeasibility could be drastically reduced if a restart occurs
after having explored the inconsistent black subtree at least once. Indeed, in this
context, the second restart will directly explore the black search tree without
even considering the white variables that have been “disculpated”.

3.4 Differences with Last Conflict Search

Although similar, we show that COS and LC are rather different. Indeed, LC
relies on a parameter k that corresponds to the maximum size of the conflict
sets that can be captured by the search process. The value of k is of importance
as setting k too low may not allow LC to capture the encountered conflict sets.
For instance, LC(k) cannot capture the conflict set in Example 1 if k is lower
than m − 2. COS, however, does not require any parameter and is able to handle
2
Similar frameworks are typically used by SAT solvers [12].
 Conflict Ordering Search for Scheduling Problems 145

conflict sets of any size. While setting the parameter k to the number of decision
variables may solve the problem, LC still suffers from resets of its conflict set
that occur each time the conflicting variables have been successfully assigned.
Conversely, COS does not forget conflicting variables and progressively reorders
those variables to give priority to the recently conflicting ones. This is particu-
larly important with restarts as LC is not designed to focus on conflict sets in
such contexts (see Example 2).

4 Experiments

We have tested our approach on RCPSP (Resource-Constrained Project Schedul-

ing Project) instances from PSPLIB [6]. We used a computer equipped with
a i7-3615QM processor running at 2.30GHz. The problem has been modeled
in the open-source solver OscaR [13], using precedence and cumulative con-
straints. Precedences are simple binary precedence constraints, and cumulative
constraints use the Time-Tabling propagator presented in [2]. Both overload
checking [19] or time-table edge-finding [17] were tested but energy-based rea-
soning does not help much on PSPLIB instances, whose optimal solutions typ-
ically waste capacity. Adding TTDR [3] helps even with learning searches, but
it makes the experiments harder to reproduce, thus we chose to not use it.

4.1 Branch-and-Bound

The goal of this first experiment is to find an optimal solution using a pure
branch-and-bound search. We compare five search solving methods. The first
is a simple min/min scheme, which selects the variable with the smallest min-
imal value and chooses the smallest value (for assignment). The second one is
the scheduling-specialized SetTimes heuristic with dominances [7], which is a
min/min scheme that simply postpones assignments (instead of making value
refutations) when branching at right, fails when a postponed task can no longer
be woken up, and assigns the tasks that are not postponed and cannot be
disturbed by other tasks. Finally, the last three heuristics correspond to conflict-
based reasoning searches, namely, LC(k) for the best value of k, our main con-
tribution COS, and COSPhase based on the variant presented in Section 15. All
these have a min/min helper heuristic.
We have observed that the ranking of these five search methods is the same
on the four RCPSP benchmarks (J30, J60, J90, J120) from PSPLIB. Results
are represented on the left part of Fig. 3 where the y-axis is the cumulated
number of solved instances and the x-axis is the CPU time. SetTimes is clearly
an improvement on min/min, as it finishes ahead and seems to continue its
course on a better slope than min/min. In turn, the well-parameterized LC
(we empirically chose the best possible value for k) fares better than SetTimes.
Finally, COS allows us to close a higher number of instances, and the variant
COSPhase improves COS even further.
146 S. Gay et al.

4.2 Branch-and-Bound with Restarts

As illustrated in Example 2, keeping the conflict ordering between restarts could
drastically reduce search efforts. The aim of this second experiment is to compare
the performance of COS if the ordering is kept between restarts or not. Experi-
mental settings are the same as before except that we use restarts and nogood
recording has explained in [10]. The first iteration is limited to 100 failures and
increases by a 1.15 factor. We compared the performance of COS and COSPhase
with and without reset (we add “rst-” as prefix for the resetting version). All
searches rely on minrnd /min – a randomized version of min/min that breaks ties
randomly – as helper heuristic.
Results are presented in the right part of Fig. 3. First, we observe that restarts
do have a positive effect as minrnd /min obtains better results than min/min in
the previous experiment. Next, we see that resetting the conflict order has a bad
effect on the search process. Indeed, these variant obtain worse results than in
the pure branch-and-bound framework. This highlights that using conflict-based
search as a full heuristic can yield much better results than using it as a repairing
patch. Finally, phase recording does not seem to help anymore.

COSPhase
235 COSPhase 237
COS
228 COS
211 rst-COSPhase
206 rst-COS
185 LC(k)
175 SetTimes 172 minrnd/min
155 min/min

0 0
0 10 ms 1 sec 1 min 0 10 ms 1 sec 1 min

Search #closed Σ Makespans Search #closed Σ Makespans

COSPhase 235 77789 COS 237 77770
COS 228 77952 COSPhase 236 77721
LC(k) 185 79010 rst-COSPhase 211 78301
SetTimes 175 80970 rst-COS 206 78476
min/min 155 80038 minrnd /min 172 79510

Fig. 3. On the left, results obtained for pure branch-and-bound, and on the right,
results obtained with branch-and-bound with restarts. Graphs at the top show the
cumulated number of solved RCPSP instances from PSPLIB120. The tables at the
bottom compare branching heuristics at the end of the 60s timeout, giving the number
of closed instances and the sum of makespans.

4.3 Destructive Lower Bounds.

We performed similar experiments for destructive lower bounds. We added the
Time-Table Edge-Finding propagator presented in [17] since it has a large impact
 Conflict Ordering Search for Scheduling Problems 147

in this case. The results are similar. We also compared COS to the recently intro-
duced Failure Directed Search [18] by implementing it directly in CP Optimizer.
Unfortunately our COS implementation in CP Optimizer was not able to obtain
results competitive with FDS.

5 Conclusion

In this paper, we have proposed a general-purpose search scheme that can be

combined with any variable ordering heuristic. Contrary to Last Conflict, Con-
flict Ordering Search is very aggressive, discarding progressively the role played
by the heuristic. Besides, by means of simple timestamps, all variables recorded
in the global conflict set stay permanently ordered, the priority being modified
at each new conflict. We have shown that on some structured known problems
our approach outperforms other generic and specific solving methods. So, COS
should be considered as one new useful technique to be integrated in the outfit
of constraint systems.

Acknowledgments. Steven Gay is financed by project Innoviris 13-R-50 of the

Brussels-Capital region. Renaud Hartert is a Research Fellow of the Fonds de la
Recherche Scientifiques - FNRS. Christophe Lecoutre benefits from the financial sup-
port of CNRS and OSEO (BPI France) within the ISI project ?Pajero?. The authors
would like to thank Petr Vilı́m for his help with the comparison to FDS in CP
Optimizer.

References
1. Boussemart, F., Hemery, F., Lecoutre, C., Sais, L.: Boosting systematic search by
weighting constraints. In: Proceedings of ECAI 2004, pp. 146–150 (2004)
2. Gay, S., Hartert, R., Schaus, P.: Simple and scalable time-table filtering for
the cumulative constraint. In: Pesant, G. (ed.) Proceedings of CP 2015. LNCS,
vol. 9255, pp. 149–157. Springer, Heidelberg (2015)
3. Gay, S., Hartert, R., Schaus, P.: Time-table disjunctive reasoning for the cumula-
tive constraint. In: Michel, L. (ed.) CPAIOR 2015. LNCS, vol. 9075, pp. 157–172.
Springer, Heidelberg (2015)
4. Gomes, C., Selman, B., Crato, N., Kautz, H.: Heavy-tailed phenomena in satisfia-
bility and constraint satisfaction problems. Journal of Automated Reasoning 24,
67–100 (2000)
5. Haralick, R.M., Elliott, G.L.: Increasing tree search efficiency for constraint satis-
faction problems. Artificial Intelligence 14, 263–313 (1980)
6. Kolisch, R., Schwindt, C., Sprecher, A.: Benchmark instances for project scheduling
problems. In: Project Scheduling, pp. 197–212. Springer (1999)
7. Le Pape, C., Couronné, P., Vergamini, D., Gosselin, V.: Time-versus-capacity com-
promises in project scheduling (1994)
8. Lecoutre, C., Sais, L., Tabary, S., Vidal, V.: Nogood recording from restarts. In:
Proceedings of IJCAI 2007, pp. 131–136 (2007)
148 S. Gay et al.

9. Lecoutre, C., Sais, L., Tabary, S., Vidal, V.: Reasonning from last conflict(s) in
constraint programming. Artificial Intelligence 173(18), 1592–1614 (2009)
10. Lecoutre, C., Sais, L., Tabary, S., Vidal, V.: Recording and minimizing nogoods
from restarts. Journal on Satisfiability, Boolean Modeling and Computation 1,
147–167 (2007)
11. Michel, L., Van Hentenryck, P.: Activity-based search for black-box constraint
programming solvers. In: Beldiceanu, N., Jussien, N., Pinson, É. (eds.) CPAIOR
2012. LNCS, vol. 7298, pp. 228–243. Springer, Heidelberg (2012)
12. Moskewicz, M.W., Madigan, C.F., Zhao, Y., Zhang, L., Malik, S. Chaff: engineering
an efficient SAT solver. In: Proceedings of DAC 2001, pp. 530–535 (2001)
13. OscaR Team. OscaR: Scala in OR (2012). bitbucket.org/oscarlib/oscar
14. Pesant, G., Quimper, C.-G., Zanarini, A.: Counting-based search: Branching
heuristics for constraint satisfaction problems. Journal of Artificial Intelligence
Research 43, 173–210 (2012)
15. Pipatsrisawat, K., Darwiche, A.: A lightweight component caching scheme for sat-
isfiability solvers. In: Marques-Silva, J., Sakallah, K.A. (eds.) SAT 2007. LNCS,
vol. 4501, pp. 294–299. Springer, Heidelberg (2007)
16. Refalo, P.: Impact-based search strategies for constraint programming. In:
Wallace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 557–571. Springer, Heidelberg
(2004)
17. Vilı́m, P.: Timetable edge finding filtering algorithm for discrete cumulative
resources. In: Achterberg, T., Beck, J.C. (eds.) CPAIOR 2011. LNCS, vol. 6697,
pp. 230–245. Springer, Heidelberg (2011)
18. Vilı́m, P., Laborie, P., Shaw, P.: Failure-directed search for constraint-based
scheduling. In: Michel, L. (ed.) CPAIOR 2015. LNCS, vol. 9075, pp. 437–453.
Springer, Heidelberg (2015)
19. Wolf, A., Schrader, G.: O(n log n) overload checking for the cumulative constraint
and its application. In: Umeda, M., Wolf, A., Bartenstein, O., Geske, U., Seipel, D.,
Takata, O. (eds.) INAP 2005. LNCS (LNAI), vol. 4369, pp. 88–101. Springer,
Heidelberg (2006)
 Simple and Scalable Time-Table Filtering
for the Cumulative Constraint

Steven Gay(B) , Renaud Hartert, and Pierre Schaus

ICTEAM, UCLouvain, Place Sainte Barbe 2, 1348 Louvain-la-neuve, Belgium

{steven.gay,renaud.hartert,pierre.schaus}@uclouvain.be

Abstract. Cumulative is an essential constraint in the CP framework,

and is present in scheduling and packing applications. The lightest filter-
ing for the cumulative constraint is time-tabling. It has been improved
several times over the last decade. The best known theoretical time com-
plexity for time-table is O(n log n)introduced by Ouellet and Quimper.
We show a new algorithm able to run in O(n), by relying on range min
query algorithms. This approach is more of theoretical rather than prac-
tical interest, because of the generally larger number of iterations needed
to reach the fixed point. On the practical side, the recent synchronized
sweep algorithm of Letort et al, with a time-complexity of O(n2 ), requires
fewer iterations to reach the fix-point and is considered as the most scal-
able approach. Unfortunately this algorithm is not trivial to implement.
In this work we present a O(n2 ) simple two step alternative approach:
first building the mandatory profile, then updating all the bounds of
the activities. Our experimental results show that our algorithm out-
performs synchronized sweep and the time-tabling implementations of
other open-source solvers on large scale scheduling instances, sometimes
significantly.

Keywords: Constraint programming · Large-scale · Scheduling ·

Cumulative constraint · Time-table

1 Preliminaries
In this paper, we focus on a single cumulative resource with a discrete finite
capacity C ∈ N and a set of n tasks Ω = {1, . . . , n}. Each task i has a start time
si ∈ Z, a fixed duration di ∈ N, and an end time ei ∈ Z such that the equality
si + di = ei holds. Moreover, each task i consumes a fixed amount of resource
ci ∈ N during its processing time. Tasks are non-preemptive, i.e., they cannot be
interrupted during their processing time. In the following, we denote by si and
si the earliest and the latest start time of task i and by ei and ei the earliest and
latest end time of task i (see Fig. 1). The cumulative constraint [1] ensures that
the accumulated resource consumptiondoes not exceed the maximum capacity
C at any time t (see Fig. 2): ∀t ∈ Z : i∈Ω : si ≤t<ei ci ≤ C.
Even tasks that are not fixed convey some information that can be used by
filtering rules. For instance, tasks with a tight execution window must consume
some resource during a specific time interval known as mandatory part.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 149–157, 2015.
DOI: 10.1007/978-3-319-23219-5 11
150 S. Gay et al.

Fig. 1. Task i is characterized by its start Fig. 2. Accumulated resource consump-

time si , its duration di , its end time ei , tion over time. The cumulative constraint
and its resource consumption ci . ensures that the maximum capacity C is
not exceeded.

Definition 1 (Mandatory part). Let us consider a task i ∈ Ω. The manda-

tory part of i is the time interval [si , ei [. Task i has a mandatory part only if its
latest start time is smaller than its earliest end time.
If task i has a mandatory part, we know that task i will consume ci units of
resource during all its mandatory part no matter its start time. Fig. 3 illustrates
the mandatory part of an arbitrary task i.

Fig. 3. Task i has a mandatory part [si , ei [ if its latest start time si is smaller than its
earliest end time ei : si < ei . Task i always consumes the resource during its mandatory
part no matter its start time.

By aggregation, mandatory parts allow us to have an optimistic view of the

resource consumption over time. This aggregation is known as the time-table.
Definition 2 (Time-Table). The time-table TTΩ is the aggregation of the
mandatory parts of all the tasks in Ω. It is defined as the following step function:

TTΩ = t ∈ Z −→ ci . (1)
i∈Ω | si ≤t<ei

The capacity of the resource is exceeded if ∃t ∈ Z : TTΩ (t) > C.

The time-table of a resource can be computed in O(n) by a sweep algorithm
given the tasks sorted by latest start time and earliest end time [2,11,15].
The time-table filtering rule is formalized as follows:

(t < ei ) ∧ (ci + TTΩ\i (t) > C) ⇒ t < si . (2)

 Simple and Scalable Time-Table Filtering for the Cumulative Constraint 151

Observe that this filtering rule only describes how to update the start time of a
task. End times are updated in a symmetrical way.
Let j be a rectangle denoted aj , bj , hj
with aj ∈ Z (resp. bj ∈ Z) its start
(resp. end) time, hj ∈ N its height, and bj − aj its duration (length). The time-
table TTΩ can be represented as a contiguous sequence of rectangles

T TΩ = −∞, a1 , 0
, a1 , b1 , h1
, . . . , am , bm , hm
, bm , ∞, 0
(3)

such that bi = ai+1 and that the following holds:

∀ aj , bj , hj
∈ T TΩ , ∀t ∈ [aj , bj [ : T TΩ (t) = hj . (4)

We assume that the sequence is minimal, i.e., no consecutive rectangles have the
same height. The maximum number of rectangles is thus limited to 2n + 1.

Definition 3 (Conflicting Rectangle). For a task i, a left-conflicting rectan-

gle is a rectangle aj , bj , hj
∈ TTΩ\i such that (aj < e)∧(bj ≥ si )∧(hj > C−ci ).
We say that the task is in left-conflict with rectangle j. Right-conflicting rectan-
gles are defined symmetrically.

The time-table filtering rule can thus be rephrased as follows:

∀i ∈ Ω, ∀ aj , bj , hj
∈ T TΩ\i : j is in left-conflict with i ⇒ bj ≤ si . (5)

Definition 4 (Time-Table Consistency). A cumulative constraint is left

(resp. right) time-table consistent if no task has a left (resp. right) conflicting
rectangle. It is time-table consistent if it is left and right time-table consistent.

2 Existing Algorithms for Time-Tabling

Using the notion of conflicting rectangles, one can design a naive time-tabling
algorithm by confronting every task to every rectangle of the profile. The fol-
lowing algorithms improve on this, mainly by avoiding fruitless confrontations
of rectangles and tasks.

Sweep-line Algorithm. The sweep-line algorithm introduced by Beldiceanu et al.

[2] introduces tasks from left to right, and builds the mandatory profile on-the-
fly. This allows to confront tasks and rectangles only if they can overlap in time.
It can factorize confrontations of a rectangle against several tasks, by organizing
tasks in a heap. It pushes tasks to the right until they have no left-conflicting
rectangle, as pictured in Figure 4(c). This algorithm runs in O(n2 ).

Idempotent Sweep-line Algorithm. The sweep-line algorithm by Letort et. al [11]

improves on building the profile on-the-fly, by taking in consideration mandatory
parts that appear dynamically as tasks are pushed. It reaches left-time-table
consistency in O(n2 ), or O(n2 log n) for its faster practical implementation.
152 S. Gay et al.

Interval Tree Algorithm. The algorithm of Ouellet and Quimper [14] first builds
the profile, then introduces rectangles and tasks in an interval tree. Rectangles
are introduced by decreasing height, tasks by increasing height. This allows tasks
and rectangles to be confronted only when their heights do conflict. For each
task introduction, the tree structure decides in log n if its time domain conflicts
with some rectangle. Its filtering is weaker, since it pushes a task i only after
left-conflicting rectangles that overlap [si , ei [, as pictured in Figure 4(b). The
algorithm has time complexity O(n log n).

Fig. 4. Filtering obtained for (a) our linear time-tabling (b) Ouellet et al [14] and (c)
Beldiceanu et al [2].

New Algorithms. In this paper, we introduce two new algorithms for time-
tabling. The first one is of theoretical interest and runs in O(n). It uses range-
max-query algorithms to determine whether a task has a conflicting rectangle.
As the algorithm of Ouellet et al [14], it confronts task i only with rectangles
overlapping [si , ei [, but only chooses the one with the largest height instead of
the largest end. Thus, it prunes even less, as depicted in Figure 4(a).
The second algorithm is practical, and runs in O(n2 ). It separates profile
building from task sweeping. To locate tasks on the profile, it exploits residues
from previous computations, and incrementally removes fixed tasks that cannot
lead to any more pruning. It uses sweeping, thus pruning as much as [2] per call,
but it updates both the earliest start and latest end times of the tasks in a single
execution.

3 A Linear Time-Table Filtering Algorithm

In order to obtain linear time complexity, we will confront task i to only one
well-chosen rectangle, for every task i.
Proposition 1. Suppose the mandatory profile does not overload the resource. Let
i be a task, and j ∗ be a highest rectangle of the profile overlapping [si , min(ei , si )
[: j ∗ = argmaxj {hj | aj , bj , hj
∈ TTΩ and [aj , bj [∩[si , min(ei , si )[= ∅} .
Then j ∗ is in left-conflict iff hj ∗ + ci > C; otherwise i has no rectangles in
left-conflict.
 Simple and Scalable Time-Table Filtering for the Cumulative Constraint 153

Proof. If i has a mandatory part, we only need to look for conflict rectangles
overlapping [si , si [, since the profile already includes the mandatory part of i.
Otherwise, we need to look at [si , ei [. If rectangle j ∗ is not in left-conflict with
i, then no other rectangle can, since it would need to be higher than j ∗ .
To retrieve the index j ∗ , we must answer the question: given a vector of values
and two indices on this vector, what is the index between those two indices that
has the highest value? This kind of query corresponds to the range max query
problem1 , it can be done in constant time, given a linear time preprocessing [7].
Example 1. Assume the vector is values = [5, 4, 2, 1, 4, 3, 0, 8, 2, 3]. The range
max query between index 4 and index 7 is 5, denoted rmq(values, 4, 7) = 5.
This is indeed at index 5 that there is the maximum value on the subvector
[1, 4, 3, 0].
In our case the vector is composed of the heights of the rectangles of the
profile heights = [h1 , h2 , . . . , hm ]. The two indices of the query are respectively:
– j1 (i) is the index j of the rectangle aj , bj , hj
s.t. si ∈ [aj , bj [.
– j2 (i) is the index j of the rectangle aj , bj , hj
s.t. min(ei , si ) − 1 ∈ [aj , bj [
The whole algorithm is given in Algorithm 1. An example of the filtering is
given in Figure 4 (a). Notice that the task is pushed after a highest conflicting
rectangle, which is not as good as the filtering of [14] (Figure 4 (b)).

Algorithm 1. MinLeftTTLinearTime(Ω,C)
Input: A set of tasks Ω, capacity C.
Output: true iff propagation failed, i.e. if the problem is infeasible.
1 initialize TTΩ // aj , bj , hj ∀i ∈ {1 . . . m}
2 if maxj∈[1;m] hj > C then return true
3 heights ← [h1 , h2 , . . . , hm ]
4 ∀i ∈ Ω compute j1 (i), j2 (i)
5 initialize rmq(heights)
6 for i ∈ Ω such that s < s do
7 j ∗ ← rmq(heights, j1 (i), j2 (i))
8 if hj ∗ + ci > C then si ← bj
9 return false

Time Complexity. As in [6], we assume that all the time points are encoded
with w − bit integers and can thus be sorted in linear time. Given the sorted
time points the profile TTΩ can be computed in linear time using a sweep line
algorithm, and all the indices j1 (i), j2 (i) can be computed in linear time as well.
The range min/max query is a well studied problem. Preprocessing in line 5
can be done in linear time, so that any subsequent query at Line 7 executes in
constant time [7]. Thus, the whole algorithm executes in O(n).
1
a straightforward variant of the well-known range min query problem.
154 S. Gay et al.

Discussion. Although the linear time complexity is an improvement over the

O(n log n)algorithm introduced in [14], we believe that this result is more of
theoretical rather than practical interest. The linear time range max query ini-
tialization hides non-negligible constants. The range max query used in Line 7
to reach this time complexity could be implemented by simply iterating on the
rectangles from j1 (i) to j2 (i). On most problems, the interval [si , min(si , e) − 1]
only overlaps a few rectangles of the profile, so the O(n) cost is not high in
practice. Another limitation of the algorithm is that (as for the one of [14]) it
may be called several times before reaching the fix-point (although it does not
suffer from the slow convergence phenomenon described in [3]) either. It may
be more efficient to continue pushing a task further to the right as in [2] rather
than limiting ourselves to only one update per task per call to the procedure.
This is precisely the objective of the algorithm introduced in the next section.

4 An Efficient O(n2 ) Time-Table Filtering

In this section, we introduce a practical algorithm for time-table filtering. It
proceeds in two main phases: first the computation of the mandatory profile,
then a per-task sweeping from left to right and from right to left. This modular
design makes the algorithm simple, and its scalability comes from being able to
exploit structures separately, for instance using sorting only on few tasks. We
review the main phases of Algorithm 2 in execution order.

Building the Profile. Line 2 computes the mandatory profile as a sequence of

rectangles. We process only those tasks in Ω that have a mandatory part. We
will try to reduce that set of tasks further ahead, reducing the work in this part.

Computing Profile Indices. Line 4 computes, for all unfixed tasks i, the pro-
file rectangle containing si . This value is saved between consecutive calls in a
residue2 ; most of the time, it is still valid and we do not have to recompute
it, if not, a dichotomic search is performed, at a cost of O(log n). Note that [2]
sorts tasks by s to locate tasks on the profile, at a theoretical cost of O(n log n).
Here we pay O(log n) only for tasks where the residue is invalid. Similarly, line
5 computes the rectangle containing the last point of i, ei − 1.

Per-Task Sweeping. The loop in line 6 looks for left and right-conflicting rect-
angles for i linearly. The main difference with the global sweeping in [2] is that
our method does not factorize sweeping according to height, wagering that the
cost of moving tasks in height-ordered heaps is higher than that of pushing every
task until no conflict remains. This main part has a worst case cost O(n2 ).

Fruitless Fixed Tasks Removal. After the main loop, line 24 removes fixed tasks
at profile extremities that can no longer contribute to pruning. This filtering is
O(n). Note that Ω is kept along the search tree using a reversible sparse-set [5].
2
this is similar to residual supports for AC algorithms [10].
 Simple and Scalable Time-Table Filtering for the Cumulative Constraint 155

Algorithm 2. ScalableTimeTable(Ω,C)
Input: A set of tasks Ω, capacity C.
Output: true iff propagation failed, i.e. if the problem is infeasible.
1 Ωu ← {i | si < si } // unfixed tasks
2 initialize TTΩ // aj , bj , hj , ∀j ∈ {1 . . . m}
3 if maxj∈[1;m] hj > C then return true
4 ∀i ∈ Ωu , compute j1 (i) such that si ∈ [aj1 (i) ; bj1 (i) [
5 ∀i ∈ Ωu , compute j2 (i) such that ei − 1 ∈ [aj2 (i) ; bj2 (i) [
6 for i ∈ Ωu do
7 j ← j1 (i)
8 s∗i ← si
9 while j ≤ m and aj < min(s∗i + di , si ) do
10 if C − ci < hj then
11 s∗i ← min(bj , si ) // j in left-conflict
12 j ←j+1
13 if s∗i > si then si ← s∗i
14
15 j ← j2 (i)
16 e∗i ← ei
17 while j ≥ 1 and bj ≥ max(e∗i − di , ei ) do
18 if C − ci < hj then
19 e∗i ← max(aj , ei ) // j in right-conflict
20 j ←j−1
21 if e∗i < ei then ei ← e∗i
u
22 smin ← mini∈Ωu si
u
23 emax ← maxi∈Ωu ei
24 Ω ← Ω \ {i ∈ Omega | ei ≤ sumin ∨ eumax ≤ si }
25 return false

5 Experiments
We have tested our ScalableTimeTable filtering against the time-table filtering
of or-tools [12], Choco3 [4] and Gecode [9] solvers. The algorithm in Choco3
and Gecode solver is the one of [2]. Similarly to our algorithm, or-tools also
builds the time-table structure before filtering. To the best of our knowledge, no
implementation of Letort et al [11] algorithm is publicly available. We have thus
implemented the heap-based variant of the algorithm, faster in practice, with the
same quality standard as our new ScalableTimeTable [13]. In the models used
for this experiment, cumulative propagators enforce only the resource constraint,
precedences are enforced by separate propagators.
We have generated randomly n (ranging from 100 to 12800) tasks with dura-
tion between 200 and 2000 and heights between 1 and 40. The capacity is fixed to
100. The search is a simple greedy heuristic selecting the current tasks with the
smallest earliest possible start, hence there is no backtrack. The search finishes
156 S. Gay et al.

when all the tasks have been placed. This simple experimental setting guarantees
that every solver has exactly the same behavior. The results are given on Figure 5.
As can be seen, the time-table implementation of Choco3 and Gecode are quickly
not able to scale well for more than 1600 tasks. The algorithm of or-tools, Letort
et al and our new algorithm are still able to handle 12800 tasks. Surprisingly, our
algorithm outperforms the one of Letort et al despite its simplicity.

298.2 s Gecode

234.8 s Choco 3
175.6 s OR-Tools
128.8 s OscaR Letort

47.8 s OscaR Scalable

0.0 s
100 200 400 800 1600 3200 6400 12800

Fig. 5. Comparison of Time-Table implementations.

6 Conclusion
We have introduced an O(n) time-table filtering using range min queries. We
believe that the usage of range min query may be useful for subsequent research
on scheduling, for instance in time-table disjunctive reasoning [8]. We introduced
simple but scalable O(n2 ) filtering for the cumulative constraint. Our results
show that despite its simplicity, it outperforms current implementations of time-
table constraints in some open-source solvers and also the recent synchronized
sweep algorithm. The resources related to this work are available here http://
bit.ly/cumulativett.

References
1. Aggoun, A., Beldiceanu, N.: Extending chip in order to solve complex scheduling
and placement problems. Mathematical and Computer Modelling 17(7), 57–73
(1993)
2. Beldiceanu, N., Carlsson, M.: A New multi-resource cumulatives constraint
with negative heights. In: Van Hentenryck, P. (ed.) CP 2002. LNCS, vol. 2470,
pp. 63–79. Springer, Heidelberg (2002)
3. Bordeaux, L., Hamadi, Y., Vardi, M.Y.: An analysis of slow convergence in inter-
val propagation. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741, pp. 790–797.
Springer, Heidelberg (2007)
4. Charles Prud’homme, X.L., Fages, J.-G.: Choco3 documentation. TASC, INRIA
Rennes, LINA CNRS UMR 6241, COSLING S.A.S. (2014)
5. de Saint-Marcq, V.C., Schaus, P., Solnon, C., Lecoutre, C.: Sparse-sets for domain
implementation. In: CP Workshop on Techniques for Implementing Constraint
Programming Systems (TRICS), pp. 1–10 (2013)
 Simple and Scalable Time-Table Filtering for the Cumulative Constraint 157

6. Fahimi, H., Quimper, C.-G.: Linear-time filtering algorithms for the disjunctive
constraint. In: Twenty-Eighth AAAI Conference on Artificial Intelligence (2014)
7. Fischer, J., Heun, V.: Theoretical and practical improvements on the RMQ-
problem, with applications to LCA and LCE. In: Lewenstein, M., Valiente, G.
(eds.) CPM 2006. LNCS, vol. 4009, pp. 36–48. Springer, Heidelberg (2006)
8. Gay, S., Hartert, R., Schaus, P.: Time-table disjunctive reasoning for the cumula-
tive constraint. In: Michel, L. (ed.) CPAIOR 2015. LNCS, vol. 9075, pp. 157–172.
Springer, Heidelberg (2015)
9. Gecode Team. Gecode: Generic constraint development environment (2006).
http://www.gecode.org
10. Lecoutre, C., Hemery, F., et al.: A study of residual supports in arc consistency.
In: IJCAI, vol. 7, pp. 125–130 (2007)
11. Letort, A., Beldiceanu, N., Carlsson, M.: A Scalable Sweep Algorithm for the cumu-
lative Constraint. In: Milano, M. (ed.) Principles and Practice of Constraint Pro-
gramming. LNCS, pp. 439–454. Springer, Heidelberg (2012)
12. Or-tools Team. or-tools: Google optimization tools (2015). https://developers.
google.com/optimization/
13. OscaR Team. OscaR: Scala in OR (2012). https://bitbucket.org/oscarlib/oscar
14. Ouellet, P., Quimper, C.-G.: Time-Table extended-edge-finding for the cumulative
constraint. In: Schulte, C. (ed.) CP 2013. LNCS, vol. 8124, pp. 562–577. Springer,
Heidelberg (2013)
15. Vilı́m, P.: Timetable edge finding filtering algorithm for discrete cumulative
resources. In: Achterberg, T., Beck, J.C. (eds.) CPAIOR 2011. LNCS, vol. 6697,
pp. 230–245. Springer, Heidelberg (2011)
 General Bounding Mechanism for Constraint
Programs

Minh Hoàng Hà1 , Claude-Guy Quimper2(B) , and Louis-Martin Rousseau1

1
Department of Mathematics and Industrial Engineering and CIRRELT,
École Polytechnique de Montréal, C.P. 6079, Succursale Centre-ville,
Montreal, QC H3C 3A7, Canada
{minhhoang.ha,louis-martin.rousseau}@cirrelt.net
2
Département d’informatique et de génie logiciel and CIRRELT,
Université Laval, Quebec, Canada
[email protected]

Abstract. Integer programming (IP) is one of the most successful

approaches for combinatorial optimization problems. Many IP solvers
make use of the linear relaxation, which removes the integrality require-
ment on the variables. The relaxed problem can then be solved using
linear programming (LP), a very efficient optimization paradigm. Con-
straint programming (CP) can solve a much wider variety of problems,
since it does not require the problem to be expressed in terms of lin-
ear equations. The cost of this versatility is that in CP there is no easy
way to automatically derive a good bound on the objective. This paper
presents an approach based on ideas from Lagrangian decomposition
(LD) that establishes a general bounding scheme for any CP. We pro-
vide an implementation for optimization problems that can be formu-
lated with knapsack and regular constraints, and we give comparisons
with pure CP approaches. Our results clearly demonstrate the benefits
of our approach on these problems.

Keywords: Constraint programming · Automatic bounding ·

Lagrangian decomposition · Knapsack constraint · Regular constraint

1 Introduction
Constraint Programming (CP) is an efficient tool for complex decision problems
arising in industry, such as vehicle routing, scheduling, and resource allocation. CP
solvers essentially solve satisfiability problems, that is, they determine whether or
not there exists a solution to a given model. When applied to optimization prob-
lems, most CP solvers solve a sequence of satisfiability problems, requiring at each
step that the solution found improves on the solution found at the previous step.
The search stops when no feasible solution can be found, proving that the pre-
vious solution was indeed optimal. The ability to compute bounds on the objec-
tive function (the optimization criterion) is crucial. It allows faster termination
of the final subproblem (by proving infeasibility more quickly), and it speeds up

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 158–172, 2015.
DOI: 10.1007/978-3-319-23219-5 12
 General Bounding Mechanism for Constraint Programs 159

the solution of all the subproblems (because filtering and pruning become possible
when a good bound on the objective is known). In all cases where CP has success-
fully solved optimization problems, the modeler had to implement good bounding
mechanisms. The lack of a good general-purpose bounding scheme is one of the
main drawbacks of CP in comparison with integer programming (IP).
In CP, the constraints are independent, and information is communicated
solely through the domain store, the set of possible remaining values for each
variable. During the search, each constraint looks at the domain store, determines
whether a solution still exists, and filters out pairs of variables and values that
no longer lead to feasible solutions. This decoupling of the problem in terms of
the independent constraints allows us to use a simple and efficient combinatorial
algorithm for each constraint. A similar concept is used in linear programming
(LP): Lagrangian decomposition (LD) relaxes the link between difficult sets of
constraints and introduces in the objective a penalty vector that acts as the glue
linking the remaining relatively simple subproblems. In CP, we observe that
there is no such glue for the constraints (or subproblems), making any default
bound computation very weak. If we look at CP through LD glasses, we see that
the relaxed constraints are actually a set of implicit constraints stating that
any given variable should have the same value in all the different constraints in
which it appears. Therefore, to apply LD techniques to a CP model, we must
penalize that different constraints assume the same variable can take different
values during search. Our goal is to develop a general bounding mechanism for
CP that is compatible with every CP model and transparent to the user.
This paper is organized as follows. The next section reviews related work on
decomposition techniques and CP, and Section 3 presents our general bounding
approach. Section 4 evaluates the method on two optimization problems: a) the
knapsack problem and b) the simple shift scheduling problem that can be mod-
eled as a combination of many regular constraints. Finally, Section 5 summarizes
our conclusions.

2 Related Work
We present here some background on LD and the global CP constraints investi-
gated in this paper.

2.1 Lagrangian Decomposition

Decomposition techniques are an efficient approach for large-scale optimization

problems. One of the most popular techniques is Lagrangian relaxation (LR),
a natural approach that exploits the problem structure. It relaxes the “compli-
cated” constraints and penalizes the constraint violation in the objective func-
tion. This simplifies the solution procedure since the resulting problem is easier
to solve.
Several authors have used LR coupled with CP to solve optimization prob-
lems. Sellmann and Fahle [20] propose an algorithm for the automatic recording
160 M.H. Hà et al.

problem. They introduce two linear substructures, and they dualize the first and
propagate the second. At convergence, they use the optimal dual information to
propagate the first substructure. Ouaja and Richards [15] embed LR and CP
into a branch-and-bound algorithm for the traffic placement problem. At each
node of the search tree, CP is used to find a solution if one exists and to prove
infeasibility otherwise, and LD indicates how close the solution is to optimality.
Benoist et al. [2] propose hybrid algorithms combining LR and CP for the trav-
eling tournament problem, which includes round-robin assignment and travel
optimization. They show that LR provides not only good bounds to limit the
search but also information to guide new solutions or efficient branching. Sell-
mann [19] investigates the theoretical foundations of CP-based LR and proves
that suboptimal Lagrangian multipliers can have stronger filtering abilities than
optimal ones.
A drawback of LR is that the problem loses its original structure since the
complicated constraints are removed from the constraint set and penalized in the
objective function. Moreover, the penalized constraints must generally be linear.
This seriously restricts the application of LR in the context of CP where most
global constraints have a nonlinear structure. For example, it is not easy to dual-
ize some popular global constraints such as the all-different, global cardinality,
and regular constraints. To overcome this restriction, Fontaine et al. [8] general-
ize LR to arbitrary high-level models using the notion of satisfiability (violation)
degree. In this method, a satisfiability degree (or violation degree) is defined for
each hard constraint and penalized in the objective function instead of in the
constraint itself. The results show that LR coupled with CP can efficiently solve
some classes of graph coloring problems.
Another way to avoid the penalization of nonlinear hard constraints is to
use LD. This was first introduced by Guignard and Kim [10]; it is also called
variable splitting. It creates “copies” of a set of variables responsible for connect-
ing important substructures in the model, using one copy per substructure and
dualizing the equality of a variable and its copy. Since the substructures of the
original problem are retained, LD provides bounds that are always at least as
tight as those from LR. The method is particularly useful if there are no apparent
complicating constraints or the complicating constraints have nonlinear forms.
The research on integrating LD into CP is limited. To the best of our knowl-
edge, only Cronholm and Ajili [5] have studied this. They propose a hybrid
algorithm for the multicast network design problem. Their approach enables the
propagation of all the substructures at every dual iteration and also provides
strong cost-based filtering. We continue their theme with our attempt to use LD
to construct an automatic bounding mechanism, thus making CP tools more effi-
cient for optimization problems. To do this, we consider each global constraint
in the CP model as a substructure, and we connect them through LD to pro-
vide valid bounds. These are used to prune the nodes in the search tree. We
investigate two global constraints: knapsack and regular constraints.
 General Bounding Mechanism for Constraint Programs 161

2.2 Knapsack and Regular Constraints

In this subsection, we discuss knapsack and regular constraints and formulate

the related optimization problems.

Knapsack Constraint. Knapsack constraints are probably the most commonly

used constraints for problems in mixed integer programming and CP. These
linear constraints have the form wx ≤ W where W is scalar, x = [x1 , x2 , ..., xn ] is
a vector of n binary variables, and w = [w1 , w2 , ..., wn ] is a vector of n coefficients.
Most CP solvers filter knapsack constraints in a straightforward manner: the
domain reduction is based on interval arithmetic and simple bounding argu-
ments. This filtering is fast and effective but requires an important branching
in practice. Another knapsack filtering has been proposed by Trick [22], who
uses a dynamic programming structure to represent the constraint and achieves
hyper-arc consistency, thus determining infeasibility before all the variables are
set. The filtering is based on a layered directed graph where the rows correspond
to W values (0 through |W |), the columns represent variable indexes (0 through
n), and the arcs represent variable-value pairs. This graph has the property that
any path from (0, 0) to the nodes of the last layer corresponds to a feasible
solution to the knapsack constraint.
The filtering can reduce the branching when dealing with the satisfiability
version of the knapsack problem. This version, the market split problem, has no
objective function (see Trick [22] for more information). An effective implemen-
tation is essential to reduce the computational time. In the next sections, we
show that integrating the filtering proposed by Trick [22] with LD reduces both
the branching and the computational time for optimization problems.
Fahle and Sellmann [7] introduce an optimization version of the knap-
sack constraint. Given a lower bound B ∈ N and an vector of n coefficients
p = [p1 , p2 , ..., pn ], the constraint enforces not only wx ≤ W but also px ≥ B.
Since achieving generalized arc consistency for this constraint is NP-hard, Fahle
and Sellmann [7] introduce the notion of relaxed consistency for optimization
constraints, and they use bounds based on LP relaxations for polynomial-time
domain filtering. Using bounds with guaranteed accuracy, Sellmann [18] exploits
an existing approximation algorithm to provide an approximate consistency. Ka-
triel et al. [11] develop an efficient incremental version of the previous algorithm.
Malitsky et al. [13] generalize the method of [11] to provide a filtering algorithm
for a more general constraint, the bounded knapsack constraint, in which the
variables are integer.
The knapsack constraint is the main component of the class of knapsack
problems, which has many variants (see Kellerer et al. [12] for more details).
The simplest is the 0/1 knapsack problem. Given a set of n items, where each
item i has a weight wi and a value pi , we must determine which items to include
in the knapsack so that the total weight is less than or equal to a given limit W
and the total value is maximized. In this paper, we focus on the multidimensional
knapsack problem (MKP), which is as follows:
162 M.H. Hà et al.


n
Maximize pi xi (1)
i=1

n
subject to wij xi ≤ Cj j = 1, . . . , m (2)
i=1
xi ∈ {0, 1} (3)

Regular Constraint. The regular global constraint is generally specified by

using a (nondeterministic) finite automaton π, which is defined as a 5-tuple
π = (Q, A, τ, q0 , F ) where:
– Q is the finite set of states;
– A is the alphabet;
– τ : Q × A × Q is the transition table;
– q0 ∈ Q is the initial state;
– F ⊆ Q is a set of final states.
The constraint regular([X1 , . . . , Xn ], π) holds if the sequence of values of
the variables X1 , . . . , Xn is a member of the regular language recognized by a
deterministic finite automaton π. The recognition is confirmed if there exists a
sequence of states qi0 , . . . , qin such that qi0 = q0 , (qik , Xi , qik+1 ) is a transition
in τ , and qin ∈ F is a final state.
Pesant [16] introduced a filtering algorithm for the regular constraint. It
constructs a layered graph similar to that proposed for the knapsack constraint
above, except that its rows correspond to states of the automaton. This approach
was later extended to the optimization constraints Soft-regular [23], Cost-
regular [6], and Multicost-regular [14] to enforce bounds on the global
cost of the assignment. The underlying solution methods compute the shortest
and longest paths in an acyclic graph.
The regular global constraint is useful in modeling complex work regulations
in the shift scheduling problem (SSP), since the deterministic finite automa-
ton can specify the rules that regulate transitions in the sequence of activi-
ties [3,4,6,17]. The problem consists in scheduling a sequence of activities (work,
break, lunch, and rest activities) for a set of employees. We will investigate a
version of the SSP in which only one employee is considered and the work regula-
tions are so complex that modeling them requires many regular constraints. Let
W be a set of work activities, T a set of periods, Π a set of deterministic finite
automata expressing work regulations, and pij the profit for the assignment of
activity i ∈ W at period j ∈ T . We must assign one and only one activity for
each period such that the transition of the activities over the periods satisfies
the work regulations defined by the automata in Π and the total profit is max-
imized. To state the model for the SSP in CP, we use the element constraint
Element(Index, Table, Value), which holds if Value is equal to the Index
of the Table, i.e., value = Table[Index], to represent the assignment. Let
Xj be the variable representing the activity assigned to period j. The problem
is then:
 General Bounding Mechanism for Constraint Programs 163

Maximize Cj (4)
j∈T

subject to regular(X, πi ) ∀πi ∈ Π (5)

element(Xj , pij , Cj ) ∀i ∈ W, j ∈ T (6)
Xj ∈ W ∀j ∈ T (7)
Cj ∈ R, min pij ≤ Cj ≤ max pij ∀j ∈ T (8)
i∈W i∈W

where constraints (6) ensure that Cj is equal to pij if and only if Xj = i. This
specifies the profit of each assignment.

3 Proposed Approach
3.1 Lagrangian Decomposition
We first recall the LD approach. Consider the problem of computing max{z =
c x|C1 (x) ∧ C2 (x)}, where x is a set of variables, c is a set of coefficients, and
C1 (x) and C2 (x) are two arbitrary constraints. One can obtain a relaxation, i.e.,
an upper bound on the solution, as follows:

 ⏐   ⏐ 
⏐ ⏐
max z = c x⏐C1 (x) ∧ C2 (x) = max c x⏐C1 (x) ∧ C2 (y) ∧ x = y
 ⏐ 
⏐
= max c x + u (x − y)⏐C1 (x) ∧ C2 (y) ∧ x = y
 ⏐ 
⏐
≤ max c x + u (x − y)⏐C1 (x) ∧ C2 (y)
 ⏐   ⏐ 
⏐ ⏐
= max (c + u )x⏐C1 (x) + max −u y ⏐C2 (y)

In LD, a set of variables x is duplicated by introducing an identical set of

variables y, and the difference (x − y) is added to the objective function with a
violation penalty cost u ≥ 0, where u is the vector of Lagrangian multipliers. The
original problem is then relaxed to two subproblems: one with C1 and the other
with C2 . The two subproblems can be solved separately. Solving the resulting
programs with a given parameter u provides a valid upper bound on the original
objective function. The non-negative multipliers u that give the best bound
can be found by solving the Lagrangian dual. The decomposition can easily be
generalized to m constraints:
   
⏐ 
m ⏐ 
m m

 ⏐  1⏐ i i 1
max z = c x⏐ Ci (x) = max c x ⏐ Ci (x ) x =x
i=1 i=1 i=2
 
m
 ⏐ 
m m

 1  1 i ⏐ i i 1
= max c x + u(i) (x − x )⏐ Ci (x ) x =x
i=2 i=1 i=2
 
m
 ⏐ 
m
 1  ⏐1 ii
≤ max c x + u(i) (x − x )⏐ Ci (x )
i=2 i=1
  
m
 ⏐ m
 ⏐

  1⏐ 1  i⏐ i
= max c + u(i) x ⏐C1 (x ) + max −u(i) x ⏐Ci (x )
i=2 i=2
164 M.H. Hà et al.

This decomposition works well for numeric variables, i.e., variables whose
domains contain scalars. In CP, we often encounter domains that contain non-
numeric values. For instance, the regular constraint described in Section 2.2
applies to variables whose domains are sets of characters. In most applications,
it makes no sense to multiply these characters with a Lagrangian multiplier. In
this situation, we do not apply the LD method to the original variables Xi but
instead to a set of binary variables xi,v ∈ {0, 1} where xi,v = 1 ⇐⇒ Xi = v.
Rather than having a Lagrangian multiplier for each variable Xi , we instead
have a multiplier for each binary variable xi,v .
Finding the optimal multipliers is the main challenge in optimizing LD since
it is nondifferentiable. There are several approaches; the most common is the
subgradient method (see Shor et al. [21]). Starting from an arbitrary value of
the multipliers u0 , it solves subproblems iteratively for different values of u.
These are updated as follows:

uk+1 = uk + tk (y k − xk ) (9)

where the index k corresponds to the iteration number, y is a copy of variable

x, and tk is the step size. The step size is computed using the distance between
the objective value of the preceding iteration, Z k−1 , and the estimated optimum
Z ∗:
λ(Z k−1 − Z ∗ )
tk = 0 ≤ λ ≤ 2. (10)

y k−1 − xk−1
2
Here, λ is a scaling factor used to control the convergence; it is normally between
0 and 2.
Our approach integrates an automatic bounding mechanism into the branch-
and-bound algorithm of CP. The idea is that, at each node of the search tree, we
use LD to yield valid bounds. The approach divides the problem into many sub-
problems; each subproblem has one global constraint and an objective function
involving Lagrangian multipliers. For each subproblem, the solver must find the
assignment that satisfies the global constraint while optimizing a linear objective
function. It is generally possible to adapt the filtering algorithm of the global
constraints to obtain an optimal support. If a constraint takes as parameter a
cost for each pair for variable/value and it constrains the cumulative cost of the
assignments to be bounded, then this constraint is compatible with the bounding
mechanism we propose. Soft constraints such as Cost-GCC and Cost-Regular [23]
can therefore be used.

3.2 The Knapsack Constraint

For the MKP, the subproblems are 0/1 knapsack problems. They can be solved
via dynamic programming, which runs in pseudo-polynomial time O(nW ), where
n is the number of items and W is the size of the knapsack. We use the algorithm
by Trick [22] to filter the constraint, and we adapt it to compute the bound. Trick
constructs an acyclic graph with n + 1 layers where Lj contains the nodes of
 General Bounding Mechanism for Constraint Programs 165

layer j for 0 ≤ j ≤ n. Each node is labeled with a pair of integers: the layer
and the accumulated weight. At layer 0, we have a single node L0 = {(0, 0)}.
At layer j, we have Lj = {(j, k) | 1 ∈ dom(Xj ) ∧ (j − 1, k − wj ) ∈ Lj−1 ∨ 0 ∈
dom(Xj ) ∧ (j − 1, k) ∈ Lj−1 }. There is an edge labeled with value 1 between the
nodes (j − 1, k − wj ) and (j, k) whenever these nodes exist and an edge labeled
with value 0 between the nodes (j − 1, k) and (j, k) whenever these nodes exist.
Trick’s algorithm filters the graph by removing all the nodes and edges that do
not lie on a path connecting the node (0, 0) to a node (n, k) for 0 ≤ k ≤ C. After
the graph has been filtered, if no edges labeled with the value 1 remain between
layer j − 1 and layer j, this value is removed from the domain of xj . Similarly,
if no edges labeled with the value 0 remain between layer j − 1 and layer j, this
value is removed from the domain of xj .
We augment this algorithm by computing costs. We assign m to each edge
connecting node (j − 1, k − wj ) to node (j, k) a cost of cj + l=2 u(l, j) for the
graph representing the first constraint (i = 1), where cj is the value of item j
and m is the number of constraints; and a cost of −u(i, j) where i ≥ 2 is the
index of the constraint for the graph representing the remaining constraints. We
use dynamic programming to compute for each node (j, k) the cost of the longest
path connecting the source node (0, 0) to the node (j, k):
⎧
⎪
⎨0 if k = j = 0
M [j, k] = −∞ if k > j = 0
⎪
⎩
max(M [j − 1, k], M [j − 1, k − wj ] + addedCost) otherwise

m
where addedCost is equal to cj + l=2 u(l, j) if i = 1 and −u(i, j) if i > 1.
The optimal bound for this constraint is given by max0≤j≤W M [n, j]. Indeed,
any path in the filtered graph connecting the source node (0, 0) to a node
(n, k) for 0 ≤ k ≤ C corresponds to a valid assignment. The expression
max0≤j≤W M [n, j] returns the largest cost of such a path and therefore the
maximum cost associated with a solution of the knapsack constraint.

3.3 The Regular Constraint

We proceed similarly with the regular constraint. Pesant [16] presents an algo-
rithm also based on a layered graph. The set Lj contains the nodes at layer
j for 0 ≤ j ≤ n. Each node is labeled with an integer representing the layer
and a state of the automaton. The first layer contains a single node, and we
have L0 = {(0, q0 )}. The other layers, for 1 ≤ j ≤ n, contain the nodes
Lj = {(j, q2 ) | (j − 1, q1 ) ∈ Lj−1 ∧ a ∈ dom(Xj ) ∧ (q1 , a, q2 ) ∈ τ }. An edge
is a triplet (n1 , n2 , a) where n1 and n2 are two nodes and a ∈ A is a label. Two
nodes can be connected to each other with multiple edges having distinct labels.
The set of edges is denoted E. There is an edge connecting node (j−1, q1 ) to node
(j, q2 ) with label a whenever these nodes exist and (q1 , a, q2 ) ∈ τ ∧ a ∈ dom(Xj )
holds. As with Trick’s algorithm, Pesant filters the graph by removing all nodes
166 M.H. Hà et al.

and edges that do not lie on a path connecting the source node (0, q0 ) to a node
(n, q) where q ∈ F is a final state. If no edges with label a connecting a node in
Lj−1 to a node in Lj remain, the value a is pruned from the domain dom(Xj ).
This filtering algorithm can be augmented by associating a cost with each
edge. We assume that there is a Lagrangian multiplier u(i, j, a) for each binary
variable xja representing the assignment of variable Xj to character a ∈ A.
Here, i ≥ 1 represents the index of the constraint. An edge (q1 , a, q2 ) has a cost of
m
paj + l=2 u(l, j, a) in the graph induced by the first constraint (i = 1), and a cost
of −u(i, j, a) in the graph induced by the remaining constraints (i > 1). Using
dynamic programming, we can compute the cost of the longest path connecting
node (0, q0 ) to a node (j, q):

0 if j = 0
R[j, q] =
max((j−1,q1 ),(j,q),a)∈E (R[j − 1, q1 ] + addedCost) otherwise
m
where addedCost is equal to paj + l=2 u(l, j, a) if i = 1 and −u(i, j, a) if i > 1.
Since every path from layer 0 to layer n in the graph corresponds to a valid
assignment for the regular constraint, the optimal bound for this constraint is
given by maxq∈F R[n, q], i.e., the greatest cost for a valid assignment.

3.4 The Subgradient Procedure

We use the subgradient procedure to solve the Lagrangian dual. The estimated
optimum Z ∗ is set to the value of the best incumbent solution, and the scaling
factor λ is set to 2. The updating strategy halves λ when the objective function
does not improve in five iterations. At each node of the search tree, the number of
iterations for the subgradient procedure, which gives the best trade-off between
the running time and the number of nodes required in the search tree, is fixed
to 60 for the MKP in which the subproblem is solved by dynamic programming
and to 10 for all other cases. The subgradient procedure is terminated as soon
as the resulting Lagrangian bound is inferior to the value of the best incumbent
solution.
The initial multipliers u0 at the root node are fixed to 1. At the other nodes,
we use an inheritance mechanism, i.e., the value of u0 at a node is taken from
the multipliers of the parent node. This mechanism is better than always fixing
the initial multipliers to a given value. We have tested two strategies: we fix u0
either to the multipliers that give the best bound or to the last multipliers of the
parent node. The results show that the latter strategy generally performs better.
This is because the limited number of subgradient iterations at each node is not
sufficient to provide tight bounds for the overall problem. Therefore, the more
iterations performed at early nodes, the better the multipliers expected at later
nodes.

4 Computational Results
This section reports some experimental results. The goal is not to present state-
of-the-art results for specific problems but to show that LD could make CP tools
 General Bounding Mechanism for Constraint Programs 167

more efficient for optimization problems. The algorithms are built around CP
Optimizer 12.6 with depth-first search. All the other parameters of the solver
are set to their default values. The criteria used to evaluate the solutions are the
number of nodes in the search tree, the computational time, and the number of
instances successfully solved to optimality.
We investigate the behavior of the algorithms in two contexts: with and
without initial lower bounds. In case A, we provide the optimal solution to the
model by adding a constraint on the objective function, setting it to be at least
the value of the optimal solution; the goal is then to prove the optimality of
this value. In case B, we start the algorithms without an initial lower bound.
These two tests provide different insights, since the presence of different bound-
ing mechanisms will impact the search and branching decisions of the solver, with
unpredictable outcomes. However, once the solver has found the optimal solu-
tion, it must always perform a last search to demonstrate optimality. A better
bounding mechanism will always contribute to this phase of the overall search.

4.1 Results for MKP

The experiments in this subsection analyze the performance of LD with two

MKP solution methods: i) direct CP using only sum constraints, and ii) the
approach of Trick [22]. For i) the Lagrangian subproblems are solved by dynamic
programming, and for ii) they are solved using the filtering graph of Trick [22].
We selected two groups of small and medium MKP instances from the OR-
Library [1] for the test: Weing (8 problems, each with 2 constraints) and Weish
(30 problems, each with 5 constraints). We limited the computational time to
one hour.
Table 1 reports the results for case B and Table 2 reports the results for
case A. The instances not included in Table 1 could not be solved by any of
the algorithms; a dash indicates that the algorithm could not solve the instance
within the time limit.
As can be seen in Table 2, the LD approach processes much fewer nodes per
seconds, but the extra effort invested in bounding significantly decreases the size
of the search tree and the time to find the optimal solution, especially for the
larger instances. The most efficient method is CP + LD: it can solve 30 instances.
However, it generally requires more time to solve the small instances. Moreover,
it is quite time-consuming compared with the original method on a few instances,
e.g., Weish6, Weish7, Weish8, and Weing15. The Trick + LD approach improves
on Trick for all the criteria, even for the small instances. This is because solving
the Lagrangian subproblems based on the information available from the filtering
algorithm is computationally less costly. Moreover, the propagation performed
at each node of the search tree is limited for the direct CP formulation, and the
computation of the Lagrangian bound significantly increases this. On the other
hand, the Trick method relies on a filtering algorithm for which the complexity is
linear in the size of the underlying graph. In this case, the additional computation
for the LD bounds has less impact.
168 M.H. Hà et al.

Table 1. Results for MKP without initial lower bound

# CP CP + LD Trick Trick + LD
Instance Nodes
Vars Nodes Time Time Nodes Time Nodes
Time Nodes Time Nodes
Time Nodes Time Nodes
Time
Weing1 28 4424 1.19 3718 860 3.95 218 4424 6.19 715 1100 6.00 183
Weing2 28 5572 1.29 4319 744 3.07 242 5572 11.94 467 744 4.30 173
Weing3 28 8650 0.55 16k 270 0.97 278 8650 15.34 564 280 1.53 183
Weing4 28 4106 1.08 3802 538 2.62 205 4106 19.72 208 538 4.14 130
Weing5 28 13k 0.58 22k 262 1.00 262 12k 21.13 615 262 1.53 171
Weing6 28 9150 1.14 8026 876 3.59 244 9150 20.50 446 1012 4.83 210
Weing7 105 - - - 32k 3410.04 9 - - - - - -
Weing8 105 - - - 19k 147.98 128 - - - 19k 655.46 29
Weish1 30 35k 1.78 20k 1320 37.31 35 35k 910.82 38 1286 59.90 21
Weish2 30 40k 1.64 24k 1280 27.14 47 40k 2481.72 16 1384 57.47 24
Weish3 30 11k 0.83 13k 674 8.47 80 11k 669.01 16 760 24.35 31
Weish4 30 2342 0.80 2927 856 11.91 72 2342 186.47 13 826 29.68 28
Weish5 30 1614 0.90 1793 728 9.19 79 1614 149.78 11 644 23.74 27
Weish6 40 1.2M 12.56 96k 4286 369.13 12 - - - 3368 320.92 10
Weish7 40 901k 15.12 60k 3888 290.79 13 - - - 3482 352.67 10
Weish8 40 1.1M 19.74 56k 4004 256.00 16 - - - 3464 392.57 9
Weish9 40 144k 3.12 46k 2426 46.78 52 - - - 2212 191.83 12
Weish10 50 28M 589.20 48k 4486 264.40 17 - - - 3969 734.65 5
Weish11 50 6M 95.84 63k 3764 159.05 24 - - - 3764 595.38 6
Weish12 50 21M 355.78 59k 4738 307.37 15 - - - 3728 651.90 6
Weish13 50 20M 338.07 59k 4208 250.62 17 - - - 4374 908.53 5
Weish14 60 - - - 8424 645.04 13 - - - 8856 2472.92 4
Weish15 60 35M 720.98 49k 13k 1363.17 10 - - - 12k 3482.11 3
Weish16 60 - - - 14k 1216.45 12 - - - 13k 3324.81 4
Weish17 60 - - - 14k 1600.78 9 - - - - - -
Weish18 70 - - - - - - - - - - - -
Weish19 70 - - - 7750 667.02 12 - - - 8907 3174.07 3
Weish20 70 - - - 18k 2610.74 7 - - - - - -
Weish21 70 - - - 15k 1642.16 9 - - - - - -
Weish22 80 - - - 15k 2905.41 5 - - - - - -
Weish23 80 - - - 12k 2002.74 6 - - - - - -

Table 2 presents the results for case A. Here, the Lagrangian approaches
are even more efficient. The algorithms with LD can successfully solve all 38
instances, and the search tree is relatively small. The computational time of CP
+ LD is still worse than that of CP on a few small instances, but the difference
has decreased significantly. The main reason for this is that the optimal solutions
used to compute the step size make the subgradient procedure more stable and
cause it to converge more quickly. This observation suggests that adding good
lower bounds as soon as possible in the solution process will improve the method.
We computed at the root of the search tree the bound that the LD provides
as well as the bound returned by CPLEX using a linear relaxation (LP) and the
bound that CPLEX computes when the integer variables are not relaxed and all
cuts are added (ILP). Due to lack of space, we do not report the bounds for each
instance. In average, the bounds for LD, LP, and ILP are 1.03%, 0,79%, and 0.22%
greater than the optimal value. Even though LD does not provide the best bound,
it represents a significative improvement to CP whose bound is very weak.
 General Bounding Mechanism for Constraint Programs 169

4.2 Results for SSP

To generate the regular constraints, we use the procedure proposed by Pesant [16]
to obtain random automata. The proportion of undefined transitions in τ is set
to 30% and the proportion of final states to 50%. To control the problem size, we
create instances with only 2 regular constraints and 50 periods. The assignment
profits are selected at random from integer values between 1 and 100. Each
instance is then generated based on two parameters: the number of activities
(nva) and the number of states (nbs) in the automata. For each pair (nva,nbs),
we randomly generate three instances. The instances are labeled nbv-nbs-i, where

Table 2. Results for MKP: proving optimality

# CP CP + LD Trick Trick + LD
Instance Nodes
Vars Nodes Time Time Nodes Time Nodes
Time Nodes Time Nodes
Time Nodes Time Nodes
Time
Weing1 28 3408 0.21 16k 24 0.62 39 3408 5.25 649 24 0.78 31
Weing2 28 5070 0.21 24k 30 0.48 63 5070 16.36 310 30 0.74 41
Weing3 28 7004 0.55 13k 32 0.32 100 7004 11.78 595 34 0.42 81
Weing4 28 2344 0.14 17k 16 0.37 43 2344 10.45 224 16 0.81 20
Weing5 28 18k 0.33 55k 26 0.32 81 18k 35.30 510 26 0.56 46
Weing6 28 8038 0.22 37k 30 0.44 68 8038 27.62 291 30 0.80 38
Weing7 105 - - - 134 4 - - - 134 174.03 1
Weing8 105 - - - 168 3.54 47 - - - 520 40.95 13
Weish1 30 11k 0.27 41k 54 2.85 19 11k 447.18 25 70 5.72 12
Weish2 30 23k 0.43 53k 66 1.98 33 23k 1679.54 14 62 9.15 7
Weish3 30 8394 0.23 36k 38 2.80 14 8394 476.57 18 46 5.59 8
Weish4 30 632 0.14 4514 34 1.26 27 632 57.19 11 38 4.64 8
Weish5 30 556 0.12 4633 34 0.87 39 556 55.93 10 36 5.67 6
Weish6 40 861k 12.92 67k 56 5.54 10 - - - 72 16.58 4
Weish7 40 456k 7.07 64k 60 8.40 7 - - - 72 17.40 4
Weish8 40 707k 10.38 68k 60 9.05 7 - - - 54 19.52 3
Weish9 40 74k 1.26 59k 48 2.28 21 - - - 74 18.18 4
Weish10 50 8.6M 132.92 65k 134 29.53 5 - - - 192 51.11 4
Weish11 50 1.4M 22.75 62k 86 10.50 8 - - - 82 28.44 3
Weish12 50 9M 135.70 66k 114 24.92 5 - - - 122 38.23 3
Weish13 50 4.1M 60.90 67k 120 17.84 7 - - - 112 35.23 3
Weish14 60 - - - 104 27.97 4 - - - 330 123.89 3
Weish15 60 9.7M 173.65 56k 92 28.20 3 - - - 146 68.70 2
Weish16 60 - - - 128 37.95 3 - - - 450 192.18 2
Weish17 60 156M 3537.25 44k 90 34.36 3 - - - 176 115.41 1.5
Weish18 70 - - - 200 87.97 2 - - - 142 116.60 1.2
Weish19 70 - - - 200 86.98 2 - - - 320 228.97 1.4
Weish20 70 - - - 174 65.84 3 - - - 596 340.87 1.7
Weish21 70 - - - 134 48.17 3 - - - 354 225.53 1.6
Weish22 80 - - - 150 87.39 2 - - - 1068 1083.83 1.0
Weish23 80 - - - 158 83.39 1.9 - - - 177 350.55 0.5
Weish24 80 - - - 146 84.90 1.7 - - - 154 194.55 0.8
Weish25 80 - - - 238 117.47 2 - - - 586 539.24 1.1
Weish26 90 - - - 178 135.75 1.3 - - - 438 567.02 0.8
Weish27 90 - - - 152 94.48 1.6 - - - 258 356.33 0.7
Weish28 90 - - - 152 96.71 1.6 - - - 266 344.16 0.8
Weish29 90 - - - 152 104.87 1.4 - - - 416 601.38 0.7
Weish30 90 - - - 152 126.96 1.2 - - - 138 372.65 0.4
170 M.H. Hà et al.

Table 3. Results for SSP without initial lower bound

Pesant Pesant + LD
Instance
Nodes Time Nodes
Time Nodes Time
Nodes
Time
10-20-01 2.2M 1232.72 1785 156k 346.92 450
10-20-02 - - - 298k 741.35 402
10-20-03 1.6M 783.29 2043 158k 332.15 476

10-80-01 256k 1325.80 193 84k 1020.53 82

10-80-02 788k 3307.39 238 238k 2834.86 84
10-80-03 847k 2344.55 361 246k 2176.73 113

20-20-01 - - - 828k 1856.10 446

20-20-02 - - - 1.3M 3404.56 382
20-20-03 2.1M 2427.72 865 164k 439.97 373

20-80-01 - - - 373k 3944.18 95

20-80-02 - - - 436k 5206.81 84
20-80-03 - - - 228k 2561.64 89

Table 4. Results for SSP: proving optimality

Pesant Pesant + LD
Instance
Nodes Time Nodes Nodes
Time Nodes Time Time
10-20-01 87k 42.20 2062 2564 5.94 432
10-20-02 407k 342.43 1189 44k 108.30 406
10-20-03 71k 28.60 2483 4118 7.48 551

10-80-01 20k 118.97 168 4546 71.16 64

10-80-02 25k 81.39 307 5466 63.19 87
10-80-03 26k 85.67 303 4762 58.34 82

20-20-01 343k 176.38 1945 2651 13.41 198

20-20-02 372k 297.97 1248 10k 51.63 194
20-20-03 53k 44.28 1197 1353 7.43 182

20-80-01 105k 486.89 216 10k 147.63 68

20-80-02 216k 1648.28 131 13k 211.23 62
20-80-03 16k 128.74 124 5128 72.33 71

i is the instance number. Because SSP instances are more difficult than MKP
instances, the computational time is increased to 2 hours.
The experiment investigates the performance of LD when combined with the
method of Pesant [16] for the SSP. As for the MKP, we test two cases: case A
has initial lower bounds and case B does not. Tables 3 and 4 present the results,
which demonstrate the performance of our approach. The LD method clearly
improves the method of Pesant [16] for all three criteria. More precisely, for case
B, it can solve 6 additional instances, reducing the size of the search tree by a
factor of about 7 and the computational time by a factor of 1.5 on average. In
case A, it works even better, reducing the size of the search tree by a factor of
about 16 and the computational time by a factor of more than 4 on average.
 General Bounding Mechanism for Constraint Programs 171

This is because we can reuse the graph constructed by the filtering algorithm to
solve the Lagrangian subproblems.

5 Conclusions and Future Work

We have introduced an automatic bounding mechanism based on the LD concept
to improve the performance of CP for optimization problems. We tested the
approach on two problems, the MKP and a synthesized version of the SSP.
These rely on two well-known global constraints: knapsack and regular.
Our next step will be to apply the approach to other global constraints
to solve real problems. One candidate is the global cardinality constraint, for
which the subproblems can be solved in polynomial time with the aid of a
graph obtained from the filtering algorithm. In addition, our approach could
be improved in several ways. First, we could use the information obtained from
solving the Lagrangian dual to design a strong cost-based filtering, as proposed
by Cron- holm and Ajili [5]. Second, by checking the feasibility of the solutions of
the Lagrangian subproblems we could find good solutions that help to limit the
search and improve the convergence of the subgradient method. Moreover, we
could use the solutions of the subproblems to guide the search by branching on
the variable with the largest difference between itself and its copies. Finally, as
reported by Guignard [9], the subgradient approach can have unpredictable con-
vergence behavior, so a more suitable algorithm could improve the performance
of our approach.

Acknowledgement. This work was financed with a Google Research Award. We

would like to thank Laurent Perron for his support.

References
1. Beasley, J.E.: OR-library (2012). http://people.brunel.ac.uk/∼mastjjb/jeb/info.
html
2. Benoist, T., Laburthe, F., Rottembourg, B.: Lagrange relaxation and constraint
programming collaborative schemes for travelling tournament problems. In: CP-
AI-OR 2001, pp. 15–26. Wye College (2001)
3. Chapados, N., Joliveau, M., L’Ecuyer, P., Rousseau, L.-M.: Retail store schedul-
ing for profit. European Journal of Operational Research 239(3), 609–624 (2014).
doi:10.1016/j.ejor.2014.05.033
4. Côté, M.-C., Gendron, B., Quimper, C.-G., Rousseau, L.-M.: Formal languages for
integer programming modeling of shift scheduling problems. Constraints 16(1),
54–76 (2011). doi:10.1007/s10601-009-9083-2
5. Cronholm, W., Ajili, F.: Strong cost-based filtering for lagrange decomposition
applied to network design. In: Wallace, M. (ed.) CP 2004. LNCS, vol. 3258,
pp. 726–730. Springer, Heidelberg (2004). http://www.springerlink.com/index/
ur3uvyqbp0216btd.pdf
6. Demassey, S., Pesant, G., Rousseau, L.-M.: A cost-regular based hybrid column
generation approach. Constraints 11(4), 315–333 (2006). http://dblp.uni-trier.de/
db/journals/constraints/constraints11.html#DemasseyPR06
172 M.H. Hà et al.

7. Fahle, T., Sellmann, M.: Cost based filtering for the constrained knapsack problem.
Annals of OR 115(1–4), 73–93 (2002). doi:10.1023/A:1021193019522
8. Fontaine, D., Michel, L.D., Van Hentenryck, P.: Constraint-based lagrangian relax-
ation. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 324–339. Springer,
Heidelberg (2014)
9. Guignard, M.: Lagrangean relaxation. Top 11(2), 151–200 (2003). doi:10.1007/
BF02579036. ISSN: 1134–5764
10. Guignard, M., Kim, S.: Lagrangean decomposition: A model yielding stronger
Lagrangean bounds. Mathematical Programming 39(2), 215–228 (1987). doi:10.
1007/BF02592954. ISSN: 00255610
11. Katriel, I., Sellmann, M., Upfal, E., Van Hentenryck, P.: Propagating knapsack
constraints in sublinear time. In: Proceedings of the Twenty-Second AAAI Confer-
ence on Artificial Intelligence, Vancouver, British Columbia, Canada, July 22–26,
pp. 231–236 (2007). http://www.aaai.org/Library/AAAI/2007/aaai07-035.php
12. Kellerer, H., Pferschy, U., Pisinger, D.: Knapsack Problems. Springer, Berlin (2004)
13. Malitsky, Y., Sellmann, M., Szymanek, R.: Filtering bounded knapsack constraints
in expected sublinear time. In: Proceedings of the Twenty-Fourth AAAI Conference
on Artificial Intelligence, AAAI 2010, Atlanta, Georgia, USA, July 11–15, 2010.
http://www.aaai.org/ocs/index.php/AAAI/AAAI10/paper/view/1855
14. Menana, J., Demassey, S.: Sequencing and counting with the multicost-regular
constraint. In: van Hoeve, W.-J., Hooker, J.N. (eds.) CPAIOR 2009. LNCS, vol.
5547, pp. 178–192. Springer, Heidelberg (2009). doi:10.1007/978-3-642-01929-6 14.
ISBN: 3642019285
15. Ouaja, W., Richards, B.: A hybrid multicommodity routing algorithm for traf-
fic engineering. Networks 43(3), 125–140 (2004). http://dblp.uni-trier.de/db/
journals/networks/networks43.html#OuajaR04
16. Pesant, G.: A regular language membership constraint for finite sequences of vari-
ables. In: Wallace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 482–495. Springer,
Heidelberg (2004). http://www.springerlink.com/content/ed24kyhg561jjthj
17. Quimper, C.-G., Rousseau, L.-M.: A large neighbourhood search approach to the
multi-activity shift scheduling problem. J. Heuristics 16(3), 373–392 (2010). doi:10.
1007/s10732-009-9106-6
18. Sellmann, M.: Approximated consistency for knapsack constraints. In: Rossi, F.
(ed.) CP 2003. LNCS, vol. 2833, pp. 679–693. Springer, Heidelberg (2003). doi:10.
1007/978-3-540-45193-8 46
19. Sellmann, M.: Theoretical foundations of CP-based lagrangian relaxation. In: Wal-
lace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 634–647. Springer, Heidelberg (2004).
http://dblp.uni-trier.de/db/conf/cp/cp/2004.html#sellmann.ISBN:3-540-23241-9
20. Sellmann, M., Fahle, T.: Constraint programming based Lagrangian relaxation
for the automatic recording problem. Annals of Operations Research 118, 17–33
(2003). doi:10.1023/A:1021845304798. ISBN: 0254-5330
21. Shor, N.Z., Kiwiel, K.C., Ruszcayński, A., Ruszcayński, A.: Minimization methods
for non-differentiable functions. Springer-Verlag New York Inc., New York (1985).
ISBN: 0-387-12763-1
22. Trick, M.A.: A dynamic programming approach for consistency and prop-
agation for knapsack constraints. Annals of OR 118(1–4), 73–84 (2003).
http://dblp.uni-trier.de/db/journals/anor/anor118.html#Trick03
23. van Hoeve, W.J., Pesant, G., Rousseau, L.-M.: On global warming: Flow-
based soft global constraints. J. Heuristics 12(4–5), 347–373 (2006). doi:10.1007/
s10732-006-6550-4
 Smallest MUS Extraction
with Minimal Hitting Set Dualization

Alexey Ignatiev1 , Alessandro Previti2 , Mark Liffiton3 ,

and Joao Marques-Silva1,2
1
INESC-ID, IST, Technical University of Lisbon, Lisboa, Portugal
2
CASL, University College Dublin, Belfield, Ireland
3
Illinois Wesleyan University, Bloomington, IL, USA

Abstract. Minimal explanations of infeasibility are of great interest

in many domains. In propositional logic, these are referred to as Mini-
mal Unsatisfiable Subsets (MUSes). An unsatisfiable formula can have
multiple MUSes, some of which provide more insights than others. Dif-
ferent criteria can be considered in order to identify a good minimal
explanation. Among these, the size of an MUS is arguably one of the
most intuitive. Moreover, computing the smallest MUS (SMUS) finds
several practical applications that include validating the quality of the
MUSes computed by MUS extractors and finding equivalent subformulae
of smallest size, among others. This paper develops a novel algorithm for
computing a smallest MUS, and we show that it outperforms all the pre-
vious alternatives pushing the state of the art in SMUS solving. Although
described in the context of propositional logic, the presented technique
can also be applied to other constraint systems.

1 Introduction

For inconsistent formulae, finding a minimal explanation of their infeasibility

is a central task in order to disclose the source of the problem. In general, an
inconsistent formula can have multiple explanations. Thus, defining a measure of
the quality of an explanation becomes necessary in order to focus on those pro-
viding more insights. For the case of a propositional formula, a natural measure
of explanation quality is the size of the computed minimal unsatisfiable subsets
(MUSes). From a query complexity perspective, extracting an MUS is in FPNP .
In contrast, deciding whether there exists an MUS of size less than or equal to
k is Σ2P -complete [8,12]. As a consequence, extracting a smallest MUS (SMUS)
P
is in FPΣ2 .
Computing an SMUS is central to a number of practical problems, includ-
ing validating MUSes computed with modern MUS extractors and finding an
equivalent subformula of minimum size.

This work is partially supported by SFI PI grant BEACON (09/IN.1/ I2618),

FCT grant POLARIS (PTDC/EIA-CCO/123051/2010) and national funds through
Fundação para a Ciência e a Tecnologia (FCT) with reference UID/CEC/50021/2013.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 173–182, 2015.
DOI: 10.1007/978-3-319-23219-5 13
174 A. Ignatiev et al.

This paper shows how recent work on iterative enumeration of hitting sets can
be adapted to computing the smallest MUS of unsatisfiable formulae, improving
upon state-of-the-art algorithms. The approach is implemented and tested on
Boolean Satisfiability instances, but the technique applies equally well to com-
puting SMUSes of any constraint system for which the satisfiability of all subsets
is well-defined.
The paper is organized as follows. In Section 2 basic definitions are provided.
Section 3 introduces an algorithm for the extraction of an SMUS. In Section 4
some optimizations are presented. Finally, Section 5 is dedicated to the presen-
tation of the experimental results, and Section 6 concludes the paper.

2 Preliminaries
This section provides common definitions that will be used throughout the paper.
Additional standard definitions are assumed (e.g. see [3]). In what follows, F
denotes a propositional formula expressed in Conjunctive Normal Form (CNF).
A formula in CNF is a conjunction of clauses, where each clause is a disjunction
of literals. A literal is either a Boolean variable or its complement. Also, we may
refer to formulae as sets of clauses and clauses as sets of literals. An assignment
to variables that satisfies formula F is said to be a model of F. A formula is
unsatisfiable when no model exists. Unless specified explicitly, F is assumed to
be unsatisfiable. The following definitions also apply.
Definition 1. A subset U ⊆ F is a minimal unsatisfiable subset (MUS) if U is
unsatisfiable and ∀U  ⊂ U, U  is satisfiable. An MUS U of F of the smallest size
is called a smallest MUS (SMUS).
Definition 2. A subset C of F is a minimal correction subset (MCS) if F \ C
is satisfiable and ∀C  ⊆ C ∧ C  = ∅, (F \ C) ∪ C  is unsatisfiable.
Definition 3. A satisfiable subset S ⊆ F is a maximal satisfiable subset (MSS)
if ∀S  ⊆ F s.t. S ⊂ S  , S  is unsatisfiable.
An MSS can also be defined as the complement of an MCS (and vice versa).
If C is an MCS, then S = F \ C represents an MSS. On the other hand, MUSes
and MCSes are related by the concept of minimal hitting set.
Definition 4. Given a collection Γ of sets from a universe U, a hitting set h
for Γ is a set such that ∀S ∈ Γ, h ∩ S = ∅.
A hitting set h is minimal if none of its subset is a hitting set. The minimal
hitting set duality between MUSes and MCSes is well-known (e.g. see [15,17]):
Proposition 1. Given a CNF formula F, let MUSes(F) and MCSes(F) be the
set of all MUSes and MCSes of F, respectively. Then the following holds:
1. A subset U of F is an MUS iff U is a minimal hitting set of MCSes(F).
2. A subset C of F is an MCS iff C is a minimal hitting set of MUSes(F).
The duality relating MUSes and MCSes is a key aspect of the algorithm
presented below. In the next section, we will describe how the smallest MUS can
be computed by exploiting this observation.
 Smallest MUS Extraction with Minimal Hitting Set Dualization 175

Algorithm 1. Basic SMUS algorithm

Input: CNF formula F
1 begin
2 H←∅
3 while true do
4 h ← MinimumHS(H)
5 F  ← {ci | ei ∈ h}
6 if not SAT(F  ) then
7 return SMUS ← F 
8 else
9 C ← grow(F  )
10 H ← H ∪ {C}
11 end

3 Basic Algorithm

This section describes the new SMUS algorithm. We start by providing the
definition of a minimum size hitting set:

Definition 5. Let Γ be a collection of sets and MHS(Γ ) the set of all minimal
hitting sets on Γ . Then a hitting set h ∈ M HS(Γ ) is said to be a minimum
hitting set if ∀ h ∈ M HS(Γ ) we have that |h| ≤ |h |.

The algorithm is based on the following observation:

Proposition 2. A set U ⊆ F is an SMUS of F if and only if U is a minimum
hitting set of MCSes(F).
By duality [17], we also have that the minimum hitting set of MUSes(F)
corresponds to a MaxSAT solution for F. This observation has already been
exploited in [6]. Algorithm 1 can be seen as the dual of the algorithm presented
in [6]. H represents a collection of sets, where each set corresponds to an MCS
on F. Thus, elements of the sets in H represent clauses of F. Let elm(H) denote
the set of elements in H. Each element e ∈ elm(H) is associated with a clause
c ∈ F. At the beginning of the algorithm H is empty (line 2). At each step, a
minimum hitting set h is computed on H (see line 4) and the induced formula F 
is tested for satisfiability at line 6. If F  is satisfiable, it is extended by the grow
procedure into an MSS containing F  , the complement of which is returned as
the MCS C. Then C is added to the collection H. If instead F  is unsatisfiable
then F  is guaranteed to be an SMUS of F as the following lemma states:

Lemma 1. Let K ⊆ MCSes(F). Then a subset U of F is an SMUS if U is a

minimum hitting set on K and U is unsatisfiable.

Proof. Since U is unsatisfiable it means that it already hits every MCS in

MCSes(F) (Proposition 1). U is also a minimum hitting set on MCSes(F), since
176 A. Ignatiev et al.

c1 = ¬s c2 = s ∨ ¬p c3 = p
c5 = ¬m ∨ n c6 = ¬n
c4 = ¬p ∨ m

c7 = ¬m ∨ l

c8 = ¬l

Fig. 1. Formula example

Table 1. Example SMUS computation

MinimumHS(H) SAT(F  ) F \ grow(F  , F ) H=H∪C

F  ← {∅} true C ← {c3 } {{c3 }}

F  ← {c3 } true C ← {c2 , c4 } {{c3 }, {c2 , c4 }}

F  ← {c2 , c3 } true C ← {c1 , c4 } {{c3 }, {c2 , c4 }, {c1 , c4 }}

F  ← {c3 , c4 } true C ← {c1 , c5 , c7 } {{c3 }, {c2 , c4 }, {c1 , c4 }, {c1 , c5 , c7 }}

F  ← {c1 , c2 , c3 } false {c1 , c2 , c3 } is an SMUS of F

it is a minimum hitting set for K ⊆ MCSes(F) and no other added MCS can
make it grow in size. Moreover, all the other hitting sets can either grow in size
or remain the same. Thus, by Proposition 2 U must be an SMUS. 

Lemma 1 states that it is not necessary to enumerate all MCSes of formula F.
Instead, it is enough to compute only those whose minimum hitting set is an
MUS. Therefore, Algorithm 1 terminates once the first MUS is computed, which
is by construction guaranteed to be of the smallest size.
It is worth noting that nothing in Algorithm 1 is specific to Boolean CNF,
and in fact the algorithm can be applied to any type of constraint system for
which the satisfiability of constraint subsets can be checked. The algorithm and
all following additions to it are constraint agnostic.
An example of a run of Algorithm 1 is shown in Table 1 (Figure 1 illustrates
the input formula). The first column contains the formula F  induced by the
minimum hitting set on H. Whenever F  is satisfiable (second column), the grow
procedure (see line 9 of Algorithm 1) returns an MCS (third column). The last
 Smallest MUS Extraction with Minimal Hitting Set Dualization 177

column shows the current H, the collected set of MCSes from which a minimum
hitting set will be found in the next iteration. In the last row, F  = {c1 , c2 , c3 }
and since the call SAT(F  ) returns false, set {c1 , c2 , c3 } represents an SMUS.
Notice that MinimumHS(H) could have returned {c3 , c4 , c5 } instead of {c1 , c2 , c3 }.
In this case, further iterations would be necessary in order to find an SMUS.

4 Additional Details

This section describes some essential details of the approach being proposed.
These include computing a minimum hitting set with at most one SAT call,
enumerating disjoint MCSes, and reporting an upper bound within a reasonably
short time.

4.1 Reducing the Number of SAT Calls

A number of different approaches can be used for computing minimum size

hitting sets [6]. This section describes a different alternative that exploits the
use of SAT solvers as well as the problem structure. The proposed approach
exploits incremental MaxSAT solving [5].
Let the current minimum hitting set (MHS) h have size k, and let C be a
new set to hit. Call a SAT solver on the resulting formula, requesting an MHS
of size k. If the formula is satisfiable, then we have a new MHS of size k that
also hits C. However, if the formula is unsatisfiable, this means there is no MHS
of size k. Thus, an MHS of size k + 1 can be constructed by adding to h any
element of C. Clearly, by construction, h does not hit any element of C. Thus,
every MHS can be obtained with a single call to a SAT solver.
Additionally, extraction of each MCS C (see line 9) requires a number of
SAT calls. Alternatively, one can avoid the computation of MCS C and use
any correction subset C  s.t. C ⊆ C  instead. Indeed, any MHS of the set of all
correction subsets of F also hits every MCS of F [17]. Moreover, observe that
a proposition similar to Lemma 1 can be proved for a partial set of correction
subsets of F. Therefore, the grow procedure can be skipped, and the complement
to F  can be directly added to H.

4.2 Disjoint MCS Enumeration

Enumeration of disjoint inconsistencies has been used in various settings in recent

years. For example, disjoint unsatisfiable core enumeration in MaxSAT was pro-
posed in [16], and nowadays it is often used in state-of-the-art MaxSAT solvers
(e.g. see [7]). Enumeration of disjoint MCSes, which act as unsatisfiable cores
over quantified constraints, has also proven its relevance for the SMUS problem
in the context of QMaxSAT [9]. Also note that disjoint MCSes are known to
have a strong impact on the performance of the branch-and-bound algorithms
for SMUS by refining the lower bound on the size of the SMUS [13].
178 A. Ignatiev et al.

Our approach based on the minimum hitting set duality [15,17] between
MCSes and MUSes of an unsatisfiable CNF formula F can also make use of dis-
joint MCSes. Indeed, since each MCS must be hit by any MUS of a CNF formula,
disjoint MCSes must be hit by the smallest MUS of the formula. Therefore, given
a set of disjoint MCSes D, one can initialize set H with D instead of the empty
set (see line 2 of Algorithm 1) in order to boost the performance of Algorithm 1.
Note that this also simplifies the computation of the first minimum hitting set,
which for a set of disjoint MCSes of size k is exactly of size k. According to the
experimental results described in Section 5, this improvement has a huge impact
on the overall performance of the proposed approach.

4.3 Finding Approximate Solutions

Although the approach being proposed performs better than the known alter-
native approaches to SMUS, this problem is computationally much harder than
extracting any MUS of a CNF formula (the decision version of the SMUS problem
is known to be Σ2P -complete, e.g. see [8,12]). One may find this complexity char-
acterization a serious obstacle for using SMUS algorithms in practice. Indeed, a
user may prefer to find an MUS close to the smallest size within a reasonable
amount of time instead of waiting until the smallest MUS is found.
In order to resolve this issue and following the ideas of [10], Algorithm 1
can be extended for computing a “good” upper bound on the exact solution of
the SMUS problem. Any MUS can be considered as an upper bound on the
smallest MUS. Therefore and since extracting one MUS is a relatively simple
task, enumerating MUSes within a given time limit and choosing the smallest
one among them can be satisfactory for a user. MUS enumeration can be done in
a way similar to the one proposed in [14]. Observe that Algorithm 1 iteratively
refines a lower bound of the SMUS, and so can serve to measure the quality of
approximate upper bounds.
This pragmatic policy of computing an upper bound before computing the
exact solution provides a user with a temporary approximate answer within a
short period of time and continues with computing the exact solution, if needed
(i.e. if the user is not satisfied with the quality of the upper bound). Otherwise,
the upper bound is enough and the user can stop the process.

5 Experimental Results

This section evaluates the approach to the smallest MUS problem proposed
in this paper. The experiments were performed in Ubuntu Linux on an Intel
Xeon E5-2630 2.60GHz processor with 64GByte of memory. The time limit was
set to 800s and the memory limit to 10GByte. The approach proposed above was
implemented in a prototype called forqes (FORmula QuintESsence extractor ).
The underlying SAT solver of forqes is Glucose 3.01 [1]. A weakened version

1
Available from http://www.labri.fr/perso/lsimon/glucose/
 Smallest MUS Extraction with Minimal Hitting Set Dualization 179

800
vbs
700 forqes
minuc
600
digger
500 forqes-w
CPU time (s)

400

300

200

100

0
300 350 400 450 500
instances

Fig. 2. Cactus plot showing the performance of forqes, minuc, and digger

of forqes, which does not use disjoint MCS enumeration and implements Algo-
rithm 1, is referred to as forqes-w.
The performance of forqes was compared to the state-of-the-art approach
to the SMUS problem that uses the quantified MaxSAT formulation of SMUS [9].
The most efficient version of the tool described in [9] performs core-guided
QMaxSAT solving; in the following it is referred to as minuc. Additionally,
a well-known branch-and-bound SMUS extractor called digger (see [13]) was
also considered. Note that the versions of minuc and digger participating in
the evaluation make use of disjoint MCS enumeration.
Several sets of benchmarks were used to assess the efficiency of the new algo-
rithm, all used in the evaluation in [9] as well. This includes a collection of
instances from automotive product configuration benchmarks [18] and two sets
of circuit diagnosis instances. Additionally, we selected instances from the com-
plete set of the MUS competitions benchmarks2 as follows. Because extracting a
smallest MUS of a CNF formula is computationally much harder than extracting
any MUS of the formula, the instances that are difficult for a state-of-the-art
MUS extractor were excluded. Instead, we considered only formulae solvable by
the known MUS extractor muser-2 (e.g. see [2]) within 10 seconds. The total
number of instances considered in the evaluation is 682.
Figure 2 shows a cactus plot illustrating the performance of the tested solvers
on the total set of instances. forqes exhibits the best performance, being able
to solve 483 instances. minuc comes second with 448 instances solved. Thus,
forqes solves 7.8% more instances than minuc. digger and forqes-w have
almost the same performance solving 371 and 367 instances, respectively.

2
See http://www.satcompetition.org/2011/
180 A. Ignatiev et al.

103 103
800 sec. timeout 800 sec. timeout

102 102

101 101
digger

minuc
100 100

10−1 10−1

800 sec. timeout

800 sec. timeout

10−2 10−2

10−3 −3 10−3 −3
10 10−2 10−1 100 101 102 103 10 10−2 10−1 100 101 102 103
forqes forqes

(a) forqes vs digger (b) forqes vs minuc

Fig. 3. Performance of forqes, minuc, and digger

More details on the solvers’ performance can be found in Figure 3a, Figure 3b,
and Figure 4a. Also note that the virtual best solver (vbs) among all the con-
sidered solvers is able to solve 485 instances, which is only 2 more than what
forqes can solve on its own. Interestingly, neither minuc nor digger contribute
to the vbs. Although the weakened version of forqes (forqes-w) performs
quite poorly, it is the only solver (besides the normal version of forqes) that
contributes to the vbs. This can be also seen in Figure 4a.
As it was described in Section 4.3, the approach being proposed can report
an upper bound, which can be used as an approximation of the SMUS if finding
the exact solution is not efficient and requires too much time. The following
evaluates the quality of the upper bound reported by this pragmatic solving
strategy. Given an upper bound U B computed within some time limit and the
optimal value opt ≤ U B, the closer value Uopt B
is to 1 the better the quality of
U B is. Since it is often hard to find the exact optimal value, one can consider
a lower bound on the exact solution instead of the optimal value in order to
estimate the quality of the upper bound. Indeed, any set of disjoint MCSes found
within a given timeout can be seen as a lower bound on the exact solution.
Given a CNF formula, an upper bound on its smallest MUS reported by
forqes within a time limit is computed with an external call to a known MUS
enumerator called marco [14]. Several timeout values were tested, namely 5,
10, and 20 seconds. Figure 4b shows a cactus plot illustrating the quality of the
upper bounds computed this way. It is not surprising that generally the more
time is given to the solver, the better upper bound is computed. For about 400
instances, value U B
LB is extremely close to 1 meaning that the upper bound is
usually almost equal to the lower bound. As one can see, the upper bound is
less than an order of magnitude larger than the lower bound for about 550, 585,
and 620 instances if computed within 5, 10, and 20 seconds, respectively. Also
 Smallest MUS Extraction with Minimal Hitting Set Dualization 181

103 103
800 sec. timeout
t = 20s
2
t = 10s
10
t = 5s

Upper bound quality (U B/LB)

101 102
forqes-w

100

10−1 101

800 sec. timeout

10−2

10−3 −3 100
10 10−2 10−1 100 101 102 103 400 450 500 550 600 650
forqes instances

(a) forqes vs forqes-w (b) UB quality in 5, 10, and 20 sec

Fig. 4. Performance of forqes-w and UB quality

note that both lower and upper bounds are relatively easy to compute, e.g. both
of them can be computed almost for 650 instances (out of 682). Moreover, they
give an idea of how large the interval is between them. Given this information,
a user can decide how critical it is to compute the exact solution.
In summary, the experimental results indicate that the proposed approach
pushes the state of the art in SMUS solving, outperforming all the previous
approaches in terms of the number of solved instances. Moreover, the scatter
plots indicate that in most of the cases the proposed approach is also the fastest
in comparison to others. Moreover, a pragmatic policy to report an upper bound
within a short period of time can be helpful when it is hard to compute the exact
solution and provide a user with an approximate solution of the SMUS problem
of reasonable size.

6 Conclusion

This paper adapts recent algorithms for implicit set problems [4,6,11,19] to the
case of computing the smallest MUS. A number of enhancements are developed
and added to the new algorithm. Experimental results, obtained on represen-
tative problems instances, show clear gains over what currently represents the
state of the art. A natural line of research is to apply the novel SMUS algorithm
in concrete practical applications of SMUSes, e.g. finding smallest equivalent
subformulae of smallest size.
182 A. Ignatiev et al.

References
1. Audemard, G., Lagniez, J.-M., Simon, L.: Improving glucose for incremental
SAT solving with assumptions: application to MUS extraction. In: Järvisalo, M.,
Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 309–317. Springer,
Heidelberg (2013)
2. Belov, A., Lynce, I., Marques-Silva, J.: Towards efficient MUS extraction. AI Com-
mun. 25(2), 97–116 (2012)
3. Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability.
Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press (2009)
4. Chandrasekaran, K., Karp, R.M., Moreno-Centeno, E., Vempala, S.: Algorithms
for implicit hitting set problems. In: SODA, pp. 614–629 (2011)
5. Cimatti, A., Griggio, A., Schaafsma, B.J., Sebastiani, R.: A modular approach
to MaxSAT modulo theories. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013.
LNCS, vol. 7962, pp. 150–165. Springer, Heidelberg (2013)
6. Davies, J., Bacchus, F.: Solving MAXSAT by solving a sequence of simpler SAT
instances. In: Lee, J. (ed.) CP 2011. LNCS, vol. 6876, pp. 225–239. Springer,
Heidelberg (2011)
7. Davies, J., Bacchus, F.: Exploiting the power of mip solvers in maxsat. In:
Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS, vol. 7962, pp. 166–181.
Springer, Heidelberg (2013)
8. Gupta, A.: Learning Abstractions for Model Checking. PhD thesis, Carnegie Mel-
lon University, June 2006
9. Ignatiev, A., Janota, M., Marques-Silva, J.: Quantified maximum satisfiability: a
core-guided approach. In: Järvisalo, M., Van Gelder, A. (eds.) SAT 2013. LNCS,
vol. 7962, pp. 250–266. Springer, Heidelberg (2013)
10. Ignatiev, A., Janota, M., Marques-Silva, J.: Towards efficient optimization in pack-
age management systems. In: ICSE, pp. 745–755 (2014)
11. Karp, R.M.: Implicit hitting set problems and multi-genome alignment. In: Amir,
A., Parida, L. (eds.) CPM 2010. LNCS, vol. 6129, pp. 151–151. Springer, Heidelberg
(2010)
12. Liberatore, P.: Redundancy in logic I: CNF propositional formulae. Artif. Intell.
163(2), 203–232 (2005)
13. Liffiton, M.H., Mneimneh, M.N., Lynce, I., Andraus, Z.S., Marques-Silva, J.,
Sakallah, K.A.: A branch and bound algorithm for extracting smallest minimal
unsatisfiable subformulas. Constraints 14(4), 415–442 (2009)
14. Liffiton, M.H., Previti, A., Malik, A., Marques-Silva, J.: Fast, flexible MUS enu-
meration. Constraints (2015). http://dx.doi.org/10.1007/s10601-015-9183-0
15. Liffiton, M.H., Sakallah, K.A.: Algorithms for computing minimal unsatisfiable
subsets of constraints. J. Autom. Reasoning 40(1), 1–33 (2008)
16. Marques-Silva, J., Planes, J.: Algorithms for maximum satisfiability using unsat-
isfiable cores. In: DATE, pp. 408–413 (2008)
17. Reiter, R.: A theory of diagnosis from first principles. Artif. Intell. 32(1), 57–95
(1987)
18. Sinz, C., Kaiser, A., Küchlin, W.: Formal methods for the validation of automotive
product configuration data. AI EDAM 17(1), 75–97 (2003)
19. Stern, R.T., Kalech, M., Feldman, A., Provan, G.M.: Exploring the duality in
conflict-directed model-based diagnosis. In: AAAI, pp. 828–834 (2012)
 Upper and Lower Bounds on the Time
Complexity of Infinite-Domain CSPs

Peter Jonsson and Victor Lagerkvist(B)

Department of Computer and Information Science, Linköping University,

Linköping, Sweden
{peter.jonsson,victor.lagerkvist}@liu.se

Abstract. The constraint satisfaction problem (CSP) is a widely stud-

ied problem with numerous applications in computer science. For infinite-
domain CSPs, there are many results separating tractable and NP-hard
cases while upper bounds on the time complexity of hard cases are virtu-
ally unexplored. Hence, we initiate a study of the worst-case time cmplex-
ity of such CSPs. We analyse backtracking algorithms and show that they
can be improved by exploiting sparsification. We present even faster algo-
rithms based on enumerating finite structures. Last, we prove non-trivial
lower bounds applicable to many interesting CSPs, under the assumption
that the strong exponential-time hypothesis is true.

1 Introduction

The constraint satisfaction problem over a constraint language Γ (CSP(Γ )) is

the problem of finding a variable assignment which satisfies a set of constraints,
where each constraint is constructed from a relation in Γ . This problem is a
widely studied computational problem and it can be used to model many classi-
cal problems such as k-colouring and the Boolean satisfiability problem. In the
context of artificial intelligence, CSPs have been used for formalizing a wide
range of problems, cf. Rossi et al. [30]. Efficient algorithms for CSP problems
are hence of great practical interest. If the domain D is finite, then a CSP(Γ )
instance I with variable set V can be solved in O(|D||V | · poly(||I||)) time by
enumerating all possible assignments. Hence, we have an obvious upper bound
on the time complexity. This bound can, in many cases, be improved if addi-
tional information about Γ is known, cf. the survey by Woeginger [36] or the
textbook by Gaspers [14]. There is also a growing body of literature concerning
lower bounds [16,20,21,33].
When it comes to CSPs over infinite domains, there is a large number of
results that identify polynomial-time solvable cases, cf. Ligozat [23] or Rossi
et al. [30]. However, almost nothing is known about the time complexity of
solving NP-hard CSP problems. One may conjecture that a large number of
practically relevant CSP problems do not fall into the tractable cases, and this
motivates a closer study of the time complexity of hard problems. Thus, we
initiate such a study in this paper. Throughout the paper, we measure time

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 183–199, 2015.
DOI: 10.1007/978-3-319-23219-5 14
184 P. Jonsson and V. Lagerkvist

complexity in the number of variables. Historically, this has been the most com-
mon way of measuring time complexity. One reason is that an instance may be
massively larger than the number of variables — a SAT instance with n variables
may contain up to 22n distinct clauses if repeated literals are disallowed — and
measuring in the instance size may give far too optimistic figures, especially since
naturally appearing test examples tend to contain a moderate number of con-
straints. Another reason is that in the finite-domain case, the size of the search
space is very closely related to the number of variables. We show that one can
reason in a similar way when it comes to the complexity of many infinite-domain
CSPs.
The relations in finite-domain CSPs are easy to represent by simply listing
the allowed tuples. When considering infinite-domain CSPs, the relations need to
be implicitly represented. A natural way is to consider disjunctive formulas over
a finite set of basic relations. Let B denote some finite set of basic relations such
that CSP(B) is tractable. Let B ∨∞ denote the closure of B under disjunctions,
and let B ∨k be the subset of B ∨∞ containing only disjunctions of length at
most k. Consider the following example: let D = {true, f alse} and let B =
{B1 , B2 } where B1 = {true} and B2 = {f alse}. It is easy to see that CSP(B ∨∞ )
corresponds to the Boolean SAT problem while CSP(B ∨k ) corresponds to the
k-SAT problem.
CSPs in certain applications such as AI are often based on binary basic rela-
tions and unions of them (instead of free disjunctive formulas). Clearly, such
relations are a subset of the relations in B ∨k and we let B ∨= denote this set
of relations. We do not explicitly bound the length of disjunctions since they
are bounded by |B|. The literature on such CSPs is voluminous and we refer
the reader to Renz and Nebel [29] for an introduction. The languages B ∨∞ and
B ∨k have been studied to a smaller extent in the literature. There are both
works studying disjunctive constraints from a general point of view [9,11] and
application-oriented studies; examples include temporal reasoning [19,31], inter-
active graphics [27], rule-based reasoning [25], and set constraints (with applica-
tions in descriptive logics) [4]. We also note (see Section 2.2 for details) that there
is a connection to constraint languages containing first-order definable relations.
Assume Γ is a finite constraint language containing relations that are first-order
definable in B, and that the first order theory of B admits quantifier elimina-
tion. Then, upper bounds on CSP(Γ ) can be inferred from results such as those
that will be presented in Sections 3 and 4. This indicates that studying the time
complexity of CSP(B ∨∞ ) is worthwhile, especially since our understanding of
first-order definable constraint languages is rapidly increasing [3].
To solve infinite-domain CSPs, backtracking algorithms are usually employed.
Unfortunately, such algorithms can be highly inefficient in the worst case. Let
p denote the maximum arity of the relations in B, let m = |B|, and let |V |
denote the number of variables in a given CSP instance. We show (in Section 3.1)
m·|V |p
·log(m·|V |p )
that the time complexity ranges from O(22 · poly(||I||)) (which
is doubly exponential with respect to the number of variables) for CSP(B ∨∞ )
m p
to O(22 ·|V | ·log m · poly(||I||)) time for B ∨= (and the markedly better bound
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 185

p
of O(2|V | log m · poly(||I||)) if B consists of pairwise disjoint relations.) The use
of heuristics can probably improve these figures in some cases, but we have
not been able to find such results in the literature and it is not obvious how
to analyse backtracking combined with heuristics. At this stage, we are mostly
interested in obtaining a baseline: we need to know the performance of simple
algorithms before we start studying more sophisticated ones. However, some of
these bounds can be improved by combining backtracking search with methods
for reducing the number of constraints. We demonstrate this with sparsifica-
tion [18] in Section 3.2.
In Section 4 we switch strategy and show that disjunctive CSP problems can
be solved significantly more efficiently via a method we call structure enumera-
tion. This method is inspired by the enumerative method for solving finite-domain
p
CSPs. With this algorithm, we obtain the upper bound O(2|V | ·m · poly(||I||)) for
∨∞
CSP(B ). If we additionally assume that B is jointly exhaustive and pairwise dis-
p
joint then the running time is improved further to O(2|V | ·log m · poly(||I||)). This
bound beats or equals every bound presented in Section 3. We then proceed to
show even better bounds for certain choices of B. In Section 4.2 we consider equal-
ity constraint languages over a countably infinite domain and show that such CSP
problems are solvable in O(|V |B|V | · poly(||I||) time, where B|V | is the |V |-th Bell
number. In Section 4.3 we focus on three well-known temporal reasoning problems
and obtain significantly improved running times.
We tackle the problem of determining lower bounds for CSP(B ∨∞ ) in
Section 5, i.e. identifying functions f such that no algorithm for CSP(B ∨∞ )
has a better running time than O(f (|V |)). We accomplish this by relating CSP
problems and certain complexity-theoretical conjectures, and obtain strong lower
bounds for the majority of the problems considered in Section 4. As an exam-
ple, we show that the temporal CSP({<, >, =}∨∞ ) problem is solvable in time
O(2|V | log |V | ·poly(||I||)) but, assuming a conjecture known as the strong exponen-
tial time hypothesis (SETH), not solvable in O(c|V | ) time for any c > 1. Hence,
even though the algorithms we present are rather straightforward, there is, in
many cases, very little room for improvement, unless the SETH fails.

2 Preliminaries
We begin by defining the constraint satisfaction problem and continue by dis-
cussing first-order definable relations.

2.1 Constraint Satisfaction

Definition 1. Let Γ be a set of finitary relations over some set D of values.
The constraint satisfaction problem over Γ (CSP(Γ )) is defined as follows:
Instance: A set V of variables and a set C of constraints of the form
R(v1 , . . . , vk ), where k is the arity of R, v1 , . . . , vk ∈ V and R ∈ Γ .
Question: Is there a function f : V → D such that (f (v1 ), . . . , f (vk )) ∈ R for
every R(v1 , . . . , vk ) ∈ C?
186 P. Jonsson and V. Lagerkvist

The set Γ is referred to as the constraint language. Observe that we do

not require Γ or D to be finite. Given an instance I of CSP(Γ ) we write ||I||
for the number of bits required to represent I. We now turn our attention
to constraint languages based on disjunctions. Let D be a set of values and
let B = {B1 , . . . , Bm } denote a finite set of relations over D, i.e. Bi ⊆ Dj
for some j ≥ 1. Let the set B ∨∞ denote the set of relations defined by dis-
junctions over B. That is, B ∨∞ contains every relation R(x1 , . . . , xp ) such that
R(x1 , . . . , xp ) if and only if Bi1 (x1 )∨· · ·∨Bit (xt ) where x1 , . . . , xt are sequences
of variables from {x1 , . . . , xp } such that the length of xj equals the arity of Bij .
We refer to Bi1 (x1 ), . . . , Bit (xt ) as the disjuncts of R. We assume, without loss
of generality, that a disjunct occurs at most once in a disjunction. We define
B ∨k , k ≥ 1, as the subset of B ∨∞ where each relation is defined by a disjunction
of length at most k. It is common, especially in qualitative temporal and spa-
tial constraint reasoning, to study a restricted variant of B ∨k when all relations
in B has the same arity p. Define B ∨= to contain every relation R such that
R(x) if and only if Bi1 (x) ∨ · · · ∨ Bit (x), where x = (x1 , . . . , xp ). For examples
of basic relations, we refer the reader to Sections 4.2 and 4.3.
We adopt a simple representation of relations in B ∨∞ : every relation R in
∨∞
B is represented by its defining disjunctive formula. Note that two objects
R, R ∈ B ∨∞ may denote the same relation. Hence, B ∨∞ is not a constraint lan-
guage in the sense of Definition 1. We avoid tedious technicalities by ignoring this
issue and view constraint languages as multisets. Given an instance I = (V, C)
of CSP(B ∨∞ ) under this representation, we let Disj(I) = {Bi1 (x1 ), . . . , Bit (xt ) |
Bi1 (x1 ) ∨ · · · ∨ Bit (xt ) ∈ C} denote the set of all disjuncts appearing in C.
We close this section by recapitulating some terminology. Let B =
{B1 , . . . , Bm } be a set of relations (over a domain D) such  that all B1 , . . . , Bm
have arity p. We say that B is jointly exhaustive (JE) if B = Dp and that B is
pairwise disjoint (PD) if Bi ∩ Bj = ∅ whenever i = j. If B is both JE and PD we
say that it is JEPD. Observe that if B1 , . . . , Bm have different arity then these
properties are clearly not relevant since the intersection between two such rela-
tions is always empty. These assumptions are common in for example qualitative
spatial and temporal reasoning, cf. [24]. Let Γ be an arbitrary set of relations
with arity p ≥ 1. We say that Γ is closed under intersection if R1 ∩ R2 ∈ Γ for
all choices of R1 , R2 ∈ Γ . Let R be an arbitrary binary relation. We define the
converse R of R such that R = {(y, x) | (x, y) ∈ R}. If Γ is a set of binary
relations, then we say that Γ is closed under converse if R ∈ Γ for all R ∈ Γ .

2.2 First-Order Definable Relations

Languages of the form B ∨∞ have a close connection with languages defined

over first-order structures admitting quantifier elimination, i.e. every first-order
definable relation can be defined by an equivalent formula without quantifiers.
We have the following lemma.

Lemma 2. Let Γ be a finite constraint language first-order definable over

a relational structure (D, R1 , . . . , Rm ) admitting quantifier elimination, where
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 187

R1 , . . . , Rm are JEPD. Then there exists a k such that (1) CSP(Γ ) is polynomial-
time reducible to CSP({R1 , . . . , Rm }∨k ) and (2) if CSP({R1 , . . . , Rm }∨k ) is solvable
in O(f (|V |)·poly(||I||)) time then CSP(Γ ) is solvable in O(f (|V |)·poly(||I||)) time.

Proof. Assume that every relation R ∈ Γ is definable through a quantifier-free

first-order formula φi over R1 , . . . , Rm . Let ψi be φi rewritten in conjunctive nor-
mal form. We need to show that every disjunction in ψi can be expressed as a dis-
junction over R1 , . . . , Rm . Clearly, if ψi only contains positive literals, then this is
trivial. Hence, assume there is at least one negative literal. Since R1 , . . . , Rm are
JEPD it is easy to see that for any negated relation in {R1 , . . . , Rm } there exists
Γ ⊆ {R1 , . . . , Rm } such that the union of Γ equals the complemented relation.
We can then reduce CSP(Γ ) to CSP({R1 , . . . , Rm }∨k ) by replacing every con-
straint by its conjunctive normal formula over R1 , . . . , Rm . This reduction can
be done in polynomial time with respect to ||I|| since each such definition can be
stored in a table of fixed size. Moreover, since this reduction does not increase the
number of variables, it follows that CSP(Γ ) is solvable in O(f (|V |) · poly(||I||))
time whenever CSP(B ∨k ) is solvable in O(f (|V |) · poly(||I||)) time. 

As we will see in Section 4, this result is useful since we can use upper
bounds for CSP(B ∨k ) to derive upper bounds for CSP(Γ ), where Γ consists
of first-order definable relations over B. There is a large number of structures
admitting quantifier elimination and interesting examples are presented in every
standard textbook on model theory, cf. Hodges [15]. A selection of problems that
are highly relevant for computer science and AI are discussed in Bodirsky [3].

3 Fundamental Algorithms
In this section we investigate the complexity of algorithms for CSP(B ∨∞ ) and
CSP(B ∨k ) based on branching on the disjuncts in constraints (Section 3.1) and
the sparsification method (Section 3.2.) Throughout this section we assume that
B is a set of basic relations such that CSP(B) is in P.

3.1 Branching on Disjuncts

Let B = {B1 , . . . , Bm } be a set of basic relations with maximum arity p ≥
1. It is easy to see that CSP(B ∨∞ ) is in NP. Assume we have an instance I
p
of CSP(B ∨∞ ) with variable set V . Such an instance contains at most 2m·|V |
distinct constraints. Each such constraint contains at most m · |V |p disjuncts so
the instance I can be solved in
m·|V |p m·|V |p
·log(m·|V |p )
O((m · |V |p )2 · poly(||I||)) = O(22 · poly(||I||))

time by enumerating all possible choices of one disjunct out of every disjunctive
constraint. The satisfiability of the resulting sets of constraints can be checked
in polynomial time due to our initial assumptions. How does such an enumer-
ative approach compare to a branching search algorithm? In the worst case,
188 P. Jonsson and V. Lagerkvist

a branching algorithm without heuristic aid will go through all of these cases so
the bound above is valid for such algorithms. Analyzing the time complexity of
branching algorithms equipped with powerful heuristics is a very different (and
presumably very difficult) problem.
Assume instead that we have an instance I of CSP(B ∨k ) with variable
set V . There are at most m · |V |p different disjuncts which leads to at most
k p i p k
i=0 (m|V | ) ≤ k · (m|V | ) distinct constraints. We can thus solve instances
k·(m|V |p )k p k
with |V | variables in O(k · poly(||I||)) = O(2k·log k·(m|V | ) · poly(||I||))
time.
Finally, let I be an instance of CSP(B ∨= ) with variable set V . It is not hard to
see that I contains at most 2m · |V |p distinct constraints, where each constraint
has length at most m. Non-deterministic guessing gives that instances of this
kind can be solved in
m
·|V |p m
·|V |p ·log m
O(m2 · poly(||I||)) = O(22 · poly(||I||))

time. This may appear to be surprisingly slow but this is mainly due to the
fact that we have not imposed any additional restrictions on the set B of basic
relations. Hence, assume that the relations in B are PD. Given two relations
R1 , R2 ∈ B ∨= , it is now clear that R1 ∩ R2 is a relation in B ∨= , i.e. B ∨=
is closed under intersection. Let I = (V, C) be an instance of CSP(B ∨= ). For
any sequence of variables (x1 , . . . , xp ), we can assume that there is at most
one constraint R(x1 , . . . , xp ) in C. This implies that we can solve CSP(B ∨= ) in
p p
O(m|V | · poly(||I||)) = O(2|V | log m · poly(||I||)) time. Combining everything so
far we obtain the following upper bounds.

Lemma 3. Let B be a set of basic relations with maximum arity p and let m =
|B|. Then
m·|V |p p
– CSP(B ∨∞ ) is solvable in O(22 ·log(m·|V | )
· poly(||I||)) time,
∨k k·log k·(m|V |p )k
– CSP(B ) is solvable in O(2 · poly(||I||)) time,
m p
– CSP(B ∨= ) is solvable in O(22 ·|V | ·log m · poly(||I||)) time, and
∨= |V |p log m
– CSP(B ) is solvable in O(2 · poly(||I||)) time if B is PD.

A bit of fine-tuning is often needed when applying highly general results like
Lemma 3 to concrete problems. For instance, Renz and Nebel [29] show that
|V |2
the RCC-8 problem can be solved in O(c 2 ) for some (unknown) c > 1. This
problem can be viewed as CSP(B ∨= ) where B contains JEPD binary relations
2
and |B| = 8. Lemma 3 implies that CSP(B ∨= ) can be solved in O(23|V | ) which
is significantly slower if c < 82 . However, it is well known that B is closed under
converse. Let I = ({x1 , . . . , xn }, C) be an instance of CSP(B ∨= ). Since B is
closed under converse, we can always assume that if R(xi , xj ) ∈ C, then i ≤ j.
|V |2 |V |2
Thus, we can solve CSP(B ∨= ) in O(m 2 ·poly(||I||)) = O(2 2 log m ·poly(||I||))
time. This figure matches the bound by Renz and Nebel better when c is small.
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 189

3.2 Sparsification
The complexity of the algorithms proposed in Section 3 is dominated by the
number of constraints. An idea for improving these running times is therefore
to reduce the number of constraints within instances. One way of accomplishing
this is by using sparsification [18]. Before presenting this method, we need a
few additional definitions. An instance of the k-Hitting Set problem consists of
a finite set U (the universe) and a collection C = {S1 , . . . , Sm } where Si ⊆ U
and |Si | ≤ k, 1 ≤ i ≤ m. A hitting set for C is a set C ⊆ U such that C ∩ Si = ∅
for each Si ∈ C. Let σ(C) be the set of all hitting sets of C. The k-Hitting Set
problem is to find a minimal size hitting set. T is a restriction of C if for each
S ∈ C there is a T ∈ T with T ⊆ S. If T is a restriction of C, then σ(T ) ⊆ σ(C).
We then have the following result1 .

Theorem 4 (Impagliazzo et al. [18]). For all ε > 0 and positive k, there
is a constant K and an algorithm that, given an instance C of k-Hitting Set on
a universe ofsize n, produces a list of t ≤ 2ε·n restrictions T1 , . . . , Tt of C so
t
that σ(C) = i=1 σ(Ti ) and so that for each Ti , |Ti | ≤ Kn. Furthermore, the
algorithm runs in time poly(n) · 2ε·n .

Lemma 5. Let B be a set of basic relations with maximum arity p and let m =
p
|B|. Then CSP(B ∨k ) is solvable in O(2(ε+K log k)·|V | ·m ·poly(||I||)) time for every
ε > 0, where K is a constant depending only on ε and k.

Proof. Let I = (V, C) be an instance of CSP(B ∨k ). We can easily reduce

CSP(B ∨k ) to k-Hitting set by letting U = Disj(I) and C be the set correspond-
ing to all disjunctions in C. Then choose some ε > 0 and let {T1 , . . . , Tt } be
the resulting sparsification. Let {T1 , . . . , Tt } be the corresponding instances of
CSP(B ∨k ). Each instance Ti contains at most K · |U | ≤ K · |V |p · m distinct
constraints, where K is a constant depending on ε and k, and can therefore be
p
solved in time O(poly(||I||) · k K·|V | ·m ) by exhaustive search à la Section 3.1.
Last, answer yes if and only if some Ti is satisfiable. This gives a total running
time of
p p p
poly(|V |p · m) · 2ε·|V | ·m
+ 2ε·|V | ·m
· k K·|V | ·m
· poly(||I||) ∈
p p p
O(2ε·|V | ·m
· 2K·|V | ·m·log k
· poly(||I||)) = O(2(ε+K log k)·|V | ·m
· poly(||I||))

since t ≤ 2ε·n . 

This procedure can be implemented using only polynomial space, just as the
enumerative methods presented in Section 3.1. This follows from the fact that the
restrictions T1 , . . . , Tt of C can be computed one after another with polynomial
delay [10, Theorem5.15]. Although this running time still might seem excessively
p k
slow observe that it is significantly more efficient than the 2k·log k·(m|V | ) algo-
∨k
rithm for CSP(B ) in Lemma 3.
1
We remark that Impagliazzo et al. [18] instead refer to the k-Hitting set problem as
the k-Set cover problem.
190 P. Jonsson and V. Lagerkvist

4 Improved Upper Bounds

In this section, we show that it is possible to obtain markedly better upper
bounds than the ones presented in Section 3. In Section 4.1 we first consider
general algorithms for CSP(B ∨∞ ) based on structure enumeration, and in Sec-
tions 4.2 and 4.3, based on the same idea, we construct even better algorithms
for equality constraint languages and temporal reasoning problems.

4.1 Structure Enumeration

We begin by presenting a general algorithm for CSP(B ∨∞ ) based on the idea of

enumerating all variable assignments that are implicitly described in instances.
As in the case of Section 3 we assume that B is a set of basic relations such that
CSP(B) is solvable in O(poly(||I||)) time.

Theorem 6. Let B be a set of basic relations with maximum arity p and let
p
m = |B|. Then CSP(B∨∞ ) is solvable in O(2m|V | · poly(||I||)) time.

Proof. Let I = (V, C) be an instance of CSP(B ∨∞ ). Let S = Disj(I) and note

that |S| ≤ m|V |p . For each subset Si of S first determine whether Si is satisfiable.
Due to the initial assumption this can be done in O(poly(||I||)) time since this
set of disjuncts can be viewed as an instance of CSP(B). Next, check whether Si
satisfies I by, for each constraint in C, determine whether at least one disjunct is
included in Si . Each such step can determined in time O(poly(||I||)) time. The
p
total time for this algorithm is therefore O(2m|V | · poly(||I||)). 

The advantage of this approach compared to the branching algorithm in

Section 3 is that enumeration of variable assignments is much less sensitive
to instances with a large number of constraints. We can speed up this result
even further by making additional assumptions on the set B. This allows us to
enumerate smaller sets of constraints than in Theorem 6.

Theorem 7. Let B be a set of basic relations with maximum arity p and let
m = |B|. Then
p
1. CSP(B∨∞ ) solvable in O(2|V | ·log m · poly(||I||))) time if B is JEPD, and
p
2. CSP(B ∨∞ ) is solvable in O(2|V | ·log(m+1) · poly(||I||))) time if B is PD.

Proof. First assume that B is JEPD and let I = (V, C) be an instance of

CSP(B ∨∞ ). Observe that every basic relation has the same arity p since B is
p
JEPD. Let F be the set of functions from |V |p to B. Clearly |F | ≤ 2|V | log m .
For every fi ∈ F let Sfi = {Bj (xj ) | xj ∈ V p , fi (xj ) = Bj }. For a set Sfi
one can then determine in O(poly(||I||)) time whether it satisfies I by, for every
constraint in C, check if at least one disjunct in every constraint is included in
Sfi . Hence, the algorithm is sound. To prove completeness, assume that g is a
satisfying assignment of I and let Sg be the set of disjuncts in C which are true
in this assignment. For every Bi (xi ) ∈ Sg define the function f as f (xi ) = Bi .
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 191

Since B is PD it cannot be the case that f (xi ) = Bi = Bj for some Bj ∈ B

distinct from Bi . Next assume that there exists xi ∈ V p but no Bi ∈ B such that
Bi (xi ) ∈ Sg . Let B = {B1 , . . . , Bm } and let f1 , . . . , fm be functions agreeing
with f for every value for which it is defined and such that fi (xi ) = Bi . Since B
is JE it holds that f satisfies I if and only if some fi satisfies I.
Next assume that B is PD but not JE. In this case we use the same construc-
tion but instead consider the set of functions F  from V p to B ∪ {Dp }. There are
p p
2|V | ·log(m+1) ) such functions, which gives the desired bound O(2|V | ·log(m+1) ·
p
poly(||I||)). The reason for adding the additional element D to the domains of
the functions is that if f ∈ F  , and if f (x) = Dp for some x ∈ V p , then this
constraint does not enforce any particular values on x. 

4.2 Equality Constraint Languages

Let E = {=, =} over some countably infinite domain D. The language E ∨∞ is a

particular case of an equality constraint language [5], i.e. sets of relations definable
through first-order formulas over the structure (D, =). Such languages are of
fundamental interest in complexity classifications for infinite domain CSPs, since
a classification of CSP problems based on first-order definable relations over some
fixed structure, always includes the classification of equality constraint language
2
CSPs. We show that the O(2|V | · poly(||I||)) time algorithm in Theorem 7 can
be improved upon quite easily. But first we need some additional machinery.
mX with n elements is a pairwise disjoint set {X1 , . . . , Xm },
A partition of a set
m ≤ n such that i=1 Xi = X. A set X with n elements has Bn partitions, where
0.792n
Bn is the n-th Bell number. Let L(n) = ln(n+1) . It is known that Bn < L(n)n [1]
and that all partitions can be enumerated in O(nBn ) time [13,32].

Theorem 8. CSP(E ∨∞ ) is solvable in O(|V |2|V |·log L(|V |) · poly(||I||)) time.

Proof. Let I = (V, C) be an instance of CSP(E ∨∞ ). For every partition S1 ∪. . .∪

Sn of V we interpret the variables in Si as being equal and having the value i, i.e.
a constraint (x = y) holds if and only if x and y belong to the same set and (x =
y) holds if and only if x and y belong to different sets. Then check in poly(||I||)
time if this partition satisfies I using the above interpretation. The complexity of
this algorithm is therefore O(|V |B|V | ·poly(||I||)) ⊆ O(|V |L(|V |)|V | ·poly(||I||)) =
O(|V |2|V |·log L(|V |) · poly(||I||)). 

2
Observe that this algorithm is much more efficient than the O(2|V | ·
poly(||I||)) algorithm in Theorem 7. It is well known that equality constraint
languages admit quantifier elimination [5]. Hence, we can use Lemma 2 to
extend Theorem 8 to cover arbitrary equality constraint languages.

Corollary 9. Let Γ be a finite set of relations first-order definable over (D, =).
Then CSP(Γ ) is solvable in O(|V |2|V |·log L(|V |) · poly(||I||)) time.
192 P. Jonsson and V. Lagerkvist

4.3 Temporal Constraint Reasoning

Let T = {<, >, =} denote the JEPD order relations on Q and recall that CSP(T )
2
is tractable [34]. Theorem 7 implies that CSP(T ∨∞ ) can be solved in O(2|V | ·log 3 ·
poly(||I||)) time. We improve this as follows.
Theorem 10. CSP(T ∨∞ ) is solvable in O(2|V | log |V | · poly(||I||)) time.
Proof. Let I = (V, C) be an instance of CSP(T ∨∞ ). Assume f : V → Q sat-
isfies this instance. It is straightforward to see that there exists some g : V →
{1, . . . , |V |} which satisfies I, too. Hence, enumerate all 2|V | log |V | functions from
V to {1, . . . , |V |} and answer yes if any of these satisfy the instance. 

It is well known that the first-order theory of (Q, <) admits quantifier elimi-
nation [6,15]. Hence, we can exploit Lemma 2 to obtain the following corollary.
Corollary 11. Let Γ be a finite temporal constraint language over (Q, <). If
CSP(Γ ) is NP-complete, then it is solvable in O(2|V | log |V | · poly(||I||)) time.
We can also obtain strong bounds for Allen’s interval algebra, which is a well-
known formalism for temporal reasoning. Here, one considers relations between
intervals of the form [x, y], where x, y ∈ R is the starting and ending point,
respectively. Let Allen be the 213 = 8192 possible unions of the set of the thirteen
relations in Table 1. For convenience we write constraints such as (p ∨ m)(x, y)
as x{p, m}y, using infix notation and omitting explicit disjunction signs. The
problem CSP(Allen) is NP-complete and all tractable fragments have been iden-
tified [22].
Given an instance I = (V, C) of CSP(Allen) we first create two fresh variables
xsi and xei for every x ∈ V , intended to represent the startpoint and endpoint of
the interval x. Then observe that a constraint x{r1 , . . . , rm }y ∈ C, where each
ri is a basic relation, can be represented as a disjunction of temporal constraints
over xs , xe , y s and y e by using the definitions of each basic relation in Table 1.
Applying Theorem 10 to the resulting instance gives the following result.

Table 1. The thirteen basic relations in Allen’s interval algebra. The endpoint relations
xs < xe and y s < y e that are valid for all relations have been omitted.

Basic relation Example Endpoints

x precedes y p xxx xe < y s
y preceded by x p−1 yyy
x meets y m xxxx xe = y s
y met-by x m−1 yyyy
x overlaps y o xxxx y s < xe ,
y overl.-by x o−1 yyyy xe < y e
x during y d xxx ys ,
y includes x d−1 yyyyyyy xe < y e
x starts y s xxx ys ,
y started by x s−1 yyyyyyy xe < y e
x finishes y f xxx xe = y e ,
y finished by x f−1 yyyyyyy ys
x equals y ≡ xxxx ys ,
yyyy xe = y e
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 193

Corollary 12. CSP(Allen) is solvable in O(22|V |(1+log |V |) · poly(||I||)) time.

Finally, we consider branching time. We define the following relations on

the set of all points in the forest containing all oriented, finite trees where the
in-degree of each node is at most one.

1. x = y if and only if there is a path from x to y and a path from y to x,

2. x < y if and only if and there is a path from x to y but no path from y to x,
3. x > y if and only if there is a path from y to x but no path from x to y,
4. x||y if and only if there is no path from x to y and no path from y to x,

These four basic relations are known as the point algebra for branching time.
We let P = {||, <, >, =}. The problem CSP(P ∨∞ ) is NP-complete and many
tractable fragments have been identified [8].

Theorem 13. CSP(P ∨∞ ) is solvable in O(2|V |+2|V | log |V | · poly(||I||)) time.

Proof. Let I = (V, C) be an instance of CSP(P ∨∞ ). We use the following algo-

rithm.

1. enumerate all directed forests over V where the in-degree of each node is at
most one,
2. for every forest F , if at least one disjunct in every constraint in C is satisfied
by F , answer yes,
3. answer no.

It is readily seen that this algorithm is sound and complete for CSP(P ∨∞ ). As
for the time complexity, recall that the number of directed labelled trees with
|V | vertices is equal to |V ||V |−2 by Cayley’s formula. These can be efficiently
enumerated by e.g. enumerating all Prüfer sequences [28] of length |V | − 2. To
enumerate all forests instead of trees, we can enumerate all labelled trees with
|V | + 1 vertices and only consider the trees where the extra vertex is connected
to all other vertices. By removing this vertex we obtain a forest with |V | vertices.
Hence, there are at most 2|V | |V ||V |−1 directed forests over V . The factor 2|V |
stems from the observation that each forest contains at most |V | edges, where
each edge has two possible directions. We then filter out the directed forests
containing a tree where the in degree of any vertex is more than one. Last, for
each forest, we enumerate all |V ||V | functions from V to the forest, and check
in poly(||I||) time whether it satisfies I. Put together this gives a complexity of
O(2|V | |V ||V |−1 |V ||V | · poly(||I||)) ⊆ O(2|V |+2|V | log |V | · poly(||I||)). 

Branching time does not admit quantifier elimination [3, Section 4.2] so
Lemma 2 is not applicable. However, there are closely connected constraint lan-
guages on trees that have this property. Examples include the triple consistency
problem with important applications in bioinformatics [7].
194 P. Jonsson and V. Lagerkvist

5 Lower Bounds
The algorithms presented in Section 4 give new upper bounds for the complexity
of CSP(B ∨∞ ). It is natural to also ask, given reasonable complexity theoretical
assumptions, how much room there is for improvement. This section is divided
into Section 5.1, where we obtain lower bounds for CSP(B ∨∞ ) and CSP(B ∨k ) for
B that are JEPD, and in Section 5.2, where we obtain lower bounds for Allen’s
interval algebra.

5.1 Lower Bounds for JEPD Languages

One of the most well-known methods for obtaining lower bounds is to exploit the
exponential-time hypothesis (ETH). The ETH states that there exists a δ > 0
such that 3-SAT is not solvable in O(2δ|V | ) time by any deterministic algorithm,
i.e. it is not solvable in subexponential time [16]. If the ETH holds, then there
is an increasing sequence s3 , s4 , . . . of reals such that k-SAT cannot be solved
in time 2sk |V | but it can be solved in 2(sk +)|V | time for arbitrary  > 0. The
strong exponential-time hypothesis (SETH) is the conjecture that the limit of the
sequence s3 , s4 , . . . equals 1, and, as a consequence, that SAT is not solvable in
time O(2δ|V | ) for any δ < 1 [16]. These conjectures have in recent years success-
fully been used for proving lower bounds of many NP-complete problems [26].
Theorem 14. Let B = {R1 , R2 , . . . , Rm } be a JEPD set of nonempty basic
relations. If the SETH holds then CSP(B ∨∞ ) cannot be solved in O(2δ|V | ) time
for any δ < 1.
Proof. If the SETH holds then SAT cannot be solved in O(2δ|V | ) time for any δ <
1. We provide a polynomial-time many-one reduction from SAT to CSP(B ∨∞ )
which only increases the number of variables by a constant — hence, if CSP(B ∨∞ )
is solvable in O(2δ|V | ) time for some δ < 1 then SAT is also solvable in O(2δ|V | )
time, contradicting the original assumption.
Let I = (V, C) be an instance of SAT, where V is a set of variables and C
a set of clauses. First observe that since m ≥ 2 and since B is JEPD, B must
be defined over a domain with two or more elements. Also note that the require-
ment that B is JEPD implies that complement of R1 (x) can be expressed as
R2 (x) ∨ . . . ∨ Rm (x). Now, let p denote the arity of the relations in B. We intro-
duce p−1 fresh variables T1 , . . . , Tp−1 and then for every clause ( 1 ∨. . .∨ k ) ∈ C
create the constraint (φ1 (x1 , T1 , . . . , Tp−1 ) ∨ . . . ∨ φk (xk , T1 , . . . , Tp−1 )), where
φi (xi , T1 , . . . , Tp−1 ) = R1 (xi , T1 , . . . , Tp−1 ) if i = xi and φi (xi , T1 , . . . , Tp−1 ) =
R2 (xi , T1 , . . . , Tp−1 ) ∨ . . . ∨ Rm (xi , T1 , . . . , Tp−1 ) if i = ¬xi . Hence, the result-
ing instance is satisfiable if and only if I is satisfiable. Since the reduction
only introduces p − 1 fresh variables it follows that SAT is solvable in time
O(2δ(|V |+p−1) ) = O(2δ|V | ). 

Even though this theorem does not rule out the possibility that CSP(B ∨k )
can be solved significantly faster for some k it is easy to see that CSP(B ∨k )
cannot be solved in subexponential time for any k ≥ 3(|B| − 1). First assume
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 195

that the ETH holds. By following the proof of Theorem 14 we can reduce 3-SAT
to CSP(B ∨3(|B|−1) ), which implies that CSP(B ∨3(|B|−1) ) cannot be solved in 2δn
time either. The bound k = 3(|B| − 1) might obviously feel a bit unsatisfactory
and one might wonder if this can be improved. We can in fact make this much
more precise by adding further restrictions to the set B. As in the case of the
equality constraint languages in Section 4.2 we let = denote the equality relation
on a given countably infinite domain.

Theorem 15. Let B = {=, R1 , . . . , Rm } be a set of binary PD, nonempty rela-

tions. If the ETH holds then CSP(B ∨k ) cannot be solved in O(2sk |V | ) time.

Proof. We prove this result by reducing k-SAT to CSP(B ∨k ) in such a way

that we at most introduce one fresh variable. Let I = (V, C) be an instance
of k-SAT, where V is a set of variables and C a set of clauses. We know that
R1 ⊆ {(a, b) | a, b ∈ D and a = b} since B is PD. Introduce one fresh variable T .
For every clause ( 1 ∨ . . . ∨ k ) ∈ C create the constraint (φ1 ∨ . . . ∨ φk ), where
φi := xj = T if i = xj and φi = R1 (xj , T ) if i = ¬xj . Let (V  , C  ) be the
resulting instance of CSP(B ∨k ). We show that (V  , C  ) has a solution if and only
if (V, C) has a solution.
Assume first that (V, C) has a solution f : V → {0, 1}. Arbitrarily choose
a tuple (a, b) ∈ R1 . We construct a solution f  : V  → {a, b} for (V  , C  ). Let
f  (T ) = b, and for all v ∈ V let f  (v) = b if f (v) = 1 and let f  (v) = a if
f (v) = 0. Arbitrarily choose a clause ( 1 ∨ . . . ∨ k ) ∈ C and assume for instance
that 1 evaluates to 1 under the solution f . If 1 = xi , then f (xi ) = 1 and
the corresponding disjunct in the corresponding disjunctive constraint in C  is
xi = T . By definition, (f  (xi ), f  (T )) = (b, b). If 1 = ¬xi , then f (xi ) = 0 and
the corresponding disjunct in the corresponding disjunctive constraint in C  is
R1 (xi , T ). By definition, (f  (xi ), f  (T )) = (a, b) and (a, b) ∈ R1 .
Assume instead that f  : V  → D is a solution to (V  , C  ), and that f  (T ) = c.
We construct a solution f : V → {0, 1} to (V, C) as follows. Arbitrarily choose a
disjunctive constraint (d1 ∨· · ·∨dk ) ∈ C  and let ( 1 ∨· · ·∨ k ) be the corresponding
clause in C  . Assume that 1 = xi . If d1 is true under f  , then let f (xi ) = 1
and, otherwise, f (xi ) = 0. If 1 = ¬xi , then do the opposite: f (xi ) = 0 if d1
is true and f (xi ) = 1 otherwise. If the function f is well-defined, then f is
obviously a solution to (V, C). We need to prove that there is no variable that
is simultaneously assigned 0 and 1. Assume this is the case. Then there is some
variable xi such that the constraints xi = T and R1 (xi , T ) are simultaneously
satisfied by f  . This is of course impossible due to the fact that R1 contains no
tuple of the form (a, a). 

If we in addition require that B is JE we obtain substantially better lower

bounds for CSP(B ∨∞ ).

Theorem 16. Let B = {=, R1 , . . . , Rm } be a set of binary JEPD relations over

a countably infinite domain. If the SETH holds then CSP(B ∨∞ ) cannot be solved
in O(c|V | ) time for any c > 1.
196 P. Jonsson and V. Lagerkvist

m First observe that the binary inequality relation = over D can be defined
Proof.
as i=1 Ri since m B is JEPD. In the the proof we therefore use = as an abbre-
viation for i=1 Ri . Let I = (V, C) be an instance of SAT with variables
V = {x1 , . . . , xn } and the set of clauses C. Let K be an integer such that
K > log c. Assume without loss of generality that n is a multiple of K. We
n n
will construct an instance of CSP(B ∨∞ ) with K + 2K = K + O(1) variables.
K
First, introduce 2 fresh variables v1 , . . . , v2K and make them different by impos-
n
ing = constraints. Second, introduce K fresh variables y1 , . . . , y Kn , and for each
n
i ∈ {1, . . . , K } impose the constraint (yi = v1 ∨ yi = v2 ∨ · · · ∨ yi = v2k ). Let
V1 , . . . , V Kn be a partition of V such that each |Vi | = K. We will represent each set
Vi of Boolean variables by one yi variable over D. To do this we will interpret each
auxiliary variable zi as a K-ary Boolean tuple. Let h : {v1 , . . . , v2K } → {0, 1}K
be an injective function which assigns a Boolean K-tuple for every variable
vi . Let g+ be a function from {1, . . . , K} to subsets of {v1 , . . . , v2K } such that
vi ∈ g(j) if and only if the j-th element in h(vi ) is equal to 1. Define g− in the
analogous way. Observe that |g+ (j)| = |g− (j)| = 2K−1 for each j ∈ {1, . . . , K}.
For the reduction, let ( i1 ∨. . .∨ in ), ij = xij or ij = ¬xij , be a clause in C.
We assume that n ≤ n since the clause contains repeated literals otherwise. For
each literal ij let Vi ⊆  V be the set of variables such that xij ∈Vi . Each literal
ij is then replaced by z∈g+ (ij ) yi = z if ij = xij , and with z∈g− (ij ) yi = z
if ij = ¬xij . This reduction can be done in polynomial time since a clause with
n literals is replaced by a disjunctive constraint with n 2K−1 disjuncts (since K
is a constant depending only on c). It follows that SAT can be solved in
n n
O(c K +O(1) · poly(||I||)) = O(2( K +O(1))·log c · poly(||I||)) = O(2δ·n · poly(||I||))

for some δ < 1, since K > log c . Thus, the SETH does not hold. 

As an illustrative use of the theorem we see that the temporal problem

CSP(T ∨∞ ) is solvable in O(2|V | log |V | · poly(||I||)) time but not in O(c|V | ) time
for any c > 1 if the SETH holds. Lower bounds can also be obtained for
the branching time problem CSP(P ∨∞ ) since there is a trivial reduction from
CSP(T )∨∞ which does not increase the number of variables: simply add a con-
straint (x < y ∨ x > y ∨ x = y) for every pair of variables in the instance.
Similarly, the equality constraint satisfaction problem CSP(E ∨∞ ) is not solvable
in O(c|V | ) time for any c > 1 either, unless the SETH fails. Hence, even though
the algorithms in Sections 4.2 and 4.3 might appear to be quite simple, there is
very little room for improvement.

5.2 Lower Bounds for Allen’s Interval Algebra

Theorems 14, 15 and 16 gives lower bounds for all the problems considered in
Sections 4.2 and 4.3 except for CSP(Allen) since unlimited use of disjunction is
not allowed in this language. It is however possible to relate the complexity of
CSP(Allen) to the Chromatic Number problem, i.e. the problem of computing
the number of colours needed to colour a given graph.
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 197

√ |V |
Theorem 17. If CSP(Allen) can be solved in O( c ) time for some c < 2,
then Chromatic Number can be solved in O((c+)|V | ) time for arbitrary  > 0.
Proof. We first present a polynomial-time many-one reduction from k-
Colourability to CSP(Allen) which introduces k fresh variables. Given an
undirected graph G = ({v1 , . . . , vn }, E), introduce the variables z1 , . . . , zk and
v1 , . . . , vn , and:

1. impose the constraints z1 {m}z2 {m} . . . {m}zk ,

2. for each vi , 1 ≤ i ≤ n, add the constraints vi {≡, s−1 }z1 , vi {p, m, f −1 , d−1 }zj
(2 ≤ j ≤ k − 1), and vi {p, m, f −1 }zk ,
3. for each (vi , vj ) ∈ E, add the constraint vi {s, s−1 }vj .

Consulting Table 1, we see that for each vi , it holds that its right endpoint
must equal the right endpoint of some zi , and its left endpoint must equal the
left endpoint of z1 . It is now obvious that the resulting instance has a solution if
and only if G is k-colourable. The result then follows since there is a polynomial-
time Turing reduction from Chromatic Number to CSP(Allen) by combining
binary search (that will evaluate log n Allen instances) with the reduction above
(recall that O(log n · cn ) ⊆ O((c + )n ) for every  > 0) . Observe that if k =√n
then the reduction introduces n fresh variables, which is where the constant c
√ |V |
in the expression O( c ) stems from. CSP(Allen). 

The exact complexity of Chromatic Number has been analysed and dis-
cussed in the literature. Björklund et al. [2] have shown that the problem is
solvable in 2|V | · poly(||I||) time. Impagliazzo and Paturi [17] poses the following
question: “Assuming SETH, can we prove a 2n−o(n) lower bound for Chromatic
√ |V |
Number?”. Hence, an O( c ), c < 2, algorithm for CSP(Allen) would also be
a major breakthrough for Chromatic Number.

6 Discussion
We have investigated several novel algorithms for solving disjunctive CSP prob-
lems, which, with respect to worst-case time complexity, are much more efficient
than e.g. backtracking algorithms without heuristics. These bounds can likely
be improved, but, due to the lower bounds in Section 5, probably not to a great
degree. Despite this, algorithms for solving infinite domain constraint satisfaction
problems are in practice used in many non-trivial applications. In light of this the
following research direction is particularly interesting: how to formally analyse
the time complexity of branching algorithms equipped with (powerful) heuristics?
In the case of finite-domain CSPs and, in particular, DPLL-like algorithms for
the k-SAT problem there are numerous results to be found in the literature,
cf. the survey by Vsemirnov et al. [35]. This is not the case for infinite-domain
CSPs, even though there is a considerable amount of empirical evidence that
infinite-domain CSPs can be efficiently solved by such algorithms, so one ought
to be optimistic about the chances of actually obtaining non-trivial bounds. Yet,
sharp formal analyses appear to be virtually nonexistent in the literature.
198 P. Jonsson and V. Lagerkvist

Another research direction is to strengthen the lower bounds in Section 5 even

further. It would be interesting to prove stronger lower bounds for CSP(B ∨k ) for
some concrete choices of B and k. As an example, consider the temporal problem
CSP(T ∨4 ). From Theorem 15 we see that CSP(T ∨4 ) is not solvable in O(2s4 |V | )
time for some s4 < log 1.6, assuming the ETH holds, since the currently best
deterministic algorithm for 4-SAT runs in O(1.6|V | ) time [12]. On the other hand,
√ |V |
if CSP(T ∨4 ) is solvable in O( c ) time for some c < 2, then Chromatic
Number can be solved in O((c + )|V | ) time for arbitrary  > 0. This can be
proven similar to the reduction in Theorem 17 but by making use of temporal
constraints instead of interval constraints. Hence, for certain choices of B and k
it might be possible to improve upon the general bounds given in Section 5.

References
1. Berend, D., Tassa, T.: Improved bounds on Bell numbers and on moments of
sums of random variables. Probability and Mathematical Statistics 30(2), 185–205
(2010)
2. Björklund, A., Husfeldt, T., Koivisto, M.: Set partitioning via inclusion-exclusion.
SIAM Journal on Computing 39(2), 546–563 (2009)
3. Bodirsky, M.: Complexity classification in infinite-domain constraint satisfaction.
Mémoire d’habilitation à diriger des recherches, Université Diderot - Paris 7.
arXiv:1201.0856 (2012)
4. Bodirsky, M., Hils, M.: Tractable set constraints. Journal of Artificial Intelligence
Research 45, 731–759 (2012)
5. Bodirsky, M., Kára, J.: The complexity of equality constraint languages. Theory
of Computing Systems 43(2), 136–158 (2008)
6. Bodirsky, M., Kára, J.: The complexity of temporal constraint satisfaction prob-
lems. Journal of the ACM 57(2), 9:1–9:41 (2010)
7. Bodirsky, M., Mueller, J.K.: The complexity of rooted phylogeny problems. Logical
Methods in Computer Science 7(4) (2011)
8. Broxvall, M., Jonsson, P.: Point algebras for temporal reasoning: Algorithms and
complexity. Artificial Intelligence 149(2), 179–220 (2003)
9. Broxvall, M., Jonsson, P., Renz, J.: Disjunctions, independence, refinements. Arti-
ficial Intelligence 140(1–2), 153–173 (2002)
10. Calabro, C.: The Exponential Complexity of Satisfiability Problems. PhD thesis,
University of California, San Diego, CA, USA (2009)
11. Cohen, D., Jeavons, P., Jonsson, P., Koubarakis, M.: Building tractable disjunctive
constraints. Journal of the ACM 47(5), 826–853 (2000)
12. Dantsin, E., Goerdt, A., Hirsch, E.A., Kannan, R., Kleinberg, J.M., Papadimitriou,
C.H., Raghavan, P., Schöning, U.: A deterministic (2 − 2/(k + 1))n algorithm for
k-SAT based on local search. Theoretical Computer Science 289(1), 69–83 (2002)
13. Djokic, B., Miyakawa, M., Sekiguchi, S., Semba, I., Stojmenovic, I.: A fast iterative
algorithm for generating set partitions. The Computer Journal 32(3), 281–282
(1989)
14. Gaspers, S.: Exponential Time Algorithms - Structures, Measures, and Bounds.
VDM (2010)
15. Hodges, W.: A Shorter Model Theory. Cambridge University Press, New York
(1997)
16. Impagliazzo, R., Paturi, R.: On the complexity of k-SAT. Journal of Computer
and System Sciences 62(2), 367–375 (2001)
 Upper and Lower Bounds on the Time Complexity of Infinite-Domain CSPs 199

17. Impagliazzo, R., Paturi, R.: Exact complexity and satisfiability. In: Gutin, G.,
Szeider, S. (eds.) IPEC 2013. LNCS, vol. 8246, pp. 1–3. Springer, heidelberg (2013)
18. Impagliazzo, R., Paturi, R., Zane, F.: Which problems have strongly exponential
complexity? Journal of Computer and System Sciences 63(4), 512–530 (2001)
19. Jonsson, P., Bäckström, C.: A unifying approach to temporal constraint reasoning.
Artificial Intelligence 102(1), 143–155 (1998)
20. Jonsson, P., Lagerkvist, V., Nordh, G., Zanuttini, B.: Complexity of SAT prob-
lems, clone theory and the exponential time hypothesis. In: Proceedings of
the 24th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA-2013),
pp. 1264–1277 (2013)
21. Kanj, I., Szeider, S.: On the subexponential time complexity of CSP. In: Proceed-
ings of the Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI-
2013) (2013)
22. Krokhin, A., Jeavons, P., Jonsson, P.: Reasoning about temporal relations: The
tractable subalgebras of Allen’s interval algebra. Journal of the ACM 50(5),
591–640 (2003)
23. Ligozat, G.: Qualitative Spatial and Temporal Reasoning. Wiley-ISTE (2011)
24. Ligozat, G., Renz, J.: What Is a qualitative calculus? a general framework. In:
Zhang, C., W. Guesgen, H., Yeap, W.-K. (eds.) PRICAI 2004. LNCS (LNAI), vol.
3157, pp. 53–64. Springer, Heidelberg (2004)
25. Liu, B., Jaffar, J.: Using constraints to model disjunctions in rule-based reasoning.
In: Proceedings of the Thirteenth National Conference on Artificial Intelligence,
AAAI 1996, Portland, Oregon, vol. 2, pp. 1248–1255 (1996)
26. Lokshtanov, D., Marx, D., Saurabh, S.: Lower bounds based on the exponential
time hypothesis. Bulletin of EATCS 3(105) (2013)
27. Marriott, K., Moulder, P., Stuckey, P.J.: Solving disjunctive constraints for interac-
tive graphical applications. In: Walsh, T. (ed.) CP 2001. LNCS, vol. 2239, p. 361.
Springer, Heidelberg (2001)
28. Prüfer, H.: Neuer beweis eines satzes über permutationen. Archiv der Mathematik
und Physik 27, 742–744 (1918)
29. Renz, J., Nebel, B.: Efficient methods for qualitative spatial reasoning. Journal of
Artificial Intelligence Research 15(1), 289–318 (2001)
30. Rossi, F., van Beek, P., Walsh, T. (eds.) Handbook of Constraint Programming.
Elsevier (2006)
31. Stergiou, K., Koubarakis, M.: Backtracking algorithms for disjunctions of temporal
constraints. Artificial Intelligence 120(1), 81–117 (2000)
32. Stojmenović, I.: An optimal algorithm for generating equivalence relations on a
linear array of processors. BIT Numerical Mathematics 30(3), 424–436 (1990)
33. Traxler, P.: The time complexity of constraint satisfaction. In: Grohe, M.,
Niedermeier, R. (eds.) IWPEC 2008. LNCS, vol. 5018, pp. 190–201. Springer,
Heidelberg (2008)
34. Vilain, M.B., Kautz, H.A.: Constraint propagation algorithms for temporal rea-
soning. In: Proceedings of the 5th National Conference on Artificial Intelligence
(AAAI 1986), pp. 377–382 (1986)
35. Vsemirnov, M., Hirsch, E., Dantsin, E., Ivanov, S.: Algorithms for SAT and upper
bounds on their complexity. Journal of Mathematical Sciences 118(2), 4948–4962
(2003)
36. Woeginger, G.: Exact algorithms for NP-hard problems: a survey. In: Juenger, M.,
Reinelt, G., Rinaldi, G. (eds.) Combinatorial Optimization - Eureka! You Shrink!.
LNCS, vol. 2570, pp. 185–207. Springer, Heidelberg (2000)
 Generalized Totalizer Encoding
for Pseudo-Boolean Constraints

Saurabh Joshi1(B) , Ruben Martins1 , and Vasco Manquinho2

1
Department of Computer Science, University of Oxford, Oxford, UK
{saurabh.joshi,ruben.martins}@cs.ox.ac.uk
2
INESC-ID/Instituto Superior Técnico, Universidade de Lisboa, Lisboa, Portugal
[email protected]

Abstract. Pseudo-Boolean constraints, also known as 0-1 Integer Lin-

ear Constraints, are used to model many real-world problems. A common
approach to solve these constraints is to encode them into a SAT for-
mula. The runtime of the SAT solver on such formula is sensitive to
the manner in which the given pseudo-Boolean constraints are encoded.
In this paper, we propose generalized Totalizer encoding (GTE), which
is an arc-consistency preserving extension of the Totalizer encoding to
pseudo-Boolean constraints. Unlike some other encodings, the number of
auxiliary variables required for GTE does not depend on the magnitudes
of the coefficients. Instead, it depends on the number of distinct combina-
tions of these coefficients. We show the superiority of GTE with respect
to other encodings when large pseudo-Boolean constraints have low num-
ber of distinct coefficients. Our experimental results also show that GTE
remains competitive even when the pseudo-Boolean constraints do not
have this characteristic.

1 Introduction

Pseudo-Boolean constraints (PBCs) or 0-1 Integer Linear constraints have been

used to model a plethora of real world problems such as computational biol-
ogy [13,24], upgradeability problems [3,15,16], resource allocation [27], schedul-
ing [26] and automated test pattern generation [22]. Due to its importance and a
plethora of applications, a lot of research has been done to efficiently solve PBCs.
One of the popular approaches is to convert PBCs into a SAT formula [7,11,21]
thus making them amenable to off-the-shelf SAT solvers. We start by formally
introducing PBC, followed by a discussion on how to convert a PBC into a SAT
formula.
A PBC is defined over a finite set of Boolean variables x1 , . . . , xn which can be
assigned a value 0 (f alse) or 1 (true). A literal li is either a Boolean variable xi
(positive literal) or its negation ¬xi (negative literal). A positive (resp. negative)
literal li is said to be assigned 1 if and only if the corresponding variable xi is
assigned 1 (resp. 0). Without a loss of generality, PBC can be defined as a linear
inequality of the following normal form:


c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 200–209, 2015.
DOI: 10.1007/978-3-319-23219-5 15
 Generalized Totalizer Encoding for Pseudo-Boolean Constraints 201

(O : o2 , o3 , o5 , o6 : 6)

(A : a2 , a3 , a5 : 5) (B : b3 , b6 : 6)

(C : l1 : 2) (D : l2 : 3) (E : l3 : 3) (F : l4 : 3)

Fig. 1. Generalized Totalizer Encoding for 2l1 + 3l2 + 3l3 + 3l4 ≤ 5


wi li ≤ k (1)

Here, wi ∈ N+ are called coefficients or weights, li are input literals and

k ∈ N+ is called the bound. Linear inequalities in other forms (e.g. other inequal-
ity, equalities or negative coefficients) can be converted into this normal form
in linear time [8]. Cardinality constraint is a special case of PBC when all the
weights have the value 1. Many different encodings have been proposed to encode
cardinality constraints[4,5,25,28]. Linear pseudo-Boolean solving (PBS) is a gen-
eralization of the SAT formulation where constraints are not restricted to clauses
and can be PBCs. A related problem to PBS is the linear pseudo-Boolean opti-
mization (PBO) problem, where all the constraints must be satisfied and the
value of a linear cost function is optimized. PBO usually requires an iterative
algorithm which solves a PBS in every iteration [11,18,19,21]. Considering that
the focus of the paper is on encodings rather than algorithms, we restrict our-
selves to the decision problem (PBS).
This paper makes the following contributions.

– We propose an arc-consistency [12] preserving extension of Totalizer encod-

ing [5] called Generalized Totalizer encoding (GTE) in Section 2.
– We compare various PBC encoding schemes that were implemented in a com-
mon framework, thus providing a fair comparison. After discussing related
work in Section 3, we show GTE as a promising encoding through its com-
petitive performance in Section 4.

2 Generalized Totalizer Encoding

The Totalizer encoding [5] is an encoding to convert cardinality constraints into a

SAT formula. In this section, the generalized Totalizer encoding (GTE) to encode
PBC into SAT is presented. GTE can be better visualized as a binary tree, as
shown in Fig. 1. With the exception of the leaves, every node is represented as
(node name : node vars : node sum). The node sum for every node represents
the maximum possible weighted sum of the subtree rooted at that node. For
any node A, a node variable aw represents a weighted sum w of the underlying
subtree. In other words, whenever the weighted sum of some of the input literals
in the subtree becomes w, aw must be set to 1. Note that for any node A, we
202 S. Joshi et al.

would need one variable corresponding to every distinct weighted sum that the
input literals under A can produce. Input literals are at the leaves, represented
as (node name : literal name : literal weight) with each of the terms being self
explanatory.
For any node P with children Q and R, to ensure that weighted sum is
propagated from Q and R to P , the following formula is built for P :
⎛ ⎞
⎛ ⎞
⎜ ⎟
⎜ ⎟ ⎜ ⎟
⎜ q (¬qw1 ∨ ¬rw2 ∨ pw3 )⎟
⎜ w1 ∈ Q.node vars ⎟ ∧ ⎝ sw ∈ (Q.node vars ∪ R.node vars) (¬sw ∨ pw )⎠ (2)
⎝ rw2 ∈ R.node vars ⎠ w=w
w3 = w1 + w2 pw ∈ P.node vars
pw3 ∈ P.node vars

The left part of Eqn. (2) ensures that, if node Q has witnessed a weighted
sum of w1 and R has witnessed a weighted sum of w2 , then P must be considered
to have witnessed the weighted sum of w3 = w1 + w2 . The right part of Eqn. (2)
just takes care of the boundary condition where weighted sums from Q and R
are propagated to P without combining it with their siblings. This represents
that Q (resp. R) has witnessed a weighted sum of w but R (resp. Q) may not
have witnessed any positive weighted sum.
Note that node O in Fig. 1 does not have variables for the weighted sums
larger than 6. Once the weighted sum goes above the threshold of k, we represent
it with k + 1. Since all the weighted sums above k would result in the constraint
being not satisfied, it is sound to represent all such sums as k + 1. This is in some
sense a generalization of k-simplification described in [9,17]. For k-simplification,
w3 in Eqn. (2) would change to w3 = min(w1 + w2 , k + 1).
Finally, to enforce that the weighted sum does not exceed the given threshold
k, we add the following constraint at the root node O :

¬ok+1 (3)

Encoding Properties: Let AIw represent the multiset of weights of all the
input literals in the
subtree rooted at node A. For any given multiset S of weights,
let W eight(S) = e∈S e. For a given multiset S, let unique(S) denote the set
with all the multiplicity removed from S. Let |S| denote the cardinality of the
set S. Hence, the total number of node variables required at node A is:

|unique ({W eight(S)|S ⊆ AIw ∧ S = ∅})| (4)

Note that unlike some other encodings [7,14] the number of auxiliary vari-
ables required for GTE does not depend on the magnitudes of the weights.
Instead, it depends on how many unique weighted sums can be generated. Thus,
we claim that for pseudo-Boolean constraints where the distinct weighted sum
combinations are low, GTE should perform better. We corroborate our claim in
Section 4 through experiments.
Nevertheless, in the worst case, GTE can generate exponentially many auxil-
iary variables and clauses. For example, if the weights of input literals l1 , . . . , ln
 Generalized Totalizer Encoding for Pseudo-Boolean Constraints 203

are respectively 20 , . . . , 2n−1 , then every possible weighted sum combination

would be unique. In this case, GTE would generate exponentially many aux-
iliary variables. Since every variable is used in at least one clause, it will also
generate exponentially many clauses.
Though GTE does not depend on the magnitudes of the weights, one can
use the magnitude of the largest weight to categorize a class of PBCs for which
GTE is guaranteed to be of polynomial size. If there are n input literals and
the largest weight is a polynomial P (n), then GTE is guaranteed to produce a
polynomial size formula. If the largest weight is P (n), then the total number
of distinct weight combinations (Eqn. (4)) is bounded by nP (n), resulting in a
polynomial size formula.
The best case for GTE occurs when all of the weights are equal, in which
case the number of auxiliary variables and clauses is, respectively, O(n log2 n)
and O(n2 ). Notice that for this best case with k-simplification, we have O(nk)
variables and clauses, since it will behave exactly as the Totalizer encoding [5].
Note also that the generalized arc consistency (GAC) [12] property of Total-
izer encoding holds for GTE as well. GAC is a property of an encoding which
allows the solver to infer maximal possible information through propagation,
thus helping the solver to prune the search space earlier. The original proof [5]
makes an inductive argument using the left subtree and the right subtree of a
node. It makes use of the fact that, if there are q input variables set to 1 in
the left child Q and r input variables are set to 1 in the right child R, then
the encoding ensures that in the parent node P , the variable pq+r is set to 1.
Similarly, GTE ensures that if the left child Q contributes w1 to the weighted
sum (qw1 is set to 1) and the right child R contributes w2 to the weighted sum
(rw2 is set to 1), then the parent node P registers the weighted sum to be at
least w3 = w2 + w1 (pw3 is set to 1). Hence, the GAC proof still holds for GTE.

3 Related Work

The idea of encoding a PBC into a SAT formula is not new. One of the first such
encoding is described in [11,30] which uses binary adder circuit like formulation
to compute the weighted sum and then compare it against the threshold k.
This encoding creates O(n log2 k) auxiliary clauses, but it is not arc-consistent.
Another approach to encode PBCs into SAT is to use sorting networks [11].
This encoding produces O(N log22 N ) auxiliary clauses, where N is bounded by
log2 w1
+ . . . + log2 wn
. This encoding is also not arc-consistent for PBCs, but
it preserves more implications than the adder encoding, and it maintains GAC
for cardinality constraints.
The Watchdog encoding [7] scheme uses the Totalizer encoding, but in a
completely different manner than GTE. It uses multiple Totalizers, one for each
bit of the binary representation of the weights. The Watchdog encoding was the
first polynomial sized encoding that maintains GAC for PBCs and it only gen-
erates O(n3 log2 n log2 wmax ) auxiliary clauses. Recently, the Watchdog encoding
has been generalized to a more abstract framework with the Binary Merger
204 S. Joshi et al.

encoding [20]. Using a different translation of the components of the Watchdog

encoding allows the Binary Merger encoding to further reduce the number of
auxiliary clauses to O(n2 log22 n log2 wmax ). The Binary Merger is also polyno-
mial and maintains GAC.
Other encodings that maintain GAC can be exponential in the worst case
scenario, such as BDD based encodings [1,6,11]. These encodings share quite a
lot of similarity to GTE, such as GAC and independence from the magnitude of
the weight. One of the differences is that GTE always has a tree like structure
amongst auxiliary variables and input literals. However, the crucial difference
lies in the manner in which auxiliary variables are generated, and what they
represent. In BDD based approaches, an auxiliary variable Di attempts to reason
about the weighted sum of the input literals either li , . . . , ln or l1 , . . . , li . On the
other hand, an auxiliary variable aw at a node A in GTE attempts to only
reason about the weighted sum of the input literals that are descendants of A.
Therefore, two auxiliary variables in two disjoint subtrees in GTE are guaranteed
to reason about disjoint sets of input literals. We believe that such a localized
reasoning could be a cause of relatively better performance of GTE as reported in
Section 4. It is worth noting that the worst case scenario for GTE, when weights
are of the form ai , where a ≥ 2, would generate a polynomial size formula for
BDD based approaches [1,6,11].
As GTE generalizes the Totalizer encoding, the Sequential Weighted Counter
(SWC) encoding [14] generalizes sequential encoding [28] for PBCs. Like BDD
based approaches and GTE, SWC can be exponential in the worst case.

4 Implementation and Evaluation

All experiments were performed on two AMD 6276 processors (2.3 GHz) run-
ning Fedora 18 with a timeout of 1,800 seconds and a memory limit of 16
GB. Similar resource limitations were used during the last pseudo-Boolean (PB)
evaluation of 20121 . For a fair comparison, we implemented GTE (gte) in the
PBLib [29] (version 1.2) open source library which contains a plethora of encod-
ings, namely, Adder Networks (adder) [11,30], Sorting Networks (sorter) [11],
watchdog (watchdog) [7], Binary Merger (bin-merger) [20], Sequential Weighted
Counter (swc) [14], and BDDs (bdd) [1]. A new encoding in PBLib can be added
by implementing encode method of the base class Encoder. Thus, all the encod-
ings mentioned above, including GTE, only differ in how encode is implemented
while they share the rest of the whole environment. PBLib provides parsing
and normalization [11] routines for PBC and uses Minisat 2.2.0 [10] as a back-
end SAT solver. When the constraint to be encoded into CNF is a cardinality
constraint, we use the default setting of PBLib that dynamically selects a car-
dinality encoding based on the number of auxiliary clauses. When the constraint
to be encoded into CNF is a PBC, we specify one of the above encodings.
Benchmarks: Out of all 355 instances from the DEC-SMALLINT-LIN cate-
gory in the last PB evaluation of 2012 (PB’12), we only considered those 214
1
http://www.cril.univ-artois.fr/PB12/
 Generalized Totalizer Encoding for Pseudo-Boolean Constraints 205

Table 1. Characteristics of pseudo-Boolean benchmarks


Benchmark #PB #lits k max wi wi #diff wi
PB’12 164.31 32.25 27.94 12.55 167.14 6.72
pedigree 1.00 10,794.13 11,106.69 456.28 4,665,237.38 2.00

Table 2. Number of solved instances

Benchmark Result sorter swc adder watchdog bin-merger bdd gte

PB’12 SAT 72 74 73 79 79 81 81
(214) UNSAT 74 77 83 85 85 84 84
pedigree SAT 2 7 6 25 43 82 83
(172) UNSAT 0 7 6 23 35 72 75
Total SAT/UNSAT 146 165 172 212 242 319 323

instances2 that contain at least 1 PBC. We also consider an additional set of

pedigree benchmarks from computational biology [13]. These benchmarks were
originally encoded in Maximum Satisfiability (MaxSAT) and were used in the
last MaxSAT Evaluation of 20143 . Any MaxSAT problem can be converted to a
corresponding equivalent pseudo-Boolean problem [2]. We generate two pseudo-
Boolean decision problems (one satisfiable, another unsatisfiable) from the opti-
mization version of each of these benchmarks. The optimization function is trans-
formed into a PBC with the value of the bound k set to a specific value. Let
the optimum value for the optimization function be kopt . The satisfiable decision
problem uses kopt as the value for the bound k, whereas the unsatisfiable deci-
sion problem uses kopt − 1 as the value for the bound k. Out of 200 generated
instances4 , 172 had at least 1 PBC and were selected for further evaluation.
Tab. 1 shows the characteristics of the benchmarks used in this evaluation.

#PB denotes the average number of PBCs per instance. #lits, k, max wi , wi
and #diff wi denote the per constraint per instance average of input literals,
bound, the largest weight, maximum possible weighted sum and the number
of distinct weights. PB’12 benchmarks are a mix of crafted as well as industrial
benchmarks, whereas all of the pedigree benchmarks are from the same biological
problem [13]. The PB’12 benchmarks have on average several PBCs, however,
they are relatively small in magnitude. In contrast, the pedigree benchmarks
contain one large PB constraint with very large total weighted sum. pedigree
benchmarks have only two distinct values of weights, thus making them good
candidates for using GTE.
Results: Tab. 2 shows the number of instances solved using different encodings.
sorter, adder and swc perform worse than the remaining encodings for both sets of
benchmarks. The first two are not arc-consistent therefore the SAT solver is not
2
Available at http://sat.inesc-id.pt/∼ruben/benchmarks/pb12-subset.zip
3
http://www.maxsat.udl.cat/14/
4
Available at http://sat.inesc-id.pt/∼ruben/benchmarks/pedigrees.zip
206 S. Joshi et al.

·107 ·107
1.2
swc
sorter
watchdog 1 swc
3 watchdog
gte
bin-merger sorter
0.8
bdd bin-merger
adder bdd
#Vars

#Vars
2 0.6 gte
adder

0.4
1
0.2

0 0
170 180 190 200 210 0 20 40 60 80 100 120 140 160
instances instances

(a) # Variables on PB’12 benchmarks (b) # Variables on pedigree benchmarks

·107 ·107
2
swc
gte swc
adder
3 watchdog
sorter 1.5 gte
watchdog sorter
bin-merger bin-merger
#Clauses

#Clauses

bdd bdd
2 1 adder

1 0.5

0 0
150 160 170 180 190 200 210 0 20 40 60 80 100 120 140 160
instances instances

(c) # Clauses on PB’12 benchmarks (d) # Clauses on pedigree benchmarks

sorter sorter
swc swc
adder adder
1,500 bin-merger 1,500
watchdog
watchdog bin-merger
bdd bdd
seconds

seconds

gte gte
1,000 1,000

500 500

0 0
100 120 140 160 0 20 40 60 80 100 120 140 160
instances instances

(e) Runtime on PB’12 benchmarks (f) Runtime on pedigree benchmarks

Fig. 2. Cactus plots of number of variables, number of clauses and runtimes

 Generalized Totalizer Encoding for Pseudo-Boolean Constraints 207

able to infer as much information as with arc-consistent encodings. swc, though

arc-consistent, generates a large number of auxiliary variables and clauses, which
deteriorates the performance of the SAT solver.
gte provides a competitive performance to bdd, bin-merger and watchdog for
PB’12. However, only the gte and bdd encodings are able to tackle pedigree
benchmarks, which contain a large number of literals and only two different
coefficients. Unlike other encodings, gte and bdd are able to exploit the charac-
teristics of these benchmarks.
swc requires significantly large number of variables as the value of k increases,
whereas bdd and gte keep the variable explosion in check due to reuse of variables
on similar combinations (Figs. 2a and 2b). This reuse of auxiliary variables
is even more evident on pedigree benchmarks (Fig. 2b) as these benchmarks
have only two different coefficients resulting in low number of combinations. k-
simplification also helps gte in keeping the number of variables low as all the
combinations weighing more than k + 1 are mapped to k + 1.
Number of clauses required for gte is quite large as compared to some other
encodings (Figs. 2c and 2d). gte requires clauses to be generated for all the
combinations even though most of them produce the same value for the weighted
sum, thus reusing the same variable. Though bdd has an exponential worst case,
in practice it appears to generate smaller formulas (Figs. 2c and 2d).
Fig. 2e shows that gte provides a competitive performance with respect to
bin-merger, watchdog and bdd. Runtime on pedigree benchmarks as shown in
Fig. 2f establishes gte as the clear winner with bdd performing a close second.
The properties that gte and bdd share help them perform better on pedigree
benchmarks as they are not affected by large magnitude of weights in the PBCs.

5 Conclusion

Many real-world problems can be formulated using pseudo-Boolean constraints

(PBC). Given the advances in SAT technology, it becomes crucial how to encode
PBC into SAT, such that SAT solvers can efficiently solve the resulting formula.
In this paper, an arc-consistency preserving generalization of the Totalizer
encoding is proposed for encoding PBC into SAT. Although the proposed encod-
ing is exponential in the worst case, the new Generalized Totalizer encoding
(GTE) is very competitive in relation with other PBC encodings. Moreover,
experimental results show that when the number of different weights in PBC
is small, it clearly outperforms all other encodings. As a result, we believe the
impact of GTE can be extensive, since one can further extend it into incremental
settings [23].

Acknowledgments. This work is partially supported by the ERC project

280053, FCT grants AMOS (CMUP-EPB/TIC/0049/2013), POLARIS (PTDC/EIA-
CCO/123051/2010), and INESC-ID’s multiannual PIDDAC UID/CEC/50021/2013.
208 S. Joshi et al.

References
1. Abı́o, I., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E., Mayer-Eichberger,
V.: A New Look at BDDs for Pseudo-Boolean Constraints. Journal of Artificial
Intelligence Research 45, 443–480 (2012)
2. Aloul, F., Ramani, A., Markov, I., Sakallah, K.: Generic ILP versus specialized
0–1 ILP: an update. In: International Conference on Computer-Aided Design,
pp. 450–457. IEEE Press (2002)
3. Argelich, J., Berre, D.L., Lynce, I., Marques-Silva, J., Rapicault, P.: Solving linux
upgradeability problems using boolean optimization. In: Workshop on Logics for
Component Configuration, pp. 11–22 (2010)
4. Ası́n, R., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E.: Cardinality Net-
works: a theoretical and empirical study. Constraints 16(2), 195–221 (2011)
5. Bailleux, O., Boufkhad, Y.: Efficient CNF encoding of boolean cardinality con-
straints. In: Rossi, F. (ed.) CP 2003. LNCS, vol. 2833, pp. 108–122. Springer,
Heidelberg (2003)
6. Bailleux, O., Boufkhad, Y., Roussel, O.: A Translation of Pseudo Boolean Con-
straints to SAT. Journal on Satisfiability, Boolean Modeling and Computation
2(1–4), 191–200 (2006)
7. Bailleux, O., Boufkhad, Y., Roussel, O.: New encodings of pseudo-boolean con-
straints into CNF. In: Kullmann, O. (ed.) SAT 2009. LNCS, vol. 5584, pp. 181–194.
Springer, Heidelberg (2009)
8. Barth, P.: Logic-based 0–1 Constraint Programming. Kluwer Academic Publishers
(1996)
9. Büttner, M., Rintanen, J.: Satisfiability planning with constraints on the number
of actions. In: International Conference on Automated Planning and Scheduling,
pp. 292–299. AAAI Press (2005)
10. Eén, N., Sörensson, N.: An extensible SAT-solver. In: Giunchiglia, E., Tacchella, A.
(eds.) SAT 2003. LNCS, vol. 2919, pp. 502–518. Springer, Heidelberg (2004)
11. Eén, N., Sörensson, N.: Translating Pseudo-Boolean Constraints into SAT. Journal
on Satisfiability, Boolean Modeling and Computation 2(1–4), 1–26 (2006)
12. Gent, I.P.: Arc consistency in SAT. In: European Conference on Artificial Intelli-
gence, pp. 121–125. IOS Press (2002)
13. Graça, A., Lynce, I., Marques-Silva, J., Oliveira, A.L.: Efficient and accurate haplo-
type inference by combining parsimony and pedigree information. In: Horimoto, K.,
Nakatsui, M., Popov, N. (eds.) ANB 2010. LNCS, vol. 6479, pp. 38–56. Springer,
Heidelberg (2012)
14. Hölldobler, S., Manthey, N., Steinke, P.: A compact encoding of pseudo-boolean
constraints into SAT. In: Glimm, B., Krüger, A. (eds.) KI 2012. LNCS, vol. 7526,
pp. 107–118. Springer, Heidelberg (2012)
15. Ignatiev, A., Janota, M., Marques-Silva, J.: Towards efficient optimization in pack-
age management systems. In: International Conference on Software Engineering,
pp. 745–755. ACM (2014)
16. Janota, M., Lynce, I., Manquinho, V.M., Marques-Silva, J.: PackUp: Tools for
Package Upgradability Solving. JSAT 8(1/2), 89–94 (2012)
17. Koshimura, M., Zhang, T., Fujita, H., Hasegawa, R.: QMaxSAT: A Partial Max-
SAT Solver. Journal on Satisfiability, Boolean Modeling and Computation 8,
95–100 (2012)
18. Le Berre, D., Parrain, A.: The Sat4j library, release 2.2. Journal on Satisfiability,
Boolean Modeling and Computation 7(2–3), 59–64 (2010)
 Generalized Totalizer Encoding for Pseudo-Boolean Constraints 209

19. Manquinho, V., Marques-Silva, J., Planes, J.: Algorithms for weighted boolean
optimization. In: Kullmann, O. (ed.) SAT 2009. LNCS, vol. 5584, pp. 495–508.
Springer, Heidelberg (2009)
20. Manthey, N., Philipp, T., Steinke, P.: A more compact translation of pseudo-
boolean constraints into cnf such that generalized arc consistency is maintained.
In: Lutz, C., Thielscher, M. (eds.) KI 2014. LNCS, vol. 8736, pp. 123–134. Springer,
Heidelberg (2014)
21. Manthey, N., Steinke, P.: npSolver - A SAT based solver for optimization problems.
In: Pragmatics of SAT (2012)
22. Marques-Silva, J.: Integer programming models for optimization problems in test
generation. In: Asia and South Pacific Design Automation, pp. 481–487. IEEE
Press (1998)
23. Martins, R., Joshi, S., Manquinho, V., Lynce, I.: Incremental cardinality con-
straints for MaxSAT. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656,
pp. 531–548. Springer, Heidelberg (2014)
24. Miranda, M., Lynce, I., Manquinho, V.: Inferring phylogenetic trees using pseudo-
Boolean optimization. AI Communications 27(3), 229–243 (2014)
25. Ogawa, T., Liu, Y., Hasegawa, R., Koshimura, M., Fujita, H.: Modulo based CNF
encoding of cardinality constraints and its application to MaxSAT solvers. In:
International Conference on Tools with Artificial Intelligence, pp. 9–17. IEEE Press
(2013)
26. Prestwich, S., Quirke, C.: Boolean and pseudo-boolean models for scheduling. In:
International Workshop on Modelling and Reformulating Constraint Satisfaction
Problems (2003)
27. Ribas, B.C., Suguimoto, R.M., Montaño, R.A.N.R., Silva, F., de Bona, L., Castilho,
M.A.: On modelling virtual machine consolidation to pseudo-boolean constraints.
In: Pavón, J., Duque-Méndez, N.D., Fuentes-Fernández, R. (eds.) IBERAMIA
2012. LNCS, vol. 7637, pp. 361–370. Springer, Heidelberg (2012)
28. Sinz, C.: Towards an optimal CNF encoding of boolean cardinality constraints. In:
van Beek, P. (ed.) CP 2005. LNCS, vol. 3709, pp. 827–831. Springer, Heidelberg
(2005)
29. Steinke, P., Manthey, N.: PBLib-A C++ Toolkit for Encoding Pseudo-Boolean
Constraints into CNF. Tech. rep., Technische Universität Dresden (2014). http://
tools.computational-logic.org/content/pblib.php
30. Warners, J.: A Linear-Time Transformation of Linear Inequalities into Conjunctive
Normal Form. Information Processing Letters 68(2), 63–69 (1998)
 Smaller Selection Networks for Cardinality
Constraints Encoding

Michal Karpiński(B) and Marek Piotrów

Institute of Computer Science, University of Wroclaw,

Joliot-Curie 15, 50-383 WroclAw, Poland
{karp,mpi}@cs.uni.wroc.pl

Abstract. Selection comparator networks have been studied for many

years. Recently, they have been successfully applied to encode cardinal-
ity constraints for SAT-solvers. To decrease the size of generated formula
there is a need for constructions of selection networks that can be effi-
ciently generated and produce networks of small sizes for the practical
range of their two parameters: n – the number of inputs (Boolean vari-
ables) and k – the number of selected items (a cardinality bound). In
this paper we give and analyze a new construction of smaller selection
networks that are based on the pairwise selection networks introduced
by Codish and Zazon-Ivry. We prove also that standard encodings of
cardinality constraints with selection networks preserve arc-consistency.

1 Introduction
Comparator networks are probably the simplest data-oblivious model for sorting-
related algorithms. The most popular construction is due to Batcher [4] and
it’s called odd-even sorting network. For all practical values, this is the best
known sorting network. However, in 1992 Parberry [10] introduced the serious
competitor to Batcher’s construction, called pairwise sorting network. In context
of sorting, pairwise network is not better than odd-even network, in fact it has
been proven that they have exactly the same size and depth. As Parberry said
himself: ”It is the first sorting network to be competitive with the odd-even sort
for all values of n“. There is a more sophisticated relation between both types
of network and their close resemblance. For overview of sorting networks, see
Knuth [8] or Parberry [9].
In recent years new applications for sorting networks have been found, for
example in encoding of pseudo Boolean constraints and cardinality constraints
for SAT-solvers. Cardinality constraints take the form x1 + x2 + . . . + xn ∼ k,
where x1 , x2 , . . . , xn are Boolean variables, k is a natural number, and ∼ is a
relation from the set {=, <, ≤, >, ≥}. Cardinality constraints are used in many
applications, the significant ones worth mentioning arise in SAT-solvers. Using
cardinality constraints with cooperation of SAT-solvers we can handle many
practical problems, for example, cumulative scheduling [11] and timetabling [3],
that are proven to be hard. Works of Ası́n et al. [1,2] describe how to use odd-
even sorting network to encode cardinality constraints into Boolean formulas. In
[7] authors do the same with pseudo Boolean constraints.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 210–225, 2015.
DOI: 10.1007/978-3-319-23219-5 16
 Smaller Selection Networks for Cardinality Constraints Encoding 211

It has already been observed that using selection networks instead of sort-
ing networks is more efficient for the encoding of cardinality constraints. Codish
and Zazon-Ivry [6] introduced pairwise cardinality networks, which are networks
derived from pairwise sorting networks that express cardinality constraints. Two
years later, same authors [5] reformulated the definition of pairwise selection net-
works and proved that their sizes are never worse than the sizes of corresponding
odd-even selection networks. To show the difference they plotted it for selected
values of n and k.
In this paper we give a new construction of smaller selection networks that
are based on the pairwise selection ones and we prove that the construction is
correct. We also estimate the size of our networks and compute the difference in
sizes between our selection networks and the corresponding pairwise ones. The
difference can be as big as n log n/2 for k = n/2. Finally, we analyze the standard
3(6)-clause encoding of a comparator and prove that such CNF encoding of
any selection network preserves arc-consistency with respect to a corresponding
cardinality constraint.
The rest of the paper is organized in the following way: in Section 2 we give
definitions and notations used in this paper. In Section 3 we recall the definition
of pairwise selection networks and define auxiliary bitonic selection networks
that we will use to estimate the sizes of our networks. In Section 4 we present
the construction of our selection networks and prove its correctness. In Section
5 we analyze the sizes of the networks and, finally, in Section 6 we examine the
arc-consistency of selection networks.

2 Preliminaries
In this section we will introduce definitions and notations used in the rest of the
paper.
Definition 1 (Input Sequence). Input sequence of length n is a sequence of
natural numbers x̄ = x1 , . . . , xn , where xi ∈ N (for all i = 1..n). We say that
x̄ ∈ Nn is sorted if xi ≥ xi+1 (for each i = 1..n − 1). Given x̄ = x1 , . . . , xn ,
ȳ = y1 , . . . , yn  we define concatenation as x̄ :: ȳ = x1 , . . . , xn , y1 , . . . , yn .
We will use the following functions from Nn to Nn/2 :

lef t(x̄) = x1 , . . . , xn/2 , right(x̄) = xn/2+1 , . . . , xn 

Let n, m ∈ N. We define a relation ’’ on Nn × Nm . Let x̄ = x1 , . . . , xn  and
ȳ = y1 , . . . , ym , then:

x̄  ȳ ⇐⇒ ∀i∈{1,...,n} ∀j∈{1,...,m} xi ≥ yj
Definition 2 (Comparator). Let x̄ ∈ Nn and let i, j ∈ N, where 1 ≤ i < j ≤ n.
A comparator is a function ci,j defined as:

ci,j (x̄) = ȳ ⇐⇒ yi = max{xi , xj } ∧ yj = min{xi , xj } ∧ ∀k=i,j xk = yk

212 M. Karpiński and M. Piotrów

Definition 3 (Comparator Network). We say that f n : Nn → Nn is a com-

parator network of order n, if it can be represented as the composition of finite
number of comparators, namely, f n = ci1 ,j1 ◦ · · · ◦ cik ,jk . The size of comparator
network (number of comparators) is denoted by |f n |. Comparator network of size
0 is denoted by idn .

Definition 4 (V-shaped and Bitonic Sequences). A sequence x̄ ∈ Nn is

called v-shaped if x1 ≥ . . . ≥ xi ≤ . . . ≤ xn for some i, where 1 ≤ i ≤ n. A
v-shaped sequence or its circular shift is traditionally called bitonic.

Definition 5 (Sorting Network). A comparator network f n is a sorting net-

work, if for each x̄ ∈ Nn , f n (x̄) is sorted.

Two types of sorting networks are of interest to us: odd-even and pairwise.
Based on their ideas, Knuth [8] (for odd-even network) and Codish and Zazon-
Ivry [5,6] (for pairwise network) showed how to transform them into selection
networks (we name them oe selkn and pw selkn respectively).

Definition 6 (Top k Sorted Sequence). A sequence x̄ ∈ Nn is top k sorted,

with k ≤ n, if x1 , . . . , xk  is sorted and x1 , . . . , xk   xk+1 , . . . , xn .

Definition 7 (Selection Network). A comparator network fkn (where k ≤ n)

is a selection network, if for each x̄ ∈ Nn , fkn (x̄) is top k sorted.

To simplify the presentation we assume that n and k are powers of 2.

A clause is a disjunction of literals (Boolean variables x or their negation
¬x). A CNF formula is a conjunction of one or more clauses.
A unit propagation (UP) is a process, that for given CNF formula, clauses are
sought in which all literals but one are false (say l) and l is undefined (initially
only clauses of size one satisfy this condition). This literal l is set to true and
the process is iterated until reaching a fix point.
Cardinality constraints are of the form x1 + . . . + xn ∼ k, where k ∈ N
and ∼ belongs to {<, ≤, =, ≥, >}. We will focus on cardinality constraints with
less-than relation, i.e. x1 + . . . + xn < k. An encoding (a CNF formula) of such
constraint preserves arc-consistency, if as soon as k − 1 variables among the xi ’s
become true, the unit propagation sets all other xi ’s to false.
In [7] authors are using sorting networks for an encoding of cardinality con-
straints, where inputs and outputs of a comparator are Boolean variables and
comparators are encoded as a CNF formula. In addition, the k-th greatest out-
put variable yk of the network is forced to be 0 by adding ¬yk as a clause to
the formula that encodes x1 + . . . + xn < k. They showed that the encoding
preserves arc-consistency.
A single comparator can be translated to a CNF formula in the following way:
let a and b be variables denoting upper and lower inputs of the comparator, and
c and d be variables denoting upper and lower outputs of a comparator, then:

f comp(a, b, c, d) ⇔ (c ⇔ a ∨ b) ∧ (d ⇔ a ∧ b)
 Smaller Selection Networks for Cardinality Constraints Encoding 213

is the full encoding of a comparator. Notice that it consists of 6 clauses. Let f

be a comparator network. Full encoding φ of f is a conjunction of full encoding
of every comparator of f .
In [2] authors observe that in case of ∼ being < or ≤, it is sufficient to use
only 3 clauses for a single comparator, namely:

hcomp(a, b, c, d) ⇔ (a ⇒ c) ∧ (b ⇒ c) ∧ (a ∧ b ⇒ d) (1)
        
(c1) (c2) (c3)

We call it: a half encoding. In [2] it is used to translate an odd-even sorting

network to an encoding that preserves arc-consistency. We show a more general
result (with respect to both [7] and [2]), that the half encoding of any selection
network preserves arc-consistency for the “<” and “≤” relations. Similar results
can be proved for the “=” relation using the full encoding of comparators and
for the “>” or “≥” relations using an encoding symmetric to hcomp(a, b, c, d),
namely: (d ⇒ a) ∧ (d ⇒ b) ∧ (c ⇒ a ∨ b).

3 Pairwise and Bitonic Selection Networks

Now we present two constructions for selection networks. First, we recall the def-
inition of pairwise selection networks by Codish and Zazon-Ivry [5,6]. Secondly,
we give the auxiliary construction of a bitonic selection network bit selkn , that
we will use to estimate the sizes of our improved pairwise selection network in
Section 5.

Definition 8 (Domination). x̄ ∈ Nn dominates ȳ ∈ Nn if xi ≥ yi (for i =

1..n).

Definition 9 (Splitter). A comparator network f n is a splitter if for any

sequence x̄ ∈ Nn , if ȳ = f n (x̄), then lef t(ȳ) dominates right(ȳ).

Observation 1. We can construct splitter splitn by joining inputs i, n/2 + i,
for i = 1..n/2, with a comparator. The size of a splitter is |splitn | = n/2.

Lemma 1. If b̄ ∈ Nn is bitonic and ȳ = splitn (b̄), then lef t(ȳ) and right(ȳ)
are bitonic and lef t(ȳ)  right(ȳ).

Proof. See Appendix B of [4].

The construction of a pairwise selection network is presented in Network 1.

Notice that since a splitter is used as the third step, in the recursive calls we need
to select k top elements from the first half of ȳ, but only k/2 top elements from
the second half. The reason is that rk/2+1 cannot be one of the first k largest
elements of ¯l :: r̄. First, rk/2+1 is not greater than any one of r1 , . . . , rk/2  (by
the definition of top k sorted sequence), and second, l1 , . . . , lk/2  dominates
r1 , . . . , rk/2 , so rk/2+1 is not greater than any one of l1 , . . . , lk/2 . Based on
these arguments we can make the following observation:
214 M. Karpiński and M. Piotrów

Network 1. pw selkn ; see [5,6]

Input: any x̄ ∈ Nn
1: if k = 1 then return maxn (x̄)
2: if k = n then return oe sortn (x̄)
3: ȳ ← split(x̄)
n/2 n/2
4: ¯
l ← pw selk (lef t(ȳ)) and r̄ ← pw selk/2 (right(ȳ))
n ¯
5: return pw mergek (l :: r̄)

Observation 2. If ¯l ∈ Nn/2 is top k sorted, r̄ ∈ Nn/2 is top k/2 sorted and

l1 , . . . , lk/2  dominates r1 , . . . , rk/2 , then k largest elements of ¯l :: r̄ are in
l1 , . . . , lk  :: r1 , . . . , rk/2 .
The last step of Network 1 merges k top elements from ¯l and k/2 top elements
from r̄ with so called pairwise merger. We will omit the construction of this
merger, because it is not relevant to our work. We would only like to note, that
its size is: |pw mergenk | = k log k − k + 1. Construction of the merger as well as
the detailed proof of correctness of network pw selkn can be found in Section 6
of [5].

Definition 10 (Bitonic Splitter). A comparator network f n is a bitonic split-

ter if for any two sorted sequences x̄, ȳ ∈ Nn/2 , if z̄ = f n (x̄ :: ȳ), then (1)
lef t(z̄)  right(z̄) and (2) lef t(z̄) and right(z̄) are bitonic.

Observation 3. We can construct bitonic splitter bit splitn by joining inputs

i, n − i + 1, for i = 1..n/2, with a comparator. The size of a bitonic splitter is
|bit splitn | = n/2.
We now present the procedure for construction of the bitonic selection net-
work. We use the odd-even sorting network oe sort and the network bit merge
(also by Batcher [4]) for sorting bitonic sequences as black-boxes. As a reminder:
bit mergen consists of two steps, first we use ȳ = splitn (x̄), then recursively com-
pute bit mergen/2 for lef t(ȳ) and right(ȳ) (base case, n = 2, consists of a single
comparator). The size of this network is: |bit mergen | = n log n/2. A bitonic
selection network bit selkn is constructed by the procedure Network 2.

Theorem 1. A comparator network bit selkn constructed by the procedure Net-

work 2 is a selection network.

Proof. Let x̄ ∈ Nn be the input to bit selkn . After step one we get sorted sequences
B1 , . . . , Bl , where l = n/k. Let lm be the value of l after m iterations. Let
B1m , . . . , Blmm
be the blocks after m iterations. We will prove by induction that:
P (m): if B1 , . . . , Bl are sorted contain k largest elements of x̄, then after m-
th iteration of the second step: lm = l/2m , B1m , . . . , Blmm
are sorted and contain
k largest elements of x̄.
If m = 0, then l0 = l, so P (0) holds. We show that ∀m≥0 (P (m) ⇒ P (m + 1)).
Consider (m+1)-th iteration of step two. By the induction hypothesis lm = l/2m ,
 Smaller Selection Networks for Cardinality Constraints Encoding 215

Network 2. bit selkn

Input: any x̄ ∈ Nn
1: l ← n/k and partition input x̄ into l consecutive blocks, each of size k, then sort
each block with oe sortk , obtaining sorted blocks B1 , . . . , Bl
2: while l > 1 do
3: Collect blocks into pairs B1 , B2 , . . . , Bl−1 , Bl 
4: for all i ∈ {1, 3, . . . , l − 1} do y¯i ← bit split2k (Bi :: Bi+1 )

5: for all i ∈ {1, 3, . . . , l − 1} do Bi/2 ← bit mergek (lef t(y¯i ))
6: l ← l/2 and relabel Bi to Bi , for 1 ≤ i ≤ l

B1m , . . . , Blm
m
are sorted and contain k largest elements of x̄. We will show that
(m + 1)-th iteration does not remove any element from k largest elements of x̄.
To see this, notice that if y¯i = bit split2k (Bim :: Bi+1 m
) (for i ∈ {1, 3, . . . , lm − 1}),
then lef t(y¯i )  right(y¯i ) and that lef t(y¯i ) is bitonic (by Definition 10). Because
of those two facts, right(y¯i ) is discarded and lef t(y¯i ) is sorted using bit mergek .
After this, lm+1 = lm /2 = l/2m+1 and blocks B1m+1 , . . . , Blm+1 m+1
are sorted. Thus
P (m + 1) is true.
Since l = n/k, then by P (m) we see that the second step will terminate after
m = log nk iterations and that B1 is sorted and contains k largest elements of
x̄.

The size of bitonic selection network is:

n n 
|bit selkn | =|oe sortk | + − 1 (|bit split2k | + |bit mergek |)
k k
1 1 1 n
= n log k + n log k + 2n − k log k − k −
2
(2)
4 4 2 k

4 New Smaller Selection Networks

As mentioned in the previous section, only the first k/2 elements from the second
half of the input are relevant when we get to the merging step in pw selkn . We
will exploit this fact to create a new, smaller merger. We will use the concept
of bitonic sequences, therefore the new merger will be called pw bit mergenk and
the new selection network: pw bit selkn . The network pw bit selkn is generated by
substituting the last step of pw selkn with pw bit mergenk . The new merger is
constructed by the procedure Network 3.

Theorem 2. The output of Network 3 consists of sorted k largest elements from

input ¯l :: r̄, assuming that ¯l ∈ Nn/2 is top k sorted and r̄ ∈ Nn/2 is top k/2 sorted
and l1 , . . . , lk/2  dominates r1 , . . . , rk/2 .

Proof. We have to prove two things: (1) b̄ is bitonic and (2) b̄ consists of k largest
elements from ¯l :: r̄.
216 M. Karpiński and M. Piotrów

Network 3. pw bit mergenk

Input: ¯ l :: r̄, where ¯ l ∈ Nn/2 is top k sorted and r̄ ∈ Nn/2 is top k/2 sorted and
l1 , . . . , lk/2  dominates r1 , . . . , rk/2 
1: ȳ ← bit splitk (lk/2+1 , . . . , lk , r1 , . . . , rk/2 ) and b̄ ← l1 , . . . , lk/2  :: y1 , . . . , yk/2 
2: return bit mergek (b̄)

(1) Let j be the last index in the sequence k/2 + 1, . . . , k, for which lj >
rk−j+1 . If such j does not exist, then y1 , . . . , yk/2  is nondecreasing, hence b̄
is bitonic (nondecreasing). Assume that j exists, then yj−k/2+1 , . . . , yk/2  is
nondecreasing and y1 , . . . , yk−j  is nonincreasing. Adding the fact that lk/2 ≥
lk/2+1 = y1 proves, that b̄ is bitonic (v-shaped).
(2) By Observation 2, it is sufficient to prove that b̄  yk/2+1 , . . . , yk .
Since ∀k/2<j≤k lk/2 ≥ lj ≥ min{lj , rk−j+1 } = y3k/2−j+1 , then l1 , . . . , lk/2  
yk/2+1 , . . . , yk  and by Definition 10: y1 , . . . , yk/2   yk/2+1 , . . . , yk . Therefore
b̄ consists of k largest elements from ¯l :: r̄.
The bitonic merger in step 2 receives a bitonic sequence, so it outputs a
sorted sequence, which completes the proof.
The first step of improved pairwise merger is illustrated in Figure 1. We
use k/2 comparators in the first step and k log k/2 comparators in the second
step. We get a merger of size k log k/2 + k/2, which is better than the previous
approach. In the following it is shown that we can do even better and eliminate
k/2 term.

Fig. 1. Making the bitonic sequence. Arrows on the right picture show directions of
inequalities. Sequence on the right is v-shaped s-dominating at point i.

The main observation is that the result of the first step of pw bit merge oper-
ation: b1 , b2 , . . . , bk  is not only bitonic, but what we call v-shaped s-dominating.

Definition 11 (S-domination). A sequence b̄ = b1 , b2 , . . . , bk  is s-dominating

if ∀1≤j≤k/2 bj ≥ bk−j+1 .

Lemma 2. If b̄ = b1 , b2 , . . . , bk  is v-shaped and s-dominating, then b̄ is non-

increasing or ∃k/2<i<k bi < bi+1 .
 Smaller Selection Networks for Cardinality Constraints Encoding 217

Proof. Assume that b̄ is not nonincreasing. Then ∃1≤j<k bj < bj+1 . Assume that
j ≤ k/2. Since b̄ is v-shaped, bj+1 must be in nondecreasing part of b̄. If follows
that bj < bj+1 ≤ . . . ≤ bk/2 ≤ . . . ≤ bk−j+1 . That means that bj < bk−j+1 . On
the other hand, b̄ is s-dominating, thus bj ≥ bk−j+1 – a contradiction.

We will say that a sequence b̄ is v-shaped s-dominating at point i if i is the

smallest index greater than k/2 such that bi < bi+1 or i = k for a nonincreasing
sequence.

Lemma 3. Let b̄ = b1 , b2 , . . . , bk  be v-shaped s-dominating at point i, then

b1 , . . . , bk/4   bk/2+1 , . . . , b3k/4 .

Proof. If b̄ is nonincreasing, then the lemma holds. Otherwise, from Lemma 2:

k/2 < i < k. If i > 3k/4, then by Definition 4: b1 ≥ . . . ≥ b3k/4 ≥ . . . ≥ bi ,
so the lemma holds. If k/2 < i ≤ 3k/4, then by Definition 4: b1 ≥ . . . ≥ bi , so
b1 , . . . , bk/4   bk/2+1 , . . . , bi . Since bi < bi+1 ≤ . . . ≤ b3k/4 , it suffices to prove
that bk/4 ≥ b3k/4 . By Definition 11 and 4: bk/4 ≥ b3k/4+1 ≥ b3k/4 .

Definition 12 (Half Splitter). A half splitter is a comparator network con-

structed by comparing inputs k/4 + 1, 3k/4 + 1, . . . , k/2, k (normal splitter
with first k/4 comparators removed). We will call it half splitk .

Lemma 4. If b̄ is v-shaped s-dominating, then half splitk (b̄) = splitk (b̄).

Proof. Directly from Lemma 3.

Lemma 5. Let b̄ be v-shaped s-dominating. The following statements are true:

(1) lef t(half splitk (b̄)) is v-shaped s-dominating; (2) right(half splitk (b̄)) is
bitonic; (3) lef t(half splitk (b̄))  right(half splitk (b̄)).

Proof. (1) Let ȳ = lef t(half splitk (b̄)). First we show that ȳ is v-shaped. If ȳ is
nonincreasing, then it is v-shaped. Otherwise, let j be the first index from the
range {1, . . . , k/2}, where yj−1 < yj . Since yj = max{bj , bj+k/2 } and yj−1 ≥
bj−1 ≥ bj , thus bj < bj+k/2 . Since b̄ is v-shaped, element bj+k/2 must be in
nondecreasing part of b̄. It follows that bj ≥ . . . ≥ bk/2 and bj+k/2 ≤ . . . ≤ bk .
From this we can see that ∀j≤j  ≤k/2 yj  = max{bj  , bj  +k/2 } = bj  +k/2 , so yj ≤
. . . ≤ yk/2 . Therefore ȳ is v-shaped.
Next we show that ȳ is s-dominating. Consider any j, where 1 ≤ j ≤ k/4.
By Definition 4 and 11: bj ≥ bk/2−j+1 and bj ≥ bk−j+1 , therefore yj = bj ≥
max{bk/2−j+1 , bk−j+1 } = yk/2−j+1 , thus proving that ȳ is s-dominating. Con-
cluding: ȳ is v-shaped s-dominating.
(2) Let z̄ = right(half splitk (b̄)). By Lemma 4: z̄ = right(splitk (b̄)). We
know that b̄ is a special case of bitonic sequence, therefore using Lemma 1 we
get that z̄ is bitonic.
(3) Let w̄ = half splitk (b̄). By Lemma 4: w̄ = splitk (b̄). We know that b̄ is
a special case of bitonic sequence, therefore using Lemma 1 we get lef t(w̄) 
right(w̄).
218 M. Karpiński and M. Piotrów

Network 4. pw hbit mergenk

Input: ¯ l :: r̄, where ¯ l ∈ Nn/2 is top k sorted, r̄ ∈ Nn/2 is top k/2 sorted and
l1 , . . . , lk/2  dominates r1 , . . . , rk/2 
1: ȳ ← bit splitk (lk/2+1 , . . . , lk , r1 , . . . , rk/2 ) and b̄ ← l1 , . . . , lk/2  :: y1 , . . . , yk/2 
2: return half bit mergek (b̄), where
3: function half bit mergek (b̄)
4: if k = 2 then return (b1 , b2 )
5: b¯ ← half split(b1 , . . . , bk )
6: return half bit mergek/2 (lef t(b¯ )) :: bit mergek/2 (right(b¯ ))

Using both half split and Batcher’s bit merge and successively applying
Lemma 5 to the resulting v-shaped s-dominating half of the output, we have
all the tools needed to construct the improved pairwise merger pw hbit mergenk
(Network 4) using half splitters and then to prove that the construction is correct.
Theorem 3. The output of Network 4 consists of sorted k largest elements from
input ¯l :: r̄, assuming that ¯l ∈ Nn/2 is top k sorted and r̄ ∈ Nn/2 is top k/2
sorted and l1 , . . . , lk/2  dominates r1 , . . . , rk/2 . Moreover, |pw hbit mergenk | =
k log k/2.
Proof. Since step 1 in Network 4 is the same as in Network 3, we can reuse
the proof of Theorem 2 to deduce, that b̄ is v-shaped and contains k largest
elements from ¯l :: r̄. Also, since ∀1≤j≤k/2 lj ≥ lk−j+1 and lj ≥ rj , then bj = lj ≥
max{lk−j+1 , rj } = bk−j+1 , so b̄ is s-dominating.
We prove by the induction on k, that if b̄ is v-shaped s-dominating, then the
sequence half bit mergek (b̄) is sorted. For the base case, consider k = 2 and a v-
shaped s-dominating sequence b1 , b2 . By Definition 11 this sequence is already
sorted and we are done. For the induction step, consider b¯ = half splitk (b̄). By
Lemma 5 we get that lef t(b¯ ) is v-shaped s-dominating and right(b¯ ) is bitonic.
Using the induction hypothesis we sort lef t(b¯ ) and using bitonic merger we
sort right(b¯ ). By Lemma 5: lef t(b¯ )  right(b¯ ), which completes the proof of
correctness.
As mentioned in Definition 12: half splitk is just splitk with the first k/4
comparators removed. So half bit mergek is just bit mergek with some com-
parators removed. Let’s count them: in each level of recursion step we take half
of comparators from splitk and additional one comparator from the base case
(k = 2). We sum them together to get:

log k−1
 


log k−2
k k  1
i
2 k 2 2 k
1+ =1+ − =1+ 2− − =
i=0
2i+2 4 i=0
2 k 4 k k 2

Therefore we have:
|pw hbit mergenk | = k/2 + k log k/2 − k/2 = k log k/2
 Smaller Selection Networks for Cardinality Constraints Encoding 219

The only difference between pw sel and our pw hbit sel is the use of improved
merger pw hbit merge rather than pw merge. By Theorem 3, we conclude that
|pw mergenk | ≥ |pw hbit mergenk |, so it follows that:
Remark 1. |pw hbit selkn | ≤ |pw selkn |

5 Sizes of New Selection Networks

In this section we estimate the size of pw hbit selkn . To this end we show that the
size of pw hbit selkn is upper-bounded by the size of bit selkn and use this fact in
our estimation. We also compute the exact difference between sizes of pw selkn
and pw hbit selkn and show that it can be as big as n log n/2. Finally we show
graphically how much smaller is our selection network on practical values of n
and k.
We have the recursive formula for the number of comparators of pw hbit selkn :

⎧ n/2 n/2
⎪
⎪ |pw hbit selk | + |pw hbit selk/2 |+
⎨
|pw hbit selkn | = +|split k| + |pw hbit merge |
n k
if k < n (3)
⎪
⎩ |oe sort
⎪ | if k = n
|maxn | if k = 1
Lemma 6. |pw hbit selkn | ≤ |bit selkn |.
Proof. Let aux selkn be the comparator network that is generated by substituting
recursive calls in pw hbit selkn by calls to bit selkn . Size of this network (for 1 <
k < n) is:

n/2 n/2
|aux selkn | = |bit selk | + |bit selk/2 | + |splitn | + |pw hbit mergek | (4)
Lemma 6 follows from Lemma 7 and Lemma 8 below, where we show that:

|pw hbit selkn | ≤ |aux selkn | ≤ |bit selkn |

Lemma 7. For 1 < k < n (both powers of 2), |aux selkn | ≤ |bit selkn |.
Proof. We compute both values from Equation 2 and 4:

1 5 1 5 3n
|aux selkn | = n log2 k + n − k log k − k −
4 2 4 4 2k
1 1 1 n
|bit selk | = n log k + n log k + 2n − k log k − k −
n 2
4 4 2 k
We simplify both sides to get the following inequality:
1 n 1
n − k − ≤ (n − k) log k
2 k 2
which can be easily proved by induction.
220 M. Karpiński and M. Piotrów

Lemma 8. For 1 ≤ k < n (both powers of 2), |pw hbit selkn | ≤ |aux selkn |.

Proof. By induction. For the base case, consider 1 = k < n. If follows by defi-
nitions that |pw hbit selkn | = |aux selkn | = n − 1. For the induction step assume
that for each (n , k  ) ≺ (n, k) (in lexicographical order) the lemma holds, we get:

|pw hbit selkn |

n/2 n/2
= |pw hbit selk/2 | + |pw hbit selk | + |splitn | + |pw hbit mergek |
(by the definition of pw hbit sel)
n/2 n/2
≤ |aux selk/2 | + |aux selk | + |splitn | + |pw hbit mergek |
(by the induction hypothesis)
n/2 n/2
≤ |bit selk/2 | + |bit selk | + |splitn | + |pw hbit mergek |
(by Lemma 7)
= |aux selkn |
(by the definition of aux sel)

Let N = 2n and K = 2k . We will compute upper bound for P (n, k) =

|pw hbit selK
N
| using B(n, k) = |bit selK
N
|.

Lemma 9. Let:

 i
 m

m−1
i    m
P (n, k, m) = (k − j)2k−j−1
+2n−i−1
+ P (n − m, k − i).
i=0 j=0
j i=0
i

Then ∀0≤m≤min(k,n−k) P (n, k, m) = P (n, k).

Proof. The lemma can be easily proved by induction on m.

 2     
k 3 m k
Lemma 10. P (n, k, m) ≤ 2n−2 k − m 2 + k + 7m
4 + 8 +2 2 2 −
m
6 −
 
n−k 3 m
2 (k + 1) − 2
k
2 .

Proof. Due to space restriction we only present schema of the proof. The lemma
can be proven directly from the inequality below, which is a consequence of
Lemma 9 and 6.

 i
 m

m−1
i    m
P (n, k, m) ≤ (k − j)2k−j−1
+2n−i−1
+ B(n − m, k − i)
i=0 j=0
j i=0
i

  
7 2
Theorem 4. For m = min(k, n−k), P (n, k) ≤ 2n−2 k− m
2 − 4 + 9k
2 + 79
16
 m  k m   
n−k 3 m
+2k 32 2 − 6 − 2 (k + 1) − 2
k
2 .
 Smaller Selection Networks for Cardinality Constraints Encoding 221

Proof. Directly from Lemma 9 and 10.

We will now present the size difference SD(n, k) between pairwise selection
network and our network. Merging step in pw selK N
costs 2k k − 2k + 1 and in
N k−1
pw hbit selK : 2 k, so the difference is given by the following equation:

⎧
⎪
⎪ 0 if n = k
⎨
0 if k = 0
SD(n, k) = (5)
⎪
⎪ 2k−1
k − 2 k
+ 1+
⎩
+SD(n − 1, k) + SD(n − 1, k − 1) if 0 < k < n
k   k−j
Theorem 5. Let Sn,k = j=0 n−k+j j 2 . Then:


n n+1 n − 2k + 1
SD(n, k) = − Sn,k − 2k (k − 1) − 1
k 2 2
Proof. By straightforward calculation
  one can verify that Sn,0 = 1, Sn,n =
2n+1 − 1, Sn−1,k−1 = 12 (Sn,k − nk ) and Sn−1,k−1 + Sn−1,k = Sn,k . It follows that
the theorem is true for k = 0 and k = n. We prove the theorem by induction
on pairs (k, n). Take any (k, n), 0 < k < n, and assume that theorem holds for
every (k  , n ) ≺ (k, n) (in lexicographical order). Then we have:

SD(n, k) = 2k−1 k − 2k + 1 + SD(n − 1, k) + SD(n − 1, k − 1)



n−1 n n−1 n
=2 k−1
k−2 +1+
k
+ − 2k (k − 1) − 1
k 2 k−1 2
n − 2k n − 2k + 2
− 2k−1 (k − 2) − 1 − (Sn−1,k + Sn−1,k−1 )

 2 2
n n n − 2k
= − Sn,k − Sn−1,k−1 − 2k (k − 1) − 1
k 2 2


n n+1 n − 2k + 1
= − Sn,k − 2k (k − 1) − 1
k 2 2

−4
Corollary 1. |pw selN/2
N
|−|pw hbit selN/2
N
| = N log N
2 +log N +2, for N = 2n .
Plots in Figure 2 show how much pw sel and the upper bound from The-
orem 4 are worse than pw hbit sel. Lines labeled pw sel are plotted from
(|pw selKN
| − |pw hbit selK
N
|)/|pw hbit selK
N
| and the ones labeled upper are plot-
ted from the formula (|upperK |−|pw hbit selK
N N
|)/|pw hbit selK
N
|, where |upperKN
|
is the upper bound from Theorem 4. Both |pw selK | and |pw hbit selK | were
N N

computed directly from recursive formulas. We can see that we save the most
number of comparators when k is larger than n/2, nevertheless for small values
of n superiority of our network is apparent for any k. As for the upper bound,
it gives a good approximation of |pw hbit selK N
| when n is small, but for larger
values of n it becomes less satisfactory.
222 M. Karpiński and M. Piotrów

100 % 100 % 100 %

pw_sel pw_sel pw_sel
upper upper upper
80 % 80 % 80 %

60 % 60 % 60 %

40 % 40 % 40 %

20 % 20 % 20 %

0% 0% 0%
0 1 2 3 4 5 6 7 0 2 4 6 8 10 12 14 0 5 10 15 20 25 30
log(K) log(K) log(K)

(a) N = 27 (b) N = 215 (c) N = 231

N
Fig. 2. The relative percentage change of the size of pw selK and the upper bound
N
given in Thm 4 with respect to the size of pw hbit selK .

6 Arc-Consistency of Selection Networks

In this section we prove that half encoding of any selection network preserves
arc-consistency with respect to “less-than” cardinality constraints. The proof
can be generalized to other types of cardinality constraints.
We introduce the convention, that x1 , . . . , xn  will denote the input and
y1 , . . . , yn  will denote the output of some order n comparator network. We
would also like to view them as sequences of Boolean variables, that can be set
to either true (1), false (0) or undefined (X).
From now on we assume that every network f is half encoded and when we
say “comparator” or “network”, we view it in terms of CNF formulas. We denote
V [φ(f )] to be the set of variables in encoding φ(f ).

Observation 4. A single comparator hcomp(a, b, c, d) has the following propa-

gation properties:

1. If a = 1 or b = 1, then UP sets c = 1 (by 1.c1 or 1.c2).

2. If a = b = 1, then UP sets c = d = 1 (by 1.c1 and 1.c3).
3. If c = 0, then UP sets a = b = 0 (by 1.c1 and 1.c2).
4. If b = 1 and d = 0, then UP sets a = 0 (by 1.c3).
5. If a = 1 and d = 0, then UP sets b = 0 (by 1.c3).

Lemma 11. Let fkn be a selection network. Assume that k − 1 inputs are set
to 1, and rest of the variables are undefined. Unit propagation will set variables
y1 , . . . , yk−1 to 1.

Proof. From propagation properties of hcomp(a, b, c, d) we can see that if com-

parator receives two 1s, then it outputs two 1s, when it receives 1 on one input
and X on the other, then it outputs 1 on the upper output and X on the lower
output. From this we conclude that a single comparator will sort its inputs, as
long as one of the inputs is set to 1. No 1 is lost, so they must all reach the out-
puts. Because the comparators comprise a selection network, the 1s will appear
at outputs y1 , . . . , yk−1 .
 Smaller Selection Networks for Cardinality Constraints Encoding 223

The process of propagating 1s we call a forward propagation. For the remain-

der of this section assume that: fkn is a selection network; k − 1 inputs are set
to 1, and the rest of the variables are undefined; forward propagation has been
performed resulting in y1 , . . . , yk−1 to be set to 1.
Definition 13 (Path). A path is a sequence of Boolean variables z1 , . . . , zm 
such that ∀1≤i≤m zi ∈ V [φ(fkn )] and for all 1 ≤ i < m there exists a comparator
hcomp(a, b, c, d) in φ(fkn ) for which zi ∈ {a, b} and zi+1 ∈ {c, d}.
Definition 14 (Propagation Path). Let x be an undefined input variable. A
path z̄x = z1 , . . . , zm  (m ≥ 1) is a propagation path, if z1 ≡ x and z2 , . . . , zm 
is the sequence of variables that would be set to 1 by UP, if we would set z1 = 1.
Lemma 12. If z̄x = z1 , . . . , zm  is a propagation path for an undefined variable
x, then zm ≡ yk .
Proof. Remember that all y1 , . . . , yk−1 are set to 1. Setting any undefined input
variable x to 1 will result in UP to set yk to 1. Otherwise fkn would not be a
selection network.
The following lemma shows that propagation paths are deterministic.
Lemma 13. Let z̄x = z1 , . . . , zm  be a propagation path. For each 1 ≤ i ≤ m
and z1 ≡ zi , if z1 , . . . , zm

  is a path that would be set to 1 by UP if we would
  
set z1 = 1, then z1 , . . . , zm   = zi , . . . , zm .

Proof. By induction on l = m−i. If l = 0, then z1 ≡ zm ≡ yk (by Lemma 12), so

the lemma holds. Let l ≥ 0 and assume that the lemma is true for zl . Consider
z1 ≡ zl−1 ≡ zm−i−1 . Set zm−i−1 = 1 and use UP to set zm−i = 1. Notice that
zm−i ≡ z2 , otherwise there would exist a comparator hcomp(a, b, c, d), for which
zm−i−1 is equivalent to either a or b and zm−i ≡ c and z2 ≡ d (or vice versa).
That would mean that a single 1 on the input produces two 1s on the outputs.
This contradicts our reasoning in the proof of Lemma 11. By the induction
hypothesis z2 , . . . , zm
  
  = zm−i , . . . , zm , so z1 , . . . , zm  = zm−i−1 , . . . , zm .

For each undefined input variable x and propagation path z̄x = z1 , . . . , zm 
we define a directed graph Px = {zi , zi+1  : 1 ≤ i < m}.
Lemma 14. Let {xi1 , . . . , xit } (t > 0) be the set of undefined input variables.
Then T = Pxi1 ∪ . . . ∪ Pxit is the tree rooted at yk .
Proof. By induction on t. If t = 1, then T = Pxi1 and by Lemma 12, Pxi1 ends
in yk , so the lemma holds. Let t > 0 and assume that the lemma is true for t.
We will show that it is true for t + 1. Consider T = Pxi1 ∪ . . . ∪ Pxit ∪ Pxit+1 .
By the induction hypothesis T  = Pxi1 ∪ . . . ∪ Pxit is a tree rooted at yk . By
Lemma 12, V (Pxit+1 ) ∩ V (T  ) = ∅. Let z ∈ V (Pxit+1 ) be the first variable, such
that z ∈ V (T  ). Since z ∈ V (T  ), there exists j (1 ≤ j ≤ t) such that z ∈ Pxij .
By Lemma 13, starting from variable z, paths Pxit+1 and Pxij are identical.
224 M. Karpiński and M. Piotrów

Graph T from the above lemma will be called a propagation tree.

Theorem 6. If we set yk = 0, then unit propagation will set all undefined input
variables to 0.

Proof. Let T be the propagation tree rooted at yk . We prove by induction on

the height h of a subtree T  of T , that (*) if the root of T  is set to 0, then UP
sets all nodes of T  to 0. It follows that if yk is set to 0 then UP sets all undefined
input variables to 0. If h = 0, then V = {yk }, so (*) is trivially true. Let h > 0
and assume that (*) holds. We will show that (*) holds for height h + 1. Let
T  be the propagation tree of height h + 1 and let r = 0 be the root. Consider
children of r in T  and a comparator hcomp(a, b, c, d) for which r ∈ {c, d}:
Case 1: r has two children. The only case is when r ≡ c = 0. Unit propagation
sets a = b = 0. Nodes a and b are roots of propagation trees of height h and are
set to 0, therefore by the induction hypothesis all nodes in T  will be set to 0.
Case 2: r has one child. Consider two cases: (i) if r ≡ c = 0 and either a or b is
the child of r, then UP sets a = b = 0 and either a or b is the root of propagation
tree of height h and is set to 0, therefore by the induction hypothesis all nodes
in T  will be set to 0, (ii) r ≡ d = 0 and either a = c = 1 and b is the child of
r or b = c = 1 and a is the child of r. Both of them will be set to 0 by UP and
again we get the root of propagation tree of height h that is set to 0, therefore
by the induction hypothesis all nodes in T  will be set to 0.

7 Conclusions
We have constructed a new family of selection networks, which are based on the
pairwise selection ones, but require less comparators to merge subsequences. The
difference in sizes grows with k and is equal to n log 2n−4 + log n + 2 for k = n/2.
In addition, we have shown that any selection network encoded in a standard
way to a CNF formula preserves arc-consistency with respect to a corresponding
cardinality constraint. This property is important, as many SAT-solvers take
advantage of the unit-propagation algorithm, making the computation signifi-
cantly faster.
It’s also worth noting that using encodings based on selection networks give
an extra edge in solving optimization problems for which we need to solve a
sequence of problems that differ only in the decreasing bound of a cardinality
constraint. In this setting we only need to add one more clause ¬yk for a new
value of k, and the search can be resumed keeping all previous clauses as it is.
This works because if a comparator network is a k-selection network, then it is
also a k  -selection network for any k  < k. This property is called incremental
strengthening and most state-of-the-art SAT-solvers provide a user interface for
doing this.
We are expecting that our smaller encodings of cardinality constraints should
improve the performance of SAT solvers, but the statement needs an experimen-
tal evaluation. We start doing the evaluation and the results will be presented
in the near future.
 Smaller Selection Networks for Cardinality Constraints Encoding 225

References
1. Ası́n, R., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E.: Cardinality
networks and their applications. In: Kullmann, O. (ed.) SAT 2009. LNCS,
vol. 5584, pp. 167–180. Springer, Heidelberg (2009)
2. Ası́n, R., Nieuwenhuis, R., Oliveras, A., Rodrı́guez-Carbonell, E.: Cardinality
networks: a theoretical and empirical study. Constraints 16(2), 195–221 (2011)
3. Ası́n, R., Nieuwenhuis, R.: Curriculum-based course timetabling with SAT and
MaxSAT. Annals of Operations Research 218(1), 71–91 (2014)
4. Batcher, K.E.: Sorting networks and their applications. In: Proc. of the April
30-May 2, 1968, Spring Joint Computer Conference, AFIPS 1968 (Spring),
pp. 307–314. ACM, New York (1968)
5. Codish, M., Zazon-Ivry, M.: Pairwise networks are superior for selection. http://
www.cs.bgu.ac.il/∼mcodish/Papers/Sources/pairwiseSelection.pdf
6. Codish, M., Zazon-Ivry, M.: Pairwise cardinality networks. In: Clarke, E.M.,
Voronkov, A. (eds.) LPAR-16 2010. LNCS, vol. 6355, pp. 154–172. Springer,
Heidelberg (2010)
7. Eén, N., Sörensson, N.: Translating pseudo-boolean constraints into sat. Journal
on Satisfiability, Boolean Modeling and Computation 2, 1–26 (2006)
8. Knuth, D.E.: The Art of Computer Programming, Sorting and Searching, vol. 3,
2nd edn. Addison Wesley Longman Publishing Co. Inc., Redwood City (1998)
9. Parberry, I.: Parallel complexity theory. Pitman, Research notes in theoretical
computer science (1987)
10. Parberry, I.: The pairwise sorting network. Parallel Processing Letters 2, 205–211
(1992)
11. Schutt, A., Feydy, T., Stuckey, P.J., Wallace, M.G.: Why Cumulative decompo-
sition is not as bad as it sounds. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732,
pp. 746–761. Springer, Heidelberg (2009)
 PREFIX-PROJECTION Global Constraint
for Sequential Pattern Mining

Amina Kemmar1 , Samir Loudni2(B) , Yahia Lebbah1 ,

Patrice Boizumault2 , and Thierry Charnois3
1
LITIO, University of Oran 1, EPSECG of Oran, Oran, Algeria
[email protected], [email protected]
2
GREYC (CNRS UMR 6072), University of Caen, Caen, France
{samir.loudni,patrice.boizumault}@unicaen.fr
3
LIPN (CNRS UMR 7030), University PARIS 13, Villetaneuse, France
[email protected]

Abstract. Sequential pattern mining under constraints is a challenging

data mining task. Many efficient ad hoc methods have been developed
for mining sequential patterns, but they are all suffering from a lack
of genericity. Recent works have investigated Constraint Programming
(CP) methods, but they are not still effective because of their encoding.
In this paper, we propose a global constraint based on the projected
databases principle which remedies to this drawback. Experiments show
that our approach clearly outperforms CP approaches and competes well
with ad hoc methods on large datasets.

1 Introduction

Mining useful patterns in sequential data is a challenging task. Sequential pat-

tern mining is among the most important and popular data mining task with
many real applications such as the analysis of web click-streams, medical or
biological data and textual data. For effectiveness and efficiency considerations,
many authors have promoted the use of constraints to focus on the most promis-
ing patterns according to the interests given by the final user. In line with [15],
many efficient ad hoc methods have been developed but they suffer from a lack
of genericity to handle and to push simultaneously sophisticated combination of
various types of constraints. Indeed, new constraints have to be hand-coded and
their combinations often require new implementations.
Recently, several proposals have investigated relationships between sequen-
tial pattern mining and constraint programming (CP) to revisit data mining
tasks in a declarative and generic way [5,9,11,12]. The great advantage of these
approaches is their flexibility. The user can model a problem and express his
queries by specifying what constraints need to be satisfied. But, all these pro-
posals are not effective enough because of their CP encoding. Consequently, the
design of new efficient declarative models for mining useful patterns in sequential
data is clearly an important challenge for CP.


c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 226–243, 2015.
DOI: 10.1007/978-3-319-23219-5 17
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 227

To address this challenge, we investigate in this paper the other side

of the cross fertilization between data-mining and constraint programming,
namely how the CP framework can benefit from the power of candidate prun-
ing mechanisms used in sequential pattern mining. First, we introduce the
global constraint Prefix-Projection for sequential pattern mining. Prefix-
Projection uses a concise encoding and its filtering relies on the principle
of projected databases [14]. The key idea is to divide the initial database into
smaller ones projected on the frequent subsequences obtained so far, then, mine
locally frequent patterns in each projected database by growing a frequent pre-
fix. This global constraint utilizes the principle of prefix-projected database to
keep only locally frequent items alongside projected databases in order to remove
infrequent ones from the domains of variables. Second, we show how the concise
encoding allows for a straightforward implementation of the frequency constraint
(Prefix-Projection constraint) and constraints on patterns such as size, item
membership and regular expressions and the simultaneous combination of them.
Finally, experiments show that our approach clearly outperforms CP approaches
and competes well with ad hoc methods on large datasets for mining frequent
sequential patterns or patterns under various constraints. It is worth noting that
the experiments show that our approach achieves scalability while it is a major
issue of CP approaches.
The paper is organized as follows. Section 2 recalls preliminaries. Section 3
provides a critical review of ad hoc methods and CP approaches for sequential
pattern mining. Section 4 presents the global constraint Prefix-Projection.
Section 5 reports experiments we performed. Finally, we conclude and draw some
perspectives.

2 Preliminaries
This section presents background knowledge about sequential pattern mining
and constraint satisfaction problems.

2.1 Sequential Patterns

Let I be a finite set of items. The language of sequences corresponds to LI = I n
where n ∈ N+ .
Definition 1 (sequence, sequence database). A sequence s over LI is an
ordered list s1 s2 . . . sn , where si , 1 ≤ i ≤ n, is an item. n is called the length
of the sequence s. A sequence database SDB is a set of tuples (sid, s), where sid
is a sequence identifier and s a sequence.
Definition 2 (subsequence,  relation). A sequence α = α1 . . . αm  is a
subsequence of s = s1 . . . sn , denoted by (α  s), if m ≤ n and there exist
integers 1 ≤ j1 ≤ . . . ≤ jm ≤ n, such that αi = sji for all 1 ≤ i ≤ m. We
also say that α is contained in s or s is a super-sequence of α. For example,
the sequence BABC is a super-sequence of AC : AC  BABC. A tuple
(sid, s) contains a sequence α, if α  s.
228 A. Kemmar et al.

Table 1. SDB1 : a sequence database example.

sid Sequence
1 ABCBC
2 BABC
3 AB
4 BCD

The cover of a sequence p in SDB is the set of all tuples in SDB in which
p is contained. The support of a sequence p in SDB is the number of tuples in
SDB which contain p.
Definition 3 (coverage, support). Let SDB be a sequence database and
p a sequence. coverSDB (p)={(sid, s) ∈ SDB | p  s} and supSDB (p) = #
coverSDB (p).
Definition 4 (sequential pattern). Given a minimum support threshold
minsup, every sequence p such that supSDB (p) ≥ minsup is called a sequen-
tial pattern [1]. p is said to be frequent in SDB.

Example 1. Table 1 represents a sequence database of four sequences where

the set of items is I = {A, B, C, D}. Let the sequence p = AC. We have
coverSDB1 (p) = {(1, s1 ), (2, s2 )}. If we consider minsup = 2, p = AC is a
sequential pattern because supSDB1 (p) ≥ 2.
Definition 5 (sequential pattern mining (SPM)). Given a sequence
database SDB and a minimum support threshold minsup. The problem of
sequential pattern mining is to find all patterns p such that supSDB (p) ≥
minsup.

2.2 SPM under Constraints

In this section, we define the problem of mining sequential patterns in a sequence
database satisfying user-defined constraints Then, we review the most usual
constraints for the sequential mining problem [15].
Problem statement. Given a constraint C(p) on pattern p and a sequence
database SDB, the problem of constraint-based pattern mining is to find the
complete set of patterns satisfying C(p). In the following, we present different
types of constraints that we explicit in the context of sequence mining. All these
constraints will be handled by our concise encoding (see Sections 4.2 and 4.5).
- The minimum size constraint size(p, min ) states that the number of items of
p must be greater than or equal to min .
- The item constraint item(p, t) states that an item t must belong (or not) to a
pattern p.
- The regular expression constraint [7] reg(p, exp) states that a pattern p must be
accepted by the deterministic finite automata associated to the regular expres-
sion exp.
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 229

2.3 Projected Databases

We now present the necessary definitions related to the concept of projected

databases [14].

Definition 6 (prefix, projection, suffix). Let β = β1 . . . βn  and α =

α1 . . . αm  be two sequences, where m ≤ n.
- Sequence α is called the prefix of β iff ∀i ∈ [1..m], αi = βi .
- Sequence β = β1 . . . βn  is called the projection of some sequence s w.r.t. α,
iff (1) β  s, (2) α is a prefix of β and (3) there exists no proper super-sequence
β  of β such that β   s and β  also has α as prefix.
- Sequence γ = βm+1 . . . βn  is called the suffix of s w.r.t. α. With the standard
concatenation operator ”concat”, we have β = concat(α, γ).

Definition 7 (projected database). Let SDB be a sequence database, the α-

projected database, denoted by SDB|α , is the collection of suffixes of sequences
in SDB w.r.t. prefix α.

[14] have proposed an efficient algorithm, called PrefixSpan, for mining

sequential patterns based on the concept of projected databases. It proceeds by
dividing the initial database into smaller ones projected on the frequent subse-
quences obtained so far; only their corresponding suffixes are kept. Then, sequen-
tial patterns are mined in each projected database by exploring only locally
frequent patterns.

Example 2. Let us consider the sequence database of Table 1 with minsup = 2.

PrefixSpan starts by scanning SDB 1 to find all the frequent items, each of them
is used as a prefix to get projected databases. For SDB 1 , we get 3 disjoint subsets
w.r.t. the prefixes A, B, and C. For instance, SDB 1 |A consists of 3 suffix
sequences: {(1, BCBC), (2, BC), (3, B)}. Consider the projected database
SDB 1 |<A> , its locally frequent items are B and C. Thus, SDB 1 |<A> can be
recursively partitioned into 2 subsets w.r.t. the two prefixes AB and AC.
The AB- and AC- projected databases can be constructed and recursively
mined similarly. The processing of a α-projected database terminates when no
frequent subsequence can be generated.

Proposition 1 establishes the support count of a sequence γ in SDB|α [14]:

Proposition 1 (Support count). For any sequence γ in SDB with prefix α
and suffix β s.t. γ = concat(α, β), supSDB (γ) = supSDB|α (β).
This proposition ensures that only the sequences in SDB grown from α need
to be considered for the support count of a sequence γ. Furthermore, only those
suffixes with prefix α should be counted.
230 A. Kemmar et al.

2.4 CSP and Global Constraints

A Constraint Satisfaction Problem (CSP) consists of a set X of n variables, a

domain D mapping each variable Xi ∈ X to a finite set of values D(Xi ), and a
set of constraints C. An assignment σ is a mapping from variables in X to values
in their domains: ∀Xi ∈ X, σ(Xi ) ∈ D(Xi ). A constraint c ∈ C is a subset of the
cartesian product of the domains of the variables that are in c. The goal is to
find an assignment such that all constraints are satisfied.
Domain Consistency (DC). Constraint solvers typically use backtracking
search to explore the space of partial assignments. At each assignment, filtering
algorithms prune the search space by enforcing local consistency properties like
domain consistency. A constraint c on X is domain consistent, if and only if, for
every Xi ∈ X and for every di ∈ D(Xi ), there is an assignment σ satisfying c
such that σ(Xi ) = di . Such an assignment is called a support.
Global constraints provide shorthands to often-used combinatorial substruc-
tures. We present two global constraints. Let X = X1 , X2 , ..., Xn  be a sequence
of n variables.
Let V be a set of values, l and u be two integers s.t. 0 ≤ l ≤ u ≤ n, the constraint
Among(X, V, l, u) states that each value a ∈ V should occur at least l times and at
most u times in X [4]. Given a deterministic finite automaton A, the constraint
Regular(X, A) ensures that the sequence X is accepted by A [16].

3 Related Works

This section provides a critical review of ad hoc methods and CP approaches for
SPM.

3.1 Ad hoc Methods for SPM

GSP [17] was the first algorithm proposed to extract sequential patterns. It uses
a generate-and test approach. Later, two major classes of methods have been
proposed:
- Depth-first search based on a vertical database format e.g. cSpade incorporat-
ing contraints (max-gap, max-span, length) [21], SPADE [22] or SPAM [2].
- Projected pattern growth such as PrefixSpan [14] and its extensions, e.g.
CloSpan for mining closed sequential patterns [19] or Gap-BIDE [10] tackling the
gap constraint.
In [7], the authors proposed SPIRIT based on GSP for SPM with regular
expressions. Later, [18] introduces Sequence Mining Automata (SMA), a new
approach based on a specialized kind of Petri Net. Two variants of SMA were
proposed: SMA-1P (SMA one pass) and SMA-FC (SMA Full Check). SMA-1P pro-
cesses by means of the SMA all sequences one by one, and enters all resulting
valid patterns in a hash table for support counting, while SMA-FC allows fre-
quency based pruning during the scan of the database. Finally, [15] provides a
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 231

survey for other constraints such as regular expressions, length and aggregates.
But, all these proposals, though efficient, are ad hoc methods suffering from a
lack of genericity. Adding new constraints often requires to develop new imple-
mentations.

3.2 CP Methods for SPM

Following the work of [8] for itemset mining, several methods have been proposed
to mine sequential patterns using CP.
Proposals. [5] have proposed a first SAT-based model for discovering a special
class of patterns with wildcards1 in a single sequence under different types of
constraints (e.g. frequency, maximality, closedness). [11] have proposed a CSP
model for SPM. Each sequence is encoded by an automaton capturing all sub-
sequences that can occur in it. [9] have proposed a CSP model for SPM with
wildcards. They show how some constraints dealing with local patterns (e.g.
frequency, size, gap, regular expressions) and constraints defining more complex
patterns such as relevant subgroups [13] and top-k patterns can be modeled using
a CSP. [12] have proposed two CP encodings for the SPM. The first one uses
a global constraint to encode the subsequence relation (denoted global-p.f),
while the second one encodes explicitly this relation using additional variables
and constraints (denoted decomposed-p.f).
All these proposals use reified constraints to encode the database. A reified
constraint associates a boolean variable to a constraint reflecting whether the
constraint is satisfied (value 1) or not (value 0). For each sequence s of SDB,
a reified constraint, stating whether (or not) the unknown pattern p is a subse-
quence of s, is imposed: (Ss = 1) ⇔ (p  s). A great consequenceis that the
encoding of the frequency measure is straightforward: f req(p) = s∈SDB Ss .
But such an encoding has a major drawback since it requires (m = #SDB)
reified constraints to encode the whole database. This constitutes a strong limi-
tation of the size of the databases that could be managed.
Most of these proposals encode the subsequence relation (p  s) using
variables P oss,j (s ∈ SDB and 1 ≤ j ≤ ) to determine a position where p
occurs in s. Such an encoding requires a large number of additional variables
(m×) and makes the labeling computationally expensive. In order to address
this drawback, [12] have proposed a global constraint exists-embedding to
encode the subsequence relation, and used projected frequency within an ad hoc
specific branching strategy to keep only frequent items before branching over
the variables of the pattern. But, this encoding still relies on reified constraints
and requires to impose m exists-embedding global constraints.
So, we propose in the next section the Prefix-Projection global con-
straint that fully exploits the principle of projected databases to encode both
the subsequence relation and the frequency constraint. Prefix-Projection
does not require any reified constraints nor any extra variables to encode the

1
A wildcard is a special symbol that matches any item of I including itself.
232 A. Kemmar et al.

subsequence relation. As a consequence, usual SPM constraints (see Section 2.2)

can be encoded in a straightforward way using directly the (global) constraints
of the CP solver.

4 PREFIX-PROJECTION Global Constraint

This section presents the Prefix-Projection global constraint for the SPM
problem.

4.1 A Concise Encoding

Let P be the unknown pattern of size  we are looking for. The symbol 2 stands
for an empty item and denotes the end of a sequence. The unknown pattern P is
encoded with a sequence of  variables P1 , P2 , . . . , P  s.t. ∀i ∈ [1 . . . ], D(Pi ) =
I ∪ {2}. There are two basic rules on the domains:
1. To avoid the empty sequence, the first item of P must be non empty, so
(2 ∈ D1 ).
2. To allow patterns with less than  items, we impose that ∀i ∈
[1..(−1)], (Pi = 2) → (Pi+1 = 2).

4.2 Definition and Consistency Checking

The global constraint Prefix-Projection ensures both subsequence relation
and minimum frequency constraint.
Definition 8 (Prefix-Projection global constraint). Let P = P1 , P2 , . . . , P 
be a pattern of size . d1 , ..., d  ∈ D(P1 ) × . . . × D(P ) is a solution of Prefix-
Projection (P, SDB, minsup) iff supSDB (d1 , ..., d ) ≥ minsup.
Proposition 2. A Prefix-Projection (P, SDB, minsup) constraint has a
solution if and only if there exists an assignment σ = d1 , ..., d  of variables
of P s.t. SDB|σ has at least minsup suffixes of σ: #SDB|σ ≥ minsup.
Proof: This is a direct consequence of proposition 1. We have straightforwardly
supSDB (σ) = supSDB|σ () = #SDB|σ . Thus, suffixes of SDB|σ are supports
of σ in the constraint Prefix-Projection (P, SDB, minsup), provided that
#SDB|σ ≥ minsup. 2
The following proposition characterizes values in the domain of unassigned
(i.e. future) variable Pi+1 that are consistent with the current assignment of
variables P1 , ..., Pi .
Proposition 3. Let σ 2 = d1 , . . . , di  be a current assignment of variables
P1 , . . . , Pi , Pi+1 be a future variable. A value d ∈ D(Pi+1 ) appears in a solu-
tion for Prefix-Projection (P, SDB, minsup) if and only if d is a frequent
item in SDB|σ :
#{(sid, γ)|(sid, γ) ∈ SDB|σ ∧ dγ} ≥ minsup
2
We indifferently denote σ by d1 , . . . , di  or by σ(P1 ), . . . , σ(Pi ).
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 233

Proof: Suppose that value d ∈ D(Pi+1 ) occurs in SDB|σ more than minsup.
From proposition 1, we have supSDB (concat(σ, d)) = supSDB|σ (d). Hence,
the assignment σ ∪ d satisfies the constraint, so d ∈ D(Pi+1 ) participates in a
solution. 2
Anti-monotonicity of the Frequency Measure. If a pattern p is not fre-
quent, then any pattern p satisfying p  p is not frequent. From proposition 3
and according to the anti-monotonicity property, we can derive the following
pruning rule:

Proposition 4. Let σ = d1 , . . . , di  be a current assignment of variables

P1 , . . . , Pi . All values d ∈ D(Pi+1 ) that are locally not frequent in SDB|σ
can be pruned from the domain of variable Pi+1 . Moreover, these values d can
also be pruned from the domains of variables Pj with j ∈ [i + 2, . . . , ].

Proof: Let σ = d1 , . . . , di  be a current assignment of variables P1 , . . . , Pi . Let

d ∈ D(Pi+1 ) s.t. σ  = concat(σ, d). Suppose that d is not frequent in SDB|σ .
According to proposition 1, supSDB|σ (d) = supSDB (σ  ) < minsup, thus σ  is
not frequent. So, d can be pruned from the domain of Pi+1 .
Suppose that the assignment σ has been extended to concat(σ, α), where α
corresponds to the assignment of variables Pj (with j > i). If d ∈ D(Pi+1 ) is not
frequent, it is straightforward that supSDB|σ (concat(α, d)) ≤ supSDB|σ (d) <
minsup. Thus, if d is not frequent in SDB|σ , it will be also not frequent in
SDB|concat(σ,α) . So, d can be pruned from the domains of Pj with j ∈ [i +
2, . . . , ]. 2

Example 3. Consider the sequence database of Table 1 with minsup = 2. Let

P = P1 , P2 , P3  with D(P1 ) = I and D(P2 ) = D(P3 ) = I ∪ {2}. Suppose that
σ(P1 ) = A, Prefix-Projection(P, SDB, minsup) will remove values A and
D from D(P2 ) and D(P3 ), since the only locally frequent items in SDB 1 |<A>
are B and C.

Proposition 4 guarantees that any value (i.e. item) d ∈ D(Pi+1 ) present

but not frequent in SDB|σ does not need to be considered when extending
σ, thus avoiding searching over it. Clearly, our global constraint encodes the
anti-monotonicity of the frequency measure in a simple and elegant way, while
CP methods for SPM have difficulties to handle this property. In [12], this is
achieved by using very specific propagators and branching strategies, making
the integration quite complex (see [12]).

4.3 Building the Projected Databases

The key issue of our approach lies in the construction of the projected databases.
When projecting a prefix, instead of storing the whole suffix as a projected sub-
sequence, one can represent each suffix by a pair (sid, start) where sid is the
sequence identifier and start is the starting position of the projected suffix in
the sequence sid. For instance, let us consider the sequence database of Table 1.
234 A. Kemmar et al.

Algorithm 1. ProjectSDB(SDB, P rojSDB, α)

Data: SDB: initial database; P rojSDB: projected sequences; α: prefix
begin
1 SDB|α ← ∅ ;
2 for each pair (sid, start) ∈ P rojSDB do
3 s ← SDB[sid] ;
4 posα ← 1; poss ← start ;
5 while (posα ≤ #α ∧ poss ≤ #s) do
6 if (α[posα ] = s[poss ]) then
7 posα ← posα + 1 ;
8 poss ← poss + 1 ;
9 if (posα = #α + 1) then
10 SDB|α ← SDB|α ∪ {(sid, poss )}

11 return SDB|α ;

As shown in example 2, SDB|A consists of 3 suffix sequences: {(1, BCBC),

(2, BC), (3, B)}. By using the pseudo-projection, SDB|A can be repre-
sented by the following three pairs: {(1, 2), (2, 3), (3, 2)}. This is the principle of
pseudo-projection, adopted in , exploited during the filtering step of our Prefix-
Projection global constraint. Algorithm 1 details this principle. It takes as
input a set of projected sequences P rojSDB and a prefix α. The algorithm pro-
cesses all the pairs (sid, start) of P rojSDB one by one (line 2), and searches
for the lowest location of α in the sequence s corresponding to the sid of that
sequence in SDB (lines 6-8).
In the worst case, ProjectSDB processes all the items of all sequences. So,
the time complexity is O( × m), with m = #SDB and  is the length of the
longest sequence in SDB. The worst case space complexity of pseudo-projection
is O(m), since we need to store for each sequence only a pair (sid, start), while
for the standard projection the space complexity is O(m×). Clearly, the pseudo-
projection takes much less space than the standard projection.

4.4 Filtering

Ensuring DC on Prefix-Projection(P, SDB, minsup) is equivalent to finding

a sequential pattern of length ( − 1) and then checking whether this pattern
remains a frequent pattern when extended to any item d in D(P ). Thus, finding
such an assignment (i.e. support) is as much as difficult than the original prob-
lem of sequential pattern mining. [20] has proved that the problem of counting
the number of maximal3 frequent patterns in a database of sequences is #P-
complete, thereby proving the NP-hardness of the problem of mining maximal
frequent sequences. The difficulty is due to the exponential number of candi-
dates that should be parsed to find the frequent patterns. Thus, finding, for
every variable Pi ∈ P and for every di ∈ D(Pi ), an assignment σ satisfying
Prefix-Projection(P, SDB, minsup) s.t. σ(Pi ) = di is of exponential nature.

3
A sequential pattern p is maximal if there is no sequential pattern q such that pq.
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 235

Algorithm 2. Filter-Prefix-Projection(SDB, σ, i, P , minsup)

Data: SDB: initial database; σ: current prefix
σ(P1 ), . . . , σ(Pi ); minsup: the minimum
support threshold; PSDB: internal data structure of Prefix-Projection for storing
pseudo-projected databases
begin
1 if (i ≥ 2 ∧ σ(Pi ) = 2) then
2 for j ← i + 1 to  do
3 Pj ← 2;
4 return True;
else
5 PSDBi ← ProjectSDB(SDB, PSDBi−1 ,
σ(Pi ));
6 if (#PSDBi < minsup) then
7 return False ;
else
8 F I ← getFreqItems(SDB, PSDBi , minsup) ;
9 for j ← i + 1 to  do
10 foreach a ∈ D(Pj ) s.t.(a = 2 ∧ a ∈
/ F I) do
11 D(Pj ) ← D(Pj ) − {a};

12 return True;

Function getFreqItems (SDB, P rojSDB, minsup) ;

Data: SDB: the initial database; P rojSDB: pseudo-projected database; minsup: the
minimum support threshold; ExistsItem, SupCount: internal data structures using
a hash table for support counting over items;
begin
13 SupCount[] ← {0, ..., 0}; F ← ∅ ;
14 for each pair (sid, start) ∈ P rojSDB do
15 ExistsItem[] ← {f alse, ..., f alse}; s ← SDB[sid] ;
16 for i ← start to #s do
17 a ← s[i] ;
18 if (¬ExistsItem[a]) then
19 SupCount[a] ← SupCount[a] + 1; ExistsItem[a] ← true;
20 if (SupCount[a] ≥ minsup) then
21 F ← F ∪ {a};

22 return F ;

So, the filtering of the Prefix-Projection constraint maintains a consis-

tency lower than DC. This consistency is based on specific properties of the
projected databases (see Proposition 3), and anti-monotonicity of the frequency
constraint (see Proposition 4), and resembles forward-checking regarding Propo-
sition 3. Prefix-Projection is considered as a global constraint, since all vari-
ables share the same internal data structures that awake and drive the filtering.
Algorithm 2 describes the pseudo-code of the filtering algorithm of the
Prefix-Projection constraint. It is an incremental filtering algorithm that
should be run when some i first variables are assigned according to the following
lexicographic ordering P1 , P2 , . . . , P  of variables of P . It exploits internal data-
structures enabling to enhance the filtering algorithm. More precisely, it uses an
incremental data structure, denoted PSDB, that stores the intermediate pseudo-
projections of SDB, where PSDBi (i ∈ [0, . . . , ]) corresponds to the σ-projected
database of the current partial assignment σ = σ(P1 ), . . . , σ(Pi ) (also called
prefix) of variables P1 , . . . , Pi , and PSDB0 = {(sid, 1)|(sid, s) ∈ SDB} is the
236 A. Kemmar et al.

initial pseudo-projected database of SDB (case where σ = ). It also uses a

hash table indexing the items I into integers (1 . . . #I) for an efficient support
counting over items (see function getFreqItems).
Algorithm 2 takes as input the current partial assignment σ = σ(P1 ), . . . , σ(Pi )
of variables P1 , . . . , Pi , the length i of σ (i.e. position of the last assigned vari-
able in P ) and the minimum support threshold minsup. It starts by checking if
the last assigned variable Pi is instantiated to 2 (line 1). In this case, the end of
sequence is reached (since value 2 can only appear at the end) and the sequence
σ(P1 ), . . . , σ(Pi ) constitutes a frequent pattern in SDB; hence the algorithm sets
the remaining ( − i) unassigned variables to 2 and returns true (lines 2-4). Oth-
erwise, the algorithm computes incrementally PSDBi from PSDBi−1 by calling
function ProjectSDB (see Algorithm 1). Then, it checks in line 6 whether the
current assignment σ is a legal prefix for the constraint (see Proposition 2). This
is done by computing the size of PSDBi . If this size is less than minsup, we stop
growing σ and we return false. Otherwise, the algorithm computes the set of locally
frequent items FI in PSDBi by calling function getFreqItems (line 8).
Function getFreqItems processes all the entries of the pseudo-projected
database one by one, counts the number of first occurrences of items a (i.e.
SupCount[a]) in each entry (sid, start), and keeps only the frequent ones (lines
13-21). This is done by using ExistsItem data structure. After the whole pseudo-
projected database has been processed, the frequent items are returned (line 22),
and Algorithm 2 updates the current domains of variables Pj with j ≥ (i + 1)
by pruning inconsistent values, thus avoiding searching over not frequent items
(lines 9-11).
Proposition 5. In the worst case, filtering with Prefix-Projection global
constraint can be achieved in O(m ×  + m × d +  × d). The worst case space
complexity of Prefix-Projection is O(m × ).
Proof: Let  be the length of the longest sequence in SDB, m = #SDB, and
d = #I. Computing the pseudo-projected database PSDBi can be done in
O(m × ): for each sequence (sid, s) of SDB, checking if σ occurs in s is O()
and there are m sequences. The total complexity of function GetFreqItems is
O(m×(+d)). Lines (9-11) can be achieved in O(×d). So, the whole complexity
is O(m×+m×(+d)+×d) = O(m×+m×d+×d). The space complexity of
the filtering algorithm lies in the storage of the PSDB internal data structure.
In the worst case, we have to store  pseudo-projected databases. Since each
pseudo-projected database requires O(m), the worst case space complexity is
O(m × ). 2

4.5 Encoding of SPM Constraints

Usual SPM constraints (see Section 2.2) can be reformulated in a straightforward
way. Let P be the unknown pattern. i=
- Minimum size constraint: size(P, min ) ≡ i=1min (Pi = )
- Item constraint:let V be a subset of items, l and u two integers s.t. 0 ≤ l ≤ u ≤
. item(P, V ) ≡ t∈V Among(P, {t}, l, u) enforces that items of V should occur
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 237

Table 2. Dataset Characteristics.

dataset #SDB #I avg (#s) maxs∈SDB (#s) type of data

Leviathen 5834 9025 33.81 100 book
Kosarak 69999 21144 7.97 796 web click stream
FIFA 20450 2990 34.74 100 web click stream
BIBLE 36369 13905 21.64 100 bible
Protein 103120 24 482 600 protein sequences
data-200K 200000 20 50 86 synthetic dataset
PubMed 17527 19931 29 198 bio-medical text

at least l times and at most u times in P . To forbid items of V to occur in P , l

and u must be set to 0.
- Regular expression constraint: let Areg be the deterministic finite automaton
encoding the regular expression exp. reg(P, exp) ≡ Regular(P, Areg ).

5 Experimental Evaluation
This section reports experiments on several real-life datasets from [3,6,18] of
large size having varied characteristics and representing different application
domains (see Table 2). Our objective is (1) to compare our approach to existing
CP methods as well as to state-of-the-art methods for SPM in terms of scalability
which is a major issue of existing CP methods, (2) to show the flexibility of our
approach allowing to handle different constraints simultaneously.
Experimental Protocol. The implementation of our approach was carried
out in the Gecode solver4 . All experiments were conducted on a machine with a
processor Intel X5670 and 24 GB of memory. A time limit of 1 hour has been
used. For each dataset, we varied the minsup threshold until the methods are
not able to complete the extraction of all patterns within the time limit.  was
set to the length of the longest sequence of SDB. The implementation and the
datasets used in our experiments are available online5 . We compare our approach
(indicated by PP) with:
1. two CP encodings [12], the most efficient CP methods for SPM: global-p.f
and decomposed-p.f;
2. state-of-the-art methods for SPM : and cSpade;
3. SMA [18] for SPM under regular expressions.
We used the author’s cSpade implementation 6 for SPM, the publicly avail-
able implementations of by Y. Tabei 7 and the SMA implementation 8 for SPM
under regular expressions. The implementation 9 of the two CP encodings was
4
http://www.gecode.org
5
https://sites.google.com/site/prefixprojection4cp/
6
http://www.cs.rpi.edu/∼zaki/www-new/pmwiki.php/Software/
7
https://code.google.com/p/prefixspan/
8
http://www-kdd.isti.cnr.it/SMA/
9
https://dtai.cs.kuleuven.be/CP4IM/cpsm/
238 A. Kemmar et al.

Fig. 1. Comparing PP with global-p.f for SPM on real-life datasets: CPU times.

carried out in the Gecode solver. All methods have been executed on the same
machine.
(a) Comparing with CP Methods for SPM. First we compare PP with
the two CP encodings global-p.f and decomposed-p.f (see Section 3.2). CPU
times (in logscale for BIBLE, Kosarak and PubMed) of the three methods are
shown on Fig. 1. First, decomposed-p.f is the least performer method. On all
the datasets, it fails to complete the extraction within the time limit for all
values of minsup we considered. Second, PP largely dominates global-p.f on
all the datasets: PP is more than an order of magnitude faster than global-p.f.
The gains in terms of CPU times are greatly amplified for low values of minsup.
On BIBLE (resp. PubMed), the speed-up is 84.4 (resp. 33.5) for minsup equal
to 1%. Another important observation that can be made is that, on most of
the datasets (except BIBLE and Kosarak), global-p.f is not able to mine for
patterns at very low frequency within the time limit. For example on FIFA, PP is
able to complete the extraction for values of minsup up to 6% in 1, 457 seconds,
while global-p.f fails to complete the extraction for minsup less than 10%.
To complement the results given by Fig. 1, Table 3 reports for different
datasets and different values of minsup, the number of calls to the propagate
routine of Gecode (column 5), and the number of nodes of the search tree (column
6). First, PP explores less nodes than global-p.f. But, the difference is not huge
(gains of 45% and 33% on FIFA and BIBLE respectively). Second, our approach
is very effective in terms of number of propagations. For PP, the number of
propagations remains small (in thousands for small values of minsup) compared
to global-p.f (in millions). This is due to the huge number of reified constraints
used in global-p.f to encode the subsequence relation. On the contrary, our
Prefix-Projection global constraint does not require any reified constraints
nor any extra variables to encode the subsequence relation.
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 239

Table 3. PP vs. global-p.f.

CPU times (s) #PROPAGATIONS #NODES
Dataset minsup (%) #PATTERNS
PP global-p.f PP global-p.f PP global-p.f
20 938 8.16 129.54 1884 11649290 1025 1873
18 1743 13.39 222.68 3502 19736442 1922 3486
16 3578 24.39 396.11 7181 35942314 3923 7151
FIFA
14 7313 44.08 704 14691 65522076 8042 14616
12 16323 86.46 1271.84 32820 126187396 18108 32604
10 40642 185.88 2761.47 81767 266635050 45452 81181
10 174 1.98 105.01 363 4189140 235 348
8 274 2.47 153.61 575 5637671 362 548
6 508 3.45 270.49 1065 8592858 669 1016
BIBLE
4 1185 5.7 552.62 2482 15379396 1575 2371
2 5311 15.05 1470.45 11104 39797508 7048 10605
1 23340 41.4 3494.27 49057 98676120 31283 46557
5 2312 8.26 253.16 4736 15521327 2833 4619
4 3625 11.17 340.24 7413 20643992 4428 7242
3 6336 16.51 536.96 12988 29940327 7757 12643
PubMed
2 13998 28.91 955.54 28680 50353208 17145 27910
1 53818 77.01 2581.51 110133 124197857 65587 107051
99.99 127 165.31 219.69 264 26731250 172 221
99.988 216 262.12 411.83 451 44575117 293 390
99.986 384 467.96 909.47 805 80859312 514 679
Protein
99.984 631 753.3 1443.92 1322 132238827 845 1119
99.982 964 1078.73 2615 2014 201616651 1284 1749
99.98 2143 2315.65 − 4485 − 2890 −

(b) Comparing with ad hoc Methods for SPM. Our second experiment
compares PP with state-of-the-art methods for SPM. Fig. 2 shows the CPU times
of the three methods. First, cSpade obtains the best performance on all datasets
(except on Protein). However, PP exhibits a similar behavior as cSpade, but it is
less faster (not counting the highest values of minsup). The behavior of cSpade
on Protein is due to the vertical representation format that is not appropriated
in the case of databases having large sequences and small number of distinct
items, thus degrading the performance of the mining process. Second, PP which
also uses the concept of projected databases, clearly outperforms on all datasets.
This is due to our filtering algorithm combined together with incremental data
structures to manage the projected databases. On FIFA, is not able to complete
the extraction for minsup less than 12%, while our approach remains feasible
until 6% within the time limit. On Protein, fails to complete the extraction for
all values of minsup we considered. These results clearly demonstrate that our
approach competes well with state-of-the-art methods for SPM on large datasets
and achieves scalability while it is a major issue of existing CP approaches.
(c) SPM under size and item constraints. Our third experiment aims at
assessing the interest of pushing simultaneously different types of constraints. We
impose on the PubMed dataset usual constraints such as the minimum frequency
and the minimum size constraints and other useful constraints expressing some
linguistic knowledge such as the item constraint. The goal is to retain sequential
patterns which convey linguistic regularities (e.g., gene - rare disease relation-
ships) [3]. The size constraint allows to remove patterns that are too small w.r.t.
the number of items (number of words) to be relevant patterns. We tested this
constraint with min set to 3. The item constraint imposes that the extracted
patterns must contain the item GENE and the item DISEASE. As no ad hoc
240 A. Kemmar et al.

Fig. 2. Comparing Prefix-Projection with state-of-the-art algorithms for SPM.

Table 4. PP vs. global-p.f under minimum size and item constraints.

CPU times (s) #PROPAGATIONS #NODES
Dataset minsup (%) #PATTERNS
PP global-p.f PP global-p.f PP global-p.f
5 279 6.76 252.36 7878 12234292 2285 4619
4 445 8.81 339.09 12091 16475953 3618 7242
3 799 12.35 535.32 20268 24380096 6271 12643
PubMed
2 1837 20.41 953.32 43088 42055022 13888 27910
1 7187 49.98 2574.42 157899 107978568 52508 107051

method exists for this combination of constraints, we only compare PP with

global-p.f. Fig. 3 shows the CPU times and the number of sequential patterns
extracted with and without constraints. First, pushing simultaneously the two
constraints enables to reduce significantly the number of patterns. Moreover, the
CPU times for PP decrease slightly whereas for global-p.f (with and without
constraints), they are almost the same. This is probably due to the weak com-
munication between the m exists-embedding reified global constraints and the
two constraints. This reduces significantly the quality of the whole filtering. Sec-
ond (see Table 4), when considering the two constraints, PP clearly dominates
global-p.f (speed-up value up to 51.5). Moreover, the number of propagations
performed by PP remains very small as compared to global-p.f. Fig. 3c com-
pares the two methods under the minimum size constraint for different values
of min , with minsup fixed to 1%. Once again, PP is always the most performer
method (speed-up value up to 53.1). These results also confirm what we observed
previously, namely the weak communication between reified global constraints
and constraints imposed on patterns (i.e., size and item constraints).
(d) SPM under regular constraints. Our last experiment compares PP-REG
against two variants of SMA: SMA-1P (SMA one pass) and SMA-FC (SMA Full
Check). Two datasets are considered from [18]: one synthetic dataset (data-
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 241

Fig. 3. Comparing PP with global-p.f under minimum size and item constraints on
PubMed.

Fig. 4. Comparing Prefix-Projection with SMA for SPM under RE constraint.

200k), and one real-life dataset (Protein). For data-200k, we used two RE: RE10 ≡
A∗ B(B|C)D∗ EF ∗ (G|H)I ∗ and RE14 ≡ A∗ (Q|BS ∗ (B|C))D∗ E(I|S)∗ (F |H)G∗ R.
For Protein, we used RE2 ≡ (S|T ) . (R|K) (where . represents any symbol).
Fig. 4 reports CPU-times comparison. On the synthetic dataset, our approach is
very effective. For RE14, our method is more than an order of magnitude faster
than SMA. On Protein, the gap between the 3 methods shrinks, but our method
remains effective. For the particular case of RE2, the Regular constraint can be
substituted by restricting the domain of the first and third variables to {S, T }
and {R, K} respectively (denoted as PP-SRE), thus improving performances.

6 Conclusion

We have proposed the global constraint Prefix-Projection for sequential

pattern mining. Prefix-Projection uses a concise encoding and provides an
efficient filtering based on specific properties of the projected databases, and anti-
monotonicity of the frequency constraint. When this global constraint is integrated
into a CP solver, it enables to handle several constraints simultaneously. Some
of them like size, item membership and regular expression are considered in this
paper. Another point of strength, is that, contrary to existing CP approaches
for SPM, our global constraint does not require any reified constraints nor any
extra variables to encode the subsequence relation. Finally, although Prefix-
Projection is well suited for constraints on sequences, it would require to be
adapted to handle constraints on subsequence relations like gap.
242 A. Kemmar et al.

Experiments performed on several real-life datasets show that our approach

clearly outperforms existing CP approaches and competes well with ad hoc meth-
ods on large datasets and achieves scalability while it is a major issue of CP
approaches. As future work, we intend to handle constraints on set of sequential
patterns such as closedness, relevant subgroup and skypattern constraints.

Acknowledgments. The authors would like to thank the anonymous referees for
their valuable comments. This work is partly supported by the ANR (French Research
National Agency) funded projects Hybride ANR-11-BS002-002.

References
1. Agrawal, R., Srikant, R.: Mining sequential patterns. In: Yu, P.S., Chen, A.L.P.
(eds.) ICDE, pp. 3–14. IEEE Computer Society (1995)
2. Ayres, J., Flannick, J., Gehrke, J., Yiu, T.: Sequential pattern mining using a
bitmap representation. In: KDD 2002, pp. 429–435. ACM (2002)
3. Béchet, N., Cellier, P., Charnois, T., Crémilleux, B.: Sequential pattern mining to
discover relations between genes and rare diseases. In: CBMS (2012)
4. Beldiceanu, N., Contejean, E.: Introducing global constraints in CHIP. Journal of
Mathematical and Computer Modelling 20(12), 97–123 (1994)
5. Coquery, E., Jabbour, S., Saı̈s, L., Salhi, Y.: A SAT-based approach for discovering
frequent, closed and maximal patterns in a sequence. In: ECAI, pp. 258–263 (2012)
6. Fournier-Viger, P., Gomariz, A., Gueniche, T., Soltani, A., Wu, C., Tseng, V.:
SPMF: A Java Open-Source Pattern Mining Library. J. of Machine Learning Resea.
15, 3389–3393 (2014)
7. Garofalakis, M.N., Rastogi, R., Shim, K.: Mining sequential patterns with regular
expression constraints. IEEE Trans. Knowl. Data Eng. 14(3), 530–552 (2002)
8. Guns, T., Nijssen, S., Raedt, L.D.: Itemset mining: A constraint programming
perspective. Artif. Intell. 175(12–13), 1951–1983 (2011)
9. Kemmar, A., Ugarte, W., Loudni, S., Charnois, T., Lebbah, Y., Boizumault, P.,
Crémilleux, B.: Mining relevant sequence patterns with cp-based framework. In:
ICTAI, pp. 552–559 (2014)
10. Li, C., Yang, Q., Wang, J., Li, M.: Efficient mining of gap-constrained subsequences
and its various applications. ACM Trans. Knowl. Discov. Data 6(1), 2:1–2:39 (2012)
11. Métivier, J.P., Loudni, S., Charnois, T.: A constraint programming approach for
mining sequential patterns in a sequence database. In: ECML/PKDD Workshop
on Languages for Data Mining and Machine Learning (2013)
12. Negrevergne, B., Guns, T.: Constraint-based sequence mining using constraint
programming. In: Michel, L. (ed.) CPAIOR 2015. LNCS, vol. 9075, pp. 288–305.
Springer, Heidelberg (2015)
13. Novak, P.K., Lavrac, N., Webb, G.I.: Supervised descriptive rule discovery: A uni-
fying survey of contrast set, emerging pattern and subgroup mining. Journal of
Machine Learning Research 10 (2009)
14. Pei, J., Han, J., Mortazavi-Asl, B., Pinto, H., Chen, Q., Dayal, U., Hsu, M.:
PrefixSpan: Mining sequential patterns by prefix-projected growth. In: ICDE,
pp. 215–224. IEEE Computer Society (2001)
15. Pei, J., Han, J., Wang, W.: Mining sequential patterns with constraints in large
databases. In: CIKM 202, pp. 18–25. ACM (2002)
 PREFIX-PROJECTION Global Constraint for Sequential Pattern Mining 243

16. Pesant, G.: A regular language membership constraint for finite sequences of vari-
ables. In: Wallace, M. (ed.) CP 2004. LNCS, vol. 3258, pp. 482–495. Springer,
Heidelberg (2004)
17. Srikant, R., Agrawal, R.: Mining sequential patterns: Generalizations and perfor-
mance improvements. In: EDBT, pp. 3–17 (1996)
18. Trasarti, R., Bonchi, F., Goethals, B.: Sequence mining automata: A new tech-
nique for mining frequent sequences under regular expressions. In: ICDM 2008,
pp. 1061–1066 (2008)
19. Yan, X., Han, J., Afshar, R.: CloSpan: mining closed sequential patterns in large
databases. In: Barbará, D., Kamath, C. (eds.) SDM. SIAM (2003)
20. Yang, G.: Computational aspects of mining maximal frequent patterns. Theor.
Comput. Sci. 362(1–3), 63–85 (2006)
21. Zaki, M.J.: Sequence mining in categorical domains: Incorporating constraints. In:
Proceedings of the 2000 ACM CIKM International Conference on Information and
Knowledge Management, McLean, VA, USA, November 6–11, pp. 422–429 (2000)
22. Zaki, M.J.: SPADE: An efficient algorithm for mining frequent sequences. Machine
Learning 42(1/2), 31–60 (2001)
 On Tree-Preserving Constraints

Shufeng Kong1 , Sanjiang Li1(B) , Yongming Li2 , and Zhiguo Long1

1
QCIS, FEIT, University of Technology Sydney, Sydney, Australia
{Shufeng.Kong,Zhiguo.Long}@student.uts.edu.au, [email protected]
2
College of Computer Science, Shaanxi Normal University, Xi’an, China
[email protected]

Abstract. Tree convex constraints are extensions of the well-known row

convex constraints. Just like the latter, every path-consistent tree convex
constraint network is globally consistent. This paper studies and com-
pares three subclasses of tree convex constraints which are called chain-,
path- and tree-preserving constraints respectively. While the tractabil-
ity of the subclass of chain-preserving constraints has been established
before, this paper shows that every chain- or path-preserving constraint
network is in essence the disjoint union of several independent connected
row convex constraint networks, and hence (re-)establish the tractability
of these two subclasses of tree convex constraints. We further prove that,
when enforcing arc- and path-consistency on a tree-preserving constraint
network, in each step, the network remains tree-preserving. This ensures
the global consistency of the tree-preserving network if no inconsistency
is detected. Moreover, it also guarantees the applicability of the par-
tial path-consistency algorithm to tree-preserving constraint networks,
which is usually more efficient than the path-consistency algorithm for
large sparse networks. As an application, we show that the class of tree-
preserving constraints is useful in solving the scene labelling problem.

1 Introduction
Constraint satisfaction problems (CSPs) have been widely used in many areas,
such as scene labeling [10], natural language parsing [15], picture processing [16],
and spatial and temporal reasoning [5,14]. Since deciding consistency of CSP
instances is NP-hard in general, lots of efforts have been devoted to identify
tractable subclasses. These subclasses are usually obtained by either restricting
the topology of the underlying graph of the constraint network (being a tree or
having treewidth bounded by a constant) or restricting the type of the allowed
constraints between variables (cf. [17]).
In this paper, we are mainly interested in the second type of restriction.
Montanari [16] shows that path-consistency is sufficient to guarantee that a
network is globally consistent if the relations are all monotone. Van Beek and
Dechter [17] generalise monotone constraints to a larger class of row convex
constraints, which are further generalised to tree convex constraints by Zhang
and Yap [20]. These constraints also have the nice property that every path-
consistent constraint network is globally consistent.

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 244–261, 2015.
DOI: 10.1007/978-3-319-23219-5 18
 On Tree-Preserving Constraints 245

However, neither row convex constraints nor tree convex constraints are
closed under composition and intersection, the main operations of path-consistent
algorithms. This means enforcing path-consistency may destroy row and tree-
convexity. Deville et al. [6] propose a tractable subclass of row convex constraints,
called connected row convex (CRC) constraints, which are closed under composi-
tion and intersection. Zhang and Freuder [18] also identify a tractable subclass for
tree convex constraints, called locally chain convex and strictly union closed con-
straints. They also propose the important notion of consecutive constraints. Kumar
[13] shows that the subclass of arc-consistent consecutive tree convex (ACCTC)
constraints is tractable by providing a polynomial time randomised algorithm. But,
for the ACCTC problems, “it is not known whether there are efficient deterministic
algorithms, neither is it known whether arc- and path-consistency ensures global
consistency on those problems.” [18]
In this paper, we study and compare three subclasses of tree convex con-
straints which are called, respectively, chain-, path- and tree-preserving con-
straints. Chain-preserving constraints are exactly “locally chain convex and
strictly union closed” constraints and ACCTC constraints are strictly contained
in the subclass of tree-preserving constraints. We first show that every chain-
or path-preserving constraint network is in essence the disjoint union of several
independent CRC constraint networks and then prove that enforcing arc- and
path-consistency on a tree-preserving constraint network ensures global consis-
tency. This provides an affirmative answer to the above open problem raised
in [18]. Note also that our result is more general than that of Kumar [13] as
we do not require the constraint network to be arc-consistent. Moreover, when
enforcing arc- and path-consistent on a tree-preserving constraint network, in
each step, the network remains tree-preserving. This guarantees the applicabil-
ity of the partial path-consistency algorithm [2] to tree-preserving constraint
networks, which is usually more efficient than the path-consistency algorithm
for large sparse networks. We further show that a large subclass of the trihedral
scene labelling problem [10,12] can be modelled by tree-preserving constraints.
In the next section, we introduce basic notations and concepts that will be
used throughout the paper. Chain-, path-, and tree-preserving constraints are
discussed in Sections 3, 4, and 5, respectively. Application of tree-preserving
constraints in the scene labelling problem is shown in Section 6. Section 7 briefly
discusses the connection with majority operators [11] and concludes the paper.

2 Preliminaries

Let D be a domain of a variable x. A graph structure can often be associated

to D such that there is a bijection between the vertices in the graph and the
values in D. If the graph is connected and acyclic, i.e. a tree, then we say it is
a tree domain of x. Tree domains arise naturally in e.g. scene labeling [18] and
combinatorial auctions [4]. We note that, in this paper, we have a specific tree
domain Dx for each variable x.
246 S. Kong et al.

In this paper, we distinguish between tree and rooted tree. Standard notions
from graph theory are assumed. In particular, the degree of a node a in a graph
G, denoted by deg(a), is the number of neighbors of a in G.

Definition 1. A tree is a connected graph without any cycle. A tree is rooted if

it has a specified node r, called the root of the tree. Given a tree T , a subgraph I
is called a subtree of T if I is connected. An empty set is a subtree of any tree.
Let T be a (rooted) tree and I a subtree of T . I is a path ( chain, resp.) in T
if each node in I has at most two neighbors (at most one child, resp.) in I. Given
two nodes p, q in T , the unique path that connects p to q is denoted by πp,q .
Suppose a is a node of a tree T . A branch of a is a connected component of
T \ {a}.

Throughout this paper, we always associate a subtree with its node set.
Definition 2. A binary constraint has the form (xδy), where x, y are two vari-
ables with domains Dx and Dy and δ is a binary relation from Dx to Dy , or
δ ⊆ Dx × Dy . For simplicity, we often denote by δ this constraint. A value
u ∈ Dx is supported if there exists a value v in Dy s.t. (u, v) ∈ δ. In this case,
we say v is a support of u. We say a subset F of Dx is unsupported if every
value in F is not supported. Given A ⊆ Dx , the image of A under δ is defined
as δ(A) = {b ∈ Dy : (∃a ∈ A)(a, b) ∈ δ}. For A = {a} that contains only one
value, without confusion, we also use δ(a) to represent δ({a}).
A binary constraint network consists of a set of variables V = {x1 , x2 , ..., xn }
with a finite domain Di for each variable xi ∈ V , and a set Δ of binary constraints
over the variables of V . The usual operations on relations, e.g., intersection (∩),
composition (◦), and inverse (−1 ), are applicable to constraints. As usual, we
assume that there is at most one constraint for any ordered pair of variables
(x, y). Write δxy for this constraint if it exists. In this paper, we always assume
δxy is the inverse of δyx , and if there is no constraint for (x, y), we assume δxy is
the universal constraint.

Definition 3. [8, 9] A constraint network Δ over n variables is k-consistent iff

any consistent instantiation of any distinct k − 1 variables can be consistently
extended to any k-th variable. We say Δ is strongly k-consistent iff it is j-
consistent for all j ≤ k; and say Δ is globally consistent if it is strongly n-
consistent. 2- and 3-consistency are usually called arc- and path-consistency
respectively.

Definition 4. Let x, y be two variables with finite tree domains Tx = (Dx , Ex )

and Ty = (Dy , Ey ) and δ a constraint from x to y. We say δ, w.r.t. Tx and Ty , is

- tree convex if the image of every value a in Dx (i.e. δ(a)) is a (possibly empty)
subtree of Ty ;
- consecutive if the image of every edge in Tx is a subtree in Ty ;
- path-preserving if the image of every path in Tx is a path in Ty .
- tree-preserving if the image of every subtree in Tx is a subtree in Ty .
 On Tree-Preserving Constraints 247

In case Tx and Ty are rooted, we say δ, w.r.t. Tx and Ty , is

- chain-preserving if the image of every chain in Tx is a chain in Ty .
Chain-preserving constraints are exactly those “locally chain convex and strictly
union closed” constraints defined in [18].
CRC constraints are special tree convex constraints defined over chain
domains. The following definition of CRC constraints is equivalent to the one
given in [6].
Definition 5. Let x, y be two variables with finite tree domains Tx and Ty ,
where Tx and Ty are chains. A constraint δ from x to y is connected row convex
(CRC), w.r.t. Tx and Ty , if both δ and δ −1 are chain-preserving.
The class of CRC constraints is tractable and closed under intersection, inverse,
and composition [6].
Definition 6. A binary constraint network Δ over variables in V and tree
domains Tx (x ∈ V ) is called tree convex, chain-, path-, or tree-preserving if
every constraint δ ∈ Δ is tree convex, chain-, path-, or tree-preserving, respec-
tively. A CRC constraint network is defined similarly.
Proposition 1. Every chain-, path-, or tree-preserving constraint (network) is
consecutive and every path-preserving constraint (network) is tree-preserving.
Moreover, every arc-consistent consecutive tree convex (ACCTC) constraint
(network) is tree-preserving.
Not every consecutive tree convex constraint (or chain-preserving constraint)
is tree-preserving, but such a constraint becomes tree-preserving if it is arc-
consistent.
Lemma
n 1. [20] Let T be a tree and suppose ti (i = 1, .., n) are subtrees of T .
Then i=1 ti is nonempty iff ti ∩ tj is nonempty for every 1 ≤ i = j ≤ n.
Lemma 2. Let T be a tree and t, t subtrees of T . Suppose {u, v} is an edge in
T . If u ∈ t and v ∈ t , then t ∪ t is a subtree of T ; if, in addition, u ∈ t and
v ∈ t, then t ∩ t = ∅.
Using Lemma 1, Zhang and Yap [20] proved
Theorem 1. A tree-convex constraint network is globally consistent if it is path-
consistent.

3 Chain-Preserving Constraints
Zhang and Freuder [18] have proved that any consistent chain-preserving network
can be transformed to an equivalent globally consistent network by enforcing arc-
and path-consistency. This implies that the class of chain-preserving constraints
is tractable. We next show that every chain-preserving constraint network Δ can
be uniquely divided into a small set of k CRC constraint networks Δ1 , ..., Δk s.t.
Δ is consistent iff at least one of Δi is consistent.
We first recall the following result used in the proof of [18, Theorem 1].
248 S. Kong et al.

Proposition 2. Let Δ be a chain-preserving constraint network over tree

domains Tx (x ∈ V ). If no inconsistency is detected, then Δ remains chain-
preserving after enforcing arc-consistency.
We note that these tree domains over variables in V may need adjustment
in the process of enforcing arc-consistency. Here by adjustment we mean adding
edges to the tree structure so that it remains connected when unsupported values
are deleted.
Definition 7. Let T be a tree with root. A chain [a, a∗ ] in T is called an irre-
ducible perfect chain (ip-chain) if (i) a is the root or has one or more siblings;
(ii) a∗ is a leaf node or has two or more children; and (iii) every node in [a, a∗ )
has only one child.
Note that it is possible that a = a∗ . In fact, this happens when a is the root or
has one or more siblings and has two or more children. An ip-chain as defined
above is a minimum chain which satisfies (1) in the following lemma.
Lemma 3. Suppose δxy and δyx are arc-consistent and chain-preserving w.r.t.
rooted trees Tx and Ty . Assume [a, a∗ ] ⊆ Tx is an ip-chain. Then

δyx (δxy ([a, a∗ ])) = [a, a∗ ] (1)

and δxy ([a, a∗ ]) is also an ip-chain in Ty .

Proof. W.l.o.g., we suppose â is the parent of a, a is a sibling of a, and
a1 , a2 , ..., ak (k ≥ 2) are the children of a∗ .
Because δxy and δyx are arc-consistent and chain-preserving, δxy ([a, a∗ ]) is a
non-empty chain in Ty , written [b, b∗ ], and so is δyx ([b, b∗ ]). Suppose δyx ([b, b∗ ])
is not [a, a∗ ]. This implies that either â or one of a1 , a2 , ..., ak is in δyx ([b, b∗ ]).
Suppose â ∈ δyx ([b, b∗ ]). Then there exists b̂ ∈ [b, b∗ ] such that (â, b̂) ∈ δxy .
By b̂ ∈ [b, b∗ ] = δxy ([a, a∗ ]), we have a+ ∈ [a, a∗ ] s.t. (a+ , b̂) ∈ δxy . Therefore,
[â, a+ ] is contained in δyx (δxy ({â})). Recall that a is a sibling of a. Because
δyx (δxy ([â, a ])) contains â, a , a+ , it cannot be a chain in Tx . A contradiction.
Therefore, â ∈ δyx ([b, b∗ ]).
Suppose, for example, a1 ∈ δyx ([b, b∗ ]). Then there exist b ∈ [b, b∗ ] s.t.
(a1 , b ) ∈ δxy and ā ∈ [a, a∗ ] s.t. (ā, b ) ∈ δxy . We have δyx (δxy ({ā}) ⊇ [ā, a1 ] and
δyx (δxy ([ā, a2 ]) contains {ā, a1 , a2 }, which is not a subset of a chain. Therefore,
ai ∈ δyx ([b, b∗ ]).
So far, we have proved δyx (δxy ([a, a∗ ])) = [a, a∗ ]. We next show [b, b∗ ] is also
an ip-chain. First, we show every node in [b, b∗ ) has only one child. Suppose not
and b ∈ [b, b∗ ) has children b1 , b2 with b1 ∈ (b , b∗ ]. Since δxy ([â, a∗ ]) is a chain
that contains [b, b∗ ], we know (â, b2 ) is not in δxy . Furthermore, as δyx ({b , b2 }) is
a chain in Tx and the image of b2 is disjoint from [a, a∗ ], we must have (ai , b2 ) ∈
δxy for some child ai of a∗ . Note that then δxy ([a, ai ]) contains [b, b∗ ] and b2 and
thus is not a chain. This contradicts the chain-preserving property of δxy . Hence,
every node in [b, b∗ ) has only one child. In other words, [b, b∗ ] is contained in an
ip-chain [u, v].
 On Tree-Preserving Constraints 249

By the result we have proved so far, we know δxy (δyx ([u, v])) = [u, v] and
δyx ([u, v]) is contained in an ip-chain in Tx . Because [a, a∗ ] = δyx ([b, b∗ ]) ⊆
δyx ([u, v]) is an ip-chain, we know δyx ([u, v]) is exactly [a, a∗ ]. Therefore, we
have [u, v] = δxy (δyx ([u, v])) = δxy ([a, a∗ ]) = [b, b∗ ]. This proves that [b, b∗ ] is an
ip-chain in Ty .

Using the above result, we can break Tx into a set of ip-chains by deleting
the edges from each node a to its children if a has two or more children. Write
Ix for the set of ip-chains of Tx . Similar operation and notation apply to Ty . It
is clear that two different ip-chains in Ix are disjoint and δxy naturally gives rise
to a bijection from Ix to Iy .
Lemma 4. Suppose Δ is an arc-consistent and chain-preserving constraint net-
work over tree domains Tx (x ∈ V ). Fix a variable x ∈ V and let Ix = {Ix1 , ..., Ixl }
be the set of ip-chains of Tx . Then, for every y = x in V , the set of ip-chains
in Ty is {δxy (Ix1 ), δxy (Ix2 ), ..., δxy (Ixl )}. Write Δi for the restriction of Δ to Ixi .
Then each Δi is a CRC constraint network and Δ is consistent iff at least one
Δi is consistent.
The following result asserts that the class of chain-preserving constraints is
tractable.
Theorem 2. Let Δ be a chain-preserving constraint network. If no inconsis-
tency is detected, then enforcing arc- and path-consistency determines the con-
sistency of Δ and transforms Δ into a globally consistent network.
Proof. First, by Proposition 2, we transform Δ into an arc-consistent and
chain-preserving constraint network if no inconsistency is detected. Second, by
Lemma 4, we reduce the consistency of Δ to the consistency of the CRC con-
straint networks Δ1 , ..., Δl . By [6], we know enforcing path-consistency trans-
forms a CRC constraint network into a globally consistent one if no inconsistency
is detected. If enforcing arc- and path-consistency does not detect any inconsis-
tency, then the result is a set of at most l globally consistent CRC networks Δi ,
the union of which is globally consistent and equivalent to Δ.

Lemma 4 also suggests that we can use the variable elimination algorithm for
CRC constraints [19] to more efficiently solve chain-preserving constraints.

4 Path-Preserving Constraints
At first glance, path-preserving constraints seem to be more general than chain-
preserving constraints, but Fig. 1(a,b) show that they are in fact incomparable.
We show the class of path-preserving constraints is also tractable by estab-
lishing its connection with CRC constraints.
We have the following simple results.
Lemma 5. Suppose δxy and δyx are path-preserving (tree-preserving) w.r.t. tree
 
domains Tx and Ty . Let t be a subtree of Tx and δxy and δyx the restrictions of
 
δxy and δyx to t. Then both δxy and δyx are path-preserving (tree-preserving).
250 S. Kong et al.

Fig. 1. (a) A chain- but not path-preserving constraint; (b) A path- but not chain-
preserving constraint; (c) A tree-preserving but neither path- nor chain-preserving
constraint.

Lemma 6. Suppose δxy is nonempty and path-preserving (tree-preserving)

w.r.t. tree domains Tx and Ty . If v ∈ Ty has no support in Tx under δyx , then
all supported nodes of Ty are in the same branch of v. That is, every node in
any other branch of v is not supported under δyx .

Proof. Suppose a, b are two supported nodes in Ty . There exist u1 , u2 in Tx s.t.

−1
u1 ∈ δyx (a) and u2 ∈ δyx (b). By δyx = δxy , we have a ∈ δxy (u1 ) and b ∈
δxy (u2 ). Hence a, b ∈ δxy (πu1 ,u2 ). Since δxy is path-preserving (tree-preserving),
δxy (πu1 ,u2 ) is a path (tree) in Ty . If a, b are in two different branches of v, then
πa,b must pass v and hence we must have v ∈ δxy (πu1 ,u2 ). This is impossible as
v has no support.


It is worth noting that this lemma does not require δyx to be path- or tree-
preserving.
The following result then follows directly.
Proposition 3. Let Δ be a path-preserving (tree-preserving) constraint network
over tree domains Tx (x ∈ V ). If no inconsistency is detected, then Δ remains
path-preserving (tree-preserving) after enforcing arc-consistency.

Proof. Enforcing arc-consistency on Δ only removes values which have no sup-

port under some constraints. For any y ∈ V , if v is an unsupported value in Ty ,
then, by Lemma 6, every supported value of Ty is located in the same branch of v.
Deleting all these unsupported values from Ty , we get a subtree t of Ty . Applying
Lemma 5, the restricted constraint network to t remains path-preserving (tree-
preserving).


Definition 8. Let T be a tree. A path π in T is maximal if there exists no path

π  in T that strictly contains π.

We need three additional lemmas to prove the main result.

Lemma 7. Suppose δxy and δyx are arc-consistent and path-preserving w.r.t.
tree domains Tx and Ty . If π is a maximal path in Tx , then δxy (π) is a maximal
path in Ty .
 On Tree-Preserving Constraints 251

Lemma 8. Suppose δxy and δyx are arc-consistent and path-preserving w.r.t. Tx
and Ty . Assume a is a node in Tx with deg(a) > 2. Then there exists a unique
node b ∈ Ty s.t. (a, b) ∈ δxy . Moreover, deg(a) = deg(b).
Proof. Suppose π = a0 a1 ...ak is a maximal path in Tx and π ∗ = b0 b1 ...bl is its
image under δxy in Ty . W.l.o.g. we assume k, l ≥ 1. Suppose ai is a node in π
s.t. deg(ai ) > 2 and a ∈ π is another node in Tx s.t. {ai , a } is an edge in Tx .
Suppose δxy (ai ) = [bj , bj  ] and j  > j.
Because π is a maximal path and π ∗ is its image, we know δxy (a ) ∩ π ∗ = ∅.
Consider the edge {a , ai }. Since δxy ({a , ai }) is a path in Ty , there exists a
node b ∈ δxy (a ) s.t. either {b , bj } or {b , bj  } is an edge in Ty . Suppose w.l.o.g.
{b , bj  } is in Ty . Note that π ∗ = [b0 , bl ] is contained in the union of δxy ([a , a0 ])
and δxy ([a , ak ]). In particular, bl is in either δxy ([a , a0 ]) or δxy ([a , ak ]). Let
us assume bl ∈ δxy ([a , a0 ]). Then bl , bj , bj  , b (which are not on any path) are
contained in the path δxy ([a , a0 ]), a contradiction. Therefore, our assumption
that δxy (ai ) = [bj , bj  ] and j  > j is incorrect. That is, the image of ai under δxy
is a singleton, say, {bj }. We next show deg(bj ) = deg(ai ).
Because δxy (ai ) = {bj }, the image of each neighbor of ai in Tx must contain
a neighbor of bj , as δxy is path-preserving. Moreover, two different neighbors
ai , ai of ai cannot map to the same neighbor bj of bj . This is because the image
of bj under δyx , which is a path in Tx , contains ai and ai , and hence also
contains ai . This contradicts the assumption δxy (ai ) = {bj }. This shows that
deg(ai ) = deg(bj ).

Definition 9. Let T be a tree. A path π from a to a∗ in T is called an irreducible
perfect path (ip-path) if (i) every node on path π has degree 1 or 2; and (ii) any
neighbour of a (or a∗ ) that is not on π has degree 3 or more.
Let Fx = {a ∈ Tx : deg(a) > 2} and Fy = {b ∈ Ty : deg(b) > 2}. Then δxy ,
when restricted to Fx , is a bijection from Fx to Fy . Removing all edges incident
to a node in Fx , we obtain a set of pairwise disjoint paths in Tx . These paths are
precisely the ip-paths of Tx . Write Px for this set. Then δxy induces a bijection
from Px to Py .
Lemma 9. Suppose Δ is an arc-consistent and path-preserving constraint net-
work over tree domains Tx (x ∈ V ). Fix a variable x ∈ V and let Px =
{πx1 , ..., πxl } be the set of ip-paths in Tx . Then, for every y = x, the set of ip-
paths in Ty is {δxy (πx1 ), ..., δxy (πxl )}. Write Δi for the restriction of Δ to πxi .
Then each Δi is a CRC constraint network and Δ is consistent iff at least one
Δi is consistent.
Thus the class of path-preserving constraints is tractable.
Theorem 3. Let Δ be a path-preserving constraint network. If no inconsistency
is detected, then enforcing arc- and path-consistency determines the consistency
of Δ and transforms Δ into a globally consistent network.
The proof is analogous to that of Theorem 2. Lemma 9 suggests that we can use
the variable elimination algorithm for CRC constraints [19] to more efficiently
solve path-preserving constraints.
252 S. Kong et al.

5 Tree-Preserving Constraints
It is easy to see that every arc-consistent chain- or path-preserving constraint is
tree-preserving, but Fig. 1(c) shows that the other direction is not always true.
In this section, we show that the class of tree-preserving constraints is
tractable. Given a tree-preserving constraint network Δ, we show that, when
enforcing arc- and path-consistency on Δ, in each step, the network remains
tree-preserving. Hence, enforcing arc- and path-consistency on Δ will transform
it to an equivalent globally consistent network if no inconsistency is detected.
Moreover, we show that the partial path-consistency algorithm (PPC) of [2] is
applicable to tree-preserving constraint networks. PPC is more efficient than
path-consistency algorithm for large sparse constraints.

5.1 Enforcing Arc- and Path-Consistency Preserves Tree-Preserving

Unlike CRC and chain-preserving constraints, removing a value from a domain

may change the tree-preserving property of a network. Instead, we need to
remove a ‘trunk’ from the tree domain or just keep one branch.

Definition 10. Suppose a = b are two nodes of a tree T that are not neighbors.
The trunk between a, b, written Mab , is defined as the connected component of
T \{a, b} which contains all internal nodes of πa,b (see Fig.2). The M-contraction
of T by Ma,b , denoted by T  Ma,b , is the tree obtained by removing nodes in
Ma,b and adding an edge {a, b} to T .

Fig. 2. Mab is a trunk of tree T .

Lemma 10. Suppose δxy and δyx are arc-consistent and tree-preserving w.r.t.
tree domains Tx and Ty . Suppose a, b are two nodes in Tx s.t. δxy (a) ∪ δxy (b)
is not connected in Ty . Then there exist r, s ∈ Ty s.t. r ∈ δxy (a), s ∈ δxy (b),
and δyx (Mr,s ) ⊆ Ma,b . Let Ty∗ be the domain obtained by deleting from Ty all
nodes v s.t. δyx (v) ⊆ Ma,b . Then Ty∗ becomes a tree if we add the edge {r, s}.
Moreover, δxy and δyx remain arc-consistent and tree-preserving when restricted
to Tx  Ma,b and Ty∗ .
 On Tree-Preserving Constraints 253

Proof. Choose r ∈ δxy (a) and s ∈ δxy (b) such that the path πr,s from r to s in
Ty is a shortest one among {πr ,s : r ∈ δxy (a), s ∈ δxy (b)}. In particular, we
have πr,s ∩ (δxy (a) ∪ δxy (b)) = {r, s}. We assert that the image of every node v
in Mr,s under δyx is contained in Ma,b . Suppose otherwise and there exists u in
Tx \ Ma,b s.t. (u, v) ∈ δxy . Assume that u is in the same connected component as
a. Since the subtree δyx (πv,s ) contains u and b, it also contains a. This implies
that there is a node v  on πv,s which is in δxy (a). This is impossible as v ∈ Mr,s
and δxy (a) ∩ πr,s = {r}. Therefore δyx (v) ⊆ Ma,b for any v ∈ Mr,s . Hence
δyx (Mr,s ) ⊆ Ma,b holds.
It is clear that, when restricted to Tx  Ma,b and Ty  Mr,s , δxy ({a, b})
is connected and so is δyx ({r, s}). For any other edge {a , b } in Tx  Ma,b , by
δyx (Mr,s ) ⊆ Ma,b , δxy ({a , b }) ∩ Mr,s = ∅ and the image of {a , b } is unchanged
(hence connected) after the M-contraction of Ty . This shows that δxy is consec-
utive when restricted to Tx  Ma,b . Furthermore, since every node in Tx  Ma,b
is supported in Ty  Mr,s , we know δxy is also tree-preserving when restricted
to Tx  Ma,b .
It is possible that there is a node v ∈ Ty  Mr,s s.t. δyx (v) ⊆ Ma,b . We assert
that any v like this has at most one branch in Ty \Mr,s s.t. there is a node v  in the
branch which is supported under δyx by a node in Tx \ Ma,b . Because δxy is tree-
preserving when restricted to Tx Ma,b , this follows immediately from Lemma 6.
This implies that, if we remove all these nodes v s.t. δyx (v) ⊆ Ma,b from Ty Mr,s ,
the domain is still connected. As a consequence, the two constraints remain arc-
consistent and tree preserving when restricted to Tx  Ma,b and Ty∗ .


In general, we have

Lemma 11. Let Δ be an arc-consistent and tree-preserving constraint network

over tree domains Tx (x ∈ V ). Suppose x ∈ V and Ma,b is a trunk in Tx . When
restricted to Tx  Ma,b and enforcing arc-consistency, Δ remains tree-preserving
if we modify each Ty (y ∈ V ) by deleting all unsupported nodes and adding some
edges.

Proof (Sketch). The result follows from Lemmas 10 and 5. One issue we need
to take care of is how two trunks interact. Suppose x, y, z are three different
variables and M , M  are trunks to be contracted from Tx and Ty respectively.
Applying Lemma 10 to the constraints between x and z and, separately, to the
constraints between y and z, we get two different trunks, say Ma,b and Mc,d ,
to be contracted from the same tree domain Tz . Can we do this (i.e. applying
Lemma 10) one by one? Does the order of the contractions matter? To answer
these questions, we need to know what is exactly the union of two trunks. There
are in essence ten configurations as shown in Fig. 3. The union of two trunks can
be the whole tree, a branch, a trunk, or two disjoint trunks. If the union is the
whole tree, then the network is inconsistent; if it is a branch, then we can remove
it directly; if it is a trunk or two disjoint trunks, then we can use Lemma 10 to
contract them one by one in either order.

254 S. Kong et al.

Fig. 3. Possible configurations of trunks Ma,b and Mc,d .


Lemma 12. Assume δxy and δxy are two arc-consistent and tree-preserving
∗ 
constraints w.r.t. trees Tx and Ty . Let δxy = δxy ∩ δxy . Suppose u ∈ Tx and
 ∗
δxy (u) ∩ δxy (u) = ∅. Then the supported values of δxy in Tx are in at most two
branches of u.
∗
Proof. Suppose u1 , u2 , u3 are three supported values of δxy in Tx that are in

three different branches of u. Take wi ∈ δxy (ui ) ∩ δxy (ui ). For each i, we have

either wi ∈ δxy (u) or wi ∈ δxy (u). Recall that πwi ,wj denotes the unique path
πwi ,wj that connects wi to wj (1 ≤ i = j ≤ 3). There are two subcases. (1) One
node is on the path that connects the other two. Suppose w.l.o.g. w3 is between
w1 and w2 . If w3 ∈ δxy (u), then there exist v1 ∈ πw1 ,w3 and v2 ∈ πw3 ,w2 s.t.
v1 , v2 ∈ δxy (u). This is because the image of πw1 ,w3 and the image of πw3 ,w2
(under δyx ) both contain u. That is, v1 , v2 ∈ δxy (u) but w3 ∈ δxy (u). Since
δxy (u) is a subtree and w3 is on the path πv1 ,v2 , this is impossible. The case when

w3 ∈ δxy (u) is analogous. (2) The three nodes are in three different branches
of a node w. In this case, we note there exists a node between any path πwi ,wj
which is a support of u under δxy . It is easy to see that w itself is a support of u

under δxy . Similarly, we can show w is also a support of u under δxy . Since both
∗
subcases lead to a contradiction, we know the supported values of δxy are in at
most two branches of u.


Actually, this lemma shows


Corollary 1. Assume δxy and δxy are two arc-consistent and tree-preserving

constraints w.r.t. trees Tx and Ty . Then those unsupported values of δxy ∩ δxy in
Tx are in a unique set of pairwise disjoint branches and trunks.
Similar to Lemma 10, we have
 On Tree-Preserving Constraints 255


Lemma 13. Suppose δxy and δxy are arc-consistent and tree-preserving con-
 ∗ 
straints w.r.t. trees Tx and Ty and so are δyx and δyx . Let δxy = δxy ∩ δxy .
∗ ∗
Assume {u, v} is an edge in Tx s.t. δxy (u) ∪ δxy (v) is disconnected in Ty . Then
∗ ∗
there exist r ∈ δxy (u) and s ∈ δxy (v) s.t. every node in Mr,s is unsupported
∗
under δyx .
∗ ∗
Proof. Write Tr = δxy (u) and Ts = δxy (v). Clearly, Tr and Ts are nonempty
subtrees of Ty . Since they are disconnected, there exist r ∈ Tr , s ∈ Ts s.t.
πr,s ∩ (Tr ∪ Ts ) = {r, s} (see Fig. 4 for an illustration). Write A = δxy (u),
 
B = δxy (v), C = δxy (u) and D = δxy (v). We show every node in Mr,s is not
∗
supported under δyx .
Suppose w is an arbitrary internal node on πr,s . We first show w is not
∗
supported under δyx . Note w ∈ A ∪ B, w ∈ C ∪ D, w ∈ A ∩ C, and w ∈ B ∩ D.
There are two cases according to whether w ∈ A. If w ∈ A, then we have w ∈ C,
w ∈ D, and w ∈ B. If w ∈ A, then we have w ∈ B, w ∈ D, and w ∈ C. Suppose
w.l.o.g. w ∈ A. By w ∈ A = δxy (u), we have u ∈ δyx (w); by w ∈ B = δxy (v), we
 
have v ∈ δyx (w). Similarly, we have u ∈ δyx (w) and v ∈ δyx (w). Thus subtree
 ∗
δyx (w) is disjoint from subtree δyx (w). This shows δyx (w) = ∅ and hence w is
∗
not supported under δyx .
Second, suppose w1 is an arbitrary node in Mr,s s.t. w1 is in a different branch
of w to r and s, i.e. πw,w1 ∩ (Tr ∪ Ts ) = ∅. We show w1 is not supported under
∗
δyx either.
Again, we assume w ∈ A. In this case, we have u ∈ δyx (w) ⊆ δyx (πw,w1 )
 
and v ∈ δyx (w) ⊆ δyx (πw,w1 ). As πw,w1 ∩ (Tr ∪ Ts ) = ∅, we have πw,w1 ∩ Tr =
πw,w1 ∩ A ∩ C = ∅. As πw,w1 ∩ A = ∅ and A ∩ C = ∅, by Lemma 1, we must
 
have πw,w1 ∩ δxy (u) = ∅. This shows u ∈ δyx (πw,w1 ). Similarly, we can show

v ∈ δyx (πw,w1 ). Thus subtree δyx (πw,w1 ) is disjoint from subtree δyx (πw,w1 ) and,
∗ ∗
hence, δyx (πw,w1 ) = ∅. This proves that w1 is not supported under δyx either.
In summary, every node in Mr,s is unsupported.


Fig. 4. Illustration of proof of Lemma 13.

Proposition 4. [18] Assume δxz and δzy are two tree-preserving constraints
w.r.t. trees Tx , Ty , and Tz . Then their composition δxz ◦ δzy is tree-preserving.
256 S. Kong et al.

At last, we give the main result of this section.

Theorem 4. Let Δ be a tree-preserving constraint network. If no inconsistency

is detected, then enforcing arc- and path-consistency determines the consistency
of Δ and transforms Δ into a globally consistent network.

Proof. If we can show that Δ is still tree-preserving after enforcing arc and
path-consistency, then by Theorem 1 the new network is globally consistent if
no inconsistency is detected.
By Proposition 3, Δ remains tree-preserving after enforcing arc-consistency.
To enforce path-consistency on Δ, we need to call the following updating rule

δxy ← δxy ∩ (δxz ◦ δzy ) (2)

for x, y, z ∈ V until the network is stable.

Suppose Δ is arc-consistent and tree-preserving w.r.t. trees Tx for x ∈ V
before applying (2). Note that if δ ∗ = δxy ∩ (δxz ◦ δzy ) (as well as its converse)
is arc-consistent, then δ ∗ (u) is nonempty for any node u in Tx . By Corollary 1,
no branches or trunks need to be pruned in either Tx or Ty . Furthermore, by
Lemma 13, δ ∗ (u) ∪ δ ∗ (v) is connected for every edge {u, v} in Tx as there are no
unsupported nodes in Ty under the converse of δ ∗ . Therefore δ ∗ is arc-consistent
and consecutive, hence, tree-preserving.
If δ ∗ is not arc consistent, then we delete all unsupported values from Tx
and Ty and enforce arc-consistency on Δ. If no inconsistency is detected then
we have an updated arc-consistent and tree-preserving network by Lemma 11.
Still write Δ for this network and recompute δ ∗ = δxy ∩ (δxz ◦ δzy ) and repeat
the above procedure until either inconsistency is detected or δ ∗ is arc-consistent.
Note that, after enforcing arc-consistency, the composition δxz ◦ δzy may have
changed.
Once arc-consistency of δ ∗ is achieved, we update δxy with δ ∗ and continue
the process of enforcing path-consistency until Δ is path-consistent or an incon-
sistency is detected.


5.2 Partial Path-Consistency

The partial path-consistency (PPC) algorithm was first proposed by Bliek and
Sam-Haroud [2]. The idea is to enforce path consistency on sparse graphs by
triangulating instead of completing them. Bliek and Sam-Haroud demonstrated
that, as far as CRC constraints are concerned, the pruning capacity of path
consistency on triangulated graphs and their completion are identical on the
common edges.
An undirected graph G = (V, E) is triangulated or chordal if every cycle of
length greater than 3 has a chord, i.e. an edge connecting two non-consecutive
vertices of the cycle. For a constraint network Δ = {vi δij vj : 1 ≤ i, j ≤ n}
over V = {v1 , ..., vn }, the constraint graph of Δ is the undirected graph G(Δ) =
(V, E(Δ)), for which we have (vi , vj ) ∈ E(Δ) iff δij is not a universal constraint.
 On Tree-Preserving Constraints 257

Given a constraint network Δ and a graph G = (V, E), we say Δ is partial path-
consistent w.r.t. G iff for any 1 ≤ i, j, k ≤ n with (vi , vj ), (vj , vk ), (vi , vk ) ∈ E
we have δik ⊆ δij ◦ δjk [2].
Theorem 5. Let Δ be a tree-preserving constraint network. Suppose G = (V, E)
is a chordal graph such that E(Δ) ⊆ E. Then enforcing partial path-consistency
on G is equivalent to enforcing path-consistency on the completion of G, in the
sense that the relations computed for the constraints in G are identical.
Proof. The proof is similar to the one given for CRC constraints [2, Theorem 3].
This is because, (i) when enforcing arc- and path-consistency on a tree-preserving
constraint network, in each step, we obtain a new tree-preserving constraint
network; and (ii) path-consistent tree convex constraint networks are globally
consistent.

Remark 1. Note that our definition and results of tree-preserving constraints
can be straightforwardly extended to domains with acyclic graph structures
(which are connected or not). We call such a structure a forest domain. Given a
tree-preserving constraint network Δ over forest domains F1 , ..., Fn of variables
v1 , ..., vn . Suppose Fi consists of trees ti,1 , ..., ti,ki . Note that the image of each
tree, say ti,1 , of Fi under constraint Rij is a subtree t of Fj . Assume t is con-
tained in the tree tj,s of forest Fj . Then the image of tj,s under constraint Rji
is a subtree of ti,1 . This establishes, for any 1 ≤ i = j ≤ n, a 1-1 correspondence
between trees in Fi and trees in Fj if the image of each tree is nonempty. In
this way, the consistency of Δ is reduced to the consistency of several parallel
tree-preserving networks over tree domains.

6 Tree-Preserving Constraints and the Scene Labelling

Problem
The scene labelling problem [10] is a classification problem where all edges in
a line-drawing picture have to be assigned a label describing them. The scene
labelling problem is NP-complete in general. This is true even in the case of the
trihedral scenes, i.e. scenes where no four planes share a point [12].
Labels used in the scene labelling problem are listed as follows:
‘+’ The edge is convex which has both of its corresponding planes visible;
‘−’ The edge is concave which has both of its corresponding planes visible;
‘→’ Only one plane associated with the edge is visible, and when one moves in
the direction indicated by the arrow, the pair of associated planes is to the
right.
In the case of trihedral scenes, there are only four basic ways in which three
plane surfaces can come together at a vertex [10]. A vertex projects in the picture
into a ‘V ’, ‘W ’, ‘Y ’ or ‘T ’-junction (each of these junction-types may appear
with an arbitrary rotation in a given picture). A complete list of the labelled
line configurations that are possible in the vicinity of a node in a picture is given
in Fig. 5.
258 S. Kong et al.

Fig. 5. Possible labelled line configurations of a junction in a picture and their corre-
sponding forest structures.

In this section, we show that (i) every instance of the trihedral scene labelling
problem can be modelled by a tree convex constraint network; (ii) a large sub-
class of the trihedral scene labelling problem can be modelled by tree-preserving
constraints; (iii) there exists a scene labelling instance which can be modelled
by tree-preserving constraints but not by chain- or CRC constraints.
A CSP for the scene labelling problem can be formulated as follows. Each
junction in the line-drawing picture is a variable. The domains of the vertices are
the possible configurations as shown in Fig. 5. The constraints between variables
are simply that, if two variables share an edge, then the edge must be labeled
the same at both ends.
Proposition 5. Every instance of the trihedral scene labelling problem can be
modelled by a tree convex constraint network. Furthermore, there are only 39
possible configurations of two neighbouring nodes in 2D projected pictures of 3D
trihedral scenes, and 29 out of these can be modelled by tree-preserving con-
straints.

Proof. The complete list of these configurations and their corresponding tree
convex or tree-preserving constraints is attached as an appendix. Note that we
do not consider T-junctions in line drawing pictures since they decompose into
unary constraints.


As a consequence, we know that these 29 configurations of the scene labelling

problem with 2D line-drawing pictures can be solved by the path-consistency
algorithm in polynomial time. Moreover, since it is NP-hard to decide if a trihe-
dral scene labelling instance is consistent, we have the following corollary.
Corollary 2. The consistency problem of tree convex constraint networks is NP-
complete.
 On Tree-Preserving Constraints 259

We next give a scene labelling instance which can be modelled by tree-

preserving constraints but not by chain-preserving or CRC constraints. Consider
the line drawing in the left of the following figure and the constraints for the
drawing listed in the right. One can easily verify that all constraints are tree-
preserving w.r.t. the forest structures listed in Fig. 5, but, for example, δ21 is
not chain-preserving for the forest structures illustrated in Fig. 5 and δ25 is not
CRC.

7 Further Discussion and Conclusion

In this paper, when formulating a CSP, we allow different variables have different
tree domains. Feder and Vardi [7] and many other authors (see e.g. [1,11]) also
considered CSPs which have a common domain D for all variables. These CSPs
are called one-sorted in [3]. For one-sorted tree-preserving CSPs, we could also
define a majority operator [11] under which the set of tree-preserving constraints
is closed. This implies that the class of one-sorted tree-preserving CSPs has
bounded strict width [7] and hence tractable. Indeed, such a majority operator ρ
is defined as follows: for any three nodes a, b, c in a tree domain T , define ρ(a, b, c)
as the node d which is the intersection of paths πa,b , πa,c , and πb,c . Following [3],
it is straightforward to extend this result to multi-sorted tree-preserving CSPs.
In this paper, we identified two new tractable subclasses of tree convex con-
straint which are called path- and tree-preserving constraints, and proved that
a chain- or path-preserving constraint network is in essence the disjoint union
of several independent CRC constraint networks, and hence (re-)established the
tractability of these constraints. More importantly, we proved that when enforc-
ing arc- and path-consistency on a tree-preserving constraint network, in each
step, the network remains tree-preserving. This implies that enforcing arc- and
260 S. Kong et al.

path-consistency will change a tree-preserving constraint network into a globally

consistent constraint network. This also implies that the efficient partial path-
consistent algorithm for large sparse networks is applicable for tree-preserving
constraint network. As an application, we showed that a large class of the tri-
hedral scene labelling problem can be modelled by tree-preserving constraints.
This shows that tree-preserving constraints are useful in real world applications.

Acknowledgments. We sincerely thank the anonymous reviewers of CP-15 and

IJCAI-15 for their very helpful comments. The majority operator was first pointed
out to us by two reviewers of IJCAI-15. This work was partially supported by ARC
(FT0990811, DP120103758, DP120104159) and NSFC (61228305).

References
1. Barto, L., Kozik, M.: Constraint satisfaction problems solvable by local consistency
methods. Journal of ACM 61(1), 3:1–3:19 (2014)
2. Bliek, C., Sam-Haroud, D.: Path consistency on triangulated constraint graphs.
In: IJCAI 1999, pp. 456–461 (1999)
3. Bulatov, Andrei A., Jeavons, Peter G.: An algebraic approach to multi-sorted
constraints. In: Rossi, Francesca (ed.) CP 2003. LNCS, vol. 2833, pp. 183–198.
Springer, Heidelberg (2003)
4. Conitzer, V., Derryberry, J., Sandholm, T.: Combinatorial auctions with structured
item graphs. In: AAAI 2004, pp. 212–218 (2004)
5. Dechter, R., Meiri, I., Pearl, J.: Temporal constraint networks. Artificial Intelli-
gence 49(1–3), 61–95 (1991)
6. Deville, Y., Barette, O., Hentenryck, P.V.: Constraint satisfaction over connected
row convex constraints. Artificial Intelligence 109(1–2), 243–271 (1999)
7. Feder, T., Vardi, M.Y.: The computational structure of monotone monadic snp and
constraint satisfaction: A study through datalog and group theory. SIAM Journal
on Computing 28(1), 57–104 (1998)
8. Freuder, E.C.: Synthesizing constraint expressions. Communications of the ACM
21(11), 958–966 (1978)
9. Freuder, E.C.: A sufficient condition for backtrack-free search. Journal of the ACM
29(1), 24–32 (1982)
10. Huffman, D.A.: Impossible objects as nonsense sentences. Machine Intelligence
6(1), 295–323 (1971)
11. Jeavons, P., Cohen, D.A., Cooper, M.C.: Constraints, consistency and closure.
Artificial Intelligence 101(1–2), 251–265 (1998)
12. Kirousis, L.M., Papadimitriou, C.H.: The complexity of recognizing polyhedral
scenes. In: FOCS 1985, pp. 175–185 (1985)
13. Kumar, T.K.S.: Simple randomized algorithms for tractable row and tree convex
constraints. In: AAAI 2006, pp. 74–79 (2006)
14. Li, S., Liu, W., Wang, S.: Qualitative constraint satisfaction problems: An extended
framework with landmarks. Artificial Intelligence 201, 32–58 (2013)
15. Maruyama, H.: Structural disambiguation with constraint propagation. In: ACL
1990, pp. 31–38 (1990)
16. Montanari, U.: Networks of constraints: Fundamental properties and applications
to picture processing. Information Sciences 7, 95–132 (1974)
 On Tree-Preserving Constraints 261

17. Van Beek, P., Dechter, R.: On the minimality and global consistency of row-convex
constraint networks. Journal of the ACM 42(3), 543–561 (1995)
18. Zhang, Y., Freuder, E.C.: Properties of tree convex constraints. Artificial Intelli-
gence 172(12–13), 1605–1612 (2008)
19. Zhang, Y., Marisetti, S.: Solving connected row convex constraints by variable
elimination. Artificial Intelligence 173(12), 1204–1219 (2009)
20. Zhang, Y., Yap, R.H.C.: Consistency and set intersection. In: IJCAI 2003,
pp. 263–270 (2003)
 Modeling and Solving Project Scheduling
with Calendars

Stefan Kreter1 , Andreas Schutt2,3(B) , and Peter J. Stuckey2,3

1
Operations Research Group, Institute of Management and Economics,
Clausthal University of Technology, 38678 Clausthal-Zellerfeld, Germany
[email protected]
2
Optimisation Research Group, National ICT Australia, Melbourne, Australia
{andreas.schutt,peter.stuckey}@nicta.com.au
3
Department of Computing and Information Systems, The University of Melbourne,
Melbourne, VIC 3010, Australia

Abstract. Resource-constrained project scheduling with the objective

of minimizing project duration (RCPSP) is one of the most studied
scheduling problems. In this paper we consider the RCPSP with gen-
eral temporal constraints and calendar constraints. Calendar constraints
make some resources unavailable on certain days in the scheduling period
and force activity execution to be delayed while resources are unavail-
able. They arise in practice from, e.g., unavailabilities of staff during
public holidays and weekends. The resulting problems are challenging
optimization problems. We develop not only four different constraint
programming (CP) models to tackle the problem, but also a specialized
propagator for the cumulative resource constraints taking the calendar
constraints into account. This propagator includes the ability to explain
its inferences so it can be used in a lazy clause generation solver. We
compare these models, and different search strategies on a challenging
set of benchmarks using a lazy clause generation solver. We close 83 of
the open problems of the benchmark set, and show that CP solutions
are highly competitive with existing Mip models of the problem.

1 Introduction
The resource-constrained project scheduling problem with general temporal and
calendar constraints (RCPSP/max-cal) is an extension of the well-known RCPSP
and RCPSP/max (see, e.g., [14, Chap. 2]) through calendars. The RCPSP/max-
cal can be given as follows. For a set of activities, which require time and renew-
able resources for their execution, execution time intervals must be determined
in a way that minimum and maximum time lags between activities are satisfied,
the prescribed resource capacities are not exceeded, and the project duration is
minimized. The difference with RCPSP/max is that a calendar is given for each
renewable resource type that describes for each time period whether the resource
type is available or unavailable. Time periods of unavailability can occur, e.g.,
due to weekends or public holidays. The activities and time lags are dependent

c Springer International Publishing Switzerland 2015
G. Pesant (Ed.): CP 2015, LNCS 9255, pp. 262–278, 2015.
DOI: 10.1007/978-3-319-23219-5 19
 Modeling and Solving Project Scheduling with Calendars 263

resource 3
3 1 6 @
- @
1  2 3 @ 3
@

{1, 2, 3} {3}
@ 2 @
@
0 @ 0 @
R
@ 1
@ -t
0 5 1 2 3 4 5 6 7 8 9 10

∅ @  ∅ resource 2
@ 3 2
R 3  - 4
6 @ @ 4
@ 3
@ @
@ @ 3
@ @
{1, 2, 3} {1, 2} @ @
@ @
(a) Logic diagram of the example project 1 @ @
@ @ -t
1 2 3 4 5 6 7 8 9 10

resource 1
6 J

6 4
Cal3
JJ
J

3
Cal2
JJ
JJ
Cal1 -t
1
-t
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10

(b) Time periods of unavailability (c) Possible resource allocation

Fig. 1. Illustrative Example for RCPSP/max-cal

on the resource calendars, too, and some activities can be interrupted for the
duration of a break while others cannot be interrupted due to technical reasons.
For the interruptible activities a start-up phase is given during which the activ-
ity is not allowed to be paused. Concerning the renewable resource types one
distinguishes resource types that stay engaged or are blocked, respectively, dur-
ing interruptions of activities that require it and resource types that are released
and can be used to carry out other activities during interruptions.
Our motivation for developing CP models for the RCPSP/max-cal and using
lazy clause generation to solve it lies in the very good results obtained by
[18,19,20] solving RCPSP and RCPSP/max by lazy clause generation.
Example 1. Figure 1 shows an illustrative example with six activities and three
renewable resource types. The project start (activity 0) and the project end
(activity 5) are fictitious activities, i.e., they do not require time or resources. A
logic diagram of the project is given in Fig. 1(a) where each activity is represented
by a node with the duration given above and the set of resource types used by
the activity below the node. The arcs between the nodes represent time lags.
The calendars of the three renewable resource types are depicted in Fig. 1(b).
If there is a box with an X for a resource type k and time t, then resource type k
is not available at time t. Resource type 1 is always available and can be thought
of as a machine. Resource types 2 and 3 can be thought of as different kinds of
staff where resource type 2 (3) has a five-day (six-day) working week. In addition,
assume that resource type 1 stays engaged or is blocked, respectively, during a
264 S. Kreter et al.

break of an activity that requires resource type 1 for its execution while resource
types 2 and 3 are released during interruptions of activities.
A possible resource allocation of the three renewable resource types is shown
in Fig. 1(c). Activity 3 requires all renewable resource types for its execution.
Since resource type 2 is not available in periods 6 and 7, activity 3 is interrupted
during these periods. While resource type 1 stays engaged during the interrup-
tion, resource type 3 can be used to carry out activity 2 in period 6. 

Few authors have dealt with calendars in project scheduling so far. A time
planning method for project scheduling with the same calendar for each resource
type is introduced in [23]. In [6] the RCPSP/max with different calendars for
each renewable resource type is investigated for the first time but the start-
up phase of the interruptible activities are not taken into account. [6] proposes
methods to determine the earliest and latest start and completion times for the
project activities and priority rule methods. Procedures to determine the earliest
and latest start times if a start-up phase is taken into account are presented in
[7] and [14, Sect. 2.11]. In addition, they sketch how priority-rule methods for
the RCPSP/max can be adapted for calendars. In the approach in [7] and [14,
Sect. 2.11] all resources stay engaged during interruptions of activities. Within
the priority-rule methods in [6,7], and [14, Sect. 2.11] the procedures to deter-
mine the earliest and latest start times must be carried out in each iteration.
Recently, a new time planning method, three binary linear model formulations,
and a scatter search procedure for the RCPSP/max-cal were developed in [9].
Moreover, Kreter et al. [9] introduce a benchmark test set which is based on
the UBO test set for RCPSP/max [8]. The time planning method determines all
time and calendar feasible start times for the activities and absolute time lags
depending on the start times of the activities once in advance and then uses this
throughout the scatter search.
In CP, the works [3,4] respectively propose calendar constraints/rules for
ILOG Schedule and Cosytech CHIP. The former [3] was generalized to intensity
functions of activities in IBM ILOG CP Optimizer, while breaks of activities
extend the length between their start and end times, only resource types that
stay engaged can be modeled directly. The latter [4] introduces constraint rules
in the global constraint diffn for parallel machine scheduling.
A practical application where calendars must be considered as well as other
additional constraints can be found in batch scheduling [21]. Problems that are
related to the RCPSP/max-cal are treated in [22,5]. An alternative approach to
include calendars into project scheduling that makes use of calendar independent
start-start, start-end, end-start, and end-end time lags is proposed in [22] and [5]
studies the RCPSP with non-preemptive activity splitting, where an activity in
process is allowed to pause only when resource levels are temporarily insufficient.

2 Problem Description
In this section we describe the RCPSP/max-cal formally and give an example
instance. We use identifiers and definitions from [9]. In what follows, we assume
 Modeling and Solving Project Scheduling with Calendars 265

that a project consists of a set V := {0, 1, . . . , n, n + 1}, n ≥ 1, of activities,

where 0 and n + 1 represent the begin and the end of the project, respectively.
Each activity i has a processing time pi ∈ N0 . Activities i with pi > 0 are called
real activities and the set of real activities is denoted by V r ⊂ V . Activities 0
and n + 1 as well as milestones, which specify significant events of the project
and have a duration of pi = 0, form the set V f = V \ V r of fictitious activities.
A project completion deadline d ∈ N has to be determined in order to define
the time horizon of the calendars and the time axis is divided into intervals
[0, 1), [1, 2), . . . , [d−1, d) where a unit length time interval [t−1, t) is also referred
to as time period t. The set of renewable resource types is denoted by R and for
each renewable resource type k ∈ R a resource capacity Rk ∈ N is given that
must not be exceeded at any point in time. The amount of resource type k that
is used constantly during the execution of activity i ∈ V is given by rik ∈ N0 .
For fictitious activities i ∈ V f rik := 0 holds for all k ∈ R. For each resource
type a resource calendar is given.
Definition 1. A calendar for resource k ∈ R is a step function Calk (·) :
[0, d) → {0, 1} continuous from the right at the jump points, where the condition

1, if period [t , t + 1 ) is a working period for k
Calk (t) :=
0, if period [t , t + 1 ) is a break period for k

is satisfied.
With Ri := {k ∈ R | rik > 0} indicating the set of resource types that is used
to carry out activity i ∈ V , an activity calendar Ci (·) : [0, d) → {0, 1} can be
determined from the resource calendars as follows:

mink∈Ri Calk (t), if Ri
= ∅
Ci (t) :=
1, otherwise.

Then, for every activity i and a point in time t ∈ T := {0, 1, . . . , d} functions

next breaki (t) and next starti (t) give the start time and the end time of the
next break after time t in calendar Ci , respectively.

next breaki (t) := min{τ ∈ T | τ > t ∧ Ci (τ ) = 0}

next starti (t) := min{τ ∈ T | τ > t ∧ Ci (τ ) = 1 ∧ Ci (τ − 1) = 0}

When calendars are present, we have to distinguish activities that can be inter-
rupted for the duration of a break in the underlying activity calendar and activ-
ities that are not allowed to be interrupted. The set of (break-)interruptible
activities is denoted by V bi ⊂ V and the set of non-interruptible activities is
given by V ni = V \ V bi , where V f ⊆ V ni holds. The execution of an activity
i ∈ V bi must be interrupted at times t with Ci (t) = 0, and the execution must
be continued at the next point in time τ > t with Ci (τ ) = 1. Si ∈ T indicates the
start time and Ei ∈ T represents the end of activity i ∈ V . Since the jump points
in the calendars Calk , k ∈ R, are all integer valued, the points in time where an
activity is interrupted or continued are integer valued, too. The completion time
266 S. Kreter et al.

t−1
of activity i ∈ V can be determined by Ei (Si ) := min{t | τ =Si Ci (τ ) = pi }.
For each activity i ∈ V a start-up phase εi ∈ N0 is given during which activity
i is not allowed to be interrupted. For all activities i ∈ V ni εi := pi holds. We
assume that the underlying project begins at time 0, i.e., S0 := 0. Then, the
project duration equals Sn+1 . In addition, we assume that no activity i ∈ V can
be in execution before the project start, i.e., Si ≥ 0, or after the project end,
i.e., Ei ≤ Sn+1 .
Between the activities a set A of minimum and maximum time lags is given.
W.l.o.g. these time lags are defined between the start times of the activities (see
[6,9]). For each time lag i, j ∈ A, a resource set Rij ⊆ R and a length δij ∈ Z
are given, from which we can compute a calendar Cij (·) : [0, d) → {0, 1} for each
time lag by 
mink∈Rij Calk (t), if Rij
= ∅
Cij (t) :=
1, otherwise
i.e., at least tu time units mustelapse after the start of activity i before activity j
t−1
can start where tu = min{t | τ =Si Cij (τ ) = δij }.
With parameter ρk we indicate whether renewable resource types k ∈ R
stay engaged or are blocked, respectively, during interruptions of activities that
require it (ρk = 1) or are released and can be used to carry out other activities
during interruptions (ρk = 0). A vector S = (S0 , S1 , . . . , Sn+1 ) of all activity
start times is called a schedule. Given a schedule S and point in time t the set
of all real activities i ∈ V r that are started before but not completed at time
t is called the active set and can be determined by A(S, t) := {i ∈ V r | Si ≤
t < Ei (Si )}. Then, the resource utilization rkcal (S, t) of resource k ∈ R at time t
according to schedule S can be computed by
 
rkcal (S, t) := rik + rik ρk .
i∈A(S,t)|Ci (t)=1 i∈A(S,t)|Ci (t)=0

With the introduced notation the following mathematical formulation for the
RCPSP/max-cal can be given (cf. [6]):

Minimize Sn+1 (1)

Si +εi −1
subject to Ci (t) = εi i∈V (2)
t=Si
Sj −1 Si −1
Cij (t) − Cij (t) ≥ δij i, j ∈ A (3)
t=Si t=Sj

rkcal (S, t) ≤ Rk k ∈ R, t ∈ T \ {d} (4)

Si ∈ T i∈V (5)

The aim of the RCPSP/max-cal is to find a schedule that minimizes the project
makespan (1) and satisfies the calendar constraints (2), time lags (3), and
resource capacities (4).
Each project can be represented by an activity-on-node network where each
activity i ∈ V is represented by a node and each time lag i, j ∈ A is given by
an arc from node i to node j with weights δij and Rij . The activity duration as
 Modeling and Solving Project Scheduling with Calendars 267

3,13,R1 - 1,1 Cal1 (t)

1  2
 -7,R 2 @ 1 r
6
0,∅ 2,1,2 0,0,3
@1,R2 -t
0,0 @ 0,0 2 4 6 8 10
@
R Cal2 (t)
0  5
-10,∅ r
6 rq r
0,0,0@ 0,∅ 0,0,0 1
2,R4 r rq -t
@ 2 4 6 8 10
@ 3,2
3,R3 - 2,2 Cal3 (t)
@R 3  4 1 r
6 qr r
1,2,1 -3,R3 2,1,0 r rq -t
2 4 6 8 10
pi , εi pj , εj
δij , Rij
Legend: i - j
ri1 , ri2 , ri3 rj1 , rj2 , rj3

Fig. 2. Activity-on-node network and resource calendars

well as the start-up phase is given above node i in an activity-on-node network

and the resource requirements of activity i ∈ V are given below node i. For
the case where time lags depend on calendars, the label-correcting and triple
algorithm (see, e.g., [2, Sects. 5.4 and 5.6]) can be adapted and integrated in
a time planning procedure that determines a set Wi for each activity i ∈ V
containing all start times that are feasible due to the time lags and calendar
constraints, i.e., this procedure determines the solution space of the resource
relaxation of the RCPSP/max-cal (problem (1)–(3), (5)) [9]. In addition to the
sets Wi , the time planning procedure in [9] determines the “absolute” durations
of each activity and time lag with respect to the activities start times. The
absolute duration of an activity i ∈ V is denoted by pi (Si ) := Ei (Si ) − Si and
the absolute time lag for i, j ∈ A by dij (t) for each t ∈ Wi .
Example 2. Figure 2 shows the problem of Ex. 1 again, but now filled with
information for the activites start-up phases and resource requirements as well
as information for the time lags.
Activities 0, 2, 4, and 5 are non-interruptible while activities 1 and 3 form
the set V bi and therefore can be interrupted for the duration of a break in the
underlying activity calendar. By applying the determination rules from above
Cal1 = C0 = C5 = C01 = C03 = C50 , Cal2 = C1 = C3 = C4 = C12 = C34 =
C43 = C45 , and Cal3 = C2 = C21 = C25 hold for the activity and time lag
calendars. Since both time lags between activities 3 and 4 depend on the same
calendar and p3 = δ34 = −δ43 , activity 4 must be started when activity 3 ends
or more precisely at the next point in time after the end of activity 3 where the
calendar equals 1. The arc from the project end (node 5) to the project start
(node 0) represents an upper bound on the planning horizon of d = 10.
For the given example the time planning procedure from [9] determines
the sets W0 = {0}, W1 = {0, 1, 2, 3, 4}, W2 = {3, 4, 5, 7, 8, 9}, W3 = {0, 2, 3},
268 S. Kreter et al.

r1cal (S, t) r2cal (S, t)

6 s rs 6 s rs
3 3
2 s sr3 s sr 2 3 s rs s rs
1 1 s sr 4 1 s sr 3 3 s sr
s rs 3 s- t s rs 1 s sr 4 s- t
5 10 5 10
r3cal (S, t)

6 s rs s rs
3
2 s sr3
1 1 s sr2 s sr
s rs 3 s rs3 s -t
5 10

Fig. 3. Resource profiles of schedule S = (0, 1, 5, 3, 8, 10)

W4 = {3, 7, 8}, and W5 = {5, 6, 7, 8, 9, 10}. For example, activity 4 cannot start
at times 5 or 6 since there is a break in calendar C4 from 5 to 7. Moreover, activ-
ity 4 cannot start at time 4 because it has to be executed without interruptions.
Due to the time lag between activities 3 and 4, activity 3 cannot start at time
1, because if activity 3 started at time 1 activity 4 must start at time 4.
For the time- and calendar-feasible schedule S = (0, 1, 5, 3, 8, 10) the resource
profiles are given in Fig. 3. As already mentioned in the introduction resource
type 1 stays engaged during interruptions (ρ1 = 1) while resource types 2 and
3 are released during interruptions (ρ2 = ρ3 = 0). If the inequality Rk ≥ 3 is
fullfilled for each k ∈ R, schedule S is resource feasible and therefore a feasible
solution for the given example. 2

3 Models for RCPSP/max-cal

In this section, we present four different ways of modeling the RCPSP/max-cal.
The first three approaches use only well-known constraints from finite domain
propagation, while a new constraint to model the resource restrictions of the
RCPSP/max-cal and a corresponding propagator are used in the fourth model.

3.1 Model timeidx (Time Indexed Formulation)

In preprocessing, the time planning procedure of [9] is used to determine the
sets Wi of all time- and calendar-feasible start times for each activity i ∈ V and
S i ∈ Wi i∈V (6)
must be satisfied. Since the absolute time lags between the activities are depen-
dent on the start time of activity i for each i, j ∈ A, element constraints are
used to ensure that the correct values are taken into account.
element(Si , dij , dij ) i, j ∈ A (7)
 Modeling and Solving Project Scheduling with Calendars 269

Thereby, dij is an array that contains for all Si ∈ Wi the corresponding dij (Si )
value. Then, the constraints modelling time lags are

Sj − Si ≥ dij i, j ∈ A (8)

Absolute durations of the activities i ∈ V are used and the correct assignment
is ensured again by element constraints, where pi is an array containing for all
Si ∈ Wi the coresponding pi (Si ) value.

element(Si , pi , pi ) i∈V (9)

We implement the resource constraints using a time-indexed decomposition with

binary variables bit for each real activity i ∈ V r and point in time t ∈ T where
bit is true when i runs at t.

bit ↔ Si ≤ t ∧ t < Si + pi i ∈ V r, t ∈ T (10)


bit rik (Ci (t) + (1 − Ci (t)) ρk ) ≤ Rk k ∈ R, t ∈ T (11)
i∈V r

Model timeidx can now be given by: Minimize Sn+1 subject to (6) − (11).

3.2 Model 2cap (Doubling Resource Capacity)

Usually global propagators should be used to implement the resource constraints,

since more information is taken into account during propagation. This model
and the next make use of the global cumulative propagator [1] that explains
its propagation [16]. If the resource k ∈ R under investigation stays engaged
during interruptions of activities that require k for their execution, i.e., ρk = 1,
the global cumulative propagator can be used directly with the absolute activity
durations. If we regard the absolute duration of each activity i ∈ V and assume
that activity i requires rik units of resource k ∈ R with ρk = 0 at each point
in time {Si , . . . , Ei (Si ) − 1}, there can be resource overloads at break times of
an activity even if the corresponding schedule is feasible. One way to handle
resources k ∈ R with ρk = 0 is to determine points in time Rtimes k where
there exist an activity that can be in execution and another activity that can
be interrupted, double the resource capacity Rk , introduce a set Vkd of dummy
activities that require exactly Rk units of resource k at each point in time t ∈
T \ Rtimes
k , and use the global cumulative propagator:

cumulative(S, p , rk , Rk ) k ∈ R : (ρk = 1 ∨ Rtimes

k = ∅) (12)
cumulative(S ∪ S d , p ∪ pd , rk ∪ rkd , 2 Rk ) k ∈ R : (ρk = 0 ∧ Rtimes
k
= ∅) (13)

Note that rk is a vector containing the resource requirements on resource k of all

activities i ∈ V and that the vectors S d , pd , and rkd contain start times, absolute
durations, and resource requirements on resource k, respectively, for all j ∈ Vkd .
270 S. Kreter et al.

In addition, some decomposed constraints from (10) and (11) are required to
enforce non-overload of resource k at times Rtimes
k .

bit ↔ Si ≤ t ∧ t < Si + pi i ∈ V r, t ∈ Rtimes
k (14)
k∈R:ρk =0

bit rik Ci (t) ≤ Rk k ∈ R : ρk = 0, t ∈ Rtimes
k (15)
i∈V r

For all k ∈ R with ρk = 0 the set Rtimes

k is defined as follows.

Rtimes
k := {t ∈ T | ∃ i, j ∈ V : rik > 0 ∧ rjk > 0 ∧ min Wi ≤ t < Ei (max Wi )∧
min Wj ≤ t < Ej (max Wj ) ∧ Ci (t)
= Cj (t)}

Model 2cap can be achieved by deleting constraints (10) and (11) from model
timeidx and adding constraints (12)–(15) instead.
Example 3. Regarding the example project from Fig. 2 on page 267, resource 3
is the only resource where Rtimes
k
= ∅. We can see in Fig. 3 on page 268 that
in time period 6 activity 2 is in execution and activity 3 is interrupted. Hence
Rtimes
3 = {5}. The solution presented in Fig. 3 is resource feasible for R3 = 3 but
cumulative does not know that activity 3 is interrupted and detects a resource
overload if resource limit R3 = 3 is used. By doubling the resource capacity and
introducing a set V3d of dummy activities requiring 3 resources in all periods
but 6, the cumulative of (13) does not detect a resource overload. The reason
for the decomposed constraint (15) for time point 5 is clear when we imagine
another activity 2 that requires resource type 3 for its execution and could be
in execution in time period 6 just like activity 2, then for any solution where
both activities 2 and 2 are in execution in time period 6 there is a resource
overload, which the cumulative does not detect when the resource capacity is
doubled. 


3.3 Model addtasks (Adding Split Tasks)

Another way to handle resources k ∈ R with ρk = 0 is to introduce for
each interruptible activity i ∈ V bi a set Addi := {ai1 , ai2 , . . . , ai|Addi | } of addi-
tional (non-interruptible) activities that cover only those points in time t ∈
{Si , . . . , Ei (Si )−1} with Ci (t) = 1, i.e., resource k is released during an interrup-
tion of activity i. For the start times and processing times of activites aij ∈ Addi
the following equalities must be guaranteed.

Sai1 = Si i ∈ V bi (16)
Saij = next starti (Saij−1 ) i ∈ V bi , j ∈ {2, . . . , |Addi |} (17)
j−1

paij = min(next breaki (Saij ), pi − paih ) i ∈ V bi , j ∈ {1, . . . , |Addi |} (18)
h=1
raij ,k = rik i ∈ V bi , j ∈ {1, . . . , |Addi |} (19)
 Modeling and Solving Project Scheduling with Calendars 271

Thereby, next breaki (t) gives the start time of the next break after time t in
calendar Ci and next starti (t) gives the end time of the next break as defined
in Sect. 2. Finally, the resource requirement of each additional activity aij ∈ Addi
is set equal to rik and the global cumulative propagator can be used:
cumulative(S, p , rk , Rk ) k ∈ R : ρk = 1 (20)
cumulative(S a , pa , rka , Rk ) k ∈ R : ρk = 0 (21)
In constraints (21), the vectors S a , pa , and rka contain not only the start times,
durations, and resource requirements of the additional activities aij , i ∈ V bi , j ∈
{1, . . . , |Addi |}, but also the start times, durations, and resource requirements
of the non-interruptible activities i ∈ V ni .
Model addtasks can be achieved by deleting constraints (10) and (11) from
model timeidx as well as adding constraints (16)–(21) instead.

3.4 Model cumucal (Global Calendar Propagator)

For our fourth model for RCPSP/max-cal, we created a global cumulative prop-
agator that takes calendars into account and named it cumulative calendar.
The fourth model (cumucal) can be achieved by deleting constraints (9), (10),
and (11) from model timeidx as well as adding constraints (22)
cumulative calendar(S, p, rk , Rk , C, ρk ) k∈R (22)
with p being the vector of all constant processing times pi and C being the vector
of all activity calendars Ci , i ∈ V .
The cumulative calendar propagator is made up of two parts, a time-table
consistency check and fil