Papers by Ronen I Brafman
Discrete Applied Mathematics, 2014
ABSTRACT We consider scenarios in which a sequence of tests is to be applied to an object; the re... more ABSTRACT We consider scenarios in which a sequence of tests is to be applied to an object; the result of a test may be that a decision (such as the classification of the object) can be made without running additional tests. Thus, one seeks an ordering of the tests that is optimal in some sense, such as minimum expected resource consumption. Sequences of tests are commonly used in computer vision (Paul A. Viola and Michael J. Jones (2001) [15]) and other applications. Finding an optimal ordering is easy when the tests are completely independent. Introducing precedence constraints, we show that the optimization problem becomes NP-hard when the constraints are given by means of a general partial order. Restrictions of the constraints to non-trivial special cases that allow for low-order polynomial-time algorithms are examined.
Proceedings of the Seventeenth …, Jan 1, 2001
We propose a directed graphical representation of utility functions, called UCP-networks, that co... more We propose a directed graphical representation of utility functions, called UCP-networks, that combines aspects of two existing preference models: generalized additive models and CP-networks. The network decomposes a utility function into a number of additive factors, with the directionality of the arcs reflecting conditional dependence in the underlying (qualitative) preference ordering under a ceteris paribus interpretation. The CP-semantics ensures that computing optimization and dominance queries is very efficient. We also demonstrate the value of this representation in decision making. Finally, we describe an interactive elicitation procedure that takes advantage of the linear nature of the constraints on "tradeoff weights" imposed by a UCP-network.
AAAI Spring Symposium on …, Jan 1, 1997
We investigate the solution of constraint-based configuration problems in which the preference fu... more We investigate the solution of constraint-based configuration problems in which the preference function over outcomes is unknown or incompletely specified. The aim is to configure a system, such as a personal computer, so that it will be optimal for a given user. The goal of this project is to develop algorithms that generate the most preferred feasible configuration by posing preference queries to the user. In order to minimize the number and the complexity of preference queries posed to the user, the algorithm reasons about the user's preferences while taking into account constraints over the set of feasible configurations. We assume that the user can structure their preferences in a particular way that, while natural in many settings, can be exploited during the optimization process. We also address in a preliminary fashion the trade-offs between computational effort in the solution of a problem and the degree of interaction with the user.
Computational …, Jan 1, 2004
Many AI tasks, such as product configuration, decision support, and the construction of autonomou... more Many AI tasks, such as product configuration, decision support, and the construction of autonomous agents, involve a process of constrained optimization, that is, optimization of behavior or choices subject to given constraints. In this paper we present an approach for constrained optimization based on a set of hard constraints and a preference ordering represented using a CP-network---a graphical model for representing qualitative preference information. This approach offers both pragmatic and computational advantages. First, it provides a convenient and intuitive tool for specifying the problem, and in particular, the decision maker's preferences. Second, it admits an algorithm for finding the most preferred feasible (Pareto optimal) outcomes that has the following anytime property: the set of preferred feasible outcomes are enumerated without backtracking. In particular, the first feasible solution generated by this algorithm is Pareto optimal.
Proceedings of the 7th ACM conference on Recommender systems - RecSys '13, 2013
ABSTRACT Users often configure complex objects with many possible internal choices. Recommendatio... more ABSTRACT Users often configure complex objects with many possible internal choices. Recommendation engines that automatically configure such objects given user preferences and constraints, may provide much value in such cases. These applications generate appropriate recommendations based on user preferences. It is likely, though, that the user will not be able to fully express her preferences and constraints, requiring a phase of manual tuning of the recommended configuration. We suggest that following this manual revision, additional constraints and preferences can be automatically collected, and the recommended configuration can be automatically improved. Specifically, we suggest a recommender component that takes as input an initial manual configuration of a complex object, deduces certain user preferences and constraints from this configuration, and constructs an alternative configuration. We show an appealing application for our method in complex trip planning, and demonstrate its usability in a user study.
ABSTRACT This paper describes a number of distributed forward search algorithms for solving multi... more ABSTRACT This paper describes a number of distributed forward search algorithms for solving multi-agent planning problems. We introduce a distributed formulation of non-optimal forward search, as well as an optimal version, MAD-A*. Our algorithms exploit the structure of multi-agent problems to not only distribute the work efficiently among different agents, but also to remove symmetries and reduce the overall workload. The algorithms ensure that private information is not shared among agents, yet computation is still efficient -- outperforming current state-of-the-art distributed planners, and in some cases even centralized search -- despite the fact that each agent has access only to partial information.
Lecture Notes in Computer Science, 2005
Learning to act in an unknown partially observable domain is a difficult variant of the reinforce... more Learning to act in an unknown partially observable domain is a difficult variant of the reinforcement learning paradigm. Research in the area has focused on model-free methods -methods that learn a policy without learning a model of the world. When sensor noise increases, model-free methods provide less accurate policies. The model-based approach -learning a POMDP model of the world, and computing an optimal policy for the learned model -may generate superior results in the presence of sensor noise, but learning and solving a model of the environment is a difficult problem. We have previously shown how such a model can be obtained from the learned policy of model-free methods, but this approach implies a distinction between a learning phase and an acting phase that is undesirable. In this paper we present a novel method for learning a POMDP model online, based on McCallums' Utile Suffix Memory (USM), in conjunction with an approximate policy obtained using an incremental POMDP solver. We show that the incrementally improving policy provides superior results to the original USM algorithm, especially in the presence of increasing sensor and action noise.
2008 IEEE 25th Convention of Electrical and Electronics Engineers in Israel, 2008
Workshop on Rich …, 2005
Agents learning to act in a partially observable domain may need to overcome the problem of noisy... more Agents learning to act in a partially observable domain may need to overcome the problem of noisy output from the agent's sensors. Research in the area has focused on model-free methods—methods that learn a policy without learning a model of the world. ...
The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2005
Computing optimal or approximate policies for partially observable Markov decision processes (POM... more Computing optimal or approximate policies for partially observable Markov decision processes (POMDPs) is a difficult task. When in addition the characteristics of the environment change over time, the problem is further compounded. A policy that was computed offline may stop being useful after sufficient changes to the environment have occurred. We present an online algorithm for incrementally improving POMDP policies, that is highly motivated by the Heuristic Search Value Iteration (HSVI) approach -locally improving the current value function after every action execution. Our algorithm adapts naturally to slow changes in the environment, without the need to explicitly model the changes. In initial empirical evaluation our algorithm shows a marked improvement over other online POMDP algorithms.
IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2000
Recent scaling up of POMDP solvers towards realistic applications is largely due to point-based m... more Recent scaling up of POMDP solvers towards realistic applications is largely due to point-based methods that quickly converge to an approximate solution for medium-sized problems. These algorithms compute a value function for a finite reachable set of belief points, using backup operations. Point based algorithms differ on the selection of the set of belief points, and on the order by which backup operations are executed on the selected belief points.
International Joint Conference on Artificial Intelligence, 2001
We present a new approach for personalized pre- sentation of web-page content. This approach is b... more We present a new approach for personalized pre- sentation of web-page content. This approach is based on preference-based constrained opti- mization techniques rooted in qualitative decision- theory. In our approach, web-page personalization is viewed as a configuration problem whose goal is to determine the optimal presentation of a web- page while taking into account the preferences of the web author,
International Joint Conference on Artificial Intelligence, 2007
Recent scaling up of POMDP solvers towards re- alistic applications is largely due to point-based... more Recent scaling up of POMDP solvers towards re- alistic applications is largely due to point-based methods which quickly converge to an approximate solution for medium-sized problems. Of this family HSVI, which uses trial-based asynchronous value iteration, can handle the largest domains. In this paper we suggest a new algorithm, FSVI, that uses the underlying MDP to traverse the belief space
PROCEEDINGS OF THE …, 2007
We present here a point-based value iteration algorithm for solving POMDPs, that orders belief st... more We present here a point-based value iteration algorithm for solving POMDPs, that orders belief state backups smartly based on a clustering of the underlying MDP states. We show our SCVI algorithm to converge faster than state of the art point-based algorithms.
Uploads
Papers by Ronen I Brafman