Submodular Optimization

We consider an online decision problem over a discrete space in which the loss function is submodular. We give algorithms which are computationally efficient and are Hannan-consistent in both the full information and partial feedback settings.

Relevance, diversity and personalization are key issues when presenting content which is apt to pique a user's interest. This is particularly true when presenting an engaging set of news stories. In this paper we propose an efficient algorithm for selecting a small subset of relevant articles from a streaming news corpus. It offers three key pieces of improvement over past work: 1) It is based on a detailed model of a user's viewing behavior which does not require explicit feedback. 2) We use the notion of submodularity to estimate the propensity of interacting with content. This improves over the classical context independent relevance ranking algorithms. Unlike existing methods, we learn the submodular function from the data. 3) We present an efficient online algorithm which can be adapted for personalization, story adaptation, and factorization models. Experiments show that our system yields a significant improvement over a retrieval system deployed in production.

Learning to rank is an important problem in machine learning and recommender systems. In a recommender system, a user is typically recommended a list of items. Since the user is unlikely to examine the entire recommended list, partial feedback arises naturally. At the same time, diverse recommendations are important because it is challenging to model all tastes of the user in practice. In this paper, we propose the first algorithm for online learning to rank diverse items from partial-click feedback. We assume that the user examines the list of recommended items until the user is attracted by an item, which is clicked, and does not examine the rest of the items. This model of user behavior is known as the cascade model. We propose an online learning algorithm, cascadelsb, for solving our problem. The algorithm actively explores the tastes of the user with the objective of learning to recommend the optimal diverse list. We analyze the algorithm and prove a gap-free upper bound on its...

We consider the adaptive influence maximization problem: given a network and a budget k, iteratively select k seeds in the network to maximize the expected number of adopters. In the full-adoption feedback model, after selecting each seed, the seed-picker observes all the resulting adoptions. In the myopic feedback model, the seed-picker only observes whether each neighbor of the chosen seed adopts. Motivated by the extreme success of greedy-based algorithms/heuristics for influence maximization, we propose the concept of greedy adaptivity gap, which compares the performance of the adaptive greedy algorithm to its non-adaptive counterpart. Our first result shows that, for submodular influence maximization, the adaptive greedy algorithm can perform up to a (1 − 1/e)-fraction worse than the non-adaptive greedy algorithm, and that this ratio is tight. More specifically, on one side we provide examples where the performance of the adaptive greedy algorithm is only a (1 − 1/e) fraction of the performance of the non-adaptive greedy algorithm in four settings: for both feedback models and both the independent cascade model and the linear threshold model. On the other side, we prove that in any submodular cascade, the adaptive greedy algorithm always outputs a (1 − 1/e)-approximation to the expected number of adoptions in the optimal non-adaptive seed choice. Our second result shows that, for the general submodular cascade model with full-adoption feedback, the adaptive greedy algorithm can outperform the non-adaptive greedy algorithm by an unbounded factor. Finally, we propose a risk-free variant of the adaptive greedy algorithm that always performs no worse than the nonadaptive greedy algorithm.

We consider stochastic influence maximization problems arising in social networks. In contrast to existing studies that involve greedy approximation algorithms with a 63% performance guarantee, our work focuses on solving the problem optimally. To this end, we introduce a new class of problems that we refer to as two-stage stochastic submodular optimization models. We propose a delayed constraint generation algorithm to find the optimal solution to this class of problems with a finite number of samples. The influence maximization problems of interest are special cases of this general problem class. We show that the submodularity of the influence function can be exploited to develop strong optimality cuts that are more effective than the standard optimality cuts available in the literature. Finally, we report our computational experiments with large-scale real-world datasets for two fundamental influence maximization problems, independent cascade and linear threshold, and show that our proposed algorithm outperforms the greedy algorithm.

Submodularity is an important concept in integer and combinatorial optimization. A classical submodular set function models the utility of selecting homogenous items from a single ground set, and such selections can be represented by binary variables. In practice, many problem contexts involve choosing heterogenous items from more than one ground set or selecting multiple copies of homogenous items, which call for extensions of submodularity. We refer to the optimization problems associated with such generalized notions of submodularity as Generalized Submodular Optimization (GSO). GSO is found in wide-ranging applications, including infrastructure design, healthcare, online marketing, and machine learning. Due to the often highly nonlinear (even non-convex and non-concave) objective function and the mixed-integer decision space, GSO is a broad subclass of challenging mixed-integer nonlinear programming problems. In this tutorial, we first provide an overview of classical submodularity. Then we introduce two subclasses of GSO, for which we present polyhedral theory for the mixed-integer set structures that arise from these problem classes. Our theoretical results lead to efficient and versatile exact solution methods that demonstrate their effectiveness in practical problems using real-world datasets.

Many sensing applications require monitoring phenomena with complex spatio-temporal dynamics spread over large spatial domains. Efficient monitoring of such phenomena would require an impractically large number of static sensors; therefore, actuated sensing-mobile robots carrying sensors-is required. Path planning for these robots, i.e., deciding on a subset of locations to observe, is critical for high fidelity monitoring of expansive areas with complex dynamics. We propose MUST-a MUltiscale approach with STochastic modeling. MUST is a hierarchical approach that models the phenomena as a stochastic Gaussian Process that is exploited to select a near-optimal subset of observation locations. We discuss in detail our proposed algorithm for the application of monitoring light intensity in a forest understory. We performed extensive empirical evaluations both in simulation using field data and on an actual cabled robotic system to validate the effectiveness of our proposed algorithm.

Bayesian inference tasks continue to pose a computational challenge. This especially holds for spatial-temporal modeling where high-dimensional latent parameter spaces are ubiquitous. The methodology of integrated nested Laplace approximations (INLA) provides a framework for performing Bayesian inference applicable to a large subclass of additive Bayesian hierarchical models. In combination with the stochastic partial differential equations (SPDE) approach it gives rise to an efficient method for spatial-temporal modeling. In this work we build on the INLA-SPDE approach, by putting forward a performant distributed memory variant, INLA DIST , for large-scale applications. To perform the arising computational kernel operations, consisting of Cholesky factorizations, solving linear systems, and selected matrix inversions, we present two numerical solver options, a sparse CPU-based library and a novel blocked GPU-accelerated approach which we propose. We leverage the recurring nonzero block structure in the arising precision (inverse covariance) matrices, which allows us to employ dense subroutines within a sparse setting. Both versions of INLA DIST are highly scalable, capable of performing inference on models with millions of latent parameters. We demonstrate their accuracy and performance on synthetic as well as real-world climate dataset applications.

Recent years have witnessed the emergence of shared sensor networks as integrated infrastructure for multiple applications. It is important to allocate multiple applications in a shared sensor network, in order to maximize the overall Quality of Monitoring (QoM) subject to resource constraints (e.g., in terms of memory and network bandwidth). The resulting constrained optimization problem is a difficult and open problem since it is discrete, nonlinear, and not in closed-form. This paper makes several important contributions towards optimal multi-application allocation in shared sensor networks. (1) We formulate the optimal application allocation problem for a common class of distributed sensing applications whose QoM can be modeled as variance reduction functions. (2) We prove key theoretical properties of the optimization problem, including the monotonicity and submodularity of the variance reduction functions and the multiple knapsack structure of constraints; (3) By exploiting these properties, we propose a local search algorithm, which is efficient and has a good approximation bound, for application allocation in shared sensor networks. Simulations based on both real-world datasets and randomly generated networks demonstrate that our algorithm is competitive against simulated annealing in term of QoM, with up to three orders of magnitude reduction in execution times, making it a practical solution towards multi-application allocation in shared sensor networks.

This paper addresses an ill-posed problem of recovering a color image from its compressively sensed measurement data. Differently from the typical 1D vector-based approach of the state-of-the-art methods, we exploit the nonlocal similarities inherently existing in images by treating each patch of a color image as a 3D tensor consisting of not only horizontal and vertical but also spectral dimensions. A group of nonlocal similar patches form a 4D tensor for which a nonlocal higher order dictionary is learned via higher order singular value decomposition. The multiple sub-dictionaries contained in the higher order dictionary decorrelate the group in each corresponding dimension, thus help the detail of color images to be reconstructed better. Furthermore, we promote sparsity of the final solution using a sparsity regularization based on a weight tensor. It can distinguish those coefficients of the sparse representation generated by the higher order dictionary which are expected to have large magnitude from the others in the optimization. Accordingly, in the iterative solution, it acts like a weighting process which is designed by approximating the minimum mean squared error filter for more faithful recovery. Experimental results confirm improvement by the proposed method over the state-of-the-art ones.

The budget allocation problem is an optimization problem arising from advertising planning. In the problem, an advertiser has limited budgets to allocate across media, and seeks to optimize the allocation such that the largest fraction of customers can be influenced. It is known that this problem admits a (1-1/e)-approximation algorithm. However, no previous studies on this problem considered adjusting the allocation adaptively based upon the effect of the past campaigns, which is a usual strategy in the real setting. Our main contribution in this paper is to analyze adaptive strategies for the budget allocation problem. We define a greedy strategy, referred to as the insensitive policy, and then give a provable performance guarantee. This result is obtained by extending the adaptive submodularity, which is a concept studied in the context of active learning and stochastic optimization, to the functions over an integer lattice.

We review recent results obtained by the authors on the approximability of a family of combinatorial problems arising in optimal experimental design. We first recall a result based on submodularity, which states that the greedy approach always gives a design within 1 − 1/e of the optimal solution. Then, we present a new result on the design found by rounding the solution of the continuous relaxed problem, an approach which has been applied by several authors: When the goal is to select n out of s experiments, the D−optimal design may be rounded to a design for which the dimension of the observable subspace is within n s of the optimum.

When monitoring spatial phenomena, such as the ecological condition of a river, deciding where to make observations is a challenging task. In these settings, a fundamental question is when an active learning, or sequential design, strategy, where locations are selected based on previous measurements, will perform significantly better than sensing at an a priori specified set of locations. For Gaussian Processes (GPs), which often accurately model spatial phenomena, we present an analysis and efficient algorithms that address this question. Central to our analysis is a theoretical bound which quantifies the performance difference between active and a priori design strategies. We consider GPs with unknown kernel parameters and present a nonmyopic approach for trading off exploration, i.e., decreasing uncertainty about the model parameters, and exploitation, i.e., near-optimally selecting observations when the parameters are (approximately) known. We discuss several exploration strategies, and present logarithmic sample complexity bounds for the exploration phase. We then extend our algorithm to handle nonstationary GPs exploiting local structure in the model. A variational approach allows us to perform efficient inference in this class of nonstationary models. We also present extensive empirical evaluation on several real-world problems.

A real-valued set function is (additively) approximately submodular if it satisfies the submodularity conditions with an additive error. Approximate submodularity arises in many settings, especially in machine learning, where the function evaluation might not be exact. In this paper we study how close such approximately submodular functions are to truly submodular functions. We show that an approximately submodular function defined on a ground set of n elements is O(n 2) pointwise-close to a submodular function. This result also provides an algorithmic tool that can be used to adapt existing submodular optimization algorithms to approximately submodular functions. To complement, we show an Ω(√ n) lower bound on the distance to submodularity. These results stand in contrast to the case of approximate modularity, where the distance to modularity is a constant, and approximate convexity, where the distance to convexity is logarithmic.

Robots can be used to collect environmental data in regions that are difficult for humans to traverse. However, limitations remain in the size of region that a robot can directly observe per unit time. We introduce a method for selecting a limited number of observation points in a large region, from which we can predict the state of unobserved points in the region. We combine a low rank model of a target attribute with an information-maximizing path planner to predict the state of the attribute throughout a region. Our approach is agnostic to the choice of target attribute and robot monitoring platform. We evaluate our method in simulation on two real-world environment datasets, each containing observations from one to two million possible sampling locations. We compare against a random sampler and four variations of a baseline sampler from the ecology literature. Our method outperforms the baselines in terms of average Fisher information gain per samples taken and performs comparably for average reconstruction error in most trials.

Billboard Advertisement has emerged as an effective out-ofhome advertisement technique and adopted by many commercial houses. In this case, the billboards are owned by some companies and they are provided to the commercial houses slot-wise on a payment basis. Now, given the database of billboards along with their slot information which k slots should be chosen to maximize the influence. Formally, we call this problem as the Influential Billboard Slot Selection Problem. In this paper, we pose this problem as a combinatorial optimization problem. Under the 'triggering model of influence', the influence function is non-negative, monotone, and submodular. However, as the incremental greedy approach for submodular function maximization does not scale well along with the size of the problem instances, there is a need to develop efficient solution methodologies for this problem. In this paper, we apply the pruned submodularity graph-based pruning technique for solving this problem. The proposed approach is divided into three phases, namely, preprocessing, pruning, and selection. We analyze the proposed solution approach for its performance guarantee and computational complexity. We conduct an extensive set of experiments with real-world datasets and compare the performance of the proposed solution approach with many baseline methods. We observe that the proposed one leads to more amount of influence compared to all the baseline methods within reasonable computational time.

We study the problem of maximizing a nonmonotone submodular function subject to a cardinality constraint in the streaming model. Our main contribution is a single-pass (semi) streaming algorithm that uses roughly [Formula: see text] memory, where k is the size constraint. At the end of the stream, our algorithm postprocesses its data structure using any off-line algorithm for submodular maximization and obtains a solution whose approximation guarantee is [Formula: see text], where α is the approximation of the off-line algorithm. If we use an exact (exponential time) postprocessing algorithm, this leads to [Formula: see text] approximation (which is nearly optimal). If we postprocess with the state-of-the-art offline approximation algorithm, whose guarantee is [Formula: see text], we obtain a 0.2779-approximation in polynomial time, improving over the previously best polynomial-time approximation of 0.1715. It is also worth mentioning that our algorithm is combinatorial and determin...

A wide variety of problems in machine learning, including exemplar clustering, document summarization, and sensor placement, can be cast as constrained submodular maximization problems. Unfortunately, the resulting submodular optimization problems are often too large to be solved on a single machine. We consider a distributed, greedy algorithm that combines previous approaches with randomization. The result is an algorithm that is embarrassingly parallel and achieves provable, constant factor, worstcase approximation guarantees. In our experiments, we demonstrate its efficiency in large problems with different kinds of constraints with objective values always close to what is achievable in the centralized setting.

A wide variety of problems in machine learning, including exemplar clustering, document summarization, and sensor placement, can be cast as constrained submodular maximization problems. A lot of recent effort has been devoted to developing distributed algorithms for these problems. However, these results suffer from high number of rounds, suboptimal approximation ratios, or both. We develop a framework for bringing existing algorithms in the sequential setting to the distributed setting, achieving near optimal approximation ratios for many settings in only a constant number of MapReduce rounds. Our techniques also give a fast sequential algorithm for non-monotone maximization subject to a matroid constraint.

Submodular optimization generalizes many classic problems in combinatorial optimization and has recently found a wide range of applications in machine learning (e.g., feature engineering and active learning). For many large-scale optimization problems, we are often concerned with the adaptivity complexity of an algorithm, which quantifies the number of sequential rounds where polynomially-many independent function evaluations can be executed in parallel. While low adaptivity is ideal, it is not sufficient for a distributed algorithm to be efficient, since in many practical applications of submodular optimization the number of function evaluations becomes prohibitively expensive. Motivated by these applications, we study the adaptivity and query complexity of adaptive submodular optimization. Our main result is a distributed algorithm for maximizing a monotone submodular function with cardinality constraint k that achieves a (1 − 1/e − ε)approximation in expectation. This algorithm runs in O(log(n)) adaptive rounds and makes O(n) calls to the function evaluation oracle in expectation. The approximation guarantee and query complexity are optimal, and the adaptivity is nearly optimal. Moreover, the number of queries is substantially less than in previous works. We also extend our results to the submodular cover problem to demonstrate the generality of our algorithm and techniques.

Consider the problem of choosing a set of actions to optimize an objective function that is a real-valued polymatroid function subject to matroid constraints. The greedy strategy provides an approximate solution to the optimization problem, and it is known to satisfy some performance bounds in terms of the total * Corresponding author.

We generalize the monotone local search approach of Fomin, Gaspers, Lokshtanov and Saurabh [J.ACM 2019], by establishing a connection between parameterized approximation and exponentialtime approximation algorithms for monotone subset minimization problems. In a monotone subset minimization problem the input implicitly describes a non-empty set family over a universe of size n which is closed under taking supersets. The task is to find a minimum cardinality set in this family. Broadly speaking, we use approximate monotone local search to show that a parameterized α-approximation algorithm that runs in c k • n O(1) time, where k is the solution size, can be used to derive an α-approximation randomized algorithm that runs in d n • n O(1) time, where d is the unique value in d ∈ 1, 1 + c−1 α such that D 1 α d−1 c−1 = ln c α and D (a b) is the Kullback-Leibler divergence. This running time matches that of Fomin et al. for α = 1, and is strictly better when α > 1, for any c > 1. Furthermore, we also show that this result can be derandomized at the expense of a sub-exponential multiplicative factor in the running time. We use an approximate variant of the exhaustive search as a benchmark for our algorithm. We show that the classic 2 n • n O(1) exhaustive search can be adapted to an α-approximate exhaustive search that runs in time 1 + exp −α • H 1 α n • n O(1) , where H is the entropy function. Further

Billboard Advertisement has emerged as an effective out-ofhome advertisement technique and adopted by many commercial houses. In this case, the billboards are owned by some companies and they are provided to the commercial houses slot-wise on a payment basis. Now, given the database of billboards along with their slot information which k slots should be chosen to maximize the influence. Formally, we call this problem as the Influential Billboard Slot Selection Problem. In this paper, we pose this problem as a combinatorial optimization problem. Under the 'triggering model of influence', the influence function is non-negative, monotone, and submodular. However, as the incremental greedy approach for submodular function maximization does not scale well along with the size of the problem instances, there is a need to develop efficient solution methodologies for this problem. In this paper, we apply the pruned submodularity graph-based pruning technique for solving this problem. The proposed approach is divided into three phases, namely, preprocessing, pruning, and selection. We analyze the proposed solution approach for its performance guarantee and computational complexity. We conduct an extensive set of experiments with real-world datasets and compare the performance of the proposed solution approach with many baseline methods. We observe that the proposed one leads to more amount of influence compared to all the baseline methods within reasonable computational time.

A product warranty is an agreement offered by a producer to a consumer to replace or repair a faulty item, or to partially or fully reimburse the consumer in the event of a failure. Warranties are very widespread and serve many purposes, including protection for producer, seller, and consumer. They are used as signals of quality and as elements of marketing strategies. In this study we review the notion of an online convex optimization algorithm and its variations, and apply it in warranty context. We introduce a class of profit functions, which are functions of warranty, and use it to formulate the problem of maximizing the company's profit over time as an online convex optimization problem. We use this formulation to present an approach to setting the warranty based on an online algorithm with low regret. Under a dynamic environment, this algorithm provides a warranty strategy for the company that maximises its profit over time.

We study the minimum connected sensor cover problem (MIN-CSC) and the budgeted connected sensor cover (Budgeted-CSC) problem, both motivated by important applications (e.g., reduce the communication cost among sensors) in wireless sensor networks. In both problems, we are given a set of sensors and a set of target points in the Euclidean plane. In MIN-CSC, our goal is to find a set of sensors of minimum cardinality, such that all target points are covered, and all sensors can communicate with each other (i.e., the communication graph is connected). We obtain a constant factor approximation algorithm, assuming that the ratio between the sensor radius and communication radius is bounded. In Budgeted-CSC problem, our goal is to choose a set of B sensors, such that the number of targets covered by the chosen sensors is maximized and the communication graph is connected. We also obtain a constant approximation under the same assumption.

This paper proposes a new attention-path planning algorithm that allows robots with limited sensing coverage to identify an unknown entity efficiently. Our focus is placed on how to plan optimal sequences of views to access more useful information needed to understand the entity. The adaptive submodular optimization technique guaranteed to achieve nearoptimal performance is used to maximize the expected information gain. We verified the validity of the proposed approach to the face recognition problem through preliminary experiments.

We consider persistent monitoring of a finite number of interconnected geographical nodes by a group of heterogeneous mobile agents. We assign to each geographical node a concave and increasing reward function that resets to zero after an agent's visit. Then, we design the optimal dispatch policy of which nodes to visit at what time and by what agent by finding a policy set that maximizes a utility that is defined as the total reward collected at visit times. We show that this optimization problem is NP-hard and its computational complexity increases exponentially with the number of the agents and the length of the mission horizon. By showing that the utility function is a monotone increasing and submodular set function of agents' policy, we propose a suboptimal dispatch policy design with a known optimality gap. To reduce the time complexity of constructing the feasible search set and also to induce robustness to changes in the operational factors, we perform our suboptimal policy design in a receding horizon fashion. Then, to compensate for the shortsightedness of the receding horizon approach we add a new term to our utility, which provides a measure of nodal importance beyond the receding horizon. This term gives the policy design an intuition to steer the agents towards the nodes with higher rewards on the patrolling graph. Finally, we discuss how our proposed algorithm can be implemented in a decentralized manner. A simulation study demonstrates our results.

In the adaptive influence maximization problem, we are given a social network and a budget $k$, and we iteratively select $k$ nodes, called seeds, in order to maximize the expected number of nodes that are reached by an influence cascade that they generate according to a stochastic model for influence diffusion. Differently from the non-adaptive influence maximization problem, where all the seeds must be selected beforehand, here nodes are selected sequentially one by one, and the decision on the $i$th seed is based on the observed cascade produced by the first $i-1$ seeds. We focus on the myopic feedback model, in which we can only observe which neighbors of previously selected seeds have been influenced and on the independent cascade model, where each edge is associated with an independent probability of diffusing influence. Previous works showed that the adaptivity gap is at most $4$, which implies that the non-adaptive greedy algorithm guarantees an approximation factor of $\fra...

Relay nodes are necessary to maintain scalability and increase longevity as the number of manufacturing industrial sensors grows. In a fixed-budget circumstance, however, the cost of purchasing the bare minimum of relay nodes to connect the network may exceed the budget. Although it is hard to establish a network that connects all sensor nodes, in this case, a network with a high level of connection is still desirable. This paper proposes two metrics for determining the connectedness of a disconnected graph of sensor nodes and determining the optimum deployment method for relay nodes in a network with the highest connectedness while staying within a budget restriction. The metrics are the number of connected graph components and the size of the most significant connected graph component. Prim's algorithm and the approximation minimum spanning tree algorithm are applied to construct a disconnected graph and discover the best relay node placement to solve these two criteria. Compared to the other metrics, simulation findings suggest that prioritizing the most significant connected components in the disconnected graph can yield superior outcomes by deploying the fewest number of relay nodes while retaining the connectedness of the graph.

We study the problem of maximizing a stochastic monotone submodular function with respect to a matroid constraint. Because of the presence of diminishing marginal values in real-world problems, our model can capture the effect of stochasticity in a wide range of applications. We show that the adaptivity gap—the ratio between the values of optimal adaptive and optimal nonadaptive policies—is bounded and is equal to e/(e − 1). We propose a polynomial-time nonadaptive policy that achieves this bound. We also present an adaptive myopic policy that obtains at least half of the optimal value. Furthermore, when the matroid is uniform, the myopic policy achieves the optimal approximation ratio of 1 − 1/e. This paper was accepted by Dimitris Bertsimas and Yinyu Ye, optimization.

We consider the problem of sparse atomic optimization, where the notion of "sparsity" is generalized to meaning some linear combination of few atoms. The definition of atomic set is very broad; popular examples include the standard basis, low-rank matrices, overcomplete dictionaries, permutation matrices, orthogonal matrices, etc. The model of sparse atomic optimization therefore includes problems coming from many fields, including statistics, signal processing, machine learning, computer vision and so on. Specifically, we consider the problem of maximizing a restricted strongly convex (or concave), smooth function restricted to a sparse linear combination of atoms. We extend recent work that establish linear convergence rates of greedy algorithms on restricted strongly concave, smooth functions on sparse vectors to the realm of general atomic sets, where the convergence rate involves a novel quantity: the "sparse atomic condition number". This leads to the stron...

Classes of set functions along with a choice of ground set are a bedrock to determine and develop corresponding variants of greedy algorithms to obtain efficient solutions for combinatorial optimization problems. The class of approximate constrained submodular optimization has seen huge advances at the intersection of good computational efficiency, versatility and approximation guarantees while exact solutions for unconstrained submodular optimization are NP-hard. What is an alternative to situations when submodularity does not hold? Can efficient and globally exact solutions be obtained? We introduce one such new frontier: The class of quasi-concave set functions induced as a dual class to monotone linkage functions. We provide a parallel algorithm with a time complexity over n processors of O(ng) + O(log log n) where n is the cardinality of the ground set and g is the complexity to compute the monotone linkage function that induces a corresponding quasi-concave set function via a ...

Our paper newly presents unsupervised feature representation method for very low-resolution (VLR) images called informative census transform (ICT) based on statistical analysis of CT binary features and submodular optimization. A new cost function is designed to measure the informativeness of each binary feature: (1) an individual informativeness of features to represent unlabeled image dataset and (2) relative informativeness between binary features to represent different binary features. Therefore, we considered informativeness of binary feature according to two relationship (1) between feature space and image space, and (2) between different features within same feature space. Moreover, two constraints are designed by considering sub-modular characteristics to guarantee theoretical performance and fast optimization via simple greedy algorithm. Experimental results show that the proposed ICT features with two constraints outperforms the traditional CT features in terms of recognition performance and computational cost at VLR problem.

and Computer Engineering Deployment of low power basestations within cellular networks can potentially increase both capacity and coverage. However, such deployments require efficient resource allocation schemes for managing interference from the low power and macro basestations that are located within each other's transmission range. In this dissertation, we propose novel and efficient dynamic resource allocation algorithms in the frequency, time and space domains. We show that the proposed algorithms perform better than the current state-of-art resource management algorithms. In the first part of the dissertation, we propose an interference management solution in the frequency domain. We introduce a distributed frequency allocation scheme that shares frequencies between macro and low power pico basestations, and guarantees a minimum average throughput to users. The scheme seeks to minimize the total number of frequencies needed to honor the minimum throughput requirements. We evaluate our scheme using detailed simulations and show that it performs on par with the centralized optimum allocation. Moreover, our proposed

The need for efficient monitoring of spatio-temporal dynamics in large environmental applications, such as the water quality monitoring in rivers and lakes, motivates the use of robotic sensors in order to achieve sufficient spatial coverage. Typically, these robots have bounded resources, such as limited battery or limited amounts of time to obtain measurements. Thus, careful coordination of their paths is required in order to maximize the amount of information collected, while respecting the resource constraints. In this paper, we present an efficient approach for near-optimally solving the NP-hard optimization problem of planning such informative paths. In particular, we first develop eSIP (efficient Single-robot Informative Path planning), an approximation algorithm for optimizing the path of a single robot. Hereby, we use a Gaussian Process to model the underlying phenomenon, and use the mutual information between the visited locations and remainder of the space to quantify the...

When monitoring spatial phenomena, such as the ecological condition of a river, deciding where to make observations is a challenging task. In these settings, a fundamental question is when an active learning, or sequential design, strategy, where locations are selected based on previous measurements, will perform significantly better than sensing at an a priori specified set of locations. For Gaussian Processes (GPs), which often accurately model spatial phenomena, we present an analysis and efficient algorithms that address this question. Central to our analysis is a theoretical bound which quantifies the performance difference between active and a priori design strategies. We consider GPs with unknown kernel parameters and present a nonmyopic approach for trading off exploration, i.e., decreasing uncertainty about the model parameters, and exploitation, i.e., near-optimally selecting observations when the parameters are (approximately) known. We discuss several exploration strategies, and present logarithmic sample complexity bounds for the exploration phase. We then extend our algorithm to handle nonstationary GPs exploiting local structure in the model. A variational approach allows us to perform efficient inference in this class of nonstationary models. We also present extensive empirical evaluation on several real-world problems.

Adaptive submodular optimization, where a sequence of items is selected adaptively to optimize a submodular function, has been found to have many applications from sensor placement to active learning. In the current paper, we extend this work to the setting of multiple queries at each time step, where the set of available queries is randomly constrained. A primary contribution of this paper is to prove the first near optimal approximation bound for a greedy policy in this setting. A natural application of this framework is to crowd-sourced active learning problem where the set of available experts and examples might vary randomly. We instantiate the new framework for multi-label learning and evaluate it in multiple benchmark domains with promising results.

Effective placement of emergency response vehicles (such as ambulances, fire trucks, police cars) to deal with medical, fire or criminal activities can reduce the incident response time by few seconds, which in turn can potentially save a human life. Owing to its adoption in Emergency Medical Services (EMSs) worldwide, existing research on improving emergency response has focused on optimizing the objective of bounded time (i.e. number of incidents served in a fixed time). Due to the dependence of this objective on temporal uncertainty, optimizing the bounded time objective is challenging. In this paper, we propose a new objective referred to as the bounded rank (which is the number of incidents served by a base station whose rank is below a bounded rank value) that has nice theoretical properties and serves as an indirect substitute for the bounded time objective. To understand the theoretical properties of this new objective in the context of the spatio-temporal uncertainty associ...

We consider optimal coverage problems for a multi-agent network aiming to maximize a joint event detection probability in an environment with obstacles. The objective function of this problem is nonconcave and no global optimum is guaranteed by gradient-based algorithms developed to date. In order to obtain a solution provably close to the global optimum, the selection of initial conditions is crucial. We first formulate the initial agent location generation as an additional optimization problem where the objective function is monotone submodular, a class of functions for which the performance obtained through a greedy algorithm solution is guaranteed to be within a provable bound relative to the optimal performance. We then derive two tighter bounds by exploiting the curvature information (total curvature and elemental curvature) of the objective function. We further show that the tightness of these lower bounds is complementary with respect to the sensing capabilities of the agents. The greedy algorithm solution can be subsequently used as an initial point of a gradient-based algorithm for the original optimal coverage problem. Simulation results are included to verify that this approach leads to significantly better performance relative to previously used algorithms.

We introduce submodular optimization to the problem of training data subset selection for statistical machine translation (SMT). By explicitly formulating data selection as a submodular program, we obtain fast scalable selection algorithms with mathematical performance guarantees, resulting in a unified framework that clarifies existing approaches and also makes both new and many previous approaches easily accessible. We present a new class of submodular functions designed specifically for SMT and evaluate them on two different translation tasks. Our results show that our best submodular method significantly outperforms several baseline methods, including the widely-used cross-entropy based data selection method. In addition, our approach easily scales to large data sets and is applicable to other data selection problems in natural language processing.

This paper presents a polynomial-time 1/2-approximation algorithm for maximizing nonnegative k-submodular functions. This improves upon the previous max{1/3, 1/(1+a)}-approximation by Ward andŽivný [15], where a = max{1, (k − 1)/4}. We also show that for monotone ksubmodular functions there is a polynomial-time k/(2k − 1)-approximation algorithm while for any ε > 0 a ((k + 1)/2k + ε)-approximation algorithm for maximizing monotone k-submodular functions would require exponentially many queries. In particular, our hardness result implies that our algorithms are asymptotically tight. We also extend the approach to provide constant factor approximation algorithms for maximizing skew-bisubmodular functions, which were recently introduced as generalizations of bisubmodular functions.

It is generally believed that submodular functions-and the more general class of γ-weakly submodular functions-may only be optimized under the nonnegativity assumption f (S) ≥ 0. In this paper, we show that once the function is expressed as the difference f = g − c, where g is monotone, non-negative, and γ-weakly submodular and c is non-negative modular, then strong approximation guarantees may be obtained. We present an algorithm for maximizing g − c under a k-cardinality constraint which produces a random feasible set S such that E [g(S)−c(S)] ≥ (1 − e −γ −)g(OPT)−c(OPT), whose running time is O(n log 2 1), i.e., independent of k. We extend these results to the unconstrained setting by describing an algorithm with the same approximation guarantees and faster O(n log 1) runtime. The main techniques underlying our algorithms are twofold: the use of a surrogate objective which varies the relative importance between g and c throughout the algorithm, and a geometric sweep over possible γ values. Our algorithmic guarantees are complemented by a hardness result showing that no polynomial-time algorithm which accesses g through a value oracle can do better. We empirically demonstrate the success of our algorithms by applying them to experimental design on the Boston Housing dataset and directed vertex cover on the Email EU dataset.

The paper addresses the problem of efficiently deploying sensors in spatial environments, e.g. buildings, for the purposes of monitoring spatio-temporal environmental phenomena. By modelling the environmental fields using spatio-temporal Gaussian processes, a new and efficient optimality-cost function of minimizing prediction uncertainties is proposed to find the best sensor locations. Though the environmental processes spatially and temporally vary, the proposed approach of choosing sensor positions is proven not to be affected by time variations, which significantly reduces computational complexity of the optimization problem. The sensor deployment optimization problem is then solved by a practical and feasible polynomial algorithm, where its solutions are theoretically proven to be guaranteed. The proposed method is also theoretically and experimentally compared with the existing works. The effectiveness of the proposed algorithm is demonstrated by implementation in a real tested space in a university building, where the obtained results are highly promising.

In this paper, we study fundamental problems of maximizing DR-submodular continuous functions that have real-world applications in the domain of machine learning, economics, operations research and communication systems. It captures a subclass of non-convex optimization that provides both theoretical and practical guarantees. Here, we focus on minimizing regret for online arriving non-monotone DR-submodular functions over different types of convex sets: hypercube, down-closed and general convex sets. First, we present an online algorithm that achieves a 1/e-approximation ratio with the regret of O(T^2/3) for maximizing DR-submodular functions over any down-closed convex set. Note that, the approximation ratio of 1/e matches the best-known guarantee for the offline version of the problem. Moreover, when the convex set is the hypercube, we propose a tight 1/2-approximation algorithm with regret bound of O(√(T)). Next, we give an online algorithm that achieves an approximation guarante...

In this paper, we study fundamental problems of maximizing DR-submodular continuous functions that have real-world applications in the domain of machine learning, economics, operations research and communication systems. It captures a subclass of non-convex optimization that provides both theoretical and practical guarantees. Here, we focus on minimizing regret for online arriving non-monotone DR-submodular functions over different types of convex sets: hypercube, down-closed and general convex sets. First, we present an online algorithm that achieves a $1/e$-approximation ratio with the regret of $O(T^{2/3})$ for maximizing DR-submodular functions over any down-closed convex set. Note that, the approximation ratio of $1/e$ matches the best-known guarantee for the offline version of the problem. Moreover, when the convex set is the hypercube, we propose a tight 1/2-approximation algorithm with regret bound of $O(\sqrt{T})$. Next, we give an online algorithm that achieves an approxim...

In the Steiner Network problem we are given a graph G with edge-costs and connectivity requirements ruv between node pairs u, v. The goal is to find a minimum-cost subgraph H of G that contains ruv edge-disjoint paths for all u, v ∈ V . In Prize-Collecting Steiner Network problems we do not need to satisfy all requirements, but are given a penalty function for violating the connectivity requirements, and the goal is to find a subgraph H that minimizes the cost plus the penalty. The case when ruv ∈ {0, 1} is the classic Prize-Collecting Steiner Forest problem. In this paper we present a novel linear programming relaxation for the Prize-Collecting Steiner Network problem, and by rounding it, obtain the first constant-factor approximation algorithm for submodular and monotone non-decreasing penalty functions. In particular, our setting includes all-or-nothing penalty functions, which charge the penalty even if the connectivity requirement is slightly violated; this resolves an open question posed in [SSW07]. We further generalize our results for element-connectivity and node-connectivity.

We consider the optimal value of information (VoI) problem, where the goal is to sequentially select a set of tests with a minimal cost, so that one can efficiently make the best decision based on the observed outcomes. Existing algorithms are either heuristics with no guarantees, or scale poorly (with exponential run time in terms of the number of available tests). Moreover, these methods assume a known distribution over the test outcomes, which is often not the case in practice. We propose an efficient sampling-based online learning framework to address the above issues. First, assuming the distribution over hypotheses is known, we propose a dynamic hypothesis enumeration strategy, which allows efficient information gathering with strong theoretical guarantees. We show that with sufficient amount of samples, one can identify a near-optimal decision with high probability. Second, when the parameters of the hypotheses distribution are unknown, we propose an algorithm which learns th...

A wide variety of problems in machine learning, including exemplar clustering, document summarization, and sensor placement, can be cast as constrained submodular maximization problems. A lot of recent effort has been devoted to developing distributed algorithms for these problems. However, these results suffer from high number of rounds, suboptimal approximation ratios, or both. We develop a framework for bringing existing algorithms in the sequential setting to the distributed setting, achieving near optimal approximation ratios for many settings in only a constant number of MapReduce rounds. Our techniques also give a fast sequential algorithm for non-monotone maximization subject to a matroid constraint.