Consistent approximation of an optimal search problem

Naval Research Logistics, 1995

A search is conducted for a target moving in discrete time among a finite number of cells according to a known Markov process. The searcher must choose one cell in which to search in each time period. The set ofcells available for search depends upon the cell chosen in the last time period. The problem is to find a search path, i.e., a sequence of search cells, that either maximizes the probability ofdetection or minimizes the mean number oftime penods required for detection. The search problem is modelled as a partially observable Markov decision process and several approximate solutions procedures are proposed. 0 search for efficient approximate solution procedures. Washburn [ 15 ] generalized Brown's algorithm to find approximate solutions when there are no path constraints. Stewart [ 121 used branch-and-bound methods and Brown's algorithm to obtain approximate solutions when both discrete effort and path constraints are allowed. Eagle and Yee [ 51 extended Stewart's work and obtained a branch-and-bound method that gives optimal solutions. An alternative approach for finding optimal solutions, suggested by Smallwood and Sondik [lo] and extended by Eagle [4], is to model the problem as a partially observable Markov decision process. This is the approach investigated here. Monahan's survey article [ 81 outlines the properties of partially observable Markov decision processes, and Lovejoy [ 71 describes the algorithms used to solve them. The difficulty with using this approach is that value iteration, which is the usual solution algorithm, takes too long and needs too much storage for some large problems. Lovejoy [ 61 attempts to overcome this difficulty by developing bounds on the iterates that are easier to calculate. Our objective is the same. We describe easy-to-calculate starting values for value iteration, as well as a pruning procedure similar to Lovejoy's bounds but developed independently.

International Journal of Robust and Nonlinear Control, 2008

Autonomous aerial vehicles play an important role in military applications such as in search, surveillance and reconnaissance. Multi-player stochastic pursuit-evasion (PE) differential game is a natural model for such operations involving intelligent moving targets with uncertainties. In this paper, some fundamental issues of stochastic PE games are addressed. We first model a general stochastic multi-player PE differential game with perfect state information. To avoid the difficulty of multiplicity of the players, we extend the iterative method for deterministic multi-player PE games to the stochastic case. Starting from certain suboptimal solutions with an improving property, the optimization based on limited look-ahead can be used for improvement. The process converges when this improvement is applied iteratively. Furthermore, we introduce a hierarchical approach that can determine a valid starting point of the iterative process. As a basis for multi-player games, stochastic two-player PE games are also addressed. We also briefly discuss the games with imperfect state information and propose a suboptimal approach from a practical point of view. Finally, we demonstrate the usefulness and the feasibility of the method through simulations.

IEEE Transactions on Control of Network Systems, 2016

This paper presents a distributed optimal control approach for managing omnidirectional sensor networks deployed to cooperatively track moving targets in a region of interest. Several authors have shown that, under proper assumptions, the performance of mobile sensors is a function of the sensor distribution. In particular, the probability of cooperative track detection, also known as track coverage, can be shown to be an integral function of a probability density function representing the macroscopic sensor network state. Thus, a mobile sensor network deployed to detect moving targets can be viewed as a multiscale dynamical system in which a time-varying probability density function can be identified as a restriction operator, and optimized subject to macroscopic dynamics represented by the advection equation. Simulation results show that the distributed control approach is capable of planning the motion of hundreds of cooperative sensors, such that their effectiveness is significantly increased compared to that of existing uniform, grid, random and stochastic gradient methods.

2015

We consider a pursuit differential game of one pursuer and one evader described by infinite system of first order differential equations. The coordinate-wise integral constraints are imposed on control functions of players. By definition pursuit is said to be completed if the state of system equals zero at some time. A sufficient condition of completion of pursuit is obtained. Strategy for the pursuer is constructed and an explicit formula for the guaranteed pursuit time is given.

Naval Research Logistics Quarterly, 1981

In this paper we analyze optimal search strategies in an environment in which multiple, independent targets arrive and depart at random. The analysis revolves around a continuous time differential equation model which captures the time dependent nature of the search process. We explore the impact on optimal strategies of nonzero travel times between regions as well as differing target arrival rates. We derive simple closed form expressions for determining if only one region should be searched.

52nd IEEE Conference on Decision and Control, 2013

This paper focuses on an optimal control problem in which the objective is to minimize the expectation of a cost functional with stochastic parameters. The inclusion of the stochastic parameters in the objective raises new theoretical and computational challenges not present in a standard nonlinear optimal control problem. In this paper, we provide a numerical framework for the solution of this uncertain optimal control problem by taking a sample average approximation approach. An independent random sample is taken from the parameter space, and the expectation is approximated by the sample average. The result is a family of standard nonlinear optimal control problems which can be solved using existing techniques. We provide an optimality function for both the uncertain optimal control problem and its approximation, and show that the approximation based on the sample average approach is consistent in the sense of Polak. We illustrate the approach with a numerical example arising in optimal search for a moving target.

Operations Research, 1981

This paper shows that solving the optimal whereabouts search problem for a moving target is equivalent to solving a finite number of optimal detection problems for moving targets. This generalizes the result of Kadane (Kadane, J. B. 1971. Optimal whereabouts search. Opns. Res. 19 894–904.) for stationary targets.

Journal of the Nigerian Society of Physical Sciences, 2021

We study a simple motion pursuit differential game of many pursuers and one evader in a Hilbert space $l_{2}$. The control functions of the pursuers and evader are subject to integral and geometric constraints respectively. Duration of the game is denoted by positive number $\theta $. Pursuit is said to be completed if there exist strategies $u_{j}$ of the pursuers $P_{j}$ such that for any admissible control $v(\cdot)$ of the evader $E$ the inequality $\|y(\tau)-x_{j}(\tau)\|\leq l_{j}$ is satisfied for some $ j\in \{1,2, \dots\}$ and some time $\tau$. In this paper, sufficient conditions for completion of pursuit were obtained. Consequently strategies of the pursuers that ensure completion of pursuit are constructed.

Sensors, 2014

The minimum time search in uncertain domains is a searching task, which appears in real world problems such as natural disasters and sea rescue operations, where a target has to be found, as soon as possible, by a set of sensor-equipped searchers. The automation of this task, where the time to detect the target is critical, can be achieved by new probabilistic techniques that directly minimize the Expected Time (ET) to detect a dynamic target using the observation probability models and actual observations collected by the sensors on board the searchers. The selected technique, described in algorithmic form in this paper for completeness, has only been previously partially tested with an ideal binary detection model, in spite of being designed to deal with complex non-linear/non-differential sensorial models. This paper covers the gap, testing its performance and applicability over different searching tasks with searchers equipped with different complex sensors. The sensorial models under test vary from stepped detection probabilities to continuous/discontinuous differentiable/non-differentiable detection probabilities dependent on distance, orientation, and structured maps. The analysis of the simulated results of several static and dynamic scenarios performed in this paper validates the applicability of the technique with different types of sensor models. Azores and many other objects. Since then, the theory has been used for search and rescue as well as for many other nonmilitary applications . Nowadays, much of the research aims at providing autonomous robots with the capability of searching and tracking.

We study stochastic motion planning problems which involve a controlled process, with possibly discontinuous sample paths, visiting certain subsets of the state-space while avoiding others in a sequential fashion. For this purpose, we first introduce two basic notions of motion planning, and then establish a connection to a class of stochastic optimal control problems concerned with sequential stopping-times. A weak dynamic programming principle (DPP) is then proposed, which characterizes the set of initial states that admit the existence of a policy enabling the process to execute the desired maneuver with probability at least as much as some pre-specified value. The proposed DPP consists of some auxiliary value functions defined in terms of discontinuous payoff functions. An application of the DPP is demonstrated in the context of controlled diffusion processes thereafter. It turns out that the aforementioned set of initial states can be characterized as the level set of a discontinuous viscosity solution to a sequence of partial differential equations, for which the first one has a known boundary condition, while the boundary conditions of the subsequent ones are determined by the solutions to the preceding steps. Finally, the generality and flexibility of the theoretical results are illustrated with the aid of an example involving biological switches.

2016

In this paper, we study a differential game of many pursuers and one evader in R2. The motions of all players are simple. An integral constraint is imposed on each coordinate of the control functions of players. We say that pursuit is completed if the state of a pursuer coincides with that of the evader at some time. The pursuers try to complete the pursuit, and the evader tries to avoid this. Sufficient conditions for completion of the differential game were obtained. The strategies of the pursuers are constructed based on the current values of control parameter of the evader. Also an illustrative example is provided.

ArXiv, 2020

We introduce a discrete-time search game, in which two players compete to find an object first. The object moves according to a time-varying Markov chain on finitely many states. The players know the Markov chain and the initial probability distribution of the object, but do not observe the current state of the object. The players are active in turns. The active player chooses a state, and this choice is observed by the other player. If the object is in the chosen state, this player wins and the game ends. Otherwise, the object moves according to the Markov chain and the game continues at the next period. We show that this game admits a value, and for any error-term $\veps>0$, each player has a pure (subgame-perfect) $\veps$-optimal strategy. Interestingly, a 0-optimal strategy does not always exist. The $\veps$-optimal strategies are robust in the sense that they are $2\veps$-optimal on all finite but sufficiently long horizons, and also $2\veps$-optimal in the discounted versio...

Proceedings of the 18th IFAC World Congress, 2011

The observation of multiple moving targets by cooperating mobile robots is a key problem in many security, surveillance and service applications. In essence, this problem is characterized by a tight coupling of target allocation and continuous trajectory planning. Optimal control of the multi-robot system generally neither permits to neglect physical motion dynamics nor to decouple or successively process target assignment and trajectory planning. In this paper, a numerically robust and stable model-predictive control strategy for solving the problem in the case of discrete-time double-integrator dynamics is presented. Optimization based on linear mixed logical dynamical system models allows for a flexible weighting of different aspects and optimal control inputs for settings of moderate size can be computed in real-time. By simulating sets of randomly generated situations, one can determine a maximum problem size solvable in real-time in terms of the number of considered robots, targets, and length of the prediction horizon. Based on this information, a decentralized control approach is proposed.

PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON MATHEMATICAL SCIENCES AND TECHNOLOGY 2018 (MATHTECH2018): Innovative Technologies for Mathematics & Mathematics for Technological Innovation

The present paper proposes differential game of pursuit described by system of infinite first order ordinary differential equations. The game focuses only on one-pursuer and one-evader differential game and the coordinate wise integral constraints are imposed on the player's control functions.The pursuit is assumed to be completed if the pursuer brings the state of the system from one state in to another state at some time. We proved a theorem on a guaranteed pursuit time. Moreover, an explicit strategy of pursuer is constructed.

Proceedings of the 2010 American Control Conference, 2010

This paper outlines a strategy for tracking evasive objects in discrete space using game theory to allocate sensor resources. One or more searchers have to allocate the effort among the discrete cells to maximize the object detection probability within a finite time horizon or minimize the expected search time to achieve the desired detection probability under a false alarm constraint. We review the standard formulations under a sequential decision setting for finding stationary objects. Then we consider both robust and optimal search strategies and extend the standard search problem to a two-person zerosum search allocation game where the object wants to hide from the searcher and the object has incomplete information about the searcher's remaining search time. We discuss how the results affect the sensor management and mission planning for cooperative unmanned aerial vehicle (UAV) search tasks and provide simulation examples to show the effectiveness of the proposed method compared with random search strategy.