Online Optimization

In this paper we present a method to calibrate the surface EMG signal-to-force-relationship online. For this, a simple biomechanical model composed of bones and muscles is used. The calibration is based on an online optimization algorithm where the error between the movement of the human and the movement computed with the biomechanical model is minimized. The proposed method will be part of a control system for an exoskeleton robot that should aid the wearer in everyday-life situations like walking, standing up and sitting down. In contrast to existing methods for the calculation of the EMG signal-to-force-relationship, we are not interested in the exact force values of every single muscle, but our model groups some muscles together and uses the EMG signal of one of those muscles as a representative for the group to simplify calculations. The performance of the presented method was investigated on the leg movement in sagittal plane without contact to the environment.

In this paper we present a method to calibrate the surface EMG signal-to-force-relationship online. For this, a simple biomechanical model composed of bones and muscles is used. The calibration is based on an online optimization algorithm where the error between the movement of the human and the movement computed with the biomechanical model is minimized. The proposed method will be part of a control system for an exoskeleton robot that should aid the wearer in everyday-life situations like walking, standing up and sitting down. In contrast to existing methods for the calculation of the EMG signal-to-force-relationship, we are not interested in the exact force values of every single muscle, but our model groups some muscles together and uses the EMG signal of one of those muscles as a representative for the group to simplify calculations. The performance of the presented method was investigated on the leg movement in sagittal plane without contact to the environment.

We propose a localized approach to multiple kernel learning that, in contrast to prevalent approaches, can be formulated as a convex optimization problem over a given cluster structure. From which we obtain the first generalization error bounds for localized multiple kernel learning and derive an efficient optimization algorithm based on the Fenchel dual representation. Experiments on real-world datasets from the application domains of computational biology and computer vision show that the convex approach to localized multiple kernel learning can achieve higher prediction accuracies than its global and non-convex local counterparts.

Properly optimizing the setting of configuration parameters can greatly improve performance, especially in the presence of changing workloads. This paper explores approaches to online optimization of the Apache web server, focusing on the MaxClients parameter (which controls the maximum number of workers). Using both empirical and analytic techniques, we show that Max-Clients has a concave upward effect on response time and hence hill climbing techniques can be used to find the optimal value of MaxClients. The underlying intuition is that MaxClients controls the trade-off between delays in the TCP Accept Queue and delays due to contention for operating system resources. We investigate two optimizers that employ hill climbing-one based on Newton's Method and the second based on fuzzy control. A third technique is a heuristic that exploits relationships between bottleneck utilizations and response time minimization. In all cases, online optimization reduces response times by a factor of 10 or more compared to using a static, default value. The trade-offs between the online schemes are as follows. Newton's method is well known but does not produce consistent results for highly variable data such as response times. Fuzzy control is more robust, but converges slowly. The heuristic works well in our prototype system, but it may be difficult to generalize because it requires knowledge of bottleneck resources and an ability to measure their utilizations.

We consider the problem of online optimization, where a learner chooses a decision from a given decision set and suffers some loss associated with the decision and the state of the environment. The learner's objective is to minimize its cumulative regret against the best fixed decision in hindsight. Over the past few decades numerous variants have been considered, with many algorithms designed to achieve sub-linear regret in the worst case. However, this level of robustness comes at a cost. Proposed algorithms are often over-conservative, failing to adapt to the actual complexity of the loss sequence which is often far from the worst case. In this paper we introduce a general algorithm that, provided with a "safe" learning algorithm and an opportunistic "benchmark", can effectively combine good worst-case guarantees with much improved performance on "easy" data. We derive general theoretical bounds on the regret of the proposed algorithm and discuss its implementation in a wide range of applications, notably in the problem of learning with shifting experts (a recent COLT open problem). Finally, we provide numerical simulations in the setting of prediction with expert advice with comparisons to the state of the art.

Markov decision processes (MPDs) have become a popular model for real-world problems of planning under uncertainty. A vide range of applications has been published within the fields of natural resources management, forestry, agricultural economics, and robotics. An MDP can be used to represent and optimize the management of an environment. A state variable is used to represent the current state of the environment and the interaction with the environment is expressed with an action variable, allowing for stochastic transition from one state to another state. A transition function is used to express the transition between the different states of the environment, and a reward function expresses the rewards received by applying an action when the environment is in a specific state. MDPs can thereby be used to optimize management strategies for problems with uncertain effects of actions, multiple and conflicting objectives, stochastic events, and stochastic environments. A common problem when optimizing management strategies for natural resources is that large areas have to be considered, rendering the state and action space of the MDP large. For example, an agricultural area consisting of thousands of crop fields, or forests consisting of thousands of stands. Also, spatial aspects may have to be considered. Some examples are management of epidemics, invasive or endangered species, management to prevent wind damage and wild fire. However, the structure of the problem can be used to model the problem on a compact form. For example is the probability of a tree being damaged by wind only on the state of the neighbouring trees. This type of local dependency and internal structure can be used to express the problem on a compact factored form. For example, instead of modelling the state of forest with a single state variable, a set of state variables is utilized, each expressing the state of a part of forest. A number of different representational models have been proposed for this type of problems. Factored MDPs and collaborative multiagent MDPs are two frameworks that have recently received a loot of attention. The graph-based Markov decision process (GMDP) framework has recently been developed specifically for modelling a range of large-scale spatiotemporal problems. As the name suggests, the GMDP uses a graph structure to represent the transition and reward functions on a compact form. Two algorithms for computing approximate solutions for GMDPs have been proposed. The algorithms are capable of solving large-scale GMDPs as their running time increases only linearly and polynomially with the size of the problem for a fixed induced width of the graph. However, the algorithms are model-dependent and require the transition and reward functions to be known. Simulation based models of the transition and reward functions can therefore not be used. Since there are a large number of simulation based models of real-world management problems, for example , the development of model-free solution algorithms is an important extension of the GMDP framework and will increase the applicability of GMDPs. In this paper, we describe a number of model-free reinforcement learning (RL) algorithms for GMDPs derived from existing model-free RL algorithms for collaborative multiagent MDPs. The collaborative multiagent MDP framework is similar to the GMDP framework in a number of aspects, and in this paper we use these similarities to propose RL algorithms for GMDPs. The performance of the algorithms is compared to model-dependent algorithms for a medium-and large-scale forest management problem. Our experimental results show that the proposed RL algorithms are able to efficiently compute policies demonstrating nearoptimal performance. The policies computed by the RL algorithms are of similar quality to policies computed by the model-dependent algorithms.

In classical models of exchange under smoothness and strict concavity assumptions that in particular only support positive quantities of goods, every equilibrium is shift-stable. This property, referring to good behavior in response to local perturbations, can be established by elementary means without resorting to techniques of differential topology. Utilization of Lagrange multipliers for the budget constraints furthermore brings to light additional features not recorded in the past.

The approximate nonlinear receding-horizon control law is used to treat the trajectory tracking control problem of rigid link robot manipulators. The derived nonlinear predictive law uses a quadratic performance index of the predicted tracking error and the predicted control effort. A key feature of this control law is that, for their implementation, there is no need to perform an online optimization, and asymptotic tracking of smooth reference trajectories is guaranteed. It is shown that this controller achieves the positions tracking objectives via link position measurements. The stability convergence of the output tracking error to the origin is proved. To enhance the robustness of the closed loop system with respect to payload uncertainties and viscous friction, an integral action is introduced in the loop. A nonlinear observer is used to estimate velocity. Simulation results for a two-link rigid robot are performed to validate the performance of the proposed controller.

The approximate nonlinear receding-horizon control law is used to treat the trajectory tracking control problem of rigid link robot manipulators. The derived nonlinear predictive law uses a quadratic performance index of the predicted tracking error and the predicted control effort. A key feature of this control law is that, for their implementation, there is no need to perform an online optimization, and asymptotic tracking of smooth reference trajectories is guaranteed. It is shown that this controller achieves the positions tracking objectives via link position measurements. The stability convergence of the output tracking error to the origin is proved. To enhance the robustness of the closed loop system with respect to payload uncertainties and viscous friction, an integral action is introduced in the loop. A nonlinear observer is used to estimate velocity. Simulation results for a two-link rigid robot are performed to validate the performance of the proposed controller.

The application of Field Programmable Gate Array (FPGA) in the development of power electronics circuits control scheme has drawn much attention lately due to its shorter design cycle, lower cost and higher density. This paper presents an FPGA-based gate signal generator for a multilevel inverter employing an online optimal PWM switching strategy to control its output voltage. FPGA is chosen for the hardware implementation of the switching strategy mainly due to its high computation speed that can ensure the accuracy of the instants that gating signals are generated. The gate signal generator has been realized by an FPGA (FLEXlOKZO) from Altera. The design and development of the FPGA based gate signal generator is described in detail. Results from the timing simulation using MAX+PLUSII software are given and verified by the results obtained from the FLEXlOK2O output.

Deep Feedforward Neural Networks' (DFNNs) weights estimation relies on the solution of a very large nonconvex optimization problem that may have many local (no global) minimizers, saddle points and large plateaus. As a consequence, optimization algorithms can be attracted toward local minimizers which can lead to bad solutions or can slow down the optimization process. Furthermore, the time needed to find good solutions to the training problem depends on both the number of samples and the number of variables. In this work, we show how Block Coordinate Descent (BCD) methods can be applied to improve performance of state-of-the-art algorithms by avoiding bad stationary points and flat regions. We first describe a batch BCD method ables to effectively tackle the network's depth and then we further extend the algorithm proposing a minibatch BCD framework able to scale with respect to both the number of variables and the number of samples by embedding a BCD approach into a minibatch framework. By extensive numerical results on standard datasets for several architecture networks, we show how the application of BCD methods to the training phase of DFNNs permits to outperform standard batch and minibatch algorithms leading to an improvement on both the training phase and the generalization performance of the networks.

Optimizing configuration parameters is time-consuming and skills-intensive. This paper proposes a generic approach to automating this task. By generic, we mean that the approach is relatively independent of the target system for which the optimization is done. Our approach uses online adjustment of configuration parameters to discover the system's performance characteristics. Doing so creates two challenges: (1) handling interdependencies between configuration parameters and (2) minimizing the deleterious effects on production workload while the optimization is underway. Our approach addresses (1) by including in the architecture a rule-based component that handles interdependencies between configuration parameters. For (2), we use a feedback mechanism for online optimization that searches the parameter space in a way that generally avoids poor performance at intermediate steps. Our studies of a DB2 Universal Database Server under an e-commerce workload indicate that our approach can be effective in practice.

We propose an algorithm for the effective solution of quadratic programming (QP) problems arising from model predictive control (MPC). MPC is a modern multivariable control method which gives the solution for a QP problem at each sample instant. Our algorithm combines the active-set strategy with the proportioning test to decide when to leave the actual active set. For the minimization in the face, we use a direct solver implemented by the Cholesky factors updates. The performance of the algorithm is illustrated by numerical experiments, and the results are compared with the state-of-the-art solvers on benchmarks from MPC.

In recent years there has been a lot of interest in designing principled classification algorithms over multiple cues, based on the intuitive notion that using more features should lead to better performance. In the domain of kernel methods, a principled way to use multiple features is the Multi Kernel Learning (MKL) approach. Here we present a MKL optimization algorithm based on stochastic gradient descent that has a guaranteed convergence rate. We directly solve the MKL problem in the primal formulation. By having a p-norm formulation of MKL, we introduce a parameter that controls the level of sparsity of the solution, while leading to an easier optimization problem. We prove theoretically and experimentally that 1) our algorithm has a faster convergence rate as the number of kernels grows; 2) the training complexity is linear in the number of training examples; 3) very few iterations are sufficient to reach good solutions. Experiments on standard benchmark databases support our claims.

In this study, we show the successful application of different model-based approaches for the maximizing of macrolactin D production by Paenibacillus polymyxa. After four initial cultivations, a family of nonlinear dynamic biological models was determined automatically and ranked by their respective Akaike Information Criterion (AIC). The best models were then used in a multi-model setup for robust product maximization. The experimental validation shows the highest product yield attained compared with the identification runs so far. In subsequent fermentations, the online measurements of CO 2 concentration, base consumption, and near-infrared spectroscopy (NIR) were used for model improvement. After model extension using expert knowledge, a single superior model could be identified. Model-based state estimation with a sigma-point Kalman filter (SPKF) was based on online measurement data, and this improved model enabled nonlinear real-time product maximization. The optimization increased the macrolactin D production even further by 28% compared with the initial robust multi-model offline optimization.

In classical optimization, called offline optimization here, it is assumed that all input data of an instance are available before solution algorithms are applied. In many applications this is not realistic. Decisions have to be made before all data are known. Such situations are often termed online. They arise in particular in processes that are continuously running for a longer period of time. For instance, in material flow systems of companies, transportation tasks arise throughout the day and decisions have to be made before all jobs have been generated. Online optimization is the task of finding "good," or "cheap," or "economic" decisions in online situations. An online algorithm provides such decisions. In practical applications online algorithms are typically subject to additional constraints. For example, they have to answer in real time, or must process a job (or request) within a given time frame, sometimes they have access to limited computing resources only. Competitiveness. A common concept to evaluate online algorithms is competitiveness. Here, an online algorithm has to act as follows: it always has to serve a request of a sequence before the next request (or the next k different requests in a model with lookahead) becomes visible. The idea behind the variants of this notion is the following: compare the solution of the online algorithm under consideration with the solution that some adversary would produce on the same set of data. The easiest case is the offline adversary, i.e., to him the complete sequence of requests is known in advance. For an algorithm X let C X s denote the cost X produces on input sequence s. We assume for notational convenience that C X s is positive for all s

T h e quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely even t that the author did not send a com plete m anuscript and there are missing pages, these will be noted. Also, if m aterial had to be rem oved, a note will indicate the deletion. Di!ss0?t&iori Publishing UM I 3580411

In this paper we propose an approach to optimization of web marketing content based on an online particle swarm optimization (PSO) model. The idea behind online PSO is to evaluate the collective user feedback as the PSO objective function which drives particles velocities in the hybrid continuous-discrete space of web content features. PSO coordinates the process of sampling collective user behavior in order to optimize the web marketing metric. To improve the performances a variation to the PSO schema is adopted, this variation consists in a restart of the algorithm if the convergence speed is not good. Experiments in the scenario of the home page of an online shop show that the method converges faster and avoid some common drawbacks such as local optimal and hybrid discrete/continuous features management; however is observed that the restart procedure improves the convergence speed of some difficult instances of the problem without affects the other ones. The proposed online optimization method is general and can be applied to other web marketing or business intelligent contexts.

We study exploration in Multi-Armed Bandits in a setting where k players collaborate in order to identify an ε-optimal arm. Our motivation comes from recent employment of bandit algorithms in computationally intensive, large-scale applications. Our results demonstrate a non-trivial tradeoff between the number of arm pulls required by each of the players, and the amount of communication between them. In particular, our main result shows that by allowing the k players to communicate only once, they are able to learn √ k times faster than a single player. That is, distributing learning to k players gives rise to a factor √ k parallel speed-up. We complement this result with a lower bound showing this is in general the best possible. On the other extreme, we present an algorithm that achieves the ideal factor k speed-up in learning performance, with communication only logarithmic in 1/ε.

Efficient memory management is crucial when designing high performance processors. Upon a miss, the conventional operation mode of a cache hierarchy is to retrieve the missing block from lower levels and to store it into all hierarchy levels. It is however difficult to assert that storing the block into intermediate levels will be really useful. In particular, this is unnecessary if a cache block is accessed only once before getting evicted-i.e. a single-usage block. This paper is typically concerned with reducing the number of single-usage blocks. Our observations reveal that single-usage blocks are significant at runtime and especially in the lowest cache level. We show that using an address-based prediction mechanism is sufficient to identify this phenomenon. Two schemes are examined to remove pollution caused by single-usage blocks: a bypass scheme and a cache replacement policy. Our results show that leveraging single-usage pollution is beneficial to memory-intensive applications running on superscalar and multi-core architectures.

Efficient memory management is crucial when designing high performance processors. Upon a miss, the conventional operation mode of a cache hierarchy is to retrieve the missing block from lower levels and to store it into all hierarchy levels. It is however difficult to assert that storing the block into intermediate levels will be really useful. In particular, this is unnecessary if a cache block is accessed only once before getting evicted-i.e. a single-usage block. This paper is typically concerned with reducing the number of single-usage blocks. Our observations reveal that single-usage blocks are significant at runtime and especially in the lowest cache level. We show that using an address-based prediction mechanism is sufficient to identify this phenomenon. Two schemes are examined to remove pollution caused by single-usage blocks: a bypass scheme and a cache replacement policy. Our results show that leveraging single-usage pollution is beneficial to memory-intensive applications running on superscalar and multi-core architectures.

The availability of process computers and the recent developments of on-line techniques, particularly in combination with fiber optic waveguides, offer quite new possibilities for remote on-line and in-line sensing and have considerably increased the scope of on-line control of product quality in polymerization reactors. In the present work an attempt is made to present, as completely as possible, a comprehensive literature survey of the recent advances made in the development of on-line hardware sensors for monitoring of polymerization reactions, as well as to investigate the robustness and accuracy of the online techniques used to measure continuously monomer conversion, copolymer composition, molecular weight and MWD and particle size distribution in polymer reaction operations. Finally, software-sensing strategies including state estimation and filtering techniques are very briefly reviewed.

An on-line optimizing control scheme that ensures the satisfaction of polymer property specifications under the influence of time-varying model parameters and unknown initial conditions is developed for the free-radical polymerization of methyl-methacrylate in a batch reactor. The control scheme combines a state/parameter estimation step based on on-line measurements of process variables that updates the values of model parameters with an optimization step that calculates the optimal operating policy. The control objectives require the satisfaction of quality specifications based on molecular weight properties (e.g. average molecular weights) of the final polymer product subject to time-varying kinetic parameters and unknown initial level of impurities in the reactor.

A promising method of automating management tasks in computing systems is to formulate them as control or optimization problems in terms of performance metrics. For an online optimization scheme to be of practical value in a distributed setting, however, it must successfully tackle the curses of dimensionality and modeling. This paper develops a hierarchical control framework to solve performance management problems in distributed computing systems operating in a data center. Concepts from approximation theory are used to reduce the computational burden of controlling such large-scale systems. The relevant approximations are made in the construction of the dynamical models to predict system behavior as well as in the solution of the associated control equations. Using a dynamic resource provisioning problem as a case study, we show that a computing system managed by the proposed control framework with approximation models realizes profit gains that are in the best case within 1% of a controller using an explicit model of the system.

This paper discusses online optimization of real-world transportation systems. We concentrate on transportation problems arising in production and manufacturing processes, in particular in company internal logistics. We describe basic techniques to design online optimization algorithms for such systems, but our main focus is decision support for the planner: which online algorithm is the most appropriate one in a particular setting? We show by means of several examples that traditional methods for the evaluation of online algorithms often do not suffice to judge the strengths and weaknesses of online algorithms. We present modifications of well-known evaluation techniques and some new methods, and we argue that the selection of an online algorithm to be employed in practice should be based on a sound combination of several theoretical and practical evaluation criteria, including simulation.

In "classical" optimization, all data of a problem instance are considered given. The standard theory and the usual algorithmic techniques apply to such cases only. Online optimization is different. Many decisions have to be made before all data are available. In addition, decisions once made cannot be changed. How should one act "best" in such an environment? In this paper we survey online problems coming up in combinatorial optimization. We first outline theoretical concepts, such as competitiveness against various adversaries, to analyze online problems and algorithms. The focus, however, lies on real-world applications. We report, in particular, on theoretical investigations and our practical experience with problems arising in transportation and the automatic handling of material.

In predictive control, a quadratic program (QP) needs to be solved at each sampling instant. We present a new warm-start strategy to solve a QP with an interior-point method whose data is slightly perturbed from the previous QP. In this strategy, an initial guess of the unknown variables in the perturbed problem is determined from the computed solution of the previous problem. We demonstrate the effectiveness of our warm-start strategy to a number of online benchmark problems. Numerical results indicate that the proposed technique depends upon the size of perturbation and it leads to a reduction of 30-74% in floating point operations compared to a cold-start interior-point method.

We present a method to coordinate a large number of underactuated robots by designing control laws on a small dimensional manifold, independent on the number and ordering of the robots. The small dimensional description of the team has a product structure of a Lie group, which captures the dependence of the ensemble on the world frame, and a shape manifold, which is an intrinsic description of the team. We design decoupled controls for group and shape. The individual control laws which are mapped to the desired collective behavior can be realized by feedback depending only on the current state of the robot and the state on the small dimensional manifold, so that the robots have to broadcast their states and only have to listen to some coordinating agent with small bandwidth.

Traditionally, offline optimization of power systems is acceptable due to the largely predictable loads and reliable generation. The increasing penetration of fluctuating renewable generation and internet-of-things devices allowing for fine-grained controllability of loads have led to the diminishing applicability of offline optimization in the power systems domain, and have redirected attention to online optimization methods. However, online optimization is a broad topic that can be applied in and motivated by different settings, operated on different time scales, and built on different theoretical foundations. This paper reviews the various types of online optimization techniques used in the power systems domain and aims to make clear the distinction between the most common techniques used. In particular, we introduce and compare four distinct techniques used covering the breadth of online optimization techniques used in the power systems domain, i.e., optimization-guided dynamic control, feedback optimization for single-period problems, Lyapunov-based optimization, and online convex optimization techniques for multi-period problems. Lastly, we recommend some potential future directions for online optimization in the power systems domain.

ZF Luftfahrttechnik GmbH (ZFL) has conducted open and closed loop IBC (Individual Blade Control) flight tests with the CH-53G IBC testbed of the German Federal Armed Forces Engineering Center for Aircraft. Over 25 flight hours had been spent in an "open loop" campaign to investigate the positive IBC effects. Early during the flight tests, it became clear that the response of the aircraft to the IBC inputs exceeded the initial expectations. ZFL has expanded the open loop IBC system to enable a closed loop operation for automatic online optimization of the required IBC inputs. The controller design was performed under Matlab/Simulink and automatically implemented on a dSPACE real-time system. The closed loop control system added to the open loop core system extends the existing cascade control structure by an outer control loop. This outer control loop was used to implement different control tasks which were then investigated during the closed loop campaign. The main focus was put on vibration reduction in the fuselage. Further tests have addressed the load reduction potential of IBC. The capabilities of IBC were demonstrated for different single-and multiple-harmonic inputs in different steady and maneuvering flight conditions at different forward speeds. The predicted improvements through the application of more than one frequency could clearly be shown. Different cost functions have been investigated with the controller using single or multiple sensor signals. It turned out that even simple cost functions based on only a few sensor signals were practical since often the not included sensor locations clearly benefited from that particular input. The amplitudes applied by the controller were on the order of 0.1 to 0.5 deg which has underlined the effectiveness of even small blade pitch inputs. The paper gives a brief overview of the architecture of the closed loop system and the implemented control strategies. The main focus is put on the test results gathered during the different closed loop flights. Beside the beneficial vibration reduction also some other IBC effects are discussed.

We develop a state-of-the-art nonlinear model predictive controller (NMPC) for periodic unstable systems, and apply the method to a dual line kite that shall fly loops. The kite is described by a nonlinear unstable ODE system (which we freely distribute), and the aim is to let the kite fly a periodic figure. Our NMPC approach is based on the "infinite horizon closed loop costing" scheme to ensure nominal stability. To be able to apply this scheme, we first determine a periodic LQR controller to stabilize the kite locally in the periodic orbit. Then, we formulate a two-stage NMPC optimal control problem penalizing deviations of the system state from the periodic orbit, which also contains a state constraint that avoids that the kite collides with the ground. To solve the optimal control problems reliably and in real-time, we apply the newly developed "real-time iteration scheme" for fast online optimization in NMPC. The optimization based NMPC leads to significantly improved performance compared to the LQR controller, in particular as it respects state constraints. The NMPC closed loop also performs well in the presence of large random disturbances and shows considerable robustness against changes in the wind direction.

Motivated by concerns that automated decision-making procedures can unintentionally lead to discriminatory behavior, we study a technical definition of fairness modeled after John Rawls' notion of "fair equality of opportunity". In the context of a simple model of online decision making, we give an algorithm that satisfies this fairness constraint, while still being able to learn at a rate that is comparable to (but necessarily worse than) that of the best algorithms absent a fairness constraint. We prove a regret bound for fair algorithms in the linear contextual bandit framework that is a significant improvement over our technical companion paper [16], which gives black-box reductions in a more general setting. We analyze our algorithms both theoretically and experimentally. Finally, we introduce the notion of a "discrimination index", and show that standard algorithms for our problem exhibit structured discriminatory behavior, whereas the "fair" ...

ZF Luftfahrttechnik GmbH (ZFL) has conducted open and closed loop IBC (Individual Blade Control) flight tests with the CH-53G IBC testbed of the German Federal Armed Forces Engineering Center for Aircraft. Over 25 flight hours had been spent in an "open loop" campaign to investigate the positive IBC effects. Early during the flight tests, it became clear that the response of the aircraft to the IBC inputs exceeded the initial expectations. ZFL has expanded the open loop IBC system to enable a closed loop operation for automatic online optimization of the required IBC inputs. The controller design was performed under Matlab/Simulink and automatically implemented on a dSPACE real-time system. The closed loop control system added to the open loop core system extends the existing cascade control structure by an outer control loop. This outer control loop was used to implement different control tasks which were then investigated during the closed loop campaign. The main focus was put on vibration reduction in the fuselage. Further tests have addressed the load reduction potential of IBC. The capabilities of IBC were demonstrated for different single-and multiple-harmonic inputs in different steady and maneuvering flight conditions at different forward speeds. The predicted improvements through the application of more than one frequency could clearly be shown. Different cost functions have been investigated with the controller using single or multiple sensor signals. It turned out that even simple cost functions based on only a few sensor signals were practical since often the not included sensor locations clearly benefited from that particular input. The amplitudes applied by the controller were on the order of 0.1 to 0.5 deg which has underlined the effectiveness of even small blade pitch inputs. The paper gives a brief overview of the architecture of the closed loop system and the implemented control strategies. The main focus is put on the test results gathered during the different closed loop flights. Beside the beneficial vibration reduction also some other IBC effects are discussed.

We consider a periodic-review, single-location, single-product inventory system with lost sales and positive replenishment lead times. It is well known that the optimal policy does not possess a simple structure. Motivated by recent results showing that base-stock policies perform well in these systems, we study the problem of finding the best base-stock policy in such a system. In contrast to the classical inventory literature, we assume that the manager does not know the demand distribution a priori but must make the replenishment decision in each period based only on the past sales (censored demand) data. We develop a nonparametric adaptive algorithm that generates a sequence of order-up-to levels whose running average of the inventory holding and lost sales penalty cost converges to the cost of the optimal base-stock policy, and we establish the cubic-root convergence rate of the algorithm. Our analysis is based on recent advances in stochastic online convex optimization and on ...

This paper presents a genetic algorithms approach for the optimization of a fed-batch penicillin fermentation process. A customized float-encoding genetic algorithm is developed and implemented to a benchmark fed-batch penicillin fermentation process. Off-line optimization of the initial conditions and set points are carried out in two stages for a single variable and multiple variables. Further investigations with online optimization have been carried out to demonstrate that the yield can be significantly improved with an optimal feed rate profile. The results have shown that the proposed approaches can be successfully applied to optimization problems of fed-batch fermentation to improve the operation of such processes.

An online optimization procedure provides the parameters of a nonlinear battery model by taking into account a few minutes of measured current-voltage data. Within a defined range in terms of charge current, state of charge (SOC), and duration of charge and discharge events, the model is able to capture the relevant battery dynamics and predict the behavior for the next few minutes. From the battery behavior during specific events, the state of the battery can be revealed, which is defined as the state of function. Validation, which is carried out on measured current-voltage profiles, shows the accuracy of prediction during the high-rate partial SOC operation. Even with the data measured during a city drive within a microhybrid electrical vehicle, the method is able to predict the voltage level during high-rate discharge pulses (cranking). Index Terms-Battery model, charge acceptance, high-rate partial state-of-charge (HRPSOC), hybrid electric vehicles (HEVs), lead-acid battery, state of function (SOF).

Emergency voltage control problems in electric power networks have stimulated the interest for the implementation of online optimal control techniques. Briefly stated, voltage instability stems from the attempt of load dynamics to restore power consumption beyond the capability of the transmission and generation system. Typically, this situation occurs after the outage of one or more components in the network, such that the system cannot satisfy the load demand with the given inputs at a physically sustainable voltage profile. For a particular network, a supervisory control strategy based on model predictive control is proposed, which provides at discrete time steps inputs and set-points to lower-layer primary controllers based on the predicted behavior of a model featuring hybrid dynamics of the loads and the generation system.

We study the contextual linear bandit problem, a version of the standard stochastic multi-armed bandit (MAB) problem where a learner sequentially selects actions to maximize a reward which depends also on a user provided per-round context. Though the context is chosen arbitrarily or adversarially, the reward is assumed to be a stochastic function of a feature vector that encodes the context and selected action. Our goal is to devise private learners for the contextual linear bandit problem. We first show that using the standard definition of differential privacy results in linear regret. So instead, we adopt the notion of joint differential privacy, where we assume that the action chosen on day t is only revealed to user t and thus needn't be kept private that day, only on following days. We give a general scheme converting the classic linear-UCB algorithm into a joint differentially private algorithm using the tree-based algorithm [10, 18]. We then apply either Gaussian noise or Wishart noise to achieve joint-differentially private algorithms and bound the resulting algorithms' regrets. In addition, we give the first lower bound on the additional regret any private algorithms for the MAB problem must incur.

Ripple correlation control (RCC) is a fast, robust online optimization technique. RCC is particularly suited for switching power converters, where the inherent ripple provides information about the system operating point. The present work examines a digital formulation that has reduced power consumption and greater robustness. A maximum power point tracker for a photovoltaic panel demonstrates greater than 99% tracking accuracy and fast convergence.

In predictive control, a quadratic program (QP) needs to be solved at each sampling instant. We present a new warm-start strategy to solve a QP with an interior-point method whose data is slightly perturbed from the previous QP. In this strategy, an initial guess of the unknown variables in the perturbed problem is determined from the computed solution of the previous problem. We demonstrate the effectiveness of our warm-start strategy to a number of online benchmark problems. Numerical results indicate that the proposed technique depends upon the size of perturbation and it leads to a reduction of 30-74% in floating point operations compared to a cold-start interior-point method.

Predictive controllers are usually implemented as part of a hierarchical structure of the process operation where the Real Time Optimization (RTO) and the Model Predictive Control (MPC) are executed in separated layers. Here, it is proposed to use the gradient of a convex function that links the optimizing targets to the MPC controller. The convex function is a nu+1 dimensional paraboloid of the process inputs, with the minimum corresponding to the input targets resulting from the RTO routine. For this MPC controller, the infinite prediction horizon and terminal state constraint are considered, and it operates according to the zone control strategy. Simulation results of a linear system are provided.

The paper presents a new approach to multiple sensor bias estimation. It is applied to a practical example: correcting radar biases for ATC applications. A novel procedure to organise and process measurements is proposed and compared with classical approaches, which are shown to achieve inferior performance. It is shown how the exploitation of all available information allowed by the proposed data arrangement improves the accuracy of estimates. Besides, efficient implementations are developed and a cost-benefit analysis of the main alternatives for this problem is presented in the results.

Low resource utilization in cloud data centers can be mitigated by overbooking but this increases the risk of performance degradation. We propose a three level Quality of Service (QoS) scheme for overbooked cloud data centers to assure high performance QoS for applications that need it. We design a controller that dynamically maps virtual cores to physical cores and whenever feasible shares physical cores among applications. Our evaluation based on real cloud applications and workloads demonstrates that performance isolation can be achieved for critical applications while overall utilization is increased thanks to overbooking.

Superfluid helium is used in the cryogenic circuit that cools down and stabilizes temperature of more than 1600 high performance, main superconducting magnets of the Large Hadron Collider (LHC)-the new particle accelerator at European Organization for Nuclear Research (CERN). This paper presents a simulation study of the application of Nonlinear Model Predictive Control (NMPC) to the Superfluid Helium Cryogenic Circuit. First, the new first principles, distributed parameter model of the circuit to be used in online optimization is reviewed. Then stabilization of the superconducting magnets temperature using NMPC based on the model and Continuation/Generalized Minimum Residual (C/GMRES) algorithm is described. Finally the small computational cost of C/GMRES solution/approximation method and resulting real-time feasibility are highlighted.

This paper addresses the problem of optimal control using search trees. We start by considering multi-armed bandit problems with continuous action spaces and propose LD-HOO, a limited depth variant of the hierarchical optimistic optimization (HOO) algorithm. We provide a regret analysis for LD-HOO and show that, asymptotically, our algorithm exhibits the same cumulative regret as the original HOO while being faster and more memory efficient. We then propose a Monte Carlo tree search algorithm based on LD-HOO for optimal control problems and illustrate the resulting approach’s application in several optimal control problems.

Superfluid helium is used in the cryogenic circuit that cools down and stabilizes temperature of more than 1600 high performance, main superconducting magnets of the Large Hadron Collider (LHC)-the new particle accelerator at European Organization for Nuclear Research (CERN). This paper presents a simulation study of the application of Nonlinear Model Predictive Control (NMPC) to the Superfluid Helium Cryogenic Circuit. First, the new first principles, distributed parameter model of the circuit to be used in online optimization is reviewed. Then stabilization of the superconducting magnets temperature using NMPC based on the model and Continuation/Generalized Minimum Residual (C/GMRES) algorithm is described. Finally the small computational cost of C/GMRES solution/approximation method and resulting real-time feasibility are highlighted.

In most real-world settings, a transportation plan requires modifications during execution. A thorough evaluation of transportation planning methods thus requires testing and comparison in a dynamic environment. We give conditions on a simulation environment that follow from this requirement, and propose a multi-agent simulator meeting these conditions. In addition, we propose a new measure that captures robustness in such dynamic settings. The multi-agent simulator and the robustness measure are then used to compare three different transportation methods (two multi-agent planners and one online optimization approach) in settings with release time uncertainty and truck breakdown incidents.