Linear Quadratic regulator Research Papers

The explicit linear quadratic regulator for constrained systems

2002, Automatica

We present a technique to compute the explicit state-feedback solution to both the xnite and inxnite horizon linear quadratic optimal control problem subject to state and input constraints. We show that this closed form solution is... more

descriptionView Paper arrow_downwardDownload

Backward stochastic differential equations and applications to optimal control

by Xuerong Mao

1993, Applied Mathematics & Optimization

Backward stochastic differential equations (in short BSDE's) were first introduced by J.M. Bismut in 1973 [7] as equation for the adjoint process in the stochastic version of Pontryagin maximum principle. Pardoux and Peng [48] generalized... more

descriptionView Paper arrow_downwardDownload

Combining Reinforcement Learning And Optimal Control For The Control Of Nonlinear Dynamical Systems

by Dr Ekaterina Abramova

2016, PhD Thesis

This thesis presents a novel hierarchical learning framework, Reinforcement Learning Optimal Control, for controlling nonlinear dynamical systems with continuous states and actions. The adapted approach mimics the neural computations that... more

Fig. 1.1 Example Optimal Control Problems. (a) From an infinite number of possible movement trajectories, humans and animals show stereotyped trajectories with smooth velocity profiles [110] and the control of redundant biomechanical systems has been shown to fall under optimal control framework [262]. (b) Processes such as manufacturing [5], assembly line and factory floor control require optimality in decisions on time consumption, tools/materials inventory, workers assignment and other factors for profit maximisation. (c) Optimal control of drug delivery is essential for currently crude/ad-hoc methods of anaesthetic drug administration during operations and insulin injections for patients with diabetes [39]. (d) The application of suitable control algorithms allowing smooth (optimal) movements of prosthetic devices can enable ease of mobility to impaired individuals [196] [215]. (e) Optimal control algorithms are applied to aerospace control problems such as those encountered in piloting: steering [191], landing [159], trajectory planning 208] [230] and in military such as missile tracking (e.g. find rocket thrust trajectory to achieve maximum altitude) [118]. (f) Space explorations and construction/repair of international space stations requires optimal actions of robotic devices.

Fig. 1.2 Closed Loop Control Frameworks. Both reinforcement learning and optimal control frameworks operate in a closed loop feedback system. (a) Reinforcement learning involves a high-level agent that makes decisions on choosing a discrete action, a, applied to the world advancing it to a next discrete state, s, giving an observable immediate reward, r. Both of these are fed back into the agent, which evaluates the effect of its chosen actions on the world for maximisation of a long term reward (aggregated, discounted future rewards) and learns appropriate state-action mapping. (b) Optimal control uses an analytical solution for linear systems to calculate continuous control signal u, which is applied to a plant in its current state x advancing it to the next continuous state x’ and producing a continuous cost, /. Both the cost and the new state are fed back into the system, which in turn applies the necessary control given the current state, in order to achieve the desired reference point (goal state).

Fig. 1.3 Open vs. Closed Loop Control Systems. (A) Open control framework — a control (input) signal is applied to a system without observing outcome of an action (e.g. a heater warming up a room without having a temperature sensor). Any disturbances are unaccounted for in advance. (B) Feedforward control framework — an open loop control is applied to a system, while any disturbances are observed and accounted for in advance of them affecting a system (e.g. an increase in an amount of heat applied by a heater when a signal of an opened window is received). (C) Feedback control framework — a closed loop control operates on the basis of sensing deviations from a desired reference point (e.g. a continuous room temperature reading supplied to a heater, which adjusts an amount of heat applied to a system based on a sensor reading). Any disturbances are unaccounted. (D) Feedforward and feedback control framework — a combination of both feeding back of a state error and an ability to account for any disturbances to a system before they take effect.

tt Aw), where pt is the mean and ¥ is the variance-covariance matrix of z, and where tr denotes a trace

Fig. 1.4 Agent-Environment Interaction in reinforcement learning. A closed-loop control applied to a system formu- lated in a Markov Decision Process framework, capable of learning optimal actions from interaction with an unknown environment. This is made possible solely by following an objective of long term (observed) reward maximisation by the agent. Note: adapted from Sutton & Barto book [239]. - Policy Search Algorithms search for an optimal policy directly using optimization methods.

'A linear system satisfies the Superposition Principle which states that the net response of two or more stimuli at a given point is the sum of each individual stimulus — i.e. where the additivity f(x + u) = f(x) +f(u) and homogeneity properties f(a@x) = af(x), f(Bu) = Bf(u) hold true for a function f(a@x + Bu) = af(x) + Bf(u). >An example of a nonlinear system of the form f(x,u) = A(x)x + B(x)u is f(2x,2u) = A(2x)2x + B(2u)2u 4 2f(x,u).

Fig. 2.1 Robotic Arm System. The 2 degrees of freedom planar robotic arm is modelled as 2 weightless rods of lengths 1, ,l2 and masses mj, mz centred near the middle of each link. The shoulder joint is ’attached’ to a point, with a planar freely moving shoulder and elbow links defined for angels positive to the right @ = (0), 0)" and restricted to a range (0,7) € R*.

Fig. 2.2 Cart Pole System. The cart-pole system consists of a cart moving on a track possessing a mass m, and a velocity V. = at a location (x,,0), a freely swinging pendulum of length / with a mass mp, angle 6 defined positive to the right, angular velocity 6 and tangential velocity V, defined at the right angle to the pendulum tip point (x,,y,). The admissible control (force) is applied to the front / rare of the cart, F € [—20,20] N. The cart-pole system has a stable equilibrium for pendulum hanging down 0 € {—2, 7} and an unstable equilibrium for pendulum being upright 6 = 0. ‘he cart-pole system depicted in Figure [2.2] is a standard, yet challenging benchmark in moder ontrol theory. The task consists of actively balancing the pole in an upright position (unstabl quilibrium), with the cart stopping in a pre-defined position on the track (z = 0), with limited contrc orces applied to the cart laterally along the horizontal plane. Since the rotation of the pole cannc e controlled directly the cart-pole is an underactuated system, making standard nonlinear contrc schniques ineffective [170].

Fig. 3.1 RLOC Conceptual Overview. A snowboarding task is used to i actuators (low-level muscle activations controlling the board/body — e.g. local a high-level reinforcement learning policy (comprised of high-level symbo optimal feedback controllers) are engaged ic actions — "Glide", "Lift", "Fly", "Land The continuous actuator state space is mapped into an RL framework using feature mapping and each low-level control corresponds to a high-level symbolic RL action. The type of the low-level controller is a design choice and in theory any permitted by the ability of the high-level RL feedback controller could be implemented (e.g. LQR, PID, H.. controllers), policy to learn to choose the best actions for the (discrete/continuous) sym| task dynamics are unknown, the local models may be learnt and moderated online through repeated trials (local dynamics learning). The RLOC algorithm learns to smartly re-use available low-level controllers, producing a global action-to-state bolic RL state space. In the case where t! lustrate RLOC’s architecture. The agent’s by ") er ne mapping, while maximising the long term reward of a given task. The learning loop between the low-level control and t high-level action selection is closed by mapping the accumulated actuator cost, J, of a chosen low-level controller acquired for controlling the system to the high-level reward signal, which drives the agent’s learning (note: cost = —reward). ne

Fig. 3.2 Hierarchical Organisation of Action Selection. Research in cognitive psycho represents sequential tasks with hierarchically organised symbolic tree-like processing units [54}{56] e.g. coffee-making task (figure adapted from |[ 41]}); and (2) solves decision-making tasks with RL algorithms ogy suggests that the brain: (1) 207 219}. On the lower level of the Central Nervous System (figure adapted from [249]), the spinal cord provides the first level of feedback processing (primarily performed by — feedback pathways) and the linear OFC theory predicts well the activation patterns of muscles in reaching movements [222}[223]. The control of actuators appears to be represented with linear motor primitives in the spinal cord [185)[257].

Fig. 3.3 RLOC Architecture. A top-level model-free RL agent operates in the symbolic space of mapped state features (x) and actions (a). It selects between its symbolic actions a), i=1,...,Nq each linked to an underlying closed-loop feedback controller c“ operating on the actuator space in continuous domain, for i = 1,...,%¢, where ng is the number of actions available to the algorithm. The agent learns a mapping between a problem defined state space and RL actions by evaluating outcomes of each controller, which generates accumulated costs that are mapped into the RL framework driving the agent’s learning. The accumulated costs are attributed to a state-action pair (s,a), thus allowing the agent to chain sequences of successful actions for global use, repeating exploration over many RL epochs.

Fig. 3.4 Arbitrary Problem Design. An arbitrary control problem is used to illustrate a mapping of choice between a low-level actuator state space 2 and a high-level symbolic state space .7, here chosen as (x) : IR‘ + N+ (as opposed to (x): R‘ — R). This mapping is performed by choosing a number of state dimensions, ny (here chosen as ny = 2 and denoted by z),z2) to be discretised into an (un)even grid, creating {s(1) 52), ang 55) symbolic states. of these true control regions (red lines). est boundaries for engaging each controller in a given region of state space. This true hypothetical oundary is continuous (blue curves), however due to the discretisation of state space in formulating the lew problem, RLOC assigns actions to a gridded symbolic state space resulting in an approximation f these true control regions (red lines).

Fig. 3.5 Example State-Action Mapping. An arbitrary optimal control problem is considered, showing an state-action pee (learnt) mapping for (a) Naive and (b) RLOC control. Each square corresponds to a discretised symbolic state {s(1) 5). sltts)} used by the RL agent for assignment of high-level actions.

Fig. 4.5 Relative Cost Difference Results, RLOC vs. LQR/iLQR — Robotic Arm.) The RCD are reported, where a negative number indicates that RLOC excels in controlling the task, when compared to a benchmark policy. The solid red circle at the position (6), 42) = [1.3, 1.3] radians indicates the target state (ns = 15). (a) The accumulated costs are compared between RLOC and iLQR, revealing that RLOC is able to achieve near-optimal performance which is on par with the solution quality of the state-of-the-art iLQR algorithm. (b) The results of comparing RLOC vs. LQR show that RLOC outperforms LQR for the states with elbow angle 0) 4 1.3, while meeting the performance of LQR for the states where 62 = 1.3. This result reflects on nonlinearities of the arm kinematics, which are contained in the elbow angle only. Heterogeneous, Hierarchical Control of Systems with Known Dynamics

Fig. 3.6 Epoch Sample Path. Each epoch sample path (denoted in blue) begins with a randomly chosen symbolic state S; (states are denoted with a white circle) and an action a; (actions are denoted with a red circle) is chosen according to policy 2. This results in the environment transitioning into a next state s;;; and producing a reward 7;+; associated with the choice of action at time r. This is recorded as a triplet {7;+1,5;,a;} and the process is continued until an absorbing state (denoted with ’T’ in a green square) is reached, at which point a full epoch sample path is obtained. Note a full MC simulation involves obtaining many such epochs of varied duration, each depending on the number of steps taken until an absorbing state is reached. The policy can either be improved at the end of each epoch or at the end of a number of epochs (termed a batch). Note: figure adapted from Sutton and Barto [239]. states (i.e. it does not bootstrap). The Monte-Carlo reinforcement learning algorithm is episodically iterative: it improves its policy 2% on an episode-by-episode basis using epochs — trajectory roll outs which record the {7;+1,5;,a;} triplets until an absorbing state is met (see Figure [3.6] for a description of each epoch). MC typically employs €-soft (i.e. @(s,a) > 0 Vs € Y,a € A(s)) policies, which allow both exploration — selecting an action deemed less successful than others, and exploitation — selecting an action which gives the highest long term reward — the greedy action. The MC algorithm is expected to converge to the true solution with a convergence rate of O (nz) based on the Law of Large Numbers, where n 7 is the number of MC simulations (epochs). Hence if one wishes to double MC result accuracy the number of simulations needs to be quadrupled.

Fig. 3.7 Overview of a PID Controller in Discrete Time Setting. The PID control involves three control parameters (P, I and D) and a measured output variable at each time step k only, hence it does not require knowledge of the system dynamics in order to control it. At each time step the error variable, e(k) = y(k) — x*, is calculated and utilised in three different ways: (1) the current error e; is weighted by proportional term K,,, (2) the history of past errors to up e;, is summed up and weighted by the integral term K; and (3) the derivative of the error over the last time step is weighted by the derivative term Ky. These three components are summed up to produce an overall plant control ux. The PID control applied to decoupled MIMO systems aims to minimize the error e(k) € R‘ at each ‘ime step k between the output y(k) € R¢ and a prescribed target state x* € R° ("set point"), calculated as e(k) = y(k) — x*. The control signal at each time step u(k) is calculated using the error variable utilised in three different ways: (1) the proportional error i.e. current error e(k), (2) the integral error .e. the sum of the history of past errors up to e(k) (i.e. up to time step k) and (3) the derivative error, .e. calculated over the last time step. These three components are weighted by the Kp, K;, Kg € ie proportional, integral and derivative gain parameters respectively (tuned by the user for each system) and are summed up to produce an overall plant control u(k) at time step k (see Figure|3.7).

Fig. 4.1 System control gain matrices of the Robotic Arm and the Cart Pole tasks. The system control gain matrice: do not show large variation with respect to different linearisation points, hence this figure is a good representation of the system control gain values of the robotic arm and cart pole linearised systems.

Fig. 4.2 Feature Mapping in RLOC Framework for Benchmark Tasks. The feature mapping 6(x) : R! > -Y for both tasks is visually displayed for the robotic arm and the cart pole tasks, where an evenly spaced grid of symbolic states sl), i=1...,ns is obtained. The linearisation points x) i = 1,...,M4gr for LQR controllers (note u\ = 0) are depicted with green circles, chosen to be equally spaced in state space for the simulations carried out in this chapter. The dashed red line symbolises a stencil for the dimension of interest, along which the LQR controller placement varies. (a) Displays the robotic arm system, with the two discretised dimensions (61, 02), resulting in a total of n; = 36 RL states. (b) Displays the cart pole task, for which the most relevant dimensions (0,6) are discretised, resulting in a total of ns = 25 RL states.

of the shoulder reflects the fact that this angle does not contribute to nonlinearities of the system as per Equation pa. The control trajectories al) (k) are produced one per linearisation point, using the gain matrices LY? and the information about the current state x; according to Equation 2.2.11] The robotic arm is controlled to the target of x* = x* = 0, with parameters initialised as per Table The control trajectories are reported in Figure |4.3a]and the corresponding elbow angle trajectories at) (k),i=1,...,40,k = 1,...,300, are displayed in Figure 4.3b| This investigation shows that the target elbow angle 02 = 0 is achieved correctly but imprecisely, while the control size is reduced to near zero consistent with what is expected from an LQR controller. This means that even though the individual locally optimal linear LQR controllers are able to solve the robotic arm reaching control task, they do so sub-optimally due to the task nonlinearities. Fig. 4.3 Control Trajectories and State Produced for n, Linearisation Points — Robotic Arm. The control a) (k) for i=1,...,n~, n, = 40, and the corresponding elbow trajectories 6S € [0,2] are displayed. (a) Near optimal, smooth controls are produced ranging between [0.8,-10] N, which is biologically plausible, indicating appropriate cost function. The amount of initial torque applied to the elbow joint increases with the size of the angle, as expected. (b) The corresponding elbow angle @) is taken to near 0 target state as intended, indicating that the appropriate controls are applied to the system. The target state is not precisely achieved however and small oscillations can be seen indicating sub-optimality. task, they do so sub-optimally due to the task nonlinearities.

Fig. 4.4 Control Trajectories Produced for nz, Linearisation Points Grouped by Similarity - Cart Pole. The state linearisation points xj = 1,...,40 are equally spread along the pendulum angle dimension 6 € [0,2]. (a) The control solution for starting angles [0,1.3]rad is smooth and continuous, while guiding the cart pole to target, as evidenced by the reduction of control to near zero size (17 trajectories). The red colour indicates control produced for lower starting angle values — i.e. least effort required for stabilisation. (b) The LQR control produces inadmissible controls for starting pole angles [1.3,1.6] rad (3 trajectories). The solid squares indicate the point after which the control signals exploded and therefore the reported trajectories were truncated before this point. (c) Oscillating control produced for the angle range [1.6,1.9] rad, means that no solution was found (4 trajectories). (d) The control trajectories produced for the angle range [1.9,7] rad can be seen to destabilise quickly, producing oscillating behaviour with inadmissible control size (16 trajectories). Fig. 4.4 Control Trajectories Produced for n; Linearisation Points Grouped by Similarity — Cart Pole. The state space other than where they were obtained from in order to solve the global nonlinear task optimally. size. The difficulty in solving the cart pole task for angles beyond 4 -Q.3 rad is due to the fact that LQR is only optimal for linear systems and the nonlinearity of the cart pole task increases the further away from target the starting angle is. Hence simple stand alone LQR controllers are unable to solve the cart pole task unless the stabilisation is performed very near to the target state of the upright angle. This brings an interesting consideration of whether the RLOC algorithm can successfully utilise LQR controllers with linearisation points beyond [0,0.3] rad and potentially re-use them in areas of states space other than where they were obtained from in order to solve the global nonlinear task optimally.

Fig. 4.7 Relative Cost Difference Results, RLOC vs. LQR —- Cart Pole. The Relative Cost Difference (RCD) are reported, where a negative num policy. The solid red circle at the compared between RLOC and i ber indicates that RLOC excels in controlling the task, when compared to a benchmark position (6,0) = [0,0] indicates the target state (ns = 13). (a) The accumulated costs are LOR, revealing that RLOC is able to outperform by up to 36% (and at worst equal) the optimal performance of the nonlinear iLQR control benchmark. (b) The results of comparing RLOC vs. LQR show that RLOC outperforms LQR for 18 out of 25 states by up to 74% for the regions of state space which are most nonlinear. The middle part (the coarse ’central band’ @ = 0 i.e. pendulum already upright but containing some amount of velocity) can clearly be seen to be learnt suboptimally when compared to the LQR controller. This result reflects the fact the LQRs are able to control the cart pole task at the vicinity of the target (even when some velocity is present). some PID controllers to operate on areas of state space. as iLQR) for the initial pendulum configurations of near upright (target) position containing initial velocity. Since these states are not as nonlinear as the rest of the state space, an LQR controller inearised at the top Xo = 0 (optimal for linear systems) is able to better solve the task for that area of state space, compared to the global solution found by RLOC policy*| One of the low-level controllers provided to RLOC was an LQR controller linearised at xo = 0. The result of the Figure/4.7b}means that this LQR controller was not selected to be activated in the central band region states when searching for the global solution, even though it would have been locally optimal. Hence RLOC finds a global yet sub-optimal) solution for the entire state space when controlling the cart-pole to target upright unstable equilibrium state. The RLOC algorithm did benefit from the inclusion of the PID controllers at the low-level of actuation when solving the cart pole task, since the high-level policy #(s) did assign some PID controllers to operate on areas of state space.

Fig. 4.8 Learning Curves — Cart Pole. Average results for 5 learning trials are reported. Note LQR/AALQR cost = —reward (a) The average error of (0,6) state values are the lowest for the RLOC control solution, Mz - While the errors for the iLQR tend to show non-stationary control solutions (@ of up to 1.5 rad). The LQR result on the other hand shows that on average it struggles to bring the cart pole to target, missing it by up to | radian (i.e. not solving the task) and supplemented by the standard error of the mean (SEM , pastel pink band). (b) The average sample of direct reward experienced by RLOC algorithm during learning (red curve) is reported against the benchmark of iLQR (average cost — green solid line) and linear optimal controller LQR (average cost — black solid line). The results show that the direct cost sample reduces as the simulation progresses indicating learning. A good solution quality is achieved after 500 epochs, where the iLQR benchmark starts to be outperformed. dictate how to best combine these for global use.

Fig. 5.1 Naive Trajectories Sub-Sampling — Cart Pole. Example depicting sampling of the cart-pole collected naive trajectories Y in order to obtain ny sub-trajectories of length h = 7, using 2 dimensions of the state trajectory (10), where / = 1,...,ny, the pendulum angle @ (x-axis) and pendulum angular velocity @ (y-axis). The figure shows how ten sub-trajectories (nz = 10), ist. arse, k ali. p) baer each of length # = 7 are sampled in the region close to the first (i = 1) linearisation centre € (1) (x\!) 0,u, 0) (red dot), p = 1,...,n4 sub-trajectory number. The sub-sampling boundaries are expertly defined for each task (red rectangle). Sub-trajectories that fall outside of the specified boundaries are not considered (black dots), while those that fall inside are suitable hence sub-sampled (each sub-trajectory is uniquely coloured). The sampled sub-trajectory entries do not repeat and all points of the sub-trajectories are within a bounding box centred at the linearisation centre. Reinforcement Learning Optimal Control for Unknown Dynamical Systems

Fig. 5.2 RLOC Performance vs. Number Of Controllers — Cart Pole. The performance (reported as average cost) of RLOC control solution learnt as a function of individual controllers (data points) with respect to engineering state of the art solutions (solid lines: PILCO — orange line, LQR — black line and NNOC — blue data points). RLOC — red data points with learning trial-to-trial variability displayed as mean and standard error of the mean of 500 learning trials, shows performance for a system with both high-level and low-level learning. The results show that on average RLOC outperforms LQR with 2 or more controllers, iLQR with 4 or more controllers and matches/outperforms the average cost of PILCO with 7 or more controllers respectively. Note the SEM of iLQR and PILCO algorithms is small, however does exist as pointed out by the zoomed in sections. Reinforcement Learning Optimal Control for Unknown Dynamical Systems

Fig. 5.3 Tukey boxplot of RLOC performance for varied number of controls — Cart Pole. The performance of RLOC control solution learnt as a function of individual controllers (data points) with respect to engineering state of the art solutions (solid lines), reported using box plot and whiskers diagram, in order to use non-parametric representation of the data. The Tukey boxplot allows to observe the following statistics for number of controllers al’) ji=1,...,Na,: lower quartile (q1), median (q2, black line) and upper quartile (q3). The whiskers are calculated using interquartile range (IQR): upper whisker data € (q3, q3+1.5 x IQR); lower whisker data € (q1, ql—1.5 x IQR); data outside of the whiskers is reported as outliers (black dots). RLOC’s performance for solving the cart-pole task stabilises with the increasing number of controllers and for ng > 6 the interquartile range and the median of the average RLOC costs are below the average cost of PILCO and iLQR (orange and green lines respectively)). Note the SEM of iLQR and PILCO algorithms is small, however does exist as pointed out by the zoomed in sections.

Fig. 5.4 Histograms of RLOC Performance - Cart Pole. (a) Depicts 10 histograms, 100 bins each, for RLOC av. costs when solving the CP task with al) ,i=1,...,10, LQR controllers. The av. costs for 500 trials of each controller outperform iLQR and PILCO as the number of controllers increases, where on average for ng >6 controllers the av. RLOC costs outperform those of iLQR 93% and PILCO 75% of the time. (b) Depicts 10 histograms, 100 bins each, for NNOC av. costs when solving the CP task with al), i=1,...,10, LQR controllers. We show that the average costs for the 500 trials of each controller, do not outperform either iLQR or PILCO, with at most being 6% better. Reinforcement Learning Optimal Control for Unknown Dynamical Systems

Fig. B.1 Value Function — Robotic Arm. An example robotic arm final policy is evaluated for ng = 5 controllers (labelled A-E) and the cost-to-go function is displayed with superimposed state space to control mapping. The high-level policy learns the area of state space (i.e. state) to which each controller should be assigned to in order to solve the robotic arm inverse kinematics task. A smooth convex value function is observed demonstrating that RLOC policy is able to solve the task from any area of state space, with the most nonlinearity observed at the fully outstretched arm (elbow joint 62 = 0).

Fig. 5.6 Value Function — Cart Pole. An example RLOC value function (cost-to-go) plot depicting state-space to contro mapping, reflecting the algorithm’s learnt choice of actions when a particular area of space is encountered during task execution. The plot demonstrates the cost of the learnt deterministic control for starting in any area of state space (discretisec to 10° states, learning for nz = 2000 epochs, using ng = 8 actions labelled A - H, any of which can be assigned by the high-level policy to any of the ns = 49 available symbolic states) and controlling the task to unstable upright equilibriur end-point, with warmer areas indicating a higher incurred cost. The action-state space assignment, learnt by the high-leve policy, is shown with letters superimposed onto an area of state space where the action is active, while the shape of the displayed cost function is dictated by the underlying dynamics of the task. The value function largely displays symmetry) reflecting the periodic nature of the task, however the learnt high-level RLOC state-action mapping (and hence the displayec cost-to-go) is not exactly symmetrical due to the nature of the abstract learning for the global solution. Reinforcement Learning Optimal Control for Unknown Dynamical Systems

Fig. 5.7 Action Sequences of an Example Learnt Policy — Cart Pole. The action sequences obtained during control of the cart pole to upright target from 100 start states while using the same policy as used to obtain the Value Function plot. Each row within the sub-plot indicates an action (labelled A-H) which, if chosen, is marked with a black dot. Each column corresponds to the cart-pole moving to a new symbolic state, where a new action (which may be the same) is chosen. Note the amount of switches depends on the cart-pole trajectory through state space, since the policy is allowed to engage a new controller once the continuous actuator state enters a new symbolic state. The figure shows that RLOC’s high-level policy dictates the linking of actions into sequences and demonstrates to not only be able to reuse low-level actions, but also to learn reusing whole concatenated action sequences for solving the task. Mostly, topologically conjunct regions of state space produce similar patterns of activation, which are clearly correlated with regions of the value function, while the switching in the action patterns can be seen to occur as the value function abruptly changes. Observing these patterns of higher level action organisation and using analogy to language sequences with similar structure (but different specific meaning), these patterns may be referred to as "action grammars". This suggests that biological learning of complex motor control tasks may take the form of concatenation of high-level controllers into complex action sequences. to the optimal value function obtained using Dynamic Programming method that has full knowledg of the dynamics (see Fig. 2a, M. Deisenroth et al., [[70]) and exhibits the expected discontinuity nea the central diagonal band. The band is given by t hose states from which a small cost is incurred fo bringing the pendulum up to the unstable equilibrium target without dropping it first. However onc the starting angle position goes beyond a critica point, a higher cost is incurred for dropping th pendulum (regardless of the force applied to the cart) and then being swung back up. The cart-pole 1 an underactuated trajectories, thus system, which means that the po e cannot be directly controlled to achieve arbitrar for the majority of the state space first enough momentum needs to be built up in th swung up stage before stabilisation can begin (while both of these phases accumulate cost).

Fig. 5.8 Example State Trajectories — Cart Pole. The figure depicts 100 example state trajectories in the state space of the cart-pole following the completion of a single learning trial. The learnt policy is able to successfully control the given task to the target (upright and stationary pendulum and the cart stopping in a pre-defined position on the track, z = 0) from 100 equally spaced starting positions (solid circles) located on a grid for comprehensiveness of admissible state space testing. The area of space containing the target is shown in a zoomed box section and demonstrates that RLOC algorithm solution provides a policy which controls the cart-pole to the target using smooth, precise, near-optimal control. RLOC algorithm successfully stabilizes the pendulum (upright and stationary) on the cart, stopping in a pre-defined position on the track, from all starting states.

Fig. A.1 Double Inverted Pendulum on Cart System. The DIPC is modelled as having two solid rods, where centre of mass is assumed to be in the middle of each rod. The cart is required to stop in a pre-defined position on the track at the origin (z = 0 meters).

Fig. A.2 Total Energy for Double Inverted Pendulum System. The tip of the second link is dropped from an angle of + (while 6; = 0) rad and the behaviour of the system is observed over a period of 5 seconds (with equivalent results for a period of 5 hours). The simulation was carried out using an ODE45 Matlab function with the absolute and relative tolerances of 10~°. (a) Displays the periodic changes in kinetic energy of the system. (b) Displays the periodic changes in potential energy of the system. (c) Displays the DIPC state trajectory generated by ODE4S for a 5 sec simulation. (d) Displays the stable non-zero Total Energy (mean 2.942, variance 6.58 x 10~!7).

Fig. A.3 Physical Air Muscle Robotic Arm. The physical air-muscle based robotic arm under the effects of gravity.

Fig. A.4 Simple example — system control matrix entries. Each entry of the system matrix A specifying the passive dynamics of the arbitrary task are designed using 5 Gaussian peaks. The points in the 3-D plot correspond to area of state space x = (x;,x2)! € [—10, 10]. (a) Element Aj;. (b) Element Aj. (c) Element A>,. (d) Element A».

Fig. A.5 Simple example — control matrix entries. Each entry of the system matrix B specifying the passive dynamics of the arbitrary task are designed using 5 Gaussian peaks. The points in the 3-D plot correspond to area of state space x = (x1,x2)" € [—10, 10]. (a) Element B,;. (b) Element B12. (c) Element B2;. (d) Element B29.

’xponential Mapping The cost-to-go function, v(x,t), is exponentially mapped into a reward-to-go unction, z(x,t) (allowing to transform the Hamilton-Jacobi-Bellman Equation|A.4.8}into a form where he dependence on the value function is linear). Given the particular quadratic penalty on control A.4.3), the linearity of the Bellman equation only holds if B(x) = C(x), thus implying that noise acts n the same sub-space as control.

Fig. B.2 Action Sequences of an Example Learnt Policy — Robotic Arm. An example robotic arm final policy 2(s) utilising ng = 5 controllers is evaluated with the quadratic cost to produce characteristic activation patterns. Each starting state is evaluated for T = 3 sec and the high-level action switches are marked with a black dot plotted vs. horizontal axis of the action switch number. The observed patterns of higher-level action organisation suggest that biological learning of complex motor control tasks may take the form of concatenation of high-level controllers into complex action sequences. are re-used by the RLOC policy when solving the task from areas of actuator state space with similar cost-to-go function. Action "C" can be seen to be preferred when brining the arm to the target, which interestingly is not the one with the shortest Euclidean distance from the target, showing that RLOC finds a globally near optimal policy rather than using an LQR controller typically offered by the control theory (i.e. one linearised at the target state). The results suggest that RLOC is able to re-use both the low-level LQR controllers and the whole concatenated action sequences for adjacent regions of state space. These correlate with regions of the value function from Figure grammar’ like rules [52]. [B.2|and suggest action BWianralD Qlhehaws hat fhe #ahntic. asa wee cnpeoehlles: canteen lad fans all af the ofdaeing erated

Fig. B.3 Example State Trajectories — Robotic Arm. The example state trajectories for the robotic arm obtained using deterministic learnt policy for control #(s). All of the 100 starting states are successfully controlled to the target of x* =[4, 4, 0, 0] ina smooth manner. In contrast to the cart-pole system, the robotic arm was controlled to perform planer reaching movements, therefore the arm is not requiring the arm to work against gravity, hence RLOC can be seen (cf. zoomed in box section) to control both the shoulder and elbow to a near stop position, as expected.

Table 2.1 Robotic Arm Simulation Parameters. 4.2 Free Swinging Pendulum on a Moving Cart

Table 2.2 Cart Pole Simulation Parameters. The control task of the cart pole represents starting from any state configuration x!” = [z,z,0,0]" where z € [0] is the position of the cart, z € [0] is the velocity of the cart, @ € [—7, 71] is the pendulum angle, 6 € [—1.57, 1.57] is the velocity of the pendulum (with 6 = 0 defined to be the upright, unstable equilibrium) and controlling the cart-pole to the target state of x* = 0 by applying admissible controls (forces) u € |[—20, 20] N to the front / rare of the cart as dictated by a control policy 7.

Table 3.1 Reinforcement Learning and Optimal Control Comparison.

Algorithm 2 An €-Greedy On-Policy Monte Carlo Pseudo Algorithm For simplicity, stationary policies are chosen for RLOC, where this mapping depends only on the current state. So for an observed state s and action a, %(s,a) € [0,1] gives the probability of choosing a in s. The critic only reinforcement learning uses the estimated Q-values to determine the policy. In particular the €-greedy policy uses estimates O(s, a) Vs € S,a € As) to update the policy 1(5:,4;) = Pr(a;,|s,;) =1—e+ wl iff a, =argmax, O* (s;,a) and Ww] otherwise, € € (0, 1].

Table 4.1 PID Parameters of RLOC Simulation.

Table 4.2 Pre-Defined (Fixed) Linearisation Centers of LQR Controllers in RLOC Simulation. igh-Level Reinforcement Learner The high-level reinforcement learner used in this chapter 1 ‘scribed in detail in Subsection]3.3.2|and the exact values of the RL parameters used in the simulation ‘RLOC algorithm are shown in Table|4.3] The reason for the specific choice of each parameter of th lonte Carlo algorithm are described below: (a) a low epsilon, €, value is used to allow a restrictes ‘gree of exploration, hence the epsilon decay constant, V, is also chosen to be low to allow for a slov te of decay; (b) a high discount factor gamma, y, makes the algorithm far-sighted, allowing it t

Table 4.3 ¢-Greedy On-Policy Monte Carlo Learning Parameters. In the RLOC control framework, the high-level policy learning is driven by the observation of low-level rewards r(t) accumulated for selecting a particular action a(t) in a given state s(t), each linked to a continuous low-level controller (PID, LQR), defined by a linear model .4 (i) = I,...0 fg It is important for the high-level learner to be able to compare the choice of actions and the observed rewards across state space .”, therefore the same quadratic cost based on Equation|2.2.4]is used across the continuous state space .2”, penalising the state deviation from the target and the amount of control used at each time step k. The accumulated quadratic cost of using low-level controller c“) engaged by selecting action a; in state s; is recorded as negative reward r;+, and is calculated using take future rewards (those further along the episode) more strongly into account when calculating the return; (c) a low learning rate, @, is used in order to reduce oscillations in the estimated action-value function from one iteration to the next; (d) the learning rate decay, , is chosen to be higher than epsilon decay rate v, to allow for a trade-off between the rates of decay. All of the above values have been found by running a small number of simulations with different parameters and keeping to the standard values in the literature [240].

Table 4.4 Simulation Parameters — RLOC. The specific task parameters of the robotic arm and the cart pole tasks are shown in Table |4.

Table 4.5 Simulation Parameters — LQR / iLQR.

Table 5.2 Simulation Parameters - RLOC (unknown dynamics), iLQR (benchmark). The algorithm’s effectiveness is demonstrated on the standard yet difficult benchmark nonlinear cart pole control task (see Section|2.4). For the analogous robotic arm reaching task results see Appendix BI Section|B.1] The system dynamics are not available to the algorithm a priori. Instead RLOC learns the dynamics from naive control experience on-line, while the simulated environment is being perturbed by an additive white Gaussian noise. All the simulation parameters defined in Tables |4.3} 4.4 4.5}|5.1 are kept constant (except for where explicitly stated in Table|5.2). The difference for the cart-pole state matrix entries W;; = 30, W22 = 3 is due to the requirement of the cart to stop in a pre-defined position on the track (z = 0), therefore requiring a penalty on the cart position and cart velocity; the entries W33 = 2000, W44 = 200 were increased in order to reflect the primary requirement of the task for an upright and stationary pole. Ybtaining Benchmark Solutions The benchmark costs in for the following figures were obtainec y averaging the individual start state costs of attempting to reach the upright unstable equilibriun urget from 100 equally spaced start positions in actuator state space (see Figure |4.2b|for an example f 25 such starting positions — center of each state). The LQR infinity (black line) used as many umber of controllers as starting states (i.e. 100 linearisation points, located at the centre of eact tart state) and displayed as the mean average cost of the deterministic control of 1 learning tria

and in the more compact form, we have Pr =s

Table B.1 Randomly Selected LQR Linearisation Centers for an RLOC Simulation. Figure |B.1]shows the value function of the robotic arm planar reaching movement task obtained ising a deterministic learnt policy utilising 5 low-level LQR controllers, labelled A—E and obtained ‘ia random linearisation points placement. The state-to-control space mapping is shown with letters uperimposed onto each area of state space where the controller is activated by the high-level policy t(s) (ns = 36 number of states). The value function is obtained for a total of 1 x 10° discretised states, paced out equally on the grid. A smooth, convex value function is observed, with the smallest cost 4 ncurred for controlling the arm from states near to the target x* = [5 5, 0, 0] (least control distance The results for the robotic arm benchmark task performed with RLOC (see Algorithm |4) are reported here and supplement the more difficult benchmark cart-pole task results discussed in Section 5.3] The protocol for obtaining the following plots is identical to the one described in Chap er [5] The task parameters are specified in Tables [4.3] [4.4] [4.5] [5.1] are kept constant (except for where explicitly stated, see Table|5.2). The target state for learning to control robotic arm with unknown dynamics was selected to be x* = [F, 4, 0, 0]', which is different from that in Chapter /4|in order to investigate RLOC performance on a reaching task to a different end point. The admissible state space range for starting joint angles and angular velocities is 6/"" € (0, z], Or" € |-1.57, 1.52], i= 1,2. The results discussed in this section use a single deterministic learnt RLOC policy 2(s) (selected at random from 500 trials for ng = 5 controllers, random controller placement). The linearisation points placement of Chapter|5] was performed randomly, and the selected policy has linearisation centers 6“ of the low-level controllers located at (xi”” = [¥, @),0,0]", ui” = 0) displayed in Table[B. 1}

descriptionView Paper arrow_downwardDownload

Optimization of extraction of phenolics from Inga edulis leaves using response surface methodology

by Hervé Rogez

2007, Separation and Purification Technology

The extraction of phenolic compounds from Inga edulis leaves was optimized by the simultaneous maximization of the yield in total phenolics (TP), total flavanoids (TFA), and total flavonols (TFO), using the response surface methodology... more

descriptionView Paper arrow_downwardDownload

Discrete-time nonlinear HJB solution using approximate dynamic programming: convergence proof

by Asma Al-Tamimi

2008, Systems, Man, and …

Fig. 1. Flowchart of the proposed algorithm.

descriptionView Paper arrow_downwardDownload

Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control

by Asma Al-Tamimi

2007, Automatica

In this paper, the optimal strategies for discrete-time linear system quadratic zero-sum games related to the H-infinity optimal control problem are solved in forward time without knowing the system dynamical matrices. The idea is to... more

The optimal action dependent game value function O* (xx, Ux, WE) is equal to the game value function V*(x,) when the policies uz, wz are optimal. Then one has where H is the matrix associated with P that solves GARE, and is derived as

where v is the vectorized function in Kronecker products. Since the matrix H+, reconstructed from h;+) is symmetric, iterating on h; is equivalent to then using the Kronecker products, the least-squares (34) becomes

with sampling time T = 0.1. The disturbance attenuation is selected to be y = 1. The F-16 short period dynamics has three states given as x =[a q 6e]', where « is the angle of attack, g is the pitch rate and 6, is the elevator deflection angle. The discrete-time plant model of this aircraft dynamics is a discretized version of the continuous-time one given in Stevens and Lewis (2003). We used standard zero-order-hold discretization techniques

Fig. 4. Online model-free convergence of P; to P.

Fig. 6. Convergence of the control action network parameters.

descriptionView Paper arrow_downwardDownload

RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm for Continuous Control of Nonlinear Dynamical Systems

by Dr Ekaterina Abramova

2019, RLOC

Nonlinear optimal control problems are often solved with numerical methods that require knowledge of system's dynamics which may be difficult to infer, and that carry a large computational cost associated with iterative calculations. We... more

descriptionView Paper arrow_downwardDownload

Comparison of Pole Placement and LQR Applied to Single Link Flexible Manipulator

by UDIT SATIJA

2012

This work presents a comparative study of two different control strategies for a flexible single-link manipulator. The dynamic model of the flexible manipulator involves modeling the rotational base and the flexible link as rigid bodies... more

Fig.1 Rotary flexible link module[10] This system is similar in nature to the control problems encountered in large geared robot joints where flexibility is exhibited in the gearbox. A rigid beam is mounted on a flexible joint that rotates via a DC motor. The joint deflection is measured using a sensor. Variable springs and mounting points are also provided to change system parameters. Rotating the base of the beam causes the entire beam to oscillate due to the joint flexibility introduced by the springs. Different springs can be used, to represent different flexibility effects. Also, a weight can be added to the beam. An optical encoder attached to the shaft of the DC motor is used to measure the angular position of the shaft @(t). The joint deflection @ (t) is measured by an encoder located at the motor end of the beam. [1]. The objective is to design a feedback controller such that the tip of the beam tracks a desired command while minimizing link deflection and resonance in the system. The system is supplied with a state feedback controller and linear quadratic regulator.

Neglecting the armature inductance Lm relative to the armature resistance, the transfer function of the servomotor is obtained from the block diagram, and has the form of the following expression:[1

A schematic of the flexible link is shown in figure (3), where the base angle @(t) and the tip angular deflection «(t) relative to the undeformed link are the generalized variables.[2,10] Fig.3 Schematic of the flexible link[10].

State-Feedback is the most important aspect of modern control system. Using an appropriate state-feedback, unstable systems can be stabilized or damping oscillatory can be improved. The approach studied in this paper is known as pole-placement design. Pole-placement design allows the displacement of the poles to specified locations, provided the system is controllable. The control is achieved by feeding back the state variables through a regulator with a constant gain vector K. The controllability of the system is conserved after integrating this regulator. In fact, the pole-placement approach consists on replacing the input u(t) of the initial system by a new entry[4]:

Table (1) shows the numerical values of the different parameters of the system. by putting the above parameter in matrix A,B and C we get the following system matrix as:

Figure (5) illustrates a schematic block presentation of the controlled system. Fig.5 Controlled system via feedback regulator [11] Above gain matrix are obtained using MATLAB.

Fig.6 Matlab simulink of single link flexible manipulator. Figure (6) shows the implementation of the corresponding model using Matlab/Simulink. The input reference is a step input, and the output is to the total angular displacement of the tip of the beam, that is y ()}=@(t) + aft)

Figure (7) and eigenvalue of the original system shows that single link flexible manipulator system is initially unstable. A State-Feedback controller and a Linear Quadratic Regulator using the tip deflection feedback measured by a strain gauge, is proposed to minimize vibrations due to the flexibility of the link. The proposed controllers are simulated and implemented, and experimental results showed that based on the tip deflection feedback, dynamic vibration of the flexible link is minimized. The closed loop Eigenvalue of system using Pole placement and LQR techniques show that the resultant system is stable. Fig.8 output angle y(t) state feedback (Pole placement)

descriptionView Paper arrow_downwardDownload

Robust LQR Control for PWM Converters: An LMI Approach

by Abdelali El Aroudi

2009, IEEE Transactions on Industrial Electronics

A consistent framework for robust linear quadratic regulators (LQRs) control of power converters is presented. Systems with conventional LQR controllers present good stability properties and are optimal with respect to a certain... more

descriptionView Paper arrow_downwardDownload

Fast and robust laser stripe extraction for 3D reconstruction in industrial environments

by Julio Molleda

2012, Machine Vision and Applications

The use of 3D reconstruction based on active laser triangulation techniques is very complex in industrial environments. The main problem is that most of these techniques are based on laser stripe extraction methods which are highly... more

Fig. 1 3D reconstruction based on laser projection

The proposed laser detection method is shown in Fig. 3. Only if the difference is greater than the threshold, will the Center of mass be calculated for the window around the max- imum. Otherwise, the laser is not detected for that column. The choice of the threshold level depends on the expected strength of the laser. level for column j and can be used to test the difference with the maximum for that column. Then, the position of the laser peak, P;, can be calculated using (1) and (2), where Ti, j] represents the pixel value at row i column j, ijhax represents the row where the maximum light intensity is located for column j, W is the window size around this maximum to calculate the Center of mass, CM, and T is the threshold. The value of W depends on the number of pixels illuminated by the laser light. The value of W depends on the number of pixels illuminated

Fig. 4 Characterization of the laser stripe projected on the background, (a) Image with the laser stripe, (b) Laser peak detected and quadratic fit

Fig. 5 Laser stripe extraction procedure on the background

Fig. 6 Detected limits of the laser stripe projected on background for three images

Fig. 7 Projection of the laser stripe from the laser focus

Fig. 9 Bounding rhombus of the laser focus position

The split begins by fitting all the laser peaks with only one segment. If linear segments are used, the first and last aser peaks are used to calculated the coefficients of the seg- ment (two points). In the case of quadratic segments, the first, the last, and the middle laser peaks are used (three points). If Akima splines are used, five points are necessary: the first, the last, and three intermediate equidistant points between the endpoints. Whenever the largest error, €max, between the approximation function and the laser peaks is greater than a preselected threshold, €, the segment is split into two parts at the location where €max occurs. The splitting procedure proceeds recursively until €max does not exceed the threshold e.

Figure 12 shows the split steps for image Fig. 1 1c using linear segments. The first step (Fig. 12a) fits all the points using a single segment called 0. The second step (Fig. 12b) splits segment 0 where the largest approximation error, €max, occurs, creating two segments: 0.1 and 0.2. Step three (Fig. 12c) splits segment 0.1 into another two, 0.1.1 and 0.1.2. Segment 0.2 remains intact because all the points within this segment are fitted correctly. The procedure continues for three more steps (Fig. 12d—f) until €max does not exceed the threshold €, in this case one pixel.

spurious point as a real laser peak. However, despite this strategy, spurious points can appear in the laser peaks, as can be seen in Fig. 11b, c. However, these spurious points can be clearly detected: they are always approximated with short segments. In the case of using linear segments, a spu- rious point is always approximated by a straight line; thus, it is a segment with only two points. This kind of segment that the more complex the function used to approximate the set of points, the fewer segments are needed. The two most notable source of errors that affect the split procedure are gaps and spurious points. During split, gaps are easily fixed because they are filled with the approxi- mation functions of their neighbors. This is why the laser peak detection prefers to leave such gaps than to detect an

Fig. 14 Results of the fill procedure, (a) Linear segments, (b) Quadratic segments, (c) Akima splines

Fig. 15 Results of the laser stripe extraction, (a) Center of mass, (b) Linear segments, (c) Quadratic segments, (d) Akima splines

Fig. 16 Accuracy evaluation experiment for flat surfaces, (a) Image with the metal workpiece on the steel plate, (b) Extracted laser stripe

Fig. 17 Accuracy evaluation experiment for curve surfaces, (a) Image with the pipeline, (b) Extracted laser stripe

Fig. 18 Histogram of the laser extraction time

Fig. 19 Architecture of the 3D reconstruction system for steel strips

Fig. 20 Images acquired from the weld seam tracking system Fig. 21 3D reconstruction using the proposed laser extraction procedure. (a) Steel strip, (b) Welding seam

Figure 20 shows several examples of images acquired from this system, where the black line represents the extracted laser stripe. As can be seen, despite the noise in the low-qual- ity images, the proposed procedure is able to extract the laser stripe correctly. the bevel is created. The workpieces are cut, cleaned, and smoothed. Sometimes this creates very reflective surfaces which act as a mirror. These areas of the images are misin- terpreted by the camera and can even hide the laser stripe completely.

Table 1 Results of the accuracy evaluation experiments is a pipeline which has a diameter of 90.45 mm calibrated with an error less than 0.2mm. Figure 17 shows the laser projected on the pipeline and the extracted laser stripe from the image. From the 3D reconstruction obtained after con- verting each laser stripe to physical dimensions, the diameter was calculated. The average of the diameter calculated from the 3D reconstruction was 91.02 mm, 1.e., there was an error of 0.67 mm. These results are similar to those obtained in the experiments for flat surfaces. The algorithm was coded in C++ and compiled for 32 bits using the Visual C++ compiler under Windows XP SP3.

descriptionView Paper arrow_downwardDownload

An Optimal Guidance Law Applied to Quadrotor Using LQR Method

by Mehran Zareh

2010, Transactions of the Japan …

The optimal guidance law of an autonomous four-rotor helicopter, called the Quadrotor, using linear quadratic regulators (LQR) is presented in this paper. The dynamic equations of the Quadrotor are considered nonlinear so to find an LQR... more

Recent developments in high density power storage, integrated miniature actuators and MEMS technology sensors have made autonomous miniaturized Quadrotor possible. Nowadays many models of Quadrotor is produced such as Draganflyer X-Pro [3], This vehicle consists of four rotors in total, with two pairs of counter-rotating, fixed-pitch blades located at the four corners of the aircraft, an example of which is shown in Fig. 2. The optimal guidance law of an autonomous four-rotor helicopter, called the Quadrotor, using linear quadratic regulators (LQR) is presented in this paper. The dynamic equations of the Quadrotor are considered nonlinear so to find an LQR controller, it is necessary that these equations be linearized in different operation points. Due to importance of energy consumption in Quadrotors, minimum energy is selected as the optimal criteria.

Figure 2: Baseline Draganflyer X-Pro 2. Nonlinear (DOF QUADROTOR DYNAMIC MODELLING The first step before the control development is an adequate dynamic system modeling. Especially for lightweight flying systems, the dynamic model ideally includes the gyroscopic effects resulting from both the rigid body rotation in space, and the four propeller’s rotation.

for In previous pitch and ro rotation coupled In mo his research commands is d approach eliminates gyroscopic effects theoretically and rol traj works [4], 1 commands with latera etermined ( [5] two motors are used (Fig. 3.a), therefore roll motion and vice versa. a new steering command using four ors together to accomplish roll and _ pitch Fig. 3.b). This steering then no interference appears between pitch and channels. 7 ind ectory correctly. This helps t he vehicle to track the Arrows in below pictures icate the direction of propellers rotation. Figure 3: a) Old coordinate system b) New coordinate system With consideration of new coordinate system (Fig. 3.b), new steering command is described as follows. The two pairs of propellers (1, 3) and (2, 4) turn in opposite directions. Increasing or decreasing the four propeller’s speeds together generates vertical motion. If the counter-clockwise rotors increase their rotational speed, the resultant induced aerodynamic moment will cause the Quadrotor to turn clockwise (yawing motion). Slowing down the front rotors (1, 2) and at the same time speeding up the rear rotors (3, 4) produces negative pitch rotation and vice versa produces Positive pitch rotation. To produce Positive roll rotation must decrease the right rotors (2, 3) and at the same time increase the left rotors (1, 4) and

vice versa produces negative roll rotation. In this manner the roll and pitch channel don’t couple in together, because in pitch rotation, the gyroscopic effect of rotor 1 is neutralized by rotor 2 and gyroscopic effect of rotor 3 is neutralized by rotor 4 and then there will be no gyroscopic effects theoretically. The Quadrotor is an under actuated mechanical system with 6 DOF and only 4 actuators. The main forces and moments acting on it are those produced by the propellers (Fig. 4.b) .We can assume that the thrust T; and the aerodynamic moment T,; are proportional to the square of the propellers rotation speed [6]. Arrows in (Fig. 4.b) show the direction of aerodynamic moments. Thus we have:

Where / is the distance between the helicopter center of gravity and the revolution center of each rotor. Angular momentum of Quadrotor includes two parts. First part is related to rotation of body and the other is related to rotors, then it can be expressed as follow: The rate change of relative to inertial reference system is given by:

Finally, the rotational motion can be rewritten in the equation (13), directly usable (after verification of its parameters) in simulation and control:

Figure 5: Guidance, navigation, and control functions are performed by the flight computer (Flight software)

Figure 7: The block diagram of optimal control law applied to linearized system

Selecting hover flight condition as the operating point, Euler angles and their rates are zero, only the Q, is acted to rotors, and the thrust vector is equal to

Equilibrium point is considered at hover then the Euler angles and rate of them are zero and only the Qyis acted to rotors. Therefore A, B matrixes are obtained as bellow: Au, for control inputs. Then A, B matrixes are given as:

weight of Quadrotor. Drag forces also are zero, Therefore matrixes A and B are obtained as bellow: Therefore matrix K for cruise and hovering flight is obtained as:

Figure 8: Position of Quadrotor during performing of simulation.

Figure 9: Euler angles of Quadrotor during performing of simulation.

Figure. 10: Velocity in direction of motion. Conclusion

descriptionView Paper arrow_downwardDownload

Pitch control system using LQR and Fuzzy Logic Controller

by nurbaiti wahid

2010

This paper presents a comparative assessment based on time response specification performance between modern and intelligent controller for a pitch control system of an aircraft system. The dynamic modeling of pitch control system is... more

Figure 1. Description of pitch control system. Fig. 2 shows the forces, moments and velocity components in the body fixed coordinate of aircraft system. The aerodynamics moment components for roll, pitch and yaw axis are represent as L, M and N. The term P; q, r represent the angular rates about roll, pitch and yaw axis while term u, v, w represent the velocity components of roll, pitch and yaw axis. o and £ are represents as the angle of attack and sideslip. In this study the data from General Aviation Airplane [1] is used in system analysis and modeling. The parameter include in dimensional derivatives Q = 36.8lb/fi2, OS = 6771lb, OS c = 38596ft.1b and (¢ /2ug ) = 0.016s is considered.

Figure 2. Definition of force, moments and velocity in body fixed coordinate.

In this section, two control schemes are proposed and describe in detail which is Linear Quadratic Regulator The transfer function can be represented in state-space form and output equation as state by (13) and (14).

Figure 4. Block diagram of the system with matrix, K and gain Nbar. In order to reduce steady state error of the system output, a value of constant gain Nbar should be added after the reference. With a full-state feedback controller all the states are feedback. The steady-state value of the states should be computed, multiply that by the chosen gain K, and used a new value as the reference for computing the input. Nbar can be found using the user- defined function which can be used in m-file code. The method used in simulation work is done by exported both value of matrix K and constant gain, Nbar as shown in Fig. 4. For this controller design, the value of constant gain, Nbar are found to be, Nbar = 22.360.

Figure 3. Full-state feedback controller with reference input. LQR is a method in modern control theory that used state-space approach to analyze such a system. Using state space methods it is relatively simple to work with a multi-output system. The system can be stabilized using full-state feedback system. The configuration of this control system is shown in Fig. 3. In designing LQR controller, /gr function in Matlab can be used to determine the value of the vector K which determined the feedback control law. This is done by choosing two parameter values, input R = / and Q = C’*C where C’ is the matrix transpose of C from state equation (14). The controller can be tuned by changing the nonzero elements in g matrix which is done in m-file code as obtained.

Figure 5. Fuzzy logic controller in feedback loop of pitch control system. Fig. 5 shows the overall closed-loop system for FLC with the pitch control of an aircraft. The inputs to the fuzzy controller are the error (e) which measures the system performance and the rate at which the error changes (Ae), whereas the output is the change of the control signal (Au). From the Fig. 5, the error (e) is computed by comparing the reference point (desired angle) with the plant output. The change of error (Ae) is generated by the derivation of the error. The error and change of error is fed to the fuzzy controller through a multiplexer.

Figure 6. Fuzzy set of the input ‘error’. In this study, the triangular and trap membership function are chosen for each fuzzy set. Fig. 6, Fig. 7 and Fig. 8 shows the fuzzy set of the input ‘error’ and ‘Aerror’ and the output ‘output 1’.

Figure 7. Fuzzy set of the input ‘Aerror’

Figure 8. Fuzzy set of the input ‘outout1’.

Figure 9. Pitch angle response with LQR controller. Fig. 10 shows the closed loop system response of the pitch angle, 8 with fuzzy logic controller. Two inputs have been applied to fuzzy logic controller which is the error (e) that computed by comparing the reference point desired angle) with the plant output and the change of error (Ae) which generated by the derivation of the error. The fuzzy logic controller provides good performance in erm of percent overshoot that is 0%. As depicted from Fig. 10, it can be observed that the pitch angle follows the reference value respectively. This controller is able to give a good response without produce any overshoot. The response is comparatively fast that give the settling time (T,) about 2.002 second and rise time (7,) about 1.03 second. The results also demonstrated that the artificial fuzzy logic controller can eliminate the effect of disturbances in the system up to 0.5%.

TABLE IIL. Figure 11. Pitch angle response with LQR and fuzzy logic controller. For comparison of controller performance, the response for pitch control of an aircraft system using LQR and fuzzy logic controller are shown with overall response of both controllers in Fig. 11 and the bar graphs in Fig. 12. The results clearly shows that LQR controller has the best performance as compared to fuzzy logic controller in term settling time (7, steady state error he LOR controlle system which is a he percent of ove he best range w obtained in Fig. of rising time (7,) that is 0.1335 second, that is 0.3655 second and percent of (E,;) about 0.01%. The results also show r is good in eliminating the error from the most tend to the zero value. However, for rshoot (%OS) fuzzy logic controller have hich is 0%. Therefore, from the result 1 and Fig. 12, it can be concluded that he LQR control he pitch angle as er provide higher ability in controlling compared to the fuzzy logic controller.

Figure 10. Pitch angle response with fuzzy logic controller.

Figure 12. Performance comparison between LQR and fuzzy logic controller.

LONGITUDINAL STABILITY DERIVATIVE PARAMETERS A few assumption need to be considered before continuing with the modeling process. First, the aircraft is steady state cruise at constant altitude and velocity, thus the thrust and drag are cancel out and the lift and weight balance out each other. Second, the change in pitch angle does not change the speed of an aircraft under any circumstance.

SUMMARY OF PERFORMANCE CHARACTERISTIC FOR PITCH ANGLE

descriptionView Paper arrow_downwardDownload

Dynamic simulations of hydro turbine and its state estimation based LQ control

by Ajay S Raghuvanshi

2006, Energy Conversion and Management

The effect of the elasticity of the water column in the penstock of a hydro power plant represents an irrational mathematical function. In this paper, this function is reduced to a lower order using the H-infinity approximant method. The... more

Fig. 2. Nonlinear regression analysis of turbine-penstock TF models: first order. To demonstrate the dynamic nonlinear effects, a multi-step change signal is applied as a gate position devi- ation. The developed turbine power simulated on the approximated H-infinity TF models for this signal var- iation is shown in Fig. 3(a). The first order linear fit relation computed from the steady state characteristic is excited with the multi-step input signal, and its response is given in Fig. 3(b). As illustrated, the response due to each fit relation is of similar dynamic behavior. This suggests that the first order linear fit equation is good enough to represent the TF models dynamically in the study. The relative performance between the linear fit relation and its corresponding fourth order TF model is shown in Fig. 3(c).

Fig. 3. Dynamics simulation with multi-step change input signal: (a) response of TF models; (b) response of first order linear fit determined from TF models and (c) comparison between first order linear fit and its corresponding fourth order TF model.

Fig. 4. Unit step time response of turbine power dynamics: (a) upper panel: first order, lower panel: second order; (b) upper panel: thir order, lower panel: fourth order and (c) ideal, loss-less model: approximated form.

Fig. 5. Bode plot: comparison between various hydro turbine models: (a) magnitude versus frequency and (b) phase versus frequency.

Fig. 6. Schematic of estimator based regulator design. Fig. 6 shows how the control law and the estimator are combined to form a compensator. The full state feedback controller LQR and estimator can be designed independently and then combined to form the plant compensator. When a linear Kalman filter is used as the estimator, the compensator is referred as a linear qua- dratic Gaussian (LQG). The name is derived from the fact that it assumes a linear system, quadratic cost func- tion and Gaussian noise. The LQG places the poles at some arbitrary points within the unit circle so that the resulting system is optimal in some sense.

Fig. 7. Initial simulated state estimate error convergence.

Fig. 8. Initial simulated response: (a) turbine speed; (b) gate position and (c) control signal.

Fig. 9. Turbine speed response: (a) closed loop and (b) open loop.

Fig. 10. Recovery of loop transfer function: (a) LQR and linear Kalman filter and (b) LQR and SDDF

the presence of uncertainties, plant parameter changes or water column elasticity effect in the penstock. TI lastic effect is represented by a delay e~°?’« in the hydraulic structure, which is an irrational term. A stud sonducted for a relatively long penstock on the basis of an inelastic water column effect will have significa1 error. Such approach in the governor design downgrades the assessment of the controller effectiveness. TI nvestigation of oscillation problems in a power plan umn effect based transfer function (TF) model is described in Ref. [4]. Resonance of the penstock’s characte istic frequency with torsional and electromechanica with Francis turbine installed using an elastic water co oscillations at higher load points, leading to excessi vibration, is identified as the main source of vibration in the plant. The full load of the unit used is limite Jue to the excessive vibration. Sanathanan [5] proposed an iterative parameter fitting method to approxima such irrational term in the turbine—-penstock TF mod of the irrational term in the turbine—penstock TF is el with second order TF. A second order approximatio also given in Ref. [6]. These research surveys imply th: mplementation of a detailed model of the hydraulic structures is of foremost importance in the design of cot ‘roller effectiveness and plant operation, as well as d en 4 = 4 =e wy - 2 ynamic characteristic studies. nse a4 — =e om oR . eae

—_—<—<“<I£ The static and dynamic properties of the hydro plant must be known to understand the nonlinear charac- teristic. A common test to visualize and approximate the nonlinearity in a hydro plant characteristic is to test the static behavior of the plant. The static behavior is established by the relationship between the steady state values of the gate position and the turbine power. Fig. 2 shows the gate position-turbine power relationshir derived from static tests at different gate positions for the Padé and A-infinity TF models. A first order curve fitted to the H-infinity TF is sufficient to model the static nonlinear behavior of the hydro turbine. The first order linear fit relation to approximate the nonlinearity between the two variables for TF models is given below:

Quantitative eigenvalue analysis of models with states estimated using SDDF

Quantitative eigenvalue analysis of models with states estimated using linear Kalman filter

Turbine-penstock transfer function approximation using Padé method

Turbine—penstock transfer function approximation using H-infinity method

descriptionView Paper arrow_downwardDownload

LQ control design of a class of hyperbolic PDE systems: Application to fixed-bed reactor

by Joseph Winkin

2009, Automatica

A general linear controller design method for a class of hyperbolic linear partial differential equation (PDEs) systems is presented. This is achieved by using an infinite-dimensional Hilbert state-space description with... more

descriptionView Paper arrow_downwardDownload

Passive and active mass damper control of the response of tall buildings to wind gustiness

by Francesco Ricciardelli

2003, Engineering Structures

The performance of passive (TMD), active (AMD) and hybrid (ATMD) mass dampers for the reduction of the buffeting response of tall buildings is investigated. First a simple 1 + 1 DoF system is considered to model the main structure... more

Different criteria can be used to choose the feedback control law. If a state feedback approach is adopted, the control force is set to u(t) = —Kx(t), K = k, k, kz ky] being a 1 x4 gain vector. Closed form expressions of the optimum gains could be found, fol- owing the approach used by Ayorinde and Warburton 8] for TMDs, by minimising the RMS response of the main structure. When applied to AMDs, however, this procedure brings a set of four non-linear algebraic equa- tions, whose analytical solution is not easy. In real life a solution is found through the implementation of an optimal control law. In particular, linear quadratic regu- ator (LQR) control proved to be effective in many cases 9], which involves finding an appropriate controller minimising the cost functional:

Fig. 1. Schematic of the 1 +1 DoF system with (a) TMD, (b) AMD, and (c) ATMD.

ATMDs can be seen either as a way of improving the performance of a TMD by adding an actuator or as a way of reducing the actuation force and power consump- tion of an AMD by adding a spring (and, eventually, a dashpot), between the auxiliary mass and the main struc- ture. In the first case, the design of the ATMD can be performed, for a white noise excitation, in closed form following a procedure similar to that suggested by A yor- inde and Warburton [8] for TMDs, if some simplifi- cations are made [10]. The spring stiffness k, and dash- pot constant c, are chosen following any criterion for the design of TMDs. Then the active control force is set proportional to the response vector [X,o Xper Xrel?, Xre being the device stroke, and the feedback control law is designed so as to minimise the RMS displacement of the main structure. Once the acceleration gain ji, is chosen, it is possible to derive the optimum values of the velocity and displacement gains. As an alternative, the complete

Fig. 2. Spectra of the AMDs control force for the 1 +1 DoF model.

Fig. 3. Sample time histories of the AMDs control force for the | +1 DoF model.

Response of the 1 +1 DoF system with AMDs and ATMDs Table 1

Fig. 4. Spectra of the main system displacements evaluated on the 1 +1 DoF model.

Fig. 5. Spectra of the device relative displacements evaluated on the 1 +1 DoF model.

Fig. 6. Spectra of the ATMDs passive reaction for the 1 +1 Dok model. In Figs. 9 and 10 the spectra of the main structure displacements and of the auxiliary mass relative dis-

Fig. 7. Spectra of the ATMDs active control force for the 1 +1 DoF model.

Fig. 8. Spectra of the ATMDs total control force for the 1 +1 DoF model.

Fig. 9. Spectra of the main system displacement evaluated on the 1 +1 DoF model.

Fig. 10. Spectra of the device relative displacement evaluated on the 1 +1 DoF model.

Fig. 11. Schematic of the 64-storey building. Sixteen, 94-min long. correlated time histories of the wind velocities at elevations of 16, 32... 256 m were gen-

Fig. 13. Spectra of building tip displacements evaluated on the 4 + 1 DoF model.

Fig. 12. Spectra of the AMDs control force for the 4 +1 DoF model.

Fig. 15. Spectra of the ATMDs active control force for the 4 +1 DoF model. Fig. 14. Spectra of building tip accelerations evaluated on the 4 + 1 DoF model.

Response of the 64-storey building with AMDs and ATMDs (41DoF model) Table 2

descriptionView Paper arrow_downwardDownload

Evaluation of the antineoplastic activity of guduchi (Tinospora cordifolia) in cultured HeLa cells

by Shekhar Chandu

1998, Cancer letters

Exposure of HeLa cells to 0, 5, 10, 25, 50 and 100 mg/ml of guduchi extracts (methanol, aqueous and methylene chloride) resulted in a dose-dependent but significant increase in cell killing, when compared to non-drug-treated controls. The... more

descriptionView Paper arrow_downwardDownload

Coupled Torsional and Bending Vibrations of Actively Controlled Drillstrings

by Ahmet S Yigit

2000, Journal of Sound and Vibration

The dynamics of actively controlled drillstrings is studied. The equations of motion are derived using a lumped parameter model in which the coupling between torsional and bending vibrations is considered. The model also includes the... more

descriptionView Paper arrow_downwardDownload

Treatment Planning for MRI Assisted Brachytherapy of Gynecologic Malignancies Based on Total Dose Constraints

by Christian Kirisits

2007, International Journal of Radiation Oncology*Biology*Physics

Purpose: To develop a method for treatment planning and optimization of magnetic resonance imaging (MRI)assisted gynecologic brachytherapy that includes biologically weighted total dose constraints. Methods and Materials: The applied... more

Fig. 1. Axial magnetic resonance imaging (MRI) sections of brachytherapy applications with superimposed isodose lines representing biologically normalized isoeffective total dose constraints for high-risk clinical target volume (HR CTV), rec- tum, sigmoid, and bladder for the (a) high-dose-rate (HDR) brachytherapy schedule with four fractions (physical dose per fraction and total isoeffective dose), and (b) pulsed-dose-rate (PDR) brachytherapy schedule with 54 pulses (physical dose per pulse and total isoeffective dose), as performed at Medical University of Vienna. Dose—volume parameters in these cases were as follows: HDR (a): D90 (HR CTV) = 8.8 Gy/f > 99 Gyagio, Dace (bladder) = 5.7 Gy/f > 83 Gyags, Do-¢ (rectum) = 4.3 Gy/f > 68 Gygg3, Dog. (sigmoid) = 4.3 Gy/f > 68 Gyyg3; PDR (b): D90 (HR CTV) = 0.77 Gy/p — 90 Gyagio, Dace (bladder) = 0.58 Gy/p > 77 Gyug3, Dace (rectum) = 0.48 Gy/p > 69 Gyug3, Dace (sigmoid) = 0.38 Gy/p > 62 Gyug3.

For LDR and medium-dose-rate (MDR) brachytherapy, the dose of fraction f, dy, is applied by a continuous irradiation with irradiation time ts. The repair function gr can be calcu- lated by a simplification of equation (4) with np = | and Xp = 0. gr is then expressed by equation (5) (8, 33). cessive pulses and during pulses needs to be taken into ac- count, and gy is expressed by equation (4) (9, 31, 32, 33).

The parameters e, f and x indicate the control variables for the fractions, with ngprt, Mgr and npg» the total number of EBRT, delivered brachytherapy and remaining brachyther- apy fractions respectively. In the case of PDR brachytherapy dy = ngdpz and d, = nd), and in the case of HDR brachy- therapy gs = 1 and g, = 1. treatment, consisting of external beam radiotherapy (EBRT) and brachytherapy (BT). Dison of EBRT (Disok.eBrr) and all fractions of brachytherapy are calculated separately, and subsequently added up, resulting in a Dj,og total dose (Djsoz,ror). The brachytherapy fractions are subdivided into those already delivered (Dj,o:,p1), and those remaining to be delivered (Diso&,REM)-

tion it is assumed that the remaining brachytherapy fractions or pulses all produce the same dose: d, = d for HDR/LDR, d,x = d, for PDR. The positive solution can be written as d in case of HDR/LDR, and d, in case of PDR. Equation (12) allows the definition of an isoeffective dose constraint for the whole treatment (EBRT + BT) for a specific DVH parameter, D),og,ror, and the recalculation of the cor- responding physical dose for the remaining brachytherapy fractions or pulses, di). Dose-volume parameters of organs at risk can thus be kept within maximum isoeffective dose constraints, and in addition, dose-volume parameters of target structures can be kept within minimum isoeffective dose limits. Necessary input parameters of the algorithm are: tissue parameter a/6, additionally T,;2 in case of LDR and PDR brachytherapy; isoeffective total dose constraint Djsoz,ror; dose and fractionation of EBRT (for calculation of Djsok,EBRT); dose, dose rate and fractionation of already delivered brachytherapy fractions (for calculation of Dj;or,B1); dose rate, fractionation and the number of remain- ing brachytherapy fractions.

Abbreviation: EBRT = external-beam radiotherapy. HDR brachytherapy schedules are as described in the text. Sched- ule 1: 25 x 1.8 Gy EBRT + 4 fractions HDR; Schedule 2: 25 x 1.8 Gy EBRT + 6 fractions HDR.

Abbreviation as in Table A, PDR brachytherapy schedules are as described in the text. Sched- ule 1:25 x 1.8 Gy EBRT + 54 pulses PDR; Schedule 2: 25 x 1.8 Gy EBRT + 80 pulses PDR. Table 3. Physical dose constraints per pulse pulsed-dose-rate (PDR) brachytherapy representing biologically normalized isoeffective total dose constraints for high-risk clinical target volume (HR CTV), bladder, rectum, and sigmoid for the PDR brachytherapy schedules

descriptionView Paper arrow_downwardDownload

Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights

by Xun Zhou

1999, IEEE Transactions on Automatic Control

A standard assumption in traditional (deterministic and stochastic) optimal (minimizing) linear quadratic regulator (LQR) theory is that the control weighting matrix in the cost functional is strictly positive definite. In the... more

descriptionView Paper arrow_downwardDownload

Self-tuning control for active steering of a railway vehicle with solid-axle wheelsets

by Hazlina Selamat

2008, Iet Control Theory and Applications

The instability caused by the conical (or profiled) shape of a solid-axle railway wheelset can be overcome by proper design of the vehicle's primary suspension system but is generally difficult as some of the wheelset parameters, namely... more

descriptionView Paper arrow_downwardDownload

NIR calibration in non-linear systems: different PLS approaches and artificial neural networks

by Marcelo Blanco

2000, Chemometrics and Intelligent Laboratory Systems

. The frequent non-linearity of the calibration models used in infrared reflectance spectroscopy NIRSS is the main source of large errors in analyte determinations with this technique. Non-linearity in this type of system arises from... more

descriptionView Paper arrow_downwardDownload

A Novel Solution for Stochastic Dynamic Game of Water Allocation from a Reservoir Using Collocation Method

by Mehran Homayounfar

Water Resources Management

In this study, a continuous model of stochastic dynamic game for water allocation from a reservoir system was developed. The continuous random variable of inflow in the state transition function was replaced with a discrete approximant... more

descriptionView Paper arrow_downwardDownload

OPTIMAL PID CONTROLLER DESIGN FOR SPEED CONTROL OF A SEPARATELY EXCITED DC MOTOR: A FIREFLY BASED OPTIMIZATION APPROACH

by International Journal of Soft Computing, Mathematics and Control (IJSCMC)

This paper presents a new approach to determine the optimal proportional-integral-derivative controller parameters for the speed control of a separately excited DC motor using firefly optimization technique. Firefly algorithm is one of... more

descriptionView Paper arrow_downwardDownload

Pengontrolan dengan Metoda Linear Quadratic Regulator (LQR) dan Metoda Algebraic Riccati Equation (ARE) untuk Mengurangi Resonansi Ketika Despin pada Dual-Spin Spacecraft

by Qanun Miladial Hikmah

descriptionView Paper arrow_downwardDownload

Active vibration control of smart piezoelectric beams: Comparison of classical and optimal feedback control strategies

by Cesar Vasques

2006, Computers & Structures

This paper presents a numerical study concerning the active vibration control of smart piezoelectric beams. A comparison between the classical control strategies, constant gain and amplitude velocity feedback, and optimal control... more

Fig. 1. Displacement field of the three-layered smart piezoelectric beam. The FE spatial model is obtained from the weak form of the equations of motion and electric charge equilibrium presented by Vasques and Rodrigues [7]. It is derived using Consider the three-layered beam illustrated in Fig. 1. The axial and transverse displacements, u(x, 2z,,¢) and w (x,t), of the metallic core (c) and two piezoelectric sur- face layers (a,b) are defined as

where A is the system matrix, B, and By are the mechanical and electrical input matrices, C is the output matrix, u,(7f) and uy(‘) are the mechanical and electrical input vec- tors and y(t) is the output vector, given by

Fig. 2. Cantilever smart piezoelectric beam. In this section the FE model is utilized in the evaluation of the classical and optimal active vibration control of a cantilever aluminum beam with an asymmetric collocated piezoelectric sensor/actuator pair mounted on the surface (Fig. 2). Furthermore, the displacement time history, for an initial displacement field and white noise force distur- bance, and point receptance at the free end are evaluated with the open- and closed-loop systems. In this case, the sensing and actuation effects of a collocated sensor/

Fig. 3. Tip displacement and control voltage for an initial displacement field with CGVF (a,c) and CAVF (b,d): «+++ , open-loop; —

Fig. 4. Tip displacement and control voltage for an initial displacement field with output y (a,c) and state x (b,d) weighting: -------.. , open-loop; — — - closed-loop (LQR); , closed-loop (LQG).

Fig. 5. Tip displacement and control voltage for white noise point force disturbance with CGVF (a,c) and CAVF (b,d): «+++ , open-loop; — closed-loop.

Fig. 6. Tip displacement and control voltage for white noise point force disturbance with output y (a,c) and state x (b,d) weighting: -+-+++++- , open-loop; — —-, closed-loop (LQR); , closed-loop (LQG).

Fig. 7. Point receptance with CGVF (a) and CAVF (b): «++++++ , open-loop; —— In order to analyze the capacity of the various contro strategies in the frequency domain, the point receptances at the free end of the beam evaluated with the open- and closed-loop control systems are presented. With that pur- pose, the displacement time histories and white noise mechanical disturbance of the previous section are utilized to estimate the frequency response functions by a signa processing analysis procedure through a fast Fourier trans- form (FFT) algorithm. The open- and closed-loop point receptances for the CGVF and CAVF control are pre- sented in Fig. 7. The CAVF control significantly reduces the magnitude of the first resonance in approximately 27 dB, and increases the magnitudes of the other modes. In comparison, the CGVF does not manage to have such a good performance in the attenuation of the first mode, with a 8.6 dB reduction, but in compensation attenuation in all modes is ac hieved, the fourt h mode is completely eliminated and the natural frequencies are shifted. Further- more, a severe spil over effect is visi the first natural frequency. That is re for the CAVF algorithm a band pas and only the polari first mode contri y of the sensing v bution was used ble in Fig. 7(b) above ated with the fact that s filter was considered oltage related with the in the closed-loop

Fig. 8. Point receptance with the LQR (a) and LQG (b) controller with output y weighting: --------. , open-loop; —

Fig. 9. Point receptance with the LQR (a) and LQG (b) controller with state x weighting: -------+. , open-loop; —

Material properties of the aluminum and PXE-5 Table 1

descriptionView Paper arrow_downwardDownload

State Feedback Controller Design via Takagi- Sugeno Fuzzy Model : LMI Approach

by Khaled Zehar

In this paper, we introduce a robust state feedback controller design using Linear Matrix Inequalities (LMIs) and guaranteed cost approach for Takagi-Sugeno fuzzy systems. The purpose on this work is to establish a systematic method to... more

To obtain the least possible upper bound using a quadratic Lyapunov function, we have the following optimization problem: Using the LMI lemma [8], we can write the above inequalities as follows:

Fig. 3. Response of the pendulum angular velocity.

descriptionView Paper arrow_downwardDownload

Positive Position Feedback Active Vibration Control of a Smart Cantilever Beam Using ANSYS

by Ayman El-Badawy

In this work, the active vibration control of a uniform cantilever beam using piezoelectric materials subjected to transverse vibrations is studied. The equation of motion of a beam bonded with the piezoelectric actuator is realized based... more

descriptionView Paper arrow_downwardDownload

SIMMECHANICS VISUALIZATION OF EXPERIMENTAL MODEL OVERHEAD CRANE,ITS LINEARIZATION AND REFERENCE TRACKING-LQR CONTROL

by International Journal of Chaos, Control, Modelling and Simulation (IJCCMS)

2013, International Journal of Chaos, Control, Modelling and Simulation (IJCCMS)

Overhead Crane experimental model using Simmechanic Visualization is presented for the robust antisway LQR control. First, 1D translational motion of overhead crane is designed with exact lab model measurements and features. Second,... more

descriptionView Paper arrow_downwardDownload

APPLICATION OF ORTHOGONAL FUNCTIONS TO PHARMACEUTICAL ANALYSIS, GENERATION OF DERIVATIVE CURVES

by Dalia Hamdy

2005

Application of orthogonal functions to certain instrumental methods of analysis depends upon the expansion of an experimental curve in terms of orthogonal polynomials. The theoretical principle of the method is discussed. The method... more

Where N is the number of points, x is the mean of x; and the values of P,,; are often conveniently scaled so as to make them integers (as small as possible) for all x;. Tables of orthogonal polynomials are available for all numerical values from P, to Ps; and all points from 3-to 75-point orthogonal polynomials (13). Since P ,; = 1 for all values of N and x, the values of P,; is not given in the tables.

Fig. 4 Gaussian band (a), its quadratic P2 convoluted curve using orthogonal polynomials (b), its second order derivative curve, D2 (c).

Fig. 6 Flow chart for the visual BASIC program used to obtain derivative curves.

Table 1. Analogies between orthogonal functions and Fourier functions.

descriptionView Paper arrow_downwardDownload

Analysis and Simulation of a Single Generator- Infinite Bus Power System under Linear Quadratic Regulator Control

by Mbanisi Kenechukwu

The thrust of this article is to develop elaborate educational material on linear quadratic regulator (LQR)-based power system control for electrical engineering students, especially senior undergraduate and fresh graduate students. The... more

descriptionView Paper arrow_downwardDownload

Large deformation three-dimensional image registration in image-guided radiation therapy

by Sha Chang and

2005, Physics in Medicine and Biology

In this paper, we present and validate a framework, based on deformable image registration, for automatic processing of serial three-dimensional CT images used in image-guided radiation therapy. A major assumption in deformable image... more

Figure 1. Visualization of prostate motion over the course of treatment for nine patients involved in our study. White contours, superimposed on an axial slice of each patient’s planning image, indicate the actual location of the prostate on each treatment day. These contours are taken from manual segmentations of treatment images. The discrepancies between the contours exhibit the effect of set-up error and organ motion on the prostate position. Note that different patients exhibit different amounts of prostate motion; compare the close contour agreement for patient 3101 with the wide contour variability for patient 3109. For some patients (3102, 3109) motion is primarily noticeable in the anterior—posterior direction; for other patients (3106, 3107) motion is primarily noticeable in the lateral direction.

Large deformation 3D image registration in image-guided radiation therapy Figure 2. First row: axial and sagittal slices from the planning image of patient 3102. Second row: the same slices (with respect to the planning image coordinate system) taken from a treatment image. Third row: the voxelwise absolute difference between the planning and treatment images. Black represents perfect intensity agreement, which is noticeable in the interior of the bones and outside the patient. Brighter regions, indicating intensity disagreement, are especially apparent: (1) in regions where gas is present in one image and absent in the other; (2) around the bladder which is large on the treatment day compared to the planning day; (3) uniformly along boundaries with high intensity gradient, indicating a global set-up error such as a translation.

Figure 3. Difference images comparing a planning to a daily image before and after deformable registration.

Large deformation 3D image registration in image-guided radiation therapy Figure 4. Example of deformable image registration. The first and last rows show axial and sagittal slices of the planning and treatment images. The second row shows the treatment image after deformable image registration, which brings the treatment image into alignment with the planning image. The improvement in soft tissue correspondence suggests that the registration procedure accurately captures internal organ motion. Note how the changes in size and shape of the bladder and rectum are accounted for.

Figure 5. Example of gas deflation. Panel (a) shows an axial slice of a treatment image containing a large region of bowel gas. Panel (b) shows the same image after automatic gas deflation. This deflated image can be accurately registered using deformable image registration.

Figure 6. Example of automatic segmentation using deformable image registration. (a) Axial slice of a planning image with the prostate labelled by a white contour. (b) The same axial slice (in terms of planning coordinates) from a treatment image. The planned prostate position is shown in white, the actual prostate in black (both contours manual). (c) The same treatment image and manual (black) contour. The white contour is automatically generated by performing deformable image registration and applying the resulting deformation to the planning segmentation. The close agreement of the contours indicates that image registration accurately captures the prostate motion.

Figure 7. Visualization of organ segmentations. Panel (a) is an anterior view of a 3D rendering displaying segmentations of the skin, prostate, rectum, bladder, seminal vesicles and femoral heads. Panel (b) shows a lateral view of the prostate, rectum and bladder of the same patient. The surfaces are constructed by tiling manually drawn contours.

Large deformation 3D image registration in image-guided radiation therapy Figure 8. Visualization of the result of image registration algorithm. The images show manual segmentations of each daily image deformed into the space of the planning image. The close agreement of the deformed segmentations with the position of the prostate in the planning images provides evidence for the accuracy of the image registration algorithm along the prostate boundary.

Large deformation 3D image registration in image-guided radiation therapy Figure 9. (a) Centroid differences measured in the lateral (X), anterior—posterior (Y) and superior— inferior (Z) directions. The horizontal lines on the box plots represent the lower quartile, median and upper quartile values. The whiskers indicate the extent of the rest of the data, except that outliers, which fall more than 1.5 times the interquartile range past the ends of the box, are denoted. with the ‘+’ symbol. (b) Euclidean distance between segmentation centroids.

Large deformation 3D image registration in image-guided radiation therapy Figure 10. Dice similarity coefficient (DSC) and logit DSC.

Figure 11. Radial distance maps, for the prostate. Map (a): mean radial distance between segmentations A and B (human raters). Map (b): mean distance between A and C4 segmentations (human and computer).

Large deformation 3D image registration in image-guided radiation therapy Figure 12. The position of the prostate in patient 3102 at day 2 compared to the time of planning. Left: the treatment image from day 2, shown with a skin contour from the planning day. Top right: planning image. Bottom right: day 2 image. On all three images, the location of the prostate is shown for both days, along with isodose curves at the 95%, 98% and 100% level.

Figure 13. (a) Daily treatment prostate DVHs for each of the first four days, compared to the planned DVH. (b) Dose—volume histograms for delivered dose, estimated over increasing sets of days. These are compared against the planned dose. All doses are normalized to a prescription dose of 78 Gy.

Large deformation 3D image registration in image-guided radiation therapy Figure 14. DVHs for planned total dose, planned BED and delivered BED. Delivered BED i: modelled from a sample of 18 out of 39 treatment days. a/B = 1.5.

Figure 15. Biologically effective DVHs, assuming four possible values of w/ and 50 Gy delivered in 15 equal fractions.

Table 1. Parameters used in the regularizing operator L = wV* + BVV- + y, along with the maximum number of iterations permitted.

Table 2. Summary statistics for centroid difference distributions. The mean, standard deviation and 95% confidence interval for the mean are reported.

Table 3. Summary statistics for centroid distance distributions.

Table 4. Summary statistics for the DSC measures.

Table 5. Comparison of automatic segmentation to manual segmenter A via the DSC and LDSC. This is the full set of segmenter-A segmentations that we have processed.

descriptionView Paper arrow_downwardDownload

Multirate adaptive temperature control of greenhouses

by Kostas Arvanitis

2000, Computers and Electronics in Agriculture

A new adaptive technique is proposed for the control of the temperature in a greenhouse whose parameters vary with operating conditions. The proposed adaptive controllers are derived by solving either the pole-placement or the linear... more

Fig. 1. The structure of the adaptive-multirate output controller. Consider applying the adaptive multirate control strategy depicted in Fig. 1, to system (1). The MROC sampling mechanism involves measuring the system output at every instant 7* = T,/N, where Ty is the frame sampling period and NeZ* is the sampling multiplicity. The value of N is constant for any particular design and can be selected using the guidelines provided by Er and Anderson (1991).

Fig. 2. Closed-loop system output under identification process (V.,..(k)) vs. nominal closed-loop system output (Vqnom(k)) for greenhouse temperature model (Eq. (30a)), in the case of linear quadratic (LQ) regulation with a constant multirate-output controller (MROC).

Fig. 3. Estimates of the system parameters for greenhouse temperature model (Eq. (30a)), in the case of linear quadratic (LQ) regulation with a constant multirate-output controller (MROC). in order to obtain the simulation results, 1n all cases, both nominal and estimated system were subjected to step changes in the set point. From the illustrative examples, it can be easily seen that the proposed MROC based adaptive controllers lead to a very satisfactory performance.

Fig. 4. Closed-loop system output under identification process (V.,.4(k)) vs. nominal closed-loop system output (Vunom(k)) for greenhouse temperature model (Eq. (30b)), in the case of pole placement with a non-prespecified gain /,,.

Fig. 5. Estimates of the system parameters «,, «3; and /, for greenhouse temperature model (Eq. (30b)), in the case of pole placement with a non-prespecified gain /,,. The feasibility of using MROC based adaptive controllers for controlling the temperature of greenhouses with large parameter variability has been investigated in this paper. Two fundamental alternative schemes have been analysed and simulated: MROC based adaptive pole placers and MROC based LQ optimal regulators. In these two adaptive schemes, feedback control relies on measurements obtained by multirate sampling of the greenhouse temperature, while introducing low order exogenous dynamics in the control loop. From the theoretical point of

descriptionView Paper arrow_downwardDownload

An advanced LMI-based-LQR design for voltage control of grid-connected wind farm

by Guy Dumont

2008, Electric Power Systems Research

This paper presents an innovative voltage control scheme for a grid-connected wind farm. In wind power generation systems, operating conditions are changing continually by wind speed fluctuations and load changes. Therefore, a robust... more

This paper presents an innovative voltage control scheme for a grid-connected wind farm. In wind power generation systems, operating conditions are changing continually by wind speed fluctuations and load changes. Therefore, a robust control mechanism is necessary. To enable a linear and robust control framework, the overall system is represented by a set of reduced-order linear systems that cover an operating range of interest determined by variations of the load. A control-design technique known as the linear quadratic regulator (LQR) can be conveniently utilized for multi-input multi-output systems. However, to make this approach applicable simultaneously to several linear systems, the LQR problem needs to be reformulated for finding a common Lyapunov function for the set of considered linear systems. This is accomplished by representing the underlying control optimization problem in terms of a system of linear-matrix-inequality (LMI) constraints and matrix equations that are simultaneously solved. The solution of LMI equations involves a form of quadratic Lyapunov function that not only gives the stability property of the controlled system but can also be used for achieving certain performance specifications. In addition, to make the control design applicable to realistic systems, with noise and disturbances in the measured signals, we consider a state observer. A candidate wind farm site on Vancouver Island, Canada, is conducted for simulation study. The proposed methodology is also flexible and readily applicable to larger wind farms of different configurations. (H.-S. Ko). quality specifications [1,2,4-6]. However, there are always costs associated with the installation and operation of supplementary devices, which makes this option less attractable. Therefore, to achieve easier grid integration and reliable voltage control, alternative active voltage control of WTs is required. Furthermore, in modern WF applications, the WFs have to contribute voltage control within a specified allowable voltage level at a specified remote location-a point of common coupling (PCC).

Fig. 1. Grid-connected wind farm system. Fig. 1 shows a simplified diagram of system considered in this paper [19]. Here, only three WTs are included to repre- sent possible dynamic interaction among the individual turbines on the WF. Each WT is equipped with a step-up 0.69/34.5kV transformer (TR). The WTs are connected in a chain using 9km cables. The WF and the utility grid are connected through the 132 kV transmission line (TL, 100km). Here, the utility grid

Fig. 2. Schematic diagram of the proposed voltage control. This section presents an advanced LMI representation of LQR as a frame to design a controller as indicated in Fig. 2.

Fig. 3. Comparison between the full-order (104th) and the reduced-order (fourth) transfer functions from the injected reactive power to the voltage at PCC. In this paper, a balanced-realization model-ord technique [26,27] is used to find the lower order transfer functions that are more suitable for purpo design. This technique is based on considering t states (modes) in the input—output behaviour of the er reduction approximate se of control he dominant system. The method uses Hankel singular values of the system, which are the common eigenvalues of controllability and observa bility Grami- ans. The reduced-order model is obtained by neglecting the The original 104th-order linearized model for control design might be possible, but is not desirable, as it would require signifi- cant computational resources. However, as the visual inspection of Fig. 3 reveals, these transfer functions may well be approx- imated in the frequency range of interest by a system of much lower order.

Fig. 4. Overall voltage control block diagram. 5. Case study

Fig. 5. Bus voltage output at the PCC. A three-phase symmetrical fault is assumed in the middle of one of the TLs. In this study, the fault is applied at t=1s and is subsequently cleared after 0.15s by removing the faulted line (post-fault). Fig. 5 shows the voltage transients at the PCC pre- dicted by the model with the robust-PI, CLQR and the ALQR. Here, CLQR is the LMI-based-LQR controller without the cross- product term, which is the second term in (18). The gains of the robust-PI controller are tuned based on Nyquist curve constraint technique given in Ref. [31] as the gain of the PI controller is obtained as ky, = 0.1711 and k; = 10.3166, and the anti-windup gain k, can be computed by k,=k;/ky. The transfer function, which was extracted fourth-order reduced model and used to tune PI gains, and overall voltage control scheme based on the robust-PI are given in Appendix A. As can be noted, the fault results in significant voltage swings that can undesirably inter- fere with the protection circuitry and possibly trip the WTs. From this point of view, it is desirable to minimize and/or sup- press the voltage swings. During the fault, the robust-PI showed

Fig. 6. Reactive power set-point and maximum at each WT.

The reduced fourth-order transfer function, which is used to design the robust-PI controller, is given in (A.5). The overall voltage control scheme based on the robust-PI controller with the proposed anti-windup is depicted in Fig. A.1.

Design parameters and control gains References Table A.2

descriptionView Paper arrow_downwardDownload

Load Disturbance Resistance Speed Controller Design for PMSM

by Cajetan Akujuobi and

2000, IEEE Transactions on Industrial Electronics

This paper presents an linear-quadratic-regulatorbased proportional-integral-differential equivalent controller design method for a permanent-magnet synchronous motor. The disturbance rejection is achieved based on a multi-objective... more

descriptionView Paper arrow_downwardDownload

Design of the roll stabilization controller, using fin stabilizers and pod propellers

by Key-pyo Rhee

2011, Applied Ocean Research

Ships experience roll motion due to waves in a seaway. Therefore, fin stabilizers are installed to stabilize such roll motion. A fin stabilizer is effective at reducing the roll motion at moderate speeds but not at low speeds. Recently,... more

The hydrodynamic coefficients and the wave frequency are non- dimensionalized as follows. In the above, g is the gravity and Lpp the length between perpen- diculars of the ship.

Fig. 4. Added mass moment of inertia/damping coefficient in the roll mode. Fig. 3. Added mass/damping coefficient in the sway mode.

Fig. 5. Added mass moment of inertia/damping coefficient in the yaw mode.

Fig. 8. Flow diagram of frequency-weighted LQR. By introducing the weighting matrices W,(jw) and W (jw) to give weight to the frequency, the performance index can be expressed as shown in Eq. (29).

Fig. 11. Simulation results of case3. The simulation results of each case are shown in Fig. 9 through to Fig. 14. These figures show the wave elevation, roll angle, yaw angle, fin angle, and pod angle with respect to time. The saturation angle of the fin stabilizer is 20° and that of the pod propeller is 5° (Figs. 10-15). At the low speed, the contribution of fin stabilizers to the con- trol forces is minor because the lift force reduces dramatically. The control force of the pod propeller is also decreased as the RPS is decreased. However, under both the fin and the pod propeller, the roll amplitude is reduced by more than 50%.

Fig. 13. Simulation results of case5. The ITTC wave spectrum is used to generate the irregular waves. The sea state is 5, the wave comes from the side of the ship, and the advance speed is 23 knots. As mentioned previously, there is no well-developed theory about how to determine the shape of the weighting function. In this paper, control systems are designed with different weight-

Fig. 15. ITTC wave spectrum. Table 5 shows the roll decrements in each case. In Case 3, the decrement of the roll motion is 80.2%, and the roll amplitude is reduced most efficiently.

ing functions @ 1, and the control efficiencies of the various cases are compared with each other. In Case 1, the weighting function is focused on the frequency of the nominal plant. In Case 2, the weighting function is focused on the frequency where the roll RAO has the maximum value. In Cases 3 and 4, the weighting function is focused on the frequency where the wave spectrum has the max- imum value. In Case 4, the magnitude of the weighting function is smaller than that in Case 3. Fig. 16 shows the magnitudes of the weighting functions with respect to the wave frequency.

Fig. 16. Magnitude of the weight function, a1. Fig. 14. Simulation results of case6.

Fig. 17. Simulation results with respect to the weight function, @11.

Fig. 20. POD turning rate in irregular waves. Fig. 19. Simulation results in irregular waves (fine view).

Fig. 18. Simulation results in irregular waves.

Fig. 21. Simulation results in irregular waves (modified pod).

Fig. 22. POD turning rate in irregular waves (modified pod). References

© 2011 Elsevier Ltd. All rights reserved.

Principal particulars of the fin stabilizer. Table 1 Table 2

descriptionView Paper arrow_downwardDownload

SIMULTANEOUS LINEAR QUADRATIC POLE PLACEMENT (LQPP) CONTROL DESIGN

by Ka C Cheok

The pole placement (PP) technique for design of a linear state feedback control system requires specification of all the closed-loop pole locations even though only a few poles dominate the system's transient response characteristics. The... more

gain that satisfies the extremum condition given by the following coupled algebraic Lyapunov equations: p, represents the unspecified system characteristics:

descriptionView Paper arrow_downwardDownload

Underactuated robot control: comparing LQR, subspace stabilization, and combined error metric approaches

by Lorinc marton and

2008, … , IEEE Transactions on

In this paper, three techniques for robust control of underactuated robots are experimentally compared on the classical ball and beam system. An adaptive tracking controller is first designed and implemented to identify the nominal... more

descriptionView Paper arrow_downwardDownload

Welfare-Based Optimal Monetary Policy with Unemployment and Sticky Prices: A Linear-Quadratic Framework

by Federico Ravenna and

2011, American Economic Journal: Macroeconomics

In this paper, we derive a linear-quadratic model for monetary policy analysis that is consistent with sticky prices and search and matching frictions in the labor market. We show that the second-order approximation to the welfare of the... more

The linearized expressions for inflation and the markup illustrate how labor market tightnes: affects inflation. A rise in labor market tightness reduces the retail price markup, increasing th marginal cost of the retail firms. This leads to a rise in inflation. Expected future labor marke tightness also affects current inflation. For a given 61, a rise in E6141 increases the markup anc reduces current inflation.> It does so through its effects on current wages. Expectations of labo: market tightness increase the incentive of firms to post vacancies. This would normally lead t« a rise in current tightness. However, since the coefficient on E641 measures the impact on pu when 6, remains constant, wages must fall to offset the rise in vacancies that would otherwis occur and keep 0, constant. Finally, there is a cost channel effect in that the real interest rate ha: a direct impact on yi, and therefore on inflation.® This arises since it is the present discountec value of expected future labor market conditions that affects the firm’s decision to post an extré vacancy. The linearized expressions for inflation and the markup illustrate how labor market tightness marginal cost of the retail firms. This leads to a rise in inflation. Expected future labor market

Figure 1: Employment and hours per employee (total private industries, HP detrended, shaded regions denote NBER recession dates)

Figure 2: Response of the efficient, flexible-price unemployment rate i and labor market tightness on to a 1% positive productivity innovation for different values of p,.

Figure 5: Impulse responses to a one unit bargaining shock under optimal policies minimizing different loss functions

Figure 6: Responses to a bargaining shock under optimal commitment and discretion

Figure 7: Contribution of labor market, expected inflation, and the real interest rate to the path of inflation net of the direct effect of the bargaining shock when policy is based on the welfare approximation.

Figure 8: Contribution of labor market, expected inflation, and the real interest rate to the path of inflation net of the direct effect of the bargaining shock when policy is based on the minimizing the exprected present value of 7? + Ati? and \ = 0.052.

Figure 9: Unemployment rate responses to a serially correlated productivity shock: US and EU calibrations

Figure 10: Responses to a unit bargaining shock for U.S. (solid line) and EU (dotted line) cali- brations. (Note: middle panel shows the rate of unemployment.)

* Cost of bargaining shocks under optimal commitment policy as percent of steady-state consumption.

* Cost of bargaining shocks under optimal discretion as percent of steady-state consumption.

Cost of bargaining shocks as percent of steady-state consumption under optimal commitment policy

descriptionView Paper arrow_downwardDownload

VIBRATION CONTROL USING ACTIVE MAGNETIC ACTUATORS WITH THE LQR CONTROL TECHNIQUE

by Douglas D Bueno and

this paper presents an approach that is numerically verified for active vibration control in a rotor using Active Magnetic Bearings (AMB). The feedback technique is used at this controlling device and the controller gain is obtained by... more

descriptionView Paper arrow_downwardDownload

Fuel cell and ultracapacitor energy system control using linear quadratic regulator proportional integral controller

by Himadry Shekhar Das

2019, Electrical Engineering

Application of fuel cell (FC) in power generation requires efficient power converters and controllers for hybridization of energy storage devices. This paper presents the control technique in a fuel cell and ultracapacitor hybrid system... more

Different types of fuel cell hybrid structures are proposed based on the needed application. For example, power gen- eration using fuel cell applies hybridization with renewable energy sources like photovoltaic, wind or biomass, and stor- age systems like battery, ultracapacitor or flywheel [10-12]. FC is hybridized with battery, ultracapacitor or flywheel for hybrid vehicle applications [13-17]. Also for small-scale power generation, FC is incorporated with battery or ultra- capacitor [18—21]. Therefore, FC with battery and/or ultra- capacitor is the commonly chosen combination for hybrid system regardless of application area. The hybridization structure in FC-based systems is shown in Fig. 1, where FC means fuel cell, B represents the battery, SC represents ultracapacitor/supercapacitor and DC/DC means DC/DC converter. The configurations of FC can be divided into series and parallel hybrid structures [5]. In series structure, the sources and converters are connected in series as shown in Fig. la, b, where these structures are inefficient and are rarely been used. Meanwhile, the sources are connected in parallel to the load in the parallel connected structure. For a two-source hybrid structure, up to two DC—DC converters can be used in the parallel connection. Figure 1c demon- strates that both sources are left uncontrolled, while only can be used in the parallel connection. Figure 1c demon-

The activation voltage loss equation is basically from the Tafel equation [43]: where R, n, F and iy are the constant values and a is the elec- tron transfer coefficient. Normally, the value of the transfer coefficient is 0.5 for both anode and cathode. The empirical formula of activation voltage drop can be shown as [42, 44]: divi, Si> $2> $3> alu $4 Gil Uv PeLaitweuiv VULELIIVIVIILDS. Using the mathematical equations, a 5.5 kW PEMFC stack is developed in MATLAB/Simulink, and the volt- age and power characteristics against change in current supplied by the PEMFC are shown in Fig. 3. It can be seen that the maximum power point is at 194 A current. It means, while the model supplies 194A current the power of the stack is at maximum (5750 W). At this time, the voltage of the block is 29.6 V. The stack is set to operate at the ohmic region and around the maximum power point by setting up the stack reference current limit between 50 A and 250 A. As a result, the stack voltage will fluctuate between 50 V and 20 V.

The non-ideal switching circuit diagram of the bidirec- tional DC—DC consists of an inductor (L) with internal resistance R,, current through inductor i, , two controlled switches (Sb1, Sb2), input and output filter capacitors (C, and C,,) with internal resistances R, and R,,, input DC voltage source (Vyc) and output load (R), output voltage of the con- verter as V,,,- From the circuit operation, the average equa- tions of the bidirectional converter are derived as Eq. (6-8):

4 Controller design of the hybrid system The energy distribution between sources is performed such that FC supplies major steady-state power and UC handles the transient power. Thus, the reference signals of the control loops are summarized as follows: Vo rep = Constant

Assuming no input capacitor is present, the updated stat space model for output voltage regulation is as follows: The control parameter d consisting of the modified state vec- tor %, is denoted by: In order to design the LQR-based control for voltage loop, the cost function is formulated as:

The controller is simulated with the fuel cell and ultraca- pacitor hybrid system to evaluate the performance of the proposed controller. The controller is aimed to regulate the output voltages of the DC—DC converters. In order to ana- lyse the compatibility of the proposed controller, steady- state and dynamic performances of the proposed controller are compared with PI and LQR controller performances. Subsequently, the LQR-PI controller is tested with differ- ent load scenarios for further evaluating the energy shar- ing capability. The source, converter, load and controller subsystems are illustrated in MATLAB/Simulink diagram (Fig. 5). The respective components of each subsystem are included. For example, the source subsystem contains FC and UC block, DC—DC converter subsystem includes one

Figure 8b demonstrates the output voltage response for the three controllers. In this case, the PI controller had longer recovery time (0.3 s) with overshoot/undershoot in

based on the variation in high power resistive load from the output responses (refer to Table 3).

5.2 Performance evaluation of the proposed LQR-PI controller The proposed FCHS was tested with the LQR-PI controller for two additional conditions to verify the ro energy sharing capability. The first scenario is the analysis of variation in DC load resistance. In this sectio sharing performance was verified in which hig was used for the verification. The second scenario is the analysis of variation in DC bus voltage. An increase in DC bus reference voltage was introduced, and the bustness and h power load n, the energy behaviour o the controller against the disturbance was observed. For the second case, both energy sharing performance and robust- ness of the controller were verified. The proposed FCHS was tested with the LQR-PI controller

At the starting point, the load current was low; thus, the FC converter has supplied the load current as well as the UC charging current. Concurrently, the UC converter has oper- ated at the buck mode and charging the UC unit. The first increase in load current occurred at t=5 s; however, the UC unit continued charging at a lower rate. This is because the FC current was adequate for meeting the load and charging the UC. The excess demand was met by the UC converter current once, and the load current has increased and became higher than the FC converter supply current (at t= 10 s). The UC converter was in the boost mode during this time point (from t= 10s to t=17s). Finally, the UC converter switched to buck mode and recharged the UC unit once the load cur- rent requirement dropped.

Figure 10c depicts the output current response of the hybrid system for the LQR-PI controllers. The load current was 16 at DC bus voltage of 80 V A, while with the step change in load voltage from 80 V to 100 V at t=6 s, the load current increased from 16 A to 20 A. The same figure also illustrates the load distribution between the FC and UC using LQR controller. At the start-up stage, the FC supply reached steady state, where power supply and the supplied current

The power control technique of a fuel cell and ultraca- pacitor hybrid energy system using LQR-PI technique was evaluated in the current study. The control algorithm used a double-loop PI controller for the fuel cell boost converter and a voltage-mode LQR controller for the ultracapacitor bidirectional converter. The PI controller and LQR con- troller enabled efficient energy sharing between fuel cell and ultracapacitor during steady-state and transient opera- tion. The compatibility of the proposed controller has been tested with both constant and step change in resistive loads. Only resistive load was tested as there was no difference in the characteristics of resistive and inductive loads of DC

Table 2 Control parameters of the hybrid system for PI, LQR and LQR-PI control

Table 1 Fuel cell boost converter design parameters

Table 3 Comparison of controller performance based on voltage and current overshoot, undershoot and recovery time Thus, LQR-PI controller was used in the further testing and application of the FCHS.

descriptionView Paper arrow_downwardDownload

Biologically effective dose distribution based on the linear quadratic model and its clinical relevance (ASTRO Research Fellowship)

by Min Leu

International Journal of Radiation OncologyBiologyPhysics

Purpose: Radiotherapy plans based on physical dose distributions do not necessarily entirely reflect the biological effects under various fractionation schemes. Over the past decade, the linear-quadratic (LQ) model has emerged as a... more

Purpose: Radiotherapy plans based on physical dose distributions do not necessarily entirely reflect the biological effects under various fractionation schemes. Over the past decade, the linear-quadratic (LQ) model has emerged as a convenient tool to quantify biological effects for radiotherapy. In this work, we set out to construct a mechanism to display biologically oriented dose distribution based on the LQ model. Methods and Materials: A computer program that converts a physical dose distribution calculated by a commercially available treatment planning system to a biologically effective dose (BED) distribution has been developed and verified against theoretical calculations. This software accepts a user's input of biological parameters for each structure of interest (linear and quadratic dose-response and repopulation kinetic parameters), as well as treatment scheme factors (number of fractions, fractional dose, and treatment time). It then presents a two-dimensional BED display in conjunction with anatomical structures. Furthermore, to facilitate clinicians' intuitive comparison with conventional fractionation regimen, a conversion of BED to normalized isoeffective dose (NID) is also allowed. Results: Two sample cases serve to illustrate the application of our tool in clinical practice. (a) For an orthogonal wedged pair of x-ray beams treating a maxillary sinus tumor, the biological effect at the ipsilateral mandible can be quantified, thus illustrates the so-called "double-trouble" effects very well. (b) For a typical four-field, evenly weighted prostate treatment using 10 MV x-rays, physical doshnetry predicts a comparable dose at the femoral necks between an alternate two-fields/day and four-fields/day setups. However, our BED display reveals an approximate 21% higher BED for the two-fields/day scheme. This excessive dose to the femoral necks can be eliminated if the treatment is delivered with a 3:2 (anterioposterior/posterio-anterior (AP/PA):bilaterally opposed (BLO)) dose weighting. With Co-60 beams, the increase of BED with alternate two-fields/day, 1:l setup was even more pronounced (26%). Conclusion: We have demonstrated the feasibility of constructing a biologically oriented dose distribution for clinical practice of radiotherapy. The discordance between physical dose distributions and the biological counterparts based on the given treatment schemes was quantified. The computerized display of BED at nonprescription points greatly enhanced the versatility of this tool. Although the routine use of tbii implementation in clinical radiotherapy should be cautiously done, depending largely on the accuracy of the published biological parameters, it may, nevertheless, help the clinicians derive an optimal treatment plan with a particular fractionation scheme or use it as a quantitative tool for outcome analysis in clinical research.

descriptionView Paper arrow_downwardDownload

Control for Satellites Formation Flying

by Norman Fitz-coy

2007, Journal of Aerospace Engineering

In this paper, a controller is designed for a satellite formation flying system around the Earth based on an uncertainty model derived from a nonlinear relative position equation. In this model, nonzero eccentricity and varying semimajor... more

Fig. 13. Relative position error in the elliptic reference orbit case () Fig. 10. Control accelerations in the circular reference orbit (p)

Fig. 12. Control accelerations in the circular reference orbit (SMC)

sented in Eq. (1), which is expressed in the local horizontal local vertical frame (LVLH) as shown in Fig. 2 and includes exogenous effects such as control inputs, J, perturbations acceleration a,,, and atmospheric drag acceleration ago. [1 = gravitational constant of the Earth. R,=[x,.y,,z,]’ and Rp=position vectors from the center of the Earth to the leader and follower, respectively, and R;, and R-=corresponding position magnitudes, p=[p;,p2,p3]" is the relative position vector from the leader to the follower satellite (i.e., p=R,-—-R,). It is worth noting that the absolute position information for both satellites is not necessary for this model of the leader-follower pair; absolute position is only required for one of the two satellites. u is the control acceleration acting on the follower satellite and assumed to have control authorities in all three directions. w=[0,0,f,]" and w=[0,0,f;,]" are angular velocity of the leader with true anomaly rate f, and true anomaly acceleration fie The J, and atmospheric drag accelerations (Pruss- ing and Conway 1993) are listed in Eqs. (2) and (3). In these equations, J,= 1.08263 X 1077, R,=6,378 km, and m=mass of a satellite. €, and Z=unit vectors in the radial direction of the fol- lower satellite and Z direction of the earth centered frame. For an orbital altitude above 500 km, the J, perturbation is the dominant one compared with the atmospheric drag. However, both are in- cluded in this paper for fidelity purposes

Fig. 2. LVLH coordinate system: (a) 7,178 km semimajor axis; (b) 7,978 km semimajor axis The CW equation is derived based on the assumption that the reference orbit is a near circular orbit. Here we will derive a linear model without this assumption. In deriving this model, the atmo- spheric drag model and J, perturbation will be neglected and regarded as functional uncertainties. They will be added into the linear system together with the control input after the linear sys- tem is obtained. Thus neglecting the exogenous part of Eq. (1) and performing the Tavlor series exnansion of &/R2R- vield

Therefore, in this paper, eccentricity and semimajor axis variations are modeled parametrically as shown in Eq. (12). The nomina model (CW equation) will be the equation without uncertainties W;A,i¢1,...,5, where ||A|| <1 and W,=uncertainties boundary for the ith parameter. The actuation noise is modeled as random variables 5, and the atmospheric drag, J, and other high order terms are modelec as random variables bounded by 6. Therefore, in this model, nonzero eccentricity, semimajor axis variations, drag, J>, and actuation nois¢ are all included. Sensor noise will be considered in the control synthesis discussed in the next section The linear dynamics model with eccentricity and semimajor axis variations can be rewritten as shown in Eq. (11), where the drag, J>, and high order terms are all included, and subscript 0 denotes the nominal value. The actuation and sensor noises will be included in the controller designs and the simulations. Comparing Eq. (11) with the CW model [Eq. (8)], the follow- ing statements can be made. First, if the eccentricity is zero with a nominal semimajor axis, they are identica’ . Second, the eigen- values of the CW equation are all on the imaginary axis. Simula- tions have been done for all eccentricity cases (e €[0;1]), and even an extremely small eccentricity wil eigenvalues to be unstable, which may consumptions in stabilizing the system. eccentricity is 10-? and true anomaly is 1°, the eigenvalue of the open loop mode 8.22 X 10°-7+j1.96 x has a pair of unstab cause the open loop induce higher fuel For example, when e eigenvalues to be 0-*, whereas in the zero eccentricity case, the eigenvalues are [0,0,+/1.11 < 10-7]. Therefore, the stability property is changed. that formation flying centricity. Based on t n areal mission, there is a high probability missions are initialized to have nonzero ec- his analysis, it is not difficult to see that the eccentricity needs to be modeled into the system when the control system is designed. Furthermore, as shown in Eq. (11) the semi- major axis affects the matrix elements

Fig. 3. Parametric absolute deviation due to nonzero eccentricity and different semimajor axis In general, J decreases the orbit altitude by 2% per orbit, and atmospheric drag in altitudes above 500 km is much less than the J, perturbation. Therefore, the J, and drag models are assumed to be bounded as 2% of the nominal system (CW model), and the actuation system and sensor noise is assumed to be 2%. As an example, for the eccentricity and semimajor axis parametric un- certainties, if the mission requires that the maximum eccentricity is bounded by 0.1 and the semimajor axis is varying between 7,178 and 7,978 km, the uncertainty magnitudes are calculated as W, =0.47, W,=2.3 X 10-7, W3=0.14,W4=0.44, and W;=0.29 as shown in Fig. 3(a and b). In Fig. 3, two extreme cases are shown for the semimajor axes to be 7,178 and 7,978 km.

Fig. 4. Open-loop model with parametric uncertainty The input—output relation for the open loop is shown in Fig. 4. u;, i€ [1,3], is the control and y;, ie [1,6] is composed of rela- tive position and velocity. w; and z;, i¢[1,7] are the inputs and outputs of the parametric uncertainties block. Without noise and other functional uncertainties, the system is written in a state

Cochran (1997), and Soltine and Li (1990). Time response and fuel consumptions for a 1-year mission will be compared with the p. controller developed previously.

LQR controller is designed as shown in Fig. 6 and the LQ track- ing states are [p’,p7, fe7dr]’ e R'**' where the error signal is the command signal minus the relative position state: e=r—p. The weighting function used in finding feedback gains are listed in Egs. (17) and (18). The weighting function Q is heavily weighted on the error signal as the controller is used to drive the error to zero. In order to achieve the same level of fuel consumption as the designed w and sliding model controllers, an extremely high weighting function is in the R matrix Simulation Results SMC and Parametric Optimization

The fuel consumptions (AV) calculated for each simulation are shown in Table 5 for the two simulation cases. Here the annual AV is calculated based on the average budget obtained from the 15 h simulation. In the circular reference orbit, all three methods require a similar AV budget for a 1 year station keeping mission. In this table, it can be seen that using the p» controller, both these two missions’ AV is less than 50 m/s/year, whereas, if SMC is The re onstrat can ac SMC ( contro stable better robust hieve a muc on the order or this simu performance to functiona ative position errors are shown in Figs. 13-15. It is dem- ed that for this challenge mission the designed wp controller h better control precision (70 m) than that of of 3 km). The control commands for the three lers are shown in Figs. 16-18. The LQR controller is un- ated mission. There are several reasons for the in p» controller. First, the LQR and SMC are uncertainties, and compared with pw control- ler, they are conservative. Second, the w controller is designed considering both ro considers the stability under uncertainty cases. Note that the LOR synthesis is only tracking the relative position, whereas SMC and bustness and performance, whereas SMC only

Fig. 7. Relative position error in the circular reference orbit case (w)

Fig. 9. Relative position error in the circular reference orbit case (SMC) Fig. 8. Relative position error in the circular reference orbit case (LQR)

In this paper, an uncertainty model with the nonzero eccentricity and varying semimajor axis consideration is derived based on the nonlinear relative position model. Because even a very small ec- centricity will change the linear system’s open loop stability attributes, the eccentricity and semimajor axis variations are mod- eled into parametric uncertainties. Furthermore, J, effects and atmospheric drag, actuation, and sensor noise are bounded by functional uncertainties. A robust controller is designed capable of achieving desired performance, robustness to uncertainties, and fuel consumption requirement. For comparison, LQR and sliding mode controllers are developed and extensively tuned to get the same order of AV budget as that of the designed pt controller. To illustrate the advantages of the proposed control method, a chal- lenge mission is chosen for analysis. In the nominal mission, the reference orbit is assumed to be an 800 km altitude (7,178 km semimajor axis) circular orbit with a 35° inclination. The de- Based on the comparison in control precision, stability, and fuel consumption, it can be seen that the designed p controller has a much better performance.

Fig. 14. Relative position error in the elliptic reference orbit case (LQR, unstable)

Fig. 15. Relative position error in the elliptic reference orbit case (SMC)

Fig. 18. Control accelerations in the elliptic reference orbit (SMC) Fig. 17. Control accelerations in the elliptic reference orbit (LQR, unstable)

space form as shown in Eq. (16) including states [p’,p7]’ e R°*!, exogenous signals [w’,u7]? ¢R!*!, uncertainty inp z<€R’*', and output signals y e R*!

Table 2. NP, RS, and RP Table 1. Gain and Phase Margins

Table 4. Steady State Errors used, the second mission’s AV is too high. Compared to the SMC controller, the controller has less computational cost and does not have implicit dynamics inversion which needs precise knowl- edge of the dynamics model and may cancel the natural relative motion and increase the fuel consumption. The AV used in the SMC (1.72 km/s/year) for the second mission is similar to the reference (Yeh et al. 2000) (4.8 km/s/year), which is reasonable because the sliding mode control is based on Lyapunov theory.

Table 5. Fuel Consumption per Year signed controller based on the nominal mission is planned to be used later in a mission with a reference orbit having a 7,978 km semimajor axis, an eccentricity equals 0.1, and a 35° inclination. The same designed w controller is able to keep the formation with almost the same fuel consumption (43.86 m/s/year) as used in the nominal system (39.65 m/s/year). However, as shown in the simulation, the designed (after extensive tuning) linear robust control (LQR) and nonlinear robust controller (SMC) have either serious fuel consumption penalties (1.72 km/s/year for SMC) or are unstable (LQR).

descriptionView Paper arrow_downwardDownload

IsoBED: a tool for automatic calculation of biologically equivalent fractionation schedules in radiotherapy using IMRT with a simultaneous integrated boost (SIB) technique

by lidia strigari

2011, Journal of Experimental & Clinical Cancer Research

Background An advantage of the Intensity Modulated Radiotherapy (IMRT) technique is the feasibility to deliver different therapeutic dose levels to PTVs in a single treatment session using the Simultaneous Integrated Boost (SIB)... more

descriptionView Paper arrow_downwardDownload

Analysis of MIR-18 results for physical and biological dosimetry: radiation shielding effectiveness in LEO

by Jerry Williams

2000, Radiation Measurements

We compare models of radiation transport and biological response to physical and biological dosimetry results from astronauts on the Mir space station. Transport models are shown to be in good agreement with physical measurements and... more

descriptionView Paper arrow_downwardDownload

Investigation of various growth mechanisms of solid tumour growth within the linear-quadratic model for radiotherapy

by Helen McAneney

2007, Physics in Medicine and Biology

The standard linear-quadratic survival model for radiotherapy is used to investigate different schedules of radiation treatment planning to study how these may be affected by different tumour repopulation kinetics between treatments. The... more

Figure 1. Scaling of the four growth mechanisms such that the rate of change of N at No is
comparable, taken for K = 1, No = 0.1K and t2 = 60 days. Part (a) depicts dN /dt versus N as
given by equations (3), (6), (11) and (16). Part (b) depicts N versus time given by equations (4),
(7), (12) and (17). Note that the exponential and restricted exponential growth mechanisms are
linear in terms of N whilst logistic and Gompertzian growth mechanisms are nonlinear.

Figure 2. Illustrative examples of the effect of the treatment scheme on the survival fraction of
a tumour over time given the four models for the re-growth mechanism. The findings at the end
of such treatment schedules are tabulated in tables 1-5 for various parameters: (a) from table 3
with w/B = 5 Gy, t2 = 30 days, No/K = 0.5, (b) from table 5 with w/f = 20 Gy, t2 = 30 days,
No/K = 0.5. Note the continued re-growth of the tumour over the weekend period.

Table 2. Clonogenic survival fraction for an optimized treatment schedule. The optimal schedule
has been found based on exponential re-growth and applied to the other re-growth models as well.
a/B = 5 Gy, a = 0.2 Gy~!, D = 2.308 Gy, t = b(@D + BD*)[g(a + b)]~! and n% = [tolt?/D]
(Wheldon et al 1977).

Table 1. Clonogenic survival fraction given a uniform treatment schedule. D = 2 Gy, t = 1 day,
n = 30,a =0.2Gy!.
* ty is the tumour doubling time which is constant in the case of exponential growth. For the
limited growth laws, this parameter is no longer constant. It is used to obtain the exponential
repopulation rate g, and the scaled growth behaviour g; of the restricted exponential model,
logistic model and Gompertz model (i = r,/, g) respectively.

Table 3. Clonogenic survival fraction for a standardized treatment schedule where treatment is
given only on weekdays. D = 2 Gy, n = 30,a = 0.2 Gy!.
Table 4. Clonogenic survival fraction given accelerated fractionation, as in case of advanced head.
and neck (O’Sullivan et al 2002). Treatment schedule given only on weekdays with additional
Tuesday and Thursday treatments. D = 2 Gy, n = 33 over 31 days, a = 0.2 Gy7!.

Table 5. Same as table 4 but with D = 1.8 Gy.

descriptionView Paper arrow_downwardDownload

Evaluation of the antineoplastic activity of guduchi ( Tinospora cordifolia) in cultured HeLa cells

by Ganesh Jagetia

1998, Cancer Letters

Exposure of HeLa cells to 0, 5, 10, 25, 50 and 100 mg/ml of guduchi extracts (methanol, aqueous and methylene chloride) resulted in a dose-dependent but significant increase in cell killing, when compared to non-drug-treated controls. The... more

descriptionView Paper arrow_downwardDownload

Assessing solar radiation models using multiple variables over Turkey

by Fatih Evrendilek

2008, Climate Dynamics

Solar radiation drives many environmental processes; however, needs to be estimated indirectly from more commonly measured meteorological variables since these data are not readily available from most climate stations. A geo-referenced... more

descriptionView Paper arrow_downwardDownload

A modelled comparison of the effects of using different ways to compensate for missed treatment days in radiotherapy

by Nick Slevin

1996, Clinical Oncology

There is much evidence for the detrimental effect on tumour control of missed treatment days during radiotherapy, amounting for example to approximately a 1.6% absolute decrease in local control probability per day of treatment... more

descriptionView Paper arrow_downwardDownload

Log In

Linear Quadratic regulator

Related Topics