Local Optimization

Mi c h a el , L a m b r e c h t , M a r c, Towill, D e ni s Roys t o n a n d Van d e Veld e, Wi m 2 0 0 8. Th e v al u e of c oo r di n a tio n in a t w o-e c h elo n s u p ply c h ai n. IIE Tr a n s a c tio n s 4 0 (3) , p p. 3 4 1-3 5 5. 1 0.

Explainable AI is an important aspect. But, to the best of knowledge, there isn't any end-to-end probabilistic explainable algorithms that is easy to work with. So, using Laplace smoothing idea of Bayes probabilities, an end-to-end probabilistic deep-learning algorithm is proposed. Also, there are three initial ideas related to the machine learning training process of this deep-learning.

Control schemes in real-time sensor-based systems often operate under tight time constraints determined by the system sampling rate. One area where uch constraints are especially severe is the sensorbased motion planning with dynamics in robotics. Though very important both theoretically and in practice, this problem has not been addressed so far. The typical sampling rates in such systems are 20 to 50 %>er second. This leaves only 20 to 50 msec for the whole cycle, including sensing, complex geometric (intelligence) analysis, calculations due to dynamics and control, and motion execution. As a Arst attempt to solve the problem, we show that the time constraints an be met by a combination of a simple model with a carefully chosen analytical solution of the dynamic equations. The resulting control scheme guarantees Convergence and safety of motion, and blends well with kinematic path planning algorithms. Two such strategies are discussed, one using a simple heuristic and the other with local optimization.

At international cat shows cats must be assigned to judges for evaluation. Many criteria must be considered in preparation of such distributions, and there can be several hundred cats signed in. The difficulty of preparing a good distribution therefore exceeds human capacity so we use the methods of artificial intelligence to optimize the distribution of cats among the judges. We present some algorithms that can be used for creating judging distributions at international cat shows. First we defined a fitness function that assigns to a distribution a numerical value that represents its quality. Then, we implemented several algorithms for finding an optimal distribution: exhaustive search, random search, several variants of greedy search, Monte Carlo tree search and local search. We obtained the judging distributions of some previous cat shows and evaluated them using our fitness function. We prepared optimized distributions using the above algorithms and compared them with actually u...

This work proposes the synthesis of aperiodic arrays of realistic antennas excited with uniform amplitude, where the mutual coupling between elements is rigorously taken into account. A cost function that involves the expression of the radiated field of the coupled array is obtained and its gradient is evaluated in order to move the elements in the corresponding direction at each iteration. The synthesis method involves the calculation of the gradient of the field radiated by the coupled array, which is obtained analytically by using a full-wave method based on the Generalized Scattering Matrix, Spherical Mode Expansions and rotation and translation properties of spherical waves.

Purpose: Intensity modulated radiotherapy (IMRT) for cervical esophageal cancer is challenging. Although IMRT techniques using inverse planning algorithms are facilitating the treatment planning process, the irradiation dose to the normal tissues can a critical issue. This study was performed to investigate the effect of beam numbers and their directions and local optimization (1) dose conformity and homogeneity to the planning target volume (PTV) and (2) dose to the organ at risks (OARs). Methods: Four upper esophageal cancer cases were randomly selected for this treatment planning study. Eight IMRT plans were for each case with the same dose-volume constraints but with different beam numbers and arrangements. Local optimization using regular structures drawn automatically around the PTV with margins from 0.5-1.5 cm was performed. IMRT plans were evaluated with respect to isodose distributions, dose-volume histograms (DVHs) parameters, homogeneity index (HI), and conformity index (CI). The statistical comparison between the types of plans was done using the One Way ANOVA test. Results: results showed that IMRT using three or five beams was not sufficient to obtain good dose optimization. The seven field plans showed the best coverage for the PTV with tolerable doses for the OARs, and the beam orientation was very critical. Increasing beams (Bs) number from 7 to 13 did not show significant differences in the PTV coverage, while the mean lung dose was increased. The PTV coverage were 95.1, 95.1, 98.1, 97.3, 97.3, 97.3, 97.0, and 97.0% for 3Bs, 5Bs, 7Bs, 9Bs, 13Bs, 7Bs(30), 7Bs(60) (beam angles were changed from 0 o to 30 o and 60 o), and 7Bs(R) (seven IMRT plans with ring), respectively. The mean heart dose did not exceed 0.36 Gy with p < 0.05. For lung doses, the best plan was the one with 9Bs which reduced lung volume doses V20Gy (%) and V30Gy (%), and reduced mean lung dose from 5.4 to 4.5 Gy with p < 0.05 for 7Bs(R) plans. IMRT improved the homogeneity indices (p > 0.05), yet conformity was better with 9Bs and 7Bs(R) IMRT plans with p < 0.05. Conclusion: Seven equispaced coplanar intensity-modulated beams with an addition of a ring structure can produce desirable dose distributions to the PTV. Moreover, dose-volume of exposed normal lung can be reduced with 9Bs and 7Bs(R) IMRT plans.

Restructuring techniques for And-Inverter Graphs (AIG), such as rewriting and refactoring, are powerful, scalable and fast, achieving highly optimized AIGs after few iterations. However, these techniques are biased by the original AIG structure and limited by single output optimizations. This paper investigates AIG optimization for area, exploring how far Boolean methods can reduce AIG nodes through local optimization. Boolean division is applied for multi-output functions using two-literal divisors and Boolean decomposition is introduced as a method for AIG optimization. Multioutput blocks are extracted from the AIG and optimized, achieving a further AIG node reduction of 7.76% on average for ITC99 and MCNC benchmarks.

Restructuring techniques for And-Inverter Graphs (AIG), such as rewriting and refactoring, are powerful, scalable and fast, achieving highly optimized AIGs after few iterations. However, these techniques are biased by the original AIG structure and limited by single output optimizations. This paper investigates AIG optimization for area, exploring how far Boolean methods can reduce AIG nodes through local optimization. Boolean division is applied for multi-output functions using two-literal divisors and Boolean decomposition is introduced as a method for AIG optimization. Multioutput blocks are extracted from the AIG and optimized, achieving a further AIG node reduction of 7.76% on average for ITC99 and MCNC benchmarks.

A simple parameter-less local optimizer able to solve deterministic problems with building blocks of bounded order is proposed in this article. The algorithm is able to learn and use linkage information during the run. The algorithm is algorithmically simple, easy to implement and with the exception of termination condition, it is completely parameterfree-there is thus no need to tune the population size and other parameters to the problem at hand. An empirical comparison on 3 decomposable functions, each with uniformly scaled building blocks of size 5 and 8, was carried out. The algorithm exhibits quadratic scaling with the problem dimensionality, but the comparison with the extended compact genetic algorithm and Bayesian optimization algorithm shows that it needs lower or comparable number of fitness function evaluations on the majority of functions for the tested problem dimensionalities. The results also suggest that the efficiency of the local optimizer compared to both the estimation-of-distribution algorithms should be better for problem sizes under at least a few hundreds of bits.

Several local search algorithms for real-valued domains (axis parallel line search, Nelder-Mead simplex search, Rosenbrock&#39;s algorithm, quasi-Newton method, NEWUOA, and VXQR) are described and thoroughly compared in this article, embedding them in a multi-start method. Their comparison aims (1) to help the researchers from the evolutionary community to choose the right opponent for their algorithm (to choose an opponent that would constitute a hard-to-beat baseline algorithm), (2) to describe individual features of these algorithms and show how they influence the algorithm on different problems, and (3) to provide inspiration for the hybridization of evolutionary algorithms with these local optimizers. The recently proposed Comparing Continuous Optimizers (COCO) methodology was adopted as the basis for the comparison. The results show that in low dimensional spaces, the old method of Nelder and Mead is still the most successful among those compared, while in spaces of higher dim...

L&#39;économie d&#39;énergie est un enjeu majeur des bâtiments en milieu tropical où les températures extérieures élevées imposent encore trop souvent d&#39;utiliser des systèmes énergivores permettant d&#39;assurer le confort des occupants d&#39;un bâtiment. Les présents travaux concernent les échangeurs de chaleurs passifs fonctionnant à partir du principe de siphon thermique et de ventilation naturelle. Le mur trombe est une application de génie climatique concrète de ce principe. L&#39;objectif est d&#39;optimiser la forme d&#39;un canal vertical chauffé asymétriquement dans lequel l&#39;air s&#39;écoule afin qu&#39;il réponde au mieux à certains objectifs physiques tel que favoriser les échanges de chaleurs entre l&#39;entrée et la sortie du canal. Ce problème physique est celui du mur trombe. On utilise pour cela une méthode d&#39;optimisation topologique adjointe où l&#39;on pénalise les équations de Navier-Stokes par un terme dit de Brinkman issue de la théorie des milieux p...

Highly coherent sensing matrices arise in discretization of continuum imaging problems such as radar and medical imaging when the grid spacing is below the Rayleigh threshold. Algorithms based on techniques of band exclusion (BE) and local optimization (LO) are proposed to deal with such coherent sensing matrices. These techniques are embedded in the existing compressed sensing algorithms such as Orthogonal Matching Pursuit (OMP), Subspace Pursuit (SP), Iterative Hard Thresholding (IHT), Basis Pursuit (BP) and Lasso, and result in the modified algorithms BLOOMP, BLOSP, BLOIHT, BP-BLOT and Lasso-BLOT, respectively. Under appropriate conditions, it is proved that BLOOMP can reconstruct sparse, widely separated objects up to one Rayleigh length in the Bottleneck distance independent of the grid spacing. One of the most distinguishing attributes of BLOOMP is its capability of dealing with large dynamic ranges. The BLO-based algorithms are systematically tested with respect to four performance metrics: dynamic range, noise stability, sparsity and resolution. With respect to dynamic range and noise stability, BLOOMP is the best performer. With respect to sparsity, BLOOMP is the best performer for high dynamic range while for dynamic range near unity BP-BLOT and Lasso-BLOT with the optimized regularization parameter have the best performance. In the noiseless case, BP-BLOT has the highest resolving power up to certain dynamic range. The algorithms BLOSP and BLOIHT are good alternatives to BLOOMP and BP/Lasso-BLOT: they are faster than both BLOOMP and BP/Lasso-BLOT and shares, to a lesser degree, BLOOMP's amazing attribute with respect to dynamic range. Detailed comparisons with existing algorithms such as Spectral Iterative Hard Thresholding (SIHT) and the frame-adapted BP are given.

Highly coherent sensing matrices arise in discretization of continuum imaging problems such as radar and medical imaging when the grid spacing is below the Rayleigh threshold. Algorithms based on techniques of band exclusion (BE) and local optimization (LO) are proposed to deal with such coherent sensing matrices. These techniques are embedded in the existing compressed sensing algorithms such as Orthogonal Matching Pursuit (OMP), Subspace Pursuit (SP), Iterative Hard Thresholding (IHT), Basis Pursuit (BP) and Lasso, and result in the modified algorithms BLOOMP, BLOSP, BLOIHT, BP-BLOT and Lasso-BLOT, respectively. Under appropriate conditions, it is proved that BLOOMP can reconstruct sparse, widely separated objects up to one Rayleigh length in the Bottleneck distance independent of the grid spacing. One of the most distinguishing attributes of BLOOMP is its capability of dealing with large dynamic ranges. The BLO-based algorithms are systematically tested with respect to four performance metrics: dynamic range, noise stability, sparsity and resolution. With respect to dynamic range and noise stability, BLOOMP is the best performer. With respect to sparsity, BLOOMP is the best performer for high dynamic range while for dynamic range near unity BP-BLOT and Lasso-BLOT with the optimized regularization parameter have the best performance. In the noiseless case, BP-BLOT has the highest resolving power up to certain dynamic range. The algorithms BLOSP and BLOIHT are good alternatives to BLOOMP and BP/Lasso-BLOT: they are faster than both BLOOMP and BP/Lasso-BLOT and shares, to a lesser degree, BLOOMP's amazing attribute with respect to dynamic range. Detailed comparisons with existing algorithms such as Spectral Iterative Hard Thresholding (SIHT) and the frame-adapted BP are given.

Purpose: Intensity modulated radiotherapy (IMRT) for cervical esophageal cancer is challenging. Although IMRT techniques using inverse planning algorithms are facilitating the treatment planning process, the irradiation dose to the normal tissues can a critical issue. This study was performed to investigate the effect of beam numbers and their directions and local optimization (1) dose conformity and homogeneity to the planning target volume (PTV) and (2) dose to the organ at risks (OARs). Methods: Four upper esophageal cancer cases were randomly selected for this treatment planning study. Eight IMRT plans were for each case with the same dose-volume constraints but with different beam numbers and arrangements. Local optimization using regular structures drawn automatically around the PTV with margins from 0.5-1.5 cm was performed. IMRT plans were evaluated with respect to isodose distributions, dose-volume histograms (DVHs) parameters, homogeneity index (HI), and conformity index (CI). The statistical comparison between the types of plans was done using the One Way ANOVA test. Results: results showed that IMRT using three or five beams was not sufficient to obtain good dose optimization. The seven field plans showed the best coverage for the PTV with tolerable doses for the OARs, and the beam orientation was very critical. Increasing beams (Bs) number from 7 to 13 did not show significant differences in the PTV coverage, while the mean lung dose was increased. The PTV coverage were 95.1, 95.1, 98.1, 97.3, 97.3, 97.3, 97.0, and 97.0% for 3Bs, 5Bs, 7Bs, 9Bs, 13Bs, 7Bs(30), 7Bs(60) (beam angles were changed from 0 o to 30 o and 60 o), and 7Bs(R) (seven IMRT plans with ring), respectively. The mean heart dose did not exceed 0.36 Gy with p < 0.05. For lung doses, the best plan was the one with 9Bs which reduced lung volume doses V20Gy (%) and V30Gy (%), and reduced mean lung dose from 5.4 to 4.5 Gy with p < 0.05 for 7Bs(R) plans. IMRT improved the homogeneity indices (p > 0.05), yet conformity was better with 9Bs and 7Bs(R) IMRT plans with p < 0.05. Conclusion: Seven equispaced coplanar intensity-modulated beams with an addition of a ring structure can produce desirable dose distributions to the PTV. Moreover, dose-volume of exposed normal lung can be reduced with 9Bs and 7Bs(R) IMRT plans.

Bayesian networks are graphical statistical models that represent inference between data. For their effectiveness and versatility, they are widely adopted to represent knowledge in different domains. Several research lines address the NP-hard problem of Bayesian network structure learning starting from data: over the years, the machine learning community delivered effective heuristics, while different Evolutionary Algorithms have been devised to tackle this complex problem. This paper presents a Memetic Algorithm for Bayesian network structure learning, that combines the exploratory power of an Evolutionary Algorithm with the speed of local search. Experimental results show that the proposed approach is able to outperform state-of-the-art heuristics on two wellstudied benchmarks.

Projection-free optimization via different variants of the Frank-Wolfe (FW), a.k.a. Conditional Gradient method has become one of the cornerstones in optimization for machine learning since in many cases the linear minimization oracle is much cheaper to implement than projections and some sparsity needs to be preserved. In a number of applications, e.g. Poisson inverse problems or quantum state tomography, the loss is given by a self-concordant (SC) function having unbounded curvature, implying absence of theoretical guarantees for the existing FW methods. We use the theory of SC functions to provide a new adaptive step size for FW methods and prove global convergence rate O(1/k) after k iterations. If the problem admits a stronger local linear minimization oracle, we construct a novel FW method with linear convergence rate for SC functions.

In this study, an interactive forest planning process corresponding to the practical demands was developed and further tested in a challenging forest planning situation in northeastern Finland. The process includes prior preparation of alternative stand-level treatments and a small amount of holding-level forest plans; an interactive planning session consisting of the primary choice of the forest owner's profile; the owner's selection of the best holding-level plan; and finally a local improvement of this plan. The method as a whole aims to bridge the gap between the prevailing planning culture that has developed for private forest planning over three decades in Finland and the planning approach suggested by multiobjective forest planning theory. The usability and characteristics of the process were evaluated through an exercise set given to both forestry students and forest professionals. Tests of the process indicated, among other things, that comparison particularly of stand-level alternatives and offering owners the possibility to make changes and truly affect the end-result of the planning process are seen as important characteristics of the process.

The l1-ball is a nicely structured feasible set that is widely used in many fields (e.g., machine learning, statistics and signal analysis) to enforce some sparsity in the model solutions. In this paper, we devise an active-set strategy for efficiently dealing with minimization problems over the l1-ball and embed it into a tailored algorithmic scheme that makes use of a non-monotone firstorder approach to explore the given subspace at each iteration. We prove global convergence to stationary points. Finally, we report numerical experiments, on two different classes of instances, showing the effectiveness of the algorithm.

Purpose: Intensity modulated radiotherapy (IMRT) for cervical esophageal cancer is challenging. Although IMRT techniques using inverse planning algorithms are facilitating the treatment planning process, the irradiation dose to the normal tissues can a critical issue. This study was performed to investigate the effect of beam numbers and their directions and local optimization (1) dose conformity and homogeneity to the planning target volume (PTV) and (2) dose to the organ at risks (OARs). Methods: Four upper esophageal cancer cases were randomly selected for this treatment planning study. Eight IMRT plans were for each case with the same dose-volume constraints but with different beam numbers and arrangements. Local optimization using regular structures drawn automatically around the PTV with margins from 0.5-1.5 cm was performed. IMRT plans were evaluated with respect to isodose distributions, dose-volume histograms (DVHs) parameters, homogeneity index (HI), and conformity index (CI). The statistical comparison between the types of plans was done using the One Way ANOVA test. Results: results showed that IMRT using three or five beams was not sufficient to obtain good dose optimization. The seven field plans showed the best coverage for the PTV with tolerable doses for the OARs, and the beam orientation was very critical. Increasing beams (Bs) number from 7 to 13 did not show significant differences in the PTV coverage, while the mean lung dose was increased. The PTV coverage were 95.1, 95.1, 98.1, 97.3, 97.3, 97.3, 97.0, and 97.0% for 3Bs, 5Bs, 7Bs, 9Bs, 13Bs, 7Bs(30), 7Bs(60) (beam angles were changed from 0 o to 30 o and 60 o), and 7Bs(R) (seven IMRT plans with ring), respectively. The mean heart dose did not exceed 0.36 Gy with p < 0.05. For lung doses, the best plan was the one with 9Bs which reduced lung volume doses V20Gy (%) and V30Gy (%), and reduced mean lung dose from 5.4 to 4.5 Gy with p < 0.05 for 7Bs(R) plans. IMRT improved the homogeneity indices (p > 0.05), yet conformity was better with 9Bs and 7Bs(R) IMRT plans with p < 0.05. Conclusion: Seven equispaced coplanar intensity-modulated beams with an addition of a ring structure can produce desirable dose distributions to the PTV. Moreover, dose-volume of exposed normal lung can be reduced with 9Bs and 7Bs(R) IMRT plans.

This article presents an immune inspired algorithm to tackle the Multiple Sequence Alignment (MSA) problem. MSA is one of the most important tasks in biological sequence analysis. Although this paper focuses on protein alignments, most of the discussion and methodology may also be applied to DNA alignments. The problem of finding the multiple alignment was investigated in the study by Bonizzoni and Vedova and Wang and Jiang, and proved to be a NP-hard (non-deterministic polynomial-time hard) problem. The presented algorithm, called Immunological Multiple Sequence Alignment Algorithm (IMSA), incorporates two new strategies to create the initial population and specific ad hoc mutation operators. It is based on the 'weighted sum of pairs' as objective function, to evaluate a given candidate alignment. IMSA was tested using both classical benchmarks of BALIBASE (versions 1.0, 2.0 and 3.0), and experimental results indicate that it is comparable with state-of-the-art multiple alignment algorithms, in terms of quality of alignments, weighted Sums-of-Pairs (SP) and Column Score (CS) values. The main novelty of IMSA is its ability to generate more than a single suboptimal alignment, for every MSA instance; this behaviour is due to the stochastic nature of the algorithm and of the populations evolved during the convergence process. This feature will help the decision maker to assess and select a biologically relevant multiple sequence alignment. Finally, the designed algorithm can be used as a local search procedure to properly explore promising alignments of the search space.

Sin clusters in the size range n=4-35 have been investigated, using a combination of global structure optimization methods with DFT and ab-initio calculations. One of the central aims is to provide explanations for the structural transition from prolate to spherical outer shapes at about n=25, as observed in ion mobility measurements. First, several empirical potentials for silicon and a newly generated variant of one of them were better adapted to small silicon clusters, by global optimization of their parameters. The best resulting empirical potentials were then employed in global cluster structure optimizations. The most promising structures from this stage were relaxed further at the DFT level with a hybrid B3LYP functional. For the resulting structures, single point energies have been calculated at the LMP2 level with cc-pVTZ basis set. Results obtained at the DFT level strongly support the shape transition form prolate to spherical structures beginning with Si26. In contrast, ...

The fossil-based energy system is transitioning towards a renewable energy system. One important aspect is the spatial and temporal mismatch between intermitted supply and continuous demand. To ensure a reliable and affordable energy system, we propose an integrated system approach that integrates electricity production, mobility, heating of buildings and water management with a major role for storage and conversion. The minimization of energy transport in such an integrated system indicates the need for local optimization. This study focuses on a comparison between different novel system designs for neighborhood energy and water systems with varying modes of system integration, including all-electric, power-to-heat and power-to-hydrogen. A simulation model is developed to determine the energy and water balance and carry out economic analysis to calculate the system costs of various scenarios. We show that system costs are the lowest in a scenario that combines a hydrogen boiler and...

Purpose: Intensity modulated radiotherapy (IMRT) for cervical esophageal cancer is challenging. Although IMRT techniques using inverse planning algorithms are facilitating the treatment planning process, the irradiation dose to the normal tissues can a critical issue. This study was performed to investigate the effect of beam numbers and their directions and local optimization (1) dose conformity and homogeneity to the planning target volume (PTV) and (2) dose to the organ at risks (OARs). Methods: Four upper esophageal cancer cases were randomly selected for this treatment planning study. Eight IMRT plans were for each case with the same dose-volume constraints but with different beam numbers and arrangements. Local optimization using regular structures drawn automatically around the PTV with margins from 0.5-1.5 cm was performed. IMRT plans were evaluated with respect to isodose distributions, dose-volume histograms (DVHs) parameters, homogeneity index (HI), and conformity index (CI). The statistical comparison between the types of plans was done using the One Way ANOVA test. Results: results showed that IMRT using three or five beams was not sufficient to obtain good dose optimization. The seven field plans showed the best coverage for the PTV with tolerable doses for the OARs, and the beam orientation was very critical. Increasing beams (Bs) number from 7 to 13 did not show significant differences in the PTV coverage, while the mean lung dose was increased. The PTV coverage were 95.1, 95.1, 98.1, 97.3, 97.3, 97.3, 97.0, and 97.0% for 3Bs, 5Bs, 7Bs, 9Bs, 13Bs, 7Bs(30), 7Bs(60) (beam angles were changed from 0 o to 30 o and 60 o), and 7Bs(R) (seven IMRT plans with ring), respectively. The mean heart dose did not exceed 0.36 Gy with p < 0.05. For lung doses, the best plan was the one with 9Bs which reduced lung volume doses V20Gy (%) and V30Gy (%), and reduced mean lung dose from 5.4 to 4.5 Gy with p < 0.05 for 7Bs(R) plans. IMRT improved the homogeneity indices (p > 0.05), yet conformity was better with 9Bs and 7Bs(R) IMRT plans with p < 0.05. Conclusion: Seven equispaced coplanar intensity-modulated beams with an addition of a ring structure can produce desirable dose distributions to the PTV. Moreover, dose-volume of exposed normal lung can be reduced with 9Bs and 7Bs(R) IMRT plans.

SFI Working Papers contain accounts of scientific work of the author(s) and do not necessarily represent the views of the Santa Fe Institute. We accept papers intended for publication in peer-reviewed journals or proceedings volumes, but not papers that have already appeared in print. Except for papers by our external faculty, papers must be based on work done at SFI, inspired by an invited visit to or collaboration at SFI, or funded by an SFI grant. ©NOTICE: This working paper is included by permission of the contributing author(s) as a means to ensure timely distribution of the scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the author(s). It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may be reposted only with the explicit permission of the copyright holder.

In recent years, voltage limit violation and power system load-generation imbalance, i.e., line loading limit violation have been responsible for several incidents of major network collapses leading to partial or even complete blackouts. Alleviation of line overloads is the suitable corrective action in this regard. The control action strategies to limit the line loading to the security limits are generation rescheduling/load shedding. In this paper, an approach based on radial basis function neural network (RBFN) is presented for corrective action planning to alleviate line overloading in an efficient manner. Effectiveness of the proposed method is demonstrated for overloading alleviation under different loading/contingency conditions in 6-bus system and 24-bus RTS system.

In recent years, voltage limit violation and power system load-generation imbalance, i.e., line loading limit violation have been responsible for several incidents of major network collapses leading to partial or even complete blackouts. Alleviation of line overloads is the suitable corrective action in this regard. The control action strategies to limit the line loading to the security limits are generation rescheduling/load shedding. In this paper, an approach based on radial basis function neural network (RBFN) is presented for corrective action planning to alleviate line overloading in an efficient manner. Effectiveness of the proposed method is demonstrated for overloading alleviation under different loading/contingency conditions in 6-bus system and 24-bus RTS system.

The most popular first-order accelerated black-box methods for solving large-scale convex optimization problems are the Fast Gradient Method (FGM) and the Fast Iterative Shrinkage Thresholding Algorithm (FISTA). FGM requires that the objective be finite and differentiable with known gradient Lipschitz constant. FISTA is applicable to the more broad class of composite objectives and is equipped with a line-search procedure for estimating the Lipschitz constant. Nonetheless, FISTA cannot increase the step size and is unable to take advantage of strong convexity. FGM and FISTA are very similar in form. Despite this, they appear to have vastly differing convergence analyses. In this work we generalize the previously introduced augmented estimate sequence framework as well as the related notion of the gap sequence. We showcase the flexibility of our tools by constructing a Generalized Accelerated Composite Gradient Method, that unites FGM and FISTA, along with their most popular variants...

We present a local convergence of two-step solvers for solving nonlinear operator equations under the generalized Lipschitz conditions for the first- and second-order derivatives and for the first order divided differences. In contrast to earlier works, we use our new idea of center average Lipschitz conditions, through which, we define a subset of the original domain that also contains the iterates. Then, the remaining average Lipschitz conditions are at least as tight as the corresponding ones in earlier works. This way, we obtain weaker sufficient convergence criteria, larger radius of convergence, tighter error estimates, and better information on the solution. These extensions require the same effort, since the new Lipschitz functions are special cases of the ones in earlier works. Finally, we give a numerical example that confirms the theoretical results, and compares favorably to the results from previous works.

Numerous problems in signal processing, statistical inference, computer vision, and machine learning, can be cast as large-scale convex optimization problems. Due to their size, many of these problems can only be addressed by first-order accelerated black-box methods. The most popular among these are the Fast Gradient Method (FGM) and the Fast Iterative Shrinkage Thresholding Algorithm (FISTA). FGM requires that the objective be finite and differentiable with known gradient Lipschitz constant. FISTA is applicable to the more broad class of composite objectives and is equipped with a line-search procedure for estimating the Lipschitz constant. Nonetheless, FISTA cannot increase the step size and is unable to take advantage of strong convexity. FGM and FISTA are very similar in form. Despite this, they appear to have vastly differing convergence analyses. In this work we generalize the previously introduced augmented estimate sequence framework as well as the related notion of the gap sequence. We showcase the flexibility of our tools by constructing a Generalized Accelerated Composite Gradient Method, that unites FGM and FISTA, along with their most popular variants. The Lyapunov property of the generalized gap sequence used in deriving our method implies that both FGM and FISTA are amenable to a Lyapunov analysis, common among optimization algorithms. We further showcase the flexibility of our tools by endowing our method with monotonically decreasing objective function values alongside a versatile line-search procedure. By simultaneously incorporating the strengths of FGM and FISTA, our method is able to surpass both in terms of robustness and usability. We support our findings with simulation results on an extensive benchmark of composite problems. Our experiments show that monotonicity has a stabilizing effect on convergence and challenge the notion present in the literature that for strongly convex objectives, accelerated proximal schemes can be reduced to fixed momentum methods.

This paper presents an accelerated composite gradient (ACG) variant, referred to as the AC-ACG method, for solving nonconvex smooth composite minimization problems. As opposed to well-known ACG variants that are either based on a known Lipschitz gradient constant or a sequence of maximum observed curvatures, the current one is based on the average of all past observed curvatures. More specifically, AC-ACG uses a positive multiple of the average of all observed curvatures until the previous iteration as a way to estimate the &quot;function curvature&quot; at the current point and then two resolvent evaluations to compute the next iterate. In contrast to other variable Lipschitz estimation variants, e.g., the ones based on the maximum curvature, AC-ACG always accepts the aforementioned iterate regardless how poor the Lipschitz estimation turns out to be. Finally, computational results are presented to illustrate the efficiency of AC-ACG on both randomly generated and real-world proble...

In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two convex terms: one is smooth and given by a black-box oracle, and another is general but simple and its structure is known. Despite to the bad properties of the sum, such problems, both in convex and nonconvex cases, can be solved with eciency typical for the good part of the objective. For convex problems of the above structure, we consider primal and dual variants of the gradient method (converge as O ‡ 1 k · ), and an accelerated multistep version with convergence rate O ‡ 1 k2 · , where k is the iteration counter. For all methods, we suggest some ecient “line search” procedures and show that the additional computational work necessary for estimating the unknown problem class parameters can only multiply the complexity of each iteration by a small constant factor. We present also the results of preliminary computational experiments, which confirm the su...

In this paper, we propose an efficient approach for solving a class of convex optimization problems in Hilbert spaces. Our feasible region is a (possibly infinite-dimensional) simple convex set, i.e. we assume that projections on this set are computationally easy to compute. The problem we consider is the minimization of a convex function over this region under the additional constraint Au ∈ T , where A is a linear operator and T is a (finite-dimensional) convex set whose dimension is small as compared to the dimension of the feasible region. In our approach, we dualize the linear constraints, solve the resulting dual problem with a purely dual gradient method and show how to reconstruct an approximate primal solution. In order to accelerate our scheme, we introduce a novel double smoothing technique that involves regularization of the dual problem to allow the use of a fast gradient method. As a result, we obtain a method with complexity O( 1 ln 1 ) gradient iterations, where is th...

In this paper, we present new second-order methods with convergence rate $$O\\left( k^{-4}\\right) $$ O k - 4 , where k is the iteration counter. This is faster than the existing lower bound for this type of schemes (Agarwal and Hazan in Proceedings of the 31st conference on learning theory, PMLR, pp. 774–792, 2018; Arjevani and Shiff in Math Program 178(1–2):327–360, 2019), which is $$O\\left( k^{-7/2} \\right) $$ O k - 7 / 2 . Our progress can be explained by a finer specification of the problem class. The main idea of this approach consists in implementation of the third-order scheme from Nesterov (Math Program 186:157–183, 2021) using the second-order oracle. At each iteration of our method, we solve a nontrivial auxiliary problem by a linearly convergent scheme based on the relative non-degeneracy condition (Bauschke et al. in Math Oper Res 42:330–348, 2016; Lu et al. in SIOPT 28(1):333–354, 2018). During this process, the Hessian of the objective function is computed once, and the...

In this paper, we study the auxiliary problems that appear in p-order tensor methods for unconstrained minimization of convex functions with ν-Hölder continuous pth derivatives. This type of auxiliary problems corresponds to the minimization of a (p + ν)-order regularization of the pth-order Taylor approximation of the objective. For the case p = 3, we consider the use of Gradient Methods with Bregman distance. When the regularization parameter is sufficiently large, we prove that the referred methods take at most O(log(−1)) iterations to find either a suitable approximate stationary point of the tensor model or an-approximate stationary point of the original objective function.

In this paper we analyze several new methods for solving optimization problems with the objective function formed as a sum of two convex terms: one is smooth and given by a black-box oracle, and another is general but simple and its structure is known. Despite to the bad properties of the sum, such problems, both in convex and nonconvex cases, can be solved with efficiency typical for the good part of the objective. For convex problems of the above structure, we consider primal and dual variants of the gradient method (converge as O(1/k)), and an accelerated multistep version with convergence rate O(1/k 2), where k is the iteration counter. For all methods, we suggest some efficient "line search" procedures and show that the additional computational work necessary for estimating the unknown problem class parameters can only multiply the complexity of each iteration by a small constant factor. We present also the results of preliminary computational experiments, which confirm the superiority of the accelerated scheme.

In this paper, an algorithm for solving a mathematical programming problem with complementarity (or equilibrium) constraints (MPEC) is introduced, which uses the active-set methodology while maintaining the complementarity restrictions throughout the procedure. Finite convergence of the algorithm to a strongly stationary point of the MPEC is established under reasonable hypotheses. The algorithm can be easily implemented by adopting any active-set code for nonlinear programming. Computational experience is included to highlight the efficacy of the proposed method in practice.

The optimal design of electrical machines can be mathematically modeled as a (mixed-integer) nonlinear optimization problem. We investigate the impact of different mathematical formulations on the results obtained using a local optimization solver which is well-known in the engineering community: MatLab's fmincon function. Our analysis is based on six different mathematically equivalent formulations for the same problem of the design of an electrical machine without slot. Our results underline the important impact that formulation differences may have on solver performance even on a small example of design.

In recent years, voltage limit violation and power system load-generation imbalance, i.e., line loading limit violation have been responsible for several incidents of major network collapses leading to partial or even complete blackouts. Alleviation of line overloads is the suitable corrective action in this regard. The control action strategies to limit the line loading to the security limits are generation rescheduling/load shedding. In this paper, an approach based on radial basis function neural network (RBFN) is presented for corrective action planning to alleviate line overloading in an efficient manner. Effectiveness of the proposed method is demonstrated for overloading alleviation under different loading/contingency conditions in 6-bus system and 24-bus RTS system.

Inspired by a method by Jones et al. (1993), we present a global optimization algorithm based on multilevel coordinate search. It is guaranteed to converge if the function is continuous in the neighborhood of a global minimizer. By starting a local search from certain good points, an improved convergence result is obtained. We discuss implementation details and give some numerical results.

Several local search algorithms for real-valued domains (axis-parallel line search, Nelder-Mead simplex search, Rosenbrock's algorithm, quasi-Newton method, NEWUOA and VXQR) are described and thoroughly compared in this article, embedding them in a multistart method. Their comparison aims (1) to help the researchers from the evolutionary community to choose the right opponent for their algorithm (to choose an opponent that would constitute a hard-to-beat baseline algorithm), (2) to describe individual features of these algorithms and show how they influence the algorithm on different problems, and (3) to provide inspiration for the hybridization of evolutionary algorithms with these local optimizers. The recently proposed Comparing Continuous Optimizers (COCO) methodology was adopted as the basis for the comparison. The results show that in low dimensional spaces, the old method of Nelder and Mead is still the most successful among those compared, while in spaces of higher dimensions it is better to choose an algorithm based on quadratic modeling, like NEWUOA or a quasi-Newton method.

Inspired by a method by Jones et al.(1993), we present a global optimization algorithm based on multilevel coordinate search. It is guaranteed to converge if the function is continuous in the neighborhood of a global minimizer. By starting a local search from certain good points, an improved convergence result is obtained. We discuss implementation details and give some numerical results.

Several local search algorithms for real-valued domains (axis-parallel line search, Nelder-Mead simplex search, Rosenbrock's algorithm, quasi-Newton method, NEWUOA and VXQR) are described and thoroughly compared in this article, embedding them in a multistart method. Their comparison aims (1) to help the researchers from the evolutionary community to choose the right opponent for their algorithm (to choose an opponent that would constitute a hard-to-beat baseline algorithm), (2) to describe individual features of these algorithms and show how they influence the algorithm on different problems, and (3) to provide inspiration for the hybridization of evolutionary algorithms with these local optimizers. The recently proposed Comparing Continuous Optimizers (COCO) methodology was adopted as the basis for the comparison. The results show that in low dimensional spaces, the old method of Nelder and Mead is still the most successful among those compared, while in spaces of higher dimensions it is better to choose an algorithm based on quadratic modeling, like NEWUOA or a quasi-Newton method.

The optimal design of electrical machines can be mathematically modeled as a (mixed-integer) nonlinear optimization problem. We investigate the impact of different mathematical formulations on the results obtained using a local optimization solver which is well-known in the engineering community: MatLab's fmincon function. Our analysis is based on six different mathematically equivalent formulations for the same problem of the design of an electrical machine without slot. Our results underline the important impact that formulation differences may have on solver performance even on a small example of design.

The paper proposes local and global optimization schemes for efficient TCP buffer allocation in an HTTP server. The proposed local optimization scheme dynamically adjusts the TCP send-buffer size to the connection and server characteristics. The global optimization scheme divides a certain amount of buffer space among all active TCP connections. These schemes are of increasing importance due to the large scale of TCP connection characteristics. The schemes are compared to the static allocation policy employed by a typical HTTP server, and shown to achieve considerable improvement to server performance and better utilization of its resources. The schemes require only minor code changes and only at the server.

We study a coordination scheme in a two echelon supply chain. It involves sharing details of replenishment rules, lead-times, demand patterns and tuning the replenishment rules to exploit the supply chain's cost structure. We examine four different coordination strategies; naïve operation, local optimisation, global optimisation and altruistic behaviour on behalf of the retailer.

This article presents an immune inspired algorithm to tackle the Multiple Sequence Alignment (MSA) problem. MSA is one of the most important tasks in biological sequence analysis. Although this paper focuses on protein alignments, most of the discussion and methodology may also be applied to DNA alignments. The problem of finding the multiple alignment was investigated in the study by Bonizzoni and Vedova and Wang and Jiang, and proved to be a NP-hard (non-deterministic polynomial-time hard) problem. The presented algorithm, called Immunological Multiple Sequence Alignment Algorithm (IMSA), incorporates two new strategies to create the initial population and specific ad hoc mutation operators. It is based on the 'weighted sum of pairs' as objective function, to evaluate a given candidate alignment. IMSA was tested using both classical benchmarks of BALIBASE (versions 1.0, 2.0 and 3.0), and experimental results indicate that it is comparable with state-of-the-art multiple alignment algorithms, in terms of quality of alignments, weighted Sums-of-Pairs (SP) and Column Score (CS) values. The main novelty of IMSA is its ability to generate more than a single suboptimal alignment, for every MSA instance; this behaviour is due to the stochastic nature of the algorithm and of the populations evolved during the convergence process. This feature will help the decision maker to assess and select a biologically relevant multiple sequence alignment. Finally, the designed algorithm can be used as a local search procedure to properly explore promising alignments of the search space.