Continuous Goal-Directed Actions (CGDA) is a robot imitation framework that encodes actions as the changes they produce on the environment. While it presents numerous advantages with respect to other robot imitation frameworks in terms of generalization and portability, final robot joint trajectories for the execution of actions are not necessarily encoded within the model. This is studied as an optimization problem, and the solution is computed through evolutionary algorithms in simulated environments. Evolutionary algorithms require a large number of evaluations, which had made the use of these algorithms in real world applications very challenging. This paper presents online evolutionary strategies, as a change of paradigm within CGDA execution. Online evolutionary strategies shift and merge motor execution into the planning loop. A concrete online evolutionary strategy, Online Evolved Trajectories (OET), is presented. OET drastically reduces computational times between motor executions, and enables working in real world dynamic environments and/or with human collaboration. Its performance has been measured against Full Trajectory Evolution (FTE) and Incrementally Evolved Trajectories (IET), obtaining the best overall results. Experimental evaluations are performed on the TEO full-sized humanoid robot with "paint" and "iron" actions that together involve vision, kinesthetic and force features.
Multiobjective Evolutionary Algorithms based on Decomposition (MOEA/D) represent a widely used class of population-based metaheuristics for the solution of multicriteria optimization problems. We introduce the MOEADr package, which offers many of these variants as instantiations of a component-oriented framework. This approach contributes for easier reproducibility of existing MOEA/D variants from the literature, as well as for faster development and testing of new composite algorithms. The package offers an standardized, modular implementation of MOEA/D based on this framework, which was designed aiming at providing researchers and practitioners with a standard way to discuss and express MOEA/D variants. In this paper we introduce the design principles behind the MOEADr package, as well as its current components. Three case studies are provided to illustrate the main aspects of the package.
Fuzzing has become the de facto standard technique for finding software vulnerabilities. However, even state-of-the-art fuzzers are not very efficient at finding hard-to-trigger software bugs. Most popular fuzzers use evolutionary guidance to generate inputs that can trigger different bugs. Such evolutionary algorithms, while fast and simple to implement, often get stuck in fruitless sequences of random mutations. Gradient-guided optimization presents a promising alternative to evolutionary guidance. Gradient-guided techniques have been shown to significantly outperform evolutionary algorithms at solving high-dimensional structured optimization problems in domains like machine learning by efficiently utilizing gradients or higher-order derivatives of the underlying function. However, gradient-guided approaches are not directly applicable to fuzzing as real-world program behaviors contain many discontinuities, plateaus, and ridges where the gradient-based methods often get stuck. We observe that this problem can be addressed by creating a smooth surrogate function approximating the discrete branching behavior of target program. In this paper, we propose a novel program smoothing technique using surrogate neural network models that can incrementally learn smooth approximations of a complex, real-world program's branching behaviors. We further demonstrate that such neural network models can be used together with gradient-guided input generation schemes to significantly improve the fuzzing efficiency. Our extensive evaluations demonstrate that NEUZZ significantly outperforms 10 state-of-the-art graybox fuzzers on 10 real-world programs both at finding new bugs and achieving higher edge coverage. NEUZZ found 31 unknown bugs that other fuzzers failed to find in 10 real world programs and achieved 3X more edge coverage than all of the tested graybox fuzzers for 24 hours running.
It has been observed that some working principles of evolutionary algorithms, in particular, the influence of the parameters, cannot be understood from results on the asymptotic order of the runtime, but only from more precise results. In this work, we complement the emerging topic of precise runtime analysis with a first precise complexity theoretic result. Our vision is that the interplay between algorithm analysis and complexity theory becomes a fruitful tool also for analyses more precise than asymptotic orders of magnitude. As particular result, we prove that the unary unbiased black-box complexity of the OneMax benchmark function class is $n \ln(n) - cn \pm o(n)$ for a constant $c$ which is between $0.2539$ and $0.2665$. This runtime can be achieved with a simple (1+1)-type algorithm using a fitness-dependent mutation strength. When translated into the fixed-budget perspective, our algorithm finds solutions which are roughly 13\% closer to the optimum than those of the best previously known algorithms. To prove our results, we formulate several new versions of the variable drift theorems, which also might be of independent interest.
Many real-world optimization problems require significant resources for objective function evaluations. This is a challenge to evolutionary algorithms, as it limits the number of available evaluations. One solution are surrogate models, which replace the expensive objective. A particular issue in this context are hierarchical variables. Hierarchical variables only influence the objective function if other variables satisfy some condition. We study how this kind of hierarchical structure can be integrated into the model based optimization framework. We discuss an existing kernel and propose alternatives. An artificial test function is used to investigate how different kernels and assumptions affect model quality and search performance.
Existing studies have shown that the conventional multi-objective evolutionary algorithms (MOEAs) based on decomposition may lose the population diversity when solving some many-objective optimization problems. In this paper, a simple decomposition-based MOEA with local iterative update (LIU) is proposed. The LIU strategy has two features that are expected to drive the population to approximate the Pareto Front with good distribution. One is that only the worst solution in the current neighborhood is swapped out by the newly generated offspring, preventing the population from being occupied by copies of a few individuals. The other is that its iterative process helps to assign better solutions to subproblems, which is beneficial to make full use of the similarity of solutions to neighboring subproblems and explore local areas in the search space. In addition, the time complexity of the proposed algorithm is the same as that of MOEA/D, and lower than that of other known MOEAs, since it considers only individuals within the current neighborhood at each update. The algorithm is compared with several of the best MOEAs on problems chosen from two famous test suites DTLZ and WFG. Experimental results demonstrate that only a handful of running instances of the algorithm on DTLZ4 lose their population diversity. What's more, the algorithm wins in most of the test instances in terms of both running time and solution quality, indicating that it is very effective in solving MaOPs.
Stochastic gradient descent is the most prevalent algorithm to train neural networks. However, other approaches such as evolutionary algorithms are also applicable to this task. Evolutionary algorithms bring unique trade-offs that are worth exploring, but computational demands have so far restricted exploration to small networks with few parameters. We implement an evolutionary algorithm that executes entirely on the GPU, which allows to efficiently batch-evaluate a whole population of networks. Within this framework, we explore the limited evaluation evolutionary algorithm for neural network training and find that its batch evaluation idea comes with a large accuracy trade-off. In further experiments, we explore crossover operators and find that unprincipled random uniform crossover performs extremely well. Finally, we train a network with 92k parameters on MNIST using an EA and achieve 97.6 % test accuracy compared to 98 % test accuracy on the same network trained with Adam. Code is available at https://github.com/jprellberg/gpuea.
Many real-world optimization problems occur in environments that change dynamically or involve stochastic components. Evolutionary algorithms and other bio-inspired algorithms have been widely applied to dynamic and stochastic problems. This survey gives an overview of major theoretical developments in the area of runtime analysis for these problems. We review recent theoretical studies of evolutionary algorithms and ant colony optimization for problems where the objective functions or the constraints change over time. Furthermore, we consider stochastic problems under various noise models and point out some directions for future research.
Large set of linear equations, especially for sparse and structured coefficient (matrix) equations, solutions using classical methods become arduous. And evolutionary algorithms have mostly been used to solve various optimization and learning problems. Recently, hybridization of classical methods (Jacobi method and Gauss-Seidel method) with evolutionary computation techniques have successfully been applied in linear equation solving. In the both above hybrid evolutionary methods, uniform adaptation (UA) techniques are used to adapt relaxation factor. In this paper, a new Jacobi Based Time-Variant Adaptive (JBTVA) hybrid evolutionary algorithm is proposed. In this algorithm, a Time-Variant Adaptive (TVA) technique of relaxation factor is introduced aiming at both improving the fine local tuning and reducing the disadvantage of uniform adaptation of relaxation factors. This algorithm integrates the Jacobi based SR method with time variant adaptive evolutionary algorithm. The convergence theorems of the proposed algorithm are proved theoretically. And the performance of the proposed algorithm is compared with JBUA hybrid evolutionary algorithm and classical methods in the experimental domain. The proposed algorithm outperforms both the JBUA hybrid algorithm and classical methods in terms of convergence speed and effectiveness.
For small number of equations, systems of linear (and sometimes nonlinear) equations can be solved by simple classical techniques. However, for large number of systems of linear (or nonlinear) equations, solutions using classical method become arduous. On the other hand evolutionary algorithms have mostly been used to solve various optimization and learning problems. Recently, hybridization of evolutionary algorithm with classical Gauss-Seidel based Successive Over Relaxation (SOR) method has successfully been used to solve large number of linear equations; where a uniform adaptation (UA) technique of relaxation factor is used. In this paper, a new hybrid algorithm is proposed in which a time-variant adaptation (TVA) technique of relaxation factor is used instead of uniform adaptation technique to solve large number of linear equations. The convergence theorems of the proposed algorithms are proved theoretically. And the performance of the proposed TVA-based algorithm is compared with the UA-based hybrid algorithm in the experimental domain. The proposed algorithm outperforms the hybrid one in terms of efficiency.
Recently hybrid evolutionary computation (EC) techniques are successfully implemented for solving large sets of linear equations. All the recently developed hybrid evolutionary algorithms, for solving linear equations, contain both the recombination and the mutation operations. In this paper, two modified hybrid evolutionary algorithms contained time-variant adaptive evolutionary technique are proposed for solving linear equations in which recombination operation is absent. The effectiveness of the recombination operator has been studied for the time-variant adaptive hybrid algorithms for solving large set of linear equations. Several experiments have been carried out using both the proposed modified hybrid evolutionary algorithms (in which the recombination operation is absent) and corresponding existing hybrid algorithms (in which the recombination operation is present) to solve large set of linear equations. It is found that the number of generations required by the existing hybrid algorithms (i.e. the Gauss-Seidel-SR based time variant adaptive (GSBTVA) hybrid algorithm and the Jacobi-SR based time variant adaptive (JBTVA) hybrid algorithm) and modified hybrid algorithms (i.e. the modified Gauss-Seidel-SR based time variant adaptive (MGSBTVA) hybrid algorithm and the modified Jacobi-SR based time variant adaptive (MJBTVA) hybrid algorithm) are comparable. Also the proposed modified algorithms require less amount of computational time in comparison to the corresponding existing hybrid algorithms. As the proposed modified hybrid algorithms do not contain recombination operation, so they require less computational effort, and also they are more efficient, effective and easy to implement.
Now-a-days, it is important to find out solutions of Multi-Objective Optimization Problems (MOPs). Evolutionary Strategy helps to solve such real world problems efficiently and quickly. But sequential Evolutionary Algorithms (EAs) require an enormous computation power to solve such problems and it takes much time to solve large problems. To enhance the performance for solving this type of problems, this paper presents a new Distributed Novel Evolutionary Strategy Algorithm (DNESA) for Multi-Objective Optimization. The proposed DNESA applies the divide-and-conquer approach to decompose population into smaller sub-population and involves multiple solutions in the form of cooperative sub-populations. In DNESA, the server distributes the total computation load to all associate clients and simulation results show that the time for solving large problems is much less than sequential EAs. Also DNESA shows better performance in convergence test when compared with other three well-known EAs.
Most real-world Planning problems are multi-objective, trying to minimize both the makespan of the solution plan, and some cost of the actions involved in the plan. But most, if not all existing approaches are based on single-objective planners, and use an aggregation of the objectives to remain in the single-objective context. Divide and Evolve (DaE) is an evolutionary planner that won the temporal deterministic satisficing track at the last International Planning Competitions (IPC). Like all Evolutionary Algorithms (EA), it can easily be turned into a Pareto-based Multi-Objective EA. It is however important to validate the resulting algorithm by comparing it with the aggregation approach: this is the goal of this paper. The comparative experiments on a recently proposed benchmark set that are reported here demonstrate the usefulness of going Pareto-based in AI Planning.
The classical Geiringer theorem addresses the limiting frequency of occurrence of various alleles after repeated application of crossover. It has been adopted to the setting of evolutionary algorithms and, a lot more recently, reinforcement learning and Monte-Carlo tree search methodology to cope with a rather challenging question of action evaluation at the chance nodes. The theorem motivates novel dynamic parallel algorithms that are explicitly described in the current paper for the first time. The algorithms involve independent agents traversing a dynamically constructed directed graph that possibly has loops. A rather elegant and profound category-theoretic model of cognition in biological neural networks developed by a well-known French mathematician, professor Andree Ehresmann jointly with a neurosurgeon, Jan Paul Vanbremeersch over the last thirty years provides a hint at the connection between such algorithms and Hebbian learning.
Population-based evolutionary algorithms (EAs) have been widely applied to solve various optimization problems. The question of how the performance of a population-based EA depends on the population size arises naturally. The performance of an EA may be evaluated by different measures, such as the average convergence rate to the optimal set per generation or the expected number of generations to encounter an optimal solution for the first time. Population scalability is the performance ratio between a benchmark EA and another EA using identical genetic operators but a larger population size. Although intuitively the performance of an EA may improve if its population size increases, currently there exist only a few case studies for simple fitness functions. This paper aims at providing a general study for discrete optimisation. A novel approach is introduced to analyse population scalability using the fundamental matrix. The following two contributions summarize the major results of the current article. (1) We demonstrate rigorously that for elitist EAs with identical global mutation, using a lager population size always increases the average rate of convergence to the optimal set; and yet, sometimes, the expected number of generations needed to find an optimal solution (measured by either the maximal value or the average value) may increase, rather than decrease. (2) We establish sufficient and/or necessary conditions for the superlinear scalability, that is, when the average convergence rate of a $(\mu+\mu)$ EA (where $\mu\ge2$) is bigger than $\mu$ times that of a $(1+1)$ EA.
This paper introduces a novel idea for representation of individuals using quaternions in swarm intelligence and evolutionary algorithms. Quaternions are a number system, which extends complex numbers. They are successfully applied to problems of theoretical physics and to those areas needing fast rotation calculations. We propose the application of quaternions in optimization, more precisely, we have been using quaternions for representation of individuals in Bat algorithm. The preliminary results of our experiments when optimizing a test-suite consisting of ten standard functions showed that this new algorithm significantly improved the results of the original Bat algorithm. Moreover, the obtained results are comparable with other swarm intelligence and evolutionary algorithms, like the artificial bees colony, and differential evolution. We believe that this representation could also be successfully applied to other swarm intelligence and evolutionary algorithms.
Many optimization problems arising in applications have to consider several objective functions at the same time. Evolutionary algorithms seem to be a very natural choice for dealing with multi-objective problems as the population of such an algorithm can be used to represent the trade-offs with respect to the given objective functions. In this paper, we contribute to the theoretical understanding of evolutionary algorithms for multi-objective problems. We consider indicator-based algorithms whose goal is to maximize the hypervolume for a given problem by distributing {\mu} points on the Pareto front. To gain new theoretical insights into the behavior of hypervolume-based algorithms we compare their optimization goal to the goal of achieving an optimal multiplicative approximation ratio. Our studies are carried out for different Pareto front shapes of bi-objective problems. For the class of linear fronts and a class of convex fronts, we prove that maximizing the hypervolume gives the best possible approximation ratio when assuming that the extreme points have to be included in both distributions of the points on the Pareto front. Furthermore, we investigate the choice of the reference point on the approximation behavior of hypervolume-based approaches and examine Pareto fronts of different shapes by numerical calculations.
Over the last decades, the amount of data of all kinds available electronically has increased dramatically. Data are accessible through a range of interfaces including Web browsers, database query languages, application-specific interfaces, built on top of a number of different data exchange formats. All these data span from un-structured to highly structured data. Very often, some of them have structure even if the structure is implicit, and not as rigid or regular as that found in standard database systems. Spreadsheet documents are prototypical in this respect. Spreadsheets are the lightweight technology able to supply companies with easy to build business management and business intelligence applications, and business people largely adopt spreadsheets as smart vehicles for data files generation and sharing. Actually, the more spreadsheets grow in complexity (e.g., their use in product development plans and quoting), the more their arrangement, maintenance, and analysis appear as a knowledge-driven activity. The algorithmic approach to the problem of automatic data structure extraction from spreadsheet documents (i.e., grid-structured and free topological-related data) emerges from the WIA project: Worksheets Intelligent Analyser. The WIA-algorithm shows how to provide a description of spreadsheet contents in terms of higher level of abstractions or conceptualisations. In particular, the WIA-algorithm target is about the extraction of i) the calculus work-flow implemented in the spreadsheets formulas and ii) the logical role played by the data which take part into the calculus. The aim of the resulting conceptualisations is to provide spreadsheets with abstract representations useful for further model refinements and optimizations through evolutionary algorithms computations.
Many optimization tasks have to be handled in noisy environments, where we cannot obtain the exact evaluation of a solution but only a noisy one. For noisy optimization tasks, evolutionary algorithms (EAs), a kind of stochastic metaheuristic search algorithm, have been widely and successfully applied. Previous work mainly focuses on empirical studying and designing EAs for noisy optimization, while, the theoretical counterpart has been little investigated. In this paper, we investigate a largely ignored question, i.e., whether an optimization problem will always become harder for EAs in a noisy environment. We prove that the answer is negative, with respect to the measurement of the expected running time. The result implies that, for optimization tasks that have already been quite hard to solve, the noise may not have a negative effect, and the easier a task the more negatively affected by the noise. On a representative problem where the noise has a strong negative effect, we examine two commonly employed mechanisms in EAs dealing with noise, the re-evaluation and the threshold selection strategies. The analysis discloses that the two strategies, however, both are not effective, i.e., they do not make the EA more noise tolerant. We then find that a small modification of the threshold selection allows it to be proven as an effective strategy for dealing with the noise in the problem.