PyTorchKR Papers

최신 논문

총 605편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1809.06674

Benchmarking five global optimization approaches for nano-optical shape optimization and parameter reconstruction

Numerical optimization is an important tool in the field of computational physics in general and in nano-optics in specific. It has attracted attention with the increase in complexity of structures that can be realized with nowadays nano-fabrication technologies for which a rational design is no longer feasible. Also, numerical resources are available to enable the computational photonic material design and to identify structures that meet predefined optical properties for specific applications. However, the optimization objective function is in general non-convex and its computation remains resource demanding such that the right choice for the optimization method is crucial to obtain excellent results. Here, we benchmark five global optimization methods for three typical nano-optical optimization problems: \removed{downhill simplex optimization, the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm, particle swarm optimization, differential evolution, and Bayesian optimization} \added{particle swarm optimization, differential evolution, and Bayesian optimization as well as multi-start versions of downhill simplex optimization and the limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm}. In the shown examples from the field of shape optimization and parameter reconstruction, Bayesian optimization, mainly known from machine learning applications, obtains significantly better results in a fraction of the run times of the other optimization methods.

arXiv:1809.03951공식 코드 · Papers with Code

Hubless keypoint-based 3D deformable groupwise registration

We present a novel algorithm for Fast Registration Of image Groups (FROG), applied to large 3D image groups. Our approach extracts 3D SURF keypoints from images, computes matched pairs of keypoints and registers the group by minimizing pair distances in a hubless way i.e. without computing any central mean image. Using keypoints significantly reduces the problem complexity compared to voxel-based approaches, and enables us to provide an in-core global optimization, similar to the Bundle Adjustment for 3D reconstruction. As we aim to register images of different patients, the matching step yields many outliers. Then we propose a new EM-weighting algorithm which efficiently discards outliers. Global optimization is carried out with a fast gradient descent algorithm. This allows our approach to robustly register large datasets. The result is a set of diffeomorphic half transforms which link the volumes together and can be subsequently exploited for computational anatomy and landmark detection. We show experimental results on whole-body CT scans, with groups of up to 103 volumes. On a benchmark based on anatomical landmarks, our algorithm compares favorably with the star-groupwise voxel-based ANTs and NiftyReg approaches while being much faster. We also discuss the limitations of our approach for lower resolution images such as brain MRI.

arXiv:1808.08818

A new Taxonomy of Continuous Global Optimization Algorithms

Surrogate-based optimization, nature-inspired metaheuristics, and hybrid combinations have become state of the art in algorithm design for solving real-world optimization problems. Still, it is difficult for practitioners to get an overview that explains their advantages in comparison to a large number of available methods in the scope of optimization. Available taxonomies lack the embedding of current approaches in the larger context of this broad field. This article presents a taxonomy of the field, which explores and matches algorithm strategies by extracting similarities and differences in their search strategies. A particular focus lies on algorithms using surrogates, nature-inspired designs, and those created by design optimization. The extracted features of components or operators allow us to create a set of classification indicators to distinguish between a small number of classes. The features allow a deeper understanding of components of the search strategies and further indicate the close connections between the different algorithm designs. We present intuitive analogies to explain the basic principles of the search algorithms, particularly useful for novices in this research field. Furthermore, this taxonomy allows recommendations for the applicability of the corresponding algorithms.

arXiv:1808.06918

On a New Improvement-Based Acquisition Function for Bayesian Optimization

Bayesian optimization (BO) is a popular algorithm for solving challenging optimization tasks. It is designed for problems where the objective function is expensive to evaluate, perhaps not available in exact form, without gradient information and possibly returning noisy values. Different versions of the algorithm vary in the choice of the acquisition function, which recommends the point to query the objective at next. Initially, researchers focused on improvement-based acquisitions, while recently the attention has shifted to more computationally expensive information-theoretical measures. In this paper we present two major contributions to the literature. First, we propose a new improvement-based acquisition function that recommends query points where the improvement is expected to be high with high confidence. The proposed algorithm is evaluated on a large set of benchmark functions from the global optimization literature, where it turns out to perform at least as well as current state-of-the-art acquisition functions, and often better. This suggests that it is a powerful default choice for BO. The novel policy is then compared to widely used global optimization solvers in order to confirm that BO methods reduce the computational costs of the optimization by keeping the number of function evaluations small. The second main contribution represents an application to precision medicine, where the interest lies in the estimation of parameters of a partial differential equations model of the human pulmonary blood circulation system. Once inferred, these parameters can help clinicians in diagnosing a patient with pulmonary hypertension without going through the standard invasive procedure of right heart catheterization, which can lead to side effects and complications (e.g. severe pain, internal bleeding, thrombosis).

arXiv:1807.09408

Deterministic Hypothesis Generation for Robust Fitting of Multiple Structures

We present a novel algorithm for generating robust and consistent hypotheses for multiple-structure model fitting. Most of the existing methods utilize random sampling which produce varying results especially when outlier ratio is high. For a structure where a model is fitted, the inliers of other structures are regarded as outliers when multiple structures are present. Global optimization has recently been investigated to provide stable and unique solutions, but the computational cost of the algorithms is prohibitively high for most image data with reasonable sizes. The algorithm presented in this paper uses a maximum feasible subsystem (MaxFS) algorithm to generate consistent initial hypotheses only from partial datasets in spatially overlapping local image regions. Our assumption is that each genuine structure will exist as a dominant structure in at least one of the local regions. To refine initial hypotheses estimated from partial datasets and to remove residual tolerance dependency of the MaxFS algorithm, iterative re-weighted L1 (IRL1) minimization is performed for all the image data. Initial weights of IRL1 framework are determined from the initial hypotheses generated in local regions. Our approach is significantly more efficient than those that use only global optimization for all the image data. Experimental results demonstrate that the presented method can generate more reliable and consistent hypotheses than random-sampling methods for estimating single and multiple structures from data with a large amount of outliers. We clearly expose the influence of algorithm parameter settings on the results in our experiments.

arXiv:1807.09210

Deterministic Fitting of Multiple Structures using Iterative MaxFS with Inlier Scale Estimation and Subset Updating

We present an efficient deterministic hypothesis generation algorithm for robust fitting of multiple structures based on the maximum feasible subsystem (MaxFS) framework. Despite its advantage, a global optimization method such as MaxFS has two main limitations for geometric model fitting. First, its performance is much influenced by the user-specified inlier scale. Second, it is computationally inefficient for large data. The presented MaxFS-based algorithm iteratively estimates model parameters and inlier scale and also overcomes the second limitation by reducing data for the MaxFS problem. Further it generates hypotheses only with top-n ranked subsets based on matching scores and data fitting residuals. This reduction of data for the MaxFS problem makes the algorithm computationally realistic. Our method, called iterative MaxFS with inlier scale estimation and subset updating (IMaxFS-ISE-SU) in this paper, performs hypothesis generation and fitting alternately until all of true structures are found. The IMaxFS-ISE-SU algorithm generates substantially more reliable hypotheses than random sampling-based methods especially as (pseudo-)outlier ratios increase. Experimental results demonstrate that our method can generate more reliable and consistent hypotheses than random sampling-based methods for estimating multiple structures from data with many outliers.

arXiv:1807.05096

Global optimization test problems based on random field composition

The development and identification of effective optimization algorithms for non-convex real-world problems is a challenge in global optimization. Because theoretical performance analysis is difficult, and problems based on models of real-world systems are often computationally expensive, several artificial performance test problems and test function generators have been proposed for empirical comparative assessment and analysis of metaheuristic optimization algorithms. These test problems however often lack the complex function structures and forthcoming difficulties that can appear in real-world problems. This communication presents a method to systematically build test problems with various types and degrees of difficulty. By weighted composition of parameterized random fields, challenging test functions with tunable function features such as, variance contribution distribution, interaction order, and nonlinearity can be constructed. The method is described, and its applicability to optimization performance analysis is described by means of a few basic examples. The method aims to set a step forward in the systematic generation of global optimization test problems, which could lead to a better understanding of the performance of optimization algorithms on problem types with particular characteristics. On request an introductive MATLAB implementation of a test function generator based on the presented method is available.

arXiv:1807.00373공식 코드 · Papers with Code

New Heuristics for Parallel and Scalable Bayesian Optimization

Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a challenge for the non-expert. In this report, I review the state-of-the-art in parallel and scalable Bayesian optimization methods. In addition, I propose practical ways to avoid a few of the pitfalls of Bayesian optimization, such as oversampling of edge parameters and over-exploitation of high performance parameters. Finally, I provide relatively simple, heuristic algorithms, along with their open source software implementations, that can be immediately and easily deployed in any computing environment.

arXiv:1806.08169

A convex method for classification of groups of examples

There are many applications where it important to perform well on a set of examples as opposed to individual examples. For example in image or video classification the question is does an object appear somewhere in the image or video while there are several candidates of the object per image or video. In this context, it is not important what is the performance per candidate. Instead the performance per group is the ultimate objective. For such problems one popular approach assumes weak supervision where labels exist for the entire group and then multiple instance learning is utilized. Another approach is to optimize per candidate, assuming each candidate is labeled, in the belief that this will achieve good performance per group. We will show that better results can be achieved if we offer a new methodology which synthesizes the aforementioned approaches and directly optimizes for the final optimization objective while consisting of a convex optimization problem which solves the global optimization problem. The benefit of grouping examples is demonstrated on an image classification task for detecting polyps in images from capsule endoscopy of the colon. The algorithm was designed to efficiently handle hundreds of millions of examples. Furthermore, modifications to the penalty function of the standard SVM algorithm, have proven to significantly improve performance in our test case.

arXiv:1107.2104

An estimation of distribution algorithm with adaptive Gibbs sampling for unconstrained global optimization

In this paper is proposed a new heuristic approach belonging to the field of evolutionary Estimation of Distribution Algorithms (EDAs). EDAs builds a probability model and a set of solutions is sampled from the model which characterizes the distribution of such solutions. The main framework of the proposed method is an estimation of distribution algorithm, in which an adaptive Gibbs sampling is used to generate new promising solutions and, in combination with a local search strategy, it improves the individual solutions produced in each iteration. The Estimation of Distribution Algorithm with Adaptive Gibbs Sampling we are proposing in this paper is called AGEDA. We experimentally evaluate and compare this algorithm against two deterministic procedures and several stochastic methods in three well known test problems for unconstrained global optimization. It is empirically shown that our heuristic is robust in problems that involve three central aspects that mainly determine the difficulty of global optimization problems, namely high-dimensionality, multi-modality and non-smoothness.

arXiv:1307.5840

Sub- Diving Labeling Method for Optimization Problem by Genetic Algorithm

In many global Optimization Problems, it is required to evaluate a global point (min or max) in large space that calculation effort is very high. In this paper is presented new approach for optimization problem with subdivision labeling method (SLM) but in this method for higher dimensional has high computational. SLM Genetic Algorithm (SLMGA) in optimization problems is one of the solutions of this problem. In proposed algorithm the initial population is crossing points and subdividing in each step is according to mutation. RSLMGA is compared with other well known algorithms: DE, PGA, Grefensstette and Eshelman and numerical results show that RSLMGA achieve global optimal point with more decision by smaller generations.

arXiv:1307.5839

A New Approach for Finding the Global Optimal Point Using Subdividing Labeling Method (SLM)

In most global optimization problems, finding global optimal point inthe multidimensional and great search space needs high computations. In this paper, we present a new approach to find global optimal point with the low computation and few steps using subdividing labeling method (SLM) which can also be used in the multi-dimensional and great search space. In this approach, in each step, crossing points will be labeled and complete label polytope search space of selected polytope will be subdivided after being selected. SLM algorithm finds the global point until h (subdivision function) turns into zero. SLM will be implemented on five applications and compared with the latest techniques such as random search, random search-walk and simulated annealing method. The results of the proposed method demonstrate that our new approach is faster and more reliable and presents an optimal time complexity O (logn).

arXiv:1307.5667

New Optimization Approach Using Clustering-Based Parallel Genetic Algorithm

In many global Optimization Problems, it is required to evaluate a global point (min or max) in large space that calculation effort is very high. In this paper is presented new approach for optimization problem with subdivision labeling method (SLM) but in this method for higher dimensional has high calculation effort. Clustering-Based Parallel Genetic Algorithm (CBPGA) in optimization problems is one of the solutions of this problem. That the initial population is crossing points and subdividing in each step is according to mutation. After labeling all of crossing points, selecting is according to polytope that has complete label. In this method we propose an algorithm, based on parallelization scheme using master-slave. SLM algorithm is implemented by CBPGA and compared the experimental results. The numerical examples and numerical results show that SLMCBPGA is improved speed up and efficiency.

arXiv:1307.5534

A New Optimization Approach Based on Rotational Mutation and Crossover Operator

Evaluating a global optimal point in many global optimization problems in large space is required to more calculations. In this paper, there is presented a new approach for the continuous functions optimization with rotational mutation and crossover operator. This proposed method (RMC) starts from the point which has best fitness value by elitism mechanism and after that rotational mutation and crossover operator are used to reach optimal point. RMC method is implemented by GA (Briefly RMCGA) and is compared with other wellknown algorithms such as: DE, PGA, Grefensstette and Eshelman[15,16] and numerical and simulating results show that RMCGA achieve global optimal point with more decision by smaller generations.

arXiv:1308.4008

A Literature Survey of Benchmark Functions For Global Optimization Problems

Test functions are important to validate and compare the performance of optimization algorithms. There have been many test or benchmark functions reported in the literature; however, there is no standard list or set of benchmark functions. Ideally, test functions should have diverse properties so that can be truly useful to test new algorithms in an unbiased way. For this purpose, we have reviewed and compiled a rich set of 175 benchmark functions for unconstrained optimization problems with diverse properties in terms of modality, separability, and valley landscape. This is by far the most complete set of functions so far in the literature, and tt can be expected this complete set of functions can be used for validation of new optimization in the future.

arXiv:1312.4044

CACO : Competitive Ant Colony Optimization, A Nature-Inspired Metaheuristic For Large-Scale Global Optimization

Large-scale problems are nonlinear problems that need metaheuristics, or global optimization algorithms. This paper reviews nature-inspired metaheuristics, then it introduces a framework named Competitive Ant Colony Optimization inspired by the chemical communications among insects. Then a case study is presented to investigate the proposed framework for large-scale global optimization.

arXiv:1312.5673

Flower Pollination Algorithm for Global Optimization

Flower pollination is an intriguing process in the natural world. Its evolutionary characteristics can be used to design new optimization algorithms. In this paper, we propose a new algorithm, namely, flower pollination algorithm, inspired by the pollination process of flowers. We first use ten test functions to validate the new algorithm, and compare its performance with genetic algorithms and particle swarm optimization. Our simulation results show the flower algorithm is more efficient than both GA and PSO. We also use the flower algorithm to solve a nonlinear design benchmark, which shows the convergence rate is almost exponential.

arXiv:1312.6969

Time series modeling by a regression approach based on a latent process

Time series are used in many domains including finance, engineering, economics and bioinformatics generally to represent the change of a measurement over time. Modeling techniques may then be used to give a synthetic representation of such data. A new approach for time series modeling is proposed in this paper. It consists of a regression model incorporating a discrete hidden logistic process allowing for activating smoothly or abruptly different polynomial regression models. The model parameters are estimated by the maximum likelihood method performed by a dedicated Expectation Maximization (EM) algorithm. The M step of the EM algorithm uses a multi-class Iterative Reweighted Least-Squares (IRLS) algorithm to estimate the hidden process parameters. To evaluate the proposed approach, an experimental study on simulated data and real world data was performed using two alternative approaches: a heteroskedastic piecewise regression model using a global optimization algorithm based on dynamic programming, and a Hidden Markov Regression Model whose parameters are estimated by the Baum-Welch algorithm. Finally, in the context of the remote monitoring of components of the French railway infrastructure, and more particularly the switch mechanism, the proposed approach has been applied to modeling and classifying time series representing the condition measurements acquired during switch operations.

arXiv:1405.7349

Radial basis function process neural network training based on generalized frechet distance and GA-SA hybrid strategy

For learning problem of Radial Basis Function Process Neural Network (RBF-PNN), an optimization training method based on GA combined with SA is proposed in this paper. Through building generalized Fr\'echet distance to measure similarity between time-varying function samples, the learning problem of radial basis centre functions and connection weights is converted into the training on corresponding discrete sequence coefficients. Network training objective function is constructed according to the least square error criterion, and global optimization solving of network parameters is implemented in feasible solution space by use of global optimization feature of GA and probabilistic jumping property of SA . The experiment results illustrate that the training algorithm improves the network training efficiency and stability.