PyTorchKR Papers

최신 논문

총 1,903편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1904.06837

Cramer-Rao Bound for Estimation After Model Selection and its Application to Sparse Vector Estimation

In many practical parameter estimation problems, such as coefficient estimation of polynomial regression, the true model is unknown and thus, a model selection step is performed prior to estimation. The data-based model selection step affects the subsequent estimation. In particular, the oracle Cram\'er-Rao bound (CRB), which is based on knowledge of the true model, is inappropriate for post-model-selection performance analysis and system design outside the asymptotic region. In this paper, we investigate post-model-selection parameter estimation of a vector with an unknown support set, where this support set represents the model. We analyze the estimation performance of coherent estimators that force unselected parameters to zero. We use the mean-squared-selected-error (MSSE) criterion and introduce the concept of selective unbiasedness in the sense of Lehmann unbiasedness. We derive a non-Bayesian Cram\'er-Raotype bound on the MSSE and on the mean-squared-error (MSE) of any coherent estimator with a specific selective-bias function in the Lehmann sense. We implement the selective CRB for the special case of sparse vector estimation with an unknown support set. Finally, we demonstrate in simulations that the proposed selective CRB is an informative lower bound on the performance of the maximum selected likelihood estimator for a general linear model with the generalized information criterion and for sparse vector estimation with one step thresholding. It is shown that for these cases the selective CRB outperforms the oracle CRB and Sando-Mitra-Stoica CRB (SMS-CRB) [1].

arXiv:1904.00211

Post-Selection Inference in Three-Dimensional Panel Data

Three-dimensional panel models are widely used in empirical analysis. Researchers use various combinations of fixed effects for three-dimensional panels. When one imposes a parsimonious model and the true model is rich, then it incurs mis-specification biases. When one employs a rich model and the true model is parsimonious, then it incurs larger standard errors than necessary. It is therefore useful for researchers to know correct models. In this light, Lu, Miao, and Su (2018) propose methods of model selection. We advance this literature by proposing a method of post-selection inference for regression parameters. Despite our use of the lasso technique as means of model selection, our assumptions allow for many and even all fixed effects to be nonzero. Simulation studies demonstrate that the proposed method is more precise than under-fitting fixed effect estimators, is more efficient than over-fitting fixed effect estimators, and allows for as accurate inference as the oracle estimator.

arXiv:1901.05397

lassopack: Model selection and prediction with regularized regression in Stata

This article introduces lassopack, a suite of programs for regularized regression in Stata. lassopack implements lasso, square-root lasso, elastic net, ridge regression, adaptive lasso and post-estimation OLS. The methods are suitable for the high-dimensional setting where the number of predictors $p$ may be large and possibly greater than the number of observations, $n$. We offer three different approaches for selecting the penalization (`tuning') parameters: information criteria (implemented in lasso2), $K$-fold cross-validation and $h$-step ahead rolling cross-validation for cross-section, panel and time-series data (cvlasso), and theory-driven (`rigorous') penalization for the lasso and square-root lasso for cross-section and panel data (rlasso). We discuss the theoretical framework and practical considerations for each approach. We also present Monte Carlo results to compare the performance of the penalization approaches.

arXiv:1901.00408

Improvement of Identification Procedure Using Hybrid Cuckoo Search Algorithm for TurbineGovernor and Excitation System

In this paper a new method is introduced in order to modify identification process of a gas power plant using a metaheuristic algorithm named Cuckoo Search (CS). Simulations play a significant role in dynamic analyses of power plants. This paper points out to a practical approach in model selection and parameter estimation of gas power plants. The identification and validation process concentrates on two subsystems: governor-turbine and exciter. Standard models GGOV1 and STB6 are preferred for the dynamical structures of governor-turbine and exciter respectively. Considering definite standard structure, main parameters of dynamical model are pre estimated via system identification methods based on field data. Then obtained parameters are tuned carefully using an iterative Cuckoo algorithm. Models must be validated by results derived via a trial and error series of simulation in comparison to measured test data. The procedure gradually yields in a valid model with precise estimated parameters. Simulation results show accuracy of identified models. Besides, a whiteness analysis has been performed in order to show the authenticity of the proposed method in another way. Despite various detailed models, practical attempts of model selection, identification, and validation in a real gas unit could rarely be found among literature. In this paper, Chabahar power plant in Iran, with total install capacity of 320 MW, is chosen as a benchmark for model validation.

arXiv:1812.06315

Breaking the bonds of weak coupling: the dynamic causal modelling of oscillator amplitudes

Models of coupled oscillators are useful in describing a wide variety of phenomena in physics, biology and economics. These models typically rest on the premise that the oscillators are weakly coupled, meaning that amplitudes can be assumed to be constant and dynamics can therefore be described purely in terms of phase differences. Whilst mathematically convenient, the restrictive nature of the weak coupling assumption can limit the explanatory power of these phase-coupled oscillator models. We therefore propose an extension to the weakly-coupled oscillator model that incorporates both amplitude and phase as dependent variables. We use the bilinear neuronal state equations of dynamic causal modelling as a foundation in deriving coupled differential equations that describe the activity of both weakly and strongly coupled oscillators. We show that weakly-coupled oscillator models are inadequate in describing the processes underlying the temporally variable signals observed in a variety of systems. We demonstrate that phase-coupled models perform well on simulations of weakly coupled systems but fail when connectivity is no longer weak. On the other hand, using Bayesian model selection, we show that our phase-amplitude coupling model can describe non-weakly coupled systems more effectively despite the added complexity associated with using amplitude as an extra dependent variable. We demonstrate the advantage of our phase-amplitude model in the context of model-generated data, as well as of a simulation of inter-connected pendula, neural local field potential recordings in rodents under anaesthesia and international economic gross domestic product data.

arXiv:1811.10676

LM-BIC Model Selection in Semiparametric Models

This paper studies model selection in semiparametric econometric models. It develops a consistent series-based model selection procedure based on a Bayesian Information Criterion (BIC) type criterion to select between several classes of models. The procedure selects a model by minimizing the semiparametric Lagrange Multiplier (LM) type test statistic from Korolev (2018) but additionally rewards simpler models. The paper also develops consistent upward testing (UT) and downward testing (DT) procedures based on the semiparametric LM type specification test. The proposed semiparametric LM-BIC and UT procedures demonstrate good performance in simulations. To illustrate the use of these semiparametric model selection procedures, I apply them to the parametric and semiparametric gasoline demand specifications from Yatchew and No (2001). The LM-BIC procedure selects the semiparametric specification that is nonparametric in age but parametric in all other variables, which is in line with the conclusions in Yatchew and No (2001). The results of the UT and DT procedures heavily depend on the choice of tuning parameters and assumptions about the model errors.

arXiv:1809.07691

A Survey on Theoretical Advances of Community Detection in Networks

Real-world networks usually have community structure, that is, nodes are grouped into densely connected communities. Community detection is one of the most popular and best-studied research topics in network science and has attracted attention in many different fields, including computer science, statistics, social sciences, among others. Numerous approaches for community detection have been proposed in literature, from ad-hoc algorithms to systematic model-based approaches. The large number of available methods leads to a fundamental question: whether a certain method can provide consistent estimates of community labels. The stochastic blockmodel (SBM) and its variants provide a convenient framework for the study of such problems. This article is a survey on the recent theoretical advances of community detection. The authors review a number of community detection methods and their theoretical properties, including graph cut methods, profile likelihoods, the pseudo-likelihood method, the variational method, belief propagation, spectral clustering, and semidefinite relaxations of the SBM. The authors also briefly discuss other research topics in community detection such as robust community detection, community detection with nodal covariates and model selection, as well as suggest a few possible directions for future research.

arXiv:1807.06993

Cross Validation Based Model Selection via Generalized Method of Moments

Structural estimation is an important methodology in empirical economics, and a large class of structural models are estimated through the generalized method of moments (GMM). Traditionally, selection of structural models has been performed based on model fit upon estimation, which take the entire observed samples. In this paper, we propose a model selection procedure based on cross-validation (CV), which utilizes sample-splitting technique to avoid issues such as over-fitting. While CV is widely used in machine learning communities, we are the first to prove its consistency in model selection in GMM framework. Its empirical property is compared to existing methods by simulations of IV regressions and oligopoly market model. In addition, we propose the way to apply our method to Mathematical Programming of Equilibrium Constraint (MPEC) approach. Finally, we perform our method to online-retail sales data to compare dynamic market model to static model.

arXiv:1805.08991

Model Selection in Time Series Analysis: Using Information Criteria as an Alternative to Hypothesis Testing

The issue of model selection in applied research is of vital importance. Since the true model in such research is not known, which model should be used from among various potential ones is an empirical question. There might exist several competitive models. A typical approach to dealing with this is classic hypothesis testing using an arbitrarily chosen significance level based on the underlying assumption that a true null hypothesis exists. In this paper we investigate how successful this approach is in determining the correct model for different data generating processes using time series data. An alternative approach based on more formal model selection techniques using an information criterion or cross-validation is suggested and evaluated in the time series environment via Monte Carlo experiments. This paper also explores the effectiveness of deciding what type of general relation exists between two variables (e.g. relation in levels or relation in first differences) using various strategies based on hypothesis testing and on information criteria with the presence or absence of unit roots.

arXiv:1805.06682

Analyzing order flows in limit order books with ratios of Cox-type intensities

We introduce a Cox-type model for relative intensities of orders flows in a limit order book. The model assumes that all intensities share a common baseline intensity, which may for example represent the global market activity. Parameters can be estimated by quasi likelihood maximization, without any interference from the baseline intensity. Consistency and asymptotic behavior of the estimators are given in several frameworks, and model selection is discussed with information criteria and penalization. The model is well-suited for high-frequency financial data: fitted models using easily interpretable covariates show an excellent agreement with empirical data. Extensive investigation on tick data consequently helps identifying trading signals and important factors determining the limit order book dynamics. We also illustrate the potential use of the framework for out-of-sample predictions.

arXiv:1709.01143

Modeling Sheep pox Disease from the 1994-1998 Epidemic in Evros Prefecture, Greece

Sheep pox is a highly transmissible disease which can cause serious loss of livestock and can therefore have major economic impact. We present data from sheep pox epidemics which occurred between 1994 and 1998. The data include weekly records of infected farms as well as a number of covariates. We implement Bayesian stochastic regression models which, in addition to various explanatory variables like seasonal and environmental/meteorological factors, also contain serial correlation structure based on variants of the Ornstein-Uhlenbeck process. We take a predictive view in model selection by utilizing deviance-based measures. The results indicate that seasonality and the number of infected farms are important predictors for sheep pox incidence.

arXiv:1708.08120

MSM lag time cannot be used for variational model selection

The variational principle for conformational dynamics has enabled the systematic construction of Markov state models through the optimization of hyperparameters by approximating the transfer operator. In this note we discuss why lag time of the operator being approximated must be held constant in the variational approach.

arXiv:1708.00385

Probabilistic models of individual and collective animal behavior

Recent developments in automated tracking allow uninterrupted, high-resolution recording of animal trajectories, sometimes coupled with the identification of stereotyped changes of body pose or other behaviors of interest. Analysis and interpretation of such data represents a challenge: the timing of animal behaviors may be stochastic and modulated by kinematic variables, by the interaction with the environment or with the conspecifics within the animal group, and dependent on internal cognitive or behavioral state of the individual. Existing models for collective motion typically fail to incorporate the discrete, stochastic, and internal-state-dependent aspects of behavior, while models focusing on individual animal behavior typically ignore the spatial aspects of the problem. Here we propose a probabilistic modeling framework to address this gap. Each animal can switch stochastically between different behavioral states, with each state resulting in a possibly different law of motion through space. Switching rates for behavioral transitions can depend in a very general way, which we seek to identify from data, on the effects of the environment as well as the interaction between the animals. We represent the switching dynamics as a Generalized Linear Model and show that: (i) forward simulation of multiple interacting animals is possible using a variant of the Gillespie's Stochastic Simulation Algorithm; (ii) formulated properly, the maximum likelihood inference of switching rate functions is tractably solvable by gradient descent; (iii) model selection can be used to identify factors that modulate behavioral state switching and to appropriately adjust model complexity to data. To illustrate our framework, we apply it to two synthetic models of animal motion and to real zebrafish tracking data.

arXiv:1706.03561

Awkt: A Physicochemical Parameter Estimation Tool for Capillary Zone Electrophoresis

We present a computational parameter estimation tool aimed at assisting molecule characterization applying nonlinear least-squares optimization to a specific closed-form solution of a capillary electrophoretic transport problem. Numerical estimates for the respective physicochemical parameters are generated. Among other features, the tool facilitates automated and user-controllable hypothesis testing and model selection. It can also be used to perform fully automated batch-processing in true concurrent mode.

arXiv:1704.07866

Identifying spatiotemporal dynamics of Ebola in Sierra Leone using virus genomes

Containing the recent West African outbreak of Ebola virus (EBOV) required the deployment of substantial global resources. Operationally, health workers and surveillance teams treated cases, collected genetic samples, and tracked case contacts. Despite the substantial progress in analyzing and modeling EBOV epidemiological data, a complete characterization of the spatiotemporal spread of Ebola cases remains a challenge. In this work, we offer a novel perspective on the EBOV epidemic that utilizes virus genome sequences to inform population-level, spatial models. Calibrated to phylogenetic linkages, these dynamic spatial models provide unique insight into the disease mobility of EBOV in Sierra Leone. Further, we developed a model selection framework that identifies important epidemiological variables influencing the spatiotemporal propagation of EBOV. Consistent with other investigations, our results show that the spread of EBOV during the beginning and middle portions of the epidemic strongly depended on the size of and distance between populations. Our analysis also revealed a substantial decline in the dependence on population size at the end of the epidemic, coinciding with the large-scale intervention campaign: Operation Western Area Surge. More generally, we believe this framework, pairing molecular diagnostics with dynamic models, has the potential to be a powerful forecasting tool along with offering operationally-relevant guidance for surveillance and sampling strategies during an epidemic.

arXiv:1703.05471공식 코드 · Papers with Code

Model selection and parameter inference in phylogenetics using Nested Sampling

Bayesian inference methods rely on numerical algorithms for both model selection and parameter inference. In general, these algorithms require a high computational effort to yield reliable estimates. One of the major challenges in phylogenetics is the estimation of the marginal likelihood. This quantity is commonly used for comparing different evolutionary models, but its calculation, even for simple models, incurs high computational cost. Another interesting challenge relates to the estimation of the posterior distribution. Often, long Markov chains are required to get sufficient samples to carry out parameter inference, especially for tree distributions. In general, these problems are addressed separately by using different procedures. Nested sampling (NS) is a Bayesian computation algorithm which provides the means to estimate marginal likelihoods together with their uncertainties, and to sample from the posterior distribution at no extra cost. The methods currently used in phylogenetics for marginal likelihood estimation lack in practicality due to their dependence on many tuning parameters and the inability of most implementations to provide a direct way to calculate the uncertainties associated with the estimates. To address these issues, we introduce NS to phylogenetics. Its performance is assessed under different scenarios and compared to established methods. We conclude that NS is a competitive and attractive algorithm for phylogenetic inference. An implementation is available as a package for BEAST 2 under the LGPL licence, accessible at https://github.com/BEAST2-Dev/nested-sampling.

arXiv:1701.07787공식 코드 · Papers with Code

Multi-locus data distinguishes between population growth and multiple merger coalescents

We introduce a low dimensional function of the site frequency spectrum that is tailor-made for distinguishing coalescent models with multiple mergers from Kingman coalescent models with population growth, and use this function to construct a hypothesis test between these model classes. The null and alternative sampling distributions of the statistic are intractable, but its low dimensionality renders them amenable to Monte Carlo estimation. We construct kernel density estimates of the sampling distributions based on simulated data, and show that the resulting hypothesis test dramatically improves on the statistical power of a current state-of-the-art method. A key reason for this improvement is the use of multi-locus data, in particular averaging observed site frequency spectra across unlinked loci to reduce sampling variance. We also demonstrate the robustness of our method to nuisance and tuning parameters. Finally we show that the same kernel density estimates can be used to conduct parameter estimation, and argue that our method is readily generalisable for applications in model selection, parameter inference and experimental design.

arXiv:1701.05627

Luria-Delbruck, revisited: The classic experiment does not rule out Lamarckian evolution

We re-examined data from the classic Luria-Delbruck fluctuation experiment, which is often credited with establishing a Darwinian basis for evolution. We argue that, for the Lamarckian model of evolution to be ruled out by the experiment, the experiment must favor pure Darwinian evolution over both the Lamarckian model and a model that allows both Darwinian and Lamarckian mechanisms. Analysis of the combined model was not performed in the original 1943 paper. The Luria-Delbruck paper also did not consider the possibility of neither model fitting the experiment. Using Bayesian model selection, we find that the Luria-Delbruck experiment, indeed, favors the Darwinian evolution over purely Lamarckian. However, our analysis does not rule out the combined model, and hence cannot rule out Lamarckian contributions to the evolutionary dynamics.

arXiv:1610.05383

Detection of intensity bursts using Hawkes processes: an application to high frequency financial data

Given a stationary point process, an intensity burst is defined as a short time period during which the number of counts is larger than the typical count rate. It might signal a local non-stationarity or the presence of an external perturbation to the system. In this paper we propose a novel procedure for the detection of intensity bursts within the Hawkes process framework. By using a model selection scheme we show that our procedure can be used to detect intensity bursts when both their occurrence time and their total number is unknown. Moreover, the initial time of the burst can be determined with a precision given by the typical inter-event time. We apply our methodology to the mid-price change in FX markets showing that these bursts are frequent and that only a relatively small fraction is associated to news arrival. We show lead-lag relations in intensity burst occurrence across different FX rates and we discuss their relation with price jumps.

arXiv:1602.06186

Noise Fit, Estimation Error and a Sharpe Information Criterion

When the in-sample Sharpe ratio is obtained by optimizing over a k-dimensional parameter space, it is a biased estimator for what can be expected on unseen data (out-of-sample). We derive (1) an unbiased estimator adjusting for both sources of bias: noise fit and estimation error. We then show (2) how to use the adjusted Sharpe ratio as model selection criterion analogously to the Akaike Information Criterion (AIC). Selecting a model with the highest adjusted Sharpe ratio selects the model with the highest estimated out-of-sample Sharpe ratio in the same way as selection by AIC does for the log-likelihood as measure of fit.