A novel framework has recently been proposed for designing the molecular structure of chemical compounds with a desired chemical property using both artificial neural networks and mixed integer linear programming. In this paper, we design a new method for inferring a polymer based on the framework. For this, we introduce a new way of representing a polymer as a form of monomer and define new descriptors that feature the structure of polymers. We also use linear regression as a building block of constructing a prediction function in the framework. The results of our computational experiments reveal a set of chemical properties on polymers to which a prediction function constructed with linear regression performs well. We also observe that the proposed method can infer polymers with up to 50 non-hydrogen atoms in a monomer form.
The "shape" of a planar curve and/or landmark configuration is considered its equivalence class under translation, rotation and scaling, its "form" its equivalence class under translation and rotation while scale is preserved. We extend generalized additive regression to models for such shapes/forms as responses respecting the resulting quotient geometry by employing the squared geodesic distance as loss function and a geodesic response function to map the additive predictor to the shape/form space. For fitting the model, we propose a Riemannian $L_2$-Boosting algorithm well suited for a potentially large number of possibly parameter-intensive model terms, which also yields automated model selection. We provide novel intuitively interpretable visualizations for (even non-linear) covariate effects in the shape/form space via suitable tensor-product factorization. The usefulness of the proposed framework is illustrated in an analysis of 1) astragalus shapes of wild and domesticated sheep and 2) cell forms generated in a biophysical model, as well as 3) in a realistic simulation study with response shapes and forms motivated from a dataset on bottle outlines.
Scientists have long aimed to discover meaningful formulae which accurately describe experimental data. A common approach is to manually create mathematical models of natural phenomena using domain knowledge, and then fit these models to data. In contrast, machine-learning algorithms automate the construction of accurate data-driven models while consuming large amounts of data. The problem of incorporating prior knowledge in the form of constraints on the functional form of a learned model (e.g., nonnegativity) has been explored in the literature. However, finding models that are consistent with prior knowledge expressed in the form of general logical axioms (e.g., conservation of energy) is an open problem. We develop a method to enable principled derivations of models of natural phenomena from axiomatic knowledge and experimental data by combining logical reasoning with symbolic regression. We demonstrate these concepts for Kepler's third law of planetary motion, Einstein's relativistic time-dilation law, and Langmuir's theory of adsorption, automatically connecting experimental data with background theory in each case. We show that laws can be discovered from few data points when using formal logical reasoning to distinguish the correct formula from a set of plausible formulas that have similar error on the data. The combination of reasoning with machine learning provides generalizeable insights into key aspects of natural phenomena. We envision that this combination will enable derivable discovery of fundamental laws of science and believe that our work is an important step towards automating the scientific method.
Many segmentation tasks for biomedical images can be modeled as the minimization of an energy function and solved by a class of max-flow and min-cut optimization algorithms. However, the segmentation accuracy is sensitive to the contrasting of semantic features of different segmenting objects, as the traditional energy function usually uses hand-crafted features in their energy functions. To address these limitations, we propose to incorporate end-to-end trainable neural network features into the energy functions. Our deep neural network features are extracted from the down-sampling and up-sampling layers with skip-connections of a U-net. In the inference stage, the learned features are fed into the energy functions. And the segmentations are solved in a primal-dual form by ADMM solvers. In the training stage, we train our neural networks by optimizing the energy function in the primal form with regularizations on the min-cut and flow-conservation functions, which are derived from the optimal conditions in the dual form. We evaluate our methods, both qualitatively and quantitatively, in a brain tumor segmentation task. As the energy minimization model achieves a balance on sensitivity and smooth boundaries, we would show how our segmentation contours evolve actively through iterations as ensemble references for doctor diagnosis.
For real-life nonlinear systems, the exact form of nonlinearity is often not known and the known governing equations are often based on certain assumptions and approximations. Such representation introduced model-form error into the system. In this paper, we propose a novel gray-box modeling approach that not only identifies the model-form error but also utilizes it to improve the predictive capability of the known but approximate governing equation. The primary idea is to treat the unknown model-form error as a residual force and estimate it using duel Bayesian filter based joint input-state estimation algorithms. For improving the predictive capability of the underlying physics, we first use machine learning algorithm to learn a mapping between the estimated state and the input (model-form error) and then introduce it into the governing equation as an additional term. This helps in improving the predictive capability of the governing physics and allows the model to generalize to unseen environment. Although in theory, any machine learning algorithm can be used within the proposed framework, we use Gaussian process in this work. To test the performance of proposed framework, case studies discussing four different dynamical systems are discussed; results for which indicate that the framework is applicable to a wide variety of systems and can produce reliable estimates of original system's states.
This paper develops the first closed-form optimal portfolio allocation formula for a spot asset whose variance follows a GARCH(1,1) process. We consider an investor with constant relative risk aversion (CRRA) utility who wants to maximize the expected utility from terminal wealth under a Heston and Nandi (2000) GARCH (HN-GARCH) model. We obtain closed formulas for the optimal investment strategy, the value function and the optimal terminal wealth. We find the optimal strategy is independent of the development of the risky asset, and the solution converges to that of a continuous-time Heston stochastic volatility model, albeit under additional conditions. For a daily trading scenario, the optimal solutions are quite robust to variations in the parameters, while the numerical wealth equivalent loss (WEL) analysis shows good performance of the Heston solution, with a quite inferior performance of the Merton solution.
This paper proposes a new Linear Fractional Transformation (LFT) modeling approach for uncertain Linear Parameter Varying (LPV) multibody systems with parameter-dependent equilibrium. Traditional multibody approaches, which consist in building the nonlinear model of the whole structure and linearizing it around equilibrium after a numerical trimming, do not allow to isolate parametric variations with the LFT form. Although additional techniques, such as polynomial fitting or symbolic linearization, can provide an LFT model, they may be time-consuming or miss worst-case configurations. The proposed approach relies on the trimming and linearization of the equations at the substructure level, before assembly of the multibody structure, which allows to only perform operations that preserve the LFT form throughout the linearization process. Since the physical origin of the parameters is retained, the linearized LFT-LPV model of the structure exactly covers all plants, in a single parametric model, without introducing conservatism or fitting errors. An application to the LFT-LPV modeling of a robotic arm is proposed; in its nominal configuration, the model obtained with the proposed approach matches the model provided by the software Simscape Multibody, but it is enhanced with parametric variations with the LFT form; a robust LPV synthesis is performed using Matlab robust control toolbox to illustrate the capacity of the proposed approach for control design.
Background: Trypanosoma brucei is the causative agent of human African sleeping sickness and Nagana in cattle. In addition to being an important pathogen T. brucei has developed into a model system in cell biology. Results: Using Stable Isotope Labelling of Amino acids in Cell culture (SILAC) in combination with mass spectrometry we determined the abundance of >1600 proteins in the long slender (LS), short stumpy (SS) mammalian bloodstream form stages relative to the procyclic (PC) insect-form stage. In total we identified 2645 proteins, corresponding to ~30% of the total proteome and for the first time present a comprehensive overview of relative protein levels in three life stages of the parasite. Conclusions: We can show the extent of pre-adaptation in the SS cells, especially at the level of the mitochondrial proteome. The comparison to a previously published report on monomorphic in vitro grown bloodstream and procyclic T. brucei indicates a loss of stringent regulation particularly of mitochondrial proteins in these cells when compared to the pleomorphic in vivo situation. In order to better understand the different levels of gene expression regulation in this organism we compared mRNA steady state abundance with the relative protein abundance-changes and detected moderate but significant correlation indicating that trypanosomes possess a significant repertoire of translational and posttranslational mechanisms to regulate protein abundance.
We study the entropic Gromov-Wasserstein and its unbalanced version between (unbalanced) Gaussian distributions with different dimensions. When the metric is the inner product, which we refer to as inner product Gromov-Wasserstein (IGW), we demonstrate that the optimal transportation plans of entropic IGW and its unbalanced variant are (unbalanced) Gaussian distributions. Via an application of von Neumann's trace inequality, we obtain closed-form expressions for the entropic IGW between these Gaussian distributions. Finally, we consider an entropic inner product Gromov-Wasserstein barycenter of multiple Gaussian distributions. We prove that the barycenter is a Gaussian distribution when the entropic regularization parameter is small. We further derive a closed-form expression for the covariance matrix of the barycenter.
Accurately grading open-ended assignments in large or massive open online courses (MOOCs) is non-trivial. Peer review is a promising solution but can be unreliable due to few reviewers and an unevaluated review form. To date, no work has 1) leveraged sentiment analysis in the peer-review process to inform or validate grades or 2) utilized aspect extraction to craft a review form from what students actually communicated. Our work utilizes, rather than discards, student data from review form comments to deliver better information to the instructor. In this work, we detail the process by which we create our domain-dependent lexicon and aspect-informed review form as well as our entire sentiment analysis algorithm which provides a fine-grained sentiment score from text alone. We end by analyzing validity and discussing conclusions from our corpus of over 6800 peer reviews from nine courses to understand the viability of sentiment in the classroom for increasing the information from and reliability of grading open-ended assignments in large courses.
Recent advancements in deep learning techniques have transformed the area of semantic text matching. However, most state-of-the-art models are designed to operate with short documents such as tweets, user reviews, comments, etc. These models have fundamental limitations when applied to long-form documents such as scientific papers, legal documents, and patents. When handling such long documents, there are three primary challenges: (i) the presence of different contexts for the same word throughout the document, (ii) small sections of contextually similar text between two documents, but dissimilar text in the remaining parts (this defies the basic understanding of "similarity"), and (iii) the coarse nature of a single global similarity measure which fails to capture the heterogeneity of the document content. In this paper, we describe CoLDE: Contrastive Long Document Encoder - a transformer-based framework that addresses these challenges and allows for interpretable comparisons of long documents. CoLDE uses unique positional embeddings and a multi-headed chunkwise attention layer in conjunction with a supervised contrastive learning framework to capture similarity at three different levels: (i) high-level similarity scores between a pair of documents, (ii) similarity scores between different sections within and across documents, and (iii) similarity scores between different chunks in the same document and across other documents. These fine-grained similarity scores aid in better interpretability. We evaluate CoLDE on three long document datasets namely, ACL Anthology publications, Wikipedia articles, and USPTO patents. Besides outperforming the state-of-the-art methods on the document matching task, CoLDE is also robust to changes in document length and text perturbations and provides interpretable results.
Despite achieving encouraging results, neural Referring Expression Generation models are often thought to lack transparency. We probed neural Referential Form Selection (RFS) models to find out to what extent the linguistic features influencing the RE form are learnt and captured by state-of-the-art RFS models. The results of 8 probing tasks show that all the defined features were learnt to some extent. The probing tasks pertaining to referential status and syntactic position exhibited the highest performance. The lowest performance was achieved by the probing models designed to predict discourse structure properties beyond the sentence level.
Hybrid beamforming is considered a key enabler to realize millimeter wave (mmWave) multiple-input multiple-output (MIMO) communications due to its capability of considerably reducing the number of costly and power-hungry radio frequency chains in the transceiver. However, in mmWave MIMO orthogonal frequency-division multiplexing (MIMO-OFDM) systems, hybrid beamforming design is challenging because the analog precoder and combiner are required to be shared across the whole employed bandwidth. In this paper, we propose closed-form solutions to the problem of designing the analog precoder/combiner in a mmWave MIMO-OFDM system by maximizing the upper bound of the spectral efficiency. The closed-form solutions facilitate the design of analog beamformers while guaranteeing state-of-art performance. Numerical results show that the proposed algorithm attains a slightly improved performance with much lower computational complexity compared to the considered benchmarks.
In this paper we introduce a sublinear conditional operator with respect to a family of possibly nondominated probability measures in presence of multiple ordered default times. In this way we generalize the results of [5], where a reduced-form framework under model uncertainty for a single default time is developed. Moreover, we use this operator for the valuation of credit portfolio derivatives under model uncertainty.
This work presents a novel approach to the mean-square analysis of the normalized least mean squares (NLMS) algorithm for circular complex colored Gaussian inputs. The analysis is based on the derivation of a closed-form expression for the Cumulative Distribution Function (CDF) of random variables of the form $(||{\bf u}||_{{\bf D}_1}^2)(||{\bf u}||_{{\bf D}_2}^2)^{-1}$ where ${\bf u}$ is an isotropic vector and ${\bf D}_1$ and ${\bf D}_2$ are diagonal matrices and using that to derive some moments of these variables. These moments in turn completely characterize the mean-square behavior of the NLMS algorithm in explicit closed-form expressions. Specifically, the transient, steady-state, and tracking mean-square behavior of the NLMS algorithm is studied.
This paper reports on a dynamic semantic mapping framework that incorporates 3D scene flow measurements into a closed-form Bayesian inference model. Existence of dynamic objects in the environment can cause artifacts and traces in current mapping algorithms, leading to an inconsistent map posterior. We leverage state-of-the-art semantic segmentation and 3D flow estimation using deep learning to provide measurements for map inference. We develop a Bayesian model that propagates the scene with flow and infers a 3D continuous (i.e., can be queried at arbitrary resolution) semantic occupancy map outperforming its static counterpart. Extensive experiments using publicly available data sets show that the proposed framework improves over its predecessors and input measurements from deep neural networks consistently.
We present evidence that the word entropy of American English has been rising steadily since around 1900, contrary to predictions from existing sociolinguistic theories. We also find differences in word entropy between media categories, with short-form media such as news and magazines having higher entropy than long-form media, and social media feeds having higher entropy still. To explain these results we develop an ecological model of the attention economy that combines ideas from Zipf's law and information foraging. In this model, media consumers maximize information utility rate taking into account the costs of information search, while media producers adapt to technologies that reduce search costs, driving them to generate higher entropy content in increasingly shorter formats.
In various applications in the field of control engineering the estimation of the state variables of dynamic systems in the presence of unknown inputs plays an important role. Existing methods require the so-called observer matching condition to be satisfied, rely on the boundedness of the state variables or exhibit an increased observer order of twice the plant order. In this article, a novel observer normal form for strongly observable linear time-invariant multivariable systems is proposed. In contrast to classical normal forms, the proposed approach also takes the unknown inputs into account. The proposed observer normal form allows for the straightforward construction of a higher-order sliding mode observer, which ensures global convergence of the estimation error within finite time even in the presence of unknown bounded inputs. Its application is not restricted to systems which satisfy the aforementioned limitations of already existing unknown input observers. The proposed approach can be exploited for the reconstruction of unknown inputs with bounded derivative and robust state-feedback control, which is shown by means of a tutorial example. Numerical simulations confirm the effectiveness of the presented work.
This paper incorporates a continuous-type network flexibility into chance constrained economic dispatch (CCED). In the proposed model, both power generations and line susceptances are continuous variables to minimize the expected generation cost and guarantee a low probability of constraint violation in terms of generations and line flows under renewable uncertainties. From the analytical form of CCED, we figure out the mechanism of network flexibility against uncertainties -- while renewable uncertainties shrink the usable line capacities and aggravate transmission congestion, network flexibility mitigates congestion by re-routing the base-case line flows and reducing the line capacity shrinkage caused by uncertainties. Further, we propose an alternate iteration solver for this problem. By duality theory, we set up a master problem in the form of second-order cone programming to optimize generation dispatch scheme and a subproblem in the form of linear programming to optimize line susceptances. A satisfactory solution can be obtained efficiently by alternately solving these two problems. The proposed method applies to both Gaussian uncertainty and non-Gaussian uncertainty by means of Gaussian mixture model. The case studies on the IEEE 14-bus system and IEEE 118-bus system suggest that network flexibility can significantly improve operational economy while ensuring security under uncertainties.
This paper specifies an extensive form as a 5-ary relation (that is, as a set of quintuples) which satisfies eight abstract axioms. Each quintuple is understood to list a player, a situation (that is, a name for an information set), a decision node, an action, and a successor node. Accordingly, the axioms are understood to specify abstract relationships between players, situations, nodes, and actions. Such an extensive form is called a "pentaform". Finally, a "pentaform game" is defined to be a pentaform together with utility functions. To ground this new specification in the literature, the paper defines the concept of a "traditional game" to represent the literature's many specifications of finite-horizon and infinite-horizon games. The paper's main result is to construct an intuitive bijection between pentaform games and traditional games. Secondary results concern disaggregating pentaforms by subsets, constructing pentaforms by unions, and initial pentaform applications to Selten subgames and perfect-recall (an extensive application to dynamic programming is in Streufert 2023, arXiv:2302.03855).