The standard closed form lower bound on $\sigma$ for providing $(\epsilon, \delta)$-differential privacy by adding zero mean Gaussian noise with variance $\sigma^2$ is $\sigma > \Delta\sqrt {2}(\epsilon^{-1}) \sqrt {\log \left( 5/4\delta^{-1} \right)}$ for $\epsilon \in (0,1)$. We present a similar closed form bound $\sigma \geq \Delta (\epsilon\sqrt{2})^{-1} \left(\sqrt{az+\epsilon} + s\sqrt{az}\right)$ for $z=-\log(4\delta(1-\delta))$ and $(a,s)=(1,1)$ if $\delta \leq 1/2$ and $(a,s)=(\pi/4,-1)$ otherwise. Our bound is valid for all $\epsilon > 0$ and is always lower (better). We also present a sufficient condition for $(\epsilon, \delta)$-differential privacy when adding noise distributed according to even and log-concave densities supported everywhere.
The recent availability of huge, many-dimensional data sets, like those arising from genome-wide association studies (GWAS), provides many opportunities for strengthening causal inference. One popular approach is to utilize these many-dimensional measurements as instrumental variables (instruments) for improving the causal effect estimate between other pairs of variables. Unfortunately, searching for proper instruments in a many-dimensional set of candidates is a daunting task due to the intractable model space and the fact that we cannot directly test which of these candidates are valid, so most existing search methods either rely on overly stringent modeling assumptions or fail to capture the inherent model uncertainty in the selection process. We show that, as long as at least some of the candidates are (close to) valid, without knowing a priori which ones, they collectively still pose enough restrictions on the target interaction to obtain a reliable causal effect estimate. We propose a general and efficient causal inference algorithm that accounts for model uncertainty by performing Bayesian model averaging over the most promising many-dimensional instrumental variable models, while at the same time employing weaker assumptions regarding the data generating process. We showcase the efficiency, robustness and predictive performance of our algorithm through experimental results on both simulated and real-world data.
The conditional moment problem is a powerful formulation for describing structural causal parameters in terms of observables, a prominent example being instrumental variable regression. A standard approach reduces the problem to a finite set of marginal moment conditions and applies the optimally weighted generalized method of moments (OWGMM), but this requires we know a finite set of identifying moments, can still be inefficient even if identifying, or can be theoretically efficient but practically unwieldy if we use a growing sieve of moment conditions. Motivated by a variational minimax reformulation of OWGMM, we define a very general class of estimators for the conditional moment problem, which we term the variational method of moments (VMM) and which naturally enables controlling infinitely-many moments. We provide a detailed theoretical analysis of multiple VMM estimators, including ones based on kernel methods and neural nets, and provide conditions under which these are consistent, asymptotically normal, and semiparametrically efficient in the full conditional moment model. We additionally provide algorithms for valid statistical inference based on the same kind of variational reformulations, both for kernel- and neural-net-based varieties. Finally, we demonstrate the strong performance of our proposed estimation and inference algorithms in a detailed series of synthetic experiments.
One of the key problems in tensor completion is the number of uniformly random sample entries required for recovery guarantee. The main aim of this paper is to study $n_1 \times n_2 \times n_3$ third-order tensor completion based on transformed tensor singular value decomposition, and provide a bound on the number of required sample entries. Our approach is to make use of the multi-rank of the underlying tensor instead of its tubal rank in the bound. In numerical experiments on synthetic and imaging data sets, we demonstrate the effectiveness of our proposed bound for the number of sample entries. Moreover, our theoretical results are valid to any unitary transformation applied to $n_3$-dimension under transformed tensor singular value decomposition.
Conventional methods in causal effect inferencetypically rely on specifying a valid set of control variables. When this set is unknown or misspecified, inferences will be erroneous. We propose a method for inferring average causal effects when all potential confounders are observed, but thecontrol variables are unknown. When the data-generating process belongs to the class of acyclical linear structural causal models, we prove that themethod yields asymptotically valid confidence intervals. Our results build upon a smooth characterization of linear directed acyclic graphs. We verify the capability of the method to produce valid confidence intervals for average causal effects using synthetic data, even when the appropriate specification of control variables is unknown.
We study high-dimensional least-squares regression within a subgaussian statistical learning framework with heterogeneous noise. It includes $s$-sparse and $r$-low-rank least-squares regression when a fraction $\epsilon$ of the labels are adversarially contaminated. We also present a novel theory of trace-regression with matrix decomposition based on a new application of the product process. For these problems, we show novel near-optimal "subgaussian" estimation rates of the form $r(n,d_{e})+\sqrt{\log(1/\delta)/n}+\epsilon\log(1/\epsilon)$, valid with probability at least $1-\delta$. Here, $r(n,d_{e})$ is the optimal uncontaminated rate as a function of the effective dimension $d_{e}$ but independent of the failure probability $\delta$. These rates are valid uniformly on $\delta$, i.e., the estimators' tuning do not depend on $\delta$. Lastly, we consider noisy robust matrix completion with non-uniform sampling. If only the low-rank matrix is of interest, we present a novel near-optimal rate that is independent of the corruption level $a$. Our estimators are tractable and based on a new "sorted" Huber-type loss. No information on $(s,r,\epsilon,a)$ are needed to tune these estimators. Our analysis makes use of novel $\delta$-optimal concentration inequalities for the multiplier and product processes which could be useful elsewhere. For instance, they imply novel sharp oracle inequalities for Lasso and Slope with optimal dependence on $\delta$. Numerical simulations confirm our theoretical predictions. In particular, "sorted" Huber regression can outperform classical Huber regression.
Human pose transfer, as a misaligned image generation task, is very challenging. Existing methods cannot effectively utilize the input information, which often fail to preserve the style and shape of hair and clothes. In this paper, we propose an adaptive human pose transfer network with two hierarchical deformation levels. The first level generates human semantic parsing aligned with the target pose, and the second level generates the final textured person image in the target pose with the semantic guidance. To avoid the drawback of vanilla convolution that treats all the pixels as valid information, we use gated convolution in both two levels to dynamically select the important features and adaptively deform the image layer by layer. Our model has very few parameters and is fast to converge. Experimental results demonstrate that our model achieves better performance with more consistent hair, face and clothes with fewer parameters than state-of-the-art methods. Furthermore, our method can be applied to clothing texture transfer.
Several scientific studies have reported the existence of the income gap among rideshare drivers based on demographic factors such as gender, age, race, etc. In this paper, we study the income inequality among rideshare drivers due to discriminative cancellations from riders, and the tradeoff between the income inequality (called fairness objective) with the system efficiency (called profit objective). We proposed an online bipartite-matching model where riders are assumed to arrive sequentially following a distribution known in advance. The highlight of our model is the concept of acceptance rate between any pair of driver-rider types, where types are defined based on demographic factors. Specially, we assume each rider can accept or cancel the driver assigned to her, each occurs with a certain probability which reflects the acceptance degree from the rider type towards the driver type. We construct a bi-objective linear program as a valid benchmark and propose two LP-based parameterized online algorithms. Rigorous online competitive ratio analysis is offered to demonstrate the flexibility and efficiency of our online algorithms in balancing the two conflicting goals, promotions of fairness and profit. Experimental results on a real-world dataset are provided as well, which confirm our theoretical predictions.
In this work, we present Lexical Unit Analysis (LUA), a framework for general sequence segmentation tasks. Given a natural language sentence, LUA scores all the valid segmentation candidates and utilizes dynamic programming (DP) to extract the maximum scoring one. LUA enjoys a number of appealing properties such as inherently guaranteeing the predicted segmentation to be valid and facilitating globally optimal training and inference. Besides, the practical time complexity of LUA can be reduced to linear time, which is very efficient. We have conducted extensive experiments on 5 tasks, including syntactic chunking, named entity recognition (NER), slot filling, Chinese word segmentation, and Chinese part-of-speech (POS) tagging, across 15 datasets. Our models have achieved the state-of-the-art performances on 13 of them. The results also show that the F1 score of identifying long-length segments is notably improved.
Anomalous change detection (ACD) is an important problem in remote sensing image processing. Detecting not only pervasive but also anomalous or extreme changes has many applications for which methodologies are available. This paper introduces a nonlinear extension of a full family of anomalous change detectors. In particular, we focus on algorithms that utilize Gaussian and elliptically contoured (EC) distribution and extend them to their nonlinear counterparts based on the theory of reproducing kernels' Hilbert space. We illustrate the performance of the kernel methods introduced in both pervasive and ACD problems with real and simulated changes in multispectral and hyperspectral imagery with different resolutions (AVIRIS, Sentinel-2, WorldView-2, and Quickbird). A wide range of situations is studied in real examples, including droughts, wildfires, and urbanization. Excellent performance in terms of detection accuracy compared to linear formulations is achieved, resulting in improved detection accuracy and reduced false-alarm rates. Results also reveal that the EC assumption may be still valid in Hilbert spaces. We provide an implementation of the algorithms as well as a database of natural anomalous changes in real scenarios http://isp.uv.es/kacd.html.
The paper deals with conditional linear information inequalities valid for entropy functions induced by discrete random variables. Specifically, the so-called conditional Ingleton inequalities are in the center of interest: these are valid under conditional independence assumptions on the inducing random variables. We discuss five inequalities of this particular type, four of which has appeared earlier in the literature. Besides the proof of the new fifth inequality, simpler proofs of (some of) former inequalities are presented. These five information inequalities are used to characterize all conditional independence structures induced by four discrete random variables.
Neural Tangent Kernel (NTK) theory is widely used to study the dynamics of infinitely-wide deep neural networks (DNNs) under gradient descent. But do the results for infinitely-wide networks give us hints about the behavior of real finite-width ones? In this paper, we study empirically when NTK theory is valid in practice for fully-connected ReLU and sigmoid DNNs. We find out that whether a network is in the NTK regime depends on the hyperparameters of random initialization and the network's depth. In particular, NTK theory does not explain the behavior of sufficiently deep networks initialized so that their gradients explode as they propagate through the network's layers: the kernel is random at initialization and changes significantly during training in this case, contrary to NTK theory. On the other hand, in the case of vanishing gradients, DNNs are in the the NTK regime but become untrainable rapidly with depth. We also describe a framework to study generalization properties of DNNs, in particular the variance of network's output function, by means of NTK theory and discuss its limits.
This paper considers uniformly valid inference for linear functionals and scalar subvectors of partially identified parameters defined by linear moment inequalities. Our proposed procedure amounts to bootstrapping the value functions of four carefully constructed "perturbed" linear programming problems, and does not require the researcher to grid over the parameter space. Our low-level conditions for uniform validity rely on an application of Sard's Theorem from differential topology. Our procedure is asymptotically conservative for the true partially identified parameter, but is valid under weak assumptions, performs well in finite samples, and is computationally simple to implement.
The energy-momentum budget of the astrophysical systems can be studied by the exact local conservation equation derived by Landau and Lifshitz. We show that a similar equation is valid for the Einstein-Cartan gravity. We reanalyze a binary pulsar system using the Landau-Lifshitz conservation equation and show that the orbital period change rate can be completely understood as a curvature backreaction process. Taking into account the detailed theoretical and observational research of relativistic binary pulsar systems, especially the system of Hulse and Taylor, we conclude that general relativity and astrophysical observations rule out the existence of gravitational radiation. We comment upon the LIGO GW events and their alternative explanation, as well as the recent pulsar timing arrays data.
A wide range of reinforcement learning (RL) problems - including robustness, transfer learning, unsupervised RL, and emergent complexity - require specifying a distribution of tasks or environments in which a policy will be trained. However, creating a useful distribution of environments is error prone, and takes a significant amount of developer time and effort. We propose Unsupervised Environment Design (UED) as an alternative paradigm, where developers provide environments with unknown parameters, and these parameters are used to automatically produce a distribution over valid, solvable environments. Existing approaches to automatically generating environments suffer from common failure modes: domain randomization cannot generate structure or adapt the difficulty of the environment to the agent's learning progress, and minimax adversarial training leads to worst-case environments that are often unsolvable. To generate structured, solvable environments for our protagonist agent, we introduce a second, antagonist agent that is allied with the environment-generating adversary. The adversary is motivated to generate environments which maximize regret, defined as the difference between the protagonist and antagonist agent's return. We call our technique Protagonist Antagonist Induced Regret Environment Design (PAIRED). Our experiments demonstrate that PAIRED produces a natural curriculum of increasingly complex environments, and PAIRED agents achieve higher zero-shot transfer performance when tested in highly novel environments.
Holographic communication is intended as an holistic way to manipulate with unprecedented flexibility the electromagnetic field generated or sensed by an antenna. This is of particular interest when using large antennas at high frequency (e.g., the millimeter wave or terahertz), whose operating condition may easily fall in the Fresnel propagation region (radiating near-field), where the classical plane wave propagation assumption is no longer valid. This paper analyzes the optimal communication involving large intelligent surfaces, realized for example with metamaterials as possible enabling technology for holographic communication. It is shown that traditional propagation models must be revised and that, when exploiting spherical wave propagation in the Fresnel region with large surfaces, new opportunities are opened, for example, in terms of the number of orthogonal communication channels.
Microdialysis is a well-established method for in vivo neurochemical measurements of small molecules, with implanted concentric-design probes offering minimized tissue damage and good temporal and spatial resolution. However, the large majority of measurements do not allow the perfusate to reach equilibrium with the brain, so that inferential methods of sample concentration correction such as zero-net-flux must be used to determine actual brain extracellular fluid glucose concentrations. In order for such methods to be valid, steady-state transfer of the analyte of interest within the brain is required, but this situation has not previously been confirmed. A first-principles mathematical model of fluid flow and analyte diffusion around an implanted microdialysis probe was developed and implemented in COMSOL in order to validate the zero-net-flux approach, using measurement of extracellular brain glucose levels as a well-explored example system against which to compare the model. Results from the model accurately reproduced and predicted results from in vivo experiments. Importantly, the model predicts that the time for an implanted probe to achieve steady-state equilibrium with the surrounding extracellular fluid is on the order of one to two minutes, supporting the validity of this technique for quantitative measurement of in vivo neurochemistry.
Elongation is a fundament process in amyloid fiber growth, which is normally characterized by a linear relationship between the fiber elongation rate and the monomer concentration. However, in high concentration regions, a sub-linear dependence was often observed, which could be explained by a universal saturation mechanism. In this paper, we modeled the saturated elongation process through a Michaelis-Menten like mechanism, which is constituted by two sub-steps -- unspecific association and dissociation of a monomer with the fibril end, and subsequent conformational change of the associated monomer to fit itself to the fibrillar structure. Typical saturation concentrations were found to be $7-70\mu M$ for A$\beta$40, $\alpha$-synuclein and etc. Furthermore, by using a novel Hamiltonian formulation, analytical solutions valid for both weak and strong saturated conditions were constructed and applied to the fibrillation kinetics of $\alpha$-synuclein and silk fibroin.
Recurrent boom-and-bust cycles are a salient feature of economic and financial history. Cycles found in the data are stochastic, often highly persistent, and span substantial fractions of the sample size. We refer to such cycles as "long". In this paper, we develop a novel approach to modeling cyclical behavior specifically designed to capture long cycles. We show that existing inferential procedures may produce misleading results in the presence of long cycles, and propose a new econometric procedure for the inference on the cycle length. Our procedure is asymptotically valid regardless of the cycle length. We apply our methodology to a set of macroeconomic and financial variables for the U.S. We find evidence of long stochastic cycles in the standard business cycle variables, as well as in credit and house prices. However, we rule out the presence of stochastic cycles in asset market data. Moreover, according to our result, financial cycles as characterized by credit and house prices tend to be twice as long as business cycles.
We consider the problem of inferring the unobserved edges of a graph from data supported on its nodes. In line with existing approaches, we propose a convex program for recovering a graph Laplacian that is approximately diagonalizable by a set of eigenvectors obtained from the second-order moment of the observed data. Unlike existing work, we incorporate prior knowledge about the distribution from where the underlying graph was drawn. In particular, we consider the case where the graph was drawn from a graphon model, and we supplement our convex optimization problem with a provably-valid regularizer on the spectrum of the graph to be recovered. We present the cases where the graphon model is assumed to be known and the more practical setting where the relevant features of the model are inferred from auxiliary network observations. Numerical experiments on synthetic and real-world data illustrate the advantage of leveraging the proposed graphon prior, even when the prior is imperfect.